✳flâneur — a map of the web's best reading

Configuration - Apache Iceberg

iceberg.apache.org · 1,604 words · saved by 1 readers

Iceberg tables support table properties to configure table behavior, like the default split size for readers. Reserved table properties are only used to control behaviors when creating or updating a table. The value of these properties are not persisted as a part of the table metadata. Iceberg catalogs support using catalog properties to configure catalog behaviors. Here is a list of commonly used catalog properties: HadoopCatalog and HiveCatalog can access the properties in their constructors. Any other custom catalog can access the properties by implementing Catalog.initialize(catalogName, catalogProperties). The properties can be manually constructed or passed in from a compute engine like Spark or Flink. Spark uses its session properties as catalog properties, see more details in the Spark configuration section. Flink passes in catalog properties through CREATE CATALOG statement, see more details in the Flink section. Here are the catalog properties related to locking. They are use

Configuration 🔗 Table properties 🔗 Iceberg tables support table properties to configure table behavior, like the default split size for readers. Read properties 🔗 Property Default Description read.split.target-size 134217728 (128 MB) Target size when combining data input splits read.split.metadata-target-size 33554432 (32 MB) Target size when combining metadata input splits read.split.planning-lookback 10 Number of bins to consider when combining input splits read.split.open-file-cost 4194304 (4 MB) The estimated cost to open a file, used as a minimum weight when combining splits. read.parq

Explore this link on the map →

saved by

related reading