Skip to content

Parquet writer ignores per-column write.parquet.bloom-filter-* table properties #3147

Description

@alessandro-nori

Is your feature request related to a problem or challenge?

ParquetWriterBuilder::from_table_properties translates write.parquet.* settings into parquet-rs WriterProperties (crates/iceberg/src/writer/file_writer/parquet_writer.rs:90-113): CDC, compression codec/level, row-group bytes, page size, page row limit, dict page size.

The bloom-filter properties are not among them, and crates/iceberg/src/spec/table_properties.rs defines no constants for them. A table that requests bloom filters gets files written without any. Reference behaviour is in TableProperties.java:184-199, consumed at parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java:652-668.

Describe the solution you'd like

the three per-column properties (write.parquet.bloom-filter-enabled.column.<col>, write.parquet.bloom-filter-fpp.column.<col> and write.parquet.bloom-filter-ndv.column.<col>) map directly onto parquet-rs.

Willingness to contribute

I would be willing to contribute to this feature with guidance from the Iceberg Rust community

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions