Skip to content

[C++][Parquet] Fix DeltaLengthByteArrayEncoder memory pool tracking, Arrow array batching, and zero-length safety #55487

Open

Description

@ryanvickr

Describe the bug, including details regarding any error messages, version, and platform.

DeltaLengthByteArrayEncoder in cpp/src/parquet/encoder.cc has three issues when encoding string/binary pages:

  1. Caller MemoryPool* is overwritten with default_memory_pool(): The constructor writes EncoderImpl(descr, Encoding::DELTA_LENGTH_BYTE_ARRAY, pool = ::arrow::default_memory_pool()) (assignment in the initializer list instead of a default parameter), resetting pool before initializing sink_(pool) and length_encoder_(nullptr, pool). Allocations on DeltaLengthByteArrayEncoder (and DeltaByteArrayEncoder::suffix_encoder_) bypass caller-provided memory pools.
  2. Unbatched PutBinaryArray without pre-reserving sink_: PutBinaryArray invokes length_encoder_.Put({len}, 1) per element and appends to sink_ without pre-reserving capacity, unlike Put(const ByteArray*, int) (which batches 256 lengths and reserves sink_ up front) and other Parquet writers (parquet-rs, Polars).
  3. Zero-length UnsafeAppend with null data pointer: Put(const ByteArray*, int) calls sink_.UnsafeAppend(src[idx].ptr, src[idx].len) unconditionally, passing nullptr to std::memcpy(dest, nullptr, 0) under UBSan on ByteArray(0, nullptr) values.

Component(s)

C++, Parquet

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions