You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[C++][Parquet] Fix DeltaLengthByteArrayEncoder memory pool tracking, Arrow array batching, and zero-length safety #55487
Describe the bug, including details regarding any error messages, version, and platform.
DeltaLengthByteArrayEncoder in cpp/src/parquet/encoder.cc has three issues when encoding string/binary pages:
Caller MemoryPool* is overwritten with default_memory_pool(): The constructor writes EncoderImpl(descr, Encoding::DELTA_LENGTH_BYTE_ARRAY, pool = ::arrow::default_memory_pool()) (assignment in the initializer list instead of a default parameter), resetting pool before initializing sink_(pool) and length_encoder_(nullptr, pool). Allocations on DeltaLengthByteArrayEncoder (and DeltaByteArrayEncoder::suffix_encoder_) bypass caller-provided memory pools.
Unbatched PutBinaryArray without pre-reserving sink_: PutBinaryArray invokes length_encoder_.Put({len}, 1) per element and appends to sink_ without pre-reserving capacity, unlike Put(const ByteArray*, int) (which batches 256 lengths and reserves sink_ up front) and other Parquet writers (parquet-rs, Polars).
Zero-length UnsafeAppend with null data pointer: Put(const ByteArray*, int) calls sink_.UnsafeAppend(src[idx].ptr, src[idx].len) unconditionally, passing nullptr to std::memcpy(dest, nullptr, 0) under UBSan on ByteArray(0, nullptr) values.
Activity