Search before asking
Motivation
Paimon C++ currently requires a BatchReader to outlive its returned ReadBatch, because batch buffers may use Arrow or ORC memory pools owned by the reader. This lifetime coupling is inconvenient for downstream engines and can cause invalid memory access if the reader is destroyed first.
Solution
Retain the required memory-pool smart pointers in the root ArrowArray, so the batch can safely outlive its reader.
Consumers must keep, move, and release the complete root ArrowArray. Moving individual child arrays out of it is unsupported because child arrays do not retain the memory-pool lifetime independently.
Anything else?
Related discussion: alibaba/paimon-cpp#126
Are you willing to submit a PR?
Search before asking
Motivation
Paimon C++ currently requires a
BatchReaderto outlive its returnedReadBatch, because batch buffers may use Arrow or ORC memory pools owned by the reader. This lifetime coupling is inconvenient for downstream engines and can cause invalid memory access if the reader is destroyed first.Solution
Retain the required memory-pool smart pointers in the root
ArrowArray,so the batch can safely outlive its reader.Consumers must keep, move, and release the complete root
ArrowArray. Moving individual child arrays out of it is unsupported because child arrays do not retain the memory-pool lifetime independently.Anything else?
Related discussion: alibaba/paimon-cpp#126
Are you willing to submit a PR?