Describe the bug
Array$create() on an R double or integer vector borrows the vector's data instead of copying
it (two Arrays created from the same vector export the same buffer address). R's vector data begins
48 bytes into its allocation, so the exported values buffer starts at offset 48 within its page and
at 48 mod 64: below the 64-byte alignment the columnar format recommends and that Arrow's own
allocator provides. Buffers arrow allocates itself, a $cast() result or a conversion to int64(),
are page aligned.
The borrow is a sensible default; this report is about the consequence for consumers that check
alignment. A C Data Interface consumer that needs aligned buffers (SIMD paths, a GPU runtime that
maps shared memory in place, mmap-based storage) gets an unaligned buffer from every R-created
numeric array, and nothing in the documentation of Array$create() says so or offers a copying
alternative.
arrow 25.0.0, R 4.5.3, macOS 26.6 (arm64). Addresses read from the exported struct ArrowArray
by a small C shim; any consumer of Array$export_to_c() sees the same numbers.
library(arrow)
x <- runif(1e6)
a1 <- Array$create(x); a2 <- Array$create(x)
# exported values-buffer address of a1 == that of a2 -> TRUE (a borrow, not a copy)
# address %% 64 -> 48
# address %% 16384 -> 48
# twenty fresh arrays at 1e6 and at 1e7 elements: offset 48 every time, page aligned 0/20 at both sizes
Array$create(x)$cast(float32()) # arrow-allocated: address %% 16384 == 0
Array$create(as.integer(x * 1e3), type = int64()) # converted: address %% 16384 == 0
Array$create(x)$cast(float64()) # same type: the same buffer, still 48
concat_arrays(Array$create(x)) # a copy: address %% 16384 == 0
Array$create(as.integer(x * 1e3)) # integer vector: also 48
Expected behavior
Either of: a note in ?Array that Array$create() borrows numeric vectors and that the buffer is
therefore 48 mod 64 aligned, with the copying route named ($cast() to the same type is not one, it
returns the same buffer; concat_arrays() of a single array is, today, at offset 0, but nothing says
so); or an option on Array$create() to copy into arrow-allocated, aligned memory.
Component(s)
R
Describe the bug
Array$create()on an Rdoubleorintegervector borrows the vector's data instead of copyingit (two Arrays created from the same vector export the same buffer address). R's vector data begins
48 bytes into its allocation, so the exported values buffer starts at offset 48 within its page and
at 48 mod 64: below the 64-byte alignment the columnar format recommends and that Arrow's own
allocator provides. Buffers arrow allocates itself, a
$cast()result or a conversion toint64(),are page aligned.
The borrow is a sensible default; this report is about the consequence for consumers that check
alignment. A C Data Interface consumer that needs aligned buffers (SIMD paths, a GPU runtime that
maps shared memory in place, mmap-based storage) gets an unaligned buffer from every R-created
numeric array, and nothing in the documentation of
Array$create()says so or offers a copyingalternative.
arrow 25.0.0, R 4.5.3, macOS 26.6 (arm64). Addresses read from the exported
struct ArrowArrayby a small C shim; any consumer of
Array$export_to_c()sees the same numbers.Expected behavior
Either of: a note in
?ArraythatArray$create()borrows numeric vectors and that the buffer istherefore 48 mod 64 aligned, with the copying route named (
$cast()to the same type is not one, itreturns the same buffer;
concat_arrays()of a single array is, today, at offset 0, but nothing saysso); or an option on
Array$create()to copy into arrow-allocated, aligned memory.Component(s)
R