Repository navigation
Description
Hi! We found a rather nasty bug with as_array when it's combined with structured ndarrays. A field of a structured array can have a byte stride that isn't a multiple of the item size, and in that case as_array silently builds a view that reads the wrong elements. With an object field, it segfaults.
The cause seems to be the stride conversion in the internal as_view, which does strides[i] as usize / itemsize. For a 9-byte stride and an 8-byte item that becomes 1, so the view steps 8 bytes instead of 9.
Here's a small repro that segfaults:
use std::ffi::CStr;
use numpy::{PyArray1, PyArrayMethods};
use pyo3::{prelude::*, types::PyDict};
const SETUP: &CStr = c"
import numpy as np
arr = np.array(
[('x', False), ('y', False), ('z', False)],
dtype=[('obj', 'O'), ('flag', '?')],
)
field = arr['obj'] # ['x', 'y', 'z'], strides (9,)
";
fn main() -> PyResult<()> {
Python::initialize();
Python::attach(|py| {
let locals = PyDict::new(py);
py.run(SETUP, None, Some(&locals))?;
let field = locals.get_item("field")?.unwrap().cast_into::<PyArray1<Py<PyAny>>>()?;
for obj in field.readonly().as_array() {
println!("{}", obj.bind(py)); // x, then segfault
}
Ok(())
})
}At first this looked like #523, but it also happens with aligned data.
Here the field is aligned, and as_array just quietly returns wrong values:
use std::ffi::CStr;
use numpy::{Complex64, PyArray1, PyArrayMethods};
use pyo3::{prelude::*, types::PyDict};
const SETUP: &CStr = c"
import numpy as np
arr = np.array(
[(1, 0), (2, 0), (3, 0)],
dtype=[('num', 'c16'), ('other', 'f8')],
)
field = arr['num'] # [1+0j, 2+0j, 3+0j], strides (24,), aligned
";
fn main() -> PyResult<()> {
Python::initialize();
Python::attach(|py| {
let locals = PyDict::new(py);
py.run(SETUP, None, Some(&locals))?;
let field = locals.get_item("field")?.unwrap().cast_into::<PyArray1<Complex64>>()?;
println!("{}", field.readonly().as_array()); // [1+0i, 0+2i, 0+0i]
Ok(())
})
}We tested this with the numpy crate 0.30.0 and the current main (d3720cd), pyo3 0.30.1, Python 3.14.8 and NumPy 2.5.4.
Since ndarray strides are in elements, I don't think such arrays can be represented as an ArrayView at all. A few ideas for fixing it:
- Make
as_arrayandas_array_mutpanic when a stride isn't a multiple of the item size, like they already do on a dimensionality mismatch. This keeps the API as it is. - Add
try_as_arrayandtry_as_array_mutthat return an error instead, likeas_sliceandtry_readonlydo. - Let
to_owned_arraycopy such arrays correctly. It goes through the same view, but since it copies anyway, it doesn't need to panic. - Check
try_as_matrixtoo. It looks like it divides the strides the same way, so it could returnNonefor such arrays.
For now we work around it by passing np.ascontiguousarray(field) instead.
Thanks for the great crate!
Activity
Icxolu commentedon Nov 5, 2026
Thanks for reporting. You are right, we can't represent arrays with strides that are not a multiple of the element size using ndarray. I opened #578 to fix the soundness issue by asserting the strides are a multiple of the element size for now.