Skip to content

Safe as_array segfaults or corrupts data when a stride is not a multiple of the item size #575

Open

Description

@MarkusSintonen

Hi! We found a rather nasty bug with as_array when it's combined with structured ndarrays. A field of a structured array can have a byte stride that isn't a multiple of the item size, and in that case as_array silently builds a view that reads the wrong elements. With an object field, it segfaults.

The cause seems to be the stride conversion in the internal as_view, which does strides[i] as usize / itemsize. For a 9-byte stride and an 8-byte item that becomes 1, so the view steps 8 bytes instead of 9.

Here's a small repro that segfaults:

use std::ffi::CStr;

use numpy::{PyArray1, PyArrayMethods};
use pyo3::{prelude::*, types::PyDict};

const SETUP: &CStr = c"
import numpy as np

arr = np.array(
    [('x', False), ('y', False), ('z', False)],
    dtype=[('obj', 'O'), ('flag', '?')],
)
field = arr['obj']  # ['x', 'y', 'z'], strides (9,)
";

fn main() -> PyResult<()> {
    Python::initialize();
    Python::attach(|py| {
        let locals = PyDict::new(py);
        py.run(SETUP, None, Some(&locals))?;
        let field = locals.get_item("field")?.unwrap().cast_into::<PyArray1<Py<PyAny>>>()?;

        for obj in field.readonly().as_array() {
            println!("{}", obj.bind(py)); // x, then segfault
        }
        Ok(())
    })
}

At first this looked like #523, but it also happens with aligned data.
Here the field is aligned, and as_array just quietly returns wrong values:

use std::ffi::CStr;

use numpy::{Complex64, PyArray1, PyArrayMethods};
use pyo3::{prelude::*, types::PyDict};

const SETUP: &CStr = c"
import numpy as np

arr = np.array(
    [(1, 0), (2, 0), (3, 0)],
    dtype=[('num', 'c16'), ('other', 'f8')],
)
field = arr['num']  # [1+0j, 2+0j, 3+0j], strides (24,), aligned
";

fn main() -> PyResult<()> {
    Python::initialize();
    Python::attach(|py| {
        let locals = PyDict::new(py);
        py.run(SETUP, None, Some(&locals))?;
        let field = locals.get_item("field")?.unwrap().cast_into::<PyArray1<Complex64>>()?;

        println!("{}", field.readonly().as_array()); // [1+0i, 0+2i, 0+0i]
        Ok(())
    })
}

We tested this with the numpy crate 0.30.0 and the current main (d3720cd), pyo3 0.30.1, Python 3.14.8 and NumPy 2.5.4.

Since ndarray strides are in elements, I don't think such arrays can be represented as an ArrayView at all. A few ideas for fixing it:

  • Make as_array and as_array_mut panic when a stride isn't a multiple of the item size, like they already do on a dimensionality mismatch. This keeps the API as it is.
  • Add try_as_array and try_as_array_mut that return an error instead, like as_slice and try_readonly do.
  • Let to_owned_array copy such arrays correctly. It goes through the same view, but since it copies anyway, it doesn't need to panic.
  • Check try_as_matrix too. It looks like it divides the strides the same way, so it could return None for such arrays.

For now we work around it by passing np.ascontiguousarray(field) instead.

Thanks for the great crate!

Activity

  1. Icxolu commented on Nov 5, 2026

    @Icxolu
    Member

    Thanks for reporting. You are right, we can't represent arrays with strides that are not a multiple of the element size using ndarray. I opened #578 to fix the soundness issue by asserting the strides are a multiple of the element size for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions