fix(python): declare read-scan methods in the datafusion type stub - #918
Conversation
`ReadBuilder.with_nested_projection`, `TableScan.with_chunk_shuffle` and `TableScan.with_shard` are implemented `#[pymethods]` and exercised in `tests/test_read.py`, but `datafusion.pyi` never declared them. The package ships `py.typed`, so the stub is the type contract: mypy/pyright reject these calls with "no attribute" and offer no completion, even though they work. The methods were added in apache#887 and apache#890; the stub was just not updated.
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
Reviewed head 0711b144. This has end-to-end value for typed Python callers: the three methods already exist in the PyO3 implementation and are exercised in tests/test_read.py, while the published py.typed stub omitted them. The added argument and fluent return types match the native signatures (String/u64, usize/usize, and Vec<Vec<String>> respectively). I found no actionable regression.
Validation: parsed the full .pyi with Python's AST and asserted all three method names, parameters, annotations, and return types; git diff --check passed; head CI is green. A mypy/pyright executable and a built Python extension are not present in this checkout, so I did not claim a live type-checker or Python runtime run for this stub-only change.
2 participants
ReadBuilder.with_nested_projection,TableScan.with_chunk_shuffleandTableScan.with_shardare implemented#[pymethods](and exercised intests/test_read.py), butdatafusion.pyinever declared them — #887 and #890 added the methods and skipped the stub. The package shipspy.typed, so a typed caller gets a mypy/pyright "no attribute" error and no completion for API that works. Declares the three with signatures matching the impl.