Skip to content

GH-35721: [Python] Recognize pandas arrays in the array() constructor for the from_pandas argument - #55477

Open
jay23606 wants to merge 1 commit into
apache:mainfrom
jay23606:pr-go/35721-python-recognize-pandas-arrays-in-the-ar
Open

jay23606 wants to merge 1 commit into
apache:mainfrom
jay23606:pr-go/35721-python-recognize-pandas-arrays-in-the-ar

Conversation

@jay23606

@jay23606 jay23606 commented Nov 10, 2026 •

Copy link
Copy Markdown

Problem

When pa.array() is called without from_pandas, it sets from_pandas=True automatically only for pandas Series and Index inputs. A pandas ExtensionArray such as pd.array([1.2, np.nan], dtype=float) was not detected, so NaN was kept as NaN instead of being converted to null, unlike the Series case.

Change

In get_values (python/pyarrow/array.pxi), the generic branch now sets is_pandas_object when the input has an extension dtype (pandas_api.is_extension_array_dtype(obj)). Plain lists and numpy arrays keep the existing behaviour.

Testing

Added test_pandas_null_sentinels_extension_array in python/pyarrow/tests/test_array.py. It checks that pa.array(pd.array([1.2, np.nan], dtype=float)) equals the from_pandas=True result and has one null. The test could not be run in this environment because pyarrow is not built here, so it is verified by CI only. Only a syntax check was run locally: python -c "import ast,sys; ast.parse(open('python/pyarrow/tests/test_array.py',encoding='utf-8').read()); print('syntax ok')".

Regression test: untested: CONTRIBUTING does not mention tests

Fixes #35721

@jay23606

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #35721 has been automatically assigned in GitHub to PR creator.

@jay23606
jay23606 marked this pull request as ready for review November 10, 2026 15:20
Copilot AI balanced review requested due to automatic review settings November 10, 2026 15:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the change aligning ExtensionArray null handling with Series and Index inputs.

No open issues 🎉

What changed in this PR

This PR addresses GH-35721 by extending pa.array()’s automatic pandas null handling to pandas ExtensionArray inputs, aligning them with Series and Index inputs.

Changes:

  • Detects extension dtypes in get_values to enable pandas semantics automatically.
  • Adds a regression test checking that a floating-point ExtensionArray converts NaN to null when from_pandas is omitted.

The new test was syntax-checked locally; runtime validation remains pending in CI.

File Description
python/​pyarrow/​tests/​test_array.py Adds regression coverage for ExtensionArray null handling.
python/​pyarrow/​array.pxi Uses extension-dtype detection to select pandas semantics.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python] Recognize pandas arrays in the array() constructor for the from_pandas argument

2 participants