Skip to content

gh-149196: Hide executor implementation details from ABI tools#153614

Open
Taeknology wants to merge 2 commits into
python:mainfrom
Taeknology:gh-149196-opaque-executor-array
Open

gh-149196: Hide executor implementation details from ABI tools#153614
Taeknology wants to merge 2 commits into
python:mainfrom
Taeknology:gh-149196-opaque-executor-array

Conversation

@Taeknology

@Taeknology Taeknology commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Hide the concrete executor array and executor object layouts from ABI analysis
while preserving their existing runtime representation and behavior.

_PyExecutorArray was concretely defined in the non-limited code object header.
This made the private _PyExecutorObject layout ABI-reachable through
PyCodeObject.co_executors.

Moving the array representation alone was not sufficient. A controlled ABI
probe showed that _PyExecutorObject remained reachable through _PyExitData
and exported executor and JIT interfaces intended for internal use.

This change therefore:

  • leaves only an incomplete declaration of _PyExecutorArray in
    Include/cpython/code.h;
  • defines a separate concrete array representation in the internal optimizer
    header;
  • introduces _PyExecutorHandle at exported executor and JIT interface
    boundaries;
  • keeps _PyExecutorObject and the concrete executor array representation
    internal;
  • preserves the existing field layout, allocation behavior, exported symbol
    names, and symbol visibility;
  • does not add a type-specific libabigail suppression.

The apparent 12-file footprint includes three files generated from
Python/bytecodes.c. The remaining consumer changes are mechanical adaptations
required by the opaque array and handle types.

ABI verification

I used a controlled probe that adds a field to _PyExecutorObject in a separate
temporary source tree.

Before this change, suppression-enabled abidiff reported the probe as an ABI
difference and exited with status 4.

After this change, suppression-enabled abidiff exited with status 0 in two
runs for each of the following configurations:

  • a standard shared debug build;
  • an experimental JIT-interpreter shared debug build.

Both direct library comparisons and ABI XML comparisons passed. The exported
executor and JIT symbols remained present.

Tests

  • Standard debug shared build
  • Standard --fast-ci: 48,060 tests
  • Free-threaded debug shared build
  • Free-threaded --fast-ci: 51,526 tests
  • Repeated executor, opcode, monitoring, tracing, C API, and code-object tests
  • Repeated focused experimental JIT-interpreter tests with PYTHON_JIT=1
  • Reference-leak checks for the core affected tests
  • Checks that generated case files were up to date after regeneration
  • make patchcheck
  • git diff --check

This PR includes no documentation changes or NEWS entry because it changes only
the visibility of private implementation details, without changing
user-visible behavior or any supported public API.

Closes #149196

Make _PyExecutorArray opaque in the non-limited code object header and keep its concrete representation internal.

Use an opaque _PyExecutorHandle at exported executor and JIT boundaries so _PyExecutorObject is no longer reachable by ABI tooling. Preserve the existing runtime layout, exported symbols, and behavior.
@Taeknology Taeknology marked this pull request as ready for review July 12, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_PyExecutorObject is exposed to the abidiff tool due to indirect use in PyCodeObject

1 participant