[NOGIL] Wheel creation and verification jobs by ojasvajain · Pull Request #2352 · confluentinc/confluent-kafka-python

Ojasva Jain (ojasvajain) · 2026-09-08T07:46:34Z

Modified existing wheel creation jobs to also create wheels for Python 3.14t.
Defined free threaded wheel verification jobs for all supported platforms. Smoke testing, done as part of verification jobs, executes all unit test cases.
Some unit tests were found to be failing/flaky on Windows. Those have also been fixed.

Successful job run with wheel verification passing: https://semaphore.ci.confluent.io/workflows/680a3b0c-18db-4c97-a66a-499158b0e9ad?pipeline_id=8b6d3812-11d5-4a63-abca-121b2a77a14b

Checklist

Contains customer facing changes? Including API/behavior changes
Did you add sufficient unit test and/or integration test coverage for this PR?
- If not, please explain why it is not required

References

JIRA:

Test & Review

Open questions / Follow-ups

confluent-cla-assistant · 2026-09-08T07:46:46Z

🎉 All Contributor License Agreements have been signed. Ready to merge.
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

sonarqube-confluent · 2026-09-08T15:06:55Z

Quality Gate failed

Failed conditions
6.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

Kaushik Raina (k-raina)

Thanks for PR! Left couple of comments

Kaushik Raina (k-raina) · 2026-09-10T13:00:45Z

+        # framework gets installed and the cpython314t interpreter is
+        # never created.
+        py_version_digits = cpython_tag[2:-1]
+        choicechanges_xml = os.path.join(script_dir, f"free-threaded-enable-{py_version_digits} .xml")


Suggested change

choicechanges_xml = os.path.join(script_dir, f"free-threaded-enable-{py_version_digits} .xml")

choicechanges_xml = os.path.join(script_dir, f"free-threaded-enable-{py_version_digits}.xml")

Fixed this.

Kaushik Raina (k-raina) · 2026-09-10T13:01:59Z

+        install_args = f"-applyChoiceChangesXML {choicechanges_xml} "
+    os.system(f"sudo installer -pkg {tmp_pkg_file_name} {install_args}-target /")
+
    os.system(install_command)


Is this command redundant?

Yes, fixed.

Kaushik Raina (k-raina) · 2026-09-10T13:03:13Z

-set CIBW_BUILD=cp38-%BW_ARCH% cp39-%BW_ARCH% cp310-%BW_ARCH% cp311-%BW_ARCH% cp312-%BW_ARCH% cp313-%BW_ARCH% cp314-%BW_ARCH%
-set CIBW_BEFORE_BUILD=python -m pip install delvewheel==1.1.4
+set CIBW_BUILD=cp38-%BW_ARCH% cp39-%BW_ARCH% cp310-%BW_ARCH% cp311-%BW_ARCH% cp312-%BW_ARCH% cp313-%BW_ARCH% cp314-%BW_ARCH% cp314t-%BW_ARCH%
+set CIBW_BEFORE_BUILD=python -m pip install delvewheel==1.10.0


Why is this change needed?

The new version of delvewheel excludes python314t.dll from getting vendored in the wheel. The change was introduced in v1.7.3: https://github.com/adang1345/delvewheel/blob/master/CHANGELOG.md#173-6-august-2024 but I upgraded it it to 1.10.0 as it is more recent.

Kaushik Raina (k-raina) · 2026-09-10T13:04:47Z

+"$PYTHON" -c "import sys, sysconfig; assert sysconfig.get_config_var('Py_GIL_DISABLED'), sys.executable + ' is not a free-threaded build'"
+
+echo "# Importing must not re-enable the GIL"
+"$PYTHON" -W error::RuntimeWarning -c "import sys, confluent_kafka; assert not sys._is_gil_enabled(), 'import re-enabled the GIL'"


-W error::RuntimeWarning Will it check all warnings or just GIL warning?

All warnings (including GIL). But that should be fine as there are only 2 lines of code that we are executing.

Kaushik Raina (k-raina) · 2026-09-10T13:11:00Z

+            - uv venv _venv314t --python 3.14t && source _venv314t/bin/activate
+            - uv pip install -r requirements/requirements-tests-install-nogil.txt
+            - artifact pull workflow artifacts
+            - cd artifacts && ls *.tgz |xargs -n1 tar -xvf && cd ..


Should we fail loudly if there are no artifacts?

I have added a check for no artifacts in all free-threaded verify jobs.

Kaushik Raina (k-raina) · 2026-09-10T13:13:15Z

        a.elect_leaders(correct_election_type, [correct_partitions]).result(timeout=1)


+def _poll_until_callback_raises(admin, timeout_s=5.0, step_s=0.2, ignore=()):


Should we move this to helper? Currently used in test_Consumer.py also

Thanks, added a helper.

Kaushik Raina (k-raina) · 2026-09-10T13:14:39Z

+            - export ARCH=x64
+            # sem-version only provides a pip to bootstrap uv; the wheel runs in the 3.14t venv uv creates.
+            - sem-version python 3.11
+            - pip install uv


Should we move common steps to helper script to reduce future maintenance?

Thanks. I created a helper script for linux and macos free threaded verify jobs.

Kaushik Raina (k-raina)

LGTM! Thanks Ojasva Jain (@ojasvajain)
On Nit

Kaushik Raina (k-raina) · 2026-09-21T09:46:08Z

            - artifact pull workflow artifacts
            - cd artifacts && ls *.tgz |xargs -n1 tar -xvf && cd ..
            - tools/test-wheels.sh artifacts/wheelhouse
+        - name: Verify (free-threaded)


nit:

Suggested change

- name: Verify (free-threaded)

- name: Verify free threaded

Kaushik Raina (k-raina) · 2026-09-21T09:46:23Z

            - artifact pull workflow artifacts
            - cd artifacts && ls *.tgz |xargs -n1 tar -xvf && cd ..
            - tools/test-wheels.sh artifacts/wheelhouse
+        - name: Verify (free-threaded)


Same as above

sonarqube-confluent · 2026-09-22T08:22:22Z

Quality Gate passed

Issues
3 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

Ojasva Jain (ojasvajain) self-assigned this Sep 8, 2026

Ojasva Jain (ojasvajain) added the nogil label Sep 8, 2026

Ojasva Jain (ojasvajain) requested review from a team and Matthew Seal (MSeal) as code owners September 8, 2026 07:46

Kaushik Raina (k-raina) requested changes Sep 10, 2026

View reviewed changes

airlock-confluentinc Bot force-pushed the dev_thread_free_support branch from aa81db7 to 9737e64 Compare September 18, 2026 10:01

airlock-confluentinc Bot force-pushed the dev_thread_free_wheels_ci branch from 90791ad to 0f7c552 Compare September 21, 2026 06:36

Base automatically changed from dev_thread_free_support to master September 21, 2026 10:53

Ojasva Jain (ojasvajain) added 5 commits September 22, 2026 11:29

Define Wheel creation and verification jobs for free threading

1b29611

Fix windows wheel verify job

2172822

Fix UTs failing/flaky on windows

5379198

Fix flaky test cases on windows

346372c

Address comments

a93a0e6

airlock-confluentinc Bot force-pushed the dev_thread_free_wheels_ci branch from c394eb1 to a93a0e6 Compare September 22, 2026 06:00

Kaushik Raina (k-raina) reviewed Sep 22, 2026

View reviewed changes

Slightly rename the verify jobs

d43ac14

Kaushik Raina (k-raina) approved these changes Sep 22, 2026

View reviewed changes

Ojasva Jain (ojasvajain) merged commit d9f08ba into master Sep 22, 2026
4 checks passed

Ojasva Jain (ojasvajain) deleted the dev_thread_free_wheels_ci branch September 22, 2026 09:37

	choicechanges_xml = os.path.join(script_dir, f"free-threaded-enable-{py_version_digits} .xml")
	choicechanges_xml = os.path.join(script_dir, f"free-threaded-enable-{py_version_digits}.xml")

		a.elect_leaders(correct_election_type, [correct_partitions]).result(timeout=1)


		def _poll_until_callback_raises(admin, timeout_s=5.0, step_s=0.2, ignore=()):

Conversation

Ojasva Jain (ojasvajain) commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

References

Test & Review

Open questions / Follow-ups

Uh oh!

confluent-cla-assistant Bot commented Sep 8, 2026

Uh oh!

sonarqube-confluent Bot commented Sep 8, 2026

Quality Gate failed

Uh oh!

Kaushik Raina (k-raina) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ojasva Jain (ojasvajain) Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ojasva Jain (ojasvajain) Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kaushik Raina (k-raina) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqube-confluent Bot commented Sep 22, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Ojasva Jain (ojasvajain) commented Sep 8, 2026 •

edited

Loading

Ojasva Jain (ojasvajain) Sep 21, 2026 •

edited

Loading

Ojasva Jain (ojasvajain) Sep 21, 2026 •

edited

Loading