[mihncalo]

Summary

The protobuf version in the generated et_def_pb2.py and the version of the protobuf package installed at runtime are chosen by two separate mechanisms. Nothing was tying them together. If the former were to be a newer version than the latter, it was possible that the generated python could come with functionalities that the runtime protobuf older version couldn't read. The first time anything imports et_def_pb2.py, the file asks the runtime package for its version and compares it with the version stamped in its own header. In the above scenario, this test was failing, leading to the error from the Issue.

The solution is to follow the rule: The gencode protobuf version needs to be older ( or at most equal ) to the runtime protobuf version. In order to achieve this, I pinned the compiler version and set the runtime floor to the gencode version. This means the two can't disagree.

The official protobuf documentation supports this decision: "New Gencode + Old Runtime = Never Allowed". For Python: "As of 6.32.0, all generated code since 3.20.0 will be supported until at least 8.x.y."

Test Plan

In the current setup, pip install . resolves protobuf 7.36.2, gencode is 6.31.1, and chakra_generator runs with exit 0 and no error.
Bumping grpcio-tools to a newer version without properly updating the floor fails with VersionError.
Lowering the floor fails with VersionError.

The smoke test introduced in the workflows ( python_tests.yml -> Protobuf Gencode/Runtime Smoke Test ) checks both the lower and the upper boundaries to make sure that the 2 versions are in sync. If they are not ( someone updated a pin but not the other ), the test fails.

Additional Notes

The two pins must be bumped together by hand, otherwise they will get out of sync ( catched by the smoke test ).
On Python 3.15, grpcio-tools 1.80.0 has no wheel, so a source build compiles it, which is slow. Not a concern for now, but could be addressed in later issues, once the repo hardening is done.

Closes #195

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[ajbalogh]

lgtm