Most tests in test/test_core.py and test/test_browser.py test emcc as a black box via CLI invocations. However, parts of test/test_other.py, test/test_sanity.py, and the test harness (test/common.py) directly import and depend on Python compiler internals (tools/*).
If we want to rewrite the compiler (or components like the linker) in another language, or run the test suite against alternative implementations, we need to eliminate these internal dependencies and make the suite purely blackbox.
Breakdown & Estimated Effort
| Area |
Current Coupling |
Proposed Fix |
Effort |
| 1. Global State Mutation |
Tests for -print-* flags in test_other.py mutate options.lto and settings.MEMORY64 to predict paths via cache.get_lib_dir(). |
Assert on CLI output paths directly without altering Python singletons. |
Small (~0.5 day) |
| 2. Binary Type Checks |
test_other.py calls building.is_wasm(), is_wasm_dylib(), is_ar(). |
Inspect magic bytes (b'\0asm') or use llvm-readobj / llvm-ar. |
Small (~1 day) |
| 3. Direct Unit Tests |
test_other.py / test_browser.py unit-test Python functions directly (binary_encode, wasm_sourcemap, response_file, closure_compiler). |
Test via CLI flags or isolate into a dedicated Python unit test directory. |
Small (~1–2 days) |
| 4. Wasm Inspection |
~15 tests in test_core.py and test_other.py use tools.webassembly.Module to check section names/features. |
Replace with standard CLI tools (llvm-readobj, llvm-objdump -h, wasm-dis). |
Medium (~2–3 days) |
| 5. Cache & Ports Introspection |
Tests query cache.get_sysroot_dir() or ports.Ports.get_dir() to find files in the cache. |
Expose paths via CLI (e.g. emcc --print-sysroot) and test ports via --use-port. |
Medium (~2 days) |
| 6. Harness Settings Filtering |
common.py imports COMPILE_TIME_SETTINGS to filter -s flags during compile-only (-c) runs. |
Make compiler ignore/warn on link settings during -c, or query capabilities via CLI. |
Medium (~2–3 days) |
| 7. Sanity Suite Partitioning |
test_sanity.py (~1,000 lines) tests the Python toolchain driver (config.py, cache locking, version checks). |
Separate Python driver tests from the language-agnostic conformance test suite. |
Medium (~1–2 days) |
Total Estimated Effort: ~2 to 3 weeks
Next Steps
- Clean up quick wins in
test_other.py (remove options/settings mutation, replace building.is_* helpers).
- Replace
tools.webassembly calls with standard binary tools (llvm-readobj).
- Decouple harness dependencies (
COMPILE_TIME_SETTINGS) and separate test_sanity.py.
Most tests in
test/test_core.pyandtest/test_browser.pytestemccas a black box via CLI invocations. However, parts oftest/test_other.py,test/test_sanity.py, and the test harness (test/common.py) directly import and depend on Python compiler internals (tools/*).If we want to rewrite the compiler (or components like the linker) in another language, or run the test suite against alternative implementations, we need to eliminate these internal dependencies and make the suite purely blackbox.
Breakdown & Estimated Effort
-print-*flags intest_other.pymutateoptions.ltoandsettings.MEMORY64to predict paths viacache.get_lib_dir().test_other.pycallsbuilding.is_wasm(),is_wasm_dylib(),is_ar().b'\0asm') or usellvm-readobj/llvm-ar.test_other.py/test_browser.pyunit-test Python functions directly (binary_encode,wasm_sourcemap,response_file,closure_compiler).test_core.pyandtest_other.pyusetools.webassembly.Moduleto check section names/features.llvm-readobj,llvm-objdump -h,wasm-dis).cache.get_sysroot_dir()orports.Ports.get_dir()to find files in the cache.emcc --print-sysroot) and test ports via--use-port.common.pyimportsCOMPILE_TIME_SETTINGSto filter-sflags during compile-only (-c) runs.-c, or query capabilities via CLI.test_sanity.py(~1,000 lines) tests the Python toolchain driver (config.py, cache locking, version checks).Total Estimated Effort: ~2 to 3 weeks
Next Steps
test_other.py(removeoptions/settingsmutation, replacebuilding.is_*helpers).tools.webassemblycalls with standard binary tools (llvm-readobj).COMPILE_TIME_SETTINGS) and separatetest_sanity.py.