Repository navigation
Phase 8.2 WP1 (Java): decode-baseline workers, launcher and session report - #89
Merged
Merged
Conversation
…eport Java side of the frozen section 13 contract (PR #87 is the Python reference; this branch is one verifiable unit of plan section 3.2): - jmlx-models test sources: BaselineAdapters (benchmark-only controlled and stock generation/step adapters, never published API), BaselineWorker (the process worker: anchored job validation, manifest re-hash, safetensors set pinning, claim/ready/report protocol, per-request priming, wall-clock watchdog with grace backstop) and BaselineWorkerTestSupport; driven through the new GenerationRequest.pretokenized seam - jmlx-benchmarks: BaselineBenchmark (spec validation, sequential fork, session report), BaselineJob (frozen job file + fork commands), BaselineReportValidator (adapter-aware report validation) and BaselineStatistics (type-7 quantiles), exposed only through the opt-in :jmlx-benchmarks:benchmarkBaseline task; benchmarkDecode, benchmarkBatchDecode and benchmarkTf32 are preserved, and ordinary check gains no downloads or Python execution - Java deviations from the Python worker (findings section 14.1): the launcher owns MLX_ENABLE_TF32 on both worker processes and the Java worker verifies it (JDK 25 removed System.setenv); the no-replace report write is an explicit existence refusal under REPORT_LOCK (Files.move ATOMIC_MOVE would replace); identity and finish-reason label mapping - Session report (findings section 14.3): mode-scoped per_runtime and first_request type-7 quantiles; reference-pair (observed, unprimed) samples excluded from the headline quantiles entirely — they belong to the WP3 pairs aggregation; outcome stubs plus the exclusion list; launcher exit 0/1/2 - Evidence: 59 pure-Java launcher tests (hermetic mini repository plus a fake starter simulating every protocol outcome), 32 jmlx-models tests (the three suites added to the macOS CI required-suite list), and an opt-in 0.6b session against the frozen manifest (both workers ok; headline ttft median 0.006971979 s over 20 measured requests, findings section 14.6)
…ncher limits and worker hardening - Summary cells are adapter- and mode-scoped on per_runtime and first_request (controlled/stock and headline/fixed-continuation never pool; Medium reproduced: the mixed 0.6b run's median TTFT 0.535 s sat between the two adapters' own medians) - Launcher: run_policy validated without defaults, ok-report-vs-job cross-check (invalid-report with violations on mismatch), fork wait limit min(timeout_s + 90 s, cell_time_budget_s) with kill plus 10 s settle, session time budget records the remainder not-started, and the frozen-manifest digest pin refuses an edited checkout (backstop test included) - Worker: per-run watchdog token (a stale watchdog drops every remaining action), the stock stop-set assertion unions the tokenizer's special-token eos, identity flags come from the JVM input arguments on both sides, and the stock drain note is corrected (the synchronous loop queues no lookahead, so no drain exists or is needed) - Fixed-continuation cells gain step_s (per-step, measured requests) - Tests: BaselineBenchmarkTest 39 e2e, BaselineWorkerProtocolTest 20 (no new suite classes, the CI required-suite list is unchanged); findings section 13.4 item 16, section 14 amended, 14.7 added; jmlx-benchmarks README key-shape wording
…ering - The stale-watchdog two-run test no longer relies on a sleep margin: run A is held on a one-shot requestHold latch (consumed by the held run's first request) and released only after run B's report is on disk; every A-vs-B ordering step is observed with bounded polls (worker-ready, latch consumption, watchdog fire) via two new read seams. A's post-loop failure handling can no longer race B's report write on a slow runner. - resetRunState() now runs inside the REPORT_LOCK critical section, after the run-token increment, so a stale watchdog's grace action landing in that gap can no longer observe the fresh state under the old token and mark a healthy new run timeout. - Stop-set assertion javadoc corrected: the stock path feeds the generation loop only the manifest frozen set; the tokenizer's eos is in the resolved set because the manifest derives the frozen set from the artifact's stop sources, tokenizer included. Same correction in the stop-set test comment. - Session budget overrun documented in the launcher javadoc: the budget gates new forks, not in-flight workers. - Findings: section 13.4 item 17 and new section 14.8; item 16(a)/ 16(b) and the section 14.7 Medium bullet corrected in place (fake-worker mini reproduction, not a 0.6b run; numbers unchanged). No contract, schema or measurement-path change; no opt-in session re-run needed.
…old on failed checks - Findings item 17(a) mischaracterized the old race twice (corrected in place, and in §14.8/the test comment): A's resumed request woke BEFORE B wrote its report, and A's poisoned report was a failed report from its error handler (the resumed request throws on the closed scope, so the post-loop timeout path is never reached), not a timeout report — B's own report write was then refused and B returned failed. - A failed check before the latch release left run A blocked on the latch for the rest of the test JVM: the cross-run test now releases the hold in a try/finally, run A is a daemon thread, and resetTestSeams releases any outstanding latch. Findings: section 13.4 item 18 and new section 14.9. Test-only; no contract, schema or measurement-path change.
…cription Item 18(a) and section 14.9 said A's resumed request in the old race "throws on the closed scope"; that is true only in the current test, where B has already finished. In the old race B was still running, so the scope was B's live one and only B's thread could use it (thread confinement, as the round-1 test comment said). Corrected in place; the conclusion (A's request throws, the post-loop timeout path is never reached) is unchanged. Findings: section 13.4 item 19 and new section 14.10. No code, contract, schema or measurement-path change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Java side of the frozen section 13 contract (PR #87 is the Python reference implementation; this branch is one verifiable unit of plan section 3.2). Full record in
req/plans/phase8-2-reference-findings.mdsection 14.What lands
jmlx-modelstest sources (benchmark-only, never published API):BaselineAdapters— the controlled and stock generation/step adapters, driven through the newGenerationRequest.pretokenizedseam.BaselineWorker— the process worker mirroring the frozen protocol: anchored job validation (manifest re-hash, repo anchoring, safetensors set pinning), claim/ready/report protocol, per-request priming (reference-pair primes nothing), wall-clock watchdog with grace backstop. Exit 0 ok / 1 failed / 3 timeout, same as the Python worker.BaselineWorkerTestSupport+ three test classes (5 / 18 / 9 tests), all added to the macOS CI required-suite list.jmlx-benchmarksmain sources:BaselineBenchmark(spec validation, sequential fork in spec order, session report),BaselineJob(frozenjmlx-decode-baseline-job-1file + fork commands),BaselineReportValidator(adapter-aware validation ofjmlx-decode-baseline-1),BaselineStatistics(type-7 quantiles).:jmlx-benchmarks:benchmarkBaselinetask only.benchmarkDecode,benchmarkBatchDecodeandbenchmarkTf32are preserved; ordinarycheckgains no downloads or Python execution.Java-specific deviations (findings §14.1)
System.setenv: the launcher ownsMLX_ENABLE_TF32on both worker processes and the Java worker verifies the process environment equals the jobprecision_envbefore any native initialization, failing the run on mismatch.Files.move(ATOMIC_MOVE)isrename(2)and silently replaces: the no-replace report guarantee (Python gets it fromos.link) is an explicitFiles.exists(report.json)refusal under the worker'sREPORT_LOCK.identity.devicefrom sysctl,identity.java {version, vendor, flags}; EOS/STOP_TOKEN→stop, MAX_TOKENS→length, CANCELLED→cancelled; Java stockunused_lookahead_forwards: 0;load_state: "materialized".Session report decisions (findings §14.3)
per_runtime[runtime]["<mode>/<case>/<len>"]andfirst_request[runtime][mode]type-7 quantiles — headline and fixed-continuation never pool.per_runtimeentirely — they belong to the WP3pairsaggregation. (An early unmodeled run pooled one such sample into the 0.6b headline cell, n=21 vs 20; the fixture tests pin the rule.)ok/failed/timeout(valid report),invalid-report(raw report embedded),dead,crashed,not-started; exclusion list in the summary. Launcher exit 0 = all ok, 1 = spec validation (nothing forked), 2 = any non-ok (session.jsonstill written). A starter exception stops the session.Evidence
:jmlx-benchmarks:check,:jmlx-models:checkand rootcheckgreen on this head.okin ~20 s; headlineqwen3-0.6b/32 — 26 requests (1 first + 5 warmups + 20 measured), 128 emitted, load 0.042 smaterialized, ttft {p10 0.00584, median 0.00697, p90 0.01367 s, n 20}, streaming ≈ 226.6/228.4/230.3 tok/s, first ttft 0.121 s; reference-pairok/observed/1 measured/prime_s 0.0; workload shae8c77301…matches the frozen manifest digest; native pins metal 0.32.4 / mlx-ca26f2d5.