Epic: expand TRTMC AIPerf model and workload coverage #1667

Activity

1. yizhuoz004

   self-assigned this

   on Oct 13, 2026
2. yizhuoz004

   added the

   Epic

   issue type on Oct 13, 2026
3. yizhuoz004

   added

   enhancementNew feature or requestNew feature or request

   on Oct 13, 2026
4. yizhuoz004

   added sub-issues on Oct 13, 2026

   • AIPerf stage 1: qualify all TRTMC single-sentence embedding and encoding models #1668
   • AIPerf stage 2: qualify all TRTMC text reranking models #1669
   • AIPerf stage 3: qualify all TRTMC recorded-clip transcription models #1670
   • AIPerf stage 4: qualify all TRTMC single-image object detection models #1671
   • AIPerf stage 5: qualify GB300 single-image vision-language models #1672
   • AIPerf stage 6: qualify all TRTMC text-to-image models #1673
   • AIPerf stage 7: qualify all TRTMC reference-image editing models #1674
   • AIPerf stage 8: qualify all TRTMC text-to-video models #1675
   • AIPerf stage 9: qualify all TRTMC audio-language question-answering models #1676
   • AIPerf stage 10: qualify all TRTMC video-language question-answering models #1677
   • AIPerf text stage A: qualify exact real-prompt inputs #1678
   • AIPerf text stage B: qualify multi-turn chat #1679
   • AIPerf text stage C: qualify production trace replay #1680
   • AIPerf text stage D: qualify controlled shared-prefix workloads #1681
   • AIPerf text stage E: qualify stateless Responses API #1682
   • AIPerf text stage F: qualify a speculative-decoding prompt corpus #1683
5. yizhuoz004 commented on Oct 13, 2026

   

   yizhuoz004

   on Oct 13, 2026

   CollaboratorAuthor

   More actions

   When this epic is complete, TRTMC should cover AIPerf's main model categories through selected, validated models and workloads. Full coverage of every model, dataset, and API option will still require follow-up work. The following entries are planned coverage, not newly completed qualifications.

   Planned model coverage

   Model category	Planned pilot model	Initial workload
   Text embeddings	BAAI/bge-small-en-v1.5	Sentence → vector
   Text reranking	nvidia/llama-nemotron-rerank-vl-1b-v2	Query + text passages → scores
   Speech recognition	openai/whisper-tiny	Recorded audio → transcript
   Object detection	YOLO11n	Image → labelled bounding boxes
   Image understanding	OpenGVLab/InternVL3-2B-hf	One image + question → answer
   Image generation	FLUX.1-schnell	Prompt → image
   Image editing	Qwen-Image-Edit-2511	Reference image + instruction → edited image
   Video generation	Wan2.1-T2V-1.3B	Prompt → video
   Audio understanding	Pilot still to be selected and natively validated	Audio + question → answer
   Video understanding	Pilot still to be selected and natively validated	Video + temporal question → answer

   Audio/video understanding cannot be considered complete until genuinely conditioned native Tasks and their pilot models have been selected, implemented where necessary, and qualified. The currently inspected Qwen3-Omni text-only path does not establish these capabilities.

   Existing text-generation coverage

   The existing text-generation workflow remains the foundation. The earlier campaign recorded in PR #1606 benchmarked these ten checkpoints through non-streaming /v1/completions:

   • Qwen/Qwen3-0.6B
   • openai-community/gpt2
   • facebook/opt-125m
   • openai/gpt-oss-20b
   • Qwen/Qwen3-4B-Instruct-2507
   • distilbert/distilgpt2
   • Qwen/Qwen3-30B-A3B
   • TinyLlama/TinyLlama-1.1B-Chat-v1.0
   • openbmb/MiniCPM5-2B
   • HuggingFaceTB/SmolLM3-3B

   These are historical benchmark results under the specific configurations documented in that PR. The ten-model campaign was not rerun after the rebase; it does not establish every model's support for every new workload or API option.

   The six text-workload stages use Qwen3-0.6B as their initial pilot:

   1. Exact real prompts from a fixed corpus.
   2. Multi-turn chat.
   3. Production trace replay.
   4. Controlled shared-prefix inputs.
   5. Stateless Responses API.
   6. A speculative-decoding prompt corpus, initially usable as ordinary generation.

   Qualifying these workloads on Qwen3-0.6B does not automatically qualify them on the other nine text models. Shared-prefix inputs do not prove cache reuse, and speculative-decoding datasets do not implement draft/target execution or acceptance telemetry.

   Comparison with AIPerf

   The target is similar coverage of the main model categories in AIPerf's Endpoint Types section, with a narrower initial selection of models and workloads. Full parity with Workloads and Data is outside the current 16 stages.

   Remaining follow-up coverage includes:

   • Additional models and datasets within each category.
   • Other API formats, such as Cohere rankings, HF TEI rankings, and NIM embeddings.
   • Batched inputs, multiple images, and streaming ASR or multimodal workloads.
   • Stateful Responses, tool calls, genuine speculative execution, and verified cache reuse.
   • Comprehensive load/capacity qualification across the new groups. AIPerf's client load generators remain available; each server workload still needs appropriate validation.

   AIPerf supplies the benchmark client; TRTMC supplies model execution and determines which native capabilities are available. Other built models may reuse a compatible endpoint, but each needs its own family correctness and actual AIPerf evidence before it is listed as qualified. Implementation should continue one model group and one workload at a time, using the published pip-installed AIPerf package.
6. yizhuoz004 commented on Oct 13, 2026

   

   yizhuoz004

   on Oct 13, 2026

   Last edited by yizhuoz004

   CollaboratorAuthor

   More actions

   Stages 1–4 are delivered to main: PR #1718 and PR #1753. All four group tasks (#1668, #1669, #1670, #1671) are closed and remain assigned to yizhuoz004. Stage 4's final merge is 9643800b3b5a951382c87ae1ebfbbeb24b1a8090.

   Coverage is now 41 canonical checkpoints: 22 embedding/encoding models, one dedicated text reranker, eight whole-clip ASR models and ten single-image object detectors. Every checkpoint passes its unchanged owning-family case and a real pip-installed AIPerf 0.13.0 run for its selected workload. The original 40-checkpoint results retain their actual tested heads and bundle identities in #1718; they are not relabelled as measurements on the new head. The additional Deformable DETR result is measured on cf93a041d8c0660f2e3e3ad0a4e4008b70cfd26c and reported with all ten detector rows in #1753. Extra precision/TP variants remain separately inventoried and unqualified by these canonical runs.

   Deformable DETR completion: corrected native aspect-preserving resize and dynamic geometry; the original fixture/reference and every numerical gate remain unchanged. The actual FP16/TP1 bundle passes native checkpoint parity, independent landscape/portrait/square geometry checks, all 100 complete native/HTTP comparisons, and all 55 client responses (five warmups plus 50 measured requests): 463 complete boxes / 2,315 coordinate and confidence values, zero errors or missing request/metric joins. The canonical FP32 reference remains unchanged.

   New detector / one original RGB image, concurrency 1	HTTP p50 / p95 ms	Public Task p50 / p95 ms	Images/s
   SenseTime/deformable-detr	80.65 / 83.72	71.36 / 74.04	12.89

   On the same 100 pinned COCO images, native/reference mAP@[.50:.95] is 46.8307% / 46.8189% and mAP@.50 is 62.8337% / 62.6492%. Both original maximum-drop gates remain 0.02 and pass. Client end-to-end latency and public_task_call_wall are separate; verification, build/load/startup and warmup are excluded from measured timing. One shared NVIDIA GB300, FP16, published AIPerf 0.13.0, TensorRT 11.1.0.106. These short concurrency-1 runs establish no capacity or performance gate. Actual served bundle SHA-256: 4001c40acaaac239e6c65f2f8eb6f38985c07f943bee200447a2311520f7879e. Full environment, pins and commands are in #1753 and the usage guide.

   Final-head validation: all expected latest checks pass on cf93a041d8c0660f2e3e3ad0a4e4008b70cfd26c: Stable and Dev Community CI, metadata, DCO, automated review, both dispatchers and the protected internal premerge gate. Each CPU lane reports 1,550 Python passes, two existing skips and 262 native passes. Dev's actual selected family/case passed with no deferred cases, including the dynamic geometry regression (checkpoint box IoU 0.9960, score delta 0.0081). GPU deletion and independent cleanup are confirmed. No review conversations remain unresolved. Only sanitized protected success is reported here; private evidence remains private.

   Remaining scope: stages 5–10 and the six separate text-generation workload stages remain open. Whole-clip ASR does not establish streaming HTTP; canonical single-request profiles do not qualify other precisions, TP variants, batching or capacity sweeps. Rebuild older Deformable DETR bundles for the corrected resize metadata and dynamic plan. The integration continues to reuse trtmc-server and the PyPI client. The benchmark container is stopped, artifacts are preserved outside source commits, and no separate Rubin reservation was identified; actual benchmark hardware was GB300.

   Historical 40-checkpoint campaign before Deformable DETR completion

   Group-wide delivery — PR #1718 is merged: merge commit e8f74e7e06bc463a66dc01530da05f1be3a1f3f1. Stages 1–3 (#1668, #1669, #1670) are closed and remain assigned to yizhuoz004. Stage 4 (#1671) stays open and assigned to yizhuoz004 for the additional Deformable DETR checkpoint.

   The completed campaign covers 22 embedding/encoding models, one dedicated text reranker, eight whole-clip ASR models and nine object detectors (40 checkpoints), using one normal full-model profile per checkpoint. All 40 pass their selected owning-family cases and real pip-installed AIPerf 0.13.0 runs: five warmups plus 50 measured requests at concurrency 1 per checkpoint. Complete native outputs and request/timing identity are checked for every corpus input and all 55 client responses, with zero client errors or missing joins. The PR contains all 40 latency/throughput rows, actual measurement heads, checkpoint pins, served-bundle hashes, environment and reproducible commands. Twenty-one extra TP profiles are inventoried separately and are not claimed as qualified.

   All eight ASR and nine detector corpus checks/reports are complete. RT-DETR's one-line metadata correction (coco-category-id → coco-contiguous-80) matches its actual output IDs. The same saved outputs on the same 100 images give native/reference mAP@[.50:.95] 53.6812% / 53.7066%; model outputs, references and both existing 0.02 maximum-drop gates are unchanged. Its public contract and direct native/reference E2E comparison do not use that YAML label metadata. Parakeet and YOLOS have no existing dataset gates; their corpus metrics remain separate from their passing owning fixture contracts.

   Exact final-head validation: all expected checks pass on 2f514da846d7843324d1ba6db1f9dd7c3407120b: Stable and Dev Community CI, source quality, documentation, ownership/impact, metadata, DCO, automated review, both dispatchers and the protected internal premerge gate. Each CPU lane reports 1,628 Python cases passed, two existing skips, and 258 native CPU cases passed. Dev GPU validates all seven selected families and all 12 requested cases, including Nemotron and RT-DETR; allocation deletion and independent cleanup are confirmed. All six review conversations are fixed and resolved. The reproduced Nemotron dependency-image conflict is fixed by a family-local compatible virtualenv pin. Request/stream prompt isolation and Unicode JSONL regressions remain covered. The 849-case local integration suite retains its actual preceding-head attribution; original model timings are not relabelled as measurements on the merge head.

   Remaining coverage: main added SenseTime/deformable-detr after the campaign, bringing the stage 1–4 inventory to 41 checkpoints / ten detectors. It has not been benchmarked and remains tracked in #1671. Stages 5–10 and the six text-generation workload stages remain open. Whole-clip ASR qualification does not establish HTTP streaming, and these short concurrency-1 runs establish no capacity or performance gate.

   The integration reuses trtmc-server and the published PyPI client. The benchmark container remains stopped at the owner's request and generated evidence is preserved outside source commits. Actual benchmark hardware was GB300; earlier references to Rubin were incorrect, and no separate Rubin reservation was identified.

   Historical pilot deliveries and their original measurements

   Stage 1 is delivered to main by #1684 and #1668 is closed. All community/review checks and the protected internal premerge gate passed on 27a34598d6cca133b419a315ba6be7a8f3ae02ea; merge commit: c7bd3db87b82e7b853a7547df344cb0af43d5a3f.

   Stage 2 is delivered to main by #1687 and #1669 is closed. All expected Stable/Dev Community checks (including GPU validation and allocation cleanup), documentation, source quality, metadata, DCO, automated review, and the protected internal premerge gate passed on fb3ed80718b8a7b8f76e2f431cea2eacc0032629. All review conversations are resolved. Merge commit: e3eeb98f2e5ba1f0ceaf40126d07a3ed02257a2e. Text generation and embeddings remain available alongside the new text reranking endpoint.

   Pilot / workload	HTTP p50 / p95 (ms)	Native Task p50 / p95 (ms)	Successful requests/s	Passages/s
   BAAI/bge-small-en-v1.5, one sentence	2.56 / 2.78	1.59 / 1.68	332.34	—
   nvidia/llama-nemotron-rerank-vl-1b-v2, one query / two text passages	3.60 / 3.76	2.81 / 2.95	253.08	506.16

   Both pilots were run serially through the combined server on fb3ed80718b8a7b8f76e2f431cea2eacc0032629. Each pilot completed 50/50 measured requests plus 5/5 warmups, concurrency 1, one persistent replica, using the checked-in fixed corpus and published AIPerf 0.13.0. All 55 BGE responses retained complete finite 384-value vectors; all 55 reranking responses retained real finite scores and original passage indices. Client/server request IDs and metric exports all matched, with zero client errors. Direct HTTP cases matched the same bundles' native operations exactly. Throughput uses the measured client request span, excluding warmups. This small run is an integration example, not a statistical performance or capacity gate; the passage list does not establish parallel GPU batching.

   The actual bundles passed their unchanged owning-family criteria: bert CLS cosine >= 0.999 and eagle_vlm strict ranking agreement 1.0. The server preserves family-produced values and rejects malformed output, including inconsistent native embedding dimensions. In stages 1–2, no family oracle, numerical threshold, mathematical topology, Task ABI, or bundle format changed.

   One NVIDIA GB300, FP16, TensorRT 11.1.0.106, CUDA toolkit 13.3, driver 580.105.08, Linux aarch64, Python 3.12.3. BGE: context 512, checkpoint 5c38ec7c405ec4b44b94cc5a9bb96e735b38267a; Nemotron: context 685, checkpoint 9e95da054312436dfc703319dd2b793a3bee2465. Client loopback latency and public_task_call_wall are separate; measured requests exclude build/load/startup/warmup, and no generation TTFT/ITL is used.

   Actual served bundle SHA-256 identifiers:

   • BGE: efe9960e76e20be00a8cf4dbe697992fa8e78bc408ba4e8d656e109686b141f0
   • Nemotron: 82309a302b674fae84b1418e660a1876981174c8f82f5acdbcc6311d131acda7

   The three-step usage guides use the PyPI client and existing build guidance. Other embedding/reranking profiles and the remaining 12 epic stages still need their own qualification. Generated reports and artifacts remain outside the commits.

   Stage 3 is delivered to main by #1707 and #1670 is closed. All expected checks, including the protected internal premerge gate, passed on 572e28db766daa0279e4dbec90e7f615420d732c; merge commit: 0524d694386810b0b5dadf29cce15239d230b959. Review conversations are resolved.

   The recorded-audio pilot uses openai/whisper-tiny, /v1/audio/transcriptions, and complete mono PCM16 WAV clips. The rebuilt bundle passed the unchanged strict native/reference transcript case and owning-family WER criteria on ten complete LibriSpeech clips: 0.0 WER to the FP32 reference; 3.59% WER to gold for both native and reference. All 55 published AIPerf responses matched their original input's complete native transcript, with zero errors or missing request/timing joins. Five warmups and ten separate verification calls are excluded from the 50 measured requests.

   Pilot / workload	HTTP p50 / p95 (ms)	Public Task p50 / p95 (ms)	Requests/s	Media throughput
   Whisper tiny, complete recorded clips, concurrency 1	67.99 / 199.53	66.01 / 196.83	11.52	105.42 audio seconds/s

   The selected clips contain 91.525 audio seconds; the measured requests contain 457.625 audio seconds. Throughput uses the measured client span. Per-request RTF p50 is 0.00975 client / 0.00937 Task. Environment: one GB300, TensorRT 11.1.0.106, CUDA 13.3, driver 580.105.08, Python 3.12.3, published AIPerf 0.13.0. Effective precision is FP16 with the profile's FP32 encoder selector 0 and family-owned FP32 normalization/GELU/output projection. Checkpoint revision: 169d4a4341b33bc18d8881c4b69c2e104e1cc0af; actual served bundle SHA-256: 4b79f533903ca8b0a060298c427effe3c3e9fab640f09a7fb66c115647e4f9d1.

   Whisper's checkpoint GELU mapping and comparison-sensitive arithmetic were corrected inside its owning family. Rebuild older Whisper bundles. The fixture, reference, zero-NED/zero-WER gates, Task ABI, and bundle format are unchanged. Streaming ASR, stereo/other codecs, translation, and audio QA remain unqualified. The three-step guide uses the PyPI client and existing build guidance. This remains a short integration example, not a performance or capacity gate.

   Stage 4 is delivered to main by #1713 and #1671 is closed. All expected current-head checks, including Dev Community GPU validation/cleanup and the protected internal premerge gate, passed on 57a992a38ad7e9173bbccf135a2ebe4349f3f4fc; merge commit: 90afeece5c210c7c12ed9f19b061c40318862986. Both review conversations are resolved.

   The single-image object-detection pilot uses YOLO11n FP16 and NIM-shaped /v1/infer through AIPerf's image_retrieval endpoint. The actual rebuilt bundle passes the unchanged owning-family checkpoint/reference case and 100-image COCO gates: native/reference mAP@[.50:.95] 39.4663% / 39.5096%, mAP@.50 50.7374% / 50.7170%; maximum drops remain 0.02/0.02. All 100 separate HTTP/native verification calls preserve the complete native detections. All 55 published AIPerf responses preserve original image bytes, numeric class IDs, every box and score: 325 boxes checked, zero errors or missing request/timing joins.

   Pilot / workload	HTTP p50 / p95 (ms)	Public Task p50 / p95 (ms)	Images/s
   YOLO11n FP16, one original RGB image, concurrency 1	15.83 / 20.13	7.52 / 9.29	60.17

   The published AIPerf 0.13.0 run uses one persistent replica, five warmups and 50 measured requests. Throughput uses the measured client span; build/load/startup, 100 verification calls and warmup are excluded. Task timing excludes transport image decoding and includes the family's preprocessing, inference and postprocessing. This short integration example establishes no performance or capacity gate.

   Environment: one GB300, Linux aarch64, TensorRT 11.1.0.106, CUDA toolkit 13.3, driver 580.105.08, Python 3.12.3. YOLO11n is FP16, TP 1, score threshold 0.25, class-aware NMS IoU 0.7; reference is FP32. Checkpoint Ultralytics/YOLO11 revision 8b8ac7d1fae7468f85dbf89670dd66f41f485aab; COCO revision cf0b22332314a937e9dc8a1957b21725430bb41d. The pinned selection has 100 original RGB validation images and one audited grayscale skip, with no input transforms. Actual served bundle SHA-256: edd06d0ef595fda55af9493e5fbd6885c2d71a926ae79a2f539ca0e309b43b82.

   Native serving tests: 7 passed; Python serving/transport: 178 passed; source quality: 307 passed; website inventory and production build passed. Image preparation runs off the event loop, enforces the pixel limit before full decoding, rejects damaged pixel data, and cleans up cancelled writes safely. The three-step guide uses the PyPI client and existing build guidance. Other detectors, document/table layout, non-RGB inputs, multiple images, streaming and capacity sweeps remain unqualified. No family implementation, oracle, numerical threshold, Task ABI or bundle format changed in stage 4.
7. yizhuoz004

   added 2 commits that reference this issue on Oct 15, 2026

   feat(aiperf): profile complete native model groups

   ...

   bb65dd1

   feat(aiperf): profile complete native model groups

   ...

   5baa7bb
8. yizhuoz004 commented on Oct 17, 2026

   

   yizhuoz004

   on Oct 17, 2026

   Last edited by yizhuoz004

   CollaboratorAuthor

   More actions

   Stage 5 complete: PR #1761 merged as 88ceb9f5217a9f9318e3393a404761e09891194c from bdf62ed9da1ba9909f20c0bb4b549bacb63c2ce3. All nine existing normal native GB300 vision-language profiles passed unchanged owning-family correctness and real PyPI AIPerf qualification (450 measured +45 warmup requests; all495 complete native/HTTP matches; zero client errors). The unchanged required Internal CI gate passed. All seven existing premerge cases passed at 1495d31 with no requested skips; model/runtime/test source is unchanged by the final CI-only fix, and the full Lance case passed again on the final source with compatible dependencies and its unchanged reference. The nine-model AIPerf timing table retains the actual measurement source epochs. No contract, model spec, threshold, test eligibility or repository merge rule was changed. Current-head source/CPU/docs/review checks passed; experimental public GPU status is recorded separately and is not relabelled as a pass.

   Existing profile	Corpus parity	HTTP p50 / p95 ms	Task p50 ms	Images/s
   internvl3-2b	9/10	304.32 / 353.62	299.35	3.344
   internvl3-8b	10/10	455.51 / 496.78	450.08	2.296
   qwen25vl-3b	10/10	74.78 / 81.31	71.10	13.218
   qwen3-vl-2b	10/10	60.61 / 180.63	55.97	9.441
   qwen3-vl-8b	10/10	89.73 / 239.72	85.54	6.880
   phi4-multimodal	9/10	100.82 / 110.81	95.17	9.745
   lance-3b-x2t-image	10/10	74.42 / 82.65	68.88	13.048
   locateanything-3b	10/10	155.38 / 405.91	149.42	4.093
   deepseek-ocr	9/10	116.28 / 119.93	113.10	8.578
9. yizhuoz004

   added a commit that references this issue on Oct 18, 2026

   feat(aiperf): profile existing GB300 vision-language models

   ...

   88ceb9f
10. yizhuoz004 commented on Oct 18, 2026

    

    yizhuoz004

    on Oct 18, 2026

    CollaboratorAuthor

    More actions

    Stages 7 and 8 are complete and merged.

    Stage 7: #1792 supports the existing full Qwen/Qwen-Image-Edit-2511 reference-image editing checkpoint. Its unchanged original full case and actual published AIPerf qualification passed: five measured requests plus one warmup, six complete native/HTTP PNG matches, zero client errors.

    Stage 8: #1802 merged as 61d2402f61e77238b250729ecace9ac13d038df7, tested head 33cfdae11cc80597c694b14485b01bc01943c441. All original full owning-family cases passed on the exact served bundles (one executed model case each, zero skips). Real published AIPerf 0.13.0 completed two measured requests plus one separately retained warmup per model, concurrency 1.

    Existing full profile	Complete clips / frames	HTTP p50 / p95 (s)	Task p50 (s)	Measured clips/s
    wan21-t2v-1.3b	3 / 51	4.0699 / 4.0712	2.5341	0.244931
    wan22-ti2v-5b	3 / 363	76.6237 / 77.2971	57.0793	0.013032
    minimax-h3-768p	3 / 372	159.3559 / 163.0757	136.5773	0.006270

    Total: nine complete clips, 786 frames checked, zero client errors. All ordered decoded frames match native output; all six canonical HTTP clips exactly match the approved native clips, and the original reference criteria and visual checks pass. Client latency includes polling/encoding/download; native Task timing is separate. These short workflow measurements do not establish serving capacity.

    The PR inventory and commands cover every existing full checkpoint, immutable pins, controls and variants. Existing Cosmos3-Nano and LTX-Video partial/L0-only profiles remain explicit gaps; no new models/specs/native capabilities were added. Family/core/reference/threshold/CI-rule files remain unchanged.

    The required sanitized internal gate passed on the exact stage-8 head before normal squash merge. Stable Community CI and CodeRabbit also passed; both review conversations are resolved. Optional experimental Dev CI was still pending at merge and is not counted as a passing check. Stages were delivered sequentially under unchanged merge rules.
11. yizhuoz004 commented on Oct 19, 2026

    

    yizhuoz004

    on Oct 19, 2026

    Last edited by yizhuoz004

    CollaboratorAuthor

    More actions

    A–F are complete and merged sequentially, with one PR per workload:

    Stage	Qualified existing-text workload	Merged PR
    A	Twelve original pinned ShareGPT prompts	#1803
    B	Six two-turn conversations using live assistant replies and explicit think time	#1804
    C	Twelve production-trace rows at original and compressed arrival schedules, retaining all completions/rejections/failures	#1805
    D	Twelve controlled prefix/suffix prompts in separate fresh/warm server invocations	#1806
    E	Six original prompts through stateless text Responses, plus actual OpenAI SDK protocol/native checks	#1807
    F	Twelve fitting original SpecBench first turns through ordinary native generation	#1808

    The fixed pilot uses the existing Qwen/Qwen3-0.6B full FP16 profile on GB300, checkpoint/tokenizer revision c1899de289a04d12100db370d81485cdf75e47ca, TP1 and the unchanged 256-token bundle context. Every stage has an actual published PyPI AIPerf 0.13.0 run and unchanged owning-family correctness on the exact served bundle. Original inputs, request/output identities, native generation work, warmup and errors are verified; client timing stays separate from native Task-call timing. Each PR contains its actual measurement revision, results, commands and environment.

    Required exact-head protected CI, Stable Community CPU, metadata, DCO and review passed before each normal merge. At the completion audit, optional Dev CI passed for D/E and remained pending for F. A–C’s optional Dev coordinators reached their five-hour completion timeout without a native model-validation result; the provisioning/queue details and successful local verification are recorded on A, B and C. Those optional results are not counted as passing. No model/spec, numerical criterion, test exclusion, CI configuration or merge rule was added or relaxed for this ladder.

    Capability limits remain explicit: D reports cross-request cache-hit telemetry as unavailable; E supports stateless nonstreaming single-user text, with unknown native input counts represented by null usage; F qualifies the bounded corpus under ordinary generation, with speculative execution unqualified and acceptance metrics unavailable. The text-workload coverage matrix records these limits and links the examples.

    Stages 9–10 remain deferred at the owner’s request, closed as not planned with not enough model; they are not reported as implemented. All sixteen sub-issues are now closed: fourteen completed and two deferred. The epic’s requested delivery scope is complete.
12. yizhuoz004

    closed this as completedon Oct 19, 2026