AIPerf stage 5: qualify GB300 single-image vision-language models #1672

Request type

New capability

Stage completion

Stage 5 complete: PR #1761 merged as 88ceb9f5217a9f9318e3393a404761e09891194c from bdf62ed9da1ba9909f20c0bb4b549bacb63c2ce3. All nine existing normal native GB300 vision-language profiles passed unchanged owning-family correctness and real PyPI AIPerf qualification (450 measured +45 warmup requests; all495 complete native/HTTP matches; zero client errors). The unchanged required Internal CI gate passed. All seven existing premerge cases passed at 1495d31 with no requested skips; model/runtime/test source is unchanged by the final CI-only fix, and the full Lance case passed again on the final source with compatible dependencies and its unchanged reference. The nine-model AIPerf timing table retains the actual measurement source epochs. No contract, model spec, threshold, test eligibility or repository merge rule was changed. Current-head source/CPU/docs/review checks passed; experimental public GPU status is recorded separately and is not relabelled as a pass.

Problem and use case

Expose the existing TRTMC single-image vision-language models through the existing native server and PyPI AIPerf. The group is defined by existing supported model specs, not upstream architecture compatibility.

Proposed outcome

• Workload: one original RGB image and one question, single-turn, nonstreaming, concurrency 1.
• Client/endpoint: PyPI AIPerf 0.13.0 built-in chat; POST /v1/chat/completions through trtmc-server.
• Hardware scope: retain the owner's GB300 restriction. Do not onboard Edge-backed checkpoints, new models or new specs to expand this stage.
• Implementation PR: feat(aiperf): profile existing GB300 vision-language models #1761; merged prerequisite fix(phi4): stage engine plans and export AIPerf tokenizer #1764 fixes the existing Phi4 model's plan lifetime and tokenizer export.

Public CI hardware failures — owner-approved handling for stages 5–8

Keep existing model contracts, references, numerical thresholds, test eligibility and repository merge-gate rules unchanged. If an in-scope model fails non-required public CI due to OOM or a demonstrated hardware limitation, execute the affected unchanged correctness case locally on capable hardware, retain a non-skipped passing result, and document the public failure and local verification in the PR. Merge only once the unchanged required merge checks pass on the current head and the PR otherwise satisfies the repository rules. A numerical/model-correctness failure is not a hardware exception. Do not add test exclusions or manufacture a passing public result. Deliver stages 5, 6, 7 and 8 sequentially, starting each implementation after the previous stage merges.

Existing-model scope — owner clarification

This epic integrates AIPerf with models and model specs already supported on upstream main. “Whole group” means every existing model/spec for this workload, rather than adding checkpoints that an architecture might also support.

• Reuse existing checkpoint identities, normal full-model profiles/manifests, build settings, references and correctness thresholds. Do not add model families, checkpoints, model specs/manifests, precision/parallel variants or new numerical gates to enlarge the group.
• HTTP/media adapters, AIPerf input/workload configuration and regression fixes needed for existing models are in scope. Keep model semantics family-owned and existing correctness criteria unchanged.
• Missing full-model implementations or native audio/video capabilities are recorded as unsupported/outside this integration. Do not implement a new encoder, model graph, temporal path or native Task to fill the inventory. L0/partial profiles do not require full-model onboarding in this epic.
• Refresh against current main only to include independently existing supported specs. Architectural compatibility, model cards and upstream model availability do not establish an in-scope TRTMC model.

Model and target details

Inventory reconciled against existing main a832817e6ed9603b100bbff25dce3f04c65beaca: nine existing normal full-model profiles, all already present before #1761. Their checkpoint pins, build controls and validation thresholds remain unchanged.

Scope correction: the seven InternVL profiles introduced by #1761 (InternVL3 1B/14B and InternVL3.5 1B/2B/4B/8B/14B), their new thresholds and profile-only extension are outside this request and have been removed from #1761 at e12fbc0441e648b9ebfa3ce8ebf7b2bea2e81ba6. Earlier 16-model completion counts are superseded by this existing-model inventory. Historical measurements remain historical evidence and do not enlarge the authorized scope. Existing distributed/diagnostic profiles are not additional checkpoint coverage, and the integration must preserve their selection and correctness controls.

Acceptance criteria

• Inventory every existing eligible normal model spec for the selected workload.
• All nine profiles have owning-family correctness and actual PyPI AIPerf evidence; retain the actual tested sources and bundle identities.
• Every profile completed 50 measured + 5 warmup requests with all 55 native/HTTP responses matched and zero client errors: 450 measured + 45 warmup requests across the scoped group.
• Remove the seven new profiles/thresholds and profile-only implementation from feat(aiperf): profile existing GB300 vision-language models #1761; verify no model manifest/spec/threshold differs from main.
• Pass focused checks and unchanged required current-head merge checks, document locally verified public hardware failures, resolve review, and merge feat(aiperf): profile existing GB300 vision-language models #1761.
• Provide the simple Qwen3-VL-2B example using an already-built bundle, existing build guidance and pip-installed AIPerf.

Retained local qualification results

These are conversion-parity/integration results, not gold VQA accuracy or cross-model performance rankings. All nine existing profiles pass their declared gates. HTTP includes transport; Task time includes family preprocessing and generation. Build/load/startup/warmup are excluded. Full responses and actual native output-token counts are retained.

Actual measurement sources remain unchanged: InternVL3-2B/8B use f6fe94a53a38db1c8d36eb032895e2fad42f91a4; Qwen3-VL-2B/8B and Phi4 use 798589ea05e6c39b15c8a48e92e565e25e7d9e86; DeepSeek uses 7a4f3239ee60075937f32b649d12ce50c364f470; Qwen2.5/Lance/LocateAnything use 0ce210596cadfa47e1b8641357a85aaa62281d93. Measurements are not relabelled as new-head runs. The Qwen3-only weight-mapping addition being removed is unused by the two existing Qwen2-backed InternVL profiles.

Additional context

• Parent epic: Epic: expand TRTMC AIPerf model and workload coverage #1667.
• Implementation/commands, dataset pins, software environment and qualification details: feat(aiperf): profile existing GB300 vision-language models #1761.
• Existing-model Phi4 prerequisite: fix(phi4): stage engine plans and export AIPerf tokenizer #1764, merged as a832817e6ed9603b100bbff25dce3f04c65beaca.
• Public CI OOM and local verification evidence: feat(aiperf): profile existing GB300 vision-language models #1761 (comment). The InternVL3.5-4B case in that historical report is outside the corrected scope. Existing DeepSeek and Qwen3-VL-8B public-CI limitations remain separately reported; no test exclusions have been approved or published for them.

Submission checks

• Existing issue retained; no duplicate created.
• No private/internal logs, artifacts, personal paths or restricted evidence included.