AIPerf stage 3: qualify all TRTMC recorded-clip transcription models #1670

Request type

New capability

Problem and use case

Text endpoints do not cover transcription. TRTMC already has upload and transcription bridge mechanics that should serve a real native ASR Task and preserve audio-specific correctness and timing.

Proposed outcome

Qualify all supported models in this group for one recorded-clip asr workload through the existing native serving entrypoint.

• Workload: One complete prerecorded WAV clip per request from a pinned LibriSpeech selection.
• Client / endpoint: AIPerf audio_transcription; multipart POST /v1/audio/transcriptions.
• Rollout / prerequisites: Model-group rollout after stage 2; qualify only recorded complete clips in this stage.

Existing-model scope — owner clarification

This epic integrates AIPerf with models and model specs already supported on upstream main. “Whole group” means every existing model/spec for this workload, rather than adding checkpoints that an architecture might also support.

• Reuse existing checkpoint identities, normal full-model profiles/manifests, build settings, references and correctness thresholds. Do not add model families, checkpoints, model specs/manifests, precision/parallel variants or new numerical gates to enlarge the group.
• HTTP/media adapters, AIPerf input/workload configuration and regression fixes needed for existing models are in scope. Keep model semantics family-owned and existing correctness criteria unchanged.
• Missing full-model implementations or native audio/video capabilities are recorded as unsupported/outside this integration. Do not implement a new encoder, model graph, temporal path or native Task to fill the inventory. L0/partial profiles do not require full-model onboarding in this epic.
• Refresh against current main only to include independently existing supported specs. Architectural compatibility, model cards and upstream model availability do not establish an in-scope TRTMC model.

Group-wide scope and completion (revised)

Cover every TRTMC-supported checkpoint in this model group for the workload below, not just one pilot. Refresh the inventory against github/main at implementation and before completion. Use family-owned manifests, actual build/runtime behavior, and public Task capabilities together; a model name, broad support declaration, or an inherited C++ interface alone does not establish the checkpoint's supported workload.

Use one normal full-model profile per distinct checkpoint and workload. Record all additional precision/quantization/TP/CP variants separately; do not count variants as extra models or claim untested configurations. L0/partial/random/regression smoke profiles do not establish full-model coverage. Checkpoints with only partial implementation remain explicit gaps, not silent omissions. Only independently existing supported specs on main join the refreshed inventory; do not add models or specs as part of this integration.

• Publish the exhaustive per-checkpoint inventory with checkpoint/profile/revision, native Task, supported controls, and any explicit capability gap; reconcile it with current main before closing.
• Every supported checkpoint passes its unchanged owning-family correctness criteria on the exact served bundle, followed by a real published pip-installed AIPerf run through trtmc-server for this one workload.
• Check meaningful complete output contents and native/HTTP identity for every model, with separately marked warmup, failures, client/Task timing, and workload-appropriate throughput.
• Publish a result row and reproducible command for every checkpoint. A pilot, source-only check, HTTP fixture, skipped test, unrun model, unsupported adapter, or unresolved failure cannot close this group-wide task.
• Preserve family ownership and existing quality criteria; fix real defects without introducing model behavior in shared serving code or relaxing gates.

Implementation scope:

1. Reuse multipart uploads and transcription mapping in the native server; decode actual audio and validate its sample rate and duration.
2. Use the existing qualification dataset/recording mechanics and retain skipped, rejected, and failed clips in reports.
3. Keep audio preprocessing, decoding semantics, and numerical validation within each owning model family.

Acceptance criteria for every model in the group:

• Transcripts from the actual native bundle satisfy the existing family correctness and WER contract unchanged.
• A real pip-installed AIPerf run sends the selected WAV files and returns correlated transcripts with visible failures.
• Report request latency and audio-duration-normalized throughput/real-time factor where measured or derived; record the calculation and separate Task-call time.
• Reuse existing serving/qualification mechanics; keep all model semantics, orchestration, and correctness family-owned, with capability-based Task dispatch.
• Account for inputs, outputs/artifacts, failures, and warmup by request identity. Keep client end-to-end latency separate from public_task_call_wall; exclude build/load/startup/warmup from steady-state timing.
• Record tested package versions, checkpoint/profile, actual bundle, dataset, endpoint/mode, hardware, effective precision, settings, commands, and public evidence. Distinguish planned, implemented, family-validated, and AIPerf-qualified status.
• Provide one concise reproducible example using published pip-installed AIPerf and links to existing build guidance; an AIPerf source checkout is not a runtime dependency.

Model and target details

Inventory baseline: TRTMC fd79ad26ca71a11b3ec086a7daaaead03e6c933f. Candidate models/profiles for this group:

The workload stays one complete recorded clip and one final transcript. Include both Nemotron checkpoints: their family implements the public whole-clip ITranscription path despite declaring a streaming-oriented bundle mode. Add generic capability dispatch where needed; leave chunking, decoding, prompts and orchestration family-owned. This stage does not claim HTTP streaming/TTFT. Audit family-specific maximum duration, sample rate, token limits and required language defaults.

Use the public native Task runtime path. Record the actual supported target GPU and software stack during implementation/qualification; no new hardware validation or benchmark is claimed by this planning issue. Keep existing numerical criteria unchanged.

Alternatives considered

Reuse apps/perf_serving protocol/media/recording mechanics and apps/aiperf_qual qualification instead of adding a third server or benchmark harness. Deliver this stage on its own short-lived branch from current github/main.

Follow-ups outside this initial stage: Streaming ASR and audio-question-answering are separate workloads.

Additional context

• Parent epic: Epic: expand TRTMC AIPerf model and workload coverage #1667
• Pilot profile
• AIPerf docs/tutorials/asr.md
• Native server
• Existing qualification harness

Research baseline: TRTMC 0bccb5e93af0e5ad346aaf44e404396f54b2e7de and AIPerf e1daadd10faa395100384358131d784d6858c5ae; published AIPerf 0.13.0. Recheck the tested published version and native capabilities when implementing. Existing source/fixtures/reference-only runs do not establish end-to-end TRTMC qualification. Keep internal logs, artifacts, and runner details private.

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.

Delivered to main by PR #1707; merge commit 0524d694386810b0b5dadf29cce15239d230b959. All current-head Stable/Dev Community checks, protected internal premerge CI, documentation, DCO, metadata, and automated review checks passed on 572e28db766daa0279e4dbec90e7f615420d732c. No review conversations remain open.

The rebuilt Whisper tiny bundle passes eight selected family tests (including the unchanged strict transcript case on SM89), ten-clip WER gates, 146 serving/transport tests, six native serving tests, 307 source checks, 26 minimum-version ASR tests, and website validation. Family-local GELU and FP32 encoder/output arithmetic corrections preserve the original fixture, reference, zero-NED/zero-WER gates, Task ABI, and bundle format. Older Whisper bundles require rebuilding.

Fresh pip-installed AIPerf 0.13.0 completed 50 measured requests plus five warmups, with all 55 complete responses matching native output and every request/timing export joined. HTTP p50/p95: 67.99/199.53 ms; public Task p50/p95: 66.01/196.83 ms; throughput: 11.52 requests/s and 105.42 audio seconds/s. Native/reference corpus WER is 0.0, with 3.59% WER to gold on both sides. Scope, revisions, actual served bundle, environment, and concise reproduction are documented in the PR and epic progress comment. Streaming ASR, other codecs, stereo input, translations, and audio QA remain unqualified.

Historical pilot delivery

The earlier pilot PR is merged. Its evidence remains valid for that named pilot, but does not complete the expanded all-model scope. This task is reopened to track the missing checkpoints.

Group completion policy revised at the owner's request: all supported checkpoints in this group must be accounted for and qualified; one successful pilot is insufficient. Existing pilot results are retained as historical evidence.