AIPerf text stage F: qualify a speculative-decoding prompt corpus #1683

Request type

New capability

Problem and use case

AIPerf documents speculative-decoding datasets, but loading those prompts does not create a speculative runtime or acceptance telemetry in TRTMC.

Proposed outcome

Qualify one existing text generation workload through the existing native serving entrypoint.

• Workload: One pinned speculative-decoding benchmark corpus selection, initially exercised as ordinary native generation unless genuine draft/target support is available.
• Client / endpoint: Existing supported native text endpoint; use the published AIPerf dataset path for the selected corpus.
• Rollout / prerequisites: Text-workload rollout after stage E. Native speculative execution is a separate capability prerequisite for acceleration claims.

Existing-model scope — owner clarification

This epic integrates AIPerf with models and model specs already supported on upstream main. “Whole group” means every existing model/spec for this workload, rather than adding checkpoints that an architecture might also support.

• Reuse existing checkpoint identities, normal full-model profiles/manifests, build settings, references and correctness thresholds. Do not add model families, checkpoints, model specs/manifests, precision/parallel variants or new numerical gates to enlarge the group.
• HTTP/media adapters, AIPerf input/workload configuration and regression fixes needed for existing models are in scope. Keep model semantics family-owned and existing correctness criteria unchanged.
• Missing full-model implementations or native audio/video capabilities are recorded as unsupported/outside this integration. Do not implement a new encoder, model graph, temporal path or native Task to fill the inventory. L0/partial profiles do not require full-model onboarding in this epic.
• Refresh against current main only to include independently existing supported specs. Architectural compatibility, model cards and upstream model availability do not establish an in-scope TRTMC model.

Implementation scope:

1. Choose and pin one supported corpus such as SpecBench or SPEED-Bench, documenting selection, exact input text, context limits, and generation settings.
2. Reuse dataset/text qualification for an actual ordinary-generation baseline; keep the runtime mode explicit in all reports.
3. Audit existing draft/target capability separately. Do not add draft/target models, specs or speculative runtime support in this epic. Acceleration and acceptance claims require already-supported family-owned orchestration, real counters and unchanged correctness criteria; otherwise qualify only the ordinary-generation corpus workload.

Acceptance criteria:

• A real pip-installed AIPerf run sends the selected corpus to the actual native bundle with prompt/output correlation, visible errors, and unchanged family correctness.
• The coverage matrix distinguishes qualified corpus input from speculative execution; ordinary generation is labelled accordingly and unsupported acceptance metrics remain unavailable.
• Any speculative acceleration/acceptance claim records the actual draft/target models, configuration, implementation, counters, matched inputs, and correctness evidence. Corpus choice alone is insufficient.
• Reuse existing serving/qualification mechanics; keep all model semantics, orchestration, and correctness family-owned, with capability-based Task dispatch.
• Account for inputs, outputs/artifacts, failures, and warmup by request identity. Keep client end-to-end latency separate from public_task_call_wall; exclude build/load/startup/warmup from steady-state timing.
• Record tested package versions, checkpoint/profile, actual bundle, dataset, endpoint/mode, hardware, effective precision, settings, commands, and public evidence. Distinguish planned, implemented, family-validated, and AIPerf-qualified status.
• Provide one concise reproducible example using published pip-installed AIPerf and links to existing build guidance; an AIPerf source checkout is not a runtime dependency.

Model and target details

Qwen/Qwen3-0.6B; qwen family; use the existing FP16 profile as the initial pilot. Pin the checkpoint and record the actual bundle/context limit and generation configuration.

Use the public native Task runtime path. Record the actual supported target GPU and software stack during implementation/qualification; no new hardware validation or benchmark is claimed by this planning issue. Keep existing numerical criteria unchanged.

Alternatives considered

Reuse apps/perf_serving protocol/media/recording mechanics and apps/aiperf_qual qualification instead of adding a third server or benchmark harness. Deliver this stage on its own short-lived branch from current github/main.

Follow-ups outside this initial stage: Additional speculative corpora, genuine family-owned draft/target execution where missing, and draft/target configuration sweeps are separate follow-ups.

Additional context

• Parent epic: Epic: expand TRTMC AIPerf model and workload coverage #1667
• Pilot profile
• AIPerf README.md#workloads-and-data
• Native server
• Existing qualification harness

Research baseline: TRTMC 0bccb5e93af0e5ad346aaf44e404396f54b2e7de and AIPerf e1daadd10faa395100384358131d784d6858c5ae; published AIPerf 0.13.0. Recheck the tested published version and native capabilities when implementing. Existing source/fixtures/reference-only runs do not establish end-to-end TRTMC qualification. Keep internal logs, artifacts, and runner details private.

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.