AIPerf text stage A: qualify exact real-prompt inputs #1678

Request type

New capability

Problem and use case

A synthetic token-length baseline does not show that real prompts are preserved end to end. Users need reproducible dataset-driven requests with the actual text and lengths visible.

Proposed outcome

Qualify one existing text generation workload through the existing native serving entrypoint.

• Workload: One pinned single-turn corpus selection, initially ShareGPT prompts, with original selected text preserved.
• Client / endpoint: Existing native text endpoint; initially AIPerf completions with POST /v1/completions.
• Rollout / prerequisites: The merged text-generation integration in PR feat(aiperf): profile text with trtmc-server #1606 is the foundation. Deliver this as its own PR, separately from model-group stages 1-10.

Existing-model scope — owner clarification

This epic integrates AIPerf with models and model specs already supported on upstream main. “Whole group” means every existing model/spec for this workload, rather than adding checkpoints that an architecture might also support.

• Reuse existing checkpoint identities, normal full-model profiles/manifests, build settings, references and correctness thresholds. Do not add model families, checkpoints, model specs/manifests, precision/parallel variants or new numerical gates to enlarge the group.
• HTTP/media adapters, AIPerf input/workload configuration and regression fixes needed for existing models are in scope. Keep model semantics family-owned and existing correctness criteria unchanged.
• Missing full-model implementations or native audio/video capabilities are recorded as unsupported/outside this integration. Do not implement a new encoder, model graph, temporal path or native Task to fill the inventory. L0/partial profiles do not require full-model onboarding in this epic.
• Refresh against current main only to include independently existing supported specs. Architectural compatibility, model cards and upstream model availability do not establish an in-scope TRTMC model.

Implementation scope:

1. Reuse apps/aiperf_qual dataset construction and text profiling; materialize one fixed selection with source/version/selection policy recorded.
2. Preserve selected prompt text, tokenizer identity, actual token lengths, and generation settings; select inputs within the actual bundle's context limit or explicitly build a suitable supported limit.
3. Keep rejected/over-limit records visible and avoid silent truncation or rewriting to satisfy requested synthetic lengths.

Acceptance criteria:

• Recorded native requests contain the exact selected prompt text; input/output correlation and actual lengths are verified.
• The actual bundle passes its unchanged family correctness checks and an actual pip-installed AIPerf run exercises the fixed real-prompt corpus.
• Report the selected endpoint/mode, errors, client latency/throughput, and Task-call timing separately; streaming metrics require genuine incremental output.
• Reuse existing serving/qualification mechanics; keep all model semantics, orchestration, and correctness family-owned, with capability-based Task dispatch.
• Account for inputs, outputs/artifacts, failures, and warmup by request identity. Keep client end-to-end latency separate from public_task_call_wall; exclude build/load/startup/warmup from steady-state timing.
• Record tested package versions, checkpoint/profile, actual bundle, dataset, endpoint/mode, hardware, effective precision, settings, commands, and public evidence. Distinguish planned, implemented, family-validated, and AIPerf-qualified status.
• Provide one concise reproducible example using published pip-installed AIPerf and links to existing build guidance; an AIPerf source checkout is not a runtime dependency.

Model and target details

Qwen/Qwen3-0.6B; qwen family; use the existing FP16 profile as the initial pilot. Pin the checkpoint and record the actual bundle/context limit and generation configuration.

Use the public native Task runtime path. Record the actual supported target GPU and software stack during implementation/qualification; no new hardware validation or benchmark is claimed by this planning issue. Keep existing numerical criteria unchanged.

Alternatives considered

Reuse apps/perf_serving protocol/media/recording mechanics and apps/aiperf_qual qualification instead of adding a third server or benchmark harness. Deliver this stage on its own short-lived branch from current github/main.

Follow-ups outside this initial stage: Math and code corpora, and different datasets or traffic modes, are separate experiments.

Additional context

• Parent epic: Epic: expand TRTMC AIPerf model and workload coverage #1667
• Pilot profile
• AIPerf README.md#workloads-and-data
• Native server
• Existing qualification harness

Research baseline: TRTMC 0bccb5e93af0e5ad346aaf44e404396f54b2e7de and AIPerf e1daadd10faa395100384358131d784d6858c5ae; published AIPerf 0.13.0. Recheck the tested published version and native capabilities when implementing. Existing source/fixtures/reference-only runs do not establish end-to-end TRTMC qualification. Keep internal logs, artifacts, and runner details private.

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.