AIPerf text stage C: qualify production trace replay #1680

Request type

New capability

Problem and use case

Steady-concurrency profiling does not establish faithful replay of arrival timing, queueing, rejections, or errors in a production-like trace.

Proposed outcome

Qualify one existing text generation workload through the existing native serving entrypoint.

• Workload: One supported trace format with pinned requests and timestamps; replay the recorded offered load.
• Client / endpoint: AIPerf trace replay against the existing native text endpoint selected for the trace.
• Rollout / prerequisites: Text-workload rollout after stage B; select one trace format before implementing its adapter.

Existing-model scope — owner clarification

This epic integrates AIPerf with models and model specs already supported on upstream main. “Whole group” means every existing model/spec for this workload, rather than adding checkpoints that an architecture might also support.

• Reuse existing checkpoint identities, normal full-model profiles/manifests, build settings, references and correctness thresholds. Do not add model families, checkpoints, model specs/manifests, precision/parallel variants or new numerical gates to enlarge the group.
• HTTP/media adapters, AIPerf input/workload configuration and regression fixes needed for existing models are in scope. Keep model semantics family-owned and existing correctness criteria unchanged.
• Missing full-model implementations or native audio/video capabilities are recorded as unsupported/outside this integration. Do not implement a new encoder, model graph, temporal path or native Task to fill the inventory. L0/partial profiles do not require full-model onboarding in this epic.
• Refresh against current main only to include independently existing supported specs. Architectural compatibility, model cards and upstream model availability do not establish an in-scope TRTMC model.

Implementation scope:

1. Select one trace format and a public or publishable sanitized dataset; preserve timestamps, request identity, and supported input/generation settings.
2. Reuse admission and metric recording to capture offered versus achieved rates, queueing, 429 responses, and other errors without implicit retries or lost records obscuring the workload.
3. Document timestamp units, replay/time-scaling policy, endpoint, execution-lane count, and any input/context restrictions.

Acceptance criteria:

• A real pip-installed AIPerf trace run sends the selected workload with the documented arrival schedule and verifies replay fidelity.
• Every offered request is accounted for as completed, rejected, or failed; queue/admission observations and offered/achieved rates remain visible.
• The actual native bundle passes unchanged family correctness; client end-to-end timing and Task-call timing are distinct and warmup/startup treatment is explicit.
• Reuse existing serving/qualification mechanics; keep all model semantics, orchestration, and correctness family-owned, with capability-based Task dispatch.
• Account for inputs, outputs/artifacts, failures, and warmup by request identity. Keep client end-to-end latency separate from public_task_call_wall; exclude build/load/startup/warmup from steady-state timing.
• Record tested package versions, checkpoint/profile, actual bundle, dataset, endpoint/mode, hardware, effective precision, settings, commands, and public evidence. Distinguish planned, implemented, family-validated, and AIPerf-qualified status.
• Provide one concise reproducible example using published pip-installed AIPerf and links to existing build guidance; an AIPerf source checkout is not a runtime dependency.

Model and target details

Qwen/Qwen3-0.6B; qwen family; use the existing FP16 profile as the initial pilot. Pin the checkpoint and record the actual bundle/context limit and generation configuration.

Use the public native Task runtime path. Record the actual supported target GPU and software stack during implementation/qualification; no new hardware validation or benchmark is claimed by this planning issue. Keep existing numerical criteria unchanged.

Alternatives considered

Reuse apps/perf_serving protocol/media/recording mechanics and apps/aiperf_qual qualification instead of adding a third server or benchmark harness. Deliver this stage on its own short-lived branch from current github/main.

Follow-ups outside this initial stage: Additional trace formats, burst/ramp experiments, and recorded agent traces are separate workloads. Replay alone does not implement an autonomous agent.

Additional context

• Parent epic: Epic: expand TRTMC AIPerf model and workload coverage #1667
• Pilot profile
• AIPerf docs/benchmark-modes/trace-replay.md
• Native server
• Existing qualification harness

Research baseline: TRTMC 0bccb5e93af0e5ad346aaf44e404396f54b2e7de and AIPerf e1daadd10faa395100384358131d784d6858c5ae; published AIPerf 0.13.0. Recheck the tested published version and native capabilities when implementing. Existing source/fixtures/reference-only runs do not establish end-to-end TRTMC qualification. Keep internal logs, artifacts, and runner details private.

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.