AIPerf text stage B: qualify multi-turn chat #1679

Request type

New capability

Problem and use case

The native single-turn chat baseline does not establish multi-turn correctness. Users need faithful roles, turn order, history, and per-turn measurements.

Proposed outcome

Qualify one existing text generation workload through the existing native serving entrypoint.

• Workload: One pinned conversational dataset with multiple ordered turns and an explicit think-time policy.
• Client / endpoint: AIPerf chat; POST /v1/chat/completions with conversation history.
• Rollout / prerequisites: Text-workload rollout after stage A; separate implementation/qualification PR.

Existing-model scope — owner clarification

This epic integrates AIPerf with models and model specs already supported on upstream main. “Whole group” means every existing model/spec for this workload, rather than adding checkpoints that an architecture might also support.

• Reuse existing checkpoint identities, normal full-model profiles/manifests, build settings, references and correctness thresholds. Do not add model families, checkpoints, model specs/manifests, precision/parallel variants or new numerical gates to enlarge the group.
• HTTP/media adapters, AIPerf input/workload configuration and regression fixes needed for existing models are in scope. Keep model semantics family-owned and existing correctness criteria unchanged.
• Missing full-model implementations or native audio/video capabilities are recorded as unsupported/outside this integration. Do not implement a new encoder, model graph, temporal path or native Task to fill the inventory. L0/partial profiles do not require full-model onboarding in this epic.
• Refresh against current main only to include independently existing supported specs. Architectural compatibility, model cards and upstream model availability do not establish an in-scope TRTMC model.

Implementation scope:

1. Add history support where the current native HTTP path restricts chat messages; preserve family-owned chat-template/tokenization semantics.
2. Reuse the existing AIPerf conversation/data and qualification machinery, recording conversation identity, each turn, history, and think time.
3. Use a dataset/context configuration that fits the actual bundle and make limit failures explicit rather than silently discarding history.

Acceptance criteria:

• Protocol and native correctness evidence verifies roles, turn order, retained history, and history-dependent answers under unchanged family criteria.
• A real pip-installed AIPerf multi-turn run completes against native chat with per-turn correlation and visible errors.
• Per-turn client latency and Task-call time are separate, think-time policy is recorded, and claimed TTFT/ITL is verified against genuine incremental native streaming.
• Reuse existing serving/qualification mechanics; keep all model semantics, orchestration, and correctness family-owned, with capability-based Task dispatch.
• Account for inputs, outputs/artifacts, failures, and warmup by request identity. Keep client end-to-end latency separate from public_task_call_wall; exclude build/load/startup/warmup from steady-state timing.
• Record tested package versions, checkpoint/profile, actual bundle, dataset, endpoint/mode, hardware, effective precision, settings, commands, and public evidence. Distinguish planned, implemented, family-validated, and AIPerf-qualified status.
• Provide one concise reproducible example using published pip-installed AIPerf and links to existing build guidance; an AIPerf source checkout is not a runtime dependency.

Model and target details

Qwen/Qwen3-0.6B; qwen family; use the existing FP16 profile as the initial pilot. Pin the checkpoint and record the actual bundle/context limit and generation configuration.

Use the public native Task runtime path. Record the actual supported target GPU and software stack during implementation/qualification; no new hardware validation or benchmark is claimed by this planning issue. Keep existing numerical criteria unchanged.

Alternatives considered

Reuse apps/perf_serving protocol/media/recording mechanics and apps/aiperf_qual qualification instead of adding a third server or benchmark harness. Deliver this stage on its own short-lived branch from current github/main.

Follow-ups outside this initial stage: Multi-turn image/audio conversations and additional conversation datasets are separate workloads.

Additional context

• Parent epic: Epic: expand TRTMC AIPerf model and workload coverage #1667
• Pilot profile
• AIPerf docs/tutorials/multi-turn.md
• Native server
• Existing qualification harness

Research baseline: TRTMC 0bccb5e93af0e5ad346aaf44e404396f54b2e7de and AIPerf e1daadd10faa395100384358131d784d6858c5ae; published AIPerf 0.13.0. Recheck the tested published version and native capabilities when implementing. Existing source/fixtures/reference-only runs do not establish end-to-end TRTMC qualification. Keep internal logs, artifacts, and runner details private.

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.