Skip to content

[Feature] Add multi-teacher OPD e2e configs and 2-step golden-metric accuracy test - #2098

Open
YanhuiDua wants to merge 3 commits into
InternLM:mainfrom
YanhuiDua:feat/opd-e2e-two-step-accuracy
Open

YanhuiDua wants to merge 3 commits into
InternLM:mainfrom
YanhuiDua:feat/opd-e2e-two-step-accuracy

Conversation

@YanhuiDua

@YanhuiDua YanhuiDua commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add two Qwen3-VL-2B multi-teacher OPD e2e recipe configs:
    • rl_qwen3_vl_2b_sampled_token_mopd_rollout_teacher.py: sampled-token k1 distillation with HTTP rollout teachers (batch 128 / seed 1234 / 1024+2048, aligned with the 20260731 baseline shape).
    • rl_qwen3_vl_2b_topk_mopd_train_teacher.py: top-k forward-KL (forward_kl_topk) with frozen FSDP teachers inside TrainingWorkers (no teacher HTTP servers).
  • Register the Qwen3-Dense-2B language tower and Qwen3VLDense2BConfig compose config (student 2B sizes) used by both recipes.
  • Add tests/rl/test_opd_two_step_accuracy.py: golden step-1/2 metric baselines (TB scalars recorded from the 50-step reference clusterx runs) plus TensorBoard reading / tolerance comparison. A finished 2-step smoke run can be checked via OPD_SMOKE_RUN_DIR / OPD_SMOKE_EXPERIMENT=topk|mopd; the checks skip when the env is unset so CI stays green.

Test Plan

  • pytest tests/rl/test_opd_two_step_accuracy.py: 13 passed, 2 skipped (env-gated).
  • Both recipe configs import cleanly against main (config construction verified end-to-end).
  • Golden step-1/2 values extracted from 50-step reference runs (train_teacher_topk ~33 min, mopd ~30 min); 2-step smoke tolerance calibration in progress.

@PengchengShi00 PengchengShi00 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants