on-policy distillation
5 stories tagged “on-policy distillation”.
-
A Reinforcement Learning Perspective on OPD: Least-Squares Policy Distillation for Sample-Efficient LLM Reasoning
LSPD recasts on-policy distillation as KL-regularized RL, using least-squares matching and replay to improve s…
-
Moving Beyond Timestamps: On-Policy Distillation Aligned with Decisions for Long-Horizon Agents
AlignOPSD moves distillation from timestamps to functional decision spans, using credit assignment to improve …
-
AdaTutoRank: Adaptive Tutoring Optimization for Document Set Reranking in RAG and Deep Research
AdaTutoRank improves RAG reranking with difficulty-matched hints and rubric rewards, reducing sparse credit in…
-
LastOPD: Mitigating Collapse in Latent On-Policy Distillation
LastOPD stops latent supervision collapse in Qwen3 on-policy distillation by briefly aligning only the last la…
-
MOPD-Router: Revisiting Teacher Routing for Multi-Teacher On-Policy Distillation
MOPD-Router replaces prompt-level hard routing with token-level routing, improving multi-teacher on-policy dis…