Implements the closed-loop distillation pipeline that automates:
1. Supervised train on current dataset (Forge + PPO data)
2. PPO self-play using supervised model as starting policy
3. Export winning PPO decisions to JSONL
4. Quality gate check (win rate threshold)
5. If passed, merge into dataset and repeat
6. If failed, discard batch, adjust hyperparameters, retry
New files:
- ml/training/distillation_loop.py — DistillationLoop orchestrator,
DistillationConfig, GenerationRecord, CLI entry point
Updated files:
- ml/config/scope.py — DistillationDefaults dataclass with all
default hyperparameters for the pipeline
Features:
- Generation tagging: data and models tagged with gen-NNN
- Convergence detection: stops when win rate plateaus over N gens
- Automatic retry with LR reduction + entropy boost on gate failure
- Per-generation history saved to JSON for observability
- Full CLI with all tunable params
- progress_callback for UI integration (Phase 5 ready)
Closes#68
rgriffith
ae00dd47e2