Implements the three approved Phase 5 architecture decisions:
1. Zone embedding boundary: add explicit documentation in StateDimensions
and StateEncoder._encode_zones making the 6144-dim zero-padded zone
contract visible to future maintainers.
2. SQLite WAL online learning store: new OnlineLearningStore class backed
by WAL-mode SQLite for concurrent read/write on single-machine baseline.
New /api/policy/decide endpoint collects decisions during inference.
Stats endpoint at /api/policy/decide/stats.
3. Forge-only default weights: change ppo_weight default from 0.5 to 0.0
across scope.py, DistillationConfig, DatasetConfig, and CLI defaults.
Add MIXED_MODE_PRESETS dict with forge_only, forge_90_10, forge_80_20
presets. Distill/start endpoint now accepts a "preset" parameter.
27 new tests covering all three decisions (all passing).
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Implements the closed-loop distillation pipeline that automates:
1. Supervised train on current dataset (Forge + PPO data)
2. PPO self-play using supervised model as starting policy
3. Export winning PPO decisions to JSONL
4. Quality gate check (win rate threshold)
5. If passed, merge into dataset and repeat
6. If failed, discard batch, adjust hyperparameters, retry
New files:
- ml/training/distillation_loop.py — DistillationLoop orchestrator,
DistillationConfig, GenerationRecord, CLI entry point
Updated files:
- ml/config/scope.py — DistillationDefaults dataclass with all
default hyperparameters for the pipeline
Features:
- Generation tagging: data and models tagged with gen-NNN
- Convergence detection: stops when win rate plateaus over N gens
- Automatic retry with LR reduction + entropy boost on gate failure
- Per-generation history saved to JSON for observability
- Full CLI with all tunable params
- progress_callback for UI integration (Phase 5 ready)
Closes#68
