Commit Graph
1 Commits
Author SHA1 Message Date
Claude a7f2e0aa79 Sharpen catalyst labels + add validation tooling
Accuracy: news_enrichment now sends a JSON schema to Ollama's `format`
(enum-locked catalyst_type, typed fields) so decoding is constrained at
generation time, and build_prompt carries 3 few-shot examples (offering,
FDA, earnings) — the biggest accuracy lever for a small local model.
classify_headline gains use_schema (default True; False = plain json).

Validation:
- research/ingestion/catalyst_eval.py — hand-labeled fixture + pure
  score_predictions (type accuracy, is_dilutive precision/recall, sentiment
  sign). catalyst_eval.py CLI A/Bs models (--model) against the same set.
- validate_catalysts.py — joins enriched labels to each stored gapper's
  open->close / open->high outcome; pure outcome_correlation splits by
  dilution flag and bullish-score bucket. Reports coverage honestly (news is
  live-only) and writes a CSV. Run before flipping the Phase 2 flags.

Tests: schema-in-request assertions, pure eval/aggregator coverage. 212 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M5HTMkaadGDnAPtXe2hWwg
2026-06-29 21:50:52 +00:00