momentum-system/plans/009-feature-population-prerequisite.md

Plan 009: Instrument the data so a conversion model gets ONE fair test — fill the three feature gaps, then re-run the go/no-go

Executor instructions: This is the prerequisite to plan 008. It fixes the data-quality reasons the 2026-07-14 signal probe was inconclusive, then re-runs that probe scoped to the live strategy. It is GATED: the final step's AUC decides whether modeling is ever worth pursuing. Run every verification; honor STOP conditions; update plans/README.md when done.

Drift check: git diff --stat d28fe25..HEAD -- research/labeler.py research/ingestion/ config.py — the labeler is the substrate; re-confirm the excerpts below if it moved.

Status

• Priority: P2 (real payoff, but a multi-week accrual play against a negative prior — read "Honest framing")
• Effort: L (mostly forward data accrual; the code is M)
• Risk: LOW to live trading — all of this is labeler/feature plumbing + a config flag; no live gate/sizing/exit changes
• Depends on: 006 (harness), 008 (the probe this re-runs)
• Category: research-infrastructure / data-quality
• Planned at: commit d28fe25, 2026-07-14

Honest framing (read before starting — this is not a quick win)

The 2026-07-14 probe (plan 008) found CV AUC 0.514 ≈ noise on 63,803 labeled setups. Investigation showed the test was crippled by data quality, not necessarily by absence of signal:

• 85% of rows are liq1 (a refuted track); the live ORB v1 is only 6,217 rows and converts at 24%.
• Per-version null audit:
  • v1 (live ORB): has gap_pct, above_vwap_flag, vwap_at_trigger, session_minute_number; missing relative_volume (100% null), catalyst, market-context. (Pullback/impulse fields are legitimately N/A for a breakout setup — do not "fix" those.)
  • pm1 (premarket): the best-instrumented track — gap_pct, relative_volume, impulse_pct all populated; only catalyst missing. Also the one unexplored strategy axis.
  • tape_regime_at_trigger and spy_intraday_return_pct_at_trigger are 100% null on every track — hardcoded None in the labeler.

This plan fills the three fillable gaps so a model gets one fair, fully-instrumented test. Set expectations honestly: even fully instrumented, the prior is strongly negative (7 refuted single levers + this joint-model probe), v1 is small (~6k) and low-converting, and the market-context + catalyst features can only accrue forward (thin history), so a fair re-test needs a few weeks of data. If the re-run still returns ~0.51, that is the definitive answer — stop filtering ORB and build the premarket entry (which this plan also leaves better-instrumented). The alternative to running this plan is to skip straight to the premarket-setup work; this plan is worth it only if the owner wants modeling to get its one clean shot first.

Current state

• Labeler: research/labeler.py. The v1/ORB SetupEvent build (~lines 288–316) computes rvol = (vol_upto / avg_vol) if avg_vol else None — so relative_volume is None whenever avg_vol (average daily volume) is 0/null, which is the norm for thin IPO-class small caps (documented blind spot; see the ignition.py comment "scan_gappers RVOL zeros out on thin IPOs, avg-daily-vol = 0").
• Market context: research/labeler.py:~322 literally sets tape_regime=None, spy_intraday_return=None. These features are declared in setup_events but never populated.
• SPY/QQQ data: SELECT count(*), count(DISTINCT session_date) FROM minute_bars WHERE symbol IN ('SPY','QQQ') → 53,452 rows across only 30 sessions. So market-context backfill is limited to ~30 days; forward-fill requires ingesting SPY/QQQ daily going forward.
• Catalyst: config.py OllamaConfig; enrichment gated by NEWS_ENRICH_ENABLED (off in the deployed compose — verified). catalyst_freshness_at_trigger / catalyst_type columns exist but are 73–97% null because enrichment isn't running. _catalyst_at(...) in the labeler already reads catalyst events IF they exist.
• The probe: plan 008's research/conversion/signal_probe.py (to be created per plan 008) — this plan re-runs it scoped to a single setup_version.
• READ-ONLY analysis + labeler rebuilds only. No live-path changes. DATABASE_URL = production; never source .env (extract DATABASE_URL only).

Commands you will need

Purpose	Command	Expected
Suite	DATABASE_URL=<url> python3 -m pytest -q	green (baseline: 15 WSL-loopback dashboard failures only)
ruff	ruff check --select F research/	clean
Rebuild a track	DATABASE_URL=<url> python3 -c "from research.labeler import build_<track>; from research.query import open_research_db; build_<track>(open_research_db('market'), rebuild=True)"	prints rows built

Scope

In scope:

• research/labeler.py — compute spy_intraday_return + tape_regime (replace the hardcoded None); add a thin-name RVOL fallback for the v1 build path
• research/ingestion/ — a daily SPY/QQQ ingest so market context has a source going forward (small addition; follow the existing daily-bar ingest pattern)
• Both compose scheduler blocks / .env.example — turn on NEWS_ENRICH_ENABLED and add SPY/QQQ ingest to the nightly job
• tests/unit/ — labeler feature-computation tests
• plans/README.md

Out of scope:

• ANY live gate/sizing/exit change; turning catalyst into a live gate (it is a FEATURE only — same rule as plan 008)
• "Fixing" impulse/pullback nulls on v1 (legitimately N/A for a breakout setup)
• Backfilling market context beyond the ~30 days of SPY data that exist (forward-fill only)
• Building the model (that is plan 008, gated on this plan's Step 5 result)

Steps

Step 1 — Compute market context (the "context outside the single-stock calculation")

In research/labeler.py, replace tape_regime=None, spy_intraday_return=None with computed values, for every track's build path that has a setup_time:

• spy_intraday_return = SPY's return from the session open to the setup's minute (from minute_bars where symbol='SPY', that session). Null when SPY bars are absent for that day (pre-30-day-window history).
• tape_regime = a small discrete classification from SPY's intraday path at trigger: e.g. trend_up / trend_down / chop based on SPY's return sign + whether it's making higher-highs vs oscillating (a simple rule; do not over-engineer — this is a feature to test, not a signal to trust).

This is the closest thing to the operator's "humanistic market context" — is the whole tape working with or against the trade. Add a daily SPY/QQQ ingest (research/ingestion/) wired into the nightly scheduler so this fills going forward.

Verify: rebuild one recent session's setups; SELECT count(*) FILTER (WHERE spy_intraday_return_pct_at_trigger IS NOT NULL), count(*) FROM setup_events WHERE session_date = '<a day with SPY data>' → non-null count > 0. Unit test: a synthetic SPY-up session yields spy_intraday_return > 0 and tape_regime='trend_up'.

Step 2 — RVOL fallback for thin names (v1)

In the v1 build path, when avg_vol is 0/null (thin IPO-class name), fall back to an alternative volume baseline rather than emitting relative_volume=None: e.g. the symbol's own prior-session total volume, or the median of available prior days, or a premarket-volume proxy — whichever the stored data supports. Record which baseline was used in a new nullable column or in an existing diagnostic field so the model can down-weight fallback RVOL. Never fabricate a baseline that isn't grounded in real volume.

Verify: after rebuild, SELECT avg((relative_volume IS NULL)::int) FROM setup_events WHERE setup_version='v1' drops well below 1.0. Unit test: a thin name with avg_vol=0 but real prior-day volume yields a non-null relative_volume.

Step 3 — Turn catalyst enrichment on (as a feature source)

• Set NEWS_ENRICH_ENABLED=true (loop + wherever the labeler builds) in both compose files + .env.example. Confirm OLLAMA_ENABLED + a reachable OLLAMA_HOST.
• Verify the enrichment actually writes catalyst events the labeler's _catalyst_at(...) reads, so catalyst_freshness_at_trigger / catalyst_type populate on newly-built setups.

Verify: after a night of enrichment (or a manual enrichment run), rebuild a recent session; SELECT avg((catalyst_freshness_at_trigger IS NULL)::int) FROM setup_events WHERE setup_version IN ('v1','pm1') AND session_date='<recent>' is materially below the current 0.97/0.96. If it stays ~1.0, STOP — enrichment isn't wired to the labeler; report the gap (that is a separate wiring fix, not a config flip).

Step 4 — Rebuild the instrumented tracks

Rebuild v1 and pm1 setups so the new features land (market context where SPY data exists; RVOL fallback; catalyst where enrichment has run). This is a labeler rebuild, not new trading.

Verify: per-version null audit (the query in Current state) shows relative_volume and spy_intraday_return null-rates dropped for v1 on the covered window; suite green.

Step 5 — GO/NO-GO: re-run the probe SCOPED to the live strategy

Re-run plan 008's signal_probe.py with a --setup-version v1 filter (and separately pm1), on the enriched, non-liq1 data:

• Same leakage-safe date-block CV, HistGradientBoostingClassifier, ROC-AUC + top-decile lift + permutation importance.

GO/NO-GO:

• v1 (or pm1) mean CV AUC ≥ ~0.55 with a consistent top-decile lift → signal exists in the instrumented live data → proceed to plan 008 Step 2 (build the model, shadow-first).
• Still ~0.50–0.52 after full instrumentation → STOP. This is the definitive answer: there is no trigger-time-knowable signal that separates ORB conversion, even fully instrumented. Redirect all effort to the premarket entry (a new setup), not a filter on ORB. Record the AUC + this conclusion in plans/README.md.

Verify: DATABASE_URL=<url> python3 -m research.conversion.signal_probe --setup-version v1 prints the scoped AUC; the go/no-go decision is recorded.

Test plan

• test_market_context_computed — synthetic SPY-up/down/chop sessions → correct spy_intraday_return sign + tape_regime.
• test_rvol_fallback_thin_name — avg_vol=0 + real prior-day volume → non-null relative_volume, fallback flagged.
• test_labeler_no_live_path_change — grep guard: the diff touches no gate/sizing/exit code (trading_execution.py, runtime/, run_live_paper.py gate paths unchanged).
• Verification: DATABASE_URL=<url> python3 -m pytest -q green; non-loopback failures = 0.

Done criteria

• tape_regime + spy_intraday_return computed (not None); SPY/QQQ daily ingest scheduled
• v1 relative_volume null-rate materially below 1.0 after rebuild
• NEWS_ENRICH_ENABLED on in both compose files; catalyst null-rate dropping on newly-built v1/pm1 (or STOP-reported if wiring is missing)
• Step 5 probe re-run scoped to v1 (and pm1); AUC + go/no-go decision recorded in plans/README.md
• git diff shows zero live gate/sizing/exit changes
• Suite green

STOP conditions

• Step 3 catalyst stays ~100% null after enabling enrichment → enrichment isn't wired to the labeler; report, don't force.
• Step 5 AUC still ~0.51 → STOP, record the definitive "no ORB signal" finding, recommend the premarket-setup pivot.
• Any step needs a live gate change → STOP (promotion is owner-gated).
• Market-context backfill tempts you to fetch >30 days of SPY history to "fix" coverage → don't; forward-fill is the honest scope, note the limited backfill window.

Maintenance notes

• Market context + catalyst accrue FORWARD; a fair Step-5 re-test wants a few weeks of post-enablement data. Re-running the probe monthly and watching AUC is the right cadence — one shot won't be decisive on the enriched features until n accrues.
• pm1 comes out of this the best-instrumented track — if the owner pivots to the premarket axis, the feature groundwork is already laid.
• The whole plan is falsifiable by design: if AUC stays flat, it ends the "surely some indicator works" thread with data, which is worth the effort even as a negative result.