# Plan 009: Instrument the data so a conversion model gets ONE fair test — fill the three feature gaps, then re-run the go/no-go

> **Executor instructions**: This is the prerequisite to plan 008. It fixes the
> data-quality reasons the 2026-07-14 signal probe was inconclusive, then re-runs
> that probe scoped to the live strategy. It is GATED: the final step's AUC
> decides whether modeling is ever worth pursuing. Run every verification; honor
> STOP conditions; update `plans/README.md` when done.
>
> **Drift check**: `git diff --stat d28fe25..HEAD -- research/labeler.py research/ingestion/ config.py` — the labeler is the substrate; re-confirm the excerpts below if it moved.

## Status

- **Priority**: P2 (real payoff, but a multi-week accrual play against a negative prior — read "Honest framing")
- **Effort**: L (mostly forward data accrual; the code is M)
- **Risk**: LOW to live trading — all of this is labeler/feature plumbing + a config flag; no live gate/sizing/exit changes
- **Depends on**: 006 (harness), 008 (the probe this re-runs)
- **Category**: research-infrastructure / data-quality
- **Planned at**: commit `d28fe25`, 2026-07-14

## Honest framing (read before starting — this is not a quick win)

The 2026-07-14 probe (plan 008) found **CV AUC 0.514 ≈ noise** on 63,803 labeled setups. Investigation showed the test was crippled by data quality, not necessarily by absence of signal:

- 85% of rows are `liq1` (a *refuted* track); the live ORB `v1` is only 6,217 rows and converts at 24%.
- Per-version null audit:
  - `v1` (live ORB): has `gap_pct`, `above_vwap_flag`, `vwap_at_trigger`, `session_minute_number`; **missing `relative_volume` (100% null), catalyst, market-context.** (Pullback/impulse fields are legitimately N/A for a breakout setup — do not "fix" those.)
  - `pm1` (premarket): the **best-instrumented** track — `gap_pct`, `relative_volume`, `impulse_pct` all populated; only catalyst missing. Also the one unexplored strategy axis.
  - `tape_regime_at_trigger` and `spy_intraday_return_pct_at_trigger` are **100% null on every track** — hardcoded `None` in the labeler.

This plan fills the three *fillable* gaps so a model gets one fair, fully-instrumented test. **Set expectations honestly:** even fully instrumented, the prior is strongly negative (7 refuted single levers + this joint-model probe), `v1` is small (~6k) and low-converting, and the market-context + catalyst features can only accrue **forward** (thin history), so a fair re-test needs a few weeks of data. If the re-run still returns ~0.51, that is the **definitive** answer — stop filtering ORB and build the premarket entry (which this plan also leaves better-instrumented). The alternative to running this plan is to skip straight to the premarket-setup work; this plan is worth it only if the owner wants modeling to get its one clean shot first.

## Current state

- Labeler: `research/labeler.py`. The v1/ORB `SetupEvent` build (~lines 288–316) computes `rvol = (vol_upto / avg_vol) if avg_vol else None` — so `relative_volume` is None whenever `avg_vol` (average daily volume) is 0/null, which is the norm for thin IPO-class small caps (documented blind spot; see the `ignition.py` comment "scan_gappers RVOL zeros out on thin IPOs, avg-daily-vol = 0").
- Market context: `research/labeler.py:~322` literally sets `tape_regime=None, spy_intraday_return=None`. These features are declared in `setup_events` but never populated.
- SPY/QQQ data: `SELECT count(*), count(DISTINCT session_date) FROM minute_bars WHERE symbol IN ('SPY','QQQ')` → 53,452 rows across **only 30 sessions**. So market-context backfill is limited to ~30 days; forward-fill requires ingesting SPY/QQQ daily going forward.
- Catalyst: `config.py` `OllamaConfig`; enrichment gated by `NEWS_ENRICH_ENABLED` (off in the deployed compose — verified). `catalyst_freshness_at_trigger` / `catalyst_type` columns exist but are 73–97% null because enrichment isn't running. `_catalyst_at(...)` in the labeler already reads catalyst events IF they exist.
- The probe: plan 008's `research/conversion/signal_probe.py` (to be created per plan 008) — this plan re-runs it scoped to a single `setup_version`.
- READ-ONLY analysis + labeler rebuilds only. No live-path changes. `DATABASE_URL` = production; never source `.env` (extract `DATABASE_URL` only).

## Commands you will need

| Purpose | Command | Expected |
|---|---|---|
| Suite | `DATABASE_URL=<url> python3 -m pytest -q` | green (baseline: 15 WSL-loopback dashboard failures only) |
| ruff | `ruff check --select F research/` | clean |
| Rebuild a track | `DATABASE_URL=<url> python3 -c "from research.labeler import build_<track>; from research.query import open_research_db; build_<track>(open_research_db('market'), rebuild=True)"` | prints rows built |

## Scope

**In scope**:
- `research/labeler.py` — compute `spy_intraday_return` + `tape_regime` (replace the hardcoded `None`); add a thin-name RVOL fallback for the v1 build path
- `research/ingestion/` — a daily SPY/QQQ ingest so market context has a source going forward (small addition; follow the existing daily-bar ingest pattern)
- Both compose scheduler blocks / `.env.example` — turn on `NEWS_ENRICH_ENABLED` and add SPY/QQQ ingest to the nightly job
- `tests/unit/` — labeler feature-computation tests
- `plans/README.md`

**Out of scope**:
- ANY live gate/sizing/exit change; turning catalyst into a live gate (it is a FEATURE only — same rule as plan 008)
- "Fixing" impulse/pullback nulls on v1 (legitimately N/A for a breakout setup)
- Backfilling market context beyond the ~30 days of SPY data that exist (forward-fill only)
- Building the model (that is plan 008, gated on this plan's Step 5 result)

## Steps

### Step 1 — Compute market context (the "context outside the single-stock calculation")

In `research/labeler.py`, replace `tape_regime=None, spy_intraday_return=None` with computed values, for every track's build path that has a `setup_time`:
- `spy_intraday_return` = SPY's return from the session open to the setup's minute (from `minute_bars` where `symbol='SPY'`, that session). Null when SPY bars are absent for that day (pre-30-day-window history).
- `tape_regime` = a small discrete classification from SPY's intraday path at trigger: e.g. `trend_up` / `trend_down` / `chop` based on SPY's return sign + whether it's making higher-highs vs oscillating (a simple rule; do not over-engineer — this is a feature to test, not a signal to trust).

This is the closest thing to the operator's "humanistic market context" — is the whole tape working with or against the trade. Add a daily SPY/QQQ ingest (`research/ingestion/`) wired into the nightly scheduler so this fills going forward.

**Verify**: rebuild one recent session's setups; `SELECT count(*) FILTER (WHERE spy_intraday_return_pct_at_trigger IS NOT NULL), count(*) FROM setup_events WHERE session_date = '<a day with SPY data>'` → non-null count > 0. Unit test: a synthetic SPY-up session yields `spy_intraday_return > 0` and `tape_regime='trend_up'`.

### Step 2 — RVOL fallback for thin names (v1)

In the v1 build path, when `avg_vol` is 0/null (thin IPO-class name), fall back to an alternative volume baseline rather than emitting `relative_volume=None`: e.g. the symbol's own prior-session total volume, or the median of available prior days, or a premarket-volume proxy — whichever the stored data supports. Record which baseline was used in a new nullable column or in an existing diagnostic field so the model can down-weight fallback RVOL. Never fabricate a baseline that isn't grounded in real volume.

**Verify**: after rebuild, `SELECT avg((relative_volume IS NULL)::int) FROM setup_events WHERE setup_version='v1'` drops well below 1.0. Unit test: a thin name with `avg_vol=0` but real prior-day volume yields a non-null `relative_volume`.

### Step 3 — Turn catalyst enrichment on (as a feature source)

- Set `NEWS_ENRICH_ENABLED=true` (loop + wherever the labeler builds) in both compose files + `.env.example`. Confirm `OLLAMA_ENABLED` + a reachable `OLLAMA_HOST`.
- Verify the enrichment actually writes catalyst events the labeler's `_catalyst_at(...)` reads, so `catalyst_freshness_at_trigger` / `catalyst_type` populate on newly-built setups.

**Verify**: after a night of enrichment (or a manual enrichment run), rebuild a recent session; `SELECT avg((catalyst_freshness_at_trigger IS NULL)::int) FROM setup_events WHERE setup_version IN ('v1','pm1') AND session_date='<recent>'` is materially below the current 0.97/0.96. If it stays ~1.0, STOP — enrichment isn't wired to the labeler; report the gap (that is a separate wiring fix, not a config flip).

### Step 4 — Rebuild the instrumented tracks

Rebuild `v1` and `pm1` setups so the new features land (market context where SPY data exists; RVOL fallback; catalyst where enrichment has run). This is a labeler rebuild, not new trading.

**Verify**: per-version null audit (the query in Current state) shows `relative_volume` and `spy_intraday_return` null-rates dropped for v1 on the covered window; suite green.

### Step 5 — GO/NO-GO: re-run the probe SCOPED to the live strategy

Re-run plan 008's `signal_probe.py` with a `--setup-version v1` filter (and separately `pm1`), on the enriched, non-`liq1` data:
- Same leakage-safe date-block CV, HistGradientBoostingClassifier, ROC-AUC + top-decile lift + permutation importance.

**GO/NO-GO:**
- **v1 (or pm1) mean CV AUC ≥ ~0.55 with a consistent top-decile lift** → signal exists in the instrumented live data → proceed to plan 008 Step 2 (build the model, shadow-first).
- **Still ~0.50–0.52 after full instrumentation** → **STOP. This is the definitive answer:** there is no trigger-time-knowable signal that separates ORB conversion, even fully instrumented. Redirect all effort to the premarket entry (a new setup), not a filter on ORB. Record the AUC + this conclusion in `plans/README.md`.

**Verify**: `DATABASE_URL=<url> python3 -m research.conversion.signal_probe --setup-version v1` prints the scoped AUC; the go/no-go decision is recorded.

## Test plan

- `test_market_context_computed` — synthetic SPY-up/down/chop sessions → correct `spy_intraday_return` sign + `tape_regime`.
- `test_rvol_fallback_thin_name` — `avg_vol=0` + real prior-day volume → non-null `relative_volume`, fallback flagged.
- `test_labeler_no_live_path_change` — grep guard: the diff touches no gate/sizing/exit code (`trading_execution.py`, `runtime/`, `run_live_paper.py` gate paths unchanged).
- Verification: `DATABASE_URL=<url> python3 -m pytest -q` green; non-loopback failures = 0.

## Done criteria

- [ ] `tape_regime` + `spy_intraday_return` computed (not `None`); SPY/QQQ daily ingest scheduled
- [ ] v1 `relative_volume` null-rate materially below 1.0 after rebuild
- [ ] `NEWS_ENRICH_ENABLED` on in both compose files; catalyst null-rate dropping on newly-built v1/pm1 (or STOP-reported if wiring is missing)
- [ ] Step 5 probe re-run scoped to v1 (and pm1); AUC + go/no-go decision recorded in `plans/README.md`
- [ ] `git diff` shows zero live gate/sizing/exit changes
- [ ] Suite green

## STOP conditions

- Step 3 catalyst stays ~100% null after enabling enrichment → enrichment isn't wired to the labeler; report, don't force.
- Step 5 AUC still ~0.51 → STOP, record the definitive "no ORB signal" finding, recommend the premarket-setup pivot.
- Any step needs a live gate change → STOP (promotion is owner-gated).
- Market-context backfill tempts you to fetch >30 days of SPY history to "fix" coverage → don't; forward-fill is the honest scope, note the limited backfill window.

## Maintenance notes

- Market context + catalyst accrue FORWARD; a fair Step-5 re-test wants a few weeks of post-enablement data. Re-running the probe monthly and watching AUC is the right cadence — one shot won't be decisive on the enriched features until n accrues.
- `pm1` comes out of this the best-instrumented track — if the owner pivots to the premarket axis, the feature groundwork is already laid.
- The whole plan is falsifiable by design: if AUC stays flat, it ends the "surely some indicator works" thread with data, which is worth the effort even as a negative result.