plans(010): premarket shadow-entry — harness-gated, fill-honest, never live until proven

The one unexplored setup (premarket/pm1). Step-1 go/no-go: does pm1 + the
validated fat-tail exit beat the ORB AFTER honest thin-book fills (real
spread_pct, fat premarket adverse)? If not, STOP — premarket is a paper-fill
mirage. Shadow-first; TRADING_SESSION_ONLY stays ON; no live order until a
forward shadow edge survives realistic slippage (owner-gated).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

plans/010-premarket-shadow-entry.md

@@ -0,0 +1,119 @@
+# Plan 010: Premarket entry — prove a FILL-ABLE edge in the harness first, then run it shadow-first, never live until it clears realistic slippage
+
+> **Executor instructions**: This plan is GATED at Step 1. Step 1 measures
+> whether premarket entry beats the ORB *after honest thin-book fills*. If it
+> doesn't, STOP — do not build the live shadow track. Nothing here trades live;
+> `TRADING_SESSION_ONLY` stays ON throughout. Run every verification; update
+> `plans/README.md` when done.
+>
+> **Drift check**: `git diff --stat 97dcef4..HEAD -- research/ strategy/ trading_execution.py` — the harness + labeler are the substrate.
+
+## Status
+
+- **Priority**: P2 (the one genuinely unexplored setup — but fill-reality is the make-or-break, so measure before building)
+- **Effort**: L
+- **Risk**: LOW to live — all shadow; no live gate/sizing/exit change; `TRADING_SESSION_ONLY` unchanged (the shadow track never submits)
+- **Depends on**: 006 (harness), 009 (exit-config intervention — the validated pm1 exit)
+- **Category**: direction / new-setup
+- **Planned at**: commit `97dcef4`, 2026-07-15
+
+## Why this matters (and why it's dangerous)
+
+The ORB's 5-min wait is deliberate — it lets the market *confirm* a level instead of guessing from premarket. Premarket entry trades the guess, and this session's evidence is mixed:
+
+- **For it:** pm1 (the premarket detector, 2,445 shadow setups) winners are big and fast — median **2.64R**, peak **22 min**, tail to 10R. A validated exit already exists (`exit_pm1_fattail`: no target + `trail_r_step=0.5`, +0.273R over the live exit).
+- **Against it — the traps that kill naive premarket backtests:**
+  1. **Alpaca paper LIES about premarket fills** — it assumes you fill at your price; real thin premarket books often don't fill, or fill far worse. A paper backtest *overstates* premarket edge, sometimes entirely.
+  2. **Thin liquidity** — wide spreads, real slippage. pm1's `entry_reference_price` assumes a clean fill at the break; premarket you pay the spread.
+  3. **Premarket signal is weak-to-anti-predictive** — the in-repo research + 7 refuted levers: premarket volume/gap are anti-predictive after RVOL; higher premarket volume → *harder* fade.
+  4. pm1 as a raw population is **~breakeven** even optimally-exited (−0.03R).
+  5. Entering premarket conflicts with `TRADING_SESSION_ONLY=1` — the gate added to stop after-hours submissions (a real past incident). The shadow track must NOT lift it.
+
+So the discipline: **the fill model is the whole experiment.** If pm1 + the fat-tail exit + *honest* thin-book fills still beats the ORB, there's a real setup. If honest fills erase the edge (likely), premarket is a mirage and we've learned it for free — no live risk.
+
+## Current state
+
+- pm1 setups: `research.labeler.build_premarket` / `compute_premarket_setups` — fire on premarket bars, carry `entry_reference_price` (the premarket break level) + `invalidation_price` (stop); `outcome_labels` measure the forward move **within premarket** (strict time split). 2,445 setups in `setup_events` (`setup_version='pm1'`).
+- Harness fill model: `research/experiments/replay.py` — `_adverse(fidelity) = adverse_ticks * TICK` (`TICK=0.01`). Interventions apply +adverse on entry AND exit. The validated pm1 exit is the `exit_config` intervention (plan 009) with `{target_r:0, trail_r_step:0.5}`.
+- Real spread data exists: `minute_bars.spread_pct` (per-bar) — the honest premarket fill model uses it instead of a flat 1-tick.
+- Session gate: `trading_execution._outside_session` returns a skip when `settings.session_only` (`TRADING_SESSION_ONLY`, default 1) and outside RTH. Live entries are blocked premarket. **Leave this ON.**
+- `DATABASE_URL` = production; READ-ONLY analysis + a shadow labeler track. No live-path change.
+
+## Commands you will need
+
+| Purpose | Command | Expected |
+|---|---|---|
+| Suite | `DATABASE_URL=<url> python3 -m pytest -q` | green (15 WSL-loopback dashboard failures only; non-loopback = 0) |
+| Register/run/adjudicate | `python3 -m research.experiments {register <json>\|run\|adjudicate}` | as documented |
+| ruff | `ruff check --select F research/` | clean |
+
+## Scope
+
+**In scope**:
+- `research/experiments/replay.py` — a **premarket-aware fill model**: a `fidelity.spread_mult`-driven adverse that uses each fire's real `minute_bars.spread_pct` (not a flat tick) for premarket fires; a new `exit_config`-style experiment param or a small helper. Additive; existing interventions unchanged.
+- `experiments/premarket_entry_v1.json` — the go/no-go experiment (Step 1)
+- `research/labeler.py` — ONLY if Step 2 runs: a real-time shadow premarket-entry emission (log would-be fires as a shadow event; never an order)
+- Both compose scheduler blocks — ONLY if Step 2 runs: nothing new needed (the harness already runs)
+- tests under `tests/unit/experiments/` and `tests/unit/`
+- `plans/README.md`
+
+**Out of scope**:
+- ANY live order premarket. `TRADING_SESSION_ONLY` stays 1. The shadow track logs, never submits.
+- Lifting the session gate, thin-liquidity live sizing, real-money premarket — all owner-gated, post-shadow-proof only.
+- Replacing the ORB. Premarket runs PARALLEL as a shadow track.
+- Trusting Alpaca paper premarket fills — the fill model must be pessimistic by construction.
+
+## Steps
+
+### Step 1 — GO/NO-GO: does premarket beat the ORB after HONEST fills?
+
+1. Add a premarket-aware fill to `replay.py`: for a fire, `adverse = max(adverse_ticks*TICK, spread_mult * (spread_pct * entry_price))` where `spread_pct` is the fire's entry-bar `minute_bars.spread_pct` (real thin-book spread). This makes premarket fills pay the actual spread, not a flat penny.
+2. Register `experiments/premarket_entry_v1.json`: population = pm1's, intervention = the validated fat-tail exit (`exit_config` `{target_r:0, breakeven_at_pct:0.05, trail_r_step:0.5}`), **fidelity with a fat premarket spread mult** (e.g. `spread_mult: 5, adverse_ticks: 2`). Pre-registered bar: its expR must beat the ORB (`v1`) baseline expR by a margin that survives the fill penalty (e.g. `expR_delta_min` vs the v1 exit baseline, `min_n: 300`).
+3. Run + adjudicate over the pm1 setup fires.
+
+**GO/NO-GO:**
+- **premarket_entry_v1 expR (with fat fills) > the ORB's, by the pre-registered margin** → a real fill-able edge → proceed to Step 2 (build the live shadow track).
+- **The fat-fill expR collapses to ≤ the ORB, or negative** → **STOP and report.** Premarket doesn't clear the slippage hurdle; the 22-min-fat-tail was a paper-fill mirage. This is a valid, complete outcome — it saves you from shipping a thin-book money-loser.
+
+**Verify**: `python3 -m research.experiments run && … adjudicate` shows `premarket_entry_v1`'s verdict + its expR vs the v1 baseline, side by side.
+
+### Step 2 — Live SHADOW premarket-entry track (only if Step 1 passed)
+
+Build a real-time shadow that, during premarket, detects the pm1 entry condition on live bars and **emits a `premarket_shadow_fire` event** (symbol, would-be entry, stop, timestamp) — logging only, NO order, `TRADING_SESSION_ONLY` untouched. This captures what a live premarket entry WOULD have done, in real time (vs the labeler's after-the-fact pm1), so forward fills are measured on the real book, not reconstructed.
+
+**Verify**: during a premarket session, `premarket_shadow_fire` events appear; `git diff` shows zero order-submission or session-gate changes; the loop still takes zero premarket live entries.
+
+### Step 3 — Forward adjudication (only after Step 2 accrues)
+
+Let the shadow fires accrue a few weeks; register a forward experiment comparing the shadow fires' realized-on-real-bars outcome (with the fat fill model) against the ORB. Re-adjudicate monthly.
+
+**GO/NO-GO:** forward shadow edge survives realistic fills → THEN (and only then) present the owner a live-promotion decision (lift session gate for premarket, thin-liquidity sizing, real-money caveats). Absent that, premarket stays shadow.
+
+## Test plan
+
+- `test_premarket_fill_pays_spread` — a fire with a wide `spread_pct` gets a bigger adverse than a 1-tick fire; a zero-spread fire matches the tick floor.
+- `test_premarket_experiment_registers_and_runs` — `premarket_entry_v1` registers, runs over synthetic pm1 fires, adjudicates.
+- `test_shadow_fire_emits_no_order` (Step 2) — the shadow path emits a `premarket_shadow_fire` event and NO order event; `TRADING_SESSION_ONLY` respected.
+- Verification: `DATABASE_URL=<url> python3 -m pytest -q` green; non-loopback failures = 0.
+
+## Done criteria
+
+- [ ] Premarket-aware fill model added (uses real `spread_pct`); `test_premarket_fill_pays_spread` green
+- [ ] `premarket_entry_v1` registered + adjudicated; its expR-vs-ORB recorded in `plans/README.md`
+- [ ] If NO-GO: the "premarket doesn't clear the fill hurdle" finding is logged — a complete outcome
+- [ ] If GO: the live shadow track emits `premarket_shadow_fire` and submits ZERO orders (`git diff` proves no order/session-gate change)
+- [ ] `TRADING_SESSION_ONLY` unchanged everywhere
+- [ ] Suite green
+
+## STOP conditions
+
+- Step 1 fat-fill expR ≤ ORB → STOP, report "premarket is a paper-fill mirage".
+- Any step tempts a live premarket order or lifting `TRADING_SESSION_ONLY` → HARD STOP (owner-gated promotion only).
+- The fill model can't source a real `spread_pct` for premarket bars (column null premarket) → report; use a conservative fixed premarket spread (e.g. 1%) and note it, do NOT fall back to a 1-tick fill (that recreates the paper-fill lie).
+- pm1 outcome_labels measure only the *within-premarket* move — if a realistic premarket strategy would hold into the regular session, the labels understate/overstate; report the mismatch before trusting the verdict.
+
+## Maintenance notes
+
+- The fill model is the load-bearing assumption. If it's too kind, this whole plan lies. Prefer pessimistic — a premarket edge that survives a fat spread is real; one that needs clean fills isn't tradeable on your account.
+- Premarket promotion, if it ever happens, is a bigger live change than any exit tweak (new session window, thin-book sizing, real-money slippage) — it gets its own owner-gated plan, never an env flip.
+- If Step 1 says NO-GO, that closes the "trade premarket early instead of the ORB" thread with data — as valuable as a yes, and it points back to: the ORB + the validated trailing exit is your strategy; the edge is discipline, not a new entry.

plans/README.md

@@ -24,6 +24,7 @@ your row when done.
 | 006 | Experiment OS — pre-registered counterfactual experiments over logged fires (flagship) | P1 | L | 004 (005 rec.) | DONE |
 | 007 | Operator control plane — dashboard truth + kill switch + effective config (step 1 = prod bug fix, ship early) | P2 | M | 001, 003 (006 for ledger data) | DONE |
 | 008 | Conversion-signal model — measure if the labeled data separates winners | P1 | L | 006, 004 | **MEASURED 2026-07-14 (corrected): the 0.514 was the liq1-dominated MIX. Scoped to live v1: CV AUC 0.641 — REAL signal, but it is ~entirely the VWAP gate the bot ALREADY runs (above-VWAP +1R 35.4% vs below 12.7%). Residual within above-VWAP = 0.526 ≈ noise. RVOL-fallback added (read-only test 2026-07-14): 0.526→0.535, within noise — RVOL adds nothing beyond VWAP. Verdict: VWAP is the whole story for current features; no second indicator. Only market-context + catalyst remain untested (forward-accrual only, negative prior). The lever is a different setup (premarket), not more ORB filters.** |
+| 010 | Premarket shadow entry — prove a FILL-ABLE edge in the harness before any live premarket order; shadow-first, session-gate stays ON | P2 | L | 006, 009 | TODO — Step-1 go/no-go = does pm1+fat-tail-exit beat the ORB after honest thin-book fills; STOP if not |
 | 009 | Feature-population prerequisite — instrument v1/pm1 (market context, RVOL fallback, catalyst-on), then re-run 008's probe | P2 | L | 006, 008 | code DONE, awaiting supervised rebuild+probe (market context + thin-name RVOL fallback + SPY/QQQ ingest + enrichment-on + committed probe all landed & tested; Steps 4-5 rebuild/AUC run supervised) |
 
 Status values: TODO | IN PROGRESS | DONE | BLOCKED (with one-line reason) | REJECTED (with one-line rationale)