Version: CI now tags images YYYY.MM.DD-<shortsha> alongside :latest/:sha
and passes GIT_SHA/BUILD_TAG build-args; the running app reports them via
dashboard /api/health and admin /api/status (TrueNAS custom apps only show
'custom/1.0.0', so the app self-reports what it actually runs).
Config parity: every scheduler-run script (nightly_tune, eod_replay,
premarket_eval, regime, market_context, experiments cli) now loads
config_overrides at startup like the live services do — a bare env would
replay under DEFAULT exits (the documented 4.5x-baseline landmine).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner preference: the bulk of TRADING_*/TRIGGER_* logic switches should live
in the DB, not the compose file (keys + connection info stay in compose —
the DB address cannot live inside the DB, and secrets don't belong in a
table the stack can SELECT). Precedence at boot: compose env < overrides.env
< config_overrides table; every entrypoint already routes through
env_overrides.load(), which now applies the table last. Admin /api/config
and profile-apply write the table (secrets whitelist unchanged); the config
page shows source=database. DB down = silent fall-through to file/compose,
boot never blocks on config. One-shot migration:
python -m runtime.env_overrides seed-from-env
run in any stack container to copy its current effective knobs into the
table, after which the compose lines are dead weight strippable at the next
redeploy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TRADING_EQUITY_FLOOR (0=off): equity at/below an absolute dollar floor
permanently halts the session and flattens the book — the total-drawdown
backstop the per-day breaker can't provide (a slow bleed of -2.9% days
never trips it). Same flatten path, loud risk event, bounce does not
re-open.
TRADING_BOOT_RECONCILE (default on): at boot, diff the broker's open
orders against today's submitted events — an open BUY on an unheld symbol
this process never submitted is a crash-restart orphan (the 2026-07-06
duplicate-order incident) and is cancelled. Orders on held symbols are
never touched (cancelling a bracket parent strips its protective stop).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dashboard backtest sources sessions from setup_events v1, which only the
labeler writes — and the labeler was never in the nightly scheduler, so
setups (and therefore backtests) froze at the last manual run (07-17).
Root blocker was _insert crashing on duplicate setup_ids on any re-run;
now ON CONFLICT DO NOTHING, so incremental nightly builds are safe.
Scheduler gains 'labeler build' + 'build-tod1' after eod_replay in both
compose files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 07-21 morning outage (loop booted post-EOD on the 20th, survived
midnight, silently blocked every entry while signals went READY) had zero
UI surface. New query_loop_session projection parses the live session id's
embedded date; a red banner fires when it isn't today's — the exact day-2
poison signature — telling the operator to restart the loop. Armed
Triggers rows now also show rvol_tod (the arming signal since 5597250)
next to legacy rvol.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 07-21 misses (CPHI +417%, DFNS +94%, GREE/WGRX +40%) were all static-gap
<10% names the validated gapper population excludes by design, but they share
a signature no refuted detector (liq1/run1) measured: rvol_tod (cum volume vs
median-20d prorated by elapsed session) at 50-1000x on a flat open. New
labeler setup tod1 fires on the rising edge of rvol_tod>=50 AND price>=
prev_close*1.20, restricted to static gap <10%, stop = 5-bar swing low,
scored via the shared _forward_labels. Shadow-only; pre-registered promotion
bar locked before forward data (median fwdmax60 >=+10%, median dd60 >-6%,
+1R>50%, top stratum rvol_tod>=200 clears alone).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SNXX 2026-07-21: 18s after a mid-morning restart the fast path bought a
15-minute-old break +1.6% above the trigger — fires() only checked
price >= trigger, so any blind window (boot, halt, feed outage) fired
instantly on already-extended moves. A trigger now fires only if the live
price was seen BELOW the trigger since arming (a real cross), with a
small at-the-break tolerance (TRIGGER_FRESH_CROSS_TOL, default 1.5%)
so a fast break between ticks still fires; beyond it the name must pull
back below the level and re-cross. Gap-through runners stay covered by
the slow marketable path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Day-22 forensic (BIYA session): 6 entry submissions, 0 fills — IREX missed
+1.7R, IRE +1.65R by one cent. Two root causes fixed:
1. Slow-path limit rested AT the trigger while price ran 2% past it, and
retries reused the same stale price. Marketable model now prices from
max(trigger, live last) via price_provider; the anti-chase extension
gate gets the live ref too (was an entry_ref==trigger no-op on this
path). TRADING_ENTRY_FILL_MODEL flipped to marketable.
2. Fast trigger path armed nothing all day: legacy rvol (cum-so-far /
full-day MEAN) is ~0.1-0.3 minutes after the open for a normal heavy
name and its mean is poisoned by reverse-split/prior-squeeze days
(BIYA legacy 1.27 vs real activity 481x median). New rvol_tod =
cum / (MEDIAN(20d) * elapsed-session fraction) drives arm eligibility
and the scanner keep-filter; legacy rvol untouched for validated
boundaries (gap-squeeze rvol>=5).
Also: stagnation time-stop (TRADING_EXIT_STAGNATION_MIN/_R, default OFF)
in simulate_exit + manage_live with sweep_exits.py variants for
validation before any live enable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner asks on the candlesticks:
- VOLUME: histogram series (bottom 20% overlay), coloured by candle direction.
- BOLLINGER BANDS: 20-period 2-sigma, computed client-side from closes; price
riding the upper band = strong, break back inside = fading (momentum context).
- EXIT markers were NEVER wired — the fills feed carries only entries
(order_submitted), and the loop fills brackets via position-diffing so no
order_filled fires. Exits live in position_closed; now surfaced as square
markers with reason + P&L (green win / red loss). Entry 'expected' (unfilled
submit) markers labelled distinctly. Fixed a self-inflicted bug: queried the
event store AFTER closing it — moved the position_closed fetch inside the
open-store block. 11 chart tests green incl. new exit-marker test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The self-heal commit (a00f967) inserted the _research method INSIDE __init__ at
method-body indent, truncating it so _change_signal/_backtest_lock/_backtest_
cache never got set -> AttributeError on backtest. Moved _research to a sibling
method; __init__ now sets all attrs. Two chart/sparkline tests updated: a None
research_con now REOPENS lazily, so 'no research db attached' is reached only
when the reopen also fails (they monkeypatch open to fail). 22 api tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The charts weren't broken — the data pipe was empty. dashboard_api opened the
research connection ONCE at boot; if the DB was unreachable then (container
started before Postgres, or a blip) it stayed None forever, so /api/chart and
/api/bars returned 'no research db attached' and every candlestick + sparkline
rendered blank. Live-verified: GET /api/chart returned bars:[] reason:'no
research db attached' on the deployed dashboard. Now DashboardState._research()
opens the connection LAZILY and reopens if None — same self-heal as the
event-store emit fix. 2 tests; chart/backtest endpoint tests still green.
No library change: Lightweight Charts v5.2 is fine, it just had nothing to draw.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test compared len(w) and list(w) as two SEPARATE locked reads, so the
churn thread could mutate between them (30 vs 31) — a bug in the test, not the
Watchlist (each read is individually consistent; there is no torn-read
guarantee ACROSS two calls). Now snapshots once and asserts that snapshot never
tears + concurrent contains/slice never raise. 5/5 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Telemetry: discovery took 6-30s ON the single scheduler, freezing ingest/arm
so the trigger read stale bars — the real 'late edge detection'. Now discovery
runs on its own daemon thread with a dedicated DB connection; the watchlist it
mutates is a lock-guarded Watchlist (list-like drop-in, no torn reads); the
scheduler discover step auto-disables (single-writer). Loop is now 3 fast
threads (entries/exits/discovery) + slow scheduler for tolerant work.
Also: nightly_tune writes a heartbeat every run, so the boot alert distinguishes
'tuner not running' (real) from 'ran, no profitable ORB config, kept params'
(expected — the strategy has no edge). Kills the misleading Telegram alarm.
543 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exits move off the single-threaded scheduler onto their own daemon thread at
1s (mirrors the fast trigger thread). Dedicated research connection (shared
facade isn't thread-safe); scheduler exits step auto-disabled when the thread
runs (single-caller invariant — no racing instance state, no double-submit);
manage() already idempotent (stop ratchets >1c up only). 3 driver tests + 539
suite green. On fast small-caps this is the 'get out fast' the owner asked for.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-07-17 20:23 postgres blip left the live loop writing zero events for
hours: emit() executed on a RAW cursor, bypassing PgConnection._run's
reconnect. cursor() doesn't round-trip so the dead conn went unnoticed until
the raw execute failed with no retry, and self._conn was never rebuilt.
Routing emit through the facade execute() wraps the round-trip in _run, so a
DB blip reconnects on the next write instead of running blind. 2 reconnect
tests + 34 storage tests green. (Deeper 'halt trading when persistently
unable to write' safety guard queued separately.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 017 Phase 0->3. Databento depth backfilled for 602 fires across 21
sessions -> depth_features table. Reweighted to true 23% population win rate,
gating on ask-overhang-above-break (keep>=0.034) lifts win rate 23->28% and
clears the random-gate null (+0.057 vs +0.05 on the conservative +1/-1
bracket). iv_depth_overhang_gate = keep-or-zero, skip-on-no-coverage (the
dilution-v1 lesson). This commit registers the REAL-exit harness run
(simulate_exit asymmetric trail) for the true live delta. Counter-intuitive
mechanism: more resting supply above the break = MORE likely to win
(absorption/fuel), invisible in bar data — the owner's 'size stacked above'
instinct, proven.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner saw a blank /backtest — it was CORRECT (the regime gate sat out every
recent session) but unexplained. Now: ?regime=0|1 override on /api/backtest,
an 'apply regime gate (as deployed)' checkbox, and an empty-state message
saying WHY there are no trades and what to click. Gated view = what the bot
would do; ungated = the raw simple criteria.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
admin_api.py called load_dotenv() unconditionally at module level, so
merely importing it (e.g. pytest collecting
tests/unit/test_admin_effective_config.py) ran python-dotenv's upward
find_dotenv() search, loaded the real repo .env (secrets included),
and polluted os.environ for the rest of the test session.
Gate the load_dotenv() call behind `if __name__ == "__main__":` at the
same source location, so it still runs before the module-level
os.environ.get(...) constants when run as the entrypoint (`python
admin_api.py` is unchanged) but never fires on a plain import.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
research/attribution.py's _connect() read os.environ["DATABASE_URL"]
directly, bypassing the storage facade fallback chain (DATABASE_URL,
PG* vars, LAN host retries) every other module goes through — and
KeyError-ing (uncaught by name, though still 500'd by the endpoint's
broad except) whenever DATABASE_URL was unset.
Delegate to storage.db.get_connection().raw instead, so DB resolution
matches EventStore/research.multi_schema, and a genuinely unreachable
DB fails with a catchable psycopg2.OperationalError rather than a
KeyError.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OrderFilledEvent had no side field, forcing api/main.py's chart() to
backfill it from the sibling order_submitted row. Add an additive-only
`side: str | None = None` field (existing payloads without it still
parse) and populate it at the sole production emit site
(alpaca_paper/execution.py _finalize) from the ExecutionRequest already
in scope. The api/main.py backfill workaround is left in place as a
harmless fallback for pre-existing events.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Null (the H2 lesson): zeroing share f of a negative baseline mechanically
yields expR_delta = f*|mean baseline|; a PASS on expR_delta_min now also
requires expR_delta_vs_null >= the same margin whenever zeroed_share >= 5%.
Exposure-shrinking can no longer masquerade as selection; non-gate
experiments unaffected. Honest disclosure: regime_meta_v1's FIRE-weighted
pass would not clear this bar either — its standing rests on the session-
level walk-forward OOS evidence, which no random gate reproduces.
H6: CKS order-flow-imbalance from quote_snapshots bid/ask SIZES —
size-weighted book imbalance over the 10-min pre-trigger window, zero
ask-tilted books; skip-on-no-coverage; judged under the null from day one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
H4 (3-book): keep-or-zero on trigger-bar volume concentration vs prior 5-bar
sum (>1.0 = completed one-shot burst). H3 (5-book, direction disputed):
fragility-camp primary — zero fires whose fill-time relative spread exceeds
2x the name's own 10-min median from quote_snapshots (which carry bid/ask
SIZES since 07-02 — the note's data-upgrade #1 was already satisfied).
Coverage design learns the dilution_veto_v1 lesson: no-quote fires are
SKIPPED (variant None, excluded from n), so thin coverage cannot false-FAIL.
H5 deliberately deferred until H1's verdict (its incremental value is defined
net of H1's catches). 6 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From docs/research/orderflow_notes.md — the order-flow shelf distillation.
H1 (5-book convergence): exit on first post-fill 1-min close back below the
trigger within 5 bars — the transitory-impact/recross signature — else the
standing exit stack. H2 (4-book): keep-or-zero on pre-trigger volume-weighted
bar-sign persistence (<0.30 over 15 bars = choppy one-shot burst, zeroed);
strictly-before-entry window, fail-open on thin history. Both pre-registered
at the house bar (+0.05R, n>=300, standard fidelity); nightly adjudication
decides. 7 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
replay_window reconstructed fires from ALL setup versions, so liq1/vr1/run1/pm1
(refuted shadow tracks, never traded live, 8:1 volume vs v1, tight-stop
artifacts) dominated /backtest: 17k trades / 45 sessions at -1.8R/trade —
garbage scale vs the harness-validated -0.08R/fire. reconstruct_fires_from_setups
gains an optional setup_version filter (default None: experiments unchanged);
replay_window pins it to v1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
premarket_eval's shadow enrichment call site queried session_date = current_date
(wrong on UTC-server Postgres at night, so it never matched any symbols) and
shared one try-block for float_cache + dilution filings (a float_cache failure
silently skipped the dilution refresh too). Fix: compute the ET session date
in Python, split into two independent try-blocks, lift into a testable
_refresh_shadow_enrichment(con, session_date). float_cache now applies its own
DDL unconditionally on use instead of only when an API key is present.
Adds iv_dilution_gate (additive, mirrors iv_regime_gate's keep-or-zero shape
keyed on symbol instead of session_date, fail-open on unfetched symbols) plus
its once-per-run preload, and registers the pre-registered dilution_veto_v1
experiment via the existing experiments/*.json self-registering cohort. Live
veto stays OFF; this only wires the shadow measurement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One playlist per add-playlist CLI call made adding a second playlist a chore.
Now YT_PLAYLIST_URLS takes a comma-separated list of playlist URLs (list=
param extracted) or bare ids; the transcripts service seeds the playlists
table from it at boot (insert-if-missing — hand-set labels are never
clobbered), and `python -m youtube_claims.run sync-playlists` does the same
on demand. Documented in both compose files; 3 tests.
Also: queue the two executor-surfaced defects (OrderFilledEvent side,
/api/attribution env hardcode) + the flaky nightly regime build-features
step in the plans index; regime_features 07-16 backfilled supervised.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a read-only /backtest page showing how the bot WOULD have traded past
sessions under the CURRENTLY configured live settings: theoretical ORB fires
from setup_events, replayed through the shared exit engine (simulate_exit +
ExitConfig.from_env(), same adverse-fill accounting the exit-experiment
harness uses), screened through the shared population gate and (optionally)
the live regime gate. No new simulation math — replay.replay_window() is pure
assembly of existing pieces (reconstruct_fires_from_setups, _fire_in_population,
_load_regime_favorable, simulate_exit); api/main.py adds the /backtest and
/api/backtest routes plus a dedicated-lock 10-min cache so a slow replay never
blocks the live dashboard's SSE stream. Also extends /api/chart with an
optional date= anchor so the trade drill-in chart modal can reach sessions
older than the existing 72h-from-now window, and adds a one-line BACKTEST nav
link to the live dashboard header.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Surfaces the entry_gate_evaluated decision receipts (plan 005) that were
write-only until now: a Decision Trace panel (per-gate value->threshold
chips, symbol filter, lazy-load + SSE refresh) and collapsed reject markers
merged into the plan-013 symbol chart's existing markers primitive.
- storage/projections.py: query_gate_receipts (case-insensitive symbol
filter, newest-first, same _hist_window idiom as query_fills_feed)
- api/main.py: GET /api/decisions (clamped limit<=500, date-scoped);
DashboardState.decisions() adds a server-computed epoch per row; factored
chart()'s inline to_epoch into a shared _to_epoch_et helper
- api/static/dashboard.html: Decision Trace panel + refreshChart now also
fetches /api/decisions and merges collapsed reject markers (same-reason
bursts within 5min -> one marker) into the single setMarkers call
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Click any symbol cell to open a 24h candlestick chart (vendored TradingView
Lightweight Charts v5.2, Apache-2.0) with buy/sell arrows at the bot's fills,
muted arrows for submitted-unfilled orders, and entry/stop/target price
lines. New read-only /api/chart endpoint (DashboardState.chart) converts
naive-ET minute_bars/event timestamps to UTC epoch seconds; query_fills_feed
gained an optional symbol filter. Chart refreshes off the existing SSE
snapshot signal (5s throttle, full replace) and tears down on close. Also
backfills order_filled markers' missing `side` (OrderFilledEvent's schema
has no side field) via an order_id join to the sibling order_submitted row.
Merged plan 012 (advisor/012-dashboard-order-limits) first — this plan
depends on its stopWorkingBy/targetBy/entryLimitBy maps.
The 07-15 live gate never made a real decision: live_session_favorable read
today's breadth_rvol from today's OWN setup_events rows, which only the
NIGHTLY labeler writes — always None intraday, fail-open every session (the
07-15 "validation block" was computed pre-guard on a zero-filled feature).
Worse, breadth_rvol was unit-corrupted: _session_context pooled all setup
versions and pm1 stores RAW cumulative premarket volume in relative_volume
(median ~261k) vs the ratio tracks (~0.8) — ~240/267 historical sessions'
breadth was actually median premarket volume, and plan 009's v1 RVOL fallback
was about to flip forward sessions to ratio units.
Fixes (TDD, 21 regime tests):
- breadth_rvol is now pm1-ONLY: median premarket volume — the dominant
historical source, unit-consistent, and strictly pre-open knowable.
- _live_breadth_rvol recomputes today's pm1 fires IN MEMORY from premarket
minute_bars (pure compute_premarket_setups, no insert); previous_close
falls back to the latest prior daily close when today's daily_bars row is
absent intraday (else the gap basis silently flips to the premarket open).
- _regime_reject: verdicts before 9:50 ET are PROVISIONAL (re-scored ~5 min;
ingest lag can truncate the premarket frame) and the first verdict at/after
9:50 is FINAL even when None — caps the scorer at a handful of runs/day
instead of ~78 on quiet days (hot-path stall guard). Throttle arithmetic
moved inside the fail-open try (naive/aware mixing must not escape).
- Live fit excludes breadth_rvol-NULL training rows (zero-fill taught a
"zero premarket volume" class the serve path can never produce).
Prod re-validation on the cleaned feature (regime_features rebuilt, scores
rewritten): regime_meta_v1 fire-weighted expR_delta +0.0570 (pre-registered
bar +0.05, n=2058, PASS — the contaminated +0.0965 was inflated), keeps 37%;
recent-80-session delta +0.149R/session (always −0.141 → gated +0.008).
First genuine live verdict 2026-07-16: UNFAVORABLE (pm breadth 612,819).
Known accepted skew: live scaled-IEX volumes vs nightly SIP tape — the
meta_live-vs-EOD comparison is the standing forward measure.
Also backfilled the missed 07-15 nightly job supervised (SIP refetch, 50 v1
+ 6 pm1 setups, SPY/QQQ ingest, regime row) — plan 009's accrual clock is
unbroken.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires the regime signal into the money path — fresh entries are BLOCKED on
sessions the walk-forward model scores unfavorable. Session-level: evaluated
once per day, cached. Enabled in both compose templates (paper).
Correctness (blocker 1 resolved): the LIVE feature set drops tape_regime (the
only non-strictly-open feature — AND measured 0.000 importance) + breadth_gap
(also 0.000). The 5 remaining features are all non-look-ahead at entry time:
breadth_rvol = median(relative_volume=vol_upto/avg_vol_20d) over the session's
setups so far (the SAME up-to-trigger calc the live bot runs, NOT day-end),
spy_* lagged to prior close, trailing from PRIOR sessions. The live single-
session fit reproduces the batch meta_v1 verdict exactly on the last 6 sessions.
Safety: fail-open everywhere — disabled, unfit model (<60 prior sessions),
missing breadth_rvol (pre-setup), or ANY exception returns None and never
blocks. The gate can only SUBTRACT trades on bad-regime days.
- research/regime.py: live_session_favorable() (refit on priors <today, predict
today; writes regime_scores['meta_live'] for forward validation) + LIVE_FEATURE_COLS
+ the breadth_rvol-missing fail-open guard.
- trading_execution.py: TRADING_REGIME_GATE_ENABLED / _P_MIN settings + _regime_reject()
(cached per session) + a REGIME gate after SESSION in _evaluate_entry_gates.
- docker-compose.yml + truenas-compose.yml: TRADING_REGIME_GATE_ENABLED=1.
- tests/unit/test_regime_gate_live.py: block / pass / fail-open(x3) / scored-once-per-day.
KNOWN train/serve skew (documented, validate forward via meta_live vs EOD label):
model trains on full-day breadth_rvol, live uses setups-so-far. Small edge on a
~266-session sample; paper only. Suite: 504 passed, 15 pre-existing loopback fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
research/regime.py: Step 0 probe (labelable-session/SPY-coverage STOP gate),
Step 1 build-features (regime_features: pre-open breadth/tape + prior-close
SPY momentum/vol + trailing bot expR, label = mean _sim_baseline_r over the
session's setup fires) + walk-forward threshold_v1 rule, Step 2 walk-forward
meta_v1 (shallow sklearn GBM), Step 4 OOS/leakage-audit helpers
(oos_split/sequential_folds/session_expr_delta/full_sample_fit_threshold).
Step 3 (HMM) deferred per scope — statsmodels stays absent.
replay.py: new iv_regime_gate intervention (keep-or-zero a fire on a
pre-computed session regime score, fail-open on no score row) wired into
_compute_fire + run(), mirroring rank_one_bullet's shape.
experiments/regime_threshold_v1.json + regime_meta_v1.json register the two
model variants against the shared population/bars/fidelity contract.
All new code is unit-tested against isolated EventStore(":memory:") schemas
only — no production DB read or written. Live-trading paths untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
iv_exit_config replays ANY target/breakeven/trail combo through the shared exit
engine vs the live-exit baseline — reusable way to adjudicate exit numbers.
Counterfactual over setup fires: v1 (slow, 1.37R median winners) gains most from
prior_low trailing (win 17->24%, +0.036R); pm1 (fast, 2.64R median, 10R tail)
gains most from NO target + step0.5 (+0.273R, win 24->34%) — the fixed target
throws away pm1's fat tail. Registered both as tracked experiments. +test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
research/conversion/{__init__,dataset,signal_probe}.py — the leakage-safe probe
plan 009 Step 5 re-runs scoped to a live setup_version:
- dataset.load_dataset: setup_events JOIN outcome_labels; label =
reached_1r_before_minus_1r; features = trigger-time-knowable setup_events
columns ONLY, dropping >70%-null and <2-distinct columns. HARD leakage rule —
every outcome_labels column is blocklisted AND asserted absent from X, so a
schema change can't leak the answer in.
- signal_probe.run_probe: expanding-window CV over ~5 sequential session-date
blocks (whole days, train strictly precedes test), HistGradientBoostingClassifier
(max_iter=250, lr=0.05, max_depth=4, l2=1.0, categorical_features=indices),
per-fold + mean ROC-AUC, top-decile +1R lift vs base, permutation importance on
the last fold. Drops columns constant within a train slice (HistGBM binning).
CLI: python3 -m research.conversion.signal_probe [--setup-version v1].
Tests: the critical leakage guard (zero outcome columns in features), AUC>0.5 on a
synthetic signal + ~0.5 on pure noise, and a static live-path guard. All :memory:.
No live-path change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 3 of plan 009 (config + code; no live-path change):
- research/ingestion/market_context.py: pulls SPY/QQQ full-session minute bars
(SIP, post-close embargo-safe) + daily bars via the existing upsert helpers, so
the labeler's market-context features have a FORWARD source (index ETFs never
enter the gapper universe the live loop ingests). Mirrors sip_backfill's shape.
- both compose scheduler blocks: run it after premarket_eval, guarded || echo.
- NEWS_ENRICH_ENABLED on: added to docker-compose x-app-env anchor (loop +
scheduler inherit it); truenas-compose + .env.example already had it. Catalyst
is a FEATURE source only, never a live gate; enabled-but-down never blocks
trading. Enrichment OUTPUT is verified by the orchestrator (needs live Ollama).
Compose statically validated (yaml.safe_load). Ingest self-checked via a
fake-client :memory: round-trip through _preload_spy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Steps 1-2 of plan 009 — fill two fillable feature gaps so a conversion model
gets a fair, instrumented test:
- market context: _spy_context computes spy_intraday_return (SPY RTH open ->
trigger-minute return) + a coarse tape_regime {trend_up,trend_down,chop} from
SPY's sign + higher-highs structure. Replaces the hardcoded
tape_regime=None/spy_intraday_return=None; wired into ALL five build paths
(v1/vr1/run1/liq1/pm1) via _preload_spy. Null when SPY has no RTH bars at/before
the trigger (pre-30d history, or a premarket fire before SPY's RTH open).
- thin-name RVOL fallback (v1): when avg_vol is 0/null (thin IPO-class, no 20d
baseline) fall back to the median of prior sessions' total volume (real
daily_bars.volume, knowable pre-open); flagged in new setup_events column
relative_volume_source ('avg_vol_20d' | 'prior_days_median_volume' | NULL).
Never fabricates a baseline.
A feature to TEST, not a signal to trust. No live gate/sizing/exit change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
eod_replay was the wrong tool to grow the harness sample — it only PRINTS a
backtest report (persists no fires) and get_most_actives is today-biased for
historical dates. The right source was already in the DB: 18,164 setup_events,
each carrying the ORB entry (entry_reference_price) + stop (invalidation_price).
reconstruct_fires_from_setups turns every setup into a simulated fire; the exit
experiments (volume_fade, time_stop, asym_exit) replay over its minute-bar path
with baseline = the LIVE exit config simulated on the same path (_sim_baseline_r,
same adverse-fill accounting) — a clean same-path delta needing no live P&L.
Gated by EXPERIMENTS_INCLUDE_SETUP_SIM (default on), capped by
EXPERIMENTS_SETUP_SIM_LIMIT for chunked backfill; rank_one_bullet excluded (it
needs live receipts). Grows every exit experiment's n from ~15 to thousands.
+test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The live loop fills resting-limit brackets via position-snapshot diffing and
emits NO order_filled (verified: 0 all-time in prod), so reconstruct_fires
skipped every fire and the ENTIRE harness replayed nothing (0 run rows).
Reconstruction now falls back to the order_submitted timestamp for entry timing
and reads entry/stop/exit from position_closed — restores the whole flagship
(0 -> 324 run rows on prod; pm1's overdue FAIL verdict now posts).
Adds the volume_fade intervention (exit to lock the gain when a bar's volume
drops below fade_frac x the entry-burst average, once in profit; original stop
still caps downside; adverse-tick fills on entry AND exit) + volume_fade_exit_v1
(pre-registered bars/futility/fidelity). Interim n=15 read: -0.68R vs baseline
(INCONCLUSIVE, min_n=100) — the harness answers 'does volume-fade help?' from
data before any live change. +6 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scheduler ran 'experiments run/adjudicate' but nothing registered the five
experiments/*.json into the experiments table, so on a fresh DB the harness
replayed over an empty set and the ledger stayed empty forever (owner hit this
on first deploy). registry.register_cohort() registers every cohort JSON
idempotently (skips already-registered, tolerates a malformed file); 'run'
calls it at startup so a fresh deployment self-bootstraps with no manual
register step. +test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compute_ratios bails when UTC now-16min precedes today's 08:00Z, i.e. nightly
~00:00-04:16 ET; the test's hardcoded 2026-07-06 bars made it KeyError at the
date rollover independent of any code change. Freeze the module clock to a
mid-session UTC time so the window is valid deterministically.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 6: tests/smoke_dashboard_headless.py renders the real page in a headless
browser and fails on any JS console/pageerror (standalone script, not
pytest-collected; CI non-blocking first). admin_api gains GET /api/config/effective
(per-key masked value + source: overrides.env vs compose, missing .env.example
keys flagged as drift) + an admin.html table (values XSS-escaped). dashboard.html
gains the Experiment Ledger panel reading the plan-006 verdicts from the snapshot
(empty-state until the tables exist). Removes the last CI ruff per-file pin
(api/main.py:F821) now that os/hmac are imported; scoped ruff is green unpinned.
Marks plan 007 DONE in plans/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DashboardState.bars() short-circuits to [] unless a research_con is attached; the
embedded orchestrator passed one but the standalone dashboard_api.py never built
it, so production sparklines were permanently dead. The missing wire was purely
the entrypoint (DATABASE_URL already reaches the market schema) — now it opens
open_research_db("market") read-only and passes it, degrading to None (charts
show "no research db") if it can't connect. Verified live: bars() returns real
minute rows through the standalone reader.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Badge now derives PAPER/LIVE from broker truth (base-URL host) instead of the
schwab_live literal that read PAPER while go-live swapped only ALPACA_BASE_URL;
live flips body.live-mode (top shimmer -> solid red) with a tooltip of host +
account. Adds the header FLATTEN ALL (two-step arm/confirm, 5s auto-disarm) and
PAUSE ENTRIES toggle (+ amber banner), per-position EXIT buttons (two-step), and
an operator-token field sent as X-Dashboard-Token on every POST (also fixes
approve/reject when a token is set). Action controls gate on
has_execution || command_bus; kill-switch controls on command_bus. No keyboard
shortcut for any destructive action. New styling is :root-token-only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds OperatorCommandEvent + a closed command vocabulary (flatten_all,
flatten_symbol, pause_entries, resume_entries). POST /api/command emits it
(token-required: 503 when DASHBOARD_TOKEN unset — commands never run
unauthenticated); the live loop's new 5s "command" step consumes them via
OperatorCommandConsumer with three replay-safety layers (in-session nonce set,
persisted operator_command_done reload, 10-min staleness window).
flatten_all routes through close_session DIRECTLY (never step_eod_flatten) so it
cannot trip plan 003's daily-restart exit. Pause is checked at the top of BOTH
request_approvals_for_ready_signals AND submit_breakout_now (missing either
leaks pause on that path). qabot gains an authorized-chat-only /flatten slash
command (prose "flatten" still refuses via _ACTION_WORDS). Snapshot exposes
broker truth + command_bus + a defensive experiment ledger.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every SSE client polled COUNT(*) on a fresh EventStore connection each ~1s
(re-applying the 473-line schema unless PG_SKIP_SCHEMA=1). Replaced with one
process-wide _ChangeSignal: a single long-lived connection reads max(timestamp)
(idx_events_timestamp tail seek) at most once per second GLOBALLY behind a lock,
so N clients cost one query/sec. EventStore.max_timestamp() backs it; reader is
injectable for the query-budget test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every POST endpoint (approve/reject/exit/watch) crashed with NameError:
do_POST references os.environ + hmac.compare_digest but neither was imported.
Adds the imports, a non-loopback regression test for the token gate (no token
-> routes; token set -> 401 without/with-wrong header, 200 with correct), and
plumbs DASHBOARD_TOKEN + PG_SKIP_SCHEMA=1 into both compose dashboard services
(+ .env.example docs for DASHBOARD_TOKEN / COMMAND_BUS_ENABLED).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 4. experiments/*.json for time_stop_10m_v1, probe_add_v1,
asym_exit_v1, rank_one_bullet_v1, pm1_v1 — bars verbatim from the plan table,
production population, fidelity floors {0.25, 3, 1} from the research doc. Also
hardened evaluate(): n==0 and any uncomputable bar metric -> INCONCLUSIVE (not a
false FAIL). Cohort registers + survives run->adjudicate end-to-end. 25 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 3. evaluate() scores runs against pre-registered bars
(_min/_max/min_n vocabulary) -> PASS/FAIL/KILLED/INCONCLUSIVE; terminal
verdicts flip status, emit an experiment_verdict event, and Telegram-ping ONCE
(terminal experiments are never re-adjudicated). Metric vocabulary covers
expR_delta/rate_1r plus pm1's median_up_60m/median_dd_60m/big_gap_rate_1r.
backfill_pm1 bridges pm1 from outcome_labels JOIN setup_events (variant +/-1,
baseline NULL). premarket_eval dual-writes the verdict through the harness while
keeping the legacy TSV. 8 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 2. Reconstructs fires from the event stream (submit[buy] -> fill
-> close), population-filters via strategy/population.py, and applies three pure
interventions counterfactually on the bar path: time_stop, probe_add, asym_exit
(the last through strategy.exits.simulate_exit ONLY). Hard fidelity: +1-tick
adverse fills on entry AND stop, spread_proxy = median quiet-bar high-low, stop
floor = max(atr_frac*ATR14, spread_mult*spread_proxy). rank_one_bullet is
receipts-only and starves (documented) without plan-005 rank. Idempotent via PK
+ ON CONFLICT. Anti-slippage-artifact regression (test #2) green: a
zero-slippage-profitable path LOSES under adverse fills.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
