Owner preference: the bulk of TRADING_*/TRIGGER_* logic switches should live
in the DB, not the compose file (keys + connection info stay in compose —
the DB address cannot live inside the DB, and secrets don't belong in a
table the stack can SELECT). Precedence at boot: compose env < overrides.env
< config_overrides table; every entrypoint already routes through
env_overrides.load(), which now applies the table last. Admin /api/config
and profile-apply write the table (secrets whitelist unchanged); the config
page shows source=database. DB down = silent fall-through to file/compose,
boot never blocks on config. One-shot migration:
python -m runtime.env_overrides seed-from-env
run in any stack container to copy its current effective knobs into the
table, after which the compose lines are dead weight strippable at the next
redeploy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 07-22 'tuner NOT running' alert could not be diagnosed remotely: the
heartbeat is a file on the scheduler container's volume, invisible to
everything else, and indistinguishable from a stale scheduler image or a
DATA_DIR mismatch. _write_heartbeat now also inserts a telemetry_events
row (best-effort); the boot staleness check falls back to the DB row when
the file is absent, so 'tuner ran, no profitable config' stops looking
like 'tuner dead'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docs/LIVE-CUTOVER.md: full flip-day procedure — staged pieces, discovered
dependencies (second paper key pair, DB seeding, learned_params copy,
telegram token, ports/dirs), 15-min cutover with the paper stack running
throughout, verification list, rollback. bootstrap_live_db.sh seeds the
live stack's Postgres from the paper nightly dump and truncates the
money-record tables so research history carries over while live P&L
starts at zero.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TRADING_EQUITY_FLOOR (0=off): equity at/below an absolute dollar floor
permanently halts the session and flattens the book — the total-drawdown
backstop the per-day breaker can't provide (a slow bleed of -2.9% days
never trips it). Same flatten path, loud risk event, bounce does not
re-open.
TRADING_BOOT_RECONCILE (default on): at boot, diff the broker's open
orders against today's submitted events — an open BUY on an unheld symbol
this process never submitted is a crash-restart orphan (the 2026-07-06
duplicate-order incident) and is cancelled. Orders on held symbols are
never touched (cancelling a bracket parent strips its protective stop).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prices each closed round-trip against the nearest quote_snapshots NBBO:
per-trade spread cost, book-walk flag (order size > displayed ask), and a
cost-adjusted P&L total. Measured this week: 0.3-0.5% median spreads on
gate-passing names, ~12% profit drag — far below the static 2-4% microcap
proxy. Coverage gaps (quote recorder only follows tracked names) reported
per-trade, not silently dropped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The stored minute-bar history is 273 sessions (~13 months); the 45-session
clamp blocked months-scale ranges now that the nightly labeler keeps
setup_events current. Requests stay serialized behind _backtest_lock; a
full-history run takes minutes by operator choice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dashboard backtest sources sessions from setup_events v1, which only the
labeler writes — and the labeler was never in the nightly scheduler, so
setups (and therefore backtests) froze at the last manual run (07-17).
Root blocker was _insert crashing on duplicate setup_ids on any re-run;
now ON CONFLICT DO NOTHING, so incremental nightly builds are safe.
Scheduler gains 'labeler build' + 'build-tod1' after eod_replay in both
compose files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The prior commit imported query_loop_session but a botched idempotency
guard skipped inserting the snapshot field (the import string itself
matched the 'already present' check) — ruff unused-import failed CI and
the stale-session banner had no data. Field added; banner live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 07-21 morning outage (loop booted post-EOD on the 20th, survived
midnight, silently blocked every entry while signals went READY) had zero
UI surface. New query_loop_session projection parses the live session id's
embedded date; a red banner fires when it isn't today's — the exact day-2
poison signature — telling the operator to restart the loop. Armed
Triggers rows now also show rvol_tod (the arming signal since 5597250)
next to legacy rvol.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 07-21 misses (CPHI +417%, DFNS +94%, GREE/WGRX +40%) were all static-gap
<10% names the validated gapper population excludes by design, but they share
a signature no refuted detector (liq1/run1) measured: rvol_tod (cum volume vs
median-20d prorated by elapsed session) at 50-1000x on a flat open. New
labeler setup tod1 fires on the rising edge of rvol_tod>=50 AND price>=
prev_close*1.20, restricted to static gap <10%, stop = 5-bar swing low,
scored via the shared _forward_labels. Shadow-only; pre-registered promotion
bar locked before forward data (median fwdmax60 >=+10%, median dd60 >-6%,
+1R>50%, top stratum rvol_tod>=200 clears alone).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SNXX 2026-07-21: 18s after a mid-morning restart the fast path bought a
15-minute-old break +1.6% above the trigger — fires() only checked
price >= trigger, so any blind window (boot, halt, feed outage) fired
instantly on already-extended moves. A trigger now fires only if the live
price was seen BELOW the trigger since arming (a real cross), with a
small at-the-break tolerance (TRIGGER_FRESH_CROSS_TOL, default 1.5%)
so a fast break between ticks still fires; beyond it the name must pull
back below the level and re-cross. Gap-through runners stay covered by
the slow marketable path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A process booted AFTER the day's EOD flatten never gets exit-after-flatten,
survives midnight, and every memory-only daily flag (_session_closed, entry
caps, benches) stays stale — the documented day-2 limitation, hit live at the
2026-07-21 open (booted 17:11 the day before; signals READY + book armed but
zero gate evaluations, both entry paths silently blocked by _session_closed).
Main loop now exits cleanly when the session date rolls over; compose restarts
a fresh process that rebuilds the new day's state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Day-22 forensic (BIYA session): 6 entry submissions, 0 fills — IREX missed
+1.7R, IRE +1.65R by one cent. Two root causes fixed:
1. Slow-path limit rested AT the trigger while price ran 2% past it, and
retries reused the same stale price. Marketable model now prices from
max(trigger, live last) via price_provider; the anti-chase extension
gate gets the live ref too (was an entry_ref==trigger no-op on this
path). TRADING_ENTRY_FILL_MODEL flipped to marketable.
2. Fast trigger path armed nothing all day: legacy rvol (cum-so-far /
full-day MEAN) is ~0.1-0.3 minutes after the open for a normal heavy
name and its mean is poisoned by reverse-split/prior-squeeze days
(BIYA legacy 1.27 vs real activity 481x median). New rvol_tod =
cum / (MEDIAN(20d) * elapsed-session fraction) drives arm eligibility
and the scanner keep-filter; legacy rvol untouched for validated
boundaries (gap-squeeze rvol>=5).
Also: stagnation time-stop (TRADING_EXIT_STAGNATION_MIN/_R, default OFF)
in simulate_exit + manage_live with sweep_exits.py variants for
validation before any live enable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner asks on the candlesticks:
- VOLUME: histogram series (bottom 20% overlay), coloured by candle direction.
- BOLLINGER BANDS: 20-period 2-sigma, computed client-side from closes; price
riding the upper band = strong, break back inside = fading (momentum context).
- EXIT markers were NEVER wired — the fills feed carries only entries
(order_submitted), and the loop fills brackets via position-diffing so no
order_filled fires. Exits live in position_closed; now surfaced as square
markers with reason + P&L (green win / red loss). Entry 'expected' (unfilled
submit) markers labelled distinctly. Fixed a self-inflicted bug: queried the
event store AFTER closing it — moved the position_closed fetch inside the
open-store block. 11 chart tests green incl. new exit-marker test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The self-heal commit (a00f967) inserted the _research method INSIDE __init__ at
method-body indent, truncating it so _change_signal/_backtest_lock/_backtest_
cache never got set -> AttributeError on backtest. Moved _research to a sibling
method; __init__ now sets all attrs. Two chart/sparkline tests updated: a None
research_con now REOPENS lazily, so 'no research db attached' is reached only
when the reopen also fails (they monkeypatch open to fail). 22 api tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The charts weren't broken — the data pipe was empty. dashboard_api opened the
research connection ONCE at boot; if the DB was unreachable then (container
started before Postgres, or a blip) it stayed None forever, so /api/chart and
/api/bars returned 'no research db attached' and every candlestick + sparkline
rendered blank. Live-verified: GET /api/chart returned bars:[] reason:'no
research db attached' on the deployed dashboard. Now DashboardState._research()
opens the connection LAZILY and reopens if None — same self-heal as the
event-store emit fix. 2 tests; chart/backtest endpoint tests still green.
No library change: Lightweight Charts v5.2 is fine, it just had nothing to draw.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test compared len(w) and list(w) as two SEPARATE locked reads, so the
churn thread could mutate between them (30 vs 31) — a bug in the test, not the
Watchlist (each read is individually consistent; there is no torn-read
guarantee ACROSS two calls). Now snapshots once and asserts that snapshot never
tears + concurrent contains/slice never raise. 5/5 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Telemetry: discovery took 6-30s ON the single scheduler, freezing ingest/arm
so the trigger read stale bars — the real 'late edge detection'. Now discovery
runs on its own daemon thread with a dedicated DB connection; the watchlist it
mutates is a lock-guarded Watchlist (list-like drop-in, no torn reads); the
scheduler discover step auto-disables (single-writer). Loop is now 3 fast
threads (entries/exits/discovery) + slow scheduler for tolerant work.
Also: nightly_tune writes a heartbeat every run, so the boot alert distinguishes
'tuner not running' (real) from 'ran, no profitable ORB config, kept params'
(expected — the strategy has no edge). Kills the misleading Telegram alarm.
543 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exits move off the single-threaded scheduler onto their own daemon thread at
1s (mirrors the fast trigger thread). Dedicated research connection (shared
facade isn't thread-safe); scheduler exits step auto-disabled when the thread
runs (single-caller invariant — no racing instance state, no double-submit);
manage() already idempotent (stop ratchets >1c up only). 3 driver tests + 539
suite green. On fast small-caps this is the 'get out fast' the owner asked for.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner: 'make all decisions as fast as possible.' The real entry-detection
ceiling was ingest — bars refreshed every 20s, so the 4s trigger thread read
data up to 20s stale and detected breakouts late. Ingest is incremental+cheap,
safe at 5s. Fresher bars + faster arm (trigger levels) + 2s trigger check +
the 4s exits shipped earlier = the whole ingest->detect->decide->exit loop
tightened. One batched data call per cycle so Alpaca rate limits are fine.
Sub-second exit management on a dedicated thread is the next (code) step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner's real complaint: late exits gave back gains. Exits ran on the slow
scheduler at 12s while entries run on the 4s fast thread — exits were 3x
slower than entries. A small-cap can spike and round-trip inside 12s. Drop
TRADING_EXIT_MANAGE_INTERVAL_SECONDS to 4s (config, reversible, both compose
files). Deeper fix (exit mgmt on the sub-second trigger thread) planned
separately. Entry chase-tail (24% fill >5% above break) is the next
investigation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sliced 2191 depth fires gap x overhang under real exit: lower-gap is WORST
(0-3% -0.831R), inverting the low-gap-edge hypothesis. Overhang helps a little
everywhere, nothing crosses zero. Exhaustive: 13 levers + regime + depth +
this grid = zero positive. No positive subpopulation in gap-ORB $2-20.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Definitive test, n=2191 real deployed exit: ask-overhang separates +0.083R
(high -0.676 vs low -0.759, direction held sniff->scale = genuinely real) but
fails the random-gate null — it shrinks losses, never creates profit. Live
gap>=10% band n=13/74 = no signal. Same shape as regime gate on a negative-EV
base. 13 conversion levers + regime + depth tested, zero make it positive.
Real-money rec: NO. Depth edge argues for a broader-population redesign as
future research, not a live fix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-07-17 20:23 postgres blip left the live loop writing zero events for
hours: emit() executed on a RAW cursor, bypassing PgConnection._run's
reconnect. cursor() doesn't round-trip so the dead conn went unnoticed until
the raw execute failed with no retry, and self._conn was never rebuilt.
Routing emit through the facade execute() wraps the round-trip in _run, so a
DB blip reconnects on the next write instead of running blind. 2 reconnect
tests + 34 storage tests green. (Deeper 'halt trading when persistently
unable to write' safety guard queued separately.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 017 Phase 0->3. Databento depth backfilled for 602 fires across 21
sessions -> depth_features table. Reweighted to true 23% population win rate,
gating on ask-overhang-above-break (keep>=0.034) lifts win rate 23->28% and
clears the random-gate null (+0.057 vs +0.05 on the conservative +1/-1
bracket). iv_depth_overhang_gate = keep-or-zero, skip-on-no-coverage (the
dilution-v1 lesson). This commit registers the REAL-exit harness run
(simulate_exit asymmetric trail) for the true live delta. Counter-intuitive
mechanism: more resting supply above the break = MORE likely to win
(absorption/fuel), invisible in bar data — the owner's 'size stacked above'
instinct, proven.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
First genuine selection signal in the project. 40 real fires, Databento
XNAS.ITCH book at each trigger: naive imbalance = noise (AUC 0.45/0.50), but
ask-size resting within 1% ABOVE the break separates winners at AUC 0.80
(winner median 0.276 vs loser 0.005; median split 27% vs 67% win). Inverted
from supply-trap intuition = absorption/fuel mechanism, invisible in OHLCV.
n=23 single-day sniff -> GO to Phase 1 recorder, not to trading. Cost: pennies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gate is damage-control, not an edge source: it can only allocate the
strategy's existing (negative) edge onto least-bad days — best case flat,
never gain. Its value is CONDITIONAL: a holding pattern until an entry edge
(plan 017 L2 depth) is found, at which point it flips to force-multiplier
(good names x good days). Owner's 'it just turns the bot off' critique is
correct at the current no-edge base.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cost-checked (no data spent): DATABENTO_API_KEY reaches XNAS.ITCH TotalView
10-level depth; every real gapper (EHGO/ATPC/MUD/LASE/AEON) covered at
$0.0001-0.0066/fire, full 20-fire sniff <$1 inside free credits. Catalogs
all 13 provisioned-but-unwired keys with a wire-on-demand triage (Databento
= game-changer; 4 news APIs = skip; rest redundant).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one untested selection axis (order-book depth = the owner's 'size stacked
above the break' instinct). Staged so $0 is spent until a free sniff test +
recorder prove the signal exists and covers the universe; paid feed is an
explicit owner gate; the signal faces the same pre-registered bar + random-gate
null as every refuted lever before it touches the live entry. Honest ceiling
stated: a PASS nudges odds, is not an autonomous oracle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner saw a blank /backtest — it was CORRECT (the regime gate sat out every
recent session) but unexplained. Now: ?regime=0|1 override on /api/backtest,
an 'apply regime gate (as deployed)' checkbox, and an empty-state message
saying WHY there are no trades and what to click. Gated view = what the bot
would do; ungated = the raw simple criteria.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
admin_api.py called load_dotenv() unconditionally at module level, so
merely importing it (e.g. pytest collecting
tests/unit/test_admin_effective_config.py) ran python-dotenv's upward
find_dotenv() search, loaded the real repo .env (secrets included),
and polluted os.environ for the rest of the test session.
Gate the load_dotenv() call behind `if __name__ == "__main__":` at the
same source location, so it still runs before the module-level
os.environ.get(...) constants when run as the entrypoint (`python
admin_api.py` is unchanged) but never fires on a plain import.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
research/attribution.py's _connect() read os.environ["DATABASE_URL"]
directly, bypassing the storage facade fallback chain (DATABASE_URL,
PG* vars, LAN host retries) every other module goes through — and
KeyError-ing (uncaught by name, though still 500'd by the endpoint's
broad except) whenever DATABASE_URL was unset.
Delegate to storage.db.get_connection().raw instead, so DB resolution
matches EventStore/research.multi_schema, and a genuinely unreachable
DB fails with a catchable psycopg2.OperationalError rather than a
KeyError.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OrderFilledEvent had no side field, forcing api/main.py's chart() to
backfill it from the sibling order_submitted row. Add an additive-only
`side: str | None = None` field (existing payloads without it still
parse) and populate it at the sole production emit site
(alpaca_paper/execution.py _finalize) from the ExecutionRequest already
in scope. The api/main.py backfill workaround is left in place as a
harmless fallback for pre-existing events.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Null (the H2 lesson): zeroing share f of a negative baseline mechanically
yields expR_delta = f*|mean baseline|; a PASS on expR_delta_min now also
requires expR_delta_vs_null >= the same margin whenever zeroed_share >= 5%.
Exposure-shrinking can no longer masquerade as selection; non-gate
experiments unaffected. Honest disclosure: regime_meta_v1's FIRE-weighted
pass would not clear this bar either — its standing rests on the session-
level walk-forward OOS evidence, which no random gate reproduces.
H6: CKS order-flow-imbalance from quote_snapshots bid/ask SIZES —
size-weighted book imbalance over the 10-min pre-trigger window, zero
ask-tilted books; skip-on-no-coverage; judged under the null from day one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
H2 exposed an adjudicator gap: on a -0.21R/fire baseline any keep-or-zero
gate zeroing ~44% of fires clears +0.05 mechanically; its own pre-registered
strata check inverts (choppy beats persistent within both gap buckets =
Simpson's). Queued: random-gate null for keep-or-zero bars. H3 accruing as
designed (n=29, skip-on-no-coverage working).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
H4 (3-book): keep-or-zero on trigger-bar volume concentration vs prior 5-bar
sum (>1.0 = completed one-shot burst). H3 (5-book, direction disputed):
fragility-camp primary — zero fires whose fill-time relative spread exceeds
2x the name's own 10-min median from quote_snapshots (which carry bid/ask
SIZES since 07-02 — the note's data-upgrade #1 was already satisfied).
Coverage design learns the dilution_veto_v1 lesson: no-quote fires are
SKIPPED (variant None, excluded from n), so thin coverage cannot false-FAIL.
H5 deliberately deferred until H1's verdict (its incremental value is defined
net of H1's catches). 6 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From docs/research/orderflow_notes.md — the order-flow shelf distillation.
H1 (5-book convergence): exit on first post-fill 1-min close back below the
trigger within 5 bars — the transitory-impact/recross signature — else the
standing exit stack. H2 (4-book): keep-or-zero on pre-trigger volume-weighted
bar-sign persistence (<0.30 over 15 bars = choppy one-shot burst, zeroed);
strictly-before-entry window, fail-open on thin history. Both pre-registered
at the house bar (+0.05R, n>=300, standard fidelity); nightly adjudication
decides. 7 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
9 sources (Hasbrouck, Bouchaud x2, Cartea, Dalton, Bellafiore, Neill, CKS,
Cetin-Danilova) read by parallel agents + adversarial synthesis, constrained
to computable data (1-min SIP bars, 15s quotes) and fenced from the 8 refuted
levers. Companion to regime_notes.md. H1 (trigger-erasure fast invalidation,
5-book convergence) is the lead candidate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
replay_window reconstructed fires from ALL setup versions, so liq1/vr1/run1/pm1
(refuted shadow tracks, never traded live, 8:1 volume vs v1, tight-stop
artifacts) dominated /backtest: 17k trades / 45 sessions at -1.8R/trade —
garbage scale vs the harness-validated -0.08R/fire. reconstruct_fires_from_setups
gains an optional setup_version filter (default None: experiments unchanged);
replay_window pins it to v1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
