One playlist per add-playlist CLI call made adding a second playlist a chore.
Now YT_PLAYLIST_URLS takes a comma-separated list of playlist URLs (list=
param extracted) or bare ids; the transcripts service seeds the playlists
table from it at boot (insert-if-missing — hand-set labels are never
clobbered), and `python -m youtube_claims.run sync-playlists` does the same
on demand. Documented in both compose files; 3 tests.
Also: queue the two executor-surfaced defects (OrderFilledEvent side,
/api/attribution env hardcode) + the flaky nightly regime build-features
step in the plans index; regime_features 07-16 backfilled supervised.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a read-only /backtest page showing how the bot WOULD have traded past
sessions under the CURRENTLY configured live settings: theoretical ORB fires
from setup_events, replayed through the shared exit engine (simulate_exit +
ExitConfig.from_env(), same adverse-fill accounting the exit-experiment
harness uses), screened through the shared population gate and (optionally)
the live regime gate. No new simulation math — replay.replay_window() is pure
assembly of existing pieces (reconstruct_fires_from_setups, _fire_in_population,
_load_regime_favorable, simulate_exit); api/main.py adds the /backtest and
/api/backtest routes plus a dedicated-lock 10-min cache so a slow replay never
blocks the live dashboard's SSE stream. Also extends /api/chart with an
optional date= anchor so the trade drill-in chart modal can reach sessions
older than the existing 72h-from-now window, and adds a one-line BACKTEST nav
link to the live dashboard header.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Surfaces the entry_gate_evaluated decision receipts (plan 005) that were
write-only until now: a Decision Trace panel (per-gate value->threshold
chips, symbol filter, lazy-load + SSE refresh) and collapsed reject markers
merged into the plan-013 symbol chart's existing markers primitive.
- storage/projections.py: query_gate_receipts (case-insensitive symbol
filter, newest-first, same _hist_window idiom as query_fills_feed)
- api/main.py: GET /api/decisions (clamped limit<=500, date-scoped);
DashboardState.decisions() adds a server-computed epoch per row; factored
chart()'s inline to_epoch into a shared _to_epoch_et helper
- api/static/dashboard.html: Decision Trace panel + refreshChart now also
fetches /api/decisions and merges collapsed reject markers (same-reason
bursts within 5min -> one marker) into the single setMarkers call
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
createSeriesMarkers(series, markers) attaches a NEW primitive to the series
on every call — refreshChart() called it on every 5s SSE-driven refresh, so
a modal left open accumulated one primitive per refresh (identical markers
overdraw so it looked fine, but it's an unbounded render leak on long-open
modals). v5 pattern: create the markers primitive once in openChart, mutate
it via markersApi.setMarkers(markers) on refresh. chart.remove() in
closeChart already tears the primitive down with the chart.
Click any symbol cell to open a 24h candlestick chart (vendored TradingView
Lightweight Charts v5.2, Apache-2.0) with buy/sell arrows at the bot's fills,
muted arrows for submitted-unfilled orders, and entry/stop/target price
lines. New read-only /api/chart endpoint (DashboardState.chart) converts
naive-ET minute_bars/event timestamps to UTC epoch seconds; query_fills_feed
gained an optional symbol filter. Chart refreshes off the existing SSE
snapshot signal (5s throttle, full replace) and tears down on close. Also
backfills order_filled markers' missing `side` (OrderFilledEvent's schema
has no side field) via an order_id join to the sibling order_submitted row.
Merged plan 012 (advisor/012-dashboard-order-limits) first — this plan
depends on its stopWorkingBy/targetBy/entryLimitBy maps.
sync.py fetches status="all" so the legacy stopBy map (kept for the
R-bar/spark stop line, unchanged) can pick up a cancelled stop leg.
Add stopWorkingBy, filtered to the same WORKING statuses as
targetBy/entryLimitBy, and point the new Positions Stop column at it
so a cancelled leg never renders as live protection.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Positions table gains Stop/Target columns from resting bracket legs
(targetBy/entryLimitBy maps, filtered to working order status). Order
Lifecycle splits the generic Price column into Type/Limit/Stop/Fill via
a client-side join to the broker order snapshot (client_order_id ==
the internal order_id we send Alpaca) since the lifecycle events
themselves never captured limit/stop/type; the truncated order-id
column is dropped in favor of a title attribute on the Symbol cell.
Approval queue drops the always-"auto" Mode column and shows explicit
entry/stop prices in the risk chip instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner-requested UI work, advisor-planned for executor handoff:
- 012: show working limit/stop/target prices on positions + order lifecycle,
cut dead columns (order-id, approval Mode)
- 013: symbol drill-in — 24h candlestick + live buy/sell fill markers
(TradingView Lightweight Charts v5.2, vendored, Apache-2.0)
- 014: live decision trace — surface entry_gate_evaluated receipts as a feed
(value→threshold per gate) + chart reject markers
- 015: /backtest replay page — theoretical entries/exits over past sessions
under CURRENT settings, assembled strictly from Experiment-OS primitives
All four are read-path only; zero live-loop surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 07-15 live gate never made a real decision: live_session_favorable read
today's breadth_rvol from today's OWN setup_events rows, which only the
NIGHTLY labeler writes — always None intraday, fail-open every session (the
07-15 "validation block" was computed pre-guard on a zero-filled feature).
Worse, breadth_rvol was unit-corrupted: _session_context pooled all setup
versions and pm1 stores RAW cumulative premarket volume in relative_volume
(median ~261k) vs the ratio tracks (~0.8) — ~240/267 historical sessions'
breadth was actually median premarket volume, and plan 009's v1 RVOL fallback
was about to flip forward sessions to ratio units.
Fixes (TDD, 21 regime tests):
- breadth_rvol is now pm1-ONLY: median premarket volume — the dominant
historical source, unit-consistent, and strictly pre-open knowable.
- _live_breadth_rvol recomputes today's pm1 fires IN MEMORY from premarket
minute_bars (pure compute_premarket_setups, no insert); previous_close
falls back to the latest prior daily close when today's daily_bars row is
absent intraday (else the gap basis silently flips to the premarket open).
- _regime_reject: verdicts before 9:50 ET are PROVISIONAL (re-scored ~5 min;
ingest lag can truncate the premarket frame) and the first verdict at/after
9:50 is FINAL even when None — caps the scorer at a handful of runs/day
instead of ~78 on quiet days (hot-path stall guard). Throttle arithmetic
moved inside the fail-open try (naive/aware mixing must not escape).
- Live fit excludes breadth_rvol-NULL training rows (zero-fill taught a
"zero premarket volume" class the serve path can never produce).
Prod re-validation on the cleaned feature (regime_features rebuilt, scores
rewritten): regime_meta_v1 fire-weighted expR_delta +0.0570 (pre-registered
bar +0.05, n=2058, PASS — the contaminated +0.0965 was inflated), keeps 37%;
recent-80-session delta +0.149R/session (always −0.141 → gated +0.008).
First genuine live verdict 2026-07-16: UNFAVORABLE (pm breadth 612,819).
Known accepted skew: live scaled-IEX volumes vs nightly SIP tape — the
meta_live-vs-EOD comparison is the standing forward measure.
Also backfilled the missed 07-15 nightly job supervised (SIP refetch, 50 v1
+ 6 pm1 setups, SPY/QQQ ingest, regime row) — plan 009's accrual clock is
unbroken.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The docker-image workflow's test gate has aborted at collection since sklearn
was introduced (test_conversion_probe.py, plan 008), so no image published.
Regime gate (plan 011) also needs sklearn at runtime in the image, else the
live gate silently fail-opens forever. Adds scikit-learn to requirements.txt
(installed by both the CI test job and the Dockerfile).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wires the regime signal into the money path — fresh entries are BLOCKED on
sessions the walk-forward model scores unfavorable. Session-level: evaluated
once per day, cached. Enabled in both compose templates (paper).
Correctness (blocker 1 resolved): the LIVE feature set drops tape_regime (the
only non-strictly-open feature — AND measured 0.000 importance) + breadth_gap
(also 0.000). The 5 remaining features are all non-look-ahead at entry time:
breadth_rvol = median(relative_volume=vol_upto/avg_vol_20d) over the session's
setups so far (the SAME up-to-trigger calc the live bot runs, NOT day-end),
spy_* lagged to prior close, trailing from PRIOR sessions. The live single-
session fit reproduces the batch meta_v1 verdict exactly on the last 6 sessions.
Safety: fail-open everywhere — disabled, unfit model (<60 prior sessions),
missing breadth_rvol (pre-setup), or ANY exception returns None and never
blocks. The gate can only SUBTRACT trades on bad-regime days.
- research/regime.py: live_session_favorable() (refit on priors <today, predict
today; writes regime_scores['meta_live'] for forward validation) + LIVE_FEATURE_COLS
+ the breadth_rvol-missing fail-open guard.
- trading_execution.py: TRADING_REGIME_GATE_ENABLED / _P_MIN settings + _regime_reject()
(cached per session) + a REGIME gate after SESSION in _evaluate_entry_gates.
- docker-compose.yml + truenas-compose.yml: TRADING_REGIME_GATE_ENABLED=1.
- tests/unit/test_regime_gate_live.py: block / pass / fail-open(x3) / scored-once-per-day.
KNOWN train/serve skew (documented, validate forward via meta_live vs EOD label):
model trains on full-day breadth_rvol, live uses setups-so-far. Small edge on a
~266-session sample; paper only. Suite: 504 passed, 15 pre-existing loopback fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both regime experiments PASS pre-registered bars over the 2041 v1-population
fires — first PASS of the milestone. meta_v1 gated expR +0.0117R/fire (from
-0.085R baseline). Adjudicated live via the admin /api/regime/run pipeline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The session-level metalabel P(our ORB profits today) — the one lever that
survives out-of-sample. Backtested on 266 prod sessions (walk-forward):
trade-always (trained subset): -0.111R/session, 36% hit
meta_v1 GATED (trades best ~32% of days): -0.004R/session, 48% hit
delta +0.107R in-sample, +0.128R OOS (recent 80 sessions), improves across folds
It does NOT make the bot profitable — it takes a bleeding strategy to roughly
flat by sitting out the worst ~2/3 of sessions. threshold_v1 rule gates too
little to matter; meta_v1 (walk-forward GBM) is the signal. Leaky full-fit
(+0.61R) >> walk-forward confirms the discipline was load-bearing; the OOS
holding is the evidence the +0.078R is real, not look-ahead.
Changes:
- research/regime.py: batch minute-bar load per session (64k per-fire round-trips
-> ~266 queries; fixed a >5min build to ~3min) + incremental label pass (skip
already-labeled sessions; --rebuild forces full). Nightly cost now ~one new day.
- docker-compose.yml + truenas-compose.yml: scheduler runs build-features + score
(both models) before `experiments run`, nightly. ALL-SHADOW — does NOT gate live
trading (that stays owner-gated Step 5).
- admin_api.py + admin.html: POST /api/regime/run (detached) + GET /api/regime/status
+ a Run/Status button — the panel had no way to trigger this.
- plans/011 + docs/research/regime_notes.md (6-book distillation) + plans 008/009 docs.
Prod state (shadow, rollback = DROP regime_features/regime_scores + DELETE
experiments WHERE id LIKE 'regime%'): 266 feature rows, 532 scores, 2 experiments
registered.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
research/regime.py: Step 0 probe (labelable-session/SPY-coverage STOP gate),
Step 1 build-features (regime_features: pre-open breadth/tape + prior-close
SPY momentum/vol + trailing bot expR, label = mean _sim_baseline_r over the
session's setup fires) + walk-forward threshold_v1 rule, Step 2 walk-forward
meta_v1 (shallow sklearn GBM), Step 4 OOS/leakage-audit helpers
(oos_split/sequential_folds/session_expr_delta/full_sample_fit_threshold).
Step 3 (HMM) deferred per scope — statsmodels stays absent.
replay.py: new iv_regime_gate intervention (keep-or-zero a fire on a
pre-computed session regime score, fail-open on no score row) wired into
_compute_fire + run(), mirroring rank_one_bullet's shape.
experiments/regime_threshold_v1.json + regime_meta_v1.json register the two
model variants against the shared population/bars/fidelity contract.
All new code is unit-tested against isolated EventStore(":memory:") schemas
only — no production DB read or written. Live-trading paths untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step 1 measured: pm1+fat-tail expR -0.50R@0.5% fill -> -1.27R@1% (realistic) ->
-2.74R@2%, vs ORB -0.11R. The 22-min fat tail can't outrun the 1-2% premarket
spread. Every entry lever this session (gap bands + premarket) is negative after
honest fills; the gated ORB is least-bad. Keep TRIGGER_GAP_MAX=35.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one unexplored setup (premarket/pm1). Step-1 go/no-go: does pm1 + the
validated fat-tail exit beat the ORB AFTER honest thin-book fills (real
spread_pct, fat premarket adverse)? If not, STOP — premarket is a paper-fill
mirage. Shadow-first; TRADING_SESSION_ONLY stays ON; no live order until a
forward shadow edge survives realistic slippage (owner-gated).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root anchor lacked the TRADING_EXIT_* block the truenas template already has, so
a fresh root deploy ran no-trail defaults (scratches v1 winners at breakeven).
prior_low trail after +1R (counterfactual: v1 win 17%->24%) + step0.25 + profit
tiers + BE@1R. Only the loop acts on these.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
iv_exit_config replays ANY target/breakeven/trail combo through the shared exit
engine vs the live-exit baseline — reusable way to adjudicate exit numbers.
Counterfactual over setup fires: v1 (slow, 1.37R median winners) gains most from
prior_low trailing (win 17->24%, +0.036R); pm1 (fast, 2.64R median, 10R tail)
gains most from NO target + step0.5 (+0.273R, win 24->34%) — the fixed target
throws away pm1's fat tail. Registered both as tracked experiments. +test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 0.514 was the liq1-dominated mix. Scoped to live v1: CV AUC 0.641, driven
~entirely by above_vwap_flag (above-VWAP +1R 35.4% vs below 12.7%, 2.8x). The
bot already enforces above-VWAP; residual within that population = 0.526 ≈ noise.
No new alpha in current features; revisit after 009 fills rvol/context/catalyst.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
research/conversion/{__init__,dataset,signal_probe}.py — the leakage-safe probe
plan 009 Step 5 re-runs scoped to a live setup_version:
- dataset.load_dataset: setup_events JOIN outcome_labels; label =
reached_1r_before_minus_1r; features = trigger-time-knowable setup_events
columns ONLY, dropping >70%-null and <2-distinct columns. HARD leakage rule —
every outcome_labels column is blocklisted AND asserted absent from X, so a
schema change can't leak the answer in.
- signal_probe.run_probe: expanding-window CV over ~5 sequential session-date
blocks (whole days, train strictly precedes test), HistGradientBoostingClassifier
(max_iter=250, lr=0.05, max_depth=4, l2=1.0, categorical_features=indices),
per-fold + mean ROC-AUC, top-decile +1R lift vs base, permutation importance on
the last fold. Drops columns constant within a train slice (HistGBM binning).
CLI: python3 -m research.conversion.signal_probe [--setup-version v1].
Tests: the critical leakage guard (zero outcome columns in features), AUC>0.5 on a
synthetic signal + ~0.5 on pure noise, and a static live-path guard. All :memory:.
No live-path change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 3 of plan 009 (config + code; no live-path change):
- research/ingestion/market_context.py: pulls SPY/QQQ full-session minute bars
(SIP, post-close embargo-safe) + daily bars via the existing upsert helpers, so
the labeler's market-context features have a FORWARD source (index ETFs never
enter the gapper universe the live loop ingests). Mirrors sip_backfill's shape.
- both compose scheduler blocks: run it after premarket_eval, guarded || echo.
- NEWS_ENRICH_ENABLED on: added to docker-compose x-app-env anchor (loop +
scheduler inherit it); truenas-compose + .env.example already had it. Catalyst
is a FEATURE source only, never a live gate; enabled-but-down never blocks
trading. Enrichment OUTPUT is verified by the orchestrator (needs live Ollama).
Compose statically validated (yaml.safe_load). Ingest self-checked via a
fake-client :memory: round-trip through _preload_spy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Steps 1-2 of plan 009 — fill two fillable feature gaps so a conversion model
gets a fair, instrumented test:
- market context: _spy_context computes spy_intraday_return (SPY RTH open ->
trigger-minute return) + a coarse tape_regime {trend_up,trend_down,chop} from
SPY's sign + higher-highs structure. Replaces the hardcoded
tape_regime=None/spy_intraday_return=None; wired into ALL five build paths
(v1/vr1/run1/liq1/pm1) via _preload_spy. Null when SPY has no RTH bars at/before
the trigger (pre-30d history, or a premarket fire before SPY's RTH open).
- thin-name RVOL fallback (v1): when avg_vol is 0/null (thin IPO-class, no 20d
baseline) fall back to the median of prior sessions' total volume (real
daily_bars.volume, knowable pre-open); flagged in new setup_events column
relative_volume_source ('avg_vol_20d' | 'prior_days_median_volume' | NULL).
Never fabricates a baseline.
A feature to TEST, not a signal to trust. No live gate/sizing/exit change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
eod_replay was the wrong tool to grow the harness sample — it only PRINTS a
backtest report (persists no fires) and get_most_actives is today-biased for
historical dates. The right source was already in the DB: 18,164 setup_events,
each carrying the ORB entry (entry_reference_price) + stop (invalidation_price).
reconstruct_fires_from_setups turns every setup into a simulated fire; the exit
experiments (volume_fade, time_stop, asym_exit) replay over its minute-bar path
with baseline = the LIVE exit config simulated on the same path (_sim_baseline_r,
same adverse-fill accounting) — a clean same-path delta needing no live P&L.
Gated by EXPERIMENTS_INCLUDE_SETUP_SIM (default on), capped by
EXPERIMENTS_SETUP_SIM_LIMIT for chunked backfill; rank_one_bullet excluded (it
needs live receipts). Grows every exit experiment's n from ~15 to thousands.
+test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The live loop fills resting-limit brackets via position-snapshot diffing and
emits NO order_filled (verified: 0 all-time in prod), so reconstruct_fires
skipped every fire and the ENTIRE harness replayed nothing (0 run rows).
Reconstruction now falls back to the order_submitted timestamp for entry timing
and reads entry/stop/exit from position_closed — restores the whole flagship
(0 -> 324 run rows on prod; pm1's overdue FAIL verdict now posts).
Adds the volume_fade intervention (exit to lock the gain when a bar's volume
drops below fade_frac x the entry-burst average, once in profit; original stop
still caps downside; adverse-tick fills on entry AND exit) + volume_fade_exit_v1
(pre-registered bars/futility/fidelity). Interim n=15 read: -0.68R vs baseline
(INCONCLUSIVE, min_n=100) — the harness answers 'does volume-fade help?' from
data before any live change. +6 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scheduler ran 'experiments run/adjudicate' but nothing registered the five
experiments/*.json into the experiments table, so on a fresh DB the harness
replayed over an empty set and the ledger stayed empty forever (owner hit this
on first deploy). registry.register_cohort() registers every cohort JSON
idempotently (skips already-registered, tolerates a malformed file); 'run'
calls it at startup so a fresh deployment self-bootstraps with no manual
register step. +test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compute_ratios bails when UTC now-16min precedes today's 08:00Z, i.e. nightly
~00:00-04:16 ET; the test's hardcoded 2026-07-06 bars made it KeyError at the
date rollover independent of any code change. Freeze the module clock to a
mid-session UTC time so the window is valid deterministically.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 6: tests/smoke_dashboard_headless.py renders the real page in a headless
browser and fails on any JS console/pageerror (standalone script, not
pytest-collected; CI non-blocking first). admin_api gains GET /api/config/effective
(per-key masked value + source: overrides.env vs compose, missing .env.example
keys flagged as drift) + an admin.html table (values XSS-escaped). dashboard.html
gains the Experiment Ledger panel reading the plan-006 verdicts from the snapshot
(empty-state until the tables exist). Removes the last CI ruff per-file pin
(api/main.py:F821) now that os/hmac are imported; scoped ruff is green unpinned.
Marks plan 007 DONE in plans/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DashboardState.bars() short-circuits to [] unless a research_con is attached; the
embedded orchestrator passed one but the standalone dashboard_api.py never built
it, so production sparklines were permanently dead. The missing wire was purely
the entrypoint (DATABASE_URL already reaches the market schema) — now it opens
open_research_db("market") read-only and passes it, degrading to None (charts
show "no research db") if it can't connect. Verified live: bars() returns real
minute rows through the standalone reader.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Badge now derives PAPER/LIVE from broker truth (base-URL host) instead of the
schwab_live literal that read PAPER while go-live swapped only ALPACA_BASE_URL;
live flips body.live-mode (top shimmer -> solid red) with a tooltip of host +
account. Adds the header FLATTEN ALL (two-step arm/confirm, 5s auto-disarm) and
PAUSE ENTRIES toggle (+ amber banner), per-position EXIT buttons (two-step), and
an operator-token field sent as X-Dashboard-Token on every POST (also fixes
approve/reject when a token is set). Action controls gate on
has_execution || command_bus; kill-switch controls on command_bus. No keyboard
shortcut for any destructive action. New styling is :root-token-only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds OperatorCommandEvent + a closed command vocabulary (flatten_all,
flatten_symbol, pause_entries, resume_entries). POST /api/command emits it
(token-required: 503 when DASHBOARD_TOKEN unset — commands never run
unauthenticated); the live loop's new 5s "command" step consumes them via
OperatorCommandConsumer with three replay-safety layers (in-session nonce set,
persisted operator_command_done reload, 10-min staleness window).
flatten_all routes through close_session DIRECTLY (never step_eod_flatten) so it
cannot trip plan 003's daily-restart exit. Pause is checked at the top of BOTH
request_approvals_for_ready_signals AND submit_breakout_now (missing either
leaks pause on that path). qabot gains an authorized-chat-only /flatten slash
command (prose "flatten" still refuses via _ACTION_WORDS). Snapshot exposes
broker truth + command_bus + a defensive experiment ledger.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every SSE client polled COUNT(*) on a fresh EventStore connection each ~1s
(re-applying the 473-line schema unless PG_SKIP_SCHEMA=1). Replaced with one
process-wide _ChangeSignal: a single long-lived connection reads max(timestamp)
(idx_events_timestamp tail seek) at most once per second GLOBALLY behind a lock,
so N clients cost one query/sec. EventStore.max_timestamp() backs it; reader is
injectable for the query-budget test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every POST endpoint (approve/reject/exit/watch) crashed with NameError:
do_POST references os.environ + hmac.compare_digest but neither was imported.
Adds the imports, a non-loopback regression test for the token gate (no token
-> routes; token set -> 401 without/with-wrong header, 200 with correct), and
plumbs DASHBOARD_TOKEN + PG_SKIP_SCHEMA=1 into both compose dashboard services
(+ .env.example docs for DASHBOARD_TOKEN / COMMAND_BUS_ENABLED).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 5. python -m research.experiments run && adjudicate after
nightly_tune in both scheduler blocks (guarded with FAILED echoes). ALL-SHADOW:
emits verdicts, never touches live gates/sizing/exits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 4. experiments/*.json for time_stop_10m_v1, probe_add_v1,
asym_exit_v1, rank_one_bullet_v1, pm1_v1 — bars verbatim from the plan table,
production population, fidelity floors {0.25, 3, 1} from the research doc. Also
hardened evaluate(): n==0 and any uncomputable bar metric -> INCONCLUSIVE (not a
false FAIL). Cohort registers + survives run->adjudicate end-to-end. 25 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 3. evaluate() scores runs against pre-registered bars
(_min/_max/min_n vocabulary) -> PASS/FAIL/KILLED/INCONCLUSIVE; terminal
verdicts flip status, emit an experiment_verdict event, and Telegram-ping ONCE
(terminal experiments are never re-adjudicated). Metric vocabulary covers
expR_delta/rate_1r plus pm1's median_up_60m/median_dd_60m/big_gap_rate_1r.
backfill_pm1 bridges pm1 from outcome_labels JOIN setup_events (variant +/-1,
baseline NULL). premarket_eval dual-writes the verdict through the harness while
keeping the legacy TSV. 8 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 2. Reconstructs fires from the event stream (submit[buy] -> fill
-> close), population-filters via strategy/population.py, and applies three pure
interventions counterfactually on the bar path: time_stop, probe_add, asym_exit
(the last through strategy.exits.simulate_exit ONLY). Hard fidelity: +1-tick
adverse fills on entry AND stop, spread_proxy = median quiet-bar high-low, stop
floor = max(atr_frac*ATR14, spread_mult*spread_proxy). rank_one_bullet is
receipts-only and starves (documented) without plan-005 rank. Idempotent via PK
+ ON CONFLICT. Anti-slippage-artifact regression (test #2) green: a
zero-slippage-profitable path LOSES under adverse fills.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rank concentration now runs on BOTH paths via the shared gate, so the warning
that it was 'enforced ONLY on the dead fast-trigger path' is false. Removed it
and folded rank top-N state into the [boot] entry-gates summary line.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New EntryGateEvaluatedEvent (symbol, source, passed, reject_reason, gates map)
emitted once per _evaluate_entry_gates decision on both paths, behind
TRADING_GATE_RECEIPTS (default on; dataclass default off so unit tests opt in).
Added to the 14-day nightly purge tuple (high-volume diagnostic) and .env.example.
Turns 'why didn't it trade X?' into a query and is the substrate plan 006 replays.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One ordered gate sequence (session, confirmation-speed, population, VWAP,
quality, concentrate-by-rank, daily cap [fast-only], dedup/cooldown/reentry,
dilution veto, anti-chase) that BOTH request_approvals_for_ready_signals and
submit_breakout_now call. Gate bodies lifted verbatim from whichever path
owned them; slow-path risk events preserved source-guarded so the fast path
gains no new events. The two documented asymmetries now close:
- fast path routes through the VWAP gate (was ungated),
- slow path honours concentrate-by-rank via a rank_map threaded from the
trigger book into tick()/request_approvals (no-op at concentrate_top_n=0,
which is the production config — STOP #3 does not apply).
Price-sanity/entry_ref coercion stays path-native (different data sources);
slow-path dedup pre-filter kept first so per-tick re-scans don't re-emit.
16 characterization tests pin both paths; the only decision diffs vs
pre-extraction are the two closures above.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pins today's slow-path (approval/event) and fast-path (skipped string)
decisions across single-condition signals, incl. the two asymmetries that
step 2 will close (fast has no VWAP gate; slow has no rank gate). Green
against unmodified trading_execution.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
from_env merges os.environ, and importing the .env-loading consumers populates
TRADING_MIN_ENTRY_GAP_PCT — so the "bare shell disables the gap gate" assertion
must delenv the key first. Test-only fix; the module behaviour is unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tuner was fiction in production: the 20:00 ET scheduler ran only
premarket_eval (+ eod_reconcile), so learned_params.json sat 26 days stale and
the whole-market dataset never grew. Both scheduler loops now run, in order,
after the eval/reconcile pass:
python eod_replay.py || echo "[scheduler] eod_replay FAILED"
python nightly_tune.py || echo "[scheduler] nightly_tune FAILED"
Order matters: replay grows the dataset first, the tuner fits after. 20:00 ET is
after close, so replaying "today" is complete data — NOT look-ahead (replay is a
labeling job, not a live gate). Deletes eod_replay.sh (dead cron-era wrapper
pointing at a nonexistent venv; compose is the runner).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- nightly_tune: write learned_params.json via tmp -> flush -> fsync ->
os.replace so a crash mid-dump can't leave the live loop parsing half a JSON
on its next boot (it reads this file on every restart).
- run_live_paper._load_learned: warns (boot log + one gated Telegram note) when
the file is older than LEARNED_PARAMS_MAX_AGE_DAYS (default 3) — the missing
signal that let the params sit 26 days stale. Loads anyway; never blocks boot.
Staleness math factored into _learned_params_age_days (naive ts = UTC).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both offline consumers now screen through strategy.population.population_reject
instead of divergent, look-ahead admits:
- learn_params.load_sessions: replaces the `gap>=0.03 OR day_rvol>=2.0` admit,
whose day_rvol was a FULL-DAY volume sum (outcome-time look-ahead, the
2026-07-08 class). gap is already a FRACTION (open vs prev close) — passed
as-is; the OPEN price is passed for the price band. Stored daily bars expose
no open-knowable volume, so volume=None (fail-open on the day-volume floor) —
no day-end volume is smuggled into admission.
- eod_replay._daily_gappers: screens on the OPEN price + STATIC open gap (was
t_close look-ahead) via the shared gate; the local gap is converted to a
FRACTION before the call, so it inherits the >=150% squeeze exception. Deletes
the stale $1-5 / 3-60% WATCHER_/TRIGGER_ constants (KEEPS RISK_PCT). The
second-stage scan_gappers is made permissive (membership already decided by
the shared gate), removing the `gap_pct <= 60%` cap that wrongly dropped
squeezes. Prod thresholds passed explicitly (bare from_env would disable gap).
Unit test replaces the live-DB smoke run: $15/+12% qualifies, $4/+45% rejected
unless >=150%+rvol5. research/labeler.py deferred to plan 006 (no locatable
price/gap admission screen); sweep_exits/sweep_filters likewise deferred.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tradeable population is now defined ONCE. trading_execution._population_reject
becomes a thin wrapper building a PopulationConfig from its settings and calling
strategy.population.population_reject — logic moved verbatim (dead-band 0.35/1.50
and squeeze rvol 5.0 are module CONSTANTS, gap floor / price band / volume floor
from config). Behaviour byte-identical: the pre-extraction characterization suite
stays green through the new public seam.
from_env mirrors ExecutionSettings.from_env (same 4 keys + the [0,1] _qfloat clamp)
so a live-shaped shell reconstructs the identical gate. gap_pct FRACTION contract
and the as_of open-knowable contract are documented for offline consumers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
11 table-driven cases pin the CURRENT behaviour of the live population gate
before plan 004 extracts it to strategy/population.py: in-band pass, below-gap,
dead band, squeeze pass/fail, both price-band sides, volume floor, missing-data
fail-open, the gap-is-a-FRACTION unit pin, and the gap-gate-disabled nesting.
These must stay green byte-for-byte across the extraction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 003 step 5 (completes plan 003). research/eod_reconcile.py diffs broker
truth (open positions, today's live orders) against event-store truth (today's
submits/fills/closes) and emits a loud eod_reconcile event + Telegram alert on
any drift — the duplicate-resting-order-after-restart class (2026-07-06) caught
automatically instead of by hand. Read-only (reports, never auto-fixes). Wired
into both scheduler blocks after premarket_eval. Pure diff core is unit-tested.
Suite: 373 passed (+22 plan-003 tests), 15 fail = WSL-loopback dashboard only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 003 step 4. A transport error AFTER the POST landed leaves a live order
resting at the broker; journaling it 'cancelled' breeds the duplicate/naked
order (2026-07-06 incident). Add client.get_order_by_client_order_id + an
after= passthrough on get_orders; on the generic-exception path, look up the
order by the client_order_id we already send and, if live, emit the normal
submitted sequence (ok=True) instead of cancelled. AlpacaApiError (real
rejection) is NOT reconciled. Fills the empty test_alpaca_client.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 003 steps 1-3.
- Loop exit(0) after a SUCCESSFUL EOD flatten (TRADING_EXIT_AFTER_EOD_FLATTEN,
default on); compose restarts fresh. Restart-churn guard: _rebuild_daily_state
detects today's eod_flatten event and seeds _session_closed/_eod_done so a
post-flatten restart neither re-flattens nor re-exits. SIGTERM = clean drain.
- Exit manager: _FETCH_FAILED sentinel distinguishes a failed get_orders from a
genuinely-empty snapshot. A failed fetch skips naked-stop enforcement (catastrophe
still exits, re-fetching); an empty-but-successful snapshot needs TWO consecutive
reads before 'naked' is trusted (Alpaca silently returns [] for symbols=). Costs
~one 12s pass on the all-naked case only; isolated naked positions unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pg_schema.sql: drop CREATE TABLE for 10 zero-reference tables (operator DROP
list in plans/002-deletions.log — production tables untouched until run).
projections.py: remove 4 zero-consumer query functions.
Clear all 'plan 002' ruff pins from the CI gate (files now F-clean);
api/main.py:F821 pin remains for plan 007. README repo-map/test-count fixed;
pydantic-settings removed (zero imports). Full receipts: plans/002-deletions.log
Suite: 351 passed (16 schwab/dead-module tests removed), 15 failures = the
known WSL-loopback dashboard set only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
grafana/ pointed at the decommissioned pre-migration DB with committed dev
credentials (rotation note in plans/002-deletions.log); its dashboards
provisioning MOVED to deploy/compose/grafana/dashboards (docker-compose.yml
bind-mount re-pointed — the in-compose grafana service keeps working).
docker-publish.yml fired on a nonexistent branch pushing a wrong image name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Finishes what d54a2a9's message claimed: schwab/ was never actually removed.
Also: comprehensive_backtest.py (crashed on import — research/connection.py
does not exist), strategy/universe.py, strategy/backtest/outcomes.py,
detect_momentum_ignition (kept _last_higher_low — imported by runner/
premarket_setup/liquid_breakout), milestone3_verify.py, tests/unit/schwab/,
SCHWAB_* keys from .env.example. All grep-gated: zero live references.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Production Postgres had NO automated backup (pg_dump appeared only in the
one-time migration script). New db-backup service in both compose files fires
20:10 ET Mon-Fri: pg_dump | gzip with pipefail + gunzip -t + tmp-then-rename
(a mid-stream death can't masquerade as a backup), prunes past
BACKUP_KEEP_DAYS, writes last_backup_ok on success. postgres:18 image =
version-matched client (app image has none). TrueNAS host is pull-only, so
its copy inlines the script. Restore: gunzip -c dump | psql $DATABASE_URL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The loop's Telegram notifier runs inside the loop process — loop-down means
alerts-down. The always-up qabot now reads max(events.timestamp) every ~120s
(ET-naive vs ET wall clock per RUNBOOK): stale > DEADMAN_STALE_MIN during
market hours (holiday-aware via runtime.market_calendar) alerts at most every
30 min; DEADMAN_DAILY_PING sends one OK-ping after 20:30 ET. Injected
clock/read/send; 6 unit tests, no network or DB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The registry image deploys to production on pull with zero test run today.
Add a test job (postgres:18 service, python 3.13 to match the Dockerfile)
that build now needs. ruff F gate is scoped to the money path with
pre-existing errors pinned per file+rule (owners: plans 002/007) so new
F errors still fail the push.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Consolidated 4-tool review (2 correctness agents + Codex + over-eng audit).
Money-path (regression-tested):
- _flatten_all: moved the fresh-broker close from close_session INTO the
shared function + added a broker-held guard on entry cancellation, so
the daily-loss CIRCUIT BREAKER (which called _flatten_all raw) no
longer strips a just-filled bracket's legs or leaves a sync-gap fill
naked. One guard, all callers.
- restart state rebuild: _fresh_entries cap, _requested_symbols dedup,
and losing benches now rebuild from today's events on boot — a
mid-session restart no longer resets the cap to 0 or re-fires a
resting order into a duplicate (the 2026-07-06 incident).
- POST /orders no longer retries a lost-response URLError (was
re-sending duplicate live orders); GET/DELETE still retry.
- expire_stale_entries now @_locked (raced the trigger thread).
- _session_open: ET conversion instead of a hardcoded 13:30-13:33Z
window that only matched EDT — the fast path armed nothing all winter.
Security:
- admin panel fail-CLOSED on empty ADMIN_TOKEN (was fail-open on a panel
fronting the docker socket); header-only token, constant-time compare.
- admin API responses redact secret override VALUES (live keys would
have echoed after a paper->live switch).
- dashboard: removed wildcard CORS (cross-origin drive-by could drive
trade actions); optional DASHBOARD_TOKEN gate on POST.
Deletions (~805MB + dead code shipped in the prod image): momentum-
rebuilt.zip, archive venv/htmlcov, and schwab/ core/ telemetry/ models/
runtime/providers/ storage/replay.py controlroom/ + three dead
evaluation stubs + package.json — all verified zero-importer.
+4 regression tests. 99 money-path/trigger/exit tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Systematic diff of the old .env against the deployed loop env found 68
code-read keys running on silent defaults — including the risk cap
($500 vs $868 observed), the VWAP gate, the exit ladder, and the
NEWS_ENRICH_ENABLED flag that kept the catalyst engine dead despite
OLLAMA_ENABLED=true (enrichment requires BOTH, config.py:1235 gate).
Template env block regenerated with the full inventory, secrets as
<FILL-ME>; a ready-to-paste block with real values generated locally
(data/loop-env-block.yaml, gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 10 era documents (phase plans, build summaries, redesign writeups, old
runbooks, completed consolidation plan) moved to docs/history/ with a
do-not-follow README.
- Root README: current validated population ($2-20, 10-35% static gap +
squeeze exception, SIP), Deployment section, automated-maintenance
reality, corrected repo map (admin panel, youtube_claims, deploy/).
- RUNBOOK rewritten for the running system: daily rhythm, health checks
per deployment, and the production failure-mode table earned this week
(env drift, pgdata mount traps, uid 10001, zombie approvals, PG_SKIP_
SCHEMA, timezone artifacts).
- deploy/README.md is now the three-mode index: compose (canonical),
systemd (local dev — unit templates now committed), helm (refresh
staged as issue #8). Old k8s doc moved to deploy/helm/README.md.
- docker/README.md: six TrueNAS gotchas, each from a real outage.
- Admin panel improvements staged as GitHub issues #4-#7.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 evening: the loop submitted three real resting entries at
18:53-20:09 ET (TDTH x2, SUNE). Root cause: NO market-phase gate ever
existed on either entry path — SIP prints extended-hours trades, so
after-hours price crosses fire the trigger book and evening-fresh
signals sail through the approve path. The hole predates the container
move; Thursday was merely the first evening conditions aligned.
_outside_session() (ET wall clock via zoneinfo, weekend-aware) now bars
fresh entries outside TRADING_SESSION_START..END (default 09:30-15:55)
on both submit_breakout_now (skipped: outside_session) and the approve
path (quiet skip). Exits and management remain ungated. Dataclass
default OFF so directly-constructed test settings are unaffected;
from_env defaults ON so every deployed config gets the gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The loop healthcheck opened a fresh connection every 60s with schema
auto-apply enabled; the CREATE-IF-NOT-EXISTS lock bursts cancelled
minute_bars autovacuum every single cycle (visible as 'canceling
autovacuum task' each minute post-migration), so the 19M-row table
never got analyzed, snapshot queries crawled, browsers timed out, and
the dashboard log filled with double BrokenPipeError tracebacks.
- healthchecks now set PG_SKIP_SCHEMA=1 (both compose files)
- api/main.py _send_json swallows BrokenPipe/ConnectionReset (client
hung up; nothing to tell it)
- live DB: ALTER SYSTEM max_wal_size=4GB, shared_buffers=1GB,
effective_cache_size=3GB, checkpoint_completion_target=0.9 (the
checkpoint-every-24s warning under migration load on NAS storage)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New admin service — LAN-only, token-authed (ADMIN_TOKEN), card-based UI
matching the owner's preferred dashboard style. Capabilities:
- health: docker containers via /var/run/docker.sock (stdlib unix-socket
HTTP client), DB heartbeat, Alpaca account/clock, Ollama, feed
- config: whitelisted strategy knobs (TRADING_/TRIGGER_/WATCHER_/...)
edited via data/overrides.env, which every entrypoint now loads with
override=True at boot (runtime/env_overrides.py) — so changes apply on
container restart without touching the deployment YAML; secrets are
presence-only and rejected server-side if edited
- profiles: named knob presets (seeded: validated-core, conservative,
squeeze-hunter) applied in one click
- restart buttons per container; paper/live switch guarded behind
ALPACA_LIVE_* keys existing
Verified live in docker: auth 401s, status/config/profiles/write paths,
secret-edit rejection, page serves. Added to both compose files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two new nightly enrichments for the day's tracked names, both SHADOW
ONLY (float was refuted as a gate; a filings-based dilution veto needs
its own pre-registered bar):
- research/ingestion/float_cache.py: FMP stable shares-float ->
float_cache table (free-plan budget guards: 7-day refresh, 200
calls/run). First real float data in the system — LHSW's +346% ran
on a 1.1M float; BJDX is 0.4M.
- research/ingestion/sec_dilution.py: SEC EDGAR full-text search (free,
keyless, fair-use paced) -> populates the previously-empty
dilution_events table with S-1/S-3/424B filings; TVRD + BJDX flag
immediately. recent_dilution() helper for future veto work.
Wired into premarket_eval nightly. Not wired: Polygon/AlphaVantage/
TwelveData/Marketstack/EODHD (redundant vs paid Alpaca SIP) and the
generic news aggregators (slower than Alpaca/Benzinga, dedup noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
apscheduler + youtube-transcript-api (transcripts service crash-looped
on TrueNAS: ModuleNotFoundError), rich + typer (momentum_cli journal
job). Found via AST sweep of all third-party imports vs the image
contents. Image rebuilt, import-verified, pushed to both registries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI now pushes fez5587/momentum-app to Docker Hub alongside GHCR (creds
via repo secrets). First push done manually and verified anonymously
pullable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
truenas-compose.yml now uses /mnt/volume/Apps/momentum and publishes the
new postgres on 5433 so it can coexist with the existing instance during
migration. deploy/docker/migrate_db.sh: dump / restore / verify steps
(pg_dump -Fc read-only source, parallel pg_restore, ANALYZE, row-count
diff) with env-overridable URLs and PG18 client autodetection —
connectivity smoke-tested against the live DB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TrueNAS custom apps cannot build: the main compose's 'build: .' (plus
relative .env/bind mounts) fails app 'up'. Added a GitHub Actions
workflow publishing ghcr.io/fez5587/momentum-app on every master push,
and deploy/docker/truenas-compose.yml — a pull-only variant with
dataset host-paths, inline environment (secrets as <FILL-ME>
placeholders, current validated strategy knobs pre-filled), and the
same healthchecks. Validated with docker compose config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
.env.example was stale (pre-migration keys, missing all current
TRADING_*/TRIGGER_* knobs). Regenerated from the live config with
secrets <REDACTED>; a leak check caught live Alpaca credentials
embedded in commented-out PowerShell lines and scrubbed them before
commit. README now documents the three config layers (protected
secrets vs promotable strategy knobs vs infra addresses) and the
external-Ollama networking expectations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the pre-migration compose file up to the current architecture:
python 3.13 image with ffmpeg + faster-whisper/yt-dlp (transcripts),
postgres 16 -> 18 (prod parity), and the real service set — loop
(--no-dashboard, heartbeat healthcheck), dashboard_api.py on :8765,
telegram qabot, youtube transcripts, and a 20:00-ET weekday scheduler
replacing the premarket-eval systemd timer. Jobs (journal/eod-replay/
nightly-tune) and Grafana keep their profiles. Self-contained by
default (in-stack DATABASE_URL); deploy/docker/README.md documents the
pg_dump migration, the transition mode against the LAN DB, and the
cutover checklist. Validated: image builds, compose config passes, and
a smoke container served the dashboard (HTTP 200) against the live
LAN Postgres.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tested the source trader's numeric selection playbook on the unbiased
(open-knowable) year universe, n=2,643 point-in-time ORB entries:
- Dead band RE-CONFIRMED on clean data (35-75% +0.006, 75-150% -0.066)
— CLRO/VTAK were anecdotes; band exclusion + squeeze exception stand.
- His $5-10 price sweet spot inverts for our system: edge RISES with
price ($10-20 best at +0.143); the $1-2 bucket is negative (-0.031,
n=609). SHIPPED: price floor $1->$2 (population gate band now
env-driven TRADING_PRICE_MIN/MAX; watcher floor raised to match).
- His rvol>=5 floor is backwards for us: rvol 2-5x at entry +0.346R vs
>=10x -0.047 (n=818) — crowded spotlight names hurt our entries.
NOT shipped (conflicts with the squeeze exception's rvol needs;
logged for forward validation via the new per-signal relative_volume).
- SHIPPED: buyout/merger headlines never rank-boost (price is pegged —
the CRNX class); keyword suppression in the catalyst multiplier.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Evening audit of 2026-07-08's post-fix entries (TVRD +4% true gap,
FRTT — both fired as "20% gappers"): the morning's static-gap fix was
defeated by a data hole. ingest_live_minute_bars' full-session first
pass was a GLOBAL one-shot, so symbols discovered mid-session only ever
received the 20-minute refresh window — their earliest known bar was a
mid-day bar, which became both their "session open" (fake overnight
gap) and the base of their "opening range" (fake ORB trigger). Exactly
the failure class the population gate exists to prevent.
Fixes: (1) step_ingest tracks per-symbol first sight — new names get
the full-session lookback, known names the incremental refresh;
(2) _session_open only trusts a first-RTH bar stamped <= 09:33 ET and
is the sole source for the static gap; (3) if the session is underway
and a candidate's true open is not yet ingested, it cannot arm (trigger
withheld until the backfill lands, typically the next pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 2026-07-08 audit (owner: "figure out if there is some kind of logic
or audit issue") found BOTH entry paths feeding the population gate the
WRONG gap. The fast path used the trigger book's dynamic gap (latest
price vs prev close), so intraday rallies masqueraded as gappers: TVRD
(6.0% true gap), SKIN (2.2%, filled at $0.98), JLHL (4.3%) all traded
outside the validated population. The slow path used max(gap, intraday)
with "open" taken from the first lookback-window bar, and had no upper
band at all — dead-band names SEER (38.6%) and IOTR (40.7%) entered.
Fixes: candidates carry gap_open_pct (session open vs prev close);
trigger eligibility + submit_breakout_now use it exclusively once the
session is open; _population_reject now implements the full validated
band (10-35% OR volume-verified >=150% squeeze, $1-20 price band) for
both paths; evaluator gap uses the session open; signals carry
relative_volume for squeeze verification. Plus momentum-testpg.service
so the test-isolation Postgres survives reboots.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LHSW gapped +278% on 100M shares, was rank 2 in the trigger book at
4:49am, ran +346%, and was the source trader's entire day — the bot's
TRIGGER_GAP_MAX=35 capped it out, same as NXTS two weeks ago, while the
bot spent its entries on the shadow round-trips (SEER/ELAB/PLUG/YRD).
Year-scale replay (n=2,879 ORB entries, 14 months, reverse-split
artifacts excluded by requiring a real volume surge): the 35-150% band
is dead (+0.02R / +0.00R — the cap earns its keep there), but REAL
>150% squeezes (n=82) ran +0.63R with 8/14 months >= 0, 4x the core
band's edge. New gap_squeeze_min/gap_squeeze_rvol re-admit gaps >=150%
only when rvol >= 5 (split repricings show rvol ~0.01 and stay out).
Live: TRIGGER_GAP_SQUEEZE_MIN=150, TRIGGER_GAP_SQUEEZE_RVOL=5.0.
Existing guards (trigger/day extension, halt, BE@+5% ratchet, sizing)
still apply to these names.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Live 2026-07-06: every fresh entry was market-exited seconds after
filling (ELAB round-tripped in 3s, SEER in 77s, PLUG in 85s) — all
logged as exit_trail. Root cause: seconds after a fill there is no
completed minute bar with timestamp >= the fill time, and _bars_since's
empty result fell back to the WHOLE DAY's bars, so manage_live's
high-water included the pre-entry HOD and the trail read as already
breached on any name trading off its high. The whole-day fallback is
removed: with no post-entry bars the manager now leaves the position
alone (the bracket's initial stop is the protection) until the first
post-entry bar lands. Regression test verified to fail on the old code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Everything hot already lives in Postgres — local data/ was 25 MB of
pre-migration relics and unbounded logs. Moved the two dead .duckdb
files (untouched since 2026-06-16) to archive/, added nightly log
rotation (copy-truncate >20 MB, safe with systemd append mode), and
updated the last vestigial "market.duckdb" docstrings to say Postgres,
closing the CONSOLIDATION_PLAN cleanup items.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The events table hit 3.5 GB / 1.8M rows, ~80% of it module_tick /
criteria_evaluated / signal_blocked / account_* snapshots that are only
useful for a few days of debugging. Trading-decision events are kept
forever. One-time purge (353k rows) + missing hot-path indexes
(raw_news_items payload_hash + fetched_at: 911k seq scans -> index-only)
and per-table autovacuum tuning applied directly to the DB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The owner runs this system fully autonomously — no approvals, no manual actions. The
control room is therefore a WATCHING instrument, not a steering one:
- Approvals tab REMOVED (vestigial: the loop auto-approves; interaction contradicts
the operating model). REST plumbing dropped.
- ACTIVITY tab added: merged live event feed (entries, fills, exits w/ realized P&L,
gate cuts, ready signals) refreshing every 4s — see the autonomous system act.
- TODAY summary strip on the Live tab: Day P&L (broker last_equity based, colored),
trades + win rate, submissions, gate cuts, and LOOP HEARTBEAT age (green <3min,
red otherwise) — at-a-glance trust that the unattended system is alive.
- Thread-safety fix: all fetchers run in io_bound THREADS but shared one psycopg2
connection — now serialized behind DB_LOCK (latent race, caught during redesign).
- Panic brake stays (emergency-only, typed confirmation) — the one deliberate
exception to zero-interaction.
Headless-verified: cards live (Day P&L, Loop 0s green), 10 ticker rows, 40-row
activity feed, zero JS errors. Service restarted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phases 1+2 of the staged control-room plan, adapted to this backend and shipped as a
PARALLEL service so the battle-tested :8765 dashboard keeps running until parity.
The owner's ask — live analysis with live tickers, realtime — is the centerpiece:
- LIVE tab: tracked/armed tickers grid with real-time SIP last-trade prices (1s,
flash-green/red on change), state, gap%, RVOL, trigger + distance, stop, quality
score, POPULATION-gate check (display mirror of the live gap/volume floors), VWAP,
catalyst — merged from the loop's trigger snapshots + ready-signal projections.
- Header: PAPER mode badge (chameleon pattern), market phase, live equity, heartbeat,
clock, and the PANIC brake (cancel ALL open orders via broker API, typed "STOP"
confirmation — the plan's conditional-friction flow).
- Positions: live broker positions w/ uP&L colors + naked-stop flag (2s).
- Gates & Attribution: today's gate-cut histogram (5s) + 7-day per-stage attribution.
- Approvals: pending queue w/ approve/reject via the existing dashboard REST + dialogs.
All DB/broker calls via run.io_bound (never block the event loop); dark theme +
semantic tokens per the plan. systemd momentum-controlroom (:8766), unit in
deploy/systemd/. Headless-verified: live header (equity, phase), 10 ticker rows
rendered, zero JS errors.
Remaining from the staged plan (next phases): manual orders, settings dials, audit
tab, connection-lost action guard, live-mode chameleon when a real-money broker lands.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
