TRADING_MIN_QUALITY_SCORE 0.60 -> 0.20.
setup_evaluator.py:164 emits "quality_score": quality.score, where quality comes
from calculate_data_quality_score(bars). That module's own docstring reads "Data
quality scoring for minute-bar coverage and integrity" -- it measures whether the
bars are complete, not whether the setup is good. The name collision meant a
coverage metric has been gating live entries.
Tested against realised outcomes over 62 trades: r = 0.0845. Grade A averaged
+0.13%, grade B -0.71%, grade C -0.17% with an 80% win rate -- grade C beat grade
A. There is no signal in it. On 2026-09-14 it blocked SNDQ ("ready below quality
gate (score 0.56 < 0.60)") purely for gappy bars.
0.20 keeps it as a guard against genuinely broken data while removing it as a
setup filter. Renaming the field so the collision cannot recur is the proper fix
and is left for a follow-up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
The 2026-09-13 exit retune is $7.50 behind the config it replaced across its
first 8 live trades. Its thesis -- surrender small frequent gains to keep a rare
large tail -- predicts exactly this early shape, and the backtest behind it was
234 trades across two stacks with train/test splits. But 8 losing live trades are
also what a wrong call looks like, and nothing in the data separates the two yet.
So pre-commit the exit rather than deciding it mid-drawdown. TRADING_EQUITY_FLOOR
halts the session and flattens the book at $215, about -11% from the 241.75 the
change started at. The check reads broker equity directly and returns early on a
0.0 reading, so it cannot trip on the overnight-maintenance nulls.
Owner decision 2026-09-16: keep the retune, on the view that the same day's
risk-equalisation fix (0.05 -> 0.015 per trade) removes most of what was actually
doing the damage -- the four widest-stop entries produced -$8.57 of the -$8.84
week, and those are the positions the sizing change shrinks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
TRADING_RISK_PER_TRADE_PCT 0.05 -> 0.015.
On a ~$240 account the 5% risk budget ($12) is unreachable: a 4.5% stop would
need $267 of notional against a $38 position cap (max_position_pct=0.16). So
calculate_position_size never bound -- the POSITION CAP did, on every trade.
The consequence is that every position was the same NOTIONAL (~$34 average) and
the dollar risk floated with stop width instead of being held constant:
measured over 64 live trades: risk ranged $0.52 to $8.79 -- a 17x spread
That is the opposite of what risk-based sizing is for, and it concentrated the
damage exactly where the stops were widest. On 2026-09-15/16 the four widest-stop
entries (CANG 13.5%, FTFT 21.6%, ZTG 13.2%, VNCE 19.8%) produced -$8.57 of an
-$8.84 week. It also interacts badly with the -10% catastrophe floor: any stop
wider than 10% is effectively a 10% stop, so a wide-stop trade cannot be saved by
its own stop but still realises a full 10%+ loss on the same notional.
At 1.5% the risk calc binds for wide stops while the position cap still binds for
narrow ones, compressing the spread to 6.8x. Replaying the same 64 trades:
-$4.09 -> -$0.32.
Checked that this is risk EQUALISATION and not merely trading smaller: against a
plain proportional shrink to the same average notional, equalising is worth
+$3.27 at 1.25%, +$2.30 at 1.0%, +$2.45 at 0.75%. The benefit holds at every
matched size.
Below 1% trades start being skipped outright as shares round to zero (1 of 64
lost at 1.0%, 13 of 64 at 0.5%) -- a hidden cost that looks like improved P&L.
1.5% is the tightest setting that drops no trades. Expressed as a percentage
rather than TRADING_MAX_RISK_DOLLARS so it scales with the account.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
Three independent tests over 2026-08-01+, all negative:
premarket news, any window 7.9% vs 8.6% ran >=50%
...controlling for PM range 11.3% vs 21.0% ran >=50%
intraday arrival, hour-matched 16.8% vs 24.6% ran >=10% forward
The feed explains it. Of 1,621 headlines delivered for screened names during
sessions: 61.7% reactive roundups ("Why X Shares Are Trading Higher", "20
Stocks Moving In..."), 2.7% law-firm spam, and 0.9% genuine catalyst PRs. The
wire reports the move after it happens, so a boost on "has a headline" buys the
late stage of a run. 11 of the 14 biggest runners (up to +912%) had no coverage
at all.
Set NEWS_RANK_BOOST to 1.0 (neutral). This also neutralises the binary fallback
restored in 9b4c8e5 -- correct in intent (a dead enrichment must not silently
disable news ranking) but it handed a flat 2x to any unclassified headline, and
62% of them are lagging roundups.
Deliberately unchanged:
- the LLM path. catalyst_score = conviction * max(0, sentiment), so a roundup
(type none, sentiment <= 0) scores 0 and earns nothing, while a real bullish
catalyst still lifts. It targets the 0.9% correctly.
- the dilution veto, which sinks confirmed offerings and is a separate,
sound mechanism.
- the enrichment-down alarm from 9b4c8e5.
Also evaluated and NOT shipped: halt-resumption as a catalyst feature. It is by
far the best signal in this universe -- a >=3min bar gap resuming >=2% up ran
>=25% afterwards 40-47% of the time vs ~11-20% baseline, holding in every hour
of the session. But a time-aware A/B of the armed set moved mean forward run
only 27.09% -> 27.84% and 165 -> 173 runners per 725 slots, because gap x rvol
already encodes it (a halting stock has high relative volume by definition).
Same collapse premarket range showed: strong univariate, near-zero marginal.
The ranking is saturated; selection is not where the remaining edge is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
Over 1,040 screened symbol-days (2026-08-01 on), asking which premarket
property predicts the session running >=50% open-to-high:
premarket range n avg run ran >=50%
>= 30% 201 34.7% 15.9%
15-30% 136 28.3% 14.7%
5-15% 303 15.2% 5.9%
< 5% 400 16.8% 4.3%
A name that already travelled >=15% premarket is 3.1x more likely to run
>=50% (15.4% vs 5.0%) -- the largest single lift in this dataset, and free:
it comes off minute bars the arm pass has already fetched.
Two findings constrain the shape:
- It is a STEP at 15%, not a gradient (15-30% and >=30% are 14.7 vs 15.9;
5-15% and <5% are 5.9 vs 4.3). So the multiplier is a threshold. A linear
scale would invent precision the data does not support.
- Premarket VOLUME carries NO signal (10.0/4.2/7.9/9.0% across participation
buckets, non-monotonic) and raw premarket volume runs inversely to the
outcome. scanner_snapshots.premarket_volume being NULL costs nothing;
range is the field that mattered all along.
Do not expect 3x. Replaying armed-set selection over the same 29 sessions,
top-5 by mean/median run and >=50% runners captured out of 145 slots:
gap x rvol (baseline) 39.77% / 10.97% 27
x3.0 if PM range >= 15% 42.00% / 13.16% 32
HARD FILTER >= 15% 37.27% / 12.67% 24 <- worse
PM range alone 32.86% / 12.65% 20 <- much worse
gap x rvol already encodes much of the same information, so the marginal gain
is ~+2 points of mean run and 4-5 extra big runners per 145 slots. It is
BOOST-only for a reason: gating on this feature scored below baseline, and it
must never replace gap/rvol. Every threshold in 10-20% and gain in 1.5-5.0 beat
baseline -- a flat plateau, so not fitted. The shipped 15%/3.0x are the measured
step and the measured lift, not the sweep's peak cell.
Fail-open: an unknown range yields a neutral 1.0, never a penalty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
criteria_evaluated + signal_blocked held 3.1 GB of a 7.6 GB events table.
On 2026-09-10 alone the watcher wrote 191,458 criteria receipts across 57
symbols -- ~3,359 per symbol per day -- and 154,260 of them (80.6%) were
for names whose static open gap was outside the tradeable band. NOK, NIO,
PATH, PLUG, SOFI, AAL and F all sit inside the $2-20 price filter, so
WATCHER_PRICE_MIN/MAX admits them; only the gap floor excludes them, and
that was checked per-tick instead of once up front.
Apply the SHARED population gate (strategy/population.population_reject --
the same function, same config the entry path already enforces) before the
evaluation and its receipts. This is behaviour-preserving for trading: those
names were being scored, emitted, and then rejected downstream anyway --
429 "outside tradeable population" risk events across 221 symbols, led by
exactly the names that top the receipt counts. The decision just moves
upstream of the write.
Details that matter:
- The verdict is recomputed each tick, never cached. A >150% gapper reads
"squeeze unverified" on thin volume and is admitted once rvol confirms;
a cached rejection would bench it for the session.
- _transition only emits on CHANGE, so a rejected symbol costs one
state event per session and is then silent.
- Fail-open throughout: an unreadable open, a missing previous close or
absent volume returns None, so a data gap can never bench a symbol.
- Gap is the STATIC OPEN gap (session open vs previous close), fixed for
the day, not the live price -- the verdict must not wander intraday.
Off by default (population=None) so every existing caller and test keeps
its current behaviour; the live loop opts in, and WATCHER_POPULATION_GATE=0
restores the old path.
Note for diagnostics: momentum_cli's criteria views now show only names
inside the population. That is the intended trade -- the excluded rows
described symbols the bot could never have taken.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
Tradier returns total_equity 0.0 during its overnight maintenance window --
sixteen times in the last fourteen days, all around 01:06. Three defects
compounded so that a zero could size a position off default_equity:
account_equity = equity or config.default_equity
`or` cannot tell "not supplied" from "read as zero", so a real 0.0 became
100000. Then the position-value cap, derived from the same equity, was
itself 0 -- and the guard
if max_position_value is not None and max_position_value > 0:
SKIPPED the cap precisely when the cap said "no room". Finally the auto
path (~96% of approval requests) had no `pos_cap <= 0` early-out, unlike
the manual path at request_entry_approval. Net effect on a ~$250 account:
risk 5% of 100k = $5,000, capped only by liquidity. The broker would have
rejected it for buying power, which is luck rather than design.
Never fired in live hours -- every zero read landed with the market closed,
and _current_equity caches the last good value -- so this is latent, not a
realised loss. The reachable path is a boot that coincides with the
maintenance window, when no good read has been cached yet.
Distinguish None from zero, refuse to size on non-positive equity, treat a
zero cap as "no room" rather than "no cap", and mirror the manual path's
early-out. TRADING_DEFAULT_EQUITY is set to 300 on the live stack so even
the untested fallback cannot oversize.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
docs/LIVE-CUTOVER.md already points at this file, so the cutover procedure
referenced something that existed only on one machine. It carries no
secrets -- keys stay in .env (gitignored) and the LAN address it names is
already in .env.example, the compose YAMLs and the systemd unit.
Tracking it also pins the two settings this branch corrects, with the
reasoning inline: the exit retune, and the Ollama endpoint that had drifted
to a port with nothing listening. Untracked, a rebuild would have quietly
restored both faults.
Note the precedence: compose env < overrides.env < config_overrides (DB).
The database wins, so editing this file alone changes nothing on a running
stack -- it is the bootstrap seed and the record of intent, not the live
switch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
Distills each gapper's premarket tape (first-hit price/time, pm high/low,
run %, open-vs-pm-high, VWAP distance, jackknife count) into a
premarket_structure table for entry-conversion analysis. Motivated by
FCUV 2026-07-31: our DB captured the full $1.98->$10.49 premarket ramp
while the ORB entered at $11.89 into the reversal.
Pre-registered hypotheses H1-H3 locked in the module docstring before any
data was examined; report command prints descriptive buckets only, with
n>=30/bucket floor and Phase-2 shadow bar required for promotion. Gates
nothing, trades nothing.
Wired into the nightly scheduler after premarket_eval. One session scan
per build (per-symbol queries were ~100x slower); partial session index
created post-close only (mid-market CONCURRENTLY build deadlocked against
live ingest).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fresh DB has no config_overrides row, so without the compose-level pin the
parallel stack falls back to the thin IEX feed and its rvol/volume decisions
diverge from the main stack (which carries feed=sip in the DB).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Self-contained shakeout stack beside the paper stack: own postgres (host
5434, disposable pgdata-tradier volume), own dashboard (host 8768), loop
with BROKER=tradier (sandbox). Data plane stays Alpaca; Telegram/deadman
disabled so main-stack alerts stay unambiguous; main data dir mounted
read-only (inherits learned_params/overrides.env). Filled local copy is
git-excluded; bootstrap_live_db.sh playbook reused for DB parity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Read-only nightly-scheduler job (Friday-gated): buckets realized P&L by entry
time (open/late-morning/midday) and, for each market_exit close, computes the
post-exit run-up capturable under a breakeven stop (0 if price undercuts entry
first; intrabar ties count as stopped). Emits a weekly_tally event + Telegram
summary. Evidence accumulator for the dead no_new_entries_after cutoff and the
weakness-exit winner-clipping question — no live trading path touched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docs/LIVE-CUTOVER.md: full flip-day procedure — staged pieces, discovered
dependencies (second paper key pair, DB seeding, learned_params copy,
telegram token, ports/dirs), 15-min cutover with the paper stack running
throughout, verification list, rollback. bootstrap_live_db.sh seeds the
live stack's Postgres from the paper nightly dump and truncates the
money-record tables so research history carries over while live P&L
starts at zero.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dashboard backtest sources sessions from setup_events v1, which only the
labeler writes — and the labeler was never in the nightly scheduler, so
setups (and therefore backtests) froze at the last manual run (07-17).
Root blocker was _insert crashing on duplicate setup_ids on any re-run;
now ON CONFLICT DO NOTHING, so incremental nightly builds are safe.
Scheduler gains 'labeler build' + 'build-tod1' after eod_replay in both
compose files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Day-22 forensic (BIYA session): 6 entry submissions, 0 fills — IREX missed
+1.7R, IRE +1.65R by one cent. Two root causes fixed:
1. Slow-path limit rested AT the trigger while price ran 2% past it, and
retries reused the same stale price. Marketable model now prices from
max(trigger, live last) via price_provider; the anti-chase extension
gate gets the live ref too (was an entry_ref==trigger no-op on this
path). TRADING_ENTRY_FILL_MODEL flipped to marketable.
2. Fast trigger path armed nothing all day: legacy rvol (cum-so-far /
full-day MEAN) is ~0.1-0.3 minutes after the open for a normal heavy
name and its mean is poisoned by reverse-split/prior-squeeze days
(BIYA legacy 1.27 vs real activity 481x median). New rvol_tod =
cum / (MEDIAN(20d) * elapsed-session fraction) drives arm eligibility
and the scanner keep-filter; legacy rvol untouched for validated
boundaries (gap-squeeze rvol>=5).
Also: stagnation time-stop (TRADING_EXIT_STAGNATION_MIN/_R, default OFF)
in simulate_exit + manage_live with sweep_exits.py variants for
validation before any live enable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Telemetry: discovery took 6-30s ON the single scheduler, freezing ingest/arm
so the trigger read stale bars — the real 'late edge detection'. Now discovery
runs on its own daemon thread with a dedicated DB connection; the watchlist it
mutates is a lock-guarded Watchlist (list-like drop-in, no torn reads); the
scheduler discover step auto-disables (single-writer). Loop is now 3 fast
threads (entries/exits/discovery) + slow scheduler for tolerant work.
Also: nightly_tune writes a heartbeat every run, so the boot alert distinguishes
'tuner not running' (real) from 'ran, no profitable ORB config, kept params'
(expected — the strategy has no edge). Kills the misleading Telegram alarm.
543 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exits move off the single-threaded scheduler onto their own daemon thread at
1s (mirrors the fast trigger thread). Dedicated research connection (shared
facade isn't thread-safe); scheduler exits step auto-disabled when the thread
runs (single-caller invariant — no racing instance state, no double-submit);
manage() already idempotent (stop ratchets >1c up only). 3 driver tests + 539
suite green. On fast small-caps this is the 'get out fast' the owner asked for.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner: 'make all decisions as fast as possible.' The real entry-detection
ceiling was ingest — bars refreshed every 20s, so the 4s trigger thread read
data up to 20s stale and detected breakouts late. Ingest is incremental+cheap,
safe at 5s. Fresher bars + faster arm (trigger levels) + 2s trigger check +
the 4s exits shipped earlier = the whole ingest->detect->decide->exit loop
tightened. One batched data call per cycle so Alpaca rate limits are fine.
Sub-second exit management on a dedicated thread is the next (code) step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner's real complaint: late exits gave back gains. Exits ran on the slow
scheduler at 12s while entries run on the 4s fast thread — exits were 3x
slower than entries. A small-cap can spike and round-trip inside 12s. Drop
TRADING_EXIT_MANAGE_INTERVAL_SECONDS to 4s (config, reversible, both compose
files). Deeper fix (exit mgmt on the sub-second trigger thread) planned
separately. Entry chase-tail (24% fill >5% above break) is the next
investigation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One playlist per add-playlist CLI call made adding a second playlist a chore.
Now YT_PLAYLIST_URLS takes a comma-separated list of playlist URLs (list=
param extracted) or bare ids; the transcripts service seeds the playlists
table from it at boot (insert-if-missing — hand-set labels are never
clobbered), and `python -m youtube_claims.run sync-playlists` does the same
on demand. Documented in both compose files; 3 tests.
Also: queue the two executor-surfaced defects (OrderFilledEvent side,
/api/attribution env hardcode) + the flaky nightly regime build-features
step in the plans index; regime_features 07-16 backfilled supervised.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires the regime signal into the money path — fresh entries are BLOCKED on
sessions the walk-forward model scores unfavorable. Session-level: evaluated
once per day, cached. Enabled in both compose templates (paper).
Correctness (blocker 1 resolved): the LIVE feature set drops tape_regime (the
only non-strictly-open feature — AND measured 0.000 importance) + breadth_gap
(also 0.000). The 5 remaining features are all non-look-ahead at entry time:
breadth_rvol = median(relative_volume=vol_upto/avg_vol_20d) over the session's
setups so far (the SAME up-to-trigger calc the live bot runs, NOT day-end),
spy_* lagged to prior close, trailing from PRIOR sessions. The live single-
session fit reproduces the batch meta_v1 verdict exactly on the last 6 sessions.
Safety: fail-open everywhere — disabled, unfit model (<60 prior sessions),
missing breadth_rvol (pre-setup), or ANY exception returns None and never
blocks. The gate can only SUBTRACT trades on bad-regime days.
- research/regime.py: live_session_favorable() (refit on priors <today, predict
today; writes regime_scores['meta_live'] for forward validation) + LIVE_FEATURE_COLS
+ the breadth_rvol-missing fail-open guard.
- trading_execution.py: TRADING_REGIME_GATE_ENABLED / _P_MIN settings + _regime_reject()
(cached per session) + a REGIME gate after SESSION in _evaluate_entry_gates.
- docker-compose.yml + truenas-compose.yml: TRADING_REGIME_GATE_ENABLED=1.
- tests/unit/test_regime_gate_live.py: block / pass / fail-open(x3) / scored-once-per-day.
KNOWN train/serve skew (documented, validate forward via meta_live vs EOD label):
model trains on full-day breadth_rvol, live uses setups-so-far. Small edge on a
~266-session sample; paper only. Suite: 504 passed, 15 pre-existing loopback fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The session-level metalabel P(our ORB profits today) — the one lever that
survives out-of-sample. Backtested on 266 prod sessions (walk-forward):
trade-always (trained subset): -0.111R/session, 36% hit
meta_v1 GATED (trades best ~32% of days): -0.004R/session, 48% hit
delta +0.107R in-sample, +0.128R OOS (recent 80 sessions), improves across folds
It does NOT make the bot profitable — it takes a bleeding strategy to roughly
flat by sitting out the worst ~2/3 of sessions. threshold_v1 rule gates too
little to matter; meta_v1 (walk-forward GBM) is the signal. Leaky full-fit
(+0.61R) >> walk-forward confirms the discipline was load-bearing; the OOS
holding is the evidence the +0.078R is real, not look-ahead.
Changes:
- research/regime.py: batch minute-bar load per session (64k per-fire round-trips
-> ~266 queries; fixed a >5min build to ~3min) + incremental label pass (skip
already-labeled sessions; --rebuild forces full). Nightly cost now ~one new day.
- docker-compose.yml + truenas-compose.yml: scheduler runs build-features + score
(both models) before `experiments run`, nightly. ALL-SHADOW — does NOT gate live
trading (that stays owner-gated Step 5).
- admin_api.py + admin.html: POST /api/regime/run (detached) + GET /api/regime/status
+ a Run/Status button — the panel had no way to trigger this.
- plans/011 + docs/research/regime_notes.md (6-book distillation) + plans 008/009 docs.
Prod state (shadow, rollback = DROP regime_features/regime_scores + DELETE
experiments WHERE id LIKE 'regime%'): 266 feature rows, 532 scores, 2 experiments
registered.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step 3 of plan 009 (config + code; no live-path change):
- research/ingestion/market_context.py: pulls SPY/QQQ full-session minute bars
(SIP, post-close embargo-safe) + daily bars via the existing upsert helpers, so
the labeler's market-context features have a FORWARD source (index ETFs never
enter the gapper universe the live loop ingests). Mirrors sip_backfill's shape.
- both compose scheduler blocks: run it after premarket_eval, guarded || echo.
- NEWS_ENRICH_ENABLED on: added to docker-compose x-app-env anchor (loop +
scheduler inherit it); truenas-compose + .env.example already had it. Catalyst
is a FEATURE source only, never a live gate; enabled-but-down never blocks
trading. Enrichment OUTPUT is verified by the orchestrator (needs live Ollama).
Compose statically validated (yaml.safe_load). Ingest self-checked via a
fake-client :memory: round-trip through _preload_spy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every POST endpoint (approve/reject/exit/watch) crashed with NameError:
do_POST references os.environ + hmac.compare_digest but neither was imported.
Adds the imports, a non-loopback regression test for the token gate (no token
-> routes; token set -> 401 without/with-wrong header, 200 with correct), and
plumbs DASHBOARD_TOKEN + PG_SKIP_SCHEMA=1 into both compose dashboard services
(+ .env.example docs for DASHBOARD_TOKEN / COMMAND_BUS_ENABLED).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 006 step 5. python -m research.experiments run && adjudicate after
nightly_tune in both scheduler blocks (guarded with FAILED echoes). ALL-SHADOW:
emits verdicts, never touches live gates/sizing/exits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tuner was fiction in production: the 20:00 ET scheduler ran only
premarket_eval (+ eod_reconcile), so learned_params.json sat 26 days stale and
the whole-market dataset never grew. Both scheduler loops now run, in order,
after the eval/reconcile pass:
python eod_replay.py || echo "[scheduler] eod_replay FAILED"
python nightly_tune.py || echo "[scheduler] nightly_tune FAILED"
Order matters: replay grows the dataset first, the tuner fits after. 20:00 ET is
after close, so replaying "today" is complete data — NOT look-ahead (replay is a
labeling job, not a live gate). Deletes eod_replay.sh (dead cron-era wrapper
pointing at a nonexistent venv; compose is the runner).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 003 step 5 (completes plan 003). research/eod_reconcile.py diffs broker
truth (open positions, today's live orders) against event-store truth (today's
submits/fills/closes) and emits a loud eod_reconcile event + Telegram alert on
any drift — the duplicate-resting-order-after-restart class (2026-07-06) caught
automatically instead of by hand. Read-only (reports, never auto-fixes). Wired
into both scheduler blocks after premarket_eval. Pure diff core is unit-tested.
Suite: 373 passed (+22 plan-003 tests), 15 fail = WSL-loopback dashboard only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
grafana/ pointed at the decommissioned pre-migration DB with committed dev
credentials (rotation note in plans/002-deletions.log); its dashboards
provisioning MOVED to deploy/compose/grafana/dashboards (docker-compose.yml
bind-mount re-pointed — the in-compose grafana service keeps working).
docker-publish.yml fired on a nonexistent branch pushing a wrong image name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Production Postgres had NO automated backup (pg_dump appeared only in the
one-time migration script). New db-backup service in both compose files fires
20:10 ET Mon-Fri: pg_dump | gzip with pipefail + gunzip -t + tmp-then-rename
(a mid-stream death can't masquerade as a backup), prunes past
BACKUP_KEEP_DAYS, writes last_backup_ok on success. postgres:18 image =
version-matched client (app image has none). TrueNAS host is pull-only, so
its copy inlines the script. Restore: gunzip -c dump | psql $DATABASE_URL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The loop's Telegram notifier runs inside the loop process — loop-down means
alerts-down. The always-up qabot now reads max(events.timestamp) every ~120s
(ET-naive vs ET wall clock per RUNBOOK): stale > DEADMAN_STALE_MIN during
market hours (holiday-aware via runtime.market_calendar) alerts at most every
30 min; DEADMAN_DAILY_PING sends one OK-ping after 20:30 ET. Injected
clock/read/send; 6 unit tests, no network or DB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Consolidated 4-tool review (2 correctness agents + Codex + over-eng audit).
Money-path (regression-tested):
- _flatten_all: moved the fresh-broker close from close_session INTO the
shared function + added a broker-held guard on entry cancellation, so
the daily-loss CIRCUIT BREAKER (which called _flatten_all raw) no
longer strips a just-filled bracket's legs or leaves a sync-gap fill
naked. One guard, all callers.
- restart state rebuild: _fresh_entries cap, _requested_symbols dedup,
and losing benches now rebuild from today's events on boot — a
mid-session restart no longer resets the cap to 0 or re-fires a
resting order into a duplicate (the 2026-07-06 incident).
- POST /orders no longer retries a lost-response URLError (was
re-sending duplicate live orders); GET/DELETE still retry.
- expire_stale_entries now @_locked (raced the trigger thread).
- _session_open: ET conversion instead of a hardcoded 13:30-13:33Z
window that only matched EDT — the fast path armed nothing all winter.
Security:
- admin panel fail-CLOSED on empty ADMIN_TOKEN (was fail-open on a panel
fronting the docker socket); header-only token, constant-time compare.
- admin API responses redact secret override VALUES (live keys would
have echoed after a paper->live switch).
- dashboard: removed wildcard CORS (cross-origin drive-by could drive
trade actions); optional DASHBOARD_TOKEN gate on POST.
Deletions (~805MB + dead code shipped in the prod image): momentum-
rebuilt.zip, archive venv/htmlcov, and schwab/ core/ telemetry/ models/
runtime/providers/ storage/replay.py controlroom/ + three dead
evaluation stubs + package.json — all verified zero-importer.
+4 regression tests. 99 money-path/trigger/exit tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Systematic diff of the old .env against the deployed loop env found 68
code-read keys running on silent defaults — including the risk cap
($500 vs $868 observed), the VWAP gate, the exit ladder, and the
NEWS_ENRICH_ENABLED flag that kept the catalyst engine dead despite
OLLAMA_ENABLED=true (enrichment requires BOTH, config.py:1235 gate).
Template env block regenerated with the full inventory, secrets as
<FILL-ME>; a ready-to-paste block with real values generated locally
(data/loop-env-block.yaml, gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 10 era documents (phase plans, build summaries, redesign writeups, old
runbooks, completed consolidation plan) moved to docs/history/ with a
do-not-follow README.
- Root README: current validated population ($2-20, 10-35% static gap +
squeeze exception, SIP), Deployment section, automated-maintenance
reality, corrected repo map (admin panel, youtube_claims, deploy/).
- RUNBOOK rewritten for the running system: daily rhythm, health checks
per deployment, and the production failure-mode table earned this week
(env drift, pgdata mount traps, uid 10001, zombie approvals, PG_SKIP_
SCHEMA, timezone artifacts).
- deploy/README.md is now the three-mode index: compose (canonical),
systemd (local dev — unit templates now committed), helm (refresh
staged as issue #8). Old k8s doc moved to deploy/helm/README.md.
- docker/README.md: six TrueNAS gotchas, each from a real outage.
- Admin panel improvements staged as GitHub issues #4-#7.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The loop healthcheck opened a fresh connection every 60s with schema
auto-apply enabled; the CREATE-IF-NOT-EXISTS lock bursts cancelled
minute_bars autovacuum every single cycle (visible as 'canceling
autovacuum task' each minute post-migration), so the 19M-row table
never got analyzed, snapshot queries crawled, browsers timed out, and
the dashboard log filled with double BrokenPipeError tracebacks.
- healthchecks now set PG_SKIP_SCHEMA=1 (both compose files)
- api/main.py _send_json swallows BrokenPipe/ConnectionReset (client
hung up; nothing to tell it)
- live DB: ALTER SYSTEM max_wal_size=4GB, shared_buffers=1GB,
effective_cache_size=3GB, checkpoint_completion_target=0.9 (the
checkpoint-every-24s warning under migration load on NAS storage)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New admin service — LAN-only, token-authed (ADMIN_TOKEN), card-based UI
matching the owner's preferred dashboard style. Capabilities:
- health: docker containers via /var/run/docker.sock (stdlib unix-socket
HTTP client), DB heartbeat, Alpaca account/clock, Ollama, feed
- config: whitelisted strategy knobs (TRADING_/TRIGGER_/WATCHER_/...)
edited via data/overrides.env, which every entrypoint now loads with
override=True at boot (runtime/env_overrides.py) — so changes apply on
container restart without touching the deployment YAML; secrets are
presence-only and rejected server-side if edited
- profiles: named knob presets (seeded: validated-core, conservative,
squeeze-hunter) applied in one click
- restart buttons per container; paper/live switch guarded behind
ALPACA_LIVE_* keys existing
Verified live in docker: auth 401s, status/config/profiles/write paths,
secret-edit rejection, page serves. Added to both compose files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI now pushes fez5587/momentum-app to Docker Hub alongside GHCR (creds
via repo secrets). First push done manually and verified anonymously
pullable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
truenas-compose.yml now uses /mnt/volume/Apps/momentum and publishes the
new postgres on 5433 so it can coexist with the existing instance during
migration. deploy/docker/migrate_db.sh: dump / restore / verify steps
(pg_dump -Fc read-only source, parallel pg_restore, ANALYZE, row-count
diff) with env-overridable URLs and PG18 client autodetection —
connectivity smoke-tested against the live DB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TrueNAS custom apps cannot build: the main compose's 'build: .' (plus
relative .env/bind mounts) fails app 'up'. Added a GitHub Actions
workflow publishing ghcr.io/fez5587/momentum-app on every master push,
and deploy/docker/truenas-compose.yml — a pull-only variant with
dataset host-paths, inline environment (secrets as <FILL-ME>
placeholders, current validated strategy knobs pre-filled), and the
same healthchecks. Validated with docker compose config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
.env.example was stale (pre-migration keys, missing all current
TRADING_*/TRIGGER_* knobs). Regenerated from the live config with
secrets <REDACTED>; a leak check caught live Alpaca credentials
embedded in commented-out PowerShell lines and scrubbed them before
commit. README now documents the three config layers (protected
secrets vs promotable strategy knobs vs infra addresses) and the
external-Ollama networking expectations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the pre-migration compose file up to the current architecture:
python 3.13 image with ffmpeg + faster-whisper/yt-dlp (transcripts),
postgres 16 -> 18 (prod parity), and the real service set — loop
(--no-dashboard, heartbeat healthcheck), dashboard_api.py on :8765,
telegram qabot, youtube transcripts, and a 20:00-ET weekday scheduler
replacing the premarket-eval systemd timer. Jobs (journal/eod-replay/
nightly-tune) and Grafana keep their profiles. Self-contained by
default (in-stack DATABASE_URL); deploy/docker/README.md documents the
pg_dump migration, the transition mode against the LAN DB, and the
cutover checklist. Validated: image builds, compose config passes, and
a smoke container served the dashboard (HTTP 200) against the live
LAN Postgres.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phases 1+2 of the staged control-room plan, adapted to this backend and shipped as a
PARALLEL service so the battle-tested :8765 dashboard keeps running until parity.
The owner's ask — live analysis with live tickers, realtime — is the centerpiece:
- LIVE tab: tracked/armed tickers grid with real-time SIP last-trade prices (1s,
flash-green/red on change), state, gap%, RVOL, trigger + distance, stop, quality
score, POPULATION-gate check (display mirror of the live gap/volume floors), VWAP,
catalyst — merged from the loop's trigger snapshots + ready-signal projections.
- Header: PAPER mode badge (chameleon pattern), market phase, live equity, heartbeat,
clock, and the PANIC brake (cancel ALL open orders via broker API, typed "STOP"
confirmation — the plan's conditional-friction flow).
- Positions: live broker positions w/ uP&L colors + naked-stop flag (2s).
- Gates & Attribution: today's gate-cut histogram (5s) + 7-day per-stage attribution.
- Approvals: pending queue w/ approve/reject via the existing dashboard REST + dialogs.
All DB/broker calls via run.io_bound (never block the event loop); dark theme +
semantic tokens per the plan. systemd momentum-controlroom (:8766), unit in
deploy/systemd/. Headless-verified: live header (equity, phase), 10 ticker rows
rendered, zero JS errors.
Remaining from the staged plan (next phases): manual orders, settings dials, audit
tab, connection-lost action guard, live-mode chameleon when a real-money broker lands.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pm1 verdict is promising-but-underpowered (n=21): the forward moves are real (+11-14%) but the
sample is too small to conclude. The live loop ingests premarket bars every trading day, so this
automates the "accumulate more days then re-validate" step:
- research/premarket_eval.py: rebuild pm1 from the grown DB, append a timestamped metrics row
(n, days, +1R rate, Wilson-95 LB, median upside/adverse, big-gap +1R, pass/fail) to
data/premarket_eval_history.tsv, and print the current validate output. Watch n climb + the
Wilson bound tighten until the locked bar is decisively cleared or failed.
- deploy/systemd/momentum-premarket-eval.{service,timer}: weekday-evening (Mon-Fri 20:00) oneshot,
Persistent=true (catches reboots). Installed to ~/.config/systemd/user and enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Single-machine equivalent of the Helm package: builds the app from the
Dockerfile, runs PostgreSQL with a persistent volume, and starts the trading
loop with its embedded dashboard on :8010.
- postgres:16 with healthcheck + named volume; app waits on it healthy
- DATABASE_URL overridden to the in-network postgres service (self-contained);
.env is optional (dry mode without ALPACA_* keys)
- journal / eod-replay / nightly-tune under a "jobs" profile, run on demand;
they share the app-data volume so learned_params.json reaches the loop
- Grafana under a "monitoring" profile with a compose-specific datasource
override pointed at the postgres service
- deploy/README.md documents the Compose workflow
Validated with `docker compose config` across all profiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCVMm2AbmZvvxwLTX6UpNR
Document reliability, security, and CI/operational improvements for the
Kubernetes package as an actionable backlog (not yet implemented). Notably
flags the loop-heartbeat liveness gap: /api/health proves only the dashboard
thread is alive, so a hung trading loop would go undetected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCVMm2AbmZvvxwLTX6UpNR
Containerizes the momentum monolith and packages every run mode for
deployment in a single Kubernetes namespace, with PostgreSQL hosted in
the cluster.
- Dockerfile (python:3.11-slim, non-root, writable /app/data) + .dockerignore
- GitHub Actions workflow publishing to ghcr.io/fez5587/momentum-system
- Helm chart (deploy/helm/momentum):
* PostgreSQL StatefulSet + PVC (single source of truth; schema
auto-creates from storage/pg_schema.sql on first connect)
* App Deployment running run_live_paper.py with the embedded dashboard
(replicas: 1, Recreate; the dashboard's write endpoints need the
in-process TradingExecutionService) + Service/Ingress on 8010, plus a
wait-for-postgres initContainer
* CronJobs for journal / eod-replay / nightly-tune (US/Eastern via
spec.timeZone) sharing the app data PVC
* ConfigMap (tuning env) + Secret (DATABASE_URL/broker keys, existingSecret
supported); DATABASE_URL built from the in-cluster Postgres service
* Optional Grafana provisioned against the in-cluster Postgres
- deploy/README.md with install, secrets, and constraints
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCVMm2AbmZvvxwLTX6UpNR
