Author SHA1 Message Date
momentum devandClaude Fable 5.1 731e5a5818 chore: close the news-scorer deviation in the live test freeze as adopted baseline
The reactive-coverage cap (3e523e1) has run unchanged since it went live at
10:10 ET on 2026-10-02, so it is part of the strategy under test rather than an
open deviation; only TNON that morning traded without it. Dated closed with a
resolution note so the daily report stops listing it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXxioW7HA18AY1gqnzA2HN
2026-10-10 10:04:04 -05:00
momentum devandClaude Fable 5.1 5f0625fb6f fix: watcher spends its 25-slot evaluation budget on in-population names first
The watcher evaluates candidates[:max_symbols] each tick. The live provider puts
dashboard/scan-injected symbols first (they never expire) and ranks the rest by
bar count, so on 2026-10-08 fifteen injected intraday movers plus liquid names
filled all 25 slots and every in-population gapper stopped being scored at 09:43
while 50 already-rejected names were re-rejected every 8 s. Entries were not
affected (the trigger book arms independently and injects its armed names), but
the state board and the judge's reasoning trail went stale for the session.

Names already in out_of_population now go to the back of the list and rotate, so
they are still re-checked when budget remains (a squeeze can be re-admitted) but
can never crowd out a name that can still trade.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXxioW7HA18AY1gqnzA2HN
2026-10-10 10:04:04 -05:00
momentum devandClaude Fable 5.1 b0fb0582d7 fix: journal round trips that open and close between two broker polls (TNON)
The trade journal learns about exits only by diffing consecutive position
snapshots. On 2026-10-02 TNON was bought at 09:38:37 and its bracket stop filled
at 09:39:26, so no snapshot ever held it and no position_closed was written: the
80-trade scorecard dropped a -1R loser and the re-entry bench fired on a trade the
journal did not have.

AlpacaPaperSync.reconcile_unseen_round_trips runs after the snapshot diff on every
sync. For a symbol open in neither snapshot that has a filled buy and a filled sell
submitted this session (ET), it emits one position_closed per unjournaled exit fill:
entry is the VWAP of the buy fills, exit the sell's fill, stop the session's stop
order, position_id the exit's broker order id. Exit ids already journaled (by the
diff path or an earlier run, seeded from the store) are skipped, so nothing is
double-counted across polls or restarts. Long round trips only; the bot is long-only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXxioW7HA18AY1gqnzA2HN
2026-10-10 10:04:04 -05:00
momentum devandClaude Fable 5.1 b5b38ab798 fix: scorecard counts round trips the broker filled but the journal never closed (TNON)
On 2026-10-02 the live bot bought 7 TNON at 4.62 and the Tradier sell-stop filled at
4.30 forty-nine seconds later, before a position record existed, so no position_closed
event was written and the 80-trade scorecard silently dropped a -1R loser. The daily
equity-vs-P&L gap ($2.31 that day, cents everywhere else) is what exposed it.

load_trades now merges the journal with a cross-check of the broker's filled orders
(account_orders_updated snapshots, one row per broker order id). For any session day and
symbol with a filled buy and sell but no journaled close, reconstruct_missing synthesises
the trade from the fills: fill VWAPs for entry/exit, the sell-stop's stop price for R.
Such trades are labelled "[broker fill; journal missed]" and the daily report alerts on a
same-day miss. The paper side of the exit A/B skips the check: that store's order
snapshots run to 2.2 GB over the window and its Alpaca journal is reconciled by
alpaca_paper/sync.

Corrected test state: 16/80, mean R -0.08, realized +$0.91 (was 15/80, -0.02, +$3.15).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXxioW7HA18AY1gqnzA2HN
2026-10-08 12:58:53 -05:00
momentum devandClaude Fable 5.1 3ea83d568b fix: run pg_dump/pg_restore windowless so closing a console cannot kill the nightly backup
The LiveDbBackup task runs under pythonw, but pg_dump.exe is a console program, so
Windows opened a visible console for it. On 2026-10-06 that window was closed and the
dump died with 0xC000013A before writing a byte (the Ross recap died the same way ten
minutes earlier). Pass CREATE_NO_WINDOW to both subprocess calls, the same fix the
judge paper trader got in 7e6f881.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXxioW7HA18AY1gqnzA2HN
2026-10-08 12:58:53 -05:00
momentum devandClaude Opus 5.5 b3f023f6e2 feat: nightly off-box backup of the live DB, and phone alerts from the daily report
The live (Tradier) stack has no backup service. Only the paper stack backs up its
own DB, so the 80-trade test's whole record lived only on the TrueNAS pool.
- deploy/backup_live_db.py, run by Task Scheduler "Momentum\LiveDbBackup" at 20:40 ET
  nightly: pg_dump -Fc (PostgreSQL 18.6 client) of the live DB to
  C:\Users\Admin\momentum-backups. It is verified with pg_restore --list (the events
  and config_overrides data must be present), keeps the newest 7, and writes
  status.json. A broken dump is deleted rather than kept.
- research/daily_report.py alerts when the last backup failed or is more than 36 h
  old. When there are alerts it sends them to Telegram, using the stack's own sender
  and the credentials in .env. --no-telegram suppresses that for dry runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-02 21:46:05 -05:00
momentum devandClaude Opus 5.5 7e6f881b17 fix: judge paper trader runs windowless and re-watches earlier symbols after a restart
On 2026-10-02 the trader died 5 times (13:15-15:05 ET). It left no traceback, and
each death came within the task's 5-min re-fire window, which fits its console
window being closed (as on 2026-09-29) while the PC was in use. Every relaunch popped
a new window.
- Task Scheduler now launches it with pythonw: no console window to close. Under
  pythonw, stdout/stderr go to data/judge/logs/live_<date>.log, so a real crash
  still leaves its traceback.
- A relaunched trader re-loads every symbol today's earlier runs were watching (from
  the journal's "add" rows). It used to know only the names still passing the
  scanner, so it stopped deciding on the others and never saved their ticks; the
  review then had nothing to compare CABO and APLX against.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-02 16:08:55 -05:00
momentum devandClaude Opus 5.5 ac92280b9a fix: daily report no longer flags a stack as silent when run mid-session
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-02 11:11:27 -05:00
momentum devandClaude Opus 5.5 1101e0e3fe chore: record the news-scorer reactive-coverage cap in the live test deviations
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-02 09:17:35 -05:00
momentum devandClaude Opus 5.5 3e523e132a fix: reactive coverage (halt notices, "why is X up") no longer scores as a catalyst
qwen3:14b rated "halted on circuit breaker to the upside, stock now up 172%" at
conviction 0.8 / sentiment +1.0, a bigger rank boost than most real news. So
names that had already run were ranked up.
- Prompt: halt notices, movers roundups and articles that only report a price move
  are not a new catalyst (conviction <= 0.1). If one names a new announcement,
  classify it with conviction <= 0.3.
- A deterministic cap that does not depend on the model: halt type or halt wording
  caps conviction at 0.1; after-the-fact wording ("why is X stock up", "X shares
  surge", "stocks moving") caps it at 0.3. Dilution flags are never touched.
On the 4,046-headline news history the pattern catches 100% of halt notices and 82%
of roundups, 0% of analyst items and 0.8% of earnings items. Against qwen3:14b on a
fixed 20-headline set: reactive mean boost 0.318 -> 0.055 (max 0.80 -> 0.21); real
catalysts 0.353 -> 0.362; offerings 3/3 still flagged dilutive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 21:53:42 -05:00
momentum devandClaude Opus 5.5 082d01e8bd chore: close the news-scoring outage in the live test record (qwen3:14b from 2026-10-01 22:37 ET)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 21:40:30 -05:00
momentum devandClaude Opus 5.5 6189885aed chore: record the qwen3:14b-on-mason restore plan for the live news scorer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 21:06:56 -05:00
momentum devandClaude Opus 5.5 4361f80e1b fix: drop unused import that failed the CI ruff gate (blocked image builds)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 21:00:44 -05:00
momentum devandClaude Opus 5.5 eed4ec0b62 fix: catalyst classifier works with qwen3 (OLLAMA_THINK) and no longer hides an empty answer
qwen3 thinks by default. With format=json, Ollama then returns "{}", which the parser
silently read as a confident "other" (sentiment 0, conviction 0), so every headline
scored zero with no error. Two changes:
- OLLAMA_THINK (true/false). It is sent as Ollama's "think" flag only when set, so other
  models are untouched. qwen3:14b needs OLLAMA_THINK=false.
- A JSON object with none of the expected keys is now a failure (None, retried
  next pass), not "other".
Checked through the live code path against qwen3:14b on mason (think=false): 1.4 s per
headline. 5/5 offerings (including an ATM sales agreement and a PIPE) flagged
dilutive. 'Stocks moving' roundups scored no catalyst. Earnings, guidance, analyst,
FDA and contract items were labelled correctly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 20:59:32 -05:00
momentum devandClaude Opus 5.5 b38edd4ca9 feat: daily after-close report and the judge paper-trial criteria
research/daily_report.py, run by Task Scheduler "Momentum\DailyReport" at 16:25 ET
on weekdays, writes data/reports/daily_<date>.md with ALERTS on top:
- the live opening-range bot's frozen 80-trade test: verdict, mean R with CI,
  P&L, equity headroom over the floor, today's trades, open deviations, settings
  drift;
- the judge v2 paper trial: judge.review for every counted day, cumulative
  paper-vs-sim execution parity, today's flagged trades, trader errors and
  restarts;
- the old-vs-new exit A/B (alerts at 50 pairs);
- stack health: down at the open, silent gaps of 10+ min, the press-release
  recorder.
Read-only everywhere (PG_SKIP_SCHEMA=1, SELECTs only). Replayed on 2026-09-30 it
flags that day's real problems: the judge error loop, the paper stack down until
13:51, and a 25-min gap.

judge/PAPER_TRIAL.md, written before any counted trade: trades count from
2026-10-02. At 60 trades the paper trial must show execution parity with the
simulator (paper minus sim >= -0.15R) before a 150-trade performance read. Real
money is only considered after that, and only with the owner's go-ahead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 14:37:18 -05:00
momentum devandClaude Opus 5.5 98805279d1 feat: real-time press-release and SEC filing recorder (prwire)
Premarket news timing could not be tested on history. Alpaca's Benzinga feed has
almost no PR-wire items for microcaps, and wire RSS feeds only keep the latest few
dozen items. So from now on, every item is recorded with our first-seen time.

- Polls PR Newswire, GlobeNewswire, Business Wire and SEC EDGAR's current 8-K, 6-K
  and 424B feeds every 20 s, in parallel. A failing feed backs off up to 10 min.
- Each new item goes to data/prwire/<date ET>.jsonl with seen_utc, feed, title,
  link, tickers (exchange-tagged, e.g. "(Nasdaq CM: ABCD)"; SEC filings via CIK)
  and form. Restarts do not re-log items already recorded.
- Runs on the workstation via Task Scheduler "Momentum\PrWireRecorder" (pythonw,
  no console window; re-fired every 10 min with IgnoreNew so it self-heals).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 08:12:10 -05:00
momentum devandClaude Opus 5.5 8dc97a53f8 feat: judge paper trader runs a software stop alongside the broker stop
It sells as soon as any trade prints at or through the stop, or below the 6% trail
once +1R, walking the new prints in order exactly as the validated simulator does.
Whichever acts first wins: the broker stop or this one.
Why: Alpaca's paper engine filled triggered stops 6-60 s late on 2026-09-30 (CMCT,
PWCM; judge.review), which makes the trial look worse than the simulation for reasons
that are not the strategy's. Brokers also run no stop orders outside regular hours,
so any premarket trading needs this.
JUDGE_SOFT_STOP=0 turns it off. judge.review treats soft_stop/soft_trail as
stop/trail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 07:40:24 -05:00
momentum devandClaude Opus 5.5 5315e99978 feat: judge.review compares each paper trade with the simulator
For a paper day, every order is lined up with what the validated simulator does on
the same recorded ticks: the sim exit from the real fill (stop, 6% trail after +1R,
20-min time stop, sell at the bid 300 ms after the trigger) vs the paper exit,
plus the sim R of orders that did not fill. Trades are flagged when R differs by
more than 0.3R, the exit reason differs, or the entry slipped. The report goes to
data/judge/reviews/<date>.md.

On 2026-09-30: paper -8.25R vs sim -3.97R on the same six fills. The gap is two
trades where Alpaca's paper engine filled a triggered stop 6-60 s late (CMCT,
PWCM; PWCM's own exit was also blocked by the stop-id bug fixed in f3c8be1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-10-01 07:11:54 -05:00
momentum devandClaude Opus 5.5 358b859c59 fix: give Postgres a 5-minute start_period in both TrueNAS composes
After an unclean shutdown, WAL replay outlasted the 5 x 10 s healthcheck window,
so compose marked Postgres unhealthy and refused to start any dependent service.
The paper app sat in 'Deploying' on 2026-09-30.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-09-30 19:06:02 -05:00
momentum devandClaude Opus 5.5 3e3388c7fd chore: note the TrueNAS Ollama outage as a known deviation in the live test
The host ran out of memory (843 MiB free; Ollama held 7.8 GB at a 131k context)
and the owner turned ollama2 off on 2026-09-30 evening. Strategy settings are
unchanged, but the LLM catalyst score is inert while it is off, and the
dilution veto falls back to its regex floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-09-30 19:06:02 -05:00
momentum devandClaude Opus 5.5 f3c8be11b6 fix: judge paper trader tracks replaced stop orders
On 2026-09-30 PWCM's stop was moved, and Alpaca's replace returns a NEW order id.
The trader kept the dead id, so reconcile treated the stop as gone and tried to
arm another one (refused with 403, shares held) about 20 times in 80 s, and the
trail exit could not sell because it cancelled the stale id.

- Adopt the id returned by replace. Reconcile follows replaced_by, and before
  arming it adopts any working sell stop already at the broker.
- Exits cancel every open order on the symbol before close_position.
- If the price is already through the stop, close instead of arming a stop the
  broker would reject.
- tests/unit/test_judge_live_exits.py covers these cases and the 2026-09-29 ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-09-30 15:58:52 -05:00
momentum devandClaude Opus 5.5 347031e2bb feat: judge paper trader survives a crash mid-session
On 2026-09-29 the trader's console was killed at 10:21 ET (exit 0xC000013A) and
it sat dead for the rest of the day. The scheduled task now re-fires every 5 min
from 08:10 to 15:00 local, and IgnoreNew makes that a no-op while the trader is
running. It also catches up if the machine was off at 08:10.

A relaunched trader now picks up where the dead one left off:
- positions still held at the broker are adopted, with the entry, original
  stop, fill time and high-water taken from today's journal and the backfilled
  tape, so the trail, time stop and 15:55 flat still apply. The working stop leg
  is reused; if none is working a new one is armed. A position with no known
  stop is closed rather than guessed at.
- the per-symbol order count is rebuilt from the journal, so the cap holds
  across restarts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3PXXn2tHBzdv6rYtGPz86
2026-09-29 21:21:42 -05:00
momentum devandClaude Opus 5.5 c518747387 fix: judge paper trader reconciles with the broker before exiting
Day 1 of the paper trial (2026-09-29) exposed two exit bugs:
- When the broker stop had already filled, a trail/time exit tried to cancel the
  filled leg, got a 422, returned without clearing the position, and retried
  every second forever (BKYI). The leg-filled check sat after a `continue` and
  never ran.
- A partial fill whose parent was cancelled took its OTO stop leg with it,
  leaving the shares unprotected (MSGY, 82 of 133 filled).

Now every 5 s, and after any failed exit or stop move, the trader checks the stop
leg and the broker position: a filled stop or a flat position is recorded as the
exit, and held shares with no working stop get a standalone stop. Partial
fills get their stop re-armed straight away. Failed exits retry every 5 s instead
of every second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:18:41 -05:00
momentum devandClaude Opus 5.5 f2c7e52c79 feat: judgement-model (v2) paper trader on the Alpaca "Momentum V2" account
A gradient-boosted model scores burst / micro-pullback / whole-dollar signals on
squeeze stocks from tick data, and only its top-scored regular-hours signals
are traded. Validated in data/ticksim before any live use:
  - v1 on a fresh 600-day sample: real ranking (AUC 0.63), thin edge. FAIL.
  - v2 (trained on Jun-Jul of samples 1+2), pre-registered, on 500 NEW Aug-Sep
    symbol-days: +1.11%/trade after the spread (day-block 90% lo +0.54%),
    n=332 over 38 days. PASS. It held at 2 s latency and +0.2% extra cost.

- judge/engine.py: the validated signals, execution gates and features, as one
  shared source.
- judge/model.py: trains and freezes v2 (cutoff = the training top 20%).
- judge/decide.py: pure decision logic.
- judge/replay.py: a live-style replay of recorded days. Parity: 40 of 41 research
  trades matched with identical model scores.
- judge/live.py: the paper trader. Watchlist from the snapshot scan with
  backfill, SIP trade/quote websocket, decisions each second, a limit at the ask
  with a broker OTO stop (cancelled if unfilled in 3 s), trail 6% after +1R,
  20-min time stop, flat by 15:55. $20 risk and $1,000 max notional per trade.
  Everything is journalled and the day's ticks are kept for paper-vs-simulation
  review. Weekends and holidays exit.
- Runs from the workstation (Task Scheduler, weekdays 09:10 ET) with the
  research venv. Paper only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3S5EFKThFVrjmN4HHsHMN
2026-09-28 18:25:26 -05:00
momentum devandClaude Opus 5.5 1cc15f5ea7 chore: restart the live test on 2026-09-29 with the squeeze exception off
TRADING_GAP_SQUEEZE_ENABLED=0 from the next session. The >=150% gap names lost
in both halves of the Jun-Sep history (-0.21R / -0.30R) and in our real fills,
while the 10-35% band made money. 2026-09-28's four trades ran with the
exception on, so they are recorded as a previous period and not counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3S5EFKThFVrjmN4HHsHMN
2026-09-28 16:33:16 -05:00
momentum devandClaude Opus 5.5 0c8a27e23f feat: switch to turn off the >=150% gap squeeze exception (default on)
TRADING_GAP_SQUEEZE_ENABLED=0 rejects >=150% gappers in the population gate
(watcher pre-filter and entry gate) and zeroes the trigger book's squeeze
arm. One switch covers all three paths.

Why: while testing the trader's "trade the leading gainers" idea on 1,093
live-band v1 ORB breaks (Jun-Sep 2026, current exits), top-3 gainers at the
break did WORSE (diff -0.48R Jun-Jul, -0.40R Aug-Sep, 90% CIs excluding 0).
That turned out to be almost entirely the >=150% squeeze-exception names:

  gap 10-35%   Jun-Jul +0.27R (n=342)   Aug-Sep +0.13R (n=327)
  gap >=150%   Jun-Jul -0.21R (n=65)    Aug-Sep -0.30R (n=53)

Our own squeeze fills: -0.31R / -0.33R realized (n=16). By the time a squeeze
name breaks its opening range the move has happened; the trader catches these
names premarket, which the bot does not. This finding was post-hoc (not the
pre-registered direction), but it held in both halves and in the real fills.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3S5EFKThFVrjmN4HHsHMN
2026-09-28 09:45:51 -05:00
momentum devandClaude Opus 5.5 efde7c721d fix: one invalid symbol can no longer stop all live bar ingestion
At 09:34 ET on 2026-09-28 Alpaca's movers screener returned 'TSX:STC', a
Toronto listing. The discovery filter only dropped symbols containing a dot,
so it joined the universe. The live ingest fetches bars for every symbol in
one batched request, and Alpaca rejects the WHOLE batch with 400 "invalid
symbol". From 09:35 every ingest pass stored 0 rows and the loop ran on
stale bars ("INGEST STALE: freshest bar 458s old").

- Discovery keeps only plain 1-5 letter US tickers from both screeners.
- The ingest never sends a non-US or invalid symbol. If Alpaca still rejects
  one ("invalid symbol: X"), it drops X and retries the rest (up to 3 times),
  so one bad name costs one symbol, not the session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3S5EFKThFVrjmN4HHsHMN
2026-09-28 08:48:51 -05:00
momentum devandClaude Opus 5.5 990ee89b19 chore: restart the live test on 2026-09-28 with the deployed fixes
The 2026-09-24 period ran on settings the 5372585 deploy changed: live volume
was inflated 80-160x, squeezes were verified on a mean volume baseline, and
the first ingest began at ~09:15. It also had no premarket scan and no
fail-fast exit (now 60s/0.3R). Trades from before the restart would mix two
systems, so the 80-trade count starts again on the first session under the
new build. The four earlier trades are kept in the freeze file for the record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3S5EFKThFVrjmN4HHsHMN
2026-09-27 19:46:50 -05:00
momentum devandClaude Opus 5.5 53725854f4 feat: fail-fast exit for breakouts that fail inside their first minute (off by default)
At the first exit-thread pass >= TRADING_EXIT_FAIL_FAST_SECONDS after the fill,
flatten if price has already traded TRADING_EXIT_FAIL_FAST_R x R below entry.
It is checked once per position. A position first seen after its window (a
restart or a carry) is never judged late. 0 seconds = off (the default).

Evidence, from tick data on 1,093 live-band v1 ORB breaks (Jun-Sep 2026) with
outcomes under the current exit rules:
  - 60 s / 0.3R: +0.060R per trade Jun-Jul (90% CI +0.015..+0.102) and +0.062R
    Aug-Sep (+0.029..+0.094). It fired on 29% of trades and cut 22 runners.
  - Every threshold from -0.15R to -0.4R at 60 s was positive in both halves,
    a plateau rather than one fitted cell.
  - On our own 194 real fills (10-s bars): +0.015R. Same sign, not yet
    significant.

It is deliberately narrow. The same study found that later "bail out below
+1R" rules kill runners: the LLM trade-manager pilot, the hybrid bailout, and
orderflow H1 (Jul 17). The first minute is the one window where failure tells
you something. Tape-flow features (aggressor buy share, print rate, big
prints, quote imbalance, spread) separated nothing (AUC 0.44-0.59, not stable
across halves).

Live-only: it needs sub-minute prices, so simulate_exit (minute bars) does not
model it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3S5EFKThFVrjmN4HHsHMN
2026-09-26 19:20:17 -05:00
momentum devandClaude Opus 5.5 3d0ee52214 Merge: nightly Ross-recap learning loop on local Ollama
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 16:24:02 -05:00
momentum devandClaude Opus 5.5 91af4c2e93 feat: nightly Ross-recap learning loop on the owner's local Ollama
Why: the pilot in data/ross showed that Ross Cameron's daily recaps, joined to
the SIP tape and to what our bot saw, answer "which of his trades could we have
taken and why didn't we". It was a set of one-off scripts run by hand; the
server-side youtube_claims pipeline has been dead since ~08-24 (YouTube blocks
the TrueNAS IP) and its captions fallback hid that. This makes the pilot a
nightly, local, cumulative loop without a paid LLM API.

youtube_claims/recap/ (python -m youtube_claims.recap.nightly):
- channel: yt-dlp flat listing of @DaytradeWarrior, m4a audio, faster-whisper
  small.en on CPU, pilot-format bundles (meta + our scanner list + transcript).
- extract: Ollama structured output (JSON schema) per EXTRACT_SPEC, chunked
  with overlap for long transcripts and merged. The verbatim rule is enforced in
  code: every quote must be found (token-level, one-word tolerance) and is
  snapped to the transcript's words; unquoted items, trades with no surviving
  quote, prices never spoken, and passed-on names never said are dropped.
- llm: endpoint pool over RECAP_OLLAMA_HOSTS (round-robin + failover, per-host
  model/ctx/think pins) since the owner runs several Ollama boxes.
- tape/join/ourside: ported join.py/ourside.py -- tickers are price-verified on
  the Alpaca SIP tape (plus an order-free check for his own named ticker), then
  paper (:5433) and live (:5434) event views are attached. Bars cached on disk.
- store + schema.sql: idempotent recap_* tables in the transcripts schema,
  upserted per video_id. report: the pilot's sections + "what changed since last
  run" against the previous day's metrics JSON.
- evaluate: scores any model/prompt against the 36 gold extractions.

Model choice (36 gold videos, final code path, P/R):
  gpt-oss:20b on the workstation GPU, two focused calls, think=low:
    trades P .84 R .70; tickers P .82 R .77; entry +-2% P .54 R .50;
    passed-on P .66 R .76; temperature 16/31; kind 30/36; ~49 s/video.
  tools:latest (4B, server) one call: trades P .81 R .72, tickers P .83 R .85,
    passed-on P .80 R .75, temperature 20/31, but it invents prices
    (entry P .42) and occasionally runs away (5-9 min). Prices drive the tape
    verification, so the more precise local model is the default.
  Iterations: prompt v2 + spoken-price rescale (699 -> 6.99) + split calls took
  gpt-oss trades F1 .74 -> .76-.78 and tickers .78 -> .79-.80; think=medium
  blew the token budget. qwen3-30b-a3b on mason only finished 5 videos (host
  saturated) -- inconclusive.

Also: youtube_claims/transcription.py::_captions_fallback now picks the
youtube-transcript-api shape by feature detection instead of retrying the
removed get_transcript on any error, and transcribe() reports both the whisper
and captions causes -- the IpBlocked error was being masked as AttributeError.

No trading code touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 16:23:18 -05:00
momentum devandClaude Opus 5.5 dff21bae29 Merge: whole-market premarket squeeze scanner (snapshot scan)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 12:13:11 -05:00
momentum devandClaude Opus 5.5 1cead3f29b feat: whole-market premarket squeeze scan via Alpaca snapshots
Discovery finds premarket squeezes far too late. Across 53 trades by a
professional small-cap momentum trader, our scanner had found only 4 of his
names before his entry, and 27 of his 28 entries were premarket (07:00-09:30
ET). KITT on 2026-09-25 ran $3.50 -> $8.39 at 09:00-09:01 ET (prior close
$0.49); the loop first "Discovered KITT via manual" at 09:35. The existing
sources (Alpaca movers / most-actives screeners) only reflect the regular
session, so a premarket squeeze is invisible until the open.

research/ingestion/snapshot_scan.py sweeps every active, tradable, listed US
equity (~11.9k names; OTC, warrants/units/rights and DISCOVER_EXCLUDE dropped;
asset list cached per ET day) through GET /v2/stocks/snapshots in 2,000-symbol
batches. The endpoint is bound by URL length rather than symbol count (8,000
worked; ~12.7k returned 414). A name qualifies when:
  - the LATEST TRADE price is inside WATCHER_PRICE_MIN/MAX (never the prior
    close, which is routinely < $1 on these names),
  - % change vs the prior close >= DISCOVER_SNAPSHOT_MIN_PCT (20),
  - it is trading now: last trade <= 180s old, latest minute bar <= 300s old
    with >= 5,000 shares, and >= 100,000 shares since 04:00 ET (summed from
    minute bars for the short list, one extra request).
Qualifiers are ranked by % change, capped per sweep, and announced once per
day. dailyBar OHLC ignores extended-hours prints (KITT's dailyBar high was
$3.24 against a premarket high of $8.39), so in premarket it can still be
yesterday's bar. The prior close is prevDailyBar.c only when dailyBar is dated
today, otherwise dailyBar.c.

run_live_paper.py runs it on its own daemon thread (04:00-16:00 ET weekdays,
every 45s), using the plan-019 pattern: its own DB connection, the thread-safe
Watchlist and the generic interval driver. Each new name emits
symbol_discovered with source="snapshot_scan" and price/prev_close/pct/minute
and cumulative volume in symbol_data, so first-sighting latency can be
measured. It gets a daily-history backfill and joins the watchlist, so the
normal new-symbol ingest path gives it its first full-session pull. This is
discovery only; no trading gate changes.

Gated by DISCOVER_SNAPSHOT_SCAN_ENABLED (default 1); knobs in .env.example.
Live read-only sweep at 13:10 ET: 11,871 names in 8 requests / 6.3s cold
(7 requests / 3.5s warm); top hits KITT +376%, MSGY +262%, INLF +60%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 12:11:45 -05:00
momentum devandClaude Opus 5.5 81e505b845 fix: verify squeezes on a median volume baseline and ingest from the premarket
KITT on 2026-09-25 (+557%, 9.3M shares, the day's biggest mover) was benched
as "gap 557% squeeze unverified (rvol 1.54)". Two separate faults:

1. The rvol baseline was the 20-day MEAN, and two 11-14M-share days on
   09-01/09-02 put it at 1.66M against a 333k median. Squeeze verification
   (gap >= 150% needs rvol >= 5) now takes the stronger of the mean-baseline
   reading and the median-baseline one. Both are cumulative volume over a
   full day's baseline, not prorated, so they share the units the rvol >= 5
   boundary was calibrated in. A reverse-split fake still reads ~0.01 on
   both. This applies in the watcher pre-filter, the trigger book's squeeze
   exception and the fast-path population gate. The ranking and the tod-rvol
   arm signal are unchanged.

2. A name's first ingest reached back only to ~09:15 (09:30 + 15 min, or
   the 20-minute refresh floor). KITT was found at 09:35, so its stored day
   began at 09:13 and missed the 09:00 squeeze and ~6M premarket shares.
   The first pass now covers the session from 04:00 ET.

With both fixed, KITT's volume at 09:35 is ~20x its median day.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 12:08:13 -05:00
momentum devandClaude Opus 5.5 7f1ebe9cab fix: read the data feed through the Tradier wrapper — live volume was 80-160x
With BROKER=tradier the loop's `client` is a TradierClient that delegates
market data to the Alpaca client in `data_client`. Two places read the feed
as getattr(client.settings, "feed", "iex"). TradierSettings has no feed, so
both read "iex" even though the container runs ALPACA_DATA_FEED=sip (verified
in the running loop and in config_overrides):

  - step_vratio kept computing IEX->SIP ratios as if on the free feed
  - ingest_live_minute_bars multiplied the real SIP volume by those ratios

Checked against the paper stack's SIP bars for the same minutes: prices are
identical and unscaled volume matches 1.00x, but scaled rows run x83 (GLND),
x117-126 (GRML) and x163 (SKYQ). The 2026-09-25 GRML signal logged a
cum_volume of 1.39 billion shares and an rvol of 64.6. For any name with a
ratio, every volume gate — rvol, the 300k day-volume floor, min dollar
volume, the 2.5%-of-volume liquidity cap, and the gap x rvol ranking — has
been reading inflated numbers since the stack went live on 2026-07-31.

data_feed() reads through data_client when present. The regression test
wraps a SIP client in a TradierClient and fails without the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 11:14:11 -05:00
momentum devandClaude Opus 5.5 e2e9687c45 feat: live-test scorecard and frozen strategy settings
Pre-registers the real-money test that starts 2026-09-24, the first session
without commission drag. PASS needs at least 80 round trips with a positive
mean R and positive P&L. FAIL is any session the equity floor halts. Changing
any strategy setting in live_test_freeze.json restarts the count, and the
scorecard reports that drift.

The settings were frozen, not tuned, because a study of 239 real fills
re-simulated under the current exit found nothing worth changing:
  - Capping how far above the trigger an entry may fill lost R at every cap
    from 0.25% to 10%, with either skip or resting-limit handling. Fills more
    than 5% above the trigger were the best bucket (+0.66R).
  - No break-time factor (spread, price, VWAP distance, stop width, L1
    bid/ask size, premarket range) kept its sign across both date halves and
    both accounts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 10:53:39 -05:00
momentum devandClaude Opus 5 42ca523053 fix: stop gating entries on minute-bar completeness
TRADING_MIN_QUALITY_SCORE 0.60 -> 0.20.

setup_evaluator.py:164 emits "quality_score": quality.score, where quality comes
from calculate_data_quality_score(bars). That module's own docstring reads "Data
quality scoring for minute-bar coverage and integrity" -- it measures whether the
bars are complete, not whether the setup is good. The name collision meant a
coverage metric has been gating live entries.

Tested against realised outcomes over 62 trades: r = 0.0845. Grade A averaged
+0.13%, grade B -0.71%, grade C -0.17% with an 80% win rate -- grade C beat grade
A. There is no signal in it. On 2026-09-14 it blocked SNDQ ("ready below quality
gate (score 0.56 < 0.60)") purely for gappy bars.

0.20 keeps it as a guard against genuinely broken data while removing it as a
setup filter. Renaming the field so the collision cannot recur is the proper fix
and is left for a follow-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-16 20:01:57 -05:00
momentum devandClaude Opus 5 f7bec45c32 chore: arm the equity-floor circuit breaker at $215
The 2026-09-13 exit retune is $7.50 behind the config it replaced across its
first 8 live trades. Its thesis -- surrender small frequent gains to keep a rare
large tail -- predicts exactly this early shape, and the backtest behind it was
234 trades across two stacks with train/test splits. But 8 losing live trades are
also what a wrong call looks like, and nothing in the data separates the two yet.

So pre-commit the exit rather than deciding it mid-drawdown. TRADING_EQUITY_FLOOR
halts the session and flattens the book at $215, about -11% from the 241.75 the
change started at. The check reads broker equity directly and returns early on a
0.0 reading, so it cannot trip on the overnight-maintenance nulls.

Owner decision 2026-09-16: keep the retune, on the view that the same day's
risk-equalisation fix (0.05 -> 0.015 per trade) removes most of what was actually
doing the damage -- the four widest-stop entries produced -$8.57 of the -$8.84
week, and those are the positions the sizing change shrinks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-16 14:58:19 -05:00
momentum devandClaude Opus 5 60e009f0c6 fix: equalise dollar risk per trade — the position cap was doing the sizing
TRADING_RISK_PER_TRADE_PCT 0.05 -> 0.015.

On a ~$240 account the 5% risk budget ($12) is unreachable: a 4.5% stop would
need $267 of notional against a $38 position cap (max_position_pct=0.16). So
calculate_position_size never bound -- the POSITION CAP did, on every trade.
The consequence is that every position was the same NOTIONAL (~$34 average) and
the dollar risk floated with stop width instead of being held constant:

    measured over 64 live trades: risk ranged $0.52 to $8.79 -- a 17x spread

That is the opposite of what risk-based sizing is for, and it concentrated the
damage exactly where the stops were widest. On 2026-09-15/16 the four widest-stop
entries (CANG 13.5%, FTFT 21.6%, ZTG 13.2%, VNCE 19.8%) produced -$8.57 of an
-$8.84 week. It also interacts badly with the -10% catastrophe floor: any stop
wider than 10% is effectively a 10% stop, so a wide-stop trade cannot be saved by
its own stop but still realises a full 10%+ loss on the same notional.

At 1.5% the risk calc binds for wide stops while the position cap still binds for
narrow ones, compressing the spread to 6.8x. Replaying the same 64 trades:
-$4.09 -> -$0.32.

Checked that this is risk EQUALISATION and not merely trading smaller: against a
plain proportional shrink to the same average notional, equalising is worth
+$3.27 at 1.25%, +$2.30 at 1.0%, +$2.45 at 0.75%. The benefit holds at every
matched size.

Below 1% trades start being skipped outright as shares round to zero (1 of 64
lost at 1.0%, 13 of 64 at 0.5%) -- a hidden cost that looks like improved P&L.
1.5% is the tightest setting that drops no trades. Expressed as a percentage
rather than TRADING_MAX_RISK_DOLLARS so it scales with the account.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-16 14:36:13 -05:00
momentum devandClaude Opus 5 11b0fcd7b7 fix: read the session open from the first regular-hours bar, not bars[0]
_population_reject computed the static open gap from bars["open"].iloc[0].
Premarket that is a PREMARKET bar, so the gap is fabricated. Live on
2026-09-14: FTFT opened 3.30 against a 2.88 previous close (+14.6%, well
inside the tradeable 10-35% band) but its premarket bars started near 2.52,
so the filter read -12.5% and benched it at 09:12 -- before the session had
even opened.

It never self-corrected. The frame keeps its premarket bars all session, so
every later pass recomputed the same wrong number, and _transition only emits
on change, so the symbol stayed out_of_population for the day. Several of the
day's rejections show negative gaps (-12.5%, -17.4%, -1.9%) that look like the
same artifact.

Trading was unaffected -- the trigger path does not consult watcher state, and
FTFT was still evaluated and rejected on its own merits (over_extended_day, it
was already +59% off the open). The damage is diagnostic: valid candidates lose
their criteria trail, which is the blank-reasoning-panel problem a previous
commit set out to fix.

Read the open off the first is_regular_hours bar; return None (fail open) when
the session has not opened, since the static gap is not yet knowable. A frame
with no session flag is treated as already regular-hours-only, which is what
tests and replay hand it.

Also drop the pm-range=N counter from step_arm's return string. The arm step
never emits a module_tick -- verified against both 2026-09-11 and 2026-09-14,
where only discovery, ingestion, enrichment and triggers do -- so it reached
container stdout and nothing else. Adding a tick purely to carry it would put
~4k events/day back after the population filter had just removed 88% of them.
pm_range_pct is already on the armed candidate rows for the board; persist it
on the approval payload if it needs to be measurable per trade.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-14 15:27:01 -05:00
momentum devandClaude Opus 5 4e0559501a fix: stop boosting names because a headline already reported the move
Three independent tests over 2026-08-01+, all negative:

  premarket news, any window      7.9% vs 8.6%  ran >=50%
  ...controlling for PM range    11.3% vs 21.0% ran >=50%
  intraday arrival, hour-matched 16.8% vs 24.6% ran >=10% forward

The feed explains it. Of 1,621 headlines delivered for screened names during
sessions: 61.7% reactive roundups ("Why X Shares Are Trading Higher", "20
Stocks Moving In..."), 2.7% law-firm spam, and 0.9% genuine catalyst PRs. The
wire reports the move after it happens, so a boost on "has a headline" buys the
late stage of a run. 11 of the 14 biggest runners (up to +912%) had no coverage
at all.

Set NEWS_RANK_BOOST to 1.0 (neutral). This also neutralises the binary fallback
restored in 9b4c8e5 -- correct in intent (a dead enrichment must not silently
disable news ranking) but it handed a flat 2x to any unclassified headline, and
62% of them are lagging roundups.

Deliberately unchanged:
  - the LLM path. catalyst_score = conviction * max(0, sentiment), so a roundup
    (type none, sentiment <= 0) scores 0 and earns nothing, while a real bullish
    catalyst still lifts. It targets the 0.9% correctly.
  - the dilution veto, which sinks confirmed offerings and is a separate,
    sound mechanism.
  - the enrichment-down alarm from 9b4c8e5.

Also evaluated and NOT shipped: halt-resumption as a catalyst feature. It is by
far the best signal in this universe -- a >=3min bar gap resuming >=2% up ran
>=25% afterwards 40-47% of the time vs ~11-20% baseline, holding in every hour
of the session. But a time-aware A/B of the armed set moved mean forward run
only 27.09% -> 27.84% and 165 -> 173 runners per 725 slots, because gap x rvol
already encodes it (a halting stock has high relative volume by definition).
Same collapse premarket range showed: strong univariate, near-zero marginal.
The ranking is saturated; selection is not where the remaining edge is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-14 07:31:37 -05:00
momentum devandClaude Opus 5 45b0386347 feat: rank gappers by premarket range, the one measured predictor
Over 1,040 screened symbol-days (2026-08-01 on), asking which premarket
property predicts the session running >=50% open-to-high:

    premarket range   n     avg run   ran >=50%
    >= 30%           201     34.7%      15.9%
    15-30%           136     28.3%      14.7%
     5-15%           303     15.2%       5.9%
      < 5%           400     16.8%       4.3%

A name that already travelled >=15% premarket is 3.1x more likely to run
>=50% (15.4% vs 5.0%) -- the largest single lift in this dataset, and free:
it comes off minute bars the arm pass has already fetched.

Two findings constrain the shape:

  - It is a STEP at 15%, not a gradient (15-30% and >=30% are 14.7 vs 15.9;
    5-15% and <5% are 5.9 vs 4.3). So the multiplier is a threshold. A linear
    scale would invent precision the data does not support.

  - Premarket VOLUME carries NO signal (10.0/4.2/7.9/9.0% across participation
    buckets, non-monotonic) and raw premarket volume runs inversely to the
    outcome. scanner_snapshots.premarket_volume being NULL costs nothing;
    range is the field that mattered all along.

Do not expect 3x. Replaying armed-set selection over the same 29 sessions,
top-5 by mean/median run and >=50% runners captured out of 145 slots:

    gap x rvol (baseline)        39.77% / 10.97%   27
    x3.0 if PM range >= 15%      42.00% / 13.16%   32
    HARD FILTER >= 15%           37.27% / 12.67%   24   <- worse
    PM range alone               32.86% / 12.65%   20   <- much worse

gap x rvol already encodes much of the same information, so the marginal gain
is ~+2 points of mean run and 4-5 extra big runners per 145 slots. It is
BOOST-only for a reason: gating on this feature scored below baseline, and it
must never replace gap/rvol. Every threshold in 10-20% and gain in 1.5-5.0 beat
baseline -- a flat plateau, so not fitted. The shipped 15%/3.0x are the measured
step and the measured lift, not the sweep's peak cell.

Fail-open: an unknown range yields a neutral 1.0, never a penalty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 19:16:17 -05:00
momentum devandClaude Opus 5 ff63b19e91 Merge: exit retune, news pipeline restore, sizing and watcher fixes
Five fixes from the 2026-09-13 live-account investigation:

  aae7269  exits stopped amputating winners (9.3% of the move captured on
           trades that ran >=20%; winners held 10 min vs 19 for losers)
  9b4c8e5  news ranking had been off since 2026-08-03, two days before the
           first real-money fill -- and could not be seen to be off
  0910e29  track the live overrides seed that LIVE-CUTOVER.md references
  9034839  a zero equity reading sized off the $100k default (latent)
  97a3a1b  watcher stops scoring names the entry path can never accept
           (3.1 GB of a 7.6 GB events table)

Config changes are already applied to config_overrides on the live stack.
This merge is what ships the CODE half to the image.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 15:37:12 -05:00
momentum devandClaude Opus 5 97a3a1b29d perf: stop the watcher scoring names that can never trade
criteria_evaluated + signal_blocked held 3.1 GB of a 7.6 GB events table.
On 2026-09-10 alone the watcher wrote 191,458 criteria receipts across 57
symbols -- ~3,359 per symbol per day -- and 154,260 of them (80.6%) were
for names whose static open gap was outside the tradeable band. NOK, NIO,
PATH, PLUG, SOFI, AAL and F all sit inside the $2-20 price filter, so
WATCHER_PRICE_MIN/MAX admits them; only the gap floor excludes them, and
that was checked per-tick instead of once up front.

Apply the SHARED population gate (strategy/population.population_reject --
the same function, same config the entry path already enforces) before the
evaluation and its receipts. This is behaviour-preserving for trading: those
names were being scored, emitted, and then rejected downstream anyway --
429 "outside tradeable population" risk events across 221 symbols, led by
exactly the names that top the receipt counts. The decision just moves
upstream of the write.

Details that matter:

  - The verdict is recomputed each tick, never cached. A >150% gapper reads
    "squeeze unverified" on thin volume and is admitted once rvol confirms;
    a cached rejection would bench it for the session.
  - _transition only emits on CHANGE, so a rejected symbol costs one
    state event per session and is then silent.
  - Fail-open throughout: an unreadable open, a missing previous close or
    absent volume returns None, so a data gap can never bench a symbol.
  - Gap is the STATIC OPEN gap (session open vs previous close), fixed for
    the day, not the live price -- the verdict must not wander intraday.

Off by default (population=None) so every existing caller and test keeps
its current behaviour; the live loop opts in, and WATCHER_POPULATION_GATE=0
restores the old path.

Note for diagnostics: momentum_cli's criteria views now show only names
inside the population. That is the intended trade -- the excluded rows
described symbols the bot could never have taken.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 14:27:06 -05:00
momentum devandClaude Opus 5 9034839661 fix: a zero equity reading sized off the $100k default
Tradier returns total_equity 0.0 during its overnight maintenance window --
sixteen times in the last fourteen days, all around 01:06. Three defects
compounded so that a zero could size a position off default_equity:

  account_equity = equity or config.default_equity

`or` cannot tell "not supplied" from "read as zero", so a real 0.0 became
100000. Then the position-value cap, derived from the same equity, was
itself 0 -- and the guard

  if max_position_value is not None and max_position_value > 0:

SKIPPED the cap precisely when the cap said "no room". Finally the auto
path (~96% of approval requests) had no `pos_cap <= 0` early-out, unlike
the manual path at request_entry_approval. Net effect on a ~$250 account:
risk 5% of 100k = $5,000, capped only by liquidity. The broker would have
rejected it for buying power, which is luck rather than design.

Never fired in live hours -- every zero read landed with the market closed,
and _current_equity caches the last good value -- so this is latent, not a
realised loss. The reachable path is a boot that coincides with the
maintenance window, when no good read has been cached yet.

Distinguish None from zero, refuse to size on non-positive equity, treat a
zero cap as "no room" rather than "no cap", and mirror the manual path's
early-out. TRADING_DEFAULT_EQUITY is set to 300 on the live stack so even
the untested fallback cannot oversize.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 13:34:53 -05:00
momentum devandClaude Opus 5 0910e29cc5 chore: track the live overrides seed
docs/LIVE-CUTOVER.md already points at this file, so the cutover procedure
referenced something that existed only on one machine. It carries no
secrets -- keys stay in .env (gitignored) and the LAN address it names is
already in .env.example, the compose YAMLs and the systemd unit.

Tracking it also pins the two settings this branch corrects, with the
reasoning inline: the exit retune, and the Ollama endpoint that had drifted
to a port with nothing listening. Untracked, a rebuild would have quietly
restored both faults.

Note the precedence: compose env < overrides.env < config_overrides (DB).
The database wins, so editing this file alone changes nothing on a running
stack -- it is the bootstrap seed and the record of intent, not the live
switch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 13:23:39 -05:00
momentum devandClaude Opus 5 9b4c8e5da5 fix: news ranking silently switched itself off for 40 days
step_arm ranks gappers by catalyst, but the enrichment feeding it stopped
on 2026-08-03 -- two days before the first real-money fill. raw_news_items
kept ingesting ~45k items/day throughout; news_events and
news_catalyst_cache have no rows after 2026-08-03. So every live trade to
date was ranked with the news multiplier pinned at 1.0.

Three faults, the third of which hid the other two:

1. Ollama was configured at a host/port with nothing listening, running a
   model that was never pulled.

2. _cat_mult returned a flat 1.0 for a symbol with no LLM advisory instead
   of falling back to the binary headline boost. With the scorer enabled
   and the cache empty that branch is reached for EVERY symbol, so turning
   on the graded ranker silently turned off the simple one it was meant to
   improve. Fall back properly, so news ranking degrades instead of
   disappearing.

3. enrich_recent_news never raises -- it reports failures in a returned
   counts dict, and enrich_loop discarded that return value. Every 120s
   for 40 days it received {enriched: 0, errors: 50} and threw it away: no
   log line, no event, nothing to distinguish a dead pipeline from an idle
   one. Consume the counts and emit a module_tick naming host and model,
   on the first failed pass and hourly thereafter.

Fault 3 is the one worth fixing; 1 and 2 are its first casualties.

No claim that news ranking pays: a binary "has a headline" flag tests
mildly NEGATIVE across 1,030 screened symbol-days (7.8% vs 10.4% ran
>=50%), and the graded classifier only overlaps bar data for five days,
too few to separate anything. Restoring the pipeline makes it measurable;
it does not make it trusted. Re-measure catalyst type and sentiment
against realised runs before giving the multiplier more authority.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 12:00:40 -05:00
momentum devandClaude Opus 5 aae7269538 fix: stop the exit stack amputating every winner
The live exit ran FOUR independent stop ratchets at once: a +5% breakeven
inherited silently from the env default, a 0.25R step-trail, a prior-low
trail, and profit-lock tiers. Stacked, they pinned the stop at or just
above entry within the first minute of any real move, so winners were
capped near +1R while losers still ran to the full stop.

Measured over 62 live Tradier + 172 Alpaca paper round trips:

  - trades whose stock ran >=20% after entry (avg +63.7%) kept 9.3% of it
  - median hold 10 min for winners vs 19 min for losers -- inverted
  - PCLA 2026-09-11: entered 12.8233, stopped 12.82, a 0.03% stop

breakeven_at_pct was the only exit lever in ExitConfig that defaulted ON
from the env (0.05) while its dataclass default and every sibling lever
defaulted to off, so the live stack ran it without ever setting it. Turn
the env default off so "not configured" means "not applied".

Replayed through simulate_exit -- same entries, same initial stops, only
the management varying -- with each stack split 70/30 by date. A 10%
trail running from entry looked best in-sample and went negative out-of-
sample; what survives is a trail armed only after +1R. The 8-20% range
all beat the old config in all four train/test cells (mean R 0.20-0.25 vs
0.125), so the live value of 12% is a plateau pick rather than a fitted
one.

Live config_overrides updated separately (market closed, no open
positions); this commit is the code default and the regression guard.
Risk per trade is unchanged and both safety backstops -- the -10%
catastrophe exit and naked-stop enforcement -- are untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
2026-09-13 12:00:29 -05:00
momentum devandClaude Fable 5 40e51c8dd5 feat: morning pick journal — prospective human/LLM forecast experiment
Tests whether a PROSPECTIVE forecaster (owner or local LLM) carries signal
the machine levers never found. Picks land premarket via dashboard panel or
research/llm_picker.py (Ollama); research/pick_scoring.py judges them under
a pre-registered bar (n>=40, >=30 sessions, mean >=+0.5%/pick net, median>0,
both halves positive). Picks after 9:30 ET stored but flagged invalid.
Shadow-only: human_picks is never read by the live loop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 10:52:39 -05:00
momentum devandClaude Fable 5 390d6660ea fix: halt-aware flatten — let a working market sell rest instead of churning
A market sell that sits unfilled means the symbol is halted (or session
closed); cancelling and resubmitting it every exit-manager pass fills no
faster and spams the broker — YXT 2026-08-05: 87 market sells across a
10-minute LULD halt, 86 cancelled, plus one exit_naked_stop journal event
every ~4s. On a halt-down the resting sell is also the position's only
protection.

cancel_protective_and_close now returns False (nothing submitted) when a
market sell for the symbol is already working, and only cancels
non-market protective sells before closing. Exit-manager paths
(catastrophe / naked-stop / first-red / trail) journal the exit only when
a close was actually submitted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 19:01:18 -05:00
momentum devandClaude Fable 5 3871bda616 fix: label broker sync events by actual execution broker (tradier stack)
Tradier stack emitted account/positions/orders events hardcoded as
broker_name=alpaca_paper while the executor filters snapshots by
client.broker_name (tradier) — position cap and re-entry block read zero
positions, and the dashboard showed 'Alpaca Paper' over real Tradier values.
Mode label now defaults from BROKER/TRADIER_ENV (tradier_live) instead of
alpaca_paper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 10:52:01 -05:00
momentum devandClaude Fable 5 485cec0372 feat: wide_stop_veto_v1 — skip entries with stops >12% below entry (shadow)
Year-replay stop-width buckets: <8% +0.072R, 8-12% +0.120R, 12-16%
-0.054R, >16% -0.092R — the wide cohort holds every recent dollar
disaster (BIYA 16.7%, FCUV day-2 17.5%), where the exit ladder's blind
spot (BE/trail only arm on the way up) rides the full catastrophe
distance. New stopwidth_gate intervention (fail-open without levels) +
registered spec, same forward bar as the pm-high gate; the two cohorts
overlap so the adjudicator attributes which does the work. In-sample
counts for nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 21:06:37 -05:00
momentum devandClaude Fable 5 9390f3c2a6 perf: gate-receipts and loop-session projections stop shipping fat rows
/api/snapshots ran 33-60s even after retention pruned the events table —
profiling each projection found query_gate_receipts (21s) and
query_loop_session (10s) fetching thousands of full payload rows to
Python to keep 100 (or ONE). Both now select newest-first with SQL
LIMIT / single-column reads: 0.12s and 0.07s against prod. Remaining
projections (watch_states 6s, daily_performance 5s, approval_queue 3.5s)
are the same class — later pass if the snapshot still feels slow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 22:33:01 -05:00
momentum devandClaude Fable 5 f0fa104511 feat: cap entry notional at broker buying power (cash-account GFV guard)
On the Tradier live cash account buying_power maps to cash_available —
SETTLED cash. Capping every entry's notional at it means the bot never
spends today's sale proceeds today, which is the entire good-faith
-violation rule. That replaces the blunt 1-trade/day cap: the live
account now trades like paper (cap back to 5/day), throughput
self-limits at ~settled/position-size sequential trades and refreshes
each morning at T+1. On margin accounts buying power is ~4x equity so
the cap never binds — one universal rule, no broker branches. Fail-open
when the field is absent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 09:21:55 -05:00
momentum devandClaude Fable 5 c7249b22e2 feat: orb_pmhigh_gate_v1 — premarket-high reclaim gate, forward OOS test
H3 (enter the pm-high break at 9:30 instead of the ORB) FAILED its
pre-registered bar on the year replay: paired delta -0.069R (n=689) —
entering earlier on the same names does not pay; the 5-minute wait stays.

The decomposition found the real signal: ORB fires where RTH price also
reclaimed the premarket high ran +0.316R (n=689) vs -0.298R (n=477) where
it never did — 100th percentile of a 2000-draw random-gate null at the
same keep-rate (null p99 +0.127), sign-consistent 14/14 months. First
lever of 15 to survive both checks in-sample.

Because the split was found in this dataset, it counts for nothing until
fresh fires confirm: new 'pmhigh_gate' intervention kind (fail-open on a
missing premarket_structure row, RTH-only reclaim check at/before the
entry bar) + registered spec with expR_delta >= 0.05 at n >= 200 forward
bar. Cohort self-registers on the next nightly experiments run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:20:25 -05:00
momentum devandClaude Fable 5 3fe0cb0891 feat: H3 replay — premarket-high break at 9:30 vs ORB, paired comparison
The pre-registered premarket hypothesis H3 (the source trader's actual
entry) made executable: both legs run per symbol-day over the year of
labeled gapper sessions, sharing the deployed exit engine (config parity)
and +1-tick adverse fills so the paired delta isolates ENTRY choice.

Comparison metric locked in the module docstring before the first run:
paired mean R delta (H3-ORB) where both fire, pass at >= +0.05R with
n >= 300 pairs, plus a stop-unit guard (median stop-distance % and
captured-move % printed for both legs — R parity from a tighter stop is
a fail, the vwap-reclaim lesson).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:13:57 -05:00
momentum devandClaude Fable 5 37757d3d47 fix: armed trigger names now get watcher scoring (blank reasoning panels)
Four of ten armed triggers had NO criteria events ever — the trigger book
arms from the live gapper scan while the watcher scores only the
discovery watchlist, so scanner-only names (TCX/ZTG class) showed an
empty reasoning panel with nothing behind it.

- arm pass injects every complete armed candidate into the watchlist
  provider's extra_symbols (its standing manual-add hook)
- provider returns injected symbols FIRST: the watcher truncates at
  max_symbols, so extras appended after a full research list were
  silently dropped — 'always include manually injected symbols' was
  untrue exactly when the list was full

60 watcher/watchlist/trigger tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:41:11 -05:00
momentum devandClaude Fable 5 72a9ecbd0b fix: reasoning panel flicker + stuck loading on live ticks
Every SSE snapshot re-render rebuilt the expanded panel's HTML back to
'loading…' and re-fetched — panels flickered per tick and never looked
settled. Rendered panels now cache per symbol: re-renders repaint from
cache instantly, background refresh at most once/min, single-flight
guard against fetch races. Watch panel and trigger panel share the
cache; toggleCriteria's duplicate direct fetch removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 20:36:31 -05:00
momentum devandClaude Fable 5 a6be643fbb ci: build image on dashboard static changes too
api/static/** ships inside the image but wasn't in the docker workflow's
paths filter — dashboard-only commits (a6da02f trigger-row reasoning,
81bf154, da17be8) silently produced no image and only reached production
by riding along with later .py commits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 20:20:46 -05:00
momentum devandClaude Fable 5 a6da02f809 feat: reasoning timeline on armed-trigger rows (click to expand)
The day's relevant names live in ARMED TRIGGERS — the reasoning panel now
opens right there: click a trigger row for the score/rvol timelines,
criteria lanes, and worded reasons, same renderer as the watch panel.
Caret + hover affordance; open panels survive snapshot re-renders.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 20:11:02 -05:00
momentum devandClaude Fable 5 2be23673be feat: reasoning-over-time drill-down — score/rvol timelines + criteria lanes + worded reasons
Clicking a watch-state row now shows HOW the verdict evolved, not just the
latest flags:
- setup score across the session vs the 60% gate (SVG line)
- relative volume vs its 2.5x floor
- per-criterion pass/fail lanes over the whole session (the variance of
  each logical step at a glance)
- each criterion's latest reason in words (rvol 4.9 (min 2.5), gap 80.9%,
  price vs breakout ...) instead of bare pass/fail marks

Backend: query_symbol_criteria gains history[] (server-side strided to
~240 samples) + detail[]. The generic LIKE scan took 12s/click over weeks
of criteria rows; now a partial expression index on the extracted symbol
(idx_events_criteria_symbol, built at boot) + 24h window + SQL-side
stride = <1s. Latest-verdict fallback keeps stale symbols and historical
views working (covered by existing projection tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 19:37:35 -05:00
momentum devandClaude Fable 5 0c28cf921f feat: one-click source links on catalyst feed headlines
Every LLM catalyst read's headline now links to the raw article
(raw_news_items.raw_url joined by headline), styled as a distinct dotted
underline per NN/g citation guidance — any AI verdict traces to its
source in one click. Rows without a matching raw item degrade to plain
text. Join verified against prod: 0.29s for the full feed query.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:40:49 -05:00
momentum devandClaude Fable 5 81bf1540ef feat: explicit + prefix on positive P&L values (CVD accessibility)
Gain/loss no longer rides on green/red alone — red-green color vision
deficiency affects ~8% of males; the minus sign already printed, now the
plus does too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:36:41 -05:00
momentum devandClaude Fable 5 da17be8969 feat: dashboard readability pass — type hierarchy, P&L masthead, trigger-row cleanup
- Two-texture typography: labels move to the UI sans (11px, brighter muted
  #93a0bd), values stay tabular mono — label/value pairs parse instead of
  dissolving into uniform mono soup
- TOTAL P&L is the one dominant instrument: 34px + green/red state wash and
  accent bar on the cell (CSS :has on the existing pos/neg classes, no JS)
- Equity tile: available funds as a small sub-line (fixes the two-line wrap
  that broke the strip height)
- Armed triggers: single-line ellipsized meta/levels (was a 3-line crumb
  wrap), 15px symbols, ARMED/WEAK state chips, subtle zebra, thinner bars,
  wider levels cell so entry/stop/now never clip
- Verified with headless before/after renders against live data; zero JS
  logic changes, node --check clean

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:19:16 -05:00
momentum devandClaude Fable 5 47a64688d8 fix: replay universe = scanner-discovered + historical gappers; add year-scale targeted backfill
eod_replay's universe was today's top-100 by ABSOLUTE volume — small-cap
gappers never crack that list, so 2026-07-28 replay found 1 gapper on a
day the live loop signaled 8 names. Universe is now the union of (a) the
session's symbol_discovered events, (b) historical_gappers for that date,
(c) most-actives only on a same-day replay (past-date most-actives would
be look-ahead). Verified: 7/28 replay now screens 6 gappers / 4 trades,
matching the live day. Historical sessions also fetch that day's
04:00-20:00 ET SIP bars explicitly instead of a from-now lookback window
that pulled the wrong day.

backfill_gapper_days.py: for each of the ~260 recorded gapper sessions,
pull SIP minute bars (premarket incl.) + 40d of daily bars for that day's
gappers only, then build premarket_structure. Idempotent (skips sessions
already covered). Fattens n for premarket H1/H2, pyramid_1r_v1, and the
orderflow experiments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 13:20:38 -05:00
momentum devandClaude Fable 5 054405dbdf feat: pyramid_1r_v1 experiment — add-into-strength, pre-registered
New 'pyramid' intervention kind in the replay engine: at +add_at_r on
initial risk, add add_frac x size (+adverse fill), whole-position stop to
blended breakeven; scratch there if a later bar trades through, else both
legs exit at the baseline exit. variant_r is per unit DEPLOYED
(P&L / ((1+add_frac) x risk)) so leverage alone cannot fake a pass.

Spec pyramid_1r_v1: add 0.5x at +1R, bar expR_delta >= +0.05 at n >= 300,
futility kill at n >= 500 below -0.02. Hypothesis notes the known failure
mode (median winners scratched at blended BE on first pullback). Shadow
only — no live-path changes; cohort self-registers on the next nightly
experiments run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 11:33:59 -05:00
momentum devandClaude Fable 5 733a182f10 feat: premarket-structure labeler (Phase 1 of premarket axis, observe-only)
Distills each gapper's premarket tape (first-hit price/time, pm high/low,
run %, open-vs-pm-high, VWAP distance, jackknife count) into a
premarket_structure table for entry-conversion analysis. Motivated by
FCUV 2026-07-31: our DB captured the full $1.98->$10.49 premarket ramp
while the ORB entered at $11.89 into the reversal.

Pre-registered hypotheses H1-H3 locked in the module docstring before any
data was examined; report command prints descriptive buckets only, with
n>=30/bucket floor and Phase-2 shadow bar required for promotion. Gates
nothing, trades nothing.

Wired into the nightly scheduler after premarket_eval. One session scan
per build (per-symbol queries were ~100x slower); partial session index
created post-close only (mid-market CONCURRENTLY build deadlocked against
live ingest).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 11:10:16 -05:00
momentum devandClaude Fable 5 3d6bc36955 feat: Equity/Avail funds tile in dashboard P&L strip
Shows total_equity and buying_power (settled available cash) from the
latest account snapshot. On the Tradier live cash account these diverge
for days after a deposit or sale (T+1) — the bot sizes off equity but
buys need settled cash, so the gap explains rejected entries at a glance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 13:44:49 -05:00
momentum devandClaude Fable 5 3a28b0f6a7 docs: add regime-gate flip requirement to Tradier live runbook
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 10:17:56 -05:00
momentum devandClaude Fable 5 4f1ccde370 docs: Tradier real-money flip runbook (00 cash-account delta)
Delta to LIVE-CUTOVER.md (Alpaca-era): 3-line env flip on the existing
tradier stack, cash-account GFV cap (1 entry/day), equity floor 270,
sandbox-record truncation before first live boot, day-1 naked-stop drill.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 10:17:33 -05:00
momentum devandClaude Fable 5 12f7fefec0 feat: nightly event retention + external telemetry push
- runtime/retention.py: prune 8 high-volume debug event types older than
  RETENTION_DAYS (default 14) once/day at RETENTION_HOUR (default 03:00).
  Audit events (orders, fills, closes, risk rules) are never pruned. The
  events table hit 6 GB in 3 weeks at ~340k rows/day; this caps it.
- runtime/telemetry_push.py: POST a JSON snapshot (equity, day realized
  P&L, trades/wins, open positions, last-event age) to TELEMETRY_PUSH_URL
  every TELEMETRY_PUSH_INTERVAL_SECONDS (default 300). Unset URL = step
  disabled. Push failures are reported in the step result, never raised —
  an external outage can't touch the loop. Works as healthchecks.io
  dead-man ping + payload log in one.
- wired both as scheduler steps in run_live_paper (retention gated by
  RETENTION_ENABLED, telemetry by TELEMETRY_PUSH_URL presence)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 14:25:09 -05:00
momentum devandClaude Fable 5 c5b43364be fix: don't fabricate last_equity for Tradier — realized P&L read $0 forever
Dashboard realized = broker equity delta (equity - last_equity). Tradier
balances carry no prior-session-close field and translate_balances echoed
current equity, making the delta permanently zero (win rate populated from
closed events while dollars showed $0). Absent last_equity now routes the
projection to its closed-events fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 09:39:48 -05:00
momentum devandClaude Fable 5 80422d91cd fix: bracket fills stay on legs; by-tag lookup surfaces entry-leg fill
General translation must not echo leg fills onto the OTOCO parent —
flatten_fills adds parent AND legs, double-counting entries in journal
reconstruction. The cancel-vs-fill reconcile still sees executed shares:
get_order_by_client_order_id surfaces the entry leg's fill on its result.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 09:36:52 -05:00
momentum devandClaude Fable 5 a2de50479a fix: three Tradier sandbox incident fixes (2026-07-28 reject loop)
Incident: entry-timeout cancels raced lazy sandbox fills -> untracked
positions; naked-stop guard then force-sold but every sell rejected
('more shares than your current long position') for hours.

1. translate_order: Tradier OTOCO/OTO parents carry no top-level
   symbol/side/qty (legs only) — inherit from the entry leg so
   symbol-scoped lookups can see brackets; get_orders symbols filter
   also matches by leg symbols. Without this the stop-leg search read
   a fully-protected position as naked.
2. close_position: cancel the symbol's resting sell orders/legs first —
   Tradier reserves held shares against open sells, so selling without
   releasing them always rejects.
3. cancel_entry (broker-agnostic): after cancelling, re-check the
   broker's final word by client_order_id; journal a fill for any
   executed shares instead of orphaning a real position (2026-07-06
   class, same race exists on Alpaca).

49 tests pass incl. new coverage for all three.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 09:11:00 -05:00
momentum devandClaude Fable 5 bd7183c861 fix: mark Tradier positions off Alpaca real-time trades, not delayed quotes
Exit manager reads position current_price for breakeven/trail ratchets and
the stop-breach guard; Tradier sandbox quotes are 15-min delayed, so marks
lagged reality. Prefer the Alpaca data client's latest trades (real-time),
fall back to Tradier /markets/quotes per missing symbol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 17:39:46 -05:00
momentum devandClaude Fable 5 8c62804808 feat: broker-aware factory for reconcile/journal/health tools
eod_reconcile, backfill_journal and the admin health block were hardwired
to AlpacaPaperClient — on a BROKER=tradier stack they would silently
reconcile/report against the wrong broker. make_trading_client() honors
BROKER env with no silent cross-broker fallback (missing Tradier keys =>
skip with reason, never fall back to Alpaca). Admin UI shows the active
broker label; JSON key stays 'alpaca' for compatibility.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 13:13:01 -05:00
momentum devandClaude Fable 5 13c4b61e77 docs: tradier stack fresh-DB gotchas (missing research tables, knob seeding)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 13:04:50 -05:00
momentum devandClaude Fable 5 52144241b4 fix: dashboard broker badge reflects BROKER env (PAPER · TRADIER etc.)
Badge text was hardcoded to ALPACA; _broker_block now reports the active
broker name and derives paper/live from the Tradier sandbox host when
BROKER=tradier, keeping the base-URL-truth rule from plan 007.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 12:38:37 -05:00
momentum devandClaude Fable 5 154e44e8d5 fix: tradier stack compose must pin ALPACA_DATA_FEED=sip
Fresh DB has no config_overrides row, so without the compose-level pin the
parallel stack falls back to the thin IEX feed and its rvol/volume decisions
diverge from the main stack (which carries feed=sip in the DB).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 12:12:14 -05:00
momentum devandClaude Fable 5 df85201c45 feat: Tradier parallel-stack compose template
Self-contained shakeout stack beside the paper stack: own postgres (host
5434, disposable pgdata-tradier volume), own dashboard (host 8768), loop
with BROKER=tradier (sandbox). Data plane stays Alpaca; Telegram/deadman
disabled so main-stack alerts stay unambiguous; main data dir mounted
read-only (inherits learned_params/overrides.env). Filled local copy is
git-excluded; bootstrap_live_db.sh playbook reused for DB parity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 08:07:32 -05:00
momentum devandClaude Fable 5 1cefb51c63 feat: Tradier broker mirror — config-switch redundancy for Alpaca execution
tradier_broker/ mirrors every trading call the live loop makes on
AlpacaPaperClient (account, clock, positions, orders, submit incl.
bracket->OTOCO, cancel, replace, close) with identical signatures and
Alpaca-shaped returns, so trading_execution / exit_manager / flatten /
execution / sync run unchanged. Market data always delegates to Alpaca.

- BROKER=tradier (+ TRADIER_* env) swaps execution client in run_live_paper;
  default path untouched
- client_order_id -> tag preserves the lost-submit reconcile
- TradierApiError subclasses AlpacaApiError so the executor's
  rejection-vs-transport fork works as-is
- sandbox-verified: balances/clock/positions/orders, OTOCO preview
  (result:true), place -> tag lookup -> cancel round trip
- 18 unit tests (translation + client paths); executor integration checked
- docs/TRADIER-FALLBACK.md switch runbook

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 11:01:50 -05:00
momentum devandClaude Fable 5 48d75816b6 feat: backtest page — live-taken vs not-taken split + per-trade live P&L column
/api/backtest now annotates each replay trade with the live loop's realized
P&L for the same (session, symbol) and returns a live_split summary (taken vs
not-taken n/sumR/avgR). Page grows two summary cells, a Live P&L column with
an amber ≠ marker when sim and live disagree on a trade's sign, and a footer
caveat: minute-bar granularity + adverse fills flip individual rows — trust
aggregates, not rows. Read-only annotation; cached replay result unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 13:58:24 -05:00
momentum devandClaude Fable 5 6c6d0cd799 feat: weekly execution tallies — midday-entry net + market_exit post-run
Read-only nightly-scheduler job (Friday-gated): buckets realized P&L by entry
time (open/late-morning/midday) and, for each market_exit close, computes the
post-exit run-up capturable under a breakeven stop (0 if price undercuts entry
first; intrabar ties count as stopped). Emits a weekly_tally event + Telegram
summary. Evidence accumulator for the dead no_new_entries_after cutoff and the
weakness-exit winner-clipping question — no live trading path touched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 22:25:14 -05:00
momentum devandClaude Fable 5 30c7232137 fix: market_context must not import runtime (research-isolation guard)
The DB-config wiring was unnecessary there anyway — market_context reads
zero trading knobs; the guard test (plan 009) rightly rejected the import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:36:29 -05:00
momentum devandClaude Fable 5 82273d9d4e feat: image version identity + DB-config parity for nightly scripts
Version: CI now tags images YYYY.MM.DD-<shortsha> alongside :latest/:sha
and passes GIT_SHA/BUILD_TAG build-args; the running app reports them via
dashboard /api/health and admin /api/status (TrueNAS custom apps only show
'custom/1.0.0', so the app self-reports what it actually runs).

Config parity: every scheduler-run script (nightly_tune, eod_replay,
premarket_eval, regime, market_context, experiments cli) now loads
config_overrides at startup like the live services do — a bare env would
replay under DEFAULT exits (the documented 4.5x-baseline landmine).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:28:06 -05:00
momentum devandClaude Fable 5 537e0c7e12 docs: cutover config model — knobs in config_overrides table, seed-from-env migration
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 20:31:33 -05:00
momentum devandClaude Fable 5 495a3fa5d6 feat: strategy knobs in the database — config_overrides table, admin-managed
Owner preference: the bulk of TRADING_*/TRIGGER_* logic switches should live
in the DB, not the compose file (keys + connection info stay in compose —
the DB address cannot live inside the DB, and secrets don't belong in a
table the stack can SELECT). Precedence at boot: compose env < overrides.env
< config_overrides table; every entrypoint already routes through
env_overrides.load(), which now applies the table last. Admin /api/config
and profile-apply write the table (secrets whitelist unchanged); the config
page shows source=database. DB down = silent fall-through to file/compose,
boot never blocks on config. One-shot migration:
  python -m runtime.env_overrides seed-from-env
run in any stack container to copy its current effective knobs into the
table, after which the compose lines are dead weight strippable at the next
redeploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 20:31:16 -05:00
momentum devandClaude Fable 5 108f61a9bc docs: config model — strategy knobs in data/overrides.env via admin panel, compose keeps infra+secrets only
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 20:21:45 -05:00
momentum devandClaude Fable 5 de1973fbff fix: telemetry heartbeat insert needs an explicit uuid id (NOT NULL, no default)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:36:19 -05:00
momentum devandClaude Fable 5 625abee66f fix: tuner heartbeat also recorded in DB — file-only heartbeat was unverifiable
The 07-22 'tuner NOT running' alert could not be diagnosed remotely: the
heartbeat is a file on the scheduler container's volume, invisible to
everything else, and indistinguishable from a stale scheduler image or a
DATA_DIR mismatch. _write_heartbeat now also inserts a telemetry_events
row (best-effort); the boot staleness check falls back to the DB row when
the file is absent, so 'tuner ran, no profitable config' stops looking
like 'tuner dead'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:35:50 -05:00
momentum devandClaude Fable 5 148e300af4 docs: live-money cutover runbook + one-command DB seed script
docs/LIVE-CUTOVER.md: full flip-day procedure — staged pieces, discovered
dependencies (second paper key pair, DB seeding, learned_params copy,
telegram token, ports/dirs), 15-min cutover with the paper stack running
throughout, verification list, rollback. bootstrap_live_db.sh seeds the
live stack's Postgres from the paper nightly dump and truncates the
money-record tables so research history carries over while live P&L
starts at zero.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 17:29:06 -05:00
momentum devandClaude Fable 5 5287c8ebfe feat: live-money prep — equity-floor kill switch + boot order reconciliation
TRADING_EQUITY_FLOOR (0=off): equity at/below an absolute dollar floor
permanently halts the session and flattens the book — the total-drawdown
backstop the per-day breaker can't provide (a slow bleed of -2.9% days
never trips it). Same flatten path, loud risk event, bounce does not
re-open.

TRADING_BOOT_RECONCILE (default on): at boot, diff the broker's open
orders against today's submitted events — an open BUY on an unheld symbol
this process never submitted is a crash-restart orphan (the 2026-07-06
duplicate-order incident) and is cancelled. Orders on held symbols are
never touched (cancelling a bracket parent strips its protective stop).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:13:26 -05:00
momentum devandClaude Fable 5 e9a1f577a9 feat: research.slippage — paper-vs-real fill variance from recorded NBBO
Prices each closed round-trip against the nearest quote_snapshots NBBO:
per-trade spread cost, book-walk flag (order size > displayed ask), and a
cost-adjusted P&L total. Measured this week: 0.3-0.5% median spreads on
gate-passing names, ~12% profit drag — far below the static 2-4% microcap
proxy. Coverage gaps (quote recorder only follows tracked names) reported
per-trade, not silently dropped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:04:47 -05:00
momentum devandClaude Fable 5 21b38109aa test: clamp test pins BACKTEST_MAX_SESSIONS env (default moved to 300)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 09:30:14 -05:00
momentum devandClaude Fable 5 9d0a9df60e backtest: raise session clamp 45 -> 300 (env BACKTEST_MAX_SESSIONS)
The stored minute-bar history is 273 sessions (~13 months); the 45-session
clamp blocked months-scale ranges now that the nightly labeler keeps
setup_events current. Requests stay serialized behind _backtest_lock; a
full-history run takes minutes by operator choice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 09:29:55 -05:00
momentum devandClaude Fable 5 ed176b3712 fix: backtest starvation — schedule the labeler nightly, make inserts idempotent
Dashboard backtest sources sessions from setup_events v1, which only the
labeler writes — and the labeler was never in the nightly scheduler, so
setups (and therefore backtests) froze at the last manual run (07-17).
Root blocker was _insert crashing on duplicate setup_ids on any re-run;
now ON CONFLICT DO NOTHING, so incremental nightly builds are safe.
Scheduler gains 'labeler build' + 'build-tod1' after eod_replay in both
compose files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 06:51:56 -05:00
momentum devandClaude Fable 5 5f33799a9a fix: actually wire loop_session into the snapshot payload
The prior commit imported query_loop_session but a botched idempotency
guard skipped inserting the snapshot field (the import string itself
matched the 'already present' check) — ruff unused-import failed CI and
the stale-session banner had no data. Field added; banner live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 05:22:49 -05:00
momentum devandClaude Fable 5 2e39b756ba dashboard: stale-session alarm banner + rvol_tod in armed triggers
The 07-21 morning outage (loop booted post-EOD on the 20th, survived
midnight, silently blocked every entry while signals went READY) had zero
UI surface. New query_loop_session projection parses the live session id's
embedded date; a red banner fires when it isn't today's — the exact day-2
poison signature — telling the operator to restart the loop. Armed
Triggers rows now also show rvol_tod (the arming signal since 5597250)
next to legacy rvol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 20:09:56 -05:00
momentum devandClaude Fable 5 9bfd458956 feat: tod1 shadow track — zero-gap intraday volume-explosion detector
The 07-21 misses (CPHI +417%, DFNS +94%, GREE/WGRX +40%) were all static-gap
<10% names the validated gapper population excludes by design, but they share
a signature no refuted detector (liq1/run1) measured: rvol_tod (cum volume vs
median-20d prorated by elapsed session) at 50-1000x on a flat open. New
labeler setup tod1 fires on the rising edge of rvol_tod>=50 AND price>=
prev_close*1.20, restricted to static gap <10%, stop = 5-bar swing low,
scored via the shared _forward_labels. Shadow-only; pre-registered promotion
bar locked before forward data (median fwdmax60 >=+10%, median dd60 >-6%,
+1R>50%, top stratum rvol_tod>=200 clears alone).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 11:59:51 -05:00
momentum devandClaude Fable 5 3fdf268227 fix: trigger fires require a fresh cross — stale breaks after a blind window are chases
SNXX 2026-07-21: 18s after a mid-morning restart the fast path bought a
15-minute-old break +1.6% above the trigger — fires() only checked
price >= trigger, so any blind window (boot, halt, feed outage) fired
instantly on already-extended moves. A trigger now fires only if the live
price was seen BELOW the trigger since arming (a real cross), with a
small at-the-break tolerance (TRIGGER_FRESH_CROSS_TOL, default 1.5%)
so a fast break between ticks still fires; beyond it the name must pull
back below the level and re-cross. Gap-through runners stay covered by
the slow marketable path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 08:50:41 -05:00
momentum devandClaude Fable 5 256bdc31dc fix: clean-exit at date rollover — post-EOD boot ran overnight, day-2 took zero entries
A process booted AFTER the day's EOD flatten never gets exit-after-flatten,
survives midnight, and every memory-only daily flag (_session_closed, entry
caps, benches) stays stale — the documented day-2 limitation, hit live at the
2026-07-21 open (booted 17:11 the day before; signals READY + book armed but
zero gate evaluations, both entry paths silently blocked by _session_closed).
Main loop now exits cleanly when the session date rolls over; compose restarts
a fresh process that rebuilds the new day's state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 08:42:04 -05:00
momentum devandClaude Fable 5 5597250ff1 fix: marketable entries price from live last + time-of-day rvol arms fast path
Day-22 forensic (BIYA session): 6 entry submissions, 0 fills — IREX missed
+1.7R, IRE +1.65R by one cent. Two root causes fixed:

1. Slow-path limit rested AT the trigger while price ran 2% past it, and
   retries reused the same stale price. Marketable model now prices from
   max(trigger, live last) via price_provider; the anti-chase extension
   gate gets the live ref too (was an entry_ref==trigger no-op on this
   path). TRADING_ENTRY_FILL_MODEL flipped to marketable.

2. Fast trigger path armed nothing all day: legacy rvol (cum-so-far /
   full-day MEAN) is ~0.1-0.3 minutes after the open for a normal heavy
   name and its mean is poisoned by reverse-split/prior-squeeze days
   (BIYA legacy 1.27 vs real activity 481x median). New rvol_tod =
   cum / (MEDIAN(20d) * elapsed-session fraction) drives arm eligibility
   and the scanner keep-filter; legacy rvol untouched for validated
   boundaries (gap-squeeze rvol>=5).

Also: stagnation time-stop (TRADING_EXIT_STAGNATION_MIN/_R, default OFF)
in simulate_exit + manage_live with sweep_exits.py variants for
validation before any live enable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 15:40:15 -05:00
momentum devandClaude Fable 5 f10a8033a1 chart: add volume histogram + Bollinger Bands + EXIT markers
Owner asks on the candlesticks:
- VOLUME: histogram series (bottom 20% overlay), coloured by candle direction.
- BOLLINGER BANDS: 20-period 2-sigma, computed client-side from closes; price
  riding the upper band = strong, break back inside = fading (momentum context).
- EXIT markers were NEVER wired — the fills feed carries only entries
  (order_submitted), and the loop fills brackets via position-diffing so no
  order_filled fires. Exits live in position_closed; now surfaced as square
  markers with reason + P&L (green win / red loss). Entry 'expected' (unfilled
  submit) markers labelled distinctly. Fixed a self-inflicted bug: queried the
  event store AFTER closing it — moved the position_closed fetch inside the
  open-store block. 11 chart tests green incl. new exit-marker test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 14:28:15 -05:00
momentum devandClaude Fable 5 e7bd8fa279 fix: repair DashboardState.__init__ (my prior commit orphaned _backtest_lock)
The self-heal commit (a00f967) inserted the _research method INSIDE __init__ at
method-body indent, truncating it so _change_signal/_backtest_lock/_backtest_
cache never got set -> AttributeError on backtest. Moved _research to a sibling
method; __init__ now sets all attrs. Two chart/sparkline tests updated: a None
research_con now REOPENS lazily, so 'no research db attached' is reached only
when the reopen also fails (they monkeypatch open to fail). 22 api tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 12:06:19 -05:00
momentum devandClaude Fable 5 a00f967bdf fix: dashboard research-DB connection self-heals — the 'TradingView failing' root cause
The charts weren't broken — the data pipe was empty. dashboard_api opened the
research connection ONCE at boot; if the DB was unreachable then (container
started before Postgres, or a blip) it stayed None forever, so /api/chart and
/api/bars returned 'no research db attached' and every candlestick + sparkline
rendered blank. Live-verified: GET /api/chart returned bars:[] reason:'no
research db attached' on the deployed dashboard. Now DashboardState._research()
opens the connection LAZILY and reopens if None — same self-heal as the
event-store emit fix. 2 tests; chart/backtest endpoint tests still green.
No library change: Lightweight Charts v5.2 is fine, it just had nothing to draw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 12:04:43 -05:00
momentum devandClaude Fable 5 5f04d5584a ui: surface hidden values + UI-REVIEW fixes 1 & 2
Owner: hover-only values were invisible. Now VISIBLE:
- Trades panel: Entry + Exit$ columns (projection now carries entry_price/
  exit_price; were hidden entirely).
- Armed triggers: entry/stop/now levels shown inline (were tooltip-only).
UI-REVIEW fix 1 (color): collapsed 3 greens->1 (#34d17e) and dup reds->
canonical #ff5a60 across CSS AND canvas markers (one --green/--red now).
UI-REVIEW fix 2 (a11y): :focus-visible outlines on all controls + aria-labels
on the money-path buttons (approve/reject/exit/flatten) — keyboard operators
were flying blind.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 11:54:21 -05:00
momentum devandClaude Fable 5 dd13c41fc8 test(019): de-flake watchlist concurrency test — assert single-snapshot consistency
The test compared len(w) and list(w) as two SEPARATE locked reads, so the
churn thread could mutate between them (30 vs 31) — a bug in the test, not the
Watchlist (each read is individually consistent; there is no torn-read
guarantee ACROSS two calls). Now snapshots once and asserts that snapshot never
tears + concurrent contains/slice never raise. 5/5 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 10:46:43 -05:00
momentum devandClaude Fable 5 b3a47d775d feat(019): discovery on its own thread + tuner heartbeat (honest stale alert)
Telemetry: discovery took 6-30s ON the single scheduler, freezing ingest/arm
so the trigger read stale bars — the real 'late edge detection'. Now discovery
runs on its own daemon thread with a dedicated DB connection; the watchlist it
mutates is a lock-guarded Watchlist (list-like drop-in, no torn reads); the
scheduler discover step auto-disables (single-writer). Loop is now 3 fast
threads (entries/exits/discovery) + slow scheduler for tolerant work.

Also: nightly_tune writes a heartbeat every run, so the boot alert distinguishes
'tuner not running' (real) from 'ran, no profitable ORB config, kept params'
(expected — the strategy has no edge). Kills the misleading Telegram alarm.
543 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 08:44:10 -05:00
momentum devandClaude Fable 5 a619a99f05 plans(019): move discovery off the scheduler — telemetry-proven 6-30s loop blocker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 08:33:49 -05:00
momentum devandClaude Fable 5 5896d291d8 feat(018): sub-second exits on a dedicated thread + fast decision pipeline
Exits move off the single-threaded scheduler onto their own daemon thread at
1s (mirrors the fast trigger thread). Dedicated research connection (shared
facade isn't thread-safe); scheduler exits step auto-disabled when the thread
runs (single-caller invariant — no racing instance state, no double-submit);
manage() already idempotent (stop ratchets >1c up only). 3 driver tests + 539
suite green. On fast small-caps this is the 'get out fast' the owner asked for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 08:32:46 -05:00
momentum devandClaude Fable 5 375973fe21 plans(018): sub-second exit thread — the last real execution-latency lever
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 22:05:33 -05:00
momentum devandClaude Fable 5 680ddc2174 loop: faster decision pipeline — ingest 20->5s, arm 20->8s, watch 15->8s, trigger 4->2s
Owner: 'make all decisions as fast as possible.' The real entry-detection
ceiling was ingest — bars refreshed every 20s, so the 4s trigger thread read
data up to 20s stale and detected breakouts late. Ingest is incremental+cheap,
safe at 5s. Fresher bars + faster arm (trigger levels) + 2s trigger check +
the 4s exits shipped earlier = the whole ingest->detect->decide->exit loop
tightened. One batched data call per cycle so Alpaca rate limits are fine.
Sub-second exit management on a dedicated thread is the next (code) step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 22:04:29 -05:00
momentum devandClaude Fable 5 c3828eff15 exits: manage every 4s not 12s — kill the give-back on fast small-cap round-trips
Owner's real complaint: late exits gave back gains. Exits ran on the slow
scheduler at 12s while entries run on the 4s fast thread — exits were 3x
slower than entries. A small-cap can spike and round-trip inside 12s. Drop
TRADING_EXIT_MANAGE_INTERVAL_SECONDS to 4s (config, reversible, both compose
files). Deeper fix (exit mgmt on the sub-second trigger thread) planned
separately. Entry chase-tail (24% fill >5% above break) is the next
investigation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 21:53:57 -05:00
momentum devandClaude Fable 5 2fd0a860ad plans(017): lower-gap redesign REFUTED — no positive cell in gap x overhang grid
Sliced 2191 depth fires gap x overhang under real exit: lower-gap is WORST
(0-3% -0.831R), inverting the low-gap-edge hypothesis. Overhang helps a little
everywhere, nothing crosses zero. Exhaustive: 13 levers + regime + depth +
this grid = zero positive. No positive subpopulation in gap-ORB $2-20.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 21:13:26 -05:00
momentum devandClaude Fable 5 bdeb2ec4b4 plans(017): VERDICT — L2 depth is real but too weak; FAIL under real exit + random-gate null
Definitive test, n=2191 real deployed exit: ask-overhang separates +0.083R
(high -0.676 vs low -0.759, direction held sniff->scale = genuinely real) but
fails the random-gate null — it shrinks losses, never creates profit. Live
gap>=10% band n=13/74 = no signal. Same shape as regime gate on a negative-EV
base. 13 conversion levers + regime + depth tested, zero make it positive.
Real-money rec: NO. Depth edge argues for a broader-population redesign as
future research, not a live fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 18:00:07 -05:00
momentum devandClaude Fable 5 5875bbd5f1 fix(events): emit self-heals on a dropped DB connection (the Friday blind-loop bug)
The 2026-07-17 20:23 postgres blip left the live loop writing zero events for
hours: emit() executed on a RAW cursor, bypassing PgConnection._run's
reconnect. cursor() doesn't round-trip so the dead conn went unnoticed until
the raw execute failed with no retry, and self._conn was never rebuilt.
Routing emit through the facade execute() wraps the round-trip in _run, so a
DB blip reconnects on the next write instead of running blind. 2 reconnect
tests + 34 storage tests green. (Deeper 'halt trading when persistently
unable to write' safety guard queued separately.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 14:37:26 -05:00
momentum devandClaude Fable 5 0da3ac0ff7 plans(017): honest status — signal real but modest (AUC 0.586), clean verdict backfilling
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 11:48:08 -05:00
momentum devandClaude Fable 5 46d3460d5c experiments: depth_overhang_v1 — L2 order-book overhang gate (first signal to clear the bar+null)
Plan 017 Phase 0->3. Databento depth backfilled for 602 fires across 21
sessions -> depth_features table. Reweighted to true 23% population win rate,
gating on ask-overhang-above-break (keep>=0.034) lifts win rate 23->28% and
clears the random-gate null (+0.057 vs +0.05 on the conservative +1/-1
bracket). iv_depth_overhang_gate = keep-or-zero, skip-on-no-coverage (the
dilution-v1 lesson). This commit registers the REAL-exit harness run
(simulate_exit asymmetric trail) for the true live delta. Counter-intuitive
mechanism: more resting supply above the break = MORE likely to win
(absorption/fuel), invisible in bar data — the owner's 'size stacked above'
instinct, proven.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 11:32:06 -05:00
momentum devandClaude Fable 5 b9828e5e06 plans(017) Phase 0: GO — ask-overhang-above-break separates winners AUC 0.80
First genuine selection signal in the project. 40 real fires, Databento
XNAS.ITCH book at each trigger: naive imbalance = noise (AUC 0.45/0.50), but
ask-size resting within 1% ABOVE the break separates winners at AUC 0.80
(winner median 0.276 vs loser 0.005; median split 27% vs 67% win). Inverted
from supply-trap intuition = absorption/fuel mechanism, invisible in OHLCV.
n=23 single-day sniff -> GO to Phase 1 recorder, not to trading. Cost: pennies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 09:26:50 -05:00
momentum devandClaude Fable 5 17870872ca research: regime gate relevance — honest answer to 'if it fails everything what's the point'
The gate is damage-control, not an edge source: it can only allocate the
strategy's existing (negative) edge onto least-bad days — best case flat,
never gain. Its value is CONDITIONAL: a holding pattern until an entry edge
(plan 017 L2 depth) is found, at which point it flips to force-multiplier
(good names x good days). Owner's 'it just turns the bot off' critique is
correct at the current no-edge base.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 09:21:34 -05:00
momentum devandClaude Fable 5 012bcd7b3b plans(017)+data: Databento L2 key verified live — Phase 0 is now a ~$0 same-day experiment
Cost-checked (no data spent): DATABENTO_API_KEY reaches XNAS.ITCH TotalView
10-level depth; every real gapper (EHGO/ATPC/MUD/LASE/AEON) covered at
$0.0001-0.0066/fire, full 20-fire sniff <$1 inside free credits. Catalogs
all 13 provisioned-but-unwired keys with a wire-on-demand triage (Databento
= game-changer; 4 news APIs = skip; rest redundant).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 09:06:33 -05:00
momentum devandClaude Fable 5 4f720fca4f plans(017): L2 order-book depth recorder — free-first, spend-gated, pre-registered
The one untested selection axis (order-book depth = the owner's 'size stacked
above the break' instinct). Staged so $0 is spent until a free sniff test +
recorder prove the signal exists and covers the universe; paid feed is an
explicit owner gate; the signal faces the same pre-registered bar + random-gate
null as every refuted lever before it touches the live entry. Honest ceiling
stated: a PASS nudges odds, is not an autonomous oracle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 08:57:06 -05:00
momentum devandClaude Fable 5 053c927cf7 backtest: explain the regime-gated empty window + toggle to view raw-criteria trades
Owner saw a blank /backtest — it was CORRECT (the regime gate sat out every
recent session) but unexplained. Now: ?regime=0|1 override on /api/backtest,
an 'apply regime gate (as deployed)' checkbox, and an empty-state message
saying WHY there are no trades and what to click. Gated view = what the bot
would do; ungated = the raw simple criteria.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 22:40:03 -05:00
momentum devandClaude Fable 5 3ce8594605 research: supply-side shelf distillation — filed for reference (parked per owner direction)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 22:02:29 -05:00
momentum devandClaude Fable 5 d4d1383e2c plans: mark queued defects fixed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 20:57:56 -05:00
momentum devandClaude Fable 5 7a82a9f256 fix(admin_api): stop leaking the real .env on import
admin_api.py called load_dotenv() unconditionally at module level, so
merely importing it (e.g. pytest collecting
tests/unit/test_admin_effective_config.py) ran python-dotenv's upward
find_dotenv() search, loaded the real repo .env (secrets included),
and polluted os.environ for the rest of the test session.

Gate the load_dotenv() call behind `if __name__ == "__main__":` at the
same source location, so it still runs before the module-level
os.environ.get(...) constants when run as the entrypoint (`python
admin_api.py` is unchanged) but never fires on a plain import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 20:53:33 -05:00
momentum devandClaude Fable 5 6e6bca1518 fix(attribution): stop KeyError-ing when DATABASE_URL is unset
research/attribution.py's _connect() read os.environ["DATABASE_URL"]
directly, bypassing the storage facade fallback chain (DATABASE_URL,
PG* vars, LAN host retries) every other module goes through — and
KeyError-ing (uncaught by name, though still 500'd by the endpoint's
broad except) whenever DATABASE_URL was unset.

Delegate to storage.db.get_connection().raw instead, so DB resolution
matches EventStore/research.multi_schema, and a genuinely unreachable
DB fails with a catchable psycopg2.OperationalError rather than a
KeyError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 20:49:55 -05:00
momentum devandClaude Fable 5 9772aca6bc fix(events): add optional side field to OrderFilledEvent
OrderFilledEvent had no side field, forcing api/main.py's chart() to
backfill it from the sibling order_submitted row. Add an additive-only
`side: str | None = None` field (existing payloads without it still
parse) and populate it at the sole production emit site
(alpaca_paper/execution.py _finalize) from the ExecutionRequest already
in scope. The api/main.py backfill workaround is left in place as a
harmless fallback for pre-existing events.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 20:47:17 -05:00
momentum devandClaude Fable 5 849bd476f9 adjudicator: random-gate null for keep-or-zero PASSes + register orderflow H6 (quote-size book imbalance)
Null (the H2 lesson): zeroing share f of a negative baseline mechanically
yields expR_delta = f*|mean baseline|; a PASS on expR_delta_min now also
requires expR_delta_vs_null >= the same margin whenever zeroed_share >= 5%.
Exposure-shrinking can no longer masquerade as selection; non-gate
experiments unaffected. Honest disclosure: regime_meta_v1's FIRE-weighted
pass would not clear this bar either — its standing rests on the session-
level walk-forward OOS evidence, which no random gate reproduces.

H6: CKS order-flow-imbalance from quote_snapshots bid/ask SIZES —
size-weighted book imbalance over the 10-min pre-trigger window, zero
ask-tilted books; skip-on-no-coverage; judged under the null from day one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:50:05 -05:00
momentum devandClaude Fable 5 5bf089cdcc research: orderflow H1/H2/H4 verdicts — all refuted in substance; H2's formal PASS is a bar artifact
H2 exposed an adjudicator gap: on a -0.21R/fire baseline any keep-or-zero
gate zeroing ~44% of fires clears +0.05 mechanically; its own pre-registered
strata check inverts (choppy beats persistent within both gap buckets =
Simpson's). Queued: random-gate null for keep-or-zero bars. H3 accruing as
designed (n=29, skip-on-no-coverage working).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:24:47 -05:00
momentum devandClaude Fable 5 07ddbd30b9 experiments: register orderflow H3 (spread-expansion gate) + H4 (volume-spike gate)
H4 (3-book): keep-or-zero on trigger-bar volume concentration vs prior 5-bar
sum (>1.0 = completed one-shot burst). H3 (5-book, direction disputed):
fragility-camp primary — zero fires whose fill-time relative spread exceeds
2x the name's own 10-min median from quote_snapshots (which carry bid/ask
SIZES since 07-02 — the note's data-upgrade #1 was already satisfied).
Coverage design learns the dilution_veto_v1 lesson: no-quote fires are
SKIPPED (variant None, excluded from n), so thin coverage cannot false-FAIL.
H5 deliberately deferred until H1's verdict (its incremental value is defined
net of H1's catches). 6 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 18:32:10 -05:00
momentum devandClaude Fable 5 46f80eccf2 experiments: register orderflow H1 (trigger-erasure exit) + H2 (flow-persistence gate)
From docs/research/orderflow_notes.md — the order-flow shelf distillation.
H1 (5-book convergence): exit on first post-fill 1-min close back below the
trigger within 5 bars — the transitory-impact/recross signature — else the
standing exit stack. H2 (4-book): keep-or-zero on pre-trigger volume-weighted
bar-sign persistence (<0.30 over 15 bars = choppy one-shot burst, zeroed);
strictly-before-entry window, fail-open on thin history. Both pre-registered
at the house bar (+0.05R, n>=300, standard fidelity); nightly adjudication
decides. 7 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 18:24:22 -05:00
momentum devandClaude Fable 5 cdb138c87d research: order-flow shelf distillation — 5 pre-registered-ready entry-conversion hypotheses
9 sources (Hasbrouck, Bouchaud x2, Cartea, Dalton, Bellafiore, Neill, CKS,
Cetin-Danilova) read by parallel agents + adversarial synthesis, constrained
to computable data (1-min SIP bars, 15s quotes) and fenced from the 8 refuted
levers. Companion to regime_notes.md. H1 (trigger-erasure fast invalidation,
5-book convergence) is the lead candidate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 18:07:21 -05:00
momentum devandClaude Fable 5 a9aa37c885 backtest: replay the LIVE track only (v1) — shadow research tracks poisoned /backtest
replay_window reconstructed fires from ALL setup versions, so liq1/vr1/run1/pm1
(refuted shadow tracks, never traded live, 8:1 volume vs v1, tight-stop
artifacts) dominated /backtest: 17k trades / 45 sessions at -1.8R/trade —
garbage scale vs the harness-validated -0.08R/fire. reconstruct_fires_from_setups
gains an optional setup_version filter (default None: experiments unchanged);
replay_window pins it to v1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:22:03 -05:00
momentum devandClaude Fable 5 d50115dea8 backtest: 1d window button (owner request)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:09:06 -05:00
momentum devandClaude Fable 5 8fd0fc7921 plans(016): record first adjudication — FAIL on pre-registered bar (3% coverage; flagged fires outperformed)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:22:19 -05:00
momentum devandClaude Fable 5 d2114a7716 plans(016): DONE — merged f6999c7; backfill seeded
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:56:26 -05:00
momentum dev 3d0f404a29 Merge branch 'worktree-agent-a15ca95a014d606fe' 2026-07-17 09:56:24 -05:00
momentum devandClaude Fable 5 f6999c7f6a dilution(016): unbreak nightly EDGAR refresh + pre-register filing-veto experiment
premarket_eval's shadow enrichment call site queried session_date = current_date
(wrong on UTC-server Postgres at night, so it never matched any symbols) and
shared one try-block for float_cache + dilution filings (a float_cache failure
silently skipped the dilution refresh too). Fix: compute the ET session date
in Python, split into two independent try-blocks, lift into a testable
_refresh_shadow_enrichment(con, session_date). float_cache now applies its own
DDL unconditionally on use instead of only when an API key is present.

Adds iv_dilution_gate (additive, mirrors iv_regime_gate's keep-or-zero shape
keyed on symbol instead of session_date, fail-open on unfetched symbols) plus
its once-per-run preload, and registers the pre-registered dilution_veto_v1
experiment via the existing experiments/*.json self-registering cohort. Live
veto stays OFF; this only wires the shadow measurement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:54:49 -05:00
momentum devandClaude Fable 5 d8d37700aa plans(016): EDGAR dilution enrichment — unbreak nightly refresh + pre-register filing-veto experiment
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:29:40 -05:00
momentum devandClaude Fable 5 92488b72bc youtube: YT_PLAYLIST_URLS — comma-separated playlists via env, synced at boot
One playlist per add-playlist CLI call made adding a second playlist a chore.
Now YT_PLAYLIST_URLS takes a comma-separated list of playlist URLs (list=
param extracted) or bare ids; the transcripts service seeds the playlists
table from it at boot (insert-if-missing — hand-set labels are never
clobbered), and `python -m youtube_claims.run sync-playlists` does the same
on demand. Documented in both compose files; 3 tests.

Also: queue the two executor-surfaced defects (OrderFilledEvent side,
/api/attribution env hardcode) + the flaky nightly regime build-features
step in the plans index; regime_features 07-16 backfilled supervised.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:14:22 -05:00
momentum dev a4e7a6d534 Merge branch 'worktree-agent-ae4d3f36e18c6e7a5' 2026-07-17 09:05:47 -05:00
momentum devandClaude Fable 5 8feac6da81 plans(015): DONE — executed + review-approved first pass, awaiting merge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 08:45:39 -05:00
momentum devandClaude Fable 5 8ac222a4ca research-tooling: /backtest replay dashboard — theoretical trades under current settings
Adds a read-only /backtest page showing how the bot WOULD have traded past
sessions under the CURRENTLY configured live settings: theoretical ORB fires
from setup_events, replayed through the shared exit engine (simulate_exit +
ExitConfig.from_env(), same adverse-fill accounting the exit-experiment
harness uses), screened through the shared population gate and (optionally)
the live regime gate. No new simulation math — replay.replay_window() is pure
assembly of existing pieces (reconstruct_fires_from_setups, _fire_in_population,
_load_regime_favorable, simulate_exit); api/main.py adds the /backtest and
/api/backtest routes plus a dedicated-lock 10-min cache so a slow replay never
blocks the live dashboard's SSE stream. Also extends /api/chart with an
optional date= anchor so the trade drill-in chart modal can reach sessions
older than the existing 72h-from-now window, and adds a one-line BACKTEST nav
link to the live dashboard header.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 08:42:43 -05:00
momentum dev 0ed0abe0e2 Merge branch 'worktree-agent-acb00535650deac1a' 2026-07-17 08:16:59 -05:00
momentum devandClaude Fable 5 822584e072 plans(014): DONE — executed + review-approved first pass, awaiting merge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 07:38:01 -05:00
momentum devandClaude Fable 5 dd81dbbda1 operator-plane: live decision trace — gate receipts feed + chart reject markers (plan 014)
Surfaces the entry_gate_evaluated decision receipts (plan 005) that were
write-only until now: a Decision Trace panel (per-gate value->threshold
chips, symbol filter, lazy-load + SSE refresh) and collapsed reject markers
merged into the plan-013 symbol chart's existing markers primitive.

- storage/projections.py: query_gate_receipts (case-insensitive symbol
  filter, newest-first, same _hist_window idiom as query_fills_feed)
- api/main.py: GET /api/decisions (clamped limit<=500, date-scoped);
  DashboardState.decisions() adds a server-computed epoch per row; factored
  chart()'s inline to_epoch into a shared _to_epoch_et helper
- api/static/dashboard.html: Decision Trace panel + refreshChart now also
  fetches /api/decisions and merges collapsed reject markers (same-reason
  bursts within 5min -> one marker) into the single setMarkers call

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 07:36:31 -05:00
momentum dev 968bcadc94 Merge branch 'worktree-agent-ae2ee0000d5e3a7eb' 2026-07-17 07:19:37 -05:00
momentum devandClaude Fable 5 e224d82291 plans(013): DONE — executed + review-approved, awaiting merge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 06:51:14 -05:00
momentum dev 709e7a4344 plan 013 fixup: stop leaking a markers primitive per SSE refresh
createSeriesMarkers(series, markers) attaches a NEW primitive to the series
on every call — refreshChart() called it on every 5s SSE-driven refresh, so
a modal left open accumulated one primitive per refresh (identical markers
overdraw so it looked fine, but it's an unbounded render leak on long-open
modals). v5 pattern: create the markers primitive once in openChart, mutate
it via markersApi.setMarkers(markers) on refresh. chart.remove() in
closeChart already tears the primitive down with the chart.
2026-07-17 06:50:41 -05:00
momentum dev b4ad3d3a66 operator-plane: symbol drill-in chart — 24h candles + live trade markers (plan 013)
Click any symbol cell to open a 24h candlestick chart (vendored TradingView
Lightweight Charts v5.2, Apache-2.0) with buy/sell arrows at the bot's fills,
muted arrows for submitted-unfilled orders, and entry/stop/target price
lines. New read-only /api/chart endpoint (DashboardState.chart) converts
naive-ET minute_bars/event timestamps to UTC epoch seconds; query_fills_feed
gained an optional symbol filter. Chart refreshes off the existing SSE
snapshot signal (5s throttle, full replace) and tears down on close. Also
backfills order_filled markers' missing `side` (OrderFilledEvent's schema
has no side field) via an order_id join to the sibling order_submitted row.

Merged plan 012 (advisor/012-dashboard-order-limits) first — this plan
depends on its stopWorkingBy/targetBy/entryLimitBy maps.
2026-07-17 06:46:52 -05:00
momentum dev 012de16f36 Merge branch 'advisor/012-dashboard-order-limits' 2026-07-17 06:16:49 -05:00
momentum devandClaude Fable 5 2d65669308 plans(012): DONE — executed + review-approved on advisor/012-dashboard-order-limits, awaiting merge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 06:12:29 -05:00
momentum devandClaude Fable 5 1f3ddcabe4 plan 012 fixup: Positions Stop column must use working-filtered stop, not legacy stopBy
sync.py fetches status="all" so the legacy stopBy map (kept for the
R-bar/spark stop line, unchanged) can pick up a cancelled stop leg.
Add stopWorkingBy, filtered to the same WORKING statuses as
targetBy/entryLimitBy, and point the new Positions Stop column at it
so a cancelled leg never renders as live protection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 06:11:40 -05:00
momentum devandClaude Fable 5 b3495187c3 plan 012: show working limit/stop/target prices, drop dead order-id + Mode columns
Positions table gains Stop/Target columns from resting bracket legs
(targetBy/entryLimitBy maps, filtered to working order status). Order
Lifecycle splits the generic Price column into Type/Limit/Stop/Fill via
a client-side join to the broker order snapshot (client_order_id ==
the internal order_id we send Alpaca) since the lifecycle events
themselves never captured limit/stop/type; the truncated order-id
column is dropped in favor of a title attribute on the Symbol cell.
Approval queue drops the always-"auto" Mode column and shows explicit
entry/stop prices in the risk chip instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 06:09:48 -05:00
momentum devandClaude Fable 5 5f2cbc4909 plans(012): revise Step 3 after executor STOP — lifecycle events never carried limit/stop; enrich client-side via client_order_id join to d.orders
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 06:05:26 -05:00
momentum devandClaude Fable 5 33708ffe6a plans(012-015): dashboard visibility milestone — order limits, symbol charts, decision trace, /backtest replay
Owner-requested UI work, advisor-planned for executor handoff:
- 012: show working limit/stop/target prices on positions + order lifecycle,
  cut dead columns (order-id, approval Mode)
- 013: symbol drill-in — 24h candlestick + live buy/sell fill markers
  (TradingView Lightweight Charts v5.2, vendored, Apache-2.0)
- 014: live decision trace — surface entry_gate_evaluated receipts as a feed
  (value→threshold per gate) + chart reject markers
- 015: /backtest replay page — theoretical entries/exits over past sessions
  under CURRENT settings, assembled strictly from Experiment-OS primitives

All four are read-path only; zero live-loop surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 06:00:45 -05:00
momentum devandClaude Fable 5 43a3ace91d Regime gate: make the live gate actually decide (it shipped inert) + fix breadth_rvol units
The 07-15 live gate never made a real decision: live_session_favorable read
today's breadth_rvol from today's OWN setup_events rows, which only the
NIGHTLY labeler writes — always None intraday, fail-open every session (the
07-15 "validation block" was computed pre-guard on a zero-filled feature).
Worse, breadth_rvol was unit-corrupted: _session_context pooled all setup
versions and pm1 stores RAW cumulative premarket volume in relative_volume
(median ~261k) vs the ratio tracks (~0.8) — ~240/267 historical sessions'
breadth was actually median premarket volume, and plan 009's v1 RVOL fallback
was about to flip forward sessions to ratio units.

Fixes (TDD, 21 regime tests):
- breadth_rvol is now pm1-ONLY: median premarket volume — the dominant
  historical source, unit-consistent, and strictly pre-open knowable.
- _live_breadth_rvol recomputes today's pm1 fires IN MEMORY from premarket
  minute_bars (pure compute_premarket_setups, no insert); previous_close
  falls back to the latest prior daily close when today's daily_bars row is
  absent intraday (else the gap basis silently flips to the premarket open).
- _regime_reject: verdicts before 9:50 ET are PROVISIONAL (re-scored ~5 min;
  ingest lag can truncate the premarket frame) and the first verdict at/after
  9:50 is FINAL even when None — caps the scorer at a handful of runs/day
  instead of ~78 on quiet days (hot-path stall guard). Throttle arithmetic
  moved inside the fail-open try (naive/aware mixing must not escape).
- Live fit excludes breadth_rvol-NULL training rows (zero-fill taught a
  "zero premarket volume" class the serve path can never produce).

Prod re-validation on the cleaned feature (regime_features rebuilt, scores
rewritten): regime_meta_v1 fire-weighted expR_delta +0.0570 (pre-registered
bar +0.05, n=2058, PASS — the contaminated +0.0965 was inflated), keeps 37%;
recent-80-session delta +0.149R/session (always −0.141 → gated +0.008).
First genuine live verdict 2026-07-16: UNFAVORABLE (pm breadth 612,819).
Known accepted skew: live scaled-IEX volumes vs nightly SIP tape — the
meta_live-vs-EOD comparison is the standing forward measure.

Also backfilled the missed 07-15 nightly job supervised (SIP refetch, 50 v1
+ 6 pm1 setups, SPY/QQQ ingest, regime row) — plan 009's accrual clock is
unbroken.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:00:07 -05:00
momentum devandClaude Opus 4.8 c8e542fe6c deps: add scikit-learn — unblocks CI image publish (broken since plan 008)
The docker-image workflow's test gate has aborted at collection since sklearn
was introduced (test_conversion_probe.py, plan 008), so no image published.
Regime gate (plan 011) also needs sklearn at runtime in the image, else the
live gate silently fail-opens forever. Adds scikit-learn to requirements.txt
(installed by both the CI test job and the Dockerfile).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 15:23:31 -05:00
momentum devandClaude Opus 4.8 acdf0019c3 Regime gate: document TRADING_REGIME_GATE_P_MIN in both compose anchors
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 15:21:49 -05:00
momentum devandClaude Opus 4.8 9082144d43 Regime gate: record live-gating (paper) in plans index
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 14:12:36 -05:00
momentum devandClaude Opus 4.8 0ac09d3574 Regime gate LIVE (plan 011): gate fresh entries on the session-level metalabel
Wires the regime signal into the money path — fresh entries are BLOCKED on
sessions the walk-forward model scores unfavorable. Session-level: evaluated
once per day, cached. Enabled in both compose templates (paper).

Correctness (blocker 1 resolved): the LIVE feature set drops tape_regime (the
only non-strictly-open feature — AND measured 0.000 importance) + breadth_gap
(also 0.000). The 5 remaining features are all non-look-ahead at entry time:
breadth_rvol = median(relative_volume=vol_upto/avg_vol_20d) over the session's
setups so far (the SAME up-to-trigger calc the live bot runs, NOT day-end),
spy_* lagged to prior close, trailing from PRIOR sessions. The live single-
session fit reproduces the batch meta_v1 verdict exactly on the last 6 sessions.

Safety: fail-open everywhere — disabled, unfit model (<60 prior sessions),
missing breadth_rvol (pre-setup), or ANY exception returns None and never
blocks. The gate can only SUBTRACT trades on bad-regime days.

- research/regime.py: live_session_favorable() (refit on priors <today, predict
  today; writes regime_scores['meta_live'] for forward validation) + LIVE_FEATURE_COLS
  + the breadth_rvol-missing fail-open guard.
- trading_execution.py: TRADING_REGIME_GATE_ENABLED / _P_MIN settings + _regime_reject()
  (cached per session) + a REGIME gate after SESSION in _evaluate_entry_gates.
- docker-compose.yml + truenas-compose.yml: TRADING_REGIME_GATE_ENABLED=1.
- tests/unit/test_regime_gate_live.py: block / pass / fail-open(x3) / scored-once-per-day.

KNOWN train/serve skew (documented, validate forward via meta_live vs EOD label):
model trains on full-day breadth_rvol, live uses setups-so-far. Small edge on a
~266-session sample; paper only. Suite: 504 passed, 15 pre-existing loopback fails.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 14:11:50 -05:00
momentum devandClaude Opus 4.8 9e7752c1fa Regime gate: fire-weighted harness PASS (meta_v1 +0.0965R, positive on live population)
Both regime experiments PASS pre-registered bars over the 2041 v1-population
fires — first PASS of the milestone. meta_v1 gated expR +0.0117R/fire (from
-0.085R baseline). Adjudicated live via the admin /api/regime/run pipeline.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 14:30:11 -04:00
momentum devandClaude Opus 4.8 691cb9592a Regime gate: record prod backtest verdict in plans index
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 14:06:44 -04:00
momentum devandClaude Opus 4.8 092d6072e8 Regime gate (plan 011): deploy-wire the shadow pipeline + on-demand admin trigger + prod backtest
The session-level metalabel P(our ORB profits today) — the one lever that
survives out-of-sample. Backtested on 266 prod sessions (walk-forward):

  trade-always (trained subset): -0.111R/session, 36% hit
  meta_v1 GATED (trades best ~32% of days): -0.004R/session, 48% hit
  delta +0.107R in-sample, +0.128R OOS (recent 80 sessions), improves across folds

It does NOT make the bot profitable — it takes a bleeding strategy to roughly
flat by sitting out the worst ~2/3 of sessions. threshold_v1 rule gates too
little to matter; meta_v1 (walk-forward GBM) is the signal. Leaky full-fit
(+0.61R) >> walk-forward confirms the discipline was load-bearing; the OOS
holding is the evidence the +0.078R is real, not look-ahead.

Changes:
- research/regime.py: batch minute-bar load per session (64k per-fire round-trips
  -> ~266 queries; fixed a >5min build to ~3min) + incremental label pass (skip
  already-labeled sessions; --rebuild forces full). Nightly cost now ~one new day.
- docker-compose.yml + truenas-compose.yml: scheduler runs build-features + score
  (both models) before `experiments run`, nightly. ALL-SHADOW — does NOT gate live
  trading (that stays owner-gated Step 5).
- admin_api.py + admin.html: POST /api/regime/run (detached) + GET /api/regime/status
  + a Run/Status button — the panel had no way to trigger this.
- plans/011 + docs/research/regime_notes.md (6-book distillation) + plans 008/009 docs.

Prod state (shadow, rollback = DROP regime_features/regime_scores + DELETE
experiments WHERE id LIKE 'regime%'): 266 feature rows, 532 scores, 2 experiments
registered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 14:05:34 -04:00
momentum devandClaude Opus 4.8 275c56b545 Plan 011 (build+test only): regime gate — session-level ORB metalabel
research/regime.py: Step 0 probe (labelable-session/SPY-coverage STOP gate),
Step 1 build-features (regime_features: pre-open breadth/tape + prior-close
SPY momentum/vol + trailing bot expR, label = mean _sim_baseline_r over the
session's setup fires) + walk-forward threshold_v1 rule, Step 2 walk-forward
meta_v1 (shallow sklearn GBM), Step 4 OOS/leakage-audit helpers
(oos_split/sequential_folds/session_expr_delta/full_sample_fit_threshold).
Step 3 (HMM) deferred per scope — statsmodels stays absent.

replay.py: new iv_regime_gate intervention (keep-or-zero a fire on a
pre-computed session regime score, fail-open on no score row) wired into
_compute_fire + run(), mirroring rank_one_bullet's shape.

experiments/regime_threshold_v1.json + regime_meta_v1.json register the two
model variants against the shared population/bars/fidelity contract.

All new code is unit-tested against isolated EventStore(":memory:") schemas
only — no production DB read or written. Live-trading paths untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:47:53 -04:00
momentum devandClaude Fable 5 1da168da45 plans(010): NO-GO — premarket entry is a paper-fill mirage (-1.3R @ realistic thin-book spread)
Step 1 measured: pm1+fat-tail expR -0.50R@0.5% fill -> -1.27R@1% (realistic) ->
-2.74R@2%, vs ORB -0.11R. The 22-min fat tail can't outrun the 1-2% premarket
spread. Every entry lever this session (gap bands + premarket) is negative after
honest fills; the gated ORB is least-bad. Keep TRIGGER_GAP_MAX=35.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 10:05:55 -04:00
momentum devandClaude Fable 5 6b6cde53db plans(010): premarket shadow-entry — harness-gated, fill-honest, never live until proven
The one unexplored setup (premarket/pm1). Step-1 go/no-go: does pm1 + the
validated fat-tail exit beat the ORB AFTER honest thin-book fills (real
spread_pct, fat premarket adverse)? If not, STOP — premarket is a paper-fill
mirage. Shadow-first; TRADING_SESSION_ONLY stays ON; no live order until a
forward shadow edge survives realistic slippage (owner-gated).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:39:06 -04:00
momentum devandClaude Fable 5 97dcef4992 compose(root): sync managed-exit config to match truenas template
Root anchor lacked the TRADING_EXIT_* block the truenas template already has, so
a fresh root deploy ran no-trail defaults (scratches v1 winners at breakeven).
prior_low trail after +1R (counterfactual: v1 win 17%->24%) + step0.25 + profit
tiers + BE@1R. Only the loop acts on these.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 22:34:27 -04:00
momentum devandClaude Fable 5 ab2810896c experiment-os: generic exit-config intervention + v1/pm1 exit experiments
iv_exit_config replays ANY target/breakeven/trail combo through the shared exit
engine vs the live-exit baseline — reusable way to adjudicate exit numbers.
Counterfactual over setup fires: v1 (slow, 1.37R median winners) gains most from
prior_low trailing (win 17->24%, +0.036R); pm1 (fast, 2.64R median, 10R tail)
gains most from NO target + step0.5 (+0.273R, win 24->34%) — the fixed target
throws away pm1's fat tail. Registered both as tracked experiments. +test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 22:24:18 -04:00
momentum devandClaude Fable 5 7a28eafeff plans(008): RVOL adds nothing beyond VWAP (0.526->0.535, noise) — VWAP is the whole story
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 21:12:26 -04:00
momentum devandClaude Fable 5 2abbf35257 plans(008): correct verdict — v1-scoped AUC 0.641 is real but is the VWAP gate already run
The 0.514 was the liq1-dominated mix. Scoped to live v1: CV AUC 0.641, driven
~entirely by above_vwap_flag (above-VWAP +1R 35.4% vs below 12.7%, 2.8x). The
bot already enforces above-VWAP; residual within that population = 0.526 ≈ noise.
No new alpha in current features; revisit after 009 fills rvol/context/catalyst.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 21:06:45 -04:00
momentum devandClaude Fable 5 1755f1f416 plans(009): status -> code DONE, awaiting supervised rebuild+probe
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:37:07 -04:00
momentum devandClaude Fable 5 dbaeb701e2 conversion(009): committed go/no-go signal probe (Step 5 instrument)
research/conversion/{__init__,dataset,signal_probe}.py — the leakage-safe probe
plan 009 Step 5 re-runs scoped to a live setup_version:

- dataset.load_dataset: setup_events JOIN outcome_labels; label =
  reached_1r_before_minus_1r; features = trigger-time-knowable setup_events
  columns ONLY, dropping >70%-null and <2-distinct columns. HARD leakage rule —
  every outcome_labels column is blocklisted AND asserted absent from X, so a
  schema change can't leak the answer in.
- signal_probe.run_probe: expanding-window CV over ~5 sequential session-date
  blocks (whole days, train strictly precedes test), HistGradientBoostingClassifier
  (max_iter=250, lr=0.05, max_depth=4, l2=1.0, categorical_features=indices),
  per-fold + mean ROC-AUC, top-decile +1R lift vs base, permutation importance on
  the last fold. Drops columns constant within a train slice (HistGBM binning).
  CLI: python3 -m research.conversion.signal_probe [--setup-version v1].

Tests: the critical leakage guard (zero outcome columns in features), AUC>0.5 on a
synthetic signal + ~0.5 on pure noise, and a static live-path guard. All :memory:.
No live-path change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:32:18 -04:00
momentum devandClaude Fable 5 5b885a719f ingest(009): nightly SPY/QQQ ingest + catalyst enrichment ON as feature source
Step 3 of plan 009 (config + code; no live-path change):

- research/ingestion/market_context.py: pulls SPY/QQQ full-session minute bars
  (SIP, post-close embargo-safe) + daily bars via the existing upsert helpers, so
  the labeler's market-context features have a FORWARD source (index ETFs never
  enter the gapper universe the live loop ingests). Mirrors sip_backfill's shape.
- both compose scheduler blocks: run it after premarket_eval, guarded || echo.
- NEWS_ENRICH_ENABLED on: added to docker-compose x-app-env anchor (loop +
  scheduler inherit it); truenas-compose + .env.example already had it. Catalyst
  is a FEATURE source only, never a live gate; enabled-but-down never blocks
  trading. Enrichment OUTPUT is verified by the orchestrator (needs live Ollama).

Compose statically validated (yaml.safe_load). Ingest self-checked via a
fake-client :memory: round-trip through _preload_spy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:27:49 -04:00
momentum devandClaude Fable 5 51f095a0c3 labeler(009): market context + thin-name RVOL fallback (feature population)
Steps 1-2 of plan 009 — fill two fillable feature gaps so a conversion model
gets a fair, instrumented test:

- market context: _spy_context computes spy_intraday_return (SPY RTH open ->
  trigger-minute return) + a coarse tape_regime {trend_up,trend_down,chop} from
  SPY's sign + higher-highs structure. Replaces the hardcoded
  tape_regime=None/spy_intraday_return=None; wired into ALL five build paths
  (v1/vr1/run1/liq1/pm1) via _preload_spy. Null when SPY has no RTH bars at/before
  the trigger (pre-30d history, or a premarket fire before SPY's RTH open).
- thin-name RVOL fallback (v1): when avg_vol is 0/null (thin IPO-class, no 20d
  baseline) fall back to the median of prior sessions' total volume (real
  daily_bars.volume, knowable pre-open); flagged in new setup_events column
  relative_volume_source ('avg_vol_20d' | 'prior_days_median_volume' | NULL).
  Never fabricates a baseline.

A feature to TEST, not a signal to trust. No live gate/sizing/exit change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:25:07 -04:00
momentum devandClaude Fable 5 d28fe25061 experiment-os: replay exit experiments over setup_events (the n-engine)
eod_replay was the wrong tool to grow the harness sample — it only PRINTS a
backtest report (persists no fires) and get_most_actives is today-biased for
historical dates. The right source was already in the DB: 18,164 setup_events,
each carrying the ORB entry (entry_reference_price) + stop (invalidation_price).

reconstruct_fires_from_setups turns every setup into a simulated fire; the exit
experiments (volume_fade, time_stop, asym_exit) replay over its minute-bar path
with baseline = the LIVE exit config simulated on the same path (_sim_baseline_r,
same adverse-fill accounting) — a clean same-path delta needing no live P&L.
Gated by EXPERIMENTS_INCLUDE_SETUP_SIM (default on), capped by
EXPERIMENTS_SETUP_SIM_LIMIT for chunked backfill; rank_one_bullet excluded (it
needs live receipts). Grows every exit experiment's n from ~15 to thousands.
+test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 15:59:18 -04:00
momentum devandClaude Fable 5 315afcf497 experiment-os: volume-fade exit experiment + fix fire reconstruction (0-fires bug)
The live loop fills resting-limit brackets via position-snapshot diffing and
emits NO order_filled (verified: 0 all-time in prod), so reconstruct_fires
skipped every fire and the ENTIRE harness replayed nothing (0 run rows).
Reconstruction now falls back to the order_submitted timestamp for entry timing
and reads entry/stop/exit from position_closed — restores the whole flagship
(0 -> 324 run rows on prod; pm1's overdue FAIL verdict now posts).

Adds the volume_fade intervention (exit to lock the gain when a bar's volume
drops below fade_frac x the entry-burst average, once in profit; original stop
still caps downside; adverse-tick fills on entry AND exit) + volume_fade_exit_v1
(pre-registered bars/futility/fidelity). Interim n=15 read: -0.68R vs baseline
(INCONCLUSIVE, min_n=100) — the harness answers 'does volume-fade help?' from
data before any live change. +6 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 14:36:16 -04:00
momentum devandClaude Fable 5 16f880d372 experiment-os: 'run' self-registers the declared cohort (fresh-DB bootstrap)
The scheduler ran 'experiments run/adjudicate' but nothing registered the five
experiments/*.json into the experiments table, so on a fresh DB the harness
replayed over an empty set and the ledger stayed empty forever (owner hit this
on first deploy). registry.register_cohort() registers every cohort JSON
idempotently (skips already-registered, tolerates a malformed file); 'run'
calls it at startup so a fresh deployment self-bootstraps with no manual
register step. +test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:14:51 -04:00
momentum devandClaude Fable 5 a12dc8a768 test: de-flake volume-ratio test (real-clock dead zone, not plan 007)
compute_ratios bails when UTC now-16min precedes today's 08:00Z, i.e. nightly
~00:00-04:16 ET; the test's hardcoded 2026-07-06 bars made it KeyError at the
date rollover independent of any code change. Freeze the module clock to a
mid-session UTC time so the window is valid deterministically.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:36:00 -04:00
momentum devandClaude Fable 5 e2b3c1ceae operator-plane: headless smoke + effective-config panel + ledger + unpin ruff
Step 6: tests/smoke_dashboard_headless.py renders the real page in a headless
browser and fails on any JS console/pageerror (standalone script, not
pytest-collected; CI non-blocking first). admin_api gains GET /api/config/effective
(per-key masked value + source: overrides.env vs compose, missing .env.example
keys flagged as drift) + an admin.html table (values XSS-escaped). dashboard.html
gains the Experiment Ledger panel reading the plan-006 verdicts from the snapshot
(empty-state until the tables exist). Removes the last CI ruff per-file pin
(api/main.py:F821) now that os/hmac are imported; scoped ruff is green unpinned.
Marks plan 007 DONE in plans/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:36:00 -04:00
momentum devandClaude Fable 5 21687c97ff operator-plane: wire research reader into standalone dashboard (sparklines)
DashboardState.bars() short-circuits to [] unless a research_con is attached; the
embedded orchestrator passed one but the standalone dashboard_api.py never built
it, so production sparklines were permanently dead. The missing wire was purely
the entrypoint (DATABASE_URL already reaches the market schema) — now it opens
open_research_db("market") read-only and passes it, degrading to None (charts
show "no research db") if it can't connect. Verified live: bars() returns real
minute rows through the standalone reader.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:21:37 -04:00
momentum devandClaude Fable 5 cb206e45cb operator-plane: truthful LIVE badge + kill-switch UI + action gating
Badge now derives PAPER/LIVE from broker truth (base-URL host) instead of the
schwab_live literal that read PAPER while go-live swapped only ALPACA_BASE_URL;
live flips body.live-mode (top shimmer -> solid red) with a tooltip of host +
account. Adds the header FLATTEN ALL (two-step arm/confirm, 5s auto-disarm) and
PAUSE ENTRIES toggle (+ amber banner), per-position EXIT buttons (two-step), and
an operator-token field sent as X-Dashboard-Token on every POST (also fixes
approve/reject when a token is set). Action controls gate on
has_execution || command_bus; kill-switch controls on command_bus. No keyboard
shortcut for any destructive action. New styling is :root-token-only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:19:51 -04:00
momentum devandClaude Fable 5 e7c34f298f operator-plane: guarded command bus (flatten / pause kill switch)
Adds OperatorCommandEvent + a closed command vocabulary (flatten_all,
flatten_symbol, pause_entries, resume_entries). POST /api/command emits it
(token-required: 503 when DASHBOARD_TOKEN unset — commands never run
unauthenticated); the live loop's new 5s "command" step consumes them via
OperatorCommandConsumer with three replay-safety layers (in-session nonce set,
persisted operator_command_done reload, 10-min staleness window).

flatten_all routes through close_session DIRECTLY (never step_eod_flatten) so it
cannot trip plan 003's daily-restart exit. Pause is checked at the top of BOTH
request_approvals_for_ready_signals AND submit_breakout_now (missing either
leaks pause on that path). qabot gains an authorized-chat-only /flatten slash
command (prose "flatten" still refuses via _ACTION_WORDS). Snapshot exposes
broker truth + command_bus + a defensive experiment ledger.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:15:00 -04:00
momentum devandClaude Fable 5 fbb84ae166 operator-plane: cheapen SSE change detector (shared once/sec signal)
Every SSE client polled COUNT(*) on a fresh EventStore connection each ~1s
(re-applying the 473-line schema unless PG_SKIP_SCHEMA=1). Replaced with one
process-wide _ChangeSignal: a single long-lived connection reads max(timestamp)
(idx_events_timestamp tail seek) at most once per second GLOBALLY behind a lock,
so N clients cost one query/sec. EventStore.max_timestamp() backs it; reader is
injectable for the query-budget test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:05:08 -04:00
momentum devandClaude Fable 5 07570fb0e4 operator-plane: fix dashboard POST NameError (import os/hmac)
Every POST endpoint (approve/reject/exit/watch) crashed with NameError:
do_POST references os.environ + hmac.compare_digest but neither was imported.
Adds the imports, a non-loopback regression test for the token gate (no token
-> routes; token set -> 401 without/with-wrong header, 200 with correct), and
plumbs DASHBOARD_TOKEN + PG_SKIP_SCHEMA=1 into both compose dashboard services
(+ .env.example docs for DASHBOARD_TOKEN / COMMAND_BUS_ENABLED).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:04:01 -04:00
momentum devandClaude Fable 5 f62ed646dc plans: 006 experiment-os -> DONE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:53:20 -04:00
momentum devandClaude Fable 5 1036c19448 experiment-os: wire harness into both compose schedulers
Plan 006 step 5. python -m research.experiments run && adjudicate after
nightly_tune in both scheduler blocks (guarded with FAILED echoes). ALL-SHADOW:
emits verdicts, never touches live gates/sizing/exits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:49:31 -04:00
momentum devandClaude Fable 5 70c9e3715b experiment-os: first-cohort registrations (5 experiments) (+test)
Plan 006 step 4. experiments/*.json for time_stop_10m_v1, probe_add_v1,
asym_exit_v1, rank_one_bullet_v1, pm1_v1 — bars verbatim from the plan table,
production population, fidelity floors {0.25, 3, 1} from the research doc. Also
hardened evaluate(): n==0 and any uncomputable bar metric -> INCONCLUSIVE (not a
false FAIL). Cohort registers + survives run->adjudicate end-to-end. 25 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:48:58 -04:00
momentum devandClaude Fable 5 dc8e635385 experiment-os: adjudicator + pm1 bridge (+tests)
Plan 006 step 3. evaluate() scores runs against pre-registered bars
(_min/_max/min_n vocabulary) -> PASS/FAIL/KILLED/INCONCLUSIVE; terminal
verdicts flip status, emit an experiment_verdict event, and Telegram-ping ONCE
(terminal experiments are never re-adjudicated). Metric vocabulary covers
expR_delta/rate_1r plus pm1's median_up_60m/median_dd_60m/big_gap_rate_1r.
backfill_pm1 bridges pm1 from outcome_labels JOIN setup_events (variant +/-1,
baseline NULL). premarket_eval dual-writes the verdict through the harness while
keeping the legacy TSV. 8 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:47:02 -04:00
momentum devandClaude Fable 5 6e9c52fb87 experiment-os: replay engine + fidelity floors (+tests)
Plan 006 step 2. Reconstructs fires from the event stream (submit[buy] -> fill
-> close), population-filters via strategy/population.py, and applies three pure
interventions counterfactually on the bar path: time_stop, probe_add, asym_exit
(the last through strategy.exits.simulate_exit ONLY). Hard fidelity: +1-tick
adverse fills on entry AND stop, spread_proxy = median quiet-bar high-low, stop
floor = max(atr_frac*ATR14, spread_mult*spread_proxy). rank_one_bullet is
receipts-only and starves (documented) without plan-005 rank. Idempotent via PK
+ ON CONFLICT. Anti-slippage-artifact regression (test #2) green: a
zero-slippage-profitable path LOSES under adverse fills.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:43:45 -04:00
momentum devandClaude Fable 5 c835861c81 experiment-os: registry + CLI (register/list/get, immutable)
Plan 006 step 1-2. registry.apply_schema honors PG_SKIP_SCHEMA; register()
validates required keys incl. fidelity floors and refuses to overwrite an
existing id (pre-registration is immutable). CLI: python -m research.experiments
{register|run|adjudicate} with deferred imports. 6 registry tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:37:27 -04:00
momentum devandClaude Fable 5 e624352328 experiment-os: schema DDL (experiments/runs/verdicts tables)
Plan 006 step 1. Idempotent, applied by the harness on first run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:37:27 -04:00
momentum devandClaude Fable 5 24c7330e58 plans: 005 entry-path unification -> DONE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:26:26 -04:00
momentum devandClaude Fable 5 07ec5a3eb8 entry-gate: drop the now-false rank-concentration boot warning (plan 005 step 4)
Rank concentration now runs on BOTH paths via the shared gate, so the warning
that it was 'enforced ONLY on the dead fast-trigger path' is false. Removed it
and folded rank top-N state into the [boot] entry-gates summary line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:20:56 -04:00
momentum devandClaude Fable 5 b6a4040e45 entry-gate: decision receipts per evaluation (plan 005 step 3)
New EntryGateEvaluatedEvent (symbol, source, passed, reject_reason, gates map)
emitted once per _evaluate_entry_gates decision on both paths, behind
TRADING_GATE_RECEIPTS (default on; dataclass default off so unit tests opt in).
Added to the 14-day nightly purge tuple (high-volume diagnostic) and .env.example.
Turns 'why didn't it trade X?' into a query and is the substrate plan 006 replays.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:20:10 -04:00
momentum devandClaude Fable 5 d10b5cc921 entry-gate: extract shared _evaluate_entry_gates for both paths (plan 005 step 2)
One ordered gate sequence (session, confirmation-speed, population, VWAP,
quality, concentrate-by-rank, daily cap [fast-only], dedup/cooldown/reentry,
dilution veto, anti-chase) that BOTH request_approvals_for_ready_signals and
submit_breakout_now call. Gate bodies lifted verbatim from whichever path
owned them; slow-path risk events preserved source-guarded so the fast path
gains no new events. The two documented asymmetries now close:
  - fast path routes through the VWAP gate (was ungated),
  - slow path honours concentrate-by-rank via a rank_map threaded from the
    trigger book into tick()/request_approvals (no-op at concentrate_top_n=0,
    which is the production config — STOP #3 does not apply).

Price-sanity/entry_ref coercion stays path-native (different data sources);
slow-path dedup pre-filter kept first so per-tick re-scans don't re-emit.
16 characterization tests pin both paths; the only decision diffs vs
pre-extraction are the two closures above.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:17:56 -04:00
momentum devandClaude Fable 5 20e6efc248 entry-gate: characterization matrix pinning both paths pre-extraction (plan 005 step 1)
Pins today's slow-path (approval/event) and fast-path (skipped string)
decisions across single-condition signals, incl. the two asymmetries that
step 2 will close (fast has no VWAP gate; slow has no rank gate). Green
against unmodified trading_execution.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:03:27 -04:00
momentum devandClaude Fable 5 124d816cb9 plans: 004 learning flywheel -> DONE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:49:58 -04:00
momentum devandClaude Fable 5 5518625bbb population test: clear env for the bare-shell from_env case
from_env merges os.environ, and importing the .env-loading consumers populates
TRADING_MIN_ENTRY_GAP_PCT — so the "bare shell disables the gap gate" assertion
must delenv the key first. Test-only fix; the module behaviour is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:48:17 -04:00
momentum devandClaude Fable 5 3e2d690b21 flywheel: schedule eod_replay + nightly_tune in both compose scheduler blocks
The tuner was fiction in production: the 20:00 ET scheduler ran only
premarket_eval (+ eod_reconcile), so learned_params.json sat 26 days stale and
the whole-market dataset never grew. Both scheduler loops now run, in order,
after the eval/reconcile pass:

  python eod_replay.py    || echo "[scheduler] eod_replay FAILED"
  python nightly_tune.py  || echo "[scheduler] nightly_tune FAILED"

Order matters: replay grows the dataset first, the tuner fits after. 20:00 ET is
after close, so replaying "today" is complete data — NOT look-ahead (replay is a
labeling job, not a live gate). Deletes eod_replay.sh (dead cron-era wrapper
pointing at a nonexistent venv; compose is the runner).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:46:06 -04:00
momentum devandClaude Fable 5 b737acfc2c flywheel: atomic learned_params write + boot-time staleness warning
- nightly_tune: write learned_params.json via tmp -> flush -> fsync ->
  os.replace so a crash mid-dump can't leave the live loop parsing half a JSON
  on its next boot (it reads this file on every restart).
- run_live_paper._load_learned: warns (boot log + one gated Telegram note) when
  the file is older than LEARNED_PARAMS_MAX_AGE_DAYS (default 3) — the missing
  signal that let the params sit 26 days stale. Loads anyway; never blocks boot.
  Staleness math factored into _learned_params_age_days (naive ts = UTC).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:45:04 -04:00
momentum devandClaude Fable 5 61ca8ae096 population: learn_params + eod_replay consume the shared gate (kills look-ahead)
Both offline consumers now screen through strategy.population.population_reject
instead of divergent, look-ahead admits:

- learn_params.load_sessions: replaces the `gap>=0.03 OR day_rvol>=2.0` admit,
  whose day_rvol was a FULL-DAY volume sum (outcome-time look-ahead, the
  2026-07-08 class). gap is already a FRACTION (open vs prev close) — passed
  as-is; the OPEN price is passed for the price band. Stored daily bars expose
  no open-knowable volume, so volume=None (fail-open on the day-volume floor) —
  no day-end volume is smuggled into admission.
- eod_replay._daily_gappers: screens on the OPEN price + STATIC open gap (was
  t_close look-ahead) via the shared gate; the local gap is converted to a
  FRACTION before the call, so it inherits the >=150% squeeze exception. Deletes
  the stale $1-5 / 3-60% WATCHER_/TRIGGER_ constants (KEEPS RISK_PCT). The
  second-stage scan_gappers is made permissive (membership already decided by
  the shared gate), removing the `gap_pct <= 60%` cap that wrongly dropped
  squeezes. Prod thresholds passed explicitly (bare from_env would disable gap).

Unit test replaces the live-DB smoke run: $15/+12% qualifies, $4/+45% rejected
unless >=150%+rvol5. research/labeler.py deferred to plan 006 (no locatable
price/gap admission screen); sweep_exits/sweep_filters likewise deferred.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:43:12 -04:00
momentum devandClaude Fable 5 66e81a48f3 population: extract strategy/population.py; live gate delegates to it
The tradeable population is now defined ONCE. trading_execution._population_reject
becomes a thin wrapper building a PopulationConfig from its settings and calling
strategy.population.population_reject — logic moved verbatim (dead-band 0.35/1.50
and squeeze rvol 5.0 are module CONSTANTS, gap floor / price band / volume floor
from config). Behaviour byte-identical: the pre-extraction characterization suite
stays green through the new public seam.

from_env mirrors ExecutionSettings.from_env (same 4 keys + the [0,1] _qfloat clamp)
so a live-shaped shell reconstructs the identical gate. gap_pct FRACTION contract
and the as_of open-knowable contract are documented for offline consumers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:36:59 -04:00
momentum devandClaude Fable 5 256481910f population: characterization tests pinning live _population_reject (pre-extraction)
11 table-driven cases pin the CURRENT behaviour of the live population gate
before plan 004 extracts it to strategy/population.py: in-band pass, below-gap,
dead band, squeeze pass/fail, both price-band sides, volume floor, missing-data
fail-open, the gap-is-a-FRACTION unit pin, and the gap-gate-disabled nesting.
These must stay green byte-for-byte across the extraction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:32:03 -04:00
momentum devandClaude Fable 5 23b3d01a00 money-path: nightly cage-count reconcile (broker vs event store) + schedule it
Plan 003 step 5 (completes plan 003). research/eod_reconcile.py diffs broker
truth (open positions, today's live orders) against event-store truth (today's
submits/fills/closes) and emits a loud eod_reconcile event + Telegram alert on
any drift — the duplicate-resting-order-after-restart class (2026-07-06) caught
automatically instead of by hand. Read-only (reports, never auto-fixes). Wired
into both scheduler blocks after premarket_eval. Pure diff core is unit-tested.

Suite: 373 passed (+22 plan-003 tests), 15 fail = WSL-loopback dashboard only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:29:00 -04:00
momentum devandClaude Fable 5 119e4c0ad1 money-path: reconcile a lost order-submit by client_order_id before journaling cancel
Plan 003 step 4. A transport error AFTER the POST landed leaves a live order
resting at the broker; journaling it 'cancelled' breeds the duplicate/naked
order (2026-07-06 incident). Add client.get_order_by_client_order_id + an
after= passthrough on get_orders; on the generic-exception path, look up the
order by the client_order_id we already send and, if live, emit the normal
submitted sequence (ok=True) instead of cancelled. AlpacaApiError (real
rejection) is NOT reconciled. Fills the empty test_alpaca_client.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:24:14 -04:00
momentum devandClaude Fable 5 d16eb85ee9 money-path: daily self-restart + SIGTERM + positive-evidence exit gate
Plan 003 steps 1-3.
- Loop exit(0) after a SUCCESSFUL EOD flatten (TRADING_EXIT_AFTER_EOD_FLATTEN,
  default on); compose restarts fresh. Restart-churn guard: _rebuild_daily_state
  detects today's eod_flatten event and seeds _session_closed/_eod_done so a
  post-flatten restart neither re-flattens nor re-exits. SIGTERM = clean drain.
- Exit manager: _FETCH_FAILED sentinel distinguishes a failed get_orders from a
  genuinely-empty snapshot. A failed fetch skips naked-stop enforcement (catastrophe
  still exits, re-fetching); an empty-but-successful snapshot needs TWO consecutive
  reads before 'naked' is trusted (Alpaca silently returns [] for symbols=). Costs
  ~one 12s pass on the all-naked case only; isolated naked positions unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 22:20:37 -04:00
momentum devandClaude Fable 5 af8137dfd7 delete: dead DDL (10 tables), dead projections, pinned dead code; README/pyproject prune
pg_schema.sql: drop CREATE TABLE for 10 zero-reference tables (operator DROP
list in plans/002-deletions.log — production tables untouched until run).
projections.py: remove 4 zero-consumer query functions.
Clear all 'plan 002' ruff pins from the CI gate (files now F-clean);
api/main.py:F821 pin remains for plan 007. README repo-map/test-count fixed;
pydantic-settings removed (zero imports). Full receipts: plans/002-deletions.log

Suite: 351 passed (16 schwab/dead-module tests removed), 15 failures = the
known WSL-loopback dashboard set only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:09:35 -04:00
momentum devandClaude Fable 5 cf431c6c2b delete: standalone grafana/ stack, telemetry/ husk, dead workflow, stale uv.lock
grafana/ pointed at the decommissioned pre-migration DB with committed dev
credentials (rotation note in plans/002-deletions.log); its dashboards
provisioning MOVED to deploy/compose/grafana/dashboards (docker-compose.yml
bind-mount re-pointed — the in-compose grafana service keeps working).
docker-publish.yml fired on a nonexistent branch pushing a wrong image name.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:02:46 -04:00
momentum devandClaude Fable 5 99ff1dc51f delete: schwab/ (23 files, dormant) + dead strategy modules
Finishes what d54a2a9's message claimed: schwab/ was never actually removed.
Also: comprehensive_backtest.py (crashed on import — research/connection.py
does not exist), strategy/universe.py, strategy/backtest/outcomes.py,
detect_momentum_ignition (kept _last_higher_low — imported by runner/
premarket_setup/liquid_breakout), milestone3_verify.py, tests/unit/schwab/,
SCHWAB_* keys from .env.example. All grep-gated: zero live references.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:02:46 -04:00
momentum devandClaude Fable 5 be35385ffc test nit: unused var in deadman ping test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:44:54 -04:00
momentum devandClaude Fable 5 0cb74c4c3b plans: 001 production safety rails -> DONE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:43:24 -04:00
momentum devandClaude Fable 5 caee1d5655 backup: nightly pg_dump service on the postgres:18 image
Production Postgres had NO automated backup (pg_dump appeared only in the
one-time migration script). New db-backup service in both compose files fires
20:10 ET Mon-Fri: pg_dump | gzip with pipefail + gunzip -t + tmp-then-rename
(a mid-stream death can't masquerade as a backup), prunes past
BACKUP_KEEP_DAYS, writes last_backup_ok on success. postgres:18 image =
version-matched client (app image has none). TrueNAS host is pull-only, so
its copy inlines the script. Restore: gunzip -c dump | psql $DATABASE_URL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:39:26 -04:00
momentum devandClaude Fable 5 5b95a9712c qabot: dead-man's switch on the loop's event heartbeat
The loop's Telegram notifier runs inside the loop process — loop-down means
alerts-down. The always-up qabot now reads max(events.timestamp) every ~120s
(ET-naive vs ET wall clock per RUNBOOK): stale > DEADMAN_STALE_MIN during
market hours (holiday-aware via runtime.market_calendar) alerts at most every
30 min; DEADMAN_DAILY_PING sends one OK-ping after 20:30 ET. Injected
clock/read/send; 6 unit tests, no network or DB.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:37:01 -04:00
momentum devandClaude Fable 5 e882658ecb ci: gate the image push on the test suite + scoped ruff F check
The registry image deploys to production on pull with zero test run today.
Add a test job (postgres:18 service, python 3.13 to match the Dockerfile)
that build now needs. ruff F gate is scoped to the money path with
pre-existing errors pinned per file+rule (owners: plans 002/007) so new
F errors still fail the push.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:34:32 -04:00
momentum devandClaude Fable 5 653f13a7d9 milestone 2026-07: plans 001-007 + ideation artifacts + architecture diagrams
Recon (8 clusters) -> ideation (48->14->7, all verified) -> plans, cold-read
verified by 8 fresh-context reviewers, 33 plan defects fixed pre-execution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:28:07 -04:00
momentum devandClaude Opus 4.8 d54a2a930c Review sweep: money-path fixes + security + delete 805MB dead weight
Consolidated 4-tool review (2 correctness agents + Codex + over-eng audit).

Money-path (regression-tested):
- _flatten_all: moved the fresh-broker close from close_session INTO the
  shared function + added a broker-held guard on entry cancellation, so
  the daily-loss CIRCUIT BREAKER (which called _flatten_all raw) no
  longer strips a just-filled bracket's legs or leaves a sync-gap fill
  naked. One guard, all callers.
- restart state rebuild: _fresh_entries cap, _requested_symbols dedup,
  and losing benches now rebuild from today's events on boot — a
  mid-session restart no longer resets the cap to 0 or re-fires a
  resting order into a duplicate (the 2026-07-06 incident).
- POST /orders no longer retries a lost-response URLError (was
  re-sending duplicate live orders); GET/DELETE still retry.
- expire_stale_entries now @_locked (raced the trigger thread).
- _session_open: ET conversion instead of a hardcoded 13:30-13:33Z
  window that only matched EDT — the fast path armed nothing all winter.

Security:
- admin panel fail-CLOSED on empty ADMIN_TOKEN (was fail-open on a panel
  fronting the docker socket); header-only token, constant-time compare.
- admin API responses redact secret override VALUES (live keys would
  have echoed after a paper->live switch).
- dashboard: removed wildcard CORS (cross-origin drive-by could drive
  trade actions); optional DASHBOARD_TOKEN gate on POST.

Deletions (~805MB + dead code shipped in the prod image): momentum-
rebuilt.zip, archive venv/htmlcov, and schwab/ core/ telemetry/ models/
runtime/providers/ storage/replay.py controlroom/ + three dead
evaluation stubs + package.json — all verified zero-importer.

+4 regression tests. 99 money-path/trigger/exit tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 13:00:10 -04:00
momentum devandClaude Opus 4.8 cad9051a96 truenas template: complete 86-key env block (kills env-drift class)
Systematic diff of the old .env against the deployed loop env found 68
code-read keys running on silent defaults — including the risk cap
($500 vs $868 observed), the VWAP gate, the exit ladder, and the
NEWS_ENRICH_ENABLED flag that kept the catalyst engine dead despite
OLLAMA_ENABLED=true (enrichment requires BOTH, config.py:1235 gate).
Template env block regenerated with the full inventory, secrets as
<FILL-ME>; a ready-to-paste block with real values generated locally
(data/loop-env-block.yaml, gitignored).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 12:26:25 -04:00
momentum devandClaude Opus 4.8 5ac2cc7b89 Docs cleanup: current-state rewrite + three deployment modes + history/
- 10 era documents (phase plans, build summaries, redesign writeups, old
  runbooks, completed consolidation plan) moved to docs/history/ with a
  do-not-follow README.
- Root README: current validated population ($2-20, 10-35% static gap +
  squeeze exception, SIP), Deployment section, automated-maintenance
  reality, corrected repo map (admin panel, youtube_claims, deploy/).
- RUNBOOK rewritten for the running system: daily rhythm, health checks
  per deployment, and the production failure-mode table earned this week
  (env drift, pgdata mount traps, uid 10001, zombie approvals, PG_SKIP_
  SCHEMA, timezone artifacts).
- deploy/README.md is now the three-mode index: compose (canonical),
  systemd (local dev — unit templates now committed), helm (refresh
  staged as issue #8). Old k8s doc moved to deploy/helm/README.md.
- docker/README.md: six TrueNAS gotchas, each from a real outage.
- Admin panel improvements staged as GitHub issues #4-#7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 10:56:53 -04:00
momentum devandClaude Opus 4.8 d4c695e213 Session-hours gate on fresh entries (the after-hours submission hole)
2026-07-09 evening: the loop submitted three real resting entries at
18:53-20:09 ET (TDTH x2, SUNE). Root cause: NO market-phase gate ever
existed on either entry path — SIP prints extended-hours trades, so
after-hours price crosses fire the trigger book and evening-fresh
signals sail through the approve path. The hole predates the container
move; Thursday was merely the first evening conditions aligned.

_outside_session() (ET wall clock via zoneinfo, weekend-aware) now bars
fresh entries outside TRADING_SESSION_START..END (default 09:30-15:55)
on both submit_breakout_now (skipped: outside_session) and the approve
path (quiet skip). Exits and management remain ungated. Dataclass
default OFF so directly-constructed test settings are unaffected;
from_env defaults ON so every deployed config gets the gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 07:50:35 -04:00
momentum devandClaude Opus 4.8 53f9a50aca admin: degrade gracefully when data volume is not writable (uid 10001)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:50:13 -04:00
momentum devandClaude Opus 4.8 befcec2965 Post-migration DB health: healthcheck DDL storm + broken-pipe fixes
The loop healthcheck opened a fresh connection every 60s with schema
auto-apply enabled; the CREATE-IF-NOT-EXISTS lock bursts cancelled
minute_bars autovacuum every single cycle (visible as 'canceling
autovacuum task' each minute post-migration), so the 19M-row table
never got analyzed, snapshot queries crawled, browsers timed out, and
the dashboard log filled with double BrokenPipeError tracebacks.

- healthchecks now set PG_SKIP_SCHEMA=1 (both compose files)
- api/main.py _send_json swallows BrokenPipe/ConnectionReset (client
  hung up; nothing to tell it)
- live DB: ALTER SYSTEM max_wal_size=4GB, shared_buffers=1GB,
  effective_cache_size=3GB, checkpoint_completion_target=0.9 (the
  checkpoint-every-24s warning under migration load on NAS storage)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:41:12 -04:00
momentum devandClaude Opus 4.8 f0fb91a069 Admin panel (:8767): config, profiles, service control, paper/live switch
New admin service — LAN-only, token-authed (ADMIN_TOKEN), card-based UI
matching the owner's preferred dashboard style. Capabilities:
- health: docker containers via /var/run/docker.sock (stdlib unix-socket
  HTTP client), DB heartbeat, Alpaca account/clock, Ollama, feed
- config: whitelisted strategy knobs (TRADING_/TRIGGER_/WATCHER_/...)
  edited via data/overrides.env, which every entrypoint now loads with
  override=True at boot (runtime/env_overrides.py) — so changes apply on
  container restart without touching the deployment YAML; secrets are
  presence-only and rejected server-side if edited
- profiles: named knob presets (seeded: validated-core, conservative,
  squeeze-hunter) applied in one click
- restart buttons per container; paper/live switch guarded behind
  ALPACA_LIVE_* keys existing
Verified live in docker: auth 401s, status/config/profiles/write paths,
secret-edit rejection, page serves. Added to both compose files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:37:18 -04:00
momentum devandClaude Opus 4.8 e7a91168b1 Float cache (FMP) + SEC EDGAR dilution filings — shadow data feeds
Two new nightly enrichments for the day's tracked names, both SHADOW
ONLY (float was refuted as a gate; a filings-based dilution veto needs
its own pre-registered bar):
- research/ingestion/float_cache.py: FMP stable shares-float ->
  float_cache table (free-plan budget guards: 7-day refresh, 200
  calls/run). First real float data in the system — LHSW's +346% ran
  on a 1.1M float; BJDX is 0.4M.
- research/ingestion/sec_dilution.py: SEC EDGAR full-text search (free,
  keyless, fair-use paced) -> populates the previously-empty
  dilution_events table with S-1/S-3/424B filings; TVRD + BJDX flag
  immediately. recent_dilution() helper for future veto work.
Wired into premarket_eval nightly. Not wired: Polygon/AlphaVantage/
TwelveData/Marketstack/EODHD (redundant vs paid Alpaca SIP) and the
generic news aggregators (slower than Alpaca/Benzinga, dedup noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:26:28 -04:00
momentum devandClaude Opus 4.8 6ce301d043 requirements: add ad-hoc venv deps the image was missing
apscheduler + youtube-transcript-api (transcripts service crash-looped
on TrueNAS: ModuleNotFoundError), rich + typer (momentum_cli journal
job). Found via AST sweep of all third-party imports vs the image
contents. Image rebuilt, import-verified, pushed to both registries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 16:28:30 -04:00
momentum devandClaude Opus 4.8 f4198cda5c Publish image to Docker Hub too; TrueNAS yaml pulls from docker.io
CI now pushes fez5587/momentum-app to Docker Hub alongside GHCR (creds
via repo secrets). First push done manually and verified anonymously
pullable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 15:34:13 -04:00
momentum devandClaude Opus 4.8 7427d14301 TrueNAS migration: dataset paths, 5433 coexistence port, dump/restore script
truenas-compose.yml now uses /mnt/volume/Apps/momentum and publishes the
new postgres on 5433 so it can coexist with the existing instance during
migration. deploy/docker/migrate_db.sh: dump / restore / verify steps
(pg_dump -Fc read-only source, parallel pg_restore, ANALYZE, row-count
diff) with env-overridable URLs and PG18 client autodetection —
connectivity smoke-tested against the live DB.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:21:21 -04:00
momentum devandClaude Opus 4.8 e563380f28 TrueNAS SCALE deployment: GHCR image publishing + pull-only compose
TrueNAS custom apps cannot build: the main compose's 'build: .' (plus
relative .env/bind mounts) fails app 'up'. Added a GitHub Actions
workflow publishing ghcr.io/fez5587/momentum-app on every master push,
and deploy/docker/truenas-compose.yml — a pull-only variant with
dataset host-paths, inline environment (secrets as <FILL-ME>
placeholders, current validated strategy knobs pre-filled), and the
same healthchecks. Validated with docker compose config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:55:24 -04:00
momentum devandClaude Opus 4.8 8ba4899108 Sanitized .env.example regenerated + config-layering guide
.env.example was stale (pre-migration keys, missing all current
TRADING_*/TRIGGER_* knobs). Regenerated from the live config with
secrets <REDACTED>; a leak check caught live Alpaca credentials
embedded in commented-out PowerShell lines and scrubbed them before
commit. README now documents the three config layers (protected
secrets vs promotable strategy knobs vs infra addresses) and the
external-Ollama networking expectations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:41:40 -04:00
momentum devandClaude Opus 4.8 8864041066 Docker stack modernized: full service set, PG 18, validated end-to-end
Brings the pre-migration compose file up to the current architecture:
python 3.13 image with ffmpeg + faster-whisper/yt-dlp (transcripts),
postgres 16 -> 18 (prod parity), and the real service set — loop
(--no-dashboard, heartbeat healthcheck), dashboard_api.py on :8765,
telegram qabot, youtube transcripts, and a 20:00-ET weekday scheduler
replacing the premarket-eval systemd timer. Jobs (journal/eod-replay/
nightly-tune) and Grafana keep their profiles. Self-contained by
default (in-stack DATABASE_URL); deploy/docker/README.md documents the
pg_dump migration, the transition mode against the LAN DB, and the
cutover checklist. Validated: image builds, compose config passes, and
a smoke container served the dashboard (HTTP 200) against the live
LAN Postgres.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:06:27 -04:00
momentum devandClaude Opus 4.8 ff3a4b46fb Selection refinements from clean-universe replay of the trader's criteria
Tested the source trader's numeric selection playbook on the unbiased
(open-knowable) year universe, n=2,643 point-in-time ORB entries:
- Dead band RE-CONFIRMED on clean data (35-75% +0.006, 75-150% -0.066)
  — CLRO/VTAK were anecdotes; band exclusion + squeeze exception stand.
- His $5-10 price sweet spot inverts for our system: edge RISES with
  price ($10-20 best at +0.143); the $1-2 bucket is negative (-0.031,
  n=609). SHIPPED: price floor $1->$2 (population gate band now
  env-driven TRADING_PRICE_MIN/MAX; watcher floor raised to match).
- His rvol>=5 floor is backwards for us: rvol 2-5x at entry +0.346R vs
  >=10x -0.047 (n=818) — crowded spotlight names hurt our entries.
  NOT shipped (conflicts with the squeeze exception's rvol needs;
  logged for forward validation via the new per-signal relative_volume).
- SHIPPED: buyout/merger headlines never rank-boost (price is pegged —
  the CRNX class); keyword suppression in the catalyst multiplier.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 18:09:54 -04:00
momentum devandClaude Opus 4.8 722add6815 Ingest: per-symbol session backfill; never arm without the true open
Evening audit of 2026-07-08's post-fix entries (TVRD +4% true gap,
FRTT — both fired as "20% gappers"): the morning's static-gap fix was
defeated by a data hole. ingest_live_minute_bars' full-session first
pass was a GLOBAL one-shot, so symbols discovered mid-session only ever
received the 20-minute refresh window — their earliest known bar was a
mid-day bar, which became both their "session open" (fake overnight
gap) and the base of their "opening range" (fake ORB trigger). Exactly
the failure class the population gate exists to prevent.

Fixes: (1) step_ingest tracks per-symbol first sight — new names get
the full-session lookback, known names the incremental refresh;
(2) _session_open only trusts a first-RTH bar stamped <= 09:33 ET and
is the sole source for the static gap; (3) if the session is underway
and a candidate's true open is not yet ingested, it cannot arm (trigger
withheld until the backfill lands, typically the next pass).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 16:49:09 -04:00
momentum devandClaude Opus 4.8 44ff36a669 Population gate audit: static overnight gap everywhere + full band
The 2026-07-08 audit (owner: "figure out if there is some kind of logic
or audit issue") found BOTH entry paths feeding the population gate the
WRONG gap. The fast path used the trigger book's dynamic gap (latest
price vs prev close), so intraday rallies masqueraded as gappers: TVRD
(6.0% true gap), SKIN (2.2%, filled at $0.98), JLHL (4.3%) all traded
outside the validated population. The slow path used max(gap, intraday)
with "open" taken from the first lookback-window bar, and had no upper
band at all — dead-band names SEER (38.6%) and IOTR (40.7%) entered.

Fixes: candidates carry gap_open_pct (session open vs prev close);
trigger eligibility + submit_breakout_now use it exclusively once the
session is open; _population_reject now implements the full validated
band (10-35% OR volume-verified >=150% squeeze, $1-20 price band) for
both paths; evaluator gap uses the session open; signals carry
relative_volume for squeeze verification. Plus momentum-testpg.service
so the test-isolation Postgres survives reboots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:34:46 -04:00
momentum devandClaude Opus 4.8 2e5346e035 Trigger book: mega-gapper squeeze exception (the LHSW/NXTS fix)
LHSW gapped +278% on 100M shares, was rank 2 in the trigger book at
4:49am, ran +346%, and was the source trader's entire day — the bot's
TRIGGER_GAP_MAX=35 capped it out, same as NXTS two weeks ago, while the
bot spent its entries on the shadow round-trips (SEER/ELAB/PLUG/YRD).

Year-scale replay (n=2,879 ORB entries, 14 months, reverse-split
artifacts excluded by requiring a real volume surge): the 35-150% band
is dead (+0.02R / +0.00R — the cap earns its keep there), but REAL
>150% squeezes (n=82) ran +0.63R with 8/14 months >= 0, 4x the core
band's edge. New gap_squeeze_min/gap_squeeze_rvol re-admit gaps >=150%
only when rvol >= 5 (split repricings show rvol ~0.01 and stay out).
Live: TRIGGER_GAP_SQUEEZE_MIN=150, TRIGGER_GAP_SQUEEZE_RVOL=5.0.
Existing guards (trigger/day extension, halt, BE@+5% ratchet, sizing)
still apply to these names.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:13:04 -04:00
momentum devandClaude Opus 4.8 842b54c802 Exit manager: never manage a fill with zero post-entry bars
Live 2026-07-06: every fresh entry was market-exited seconds after
filling (ELAB round-tripped in 3s, SEER in 77s, PLUG in 85s) — all
logged as exit_trail. Root cause: seconds after a fill there is no
completed minute bar with timestamp >= the fill time, and _bars_since's
empty result fell back to the WHOLE DAY's bars, so manage_live's
high-water included the pre-entry HOD and the trail read as already
breached on any name trading off its high. The whole-day fallback is
removed: with no post-entry bars the manager now leaves the position
alone (the bracket's initial stop is the protection) until the first
post-entry bar lands. Regression test verified to fail on the old code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 12:25:06 -04:00
momentum devandClaude Opus 4.8 857c64ea48 Local-data cleanup: archive dead DuckDB relics, rotate logs, fix labels
Everything hot already lives in Postgres — local data/ was 25 MB of
pre-migration relics and unbounded logs. Moved the two dead .duckdb
files (untouched since 2026-06-16) to archive/, added nightly log
rotation (copy-truncate >20 MB, safe with systemd append mode), and
updated the last vestigial "market.duckdb" docstrings to say Postgres,
closing the CONSOLIDATION_PLAN cleanup items.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 15:26:40 -04:00
momentum devandClaude Opus 4.8 bbbcc4e18b Nightly events retention: purge observability-spam events >14 days
The events table hit 3.5 GB / 1.8M rows, ~80% of it module_tick /
criteria_evaluated / signal_blocked / account_* snapshots that are only
useful for a few days of debugging. Trading-decision events are kept
forever. One-time purge (353k rows) + missing hot-path indexes
(raw_news_items payload_hash + fetched_at: 911k seq scans -> index-only)
and per-table autovacuum tuning applied directly to the DB.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 13:57:52 -04:00
momentum devandClaude Opus 4.8 20695d5e87 Control room: autonomous-ops redesign (owner directive: zero-interaction)
The owner runs this system fully autonomously — no approvals, no manual actions. The
control room is therefore a WATCHING instrument, not a steering one:

- Approvals tab REMOVED (vestigial: the loop auto-approves; interaction contradicts
  the operating model). REST plumbing dropped.
- ACTIVITY tab added: merged live event feed (entries, fills, exits w/ realized P&L,
  gate cuts, ready signals) refreshing every 4s — see the autonomous system act.
- TODAY summary strip on the Live tab: Day P&L (broker last_equity based, colored),
  trades + win rate, submissions, gate cuts, and LOOP HEARTBEAT age (green <3min,
  red otherwise) — at-a-glance trust that the unattended system is alive.
- Thread-safety fix: all fetchers run in io_bound THREADS but shared one psycopg2
  connection — now serialized behind DB_LOCK (latent race, caught during redesign).
- Panic brake stays (emergency-only, typed confirmation) — the one deliberate
  exception to zero-interaction.

Headless-verified: cards live (Day P&L, Loop 0s green), 10 ticker rows, 40-row
activity feed, zero JS errors. Service restarted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 10:06:18 -04:00
momentum devandClaude Opus 4.8 90b4aae1b8 Trading Control Room: NiceGUI live-analysis dashboard (Milestone 5 core, :8766)
Phases 1+2 of the staged control-room plan, adapted to this backend and shipped as a
PARALLEL service so the battle-tested :8765 dashboard keeps running until parity.

The owner's ask — live analysis with live tickers, realtime — is the centerpiece:
- LIVE tab: tracked/armed tickers grid with real-time SIP last-trade prices (1s,
  flash-green/red on change), state, gap%, RVOL, trigger + distance, stop, quality
  score, POPULATION-gate check (display mirror of the live gap/volume floors), VWAP,
  catalyst — merged from the loop's trigger snapshots + ready-signal projections.
- Header: PAPER mode badge (chameleon pattern), market phase, live equity, heartbeat,
  clock, and the PANIC brake (cancel ALL open orders via broker API, typed "STOP"
  confirmation — the plan's conditional-friction flow).
- Positions: live broker positions w/ uP&L colors + naked-stop flag (2s).
- Gates & Attribution: today's gate-cut histogram (5s) + 7-day per-stage attribution.
- Approvals: pending queue w/ approve/reject via the existing dashboard REST + dialogs.
All DB/broker calls via run.io_bound (never block the event loop); dark theme +
semantic tokens per the plan. systemd momentum-controlroom (:8766), unit in
deploy/systemd/. Headless-verified: live header (equity, phase), 10 ticker rows
rendered, zero JS errors.

Remaining from the staged plan (next phases): manual orders, settings dials, audit
tab, connection-lost action guard, live-mode chameleon when a real-money broker lands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 09:51:52 -04:00
momentum devandClaude Opus 4.8 c72bb4e43b Population gate: trade only real gappers — the year-long +EV boundary
The gapper-history reconstruction (n=3,008 symbol-days / 14 months) REFUTED the 15-min
confirmation gate (late crosses beat early 12/14 months incl June 2026 — the 19-day
"+0.42R early" was a definition artifact) and found the first at-scale +EV configuration:
top-20 ranked gappers (gap >= 10%, $1-20, day volume >= 300k) are +0.05..+0.15R under the
live BE@+5% ratchet exits in EVERY rank x timing segment — including rank 6-20 ("D tier",
owner-approved: +0.153R early, the best cell). The live book bled by trading OUTSIDE that
population (low-gap, thin names), not by mistiming.

- ExecutionSettings.min_entry_gap_pct / min_day_volume (TRADING_MIN_ENTRY_GAP_PCT=0.10,
  TRADING_MIN_DAY_VOLUME=300000): POPULATION gate on BOTH entry paths — auto path skips +
  logs rule_type=population_below; fast path returns skipped=population_below. Fail-open
  on missing data (house rule: never block on unknowns).
- evaluate_setup now surfaces the true overnight gap_pct in signal_data; ready-signal
  snapshot passes it through.
- fast path: submit_breakout_now takes gap_pct (trigger book stores % units; call site
  normalizes to fraction); TRIGGER_GAP_MIN raised 3 -> 10 to align the arm floor.
- confirmation gate flipped OFF via env (code stays, 0=disabled).
- rank concentration stays OFF (owner call, data-consistent: rank 6-20 converts fine).

Tests: population skip+log, real gapper passes, fail-open on missing data. 52 pass.
Monday config: population gate + cap 10 fills + quality C+ + VWAP + BE@+5% ratchet + SIP.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 21:04:38 -04:00
momentum devandClaude Opus 4.8 05508e6345 Year-long gapper-history reconstruction (research/gapper_history.py)
Every conclusion so far -- including the LIVE confirmation gate -- rests on 19-26 days of
history; the weakest link in the whole evidence base. The paid plan's 9+ years of SIP data
fixes it surgically: pull ~1yr of DAILY bars for the full listed universe (11,693 symbols
after junk filtering, via new client.get_assets), reconstruct each day's TOP-20 gappers
(gap>=10%, open $1-20, vol>=300k) into historical_gappers, pull SIP minute bars
(premarket included) for JUST those ~4-5k gapper symbol-days, then replay the ORB across
the year: conversion by CONFIRMATION-MINUTE bucket + month-by-month consistency.

PRE-REGISTERED BAR (locked before the data): the gate generalizes if the early(<=15min)
bucket beats late by >= +0.25R expectancy AND wins >= 70% of months. Steps are individually
resumable (minutes skips already-present symbol-days).

Side-effect handled: labeler._preload_daily now scoped to symbols with minute bars (the
whole-market daily universe would balloon the unscoped preload to ~3M dict entries).
Bonus: the historical gapper minute bars (incl premarket) massively grow the pm1 premarket
track's future sample.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 17:44:18 -04:00
momentum devandClaude Opus 4.8 3b692586f2 SIP cutover guards: never double-scale a consolidated feed
The owner bought Alpaca Algo Trader Plus (real-time SIP; entitlement propagating on the
same keys — latest-trade already SIP-live, quotes/bars still 403). Before flipping
ALPACA_DATA_FEED=sip, two guards so the free-tier ratio estimator can never double-count:
- ingest_live_minute_bars resolves the EFFECTIVE feed and refuses volume_scale for
  anything non-IEX (provider marked alpaca_<feed>).
- step_vratio self-disables when client.settings.feed != iex (saves the API calls too).
The estimator stays as the dormant free-tier fallback (plan lapse -> flip env back).
Test: SIP client + x25 ratios -> volumes NOT re-scaled, provider alpaca_sip. 4 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 23:02:13 -04:00
momentum devandClaude Opus 4.8 1210ac8ef6 Ratio-corrected IEX volume: free fix for the 2-5% live undercount (estimator + audit)
With the nightly SIP backfill making all BASELINES consolidated-scale, the still-IEX live
ingest would make Monday's RVOL read iex_intraday/sip_baseline ~= 25-50x too LOW and choke
the scanner. Free-key SIP is only 15-min DELAYED, so each morning the true per-name capture
ratio is directly measurable:

- research/ingestion/volume_ratio.py: compute_ratios fetches BOTH feeds for the identical
  embargo-capped window (never vs our DB bars — those are scaled; that would double-correct);
  guardrails (min samples, clamp [1,300]); persist_ratios records per-day ratios.
- ingest_live_minute_bars(volume_scale=...): scales bar volumes per symbol at ingest, marks
  source_provider=alpaca_iex_scaled. Missing symbol -> 1.0 (conservative: reads LOW, blocks).
  Constant per-symbol scale leaves cumulative VWAP unchanged (cancels in the weighted mean).
- run_live_paper: "vratio" step every 300s (VOLUME_RATIO_ENABLED/INTERVAL) merges fresh
  ratios (a name gone thin keeps its last good ratio); step_ingest applies them.
- premarket_eval nightly audit: snapshots the day's scaled ESTIMATES before the SIP truth
  overwrites, then logs median/worst error to data/volume_estimator_audit.tsv — the
  estimator's accuracy is self-measured daily, which is exactly the evidence that decides
  whether the $99/mo real-time SIP plan is ever needed.

Live check (2026-07-02 full session): ratios are hugely name-dependent — DSY x300(clamped),
CETX x286, SOFI x45, SNDQ x42, NOK x17 — validating per-name daily measurement over any
global constant. 3 pure unit tests (ratio math + thin-sample skip; ingest scaling + provider
marking; no-ratio unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 22:58:01 -04:00
momentum devandClaude Opus 4.8 f8f85b3963 SIP re-backfill: fix the 2-5% IEX volume distortion across the whole dataset
MEASURED 2026-07-02: our entire bar history was IEX-only. IEX sees ~2-5% of consolidated
volume (SNDQ 7/01: IEX 188 bars/1.5M shares vs SIP 391 bars/60.1M — 39x) and misses whole
minutes on thin names. Every volume-based signal (RVOL, liquidity floors, volume bursts,
dollar-volume strata) and several research verdicts ("untradeable dust", "thin premarket")
were computed on that sample. Daily bars had the same defect. Historical SIP is FREE on our
keys (15-min embargo) — only real-time SIP needs the paid plan.

- client.get_daily_bars: feed param (None -> settings.feed; pass "sip" for consolidated).
- research/sip_backfill.py: re-fetch minute sessions (04:00-20:00 ET) + dailies from SIP
  via the existing idempotent upserts; embargo-caps the end when refetching today;
  refresh_rolling_volume rebuild after (this morning's 20d averages were IEX-scaled too).
  CLI: full history < today, or --date D for one session.
- premarket_eval.py: nightly job now SIP-upgrades today's bars BEFORE the pm1 rebuild, so
  every finished session lands on true volume automatically (live loop stays IEX real-time
  by design — free-key SIP is embargoed 15 min).

Chain running: full backfill -> verification -> rebuild liquid/premarket/runner shadow
tracks + re-validate (their volume gates were IEX-calibrated; verdicts may change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 13:56:01 -04:00
momentum devandClaude Opus 4.8 97b318dd6e Quote-size snapshot recorder — free order-book-imbalance proxy (recording only)
Owner's idea from the CETX day: unusually large resting order volume just above/below price
(the trader's "big seller at $6" read) may predict pop-vs-fade. Full L2 depth needs a paid
feed, but Alpaca NBBO quotes carry bid/ask SIZES — a workable free proxy. We stored no quotes,
so the idea was untestable; this makes it accumulate.

- quote_snapshots table (market schema): symbol, ts, bid/ask price + SIZE; (symbol, ts) index.
- step_quotes in the live loop (QUOTE_SNAPSHOT_INTERVAL_SECONDS=15, QUOTE_SNAPSHOT_ENABLED=1):
  records tracked (trigger-book) + held names each pass, premarket included (where the edge
  data lives). Recording ONLY — zero trading impact; fetch failures degrade to a log line.
- Caveat carried in the docstring: free IEX-feed NBBO is thinner than SIP (thin names often
  print 100-share round lots), so persistent imbalance is the signal, not single snapshots.

Verified live: 10 names/pass recording every 15s with real sizes (e.g. RIVN 1300 bid / 3200 ask).
A shadow study tests ask/bid-size imbalance as a pop-vs-fade predictor once data accumulates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 13:24:20 -04:00
momentum devandClaude Opus 4.8 4fc02a8c71 Discovery: merge %-gainer movers (the wildcards volume-ranking can't see)
Forensic from the source trader's 2026-07-02 recap: his ENTIRE day (+$8.9k) was CX — a thin
reverse-split name that squeezed $3->$6 on a headline (his "wildcard effect": on dead pre-holiday
days the one mover gets everyone's attention). Our system NEVER SAW IT: zero bars, zero discovery
events. Root cause: discovery ranks by ABSOLUTE SHARE VOLUME (most-actives), and a thin %-squeezer
never cracks a volume-ranked top-N. Every other name he called out we saw and judged the same way
he did (DSY passed/backed-out, CWD too cheap, AHMA rejected) — the gap was pure discovery scope.

- alpaca_paper/client.py: get_movers(top) — /v1beta1/screener/stocks/movers top %-gainers
  (verified working with our keys: CLRO +114%, CWD +73% live).
- discover_active_symbols: wildcard gainers are fetched FIRST (warrants/units filtered:
  '.'-suffixed + 5-char W-endings), deduped with most-actives, same price-band filter.
  Additive + fail-open: a movers failure never breaks the existing actives path.

Discovery/ingestion scope ONLY — every trading gate (confirmation window, quality, VWAP, cap)
still applies unchanged. The immediate value is that shadow tracks + the labeler finally get
BARS for wildcard movers, so the next CX is measurable instead of invisible.

Verified live: universe now leads with gainer-sourced names (CLRO, CWD, USDE, YRD, DSY). 16 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 13:02:50 -04:00
momentum devandClaude Opus 4.8 6c36d04f7a Component attribution: per-stage scorecard (CLI + /api/attribution + dashboard + board pulse)
The owner's modularity/monitoring ask: rate each pipeline component so what's earning (and
what's cutting) is visible instead of inferred. Everything derives from the existing event
stream — no pipeline changes.

- research/attribution.py: compute_attribution(days) -> per-day per-stage metrics:
  DISCOVERY (symbols found + precision = fraction that ever go ready), SIGNALS (ready split
  EARLY-vs-LATE against the confirmation window — the only +EV ones), GATES (risk_rule
  histogram: what each gate cut), EXECUTION (submissions vs cancels = fill efficiency),
  OUTCOMES (realized P&L, win rate, per-close detail). CLI: python -m research.attribution.
  All queries filter the native timestamp column first (payload casts on the 676k-row events
  table time out).
- api/main.py: GET /api/attribution?days=N.
- dashboard.html: "Component Attribution" panel, lazy-loads on open, per-day table with
  gate-histogram hover.
- momentum_cli board header: live pulse line — today's early/late signal split + top gate
  cuts (two tiny date-scoped queries).

Verified live: CLI scorecard over 7 days, endpoint returns real ratings (e.g. 2026-07-02:
discovery 45%, early 38%, fill 39%, win 50%, P&L +$29), board renders the pulse line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 12:18:57 -04:00
momentum devandClaude Opus 4.8 f2239ccbe6 Backfill + maintain daily_bars.rolling_avg_volume_20d (was NULL for all 89,899 rows)
The column was never populated by upsert_daily_bars, so every downstream consumer
(scan_gappers RVOL, the quality gate's relative_volume component, labeler baselines)
silently fell back to weak within-session proxies — degrading the very signals the
2026-07-02 backtest showed carry the edge.

refresh_rolling_volume(con, symbol=None): window-function UPDATE computing the average
of the PRIOR 20 sessions (excluding the row itself so it's knowable at that day's open,
>=5 prior sessions required). Called per-symbol after each upsert_daily_bars so new rows
stay populated; ran once with symbol=None to backfill.

VERIFIED against prod: 0 -> 79,778/89,899 non-null; spot-check matches an independent
pandas shift(1).rolling(20,min_periods=5).mean() exactly (0 mismatches).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 12:14:08 -04:00
momentum devandClaude Opus 4.8 9bdb891738 Confirmation-speed entry gate — the first backtested POSITIVE-EV filter
Why the book is break-even (proven over 7 refuted levers): the ORB entry converts poorly and
the profitable trades are diluted by the negative majority. The 2026-07-02 quality-gate backtest
(6217 setups, adversarially verified across 3 seeds) isolated where the profit actually lives:
setups whose breakout CONFIRMS within ~10 bars of the open run +1R 71% / +0.42R expectancy —
even C-grade ones — while late-confirming setups are dead regardless of grade (bar-26+: 9% +1R,
-0.81R; time-of-day expectancy decays monotonically to -0.94R late-day). The edge is
confirmation TIMING, not the grade (the B-grade lift was mostly this confound).

ExecutionSettings.entry_confirm_by_minute (env TRADING_ENTRY_CONFIRM_BY_MINUTE, 0=off):
- auto path: request_approvals_for_ready_signals skips signals whose ready-timestamp landed
  more than N minutes after the 09:30 open, shadow-logged rule_type=late_confirmation (visible,
  measurable). Midday re-triggers are late by construction -> also cut (they were -0.75R).
- fast path: submit_breakout_now gates on now() (a live trigger fires in near-real-time, so
  now() IS the confirmation time) -> skipped: late_confirmation. This also closes the fast
  path's no-time-gate hole the all-day analysis flagged.
Deliberately KEEPS min_quality_score at 0.50 (C+): the gate retains the profitable early
C-band (+0.42R) that a B-grade gate would wrongly discard. Fail-open on unparseable timestamps.
4 new tests (late skip + logged, early passes, off-by-default, fast-path block); 50 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 11:36:57 -04:00
momentum devandClaude Opus 4.8 8f6ce2cc04 Daily entry cap: count FILLS not submissions (free slot on back-out)
The behavior half of the cap fix. Root-caused 2026-07-01 and re-confirmed live 2026-07-02
(now visible via the auto_rejected logging): the cap incremented on any broker-ACCEPTED
submission incl. resting limits that never fill, and never decremented on back-out -- so
unfilled/canceled entries permanently burned slots. Both days the bot capped after ~3 real
trades because 2 phantom unfilled limits ate the budget, with good setups still on the board.

Fix: mark each counted entry's armed record ("counted"); when expire_stale_entries backs it
out UNFILLED (timeout, price-break, or broker-cancel), _release_cap_slot decrements the counter.
Fills keep their slot (their armed record is popped without releasing). Net: the cap now means
"N real filled entries/day", not "N submission attempts". New test drives 2 resting entries to
the cap then times them out and asserts both slots free. 46 tests pass.

The cap VALUE (currently 5) and the all-day question are separate (data-driven) decisions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 10:08:42 -04:00
momentum devandClaude Opus 4.8 e5da67c928 YouTube extraction: normalize NULL/spaced tickers + asset-specific filter
Two data-quality fixes surfaced by inspecting the first 94 claims:
- _clean_ticker: coerce the literal 'NULL'/'NONE'/'$AAPL' strings the LLM sometimes returns to
  a real None, and collapse ASR space-mangling ('B VC' -> 'BVC').
- require_asset filter: drop claims with neither a ticker nor an asset_name -- generic trading
  commentary ('buy the dip') from tutorial content is not an asset claim. On the backlog this
  cut 94 -> 58 claims (36 generic), leaving clean per-asset signal.
Added the $2,000 Small Account Challenge recap playlist (the daily per-asset trade breakdowns) --
far higher claim density than the evergreen 'Anatomy of a Trade' tutorials. 7 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 22:16:05 -04:00
momentum devandClaude Opus 4.8 25e822b442 Remove stray systemd unit copy from repo root (canonical is deploy/systemd/)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 18:48:34 -04:00
momentum devandClaude Opus 4.8 b523b962ca Premarket track: scheduled accumulate-then-re-validate automation
The pm1 verdict is promising-but-underpowered (n=21): the forward moves are real (+11-14%) but the
sample is too small to conclude. The live loop ingests premarket bars every trading day, so this
automates the "accumulate more days then re-validate" step:

- research/premarket_eval.py: rebuild pm1 from the grown DB, append a timestamped metrics row
  (n, days, +1R rate, Wilson-95 LB, median upside/adverse, big-gap +1R, pass/fail) to
  data/premarket_eval_history.tsv, and print the current validate output. Watch n climb + the
  Wilson bound tighten until the locked bar is decisively cleared or failed.
- deploy/systemd/momentum-premarket-eval.{service,timer}: weekday-evening (Mon-Fri 20:00) oneshot,
  Persistent=true (catches reboots). Installed to ~/.config/systemd/user and enabled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 18:48:23 -04:00
momentum devandClaude Opus 4.8 7e4925d778 Premarket setup detector + pm1 shadow track — the one unexplored lane
Every RTH selection/entry lever has now been refuted (6 of them separate a population but
don't convert through the entry). The one axis never touched is PREMARKET — the source
trader's actual edge (his words: news drops pre/after-hours, NO circuit-breaker halts pre-market
so moves run clean, and RTH after 9:30 is choppy/unpredictable — the exact window our bot trades).

strategy/evaluation/premarket_setup.py::detect_premarket_setup — pure, operates on PREMARKET bars:
gap vs prev_close + real premarket volume floor (the DXST-vs-CF tell: low pm volume = fail) +
price floor + fresh premarket-high breakout + premarket-VWAP hold. Calibrated to the thin pm tape
(cum pm volume p50 ~12k shares, so a 20k floor catches the genuine movers not 1%-gap noise).
Real-tape validated + SELECTIVE: 6/30 fires JEM(245%)/CELZ(143%); 7/1 fires LHAI(166%)/SOC/SNDQ —
2-3 names/day (~8% of pm names), vs the liquid detector's 50%. 6 unit tests green.

labeler pm1 track (build/report/validate-premarket): slides the detector over premarket bars,
scores forward on PREMARKET bars via the same _forward_labels (the trader enters+exits pre-9:30),
stratifies +1R by gap. Pre-registered bar locked 2026-07-01 (median pm upside >=+10%, adverse >-6%,
+1R >50%, big-gap subset clears). SHADOW-ONLY. Real caveats it sets aside: thin premarket fills, and
live premarket needs extended_hours execution the bot doesn't have. Full 26-day backtest building.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 18:03:53 -04:00
momentum devandClaude Opus 4.8 23ae0b4a28 YouTube pipeline: pass explicit ffmpeg_location (systemd PATH omits /usr/bin)
The systemd user service runs with a minimal PATH that excludes /usr/bin, so yt-dlp's audio
postprocessor reported 'ffprobe and ffmpeg not found' and every video silently fell back to
captions. Pass ffmpeg_location=config.FFMPEG_LOCATION (shutil.which with /usr/bin fallback) to
yt-dlp so the Whisper path works under systemd. Verified: yt-dlp download + ffmpeg extract now
produces a wav. Reset the 4 caption-fallback videos to pending so the backlog gets Whisper.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 15:23:15 -04:00
momentum devandClaude Opus 4.8 26013893d1 YouTube pipeline: harden claim quote-matching; add tests + systemd unit
Extraction fix: the anti-hallucination check (a claim's verbatim_quote must appear in the
source window) was too brittle against auto-captions — fragmented segments joined with
newlines meant a valid quote spanning a join was dropped (0 claims on a 623-segment captions
transcript). Now whitespace-normalized matching (_norm + _quote_supported, ~40-char prefix):
the SAME transcript yields 6 real claims (WETG +160% reverse-split+news; 4 AURC trade claims
with prices; a rejection concern), each quote verifiably present. Whisper output is cleaner
still; this just makes the captions fallback usable too.

- tests/unit/test_youtube_extraction.py: 5 pure-helper tests (norm, quote-support across joins,
  hallucination rejection, verbatim-quote-mandatory drop, chunk time-spans).
- youtube_claims/momentum-transcripts.service: reboot-surviving systemd user unit (Restart=always,
  loads secrets from .env; Whisper option A/CPU, env-flippable to B/GPU).

END-TO-END VERIFIED (captions path, no ffmpeg): detection -> claim -> transcribe -> extract ->
6 claim rows in Postgres. Full Whisper path needs `sudo apt install ffmpeg`, then
`systemctl --user enable --now momentum-transcripts` to process the 11-video backlog at
Whisper quality + poll for new uploads.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 15:08:41 -04:00
momentum devandClaude Opus 4.8 4a54b2adcc YouTube -> transcript -> claim-extraction pipeline (youtube_claims/)
A descriptive ingestion layer feeding the downstream reasoning model: monitors YouTube
playlists, transcribes new uploads, and extracts structured, verifiable CLAIM records
(with provenance + a mandatory verbatim quote). It records WHAT was claimed; it never
judges correctness or tradeability -- all judgment stays downstream, in one place.

Built per the operator's spec. Two clocks coordinated through Postgres (§2): a 30-min
detection poll + a paced concurrency-1 transcription/extraction worker. Reuses the app's
infra: shared Postgres (isolated `transcripts` schema), the app's Ollama for extraction
(free, on-LAN, private -- transcripts never leave the network), the config.py host resolver.

  youtube_claims/schema.sql    playlists / videos / claims (§7-8); claims.verbatim_quote NOT NULL
  youtube_claims/db.py         psycopg2; claim-by-atomic-status-flip under FOR UPDATE SKIP LOCKED
  youtube_claims/detection.py  playlistItems.list over urllib; dedupe by video_id (§5)
  youtube_claims/extraction.py local Ollama -> descriptive claims; drops any claim lacking an
                               exact verbatim quote present in the source window (anti-hallucination)
  youtube_claims/transcription.py yt-dlp -> faster-whisper (medium/cpu/int8, option A;
                               initial_prompt=watchlist) with youtube-transcript-api captions fallback
  youtube_claims/worker.py     paced drain; per-video state machine + retries/backoff
  youtube_claims/run.py        APScheduler poll + worker loop in one process; CLI for setup

VERIFIED LIVE: schema applied; playlist "Anatomy of a Trade" (Ross Cameron) added; poll
ingested all 12 videos (1 API unit); Ollama extraction pulled clean descriptive claims
(TC/DXST/CF) with exact verbatim quotes from a real transcript. Whisper transcription pending
ffmpeg (`apt install ffmpeg`, needs sudo) + the yt-dlp/faster-whisper deps (installing via uv).
Secrets (YOUTUBE_API_KEY) live in gitignored .env only. Whisper is option A (CPU); env-flippable
to option B (LAN GPU large-v3) later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 15:00:09 -04:00
momentum devandClaude Opus 4.8 a7f63a5b11 Liquid breakout shadow track wired into labeler (build/report/validate-liquid)
Mirrors the runner track: slides detect_liquid_intraday_breakout over RTH history, emits
non-tradeable liq1 setups scored by _forward_labels, stratifies +1R by rvol + cumulative
dollar-volume. Pre-registered bar locked 2026-07-01. Early in-sample read (5 days, 1511 fires):
+1R 51% beats ORB 35% on same days, but median 60m upside only +0.6% (tight-stop artifact);
rvol/liquidity strata flat. SHADOW-ONLY. Full 26-day out-of-sample build pending.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 14:10:30 -04:00
momentum devandClaude Opus 4.8 a25e918240 Liquid intraday-RVOL breakout detector (prototype, shadow-only)
The 2026-07-01 forensic + the source trader's day-14 notes point at a thesis-scope gap:
the day's only clean, held, TRADEABLE +2R moves (MSTR, SSPC) were LIQUID names that ignited
intraday with ~zero premarket gap -- invisible to the premarket-gapper universe -- while the
cheap "held trenders" were untradeable dust and the liquid gappers the bot chased whipsawed.

strategy/evaluation/liquid_breakout.py::detect_liquid_intraday_breakout -- a pure detector that
makes LIQUIDITY (cumulative + per-bar dollar-volume) a first-class gate (the thing runner.py
lacked, which let it fire on dust), plus a price floor + fresh-session-high breakout + VWAP hold.
NO gap, NO blue-sky, NO float. Validated on the real tape: fires on MSTR (@bar15) and SSPC (@bar90),
correctly silent on the sub-$2 dust (DXF) and the <$1M single-print names.

Design finding baked in: RVOL is RECORDED, not a hard gate. MSTR/SSPC were NOT within-session
volume surges -- MSTR was liquid from the open (recent/early vol ratio ~0.5-1.0 the whole grind),
so a hard rvol>=3x "surge" gate structurally kills the steadily-liquid grinders this lane exists to
catch. rvol is stored as a stratification feature (the shadow track will measure whether high-rvol
fires convert better, like the runner GRADE); it only gates if rvol_min>0 is passed.

Selectivity is broad (~50% of the liquid universe fires at least once/day) -- this is a candidate
POPULATION definer, not a precise signal. The decisive question (does it convert to +1R better than
the ORB, and does any rvol/liquidity stratum beat it) is for the shadow track + a pre-registered
forward bar. SHADOW-ONLY: never routes to approve_order; 6th selection signal, must prove conversion
after cost before any live gating. See memory thesis-scope-liquid-intraday-momentum.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 13:34:56 -04:00
momentum devandClaude Opus 4.8 4f40d512e3 Log visibility: stop counting entry rejections as executions
Root-caused (workflow wjy4nzpbw, adversarially verified) why the live loop's per-cycle
log printed `execute: ... auto_executed=1` ~951 times after 09:45 ET on 2026-07-01 with
ZERO matching broker orders: reject_order() returns a truthy {"ok": True} dict that
process_auto_approvals() appends unconditionally, and step_execute logged len() of that
list as auto_executed. So every daily-entry-cap rejection masqueraded as an execution,
hiding the fact that the bot was hard-capped at 5 fresh entries for the rest of the day.

This is the log-VISIBILITY half only (safe, zero trading-behavior change):
- reject_order() tags its return rejected=True + reason, so a rejection is distinguishable
  from a real execution (both carry ok=True).
- step_execute() counts auto_executed = results with a real broker_order_id, and appends
  `auto_rejected=N (reason x N, ...)` so a tripped cap is VISIBLE instead of silent, e.g.
  `auto_executed=0 auto_rejected=1 (daily_entry_capx1)`. ASCII-safe for the systemd log sink.

NOT included (deferred): the behavior fix (cap counts accepted submissions incl. unfilled
resting limits that get backed out, never decrements — so 4 phantom orders burned the whole
day's entry budget). That change lets the bot take more real shots, which given the live-
confirmed entry-conversion weakness likely means more churn — pair it with entry-quality
work, don't ship solo. See memory daily-entry-cap-counts-submissions.

Takes effect on the next loop boot; deliberately NOT restarting mid-session (the cap counter
is in-memory, so a restart would reset it to 0 and re-open entries into today's tape).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 12:52:06 -04:00
momentum devandClaude Opus 4.8 ee6991d05a Runner track: match wgcbvb3ce grade signature + attach grade-stratified conversion
Fidelity pass against the leading-gainer-runner workflow (wgcbvb3ce) synthesis:

- runner_quality.py: reorder calculate_runner_quality signature to the spec's
  (gap_pct, vol_burst, velocity, above_vwap, ...) — behavior identical (all callers
  use keywords), now matches the documented signature so a positional caller reading
  the spec is correct.

- labeler.py: close build_plan step (3) "attach the runner grade". The grade is a pure
  function of components already persisted on each fire (gap/velocity/burst/vwap), so
  it is recomputed exactly at report/validate time (catalyst-OFF) — no 34-min rebuild.
  report-runner now stratifies forward +1R and median 60m upside BY GRADE; validate-runner
  adds the A/B-subset test (does concentrating on top grades salvage the locked bar?).

FINDING (385 fires, 25 session-days): the detector still FAILS all 5 pre-registered
bars (the workflow predicted this — SVRE/JEM put 100% of the move pre-market, RTH fires
confirm at/after the top). BUT the grade is MONOTONE with forward outcome:
  grade A: +1R 45%, median max-upside 60m 17.7%   (n=73)
  grade B: +1R 41%, median 9.2%                    (n=186)
  grade C: +1R 35%, median 7.8%                    (n=106)
  grade F: +1R 30%, median 3.5%                    (n=20)
Grade A ALONE clears the +15% upside bar (17.7%) but not +1R>50% (45%): A-runners have
real continuation, the naive last-close entry still whipsaws. Confirms the grade earns
its keep as a RANKER/sizer (concentrate on A) while the bottleneck stays ENTRY-conversion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 00:58:46 -04:00
momentum devandClaude Opus 4.8 a33996f91c Runner-aware GRADE + leading-gainer runner DETECTOR, shadow-only
The default ORB grade (quality.py) pins catalyst RUNNERS at F/C: its two heaviest
components structurally break on a vertical spike -- STRUCTURE (30%) reads 0 (no
pullback to classify) and adv-RVOL (25%) reads 1.0x (NULL rolling-20d on fresh
reverse-split names). On the 2026-06-30 tape the source trader's winners (SVRE
+247%, JEM +348%, CELZ +472%) all graded C 0.565, indistinguishable from the bot's
F-grade chop. Two parallel, shadow-only artifacts fix this without touching the live
ORB path:

- strategy/evaluation/runner_quality.py::calculate_runner_quality -- a PARALLEL
  grade that drops the two broken components and scores the traits that actually
  separate runners from chop (gap saturating at +100% not +10%, price velocity,
  own-name volume burst, above-VWAP, catalyst). Validated on the real tape:
  CELZ/JEM/SVRE -> B/A, chop (SLS/RIVN/CCXI/BB/NVDX) -> F. quality.py untouched.

- strategy/evaluation/runner.py::detect_leading_gainer_runner -- catches the mover
  detect_momentum_ignition is blind to (it requires blue-sky > prior ATH, so it
  fires ZERO on beaten-down catalyst pops). Blue-sky deleted; gates = leading-gainer
  run + verticality + own-name burst + holding session-cumulative VWAP; PM-inclusive;
  tags (never gates) PM-exhaustion. Real-tape in-sample: 5/5 runners fire, 7/7 chop
  silent, zero false-fires. Cumulative VWAP is a prefix function, so a precomputed
  session_vwap makes a bar-by-bar slide O(n) not O(n^2) (identity verified exact).

- research/labeler.py -- build-runner/report-runner/validate-runner shadow track,
  mirroring vwap_reclaim: emit a non-tradeable setup on each fresh fire, score with
  the SAME _forward_labels. Pre-registered promotion bar LOCKED before forward data
  (median fwd-max >= +15%, median adverse > -8%, +1R > 50%, beat vwap_reclaim,
  PM-exhaustion tag must earn its keep). SHADOW-ONLY: never routes to approve_order,
  never gates live until the bar clears out-of-sample. The honest open question the
  track exists to answer: the detector confirms AT/AFTER the top for PM-driven names
  (SVRE fires at +242% already run) -- does that late entry actually convert to +1R?

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 00:10:19 -04:00
momentum devandClaude Opus 4.8 74160bdf37 Fix Ollama connectivity: boot-time host fallback (same WSL flip as the DB)
Ollama was unreachable — the enrichment/qabot pointed at the 127.0.0.1:30069 netsh tunnel,
which the WSL network no longer routes (same reboot-flip landmine as the DB), while the
direct 192.168.1.5:30068 answers. config.OllamaConfig.from_env now resolves the host: probe
the configured endpoint, else the first reachable known candidate (tunnel 30069 <-> direct
30068 — note the DIFFERENT ports). So enrichment self-heals on boot instead of silently
failing. .env OLLAMA_HOST also repointed at the direct host.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 23:26:13 -04:00
momentum devandClaude Opus 4.8 a114579cc1 Quality-grade entry gate: trade fewer, higher-quality setups (config, off by default)
Forensic on 2026-06-30: of the bot's 8 traded names, SEVEN graded F (<0.50) and one B —
it sprayed low-quality chop (SLS F0.12 -$412, RIVN F0.19, BB F0.21, CCXI F0.26 +$374,
NVDX F0.33, NOK F0.48, RDW F0.49; GVH B0.71). The trader made +$616 on SVRE — which the
bot SAW but its 35% gap cap excluded; and even graded, SVRE is F (0.35): low computed
RVOL + no clean pullback structure. So the grade cuts chop but does NOT recognise a
vertical catalyst RUNNER.

Adds ExecutionSettings.min_quality_score (env TRADING_MIN_QUALITY_SCORE, default 0.0 =
off) and a gate in request_approvals_for_ready_signals mirroring the VWAP gate: a signal
scoring below the threshold is skipped and logged rule_type=quality_below; fail-open on a
missing score. 3 tests (skip-low-when-enforced / allow-when-off / allow-at-threshold);
44/44 trading-execution tests green.

Honest scope: this is the "trade less chop" lever (evidence: the bot's own labeled data
has q>=0.55 win ~71% vs q<0.55 ~36%). It will NOT catch SVRE-class leading gainers — the
grade rewards pullback structure the runners lack, and the gap cap excludes them upstream.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 23:16:58 -04:00
momentum devandClaude Opus 4.8 0870bb780f Exit manager: re-anchor a breakeven carry's +0.25R trail to TODAY's opening range
Per the operator: a carried position sitting at breakeven shouldn't ladder off its stale
overnight entry. When a held position is at breakeven+ with no real R, the +0.25R trail
now RE-ANCHORS to today's opening range — reference = the session open, R = the first
5 RTH bars' high-low — so the stop tracks today's move from the open and locks in tighter
than the wide entry-based synthetic R. Until the opening range forms (first ~5 min) it
trails off the synthetic fallback R, then re-anchors once (one-shot, tracked in
_open_anchored; cleaned up when the position closes). The stop only ever ratchets UP and
never drops below the live breakeven. 1 test added (re-anchors + ladders tighter than the
synthetic); 11/11 exit-manager tests green.

For KEEL: at tomorrow's open its stop ladders up from the open's range instead of dangling
at the $5.56 breakeven.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 09:38:13 -04:00
momentum devandClaude Opus 4.8 f5dc7a8faa Cleanup: tidy the exit-manager / exits / db_pg changes (post-review)
Focused review pass on the session's safety-critical additions, applying the safe wins:
- exit_manager: drop the dead `_Tracked.last_stop` field (written, never read) and its
  assignment; guard `_rest_protective_stop` against a degenerate qty<1 (a 0-qty submit
  is rejected and would churn) + document the cur-falsy->breakeven fallback; reword the
  synthetic-R comment (the branch is also hit on a healthy trade once breakeven ratchets
  the stop to entry, not only the stripped-stop case).
- exits: surface default_trail_r_pct in ExitConfig.describe() (boot-log observability)
  and clamp it to [0, 0.95] in from_env (a >=1 / negative synthetic R would put the stop
  at/above entry).
- db_pg: comment the gate-vs-try-order distinction of _LAN_DB_HOSTS (superset incl.
  localhost) vs _LAN_DB_FALLBACKS so a future edit can't break the non-LAN-DSN guard.
- momentum_cli: drop an unused `import sys`.
Behavior-preserving; 29/29 exit/exits tests green. Deliberately did NOT extract a shared
flatten-vs-rest helper for the catastrophe/naked blocks — the explicit blocks read
clearer for this safety code and the tests guard against drift.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 22:53:24 -04:00
momentum devandClaude Opus 4.8 3076d22375 Exit manager: trail a breakeven-protected position UP via a synthetic R (open-only)
KEEL was left at a frozen breakeven stop because its original R was lost — the bracket
stop leg was invalid (the limit filled below the ORB-low, so the stop sat ABOVE entry
and Alpaca cancelled it, which is what left it naked). The manager skipped any position
already at breakeven+, so it never trailed: a winner could only scratch or run
unprotected-above-breakeven.

Now a position first seen at breakeven+ with no real risk distance is tracked on a
SYNTHETIC R (ExitConfig.default_trail_r_pct, default 10% of entry, env
TRADING_EXIT_DEFAULT_TRAIL_R_PCT), so the SAME tested +0.25R step-trail / profit-lock
ratchet the broker stop UP as it rises (only ever up, never below the live stop).
default_trail_r_pct=0 keeps the old frozen behaviour; positions with a real sub-entry
stop are unchanged.

TRAILING is gated on the session being OPEN (one broker-clock check per pass): after
hours there are no new highs to trail under AND a resting GTC stop is often not
replaceable ("accepted" status -> Alpaca 422), so attempting it just churns. The stop
stays protective and ratchets up at the open. 4 tests added (synthetic-R trails up;
disabled=frozen; closed=no-attempt); 10/10 exit tests green.

Live now (after-hours): KEEL is protected at breakeven $5.56; the loop holds that stop
without churning and will trail it up once the session opens.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 20:32:11 -04:00
momentum devandClaude Opus 4.8 b5909bc793 Naked-stop & catastrophe enforcement: rest a protective stop when the market is closed
KEEL was carried into after-hours as a naked long (its bracket stop leg never
attached — 0 stop orders in its whole history). The enforcement tried to FLATTEN it
with a market order every pass, but a market sell can't fill when the market is
closed, so it churned submit->cancel ~every 30s (76 market-sells / 79 cancels, the
"naked-stop enforced exit KEEL xN" spam) and left the position genuinely unprotected.

Fix (runtime/exit_manager.py): both the naked-stop enforcement AND the catastrophe
backstop now branch on the broker clock. Market OPEN -> flatten (a market exit fills,
unchanged). Market CLOSED -> REST a protective sell-stop instead (GTC) — at breakeven
when in profit, a catastrophe-distance stop below market when underwater — so the
position survives to the open PROTECTED and the next pass sees the stop and stops
enforcing. Catastrophe with an existing (loose) stop while closed just defers to it
(no market churn, no double stop). New client bits: AlpacaPaperClient.get_clock() and
a standalone stop_price on submit_order. 3 exit-manager tests added (open->flatten,
closed->breakeven-rest, closed-underwater->catastrophe-rest); 7/7 green.

Operationally: KEEL was protected at breakeven ($5.56) immediately and the loop now
leaves that stop in place instead of churning.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 20:16:24 -04:00
momentum devandClaude Opus 4.8 6a6b38326b Streamline the watch CLI board + dashboard for glance-ability
From a design pass (CLI spec + dashboard spec + an adversarial glance-safety review)
that read the real output of both surfaces. Principle: each safety/state number appears
ONCE on the first screen, every list has a bounded height capped by relevance (active/open
first), and load-bearing signals (the UNPROTECTED banner, heartbeat, day P&L, circuit
breaker) are never trimmed or hidden.

CLI board (momentum_cli.py _render_board): drop the static 2-line strategy subtitle (never
changed, redraws every 3s); trim the heartbeat to "Ns"; armed table shows actionable
(armed/fired/long) rows first then the top-4 scanning by rvol, dropping corrupt >1000%-gap /
0-volume rows with a "+N more scanning" footer; remove the redundant ready-signals panel
(restated the armed table) and the day-P&L echo in the trades title; trades float open lots
to the top (never capped off) and cap closed at 6; risk collapses consecutive identical
fires into one "×N" row (the 12 duplicate KEEL lines -> one) and is promoted ABOVE trades.

Dashboard (api/static/dashboard.html): cap the long "runs" — ready-signals to top 15 by
quality (also kills ~80 per-refresh chart fetches), activity feed 20, catalyst feed 20, order
lifecycle shows only actionable orders (raw 1000s incl rejected/cancelled read as an alarm),
watch-states hide the score-gate 'blocked' rows; drop redundant strip cells (Positions dup of
pnl Open/Closed, Gross) and the Account panel's Buying-Power/Synced; demote Account to a
collapsed half-width panel; drop the ~constant ready-signals "Setup" column. JS syntax
validated, no dangling refs, serves 200.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 19:01:11 -04:00
momentum devandClaude Opus 4.8 fdb2841ae4 Resilient DB host + collapsible dashboard (ops: keep the loop + website up)
DB self-heal: the LAN Postgres is reached from WSL via EITHER the 127.0.0.1 netsh
portproxy OR a direct route to 192.168.1.5, and which one works FLIPS across WSL/host
reboots and network-mode changes — the recurring crash-loop landmine. storage/db_pg.py
_connect now tries the configured LAN host and then the alternate, so the loop boots
regardless of the current tunnel state ("connection refused" is instant, so it's cheap).

Dashboard UI: api/static/dashboard.html panels are now collapsible <details> (native,
no JS for the toggle; inner ids preserved so the live rendering JS is untouched). The
live/actionable panels (Armed Triggers, Open Positions, Approval Queue, Ready Signals,
Account) start OPEN and lead the layout; the big/historical ones (Catalyst Engine,
Daily Performance, Trades, Watch States, Activity Feed, Order Lifecycle, All-Time Score)
start COLLAPSED so the page is compact and scannable. Open/closed state persists per
panel in localStorage; an EXPAND/COLLAPSE-ALL header button toggles everything.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 18:41:48 -04:00
momentum devandClaude Opus 4.8 f170305a41 No-chase entry shadow-validation: VWAP-pullback halves the ORB loss but FAILS the locked bar
The mechanics study found the no-chase entry (buy the VWAP reclaim, not the breakout high)
was the dominant lever. This codifies it as a permanent, pre-registered shadow-validation —
the disciplined close-out of the 4-lever investigation arc.

- strategy/evaluation/vwap_pullback_entry.py: pure find_vwap_pullback_entry — first forward
  bar that dips to the CUMULATIVE session VWAP (calculate_vwap, NOT the per-bar minute_bars.vwap
  midpoint, which would make 'reclaim' trivially true) and closes back above it; entry = its
  close, stop = its low (the tight no-chase structure stop). 4 tests.
- research/labeler.py validate_no_chase_entry + `validate-entry`: simulates it vs the ORB-high
  breakout via the live simulate_exit on real forward bars, judged in PERCENT net of a REALISTIC
  per-name cost proxy (4% sub-$3 / 2% else — no NBBO in bars). PRE-REGISTERED bar locked: mean
  net% must be STRICTLY POSITIVE, 'less negative than ORB' is NOT a pass.

IN-SAMPLE RESULT (v1 gappers, 19 autocorrelated days): the no-chase entry HALVES the loss
(ORB -7.94% -> no-chase -3.61% net, median -4.23%, offered on 75%), robustly (day-jackknife
[-3.83%,-3.48%]) -- confirming not-chasing the spike-top is the strongest mechanic -- but it
still does NOT cross zero under realistic cost: PRE-REGISTERED BAR = FAIL. Consistent with the
whole arc: no mechanizable entry/exit/selection lever beats cost on the full gapper book; the
path to positive is FEWER/cleaner setups (cut the ~70% fader population) COMBINED with this
no-chase entry. validate-entry re-runs forward; promote only if it clears the bar out-of-sample.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 18:08:56 -04:00
momentum devandClaude Opus 4.8 97e3bf91a6 Runner-propensity selection lever: cheap-price + premarket-gap, shadow-measured
The float audit and the labeler both point at the SAME free, available-today selection
signal the bot underuses: gap magnitude + cheap absolute price separate runners from
faders, while the bot's gap%×RVOL rank leans on RVOL (labeler-proven FLAT) and CAPS gap.

- strategy/evaluation/runner_propensity.py: pure, transparent, FROZEN-threshold primitive
  -> a coarse runner-propensity tier = gap_tier(0-3) + price_tier(0-2). Gap is weighted
  heavier BY CONSTRUCTION because the data says it dominates; premarket gap preferred over
  RTH gap; cheap price is a secondary tie-breaker (the two are correlated). 6 unit tests.
- research/labeler.py runner_rank_report + `runner-rank` CLI: measures the lever on the
  v1 ORB labeled set. SHADOW/measurement only, not wired into live selection.

MEASURED (v1 gappers, gap>=10%, n=187, 19 autocorrelated days -- small, caveated):
the combined tier ladders monotonically: tier1 2% -> tier3 35% -> tier5 (cheap+big-gap)
48% runner (1.63x base). Gap dominates (>=50% -> 48%, 1.64x); cheap<$2 a real secondary
(1.29x). OPERATIONAL KICKER: the live ~35% TRIGGER_GAP_MAX would EXCLUDE 73 gappers whose
runner rate is 44% vs 20% for the kept <=35% -- the cap discards the best runners (this is
how NXTS at +81% got excluded). Caveat: +1R/+2R separate far less than the +20% runner
label, and gap/price are correlated; a hypothesis to shadow-confirm + adversarially
validate before touching the live cap, not a validated rank.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 18:08:56 -04:00
fez5587 ae0651b769 Merge pull request #3 from fez5587/claude/float-recreation-trading-gcn42l
spread_pct P0: real decision-time bid/ask spread as an observe-only metric
2026-06-29 16:48:39 -05:00
fez5587 6bb5e135c3 Merge pull request #1 from fez5587/claude/kubernetes-image-package-kkdkom
Add Kubernetes image + Helm package with in-cluster Postgres
2026-06-29 16:47:26 -05:00
Claude a76c0b5376 Add Docker Compose stack for local development
Single-machine equivalent of the Helm package: builds the app from the
Dockerfile, runs PostgreSQL with a persistent volume, and starts the trading
loop with its embedded dashboard on :8010.

- postgres:16 with healthcheck + named volume; app waits on it healthy
- DATABASE_URL overridden to the in-network postgres service (self-contained);
  .env is optional (dry mode without ALPACA_* keys)
- journal / eod-replay / nightly-tune under a "jobs" profile, run on demand;
  they share the app-data volume so learned_params.json reaches the loop
- Grafana under a "monitoring" profile with a compose-specific datasource
  override pointed at the postgres service
- deploy/README.md documents the Compose workflow

Validated with `docker compose config` across all profiles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCVMm2AbmZvvxwLTX6UpNR
2026-06-29 21:40:19 +00:00
Claude 4173acc9d5 docs: add future-hardening roadmap to deploy README
Document reliability, security, and CI/operational improvements for the
Kubernetes package as an actionable backlog (not yet implemented). Notably
flags the loop-heartbeat liveness gap: /api/health proves only the dashboard
thread is alive, so a hung trading loop would go undetected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCVMm2AbmZvvxwLTX6UpNR
2026-06-29 21:35:38 +00:00
Claude e91979a622 Add Kubernetes image + Helm package with in-cluster Postgres
Containerizes the momentum monolith and packages every run mode for
deployment in a single Kubernetes namespace, with PostgreSQL hosted in
the cluster.

- Dockerfile (python:3.11-slim, non-root, writable /app/data) + .dockerignore
- GitHub Actions workflow publishing to ghcr.io/fez5587/momentum-system
- Helm chart (deploy/helm/momentum):
  * PostgreSQL StatefulSet + PVC (single source of truth; schema
    auto-creates from storage/pg_schema.sql on first connect)
  * App Deployment running run_live_paper.py with the embedded dashboard
    (replicas: 1, Recreate; the dashboard's write endpoints need the
    in-process TradingExecutionService) + Service/Ingress on 8010, plus a
    wait-for-postgres initContainer
  * CronJobs for journal / eod-replay / nightly-tune (US/Eastern via
    spec.timeZone) sharing the app data PVC
  * ConfigMap (tuning env) + Secret (DATABASE_URL/broker keys, existingSecret
    supported); DATABASE_URL built from the in-cluster Postgres service
  * Optional Grafana provisioned against the in-cluster Postgres
- deploy/README.md with install, secrets, and constraints

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCVMm2AbmZvvxwLTX6UpNR
2026-06-19 17:22:06 +00:00
390 changed files with 37703 additions and 4580 deletions

No files matched your search

+48
View File
@@ -0,0 +1,48 @@
# VCS / CI
.git
.github
.gitignore
# Secrets and runtime data (never bake into the image; provided via Secret/PVC)
.env
.env.*
data/
*.duckdb
*.wal
# Local Grafana stack (provisioned via the Helm chart instead)
grafana/
# Deploy artifacts don't belong inside the app image
deploy/
Dockerfile
.dockerignore
# Python caches / build / venvs
__pycache__/
*.py[cod]
*.egg-info/
build/
dist/
.venv
venv/
ENV/
env/
.ruff_cache/
# Tests and coverage
tests/
.pytest_cache/
.coverage
htmlcov/
# Editor / OS cruft
.vscode/
.idea/
*.swp
.DS_Store
# Large artifacts
*.zip
archive/
uv.lock
+229 -172
View File
@@ -1,197 +1,254 @@
# ============================================================
# Momentum — environment configuration (copy to .env and fill)
# Regenerated 2026-07-09 from the live config. Values kept here
# are the current tuned defaults; <REDACTED> = secret, fill from
# your protected store. See deploy/docker/README.md for which
# layer (compose vs .env) each group belongs in.
# ============================================================
# ---- Database (PostgreSQL) — the single datastore (event store + bar warehouse)
# Either a full URL:
DATABASE_URL=postgresql://USER:PASSWORD@HOST:5432/momentum
# ...or individual vars (used only if DATABASE_URL is unset):
# PGHOST=127.0.0.1
# PGPORT=5432
# PGUSER=admin
# PGPASSWORD=
# PGDATABASE=momentum
TOKEN_ENCRYPTION_KEY=<REDACTED>
# ---- Alpaca (market data + paper broker) — REQUIRED for live paper trading
ALPACA_API_KEY=
ALPACA_SECRET_KEY=
# Alpaca paper trading endpoint (default is fine)
APCA_API_BASE_URL=https://paper-api.alpaca.markets
# These two aliases are also read by the paper client:
APCA_API_KEY_ID=
APCA_API_SECRET_KEY=
DASHBOARD_PORT=8765
# Dashboard POST auth + operator command bus (the :8765 kill switch).
# DASHBOARD_TOKEN is REQUIRED-when-commands-on: /api/command returns 503 unless
# it is set (commands never run unauthenticated); it also gates approve/reject/
# exit POSTs. Empty = read-only dashboard, no command bus. Keep it out of logs.
DASHBOARD_TOKEN=
# Enable the operator command bus (flatten-all / flatten-symbol / pause / resume).
# Needs DASHBOARD_TOKEN set too; the loop consumes commands on its 5s guard.
COMMAND_BUS_ENABLED=0
#curl.exe -H "APCA-API-KEY-ID: $key" -H "APCA-API-SECRET-KEY: $secret" https://paper-api.alpaca.markets/v2/account
# Alpaca API (primary data provider)
ALPACA_API_KEY=<REDACTED>
ALPACA_SECRET_KEY=<REDACTED>
ALPACA_BASE_URL=https://paper-api.alpaca.markets/v2
# ---- Schwab (live broker, Milestone 3) — OPTIONAL for paper trading
SCHWAB_MARKET_DATA_APP_KEY=
SCHWAB_MARKET_DATA_APP_SECRET=
SCHWAB_BROKER_APP_KEY=
SCHWAB_BROKER_APP_SECRET=
SCHWAB_REDIRECT_URI=
SCHWAB_TOKEN_PATH=data/schwab_tokens.json
# Polygon (optional fallback)
POLYGON_API_KEY=<REDACTED>
# ---- Optional research / news providers
POLYGON_API_KEY=
FINNHUB_API_KEY=
Alpha_Vantage_Api_Key=
# Additional news and filing sources
FINNHUB_API_KEY=<REDACTED>
FINNHUB_EMAIL=fez5587@gmail.com
SEC_API_KEY=<REDACTED>
DATABENTO_API_KEY=<REDACTED>
# News-catalyst discovery (optional): RSS feeds as name=url, comma-separated.
# When set, tickers mentioned in recent news ($TICKER or (NASDAQ:TICKER)) are
# added to the watchlist. Unset = disabled (no effect on the loop).
# NEWS_FEEDS=prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==
# Data directory for parquet files and DuckDB
DATA_DIR=./data
# Boot warns (log + one Telegram note) if data/learned_params.json is older than
# this many days — a stale file means the nightly tuner stopped running. Never blocks boot.
LEARNED_PARAMS_MAX_AGE_DAYS=3
# Alpaca News API ingestion (Benzinga) -> raw_news_items. Each item carries an
# authoritative symbols list (no regex), feeding recent_news_map catalyst
# prioritisation + the Ollama enrichment. Uses the existing Alpaca data keys.
ALPACA_NEWS_ENABLED=1
ALPACA_NEWS_INTERVAL_SECONDS=120
ALPACA_NEWS_LIMIT=50
DISCORD_WEBHOOK_URL=<REDACTED>
DISCORD_TRADING_WEBHOOK_URL=<REDACTED>
DISCORD_DEVELOPER_WEBHOOK_URL=<REDACTED>
# ---- Local LLM news/catalyst enrichment (Ollama) ----
# A local model classifies ingested headlines (catalyst type, sentiment,
# conviction, dilution) into news_catalyst_cache — surfaced on the dashboard
# approval queue. Runs off the hot path; degrades to a no-op if Ollama is down.
# OLLAMA_ENABLED=1
# OLLAMA_HOST=http://localhost:11434
# OLLAMA_MODEL=qwen2.5:7b-instruct # fits 12GB; best small-model JSON.
# # alts: llama3.1:8b-instruct-q4_K_M, mistral:7b-instruct
# NEWS_ENRICH_ENABLED=1
# NEWS_ENRICH_INTERVAL_SECONDS=120
# NEWS_ENRICH_LOOKBACK_HOURS=12
# NEWS_ENRICH_BATCH_LIMIT=50 # max headlines classified per pass (GPU budget)
# Phase 2 (automation — each ships OFF; separate from the advisory above so the
# dashboard never silently changes scoring):
# score blend — fold the catalyst into the setup quality score AND graded arming
# rank (bullish catalyst boosts, confirmed dilution sinks below the armed cut).
# NEWS_CATALYST_SCORE_ENABLED=0
# NEWS_RANK_BOOST_GAIN=2.0 # arming-rank multiplier = 1 + gain*score
# NEWS_DILUTIVE_PENALTY=0.1 # arming-rank multiplier for a confirmed dilutive name
# dilution veto — block confirmed dilutive offerings before sizing.
# NEWS_DILUTION_VETO_ENABLED=0
# NEWS_DILUTION_VETO_CONVICTION=0.6
# Telegram notifier (bot @traitor_momentum_bot) — gated; no-op if unset
TELEGRAM_BOT_TOKEN=<REDACTED>
TELEGRAM_CHAT_ID='2082795871'
# Dead-man's switch (qabot container): the loop's own notifier dies WITH the
# loop, so the always-up qabot watches max(events.timestamp) from outside.
DEADMAN_STALE_MIN=5 # alert when no events for this many minutes during market hours
DEADMAN_DAILY_PING=1 # nightly OK-ping after 20:30 ET — proves the watchdog itself is alive
# ============================================================
# Live paper-trading orchestrator (run_live_paper.py)
# ============================================================
# Nightly pg_dump (db-backup compose service, 20:10 ET Mon-Fri) — the only
# automated backup of the production Postgres. See deploy/docker/backup_db.sh.
BACKUP_DIR=/data/backups # inside the db-backup container (app-data volume)
BACKUP_KEEP_DAYS=14 # prune dumps older than this many days
# Execution destination: alpaca_paper (safe) or schwab_live (REAL MONEY)
TRADING_EXECUTION_MODE=alpaca_paper
TRADING_EXECUTION_ENABLED=1
TRADING_EXECUTION_INTERVAL_SECONDS=30
TRADING_AUTO_APPROVE=0 # 0 = manual approval in the dashboard, 1 = auto-arm (no approval)
TRADING_MAX_ORDERS_PER_TICK=1
# NOTE: concurrency is set ONCE in the risk section below (was duplicated here).
TRADING_RISK_PER_TRADE_PCT=0.01 # fraction of equity risked per trade
# --- entry mechanism (Ross-Cameron-style auto-arm; all tunable) ---
TRADING_REWARD_MULTIPLE=2.0 # target = entry + this * (entry - stop); 2R default
TRADING_ENTRY_ORDER_TYPE=limit # limit = rest at trigger (cancellable); market = fill now
TRADING_ENTRY_TIMEOUT_BARS=2 # cancel an unfilled entry after N minutes resting (0 = off)
TRADING_ENTRY_INVALIDATE_PCT=0.0 # cancel if price breaks this far below entry (0 = any break; <0 = off)
TRADING_ENTRY_GUARD_INTERVAL_SECONDS=5 # how often to re-check armed entries vs the LIVE last-trade price
TRADING_MAX_DAILY_LOSS_PCT=0.03 # daily-loss circuit breaker
#Market Data these are free plans so work within days/months capacity.
Alpha_Vantage_Api_Key=<REDACTED>
Twelve_Data_Api_Key=<REDACTED>
Marketstack_Api_Key=<REDACTED>
Financial_Modeling_Prep_Api_Key=<REDACTED>
EOD_Historical_Data_Api_Key=<REDACTED>
#News Data these are free plans so work within days/months capacity.
NewsAPI_Api_Key=<REDACTED>
NewsData.io_Api_Key=<REDACTED>
Marketaux_Api_Key=<REDACTED>
MediaStack_Api_Key=<REDACTED>
#ngrok key
ngrok_key=<REDACTED>
# Intraday watcher (signal engine)
WATCHER_ENABLED=1
WATCHER_INTERVAL_SECONDS=60
# WATCHER_EVENT_DB_PATH is legacy and ignored — the datastore is PostgreSQL (see DATABASE_URL above)
WATCHER_MAX_SYMBOLS=25
WATCHER_MIN_BARS=30
WATCHER_MIN_QUALITY=0.30
# Comma-separated symbols to follow all day. If empty, candidates are
# discovered from whatever has minute bars in market.duckdb for today.
WATCHER_SYMBOLS=AAPL,TSLA,NVDA,AMD,SOFI
# Price band for auto-discovered candidates (Ross Cameron default 1-20).
WATCHER_PRICE_MIN=1
WATCHER_PRICE_MAX=20
# Active strategy setups (comma-separated). The backtest sweep found the
# opening-range breakout is the only profitable one; empty = all setups.
# WATCHER_MIN_BARS / WATCHER_READY_SCORE are left to the nightly tuner
# (data/learned_params.json). Uncomment to pin a manual override.
# WATCHER_MIN_BARS=5
# --- faster loop cadence: react sooner once a setup forms (helps at the margin;
# the real lateness is the setup design — see learn_params.py) ---
LIVE_BARS_INTERVAL_SECONDS=20
# Live ingest only re-pulls the last few minutes (closed bars never change); the
# first pass still covers the whole session so the opening range is present.
# This is what fixed the ~46s ingest that froze the single-threaded loop.
LIVE_BARS_REFRESH_MINUTES=20
WATCHER_INTERVAL_SECONDS=15
DISCOVER_INTERVAL_SECONDS=120
# --- whole-market premarket squeeze scan (research/ingestion/snapshot_scan.py) ---
# The movers/most-actives screeners only see the regular session, so premarket
# squeezes were invisible until 09:30 (KITT 2026-09-25 ran at 09:00, found 09:35).
# This sweeps every listed, tradable US equity through Alpaca snapshots
# 04:00-16:00 ET on its own thread and adds qualifying names to the watchlist
# (symbol_discovered source=snapshot_scan). Discovery only; all trading gates apply.
# Price band = WATCHER_PRICE_MIN/MAX on the LATEST trade (not the prior close).
DISCOVER_SNAPSHOT_SCAN_ENABLED=1
DISCOVER_SNAPSHOT_INTERVAL_SECONDS=45 # one full sweep (~7 requests)
DISCOVER_SNAPSHOT_MIN_PCT=20 # % change vs prior close to qualify
DISCOVER_SNAPSHOT_MAX_PER_SWEEP=5 # new names injected per sweep (top by %)
DISCOVER_SNAPSHOT_MIN_BAR_VOLUME=5000 # shares in the latest minute bar
DISCOVER_SNAPSHOT_MIN_CUM_VOLUME=100000 # shares traded today since 04:00 ET
DISCOVER_SNAPSHOT_MAX_TRADE_AGE_SECONDS=180
DISCOVER_SNAPSHOT_MAX_BAR_AGE_SECONDS=300
DISCOVER_SNAPSHOT_BATCH=2000 # symbols per request (URL-length bound; 414 at ~12.7k)
# Active setups. The backtest sweep found the opening-range breakout is the only
# profitable one (others lose and fire later); empty = all setups. See learn_params.py.
STRATEGY_SETUPS=opening_range_break
# Live minute-bar ingestion (feeds the watcher with active market data)
LIVE_BARS_ENABLED=1
LIVE_BARS_INTERVAL_SECONDS=60
LIVE_BARS_LOOKBACK_MINUTES=120
# Data feed: live realtime uses iex (free, realtime, but ~2-5% of true volume).
# Backfill/backtest uses sip (full consolidated tape) so rvol/$-volume reflect
# REAL volume — the iex thinness silently corrupts every volume-based decision.
ALPACA_DATA_FEED=iex
# --- news / catalyst ranking ------------------------------------------------
# A fresh catalyst (PR/earnings/FDA/offering) is WHY a small-cap gaps and runs.
# These RSS feeds are ingested each discover pass; tickers with recent news are
# (1) folded into the watch universe (caught earlier) and (2) boosted up the
# armed-trigger ranking. Format: name=url,name=url. Empty = news disabled.
NEWS_FEEDS=prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_RANK_BOOST=2.0 # multiply a catalyst-backed gapper's rank score
NEWS_LOOKBACK_HOURS=8 # how fresh the catalyst must be
# Manual per-symbol priority nudge (a SMALL boost to names you want watched).
# Keep modest (1.1-1.3) so it doesn't override the gap/volume logic. Empty = off.
# e.g. SYMBOL_WEIGHTS=SOFI:1.2,RIVN:1.15
SYMBOL_WEIGHTS=
# --- fast armed-trigger path (runtime/triggers.py) ---------------------------
# Keep the N most-promising gappers "armed" with a pre-computed opening-range
# trigger/stop and FIRE the instant live price crosses (a marketable order that
# actually fills on a runner) — instead of waiting for a 1-min bar to close and
# resting a limit that never fills. This is what catches the move in time.
TRIGGER_ENABLED=1
TRIGGER_MAX_ARMED=10 # how many gappers to keep ready at once (batched price fetch scales fine)
TRIGGER_INTERVAL_SECONDS=4 # how often to check live prices / fire
TRIGGER_ARM_INTERVAL_SECONDS=20 # how often to re-rank + recompute triggers
TRIGGER_GAP_MIN=10.0 # aligned with TRADING_MIN_ENTRY_GAP_PCT (population gate) 2026-07-03
TRIGGER_RVOL_MIN=2.0 # min relative volume to be eligible to fire
# Entry-quality filters (diagnostic + out-of-sample verified, verify-entry-filters):
# capping price at $5 and skipping blow-off gaps (>60%) ~DOUBLED held-out
# expectancy (+0.76 vs +0.39R, win 86% vs 71%). $10-20 stocks were net losers;
# >150% gaps fade. Fewer, much higher-quality trades.
TRIGGER_GAP_MAX=35 # 35-150% band is dead (+0.02R year-scale) — cap earns its keep there
# mega-gapper exception (2026-07-06, the LHSW/NXTS fix): re-admit REAL >150% squeezes.
# Year replay n=2,879 w/ split artifacts removed: >150% & rvol>=5 ran +0.63R, 8/14 months >=0.
TRIGGER_GAP_SQUEEZE_MIN=150
TRIGGER_GAP_SQUEEZE_RVOL=5.0
WATCHER_PRICE_MIN=2 # raised $1->$2 2026-07-09: $1-2 bucket -0.031R over 609 unbiased entries
TRADING_PRICE_MIN=2.0
TRADING_PRICE_MAX=20.0
WATCHER_PRICE_MAX=20 # RESTORED to the original $1-20 (2026-06-22). Was narrowed to 5 by a prior backtest ("$10-20 net losers") that the LIVE record contradicts + is n=2 noise; and $5 is why the bot missed NXTS ($9.21, today's +250% winner). Downside is now $500/trade-capped so the wider net is a bounded experiment — measure by price band on the clean sample before re-narrowing.
ORB_BARS=5 # opening range = first N regular-hours minutes
TRADING_TRIGGER_SLIP_PCT=0.004 # cap chase above the trigger on the fill
WATCHER_STATIC_SYMBOLS=TSLA,NVDA,AMD,MSFT,AAPL
ACCOUNT_SYNC_ENABLED=0
ORDER_SYNC_ENABLED=0
# --- LIVE PAPER TRADING (enabled for market open) ---
ALPACA_PAPER_SYNC_ENABLED=1
TRADING_EXECUTION_ENABLED=1
TRADING_AUTO_APPROVE=1 # 1 = auto-arm bracket orders; 0 = approve each in the dashboard
TRADING_MAX_DAILY_LOSS_PCT=0.05 # RE-ARMED 2026-06-22 (was 0.50 testing): circuit breaker halts entries + flattens at -5% equity (~-$4.5k on $90k). Caps a runaway day; per-trade risk is already $500-capped so ~all-stop-out ≈ -5%. Original design was 0.03 — tighten if you want a harder stop.
TRADING_RISK_PER_TRADE_PCT=0.01 # risk 1% of equity per trade
TRADING_MAX_RISK_DOLLARS=500 # HARD dollar-risk ceiling/trade: risk = min(1% equity, $500). On the ~$90k paper acct this caps the wide-stop ~$900-1k losers from the loss diagnosis at $500. 0=off (small live acct uses the % only)
TRADING_MAX_FRESH_ENTRIES_PER_DAY=10 # EXPERIMENT 2026-07-02 (owner-authorized): raised 5->10 real FILLED entries/day (behavior fix counts fills not submits). Data says all-day/more-trades is -EV (afternoon -0.94R); paper A/B only. C+ quality gate + re-entry-on-winners still in force.
TRADING_MAX_CONCURRENT_POSITIONS=10 # raised 6->10; the 60% gross-notional cap below still hard-bounds TOTAL exposure (so this = more, smaller positions, not more risk)
TRADING_REQUIRE_ABOVE_VWAP=1 # VWAP entry gate: only take breakouts trading >= session VWAP. The ONE validated selection signal (above-VWAP reaches +1R ~1.5x as often, n=3,110 from the labeler). 0=shadow-only (logs below-VWAP signals as rule_type=vwap_below but still enters). Set 0 to revert to pre-gate behavior.
TRADING_UNIFIED_ENTRY=1 # run the fast path's anti-chase gates (day-extension: don't enter a name whose breakout level is >30% above the session open; halt) on the LIVE auto path too. Closes the gap where those guards only ran on the rarely-used fast trigger path. Fail-open on missing data. 0=pre-unification (auto path skips these gates).
TRADING_GATE_RECEIPTS=1 # emit one entry_gate_evaluated event per gate evaluation (both paths) with the full pass/reject gate map — turns 'why didn't it trade X?' into a query. High-volume diagnostic, purged after 14 days like other observability spam. 0=off (no receipts).
TRADING_ENTRY_FILL_MODEL=resting # auto-path fill: 'resting' = rest a limit AT the breakout level (fills only on a pullback, misses runners that gap up). 'marketable' = limit a hair above the trigger so it fills as price breaks UP through it (catches runners, the fast path's behavior; the limit still caps the fill so no gap-chase). VALIDATED LIVE 2026-06-24 open (NOK/SOFI/SSPC filled near-trigger, no chase) then reverted to resting (safe default). Flip to 'marketable' to use it.
# Account-aware sizing: positions size off REAL broker equity (auto — so a $300
# live account sizes for $300, not the $100k default). One position's value is
# capped to this fraction of equity so N positions fit buying power (~1/N).
# 0.16 so 6 positions fit (6 x 16% ~ full account) — esp. the $300 cash account.
TRADING_MAX_POSITION_PCT=0.16
# Liquidity cap (shares <= this % of the symbol's day volume) — OFF at small
# size; enable (e.g. 0.005) only when scaling to real money to avoid slippage.
TRADING_LIQUIDITY_MAX_VOLUME_PCT=0.025
# End-of-day auto-flatten: close everything ~5 min before the bell so the book
# never carries overnight gap risk (a day-trading strategy shouldn't hold).
TRADING_EOD_FLATTEN_ENABLED=1
TRADING_EOD_FLATTEN_TIME=15:55
# The process is one trading session: after a SUCCESSFUL EOD flatten the loop
# exits 0 and compose restarts it fresh, so memory-only daily state (entry cap,
# benches, session-closed flags) can never go stale across a date rollover.
TRADING_EXIT_AFTER_EOD_FLATTEN=1
TRADING_MAX_ORDERS_PER_TICK=1
TRADING_ENTRY_ORDER_TYPE=limit # rest at the breakout trigger
# --- managed exit (swept + out-of-sample verified: trailLow_a1) -------------
# The exit sweep over 187 sessions with a held-out test split found that
# trailing the stop under the prior bar's low (after +1R) nearly DOUBLES the
# out-of-sample expectancy vs the old fixed +2R target (+0.128R vs +0.067R,
# win 49% vs 38%) and is robust, not overfit. So: a far backstop target and let
# the trailing stop do the work. See sweep_exits.py / strategy/exits.py.
TRADING_REWARD_MULTIPLE=10.0 # far take-profit backstop; the trail is the real exit
TRADING_EXIT_TRAIL_MODE=prior_low # trail the stop under each prior bar's low
TRADING_EXIT_TRAIL_AFTER_R=1.0 # start trailing once the trade is +1R in profit
# Profit-lock checkpoints (percentage of ENTRY): once a trade has been up gain%,
# the stop is pinned so it can't close for less than lock% — you always keep a
# minimum gain once it's working, and a fade back down sells you out in profit.
# Format: gain:lock,gain:lock (whole %). Tunable; raise/lower to taste.
TRADING_EXIT_PROFIT_TIERS=8:3,15:9,25:18,40:30
TRADING_EXIT_BREAKEVEN_R=1.0 # ON: lock breakeven once +1R (forward safeguard)
TRADING_EXIT_TRAIL_R_STEP=0.25 # 0.25R STEP-TRAIL from the buy price (2026-06-24, supersedes a standalone +2% breakeven): once +0.25R is reached the stop moves to entry (breakeven), +0.5R->+0.25R, +0.75R->+0.5R, ... It locks a small green pop from round-tripping to -1R (the SOFI +$281->-$118 case) AND keeps climbing under a runner. Coexists with the prior-low trail (stop = max of all). Tradeoff: scratches some winners at breakeven on noise (sim ~+0.05R net). Widen to 0.5 if too tight.
# TRADING_EXIT_SCALE_OUT_R / _PCT # (off) scaling didn't help out-of-sample
# TRADING_EXIT_FIRST_RED=0 # (off) first-red exits were NEGATIVE expectancy here
# How far price may dip below the entry trigger before we CANCEL an unfilled
# entry. 0.0 (the old default) cancels on any sub-cent wobble -> a breakout that
# oscillates around the level gets cancelled before it can fill = pure churn,
# never holds a trade. Give it room; the bracket STOP (at the opening-range low)
# is the real protection, and risk is capped at 1% per trade by sizing.
TRADING_ENTRY_INVALIDATE_PCT=0.015
TRADING_ENTRY_TIMEOUT_BARS=2 # cancel an entry still unfilled after ~2 min
# ---- Database (PostgreSQL) — single datastore via local port-proxy to the LAN instance
DATABASE_URL=<REDACTED> # direct to the LAN host; storage/db_pg.py auto-falls-back between 192.168.1.5 and the 127.0.0.1 netsh tunnel (whichever the WSL net can reach — it flips across reboots).
# anti-thrash: bench a symbol after it backs out (unfilled entry)
TRADING_BACKOUT_COOLDOWN_SECONDS=180
TRADING_MAX_BACKOUTS_PER_SYMBOL=3
# one round-trip per symbol per day: no re-entry after a position closes
TRADING_BLOCK_REENTRY_AFTER_EXIT=1
# liquidity gate: skip breakouts on thin names (recent 5-bar $-volume floor)
TRIGGER_MIN_DOLLAR_VOL=8000
# only bench re-entry after a real losing exit (>1%), not a scratch/win
TRADING_REENTRY_MIN_LOSS_PCT=0.01
TRADING_MAX_GROSS_NOTIONAL_PCT=0.60
ALPACA_BACKFILL_FEED=sip
# Freshness watchdog: warn (emit + display) when the newest bar is older than
# this during RTH. INGEST_STALE_BLOCK=1 also refuses to ARM on stale data
# (off by default so a mis-tuned threshold can't silently halt trading).
INGEST_STALE_SECONDS=180
INGEST_STALE_BLOCK=0
# Broker state sync
ALPACA_PAPER_SYNC_ENABLED=1
ALPACA_PAPER_SYNC_INTERVAL_SECONDS=60
ACCOUNT_SYNC_ENABLED=0 # Schwab live account sync
ORDER_SYNC_ENABLED=0 # Schwab live order sync
# Dashboard
DASHBOARD_HOST=0.0.0.0
DASHBOARD_PORT=8010
# ============================================================
# Armed-trigger fast path (runtime/triggers.py) + entry quality
# ============================================================
TRIGGER_ENABLED=1
TRIGGER_MAX_ARMED=10 # gappers kept ready to fire at once
TRIGGER_INTERVAL_SECONDS=4 # fast path: live-price check / fire cadence
TRIGGER_ARM_INTERVAL_SECONDS=20 # re-rank + recompute triggers cadence
TRIGGER_GAP_MIN=3.0 # min overnight gap %
# Cap the gap: blow-off gappers (>35%) fade — they're the worst-expectancy
# bucket. Train/test validated on 14 SIP-backfilled sessions: lowering 60->35
# lifted held-out TEST expectancy 0.19R -> 0.46R/trade AND raised total R.
TRIGGER_GAP_MAX=35
TRIGGER_RVOL_MIN=2.0 # min RELATIVE volume to fire
# Absolute liquidity gate: rvol is relative, so 2x of near-nothing still
# qualifies and we fill at the top of a thin one-bar spike that reverses. This
# requires recent 5-bar $-volume >= the floor so thin names go WEAK, not armed.
TRIGGER_MIN_DOLLAR_VOL=8000
# Do not fire a trigger on a price older than this (seconds) — a stalled feed
# must not fire a breakout on a frozen quote into an already-reverted market.
TRIGGER_PRICE_MAX_AGE=12
ORB_BARS=5 # opening range = first N regular-hours minutes
TRADING_TRIGGER_SLIP_PCT=0.004 # cap the chase above the trigger on the fill
# Account-aware sizing (matters most on a small REAL account):
TRADING_MAX_POSITION_PCT=0.16 # one position's $ value <= this fraction of equity
# PORTFOLIO gross-notional cap: total $ across ALL open+pending positions stays
# under this fraction of equity. Without it, 6 concurrent x 16% = ~96% gross in
# correlated small-cap gappers. With it, later entries shrink/blocked to fit.
TRADING_MAX_GROSS_NOTIONAL_PCT=0.60
# Liquidity-aware sizing: shares <= this fraction of the name's volume, so a
# thin name is sized DOWN automatically instead of taking a full position it
# can't absorb. 0 = off (fine at tiny size).
TRADING_LIQUIDITY_MAX_VOLUME_PCT=0.025
# ============================================================
# Risk discipline (added 2026-06-18 — see git history for the why)
# Local LLM news/catalyst enrichment (Ollama) — TEST SPOT
# Detected running at the Windows host (192.168.1.9:11434).
# Phase-1 advisory only; trading behaviour does NOT change. If Ollama is down or
# the model is missing it degrades to a no-op on its own thread (never impedes).
# ============================================================
# Anti-thrash: a breakout that won't fill on a marketable limit used to re-arm
# every pass forever (observed: NEOV armed+backed-out 68x/session). Bench it.
TRADING_BACKOUT_COOLDOWN_SECONDS=180 # bench this long after an unfilled backout
TRADING_MAX_BACKOUTS_PER_SYMBOL=3 # after N backouts, bench for the session
# Re-entry discipline: don't buy back a name that already round-tripped today...
TRADING_BLOCK_REENTRY_AFTER_EXIT=1
# ...but only if it exited at a real LOSS (>1%), so a scratch/win can re-enter
# (the blunt version killed a +513 re-entry winner in replay). 0 = bench any close.
TRADING_REENTRY_MIN_LOSS_PCT=0.01
# End-of-day flatten — a day-trading book shouldn't carry overnight gap risk.
TRADING_EOD_FLATTEN_ENABLED=1
TRADING_EOD_FLATTEN_TIME=15:55 # flatten in the window [time, 16:00)
OLLAMA_ENABLED=1
OLLAMA_HOST=http://192.168.1.5:30068 # direct to host (WSL tunnel flips; client also falls back)
OLLAMA_MODEL=qwen2.5:7b-instruct
OLLAMA_TIMEOUT_SECONDS=60
NEWS_ENRICH_ENABLED=1
NEWS_ENRICH_INTERVAL_SECONDS=120
NEWS_ENRICH_LOOKBACK_HOURS=12
NEWS_ENRICH_BATCH_LIMIT=50
# Phase-2 automation stays OFF until you've watched a clean advisory run:
NEWS_CATALYST_SCORE_ENABLED=1
NEWS_DILUTION_VETO_ENABLED=1
OLLAMA_TEMPERATURE=0.1
# ============================================================
# Managed exits (strategy/exits.py — swept + out-of-sample verified)
# The far take-profit is a backstop; the TRAILING STOP is the real exit.
# ============================================================
TRADING_REWARD_MULTIPLE=10.0 # far take-profit backstop (NOT 2.0; trail does the work)
TRADING_EXIT_TRAIL_MODE=prior_low # trail the stop under each prior bar's low
TRADING_EXIT_TRAIL_AFTER_R=1.0 # start trailing once the trade is +1R
TRADING_EXIT_BREAKEVEN_R=1.0 # lock breakeven once +1R (give-back safeguard)
TRADING_EXIT_PROFIT_TIERS=8:3,15:9,25:18,40:30 # +gain% -> lock% profit-lock ladder
TRADING_MIN_ENTRY_GAP_PCT=0.10 # POPULATION gate (year-long +EV boundary, n=3008/14mo): only real gappers (open vs prev close >= 10%) are tradeable; all rank x timing segments +0.05..+0.15R under the BE@+5% ratchet
TRADING_MIN_DAY_VOLUME=300000 # population volume floor (shares, consolidated SIP): thin names whipsaw; fail-open on missing data
TRADING_ENTRY_CONFIRM_BY_MINUTE=0 # OFF 2026-07-03: year-long replay (n=3008, 14mo) REFUTED the 15-min gate — late crosses beat early 12/14 months; population is the real +EV boundary (see below)
TRADING_MIN_QUALITY_SCORE=0.60 # C or higher (owner 2026-07-07 after F-tier experiment; grade map: A>=.9 B>=.75 C>=.6 D>=.4). Old 0.50 was mislabeled C+ — actually mid-D
# NOTE: several defaults above intentionally override the simpler values earlier
# in this file (e.g. MAX_CONCURRENT_POSITIONS, REWARD_MULTIPLE) — these reflect
# the current live-verified config. Set MAX_CONCURRENT_POSITIONS=6 for the live book.
TRADING_MAX_CONCURRENT_POSITIONS=6
# !!! Before REAL-money trading, set TRADING_MAX_DAILY_LOSS_PCT back to 0.03 !!!
# YouTube claim-extraction pipeline
YOUTUBE_API_KEY=<REDACTED>
YT_WATCHLIST=
ALPACA_DATA_FEED=sip # Algo Trader Plus (paid 2026-07-02, entitlement verified 07-03): real-time consolidated tape. Flip back to iex if the plan lapses — the ratio estimator self-reactivates.
+83
View File
@@ -0,0 +1,83 @@
name: docker-image
# Build + push the app image to GHCR on every master push, so pull-only
# hosts (TrueNAS SCALE custom apps) can deploy without a repo checkout.
on:
push:
branches: [master]
paths:
- "Dockerfile"
- "requirements.txt"
- "**.py"
- "api/static/**"
- ".github/workflows/docker.yml"
workflow_dispatch:
jobs:
# Gate: no image reaches a registry unless the suite is green. Production
# (TrueNAS) pulls :latest blindly, so this job is the only test run between
# a master push and the live loop.
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:18 # keep matched to the compose files' server image
env:
POSTGRES_USER: momentum
POSTGRES_PASSWORD: momentum
POSTGRES_DB: momentum
ports: ["5432:5432"]
options: >-
--health-cmd "pg_isready -U momentum" --health-interval 5s
--health-timeout 5s --health-retries 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.13" } # match Dockerfile's python:3.13-slim
- run: pip install -r requirements.txt ruff
# ruff F gate, scoped to the money path (tests/ and root-level scratch
# scripts excluded). All per-file pins have been cleared as their owning
# plans landed (api/main.py F821 os/hmac -> plan 007), so this is unpinned.
- run: >-
ruff check --select F
run_live_paper.py trading_execution.py telegram_bot.py
api/ alpaca_paper/ runtime/ storage/ strategy/ research/
- run: python -m pytest -q
env:
DATABASE_URL: postgresql://momentum:momentum@localhost:5432/momentum
# tests were written under an ET wall clock (the app image pins it too)
TZ: America/New_York
build:
needs: test
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- id: ver
run: echo "tag=$(date -u +%Y.%m.%d)-${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT"
- uses: docker/build-push-action@v6
with:
context: .
push: true
build-args: |
GIT_SHA=${{ github.sha }}
BUILD_TAG=${{ steps.ver.outputs.tag }}
tags: |
ghcr.io/fez5587/momentum-app:latest
ghcr.io/fez5587/momentum-app:${{ github.sha }}
ghcr.io/fez5587/momentum-app:${{ steps.ver.outputs.tag }}
fez5587/momentum-app:latest
fez5587/momentum-app:${{ github.sha }}
fez5587/momentum-app:${{ steps.ver.outputs.tag }}
+4
View File
@@ -58,3 +58,7 @@ momentum-rebuilt.zip
*.zip
archive/
uv.lock
.firecrawl/
# large research PDFs (local only)
books/
+52
View File
@@ -0,0 +1,52 @@
# Momentum System — container image
#
# Single image for every run mode (the app is a monolith run from the repo root
# with PYTHONPATH=/app, not a pip-installed wheel — see pyproject.toml
# `package = false`). The Kubernetes workloads override CMD:
# - app Deployment : python run_live_paper.py (loop + embedded dashboard)
# - CronJobs : python eod_replay.py / nightly_tune.py / momentum_cli.py journal
FROM python:3.13-slim
# psycopg2-binary ships its own libpq, so no build toolchain is needed; we only
# add a tiny client for healthchecks/debugging and clean up the apt cache.
# ffmpeg: yt-dlp audio extraction for the transcripts pipeline; curl: healthchecks
RUN apt-get update \
&& apt-get install -y --no-install-recommends libpq5 ffmpeg curl \
&& rm -rf /var/lib/apt/lists/*
# Image identity: CI passes the git sha + a date tag so the RUNNING app can
# report exactly what it is (TrueNAS custom apps show only 'custom/1.0.0').
ARG GIT_SHA=dev
ARG BUILD_TAG=dev
ENV APP_GIT_SHA=$GIT_SHA \
APP_BUILD_TAG=$BUILD_TAG
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH=/app \
PIP_NO_CACHE_DIR=1 \
TZ=America/New_York
WORKDIR /app
# Install dependencies first so they cache across code-only changes.
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt \
# transcripts pipeline (CPU whisper); heavy layer, cached separately
&& pip install --no-cache-dir faster-whisper yt-dlp
# Application code.
COPY . .
# Run as a non-root user that owns the writable data dir (DuckDB research file,
# logs, learned_params.json, schwab tokens). In K8s this path is a mounted PVC.
RUN useradd --create-home --uid 10001 app \
&& mkdir -p /app/data \
&& chown -R app:app /app/data
USER app
EXPOSE 8765
# Default to the headless loop; the chart's app Deployment runs it WITH the
# dashboard, and the CronJobs override this entirely.
CMD ["python", "run_live_paper.py", "--no-dashboard"]
+39 -20
View File
@@ -1,15 +1,17 @@
# Momentum
Event-sourced small-cap **momentum day-trading** system (Ross Cameron–style
gap-and-go / opening-range breakout). It screens the day's most-active gappers
($1–5, gapping ≥3% on ≥2× relative volume), arms an opening-range-breakout
trigger the moment the range forms, fires a bracket order on the live cross,
and manages the exit with a trailing stop + profit-lock ladder. Execution runs
on an **Alpaca paper** account; everything is recorded in a **PostgreSQL**
append-only event store and replayable.
gap-and-go / opening-range breakout). It screens the day's top overnight
gappers — the year-scale-validated population: **$2–20, static open gap
10–35%, real volume**, plus a volume-verified **>150% squeeze exception** —
arms an opening-range-breakout trigger the moment the range forms, fires a
bracket order on the live cross (regular session only), and manages the exit
with a breakeven ratchet + trailing profit-lock ladder. Execution runs on an
**Alpaca paper** account over the **SIP consolidated feed**; everything is
recorded in a **PostgreSQL** append-only event store and replayable.
```
Alpaca IEX bars ─► research DBs (minute+daily) ─► screen gappers ─► ArmedTriggerBook
Alpaca SIP bars ─► research DBs (minute+daily) ─► screen gappers ─► ArmedTriggerBook
(1-min, live) │ live price
▼ crosses trigger
PostgreSQL event store ◄────────────────────── TradingExecutionService ─► Alpaca paper
@@ -36,6 +38,15 @@ python run_live_paper.py # full loop (+ embedded dashboard)
No keys? `python run_live_paper.py --once` runs a single dry pass to verify wiring.
## Deployment
Three supported shapes — **Docker Compose is canonical** (production runs the
full stack self-contained on TrueNAS SCALE: postgres, loop, dashboard :8765,
admin panel :8767, telegram bot, transcripts, nightly scheduler). Local
systemd units remain the development shape; a Helm refresh is planned.
**See [deploy/README.md](deploy/README.md)** for all three, including the
behavior flags every deployment must set explicitly.
## Commands
Everything is driven from the repo root. The `./momentum` wrapper runs the
@@ -99,8 +110,8 @@ The `.sh` wrappers set the venv + cwd so cron can call them directly:
```bash
DATABASE_URL=postgresql://user:pass@host:5432/momentum python -m pytest
# ~130 tests: strategy, storage, triggers, execution (cooldown/re-entry/EOD
# flatten/naked-stop guards), exits, journal P&L, schwab, api, research.
# ~365 tests: strategy, storage, triggers, execution (cooldown/re-entry/EOD
# flatten/naked-stop guards), exits, journal P&L, api, research.
```
Tests isolate via throwaway `mem_*` Postgres schemas; `tests/conftest.py` sweeps
@@ -111,8 +122,11 @@ them at session end.
All behavior is environment-driven — see **`.env.example`** for every key with
the *why*. The knobs that matter most:
- **Entry:** `WATCHER_PRICE_MIN/MAX` (1–5), `TRIGGER_GAP_MIN/MAX`, `TRIGGER_RVOL_MIN`,
`TRIGGER_MIN_DOLLAR_VOL` (absolute liquidity floor — skip thin spikes).
- **Entry population** (year-scale validated 2026-07): `TRADING_PRICE_MIN/MAX`
(2–20), `TRADING_MIN_ENTRY_GAP_PCT` (0.10, static open gap), `TRIGGER_GAP_MAX`
(35 — the 35-150% band tested dead), `TRIGGER_GAP_SQUEEZE_MIN/RVOL` (150/5 —
re-admits real squeezes), `TRADING_MIN_DAY_VOLUME`, `TRADING_SESSION_ONLY`
(bars after-hours entries).
- **Sizing:** `TRADING_RISK_PER_TRADE_PCT` (1%), `TRADING_MAX_POSITION_PCT`,
`TRADING_LIQUIDITY_MAX_VOLUME_PCT` (size down thin names).
- **Exits:** `TRADING_EXIT_TRAIL_MODE=prior_low`, `TRADING_EXIT_TRAIL_AFTER_R`,
@@ -124,8 +138,9 @@ the *why*. The knobs that matter most:
## Database & performance
The single datastore is **PostgreSQL** (`DATABASE_URL`); research bars live in
DuckDB under `data/research/`. The event store is one append-only `events`
The single datastore is **PostgreSQL** (`DATABASE_URL`) — bars, events,
research tables, everything (the DuckDB era is over; `data/` holds only logs,
transcripts, and boot-time state). The event store is one append-only `events`
table that every projection reads.
**Hot-path index.** Projections overwhelmingly run
@@ -149,10 +164,10 @@ For ongoing tuning:
+ time partitioning + compression). Not required at current scale, and must be
installed on the server.
**Maintenance.** ~80% of `events` rows are high-volume diagnostics
(`criteria_evaluated`, `signal_blocked`, `module_tick`). A retention/prune of old
diagnostic events keeps the table small and every query fast. Run `ANALYZE events`
after large ingests so the planner has fresh stats.
**Maintenance is automated.** The nightly job (scheduler service / systemd
timer) prunes diagnostic events older than 14 days, rotates logs, refreshes the
day's bars to SIP truth, and updates the float/dilution shadow caches.
Autovacuum is tuned per-table; run `ANALYZE` manually only after bulk imports.
## Repository map
@@ -165,15 +180,19 @@ strategy/ pure strategy engine: evaluation, structure, exits, backte
storage/ event schema + store (PostgreSQL), projections
research/ research DBs, ingestion (bars, RSS, gappers), screeners
alpaca_paper/ Alpaca paper client, executor, account/order sync
schwab/ Schwab OAuth, market data, positions/orders, health
api/ + dashboard_api.py dashboard JSON API + static UI
api/ + dashboard_api.py dashboard JSON API + static UI (:8765)
admin_api.py admin panel (:8767): health, config overrides, profiles, restarts
youtube_claims/ trader-video transcript -> claim extraction pipeline
controlroom/ NiceGUI control room (disabled by owner preference)
deploy/ compose (canonical) / systemd (local) / helm (planned)
eod_replay.py end-of-day whole-market replay
docs/history/ point-in-time plans from earlier eras — don't follow them
tests/ pytest suite (unit + integration)
```
## Security
`.env` and all of `data/` (DBs, `data/schwab_tokens.json`) are git-ignored and
`.env` and all of `data/` are git-ignored and
never committed; token files are written `0600`. The `.env` holds **live broker /
database / webhook credentials** — keep it local. If any secret was ever
committed to a repo's history, **rotate it** (re-issue Alpaca keys / re-run the
+49 -209
View File
@@ -1,227 +1,67 @@
# RUNBOOK
Operational guide for running the momentum system. This file replaces the
original MILESTONE*.md documents, which were unrecoverable during the
rebuild.
Operating the Momentum stack day to day. Deployment shapes and setup live in
[deploy/README.md](deploy/README.md); this is "it's running — now what."
---
## Daily rhythm (all times ET)
## 1. Why live paper trading never worked (root causes, now fixed)
1. **Minute bars were never ingested.** The watcher reads 1-minute bars from
`data/research/market.duckdb`, but the old ingestion only wrote *daily*
bars for a handful of mega-caps. Fix: `research/ingestion/market_data.py`
ingests 1-minute Alpaca IEX bars continuously, and
`run_live_paper.py` schedules it every `LIVE_BARS_INTERVAL_SECONDS`.
2. **Session flags were hardcoded.** Every old row had
`is_regular_hours = TRUE`. Fix: flags are derived from the bar timestamp
in US/Eastern (premarket 04:00–09:30, regular 09:30–16:00, afterhours
16:00–20:00) — see `classify_session()` and its tests.
3. **The candidate filter excluded everything ingested.** The query had a
hardcoded `close BETWEEN 1 AND 20` while only mega-caps (price ≫ 20) were
in the database, so the watchlist was always empty. Fix: the band is
configurable (`WATCHER_PRICE_MIN` / `WATCHER_PRICE_MAX`) and the default
symbol list now actually gets ingested.
4. **No orchestrator.** Ingestion, watcher, broker sync, and execution were
never wired into one loop. Fix: `run_live_paper.py`.
5. **Broken source files.** `runtime/providers/registry.py`,
`schwab/market/models.py`, `option_chain_service.py`, and
`schwab/streaming/client.py` had syntax errors that crashed imports.
All repaired/rewritten.
6. **Schwab tokens were committed to git.** `data/schwab_tokens.json` is now
git-ignored and written `chmod 0600`. **Action required:** if that file
exists anywhere in your repo history, revoke the refresh token in the
Schwab developer portal and re-run the OAuth flow. Scrub history with
`git filter-repo --path data/schwab_tokens.json --invert-paths` if the
repo was ever shared.
---
## 2. Environment setup
```bash
pip install -r requirements.txt
cp .env.example .env
```
Minimum for paper trading: `ALPACA_API_KEY`, `ALPACA_SECRET_KEY`
(free keys are fine — the system uses the IEX feed). Everything else has a
sensible default; see `.env.example` for the full catalog.
Key knobs:
| Variable | Default | Meaning |
| When | What happens | Verify |
|---|---|---|
| `TRADING_EXECUTION_MODE` | `alpaca_paper` | `schwab_live` routes to real money — leave on paper |
| `TRADING_AUTO_APPROVE` | `0` | `1` = orders submit without dashboard approval (or pass `--auto-approve`) |
| `TRADING_MAX_ORDERS_PER_TICK` | `1` | throttle new entries |
| `TRADING_MAX_CONCURRENT_POSITIONS` | `3` | cap on simultaneous open positions |
| `TRADING_RISK_PER_TRADE_PCT` | `0.01` | 1% of equity risked per trade (sizing = risk / (entry − stop)) |
| `TRADING_REWARD_MULTIPLE` | `2.0` | profit target = entry + this × risk (2R by default) |
| `TRADING_ENTRY_ORDER_TYPE` | `limit` | `limit` rests at the trigger so an unfilled entry can be cancelled; `market` fills instantly |
| `TRADING_ENTRY_TIMEOUT_BARS` | `2` | cancel an unfilled entry after this many minutes resting (wall-clock; `0` disables the time box) |
| `TRADING_ENTRY_INVALIDATE_PCT` | `0.0` | cancel an unfilled entry if the **live last-trade price** trades this fraction below the trigger (`0.0` = any break below entry; `0.005` = 0.5% below; negative disables) |
| `TRADING_ENTRY_GUARD_INTERVAL_SECONDS` | `5` | how often the fast guard re-checks armed entries against the live price — lower = faster break detection, more quote calls |
| `TRADING_MAX_DAILY_LOSS_PCT` | `0.03` | daily-loss circuit breaker |
| `WATCHER_SYMBOLS` | `AAPL,TSLA,…` | static watchlist; `--discover` adds screener actives |
| `WATCHER_PRICE_MIN/MAX` | `1` / `20` | candidate price band |
| `WATCHER_EVENT_DB_PATH` | `./data/momentum.duckdb` | the event store |
| `DASHBOARD_HOST/PORT` | `127.0.0.1` / `8010` | UI address |
| ~04:00+ | premarket ingest + discovery begin | trigger board fills with gappers |
| 09:30 | session gate opens; entries allowed 09:30–15:55 | first `TRIGGER FIRED` / auto-approved orders |
| 09:35 | opening ranges complete → triggers arm | board states flip waiting→armed |
| 15:55 | fresh entries barred | — |
| 16:00± | EOD flatten of open positions | positions = 0 |
| 20:00 Mon–Fri | nightly job: SIP re-fetch, event retention, log rotation, shadow tracks, float/dilution caches | `appended:` line in scheduler log |
---
## 3. Start commands
**Full live paper loop (recommended):**
## Health checks
**Compose (production):**
```bash
python run_live_paper.py
# dashboard: http://127.0.0.1:8010
# flags: --once --symbols SNDL,COSM --auto-approve --discover --no-dashboard
docker ps # all momentum containers Up, loop (healthy)
docker logs -f <loop> # discover/ingest/watch/execute cycle lines
curl -s localhost:8765/api/snapshots | head -c 200 # dashboard serves
# admin panel: http://<host>:8767 — services, DB, Alpaca, Ollama cards
```
Each tick prints one status line:
**Local (systemd):** `systemctl --user status 'momentum-*'` + the same curls.
**Database truth (any shape):**
```sql
SELECT max(timestamp) FROM events; -- heartbeat: < ~2 min during any hour
SELECT count(*) FROM minute_bars; -- history intact: ~19M+ (2026-07)
```
[09:31:02] ingest: 240 minute rows | watch: evaluated=5 ready=['SNDL'] blocked=3 | sync: ok | execute: approvals_requested=['…'] auto_executed=0
```
Timestamps are **ET-naive**: compare against ET now, not UTC (a constant
"4-hour stale" reading is a timezone artifact, not an outage).
**Standalone read-only dashboard** (e.g. to review a past session):
## Failure modes seen in production (and the fix)
```bash
python dashboard_api.py
```
| Symptom | Cause | Fix |
|---|---|---|
| Entries queue as "manual" in the dashboard | `TRADING_AUTO_APPROVE` missing from the loop's env | add `=1` to the loop service env; stale queue rows are zombies that clear with the session |
| Orders submitted after hours | (fixed 2026-07-10) missing session gate | `TRADING_SESSION_ONLY=1` — default on; verify if reintroduced |
| Duplicate resting orders after a loop crash/restart | armed/submitted state is in-memory | diff Alpaca open orders after any loop restart; cancel dupes (`get_orders` unscoped — `symbols=` can silently return empty) |
| `PermissionError` on `/app/data/...` | host dir not owned by container uid | `chown -R 10001:10001 <data dir>` |
| Postgres serving a tiny DB when history should exist | wrong pgdata mount (path typo/case, nested cluster, stale container) | `docker inspect <postgres> --format '{{range .Mounts}}...'`; verify by **size at destination**; recreate containers after volume edits (restart is not enough) |
| Autovacuum cancelled every 60s on a big table | a frequent connector re-applying schema DDL | healthchecks/one-shot scripts must set `PG_SKIP_SCHEMA=1` |
| Env edits in the YAML do nothing | TrueNAS expanded the anchors; `x-app` is decorative | put env in each **service's** block; verify with `docker exec <svc> sh -c 'echo $VAR'` |
| Loop trading names it shouldn't | check gates in order: static open gap 10–35% (or ≥150% + rvol≥5), $2–20, day-vol, quality ≥0.60, VWAP, session window | `./momentum inspect criteria SYM` or the events `risk_rule_triggered` payloads |
Approve/reject/exit buttons require the orchestrator (it attaches the
execution service to the same server); the standalone runner returns 503 on
actions by design.
## Safety posture
**Other tools:**
- Paper account; bracket orders carry the protective stop; the exit manager
enforces stops on anything naked, ratchets breakeven at +5%, catastrophe-
stops at −10%/1.5×R.
- **When in doubt, stop the loop container** — positions stay protected by
broker-side stop legs; a missed day costs nothing (the no-trade day is a
legitimate outcome). Never run two loops against one account.
- Config experiments: use the admin panel's profiles; treat each switch as a
hypothesis and compare the day against `validated-core` behavior.
```bash
python fetch_minute_bars.py SNDL COSM --lookback 390 # ad-hoc ingestion
python research_cli.py symbols --date 2026-06-11 # what's in the DB
python research_cli.py gappers --date 2026-06-11 # gap scan → scanner_snapshots
python backtest_cli.py --date 2026-06-10 # backtest a stored session
python milestone3_verify.py # Schwab health walk
python -m pytest # full test suite
```
## Where things live
---
## 4. The trading day, end to end
1. **Ingest** — every 60 s, 1-minute IEX bars for the watchlist land in
`data/research/market.duckdb` with correct session flags. On boot, a
30-day daily backfill gives gap % and relative volume a baseline.
2. **Watch** — every 30 s the watcher pulls in-band candidates, evaluates
nine weighted criteria (gap, RVOL, impulse, pullback, pullback volume,
VWAP, candle quality, breakout, data sufficiency) plus structure
detection, and drives `discovered → watching → ready | blocked | late`.
A `signal_ready` is emitted at most once per symbol per session.
3. **Sync** — Alpaca account, positions, and orders snapshot into the event
store so projections and risk guards see broker truth.
4. **Execute** — ready signals become orders sized at `risk_per_trade_pct`
of equity, with the stop from the setup and the target at
entry + `reward_multiple` × risk (2R by default). Manual mode parks them
in the dashboard queue; **auto mode (`--auto-approve`) submits
immediately, no approval.** Guards: max concurrent positions, no re-entry
into held symbols, one request per symbol per session.
5. **Manage** — approve/reject from the dashboard; **Exit** sends a market
sell for the synced quantity.
### The entry lifecycle (auto-arm → fill or back out)
The earlier system would "see" a setup as something that had *already
happened* rather than a trade about to happen, and never pull the trigger.
The execution model now separates recognition from conviction from
discipline:
1. **Recognize** — when all the criteria pass, a `signal_ready` fires. The
setup is evaluated against the **last bar's timestamp**, not the wall
clock, so a live 9:40 breakout is judged against the 9:40 cutoff instead
of being dismissed as hours stale.
2. **Arm with conviction** — in auto mode the order is submitted right away
as a **bracket** (entry + stop + target placed atomically). With
`TRADING_ENTRY_ORDER_TYPE=limit` (default) the entry rests at the trigger,
so "unfilled" is a real, observable state.
3. **Back out at a defined point** — a fast **guard loop**
(`TRADING_ENTRY_GUARD_INTERVAL_SECONDS`, default 5 s) re-checks every
armed-but-unfilled entry and cancels it if either:
- the **live last-trade price** (pulled tick-by-tick from Alpaca's data
API, not a bar close) trades back below the trigger by
`TRADING_ENTRY_INVALIDATE_PCT`, or
- it has rested longer than `TRADING_ENTRY_TIMEOUT_BARS` minutes
(wall-clock — running the guard often does **not** shorten this window).
A cancel frees the risk budget and the concurrent-position slot, and emits
a `risk_rule_triggered` (`entry_backout`) you can see in the activity feed.
Once the order **fills**, it stops being tracked and the bracket's stop and
target manage the open position — no second-guessing.
Tuning cheat-sheet: react to a fakeout faster → lower
`TRADING_ENTRY_GUARD_INTERVAL_SECONDS` (e.g. `2`); more patience on the fill →
raise `TRADING_ENTRY_TIMEOUT_BARS`; give the breakout room to wobble → set
`TRADING_ENTRY_INVALIDATE_PCT` to something like `0.005`; chase fills instead
of resting → `TRADING_ENTRY_ORDER_TYPE=market` (note: market fills can't time
out); bigger winners → raise `TRADING_REWARD_MULTIPLE`.
> **Quote rate limits.** The guard makes one latest-trade call per armed
> symbol per pass (a ~2 s cache collapses duplicates). At the 5 s default with
> a few open entries you're well inside Alpaca's free-tier 200 req/min. If you
> drop the interval to 1 s with several armed entries, watch the budget —
> bars ingestion and account sync also draw from it.
### Reading the dashboard
- **Badge** — blue `PAPER · ALPACA` vs red `LIVE · SCHWAB`. If it's red and
you didn't mean it, stop and check `TRADING_EXECUTION_MODE`.
- **P&L strip** (top) — session realized / unrealized / total, win rate,
average R, open/closed counts. Green/red signed.
- **Approval Queue** — pending entries with a **risk preview** ($ risk,
shares, notional, stop %) and Approve/Reject. Keyboard: `A` / `R` act on the
top row.
- **Ready Signals / Open Positions** — each row has a live **sparkline** (last
~40 one-minute closes) with dashed entry (blue) and stop (red) lines.
- **Watch States** — click any row to expand the **nine-criteria breakdown**
(✓ pass / ✕ fail / · not-evaluated). The add-symbol box (or `/`) injects a
ticker onto the watchlist for the rest of the session.
- **Activity Feed** — submissions and fills as they happen.
- **Order Lifecycle** — per-order status from the event store.
- **Connection dot** (top right) — green pulse = live SSE stream; it falls
back to 4 s polling automatically if the stream drops.
- **Sound** — toggle the button or press `S` for an audio + toast alert when a
new signal goes ready. Press `?` for the shortcut list.
Streams from `GET /api/stream` (SSE); also exposes `/api/snapshots`,
`/api/criteria?symbol=`, `/api/bars?symbol=&minutes=`, and
`POST /api/watch/add`. Charts require the research DB, which the orchestrator
wires in automatically; the standalone `dashboard_api.py` has no bars source,
so sparklines there show "no bars".
---
## 5. Troubleshooting
| Symptom | Likely cause / fix |
|---|---|
| `watch: evaluated=0` all day | No minute bars for today. Check keys; run `python research_cli.py symbols`. Outside market hours there are no new bars — backfill with `fetch_minute_bars.py --lookback 1440`. |
| Candidates exist but never `ready` | Score < 60%. Watch the `Watch States` panel score column; criteria details are in `criteria_evaluated` events. |
| `ingest: … 403` errors | Free keys are IEX-only — ensure `ALPACA_DATA_FEED=iex` (default) and that the request window ends ≥30 s in the past (handled automatically). |
| Approve button does nothing | You're on `dashboard_api.py` (read-only). Use `run_live_paper.py`. |
| Port 8010 busy | Set `DASHBOARD_PORT`, or `--no-dashboard`. |
| Schwab shows `SCHWAB-UNAUTH` | Expected without OAuth. Run the flow in `schwab/auth/oauth2_flow_manager.py`; verify with `milestone3_verify.py`. |
| Start fresh | Delete `data/momentum.duckdb` (events) and/or `data/research/` (bars). Both rebuild automatically. |
---
## 6. Going live on Schwab (when ready)
Paper-trade until the stats earn it. Then: complete OAuth
(`milestone3_verify.py` all green), set `TRADING_EXECUTION_MODE=schwab_live`,
keep `TRADING_AUTO_APPROVE=0`, and start with the smallest viable
`TRADING_RISK_PER_TRADE_PCT`. The dashboard badge turns red in live mode.
- **Everything queryable**: PostgreSQL (`DATABASE_URL`) — events, bars,
research, shadow tracks, caches.
- **Local disk / app-data volume**: logs (rotated nightly), transcripts,
`learned_params.json`, `overrides.env` (admin panel), `profiles/`.
- **Historical plans/era docs**: `docs/history/` — don't follow them.
+437
View File
@@ -0,0 +1,437 @@
"""Momentum ADMIN panel (:8767) — configure, validate, and control the stack.
LAN-only by intent; every mutating endpoint (and the page itself) requires the
ADMIN_TOKEN (env). Capabilities:
- service health: docker containers (via /var/run/docker.sock), DB, Alpaca,
Ollama, loop heartbeat, market clock, data feed
- config: read/edit the whitelisted strategy knobs via data/overrides.env
(beats compose env on next boot; see runtime/env_overrides.py); secrets are
shown as present/absent only, never values
- profiles: named knob presets (data/profiles/*.json) applied in one click
- restart: bounce individual containers so config changes take effect
- paper/live switch: guarded — refuses unless ALPACA_LIVE_* keys are set
Run: python admin_api.py (ADMIN_HOST/ADMIN_PORT, default 0.0.0.0:8767)
"""
from __future__ import annotations
import hmac
import http.client
import json
import os
import socket
import subprocess
import sys
import urllib.request
from datetime import datetime
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from dotenv import load_dotenv
if __name__ == "__main__":
# Load the repo .env ONLY when this file is run as the entrypoint
# (`python admin_api.py`). A bare module-level load_dotenv() here used to
# fire on every import too -- including pytest collection -- leaking the
# real repo .env (secrets included) into any process that merely imports
# this module. This runs before the os.environ.get(...) constants below,
# so `python admin_api.py` still behaves identically.
load_dotenv()
from runtime.env_overrides import load as _load_overrides # noqa: E402
_load_overrides()
from runtime import env_overrides # noqa: E402
DOCKER_SOCK = os.environ.get("DOCKER_SOCK", "/var/run/docker.sock")
# Regime backtest (plan 011) on-demand trigger — logs to the app-data volume so
# GET /api/regime/status can tail it (the panel had no way to fire this).
REGIME_LOG = os.environ.get(
"REGIME_RUN_LOG", os.path.join(os.path.dirname(__file__), "data", "regime_run.log"))
_REGIME_DONE = "REGIME PIPELINE DONE"
def run_regime_pipeline() -> int:
"""Fire the full regime shadow pipeline DETACHED (it takes a few min): rebuild
session labels -> walk-forward score both models -> replay + adjudicate the
regime_gate experiments. Returns the child pid. Shadow only — never touches a
live order. stdout+stderr -> REGIME_LOG (truncated each run so status shows
the CURRENT run); a trailing marker lets status report completion."""
py = sys.executable
steps = " && ".join([
f"{py} -m research.regime build-features",
f"{py} -m research.regime score --model threshold_v1",
f"{py} -m research.regime score --model meta_v1",
f"{py} -m research.experiments run",
f"{py} -m research.experiments adjudicate",
f"echo {_REGIME_DONE}",
])
os.makedirs(os.path.dirname(REGIME_LOG), exist_ok=True)
log = open(REGIME_LOG, "wb") # noqa: SIM115 — handed to the detached child
proc = subprocess.Popen(["/bin/sh", "-c", steps], stdout=log,
stderr=subprocess.STDOUT, start_new_session=True)
return proc.pid
def regime_status() -> dict:
"""Tail REGIME_LOG + whether the last run finished (the marker) — no DB query
needed; the adjudicate step prints each experiment's verdict into the log."""
try:
with open(REGIME_LOG, encoding="utf-8", errors="replace") as fh:
lines = fh.read().splitlines()
except FileNotFoundError:
return {"log_tail": [], "running": False, "ran": False}
return {"log_tail": lines[-40:], "ran": True,
"running": _REGIME_DONE not in lines[-3:] if lines else False}
PROFILES_DIR = os.environ.get(
"PROFILES_DIR", os.path.join(os.path.dirname(__file__), "data", "profiles"))
ADMIN_TOKEN = os.environ.get("ADMIN_TOKEN", "")
# knobs the UI may read+write — everything else is off-limits by construction
EDITABLE_PREFIXES = ("TRADING_", "TRIGGER_", "WATCHER_", "LIVE_BARS_", "NEWS_",
"OLLAMA_", "DISCOVER_", "SYMBOL_WEIGHTS")
EDITABLE_EXACT = {"ALPACA_DATA_FEED"}
SECRET_MARKERS = ("KEY", "SECRET", "TOKEN", "PASSWORD", "WEBHOOK", "URL")
SEED_PROFILES = {
"validated-core": {
"_note": "The year-validated population config (2026-07-09 state).",
"TRADING_MIN_QUALITY_SCORE": "0.60", "TRADING_MAX_FRESH_ENTRIES_PER_DAY": "10",
"TRADING_MIN_ENTRY_GAP_PCT": "0.10", "TRADING_MIN_DAY_VOLUME": "300000",
"TRADING_PRICE_MIN": "2.0", "TRADING_PRICE_MAX": "20.0",
"TRIGGER_GAP_MIN": "10.0", "TRIGGER_GAP_MAX": "35",
"TRIGGER_GAP_SQUEEZE_MIN": "150", "TRIGGER_GAP_SQUEEZE_RVOL": "5.0",
},
"conservative": {
"_note": "Fewer, stricter entries: B-grade floor, half the daily cap, no squeeze lottery.",
"TRADING_MIN_QUALITY_SCORE": "0.75", "TRADING_MAX_FRESH_ENTRIES_PER_DAY": "5",
"TRADING_MIN_ENTRY_GAP_PCT": "0.10", "TRADING_MIN_DAY_VOLUME": "300000",
"TRADING_PRICE_MIN": "2.0", "TRADING_PRICE_MAX": "20.0",
"TRIGGER_GAP_MIN": "10.0", "TRIGGER_GAP_MAX": "35",
"TRIGGER_GAP_SQUEEZE_MIN": "0",
},
"squeeze-hunter": {
"_note": "Experiment: lean into the >150% volume-verified squeeze band + looser quality.",
"TRADING_MIN_QUALITY_SCORE": "0.50", "TRADING_MAX_FRESH_ENTRIES_PER_DAY": "10",
"TRADING_MIN_ENTRY_GAP_PCT": "0.10", "TRADING_MIN_DAY_VOLUME": "300000",
"TRADING_PRICE_MIN": "2.0", "TRADING_PRICE_MAX": "20.0",
"TRIGGER_GAP_MIN": "10.0", "TRIGGER_GAP_MAX": "35",
"TRIGGER_GAP_SQUEEZE_MIN": "150", "TRIGGER_GAP_SQUEEZE_RVOL": "3.0",
},
}
def _safe_overrides() -> dict:
"""Overrides for API responses with secret VALUES redacted — a /api/mode live
switch writes ALPACA_LIVE_* into overrides.env; without this they'd echo back
in every /api/status and /api/config response."""
merged = {**env_overrides.read_overrides(), **env_overrides.read_db_overrides()}
return {k: ("***" if any(m in k for m in SECRET_MARKERS) else v)
for k, v in merged.items()}
def _editable(key: str) -> bool:
if key in EDITABLE_EXACT:
return True
return (key.startswith(EDITABLE_PREFIXES)
and not any(m in key for m in SECRET_MARKERS))
def _mask_value(key: str, value: str) -> str:
"""Mask secret VALUES (*** + last 4). Uses the broader SECRET_MARKERS set so
DATABASE_URL / *_WEBHOOK_URL are masked too — narrowing to KEY|SECRET|TOKEN|
PASSWORD would leak the DB password in the config table."""
if any(m in key.upper() for m in SECRET_MARKERS):
tail = value[-4:] if value and len(value) >= 4 else ""
return "***" + tail
return value
def _env_example_keys() -> set[str]:
"""Keys documented in .env.example — the 'required' config surface used for
drift detection (a documented key absent from the effective env is drift)."""
path = os.path.join(os.path.dirname(__file__), ".env.example")
keys: set[str] = set()
try:
with open(path) as fh:
for line in fh:
line = line.strip()
if line and not line.startswith("#") and "=" in line:
keys.add(line.split("=", 1)[0].strip())
except Exception: # noqa: BLE001
pass
return keys
def effective_config() -> dict:
"""Per-key effective value + source for the admin config table.
Merge order baked < compose < overrides.env (< learned). Sources we can prove
at runtime: ``overrides.env`` (from data/overrides.env) vs ``compose`` (the
process env — baked and compose are indistinguishable inside the container, so
both surface as ``compose``); a documented .env.example key that is absent
surfaces as ``missing`` (drift). Secret values are masked."""
overrides = env_overrides.read_overrides()
db_overrides = env_overrides.read_db_overrides()
example = _env_example_keys()
keys = (set(example) | set(overrides) | set(db_overrides)
| {k for k in os.environ if _editable(k)})
out: dict[str, dict] = {}
for key in sorted(keys):
value = os.environ.get(key)
if value is None:
out[key] = {"value_masked": None,
"source": "missing" if key in example else "unset",
"drift": key in example}
continue
source = ("database" if key in db_overrides
else "overrides.env" if key in overrides else "compose")
out[key] = {"value_masked": _mask_value(key, str(value)), "source": source}
return out
# ---------- docker over the unix socket (stdlib only) ----------
class _SockConn(http.client.HTTPConnection):
def __init__(self):
super().__init__("localhost")
def connect(self):
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect(DOCKER_SOCK)
self.sock = s
def docker(method: str, path: str) -> tuple[int, object]:
try:
c = _SockConn()
c.request(method, path)
r = c.getresponse()
body = r.read()
return r.status, (json.loads(body) if body else None)
except Exception as exc: # noqa: BLE001
return 0, {"error": str(exc)}
def list_containers() -> list[dict]:
st, data = docker("GET", "/containers/json?all=1")
if st != 200 or not isinstance(data, list):
return []
out = []
for c in data:
name = (c.get("Names") or ["?"])[0].lstrip("/")
if "momentum" not in name and "postgres" not in name:
continue
out.append({
"name": name,
"state": c.get("State"),
"status": c.get("Status"),
"health": ("healthy" if "(healthy)" in (c.get("Status") or "")
else "unhealthy" if "(unhealthy)" in (c.get("Status") or "")
else None),
"image": c.get("Image"),
"id": c.get("Id", "")[:12],
})
return sorted(out, key=lambda x: x["name"])
# ---------- health checks ----------
def checks() -> dict:
out: dict = {}
try:
from research.multi_schema import open_research_db
con = open_research_db("market")
r = con.execute("SELECT max(timestamp) FROM events").fetchall()[0][0]
age = (datetime.now() - r).total_seconds() if r else None
out["db"] = {"ok": True, "newest_event_age_s": round(age) if age else None}
except Exception as exc: # noqa: BLE001
out["db"] = {"ok": False, "error": str(exc)[:120]}
try:
# active execution broker (BROKER env) — JSON key stays "alpaca" for
# UI compatibility; "name" carries the real broker label
from tradier_broker.factory import make_trading_client
c, label = make_trading_client()
if c is None:
out["alpaca"] = {"ok": False, "error": label}
else:
clock = c.get_clock()
acct = c.get_account()
out["alpaca"] = {"ok": True, "market_open": clock.get("is_open"),
"equity": acct.get("equity"), "name": label,
"mode": "LIVE" if label.endswith("LIVE") else "paper"}
except Exception as exc: # noqa: BLE001
out["alpaca"] = {"ok": False, "error": str(exc)[:120]}
try:
host = os.environ.get("OLLAMA_HOST", "http://192.168.1.9:11434")
with urllib.request.urlopen(host + "/api/tags", timeout=4) as r:
out["ollama"] = {"ok": r.status == 200}
except Exception: # noqa: BLE001
out["ollama"] = {"ok": False, "note": "enrichment fails safe; trading unaffected"}
out["feed"] = os.environ.get("ALPACA_DATA_FEED", "iex")
return out
# ---------- profiles ----------
def seed_profiles() -> None:
try:
os.makedirs(PROFILES_DIR, exist_ok=True)
for name, body in SEED_PROFILES.items():
p = os.path.join(PROFILES_DIR, f"{name}.json")
if not os.path.exists(p):
json.dump(body, open(p, "w"), indent=2)
except PermissionError:
# host volume not writable by the container user (uid 10001) — degrade
# instead of crash-looping; profiles/config editing will 500 until the
# host dir is chown'd (see deploy/docker/README.md)
print(f"[admin] WARNING: {PROFILES_DIR} not writable (chown -R 10001:10001 "
"the data dir on the host); profiles/config editing disabled")
def profiles() -> dict:
seed_profiles()
out = {}
for f in sorted(os.listdir(PROFILES_DIR)):
if f.endswith(".json"):
try:
out[f[:-5]] = json.load(open(os.path.join(PROFILES_DIR, f)))
except Exception: # noqa: BLE001
pass
return out
# ---------- HTTP ----------
class Handler(BaseHTTPRequestHandler):
def _send(self, code: int, body, ctype="application/json"):
raw = body if isinstance(body, bytes) else json.dumps(body, default=str).encode()
self.send_response(code)
self.send_header("Content-Type", ctype)
self.send_header("Content-Length", str(len(raw)))
self.end_headers()
self.wfile.write(raw)
def _authed(self) -> bool:
# FAIL CLOSED: no token configured => deny (this panel fronts the docker
# socket + the paper/live switch; an unset env var must not open it to the
# LAN). Header only — a ?token= query would leak into proxy/access logs.
if not ADMIN_TOKEN:
return False
return hmac.compare_digest(self.headers.get("X-Admin-Token", ""), ADMIN_TOKEN)
def do_GET(self): # noqa: N802
path = self.path.split("?")[0]
if path == "/":
page = os.path.join(os.path.dirname(__file__), "api", "static", "admin.html")
return self._send(200, open(page, "rb").read(), "text/html")
if not self._authed():
return self._send(401, {"error": "bad or missing X-Admin-Token"})
if path == "/api/status":
return self._send(200, {"containers": list_containers(), "checks": checks(),
"overrides": _safe_overrides(),
"version": os.environ.get("APP_BUILD_TAG", "dev"),
"git_sha": os.environ.get("APP_GIT_SHA", "dev")[:12]})
if path == "/api/config":
eff = {k: v for k, v in sorted(os.environ.items()) if _editable(k)}
secrets = {k: bool(os.environ.get(k)) for k in
("ALPACA_API_KEY", "ALPACA_SECRET_KEY", "ALPACA_LIVE_API_KEY",
"ALPACA_LIVE_SECRET_KEY", "TELEGRAM_BOT_TOKEN", "YOUTUBE_API_KEY",
"FMP_API_KEY")}
return self._send(200, {"effective": eff, "secrets_present": secrets,
"overrides": _safe_overrides()})
if path == "/api/profiles":
return self._send(200, profiles())
if path == "/api/config/effective":
return self._send(200, {"config": effective_config()})
if path == "/api/regime/status":
return self._send(200, regime_status())
return self._send(404, {"error": "unknown route"})
def do_POST(self): # noqa: N802
if not self._authed():
return self._send(401, {"error": "bad or missing X-Admin-Token"})
n = int(self.headers.get("Content-Length") or 0)
try:
body = json.loads(self.rfile.read(n) or b"{}")
except Exception: # noqa: BLE001
return self._send(400, {"error": "bad json"})
path = self.path.split("?")[0]
if path == "/api/config":
updates = body.get("set") or {}
deletes = body.get("delete") or []
bad = [k for k in list(updates) + list(deletes) if not _editable(k)]
if bad:
return self._send(400, {"error": f"not editable: {bad}"})
# knobs go to the DATABASE (owner-facing store; whitelist above
# already refused secrets). overrides.env keeps secrets/bootstrap.
try:
merged = env_overrides.write_db_overrides(
{**{k: str(v) for k, v in updates.items()},
**{k: None for k in deletes}})
except Exception as exc: # noqa: BLE001
return self._send(502, {"error": f"db write failed: {exc}"})
return self._send(200, {"ok": True, "overrides": merged,
"note": "restart affected services to apply"})
if path == "/api/profiles/apply":
name = body.get("name", "")
prof = profiles().get(name)
if not prof:
return self._send(404, {"error": f"no profile {name!r}"})
knobs = {k: str(v) for k, v in prof.items() if not k.startswith("_")}
merged = env_overrides.write_db_overrides(knobs)
return self._send(200, {"ok": True, "applied": name, "overrides": merged,
"note": "restart the loop to apply"})
if path == "/api/restart":
name = body.get("service", "")
target = next((c for c in list_containers() if c["name"] == name), None)
if not target:
return self._send(404, {"error": f"no container {name!r}"})
st, _ = docker("POST", f"/containers/{target['id']}/restart?t=10")
return self._send(200 if st in (200, 204) else 502,
{"ok": st in (200, 204), "docker_status": st})
if path == "/api/mode":
# paper/live switch — guarded: live requires ALPACA_LIVE_* keys present
mode = body.get("mode")
if mode == "live":
if not (os.environ.get("ALPACA_LIVE_API_KEY")
and os.environ.get("ALPACA_LIVE_SECRET_KEY")):
return self._send(400, {"error": "ALPACA_LIVE_API_KEY/SECRET not set — "
"refusing to switch to live"})
merged = env_overrides.write_overrides({
"ALPACA_BASE_URL": "https://api.alpaca.markets",
"ALPACA_API_KEY": os.environ["ALPACA_LIVE_API_KEY"],
"ALPACA_SECRET_KEY": os.environ["ALPACA_LIVE_SECRET_KEY"]})
elif mode == "paper":
merged = env_overrides.write_overrides({
"ALPACA_BASE_URL": None, "ALPACA_API_KEY": None,
"ALPACA_SECRET_KEY": None})
else:
return self._send(400, {"error": "mode must be paper|live"})
return self._send(200, {"ok": True, "mode": mode, "overrides": merged,
"note": "restart the loop to apply"})
if path == "/api/regime/run":
try:
pid = run_regime_pipeline()
except Exception as e: # noqa: BLE001
return self._send(500, {"ok": False, "error": str(e)[:200]})
return self._send(200, {"ok": True, "pid": pid, "log": REGIME_LOG,
"note": "regime shadow pipeline started (~a few min); "
"poll GET /api/regime/status"})
return self._send(404, {"error": "unknown route"})
def log_message(self, *a): # quiet
pass
def main() -> None:
host = os.environ.get("ADMIN_HOST", "0.0.0.0")
port = int(os.environ.get("ADMIN_PORT", "8767"))
seed_profiles()
print(f"[admin] serving http://{host}:{port} "
f"(token {'SET' if ADMIN_TOKEN else 'NOT SET — open access'}; "
f"docker sock {'ok' if os.path.exists(DOCKER_SOCK) else 'MISSING'})")
ThreadingHTTPServer((host, port), Handler).serve_forever()
if __name__ == "__main__":
main()
+60 -6
View File
@@ -137,8 +137,14 @@ class AlpacaPaperClient:
continue
raise AlpacaApiError(exc.code, exc.read().decode(errors="replace")) from exc
except urllib.error.URLError:
# transient network/DNS (e.g. gaierror) — retry a couple times
if attempt >= self.max_retries:
# transient network/DNS (e.g. gaierror) — retry a couple times.
# NOT for POST: urllib wraps a lost RESPONSE (read timeout/reset)
# in URLError too, so attempt 1 may have SUCCEEDED at the broker
# with the reply lost; re-sending POST /orders then rests a
# duplicate live order (or 422s and looks like a failure while a
# real bracket rests untracked). Idempotent GET/DELETE retry; POST
# raises so the caller reconciles against broker positions.
if method == "POST" or attempt >= self.max_retries:
raise
_time.sleep(0.4 * (attempt + 1))
raise RuntimeError("request retries exhausted") # unreachable
@@ -162,6 +168,11 @@ class AlpacaPaperClient:
def get_account(self) -> dict:
return self._cached("account", lambda: self._trading("GET", "/account"))
def get_clock(self) -> dict:
"""Broker market clock: {is_open, next_open, next_close, timestamp}. The
authoritative open/closed check (handles weekends, holidays, half-days)."""
return self._trading("GET", "/clock")
def get_positions(self, fresh: bool = False) -> list[dict]:
# fresh=True bypasses the TTL cache — REQUIRED for the naked-stop guard,
# which must never decide an order is unfilled off a stale snapshot.
@@ -172,9 +183,23 @@ class AlpacaPaperClient:
return val
return self._cached("positions", lambda: self._trading("GET", "/positions"))
def get_order_by_client_order_id(self, client_order_id: str) -> dict | None:
"""The single order carrying this client_order_id, or None if unknown.
Used to reconcile a LOST submit response (network drop after the POST
left): the order may actually be resting at the broker, so we look it up
by the client_order_id we sent before assuming it was cancelled. A 404
(never landed) surfaces as an AlpacaApiError the caller treats as 'not
found'. `_trading` already prefixes /v2; the literal ':' in the path is
Alpaca's by-client-id sub-resource, not a typo."""
return self._trading(
"GET", "/orders:by_client_order_id",
params={"client_order_id": client_order_id},
)
def get_orders(
self, status: str = "all", limit: int = 500, nested: bool = True,
symbols: list[str] | None = None,
symbols: list[str] | None = None, after: str | None = None,
) -> list[dict]:
# nested=true includes bracket child legs (stop/target) so working risk
# isn't under-reported in the orders snapshot.
@@ -188,6 +213,8 @@ class AlpacaPaperClient:
# the exit manager / safety checks must do.
params: dict = {"status": status, "limit": min(limit, 500),
"nested": str(nested).lower(), "direction": "desc"}
if after: # only orders submitted after this ISO timestamp (exclusive) — the cage-count reconcile scopes to today
params["after"] = after
if symbols:
# scoped to a few names — well under the cap, one page is complete
params["symbols"] = ",".join(symbols)
@@ -224,6 +251,7 @@ class AlpacaPaperClient:
order_type: str = "market",
time_in_force: str = "day",
limit_price: float | None = None,
stop_price: float | None = None,
stop_loss_price: float | None = None,
take_profit_price: float | None = None,
client_order_id: str | None = None,
@@ -238,6 +266,8 @@ class AlpacaPaperClient:
}
if limit_price is not None:
body["limit_price"] = str(round(limit_price, 2))
if stop_price is not None: # standalone stop / stop-limit trigger
body["stop_price"] = str(round(stop_price, 2))
if client_order_id:
body["client_order_id"] = client_order_id
if stop_loss_price is not None or take_profit_price is not None:
@@ -353,6 +383,26 @@ class AlpacaPaperClient:
)
return payload.get("most_actives") or []
def get_assets(self, status: str = "active") -> list[dict]:
"""All tradable assets (trading API). Used by the gapper-history reconstruction
to enumerate the listed US-equity universe (~10-11k symbols)."""
payload = self._trading("GET", "/assets",
params={"status": status, "asset_class": "us_equity"})
return payload if isinstance(payload, list) else []
def get_movers(self, top: int = 15) -> list[dict]:
"""Top %-change gainers (the 'wildcard' movers). Ranked by PERCENT change, not
share volume — a thin reverse-split name squeezing on a headline (CX on
2026-07-02: $3->$6, the source trader's whole day, invisible to
most-actives-by-volume) shows up here and nowhere else.
Items: {symbol, percent_change, change, price}."""
payload = self._data(
"GET",
"/v1beta1/screener/stocks/movers",
params={"top": top},
)
return payload.get("gainers") or []
def get_news(
self,
symbols: list[str] | None = None,
@@ -376,9 +426,13 @@ class AlpacaPaperClient:
return payload.get("news") or []
def get_daily_bars(
self, symbols: list[str], start_iso: str, end_iso: str | None = None
self, symbols: list[str], start_iso: str, end_iso: str | None = None,
feed: str | None = None,
) -> dict[str, list[dict]]:
"""Daily bars, batching symbols (cap-safe) + following pagination."""
"""Daily bars, batching symbols (cap-safe) + following pagination.
`feed`: None -> settings.feed (iex on free keys, ~2-5% of true volume).
Pass "sip" for consolidated-tape volume (free keys allow historical SIP)."""
all_bars: dict[str, list[dict]] = {}
for chunk in self._chunked(symbols, 100):
page_token = None
@@ -392,7 +446,7 @@ class AlpacaPaperClient:
"start": start_iso,
"end": end_iso,
"limit": 10_000,
"feed": self.settings.feed,
"feed": feed or self.settings.feed,
"adjustment": "raw",
"page_token": page_token,
},
+85 -4
View File
@@ -93,6 +93,9 @@ class AlpacaPaperExecutor:
):
self.store = store
self.client = client or AlpacaPaperClient()
# honest broker label in journaled events when a mirror client
# (e.g. TradierClient) is injected
self.broker_name = getattr(self.client, "broker_name", "alpaca_paper")
self.session_id = session_id
self.mode = mode
@@ -122,12 +125,47 @@ class AlpacaPaperExecutor:
ok=False, order_id=request.order_id, error=str(exc)
)
except Exception as exc:
# LOST RESPONSE: the POST may have LANDED before the connection dropped,
# leaving a real order resting at the broker. Reconcile by the
# client_order_id we sent before assuming it was cancelled — journalling
# a live order as 'cancelled' is how a duplicate/naked order is born
# (the 2026-07-06 incident). Only the generic/transport error path
# reconciles; an AlpacaApiError above is a genuine broker rejection.
logger.exception("alpaca order failed")
self._emit_cancelled(request, f"submit error: {exc}")
recovered = self._reconcile_lost_submit(request)
if recovered is not None:
logger.warning("recovered lost submit for %s via client_order_id "
"(broker status=%s)", request.symbol, recovered.get("status"))
return self._finalize(request, recovered)
self._emit_cancelled(request, f"submit error (reconcile: not found): {exc}")
return ExecutionResult(
ok=False, order_id=request.order_id, error=str(exc)
)
return self._finalize(request, raw)
_LIVE_STATUSES = {"new", "accepted", "pending_new", "partially_filled",
"filled", "held", "accepted_for_bidding"}
def _reconcile_lost_submit(self, request: "ExecutionRequest") -> dict | None:
"""Look up the order by the client_order_id we sent; return it only if it
is actually LIVE at the broker (so a lost response isn't journaled as a
cancel). Returns None on 404/lookup error — then the caller emits cancelled."""
getter = getattr(self.client, "get_order_by_client_order_id", None)
if getter is None:
return None
try:
found = getter(request.order_id[:48])
except Exception: # noqa: BLE001 — 404 (never landed) or transport error
return None
if found and str(found.get("status")) in self._LIVE_STATUSES:
return found
return None
def _finalize(self, request: "ExecutionRequest", raw: dict) -> ExecutionResult:
"""Record a submission's outcome from the broker response (or a recovered
order) and return the ExecutionResult — one event sequence for the normal
and the reconcile-recovered paths alike."""
broker_order_id = raw.get("id")
status = raw.get("status")
@@ -147,7 +185,7 @@ class AlpacaPaperExecutor:
correlation_id=self.session_id,
message=(
f"Submitted {request.side} {request.quantity} {request.symbol} "
f"to alpaca_paper ({status})"
f"to {self.broker_name} ({status})"
),
symbol=request.symbol,
order_id=request.order_id,
@@ -181,11 +219,13 @@ class AlpacaPaperExecutor:
order_id=request.order_id,
fill_price=fill_price or 0.0,
fill_quantity=filled_qty,
side=request.side,
payload={
"symbol": request.symbol,
"order_id": request.order_id,
"fill_price": fill_price,
"fill_quantity": filled_qty,
"side": request.side,
},
)
)
@@ -211,16 +251,57 @@ class AlpacaPaperExecutor:
def cancel_entry(
self, order_id: str, broker_order_id: str | None, symbol: str, reason: str
) -> ExecutionResult:
"""Cancel a resting (unfilled) entry order at the broker and record it."""
"""Cancel a resting (unfilled) entry order at the broker and record it.
CANCEL-vs-FILL RACE: the entry can fill at the broker before (or as)
the cancel lands — the cancel then no-ops but journaling only a cancel
orphans a REAL position (untracked shares, the 2026-07-06 class;
happened live on Tradier sandbox 2026-07-28: lazy fills + entry
timeouts). After cancelling, re-check the broker's final word and emit
the fill for any executed shares so the position is tracked."""
try:
if broker_order_id:
self.client.cancel_order(broker_order_id)
except AlpacaApiError as exc:
# already filled/cancelled at the broker, or transient — log and
# still emit the cancel so our state stops tracking it
# continue to the fill-race check below
logger.warning("cancel of %s returned: %s", order_id, exc)
except Exception as exc: # noqa: BLE001
logger.warning("cancel of %s errored: %s", order_id, exc)
filled_qty, fill_price, side = 0, None, "buy"
getter = getattr(self.client, "get_order_by_client_order_id", None)
if getter is not None:
try:
found = getter(order_id[:48]) or {}
filled_qty = int(float(found.get("filled_qty") or 0))
fill_price = float(found.get("filled_avg_price") or 0) or None
side = found.get("side") or "buy"
except Exception: # noqa: BLE001 — lookup failure: keep cancel-only path
pass
if filled_qty > 0:
logger.warning("cancel of %s raced a fill: %d %s executed @ %s — "
"journaling the fill", order_id, filled_qty, symbol, fill_price)
self.store.emit(
OrderFilledEvent(
timestamp=datetime.now(),
mode=self.mode,
correlation_id=self.session_id,
message=f"Filled {symbol} {filled_qty} @ {fill_price} "
f"(cancel raced fill)",
symbol=symbol,
order_id=order_id,
fill_price=fill_price or 0.0,
fill_quantity=filled_qty,
side=side,
payload={"symbol": symbol, "order_id": order_id,
"fill_price": fill_price, "fill_quantity": filled_qty,
"side": side, "cancel_raced_fill": True},
)
)
return ExecutionResult(ok=True, order_id=order_id, status="filled")
self._emit_cancelled(
ExecutionRequest(symbol=symbol, side="buy", quantity=0, order_id=order_id),
reason,
+116 -9
View File
@@ -8,6 +8,7 @@ from __future__ import annotations
import logging
from datetime import datetime
from zoneinfo import ZoneInfo
import json
@@ -31,6 +32,7 @@ _EXIT_REASON = {
logger = logging.getLogger(__name__)
BROKER_NAME = "alpaca_paper"
_ET = ZoneInfo("America/New_York")
class AlpacaPaperSync:
@@ -45,10 +47,21 @@ class AlpacaPaperSync:
self.client = client or AlpacaPaperClient()
self.session_id = session_id
self.mode = mode
# Label events by the ACTUAL execution broker (BROKER=tradier swaps the
# client): the executor filters position snapshots by client.broker_name,
# so a hardcoded "alpaca_paper" here made those queries return empty on
# the Tradier stack (position cap / re-entry block saw zero positions).
self.broker_name = getattr(self.client, "broker_name", BROKER_NAME)
self.account_desc = (
"Tradier" if self.broker_name == "tradier" else "Alpaca Paper")
# last seen open positions, to detect closes by diffing snapshots. None
# until the first sync, when it's seeded from the last PERSISTED snapshot
# so a close that happened while the loop was down is still journaled.
self._prev_positions: list[dict] | None = None
# exit order ids already journaled as position_closed (position_id), so the
# orders-based pass below never double-counts an exit the snapshot diff
# (or an earlier run) already recorded. None until seeded from the store.
self._journaled_exits: set[str] | None = None
def _emit_health(self, reason: str) -> None:
"""Make a broker sync failure VISIBLE (else stale snapshots look current)."""
@@ -57,8 +70,8 @@ class AlpacaPaperSync:
timestamp=datetime.now(),
mode=self.mode,
correlation_id=self.session_id,
message=f"alpaca_paper sync degraded: {reason}",
broker_name=BROKER_NAME,
message=f"{self.broker_name} sync degraded: {reason}",
broker_name=self.broker_name,
previous_health="healthy",
new_health="degraded",
health_reason=reason,
@@ -78,10 +91,10 @@ class AlpacaPaperSync:
timestamp=datetime.now(),
mode=self.mode,
correlation_id=self.session_id,
message=f"Alpaca paper equity {account.get('equity')}",
broker_name=BROKER_NAME,
message=f"{self.account_desc} equity {account.get('equity')}",
broker_name=self.broker_name,
account_id=account_id,
account_desc="Alpaca Paper",
account_desc=self.account_desc,
total_equity=float(account.get("equity") or 0),
cash_balance=float(account.get("cash") or 0),
buying_power=float(account.get("buying_power") or 0),
@@ -116,8 +129,8 @@ class AlpacaPaperSync:
timestamp=datetime.now(),
mode=self.mode,
correlation_id=self.session_id,
message=f"Alpaca paper positions: {len(positions)}",
broker_name=BROKER_NAME,
message=f"{self.account_desc} positions: {len(positions)}",
broker_name=self.broker_name,
account_id="paper",
positions=positions,
)
@@ -166,8 +179,8 @@ class AlpacaPaperSync:
timestamp=datetime.now(),
mode=self.mode,
correlation_id=self.session_id,
message=f"Alpaca paper orders: {len(orders)}",
broker_name=BROKER_NAME,
message=f"{self.account_desc} orders: {len(orders)}",
broker_name=self.broker_name,
account_id="paper",
orders=orders,
)
@@ -223,6 +236,92 @@ class AlpacaPaperSync:
reason = _EXIT_REASON.get(str(f.get("type") or ""), "closed")
return px, reason, stop_level, f.get("broker_order_id")
def _load_journaled_exit_ids(self) -> set[str]:
"""position_ids of the closes already in the store (an exit's broker order
id, see reconcile_closed) so a restart does not re-journal them."""
try:
rows = self.store.con.execute(
"SELECT payload_json FROM events WHERE event_type = 'position_closed' "
"ORDER BY timestamp DESC LIMIT 2000").fetchall()
return {str(json.loads(r[0] or "{}").get("position_id") or "") for r in rows} - {""}
except Exception: # noqa: BLE001
return set()
@staticmethod
def _same_session(submitted_at, today) -> bool:
"""True if an order's submitted_at (ISO, UTC) falls on ``today`` (ET), or is
unparseable (Tradier's list is already scoped to the current session)."""
if not submitted_at:
return True
try:
ts = datetime.fromisoformat(str(submitted_at).replace("Z", "+00:00"))
return ts.astimezone(_ET).date() == today
except ValueError:
return True
def reconcile_unseen_round_trips(self, prev_positions, new_positions, orders,
today=None) -> int:
"""Journal round trips that opened AND closed between two position polls.
2026-10-02 TNON: bought at 09:38:37, the bracket stop filled at 09:39:26,
and no positions snapshot ever held it, so the snapshot diff in
reconcile_closed had nothing to diff and the trade vanished from the
journal (the 80-trade scorecard dropped a -1R loser). The broker's order
list still shows both fills, so for any symbol that is open in NEITHER
snapshot but has a filled buy and a filled sell submitted this session,
emit a position_closed per unjournaled exit fill: entry = VWAP of the buy
fills, exit = that sell's fill price, stop = the session's stop order.
Long round trips only (the bot is long-only). Returns the number emitted."""
if self._journaled_exits is None:
self._journaled_exits = self._load_journaled_exit_ids()
today = today or datetime.now(_ET).date()
open_syms = {p.get("symbol") for p in (prev_positions or []) + (new_positions or [])
if abs(float(p.get("quantity") or 0)) > 0}
by_sym: dict[str, list[dict]] = {}
for o in orders or []:
sym = o.get("symbol")
if (not sym or sym in open_syms or str(o.get("status") or "").lower() != "filled"
or not o.get("type") or o.get("filled_avg_price") in (None, "")
or float(o.get("filled_quantity") or 0) <= 0
or not self._same_session(o.get("submitted_at"), today)):
continue
by_sym.setdefault(sym, []).append(o)
emitted = 0
for sym, fills in by_sym.items():
buys = [o for o in fills if str(o.get("side") or "").lower() == "buy"]
sells = [o for o in fills if str(o.get("side") or "").lower() == "sell"
and str(o.get("broker_order_id") or "") not in self._journaled_exits]
if not buys or not sells:
continue
bought = sum(float(o["filled_quantity"]) for o in buys)
entry = sum(float(o["filled_quantity"]) * float(o["filled_avg_price"]) for o in buys) / bought
stop_level = None
for o in fills:
if str(o.get("type") or "") in ("stop", "stop_limit") and o.get("stop_price") not in (None, ""):
try:
stop_level = float(o["stop_price"])
except (TypeError, ValueError):
pass
for f in sorted(sells, key=lambda o: str(o.get("submitted_at") or "")):
qty = min(float(f["filled_quantity"]), bought)
exit_price = float(f["filled_avg_price"])
reason = _EXIT_REASON.get(str(f.get("type") or ""), "closed")
realized = (exit_price - entry) * qty
oid = str(f.get("broker_order_id") or f"{sym}-{datetime.now().isoformat()}")
self.store.emit(PositionClosedEvent(
timestamp=datetime.now(), mode=self.mode, correlation_id=self.session_id,
message=(f"{sym} closed @ {exit_price:.4f} ({reason}) pnl={realized:+.2f} "
f"[round trip between polls]"),
position_id=oid, symbol=sym, exit_price=round(exit_price, 4),
exit_reason=reason, realized_pnl=round(realized, 2),
entry_price=round(entry, 4),
stop_loss_price=round(stop_level, 4) if stop_level else None,
side="buy", quantity=qty,
))
self._journaled_exits.add(oid)
emitted += 1
return emitted
def reconcile_closed(self, prev_positions, new_positions, orders) -> int:
"""Emit a position_closed event for every symbol that was open in the
previous snapshot and is gone (or flat) in the new one. This is the ONLY
@@ -257,6 +356,10 @@ class AlpacaPaperSync:
stop_loss_price=round(stop_level, 4) if stop_level else None,
side=("sell" if is_short else "buy"), quantity=qty,
))
if oid:
if self._journaled_exits is None:
self._journaled_exits = self._load_journaled_exit_ids()
self._journaled_exits.add(str(oid))
emitted += 1
return emitted
@@ -274,5 +377,9 @@ class AlpacaPaperSync:
self.reconcile_closed(self._prev_positions, positions, orders or [])
except Exception as exc: # noqa: BLE001 — never let journaling break sync
logger.warning("position-close reconcile failed: %s", exc)
try:
self.reconcile_unseen_round_trips(self._prev_positions, positions, orders or [])
except Exception as exc: # noqa: BLE001
logger.warning("round-trip reconcile failed: %s", exc)
self._prev_positions = positions
return {"account": account, "positions": positions, "orders": orders}
+605 -15
View File
@@ -8,10 +8,17 @@ GET /api/stream Server-Sent Events: pushes snapshots on
GET /api/approval-queue pending approvals only
GET /api/criteria?symbol=XYZ per-criterion pass/fail breakdown
GET /api/bars?symbol=XYZ[&minutes=60] recent 1-min bars for a sparkline
GET /api/chart?symbol=XYZ[&hours=24][&date=YYYY-MM-DD] candles + fill markers + levels;
date= anchors the window to that session (else "now")
GET /api/decisions[?symbol=&limit=&date=] entry_gate_evaluated decision receipts (plan 014)
GET /backtest theoretical-replay dashboard HTML (plan 015)
GET /api/backtest[?days=10|since=&until=] theoretical trades under CURRENT settings (plan 015)
GET /static/vendor/... vendored JS libs (e.g. Lightweight Charts)
POST /api/trading/approvals/approve {"order_id": ..., "approved_by"?: ...}
POST /api/trading/approvals/reject {"order_id": ..., "reason"?: ...}
POST /api/trading/exit-order {"symbol": ...}
POST /api/watch/add {"symbol": ...} (manual watchlist inject)
POST /api/command {"command": ..., "symbol"?: ...} (operator kill switch; requires DASHBOARD_TOKEN)
The server holds an EventStore for reads; trade actions route through a
TradingExecutionService when one is attached (the orchestrator does this).
@@ -21,14 +28,18 @@ dashboard still works read-only.
from __future__ import annotations
import hmac
import json
import os
import threading
import time
from datetime import datetime, timezone
import uuid
from datetime import date, datetime, timedelta, timezone
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import parse_qs, urlparse
from storage.event_schema import EventMode, OperatorCommandEvent, OPERATOR_COMMANDS
from storage.event_store import EventStore
from storage.projections import (
query_account_orders_snapshot,
@@ -40,7 +51,9 @@ from storage.projections import (
query_catalyst_feed,
query_daily_performance,
query_fills_feed,
query_gate_receipts,
query_armed_triggers,
query_loop_session,
query_equity_curve,
query_order_lifecycle_snapshot,
query_ready_signals_snapshot,
@@ -51,9 +64,46 @@ from storage.projections import (
query_watch_states_snapshot,
)
import logging
from research.multi_schema import open_research_db
logger = logging.getLogger(__name__)
STATIC_DIR = Path(__file__).parent / "static"
def _command_bus_enabled() -> bool:
"""The operator command bus is live only when a token guards it AND it is
explicitly enabled — so a stray COMMAND_BUS_ENABLED never opens an
unauthenticated kill switch."""
return bool(os.environ.get("DASHBOARD_TOKEN")) and (
os.environ.get("COMMAND_BUS_ENABLED", "").strip().lower()
in {"1", "true", "yes", "on"}
)
def _broker_block() -> dict:
"""Broker truth for the LIVE badge, derived from the base URL the services
already have (not the execution_mode literal, which lied). Alpaca's paper host
is ``paper-api.alpaca.markets``; Tradier's sandbox host starts with
``sandbox.``; anything else is live money."""
from urllib.parse import urlparse as _urlparse
if os.environ.get("BROKER", "alpaca").strip().lower() == "tradier":
url = os.environ.get("TRADIER_BASE_URL") or (
"https://api.tradier.com/v1"
if os.environ.get("TRADIER_ENV", "sandbox").strip().lower() == "live"
else "https://sandbox.tradier.com/v1")
host = _urlparse(url).hostname or url
return {"name": "TRADIER", "base_url_host": host,
"is_paper": host.startswith("sandbox.")}
url = os.environ.get("ALPACA_BASE_URL") or "https://paper-api.alpaca.markets"
host = _urlparse(url).hostname or url
return {"name": "ALPACA", "base_url_host": host,
"is_paper": host.startswith("paper-api")}
def _eastern_session_date():
"""Today's date in US/Eastern — matches the ingestion pipeline's flags."""
from zoneinfo import ZoneInfo
@@ -61,6 +111,19 @@ def _eastern_session_date():
return datetime.now(ZoneInfo("America/New_York")).date()
def _to_epoch_et(ts) -> int:
"""Naive US/Eastern timestamp (datetime or ISO string) -> UTC epoch seconds.
Event timestamps (and ``minute_bars``) are stored naive-ET; both /api/chart
(plan 013) and /api/decisions (plan 014) hand the browser epoch seconds so
Lightweight Charts — and the JS merging the two marker sets — never does
timezone math."""
from zoneinfo import ZoneInfo
dt = ts if isinstance(ts, datetime) else datetime.fromisoformat(str(ts))
return int(dt.replace(tzinfo=ZoneInfo("America/New_York")).timestamp())
def _available_dates(store) -> list[str]:
"""Distinct session dates with account data — populates the date picker so
you can switch from LIVE to review a past day."""
@@ -105,6 +168,87 @@ def _catalyst_engine(store, catalyst: dict) -> dict:
"dilutive": dilutive, "vetoes_today": vetoes}
class _ChangeSignal:
"""One process-wide event-change signal shared by ALL SSE clients.
Before this, every SSE client ran ``COUNT(*)`` on a FRESH EventStore
connection every ~1s (re-applying the 473-line schema unless
``PG_SKIP_SCHEMA=1``). Now a single long-lived connection polls
``max(timestamp)`` (an index-tail seek) at most once per second GLOBALLY,
behind a lock — N clients cost one query/sec, not N. ``reader`` is injectable
so tests can count underlying queries without a DB.
"""
def __init__(self, db_path: str | None = None, *, reader=None, min_interval: float = 1.0):
self._db_path = db_path
self._reader = reader
self._min_interval = float(min_interval)
self._lock = threading.Lock()
self._store = None
self._stamp = None
self._checked_at = 0.0
self.query_count = 0 # test hook: number of underlying reads performed
def _read(self):
self.query_count += 1
if self._reader is not None:
return self._reader()
if self._store is None:
self._store = EventStore(self._db_path) # long-lived; schema skipped via PG_SKIP_SCHEMA
return self._store.max_timestamp()
def current(self):
"""Latest change token (max event timestamp), refreshed at most once per
``min_interval`` seconds across all callers."""
now = time.monotonic()
with self._lock:
if self._checked_at == 0.0 or (now - self._checked_at) >= self._min_interval:
try:
self._stamp = self._read()
except Exception: # noqa: BLE001 — a transient read must not kill the stream
pass
self._checked_at = now
return self._stamp
def _maybe_json(v):
"""JSONB columns come back from psycopg2 as str OR already-parsed dict; make it
a dict either way so the ledger never double-encodes."""
if isinstance(v, str):
try:
return json.loads(v)
except (ValueError, TypeError):
return {}
return v or {}
def _experiment_ledger(store) -> list[dict]:
"""Latest verdict per experiment for the ledger panel (plan 006 tables).
Returns [] on ANY error — the experiments/experiment_verdicts tables live in a
separate schema file applied by the registry, so on a bare/test database they
simply don't exist and the panel shows its empty state."""
try:
rows = store.con.execute(
"SELECT v.experiment_id, v.as_of, v.n, v.verdict, v.metrics, "
" e.hypothesis, e.bars, e.status "
"FROM (SELECT DISTINCT ON (experiment_id) experiment_id, as_of, n, verdict, metrics "
" FROM experiment_verdicts ORDER BY experiment_id, as_of DESC) v "
"LEFT JOIN experiments e ON e.id = v.experiment_id "
"ORDER BY v.as_of DESC"
).fetchall()
except Exception: # noqa: BLE001 — tables absent on a bare DB
return []
out = []
for r in rows:
out.append({
"experiment": r[0], "as_of": str(r[1]), "n": r[2], "verdict": r[3],
"metrics": _maybe_json(r[4]), "hypothesis": r[5],
"bars": _maybe_json(r[6]), "status": r[7],
})
return out
class DashboardState:
"""Shared state: event store path + optional live execution service."""
@@ -123,6 +267,41 @@ class DashboardState:
self.research_con = research_con
self.watch_inject = watch_inject
self._lock = threading.Lock()
self._research_lock = threading.Lock()
# lazily-built, process-wide SSE change signal (shared by all clients)
self._change_signal: _ChangeSignal | None = None
# backtest replay cache (plan 015) — a DEDICATED lock, not self._lock,
# so a slow replay (10-30s) never blocks the live snapshot/SSE path.
# key = (since, until, repr(settings)) -> (result, monotonic timestamp).
self._backtest_lock = threading.Lock()
self._backtest_cache: dict = {}
def _research(self):
"""The research DB connection, opened LAZILY and self-healing. It used
to be opened once at boot; if the DB was unreachable then (container
started before Postgres, or a blip), it stayed None forever and every
chart/sparkline/backtest returned 'no research db attached' — which
read as broken TradingView charts when the real cause was an empty data
pipe. Now a None connection is (re)opened on demand; the facade itself
auto-reconnects a dropped-but-open one."""
if self.research_con is not None:
return self.research_con
with self._research_lock:
if self.research_con is None:
try:
self.research_con = open_research_db("market")
except Exception as exc: # noqa: BLE001 — degrade to None, retry next call
logger.warning("research DB open failed (charts empty): %s", exc)
self.research_con = None
return self.research_con
def change_stamp(self):
"""SSE change token from the shared once-per-second signal (see _ChangeSignal)."""
if self._change_signal is None:
with self._lock:
if self._change_signal is None:
self._change_signal = _ChangeSignal(self.event_db_path)
return self._change_signal.current()
def open_store(self) -> EventStore:
return EventStore(self.event_db_path)
@@ -146,6 +325,11 @@ class DashboardState:
"generated_at": datetime.now(timezone.utc).isoformat(),
"execution_mode": self.execution_mode,
"has_execution": self.execution_service is not None,
# broker truth (LIVE badge) + operator command-bus availability
"broker": _broker_block(),
"command_bus": _command_bus_enabled(),
# experiment ledger (plan 006 tables) — [] until they exist
"experiments": _experiment_ledger(store) if is_live else [],
"view_date": None if is_live else for_date,
"available_dates": _available_dates(store),
"pnl": query_session_pnl(store, for_date=for_date),
@@ -155,6 +339,9 @@ class DashboardState:
"daily": query_daily_performance(store),
"risk": query_risk_state(store, for_date=for_date),
"triggers": query_armed_triggers(store) if is_live else {"triggers": [], "armed": 0},
# loop session identity + day-2 staleness alarm (memory-only
# daily flags survive midnight only to silently block entries)
"loop_session": query_loop_session(store) if is_live else {},
"equity_curve": query_equity_curve(store, for_date=for_date),
"catalyst": catalyst,
"catalyst_engine": _catalyst_engine(store, catalyst) if is_live else {},
@@ -208,22 +395,23 @@ class DashboardState:
pipeline). If today has no bars yet (pre-market, weekend, or replay of
an older session), falls back to the most recent session that does.
"""
if self.research_con is None:
con = self._research()
if con is None:
return {"symbol": symbol, "bars": [], "reason": "no research db attached"}
from research.query import query_minute_bars
with self._lock:
try:
session_date = _eastern_session_date()
df = query_minute_bars(self.research_con, symbol, session_date)
df = query_minute_bars(con, symbol, session_date)
if df is None or df.empty:
# fall back to the latest session with bars for this symbol
row = self.research_con.execute(
row = con.execute(
"SELECT MAX(session_date) FROM minute_bars WHERE symbol = ?",
[symbol],
).fetchone()
if row and row[0] is not None:
df = query_minute_bars(self.research_con, symbol, row[0])
df = query_minute_bars(con, symbol, row[0])
except Exception as exc: # noqa: BLE001
return {"symbol": symbol, "bars": [], "reason": str(exc)}
if df is None or df.empty:
@@ -242,6 +430,266 @@ class DashboardState:
]
return {"symbol": symbol, "bars": bars}
def chart(self, symbol: str, hours: int = 24, anchor_date: str | None = None) -> dict:
"""24h candlestick bars + trade markers + entry/stop/target levels for
the symbol drill-in chart (plan 013). Bars come from the research DB;
markers from the fills feed; levels from the same account snapshots
``snapshots()`` already loads. Timestamps in ``minute_bars``/``events``
are naive US/Eastern — every timestamp returned here is a UTC epoch
second (what Lightweight Charts wants).
``anchor_date`` (plan 015): anchors the window to that session's close
instead of "now" — the live modal never passes it (unchanged
behavior), but the backtest replay's drill-in needs bars from
arbitrary past sessions the plain ``hours``-back-from-now window can
never reach (it's capped at 72h).
"""
con = self._research()
if con is None:
return {"symbol": symbol, "bars": [], "reason": "no research db attached"}
from zoneinfo import ZoneInfo
et = ZoneInfo("America/New_York")
hours = max(1, min(int(hours or 24), 72))
if anchor_date:
d = date.fromisoformat(str(anchor_date))
now_et = datetime(d.year, d.month, d.day, 20, 0, tzinfo=et) # well after the close
else:
now_et = datetime.now(et)
cutoff_et = now_et - timedelta(hours=hours)
cutoff_naive = cutoff_et.replace(tzinfo=None) # naive ET, matches storage
cutoff_epoch = int(cutoff_et.timestamp())
to_epoch = _to_epoch_et
with self._lock:
try:
rows = con.execute(
"SELECT timestamp, open, high, low, close, volume "
"FROM minute_bars WHERE symbol = ? AND timestamp >= ? "
"ORDER BY timestamp",
[symbol, cutoff_naive],
).fetchall()
except Exception as exc: # noqa: BLE001
return {"symbol": symbol, "bars": [], "reason": str(exc)}
store = self.open_store()
try:
raw_fills = query_fills_feed(store, limit=200, symbol=symbol)
positions = query_account_positions_snapshot(store)
orders = query_account_orders_snapshot(store)
try: # exits for the marker loop (store is open only in here)
closed_events = store.query_events(event_type="position_closed", limit=200)
except Exception: # noqa: BLE001
closed_events = []
finally:
store.close()
bars = [
{"time": to_epoch(r[0]), "open": float(r[1]), "high": float(r[2]),
"low": float(r[3]), "close": float(r[4]), "volume": int(r[5] or 0)}
for r in rows
]
# OrderFilledEvent carries no `side` field (schema gap, not this plan's
# to fix — alpaca_paper/execution.py is out of scope) — backfill a
# fill's side from its sibling order_submitted row via order_id so the
# marker can still be colored buy/sell.
side_by_order = {r["order_id"]: r["side"] for r in raw_fills if r.get("side")}
markers = []
for r in raw_fills:
ts = r.get("timestamp")
if not ts:
continue
epoch = to_epoch(ts)
if epoch < cutoff_epoch:
continue
markers.append({
"time": epoch,
"side": r.get("side") or side_by_order.get(r.get("order_id")),
"kind": r.get("kind"),
"price": r.get("price"),
"quantity": r.get("quantity"),
})
# EXIT markers — the fills feed carries entries (order_submitted); exits
# live in position_closed (the loop fills brackets via position-diffing,
# so no order_filled ever fires). Without this the chart showed entries
# but never where a trade closed. kind='exit' so the UI can mark it
# distinctly (X / down-triangle) with the exit reason + P&L.
for e in closed_events:
p = json.loads(e.get("payload_json", "{}"))
if p.get("symbol") != symbol:
continue
ts = e.get("timestamp")
epoch = to_epoch(ts) if ts else None
if epoch is None or epoch < cutoff_epoch:
continue
markers.append({
"time": epoch, "kind": "exit", "side": "sell",
"price": p.get("exit_price"), "quantity": p.get("quantity"),
"exit_reason": p.get("exit_reason"),
"pnl": p.get("realized_pnl"),
})
markers.sort(key=lambda m: m["time"])
levels = {"avg_entry": None, "stop": None, "target": None, "entry_limit": None}
for snap in positions:
for pos in snap.get("positions", []):
if pos.get("symbol") != symbol:
continue
avg = pos.get("avg_entry_price")
if avg is None:
avg = pos.get("average_price")
if avg is not None:
levels["avg_entry"] = float(avg)
working = {"new", "accepted", "pending_new", "partially_filled"}
for snap in orders:
for o in snap.get("orders", []):
if o.get("symbol") != symbol or o.get("status") not in working:
continue
if (o.get("side") == "sell" and o.get("type") in ("stop", "stop_limit")
and o.get("stop_price") is not None and levels["stop"] is None):
levels["stop"] = float(o["stop_price"])
elif (o.get("side") == "sell" and o.get("type") == "limit"
and o.get("limit_price") is not None and levels["target"] is None):
levels["target"] = float(o["limit_price"])
elif (o.get("side") == "buy" and o.get("type") == "limit"
and o.get("limit_price") is not None and levels["entry_limit"] is None):
levels["entry_limit"] = float(o["limit_price"])
return {"symbol": symbol, "bars": bars, "markers": markers, "levels": levels}
def decisions(self, symbol: str | None = None, limit: int = 100,
for_date: str | None = None) -> list[dict]:
"""Live decision-trace feed (plan 014): entry_gate_evaluated receipts
with a server-computed ``epoch`` per row (naive-ET storage; the chart
overlay and the panel never do timezone math client-side)."""
with self._lock:
store = self.open_store()
try:
rows = query_gate_receipts(store, symbol=symbol, limit=limit,
for_date=for_date)
finally:
store.close()
for row in rows:
ts = row.get("timestamp")
row["epoch"] = _to_epoch_et(ts) if ts else None
return rows
def backtest(self, days: int | None = 10, since: str | None = None,
until: str | None = None, regime_override: bool | None = None) -> dict:
"""Theoretical trades under CURRENT live settings (plan 015) —
assembly only, the simulation lives in research.experiments.replay.
``days`` (default path): the most recent N trading sessions with
setup_events, counted backwards. ``since``/``until``: an explicit
date range instead. Either way capped at 45 sessions (``clamped``
reports when the request exceeded it). Cached 10 min behind a
DEDICATED lock (not self._lock) so a slow replay never blocks the
live snapshot/SSE path — two concurrent identical requests compute
the replay exactly once.
"""
from strategy.exits import ExitConfig
from strategy.population import PopulationConfig
# cap protects the (serialized) request thread; 300 spans the full stored
# minute-bar history (~273 sessions) so months-scale ranges work via
# since/until — a full-history request runs minutes, operator's choice.
MAX_SESSIONS = int(os.environ.get("BACKTEST_MAX_SESSIONS", "300"))
pop = PopulationConfig.from_env()
regime_on = os.environ.get("TRADING_REGIME_GATE_ENABLED", "0").strip().lower() in {
"1", "true", "yes", "on"}
if regime_override is not None:
# /backtest UX: let the page show RAW-criteria trades when the gate
# sits out the whole window (owner: 'no data in backtest').
regime_on = regime_override
fidelity = {"atr_frac": 0.25, "spread_mult": 3, "adverse_ticks": 1}
settings = {
"population": {"price_min": pop.price_min, "price_max": pop.price_max,
"min_entry_gap_pct": pop.min_entry_gap_pct,
"min_day_volume": pop.min_day_volume},
"regime_gate": regime_on,
"fidelity": fidelity,
"target_r": ExitConfig.from_env().target_r,
}
empty = {"trades": [], "daily": [], "skipped": 0,
"summary": {"n": 0, "sum_r": 0.0, "avg_r": 0.0, "hit_rate": 0.0, "sessions": 0},
"screened": {"population": 0, "regime": 0},
"live_split": {"taken": {"n": 0, "sum_r": 0.0, "avg_r": 0.0},
"not_taken": {"n": 0, "sum_r": 0.0, "avg_r": 0.0}}}
store = self.open_store()
try:
base_q = ("SELECT DISTINCT session_date FROM setup_events WHERE "
"entry_reference_price IS NOT NULL AND invalidation_price IS NOT NULL")
if since and until:
rows = store.con.execute(
base_q + " AND session_date >= ? AND session_date <= ? "
"ORDER BY session_date DESC", [since, until]).fetchall()
all_dates = sorted(r[0] for r in rows)
clamped = len(all_dates) > MAX_SESSIONS
dates = all_dates[-MAX_SESSIONS:] if clamped else all_dates
sessions_requested = len(all_dates)
else:
requested = int(days or 10)
clamped = requested > MAX_SESSIONS
n_sessions = min(max(requested, 1), MAX_SESSIONS)
rows = store.con.execute(
base_q + " ORDER BY session_date DESC LIMIT ?", [n_sessions]).fetchall()
dates = sorted(r[0] for r in rows)
sessions_requested = n_sessions
if not dates:
return {**empty, "settings": settings, "clamped": clamped,
"sessions_requested": sessions_requested}
win_since, win_until = str(dates[0]), str(dates[-1])
key = (win_since, win_until, repr(settings))
with self._backtest_lock:
cached = self._backtest_cache.get(key)
if cached and (time.monotonic() - cached[1]) < 600:
result = cached[0]
else:
from research.experiments.replay import replay_window
result = replay_window(store, win_since, win_until, population_cfg=pop,
regime_gate=regime_on, fidelity=fidelity)
for t in result["trades"]:
t["entry_epoch"] = _to_epoch_et(t["entry_ts"])
t["exit_epoch"] = _to_epoch_et(t["exit_ts"])
self._backtest_cache[key] = (result, time.monotonic())
# Live-taken vs skipped split (2026-07-25): which replay fires did the
# live loop actually trade? Closes for past sessions are immutable, so
# re-annotating the cached result on every request is idempotent.
rows = store.con.execute(
"SELECT timestamp, payload_json FROM events "
"WHERE event_type = 'position_closed' AND timestamp >= ? AND timestamp < ?",
[win_since,
(date.fromisoformat(win_until) + timedelta(days=1)).isoformat()],
).fetchall()
live_pnl: dict = {}
for ts, pj in rows:
p = pj if isinstance(pj, dict) else json.loads(pj or "{}")
sym = p.get("symbol")
if sym and p.get("realized_pnl") is not None:
k = (str(ts.date()), sym)
live_pnl[k] = round(live_pnl.get(k, 0.0) + float(p["realized_pnl"]), 2)
live_split = {"taken": {"n": 0, "sum_r": 0.0},
"not_taken": {"n": 0, "sum_r": 0.0}}
for t in result["trades"]:
pnl = live_pnl.get((str(t["session_date"]), t["symbol"]))
t["live_pnl"] = pnl
grp = live_split["taken" if pnl is not None else "not_taken"]
grp["n"] += 1
grp["sum_r"] += t["r_multiple"]
for grp in live_split.values():
grp["sum_r"] = round(grp["sum_r"], 3)
grp["avg_r"] = round(grp["sum_r"] / grp["n"], 3) if grp["n"] else 0.0
finally:
store.close()
return {**result, "settings": settings, "clamped": clamped,
"live_split": live_split, "sessions_requested": sessions_requested}
def _with_service(self, fn):
if self.execution_service is None:
return None
@@ -261,6 +709,28 @@ class DashboardState:
def exit_position(self, symbol: str) -> dict | None:
return self._with_service(lambda svc: svc.submit_exit_order(symbol))
def emit_command(self, command: str, symbol: str | None = None,
requested_by: str = "dashboard") -> dict:
"""Emit a guarded operator command event (the loop consumes + executes it).
Works even when execution_service is None — the standalone dashboard is a
thin emitter; the loop process is the single execution authority. Callers
gate on DASHBOARD_TOKEN before reaching here (commands never run
unauthenticated)."""
nonce = uuid.uuid4().hex
with self._lock:
store = self.open_store()
try:
store.emit(OperatorCommandEvent(
timestamp=datetime.now(), mode=EventMode.PAPER,
message=f"operator command {command}"
+ (f" {symbol}" if symbol else ""),
command=command, symbol=(symbol or None),
requested_by=requested_by, nonce=nonce))
finally:
store.close()
return {"ok": True, "command": command, "symbol": symbol, "nonce": nonce}
def add_watch(self, symbol: str) -> dict | None:
"""Manually inject a symbol onto the watchlist (if supported)."""
if not self.watch_inject or not symbol:
@@ -271,6 +741,33 @@ class DashboardState:
except Exception as exc: # noqa: BLE001
return {"ok": False, "error": str(exc)}
def add_pick(self, body: dict) -> dict:
"""Morning pick journal (research/pick_journal.py) — shadow forecast
experiment; never read by the live loop."""
from research.pick_journal import add_pick as _add
store = self.open_store()
try:
row = _add(store.con, str(body.get("symbol") or ""),
forecaster=str(body.get("forecaster") or "phil"),
direction=str(body.get("direction") or "long"),
conviction=int(body.get("conviction") or 2),
note=body.get("note"))
return {"ok": True, "valid": row["valid"], "symbol": row["symbol"],
"picked_at": row["picked_at"].isoformat()}
except ValueError as exc:
return {"ok": False, "error": str(exc)}
finally:
store.con.close()
def picks(self, for_date: str | None = None) -> list[dict]:
from research.pick_journal import list_picks
store = self.open_store()
try:
d = date.fromisoformat(for_date) if for_date else None
return list_picks(store.con, d)
finally:
store.con.close()
def _json_default(obj):
if isinstance(obj, datetime):
@@ -293,10 +790,27 @@ class DashboardHandler(BaseHTTPRequestHandler):
def _send_json(self, payload, status: int = 200) -> None:
body = _dump(payload)
self.send_response(status)
self.send_header("Content-Type", "application/json")
try:
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
except (BrokenPipeError, ConnectionResetError):
# the client hung up while we were rendering (slow query + browser
# timeout) — nothing to send it; don't double-traceback the log
pass
def _send_static(self, path: Path, content_type: str) -> None:
"""Serve a static asset (e.g. the vendored chart lib) — cacheable, unlike
the dashboard shell."""
if not path.exists():
self._send_json({"error": "not found"}, 404)
return
body = path.read_bytes()
self.send_response(200)
self.send_header("Content-Type", content_type)
self.send_header("Content-Length", str(len(body)))
self.send_header("Access-Control-Allow-Origin", "*")
self.end_headers()
self.wfile.write(body)
@@ -341,18 +855,18 @@ class DashboardHandler(BaseHTTPRequestHandler):
self.send_header("Content-Type", "text/event-stream")
self.send_header("Cache-Control", "no-cache")
self.send_header("Connection", "keep-alive")
self.send_header("Access-Control-Allow-Origin", "*")
self.end_headers()
shutdown = getattr(type(self), "shutdown_event", None)
last_count = -1
_sentinel = object()
last_stamp = _sentinel
last_heartbeat = 0.0
try:
while shutdown is None or not shutdown.is_set():
count = self.state.event_count()
stamp = self.state.change_stamp()
now = time.monotonic()
if count != last_count:
last_count = count
if stamp != last_stamp: # last_stamp starts as a unique sentinel
last_stamp = stamp
payload = _dump(self.state.snapshots()).decode()
self.wfile.write(f"event: snapshot\ndata: {payload}\n\n".encode())
self.wfile.flush()
@@ -376,6 +890,17 @@ class DashboardHandler(BaseHTTPRequestHandler):
route = urlparse(self.path).path
if route in ("/", "/index.html", "/dashboard"):
self._send_html(STATIC_DIR / "dashboard.html")
elif route == "/backtest":
self._send_html(STATIC_DIR / "backtest.html")
elif route.startswith("/static/vendor/"):
# path-traversal guard: resolve then require it stays inside
# STATIC_DIR/vendor (e.g. "/static/vendor/../dashboard.html" 404s).
vendor_dir = (STATIC_DIR / "vendor").resolve()
requested = (vendor_dir / route[len("/static/vendor/"):]).resolve()
if requested.is_relative_to(vendor_dir) and requested.is_file():
self._send_static(requested, "application/javascript")
else:
self._send_json({"error": "not found"}, 404)
elif route == "/api/stream":
self._stream()
elif route == "/api/snapshots":
@@ -391,14 +916,54 @@ class DashboardHandler(BaseHTTPRequestHandler):
symbol = (q.get("symbol") or [""])[0]
minutes = int((q.get("minutes") or ["60"])[0])
self._send_json(self.state.bars(symbol, minutes))
elif route == "/api/chart":
q = self._query()
symbol = (q.get("symbol") or [""])[0]
hours = int((q.get("hours") or ["24"])[0])
anchor = (q.get("date") or [""])[0] or None
self._send_json(self.state.chart(symbol, hours, anchor_date=anchor))
elif route == "/api/decisions":
q = self._query()
symbol = (q.get("symbol") or [""])[0] or None
limit = min(int((q.get("limit") or ["100"])[0]), 500)
d = (q.get("date") or [""])[0] or None
self._send_json({"decisions":
self.state.decisions(symbol=symbol, limit=limit, for_date=d)})
elif route == "/api/backtest":
q = self._query()
since = (q.get("since") or [""])[0] or None
until = (q.get("until") or [""])[0] or None
days = (q.get("days") or [""])[0]
rg = (q.get("regime") or [""])[0]
self._send_json(self.state.backtest(
days=int(days) if days else 10, since=since, until=until,
regime_override=(rg == "1") if rg in ("0", "1") else None))
elif route == "/api/attribution":
# per-component stage ratings (research/attribution.py); computed on
# demand — a handful of indexed event queries, fine per click
from research.attribution import compute_attribution
days = int((self._query().get("days") or ["7"])[0])
self._send_json(compute_attribution(days=days))
elif route == "/api/picks":
d = (self._query().get("date") or [""])[0] or None
self._send_json({"picks": self.state.picks(d)})
elif route == "/api/health":
self._send_json({"ok": True, "mode": self.state.execution_mode})
self._send_json({"ok": True, "mode": self.state.execution_mode,
"version": os.environ.get("APP_BUILD_TAG", "dev"),
"git_sha": os.environ.get("APP_GIT_SHA", "dev")[:12]})
else:
self._send_json({"error": "not found"}, 404)
except Exception as exc: # noqa: BLE001
self._send_json({"error": str(exc)}, 500)
def do_POST(self): # noqa: N802
# optional shared-secret gate on trade actions (approve/reject/exit/watch).
# Off by default (autonomous stack auto-approves); set DASHBOARD_TOKEN to
# require X-Dashboard-Token on every POST. CORS wildcard was removed so a
# cross-origin LAN page can't drive these either way.
_tok = os.environ.get("DASHBOARD_TOKEN")
if _tok and not hmac.compare_digest(self.headers.get("X-Dashboard-Token", ""), _tok):
return self._send_json({"error": "bad or missing X-Dashboard-Token"}, 401)
try:
route = urlparse(self.path).path
body = self._read_body()
@@ -418,6 +983,32 @@ class DashboardHandler(BaseHTTPRequestHandler):
result = self.state.exit_position(str(body.get("symbol") or ""))
elif route == "/api/watch/add":
result = self.state.add_watch(str(body.get("symbol") or ""))
elif route == "/api/picks":
self._send_json(self.state.add_pick(body))
return
elif route == "/api/command":
# The command bus is token-gated by construction: without
# DASHBOARD_TOKEN a command could run unauthenticated, so refuse
# outright (the top-of-do_POST gate only bites WHEN a token is set).
if not os.environ.get("DASHBOARD_TOKEN"):
self._send_json(
{"ok": False, "error": "command bus disabled: DASHBOARD_TOKEN not set"},
503)
return
command = str(body.get("command") or "")
if command not in OPERATOR_COMMANDS:
self._send_json(
{"ok": False, "error": f"unknown command {command!r}"}, 400)
return
symbol = body.get("symbol")
symbol = str(symbol).upper() if symbol else None
if command == "flatten_symbol" and not symbol:
self._send_json(
{"ok": False, "error": "flatten_symbol requires a symbol"}, 400)
return
self._send_json(
self.state.emit_command(command, symbol=symbol), 200)
return
else:
self._send_json({"error": "not found"}, 404)
return
@@ -432,7 +1023,6 @@ class DashboardHandler(BaseHTTPRequestHandler):
def do_OPTIONS(self): # noqa: N802
self.send_response(204)
self.send_header("Access-Control-Allow-Origin", "*")
self.send_header("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
self.send_header("Access-Control-Allow-Headers", "Content-Type")
self.end_headers()
+155
View File
@@ -0,0 +1,155 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Momentum Admin</title>
<style>
:root{--bg:#f6f7fb;--card:#fff;--ink:#1a2233;--mut:#68738a;--line:#e5e8f0;
--ok:#16a34a;--warn:#d97706;--bad:#dc2626;--acc:#2563eb;--accbg:#eff4ff;--r:14px}
@media (prefers-color-scheme:dark){:root{--bg:#0f1420;--card:#171e2e;--ink:#e8ecf6;
--mut:#8b96ad;--line:#232c42;--accbg:#1a2540}}
*{box-sizing:border-box;margin:0}
body{font:15px/1.5 -apple-system,'Segoe UI',Roboto,sans-serif;background:var(--bg);color:var(--ink);padding:24px}
.wrap{max-width:1100px;margin:0 auto}
header{display:flex;align-items:center;gap:14px;margin-bottom:22px;flex-wrap:wrap}
h1{font-size:20px;font-weight:700}
.badge{padding:3px 12px;border-radius:99px;font-size:12px;font-weight:700;letter-spacing:.4px}
.paper{background:#e0edff;color:#1d4ed8}.live{background:#fee2e2;color:#b91c1c}
.spacer{flex:1}
input[type=password],input[type=text]{background:var(--card);border:1px solid var(--line);
border-radius:8px;padding:7px 10px;color:var(--ink);font-size:14px}
h2{font-size:13px;text-transform:uppercase;letter-spacing:.8px;color:var(--mut);margin:26px 0 10px}
.grid{display:grid;gap:12px}.g3{grid-template-columns:repeat(auto-fill,minmax(240px,1fr))}
.g4{grid-template-columns:repeat(auto-fill,minmax(200px,1fr))}
.card{background:var(--card);border:1px solid var(--line);border-radius:var(--r);padding:14px 16px}
.row{display:flex;align-items:center;gap:10px}
.dot{width:9px;height:9px;border-radius:99px;flex:none}
.ok{background:var(--ok)}.warn{background:var(--warn)}.bad{background:var(--bad)}
.mut{color:var(--mut);font-size:13px}
button{background:var(--acc);color:#fff;border:0;border-radius:8px;padding:7px 14px;
font-size:13px;font-weight:600;cursor:pointer}
button.sec{background:var(--accbg);color:var(--acc)}
button.danger{background:var(--bad)}
button:disabled{opacity:.45;cursor:default}
table{width:100%;border-collapse:collapse;font-size:14px}
td,th{padding:8px 10px;border-bottom:1px solid var(--line);text-align:left}
th{color:var(--mut);font-size:12px;text-transform:uppercase;letter-spacing:.5px}
td input{width:130px}
.tag{font-size:11px;background:var(--accbg);color:var(--acc);border-radius:6px;padding:1px 7px;font-weight:700}
#toast{position:fixed;bottom:22px;left:50%;transform:translateX(-50%);background:var(--ink);
color:var(--bg);padding:10px 20px;border-radius:10px;font-size:14px;opacity:0;transition:.3s}
</style>
</head>
<body><div class="wrap">
<header>
<h1>Momentum Admin</h1>
<span id="mode" class="badge paper">PAPER</span>
<span id="mkt" class="mut"></span>
<div class="spacer"></div>
<input type="password" id="token" placeholder="admin token" size="14">
<button class="sec" onclick="saveToken()">Connect</button>
</header>
<h2>Health</h2>
<div class="grid g4" id="health"></div>
<h2>Services</h2>
<div class="grid g3" id="services"></div>
<h2>Strategy profiles</h2>
<div class="grid g3" id="profiles"></div>
<h2>Configuration <span class="mut" style="text-transform:none;letter-spacing:0">— edits go to overrides.env; restart the loop to apply. <span class="tag">override</span> = differs from deployed env</span></h2>
<div class="card"><table id="cfg"><thead><tr><th>key</th><th>value</th><th></th></tr></thead><tbody></tbody></table>
<div class="row" style="margin-top:12px"><button onclick="saveCfg()">Save changes</button>
<span class="mut" id="cfgnote"></span></div></div>
<h2>Effective config <span class="mut" style="text-transform:none;letter-spacing:0">— resolved value + source per key; <span class="tag">overrides.env</span> beats compose. Secrets masked. A red row = a documented .env.example key is missing (drift).</span></h2>
<div class="card"><table id="effcfg"><thead><tr><th>key</th><th>effective value</th><th>source</th></tr></thead><tbody></tbody></table></div>
<h2>Trading mode</h2>
<div class="card row">
<div style="flex:1"><b>Paper ⇄ Live switch</b><div class="mut">Live requires ALPACA_LIVE_API_KEY / ALPACA_LIVE_SECRET_KEY in the deployment env. Switching restarts nothing by itself — restart the loop after.</div></div>
<button class="sec" onclick="setMode('paper')">Paper</button>
<button class="danger" onclick="if(confirm('Switch the loop to LIVE trading with real money on next restart. Are you sure?'))setMode('live')">Go Live</button>
</div>
<h2>Research jobs</h2>
<div class="card row">
<div style="flex:1"><b>Regime backtest <span class="mut" style="text-transform:none;letter-spacing:0">— plan 011, SHADOW</span></b>
<div class="mut">Rebuild session labels → walk-forward score both models → replay + adjudicate the regime_gate experiments. Runs a few min; does NOT touch live trading.</div></div>
<button class="sec" onclick="runRegime()">Run</button>
<button class="sec" onclick="regimeStatus()">Status</button>
</div>
<div class="card" id="regimeout" style="display:none"><pre id="regimelog" class="mut" style="white-space:pre-wrap;max-height:240px;overflow:auto;margin:0;font-size:12px"></pre></div>
<h2>Secrets present</h2>
<div class="card" id="secrets"></div>
<div id="toast"></div>
</div>
<script>
const $=q=>document.querySelector(q);let staged={};
const esch=s=>String(s==null?'':s).replace(/[&<>"']/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
const tok=()=>localStorage.getItem('admintok')||'';
function saveToken(){localStorage.setItem('admintok',$('#token').value);load()}
async function api(p,opt={}){opt.headers={...(opt.headers||{}),'X-Admin-Token':tok(),'Content-Type':'application/json'};
const r=await fetch(p,opt);if(r.status===401){toast('bad token');throw 0}return r.json()}
function toast(m){const t=$('#toast');t.textContent=m;t.style.opacity=1;setTimeout(()=>t.style.opacity=0,2600)}
async function runRegime(){toast('starting regime pipeline…');
try{const r=await api('/api/regime/run',{method:'POST',body:'{}'});
toast(r.ok?('started (pid '+r.pid+') — poll Status'):('failed: '+(r.error||'')));
setTimeout(regimeStatus,2000)}catch(e){toast('failed')}}
async function regimeStatus(){const r=await api('/api/regime/status');
$('#regimeout').style.display='block';
$('#regimelog').textContent=(r.log_tail&&r.log_tail.length?r.log_tail.join('\n'):'(no run yet — click Run)')
+(r.running?'\n\n… running …':(r.ran?'\n\n[finished]':''));
if(r.running)setTimeout(regimeStatus,4000)}
function dot(ok){return `<span class="dot ${ok===true?'ok':ok===false?'bad':'warn'}"></span>`}
async function load(){
const s=await api('/api/status');
const c=s.checks||{};
$('#health').innerHTML=[
`<div class="card row">${dot(c.db?.ok)}<div><b>Database</b><div class="mut">${c.db?.ok?`event ${c.db.newest_event_age_s}s ago`:c.db?.error||''}</div></div></div>`,
`<div class="card row">${dot(c.alpaca?.ok)}<div><b>${c.alpaca?.name||'Broker'}</b><div class="mut">${c.alpaca?.ok?`$${(+c.alpaca.equity).toLocaleString()} · market ${c.alpaca.market_open?'open':'closed'}`:c.alpaca?.error||''}</div></div></div>`,
`<div class="card row">${dot(c.ollama?.ok)}<div><b>Ollama</b><div class="mut">${c.ollama?.ok?'enrichment up':'down (fails safe)'}</div></div></div>`,
`<div class="card row">${dot(c.feed==='sip')}<div><b>Data feed</b><div class="mut">${c.feed}</div></div></div>`].join('');
$('#mode').className='badge '+(c.alpaca?.mode==='LIVE'?'live':'paper');
$('#mode').textContent=c.alpaca?.mode==='LIVE'?'LIVE':'PAPER';
$('#mkt').textContent=c.alpaca?.market_open?'market open':'market closed';
$('#services').innerHTML=(s.containers||[]).map(x=>`<div class="card row">
${dot(x.health?x.health==='healthy':x.state==='running')}
<div style="flex:1"><b>${x.name.replace(/^ix-|-1$/g,'')}</b><div class="mut">${x.status||x.state}</div></div>
<button class="sec" onclick="restart('${x.name}')">restart</button></div>`).join('');
const cfg=await api('/api/config');staged={};
$('#cfg tbody').innerHTML=Object.entries(cfg.effective).map(([k,v])=>`<tr>
<td>${k} ${cfg.overrides[k]!==undefined?'<span class="tag">override</span>':''}</td>
<td><input type="text" value="${v}" oninput="staged['${k}']=this.value"></td>
<td>${cfg.overrides[k]!==undefined?`<button class="sec" onclick="delKey('${k}')">reset</button>`:''}</td></tr>`).join('');
const eff=await api('/api/config/effective');
$('#effcfg tbody').innerHTML=Object.entries(eff.config||{}).map(([k,v])=>`<tr${v.drift?' style="background:rgba(220,38,38,.12)"':''}>
<td>${esch(k)}</td>
<td>${v.value_masked==null?'<span class="mut">—</span>':esch(v.value_masked)}</td>
<td><span class="tag">${esch(v.source)}</span></td></tr>`).join('');
$('#secrets').innerHTML=Object.entries(cfg.secrets_present).map(([k,v])=>
`<span style="margin-right:18px">${v?'✅':'❌'} ${k}</span>`).join('');
const prof=await api('/api/profiles');
$('#profiles').innerHTML=Object.entries(prof).map(([n,p])=>`<div class="card">
<b>${n}</b><div class="mut" style="min-height:40px">${p._note||''}</div>
<button onclick="applyProfile('${n}')">Apply</button></div>`).join('');
}
async function saveCfg(){if(!Object.keys(staged).length)return toast('nothing changed');
await api('/api/config',{method:'POST',body:JSON.stringify({set:staged})});
toast('saved — restart the loop to apply');load()}
async function delKey(k){await api('/api/config',{method:'POST',body:JSON.stringify({delete:[k]})});load()}
async function applyProfile(n){await api('/api/profiles/apply',{method:'POST',body:JSON.stringify({name:n})});
toast(`profile ${n} staged — restart the loop to apply`);load()}
async function restart(n){toast('restarting '+n+'…');
const r=await api('/api/restart',{method:'POST',body:JSON.stringify({service:n})});
toast(r.ok?'restarted '+n:'restart failed');setTimeout(load,3000)}
async function setMode(m){const r=await api('/api/mode',{method:'POST',body:JSON.stringify({mode:m})});
toast(r.error||('mode -> '+m+' — restart the loop to apply'));load()}
$('#token').value=tok();load().catch(()=>{});setInterval(()=>load().catch(()=>{}),15000);
</script>
</body>
</html>
+434
View File
@@ -0,0 +1,434 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Momentum — Backtest Replay</title>
<style>
:root {
--bg: #080a10;
--panel: #111622;
--panel-2: #161c2a;
--panel-hi: #1c2435;
--border: #232c41;
--border-soft: #18202f;
--text: #e3e8f2;
--muted: #7884a2;
--accent: #5ba3ff;
--accent-dim: #2a4a78;
--green: #34d17e;
--red: #ff5a60;
--amber: #f7b13b;
--purple: #c2a6e2;
--glow-green: rgba(52,209,126,.5);
--glow-red: rgba(255,90,96,.45);
--shadow: 0 8px 24px -12px rgba(0,0,0,.7), 0 1px 0 0 rgba(255,255,255,.035) inset;
--mono: "SF Mono", "Cascadia Code", "JetBrains Mono", Menlo, Consolas, monospace;
}
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
background:
radial-gradient(1100px 560px at 82% -12%, rgba(30,58,110,.28), transparent 60%),
radial-gradient(820px 480px at -5% -5%, rgba(60,40,120,.16), transparent 55%),
var(--bg);
background-attachment: fixed;
color: var(--text);
font: 14px/1.5 -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
min-height: 100vh;
}
a { color: var(--accent); }
header { display: flex; align-items: center; gap: 14px; padding: 14px 20px;
border-bottom: 1px solid var(--border); flex-wrap: wrap; }
header h1 { font-size: 16px; letter-spacing: .3px; }
header h1 a { color: var(--muted); text-decoration: none; font-size: 12px; font-weight: 600;
border: 1px solid var(--border); border-radius: 4px; padding: 3px 8px; margin-left: 10px; }
header h1 a:hover { color: var(--text); border-color: var(--accent); }
.picker { display: flex; align-items: center; gap: 6px; margin-left: auto; flex-wrap: wrap; }
.picker button { background: var(--panel-2); border: 1px solid var(--border); color: var(--text);
border-radius: 5px; padding: 6px 12px; font: 600 12px var(--mono); cursor: pointer; }
.picker button:hover { border-color: var(--accent); }
.picker button.active { background: var(--accent-dim); border-color: var(--accent); color: #fff; }
.picker input[type=date] { background: var(--panel-2); border: 1px solid var(--border); color: var(--text);
border-radius: 5px; padding: 5px 8px; font: 600 12px var(--mono); }
.picker .run { background: var(--accent-dim); border-color: var(--accent); }
.banner { margin: 14px 20px 0; padding: 10px 14px; border-radius: 8px;
background: rgba(247,177,59,.10); border: 1px solid rgba(247,177,59,.35);
color: var(--amber); font: 700 12px/1.4 var(--mono); letter-spacing: .2px; }
.chips { display: flex; gap: 8px; flex-wrap: wrap; margin: 12px 20px 0; }
.chip { background: var(--panel-2); border: 1px solid var(--border); border-radius: 20px;
padding: 4px 12px; font: 600 11px var(--mono); color: var(--muted); }
.chip.on { color: var(--green); border-color: rgba(52,209,126,.4); }
.chip.off { color: var(--muted); }
.loading { margin: 40px 20px; text-align: center; color: var(--muted); font: 600 13px var(--mono); }
.loading .spin { width: 30px; height: 30px; margin: 0 auto 14px; border: 3px solid var(--border);
border-top-color: var(--accent); border-radius: 50%; animation: spin .8s linear infinite; }
@keyframes spin { to { transform: rotate(360deg); } }
.error { margin: 20px; padding: 14px; border-radius: 8px; background: rgba(255,90,96,.08);
border: 1px solid rgba(255,90,96,.35); color: var(--red); font: 600 12px var(--mono); }
.hidden { display: none !important; }
main { padding: 16px 20px 40px; }
.summary { display: grid; grid-template-columns: repeat(8, 1fr); gap: 1px; background: var(--border);
border: 1px solid var(--border); border-radius: 10px; overflow: hidden; margin-bottom: 16px; }
.summary .cell { background: var(--panel); padding: 12px 14px; }
.summary .label { font: 600 10px var(--mono); color: var(--muted); text-transform: uppercase; letter-spacing: .5px; }
.summary .value { font: 700 20px var(--mono); margin-top: 4px; }
.summary .value.pos { color: var(--green); }
.summary .value.neg { color: var(--red); }
.panel { background: var(--panel); border: 1px solid var(--border); border-radius: 10px;
box-shadow: var(--shadow); margin-bottom: 16px; }
.panel h2 { font-size: 13px; padding: 12px 16px; border-bottom: 1px solid var(--border-soft);
display: flex; align-items: center; gap: 8px; }
.panel h2 .sub { font-weight: 500; color: var(--muted); font-size: 11px; }
.panel-body { padding: 12px 16px; overflow-x: auto; }
.empty { color: var(--muted); font: 600 12px var(--mono); padding: 20px 0; display: block; text-align: center; }
.daily-svg { display: block; }
.daily-col { cursor: pointer; }
.daily-hit { fill: transparent; }
.daily-col:hover .daily-hit { fill: rgba(91,163,255,.06); }
.daily-lbl { font: 600 9px var(--mono); fill: var(--muted); text-anchor: middle; }
.daily-lbl.daily-sel { fill: var(--accent); }
.daily-col.daily-sel .daily-hit { fill: rgba(91,163,255,.10); }
table { width: 100%; border-collapse: collapse; font-size: 12px; }
th { text-align: left; font: 600 10px var(--mono); color: var(--muted); text-transform: uppercase;
letter-spacing: .4px; padding: 6px 10px; border-bottom: 1px solid var(--border-soft); position: sticky; top: 0; background: var(--panel); }
td { padding: 7px 10px; border-bottom: 1px solid var(--border-soft); font: 500 12px var(--mono); white-space: nowrap; }
tbody tr { cursor: pointer; }
tbody tr:hover { background: var(--panel-hi); }
td.sym { font-weight: 700; color: var(--accent); }
td.r.pos { color: var(--green); font-weight: 700; }
td.r.neg { color: var(--red); font-weight: 700; }
footer { padding: 10px 20px 30px; color: var(--muted); font: 500 10px/1.6 var(--mono); }
/* chart modal — imitates dashboard.html's plan-013 drill-in */
.chart-modal { position: fixed; inset: 0; background: rgba(6,8,12,.82); display: none; place-items: center; z-index: 70; }
.chart-modal.show { display: grid; }
.chart-card { background: var(--panel); border: 1px solid var(--border); border-radius: 10px;
width: min(920px, 94vw); height: min(560px, 88vh); display: flex; flex-direction: column; overflow: hidden; }
.chart-hdr { display: flex; align-items: center; gap: 10px; padding: 10px 16px;
border-bottom: 1px solid var(--border); flex-shrink: 0; }
.chart-hdr .sym { font: 700 15px var(--mono); letter-spacing: .3px; }
.chart-hdr .meta { color: var(--muted); font: 600 11px var(--mono); }
.chart-hdr .close { margin-left: auto; background: transparent; border: 1px solid var(--border);
color: var(--muted); border-radius: 4px; padding: 4px 10px; cursor: pointer; font: 600 11px var(--mono); }
.chart-hdr .close:hover { color: var(--text); border-color: var(--accent); }
#chart-container { flex: 1; min-height: 0; }
.chart-ftr { padding: 6px 16px; border-top: 1px solid var(--border-soft); font: 500 10px var(--mono);
color: var(--muted); flex-shrink: 0; }
@media (max-width: 900px) { .summary { grid-template-columns: repeat(3, 1fr); } }
@media (max-width: 640px) { .summary { grid-template-columns: repeat(2, 1fr); } header { padding: 10px 14px; } }
</style>
</head>
<body>
<header>
<h1>Backtest Replay <a href="/">← live dashboard</a></h1>
<div class="picker">
<button data-days="1">1d</button>
<button data-days="5">5d</button>
<button data-days="10">10d</button>
<button data-days="20">20d</button>
<button data-days="45">45d</button>
<input type="date" id="since-input">
<input type="date" id="until-input">
<button class="run" id="run-custom">Run range</button>
<label style="font: 600 11px var(--mono); color: var(--muted); display:flex; align-items:center; gap:5px; margin-left:8px">
<input type="checkbox" id="regime-toggle" checked> apply regime gate (as deployed)
</label>
</div>
</header>
<div class="banner">THEORETICAL — simulated fills (adverse-tick fidelity), setup-sourced entries; not live P&amp;L.</div>
<div class="chips" id="chips"></div>
<div class="loading hidden" id="loading"><div class="spin"></div><span id="loading-msg">replaying…</span></div>
<div class="error hidden" id="error"></div>
<main class="hidden" id="results">
<div class="summary" id="summary"></div>
<div class="panel">
<h2>Equity curve (R) <span class="sub">click a day to filter the trade table</span></h2>
<div class="panel-body"><div id="daily-curve"></div></div>
</div>
<div class="panel">
<h2>Trades <span class="sub" id="trades-sub"></span></h2>
<div class="panel-body">
<table>
<thead><tr><th>Date</th><th>Symbol</th><th>Entry</th><th>Exit</th><th>Reason</th><th>R</th><th>Live P&amp;L</th></tr></thead>
<tbody id="trades-body"></tbody>
</table>
</div>
</div>
</main>
<footer id="footer" class="hidden">
<div>entries are setup_events-sourced ORB triggers (the labeler's population); entry fill = trigger + adverse ticks — optimistic vs real slippage on thin names.</div>
<div>per-trade sim &ne; per-trade reality (minute-bar granularity + adverse fills can flip individual rows, e.g. a live +$200 winner simming as a &minus;1R stop) — trust the aggregates and the live-taken/not-taken split, not single rows.</div>
<div id="coverage-line"></div>
</footer>
<div id="chart-modal" class="chart-modal">
<div class="chart-card">
<div class="chart-hdr">
<span class="sym" id="chart-sym"></span>
<span class="meta" id="chart-meta"></span>
<button class="close" id="chart-close">CLOSE (Esc)</button>
</div>
<div id="chart-container"></div>
<div class="chart-ftr">blue = theoretical entry · purple = theoretical exit (reason labeled) · Charts by TradingView</div>
</div>
</div>
<script src="/static/vendor/lightweight-charts.standalone.production.js" defer></script>
<script>
const $ = (id) => document.getElementById(id);
const esc = (s) => String(s ?? "").replace(/[&<>"']/g, c => ({"&":"&amp;","<":"&lt;",">":"&gt;",'"':"&quot;","'":"&#39;"}[c]));
const fmtR = (v) => (v == null || isNaN(+v)) ? "—" : (v >= 0 ? "+" : "") + (+v).toFixed(2) + "R";
const fmtPx = (v) => (v == null || isNaN(+v)) ? "—" : (+v).toFixed(2);
const fmtTs = (iso) => { try { return iso.replace("T", " ").slice(5, 16); } catch { return iso || "—"; } };
let state = { data: null, viewDate: null };
// -------------------------------------------------------------------------
// fetch + orchestration
// -------------------------------------------------------------------------
function setLoading(msg) {
$("loading-msg").textContent = msg;
$("loading").classList.remove("hidden");
$("error").classList.add("hidden");
$("results").classList.add("hidden");
$("footer").classList.add("hidden");
}
async function runBacktest(params, activeBtn) {
document.querySelectorAll(".picker button[data-days]").forEach(b => b.classList.toggle("active", b === activeBtn));
state.lastParams = params; state.lastBtn = activeBtn;
// the gate toggle rides every request: unchecked = raw-criteria trades
params = { ...params, regime: $("regime-toggle").checked ? "1" : "0" };
const n = params.days || "the requested";
setLoading("replaying " + n + " session" + (n === 1 ? "" : "s") + "… (can take up to 30s)");
try {
const qs = new URLSearchParams(params).toString();
const r = await fetch("/api/backtest?" + qs);
const data = await r.json();
if (!r.ok) throw new Error(data.error || ("HTTP " + r.status));
state.data = data;
state.viewDate = null;
render(data);
} catch (e) {
$("loading").classList.add("hidden");
$("error").textContent = "backtest failed: " + e.message;
$("error").classList.remove("hidden");
}
}
document.querySelectorAll(".picker button[data-days]").forEach(btn => {
btn.onclick = () => runBacktest({ days: btn.dataset.days }, btn);
});
$("regime-toggle").onchange = () => runBacktest(state.lastParams || { days: 10 }, state.lastBtn || null);
$("run-custom").onclick = () => {
const since = $("since-input").value, until = $("until-input").value;
if (!since || !until) return;
runBacktest({ since, until }, null);
};
// -------------------------------------------------------------------------
// render
// -------------------------------------------------------------------------
function render(data) {
$("loading").classList.add("hidden");
$("results").classList.remove("hidden");
$("footer").classList.remove("hidden");
renderChips(data.settings, data.clamped);
renderSummary(data.summary, data.skipped, data.screened, data.live_split);
renderDaily(data.daily);
renderTrades(data.trades, null);
const y = data.sessions_requested;
const x = (data.daily || []).length;
$("coverage-line").textContent = "sessions replayed: " + x + " of " + y + " requested" +
(data.clamped ? " (clamped to the 45-session cap)" : "") +
(x < y ? " — a nightly setup_events build gap may explain the shortfall" : "");
}
function renderChips(s, clamped) {
if (!s) { $("chips").innerHTML = ""; return; }
const gap = s.population.min_entry_gap_pct > 0 ? "gap≥" + (s.population.min_entry_gap_pct * 100).toFixed(0) + "%" : "gap gate off";
const chips = [
"target_r=" + s.target_r,
"price $" + s.population.price_min + "-" + s.population.price_max,
gap,
"regime gate " + (s.regime_gate ? "ON" : "OFF"),
"fidelity atr×" + s.fidelity.atr_frac + " spread×" + s.fidelity.spread_mult + " adverse=" + s.fidelity.adverse_ticks + "tick",
];
if (clamped) chips.push("CLAMPED to 45 sessions");
$("chips").innerHTML = chips.map(c => '<span class="chip ' + (c.includes("ON") ? "on" : "") + '">' + esc(c) + '</span>').join("");
}
function renderSummary(sum, skipped, screened, split) {
sum = sum || { n: 0, sum_r: 0, avg_r: 0, hit_rate: 0, sessions: 0 };
const cell = (label, val, cls) => '<div class="cell"><div class="label">' + label +
'</div><div class="value ' + (cls || "") + '">' + val + '</div></div>';
const sp = (g) => g ? fmtR(g.sum_r) + ' <span style="font-size:10px;color:var(--muted)">avg ' +
fmtR(g.avg_r) + '</span>' : "—";
const taken = split && split.taken, not_taken = split && split.not_taken;
$("summary").innerHTML =
cell("Trades", sum.n) +
cell("Sum R", fmtR(sum.sum_r), sum.sum_r >= 0 ? "pos" : "neg") +
cell("Avg R", fmtR(sum.avg_r), sum.avg_r >= 0 ? "pos" : "neg") +
cell("Hit Rate", (sum.hit_rate * 100).toFixed(0) + "%") +
cell("Sessions", sum.sessions) +
cell("Skipped", skipped + " <span style=\"font-size:10px;color:var(--muted)\">/ pop " +
(screened ? screened.population : 0) + " · regime " + (screened ? screened.regime : 0) + "</span>") +
cell("Live-taken (" + (taken ? taken.n : 0) + ")", sp(taken),
taken ? (taken.sum_r >= 0 ? "pos" : "neg") : "") +
cell("Not taken (" + (not_taken ? not_taken.n : 0) + ")", sp(not_taken),
not_taken ? (not_taken.sum_r >= 0 ? "pos" : "neg") : "");
}
// day-over-day R curve — waterfall adapted from dashboard.html's renderDaily
// (cum_pnl -> cum_r, day_pnl -> sum_r); click a day filters the trade table.
function renderDaily(days) {
const host = $("daily-curve");
if (!days || !days.length) {
const reg = ((state.data || {}).screened || {}).regime || 0;
host.innerHTML = '<span class="empty">' + (reg > 0
? 'No trades: the regime gate sat out every replayed session in this window — the bot would not have traded these days at all. Uncheck \u201capply regime gate\u201d above to see what the raw criteria would have done.'
: 'No trades in this window (no qualifying setups).') + '</span>';
return;
}
const n = days.length;
const W = Math.max(host.clientWidth || 680, n * 48);
const H = 184, padT = 16, padB = 30, padX = 12, plotH = H - padT - padB;
const cums = days.map(d => +d.cum_r || 0);
const lo = Math.min(0, ...cums), hi = Math.max(0, ...cums), rng = (hi - lo) || 1;
const Y = v => padT + plotH * (1 - (v - lo) / rng);
const slot = (W - 2 * padX) / n, cx = i => padX + slot * (i + 0.5);
const barW = Math.min(slot * 0.5, 34);
const z = Y(0).toFixed(1);
let bars = "", labels = "", dots = "", prev = 0;
days.forEach((d, i) => {
const cur = cums[i], up = (+d.sum_r || 0) >= 0, col = up ? "#34d17e" : "#ff5a60";
const y1 = Y(prev), y2 = Y(cur), top = Math.min(y1, y2), h = Math.max(1, Math.abs(y1 - y2));
const x = (cx(i) - barW / 2).toFixed(1);
const sel = state.viewDate === d.date ? " daily-sel" : "";
const tip = d.date + " · " + d.n + " trades · sum " + fmtR(d.sum_r) + " · cum " + fmtR(cur);
bars += '<g class="daily-col' + sel + '" onclick="filterByDay(\'' + d.date + '\')"><title>' + esc(tip) + '</title>'
+ '<rect class="daily-hit" x="' + (padX + slot * i).toFixed(1) + '" y="' + padT + '" width="' + slot.toFixed(1) + '" height="' + plotH + '"/>'
+ '<rect x="' + x + '" y="' + top.toFixed(1) + '" width="' + barW.toFixed(1) + '" height="' + h.toFixed(1) + '" rx="2" fill="' + col + '" opacity=".5"/>'
+ '</g>';
labels += '<text class="daily-lbl' + sel + '" x="' + cx(i).toFixed(1) + '" y="' + (H - 10) + '">' + d.date.slice(5) + '</text>';
dots += '<circle cx="' + cx(i).toFixed(1) + '" cy="' + y2.toFixed(1) + '" r="2.8" fill="#5ba3ff"/>';
prev = cur;
});
const line = days.map((d, i) => cx(i).toFixed(1) + "," + Y(cums[i]).toFixed(1)).join(" ");
host.innerHTML = '<svg width="' + W + '" height="' + H + '" viewBox="0 0 ' + W + ' ' + H + '" class="daily-svg">'
+ '<line x1="' + padX + '" y1="' + z + '" x2="' + (W - padX) + '" y2="' + z + '" stroke="#30363d" stroke-width="1" stroke-dasharray="3 3"/>'
+ bars + '<polyline points="' + line + '" fill="none" stroke="#5ba3ff" stroke-width="1.6"/>' + dots + labels + '</svg>';
}
function filterByDay(date) {
state.viewDate = (state.viewDate === date) ? null : date;
renderDaily(state.data.daily);
renderTrades(state.data.trades, state.viewDate);
}
function renderTrades(trades, viewDate) {
trades = trades || [];
const rows = viewDate ? trades.filter(t => t.session_date === viewDate) : trades;
$("trades-sub").textContent = viewDate ? (rows.length + " on " + viewDate + " — click the day again to clear") :
(rows.length + " total");
if (!rows.length) { $("trades-body").innerHTML = '<tr><td colspan="7" class="empty">No trades.</td></tr>'; return; }
$("trades-body").innerHTML = rows.map((t, i) => {
const idx = trades.indexOf(t);
const cls = t.r_multiple >= 0 ? "pos" : "neg";
let live = '<span style="color:var(--muted)">—</span>';
if (t.live_pnl != null) {
const lcls = t.live_pnl >= 0 ? "pos" : "neg";
// sign disagreement = the per-trade caveat made visible (minute-bar
// granularity + adverse fills); aggregates are the trustworthy view
const diverged = (t.live_pnl >= 0) !== (t.r_multiple >= 0)
? ' <span style="color:var(--amber)" title="sim and live disagree on this trade — minute-bar granularity + adverse fills; trust aggregates, not rows">&ne;</span>' : '';
live = '<span class="r ' + lcls + '">' + (t.live_pnl >= 0 ? "+$" : "−$") +
Math.abs(t.live_pnl).toFixed(0) + '</span>' + diverged;
}
return '<tr onclick="openTradeChart(' + idx + ')">' +
'<td>' + esc(t.session_date) + '</td>' +
'<td class="sym">' + esc(t.symbol) + '</td>' +
'<td>' + fmtTs(t.entry_ts) + ' @ ' + fmtPx(t.entry_fill) + '</td>' +
'<td>' + fmtTs(t.exit_ts) + ' @ ' + fmtPx(t.fills.length ? t.fills[t.fills.length - 1].price : null) + '</td>' +
'<td>' + esc(t.exit_reason) + '</td>' +
'<td class="r ' + cls + '">' + fmtR(t.r_multiple) + '</td>' +
'<td>' + live + '</td>' +
'</tr>';
}).join("");
}
// -------------------------------------------------------------------------
// chart drill-in — imitates dashboard.html's openChart/refreshChart (plan 013)
// -------------------------------------------------------------------------
let chartState = { chart: null, series: null, markersApi: null };
function closeChart() {
$("chart-modal").classList.remove("show");
if (chartState.chart) chartState.chart.remove();
chartState = { chart: null, series: null, markersApi: null };
}
async function openTradeChart(idx) {
if (!window.LightweightCharts) return;
const t = state.data.trades[idx];
if (!t) return;
if (chartState.chart) closeChart();
$("chart-sym").textContent = t.symbol;
$("chart-meta").textContent = t.session_date + " · " + t.exit_reason + " · " + fmtR(t.r_multiple);
$("chart-modal").classList.add("show");
const container = $("chart-container");
container.innerHTML = "";
const chart = LightweightCharts.createChart(container, {
autoSize: true,
layout: { background: { color: "#111622" }, textColor: "#e3e8f2" },
grid: { vertLines: { color: "#18202f" }, horzLines: { color: "#18202f" } },
timeScale: { timeVisible: true, secondsVisible: false },
});
const series = chart.addSeries(LightweightCharts.CandlestickSeries, {
upColor: "#2fbf71", downColor: "#e5484d", borderVisible: false,
wickUpColor: "#2fbf71", wickDownColor: "#e5484d",
});
const markersApi = LightweightCharts.createSeriesMarkers(series, []);
chartState = { chart, series, markersApi };
try {
const r = await fetch("/api/chart?symbol=" + encodeURIComponent(t.symbol) +
"&hours=24&date=" + encodeURIComponent(t.session_date));
const data = await r.json();
series.setData((data.bars || []).map(b => ({ time: b.time, open: b.open, high: b.high, low: b.low, close: b.close })));
const markers = [
{ time: t.entry_epoch, position: "belowBar", shape: "arrowUp", color: "#5ba3ff", text: "entry " + fmtPx(t.entry_fill) },
{ time: t.exit_epoch, position: "aboveBar", shape: "arrowDown", color: "#c2a6e2", text: t.exit_reason },
].sort((a, b) => a.time - b.time);
markersApi.setMarkers(markers);
} catch (e) { /* leave the empty chart on screen */ }
}
document.addEventListener("click", (e) => { if (e.target.id === "chart-modal") closeChart(); });
document.addEventListener("keydown", (e) => { if (e.key === "Escape") closeChart(); });
$("chart-close").onclick = closeChart;
// -------------------------------------------------------------------------
// init — default 10-session window
// -------------------------------------------------------------------------
runBacktest({ days: 10 }, document.querySelector('.picker button[data-days="10"]'));
</script>
</body>
</html>
+892 -136
View File
File diff suppressed because it is too large. Load diff
+2
View File
@@ -0,0 +1,2 @@
TradingView Lightweight Charts™
Copyright (с) 2025 TradingView, Inc. https://www.tradingview.com/
@@ -0,0 +1,7 @@
/*!
* @license
* TradingView Lightweight Charts™ v5.2.0
* Copyright (c) 2026 TradingView, Inc.
* Licensed under Apache License 2.0 https://www.apache.org/licenses/LICENSE-2.0
*/
!function(){"use strict";const t={title:"",visible:!0,hitTestTolerance:3,lastValueVisible:!0,priceLineVisible:!0,priceLineSource:0,priceLineWidth:1,priceLineColor:"",priceLineStyle:2,baseLineVisible:!0,baseLineWidth:1,baseLineColor:"#B2B5BE",baseLineStyle:0,priceFormat:{type:"price",precision:2,minMove:.01}};var i,n;function s(t,i){const n=function(t,i){switch(t){case 0:default:return[];case 1:return[i,i];case 2:return[2*i,2*i];case 3:return[6*i,6*i];case 4:return[i,4*i]}}(i,t.lineWidth);return t.setLineDash(n),n}function e(t,i,n,s){t.beginPath();const e=t.lineWidth%2?.5:0;t.moveTo(n,i+e),t.lineTo(s,i+e),t.stroke()}function r(t,i){if(!t)throw new Error("Assertion failed"+(i?": "+i:""))}function h(t){if(void 0===t)throw new Error("Value is undefined");return t}function a(t){if(null===t)throw new Error("Value is null");return t}function l(t){return a(h(t))}!function(t){t[t.Simple=0]="Simple",t[t.WithSteps=1]="WithSteps",t[t.Curved=2]="Curved"}(i||(i={})),function(t){t[t.Solid=0]="Solid",t[t.Dotted=1]="Dotted",t[t.Dashed=2]="Dashed",t[t.LargeDashed=3]="LargeDashed",t[t.SparseDotted=4]="SparseDotted"}(n||(n={}));class o{constructor(){this.t=[]}i(t,i,n){const s={h:t,l:i,o:!0===n};this.t.push(s)}_(t){const i=this.t.findIndex((i=>t===i.h));i>-1&&this.t.splice(i,1)}u(t){this.t=this.t.filter((i=>i.l!==t))}p(t,i,n){const s=[...this.t];this.t=this.t.filter((t=>!t.o)),s.forEach((s=>s.h(t,i,n)))}v(){return this.t.length>0}m(){this.t=[]}}function _(t,...i){for(const n of i)for(const i in n)void 0!==n[i]&&Object.prototype.hasOwnProperty.call(n,i)&&!["__proto__","constructor","prototype"].includes(i)&&("object"!=typeof n[i]||void 0===t[i]||Array.isArray(n[i])?t[i]=n[i]:_(t[i],n[i]));return t}function u(t){return"number"==typeof t&&isFinite(t)}function c(t){return"number"==typeof t&&t%1==0}function d(t){return"string"==typeof t}function f(t){return"boolean"==typeof t}function p(t){const i=t;if(!i||"object"!=typeof i)return i;let n,s,e;for(s in n=Array.isArray(i)?[]:{},i)i.hasOwnProperty(s)&&(e=i[s],n[s]=e&&"object"==typeof e?p(e):e);return n}function v(t){return null!==t}function m(t){return null===t?void 0:t}const w="-apple-system, BlinkMacSystemFont, 'Trebuchet MS', Roboto, Ubuntu, sans-serif";function g(t,i,n){return void 0===i&&(i=w),`${n=void 0!==n?`${n} `:""}${t}px ${i}`}class M{constructor(t){this.M={S:1,C:5,k:NaN,P:"",T:"",R:"",D:"",I:0,V:0,B:0,A:0,L:0},this.O=t}N(){const t=this.M,i=this.F(),n=this.W();return t.k===i&&t.T===n||(t.k=i,t.T=n,t.P=g(i,n),t.A=2.5/12*i,t.I=t.A,t.V=i/12*t.C,t.B=i/12*t.C,t.L=0),t.R=this.H(),t.D=this.U(),this.M}H(){return this.O.N().layout.textColor}U(){return this.O.$()}F(){return this.O.N().layout.fontSize}W(){return this.O.N().layout.fontFamily}}function b(t){return t<0?0:t>255?255:Math.round(t)||0}function S(t){return.199*t[0]+.687*t[1]+.114*t[2]}class x{constructor(t,i){this.j=new Map,this.q=t,i&&(this.j=i)}Y(t,i){if("transparent"===t)return t;const n=this.K(t),s=n[3];return`rgba(${n[0]}, ${n[1]}, ${n[2]}, ${i*s})`}Z(t){const i=this.K(t);return{G:`rgb(${i[0]}, ${i[1]}, ${i[2]})`,X:S(i)>160?"black":"white"}}J(t){return S(this.K(t))}tt(t,i,n){const[s,e,r,h]=this.K(t),[a,l,o,_]=this.K(i),u=[b(s+n*(a-s)),b(e+n*(l-e)),b(r+n*(o-r)),(c=h+n*(_-h),c<=0||c>1?Math.min(Math.max(c,0),1):Math.round(1e4*c)/1e4)];var c;return`rgba(${u[0]}, ${u[1]}, ${u[2]}, ${u[3]})`}K(t){const i=this.j.get(t);if(i)return i;const n=function(t){const i=document.createElement("div");i.style.display="none",document.body.appendChild(i),i.style.color=t;const n=window.getComputedStyle(i).color;return document.body.removeChild(i),n}(t),s=n.match(/^rgba?\s*\((\d+),\s*(\d+),\s*(\d+)(?:,\s*(\d*\.?\d+))?\)$/);if(!s){if(this.q.length)for(const i of this.q){const n=i(t);if(n)return this.j.set(t,n),n}throw new Error(`Failed to parse color: ${t}`)}const e=[parseInt(s[1],10),parseInt(s[2],10),parseInt(s[3],10),s[4]?parseFloat(s[4]):1];return this.j.set(t,e),e}}class C{constructor(){this.it=[]}nt(t){this.it=t}st(t,i,n){this.it.forEach((s=>{s.st(t,i,n)}))}}class y{st(t,i,n){t.useBitmapCoordinateSpace((t=>this.et(t,i,n)))}}class k extends y{constructor(){super(...arguments),this.rt=null}ht(t){this.rt=t}et({context:t,horizontalPixelRatio:i,verticalPixelRatio:n}){if(null===this.rt||null===this.rt.lt)return;const s=this.rt.lt,e=this.rt,r=Math.max(1,Math.floor(i))%2/2,h=h=>{t.beginPath();for(let a=s.to-1;a>=s.from;--a){const s=e.ot[a],l=Math.round(s._t*i)+r,o=s.ut*n,_=h*n+r;t.moveTo(l,o),t.arc(l,o,_,0,2*Math.PI)}t.fill()};e.ct>0&&(t.fillStyle=e.dt,h(e.ft+e.ct)),t.fillStyle=e.vt,h(e.ft)}}function P(){return{ot:[{_t:0,ut:0,wt:0,gt:0}],vt:"",dt:"",ft:0,ct:0,lt:null}}const T={from:0,to:1};class R{constructor(t,i,n){this.Mt=new C,this.bt=[],this.St=[],this.xt=!0,this.O=t,this.Ct=i,this.yt=n,this.Mt.nt(this.bt)}kt(t){this.Pt(),this.xt=!0}Tt(){return this.xt&&(this.Rt(),this.xt=!1),this.Mt}Pt(){const t=this.yt.Dt();t.length!==this.bt.length&&(this.St=t.map(P),this.bt=this.St.map((t=>{const i=new k;return i.ht(t),i})),this.Mt.nt(this.bt))}Rt(){consLine truncated
+102
View File
@@ -0,0 +1,102 @@
#!/usr/bin/env python3
"""Targeted historical backfill: minute bars (premarket included, SIP) for the
year of recorded gapper-days in ``historical_gappers``.
Purpose: fatten n for the sample-starved queues — premarket-structure H1/H2,
pyramid_1r_v1, orderflow experiments — by giving eod_replay and the premarket
labeler real bar paths for ~260 historical sessions. Pulls ONLY each session's
recorded gappers (the population we trade), never the whole market.
PYTHONPATH=. python backfill_gapper_days.py --limit 20 # newest 20 days
PYTHONPATH=. python backfill_gapper_days.py # all recorded days
Per session: fetch 04:00-20:00 ET SIP minute bars for that day's gappers,
upsert, fetch ~40 calendar days of daily bars ending that session (the daily
screen needs prior-20-day volume), then build premarket_structure for the day.
Idempotent: sessions whose gappers already have premarket bars are skipped.
Replay/experiments consume the bars via their own runs afterwards:
for d in $(dates); do PYTHONPATH=. python eod_replay.py $d; done
"""
from __future__ import annotations
import argparse
import sys
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
from dotenv import load_dotenv
from runtime.env_overrides import load as _load_cfg_overrides
_load_cfg_overrides()
load_dotenv()
from alpaca_paper.client import AlpacaPaperClient # noqa: E402
from alpaca_paper.settings import AlpacaPaperSettings # noqa: E402
from research.ingestion.market_data import ( # noqa: E402
upsert_daily_bars,
upsert_minute_bars,
)
from research.multi_schema import open_research_db # noqa: E402
from research.premarket_structure import build as build_pm # noqa: E402
ET = ZoneInfo("America/New_York")
def backfill_session(con, client, sess, symbols: list[str], feed: str) -> dict:
start = datetime(sess.year, sess.month, sess.day, 4, 0, tzinfo=ET)
end = datetime(sess.year, sess.month, sess.day, 20, 0, tzinfo=ET)
by_sym = client.get_minute_bars(symbols, start_iso=start.isoformat(),
end_iso=end.isoformat(), feed=feed)
minute_rows = sum(upsert_minute_bars(con, s, b, source_provider="alpaca_sip_backfill")
for s, b in by_sym.items() if b)
d_start = (start - timedelta(days=40)).isoformat()
daily = client.get_daily_bars(symbols, start_iso=d_start,
end_iso=end.isoformat(), feed=feed)
daily_rows = sum(upsert_daily_bars(con, s, b) for s, b in daily.items() if b)
pm = build_pm(con, session_date=str(sess))
return {"minute": minute_rows, "daily": daily_rows, "pm_rows": pm["rows"]}
def main(argv=None) -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--limit", type=int, help="only the newest N recorded sessions")
ap.add_argument("--force", action="store_true", help="re-fetch even if bars exist")
args = ap.parse_args(argv)
con = open_research_db("market")
client = AlpacaPaperClient(AlpacaPaperSettings.from_env())
feed = client.settings.backfill_feed
lim = f" LIMIT {int(args.limit)}" if args.limit else ""
days = con.execute(
"SELECT session_date, array_agg(DISTINCT symbol) FROM historical_gappers "
f"GROUP BY session_date ORDER BY session_date DESC{lim}").fetchall()
print(f"{len(days)} recorded gapper sessions; feed={feed}")
done = skipped = 0
for sess, symbols in days:
symbols = [s for s in (symbols or []) if s]
if not symbols:
continue
if not args.force:
have = con.execute(
"SELECT count(DISTINCT symbol) FROM minute_bars "
"WHERE session_date = ? AND is_premarket AND symbol = ANY(?)",
[str(sess), symbols]).fetchall()[0][0]
if have >= max(1, len(symbols) // 2): # most already present -> skip
skipped += 1
continue
try:
out = backfill_session(con, client, sess, symbols, feed)
done += 1
print(f"{sess}: {len(symbols)} syms, {out['minute']} minute rows, "
f"{out['daily']} daily rows, {out['pm_rows']} pm-structure rows", flush=True)
except Exception as exc: # noqa: BLE001 — one bad day must not kill the sweep
print(f"{sess}: FAILED {exc}", flush=True)
print(f"done: {done} sessions backfilled, {skipped} already present")
return 0
if __name__ == "__main__":
sys.exit(main())
+5 -3
View File
@@ -45,16 +45,18 @@ def main() -> None:
args = ap.parse_args()
load_dotenv()
from alpaca_paper.client import AlpacaPaperClient
from storage.event_schema import EventMode, PositionClosedEvent
from storage.event_store import EventStore
from storage.journal_backfill import flatten_fills, reconstruct_round_trips
from tradier_broker.factory import make_trading_client
store = EventStore("momentum")
client = AlpacaPaperClient()
client, label = make_trading_client()
if client is None:
raise SystemExit(f"[backfill_journal] {label} — aborting")
raw = client.get_orders(status="closed", limit=500, nested=True)
print(f"fetched {len(raw)} historical orders from Alpaca")
print(f"fetched {len(raw)} historical orders from {label}")
trips = reconstruct_round_trips(flatten_fills(raw))
print(f"reconstructed {len(trips)} completed round-trips")
-326
View File
@@ -1,326 +0,0 @@
#!/usr/bin/env python3
"""Multi-week comprehensive backtest with detailed statistical analysis."""
from datetime import datetime, timedelta, date
import statistics
import pandas as pd
from rich.console import Console
from rich.table import Table
from config import Config
from research.connection import ResearchConnection
from research.query import query_minute_bars
from strategy.backtest.engine import BacktestEngine
console = Console()
def get_trading_days(start: date, end: date) -> list[date]:
"""Get all weekdays between start and end (trading days only)."""
days = []
current = start
while current <= end:
# Only include weekdays (0=Monday, 6=Sunday)
if current.weekday() < 5:
days.append(current)
current += timedelta(days=1)
return days
def run_backtest_multi_day(start_date: date, end_date: date, symbols: list[str] | None = None):
"""Run backtest across multiple days and aggregate results."""
config = Config()
con = ResearchConnection()
if symbols is None:
symbols = ["SNDL", "MARA", "RIOT", "BBAI", "SOUN", "PLUG"]
console.print(f"\n[bold blue]Comprehensive Backtest: {start_date} to {end_date}[/bold blue]")
console.print(f"[dim]Testing {len(symbols)} symbols across trading days[/dim]\n")
trading_days = get_trading_days(start_date, end_date)
console.print(f"[cyan]{len(trading_days)} trading days in range[/cyan]\n")
# Aggregate results
all_trades: list[dict] = []
daily_results: dict[date, dict] = {}
symbol_stats: dict[str, dict] = {}
engine = BacktestEngine(config)
# Progress bar
processed = 0
for trading_day in trading_days:
# Query minute bars for all symbols that day
day_symbols_with_data = []
day_bars = {}
for symbol in symbols:
try:
bars = query_minute_bars(con, symbol, trading_day)
if len(bars) > 0:
day_symbols_with_data.append(symbol)
day_bars[symbol] = bars
except Exception as e:
pass # No data for this day
if not day_symbols_with_data:
processed += 1
continue
# Initialize daily results if needed
if trading_day not in daily_results:
daily_results[trading_day] = {"trades": [], "pnl": 0.0}
# Run backtest for each symbol with data
for symbol in day_symbols_with_data:
bars = day_bars[symbol]
try:
result = engine.run(bars, symbol)
# Aggregate
daily_results[trading_day]["pnl"] += result.total_pnl
# Initialize symbol stats if needed
if symbol not in symbol_stats:
symbol_stats[symbol] = {"trades": [], "wins": 0, "losses": 0, "pnl": 0.0}
for trade in result.trades:
trade_data = {
"date": trading_day.isoformat(),
"symbol": symbol,
"entry_time": trade.entry_time,
"entry_price": trade.entry_price,
"stop_price": trade.stop_price,
"target_price": trade.target_price,
"quantity": trade.quantity,
"exit_price": trade.exit_price,
"exit_reason": trade.exit_reason,
"pnl": trade.realized_pnl or 0.0,
"r_multiple": trade.r_multiple or 0.0,
}
all_trades.append(trade_data)
daily_results[trading_day]["trades"].append(trade_data)
# Symbol stats
symbol_stats[symbol]["trades"].append(trade_data)
symbol_stats[symbol]["pnl"] += trade.realized_pnl or 0.0
if (trade.realized_pnl or 0) > 0:
symbol_stats[symbol]["wins"] += 1
else:
symbol_stats[symbol]["losses"] += 1
except Exception as e:
console.print(f"[red]Error backtesting {symbol} on {trading_day}: {e}[/red]")
processed += 1
console.print(f"[dim]Processed {processed}/{len(trading_days)} days[/dim]", end="\r")
console.print()
# === SUMMARY STATISTICS ===
if not all_trades:
console.print("[yellow]No trades found in backtest range[/yellow]")
return
# Daily summary
console.print("\n[bold cyan]=== DAILY SUMMARY ===[/bold cyan]")
daily_table = Table(title="P&L by Day")
daily_table.add_column("Date", style="cyan")
daily_table.add_column("Trades", justify="right")
daily_table.add_column("P&L", justify="right")
daily_pnls = []
for day in sorted(daily_results.keys()):
daily_pnl = daily_results[day]["pnl"]
daily_pnls.append(daily_pnl)
color = "green" if daily_pnl > 0 else "red"
daily_table.add_row(
str(day),
str(len(daily_results[day]["trades"])),
f"[{color}]${daily_pnl:,.2f}[/{color}]"
)
console.print(daily_table)
# Symbol summary
console.print("\n[bold cyan]=== SYMBOL SUMMARY ===[/bold cyan]")
symbol_table = Table(title="Results by Symbol")
symbol_table.add_column("Symbol", style="cyan")
symbol_table.add_column("Trades", justify="right")
symbol_table.add_column("Wins", justify="right")
symbol_table.add_column("Win %", justify="right")
symbol_table.add_column("P&L", justify="right")
symbol_table.add_column("Avg R", justify="right")
for symbol in sorted(symbol_stats.keys()):
stats = symbol_stats[symbol]
trades = stats["trades"]
if not trades:
continue
win_pct = (stats["wins"] / len(trades)) * 100 if trades else 0
avg_r = statistics.mean([t["r_multiple"] for t in trades]) if trades else 0
color = "green" if stats["pnl"] > 0 else "red"
symbol_table.add_row(
symbol,
str(len(trades)),
str(stats["wins"]),
f"{win_pct:.1f}%",
f"[{color}]${stats['pnl']:,.2f}[/{color}]",
f"{avg_r:.2f}R"
)
console.print(symbol_table)
# Overall stats
total_trades = len(all_trades)
total_pnl = sum(t["pnl"] for t in all_trades)
wins = len([t for t in all_trades if t["pnl"] > 0])
losses = len([t for t in all_trades if t["pnl"] < 0])
win_rate = (wins / total_trades) * 100 if total_trades else 0
r_multiples = [t["r_multiple"] for t in all_trades]
avg_r = statistics.mean(r_multiples) if r_multiples else 0
# Separate wins and losses for analysis
win_trades = [t for t in all_trades if t["pnl"] > 0]
loss_trades = [t for t in all_trades if t["pnl"] < 0]
avg_win = statistics.mean([t["pnl"] for t in win_trades]) if win_trades else 0
avg_loss = statistics.mean([t["pnl"] for t in loss_trades]) if loss_trades else 0
max_loss = min([t["pnl"] for t in loss_trades]) if loss_trades else 0
profit_factor = (sum([t["pnl"] for t in win_trades]) / abs(sum([t["pnl"] for t in loss_trades]))) if loss_trades and sum([t["pnl"] for t in loss_trades]) < 0 else 0
console.print("\n[bold cyan]=== OVERALL STATISTICS ===[/bold cyan]")
console.print(f"Total Trades: {total_trades}")
console.print(f"Wins: {wins} | Losses: {losses}")
console.print(f"Win Rate: {win_rate:.1f}%")
console.print(f"Total P&L: [{'green' if total_pnl > 0 else 'red'}]${total_pnl:,.2f}[/{'green' if total_pnl > 0 else 'red'}]")
console.print(f"Avg Win: ${avg_win:,.2f} | Avg Loss: ${avg_loss:,.2f}")
console.print(f"Max Loss: ${max_loss:,.2f}")
if profit_factor > 0:
console.print(f"Profit Factor: {profit_factor:.2f}x")
console.print(f"Avg R-Multiple: {avg_r:.2f}R")
# === LOSS ANALYSIS ===
if loss_trades:
console.print("\n[bold yellow]=== LOSS ANALYSIS ===[/bold yellow]")
console.print(f"Total Losing Trades: {len(loss_trades)}")
loss_table = Table(title="Top 10 Losses (for pattern analysis)")
loss_table.add_column("Date", style="cyan")
loss_table.add_column("Symbol", style="cyan")
loss_table.add_column("Entry", justify="right")
loss_table.add_column("Stop", justify="right")
loss_table.add_column("Exit", justify="right")
loss_table.add_column("P&L", justify="right")
loss_table.add_column("Reason")
sorted_losses = sorted(loss_trades, key=lambda t: t["pnl"])
for trade in sorted_losses[:10]:
loss_table.add_row(
trade["date"],
trade["symbol"],
f"${trade['entry_price']:.2f}",
f"${trade['stop_price']:.2f}",
f"${trade['exit_price']:.2f}" if trade["exit_price"] else "-",
f"[red]${trade['pnl']:,.2f}[/red]",
trade["exit_reason"] or "?"
)
console.print(loss_table)
# Loss metrics
stop_hits = len([t for t in loss_trades if "stop" in (t["exit_reason"] or "").lower()])
console.print(f"\n[dim]Losses by reason:[/dim]")
console.print(f" Hit stop: {stop_hits}/{len(loss_trades)}")
# Gap-through analysis
for trade in sorted_losses[:5]:
entry = trade["entry_price"]
stop = trade["stop_price"]
exit_price = trade["exit_price"]
if exit_price and exit_price < stop:
gap_pct = ((stop - exit_price) / stop) * 100
console.print(f" [yellow]{trade['symbol']} {trade['date']}: gap-through {gap_pct:.2f}% ({stop:.2f} → {exit_price:.2f})[/yellow]")
# === WIN ANALYSIS ===
if win_trades:
console.print("\n[bold green]=== WIN ANALYSIS ===[/bold green]")
console.print(f"Total Winning Trades: {len(win_trades)}")
win_table = Table(title="Recent Wins")
win_table.add_column("Date", style="cyan")
win_table.add_column("Symbol", style="cyan")
win_table.add_column("Entry", justify="right")
win_table.add_column("Target", justify="right")
win_table.add_column("Exit", justify="right")
win_table.add_column("P&L", justify="right")
win_table.add_column("R")
sorted_wins = sorted(win_trades, key=lambda t: t["pnl"], reverse=True)
for trade in sorted_wins[:5]:
win_table.add_row(
trade["date"],
trade["symbol"],
f"${trade['entry_price']:.2f}",
f"${trade['target_price']:.2f}",
f"${trade['exit_price']:.2f}" if trade["exit_price"] else "-",
f"[green]${trade['pnl']:,.2f}[/green]",
f"{trade['r_multiple']:.2f}R"
)
console.print(win_table)
# === RISK ASSESSMENT ===
console.print("\n[bold yellow]=== RISK ASSESSMENT ===[/bold yellow]")
if avg_loss != 0:
expectancy = (win_rate / 100 * avg_win) + ((1 - win_rate / 100) * avg_loss)
console.print(f"Expected value per trade: ${expectancy:,.2f}")
if profit_factor > 0:
if profit_factor < 1.5:
console.print("[yellow]⚠️ Profit factor < 1.5 indicates marginal edge or slippage issues[/yellow]")
else:
console.print("[green]✓ Profit factor suggests adequate win/loss ratio[/green]")
# Check for consistent losses
max_win = max([t["pnl"] for t in win_trades]) if win_trades else 0
if max_loss < 0 and abs(max_loss) > max_win * 1.5:
console.print(f"[red]⚠️ Max loss (${max_loss:,.2f}) >> max win (${max_win:,.2f}) — stop is too wide or slippage is severe[/red]")
# Sharpe-ish ratio (simplified)
if len(daily_pnls) > 1:
daily_std = statistics.stdev(daily_pnls) if len(daily_pnls) > 1 else 0
if daily_std > 0:
daily_mean = statistics.mean(daily_pnls)
sharpe_approx = daily_mean / daily_std if daily_std > 0 else 0
console.print(f"Daily Sharpe (approx): {sharpe_approx:.2f}")
console.print("\n[dim]Recommendations:[/dim]")
if win_rate < 40:
console.print("[red]• Win rate < 40% — strategy may lack edge even with profitable trades[/red]")
if profit_factor < 1.3 and profit_factor > 0:
console.print("[yellow]• Profit factor < 1.3 — consider tightening stops or improving entry[/yellow]")
if len(loss_trades) > 0 and max_loss < avg_loss / 2:
console.print("[yellow]• Losses are inconsistent — gap-throughs may be a factor[/yellow]")
console.print()
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Multi-week comprehensive backtest")
parser.add_argument("--start", type=str, default="2026-06-05", help="Start date (YYYY-MM-DD)")
parser.add_argument("--end", type=str, default="2026-06-12", help="End date (YYYY-MM-DD)")
parser.add_argument("--symbols", type=str, default="SNDL,MARA,RIOT,BBAI,SOUN,PLUG", help="Comma-separated symbols")
args = parser.parse_args()
start = datetime.strptime(args.start, "%Y-%m-%d").date()
end = datetime.strptime(args.end, "%Y-%m-%d").date()
symbols = args.symbols.split(",")
run_backtest_multi_day(start, end, symbols)
+31 -1
View File
@@ -7,6 +7,31 @@ from typing import Any
import yaml
from pydantic import BaseModel, ConfigDict
# Ollama runs on the LAN host reached via EITHER the 127.0.0.1 netsh tunnel OR the direct
# 192.168.1.5 — and which works FLIPS across WSL/host reboots (same landmine as the DB).
# The two use DIFFERENT ports (tunnel 30069 -> direct 30068), so we probe known endpoints
# and pick the first reachable, so enrichment self-heals on boot instead of silently failing.
_OLLAMA_CANDIDATES = ("http://192.168.1.5:30068", "http://127.0.0.1:30069")
def _ollama_ok(base: str) -> bool:
import urllib.request
try:
with urllib.request.urlopen(base.rstrip("/") + "/api/tags", timeout=2) as r:
return r.status == 200
except Exception: # noqa: BLE001
return False
def _resolve_ollama_host(host: str) -> str:
"""Return the configured host if reachable, else the first reachable known endpoint."""
if _ollama_ok(host):
return host
for cand in _OLLAMA_CANDIDATES:
if cand != host and _ollama_ok(cand):
return cand
return host
class SlippageConfig(BaseModel):
base_spread_pct: float = 0.0015
@@ -158,6 +183,9 @@ class OllamaConfig(BaseModel):
timeout_seconds: int = 30
max_tokens: int = 256
temperature: float = 0.3
# Ollama "think" flag. None = not sent (models without a thinking mode). qwen3 thinks by default, and with
# format=json Ollama then returns an empty "{}", so set OLLAMA_THINK=false for qwen3.
think: bool | None = None
# News/catalyst enrichment (advisory layer).
enrichment_enabled: bool = False
enrichment_interval_seconds: int = 120
@@ -197,11 +225,13 @@ class OllamaConfig(BaseModel):
return cls(
enabled=b("OLLAMA_ENABLED", False),
host=v.get("OLLAMA_HOST", cls.model_fields["host"].default),
host=_resolve_ollama_host(v.get("OLLAMA_HOST", cls.model_fields["host"].default)),
model=v.get("OLLAMA_MODEL", cls.model_fields["model"].default),
timeout_seconds=i("OLLAMA_TIMEOUT_SECONDS", 30),
max_tokens=i("OLLAMA_MAX_TOKENS", 256),
temperature=f("OLLAMA_TEMPERATURE", 0.3),
think=(None if not (v.get("OLLAMA_THINK") or "").strip()
else v["OLLAMA_THINK"].strip().lower() in {"1", "true", "yes", "on"}),
enrichment_enabled=b("NEWS_ENRICH_ENABLED", False),
enrichment_interval_seconds=i("NEWS_ENRICH_INTERVAL_SECONDS", 120),
enrichment_lookback_hours=i("NEWS_ENRICH_LOOKBACK_HOURS", 12),
-259
View File
@@ -1,259 +0,0 @@
"""Module contracts and interfaces for the momentum trading system.
Every module must implement the ModuleContract protocol, defining:
- input_schema: What the module expects
- output_schema: What the module produces
- run(): Execute the module
- validate_input(): Ensure data meets contracts
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from datetime import datetime, date
from typing import Any, Protocol, TypeVar
T_Input = TypeVar("T_Input")
T_Output = TypeVar("T_Output")
@dataclass
class ModuleMetrics:
"""Standard metrics every module should track."""
module_name: str
timestamp: datetime
duration_ms: float
input_count: int = 0
output_count: int = 0
error_count: int = 0
warnings: list[str] = field(default_factory=list)
custom_metrics: dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> dict:
return {
"module_name": self.module_name,
"timestamp": self.timestamp.isoformat(),
"duration_ms": self.duration_ms,
"input_count": self.input_count,
"output_count": self.output_count,
"error_count": self.error_count,
"warnings": self.warnings,
"custom_metrics": self.custom_metrics,
}
@dataclass
class ValidationFailure:
"""Single validation failure with context."""
field: str
value: Any
reason: str
severity: str = "error" # "error" or "warning"
@dataclass
class ModuleResult:
"""Base class for all module results."""
success: bool
timestamp: datetime
metrics: ModuleMetrics
validation_failures: list[ValidationFailure] = field(default_factory=list)
def has_errors(self) -> bool:
return bool(self.validation_failures or not self.success)
def error_summary(self) -> str:
"""Human-readable error summary."""
if not self.validation_failures:
return "No errors"
errors = [f for f in self.validation_failures if f.severity == "error"]
if not errors:
return f"{len(self.validation_failures)} warnings only"
return f"{len(errors)} error(s), {len([f for f in self.validation_failures if f.severity == 'warning'])} warning(s)"
class ModuleContract(Protocol):
"""Contract every module must fulfill."""
def run(self, input: Any) -> ModuleResult:
"""Execute the module.
Args:
input: Module-specific input conforming to input_schema
Returns:
Module-specific result inheriting from ModuleResult
"""
...
def validate_input(self, input: Any) -> tuple[bool, list[ValidationFailure]]:
"""Validate input before processing.
Args:
input: Data to validate
Returns:
(is_valid, failures)
"""
...
def get_metrics(self) -> ModuleMetrics:
"""Get last execution metrics."""
...
# ============================================================================
# INGESTION MODULE CONTRACTS
# ============================================================================
@dataclass
class BarIngestRequest:
"""Input contract for ingestion module."""
symbols: list[str]
session_date: date
lookback_minutes: int = 120 # how many minutes of history to fetch
source: str = "alpaca" # "alpaca", "test", "file"
@dataclass
class BarIngestResult(ModuleResult):
"""Output contract for ingestion module."""
minute_rows_inserted: int = 0
symbols_updated: list[str] = field(default_factory=list)
source_health: dict[str, str] = field(default_factory=dict) # {source: "healthy"|"error"}
# ============================================================================
# RESEARCH MODULE CONTRACTS
# ============================================================================
@dataclass
class ResearchRequest:
"""Input contract for research module."""
symbols: list[str]
session_date: date
lookback_days: int = 30 # for daily history
@dataclass
class ResearchResult(ModuleResult):
"""Output contract for research module."""
symbols_researched: list[str] = field(default_factory=list)
symbols_with_data: list[str] = field(default_factory=list)
symbols_missing_data: list[str] = field(default_factory=list)
total_bars_available: int = 0
# ============================================================================
# EVALUATION MODULE CONTRACTS
# ============================================================================
@dataclass
class CriterionDetail:
"""Result of evaluating one criterion."""
name: str
passed: bool
value: float # actual metric value
threshold: float | None = None # minimum to pass
reason: str = ""
confidence: float = 1.0 # 0-1, how confident in this result
@dataclass
class EvaluationRequest:
"""Input contract for evaluation module."""
symbol: str
bars_df: Any # pd.DataFrame with {timestamp, open, high, low, close, volume}
previous_close: float
avg_daily_volume: float
session_date: date
@dataclass
class EvaluationResult(ModuleResult):
"""Output contract for evaluation module."""
symbol: str = ""
status: str = "late" # "ready", "blocked", "late", "error"
score: float = 0.0 # 0-100
criteria: list[CriterionDetail] = field(default_factory=list)
entry_price: float | None = None
stop_loss: float | None = None
target_price: float | None = None
quality_score: float = 0.0 # 0-1
blocking_reason: str | None = None
metrics_dict: dict[str, float] = field(default_factory=dict) # gap%, rvol, etc.
# ============================================================================
# EXECUTION MODULE CONTRACTS
# ============================================================================
@dataclass
class ExecutionRequest:
"""Input contract for execution module."""
symbol: str
entry_price: float
stop_price: float
target_price: float
score: float
criteria_count: int
account_equity: float
current_positions: int
today_realized_pnl: float
@dataclass
class ExecutionResult(ModuleResult):
"""Output contract for execution module."""
symbol: str = ""
status: str = "" # "approved", "rejected", "pending_approval", "error"
reason: str = "" # why it was approved/rejected
order_id: str | None = None
shares: int | None = None
risk_dollars: float | None = None
risk_checks_applied: list[str] = field(default_factory=list)
# ============================================================================
# BROKER MODULE CONTRACTS
# ============================================================================
@dataclass
class BrokerRequest:
"""Input contract for broker module."""
symbol: str
order_id: str
entry_price: float
stop_price: float
target_price: float
shares: int
order_type: str = "limit" # "limit" or "market"
@dataclass
class BrokerResult(ModuleResult):
"""Output contract for broker module."""
symbol: str = ""
status: str = "" # "submitted", "rejected", "error"
broker_order_id: str | None = None
submitted_at: datetime | None = None
-274
View File
@@ -1,274 +0,0 @@
"""Telemetry collection and event emission for module observability."""
from __future__ import annotations
import time
import logging
from dataclasses import dataclass, field
from datetime import datetime
from typing import TYPE_CHECKING, Any, Callable
if TYPE_CHECKING:
from core.contracts import ModuleMetrics
logger = logging.getLogger(__name__)
@dataclass
class TelemetryCollector:
"""Collect metrics during module execution."""
module_name: str
start_time: float = field(default_factory=time.time)
input_count: int = 0
output_count: int = 0
error_count: int = 0
warnings: list[str] = field(default_factory=list)
custom_metrics: dict[str, Any] = field(default_factory=dict)
def record_input(self, count: int = 1) -> None:
"""Record input items processed."""
self.input_count += count
def record_output(self, count: int = 1) -> None:
"""Record output items produced."""
self.output_count += count
def record_error(self) -> None:
"""Record an error occurred."""
self.error_count += 1
def record_warning(self, message: str) -> None:
"""Record a warning."""
self.warnings.append(message)
def record_metric(self, name: str, value: float | int | str) -> None:
"""Record a custom metric."""
self.custom_metrics[name] = value
def finalize(self) -> ModuleMetrics:
"""Finalize and return metrics."""
duration_ms = (time.time() - self.start_time) * 1000
return ModuleMetrics(
module_name=self.module_name,
timestamp=datetime.now(),
duration_ms=duration_ms,
input_count=self.input_count,
output_count=self.output_count,
error_count=self.error_count,
warnings=self.warnings,
custom_metrics=self.custom_metrics,
)
class TelemetryEmitter:
"""Emit telemetry events to event store and observability systems."""
def __init__(self, event_store=None):
"""Initialize emitter.
Args:
event_store: Optional EventStore for persisting telemetry events
"""
self.event_store = event_store
self.callbacks: list[Callable[[dict], None]] = []
def add_callback(self, callback: Callable[[dict], None]) -> None:
"""Add a callback to receive telemetry events."""
self.callbacks.append(callback)
def emit_module_tick(
self,
module_name: str,
stage: str,
metrics: ModuleMetrics | None = None,
errors: list[dict] | None = None,
correlation_id: str | None = None,
) -> None:
"""Emit a module tick event.
Args:
module_name: Name of module (e.g., "ingestion", "evaluation")
stage: Stage name (e.g., "started", "processing", "completed")
metrics: Optional ModuleMetrics object
errors: Optional list of error details
correlation_id: Correlation ID linking to signal_ready event
"""
event = {
"type": "module_tick",
"timestamp": datetime.now().isoformat(),
"module": module_name,
"stage": stage,
"metrics": metrics.to_dict() if metrics else None,
"errors": errors or [],
"correlation_id": correlation_id,
}
self._emit(event)
def emit_data_validation(
self,
module_name: str,
validation_type: str,
valid: bool,
details: dict,
correlation_id: str | None = None,
) -> None:
"""Emit a data validation event.
Args:
module_name: Name of module performing validation
validation_type: Type of validation ("schema", "checksum", "business_rules")
valid: Whether validation passed
details: Validation details (errors, warnings, statistics)
correlation_id: Optional correlation ID
"""
event = {
"type": "data_validation",
"timestamp": datetime.now().isoformat(),
"module": module_name,
"validation_type": validation_type,
"valid": valid,
"details": details,
"correlation_id": correlation_id,
}
self._emit(event)
def emit_symbol_evaluation_detail(
self,
symbol: str,
stage: str,
metrics: dict[str, Any],
passed: bool,
blocking_reason: str | None = None,
correlation_id: str | None = None,
) -> None:
"""Emit detailed evaluation event for a symbol.
Args:
symbol: Symbol being evaluated
stage: Evaluation stage (e.g., "bars_loaded", "gap_checked", "rvol_calculated")
metrics: Metrics at this stage
passed: Whether this stage passed
blocking_reason: If not passed, why
correlation_id: Optional correlation ID
"""
event = {
"type": "symbol_evaluation_detail",
"timestamp": datetime.now().isoformat(),
"symbol": symbol,
"stage": stage,
"metrics": metrics,
"passed": passed,
"blocking_reason": blocking_reason,
"correlation_id": correlation_id,
}
self._emit(event)
def _emit(self, event: dict) -> None:
"""Emit event to all registered callbacks and event store."""
logger.debug(f"Telemetry: {event['type']} - {event.get('module', event.get('symbol', 'n/a'))}")
# Call all registered callbacks
for callback in self.callbacks:
try:
callback(event)
except Exception as e:
logger.error(f"Telemetry callback failed: {e}")
# Emit to event store if available
if self.event_store:
try:
# Import here to avoid circular dependency
from storage.event_schema import ModuleTickEvent, DataValidationEvent
if event["type"] == "module_tick":
self.event_store.emit(
ModuleTickEvent(
timestamp=datetime.now(),
module=event["module"],
stage=event["stage"],
duration_ms=event["metrics"]["duration_ms"]
if event["metrics"]
else 0,
input_count=event["metrics"]["input_count"]
if event["metrics"]
else 0,
output_count=event["metrics"]["output_count"]
if event["metrics"]
else 0,
metrics=event["metrics"]["custom_metrics"]
if event["metrics"]
else {},
errors=event["errors"],
correlation_id=event.get("correlation_id"),
)
)
elif event["type"] == "data_validation":
self.event_store.emit(
DataValidationEvent(
timestamp=datetime.now(),
module=event["module"],
validation_type=event["validation_type"],
valid=event["valid"],
details=event["details"],
correlation_id=event.get("correlation_id"),
)
)
except Exception as e:
logger.error(f"Failed to emit telemetry event to store: {e}")
# Global instance (can be overridden in tests)
_telemetry_emitter: TelemetryEmitter | None = None
def set_telemetry_emitter(emitter: TelemetryEmitter) -> None:
"""Set the global telemetry emitter."""
global _telemetry_emitter
_telemetry_emitter = emitter
def get_telemetry_emitter() -> TelemetryEmitter:
"""Get the global telemetry emitter."""
global _telemetry_emitter
if _telemetry_emitter is None:
_telemetry_emitter = TelemetryEmitter()
return _telemetry_emitter
def emit_module_tick(
module_name: str,
stage: str,
metrics: ModuleMetrics | None = None,
errors: list[dict] | None = None,
correlation_id: str | None = None,
) -> None:
"""Convenience function to emit module tick."""
get_telemetry_emitter().emit_module_tick(module_name, stage, metrics, errors, correlation_id)
def emit_data_validation(
module_name: str,
validation_type: str,
valid: bool,
details: dict,
correlation_id: str | None = None,
) -> None:
"""Convenience function to emit data validation."""
get_telemetry_emitter().emit_data_validation(
module_name, validation_type, valid, details, correlation_id
)
def emit_symbol_evaluation_detail(
symbol: str,
stage: str,
metrics: dict[str, Any],
passed: bool,
blocking_reason: str | None = None,
correlation_id: str | None = None,
) -> None:
"""Convenience function to emit symbol evaluation detail."""
get_telemetry_emitter().emit_symbol_evaluation_detail(
symbol, stage, metrics, passed, blocking_reason, correlation_id
)
-284
View File
@@ -1,284 +0,0 @@
"""Data validation utilities for module boundaries."""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Callable
if TYPE_CHECKING:
from core.contracts import ValidationFailure
@dataclass
class SchemaValidator:
"""Validate data against a schema."""
required_fields: dict[str, type] # {field_name: expected_type}
optional_fields: dict[str, type] | None = None
def __post_init__(self):
if self.optional_fields is None:
self.optional_fields = {}
def validate(self, data: dict) -> tuple[bool, list[dict]]:
"""Validate data against schema.
Args:
data: Dictionary to validate
Returns:
(is_valid, list of failure dicts)
"""
failures = []
# Check required fields
for field_name, expected_type in self.required_fields.items():
if field_name not in data:
failures.append(
{
"field": field_name,
"value": None,
"reason": "Required field missing",
"severity": "error",
}
)
elif not isinstance(data[field_name], expected_type):
failures.append(
{
"field": field_name,
"value": data[field_name],
"reason": f"Expected {expected_type.__name__}, got {type(data[field_name]).__name__}",
"severity": "error",
}
)
# Check optional fields if present
if self.optional_fields:
for field_name, expected_type in self.optional_fields.items():
if field_name in data and not isinstance(data[field_name], expected_type):
failures.append(
{
"field": field_name,
"value": data[field_name],
"reason": f"Expected {expected_type.__name__}, got {type(data[field_name]).__name__}",
"severity": "warning",
}
)
return len([f for f in failures if f.severity == "error"]) == 0, failures
class RangeValidator:
"""Validate numeric values are within expected ranges."""
def __init__(self, min_val: float | None = None, max_val: float | None = None):
self.min_val = min_val
self.max_val = max_val
def validate(self, value: float, field_name: str = "value") -> tuple[bool, dict | None]:
"""Validate value is in range.
Args:
value: Numeric value to validate
field_name: Name of field for error message
Returns:
(is_valid, failure dict or None)
"""
if self.min_val is not None and value < self.min_val:
return False, {
"field": field_name,
"value": value,
"reason": f"Value {value} below minimum {self.min_val}",
"severity": "error",
}
if self.max_val is not None and value > self.max_val:
return False, {
"field": field_name,
"value": value,
"reason": f"Value {value} above maximum {self.max_val}",
"severity": "error",
}
return True, None
class ChecksumValidator:
"""Validate data integrity using checksums."""
@staticmethod
def compute_checksum(data: dict | list) -> str:
"""Compute simple checksum for data.
Args:
data: Dictionary or list to checksum
Returns:
Hex checksum string
"""
import hashlib
import json
data_str = json.dumps(data, sort_keys=True, default=str)
return hashlib.md5(data_str.encode()).hexdigest()
@staticmethod
def validate_checksum(
data: dict | list, expected_checksum: str
) -> tuple[bool, dict | None]:
"""Validate data matches checksum.
Args:
data: Data to validate
expected_checksum: Expected checksum value
Returns:
(is_valid, failure dict or None)
"""
actual_checksum = ChecksumValidator.compute_checksum(data)
if actual_checksum != expected_checksum:
return False, {
"field": "checksum",
"value": actual_checksum,
"reason": f"Checksum mismatch: expected {expected_checksum}, got {actual_checksum}",
"severity": "error",
}
return True, None
class BusinessRuleValidator:
"""Validate data against business logic rules."""
def __init__(self):
self.rules: dict[str, Callable[[Any], tuple[bool, str]]] = {}
def add_rule(self, rule_name: str, rule_fn: Callable[[Any], tuple[bool, str]]) -> None:
"""Add a business rule.
Args:
rule_name: Name of the rule
rule_fn: Function that takes data and returns (is_valid, reason)
"""
self.rules[rule_name] = rule_fn
def validate(self, data: Any) -> tuple[bool, list[dict]]:
"""Apply all business rules.
Args:
data: Data to validate
Returns:
(is_valid, list of failure dicts)
"""
failures = []
for rule_name, rule_fn in self.rules.items():
try:
is_valid, reason = rule_fn(data)
if not is_valid:
failures.append(
{
"field": "business_rule",
"value": None,
"reason": f"{rule_name}: {reason}",
"severity": "error",
}
)
except Exception as e:
failures.append(
{
"field": "business_rule",
"value": None,
"reason": f"{rule_name} raised exception: {e}",
"severity": "error",
}
)
return len(failures) == 0, failures
# Concrete validators for common cases
class BarDataValidator:
"""Validate OHLCV bar data."""
@staticmethod
def validate_bar(bar: dict) -> tuple[bool, list[dict]]:
"""Validate a single bar.
Args:
bar: Bar dictionary with {timestamp, open, high, low, close, volume}
Returns:
(is_valid, list of failure dicts)
"""
failures = []
# Check required fields
required = {"timestamp", "open", "high", "low", "close", "volume"}
for field in required:
if field not in bar:
failures.append(
{
"field": field,
"value": None,
"reason": "Missing required field",
"severity": "error",
}
)
if failures:
return False, failures
# Check value consistency
open_val = bar["open"]
high_val = bar["high"]
low_val = bar["low"]
close_val = bar["close"]
volume_val = bar["volume"]
# High must be >= all others
if high_val < open_val or high_val < close_val or high_val < low_val:
failures.append(
{
"field": "high",
"value": high_val,
"reason": f"High ({high_val}) must be >= open ({open_val}), close ({close_val}), low ({low_val})",
"severity": "error",
}
)
# Low must be <= all others
if low_val > open_val or low_val > close_val or low_val > high_val:
failures.append(
{
"field": "low",
"value": low_val,
"reason": f"Low ({low_val}) must be <= open ({open_val}), close ({close_val}), high ({high_val})",
"severity": "error",
}
)
# Volume should be non-negative
if volume_val < 0:
failures.append(
{
"field": "volume",
"value": volume_val,
"reason": "Volume cannot be negative",
"severity": "error",
}
)
# Warn on zero volume
if volume_val == 0:
failures.append(
{
"field": "volume",
"value": volume_val,
"reason": "Zero volume bar",
"severity": "warning",
}
)
return len([f for f in failures if f.severity == "error"]) == 0, failures
+17 -2
View File
@@ -19,6 +19,8 @@ from dotenv import load_dotenv
# 192.168.1.5:5432) — the event store reads it. Without this the standalone
# dashboard fell back to the legacy local DuckDB path and showed nothing.
load_dotenv()
from runtime.env_overrides import load as _load_overrides
_load_overrides() # admin-panel overrides beat compose env
from api.main import DashboardState, create_server # noqa: E402
@@ -30,8 +32,21 @@ def main() -> None:
# WATCHER_EVENT_DB_PATH is a legacy fallback only (the path is ignored).
db_path = os.environ.get("DATABASE_URL") or os.environ.get(
"WATCHER_EVENT_DB_PATH", "momentum")
mode = os.environ.get("TRADING_EXECUTION_MODE", "alpaca_paper")
state = DashboardState(db_path, execution_service=None, execution_mode=mode)
from trading_mode import TradingModeSettings
mode = TradingModeSettings.from_env().execution_mode
# Sparklines: the standalone dashboard needs the SAME research-DB reader the
# embedded (orchestrator) mode passes — without it DashboardState.bars() short-
# circuits to []. The missing wire was purely that this entrypoint never built
# one (DATABASE_URL already reaches the market schema). Read-only use; degrade
# to None (charts show "no research db") if the connection can't be opened.
research_con = None
try:
from research.multi_schema import open_research_db
research_con = open_research_db("market")
except Exception as exc: # noqa: BLE001
print(f"[dashboard] research DB unavailable, sparklines disabled: {exc}")
state = DashboardState(db_path, execution_service=None, execution_mode=mode,
research_con=research_con)
server = create_server(state, host, port)
db_disp = db_path.split("@")[-1] if "@" in db_path else db_path
print(f"[dashboard] serving http://{host}:{port} (event db: {db_disp}, mode: {mode})")
+48
View File
@@ -0,0 +1,48 @@
# Deploying Momentum
Three supported shapes, one image/codebase. Pick by where it runs:
| Mode | Status | Use when | Docs |
|---|---|---|---|
| **Docker Compose** | ✅ **canonical** (production since 2026-07-09) | Any always-on Linux host; TrueNAS SCALE | [docker/README.md](docker/README.md), `docker-compose.yml` (root), [docker/truenas-compose.yml](docker/truenas-compose.yml) |
| **Local (systemd)** | ✅ maintained | Development machine, WSL, bare metal without Docker | [systemd/README.md](systemd/README.md) + unit templates in `deploy/systemd/` |
| **Helm / Kubernetes** | 🚧 planned refresh | Future k8s hosting | [helm/](helm/) — the chart predates the 2026-07 architecture; starting point, not deployable |
## Docker Compose (canonical)
The root `docker-compose.yml` runs the full stack self-contained: postgres 18,
trading loop, dashboard (:8765), admin panel (:8767), telegram bot, transcripts
pipeline, nightly scheduler, on-demand jobs, optional Grafana. The image is
published to `docker.io/fez5587/momentum-app` and `ghcr.io/fez5587/momentum-app`
by CI on every master push.
For TrueNAS SCALE use `docker/truenas-compose.yml` (pull-only, dataset host
paths, inline env). **Read the gotchas in docker/README.md before editing the
YAML** — anchor expansion, per-service env, and path case-sensitivity have each
caused an outage.
## Local (systemd user services)
The original deployment: unit templates in `deploy/systemd/` running a git
checkout + venv, config from `.env`. Still the best loop for development —
tests, backtests, and research scripts run here. Keep the units **disabled**
while the compose stack owns production: two loops against one Alpaca account
create duplicate orders.
## Helm (future)
`deploy/helm/` holds the original chart (single app deployment + CronJobs). It
predates the service split, the session gate, and the env-flag inventory; the
refresh is tracked in GitHub issues. Until then, compose is the way.
## Config in all modes
- `.env.example` (committed, sanitized) is the full variable inventory and
says which layer each belongs in.
- Behavior flags that MUST be set explicitly in any deployment — silent
defaults bite: `TRADING_AUTO_APPROVE=1` (else entries queue for manual
approval), `ALPACA_DATA_FEED=sip`, `OLLAMA_ENABLED=true` (catalyst engine),
`FMP_API_KEY` (float cache). `TRADING_SESSION_ONLY` defaults ON in
deployed configs (bars after-hours entries).
- Runtime overrides: the admin panel writes `data/overrides.env`, which every
service loads over its baked env at boot (`runtime/env_overrides.py`).
+77
View File
@@ -0,0 +1,77 @@
"""Nightly off-box backup of the LIVE (Tradier) Postgres, from the owner's workstation.
The live stack on TrueNAS has no db-backup service; the paper stack backs up only its own database. This keeps the
live test's full record (events, trades, settings) somewhere other than the TrueNAS pool.
- pg_dump -Fc (compressed, restorable with pg_restore) of the live database (DATABASE_URL host, port 5434)
- verified: pg_restore --list must read it and it must contain the events and config_overrides tables
- keeps the newest KEEP dumps (default 7) in LIVE_BACKUP_DIR (default C:\\Users\\Admin\\momentum-backups)
- writes <dir>/status.json for research.daily_report to alert on a missed or failed backup
usage: python deploy/backup_live_db.py (Task Scheduler "Momentum\\LiveDbBackup", nightly)
restore: pg_restore -d <empty database> --no-owner <dump>
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import time
from datetime import datetime
from pathlib import Path
from urllib.parse import urlparse
from zoneinfo import ZoneInfo
# pg_dump/pg_restore are console programs: launched from pythonw they would pop a console window,
# and closing that window kills the dump (exit 0xC000013A, seen 2026-10-06). Run them windowless.
NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0)
ROOT = Path(__file__).resolve().parent.parent
PG_BIN = Path(os.environ.get("PG_BIN", r"C:\Users\Admin\pgsql18\bin"))
OUT = Path(os.environ.get("LIVE_BACKUP_DIR", r"C:\Users\Admin\momentum-backups"))
KEEP = int(os.environ.get("LIVE_BACKUP_KEEP", "7"))
PORT = int(os.environ.get("LIVE_DB_PORT", "5434"))
ET = ZoneInfo("America/New_York")
MUST_HAVE = ("TABLE DATA public events", "TABLE DATA public config_overrides")
def main() -> int:
from dotenv import dotenv_values
u = urlparse(dotenv_values(ROOT / ".env")["DATABASE_URL"])
OUT.mkdir(parents=True, exist_ok=True)
stamp = datetime.now(ET).strftime("%Y-%m-%d_%H%M")
dump = OUT / f"live_{stamp}.dump"
env = dict(os.environ, PGPASSWORD=u.password or "")
status = dict(at=datetime.now(ET).isoformat(timespec="seconds"), file=str(dump), ok=False)
t0 = time.time()
try:
subprocess.run([str(PG_BIN / "pg_dump.exe"), "-h", u.hostname, "-p", str(PORT), "-U", u.username,
"-d", u.path[1:], "-Fc", "-Z", "6", "-f", str(dump)],
env=env, check=True, capture_output=True, text=True, timeout=3600,
creationflags=NO_WINDOW)
listing = subprocess.run([str(PG_BIN / "pg_restore.exe"), "--list", str(dump)],
check=True, capture_output=True, text=True, timeout=600,
creationflags=NO_WINDOW).stdout
missing = [m for m in MUST_HAVE if m not in listing]
if missing:
raise RuntimeError(f"dump is missing {missing}")
status.update(ok=True, size_mb=round(dump.stat().st_size / 1e6, 1), seconds=round(time.time() - t0),
entries=listing.count("\n"))
except subprocess.CalledProcessError as e:
status["error"] = (e.stderr or str(e))[-400:]
except Exception as e: # noqa: BLE001
status["error"] = str(e)[-400:]
if not status["ok"] and dump.exists():
dump.unlink() # never keep a broken dump around as if it were good
dumps = sorted(OUT.glob("live_*.dump"))
for old in dumps[:-KEEP]:
old.unlink()
status["kept"] = [p.name for p in sorted(OUT.glob("live_*.dump"))]
(OUT / "status.json").write_text(json.dumps(status, indent=1), encoding="utf-8")
print(json.dumps(status, indent=1))
return 0 if status["ok"] else 1
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,19 @@
# Grafana datasource for the docker-compose stack — points at the in-network
# `postgres` service (not the LAN IP used by grafana/provisioning/datasources).
# Credentials are interpolated from the PG_* env vars set on the grafana service.
apiVersion: 1
datasources:
- name: Momentum
uid: momentum_pg
type: postgres
access: proxy
url: postgres:5432
user: ${PG_USER}
jsonData:
database: ${PG_DATABASE}
sslmode: disable
postgresVersion: 1600
secureJsonData:
password: ${PG_PASSWORD}
isDefault: true
editable: true
+117
View File
@@ -0,0 +1,117 @@
# Running the Momentum stack in Docker Compose
One image, six services: `postgres`, `loop`, `dashboard` (:8765), `qabot`
(Telegram), `transcripts`, `scheduler` (20:00 ET weekday housekeeping), plus
on-demand `jobs` and an optional `monitoring` (Grafana) profile.
```bash
docker compose build
docker compose up -d
docker compose ps # healthchecks: loop heartbeat, dashboard HTTP, pg_isready
docker compose logs -f loop
```
Secrets live in `.env` (gitignored, never in the image). The compose file
overrides `DATABASE_URL` to the in-stack postgres so the stack is
self-contained.
## Migrating the production DB into the stack
The live DB today is native Postgres 18 on 192.168.1.5. To move it into the
compose volume:
```bash
# 1. dump the live DB (~4 GB as of 2026-07; events retention keeps it bounded)
pg_dump -h 192.168.1.5 -U admin -Fc momentum > momentum.dump
# 2. start ONLY postgres, restore into it
docker compose up -d postgres
docker compose exec -T postgres pg_restore -U momentum -d momentum --no-owner < momentum.dump
# 3. bring up the rest
docker compose up -d
```
Until then you can run the app services against the LAN DB (transition mode):
comment out the `DATABASE_URL` line in the `x-app-env` anchor and `.env`'s
value (192.168.1.5) flows through.
## Where to host it
Recommended: **the DB box (192.168.1.5) or any always-on Linux host** — NOT
the WSL machine. The week of 2026-07-06 produced three trading-hours outages
from WSL restarts and networking-mode changes (see memory:
wsl-virtioproxy-networking). A dedicated Linux host removes that whole class.
WSL + Docker works for development/validation but inherits WSL's fragility.
Cutover checklist (from the systemd stack):
1. `docker compose build && docker compose up -d` on the target host
2. Verify: dashboard on :8765, `docker compose ps` all healthy, loop log shows
ingest/watch/execute cycles, Telegram bot answers
3. Diff Alpaca open orders — a crash-restart on cutover can resubmit resting
entries (memory: momentum-live-loop-architecture)
4. Disable the WSL systemd units: `systemctl --user disable --now
momentum-loop momentum-dashboard momentum-qabot momentum-transcripts
momentum-premarket-eval.timer`
5. Keep `momentum-testpg` on the dev machine (tests only, not production)
Notes:
- TZ inside containers is America/New_York (market-clock logic is ET).
- `data/` (logs, learned_params.json, transcripts) lives in the `app-data`
named volume, shared by all app services.
- Ollama stays external (192.168.1.9, `OLLAMA_*` in .env); enrichment
fails safe when it's down.
- Do the cutover OUTSIDE market hours.
## Config layering: what goes where
`.env.example` is the committed, sanitized inventory of every variable
(regenerated 2026-07-09). Three layers:
1. **Secrets — keep in `.env` in a protected space** (root-owned `chmod 600`,
or a Docker secret): `ALPACA_API_KEY`/`ALPACA_SECRET_KEY`,
`TELEGRAM_BOT_TOKEN`, `YOUTUBE_API_KEY`, `POSTGRES_PASSWORD`/`DATABASE_URL`,
Discord webhooks, Schwab keys. The compose file reads them via `env_file`;
they are never baked into the image.
2. **Strategy knobs — fine to promote into the compose `environment:` block**
if you want them versioned with the deployment: `TRADING_*`, `TRIGGER_*`
(gap band, squeeze exception, quality floor, caps), `WATCHER_*`, `LIVE_*`.
`environment:` overrides `env_file`, so promoted values win.
3. **Infra addresses**: `OLLAMA_HOST` (external box, see below),
`ALPACA_DATA_FEED=sip`, dashboard host/port — either layer works.
## External Ollama (news enrichment)
Ollama is the ONE external dependency: `OLLAMA_HOST=http://192.168.1.9:11434`
reaches the LAN box from inside containers via normal outbound NAT — no
compose changes needed. If you ever run Ollama on the docker HOST itself, use
`http://host.docker.internal:11434` and add
`extra_hosts: ["host.docker.internal:host-gateway"]` to the app anchor.
Enrichment is fully fail-safe: if Ollama is down, trading is unaffected
(news ranking just loses the catalyst-type boost).
NOTE (2026-07-09): the live `.env` contains two commented-out PowerShell lines
embedding the real Alpaca credentials — harmless there, but they nearly leaked
into the committed `.env.example` (caught by the regeneration leak check).
Consider deleting them from `.env`.
## TrueNAS gotchas (each one caused a real outage — read before editing)
1. **Anchors get expanded.** TrueNAS renders `x-app:`/`<<: *app` into inline
per-service blocks. After the first save, the `x-app` block is decorative —
env/volume edits there reach nothing. Edit each service's own block.
2. **Verify env landed** with `docker exec <svc> sh -c 'echo $VAR'` — not by
reading the YAML, and not via the admin panel (it reports its own container).
3. **Paths are case-sensitive** and Docker auto-creates missing bind sources:
`/mnt/Volume` vs `/mnt/volume` silently forked the database into two worlds.
After any data move, verify by **size at destination** (`du -sh`), never `ls`.
4. **Volume changes need container recreation** — Stop/Start or Update the app;
a container restart keeps the old mounts (and a running postgres keeps
writing to *deleted* files if you move its dir underneath it).
5. **First deploy initializes postgres** — `POSTGRES_PASSWORD` only applies to
an empty pgdata. A failed first attempt leaves a cluster with the old
password; fix with `ALTER USER` via `docker exec ... psql -U momentum` or
wipe before the real deploy.
6. **uid 10001**: the app containers run non-root; `chown -R 10001:10001` the
host data dir or every file-writer (scheduler TSVs, admin profiles,
transcripts) hits `PermissionError`.
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Nightly logical backup of the momentum Postgres. Runs inside the db-backup
# compose service — the postgres:18 image, so pg_dump always matches the
# server major (the app image has NO pg_dump, and a Debian-stock client is an
# older major that refuses to dump a newer server).
#
# pg_dump | gzip -> $BACKUP_DIR/momentum-YYYY-MM-DD.sql.gz
# prune anything older than BACKUP_KEEP_DAYS (default 14)
# write $BACKUP_DIR/last_backup_ok on success (freshness marker)
#
# Restore: gunzip -c momentum-<date>.sql.gz | psql "$DATABASE_URL"
#
# Run with bash (not sh): pipefail is what catches a mid-stream pg_dump death
# that would otherwise leave a truncated-but-valid gzip behind.
set -euo pipefail
BACKUP_DIR="${BACKUP_DIR:-/backups}"
BACKUP_KEEP_DAYS="${BACKUP_KEEP_DAYS:-14}"
: "${DATABASE_URL:?DATABASE_URL is required}"
mkdir -p "$BACKUP_DIR"
out="$BACKUP_DIR/momentum-$(date +%F).sql.gz"
echo "[backup] $(date) -> $out"
pg_dump "$DATABASE_URL" | gzip > "$out.tmp"
gunzip -t "$out.tmp"
mv "$out.tmp" "$out"
find "$BACKUP_DIR" -name 'momentum-*.sql.gz*' -mtime +"$BACKUP_KEEP_DAYS" -delete # *: also sweep stale .tmp from failed runs
date > "$BACKUP_DIR/last_backup_ok"
echo "[backup] OK $(du -h "$out" | cut -f1)"
+41
View File
@@ -0,0 +1,41 @@
#!/bin/bash
# Seed the LIVE stack's fresh Postgres from the paper stack's nightly dump,
# then wipe the money-record tables so live P&L starts at zero while every
# research baseline (minute/daily bars, rvol + regime history, learned
# setups) carries over. Run ON CUBE after the live stack's postgres is up.
#
# ./bootstrap_live_db.sh <backup.sql.gz> <live-postgres-container>
# e.g. ./bootstrap_live_db.sh \
# /mnt/Volume/Storage/Apps/momentum/data/backups/momentum-2026-07-28.sql.gz \
# ix-momentum-live-postgres-1
#
# Idempotent-ish: restoring over a non-empty DB will error on existing objects;
# for a re-run, recreate the live stack's pgdata volume first (it is DISPOSABLE
# until the flip — that's the point of staging).
set -euo pipefail
BACKUP="${1:?usage: bootstrap_live_db.sh <backup.sql.gz> <live-pg-container>}"
CONTAINER="${2:?usage: bootstrap_live_db.sh <backup.sql.gz> <live-pg-container>}"
PSQL=(docker exec -i "$CONTAINER" psql -U momentum -d momentum -v ON_ERROR_STOP=0 -q)
echo "[1/3] restoring $BACKUP into $CONTAINER ..."
gunzip -c "$BACKUP" | "${PSQL[@]}" >/dev/null
echo "[2/3] zeroing money-record tables (all trades/P&L derive from events) ..."
docker exec -i "$CONTAINER" psql -U momentum -d momentum -v ON_ERROR_STOP=1 <<'SQL'
TRUNCATE events, telemetry_events;
-- paper-only artifacts that must not leak into the live stack's views
TRUNCATE simulated_orders, simulated_fills, paper_missed_trades, paper_sessions;
SQL
echo "[3/3] sanity ..."
docker exec -i "$CONTAINER" psql -U momentum -d momentum -t <<'SQL'
SELECT 'minute_bars ' || count(*) || ' (want millions)' FROM minute_bars
UNION ALL SELECT 'daily_bars ' || count(*) FROM daily_bars
UNION ALL SELECT 'setup_events ' || count(*) FROM setup_events
UNION ALL SELECT 'regime_scores ' || count(*) FROM regime_scores
UNION ALL SELECT 'events (MUST be 0) ' || count(*) FROM events;
SQL
echo "done. Copy the paper stack's data/learned_params.json into the live"
echo "stack's data volume, then start the loop."
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
# Migrate the momentum Postgres from the existing instance to the TrueNAS
# stack's new one, with verification. Safe to practice: the SOURCE is only
# ever read (pg_dump), never written.
#
# ./deploy/docker/migrate_db.sh dump # 1. dump source -> momentum.dump
# ./deploy/docker/migrate_db.sh restore # 2. restore dump -> target
# ./deploy/docker/migrate_db.sh verify # 3. row-count diff source vs target
#
# Override via env: SRC_URL, DST_URL, DUMP_FILE, PG_BIN.
# Requires postgres 18 client tools (server is 18.4) — defaults to the local
# PG18 install at /home/philip/pg/bin when present.
set -euo pipefail
PG_BIN="${PG_BIN:-$([ -x /home/philip/pg/bin/pg_dump ] && echo /home/philip/pg/bin || echo "")}"
DUMP_FILE="${DUMP_FILE:-./momentum.dump}"
SRC_URL="${SRC_URL:-postgresql://admin:CHANGE_ME@192.168.1.5:5432/momentum}"
# target = the TrueNAS compose postgres, published on 5433 (see truenas-compose.yml)
DST_URL="${DST_URL:-postgresql://momentum:CHANGE_ME@192.168.1.5:5433/momentum}"
pgdump="${PG_BIN:+$PG_BIN/}pg_dump"
pgrestore="${PG_BIN:+$PG_BIN/}pg_restore"
psql="${PG_BIN:+$PG_BIN/}psql"
# The big tables that matter: events (trading history), minute_bars/daily_bars
# (the year of backtest truth), plus every small state table. Nothing excluded
# by default; uncomment to skip pure observability bulk and shrink the dump:
EXCLUDES=()
# EXCLUDES+=(--exclude-table-data=scanner_snapshots --exclude-table-data=raw_fetch_attempts)
case "${1:-}" in
dump)
echo ">> dumping $SRC_URL -> $DUMP_FILE (custom format, compressed)"
time "$pgdump" -Fc --no-owner --no-privileges "${EXCLUDES[@]}" \
-d "$SRC_URL" -f "$DUMP_FILE"
ls -lh "$DUMP_FILE"
;;
restore)
echo ">> restoring $DUMP_FILE -> $DST_URL (4 parallel jobs)"
time "$pgrestore" --no-owner --no-privileges --clean --if-exists -j 4 \
-d "$DST_URL" "$DUMP_FILE"
echo ">> ANALYZE for fresh planner stats"
"$psql" -d "$DST_URL" -c "ANALYZE;"
;;
verify)
echo ">> row counts, source vs target (should match):"
q="SELECT relname, n_live_tup FROM pg_stat_user_tables
WHERE schemaname='public' ORDER BY n_live_tup DESC LIMIT 12"
paste <("$psql" -d "$SRC_URL" -At -F' ' -c "$q") \
<("$psql" -d "$DST_URL" -At -F' ' -c "$q") \
| awk '{printf " %-22s src=%-10s | %-22s dst=%s\n", $1, $2, $3, $4}'
echo ">> newest event on each side:"
for u in "$SRC_URL" "$DST_URL"; do
"$psql" -d "$u" -At -c "SELECT max(timestamp) FROM events" || true
done
;;
*)
echo "usage: $0 dump|restore|verify"; exit 1 ;;
esac
+160
View File
@@ -0,0 +1,160 @@
# Seed for /app/data/overrides.env on the LIVE stack — the admin panel
# (port 8867.. n/a on live; edit via file or add admin service) manages this.
# Every service loads it at boot; overrides BEAT compose env.
ALPACA_DATA_FEED=sip
DISCOVER_INTERVAL_SECONDS=120
DISCOVER_THREAD_ENABLED=1
LIVE_BARS_INTERVAL_SECONDS=5
LIVE_BARS_REFRESH_MINUTES=20
NEWS_CATALYST_SCORE_ENABLED=1
NEWS_DILUTION_VETO_ENABLED=1
NEWS_ENRICH_BATCH_LIMIT=50
NEWS_ENRICH_ENABLED=1
NEWS_ENRICH_INTERVAL_SECONDS=120
NEWS_ENRICH_LOOKBACK_HOURS=12
NEWS_FEEDS=prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS=8
# BLIND HEADLINE BOOST DISABLED (1.0 = neutral), 2026-09-14. Measured three ways
# over 2026-08-01+: premarket news 7.9% vs 8.6% ran >=50%; controlling for premarket
# range 11.3% vs 21.0%; and an hour-matched event study of intraday arrivals, 16.8%
# vs 24.6% ran >=10% forward. News arrival predicts LOWER forward returns, every time.
# The cause is the feed: of 1,621 headlines it delivered for screened names during
# sessions, 61.7% were reactive roundups ("Why X Shares Are Trading Higher") and 0.9%
# were genuine catalyst PRs. It reports the move after it happens, so boosting on it
# buys the late stage. The LLM path is UNAFFECTED and stays on: catalyst_score is
# conviction * max(0, sentiment), so a roundup scores 0 and earns no boost, while the
# dilution veto still sinks confirmed offerings. Only the blind "has a headline" 2x
# is removed. Restore by setting this to 2.0.
NEWS_RANK_BOOST=1.0
# Ollama lives on 11434 (the standard port). The stack pointed at 30068/4000 with
# a model that was never pulled, so catalyst enrichment produced nothing from
# 2026-08-03 to 2026-09-13 — silently, because enrich_recent_news reports failure
# in its return value rather than raising. Smoke-tested 2026-09-13: tools:latest
# parses 8/8 headlines in ~2.8s each and classifies offering_dilution / fda_approval
# correctly. Verify the model is pulled before changing either line.
OLLAMA_ENABLED=true
OLLAMA_HOST=http://192.168.1.5:11434
OLLAMA_MODEL=tools:latest
OLLAMA_TEMPERATURE=0.1
OLLAMA_TIMEOUT_SECONDS=60
TRADING_AUTO_APPROVE=1
TRADING_BACKOUT_COOLDOWN_SECONDS=180
TRADING_BLOCK_REENTRY_AFTER_EXIT=1
TRADING_ENTRY_CONFIRM_BY_MINUTE=0
TRADING_ENTRY_FILL_MODEL=marketable
TRADING_ENTRY_INVALIDATE_PCT=0.015
TRADING_ENTRY_ORDER_TYPE=limit
TRADING_ENTRY_TIMEOUT_BARS=2
TRADING_EOD_FLATTEN_ENABLED=1
TRADING_EOD_FLATTEN_TIME=15:55
# CIRCUIT BREAKER (2026-09-16). The exit retune of 2026-09-13 is running $7.50
# behind the config it replaced over its first 8 live trades. That is the shape the
# change predicted -- bleed small, wait for a tail -- but it is also what a wrong
# call looks like at n=8, and the two are not separable yet. Rather than leave the
# decision to a judgement call on a bad afternoon, halt automatically: at $215
# (-11% from the 241.75 the change started at) trading stops for the session and the
# book flattens. Reads real broker equity; a 0.0 reading short-circuits earlier and
# cannot trip it spuriously.
TRADING_EQUITY_FLOOR=215
TRADING_EXECUTION_ENABLED=1
# EXIT MANAGEMENT — retuned 2026-09-13. The previous config stacked FOUR
# independent ratchets (BE@+5% from the env default, step-trail every 0.25R,
# prior-low trail, and profit-lock tiers). Together they capped every winner at
# ~1R: over 62 live + 172 paper trades the bot kept only 9.3% of the move on
# trades that ran >=20%, and held winners a median 10 min vs 19 min for losers.
# Replaced with a single percentage giveback that starts only after +1R. The
# 8/12/15/20% neighbourhood all beat the old config on BOTH stacks in BOTH
# train and test splits (mean R 0.20-0.25 vs 0.125), so 12% is a plateau pick,
# not a fitted one. Re-sweep with sweep_exits.py before changing.
TRADING_EXIT_BREAKEVEN_R=1.0
TRADING_EXIT_BREAKEVEN_PCT=0.0
TRADING_EXIT_MANAGE_INTERVAL_SECONDS=4
TRADING_EXIT_PROFIT_TIERS=
TRADING_EXIT_THREAD_ENABLED=1
TRADING_EXIT_THREAD_INTERVAL_SECONDS=1.0
TRADING_EXIT_TRAIL_AFTER_R=1.0
TRADING_EXIT_TRAIL_MODE=pct
TRADING_EXIT_TRAIL_PCT=0.12
TRADING_EXIT_TRAIL_R_STEP=0.0
TRADING_LIQUIDITY_MAX_VOLUME_PCT=0.025
TRADING_MAX_BACKOUTS_PER_SYMBOL=3
TRADING_MAX_CONCURRENT_POSITIONS=10
TRADING_MAX_DAILY_LOSS_PCT=0.05
TRADING_MAX_FRESH_ENTRIES_PER_DAY=10
TRADING_MAX_GROSS_NOTIONAL_PCT=0.60
TRADING_MAX_ORDERS_PER_TICK=1
TRADING_MAX_POSITION_PCT=0.16
# Boot-time sizing fallback. calculate_position_size falls back to this ONLY when
# equity was never readable; at the old 100000 a boot inside Tradier's ~01:06
# maintenance window (equity reads 0.0 there — 16 times in 14 days) would have
# sized 5% of 100k on a ~$250 account. Keep it BELOW real account equity so the
# fallback can only ever under-size (live equity 241.75 on 2026-09-13).
TRADING_DEFAULT_EQUITY=200
TRADING_MAX_RISK_DOLLARS=50000
TRADING_MIN_DAY_VOLUME=300000
TRADING_MIN_ENTRY_GAP_PCT=0.10
# 0.60 -> 0.20 (2026-09-16). This gates entries on setup_evaluator's quality_score,
# which is calculate_data_quality_score(bars) -- a MINUTE-BAR COVERAGE metric, not a
# setup metric ("Data quality scoring for minute-bar coverage and integrity"). Tested
# against realised outcomes over 62 trades it correlates r=0.0845, and grade C beat
# grade A. It blocked SNDQ on 2026-09-14 for having slightly gappy bars. Dropped to
# 0.20 so it only rejects genuinely broken data instead of acting as a setup filter.
TRADING_MIN_QUALITY_SCORE=0.20
# PREMARKET-RANGE RANK BOOST (2026-09-13). A name that already travelled >=15%
# premarket ran >=50% on the day 15.4% of the time vs 5.0% for the rest -- a 3.1x
# lift, the largest single predictor measured over 1,040 screened symbol-days.
# Replaying the armed-set selection, the gain is real but modest (mean run of the
# top-5 armed 39.8% -> 42.0%, 27 -> 32 big runners per 145 slots) because gap x
# rvol already encodes much of it. BOOST only: a hard filter on the same feature
# scored WORSE than baseline. Set TRADING_PM_RANGE_BOOST=0 to disable.
TRADING_PM_RANGE_BOOST=1
TRADING_PM_RANGE_MIN_PCT=15.0
TRADING_PM_RANGE_GAIN=3.0
TRADING_PRICE_MAX=20.0
TRADING_PRICE_MIN=2.0
TRADING_REENTRY_MIN_LOSS_PCT=0.01
TRADING_REGIME_GATE_ENABLED=0
TRADING_REGIME_GATE_P_MIN=0.5
TRADING_REQUIRE_ABOVE_VWAP=1
TRADING_REWARD_MULTIPLE=10.0
# RISK PER TRADE cut 0.05 -> 0.015 (2026-09-16). At 5% the risk budget was
# unreachable: 5% of ~$240 is $12, which on a 4.5% stop needs $267 of notional
# against a $38 position cap (TRADING_MAX_POSITION_PCT=0.16). So the POSITION CAP
# bound on every trade and the risk calc never did — every position was the same
# NOTIONAL and the dollar risk floated with stop width, a measured 17x spread
# ($0.52 to $8.79) across 64 live trades. Wide-stop names carried 3-8x the risk of
# narrow-stop ones; on 2026-09-15/16 the four widest-stop trades produced -$8.57 of
# an -$8.84 week.
#
# At 1.5% the risk calc binds for wide stops while the position cap still binds for
# narrow ones, compressing the spread to 6.8x. Replayed over 64 live trades:
# -$4.09 -> -$0.32, and the gain is NOT just smaller size — against a plain
# shrink to the same average notional it is worth +$3.1 to +$3.3. Below 1% trades
# start being skipped entirely (shares round to zero: 1 lost at 1.0%, 13 at 0.5%),
# so 1.5% is the tightest value that drops nothing. It scales with equity, unlike
# TRADING_MAX_RISK_DOLLARS.
TRADING_RISK_PER_TRADE_PCT=0.015
TRADING_SESSION_ONLY=1
TRADING_TRIGGER_SLIP_PCT=0.004
TRADING_UNIFIED_ENTRY=1
TRIGGER_ARM_INTERVAL_SECONDS=8
TRIGGER_ENABLED=1
TRIGGER_GAP_MAX=35
TRIGGER_GAP_MIN=10.0
TRIGGER_GAP_SQUEEZE_MIN=150
TRIGGER_GAP_SQUEEZE_RVOL=5.0
TRIGGER_INTERVAL_SECONDS=2
TRIGGER_MAX_ARMED=10
TRIGGER_MIN_DOLLAR_VOL=8000
TRIGGER_PRICE_MAX_AGE=12
TRIGGER_RVOL_MIN=2.0
# Apply the shared tradeable-population gate before the watcher scores anything.
# Without it the watcher wrote 191,458 criteria receipts across 57 symbols on
# 2026-09-10, 80.6% of them for names outside the gap band. Set 0 to restore the
# old evaluate-everything behaviour.
WATCHER_POPULATION_GATE=1
WATCHER_ENABLED=1
WATCHER_INTERVAL_SECONDS=8
WATCHER_PRICE_MAX=20
WATCHER_PRICE_MIN=2
+687
View File
@@ -0,0 +1,687 @@
# TrueNAS SCALE custom-app compose - FLATTENED (no YAML anchors).
# TrueNAS's app validator REJECTS anchors (x-app / <<: *app) with
# 'Invalid YAML provided', so every service carries its own full env.
# Anchored/DRY source of truth: deploy/docker/truenas-compose.yml.
# Fill every <FILL-ME-*> before deploying (PG password repeats in all
# DATABASE_URL blocks + postgres + db-backup). Regenerated? keep both in sync.
#
services:
postgres:
image: postgres:18
environment:
POSTGRES_USER: momentum
POSTGRES_PASSWORD: <FILL-ME-PG-PASSWORD>
POSTGRES_DB: momentum
PGDATA: /var/lib/postgresql/data/pgdata
volumes:
- /mnt/volume/Apps/momentum/pgdata:/var/lib/postgresql/data
ports:
- 5433:5432
healthcheck:
test:
- CMD-SHELL
- pg_isready -U momentum -d momentum
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
loop:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- run_live_paper.py
- --no-dashboard
environment:
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_BACKFILL_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets/v2
ALPACA_DATA_FEED: sip
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: <FILL-ME-FMP-KEY>
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '20'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: http://192.168.1.5:30068
OLLAMA_MODEL: qwen2.5:7b-instruct
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '1'
STRATEGY_SETUPS: opening_range_break
TELEGRAM_BOT_TOKEN: <FILL-ME-TELEGRAM-TOKEN>
TELEGRAM_CHAT_ID: <FILL-ME-TELEGRAM-CHAT-ID>
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: marketable
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: limit
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: 8:3,15:9,25:18,40:30
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: prior_low
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REGIME_GATE_ENABLED: '1'
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '20'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '4'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: America/New_York
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '15'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: <FILL-ME-YOUTUBE-KEY>
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
healthcheck:
test:
- CMD
- python
- -c
- import os; os.environ['PG_SKIP_SCHEMA']='1'; import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)
interval: 60s
timeout: 20s
retries: 5
start_period: 180s
dashboard:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- dashboard_api.py
environment:
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_BACKFILL_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets/v2
ALPACA_DATA_FEED: sip
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: <FILL-ME-FMP-KEY>
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '20'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: http://192.168.1.5:30068
OLLAMA_MODEL: qwen2.5:7b-instruct
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '1'
STRATEGY_SETUPS: opening_range_break
TELEGRAM_BOT_TOKEN: <FILL-ME-TELEGRAM-TOKEN>
TELEGRAM_CHAT_ID: <FILL-ME-TELEGRAM-CHAT-ID>
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: marketable
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: limit
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: 8:3,15:9,25:18,40:30
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: prior_low
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REGIME_GATE_ENABLED: '1'
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '20'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '4'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: America/New_York
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '15'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: <FILL-ME-YOUTUBE-KEY>
DASHBOARD_HOST: 0.0.0.0
DASHBOARD_PORT: '8765'
DASHBOARD_TOKEN: <FILL-ME-DASHBOARD-TOKEN>
COMMAND_BUS_ENABLED: '1'
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
ports:
- 8765:8765
qabot:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- telegram_bot.py
environment:
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_BACKFILL_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets/v2
ALPACA_DATA_FEED: sip
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: <FILL-ME-FMP-KEY>
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '20'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: http://192.168.1.5:30068
OLLAMA_MODEL: qwen2.5:7b-instruct
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '1'
STRATEGY_SETUPS: opening_range_break
TELEGRAM_BOT_TOKEN: <FILL-ME-TELEGRAM-TOKEN>
TELEGRAM_CHAT_ID: <FILL-ME-TELEGRAM-CHAT-ID>
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: marketable
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: limit
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: 8:3,15:9,25:18,40:30
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: prior_low
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REGIME_GATE_ENABLED: '1'
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '20'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '4'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: America/New_York
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '15'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: <FILL-ME-YOUTUBE-KEY>
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
transcripts:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- -m
- youtube_claims.run
- run
environment:
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_BACKFILL_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets/v2
ALPACA_DATA_FEED: sip
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: <FILL-ME-FMP-KEY>
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '20'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: http://192.168.1.5:30068
OLLAMA_MODEL: qwen2.5:7b-instruct
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '1'
STRATEGY_SETUPS: opening_range_break
TELEGRAM_BOT_TOKEN: <FILL-ME-TELEGRAM-TOKEN>
TELEGRAM_CHAT_ID: <FILL-ME-TELEGRAM-CHAT-ID>
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: marketable
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: limit
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: 8:3,15:9,25:18,40:30
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: prior_low
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REGIME_GATE_ENABLED: '1'
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '20'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '4'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: America/New_York
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '15'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: <FILL-ME-YOUTUBE-KEY>
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
admin:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- admin_api.py
environment:
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_BACKFILL_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets/v2
ALPACA_DATA_FEED: sip
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: <FILL-ME-FMP-KEY>
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '20'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: http://192.168.1.5:30068
OLLAMA_MODEL: qwen2.5:7b-instruct
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '1'
STRATEGY_SETUPS: opening_range_break
TELEGRAM_BOT_TOKEN: <FILL-ME-TELEGRAM-TOKEN>
TELEGRAM_CHAT_ID: <FILL-ME-TELEGRAM-CHAT-ID>
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: marketable
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: limit
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: 8:3,15:9,25:18,40:30
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: prior_low
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REGIME_GATE_ENABLED: '1'
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '20'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '4'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: America/New_York
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '15'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: <FILL-ME-YOUTUBE-KEY>
ADMIN_PORT: '8767'
ADMIN_TOKEN: <FILL-ME-ADMIN-TOKEN>
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
ports:
- 8767:8767
scheduler:
image: docker.io/fez5587/momentum-app:latest
command:
- /bin/sh
- -c
- "echo \"[scheduler] up; fires 20:00 ET Mon-Fri\"\nwhile true; do\n if [ \"$$(date +%H:%M)\" = \"20:00\" ] && [ \"$$(date +%u)\" -le 5 ]; then\n python -m research.premarket_eval || echo \"[scheduler] eval FAILED\"\n python -m research.ingestion.market_context || echo \"[scheduler] market_context FAILED\"\n python -m research.eod_reconcile || echo \"[scheduler] reconcile FAILED\"\n python eod_replay.py || echo \"[scheduler] eod_replay FAILED\"\n python -m research.labeler build || echo \"[scheduler] labeler build FAILED\"\n python -m research.labeler build-tod1 || echo \"[scheduler] labeler build-tod1 FAILED\"\n python nightly_tune.py || echo \"[scheduler] nightly_tune FAILED\"\n python -m research.regime build-features || echo \"[scheduler] regime build FAILED\"\n python -m research.regime score --model threshold_v1 || echo \"[scheduler] regime threshold FAILED\"\n python -m research.regime score --model meta_v1 || echo \"[scheduler] regime meta FAILED\"\n python -m research.experiments run || echo \"[scheduler] experiments run FAILED\"\n python -m research.experiments adjudicate || echo \"[scheduler] experiments adjudicate FAILED\"\n python -m research.weekly_tallies || echo \"[scheduler] weekly_tallies FAILED\"\n sleep 120\n fi\n sleep 20\ndone\n"
environment:
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_BACKFILL_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets/v2
ALPACA_DATA_FEED: sip
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: <FILL-ME-FMP-KEY>
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '20'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: http://192.168.1.5:30068
OLLAMA_MODEL: qwen2.5:7b-instruct
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '1'
STRATEGY_SETUPS: opening_range_break
TELEGRAM_BOT_TOKEN: <FILL-ME-TELEGRAM-TOKEN>
TELEGRAM_CHAT_ID: <FILL-ME-TELEGRAM-CHAT-ID>
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: marketable
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: limit
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: 8:3,15:9,25:18,40:30
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: prior_low
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REGIME_GATE_ENABLED: '1'
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '20'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '4'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: America/New_York
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '15'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: <FILL-ME-YOUTUBE-KEY>
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
db-backup:
image: postgres:18
restart: unless-stopped
environment:
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
BACKUP_DIR: /backups
BACKUP_KEEP_DAYS: '14'
TZ: America/New_York
volumes:
- /mnt/volume/Apps/momentum/data/backups:/backups
depends_on:
postgres:
condition: service_healthy
command:
- /bin/bash
- -c
- "set -o pipefail\necho \"[backup] up; fires 20:10 ET Mon-Fri\"\nwhile true; do\n if [ \"$$(date +%H:%M)\" = \"20:10\" ] && [ \"$$(date +%u)\" -le 5 ]; then\n out=\"$$BACKUP_DIR/momentum-$$(date +%F).sql.gz\"\n mkdir -p \"$$BACKUP_DIR\" && pg_dump \"$$DATABASE_URL\" | gzip > \"$$out.tmp\" && gunzip -t \"$$out.tmp\" && mv \"$$out.tmp\" \"$$out\" && echo \"[backup] OK\" || echo \"[backup] FAILED\"\n sleep 120\n fi\n sleep 20\ndone\n"
+116
View File
@@ -0,0 +1,116 @@
# TRADIER PARALLEL STACK — template (fill <FILL-ME-*>, save as
# truenas-compose.tradier.local.yaml, never commit the filled copy).
#
# Runs the SAME image/strategy as the paper stack but executes through the
# Tradier mirror (BROKER=tradier, sandbox by default). Fully self-contained:
# own postgres (host 5434), own dashboard (host 8768). Market data stays on
# Alpaca (keys still required). Telegram + deadman are OFF here so the main
# stack's alerts stay unambiguous.
#
# Shares the main stack's data dir READ-ONLY: the loop inherits the nightly
# learned_params.json and overrides.env without being able to clobber them.
#
# Fresh-DB bootstrap (same playbook as live staging), run on cube:
# ./bootstrap_live_db.sh <nightly-backup.sql.gz> <this-stack-postgres-container>
# or skip it and let the loop start from an empty DB (research baselines absent,
# config knobs at code defaults — fine for a first shakeout).
#
# Requires an image built from >= 1cefb51 (tradier_broker in image).
#
# Sandbox caveat: Tradier sandbox fills are simulated and can be slow/absent —
# an order parked in 'pending_new' is usually the sandbox, not the code.
services:
postgres:
image: postgres:18
environment:
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_DB: momentum
POSTGRES_PASSWORD: <FILL-ME-PG-PASSWORD>
POSTGRES_USER: momentum
TZ: America/New_York
healthcheck:
interval: 10s
retries: 5
test:
- CMD-SHELL
- pg_isready -U momentum -d momentum
timeout: 5s
# crash recovery after an unclean shutdown can outlast 5 x 10s (see truenas-compose.yml)
start_period: 300s
ports:
- 5434:5432
restart: unless-stopped
volumes:
- /mnt/Volume/Storage/Apps/momentum/pgdata-tradier:/var/lib/postgresql/data
loop:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- run_live_paper.py
- --no-dashboard
depends_on:
postgres:
condition: service_healthy
environment: &tradier-env
BROKER: tradier
TRADIER_ACCESS_TOKEN: <FILL-ME-TRADIER-TOKEN>
TRADIER_ACCOUNT_ID: <FILL-ME-TRADIER-ACCOUNT>
TRADIER_ENV: sandbox
ALPACA_API_KEY: <FILL-ME-ALPACA-KEY>
ALPACA_SECRET_KEY: <FILL-ME-ALPACA-SECRET>
ALPACA_BACKFILL_FEED: sip
# fresh DB has no config_overrides, so compose must set the paid SIP
# feed explicitly or this stack silently runs thin IEX volume
ALPACA_DATA_FEED: sip
ALPACA_BASE_URL: https://paper-api.alpaca.markets
ALPACA_PAPER_SYNC_ENABLED: '1'
DATABASE_URL: postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum
# alerts stay with the MAIN stack — no double Telegram / deadman pings
TELEGRAM_BOT_TOKEN: ''
TELEGRAM_CHAT_ID: ''
DEADMAN_DAILY_PING: '0'
DEADMAN_STALE_MIN: '5'
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '0'
QUOTE_SNAPSHOT_INTERVAL_SECONDS: '8'
STRATEGY_SETUPS: opening_range_break
DEPTH_RECORD_ENABLED: '0'
TZ: America/New_York
healthcheck:
interval: 60s
retries: 5
start_period: 180s
test:
- CMD
- python
- -c
- import os; os.environ['PG_SKIP_SCHEMA']='1'; import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)
timeout: 20s
restart: unless-stopped
volumes:
- /mnt/Volume/Storage/Apps/momentum/data:/app/data:ro
dashboard:
image: docker.io/fez5587/momentum-app:latest
command:
- python
- -u
- dashboard_api.py
depends_on:
postgres:
condition: service_healthy
environment:
<<: *tradier-env
COMMAND_BUS_ENABLED: '1'
DASHBOARD_HOST: 0.0.0.0
DASHBOARD_PORT: '8765'
DASHBOARD_TOKEN: <FILL-ME-DASHBOARD-TOKEN>
ports:
- 8768:8765
restart: unless-stopped
volumes:
- /mnt/Volume/Storage/Apps/momentum/data:/app/data:ro
+288
View File
@@ -0,0 +1,288 @@
# Momentum stack for TrueNAS SCALE (Apps -> Discover Apps -> Custom App -> YAML).
#
# PULL-ONLY: TrueNAS custom apps cannot `build:` — the image comes from GHCR
# (published by .github/workflows/docker.yml on every master push). Make the
# ghcr.io/fez5587/momentum-app package PUBLIC once (GitHub -> Packages ->
# momentum-app -> Package settings -> Change visibility), or add registry
# credentials in TrueNAS (Apps -> Settings -> Manage Container Images).
#
# BEFORE DEPLOYING:
# 1. Create a dataset for the stack, e.g. /mnt/<pool>/apps/momentum
# with subdirs: data/ pgdata/
# 2. Replace /mnt/tank/apps/momentum below with your dataset path.
# 3. Fill every <FILL-ME> in the environment block (these are the secrets
# you wanted to manage from a protected space — TrueNAS stores the YAML
# server-side, or mount an env file from a restricted dataset instead).
#
# TZ is America/New_York: the loop's market clock thinks in ET.
x-app: &app
image: docker.io/fez5587/momentum-app:latest # also published to ghcr.io/fez5587/momentum-app
environment: &app-env
ALPACA_API_KEY: '<FILL-ME>'
ALPACA_BACKFILL_FEED: 'sip'
ALPACA_BASE_URL: 'https://paper-api.alpaca.markets/v2'
ALPACA_DATA_FEED: 'sip'
ALPACA_PAPER_SYNC_ENABLED: '1'
ALPACA_SECRET_KEY: '<FILL-ME>'
DATABASE_URL: 'postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum'
DEADMAN_DAILY_PING: '1'
DEADMAN_STALE_MIN: '5'
DISCOVER_INTERVAL_SECONDS: '120'
FMP_API_KEY: '<FILL-ME>'
INGEST_STALE_BLOCK: '0'
INGEST_STALE_SECONDS: '180'
LIVE_BARS_INTERVAL_SECONDS: '5'
LIVE_BARS_REFRESH_MINUTES: '20'
NEWS_CATALYST_SCORE_ENABLED: '1'
NEWS_DILUTION_VETO_ENABLED: '1'
NEWS_ENRICH_BATCH_LIMIT: '50'
NEWS_ENRICH_ENABLED: '1'
NEWS_ENRICH_INTERVAL_SECONDS: '120'
NEWS_ENRICH_LOOKBACK_HOURS: '12'
NEWS_FEEDS: 'prnewswire=https://www.prnewswire.com/rss/news-releases-list.rss,businesswire=https://feed.businesswire.com/rss/home/?rss=G1QFDERJXkJeGVtRWQ==,globenewswire=https://www.globenewswire.com/RssFeed/orgclass/1/feedTitle/GlobeNewswire%20-%20News%20about%20Public%20Companies'
NEWS_LOOKBACK_HOURS: '8'
NEWS_RANK_BOOST: '2.0'
OLLAMA_ENABLED: '1'
OLLAMA_HOST: 'http://192.168.1.5:30068'
OLLAMA_MODEL: 'qwen2.5:7b-instruct'
OLLAMA_TEMPERATURE: '0.1'
OLLAMA_TIMEOUT_SECONDS: '60'
ORB_BARS: '5'
PG_SKIP_SCHEMA: '0'
STRATEGY_SETUPS: 'opening_range_break'
TELEGRAM_BOT_TOKEN: '<FILL-ME>'
TELEGRAM_CHAT_ID: "'2082795871'"
TRADING_AUTO_APPROVE: '1'
TRADING_BACKOUT_COOLDOWN_SECONDS: '180'
TRADING_BLOCK_REENTRY_AFTER_EXIT: '1'
TRADING_ENTRY_CONFIRM_BY_MINUTE: '0'
TRADING_ENTRY_FILL_MODEL: 'resting'
TRADING_ENTRY_INVALIDATE_PCT: '0.015'
TRADING_ENTRY_ORDER_TYPE: 'limit'
TRADING_ENTRY_TIMEOUT_BARS: '2'
TRADING_EOD_FLATTEN_ENABLED: '1'
TRADING_EOD_FLATTEN_TIME: '15:55'
TRADING_EXECUTION_ENABLED: '1'
TRADING_EXIT_BREAKEVEN_R: '1.0'
TRADING_EXIT_PROFIT_TIERS: '8:3,15:9,25:18,40:30'
# Regime gate (plan 011) LIVE: block fresh entries on sessions the walk-forward
# metalabel scores unfavorable (sit out bad regimes). Only the LOOP acts on it;
# fail-open (unfit model / error never blocks). Needs regime_features populated
# (the scheduler builds them nightly). Set 0 to disable.
TRADING_REGIME_GATE_ENABLED: '1'
# Favorable threshold: block a session when P(profit) < this. 0.5 = code
# default (matches the backtest). Lower (e.g. 0.4) trades MORE days; raise to
# trade fewer, higher-confidence days. Optional — omit to use the 0.5 default.
TRADING_REGIME_GATE_P_MIN: '0.5'
TRADING_EXIT_TRAIL_AFTER_R: '1.0'
TRADING_EXIT_TRAIL_MODE: 'prior_low'
TRADING_EXIT_MANAGE_INTERVAL_SECONDS: '4' # 12->4s: react to fast small-cap round-trips (was giving back gains)
TRADING_EXIT_THREAD_ENABLED: '1'
DISCOVER_THREAD_ENABLED: '1'
TRADING_EXIT_THREAD_INTERVAL_SECONDS: '1.0'
TRADING_EXIT_TRAIL_R_STEP: '0.25'
TRADING_LIQUIDITY_MAX_VOLUME_PCT: '0.025'
TRADING_MAX_BACKOUTS_PER_SYMBOL: '3'
TRADING_MAX_CONCURRENT_POSITIONS: '10'
TRADING_MAX_DAILY_LOSS_PCT: '0.05'
TRADING_MAX_FRESH_ENTRIES_PER_DAY: '10'
TRADING_MAX_GROSS_NOTIONAL_PCT: '0.60'
TRADING_MAX_ORDERS_PER_TICK: '1'
TRADING_MAX_POSITION_PCT: '0.16'
TRADING_MAX_RISK_DOLLARS: '500'
TRADING_MIN_DAY_VOLUME: '300000'
TRADING_MIN_ENTRY_GAP_PCT: '0.10'
TRADING_MIN_QUALITY_SCORE: '0.60'
TRADING_PRICE_MAX: '20.0'
TRADING_PRICE_MIN: '2.0'
TRADING_REENTRY_MIN_LOSS_PCT: '0.01'
TRADING_REQUIRE_ABOVE_VWAP: '1'
TRADING_REWARD_MULTIPLE: '10.0'
TRADING_RISK_PER_TRADE_PCT: '0.01'
TRADING_SESSION_ONLY: '1'
TRADING_TRIGGER_SLIP_PCT: '0.004'
TRADING_UNIFIED_ENTRY: '1'
TRIGGER_ARM_INTERVAL_SECONDS: '8'
TRIGGER_ENABLED: '1'
TRIGGER_GAP_MAX: '35'
TRIGGER_GAP_MIN: '10.0'
TRIGGER_GAP_SQUEEZE_MIN: '150'
TRIGGER_GAP_SQUEEZE_RVOL: '5.0'
TRIGGER_INTERVAL_SECONDS: '2'
TRIGGER_MAX_ARMED: '10'
TRIGGER_MIN_DOLLAR_VOL: '8000'
TRIGGER_PRICE_MAX_AGE: '12'
TRIGGER_RVOL_MIN: '2.0'
TZ: 'America/New_York'
WATCHER_ENABLED: '1'
WATCHER_INTERVAL_SECONDS: '8'
WATCHER_PRICE_MAX: '20'
WATCHER_PRICE_MIN: '2'
YOUTUBE_API_KEY: '<FILL-ME>'
# comma-separated playlist URLs (or bare ids); synced into the playlists
# table at transcripts-service boot — edit here to add/track more playlists
YT_PLAYLIST_URLS: ''
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
services:
postgres:
image: postgres:18
environment:
POSTGRES_USER: momentum
POSTGRES_PASSWORD: <FILL-ME-PG-PASSWORD>
POSTGRES_DB: momentum
PGDATA: /var/lib/postgresql/data/pgdata
volumes:
- /mnt/volume/Apps/momentum/pgdata:/var/lib/postgresql/data
ports:
# published on 5433 so it can coexist with the EXISTING postgres (5432)
# during migration — pgAdmin/pg_restore reach the new instance on 5433.
# In-stack services still use postgres:5432 (the container port).
- "5433:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U momentum -d momentum"]
interval: 10s
timeout: 5s
retries: 5
# crash recovery (WAL replay) after an unclean shutdown can outlast 5 x 10s; without a grace period
# compose marks postgres unhealthy and refuses to start every dependent service (2026-09-30)
start_period: 300s
restart: unless-stopped
loop:
<<: *app
command: ["python", "-u", "run_live_paper.py", "--no-dashboard"]
healthcheck:
# PG_SKIP_SCHEMA=1: a healthcheck must NOT re-apply schema DDL — the 60s
# CREATE-IF-NOT-EXISTS lock burst cancelled minute_bars autovacuum forever
# (2026-07-09 post-migration: stats rotted, dashboard queries crawled)
test: ["CMD", "python", "-c", "import os; os.environ['PG_SKIP_SCHEMA']='1'; import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)"]
interval: 60s
timeout: 20s
retries: 5
start_period: 180s
dashboard:
<<: *app
command: ["python", "-u", "dashboard_api.py"]
environment:
<<: *app-env
DASHBOARD_HOST: "0.0.0.0"
DASHBOARD_PORT: "8765"
# read-only viewer: never re-apply the 473-line schema (the SSE change
# detector holds one long-lived connection) — mirrors the loop healthcheck.
PG_SKIP_SCHEMA: '1'
# Shared secret required on every POST (approve/reject/exit + the command
# bus). Empty = commands disabled + POSTs unauthenticated; fill to arm the
# kill switch. Reference by name only — never inline the secret value.
DASHBOARD_TOKEN: <FILL-ME-DASHBOARD-TOKEN>
# Gate the operator command bus on (also needs DASHBOARD_TOKEN set).
COMMAND_BUS_ENABLED: '1'
qabot:
<<: *app
command: ["python", "-u", "telegram_bot.py"]
transcripts:
<<: *app
command: ["python", "-u", "-m", "youtube_claims.run", "run"]
admin:
<<: *app
command: ["python", "-u", "admin_api.py"]
environment:
<<: *app-env
ADMIN_PORT: "8767"
ADMIN_TOKEN: <FILL-ME-ADMIN-TOKEN>
ports:
- "8767:8767"
volumes:
- /mnt/volume/Apps/momentum/data:/app/data
- /var/run/docker.sock:/var/run/docker.sock # service status + restarts
scheduler:
<<: *app
command:
- /bin/sh
- -c
- |
echo "[scheduler] up; fires 20:00 ET Mon-Fri"
while true; do
if [ "$$(date +%H:%M)" = "20:00" ] && [ "$$(date +%u)" -le 5 ]; then
python -m research.premarket_eval || echo "[scheduler] eval FAILED"
# Premarket-structure labeler (Phase 1, observe-only): distill each
# gapper's premarket tape for entry-conversion analysis (H1-H3
# pre-registered in the module docstring). Gates nothing.
python -m research.premarket_structure build || echo "[scheduler] premarket_structure FAILED"
# SPY/QQQ ingest -> forward source for the labeler's market-context features (plan 009)
python -m research.ingestion.market_context || echo "[scheduler] market_context ingest FAILED"
python -m research.eod_reconcile || echo "[scheduler] reconcile FAILED"
# Flywheel: grow the whole-market dataset first, then fit the tuner on
# it. 20:00 ET is after close, so replay of "today" is complete data —
# NOT look-ahead (replay is a labeling job, not a live gate).
python eod_replay.py || echo "[scheduler] eod_replay FAILED"
python nightly_tune.py || echo "[scheduler] nightly_tune FAILED"
# Regime gate (plan 011, ALL-SHADOW): a session-level metalabel
# P(our ORB profits today). build-features labels each session from its
# OWN realized expR; score writes the WALK-FORWARD favorable flag the
# experiments' regime_gate intervention reads. Must run BEFORE
# experiments run. Does NOT gate live trading (that is owner-gated).
python -m research.regime build-features || echo "[scheduler] regime build-features FAILED"
python -m research.regime score --model threshold_v1 || echo "[scheduler] regime score threshold FAILED"
python -m research.regime score --model meta_v1 || echo "[scheduler] regime score meta FAILED"
# Experiment OS (plan 006, ALL-SHADOW): replay running experiments over
# newly-logged fires, then adjudicate against pre-registered bars.
python -m research.experiments run || echo "[scheduler] experiments run FAILED"
python -m research.experiments adjudicate || echo "[scheduler] experiments adjudicate FAILED"
# Weekly execution tallies (read-only): midday-entry net + market_exit
# post-run. Script self-gates to Friday; other nights it no-ops.
python -m research.weekly_tallies || echo "[scheduler] weekly_tallies FAILED"
sleep 120
fi
sleep 20
done
db-backup:
# Nightly pg_dump at 20:10 ET Mon-Fri, on the postgres:18 image so the
# client always matches the server major (the app image has NO pg_dump).
# This host is pull-only (no repo checkout), so the backup logic is INLINE
# — keep it in sync with deploy/docker/backup_db.sh. Restore:
# gunzip -c momentum-<date>.sql.gz | psql "$DATABASE_URL"
image: postgres:18
restart: unless-stopped
environment:
DATABASE_URL: 'postgresql://momentum:<FILL-ME-PG-PASSWORD>@postgres:5432/momentum'
BACKUP_DIR: '/backups'
BACKUP_KEEP_DAYS: '14'
TZ: 'America/New_York' # postgres image defaults to UTC; the 20:10 gate is ET
volumes:
- /mnt/volume/Apps/momentum/data/backups:/backups
depends_on:
postgres:
condition: service_healthy
command:
- /bin/bash
- -c
- |
set -o pipefail
echo "[backup] up; fires 20:10 ET Mon-Fri"
while true; do
if [ "$$(date +%H:%M)" = "20:10" ] && [ "$$(date +%u)" -le 5 ]; then
out="$$BACKUP_DIR/momentum-$$(date +%F).sql.gz"
mkdir -p "$$BACKUP_DIR" \
&& pg_dump "$$DATABASE_URL" | gzip > "$$out.tmp" \
&& gunzip -t "$$out.tmp" && mv "$$out.tmp" "$$out" \
&& find "$$BACKUP_DIR" -name 'momentum-*.sql.gz*' -mtime +"$$BACKUP_KEEP_DAYS" -delete \
&& date > "$$BACKUP_DIR/last_backup_ok" \
&& echo "[backup] OK $$out" \
|| echo "[backup] FAILED"
sleep 120
fi
sleep 20
done
+182
View File
@@ -0,0 +1,182 @@
# Kubernetes deployment
A container image and Helm chart that run the whole Momentum System — including
its **PostgreSQL datastore** — inside a single Kubernetes namespace.
## What gets deployed
| Workload | Kind | Notes |
| --- | --- | --- |
| PostgreSQL | StatefulSet + PVC | Single source of truth. Schema auto-creates on the app's first connection from `storage/pg_schema.sql`. |
| App loop + dashboard | Deployment (`replicas: 1`, `Recreate`) | `run_live_paper.py` with the **embedded** dashboard on :8010. Stateful singleton — never scale up. |
| journal / eod-replay / nightly-tune | CronJobs | US/Eastern (`spec.timeZone`), weekdays. Share the app's `data/` PVC. |
| Grafana | Deployment + Service | Optional (`grafana.enabled`). Datasource auto-points at the in-cluster Postgres. |
The app loop and the CronJobs run the **same image** (built from the repo
`Dockerfile`), just with different commands.
## Local development with Docker Compose
For a single-machine stack (same components, no Kubernetes) use the root
`docker-compose.yml`. It builds the app from the `Dockerfile`, brings up
PostgreSQL with a persistent volume, and runs the loop with its embedded
dashboard. Config comes from a local `.env` (optional — without `ALPACA_*` keys
the loop runs in dry mode; the dashboard and DB still work). `DATABASE_URL` is
overridden to the in-network `postgres` service, so the stack is self-contained.
```bash
cp .env.example .env # optional: add ALPACA_* keys for live data
docker compose up -d --build # postgres + app loop + dashboard (:8010)
docker compose logs -f app # watch the loop
open http://localhost:8010 # dashboard
docker compose --profile monitoring up -d # also start Grafana (:3000)
```
The scheduled jobs sit under the `jobs` profile so they don't auto-start; run
them on demand (or from host cron / a scheduler on the ET times in `.env`):
```bash
docker compose run --rm nightly-tune
docker compose run --rm eod-replay
docker compose run --rm journal
```
Postgres credentials default to `momentum`/`momentum`/`momentum`; override with
`POSTGRES_USER` / `POSTGRES_PASSWORD` / `POSTGRES_DB` in `.env`. Named volumes
`pg-data`, `app-data`, and `grafana-data` persist across restarts (the jobs share
`app-data` with the app so `nightly_tune.py`'s `learned_params.json` reaches the
loop). The Compose Grafana uses its own datasource override at
`deploy/compose/grafana/datasources/momentum.yaml` (pointed at the `postgres`
service); the legacy `grafana/docker-compose.yml` is unrelated and left for
Grafana-only use against an external DB.
## Build & publish the image
Pushes to the working branch / `main` / `v*` tags trigger
`.github/workflows/docker-publish.yml`, which builds and pushes to
`ghcr.io/fez5587/momentum-system`. Point the chart at a pushed tag via
`image.tag` (a commit SHA, or `latest` on the default branch).
Local build:
```bash
docker build -t momentum:dev .
```
## Install
```bash
helm install momentum ./deploy/helm/momentum \
-n momentum --create-namespace \
--set image.tag=latest \
--set secrets.alpacaApiKey=YOUR_KEY \
--set secrets.alpacaSecretKey=YOUR_SECRET
```
Without Alpaca keys the loop runs in **dry mode** (no live data/orders); the
dashboard and database still work.
### Secrets
The chart builds `DATABASE_URL` from the in-cluster Postgres service and a
password that is generated on first install (or set `postgres.password` to pin
it). Broker keys come from `secrets.*`.
For real keys, prefer an existing Secret over `values.yaml`:
```bash
kubectl -n momentum create secret generic momentum-broker \
--from-literal=DATABASE_URL=... \
--from-literal=ALPACA_API_KEY=... --from-literal=ALPACA_SECRET_KEY=... \
--from-literal=APCA_API_KEY_ID=... --from-literal=APCA_API_SECRET_KEY=...
helm install momentum ./deploy/helm/momentum -n momentum \
--set secrets.existingSecret=momentum-broker
```
When `secrets.existingSecret` is set the chart renders no Secret of its own and
expects that one to carry `DATABASE_URL`, the `ALPACA_*`/`APCA_*` keys, and
`POSTGRES_PASSWORD`.
## Reach the dashboard
```bash
kubectl -n momentum port-forward svc/momentum-app 8010:8010
open http://localhost:8010
```
Set `ingress.enabled=true` (with `ingress.host`/`className`) to expose it
instead.
## Verify
```bash
helm lint deploy/helm/momentum
helm template momentum deploy/helm/momentum | less
# After install:
kubectl -n momentum get pods
kubectl -n momentum exec sts/momentum-postgres -- psql -U momentum -d momentum -c '\dt'
kubectl -n momentum create job --from=cronjob/momentum-nightly-tune tune-test
```
## Notes & constraints
- **Singleton loop.** The trading loop holds in-memory state and one broker
session; `replicas` must stay `1` and the strategy is `Recreate`.
- **Shared `data/` PVC is ReadWriteOnce.** The CronJob pods mount the same PVC
as the app (so `nightly_tune.py` can write `learned_params.json` for the loop
to read), which schedules them onto the app pod's node. For multi-node
clusters, use an RWX StorageClass or split the volumes.
- **CronJob `spec.timeZone`** requires Kubernetes ≥ 1.27.
- The legacy `grafana/docker-compose.yml` is left in place for local use; the
chart provisions its own Grafana against the in-cluster Postgres.
## Future hardening (roadmap)
The package deploys and runs, but the items below separate "it comes up" from
"runs reliably for a trading system." They are **not implemented yet** — captured
here so they can be picked up later. Each notes what, why, and the rough approach.
### Reliability
- **Loop-heartbeat liveness** *(needs an app-code change).* Today `/api/health`
(`api/main.py:343`) returns a static `{"ok": true}`, which only proves the
dashboard *thread* is alive. The trading loop runs in the main thread
(`run_live_paper.py:328`), so a **crashed** loop exits the process and Kubernetes
restarts it (fine), but a **hung** loop (stuck network call, deadlock) keeps
answering health checks `200` and runs undetected. Approach: have the loop stamp a
last-tick timestamp into `DashboardState` each iteration, make `/api/health` (or a
new `/healthz`) return `503` when the last tick is older than a threshold, and
point the Deployment's `livenessProbe` at it. Highest-value item.
- **Postgres backup CronJob.** The event store is the source of truth and nothing
backs it up. Approach: a chart CronJob running `pg_dump` (postgres image, creds
from the Secret) to the app `data` PVC with N-day retention; optionally offload to
object storage.
- **Schema bootstrap hook.** The schema auto-applies on the app's first connection,
so a first-run race between the app and a CronJob is possible. Approach: a
`pre-install`/`pre-upgrade` Helm-hook Job that applies `storage/pg_schema.sql`
deterministically, with `PG_SKIP_SCHEMA=1` set on the app and CronJobs.
### Security
- **Pod hardening.** Add a `securityContext` across workloads: `runAsNonRoot`, drop
ALL capabilities, `seccompProfile: RuntimeDefault`, and `readOnlyRootFilesystem`
where feasible (the app needs `/app/data` writable — already a mounted volume).
- **NetworkPolicy.** Default-deny in the namespace; allow only the app, CronJobs, and
Grafana to reach Postgres:5432, and restrict who can reach the dashboard.
- **Grafana auth.** The chart inherits anonymous-admin from the original compose
(`templates/grafana.yaml` env). Make it off by default in-cluster, with a real
admin password sourced from the Secret (`GF_SECURITY_ADMIN_PASSWORD`).
### CI / operational
- **Chart CI validation.** Add a CI job running `helm lint` + `helm template` +
`kubeconform` on every push — catches template/schema errors automatically (the
chart could not be rendered in the sandbox where it was authored, since image-blob
CDNs were egress-blocked).
- **Image digest pinning.** For production installs, pin `image` by SHA digest rather
than the mutable `latest` tag.
- **PodDisruptionBudget / singleton guard.** A PDB for the singleton loop plus a
guard so it is never scaled above `1` or evicted to zero unexpectedly.
+5
View File
@@ -0,0 +1,5 @@
.DS_Store
.git
*.tmp
*.bak
*.orig
+14
View File
@@ -0,0 +1,14 @@
apiVersion: v2
name: momentum
description: Small-cap momentum research & paper-trading system, with an in-cluster PostgreSQL datastore.
type: application
# Chart version is independent of the app; bump on chart changes.
version: 0.1.0
# Tracks the application release (pyproject.toml).
appVersion: "0.4.0"
kubeVersion: ">=1.27.0-0"
keywords:
- trading
- postgres
- momentum
home: https://github.com/fez5587/momentum-system
+137
View File
@@ -0,0 +1,137 @@
{
"uid": "momentum-live",
"title": "Momentum — Live",
"tags": ["momentum"],
"timezone": "browser",
"schemaVersion": 39,
"version": 1,
"refresh": "5s",
"time": { "from": "now/d", "to": "now" },
"templating": { "list": [] },
"annotations": { "list": [] },
"panels": [
{
"id": 1,
"type": "stat",
"title": "Account equity",
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 0 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"fieldConfig": { "defaults": { "unit": "currencyUSD" }, "overrides": [] },
"options": { "reduceOptions": { "calcs": ["lastNotNull"] }, "colorMode": "value", "graphMode": "none" },
"targets": [{
"refId": "A",
"format": "table",
"rawQuery": true,
"rawSql": "SELECT (payload_json::jsonb->>'total_equity')::float AS equity FROM events WHERE event_type='account_summary_updated' ORDER BY timestamp DESC LIMIT 1"
}]
},
{
"id": 2,
"type": "stat",
"title": "Ready signals (today)",
"gridPos": { "h": 4, "w": 6, "x": 6, "y": 0 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"fieldConfig": { "defaults": { "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "blue", "value": 1 } ] } }, "overrides": [] },
"options": { "reduceOptions": { "calcs": ["lastNotNull"] }, "colorMode": "background", "graphMode": "none" },
"targets": [{
"refId": "A",
"format": "table",
"rawQuery": true,
"rawSql": "SELECT count(DISTINCT payload_json::jsonb->>'symbol')::int AS ready FROM events WHERE event_type='signal_ready'"
}]
},
{
"id": 6,
"type": "stat",
"title": "Symbols evaluated",
"gridPos": { "h": 4, "w": 6, "x": 12, "y": 0 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"options": { "reduceOptions": { "calcs": ["lastNotNull"] }, "colorMode": "value", "graphMode": "none" },
"targets": [{
"refId": "A",
"format": "table",
"rawQuery": true,
"rawSql": "SELECT count(DISTINCT payload_json::jsonb->>'symbol')::int AS evaluated FROM events WHERE event_type='criteria_evaluated'"
}]
},
{
"id": 3,
"type": "table",
"title": "Evaluation board — what it's looking at + the determination",
"gridPos": { "h": 14, "w": 14, "x": 0, "y": 4 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"fieldConfig": {
"defaults": {},
"overrides": [
{ "matcher": { "id": "byName", "options": "status" },
"properties": [ { "id": "custom.cellOptions", "value": { "type": "color-background" } },
{ "id": "mappings", "value": [
{ "type": "value", "options": { "ready": { "color": "green", "index": 0 }, "late": { "color": "yellow", "index": 1 }, "blocked": { "color": "red", "index": 2 } } } ] } ] },
{ "matcher": { "id": "byName", "options": "gap" }, "properties": [ { "id": "unit", "value": "percentunit" } ] },
{ "matcher": { "id": "byName", "options": "score" }, "properties": [ { "id": "unit", "value": "percent" } ] }
]
},
"options": { "sortBy": [ { "displayName": "score", "desc": true } ] },
"targets": [{
"refId": "A",
"format": "table",
"rawQuery": true,
"rawSql": "SELECT * FROM ( SELECT DISTINCT ON (sym) sym AS symbol, status, score, gap, rvol, reason FROM ( SELECT payload_json::jsonb->>'symbol' sym, payload_json::jsonb->'criteria_results'->>'status' status, (payload_json::jsonb->>'success_score_pct')::float score, (payload_json::jsonb->'criteria_results'->>'gap_pct')::float gap, (payload_json::jsonb->'criteria_results'->>'relative_volume')::float rvol, payload_json::jsonb->'criteria_results'->>'reason' reason, timestamp ts FROM events WHERE event_type='criteria_evaluated' ) t ORDER BY sym, ts DESC ) x ORDER BY score DESC"
}]
},
{
"id": 4,
"type": "table",
"title": "Ready signals",
"gridPos": { "h": 7, "w": 10, "x": 14, "y": 4 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"targets": [{
"refId": "A",
"format": "table",
"rawQuery": true,
"rawSql": "SELECT timestamp AS time, payload_json::jsonb->>'symbol' AS symbol, message FROM events WHERE event_type='signal_ready' ORDER BY timestamp DESC LIMIT 25"
}]
},
{
"id": 5,
"type": "table",
"title": "Risk events (breaker / backouts)",
"gridPos": { "h": 7, "w": 10, "x": 14, "y": 11 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"targets": [{
"refId": "A",
"format": "table",
"rawQuery": true,
"rawSql": "SELECT timestamp AS time, payload_json::jsonb->>'rule_type' AS rule, message FROM events WHERE event_type='risk_rule_triggered' ORDER BY timestamp DESC LIMIT 25"
}]
},
{
"id": 7,
"type": "timeseries",
"title": "Minute bars ingested",
"gridPos": { "h": 7, "w": 12, "x": 0, "y": 18 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"fieldConfig": { "defaults": { "custom": { "drawStyle": "bars", "fillOpacity": 40 } }, "overrides": [] },
"targets": [{
"refId": "A",
"format": "time_series",
"rawQuery": true,
"rawSql": "SELECT timestamp AS time, (payload_json::jsonb->>'output_count')::float AS minute_rows FROM events WHERE event_type='module_tick' AND payload_json::jsonb->>'module'='ingestion' AND $__timeFilter(timestamp) ORDER BY 1"
}]
},
{
"id": 8,
"type": "timeseries",
"title": "Account equity",
"gridPos": { "h": 7, "w": 12, "x": 12, "y": 18 },
"datasource": { "type": "postgres", "uid": "momentum_pg" },
"fieldConfig": { "defaults": { "unit": "currencyUSD" }, "overrides": [] },
"targets": [{
"refId": "A",
"format": "time_series",
"rawQuery": true,
"rawSql": "SELECT timestamp AS time, (payload_json::jsonb->>'total_equity')::float AS equity FROM events WHERE event_type='account_summary_updated' AND $__timeFilter(timestamp) ORDER BY 1"
}]
}
]
}
+28
View File
@@ -0,0 +1,28 @@
Momentum System deployed as release "{{ .Release.Name }}" in namespace "{{ .Release.Namespace }}".
Components:
- PostgreSQL (StatefulSet, single source of truth) : svc/{{ include "momentum.postgresName" . }}:{{ .Values.postgres.port }}
- App loop + embedded dashboard (replicas: 1) : svc/{{ include "momentum.appName" . }}:{{ .Values.service.port }}
{{- if .Values.cronjobs.enabled }}
- CronJobs (journal, eod-replay, nightly-tune) : timeZone {{ .Values.cronjobs.timeZone }}
{{- end }}
{{- if .Values.grafana.enabled }}
- Grafana : svc/{{ include "momentum.grafanaName" . }}:{{ .Values.grafana.service.port }}
{{- end }}
Reach the dashboard:
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ include "momentum.appName" . }} {{ .Values.service.port }}:{{ .Values.service.port }}
open http://localhost:{{ .Values.service.port }}
{{- if not .Values.secrets.existingSecret }}
{{- if or (not .Values.secrets.alpacaApiKey) (not .Values.secrets.alpacaSecretKey) }}
WARNING: Alpaca keys are not set — the loop runs in dry mode (no live data/orders).
Supply them: helm upgrade {{ .Release.Name }} ... \
--set secrets.alpacaApiKey=YOUR_KEY --set secrets.alpacaSecretKey=YOUR_SECRET
{{- end }}
{{- end }}
The DB schema auto-creates on the app's first connection. Verify:
kubectl -n {{ .Release.Namespace }} exec sts/{{ include "momentum.postgresName" . }} -- \
psql -U {{ .Values.postgres.username }} -d {{ .Values.postgres.database }} -c '\dt'
@@ -0,0 +1,69 @@
{{/* Base name, overridable. */}}
{{- define "momentum.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/* Fully qualified release name. */}}
{{- define "momentum.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{/* Common labels. */}}
{{- define "momentum.labels" -}}
app.kubernetes.io/name: {{ include "momentum.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
{{- end -}}
{{/* Component-scoped selector labels. Call with (dict "ctx" . "component" "app"). */}}
{{- define "momentum.selectorLabels" -}}
app.kubernetes.io/name: {{ include "momentum.name" .ctx }}
app.kubernetes.io/instance: {{ .ctx.Release.Name }}
app.kubernetes.io/component: {{ .component }}
{{- end -}}
{{/* Names of shared resources. */}}
{{- define "momentum.postgresName" -}}{{ printf "%s-postgres" (include "momentum.fullname" .) }}{{- end -}}
{{- define "momentum.appName" -}}{{ printf "%s-app" (include "momentum.fullname" .) }}{{- end -}}
{{- define "momentum.grafanaName" -}}{{ printf "%s-grafana" (include "momentum.fullname" .) }}{{- end -}}
{{- define "momentum.configMapName" -}}{{ printf "%s-config" (include "momentum.fullname" .) }}{{- end -}}
{{- define "momentum.appDataPvcName" -}}{{ printf "%s-app-data" (include "momentum.fullname" .) }}{{- end -}}
{{/* Name of the Secret holding DB URL + broker keys (existing or chart-managed). */}}
{{- define "momentum.secretName" -}}
{{- if .Values.secrets.existingSecret -}}
{{- .Values.secrets.existingSecret -}}
{{- else -}}
{{- printf "%s-secrets" (include "momentum.fullname" .) -}}
{{- end -}}
{{- end -}}
{{/*
Resolve the Postgres password once: explicit value, else reuse the one already
stored in the chart-managed Secret (so upgrades don't rotate it), else generate.
*/}}
{{- define "momentum.postgresPassword" -}}
{{- if .Values.postgres.password -}}
{{- .Values.postgres.password -}}
{{- else -}}
{{- $existing := lookup "v1" "Secret" .Release.Namespace (printf "%s-secrets" (include "momentum.fullname" .)) -}}
{{- if and $existing $existing.data (index $existing.data "POSTGRES_PASSWORD") -}}
{{- index $existing.data "POSTGRES_PASSWORD" | b64dec -}}
{{- else -}}
{{- randAlphaNum 24 -}}
{{- end -}}
{{- end -}}
{{- end -}}
@@ -0,0 +1,16 @@
{{- if .Values.app.persistence.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ include "momentum.appDataPvcName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
spec:
accessModes: ["{{ .Values.app.persistence.accessMode }}"]
{{- if .Values.app.persistence.storageClass }}
storageClassName: {{ .Values.app.persistence.storageClass | quote }}
{{- end }}
resources:
requests:
storage: {{ .Values.app.persistence.size | quote }}
{{- end }}
@@ -0,0 +1,80 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "momentum.appName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
app.kubernetes.io/component: app
spec:
# Stateful singleton: the loop holds in-memory trading state and a 1:1 broker
# session. Never scale > 1. Recreate avoids two loops overlapping on rollout.
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "app") | nindent 6 }}
template:
metadata:
labels:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "app") | nindent 8 }}
annotations:
# Roll the pod when config/secret content changes.
checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
initContainers:
# Block until Postgres accepts connections so the first app connection
# (which auto-applies storage/pg_schema.sql) succeeds.
- name: wait-for-postgres
image: {{ .Values.postgres.image | quote }}
command:
- sh
- -c
- |
until pg_isready -h {{ include "momentum.postgresName" . }} -p {{ .Values.postgres.port }} -U {{ .Values.postgres.username }}; do
echo "waiting for postgres..."; sleep 2;
done
containers:
- name: app
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: {{ toYaml .Values.app.command | nindent 12 }}
ports:
- name: http
containerPort: 8010
envFrom:
- configMapRef:
name: {{ include "momentum.configMapName" . }}
- secretRef:
name: {{ include "momentum.secretName" . }}
volumeMounts:
- name: app-data
mountPath: /app/data
{{- if .Values.app.probes.enabled }}
readinessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: {{ .Values.app.probes.initialDelaySeconds }}
periodSeconds: {{ .Values.app.probes.periodSeconds }}
livenessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: {{ .Values.app.probes.initialDelaySeconds }}
periodSeconds: {{ .Values.app.probes.periodSeconds }}
{{- end }}
resources:
{{- toYaml .Values.app.resources | nindent 12 }}
volumes:
- name: app-data
{{- if .Values.app.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ include "momentum.appDataPvcName" . }}
{{- else }}
emptyDir: {}
{{- end }}
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "momentum.appName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
selector:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "app") | nindent 4 }}
ports:
- name: http
port: {{ .Values.service.port }}
targetPort: http
@@ -0,0 +1,10 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "momentum.configMapName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
data:
{{- range $key, $val := .Values.config }}
{{ $key }}: {{ $val | quote }}
{{- end }}
@@ -0,0 +1,59 @@
{{- if .Values.cronjobs.enabled }}
{{- $root := . }}
{{- $jobs := dict "journal" .Values.cronjobs.journal "eod-replay" .Values.cronjobs.eodReplay "nightly-tune" .Values.cronjobs.nightlyTune }}
{{- range $key, $job := $jobs }}
{{- if $job.enabled }}
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: {{ include "momentum.fullname" $root }}-{{ $key }}
labels:
{{- include "momentum.labels" $root | nindent 4 }}
app.kubernetes.io/component: cron-{{ $key }}
spec:
schedule: {{ $job.schedule | quote }}
# US/Eastern so the market-clock schedules in .env's *.sh comments hold.
timeZone: {{ $root.Values.cronjobs.timeZone | quote }}
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 1
template:
metadata:
labels:
{{- include "momentum.selectorLabels" (dict "ctx" $root "component" (printf "cron-%s" $key)) | nindent 12 }}
spec:
restartPolicy: Never
{{- with $root.Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 12 }}
{{- end }}
containers:
- name: {{ $key }}
image: "{{ $root.Values.image.repository }}:{{ $root.Values.image.tag }}"
imagePullPolicy: {{ $root.Values.image.pullPolicy }}
command: {{ toYaml $job.command | nindent 16 }}
envFrom:
- configMapRef:
name: {{ include "momentum.configMapName" $root }}
- secretRef:
name: {{ include "momentum.secretName" $root }}
volumeMounts:
- name: app-data
mountPath: /app/data
volumes:
- name: app-data
{{- if $root.Values.app.persistence.enabled }}
# Shared with the app pod (nightly-tune writes learned_params.json
# the loop reads on boot). RWO ⇒ cron pods schedule onto the app's node.
persistentVolumeClaim:
claimName: {{ include "momentum.appDataPvcName" $root }}
{{- else }}
emptyDir: {}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
+158
View File
@@ -0,0 +1,158 @@
{{- if .Values.grafana.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "momentum.grafanaName" . }}-provisioning
labels:
{{- include "momentum.labels" . | nindent 4 }}
data:
# Datasource points at the in-cluster Postgres service (not the old LAN IP).
datasource.yaml: |
apiVersion: 1
datasources:
- name: Momentum
uid: momentum_pg
type: postgres
access: proxy
url: {{ include "momentum.postgresName" . }}:{{ .Values.postgres.port }}
user: {{ .Values.postgres.username }}
jsonData:
database: {{ .Values.postgres.database }}
sslmode: disable
postgresVersion: 1600
secureJsonData:
# Resolved at runtime from the env var below so it always matches the
# password stored in the Secret (avoids re-deriving it here).
password: $POSTGRES_PASSWORD
isDefault: true
editable: true
provider.yaml: |
apiVersion: 1
providers:
- name: momentum
orgId: 1
type: file
disableDeletion: false
updateIntervalSeconds: 10
allowUiUpdates: true
options:
path: /etc/grafana/provisioning/dashboards
foldersFromFilesStructure: false
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "momentum.grafanaName" . }}-dashboards
labels:
{{- include "momentum.labels" . | nindent 4 }}
data:
momentum.json: |-
{{- .Files.Get "files/dashboard.json" | nindent 4 }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "momentum.grafanaName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
app.kubernetes.io/component: grafana
spec:
replicas: 1
selector:
matchLabels:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "grafana") | nindent 6 }}
template:
metadata:
labels:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "grafana") | nindent 8 }}
spec:
containers:
- name: grafana
image: {{ .Values.grafana.image | quote }}
ports:
- name: http
containerPort: 3000
env:
- name: GF_AUTH_ANONYMOUS_ENABLED
value: "true"
- name: GF_AUTH_ANONYMOUS_ORG_ROLE
value: "Admin"
- name: GF_AUTH_DISABLE_LOGIN_FORM
value: "true"
- name: GF_DASHBOARDS_MIN_REFRESH_INTERVAL
value: "2s"
# Interpolated into the provisioned datasource (secureJsonData.password).
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: {{ include "momentum.secretName" . }}
key: POSTGRES_PASSWORD
volumeMounts:
- name: datasources
mountPath: /etc/grafana/provisioning/datasources
- name: dashboard-provider
mountPath: /etc/grafana/provisioning/dashboards/provider.yaml
subPath: provider.yaml
- name: dashboards
mountPath: /etc/grafana/provisioning/dashboards/momentum.json
subPath: momentum.json
- name: storage
mountPath: /var/lib/grafana
resources:
{{- toYaml .Values.grafana.resources | nindent 12 }}
volumes:
- name: datasources
configMap:
name: {{ include "momentum.grafanaName" . }}-provisioning
items:
- key: datasource.yaml
path: datasource.yaml
- name: dashboard-provider
configMap:
name: {{ include "momentum.grafanaName" . }}-provisioning
items:
- key: provider.yaml
path: provider.yaml
- name: dashboards
configMap:
name: {{ include "momentum.grafanaName" . }}-dashboards
- name: storage
{{- if .Values.grafana.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ include "momentum.grafanaName" . }}
{{- else }}
emptyDir: {}
{{- end }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "momentum.grafanaName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
spec:
type: {{ .Values.grafana.service.type }}
selector:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "grafana") | nindent 4 }}
ports:
- name: http
port: {{ .Values.grafana.service.port }}
targetPort: http
{{- if .Values.grafana.persistence.enabled }}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ include "momentum.grafanaName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
spec:
accessModes: ["ReadWriteOnce"]
{{- if .Values.grafana.persistence.storageClass }}
storageClassName: {{ .Values.grafana.persistence.storageClass | quote }}
{{- end }}
resources:
requests:
storage: {{ .Values.grafana.persistence.size | quote }}
{{- end }}
{{- end }}
@@ -0,0 +1,31 @@
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "momentum.appName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.className }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- with .Values.ingress.tls }}
tls:
{{- toYaml . | nindent 4 }}
{{- end }}
rules:
- host: {{ .Values.ingress.host | quote }}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: {{ include "momentum.appName" . }}
port:
number: {{ .Values.service.port }}
{{- end }}
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "momentum.postgresName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
spec:
# Headless: stable DNS for the single StatefulSet pod.
clusterIP: None
selector:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "postgres") | nindent 4 }}
ports:
- name: postgres
port: {{ .Values.postgres.port }}
targetPort: postgres
@@ -0,0 +1,67 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "momentum.postgresName" . }}
labels:
{{- include "momentum.labels" . | nindent 4 }}
app.kubernetes.io/component: postgres
spec:
serviceName: {{ include "momentum.postgresName" . }}
replicas: 1
selector:
matchLabels:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "postgres") | nindent 6 }}
template:
metadata:
labels:
{{- include "momentum.selectorLabels" (dict "ctx" . "component" "postgres") | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: postgres
image: {{ .Values.postgres.image | quote }}
ports:
- name: postgres
containerPort: {{ .Values.postgres.port }}
env:
- name: POSTGRES_USER
value: {{ .Values.postgres.username | quote }}
- name: POSTGRES_DB
value: {{ .Values.postgres.database | quote }}
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: {{ include "momentum.secretName" . }}
key: POSTGRES_PASSWORD
# Keep PGDATA in a subdir so the volume root (lost+found) doesn't clash.
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
readinessProbe:
exec:
command: ["pg_isready", "-U", {{ .Values.postgres.username | quote }}, "-d", {{ .Values.postgres.database | quote }}]
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
exec:
command: ["pg_isready", "-U", {{ .Values.postgres.username | quote }}, "-d", {{ .Values.postgres.database | quote }}]
initialDelaySeconds: 30
periodSeconds: 15
resources:
{{- toYaml .Values.postgres.resources | nindent 12 }}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: ["{{ .Values.postgres.persistence.accessMode }}"]
{{- if .Values.postgres.persistence.storageClass }}
storageClassName: {{ .Values.postgres.persistence.storageClass | quote }}
{{- end }}
resources:
requests:
storage: {{ .Values.postgres.persistence.size | quote }}
@@ -0,0 +1,26 @@
{{- if not .Values.secrets.existingSecret }}
{{- $pgPassword := include "momentum.postgresPassword" . }}
{{- $pgHost := include "momentum.postgresName" . }}
{{- $databaseUrl := printf "postgresql://%s:%s@%s:%v/%s" .Values.postgres.username $pgPassword $pgHost .Values.postgres.port .Values.postgres.database }}
apiVersion: v1
kind: Secret
metadata:
name: {{ include "momentum.fullname" . }}-secrets
labels:
{{- include "momentum.labels" . | nindent 4 }}
type: Opaque
stringData:
# Consumed by the Postgres StatefulSet.
POSTGRES_PASSWORD: {{ $pgPassword | quote }}
# Consumed by the app + CronJobs (storage/db_pg.py prefers DATABASE_URL).
DATABASE_URL: {{ $databaseUrl | quote }}
# Alpaca paper broker (the client reads both the ALPACA_* and APCA_* names).
ALPACA_API_KEY: {{ .Values.secrets.alpacaApiKey | quote }}
ALPACA_SECRET_KEY: {{ .Values.secrets.alpacaSecretKey | quote }}
APCA_API_KEY_ID: {{ .Values.secrets.alpacaApiKey | quote }}
APCA_API_SECRET_KEY: {{ .Values.secrets.alpacaSecretKey | quote }}
APCA_API_BASE_URL: {{ .Values.secrets.alpacaBaseUrl | quote }}
{{- range $k, $v := .Values.secrets.schwab }}
{{ $k }}: {{ $v | quote }}
{{- end }}
{{- end }}
+162
View File
@@ -0,0 +1,162 @@
# Default values for the momentum chart.
# Override per-environment with `-f my-values.yaml` or `--set key=value`.
# -- Container image for every workload (app loop, dashboard, CronJobs).
image:
repository: ghcr.io/fez5587/momentum-system
# Pin to a pushed tag (commit SHA or `latest`) from the docker-publish workflow.
tag: latest
pullPolicy: IfNotPresent
# Optional image pull secrets for a private GHCR package.
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
# ----------------------------------------------------------------------------
# App: the live trading loop WITH its embedded dashboard (port 8010).
# It is a stateful singleton — do not raise replicas above 1.
# ----------------------------------------------------------------------------
app:
# Runs the loop with the dashboard attached (the dashboard's approve/reject/
# exit endpoints require the in-process TradingExecutionService).
command: ["python", "run_live_paper.py"]
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 1Gi
# Shared writable dir (DuckDB research file, logs, learned_params.json, tokens).
# The nightly-tune CronJob writes learned_params.json here for the loop to read.
persistence:
enabled: true
size: 5Gi
storageClass: ""
accessMode: ReadWriteOnce
# HTTP probes hit the dashboard root.
probes:
enabled: true
initialDelaySeconds: 30
periodSeconds: 30
# Dashboard service (ClusterIP by default; port-forward to reach it).
service:
type: ClusterIP
port: 8010
ingress:
enabled: false
className: ""
annotations: {}
host: momentum.local
tls: []
# ----------------------------------------------------------------------------
# In-cluster PostgreSQL — the single source of truth (StatefulSet + PVC).
# Schema auto-creates on first app connection from storage/pg_schema.sql.
# ----------------------------------------------------------------------------
postgres:
image: postgres:16
database: momentum
username: momentum
# If empty, a random password is generated on first install and stored in the
# chart-managed Secret. Set explicitly to keep it stable across upgrades.
password: ""
port: 5432
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 1Gi
persistence:
size: 10Gi
storageClass: ""
accessMode: ReadWriteOnce
# ----------------------------------------------------------------------------
# Secrets — DB URL is built from the in-cluster Postgres service automatically.
# Provide broker keys via --set or, better, an existingSecret (sealed-secrets,
# external-secrets, etc). Never commit real keys into values.yaml.
# ----------------------------------------------------------------------------
secrets:
# If set, the chart will NOT render its own Secret and instead expects this
# one to already contain DATABASE_URL, ALPACA_*/APCA_*, and any SCHWAB_* keys.
existingSecret: ""
alpacaApiKey: ""
alpacaSecretKey: ""
alpacaBaseUrl: "https://paper-api.alpaca.markets"
schwab: {} # e.g. SCHWAB_MARKET_DATA_APP_KEY: "..."
# ----------------------------------------------------------------------------
# Non-secret tuning env (ConfigMap). Mirrors .env.example; override freely.
# ----------------------------------------------------------------------------
config:
TRADING_EXECUTION_MODE: "alpaca_paper"
TRADING_EXECUTION_ENABLED: "1"
TRADING_AUTO_APPROVE: "0"
TRADING_MAX_CONCURRENT_POSITIONS: "6"
TRADING_RISK_PER_TRADE_PCT: "0.01"
TRADING_MAX_DAILY_LOSS_PCT: "0.03"
TRADING_EOD_FLATTEN_ENABLED: "1"
WATCHER_ENABLED: "1"
WATCHER_INTERVAL_SECONDS: "60"
WATCHER_SYMBOLS: "AAPL,TSLA,NVDA,AMD,SOFI"
WATCHER_PRICE_MIN: "1"
WATCHER_PRICE_MAX: "20"
STRATEGY_SETUPS: "opening_range_break"
LIVE_BARS_ENABLED: "1"
LIVE_BARS_INTERVAL_SECONDS: "60"
ALPACA_DATA_FEED: "iex"
ALPACA_BACKFILL_FEED: "sip"
TRIGGER_ENABLED: "1"
TRIGGER_GAP_MIN: "3.0"
TRIGGER_GAP_MAX: "35"
TRIGGER_RVOL_MIN: "2.0"
ORB_BARS: "5"
DASHBOARD_HOST: "0.0.0.0"
DASHBOARD_PORT: "8010"
# Leave schema auto-creation on (storage/db_pg.py applies pg_schema.sql).
PG_SKIP_SCHEMA: "0"
# ----------------------------------------------------------------------------
# Scheduled jobs (CronJobs). Times are US/Eastern via spec.timeZone (K8s >=1.27),
# weekdays only — matching the existing *.sh crontab comments.
# ----------------------------------------------------------------------------
cronjobs:
enabled: true
timeZone: America/New_York
journal:
enabled: true
schedule: "2 16 * * 1-5"
command: ["python", "momentum_cli.py", "journal"]
eodReplay:
enabled: true
schedule: "5 16 * * 1-5"
command: ["python", "eod_replay.py"]
nightlyTune:
enabled: true
schedule: "30 17 * * 1-5"
command: ["python", "nightly_tune.py"]
# ----------------------------------------------------------------------------
# Grafana (optional) — provisioned against the in-cluster Postgres service.
# ----------------------------------------------------------------------------
grafana:
enabled: true
image: grafana/grafana:11.2.0
service:
type: ClusterIP
port: 3000
persistence:
enabled: true
size: 1Gi
storageClass: ""
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
memory: 512Mi
+30
View File
@@ -0,0 +1,30 @@
# Local deployment (systemd user services)
Unit templates for running the stack from a git checkout + venv (the original
production shape; now the development shape). Paths inside the units assume
`/mnt/c/code/momentum` + `/home/philip/.venvs/momentum` — adjust to your box.
```bash
cp deploy/systemd/momentum-*.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now momentum-loop momentum-dashboard momentum-qabot momentum-transcripts
systemctl --user enable --now momentum-premarket-eval.timer # 20:00 ET weekdays
loginctl enable-linger $USER # start on boot without login
```
| Unit | What |
|---|---|
| momentum-loop | trading loop (`run_live_paper.py --no-dashboard`) |
| momentum-dashboard | web dashboard :8765 (`dashboard_api.py`) |
| momentum-qabot | Telegram Q&A bot (read-only) |
| momentum-transcripts | YouTube → transcript → claims pipeline |
| momentum-premarket-eval(.timer) | nightly SIP refetch + housekeeping + shadow-track re-validation |
| momentum-testpg | local Postgres for test isolation (`/home/philip/pgrun` socket) — keep even when production is elsewhere |
| momentum-controlroom | NiceGUI control room :8766 — **disabled by owner preference**; kept for reference |
Config comes from `.env` in the repo root (see `.env.example`). The same
behavior-flag warnings as compose apply (`TRADING_AUTO_APPROVE=1` etc. — see
`deploy/README.md`).
**Never run these enabled while the compose stack owns production** — two loops
on one Alpaca account duplicate orders.
+21
View File
@@ -0,0 +1,21 @@
[Unit]
Description=Momentum dashboard (read-only web UI on :8765)
# Self-heal forever: a DB blip or crash must not leave the website down. Decoupled
# from the loop on purpose -- a loop restart never takes the dashboard down and vice versa.
StartLimitIntervalSec=0
[Service]
Type=simple
WorkingDirectory=/mnt/c/code/momentum
# dashboard_api.py loads .env (DATABASE_URL -> the LAN Postgres, with the host fallback)
# and binds DASHBOARD_HOST (default 0.0.0.0) : DASHBOARD_PORT (.env=8765) so it is reachable
# from the Windows browser via the netsh portproxy.
ExecStart=/home/philip/.venvs/momentum/bin/python -u dashboard_api.py
Restart=always
RestartSec=15
StandardOutput=append:/mnt/c/code/momentum/data/dashboard.log
StandardError=append:/mnt/c/code/momentum/data/dashboard.log
SyslogIdentifier=momentum-dashboard
[Install]
WantedBy=default.target
+24
View File
@@ -0,0 +1,24 @@
[Unit]
Description=Momentum live paper-trading loop
# Self-heal: if the loop crashes or the box reboots, systemd brings it back.
# The DB host (192.168.1.5) can go fully unreachable (a network/host outage) and
# the loop crashes on connect — so retry FOREVER (no give-up limit) and it
# auto-reconnects the moment the host returns. StartLimitIntervalSec=0 disables
# the "give up after N starts" cap entirely.
StartLimitIntervalSec=0
[Service]
Type=simple
WorkingDirectory=/mnt/c/code/momentum
# the loop calls load_dotenv() -> finds .env in WorkingDirectory -> DATABASE_URL
# and all secrets load from there; no secrets in this unit file.
ExecStart=/home/philip/.venvs/momentum/bin/python -u run_live_paper.py --no-dashboard
Restart=always
RestartSec=30
# keep the same log file the operator + the review crons already tail
StandardOutput=append:/mnt/c/code/momentum/data/live_run.log
StandardError=append:/mnt/c/code/momentum/data/live_run.log
SyslogIdentifier=momentum-loop
[Install]
WantedBy=default.target
@@ -0,0 +1,10 @@
[Unit]
Description=Momentum premarket (pm1) shadow track rebuild + re-validate
[Service]
Type=oneshot
WorkingDirectory=/mnt/c/code/momentum
# DATABASE_URL loads from the repo's gitignored .env via research.premarket_eval (python-dotenv)
ExecStart=/home/philip/.venvs/momentum/bin/python -m research.premarket_eval
StandardOutput=append:/mnt/c/code/momentum/data/premarket_eval.log
StandardError=append:/mnt/c/code/momentum/data/premarket_eval.log
@@ -0,0 +1,10 @@
[Unit]
Description=Weekday-evening rebuild + re-validate of the premarket (pm1) shadow track
[Timer]
# after each trading day's premarket data is ingested; Persistent catches missed runs (reboots)
OnCalendar=Mon-Fri *-*-* 20:00:00
Persistent=true
[Install]
WantedBy=timers.target
+17
View File
@@ -0,0 +1,17 @@
[Unit]
Description=Momentum Telegram Q&A bot
# retry forever through a DB-host outage (see momentum-loop.service)
StartLimitIntervalSec=0
[Service]
Type=simple
WorkingDirectory=/mnt/c/code/momentum
ExecStart=/home/philip/.venvs/momentum/bin/python -u telegram_bot.py
Restart=always
RestartSec=30
StandardOutput=append:/mnt/c/code/momentum/data/telegram_bot.log
StandardError=append:/mnt/c/code/momentum/data/telegram_bot.log
SyslogIdentifier=momentum-qabot
[Install]
WantedBy=default.target
+11
View File
@@ -0,0 +1,11 @@
[Unit]
Description=Local Postgres for momentum test isolation (socket /home/philip/pgrun)
[Service]
Type=forking
ExecStart=/home/philip/pg/bin/pg_ctl -D /home/philip/pgdata -o "-k /home/philip/pgrun" -l /home/philip/pgdata/logfile start
ExecStop=/home/philip/pg/bin/pg_ctl -D /home/philip/pgdata stop -m fast
Restart=on-failure
[Install]
WantedBy=default.target
@@ -0,0 +1,19 @@
[Unit]
Description=Momentum YouTube transcript -> claim-extraction pipeline
StartLimitIntervalSec=0
[Service]
Type=simple
WorkingDirectory=/mnt/c/code/momentum
# secrets (DATABASE_URL, YOUTUBE_API_KEY, YT_WATCHLIST) load from the repo's gitignored .env
# via youtube_claims.config (python-dotenv). Whisper is option A (CPU) by default; flip to the
# LAN GPU (option B) with YT_WHISPER_MODEL=large-v3 YT_WHISPER_DEVICE=cuda YT_WHISPER_COMPUTE_TYPE=float16.
ExecStart=/home/philip/.venvs/momentum/bin/python -u -m youtube_claims.run run
Restart=always
RestartSec=30
StandardOutput=append:/mnt/c/code/momentum/data/transcripts_pipeline.log
StandardError=append:/mnt/c/code/momentum/data/transcripts_pipeline.log
SyslogIdentifier=momentum-transcripts
[Install]
WantedBy=default.target
+282
View File
@@ -0,0 +1,282 @@
# Full self-contained stack for the Momentum System.
#
# docker compose up -d # postgres + loop + dashboard + qabot + transcripts + scheduler
# docker compose --profile monitoring up -d # also start Grafana
# docker compose run --rm nightly-tune # run a job on demand
#
# Secrets come from .env (never baked into the image). DATABASE_URL is
# overridden below to the in-stack postgres so the stack is self-contained —
# migrating from the LAN Postgres (192.168.1.5) is a pg_dump/restore into the
# pg-data volume; see deploy/docker/README.md. To run the app services against
# the LAN Postgres instead (transition mode), comment out the DATABASE_URL
# override and let .env's value through.
#
# TZ is America/New_York in the image: the loop's market clock thinks in ET.
x-app: &app
build: .
image: momentum-app:latest
env_file:
- path: .env
required: false
volumes:
- app-data:/app/data
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
logging:
driver: json-file
options: { max-size: "20m", max-file: "3" }
x-app-env: &app-env
# Self-contained default: the in-stack postgres. Overrides .env.
DATABASE_URL: postgresql://${POSTGRES_USER:-momentum}:${POSTGRES_PASSWORD:-momentum}@postgres:5432/${POSTGRES_DB:-momentum}
PG_SKIP_SCHEMA: "0"
LIVE_BARS_INTERVAL_SECONDS: "5"
TRIGGER_ARM_INTERVAL_SECONDS: "8"
WATCHER_INTERVAL_SECONDS: "8"
TRIGGER_INTERVAL_SECONDS: "2"
# catalyst enrichment ON as a labeler FEATURE source (plan 009) — loop + scheduler
# (labeler builds) both inherit this anchor. Needs a reachable Ollama (see .env);
# enabled-but-down never impedes trading (fully gated).
NEWS_ENRICH_ENABLED: "1"
# Managed-exit config — matches deploy/docker/truenas-compose.yml so both
# templates agree. Without these the loop runs code defaults (target 2R,
# BE@+5%, NO trail) which scratches slow v1 winners at breakeven. prior_low
# trailing after +1R lifts the win rate (counterfactual: v1 17%->24%). Only the
# LOOP acts on these (ExitConfig.from_env, run_live_paper.py); other services
# inherit them harmlessly.
TRADING_EXIT_TRAIL_MODE: "prior_low"
TRADING_EXIT_TRAIL_AFTER_R: "1.0"
TRADING_EXIT_MANAGE_INTERVAL_SECONDS: '4' # 12->4s: react to fast small-cap round-trips (was giving back gains)
TRADING_EXIT_THREAD_ENABLED: '1'
DISCOVER_THREAD_ENABLED: '1'
TRADING_EXIT_THREAD_INTERVAL_SECONDS: '1.0'
TRADING_EXIT_TRAIL_R_STEP: "0.25"
TRADING_EXIT_BREAKEVEN_R: "1.0"
TRADING_EXIT_PROFIT_TIERS: "8:3,15:9,25:18,40:30"
# Regime gate (plan 011) LIVE: block fresh entries on sessions the walk-forward
# metalabel scores unfavorable (sit out bad regimes). Only the LOOP acts on it;
# fail-open (unfit model / error never blocks). Needs regime_features populated
# (the scheduler builds them nightly). Set 0 to disable.
TRADING_REGIME_GATE_ENABLED: "1"
# Favorable threshold: block a session when P(profit) < this. 0.5 = code default
# (matches the backtest). Lower (e.g. 0.4) trades MORE days; raise for fewer,
# higher-confidence days. Optional — omit to use the 0.5 default.
TRADING_REGIME_GATE_P_MIN: "0.5"
services:
postgres:
image: postgres:18
environment:
POSTGRES_USER: ${POSTGRES_USER:-momentum}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-momentum}
POSTGRES_DB: ${POSTGRES_DB:-momentum}
PGDATA: /var/lib/postgresql/data/pgdata
volumes:
- pg-data:/var/lib/postgresql/data
ports:
- "5432:5432" # psql/Grafana-on-host debugging; remove if not needed
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-momentum} -d ${POSTGRES_DB:-momentum}"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
loop:
<<: *app
container_name: momentum-loop
command: ["python", "-u", "run_live_paper.py", "--no-dashboard"]
environment:
<<: *app-env
healthcheck:
# heartbeat: the loop must have written SOME event in the last 5 minutes
# (module ticks flow around the clock, not just market hours)
# PG_SKIP_SCHEMA=1: a healthcheck must NOT re-apply schema DDL — the 60s
# CREATE-IF-NOT-EXISTS lock burst cancelled minute_bars autovacuum forever
# (2026-07-09 post-migration: stats rotted, dashboard queries crawled)
test: ["CMD", "python", "-c", "import os; os.environ['PG_SKIP_SCHEMA']='1'; import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)"]
interval: 60s
timeout: 20s
retries: 5
start_period: 180s
dashboard:
<<: *app
container_name: momentum-dashboard
command: ["python", "-u", "dashboard_api.py"]
environment:
<<: *app-env
DASHBOARD_HOST: "0.0.0.0"
DASHBOARD_PORT: "8765"
# read-only viewer: never re-apply the 473-line schema (the SSE change
# detector holds one long-lived connection) — mirrors the loop healthcheck.
PG_SKIP_SCHEMA: "1"
# Shared secret required on every POST (approve/reject/exit + the command
# bus). Empty = commands disabled and POSTs unauthenticated; set a value to
# arm the kill switch. Reference by name only — never inline the secret.
DASHBOARD_TOKEN: "${DASHBOARD_TOKEN:-}"
# Gate the operator command bus on (also needs DASHBOARD_TOKEN set).
COMMAND_BUS_ENABLED: "${COMMAND_BUS_ENABLED:-0}"
ports:
- "8765:8765"
healthcheck:
test: ["CMD", "curl", "-sf", "http://localhost:8765/"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
qabot:
<<: *app
container_name: momentum-qabot
command: ["python", "-u", "telegram_bot.py"]
environment:
<<: *app-env
transcripts:
<<: *app
container_name: momentum-transcripts
command: ["python", "-u", "-m", "youtube_claims.run", "run"]
environment:
<<: *app-env
admin:
<<: *app
container_name: momentum-admin
command: ["python", "-u", "admin_api.py"]
environment:
<<: *app-env
ADMIN_PORT: "8767"
# set ADMIN_TOKEN in .env — without it the panel is open on the LAN
ports:
- "8767:8767"
volumes:
- app-data:/app/data
- /var/run/docker.sock:/var/run/docker.sock # service status + restarts
scheduler:
# Weekday-evening premarket eval + DB/log housekeeping at 20:00 ET —
# replaces the momentum-premarket-eval systemd timer.
<<: *app
container_name: momentum-scheduler
environment:
<<: *app-env
command:
- /bin/sh
- -c
- |
echo "[scheduler] up; will fire 20:00 ET Mon-Fri"
while true; do
if [ "$$(date +%H:%M)" = "20:00" ] && [ "$$(date +%u)" -le 5 ]; then
echo "[scheduler] premarket_eval $$(date)"
python -m research.premarket_eval || echo "[scheduler] eval FAILED"
# SPY/QQQ ingest -> forward source for the labeler's market-context features (plan 009)
python -m research.ingestion.market_context || echo "[scheduler] market_context ingest FAILED"
python -m research.eod_reconcile || echo "[scheduler] reconcile FAILED"
# Flywheel: grow the whole-market dataset first, then fit the tuner on
# it. 20:00 ET is after close, so replay of "today" is complete data —
# NOT look-ahead (replay is a labeling job, not a live gate).
python eod_replay.py || echo "[scheduler] eod_replay FAILED"
python nightly_tune.py || echo "[scheduler] nightly_tune FAILED"
# Regime gate (plan 011, ALL-SHADOW): a session-level metalabel
# P(our ORB profits today). build-features labels each session from its
# OWN realized expR; score writes the WALK-FORWARD favorable flag the
# experiments' regime_gate intervention reads. Must run BEFORE
# experiments run. Does NOT gate live trading (that is owner-gated).
python -m research.regime build-features || echo "[scheduler] regime build-features FAILED"
python -m research.regime score --model threshold_v1 || echo "[scheduler] regime score threshold FAILED"
python -m research.regime score --model meta_v1 || echo "[scheduler] regime score meta FAILED"
# Experiment OS (plan 006, ALL-SHADOW): replay running experiments over
# newly-logged fires, then adjudicate against pre-registered bars.
python -m research.experiments run || echo "[scheduler] experiments run FAILED"
python -m research.experiments adjudicate || echo "[scheduler] experiments adjudicate FAILED"
sleep 120
fi
sleep 20
done
db-backup:
# Nightly pg_dump at 20:10 ET Mon-Fri (after the 20:00 scheduler pass).
# Runs on the postgres:18 image — a version-matched pg_dump for free; the
# app image has none. Backups land in the app-data volume under backups/.
image: postgres:18
container_name: momentum-db-backup
restart: unless-stopped
environment:
# in-stack postgres, same value the app services use (NOT .env's
# DATABASE_URL, which may point at the old LAN instance)
DATABASE_URL: postgresql://${POSTGRES_USER:-momentum}:${POSTGRES_PASSWORD:-momentum}@postgres:5432/${POSTGRES_DB:-momentum}
BACKUP_DIR: /data/backups
BACKUP_KEEP_DAYS: "14"
TZ: America/New_York # the postgres image defaults to UTC; the 20:10 gate is ET
volumes:
- ./deploy/docker/backup_db.sh:/backup_db.sh:ro
- app-data:/data
depends_on:
postgres:
condition: service_healthy
command:
- /bin/bash
- -c
- |
echo "[backup] up; fires 20:10 ET Mon-Fri"
while true; do
if [ "$$(date +%H:%M)" = "20:10" ] && [ "$$(date +%u)" -le 5 ]; then
bash /backup_db.sh || echo "[backup] FAILED"
sleep 120
fi
sleep 20
done
# ---- On-demand jobs: docker compose run --rm <name>
journal:
<<: *app
profiles: ["jobs"]
command: ["python", "momentum_cli.py", "journal"]
environment: *app-env
restart: "no"
eod-replay:
<<: *app
profiles: ["jobs"]
command: ["python", "eod_replay.py"]
environment: *app-env
restart: "no"
nightly-tune:
<<: *app
profiles: ["jobs"]
command: ["python", "nightly_tune.py"]
environment: *app-env
restart: "no"
# ---- Grafana (optional): docker compose --profile monitoring up -d
grafana:
image: grafana/grafana:11.2.0
profiles: ["monitoring"]
environment:
GF_AUTH_ANONYMOUS_ENABLED: "true"
GF_AUTH_ANONYMOUS_ORG_ROLE: "Admin"
GF_AUTH_DISABLE_LOGIN_FORM: "true"
GF_DASHBOARDS_MIN_REFRESH_INTERVAL: "2s"
PG_USER: ${POSTGRES_USER:-momentum}
PG_PASSWORD: ${POSTGRES_PASSWORD:-momentum}
PG_DATABASE: ${POSTGRES_DB:-momentum}
ports:
- "3000:3000"
volumes:
- ./deploy/compose/grafana/datasources:/etc/grafana/provisioning/datasources:ro
- ./deploy/compose/grafana/dashboards:/etc/grafana/provisioning/dashboards:ro
- grafana-data:/var/lib/grafana
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
volumes:
pg-data:
app-data:
grafana-data:
+110
View File
@@ -0,0 +1,110 @@
# Live-Money Cutover Runbook
Staged 2026-07-22. Everything below is pre-built; flip day is configuration,
not construction. Target: minimal downtime, zero blending of paper and live
records.
## Architecture decision
Real money runs as a **second, fully isolated compose stack** ("momentum-live"):
own Postgres, own volumes, own dashboard. Rationale: the ~30 dashboard/P&L
projections do not filter by event mode — one missed filter would silently
blend money records. A separate database makes blending impossible by
construction. The paper stack stays untouched as the research substrate
(nightly labeler, experiments, backtests, regime scoring).
## What is already staged
| Piece | Where | State |
|---|---|---|
| Live compose (paper keys, current settings) | `deploy/docker/truenas-compose.live.local.yaml` (LOCAL, secrets, never commit) | ready; flip checklist in its header |
| DB seed script | `deploy/docker/bootstrap_live_db.sh` | ready |
| Equity-floor kill switch | `TRADING_EQUITY_FLOOR` (commit 5287c8e) | in image, 0=off |
| Boot order reconciliation | `TRADING_BOOT_RECONCILE` (default on, 5287c8e) | in image |
| Stale-session dashboard alarm | commit 2e39b75/5f33799 | in image |
## Known dependencies (discovered during staging)
1. **One paper account cannot serve two stacks.** Both composes currently
carry the same paper keys. Running both would double-trade one account AND
each stack's boot-reconcile would cancel the other's resting orders as
orphans. Before first boot: create a second Alpaca paper key pair for the
live stack's shakedown (or pause the paper stack's loop during it).
2. **Fresh DB must be seeded, not born empty.** Empty DB = rvol baselines
missing (~1 day to self-heal) and, worse, the regime gate fail-opens for
weeks (no premarket history) — the one validated protection would be inert
exactly at go-live. `bootstrap_live_db.sh` seeds from the paper stack's
nightly dump and truncates the money-record tables (`events`,
`telemetry_events`, simulated/paper tables), so research history carries
over while live P&L starts at zero. The dump also carries every index.
3. **`data/learned_params.json`** — copy from the paper stack's data dir into
`momentum-live/data/` (nightly tune runs only on the paper stack).
4. **Telegram** — both stacks share one chat; create a second bot token for
the live stack (`TELEGRAM_BOT_TOKEN` in its compose) or alerts interleave
indistinguishably.
5. Host prerequisites: dirs `/mnt/Volume/Storage/Apps/momentum-live/data/backups`
created; host ports **8865** (dashboard) and **5434** (postgres) free.
## Config model (added 2026-07-22)
Strategy knobs live in the **`config_overrides` database table** (commit
495a3fa), edited via the admin panel (paper :8767, live :8867) with a
whitelist (`TRADING_*`, `TRIGGER_*`, `WATCHER_*`, `NEWS_*`, `OLLAMA_*`,
`DISCOVER_*`; secrets refused). Boot precedence: compose env <
`data/overrides.env` file < database — every service applies the table last;
DB down falls through silently so boot never blocks on config. Compose and
the overrides file retain only infra + secrets (keys, URLs, DB address — the
DB address cannot live inside the DB). One-shot migration inside any stack
container:
python -m runtime.env_overrides seed-from-env
copies that container's effective knobs into the table, after which the
compose knob lines are dead weight (strip at any redeploy). For the live
stack: run the seed after `bootstrap_live_db.sh` (the seeded table travels
with the DB restore otherwise — which is fine too, since the paper dump's
table carries the same knobs; just re-check flip-day values). Flip-day knob
changes (risk, regime gate, equity floor) happen in the admin UI; only keys +
ALPACA_BASE_URL remain compose edits (secret-marked, deliberately not
UI-editable).
## Cutover procedure (~15 min, paper stack keeps running throughout)
1. `mkdir -p /mnt/Volume/Storage/Apps/momentum-live/data/backups`
2. Add the TrueNAS app from `truenas-compose.live.local.yaml`; start ONLY the
postgres service (or start all and let the loop crash-loop until step 3 —
harmless, `restart: unless-stopped`).
3. Seed: `./bootstrap_live_db.sh <latest paper backup> <live-pg-container>`
(paper backups land nightly 20:10 ET in
`/mnt/Volume/Storage/Apps/momentum/data/backups/`).
4. `cp .../momentum/data/learned_params.json .../momentum-live/data/` and
`cp deploy/docker/overrides.live.seed.env .../momentum-live/data/overrides.env`
5. Shakedown on the SECOND paper key pair for 1-2 sessions: watch the 8865
dashboard — armed triggers populate, entries fire, no stale-session banner,
boot-reconcile quiet.
6. **The flip** (edit live compose env, both loop + dashboard, then redeploy):
- `ALPACA_BASE_URL: https://api.alpaca.markets`
- `ALPACA_API_KEY` / `ALPACA_SECRET_KEY`: live keys
- `TRADING_RISK_PER_TRADE_PCT: '0.0025'` (start tiny)
- `TRADING_MAX_RISK_DOLLARS: '250'` (hard ceiling back on)
- `TRADING_REGIME_GATE_ENABLED: '1'` (the one validated lever)
- `TRADING_EQUITY_FLOOR:` 0.90 x starting equity (absolute dollars)
7. Verify within the first hour: dashboard LIVE badge, one order's audit trail
end-to-end, `python -m research.slippage --since <today>` after the first
close, Telegram alert received.
## Rollback
Point the live compose back at paper keys/URL and redeploy — the stack itself
is the same code either way. The paper stack was never touched. If the live DB
is ever suspect, it is disposable pre-flip: destroy pgdata, re-run the seed.
## Notes
- FINRA PDT rule retired June 4 2026 (Alpaca Intraday Margin Framework) — no
$25k floor; standard margin minimums apply.
- Measured fill reality on gate-passing names: 0.3-0.5% median spread, ~12%
profit drag vs paper; p90 tail 2.4-3.6% and book-walks are the real risks
(`research/slippage.py` is the standing report).
- Gate for the flip: 2-3 consecutive green regime-gated paper weeks on the
fixed stack (week of 07-20: +$1,732, 8-for-8, regime OFF/watch-mode).
+80
View File
@@ -0,0 +1,80 @@
# Tradier broker fallback
Redundancy path if the Alpaca live account stays blocked. `tradier_broker/`
mirrors every trading call the live loop makes on `AlpacaPaperClient` — same
method names, signatures, and Alpaca-shaped return dicts — so
`trading_execution`, `runtime/exit_manager`, `runtime/flatten`,
`alpaca_paper.execution` and `alpaca_paper.sync` run unchanged.
## Switch (config only)
```
BROKER=tradier
TRADIER_ACCESS_TOKEN=... # sandbox or live token
TRADIER_ACCOUNT_ID=... # e.g. VA71549758
TRADIER_ENV=sandbox # 'live' for real money — explicit opt-in
```
`BROKER` unset/`alpaca` = today's behavior, byte-for-byte. Alpaca keys stay
required either way: **market data (bars, screeners, quotes, news) always
stays on Alpaca** — the Tradier client only replaces order execution and
account/position state, delegating all data calls to the Alpaca client.
## Verify before trusting
```
python3 -m tradier_broker.smoke # balances/clock/positions/orders
python3 -m tradier_broker.smoke --preview AAPL # broker-validates bracket form, never submits
python3 -m tradier_broker.smoke --place-cancel AAPL # full submit -> tag lookup -> cancel
```
Smoke refuses order exercises against the live base URL.
## Mapping notes
- Alpaca bracket -> Tradier OTOCO (entry + OCO take-profit limit / stop);
oto -> OTO; standalone gtc stop sells map directly.
- `client_order_id` -> Tradier `tag` (lost-submit reconcile works via tag lookup).
- Tradier positions carry no mark; `get_positions` prices them off
`/markets/quotes` (sandbox quotes are 15-min delayed — fine for redundancy
checks, and live tokens get real-time).
- `replace_order` maps to PUT (moves stop/limit without cancelling — OCO stays
intact); Tradier cannot change qty on a working order (raises, cancel+resubmit).
- `get_orders` is session-scoped at Tradier (today's orders) and status/symbol
filters are client-side; the live loop only queries today + held symbols, so
behavior matches.
## Parallel shakeout stack (TrueNAS)
`deploy/docker/truenas-compose.tradier.yaml` is the committed template; the
filled copy lives (git-excluded) at `truenas-compose.tradier.local.yaml`.
Runs beside the paper stack, fully isolated:
- own postgres (host **5434**, fresh `pgdata-tradier` volume), own dashboard
(host **8768**), loop with `BROKER=tradier` against the sandbox account
- market data still Alpaca; Telegram + deadman OFF (main stack keeps alerts)
- main data dir mounted **read-only** — inherits nightly
`learned_params.json` / `overrides.env`, cannot clobber them; if a service
ever crashes on a read-only-filesystem error, switch that mount to a
writable copy
- fresh DB starts empty (knobs at code defaults). For research-baseline +
knob parity with the paper stack, reuse the live-staging playbook:
`./bootstrap_live_db.sh <nightly-backup.sql.gz> <tradier-postgres-container>`
- fresh DB also lacks the research tables the dashboard polls
(`experiment_verdicts` etc. — postgres logs `relation ... does not exist`
on every poll). Harmless, but either bootstrap as above or create the
missing tables schema-only; done by hand on the first deploy 2026-07-27
(18 tables), plus a copy of the paper stack's `config_overrides` rows
(75 knobs — without them the stack runs factory defaults and entries
pile up in the manual approval queue)
- needs an image built from >= 1cefb51 (`tradier_broker/` in image)
- sandbox fills are simulated and can be slow or absent — an order parked in
`pending_new` is usually the sandbox, not the code
## Before flipping live money to Tradier
1. Rotate the sandbox token (it transited chat) and mint a live token.
2. Run a full paper week on sandbox with `BROKER=tradier` in the paper stack.
3. Re-run the naked-stop drill from docs/LIVE-CUTOVER.md — leg-stripping
semantics differ per broker; verify a filled OTOCO's stop leg survives an
entry-order cancel attempt.
+61
View File
@@ -0,0 +1,61 @@
# Tradier Real-Money Flip — delta to LIVE-CUTOVER.md
LIVE-CUTOVER.md was written for Alpaca live (rejected 2026-07-27). Plan of
record is now Tradier real money on the EXISTING tradier parallel stack
(postgres 5434, dashboard 8768). This doc is only the delta.
## The flip (3 env lines in truenas-compose.tradier.local.yaml, loop + dashboard)
TRADIER_ENV: live # switches base URL to api.tradier.com
TRADIER_ACCESS_TOKEN: <live token> # from the REAL account, never the sandbox token
TRADIER_ACCOUNT_ID: <live account id>
Market data stays on Alpaca (unchanged). Rollback = set `TRADIER_ENV: sandbox`
+ sandbox creds, redeploy.
## Before the flip — owner tasks (nobody else can do these)
1. Open a Tradier **cash** account (margin needs $2k; $300 = cash), fund $300.
2. Create an API token on the live account. Treat as secret: compose file
only, never the repo, never the config_overrides table.
3. Confirm account approvals: API access enabled on the live account
(Tradier sometimes gates API on request).
## Before the flip — config (admin UI :8768 or config_overrides on 5434)
| Knob | Value | Why |
|---|---|---|
| `TRADING_MAX_FRESH_ENTRIES_PER_DAY` | `1` | **Cash account T+1: >1 round trip/day risks good-faith violations; 3 GFVs = 90-day freeze.** Raise only with equity growth or margin. |
| `TRADING_EQUITY_FLOOR` | `270` | Kill switch at 0.90 x $300. |
| `TRADING_MAX_RISK_DOLLARS` | `1000` (keep) | Inert at this equity (16% notional cap = ~$48 positions binds first); harmless. |
| `TRADING_RISK_PER_TRADE_PCT` | `0.05` (keep) | $15 budget; the $48 position cap means actual risk/trade lands ~$2-8. |
| `TRADING_REGIME_GATE_ENABLED` | `1` | The one lever that survived OOS validation (plan 011); flip-day requirement per LIVE-CUTOVER. |
## Before the flip — DB hygiene
The tradier DB currently holds SANDBOX trade records. Live must start clean
(cutover principle: no blending). Before first live boot:
TRUNCATE events, telemetry_events; -- on 5434; keep research tables
(Same pattern as bootstrap_live_db.sh. minute_bars/daily_bars/volume_ratios
stay — that history is real market data and feeds rvol baselines.)
## Day-1 verification (first hour)
1. Dashboard badge shows tradier-LIVE (factory label), not sandbox.
2. First fill: check the bracket legs exist at Tradier (stop visible via
by-symbol order lookup) — the 2026-07-28 incident class.
3. **Naked-stop drill (memory: required pre-flip, still undone):** with a
1-share position open, manually cancel its stop leg in the Tradier UI;
the guard must force-exit within 2 passes. This is the drill on real
plumbing with pocket-change size.
4. Telegram alert received; `python -m research.slippage --since <today>`
after close — real-fill slippage vs the 0.3-0.5% paper-measured spread.
## Expectations at $300
Positions ~$48 (16% cap), 1-20 shares on $2-20 names. P&L per trade: single
dollars. This proves execution (fills, brackets, stops, GFV discipline), not
income. Sizing scales automatically as equity grows — the knobs above are the
only things to revisit.
+106
View File
@@ -0,0 +1,106 @@
# UI Review — Momentum Dashboard (`api/static/dashboard.html`)
**Audited**: 2026-07-20 · retroactive 6-pillar visual audit (no UI-SPEC exists →
audited against abstract standards). Adversarial stance: assume failure until
the code proves otherwise. Scores are 1–4, not averaged upward.
**Scope**: the primary operator dashboard (`dashboard.html`, ~1600 lines,
single-file inline CSS/JS) plus its siblings `backtest.html` / `admin.html`.
Method: static analysis of color/spacing/typography tokens, structure, and
interaction states (the auditor's grep-based method; a headless render pass is
the standing follow-up).
## Scorecard
| Pillar | Score | One-line verdict |
|---|---|---|
| 1. Copywriting | **4** | Domain-precise, explains *why* — not generic |
| 2. Visuals / hierarchy | **3** | Clear strip→panel structure; dense but legible |
| 3. Color | **3** | Real token system, undermined by duplicate semantic colors + 90 hardcoded values |
| 4. Typography | **3** | Sensible mono scale; 9px floor is too small |
| 5. Spacing | **2** | No systematic scale — 20 distinct px values incl. arbitrary 7/9/11/13 |
| 6. Experience design | **4** | Standout — confirm-to-destruct, full empty/loading/error/connection states |
**Overall**: a genuinely competent, purpose-built operator UI. The failures are
consistency-and-accessibility drift, not structural — this is a good dashboard
with three fixable rough edges, not a rebuild.
## Findings
### Pillar 1 — Copywriting · 4
Labels state intent, not just nouns: "closest to fire on top", "why each entry
was taken or refused · value → threshold per gate", "held long with no / short
protective stop". Empty states teach ("No armed gappers — waiting for the open").
No lorem, no vague "Loading…" without context. **No fix needed.**
### Pillar 2 — Visuals / hierarchy · 3
Strong top-down hierarchy: risk-strip → P&L-strip → collapsible `<details>`
panels, colour-coded side pills, sparklines, R-bars. Weakness: extreme density
(10–11px body across 51 elements) with little whitespace — correct for a trading
terminal, but the eye has few rest points. **WARNING** — acceptable for the
audience; revisit if non-expert users are ever a target.
### Pillar 3 — Color · 3
Real foundation: 17 CSS custom properties, 149 `var(--…)` uses, coherent dark
theme, defensible 60/30/10 (bg dominant / panels secondary / `#5ba3ff` accent on
interactive). **But** two concrete defects:
- **Duplicate semantic colors** — THREE greens (`#2fbf71`, `#2ecc71`, `#34d17e`)
and FOUR reds (`#e5484d`, `#ff4d4f`, `#ff5a60`, `#ff8589`) all meaning
"up/good" and "down/bad". These should be one `--green` / `--red` (+ one
intentional bright variant), not near-identical one-offs.
- **90 hardcoded hex values** bypass the token system that already exists.
**WARNING** — the system is right; application drifted.
### Pillar 4 — Typography · 3
One monospace family (correct for aligned numeric data), a real size ladder
(9→27px), weight used for hierarchy. Defect: the **9px floor** (4 elements) is
below comfortable-read and fails WCAG minimums; 10–11px carries most of the UI,
which is tight. **WARNING** — raise the 9px labels to ≥11px.
### Pillar 5 — Spacing · 2 (lowest)
**20 distinct padding/margin values** including arbitrary `7px`, `9px`, `11px`,
`13px` with no discernible 4/8 rhythm. Dense layout justifies *tightness*, not
*arbitrariness* — the values look hand-tuned per element rather than drawn from a
scale. **WARNING** — adopt a spacing scale (e.g. 2/4/8/12/16/24) and snap values
to it; the visual result changes little but future edits stay consistent.
### Pillar 6 — Experience design · 4 (standout)
This is where the dashboard earns its keep:
- **Confirm-to-destruct**: EXIT/FLATTEN/PAUSE use an arm-then-confirm pattern
(`armButton`, 8 sites) — no accidental market flatten.
- **Full state coverage**: 38 loading/loader refs, 20 explicit empty states, 7
error/connection-lost handlers ("stream lost — polling", "render error").
- **Resilience**: SSE with polling fallback, degrade-gracefully panels.
- **3 `@media` breakpoints** — responsive to phone/tablet.
**No fix needed** — this pillar is a model for the rest.
### Cross-cutting — Accessibility (drags 2 & 6)
**0 ARIA roles/labels, only 2 `:focus` styles** on a dashboard whose buttons
submit real orders (Approve / Reject / EXIT / FLATTEN). Keyboard and
screen-reader users cannot see focus or understand controls. Not a visual defect
but a real usability gap for the money-path controls. **WARNING (borderline
BLOCKER for keyboard operators).**
## Top 3 priority fixes
1. **Collapse duplicate semantic colors + migrate hardcoded values** (Pillar 3).
Define `--green`/`--red` (+ one bright variant each), replace the 3 greens / 4
reds and the 90 hardcoded hex. Highest consistency-per-effort. *WARNING.*
2. **Add focus + ARIA to money-path controls** (Accessibility). `:focus-visible`
outlines and `aria-label` on Approve/Reject/EXIT/FLATTEN. A keyboard operator
currently flies blind on the destructive buttons. *WARNING → BLOCKER for
keyboard use.*
3. **Adopt a spacing scale** (Pillar 5). Snap the 20 ad-hoc px values to a
2/4/8/12/16/24 ladder; raise the 9px type floor to ≥11px. *WARNING.*
## What NOT to touch
Experience design (Pillar 6) and copywriting (Pillar 1) are strengths — don't
regress the confirm-to-destruct flow or the explanatory empty states while
refactoring color/spacing. The density is a deliberate, correct choice for the
trading-terminal audience; do not "add whitespace" reflexively.
---
*No UI-SPEC exists for this project. If the dashboard becomes a maintained
surface, the highest-leverage next step is extracting the existing `:root` tokens
into a small declared design contract so future changes have a baseline to audit
against — the token system is 80% there already.*
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+6
View File
@@ -0,0 +1,6 @@
# Historical documents
Point-in-time plans, phase writeups, and runbooks from earlier eras of the
project (DuckDB backend, WSL/systemd-only deployment, pre-migration
architecture pushes). Kept for archaeology; **do not follow them** — the
living docs are the root README.md, RUNBOOK.md, and deploy/README.md.
File renamed without changes.
+227
View File
@@ -0,0 +1,227 @@
# RUNBOOK
Operational guide for running the momentum system. This file replaces the
original MILESTONE*.md documents, which were unrecoverable during the
rebuild.
---
## 1. Why live paper trading never worked (root causes, now fixed)
1. **Minute bars were never ingested.** The watcher reads 1-minute bars from
`data/research/market.duckdb`, but the old ingestion only wrote *daily*
bars for a handful of mega-caps. Fix: `research/ingestion/market_data.py`
ingests 1-minute Alpaca IEX bars continuously, and
`run_live_paper.py` schedules it every `LIVE_BARS_INTERVAL_SECONDS`.
2. **Session flags were hardcoded.** Every old row had
`is_regular_hours = TRUE`. Fix: flags are derived from the bar timestamp
in US/Eastern (premarket 04:00–09:30, regular 09:30–16:00, afterhours
16:00–20:00) — see `classify_session()` and its tests.
3. **The candidate filter excluded everything ingested.** The query had a
hardcoded `close BETWEEN 1 AND 20` while only mega-caps (price ≫ 20) were
in the database, so the watchlist was always empty. Fix: the band is
configurable (`WATCHER_PRICE_MIN` / `WATCHER_PRICE_MAX`) and the default
symbol list now actually gets ingested.
4. **No orchestrator.** Ingestion, watcher, broker sync, and execution were
never wired into one loop. Fix: `run_live_paper.py`.
5. **Broken source files.** `runtime/providers/registry.py`,
`schwab/market/models.py`, `option_chain_service.py`, and
`schwab/streaming/client.py` had syntax errors that crashed imports.
All repaired/rewritten.
6. **Schwab tokens were committed to git.** `data/schwab_tokens.json` is now
git-ignored and written `chmod 0600`. **Action required:** if that file
exists anywhere in your repo history, revoke the refresh token in the
Schwab developer portal and re-run the OAuth flow. Scrub history with
`git filter-repo --path data/schwab_tokens.json --invert-paths` if the
repo was ever shared.
---
## 2. Environment setup
```bash
pip install -r requirements.txt
cp .env.example .env
```
Minimum for paper trading: `ALPACA_API_KEY`, `ALPACA_SECRET_KEY`
(free keys are fine — the system uses the IEX feed). Everything else has a
sensible default; see `.env.example` for the full catalog.
Key knobs:
| Variable | Default | Meaning |
|---|---|---|
| `TRADING_EXECUTION_MODE` | `alpaca_paper` | `schwab_live` routes to real money — leave on paper |
| `TRADING_AUTO_APPROVE` | `0` | `1` = orders submit without dashboard approval (or pass `--auto-approve`) |
| `TRADING_MAX_ORDERS_PER_TICK` | `1` | throttle new entries |
| `TRADING_MAX_CONCURRENT_POSITIONS` | `3` | cap on simultaneous open positions |
| `TRADING_RISK_PER_TRADE_PCT` | `0.01` | 1% of equity risked per trade (sizing = risk / (entry − stop)) |
| `TRADING_REWARD_MULTIPLE` | `2.0` | profit target = entry + this × risk (2R by default) |
| `TRADING_ENTRY_ORDER_TYPE` | `limit` | `limit` rests at the trigger so an unfilled entry can be cancelled; `market` fills instantly |
| `TRADING_ENTRY_TIMEOUT_BARS` | `2` | cancel an unfilled entry after this many minutes resting (wall-clock; `0` disables the time box) |
| `TRADING_ENTRY_INVALIDATE_PCT` | `0.0` | cancel an unfilled entry if the **live last-trade price** trades this fraction below the trigger (`0.0` = any break below entry; `0.005` = 0.5% below; negative disables) |
| `TRADING_ENTRY_GUARD_INTERVAL_SECONDS` | `5` | how often the fast guard re-checks armed entries against the live price — lower = faster break detection, more quote calls |
| `TRADING_MAX_DAILY_LOSS_PCT` | `0.03` | daily-loss circuit breaker |
| `WATCHER_SYMBOLS` | `AAPL,TSLA,…` | static watchlist; `--discover` adds screener actives |
| `WATCHER_PRICE_MIN/MAX` | `1` / `20` | candidate price band |
| `WATCHER_EVENT_DB_PATH` | `./data/momentum.duckdb` | the event store |
| `DASHBOARD_HOST/PORT` | `127.0.0.1` / `8010` | UI address |
---
## 3. Start commands
**Full live paper loop (recommended):**
```bash
python run_live_paper.py
# dashboard: http://127.0.0.1:8010
# flags: --once --symbols SNDL,COSM --auto-approve --discover --no-dashboard
```
Each tick prints one status line:
```
[09:31:02] ingest: 240 minute rows | watch: evaluated=5 ready=['SNDL'] blocked=3 | sync: ok | execute: approvals_requested=['…'] auto_executed=0
```
**Standalone read-only dashboard** (e.g. to review a past session):
```bash
python dashboard_api.py
```
Approve/reject/exit buttons require the orchestrator (it attaches the
execution service to the same server); the standalone runner returns 503 on
actions by design.
**Other tools:**
```bash
python fetch_minute_bars.py SNDL COSM --lookback 390 # ad-hoc ingestion
python research_cli.py symbols --date 2026-06-11 # what's in the DB
python research_cli.py gappers --date 2026-06-11 # gap scan → scanner_snapshots
python backtest_cli.py --date 2026-06-10 # backtest a stored session
python milestone3_verify.py # Schwab health walk
python -m pytest # full test suite
```
---
## 4. The trading day, end to end
1. **Ingest** — every 60 s, 1-minute IEX bars for the watchlist land in
`data/research/market.duckdb` with correct session flags. On boot, a
30-day daily backfill gives gap % and relative volume a baseline.
2. **Watch** — every 30 s the watcher pulls in-band candidates, evaluates
nine weighted criteria (gap, RVOL, impulse, pullback, pullback volume,
VWAP, candle quality, breakout, data sufficiency) plus structure
detection, and drives `discovered → watching → ready | blocked | late`.
A `signal_ready` is emitted at most once per symbol per session.
3. **Sync** — Alpaca account, positions, and orders snapshot into the event
store so projections and risk guards see broker truth.
4. **Execute** — ready signals become orders sized at `risk_per_trade_pct`
of equity, with the stop from the setup and the target at
entry + `reward_multiple` × risk (2R by default). Manual mode parks them
in the dashboard queue; **auto mode (`--auto-approve`) submits
immediately, no approval.** Guards: max concurrent positions, no re-entry
into held symbols, one request per symbol per session.
5. **Manage** — approve/reject from the dashboard; **Exit** sends a market
sell for the synced quantity.
### The entry lifecycle (auto-arm → fill or back out)
The earlier system would "see" a setup as something that had *already
happened* rather than a trade about to happen, and never pull the trigger.
The execution model now separates recognition from conviction from
discipline:
1. **Recognize** — when all the criteria pass, a `signal_ready` fires. The
setup is evaluated against the **last bar's timestamp**, not the wall
clock, so a live 9:40 breakout is judged against the 9:40 cutoff instead
of being dismissed as hours stale.
2. **Arm with conviction** — in auto mode the order is submitted right away
as a **bracket** (entry + stop + target placed atomically). With
`TRADING_ENTRY_ORDER_TYPE=limit` (default) the entry rests at the trigger,
so "unfilled" is a real, observable state.
3. **Back out at a defined point** — a fast **guard loop**
(`TRADING_ENTRY_GUARD_INTERVAL_SECONDS`, default 5 s) re-checks every
armed-but-unfilled entry and cancels it if either:
- the **live last-trade price** (pulled tick-by-tick from Alpaca's data
API, not a bar close) trades back below the trigger by
`TRADING_ENTRY_INVALIDATE_PCT`, or
- it has rested longer than `TRADING_ENTRY_TIMEOUT_BARS` minutes
(wall-clock — running the guard often does **not** shorten this window).
A cancel frees the risk budget and the concurrent-position slot, and emits
a `risk_rule_triggered` (`entry_backout`) you can see in the activity feed.
Once the order **fills**, it stops being tracked and the bracket's stop and
target manage the open position — no second-guessing.
Tuning cheat-sheet: react to a fakeout faster → lower
`TRADING_ENTRY_GUARD_INTERVAL_SECONDS` (e.g. `2`); more patience on the fill →
raise `TRADING_ENTRY_TIMEOUT_BARS`; give the breakout room to wobble → set
`TRADING_ENTRY_INVALIDATE_PCT` to something like `0.005`; chase fills instead
of resting → `TRADING_ENTRY_ORDER_TYPE=market` (note: market fills can't time
out); bigger winners → raise `TRADING_REWARD_MULTIPLE`.
> **Quote rate limits.** The guard makes one latest-trade call per armed
> symbol per pass (a ~2 s cache collapses duplicates). At the 5 s default with
> a few open entries you're well inside Alpaca's free-tier 200 req/min. If you
> drop the interval to 1 s with several armed entries, watch the budget —
> bars ingestion and account sync also draw from it.
### Reading the dashboard
- **Badge** — blue `PAPER · ALPACA` vs red `LIVE · SCHWAB`. If it's red and
you didn't mean it, stop and check `TRADING_EXECUTION_MODE`.
- **P&L strip** (top) — session realized / unrealized / total, win rate,
average R, open/closed counts. Green/red signed.
- **Approval Queue** — pending entries with a **risk preview** ($ risk,
shares, notional, stop %) and Approve/Reject. Keyboard: `A` / `R` act on the
top row.
- **Ready Signals / Open Positions** — each row has a live **sparkline** (last
~40 one-minute closes) with dashed entry (blue) and stop (red) lines.
- **Watch States** — click any row to expand the **nine-criteria breakdown**
(✓ pass / ✕ fail / · not-evaluated). The add-symbol box (or `/`) injects a
ticker onto the watchlist for the rest of the session.
- **Activity Feed** — submissions and fills as they happen.
- **Order Lifecycle** — per-order status from the event store.
- **Connection dot** (top right) — green pulse = live SSE stream; it falls
back to 4 s polling automatically if the stream drops.
- **Sound** — toggle the button or press `S` for an audio + toast alert when a
new signal goes ready. Press `?` for the shortcut list.
Streams from `GET /api/stream` (SSE); also exposes `/api/snapshots`,
`/api/criteria?symbol=`, `/api/bars?symbol=&minutes=`, and
`POST /api/watch/add`. Charts require the research DB, which the orchestrator
wires in automatically; the standalone `dashboard_api.py` has no bars source,
so sparklines there show "no bars".
---
## 5. Troubleshooting
| Symptom | Likely cause / fix |
|---|---|
| `watch: evaluated=0` all day | No minute bars for today. Check keys; run `python research_cli.py symbols`. Outside market hours there are no new bars — backfill with `fetch_minute_bars.py --lookback 1440`. |
| Candidates exist but never `ready` | Score < 60%. Watch the `Watch States` panel score column; criteria details are in `criteria_evaluated` events. |
| `ingest: … 403` errors | Free keys are IEX-only — ensure `ALPACA_DATA_FEED=iex` (default) and that the request window ends ≥30 s in the past (handled automatically). |
| Approve button does nothing | You're on `dashboard_api.py` (read-only). Use `run_live_paper.py`. |
| Port 8010 busy | Set `DASHBOARD_PORT`, or `--no-dashboard`. |
| Schwab shows `SCHWAB-UNAUTH` | Expected without OAuth. Run the flow in `schwab/auth/oauth2_flow_manager.py`; verify with `milestone3_verify.py`. |
| Start fresh | Delete `data/momentum.duckdb` (events) and/or `data/research/` (bars). Both rebuild automatically. |
---
## 6. Going live on Schwab (when ready)
Paper-trade until the stats earn it. Then: complete OAuth
(`milestone3_verify.py` all green), set `TRADING_EXECUTION_MODE=schwab_live`,
keep `TRADING_AUTO_APPROVE=0`, and start with the smallest viable
`TRADING_RISK_PER_TRADE_PCT`. The dashboard badge turns red in live mode.
File renamed without changes.
@@ -0,0 +1,293 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Ideation: Next Enhancement Milestone — Momentum</title>
<style>
:root {
--bg: #fafaf8; --fg: #1a1d21; --muted: #5c6470; --line: #dcdfe4;
--card: #ffffff; --accent: #0f6b54; --accent-soft: #e6f2ee; --accent-text: #0b5442;
--warn: #9a5b00; --warn-soft: #fdf3e3; --info: #274f8f; --info-soft: #e9effa;
--danger: #a33; --danger-soft: #faecec; --chip: #eef0f3; --code: #f2f3f5;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #14171a; --fg: #e6e8ea; --muted: #9aa3ad; --line: #2c3238;
--card: #1b1f23; --accent: #4fbf9f; --accent-soft: #14312a; --accent-text: #7dd6bc;
--warn: #e0a94f; --warn-soft: #33270f; --info: #7ea6e0; --info-soft: #17233a;
--danger: #e08080; --danger-soft: #351b1b; --chip: #262b30; --code: #22272c;
}
}
* { box-sizing: border-box; }
body { margin: 0; background: var(--bg); color: var(--fg);
font: 15px/1.55 -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; }
.page { max-width: 920px; margin-inline: auto; padding: 2.2rem 1.4rem 4rem; }
p, li, dd { max-width: 72ch; }
h1 { font-size: 1.7rem; line-height: 1.25; margin: .2rem 0 .4rem; }
h2 { font-size: 1.25rem; margin: 2.4rem 0 .8rem; padding-top: 1rem; border-top: 1px solid var(--line); }
h3 { font-size: 1.05rem; margin: 1.6rem 0 .5rem; }
code { background: var(--code); padding: .1em .35em; border-radius: 4px; font-size: .88em; }
a { color: var(--info); }
.eyebrow { font-size: .72rem; letter-spacing: .12em; text-transform: uppercase; color: var(--muted); }
.meta { display: flex; flex-wrap: wrap; gap: .4rem .6rem; margin: .6rem 0 0; padding: 0; list-style: none; }
.meta li { background: var(--chip); border-radius: 999px; padding: .18rem .7rem; font-size: .8rem; color: var(--muted); }
.meta li b { color: var(--fg); font-weight: 600; }
.stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(140px, 1fr)); gap: .7rem; margin: 1.6rem 0; }
.stat { background: var(--card); border: 1px solid var(--line); border-radius: 10px; padding: .7rem .9rem; }
.stat .n { font-size: 1.35rem; font-weight: 700; }
.stat .l { font-size: .78rem; color: var(--muted); }
article.idea { background: var(--card); border: 1px solid var(--line); border-radius: 12px;
padding: 1.1rem 1.3rem; margin: 1.1rem 0; }
article.idea h3 { margin: 0 0 .5rem; }
.chips { display: flex; flex-wrap: wrap; gap: .35rem; margin: 0 0 .8rem; }
.chips span { border-radius: 999px; padding: .14rem .6rem; font-size: .74rem; font-weight: 600; }
.chip-axis { background: var(--info-soft); color: var(--info); }
.chip-conf { background: var(--accent-soft); color: var(--accent-text); }
.chip-cx { background: var(--chip); color: var(--muted); }
.chip-cx.high { background: var(--warn-soft); color: var(--warn); }
dl.fields { margin: 0; }
dl.fields dt { font-size: .72rem; letter-spacing: .1em; text-transform: uppercase; color: var(--muted); margin-top: .7rem; }
dl.fields dd { margin: .15rem 0 0; }
.jump { background: var(--card); border: 1px solid var(--line); border-radius: 10px; padding: .8rem 1.1rem; }
.jump ol { margin: .3rem 0 0 1.2rem; padding: 0; }
table { border-collapse: collapse; width: 100%; font-size: .88rem; }
th, td { border: 1px solid var(--line); padding: .45rem .6rem; text-align: left; vertical-align: top; }
th { background: var(--chip); font-size: .78rem; letter-spacing: .05em; text-transform: uppercase; }
.tablewrap { overflow-x: auto; }
figure { margin: 1rem 0 .3rem; }
figcaption { font-size: .78rem; color: var(--muted); margin-top: .3rem; }
footer.composition-signal { margin-top: 3rem; padding-top: 1rem; border-top: 1px solid var(--line);
font-size: .78rem; color: var(--muted); }
svg text { font-family: inherit; }
</style>
</head>
<body>
<div class="page">
<div class="eyebrow">Ideation · repo-grounded</div>
<h1>Next Enhancement Milestone — Momentum</h1>
<ul class="meta">
<li><b>date</b> 2026-07-13</li>
<li><b>topic</b> next-enhancement-milestone</li>
<li><b>mode</b> repo-grounded</li>
<li><b>repo</b> momentum @ d54a2a9</li>
<li><b>focus</b> read everything, brainstorm enhancements, pick the highest-leverage build</li>
</ul>
<div class="stats">
<div class="stat"><div class="n">48</div><div class="l">raw ideas · 5 frames</div></div>
<div class="stat"><div class="n">14</div><div class="l">merged candidates</div></div>
<div class="stat"><div class="n">14/14</div><div class="l">verified sound (fresh-context spot-checks)</div></div>
<div class="stat"><div class="n">7</div><div class="l">survivors · 5/5 axes covered</div></div>
</div>
<h2 id="grounding-context">Grounding Context</h2>
<p><strong>Codebase context.</strong> Event-sourced small-cap momentum day-trading system (gap-and-go / opening-range breakout). Alpaca <em>paper</em> account over the SIP feed; single PostgreSQL datastore (append-only <code>events</code> table + projections, ~19M minute bars); production is a Docker Compose stack on TrueNAS SCALE (postgres 18, trading loop, dashboard :8765, admin :8767, telegram bot, nightly scheduler); 376 pytest tests against throwaway <code>mem_*</code> schemas. Solo owner-operator.</p>
<p>Grounding came from an 8-agent recon sweep (2026-07-13) that read every subsystem — live loop, strategy, storage, research/data, dashboard, ops/deploy, brokers, tests — and produced ~95 evidence-cited findings, plus the in-repo external research pass <code>docs/research/entry_conversion_research.md</code> (academic + backtest literature on breakout entry conversion, with a ranked experiment shortlist). A fresh-context verifier then spot-checked every load-bearing claim against HEAD with ~25 targeted reads; <strong>every checked citation matched</strong>.</p>
<p><strong>Settled constraints honored throughout:</strong> the validated entry population stays ($2–20, static open gap 10–35%, real volume, ≥150%-gap &amp; rvol≥5 squeeze exception); seven selection levers are refuted at year scale and are not re-proposed; scale-out partials and retest/pullback entries are refuted; the :8765 dashboard is the only UI to extend; the system's proven bottleneck is <strong>entry-conversion</strong>, not selection.</p>
<h2 id="topic-axes">Topic Axes</h2>
<ul>
<li><strong>money-path-correctness</strong> — session lifecycle, broker-truth handling, order reconcile, entry-path drift, exit-manager safety</li>
<li><strong>learning-flywheel</strong> — tuning/replay scheduling, look-ahead &amp; population parity, dataset growth, experiment machinery</li>
<li><strong>entry-conversion</strong> — execution experiments on the proven bottleneck (time-stop, probe-add, asymmetric exit), shadow-track infrastructure</li>
<li><strong>operability-delivery</strong> — CI gate, alerting/dead-man, backup, retention, env drift, dead-weight deletion</li>
<li><strong>operator-surface</strong> — :8765 dashboard, :8767 admin, telegram: repairs, visibility, manual controls</li>
</ul>
<h2 id="ranked-ideas">Ranked Ideas</h2>
<div class="jump">
<div class="eyebrow">Jump to</div>
<ol>
<li><a href="#i1">Experiment OS — pre-registered, auto-adjudicated counterfactual experiments over logged fires</a></li>
<li><a href="#i2">Learning-flywheel resurrection — one population function + scheduled tune/replay</a></li>
<li><a href="#i3">Entry-path unification + decision receipts</a></li>
<li><a href="#i4">Money-path hardening — self-restart, positive-evidence contract, EOD cage count</a></li>
<li><a href="#i5">Production safety rails — CI gate, dead-man's switch, automated backup</a></li>
<li><a href="#i6">Operator control plane — dashboard repair, kill switch, effective-config panel</a></li>
<li><a href="#i7">Dead-weight deletion sweep — every table names its reader or dies</a></li>
</ol>
</div>
<article class="idea" id="i1">
<h3>1. Experiment OS — pre-registered, auto-adjudicated counterfactual experiments over logged fires</h3>
<div class="chips"><span class="chip-axis">entry-conversion</span><span class="chip-conf">Confidence 75%</span><span class="chip-cx high">Complexity High</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>One parametric experiment harness replacing the four hand-built shadow-track triples in <code>research/labeler.py</code> (vwap_reclaim / runner / liquid / premarket — ~1,467 lines of structurally near-identical build/report/validate code). An experiment becomes a config row: population, intervention (an entry or exit policy variant), pre-registered pass/fail bars, minimum n, and futility criteria. A replay engine runs every logged live fire counterfactually against its recorded bar path — the Tesla shadow-mode pattern — including a <em>counterfactual exit ledger</em> (every real trade re-run under each exit variant). A verdict engine auto-adjudicates at the pre-registered thresholds (the clinical-trial DSMB pattern, with futility stops) and emits PASS / FAIL / INCONCLUSIVE verdict events that flow to Telegram and a dashboard experiment-ledger panel. First cohort ships with the harness: the three literature-backed entry-conversion experiments (idea card behind them in <code>docs/research/entry_conversion_research.md</code>): time-boxed entries + 10-minute time-stop on unconfirmed positions, probe-then-add 0.5×+0.5× sizing, asymmetric exit (volatility-fractional stop with a spread floor, no target, trail to EOD) — plus the rank-concentration "one-bullet" counterfactual and an immediate adjudication of pm1.</dd>
<dt>Basis</dt>
<dd><code>direct:</code> <code>data/premarket_eval_history.tsv</code> latest row (2026-07-09) shows pm1 at n=2377 with rate_1r=0.443 vs the pre-registered &gt;0.50 bar and median_up_60m=0.0433 vs the ≥0.10 bar — failing 3 of 4 pre-registered criteria while the code's own docstring still frames it as "n=21 is far too small… watch n climb". Seven prior levers each required a human to notice a threshold crossing in a log file. <code>external:</code> the three first-cohort experiments carry academic + replicated-backtest support (Zarattini/Barbon/Aziz SSRN 4729284, Osler NY Fed SR150, Mesfin arXiv 2605.04004) per the in-repo research doc. Verifier confirmed none of the harness/experiment machinery exists yet.</dd>
<dt>Rationale</dt>
<dd>The system's bottleneck is proven to be entry-conversion, and the three highest-priority interventions are already researched, pre-registered, and testable for <em>free</em> against fires the system has already logged — zero live risk. The blocker is machinery, not ideas: every new experiment today costs a hand-built ~300-line triple plus a human reading a TSV. Build the machinery once and every future experiment (premarket module included) drops to a config row with an automatic verdict.</dd>
<dt>Downsides</dt>
<dd>Biggest build in the set; replay fidelity is a real risk (fills, spreads, halts must be modeled honestly — the tight-stop zero-slippage artifact has already fooled two shadow tracks here, so the harness must enforce adverse-fill floors by construction). Requires the population module from idea 2 to avoid baking look-ahead into every experiment.</dd>
</dl>
<figure>
<svg viewBox="0 0 860 190" role="img" aria-label="Experiment OS flow">
<defs><marker id="arr" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0 L10,5 L0,10 z" fill="#8892a0"/></marker></defs>
<rect x="10" y="55" width="150" height="80" rx="10" fill="var(--info-soft)" stroke="var(--info)"/>
<text x="85" y="88" text-anchor="middle" font-size="13" font-weight="600" fill="var(--fg)">Logged fires</text>
<text x="85" y="106" text-anchor="middle" font-size="11" fill="var(--muted)">+ recorded bar paths</text>
<rect x="230" y="20" width="170" height="60" rx="10" fill="var(--card)" stroke="var(--line)"/>
<text x="315" y="45" text-anchor="middle" font-size="12.5" font-weight="600" fill="var(--fg)">Experiment configs</text>
<text x="315" y="62" text-anchor="middle" font-size="11" fill="var(--muted)">population · policy · bars · n</text>
<rect x="230" y="105" width="170" height="65" rx="10" fill="var(--accent-soft)" stroke="var(--accent)"/>
<text x="315" y="132" text-anchor="middle" font-size="12.5" font-weight="600" fill="var(--fg)">Replay engine</text>
<text x="315" y="149" text-anchor="middle" font-size="11" fill="var(--muted)">counterfactual P&amp;L per fire</text>
<rect x="470" y="70" width="160" height="65" rx="10" fill="var(--warn-soft)" stroke="var(--warn)"/>
<text x="550" y="97" text-anchor="middle" font-size="12.5" font-weight="600" fill="var(--fg)">Verdict engine</text>
<text x="550" y="114" text-anchor="middle" font-size="11" fill="var(--muted)">pre-registered bars · futility</text>
<rect x="700" y="20" width="150" height="55" rx="10" fill="var(--card)" stroke="var(--line)"/>
<text x="775" y="43" text-anchor="middle" font-size="12" font-weight="600" fill="var(--fg)">Telegram ping</text>
<text x="775" y="60" text-anchor="middle" font-size="11" fill="var(--muted)">PASS / FAIL / INCONCLUSIVE</text>
<rect x="700" y="115" width="150" height="55" rx="10" fill="var(--card)" stroke="var(--line)"/>
<text x="775" y="138" text-anchor="middle" font-size="12" font-weight="600" fill="var(--fg)">Ledger panel</text>
<text x="775" y="155" text-anchor="middle" font-size="11" fill="var(--muted)">:8765 dashboard</text>
<line x1="160" y1="95" x2="222" y2="130" stroke="#8892a0" stroke-width="1.6" marker-end="url(#arr)"/>
<line x1="315" y1="80" x2="315" y2="98" stroke="#8892a0" stroke-width="1.6" marker-end="url(#arr)"/>
<line x1="400" y1="130" x2="462" y2="108" stroke="#8892a0" stroke-width="1.6" marker-end="url(#arr)"/>
<line x1="630" y1="88" x2="692" y2="55" stroke="#8892a0" stroke-width="1.6" marker-end="url(#arr)"/>
<line x1="630" y1="115" x2="692" y2="138" stroke="#8892a0" stroke-width="1.6" marker-end="url(#arr)"/>
</svg>
<figcaption>Every logged fire becomes reusable evidence; every experiment becomes a config row with an automatic verdict.</figcaption>
</figure>
</article>
<article class="idea" id="i2">
<h3>2. Learning-flywheel resurrection — one population function + scheduled tune/replay</h3>
<div class="chips"><span class="chip-axis">learning-flywheel</span><span class="chip-conf">Confidence 90%</span><span class="chip-cx">Complexity Medium</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>Extract the live population gate into one shared pure module with an as-of-time contract (open-knowable inputs only: static open gap, price band, premarket-knowable volume, squeeze exception) and make every offline consumer — <code>learn_params.py</code>, the sweeps, <code>eod_replay.py</code>, the labeler — consume it instead of re-implementing it. Then actually schedule the flywheel in production: <code>eod_replay</code> after each close (unbiased whole-market dataset growth), <code>nightly_tune</code> after it (atomic tmp+rename write, staleness alert if <code>learned_params.json</code> ages past 2 days).</dd>
<dt>Basis</dt>
<dd><code>direct:</code> verifier-confirmed at HEAD: the TrueNAS scheduler runs <em>only</em> <code>research.premarket_eval</code>; <code>data/learned_params.json</code> is stamped as_of 2026-06-17 — 26 days stale — while the boot log still claims self-tuning; <code>learn_params.py:61,64</code> admits day-end <code>day_rvol</code> (outcome-time look-ahead); <code>eod_replay.py:44-48</code> screens $1–5 under a "live verified config" comment while production trades $2–20 (<code>truenas-compose.yml:105-106</code>); <code>eod_replay.sh</code> points at a dead venv path; <code>nightly_tune.py:91-92</code> writes non-atomically.</dd>
<dt>Rationale</dt>
<dd>The project memory says "nightly_tune self-tunes params, applied on boot" — that story is currently fiction in production, and the offline population drift means even a re-enabled tuner would fit a universe the bot doesn't trade. This is the data-integrity substrate under idea 1: experiments adjudicated against a look-ahead population are worthless. The owner has already been burned twice by exactly this class (the 2026-07-08 look-ahead correction).</dd>
<dt>Downsides</dt>
<dd>Population-module extraction touches the live gate — needs characterization tests before refactor. A deliberate alternative is to <em>delete</em> the tuner and stop claiming self-tuning; the wrong option is the current silent limbo.</dd>
</dl>
<figure>
<svg viewBox="0 0 860 175" role="img" aria-label="One population module fanning out to five consumers">
<defs><marker id="arr2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0 L10,5 L0,10 z" fill="#8892a0"/></marker></defs>
<text x="115" y="20" text-anchor="middle" font-size="11.5" font-weight="700" fill="var(--danger)">TODAY: 5 diverging copies</text>
<rect x="30" y="32" width="170" height="26" rx="6" fill="var(--danger-soft)" stroke="var(--danger)"/><text x="115" y="49" text-anchor="middle" font-size="11" fill="var(--fg)">live gate — $2–20, open gap</text>
<rect x="30" y="64" width="170" height="26" rx="6" fill="var(--danger-soft)" stroke="var(--danger)"/><text x="115" y="81" text-anchor="middle" font-size="11" fill="var(--fg)">tuner — day-end rvol (look-ahead)</text>
<rect x="30" y="96" width="170" height="26" rx="6" fill="var(--danger-soft)" stroke="var(--danger)"/><text x="115" y="113" text-anchor="middle" font-size="11" fill="var(--fg)">eod_replay — $1–5, no squeeze</text>
<rect x="30" y="128" width="170" height="26" rx="6" fill="var(--danger-soft)" stroke="var(--danger)"/><text x="115" y="145" text-anchor="middle" font-size="11" fill="var(--fg)">sweeps · labeler — own filters</text>
<text x="580" y="20" text-anchor="middle" font-size="11.5" font-weight="700" fill="var(--accent-text)">TARGET: one as-of-time module</text>
<rect x="360" y="65" width="190" height="56" rx="10" fill="var(--accent-soft)" stroke="var(--accent)"/>
<text x="455" y="88" text-anchor="middle" font-size="12.5" font-weight="600" fill="var(--fg)">population.py</text>
<text x="455" y="105" text-anchor="middle" font-size="11" fill="var(--muted)">open-knowable inputs only</text>
<rect x="660" y="28" width="170" height="24" rx="6" fill="var(--card)" stroke="var(--line)"/><text x="745" y="44" text-anchor="middle" font-size="11" fill="var(--fg)">live gate</text>
<rect x="660" y="58" width="170" height="24" rx="6" fill="var(--card)" stroke="var(--line)"/><text x="745" y="74" text-anchor="middle" font-size="11" fill="var(--fg)">tuner + sweeps</text>
<rect x="660" y="88" width="170" height="24" rx="6" fill="var(--card)" stroke="var(--line)"/><text x="745" y="104" text-anchor="middle" font-size="11" fill="var(--fg)">eod_replay (scheduled)</text>
<rect x="660" y="118" width="170" height="24" rx="6" fill="var(--card)" stroke="var(--line)"/><text x="745" y="134" text-anchor="middle" font-size="11" fill="var(--fg)">labeler / Experiment OS</text>
<line x1="550" y1="80" x2="652" y2="42" stroke="#8892a0" stroke-width="1.5" marker-end="url(#arr2)"/>
<line x1="550" y1="90" x2="652" y2="70" stroke="#8892a0" stroke-width="1.5" marker-end="url(#arr2)"/>
<line x1="550" y1="100" x2="652" y2="100" stroke="#8892a0" stroke-width="1.5" marker-end="url(#arr2)"/>
<line x1="550" y1="110" x2="652" y2="130" stroke="#8892a0" stroke-width="1.5" marker-end="url(#arr2)"/>
</svg>
<figcaption>Every backtest, tune, and experiment verdict becomes trustworthy by construction when all consumers share the live gate.</figcaption>
</figure>
</article>
<article class="idea" id="i3">
<h3>3. Entry-path unification + decision receipts</h3>
<div class="chips"><span class="chip-axis">money-path-correctness</span><span class="chip-conf">Confidence 85%</span><span class="chip-cx">Complexity Medium</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>Collapse the two drifted entry paths — slow auto-approval (<code>request_approvals_for_ready_signals</code>) and fast trigger (<code>submit_breakout_now</code>) — onto one shared pure gate function. Every gate evaluation emits a compact <em>decision receipt</em> event (gate name, inputs, pass/fail), so "why did/didn't it trade X?" becomes a query instead of a forensic session.</dd>
<dt>Basis</dt>
<dd><code>direct:</code> verifier-confirmed: <code>_population_reject</code>/<code>_outside_session</code>/<code>_anti_chase_skip</code> are already shared by both paths, but the VWAP check exists only in the slow path (~<code>trading_execution.py:938</code>) and <code>rank_risk_factor</code> only in the fast path (<code>:1228</code>); the boot log itself warns "rank-concentration is enforced ONLY on the dead fast-trigger path, NOT the live auto-approval path" (<code>run_live_paper.py:1026-1028</code>). The daily-cap counted 44-vs-5 incident (fixed ac1ac50) was this same drift class.</dd>
<dt>Rationale</dt>
<dd>Half the convergence is already done — this finishes it so every future gate (including any Experiment-OS winner promoted to live) lands on both paths by construction. Receipts double as the raw material Experiment OS replays and the operator reads on the dashboard.</dd>
<dt>Downsides</dt>
<dd>Touches the live money path — needs the CI gate (idea 5) and characterization tests first. Receipt events add volume to the events table (bounded; they replace ad-hoc diagnostic events that retention already prunes).</dd>
</dl>
</article>
<article class="idea" id="i4">
<h3>4. Money-path hardening — self-restart, positive-evidence contract, EOD cage count</h3>
<div class="chips"><span class="chip-axis">money-path-correctness</span><span class="chip-conf">Confidence 95%</span><span class="chip-cx">Complexity Low</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>Three small, high-severity fixes sharing one principle — never act on assumed state. (a) <strong>Daily self-restart</strong>: <code>exit(0)</code> after a successful EOD flatten and let compose's <code>restart: unless-stopped</code> resurrect a fresh process — the process <em>is</em> the trading session. (b) <strong>Positive-evidence contract</strong>: no destructive market action on an ambiguous broker read — the exit manager must distinguish "orders call failed / silently empty" from "position confirmed naked" before market-flattening, and a lost order-POST must reconcile via <code>client_order_id</code> lookup before journaling "cancelled". (c) <strong>EOD cage count</strong>: a nightly reconcile that diffs broker positions/orders/fills against event-store truth and alerts on any discrepancy, plus SIGTERM handled as clean shutdown (<code>docker stop</code> currently skips the shutdown path).</dd>
<dt>Basis</dt>
<dd><code>direct:</code> verifier-confirmed: <code>_eod_done</code>/<code>_catchup_done</code> (<code>run_live_paper.py:1064,1113</code>) and <code>_session_closed</code>/<code>_daily_loss_breach</code> (<code>trading_execution.py:398,612</code>) never reset — an uninterrupted day-2 takes zero entries and has no EOD flatten, masked today only by frequent redeploys; <code>exit_manager.py:71-72</code> returns None on any exception and treats it identically to "naked", flattening after ~24s grace; <code>execution.py:124-129</code> journals lost POSTs as cancelled with no reconcile (no <code>by_client_order_id</code> method exists in the client); the 2026-07-06 duplicate-resting-order incident is corroborated by d54a2a9's own commit message; recovery-on-boot (<code>_rebuild_daily_state</code>) already exists and is tested.</dd>
<dt>Rationale</dt>
<dd>Highest severity-to-effort ratio in the whole set. (a) alone converts an entire recurring bug class (session gate, after-hours submissions, bench state — each patched separately before) into a non-issue by construction, in ~5 lines plus tests.</dd>
<dt>Downsides</dt>
<dd>Self-restart must fire only after a <em>successful</em> flatten (exit-on-failure would orphan positions — though broker-side stop legs keep them protected); the positive-evidence branch adds one more state to exit-manager logic that is already subtle.</dd>
</dl>
</article>
<article class="idea" id="i5">
<h3>5. Production safety rails — CI gate, dead-man's switch, automated backup</h3>
<div class="chips"><span class="chip-axis">operability-delivery</span><span class="chip-conf">Confidence 95%</span><span class="chip-cx">Complexity Low</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>The "vacation test" bundle: (a) CI runs the 376-test suite (plus <code>ruff check --select F</code> for the untested script layer) <em>before</em> the image workflow pushes <code>:latest</code> to GHCR/Docker Hub; (b) the always-up qabot container gets a dead-man's switch — alert when the event-store heartbeat goes stale during market hours, plus a nightly OK-ping (the loop can't report its own death; the qabot is the independent observer); (c) a scheduled <code>pg_dump</code> (or documented ZFS snapshot task) — production currently has <strong>no automated backup</strong>.</dd>
<dt>Basis</dt>
<dd><code>direct:</code> verifier-confirmed: <code>.github/workflows/docker.yml</code> has zero test step and ships on every master push; <code>api/main.py</code> at HEAD references <code>os</code>/<code>hmac</code> without importing them (AST-confirmed) — an untested regression that already shipped; loop-down currently means alerts-down (the notifier lives inside the loop process, <code>run_live_paper.py:1034</code>); <code>pg_dump</code> appears only in the one-time migration script.</dd>
<dt>Rationale</dt>
<dd>This is the lever that makes every other lever safe to ship. Every subsequent idea in this list lands as an image push; today each push is an untested bet against the production money path.</dd>
<dt>Downsides</dt>
<dd>CI needs a Postgres service container for the suite (~3–4 min per push). Dead-man thresholds need market-calendar awareness to avoid weekend false alarms — the holiday-aware calendar already exists in <code>runtime/market_calendar.py</code>.</dd>
</dl>
</article>
<article class="idea" id="i6">
<h3>6. Operator control plane — dashboard repair, kill switch, effective-config panel</h3>
<div class="chips"><span class="chip-axis">operator-surface</span><span class="chip-conf">Confidence 85%</span><span class="chip-cx">Complexity Medium</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>Make the production :8765 surface tell the truth and accept commands. Repairs: fix the POST <code>NameError</code> (unimported <code>os</code>/<code>hmac</code>); wire the research DB into the standalone dashboard so sparklines work in production; gate action buttons on the <code>has_execution</code> flag the API already ships; derive the LIVE badge from broker truth (it currently keys on a <code>schwab_live</code> literal the admin go-live flow never sets — it would read PAPER while trading real money). Capability: manual flatten (all / per-symbol) and pause-entries as guarded <em>operator-command events</em> through the store — the loop polls its own 5s guard cadence and executes with the existing <code>cancel_protective_and_close</code> helper; dashboard and qabot become thin command emitters. Efficiency: replace the per-client per-second fresh-connection + full-DDL + <code>COUNT(*)</code> SSE change detector with one shared cached signal. Visibility: an effective-config panel showing the merged config per service with per-key provenance (baked env → compose env → <code>overrides.env</code> → learned params) and drift-vs-inventory warnings.</dd>
<dt>Basis</dt>
<dd><code>direct:</code> verifier-confirmed: <code>telegram_bot.py:29</code> hard-refuses action words; <code>dashboard_api.py:36</code> builds the production dashboard with <code>execution_service=None</code> — no manual flatten path exists anywhere in the production UI; <code>dashboard.html:1061</code> keys LIVE on <code>schwab_live</code> while <code>admin_api.py:276-295</code> only swaps Alpaca URL/keys; SSE poll confirmed at <code>api/main.py:355</code>; concurrency-gauge default drift (3 vs 6) confirmed at <code>trading_execution.py:75</code> vs <code>projections.py:869</code>.</dd>
<dt>Rationale</dt>
<dd>The operator's two moments of need — "something is wrong, get me flat NOW" and "what is this thing actually running?" — both currently require shelling into a container. Command-as-event reuses the store as the bus (no new API surface into the loop) and leaves an audit trail by construction.</dd>
<dt>Downsides</dt>
<dd>A kill switch is a live-money control: needs HMAC/token guard, idempotency, and a test that a replayed or stale command cannot re-fire. Command latency is bounded by the 5s guard cadence (acceptable; broker stops remain the fast layer).</dd>
</dl>
</article>
<article class="idea" id="i7">
<h3>7. Dead-weight deletion sweep — every table names its reader or dies</h3>
<div class="chips"><span class="chip-axis">operability-delivery</span><span class="chip-conf">Confidence 90%</span><span class="chip-cx">Complexity Low</span></div>
<dl class="fields">
<dt>Description</dt>
<dd>Finish the deletion d54a2a9 claimed: <code>schwab/</code> is still fully tracked (23 files) despite the commit message. Then the rest of the confirmed-dead surface: <code>comprehensive_backtest.py</code> (crashes on import — <code>research/connection.py</code> doesn't exist), <code>strategy/regime.py</code>, <code>universe.py</code>, <code>backtest/outcomes.py</code>, the ignition detector, the <code>grafana/</code> stack (points at the decommissioned pre-migration DB with committed plaintext dev credentials), the <code>telemetry/</code> husk, <code>docker-publish.yml</code> (fires on a nonexistent <code>main</code> branch, pushes a wrong image name), stale <code>uv.lock</code>, the empty <code>test_alpaca_client.py</code>, 12 zero-reference tables and 6 dead projections. Standing rule going forward: every table names its reader or dies — <code>quote_snapshots</code> gets retention or deletion; <code>float_cache</code>/<code>dilution_events</code> get wired into the labeler's features or stop being collected nightly.</dd>
<dt>Basis</dt>
<dd><code>direct:</code> verifier-confirmed each item at HEAD, including <code>git ls-files schwab/</code> (23 tracked files) vs d54a2a9's message, the import crash, the grafana datasource credentials, and grep-confirmed zero readers for the write-only tables.</dd>
<dt>Rationale</dt>
<dd>Every dead artifact taxes every future session ("which one is real?") and the grafana credentials are a small standing security smell. Deletion is the cheapest enhancement in the set and makes every subsequent plan smaller.</dd>
<dt>Downsides</dt>
<dd>The schwab/ decision should be explicit (its only non-delete rationale is the dormant human-vs-bot comparison); deleting tables needs a backup first (idea 5c) and a one-session soak.</dd>
</dl>
</article>
<h2 id="rejection-summary">Rejection Summary</h2>
<p>Nothing was refuted: the fresh-context verifier confirmed all 14 merged candidates sound, and all 48 raw ideas were absorbed into the 7 survivors. The table records the merges and the judgment calls.</p>
<div class="tablewrap">
<table>
<tr><th>#</th><th>Idea</th><th>Disposition</th></tr>
<tr><td>1</td><td>Daily self-restart (raw #1, 9, 18, 25, 33 — five frames converged independently)</td><td>Merged into survivor 4a — strongest single convergence in the run</td></tr>
<tr><td>2</td><td>Broker-ambiguity fixes / MCAS rule / "empty is not naked" (raw #2, 26, 37)</td><td>Merged into survivor 4b as one shared contract</td></tr>
<tr><td>3</td><td>EOD cage-count reconciliation (raw #27)</td><td>Folded into survivor 4c — thinnest-sourced candidate (analogy + one past incident); kept as backstop, not standalone</td></tr>
<tr><td>4</td><td>Verdict watchdog / DSMB / experiment ledger / experiment-as-config / Tesla shadow mode / counterfactual exit ledger (raw #4, 13, 22, 30, 34, 43, 44)</td><td>Merged into survivor 1 (Experiment OS) — the run's flagship synthesis</td></tr>
<tr><td>5</td><td>Entry-conversion shortlist experiments (raw #5, 38) and one-bullet rank counterfactual (raw #48)</td><td>Merged into survivor 1 as the first experiment cohort</td></tr>
<tr><td>6</td><td>Population parity / flywheel scheduling / "replay whole market nightly" (raw #3, 11, 12, 20, 21, 28, 35, 47)</td><td>Merged into survivor 2</td></tr>
<tr><td>7</td><td>Kill switch variants: qabot /flatten, fire handle, command-bus events (raw #8, 15, 32, 36)</td><td>Merged into survivor 6 — command-as-event chosen as the unifying mechanism</td></tr>
<tr><td>8</td><td>LISTEN/NOTIFY dashboard change signal (raw #16)</td><td>Demoted to implementation option inside survivor 6 — shared cached signal may be enough (lazy option first)</td></tr>
<tr><td>9</td><td>Effective-config panel (raw #46)</td><td>Merged into survivor 6</td></tr>
<tr><td>10</td><td>CI gate / vacation test / eye-in-the-sky (raw #6, 7, 17, 31, 41, 42)</td><td>Merged into survivor 5</td></tr>
<tr><td>11</td><td>Deletion sweeps (raw #14, 24, 39)</td><td>Merged into survivor 7</td></tr>
<tr><td>—</td><td>Axis coverage</td><td>All five axes have survivors; no recovery dispatch was needed</td></tr>
</table>
</div>
<footer class="composition-signal">Composed 2026-07-13 by ce-ideate (repo-grounded) from an 8-agent recon sweep + 5-frame ideation fleet + fresh-context basis verification, at momentum commit <code>d54a2a9</code>. Scratch: <code>/tmp/compound-engineering/ce-ideate/a3f9c2e1/</code></footer>
</div>
</body>
</html>
@@ -0,0 +1,142 @@
<?xml version="1.0" encoding="UTF-8"?>
<mxfile host="drawio" version="26.0.0">
<diagram name="Current architecture">
<mxGraphModel dx="1420" dy="900" grid="1" gridSize="10" page="1" pageWidth="1460" pageHeight="920">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="title" value="Momentum — current production architecture (TrueNAS compose, 2026-07 @ d54a2a9)" style="text;html=1;fontSize=18;fontStyle=1;align=left;" vertex="1" parent="1">
<mxGeometry x="40" y="0" width="900" height="30" as="geometry" />
</mxCell>
<mxCell id="browser" value="Operator browser" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#666666;" vertex="1" parent="1">
<mxGeometry x="60" y="60" width="180" height="60" as="geometry" />
</mxCell>
<mxCell id="telegram" value="Telegram" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#666666;" vertex="1" parent="1">
<mxGeometry x="300" y="60" width="160" height="60" as="geometry" />
</mxCell>
<mxCell id="feeds" value="Data feeds&#xa;Alpaca news &#183; RSS &#183; FMP float &#183; SEC EDGAR" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#666666;" vertex="1" parent="1">
<mxGeometry x="700" y="60" width="240" height="60" as="geometry" />
</mxCell>
<mxCell id="alpaca" value="Alpaca Paper API&#xa;orders &#183; account &#183; SIP bars/quotes" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" vertex="1" parent="1">
<mxGeometry x="1040" y="60" width="240" height="60" as="geometry" />
</mxCell>
<mxCell id="compose" value="TrueNAS SCALE &#8212; Docker Compose stack (image: fez5587/momentum-app)" style="swimlane;startSize=30;html=1;fillColor=none;strokeColor=#666666;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="40" y="180" width="1340" height="580" as="geometry" />
</mxCell>
<mxCell id="dashboard" value="Dashboard :8765&#xa;static JS + SSE&#xa;(read-only projections)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="compose">
<mxGeometry x="30" y="60" width="210" height="80" as="geometry" />
</mxCell>
<mxCell id="admin" value="Admin :8767&#xa;config overrides &#183; profiles&#xa;service control" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="compose">
<mxGeometry x="30" y="170" width="210" height="80" as="geometry" />
</mxCell>
<mxCell id="qabot" value="Telegram qabot&#xa;status Q&amp;A (read-only)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="compose">
<mxGeometry x="30" y="280" width="210" height="80" as="geometry" />
</mxCell>
<mxCell id="pg" value="PostgreSQL 18&#xa;events (append-only)&#xa;minute_bars (~19M) &#183; research tables" style="shape=cylinder3;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;boundedLbl=1;" vertex="1" parent="compose">
<mxGeometry x="480" y="160" width="240" height="130" as="geometry" />
</mxCell>
<mxCell id="nightly" value="Nightly scheduler&#xa;premarket_eval 20:00 ET&#xa;retention &#183; shadow tracks &#183; caches" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="compose">
<mxGeometry x="480" y="380" width="240" height="80" as="geometry" />
</mxCell>
<mxCell id="loop" value="Trading loop &#8212; run_live_paper.py" style="swimlane;startSize=30;html=1;fillColor=#f8f8f8;strokeColor=#6c8ebf;fontStyle=1;" vertex="1" parent="compose">
<mxGeometry x="810" y="50" width="490" height="350" as="geometry" />
</mxCell>
<mxCell id="schedthread" value="Scheduler thread&#xa;discover &#183; ingest &#183; watch &#183; arm &#183; guard" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="loop">
<mxGeometry x="20" y="40" width="210" height="70" as="geometry" />
</mxCell>
<mxCell id="trigthread" value="Trigger thread (4s)&#xa;live cross &#8594; submit_breakout_now" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="loop">
<mxGeometry x="260" y="40" width="210" height="70" as="geometry" />
</mxCell>
<mxCell id="exitmgr" value="LiveExitManager (12s)&#xa;BE ratchet &#183; trail &#183; EOD flatten" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="loop">
<mxGeometry x="20" y="140" width="210" height="70" as="geometry" />
</mxCell>
<mxCell id="execsvc" value="TradingExecutionService&#xa;gates &#183; sizing &#183; bracket orders" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="loop">
<mxGeometry x="260" y="140" width="210" height="70" as="geometry" />
</mxCell>
<mxCell id="syncsvc" value="AlpacaPaperSync (60s)&#xa;account &#183; positions &#183; orders &#183; journal" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="loop">
<mxGeometry x="140" y="250" width="210" height="70" as="geometry" />
</mxCell>
<mxCell id="e1" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="1" source="browser" target="dashboard">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="e2" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.1;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="browser" target="admin">
<mxGeometry relative="1" as="geometry">
<Array as="points">
<mxPoint x="50" y="390" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="e3" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="telegram" target="qabot">
<mxGeometry relative="1" as="geometry">
<Array as="points">
<mxPoint x="380" y="500" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="e4" value="projections (read)" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" edge="1" parent="1" source="dashboard" target="pg">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="e5" value="reads" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.75;entryDx=0;entryDy=0;" edge="1" parent="1" source="qabot" target="pg">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="e6" value="events &#183; bars" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="loop" target="pg">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="e7" value="writes &#183; purges" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="nightly" target="pg">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="e8" value="REST: bars &#183; quotes &#183; orders &#183; fills" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="1" source="alpaca" target="loop">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="e9" value="ingest" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.1;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="1" source="feeds" target="loop">
<mxGeometry relative="1" as="geometry">
<mxPoint as="offset" x="-10" y="0" />
</mxGeometry>
</mxCell>
<mxCell id="e10" value="overrides.env &#183; restart (docker sock)" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;dashed=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.9;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="admin" target="loop">
<mxGeometry relative="1" as="geometry">
<Array as="points">
<mxPoint x="460" y="690" />
<mxPoint x="1100" y="690" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="legend" value="Legend" style="rounded=0;whiteSpace=wrap;html=1;fillColor=none;strokeColor=#666666;verticalAlign=top;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="60" y="790" width="200" height="120" as="geometry" />
</mxCell>
<mxCell id="leg1" value="" style="rounded=0;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="legend">
<mxGeometry x="10" y="26" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="leg1t" value="Service / thread" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="24" width="140" height="18" as="geometry" />
</mxCell>
<mxCell id="leg2" value="" style="rounded=0;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="legend">
<mxGeometry x="10" y="48" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="leg2t" value="Datastore" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="46" width="140" height="18" as="geometry" />
</mxCell>
<mxCell id="leg3" value="" style="rounded=0;html=1;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="legend">
<mxGeometry x="10" y="70" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="leg3t" value="Scheduled job" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="68" width="140" height="18" as="geometry" />
</mxCell>
<mxCell id="leg4" value="" style="rounded=0;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" vertex="1" parent="legend">
<mxGeometry x="10" y="92" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="leg4t" value="Broker API (money path)" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="90" width="140" height="18" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>
Binary file not shown.

After

Width:  |  Height:  |  Size: 82 KiB

@@ -0,0 +1,117 @@
<?xml version="1.0" encoding="UTF-8"?>
<mxfile host="drawio" version="26.0.0">
<diagram name="Milestone feature set">
<mxGraphModel dx="1420" dy="900" grid="1" gridSize="10" page="1" pageWidth="1540" pageHeight="900">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="title" value="Momentum — next-milestone feature set: 7 workstreams and their dependencies" style="text;html=1;fontSize=18;fontStyle=1;align=left;" vertex="1" parent="1">
<mxGeometry x="40" y="0" width="960" height="30" as="geometry" />
</mxCell>
<mxCell id="ph0" value="FOUNDATIONS &#8212; ship first" style="text;html=1;fontSize=12;fontStyle=1;fontColor=#888888;align=center;" vertex="1" parent="1">
<mxGeometry x="60" y="50" width="300" height="20" as="geometry" />
</mxCell>
<mxCell id="ph1" value="SUBSTRATE &#8212; trust the data" style="text;html=1;fontSize=12;fontStyle=1;fontColor=#888888;align=center;" vertex="1" parent="1">
<mxGeometry x="500" y="50" width="300" height="20" as="geometry" />
</mxCell>
<mxCell id="ph2" value="FLAGSHIP" style="text;html=1;fontSize=12;fontStyle=1;fontColor=#888888;align=center;" vertex="1" parent="1">
<mxGeometry x="920" y="50" width="280" height="20" as="geometry" />
</mxCell>
<mxCell id="ph3" value="OPERATOR SURFACE" style="text;html=1;fontSize=12;fontStyle=1;fontColor=#888888;align=center;" vertex="1" parent="1">
<mxGeometry x="1280" y="50" width="220" height="20" as="geometry" />
</mxCell>
<mxCell id="ws5" value="WS5 &#183; Production safety rails [S]&#xa;&#xa;CI pytest gate before :latest&#xa;qabot dead-man's switch&#xa;scheduled pg_dump backup" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1">
<mxGeometry x="60" y="90" width="300" height="120" as="geometry" />
</mxCell>
<mxCell id="ws7" value="WS7 &#183; Dead-weight deletion [S]&#xa;&#xa;schwab/ &#183; grafana/ &#183; telemetry/ &#183; dead strategy modules&#xa;12 zero-ref tables &#183; 6 dead projections&#xa;rule: every table names its reader or dies" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1">
<mxGeometry x="60" y="280" width="300" height="130" as="geometry" />
</mxCell>
<mxCell id="ws4" value="WS4 &#183; Money-path hardening [S]&#xa;&#xa;daily self-restart (exit 0 after EOD flatten)&#xa;positive-evidence contract (empty &#8800; naked;&#xa;lost POST &#8594; by_client_order_id reconcile)&#xa;EOD broker-vs-events cage count &#183; SIGTERM" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1">
<mxGeometry x="60" y="480" width="300" height="140" as="geometry" />
</mxCell>
<mxCell id="ws2" value="WS2 &#183; Learning-flywheel resurrection [M]&#xa;&#xa;ONE population module (as-of-time, open-knowable)&#xa;consumers: live gate &#183; tuner &#183; sweeps &#183; replay &#183; labeler&#xa;schedule eod_replay + nightly_tune in production&#xa;atomic learned_params write + staleness alert" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1">
<mxGeometry x="500" y="120" width="330" height="150" as="geometry" />
</mxCell>
<mxCell id="ws3" value="WS3 &#183; Entry-path unification [M]&#xa;&#xa;one shared gate function, both entry paths&#xa;decision-receipt event per gate pass/fail&#xa;(closes VWAP/quality vs rank drift)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1">
<mxGeometry x="500" y="420" width="330" height="130" as="geometry" />
</mxCell>
<mxCell id="ws1" value="WS1 &#183; Experiment OS [L]&#xa;&#xa;experiment = config row (population &#183; intervention &#183;&#xa;pre-registered bars &#183; min-n &#183; futility)&#xa;counterfactual replay of every logged fire&#xa;counterfactual exit ledger (all exit variants per trade)&#xa;auto verdicts: PASS / FAIL / INCONCLUSIVE&#xa;&#xa;first cohort: 10-min time-stop &#183; probe-then-add&#xa;asymmetric exit (spread floor) &#183; one-bullet rank &#183; pm1 verdict" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;fontStyle=1" vertex="1" parent="1">
<mxGeometry x="920" y="220" width="320" height="230" as="geometry" />
</mxCell>
<mxCell id="ws6" value="WS6 &#183; Operator control plane [M]&#xa;&#xa;dashboard POST fix &#183; sparklines &#183; LIVE badge truth&#xa;kill switch: operator-command events&#xa;(flatten all / symbol &#183; pause entries)&#xa;experiment ledger panel&#xa;effective-config panel (per-key provenance)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1">
<mxGeometry x="1340" y="230" width="190" height="210" as="geometry" />
</mxCell>
<mxCell id="d1" value="trusted population" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" edge="1" parent="1" source="ws2" target="ws1">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="d2" value="decision receipts" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.75;entryDx=0;entryDy=0;" edge="1" parent="1" source="ws3" target="ws1">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="d3" value="verdict events" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="ws1" target="ws6">
<mxGeometry relative="1" as="geometry">
<mxPoint as="offset" x="0" y="-18" />
</mxGeometry>
</mxCell>
<mxCell id="d4" value="safe ships" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;dashed=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="ws5" target="ws2">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="d5" value="smaller surface" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;dashed=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="ws7" target="ws3">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="d6" value="promoted winners land on the unified gate" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;dashed=1;labelBackgroundColor=#ffffff;fontSize=11;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="ws1" target="ws3">
<mxGeometry relative="1" as="geometry">
<Array as="points">
<mxPoint x="1090" y="620" />
<mxPoint x="665" y="620" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="note4" value="WS4 is independent &#8212; ship in week 1 alongside WS5/WS7" style="text;html=1;fontSize=11;fontColor=#888888;align=left;" vertex="1" parent="1">
<mxGeometry x="60" y="630" width="330" height="20" as="geometry" />
</mxCell>
<mxCell id="legend" value="Legend" style="rounded=0;whiteSpace=wrap;html=1;fillColor=none;strokeColor=#666666;verticalAlign=top;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="500" y="680" width="360" height="140" as="geometry" />
</mxCell>
<mxCell id="lg1" value="" style="rounded=0;html=1;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="legend">
<mxGeometry x="10" y="26" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="lg1t" value="Foundation (ops / hygiene)" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="24" width="300" height="18" as="geometry" />
</mxCell>
<mxCell id="lg2" value="" style="rounded=0;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" vertex="1" parent="legend">
<mxGeometry x="10" y="48" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="lg2t" value="Money-path correctness" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="46" width="300" height="18" as="geometry" />
</mxCell>
<mxCell id="lg3" value="" style="rounded=0;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="legend">
<mxGeometry x="10" y="70" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="lg3t" value="Substrate (data / gate integrity)" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="68" width="300" height="18" as="geometry" />
</mxCell>
<mxCell id="lg4" value="" style="rounded=0;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="legend">
<mxGeometry x="10" y="92" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="lg4t" value="Flagship build (Experiment OS)" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="90" width="300" height="18" as="geometry" />
</mxCell>
<mxCell id="lg5" value="" style="rounded=0;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="legend">
<mxGeometry x="10" y="114" width="30" height="14" as="geometry" />
</mxCell>
<mxCell id="lg5t" value="Operator surface (:8765 / :8767 / qabot)" style="text;html=1;align=left;verticalAlign=middle;fontSize=11;" vertex="1" parent="legend">
<mxGeometry x="50" y="112" width="300" height="18" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>
Binary file not shown.

After

Width:  |  Height:  |  Size: 132 KiB

+29
View File
@@ -0,0 +1,29 @@
# Data-source key inventory (2026-07-18)
All keys below are in `.env` (gitignored). Most were provisioned but never
wired into code — the system uses only Alpaca (bars/quotes), FMP (float),
EDGAR/SEC full-text (filings), and RSS+Ollama (news). This catalogs what's
available so a future need pulls the right one instead of assuming absence.
**Triage — what's worth wiring, and what's redundant.** Do NOT wire all of
these; more sources = more complexity, not more edge (news was already only
weakly useful). Wire a source only when a specific hypothesis needs it.
| Key | What it is | Verdict |
|---|---|---|
| `DATABENTO_API_KEY` | **Nasdaq TotalView L2 depth (MBP-10)**, pay-as-you-go + free credits | **GAME-CHANGER — unblocks plan 017 Phase 0 at ~$0. Verified live 2026-07-18: covers the real gapper universe, fractions of a cent per fire.** |
| `POLYGON_API_KEY` | aggregated bars/quotes, some depth | Nice-to-have: cross-check / fallback for Alpaca bars; not urgent |
| `SEC_API_KEY` | sec-api.io — STRUCTURED filings (vs raw EDGAR full-text the dilution feed uses now) | Minor upgrade to `research/ingestion/sec_dilution.py` if that feed ever needs form-subtype precision; the raw feed already works |
| `Financial_Modeling_Prep_Api_Key` | FMP — float/fundamentals | **Already wired** (float_cache) |
| `FINNHUB_API_KEY` | news + fundamentals + some float | Redundant with FMP/news pipe |
| `Alpha_Vantage_Api_Key`, `Twelve_Data_Api_Key`, `Marketstack_Api_Key`, `EOD_Historical_Data_Api_Key` | bars/fundamentals/float, free tiers | Redundant — float refuted as a gate; bars come from Alpaca SIP |
| `NewsAPI_Api_Key`, `NewsData.io_Api_Key`, `Marketaux_Api_Key`, `MediaStack_Api_Key` | news feeds, free tiers | SKIP for now — 4 news APIs is complexity, not edge; RSS+Ollama already covers catalyst. Revisit only if a news-LATENCY hypothesis is registered |
**Bottom line:** one key (Databento) materially advances the project's founding
question (can premarket data give autonomous selection confidence — via the
order book). The rest are redundancy or unproven-need. Discipline over
accumulation: wire on demand, not on availability.
**Security:** these keys were pasted into a chat session 2026-07-18 — rotate
Databento + Polygon (billing feeds) when convenient. `.env` is gitignored;
never commit this file's real values (it holds none — names only).
+156
View File
@@ -0,0 +1,156 @@
# Entry-conversion research: what demonstrably improves breakout entry conversion in thin small-caps
**Date:** 2026-07-02
**Method:** deep-research fan-out — 5 parallel search angles (entry timing, confirmation filters, position management, stop placement, breakout-failure microstructure), ~20 primary sources fetched, load-bearing claims adversarially re-verified against primary sources.
**Context this feeds:** the bot's proven bottleneck is ENTRY-CONVERSION (7 refuted selection levers; 76–84% of ORB fires poke-then-stop; the one +EV subset found in-house is breakouts confirming within ~10–15 min of the open, +0.42R).
**Evidence-quality labels used throughout:** `ACADEMIC` (peer-reviewed or serious preprint), `BACKTEST#` (non-academic but real numbers, reproducible-ish), `GURU` (practitioner claim, no verifiable numbers, often selling something), `SEO` (content-farm fluff).
---
## Headline synthesis
Three independent literatures converge on one mechanism for why our fires poke-then-die:
> The breakout level is where resting **supply** is maximal (take-profit clusters, ATM-offering desks, distributing pumpers), while stop-loss orders cluster just beyond it. Merely *reaching* the level most often produces a rejection — the poke is the statistically expected outcome — while a genuine *crossing* that absorbs the shelf ignites a real trend. The reach-vs-cross question resolves fast (rejections play out within ~30 min).
- Osler (NY Fed, `ACADEMIC`, numbers verified against the paper): take-profits cluster ON round-number levels (9.9% of TP orders vs 3.8% baseline), stop-losses just beyond (14.3% of stop-buys priced in the 01–10 digit band vs 6.9% in 90–99). Prices reverse abnormally often upon merely *reaching* a level (59.3% vs 54.8% at arbitrary prices), and that rejection completes within ~30 min; genuine crossings, by contrast, trend significantly for **at least two hours** (FX data). Tradable content: poke-rejection is fast and the base case; a confirmed cross carries durable momentum. [sr150.pdf](https://www.newyorkfed.org/medialibrary/media/research/staff_reports/sr150.pdf)
- SmallCapLab (~3,000 small-cap gap-up events, `BACKTEST#`, short-biased vendor; verified with caveats): 64.2% of gap-ups close below their RTH open, ~72% below VWAP, **46.6% of highs-of-day print in the first 15 min, ~70% by 10:30** (the page's "85%+ by 10:30" caption contradicts its own cumulative chart at 69.6%; ~86% by 13:00); fade-severity buckets imply ~65% collapse ≥20% off HOD. The fade is the base case; the whole long game is the open. Universe caveat: gap ≥45% vs prior close, price ≥$0.30 — *bigger* gaps than the bot's ≤35%-gap universe, so directional only. [smallcaplab.com/research](https://www.smallcaplab.com/research)
- Zarattini/Barbon/Aziz (`ACADEMIC` working paper): 5-min opening range Sharpe 2.81 vs 15-min 1.43 vs 30-min 0.21 — the edge **decays fast with later entry**. [SSRN 4729284](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4729284)
All three externally corroborate the in-house +0.42R early-confirm finding. The literature's answer to "how do winners enter?" is *not* a cleverer trigger — every credibly profitable published system enters **immediately on the break** — it is (a) only trading the first minutes, (b) an asymmetric exit structure (tight-ish stop, no target, let the fat tail run to EoD), and (c) not paying full risk on unconfirmed pokes.
---
## 1. Breakout entry timing: on-break vs retest vs first-pullback
### What the evidence says
**Enter-on-break is the only variant with credible supporting numbers.**
- Zarattini/Barbon/Aziz 2024 ([SSRN 4729284](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4729284), `ACADEMIC` working paper, verified against source): resting stop order AT the 5-min OR high — no confirmation, no retest. Top-20 opening-RVOL "Stocks in Play", 2016–2023, 7,000+ stocks: **+1,637% net, Sharpe 2.81, ann. alpha ~36%**. Independent QuantConnect replication: Sharpe 2.40, **~17% per-trade win rate** (profit rides rare 10R+ winners) ([QuantConnect study](https://www.quantconnect.com/research/18444/opening-range-breakout-for-stocks-in-play/), `BACKTEST#`). Caveats: universe is price>$5 / >1M sh avg volume / ATR>$0.50 — *liquid* stocks-in-play, not our thin sub-$5 tier; **zero slippage assumed**; both authors sell trading education.
- The companion QQQ paper ([SSRN 4416622](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4416622)) likewise enters at the open of the 2nd 5-min candle with zero confirmation. Baseline: 24% win rate, +0.13R/trade.
**Retest / wait-for-pullback entries: the only direct head-to-head found says they are much worse.**
- Mesfin 2026 ([arXiv 2605.04004](https://arxiv.org/abs/2605.04004), `ACADEMIC` preprint, falsification-style with friction; numbers verified exact): MNQ futures, 947 days, 5-min bars, 25-min opening range. Immediate ORB entry: 51.9% win (fails net of friction; best immediate variant +2.82 pts at bar+15, still fails T≥2). **Pullback/retest entry: N=83, 19.3% win rate, 80.7% stop-out rate, −4.44 pts.** Paper's verdict: pullback entries are "systematically wrong" because "a high proportion of apparent breakouts simply fail and reverse." Caveats: index futures, unconditioned signal, small N, and the comparison is not perfectly apples-to-apples (pullback variant used a 20-pt stop while immediate variants used fixed-horizon exits) — but it is the only rigorous break-vs-retest comparison found anywhere.
- Bulkowski (`BACKTEST#`, informal daily-bar pattern stats): throwbacks/pullbacks occur after ~60–74% of breakouts and patterns **with** throwbacks *underperform* those without — waiting for the retest adversely selects the weak breakouts and misses the ~30–40% strongest moves that never come back ([thepatternsite.com](https://thepatternsite.com/fallwedge.html)).
- Mechanistically (Mesfin): entering a bar or more after the trigger is structurally post-exhaustion — expansion-bar continuation tested *actively wrong* (T=−10.96 at bar+1; measured in the Asia session, so context caveat applies); the move is largely contained in the trigger bar. Entry latency consumes the edge.
**First-pullback / bull-flag advocacy is evidence-free.** All numeric claims circulating for intraday small-cap flags (~52–75% win rates) trace to `SEO`/`GURU` content with no reproducible methodology. Warrior Trading's Ross Cameron publishes CPA-reviewed *aggregate* P&L ($18.8M 2017–2025 from $583) but **no audited per-setup statistics** — and the FTC settled with Warrior Trading for **$3M in 2022 over deceptive earnings claims** ([warriortrading.com earnings page](https://www.warriortrading.com/ross-camerons-verified-day-trading-earnings/), [FTC action]) — n=1 discretionary trader, course-seller conflict. This is consistent with our own no-chase VWAP-pullback backtest coming out negative after cost, and with the VWAP-reclaim shadow track's do-not-promote verdict.
### Verdict
| Variant | Evidence | Direction |
|---|---|---|
| On-break (stop order at trigger) | ACADEMIC + replication | Only variant with a documented net edge (when paired with selection + exit structure) |
| Wait-for-retest | ACADEMIC (1 study) + BACKTEST# | Materially worse; adverse selection |
| First-pullback (flag) | GURU only | Untested anywhere credible; our own test negative |
**Testable rule (mostly a stop-doing rule):** do not build retest/pullback entry logic; keep the on-break trigger and attack conversion via the time window, exit structure, and sizing below.
---
## 2. Confirmation at the moment of entry (1-min OHLCV computable)
**The only confirmation filter with credible published numbers is opening relative volume**, and it is a *selection* filter, not a trigger filter: Zarattini et al. define RVOL = first-5-min volume ÷ 14-day average first-5-min volume. Trades with RVOL<100% averaged **−0.02R**; RVOL>100% averaged **+0.08R** (`ACADEMIC`, verified). Computable: yes (needs 14 days of per-symbol history). Note: this exact definition (same-minutes-of-day baseline) may differ from the bot's current RVOL — worth checking the implementation matches before assuming this box is ticked.
**Bar-close-above-trigger vs intrabar poke: empirically OPEN.** No published head-to-head with numbers exists despite targeted searching. Every "wait for the close above" claim traces to unbacktested blog assertions (`SEO`). Meanwhile the best-documented ORB system fills *intrabar* on the poke and works. Given Mesfin's latency finding (one bar late = post-exhaustion), waiting 1 minute for a close plausibly costs more than it saves — but this is cheaply testable in-house on logged fires.
**Close-location-in-range (CLR = (close−low)/(high−low)) carries a warning label:** on daily bars in liquid stocks, closing near the high is a *mean-reversion sell* signal — IBS>0.8 fade has worked for two decades (Alvarez, `BACKTEST#`, [IBS post](https://alvarezquanttrading.com/blog/internal-bar-strength-for-mean-reversion/)). High-RVOL gapper minutes may flip the sign, but "strong close = confirmation" is not a market universal.
**Academic base rate for volume confirmation is directional only:** Lo/Mamaysky/Wang 2000 and Gervais/Kaniel/Mingelgrin 2001 (`ACADEMIC`, peer-reviewed) show volume carries incremental information — at day-to-month horizons on 1962–1996 liquid stocks. No intraday entry-bar numbers. The practitioner heuristics (breakout-bar volume ≥1.2–1.3× 20-bar average; price above rising VWAP; CLR in top 30%) are coherent mechanics with **zero published before/after deltas** (`GURU`/`SEO`).
**Testable rules (pre-register, shadow only):**
1. `confirm_close`: require the trigger-minute bar to CLOSE above trigger (vs current intrabar fire) — measure fill-price cost vs whipsaw avoided on logged fires.
2. `vol_surge`: breakout-bar volume ≥ 2× median of prior 20 one-min bars.
3. `clr`: trigger-bar CLR ≥ 0.7.
Each judged on the standard bars (+1R rate, expectancy net of cost, vs same-day ORB baseline). Expectation set LOW: these are folklore until proven on our fires.
---
## 3. Position management: scaling in/out, probes, pyramids
**Scale-OUT (Warrior-style "sell partials into strength") is the best-refuted idea in this whole research pass.** Every source with actual numbers says it reduces expectancy; it buys variance-reduction and win-rate optics:
- Howard Bandy via Bulkowski (`BACKTEST#`, 4,176 trades, 72 liquid tickers 2003–2008): the closer the first target, the more average profit is reduced; BE-stop-after-partial cut profits to **one-third**. Bulkowski: "If you want to make less profit, then scale out of a trade." ([thepatternsite.com/ScalingOut.html](https://thepatternsite.com/ScalingOut.html))
- Alvarez (`BACKTEST#`): scaling out → "large drop in CAR, no change or larger MDD" ([post](https://alvarezquanttrading.com/blog/adding-stops-and-scaling-out-to-a-mean-reversion-strategy/)).
- Kevin Davey (`BACKTEST#`, 567k exit backtests, 40 futures markets): all six *combination* exits were consistently worse than simple single exits ([kjtradingsystems.com](https://kjtradingsystems.com/algo-trading-exits.html)).
- The arithmetic identity: taking half off at +1R turns 2R winners into 1.5R; it only pays if edge *shrinks* as the move extends. Our book is the opposite regime — rare fat winners pay for everything (runner-grade forensics) — which makes the anti-scale-out math *stronger* for us. Warrior Trading's scale-out advice optimizes a discretionary human's psychology, not a bot's expectancy; they publish no backtest.
**Scale-IN (half-size probe at trigger + add on confirmation): theoretically right, empirically unpublished.** Kelly logic supports staged betting when confirmation genuinely raises P(win); trend-following pyramiding literature (Concretum, `BACKTEST#`, multi-decade futures) finds adds lower win rate but raise profit factor via 2–3× larger winners. **No credible published backtest of probe-then-add for intraday small-cap momentum exists** — either way. But it attacks exactly our measured problem: paying full risk on the 76–84% of fires that are unconfirmed pokes, while our own data says confirmation-within-10–15-min flips the subset to +0.42R.
**Testable rule (`probe_add`):** enter 0.5× normal size at trigger; add the remaining 0.5× only if within 10 min the position (a) has not touched the stop and (b) prints a new high above the trigger bar's high (or holds above VWAP). Compare blended R and net expectancy vs all-in baseline on the same fires. Trivially implementable with 1-min bars + two orders.
---
## 4. Stop placement for whipsaw-prone names
**Theory anchor (Kaminski & Lo, `ACADEMIC`, J. Financial Markets 2014):** under a random walk any stop-loss rule strictly lowers expected return; under positive serial correlation (momentum) stops add value ([SSRN 968338](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=968338)). So Alvarez's famous "stops hurt" results (`BACKTEST#`, mean-reversion daily bars: every 3–15% stop reduced CAR — [post](https://alvarezquanttrading.com/blog/maximum-loss-stops-do-you-really-need-them/)) do NOT transfer to our momentum entries; expect the opposite sign. Keep stops; the question is width and type.
**The best-evidenced production configuration is: tight volatility-fractional stop + NO profit target + end-of-day exit.**
- Stocks-in-play paper: stop = **10% of 14-day ATR**, no target, EoD exit, size = 1% risk ÷ stop distance with 4× leverage cap → Sharpe 2.81. Per-trade win rate is LOW (~17% in the replication); the system is a low-win-rate, fat-right-tail machine. EoD-exit beat every fixed R-target in both Zarattini papers.
- **The tight-stop artifact is published and author-admitted:** in the QQQ paper's sweep, the best cell was the tight end of the stop range (5% of ATR ≈ $0.08 on TQQQ, +9,350%) under **zero slippage**, and the authors themselves flag it verbatim as "unrealistic as our model assumed no slippage… will likely be exceeded." This is exactly the artifact we hit with the VR track. Any in-house tight-stop test must enforce a spread-aware floor: `stop_width ≥ max(k_ATR × ATR14, m × spread_proxy)` where spread_proxy for thin names can be estimated from 1-min high-low of quiet bars.
- Note the accounting identity: with fixed-fractional sizing, stop width and position size are the same knob until a leverage/notional cap binds. "Tight stop, small dollar loss, huge size" is what makes the Zarattini machine work on liquid names — and what slippage destroys on thin ones. A calibrated middle (e.g. 20–30% of ATR14 with the spread floor) is the honest test.
**Breakeven moves: near-noise in published tests** (0.3% of trades touched in one study; "heavily cuts profitability" in another — `BACKTEST#`/`GURU` mixed). On whipsaw small caps a BE@+5% will scratch far more often — our shipped BE rule is worth measuring, not assuming.
**Time stops: evidence gap.** No credible published study of "exit if no progress in N minutes" for intraday small caps. EoD exit is the only well-evidenced time stop. But Osler's fast poke-rejection window (~30 min) + SmallCapLab's HOD-in-first-15-min curve + our own +0.42R early-confirm subset all point the same direction, making this the highest-prior in-house experiment:
**Testable rules:**
1. `time_stop`: exit at market if the trade has not reached +0.5R (alt: has not printed a new post-entry high) within 10 min of fill. Measures: expectancy delta, avg loss on scratched trades vs their counterfactual stop-outs.
2. `atr_stop_floor`: stop = max(0.25 × ATR14_daily_fraction, 3 × spread_proxy), no profit target, trail-to-EoD — Zarattini structure adapted with an honest fill floor; pre-register against the tight-stop artifact (require the edge to survive +1 tick adverse fill on entry AND stop).
3. `eod_runner`: for confirmed trades (passed the 10-min gate), remove the profit target; hold with trailed structure stop into the last hour (Gao et al. intraday momentum: payoff concentrates late day — `ACADEMIC`, index-level caveat).
---
## 5. Why breakouts fail (microstructure) and what precedes the ones that hold
- **Reach-then-reject is the base case; real crossings trend** (Osler, `ACADEMIC`): stops cluster just beyond salient levels, take-profits sit on them. Prices that merely *reach* a level reverse abnormally often, with the rejection completing within ~30 min; prices that genuinely *cross* keep trending for hours (FX). The whipsaw is not noise — it is the statistically dominant outcome of touching a supply shelf, and the fast resolution window is why early confirmation is the +EV subset. (Note: an earlier read of this paper as "cascade fuel gone by 60 min" was checked against the source and inverted — post-crossing trends persist; it is the *rejection* that is fast.)
- **The level itself is a supply shelf**: take-profits cluster ON round numbers (Osler); in dilution-overhung small caps, ATM desks sell into spikes (DilutionTracker, `GURU`, unquantified but mechanistically sound — and EDGAR S-3/424B5 presence is checkable, ties to the existing EDGAR-dilution roadmap item); in pumped names the breakout print *is* the manipulator's exit liquidity (Korea order-book study, `ACADEMIC`, [SSRN 4173353](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4173353)); heavily retail-herded names average **−4.7%** 20-day abnormal returns (Barber et al. JF 2022, `ACADEMIC`).
- **Base rates say fade is the default** (SmallCapLab, `BACKTEST#`, vendor-biased, gap ≥45% universe): 64% of gappers close red vs open; only 41.5% ever break the premarket high; bigger gaps collapse harder; **higher premarket volume ⇒ harder fade** (56.2% → 65.6% → 71.5% across PM-volume buckets, small n in places) — externally corroborates our refuted RVOL/gap selection levers.
- **Conditions preceding holds, computable from our bars:** minutes-since-open (dominant), premarket-high break (binary level of real significance), distance of trigger to round numbers ($0.50/$1.00), gap size (inverse), price tier ($10+ most resilient, sub-$1 fades hardest). Conditions that do NOT predict holding: premarket volume, RVOL beyond the >100% floor, float — consistent with all seven in-house refutations.
**Testable rule (`round_number`):** log distance from trigger price to nearest $0.50/$1.00 increment on every fire; test whether fires triggering just *below* a round number (into the take-profit shelf) underperform fires triggering just *above* one (through the stop cluster). Free to log, zero risk.
---
## Ranked shortlist: 3 most promising, implementable on 1-min OHLCV
**1. Time-boxed confirmation: entries only in the opening window + a 10-minute time-stop on unconfirmed positions.**
Strongest convergence in the entire pass: our own +0.42R early-confirm subset, Osler's ~30-min poke-rejection resolution (`ACADEMIC`), SmallCapLab's 47%-of-HODs-in-first-15-min curve (`BACKTEST#`), and Zarattini's 5-min≫15-min≫30-min Sharpe decay (`ACADEMIC`). Rule: take ORB fires only ~9:30–9:50; exit at market any position that hasn't made a new post-entry high (or +0.5R) within 10 min. Directly converts the known 76–84% poke-then-die population from full-stop losses into small scratches. Cheapest to test: pure replay of logged fires.
**2. Probe-then-add sizing: 0.5× at trigger, add 0.5× only on 10-minute confirmation.**
The only technique that structurally exploits the discovered +EV confirm-subset without needing to *predict* it. Theory-consistent (Kelly staged betting, pyramiding profit-factor results); no published intraday small-cap evidence either way, so shadow-first with pre-registered bars. Halves risk on the ~80% dead pokes at the cost of a worse blended entry on winners — the trade-off is measurable on existing fire logs before any live change.
**3. Asymmetric exit restructure: volatility-fractional stop with a spread floor, NO profit target, trail to EoD; kill any scale-out.**
The published edge (Sharpe 2.81, replicated) lives in the exit asymmetry, not the entry: tight-ish stop, ~17–25% win rate, fat right tail runs to the close. Scale-out is refuted by every numeric source (Bandy/Alvarez/Davey). MUST be tested with an adverse-fill/spread floor (`stop ≥ max(0.25×ATR14, 3×spread_proxy)`, +1-tick adverse fills) — the tightest-stop-wins result is a documented zero-slippage artifact, the same artifact class that killed the VR and liq1 reads.
**Explicit do-nots from this pass:** no retest or first-pullback entry logic (refuted/evidence-free; matches our negative in-house test); no scale-out partials; no adoption of close-above-trigger or volume-multiple confirmation from folklore without an in-house pre-registered test.
---
## Source list
| Source | Type | URL |
|---|---|---|
| Zarattini, Barbon & Aziz — Profitable Day Trading Strategy (stocks-in-play ORB) | Academic WP | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4729284 |
| Zarattini & Aziz — Can Day Trading Really Be Profitable? (QQQ ORB) | Academic WP | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4416622 |
| QuantConnect replication of stocks-in-play ORB | Backtest | https://www.quantconnect.com/research/18444/opening-range-breakout-for-stocks-in-play/ |
| Mesfin — ORB/retest falsification study, MNQ | Academic preprint | https://arxiv.org/abs/2605.04004 |
| Osler — Stop-Loss Orders and Price Cascades (NY Fed SR150) | Academic | https://www.newyorkfed.org/medialibrary/media/research/staff_reports/sr150.pdf |
| Kaminski & Lo — When Do Stop-Loss Rules Stop Losses? | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=968338 |
| Gao, Han, Li & Zhou — Intraday Momentum | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2552752 |
| Heston, Korajczyk & Sadka — intraday periodicity | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1107590 |
| Lo, Mamaysky & Wang — Foundations of Technical Analysis | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=228099 |
| Gervais, Kaniel & Mingelgrin — High-Volume Return Premium | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=146468 |
| Barber, Huang, Odean & Schwarz — Attention-Induced Trading (Robinhood herding) | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3715077 |
| Lee/Lee/Kim — pump-and-dump order-book study (Korea) | Academic | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4173353 |
| Bulkowski — Scaling Out (incl. Bandy study) | Backtest | https://thepatternsite.com/ScalingOut.html |
| Alvarez — Adding Stops and Scaling Out (MR) | Backtest | https://alvarezquanttrading.com/blog/adding-stops-and-scaling-out-to-a-mean-reversion-strategy/ |
| Alvarez — Maximum Loss Stops | Backtest | https://alvarezquanttrading.com/blog/maximum-loss-stops-do-you-really-need-them/ |
| Alvarez — Internal Bar Strength | Backtest | https://alvarezquanttrading.com/blog/internal-bar-strength-for-mean-reversion/ |
| Davey — exit study (567k backtests) | Backtest | https://kjtradingsystems.com/algo-trading-exits.html |
| Concretum — pyramiding vs vol targeting | Backtest | https://concretumgroup.com/position-sizing-in-trend-following-comparing-volatility-targeting-volatility-parity-and-pyramiding/ |
| SmallCapLab — gapper base-rate research | Vendor backtest | https://www.smallcaplab.com/research |
| Warrior Trading — verified earnings page | Guru (aggregate audited) | https://www.warriortrading.com/ross-camerons-verified-day-trading-earnings/ |
| CXO Advisory — ORB review | Review | https://www.cxoadvisory.com/technical-trading/day-trading-with-an-opening-range-breakout-strategy/ |
+181
View File
@@ -0,0 +1,181 @@
<!-- Distilled 2026-07-17 from books/order_flow_microstructure/ (9 sources; O'Hara
image-only PDF unreadable). 9 parallel reader agents + adversarial synthesis, hard-constrained
to data the system actually has (1-min SIP bars, 15s quote snapshots) and fenced from the
8 refuted levers. Companion to regime_notes.md (plan 011's source). Next step: pick
hypotheses, register as pre-registered experiments (experiments/*.json), let nightly
adjudication decide. -->
> **VERDICTS (2026-07-17, first adjudication under deployed exit config, n=2044 v1 fires):**
> - **H1 trigger-erasure: FAIL, mechanism INVERTED here.** 76% of all fires close back below
> the trigger within 5 bars — on $2-20 gappers the recross is NORMAL price action, not a
> death signature. The rule clipped 46% of eventual winners (kill bar: 15%); delta −0.87R.
> Fast recross-invalidation is proven HARMFUL on this population. This also pre-kills H5's
> stall-tightening leg unless it behaves fundamentally differently.
> - **H2 flow-persistence: formal PASS (+0.107R) — REFUTED IN SUBSTANCE by its own
> pre-registered checks.** The kept population still loses (−0.19R/fire vs zeroed −0.24 —
> barely separated), and within BOTH gap strata the sign INVERTS (choppy beats persistent:
> 10-20% −0.03 vs −0.09; 20%+ −0.06 vs −0.26). The pooled tercile gradient is composition,
> not signal (Simpson's). Do not promote. STRUCTURAL LESSON: on a −0.21R baseline, ANY gate
> that zeroes ~44% of fires clears the +0.05 bar mechanically (908/2069 x 0.24 = +0.105);
> keep-or-zero experiments need a random-gate null, not just expR_delta_min.
> - **H4 volume-spike: FAIL clean** (−0.008R; concentration gate separates nothing).
> - **H3 spread-expansion: INCONCLUSIVE (n=29), accruing ~30 fires/day by design** — the
> skip-on-no-coverage guard is working; decisive in ~2 weeks.
> Running tally: conversion levers tested 12, surviving 0 (the regime gate remains the only
> OOS survivor, and it is session-level, not conversion). The transitory-impact mechanism
> (convergent mechanism #1) is looking less like a filterable signature and more like the
> population's base nature.
# Microstructure Shelf → ORB Entry-Conversion: Synthesis Note
The bot's proven #1 problem is entry conversion: triggers fire but only ~35-48% of breakouts reach +1R before -1R, and eight selection-side levers have been refuted — the population is fine, what happens AT and AFTER the breakout is the bottleneck. This shelf (Hasbrouck, Bouchaud et al., Cartea et al., Cont-Kukanov-Stoikov, Dalton, Bellafiore, Neill, Cetin-Danilova) is unusually well-matched to that problem, because market microstructure is precisely the study of which price moves stick (permanent, information-driven impact) and which revert (transitory, liquidity-driven impact). The academic and practitioner books converge, from opposite directions, on the same handful of observable signatures — and critically, most are computable from 1-min bars and 15s quote snapshots without tick data. Everything below is either a conditioning gate on already-fired entries or an exit-config variant; nothing re-opens the refuted selection axis.
## Convergent mechanisms
**1. Failed breakouts are transitory impact; the default fate of a one-shot volume burst is reversion.** (Hasbrouck ch.8/9/15; Bouchaud ch.12.4; Cartea ch.4/10.4) Price moves decompose into a permanent (information) component and a transitory (liquidity) component that mean-reverts once the initiating flow stops. Bouchaud is bluntest: after a metaorder completes, impact decays abruptly — reversion is the empirically *normal* outcome. A ~50-65% breakout failure rate is not a bug in the setup; it is the base rate for one-shot bursts. Continuation requires *sustained* flow.
**2. Return into the broken range is the structural invalidation signal, and it arrives before the stop.** (Dalton — balance-area re-entry; Bellafiore — fake-breakout recross at AIG $25/AMZN $86.60; Neill — false moves; Hasbrouck — the transitory signature *is* the round-trip) Four independent traditions say the same thing: once price trades back through the level it broke, the participants who drove the break are gone and the trade thesis is dead — waiting for the fixed stop is paying extra for the same information.
**3. Continuation is fed by persistent same-side flow, not by the size of the trigger print.** (Bouchaud ch.10 — long-memory order flow from order splitting/herding; Cartea ch.4/12 — same-side MO clustering, imbalance regimes with 0.97-0.98 persistence; Hasbrouck ch.9 — momentum-feedback impulse responses that keep rising; Neill — volume growing on the advance) The discriminator between real and fake breakouts is whether one-directional participation was *ongoing* into the trigger, versus a single isolated spike.
**4. Spread dynamics around the trigger carry real-time information — but the books disagree on the sign.** (Cartea ch.2, Hasbrouck ch.5: widening = Glosten-Milgrom adverse-selection pricing = informed flow = bullish for follow-through. Bouchaud ch.22, Cont-Kukanov-Stoikov (β~1/D), Bellafiore: widening = liquidity pulling back / thin book / chop-zone fragility = bearish.) Five sources agree the spread's *relative* behavior at trigger is informative; the direction is an empirical question and must be pre-registered two-sided.
**5. Speed and shape of the initial rejection through the level grades conviction.** (Dalton — Open-Drive vs Open-Auction taxonomy, tails "made by time" carry no conviction; Neill — gradual advance on steady volume = durable buying, vertical spike = short-covering; Bellafiore — held-bid absorption precedes real breaks) A decisive drive that doesn't look back differs measurably, at the moment it happens, from a limp tag of the level — this is same-bar shape, not delayed confirmation.
**6. Heavy volume with no price progress = exhaustion, at any timescale.** (Neill — "churning"; Dalton — time-beyond-level without acceptance; Bellafiore — chop zones) A *joint* volume-high + price-stall signature — explicitly distinct from the refuted volume-decline fade.
**7. The open is a structurally thin, high-impact regime; every quote-based feature must be normalized to the name's own trailing baseline.** (Bouchaud ch.4.2 intraday U-shape; CKS — depth 2x lower, impact 5x higher 9:30-10:00; Bellafiore — time-of-day regimes) Not a lever itself — a normalization requirement that guards every hypothesis below against re-deriving the refuted liquidity floor.
## Ranked hypotheses
### H1 — Trigger-erasure / recross fast invalidation (exit_config)
**Convergence: 5 books (Dalton, Bellafiore, Neill, Hasbrouck, Bouchaud). Strongest on the shelf.**
*Statement:* Fires where a post-entry 1-min bar closes back below the ORB trigger price within the first 5 minutes are exhibiting the transitory-impact / balance-re-entry / false-move signature and are disproportionately headed to -1R; exiting on the first such close (rather than riding to the stop) reduces average realized loss without materially truncating winners.
*Computable features (1-min bars only):* `trigger_price` from entry-gate receipts; per post-fill bar, `erased = close < trigger_price`; `first_erasure_bar_index` within a 5-bar window; secondary variant `recross_count` = number of bars whose low trades back through trigger. Trigger must be defined from the OR-high, independent of the stop price (no tautology).
*Draft pre-registration:*
```
population: {min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}
intervention: {kind: exit_config, rule: "exit at market on first 1-min bar CLOSE < trigger_price within 5 bars post-fill; else standing exit stack", sweep: {window_bars: [3, 5], condition: [close_below, low_through]}}
bars: {expR_delta_min: 0.05, min_n: 300}
fidelity: {atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}
```
*Expected direction:* Erasure fires convert to +1R at a materially lower rate; the early-exit variant lifts fire-weighted expR by shrinking the loser tail.
*Kill criteria:* expR delta < +0.05 across the sweep; OR lost-winner rate > 15% of eventual +1R trades exited by the rule (erasure-then-recovery fakeouts eating the gain); OR the rule's benefit vanishes when BE@+5% ratchet is active (redundant with existing protection).
*Confound guards:* Family resemblance to refuted volume-fade exits — but this is price-structure, not volume, and must independently clear the same bar. Measure lost-winner rate explicitly, not just loss reduction. Use consolidated bar closes, not quote mids, to avoid bid-ask bounce on thin names. Report separately by OR-width tercile so wide-range chop doesn't drive the pooled result.
### H2 — Pre-trigger order-flow persistence gate (keep-or-zero)
**Convergence: 4 books (Bouchaud, Cartea, Neill, Hasbrouck).**
*Statement:* Fires preceded by persistent same-direction bar-level flow (volume-weighted sign agreement over the run-up window) are the signature of an ongoing metaorder/herd and convert better than fires whose pre-trigger flow was choppy/balanced — the trigger is a mid-stream observation of sustained flow, not a one-off spike.
*Computable features (1-min bars only):* Over the last 15 closed pre-trigger bars: `sign_i = sign(close_i − open_i)`; `imbalance_persistence = |Σ sign_i · vol_i| / Σ vol_i`; secondary `agree_frac` = fraction of bars with sign matching breakout direction. No look-ahead into the trigger bar.
*Draft pre-registration:*
```
population: {min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}
intervention: {kind: keep_or_zero_gate, rule: "zero fires in the bottom imbalance_persistence tercile", stratify_by: [gap_pct_bucket, rvol_bucket]}
bars: {expR_delta_min: 0.05, min_n: 300}
fidelity: {atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}
```
*Expected direction:* Top tercile beats bottom tercile on +1R-before--1R rate; zeroing the bottom tercile lifts kept-population fire-weighted expR ≥ +0.05.
*Kill criteria:* Monotonicity fails across terciles; OR the spread between top and bottom tercile collapses inside gap%/RVOL strata (i.e., it was a refuted selection lever in disguise); OR fewer than min_n fires per tercile.
*Confound guards:* Bar-sign is a crude tick-rule proxy — accept the noise, demand the effect survive stratification by gap% and RVOL (the refuted axes) before believing it. Check redundancy against the surviving runner-grade before shipping as an independent lever.
### H3 — Spread-expansion-ratio at trigger (keep-or-zero, two-sided)
**Convergence: 5 books cite the mechanism (Cartea, Hasbrouck, Bouchaud, CKS, Bellafiore) — but they split on direction, so this is registered two-sided.**
*Statement:* The spread's behavior at trigger, normalized to the name's own trailing baseline, discriminates converting from failing breakouts. Adverse-selection theory (Cartea/Hasbrouck) predicts widening = informed flow = keep; fragility theory (Bouchaud/CKS/Bellafiore) predicts widening = liquidity vacuum = zero. The test determines which regime dominates $2-20 gappers.
*Computable features (15s quote snapshots):* `spread_baseline = median((ask−bid)/mid) over the 10 min pre-trigger`; `spread_at_trigger` from the last snapshot at/before trigger; `expansion_ratio = spread_at_trigger / spread_baseline`; secondary post-fill variant `post_ratio(1min) = spread at t0+1min / baseline`.
*Draft pre-registration:*
```
population: {min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}
intervention: {kind: keep_or_zero_gate, rule: "two-sided: compare top vs bottom expansion_ratio tercile; zero the losing tercile only if pre-registered margin met", normalize: per_name_baseline, stratify_by: [price_bucket, rvol_bucket, minutes_since_open]}
bars: {expR_delta_min: 0.05, min_n: 300}
fidelity: {atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}
```
*Expected direction:* Two-sided by design; a tercile spread in conversion rate that is stable across the backtest window and survives stratification, in either direction.
*Kill criteria:* No stable tercile ordering across time halves of the window; OR the effect disappears within price/RVOL strata (= liquidity floors relitigated → discard permanently); OR >20% of fires lack a usable pre-trigger snapshot baseline.
*Confound guards:* Ratio-to-own-baseline only, never absolute bps (absolute spread = refuted liquidity floor). Stratify by minutes-since-open (Bouchaud/CKS U-shape mechanically widens everyone's open spread). Test for independent value vs H1 — post-fill spread blowout and price erasure are correlated symptoms of the same reversal.
### H4 — Volume ramp vs isolated-spike trigger (keep-or-zero)
**Convergence: 3 books (Bouchaud, Neill, Hasbrouck).**
*Statement:* Fires where volume built progressively across the pre-trigger bars (a ramp — consistent with an unfinished metaorder still executing) convert better than fires where nearly all volume arrived in the single trigger bar with no build-up (a completed one-shot burst whose impact, per Bouchaud, reverts once done; per Neill, the vertical short-covering spike that "dies at the top").
*Computable features (1-min bars only):* `concentration_ratio = trigger_bar_volume / Σ(volume of preceding 5 bars)`; `ramp_slope = sign of linear volume trend over the last 5 pre-trigger bars`. Strictly pre-trigger — no post-entry volume conditioning (keeps it categorically distinct from the refuted volume-fade exit).
*Draft pre-registration:*
```
population: {min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}
intervention: {kind: keep_or_zero_gate, rule: "zero fires in the top concentration_ratio tercile (isolated-spike triggers)", stratify_by: [rvol_bucket, gap_pct_bucket]}
bars: {expR_delta_min: 0.05, min_n: 300}
fidelity: {atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}
```
*Expected direction:* Isolated-spike (high concentration) fires convert worse; ramp fires convert better; zeroing spikes lifts kept expR.
*Kill criteria:* No monotone relation across concentration terciles; OR effect collapses within RVOL strata; OR feature is >0.7 correlated with H2's imbalance_persistence on the fire population (redundant — keep only the stronger).
*Confound guards:* Must survive RVOL stratification (day-level RVOL is a refuted axis; this is intrabar structure). Check against how the gap formed premarket (instant-gap vs premarket-ramp names) to avoid re-deriving runner-grade. Single odd-lot block prints on thin names can fake a spike — winsorize trigger-bar volume at a per-name percentile.
### H5 — Post-entry churn/stall exhaustion tightening (exit_config)
**Convergence: 3 books (Neill, Dalton, Bellafiore).**
*Statement:* Positions showing, within the first 3 minutes post-fill, either (a) no new 1-min bar high above the entry bar's high (Dalton: no range extension = weak other-timeframe control), or (b) volume spiking above baseline while price makes no net progress (Neill: churning), are disproportionately failures; tightening the stop to breakeven-minus-buffer on that signal cuts loser size without materially cutting winners.
*Computable features (1-min bars only):* `entry_bar_high`; `first_new_high_bar_index` within 3 bars; joint churn flag on bars t0+1, t0+2: `vol_ratio = bar_vol / trailing_15bar_mean > 2.0 AND |close − entry_price|/entry_price < 0.5 × trailing_bar_range_mean`.
*Draft pre-registration:*
```
population: {min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}
intervention: {kind: exit_config, rule: "on stall (no new high in N bars) OR churn flag, tighten stop to entry − 0.25×ATR; sweep N in [2,3,4]", baseline: current_stop_stack}
bars: {expR_delta_min: 0.05, min_n: 300}
fidelity: {atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}
```
*Expected direction:* Average realized loss on eventual losers shrinks; eventual-winner capture rate stays within 10% of baseline; net fire-weighted expR ≥ +0.05.
*Kill criteria:* Winner-capture drops >10% at every N in the sweep (whipsaw cost exceeds loss savings); OR the churn leg alone reproduces the refuted volume-fade result (volume component fires without the price-stall component adding anything — then drop the volume leg and keep only price-stall, or kill entirely); OR gains are fully redundant with H1 erasure exits on the same trades.
*Confound guards:* This is a *joint* high-volume+stall or no-new-high signature, never volume-decline (the refuted fade). Run H1 first; only fires H1 doesn't already catch count toward H5's incremental value. Sensitivity-sweep N — real winners on small caps routinely chop 2-3 bars before extending.
## Rejected during synthesis
- **Gap-still-open-at-trigger gate (Dalton)** — single source; heavily confounded with time-since-open and gap% (refuted axis); clean but keep on the bench behind the five above.
- **Round-number proximity gate (Cartea)** — single source; evidence from a $450 large-cap at penny resolution, transfer to $2-20 names unvalidated; mechanically confounded with price level.
- **Time-since-open depth-regime gate (CKS)** — single source; collides with the dead 10:30-cutoff history and with early-trigger/gap% correlation; refuted-adjacent selection flavor.
- **Pre-trigger quote-size imbalance (Cartea/CKS)** — hard dependency on bid_size/ask_size existing in the snapshot logs; unverified = dead on arrival; the paper's OFI variable explicitly needs queue data we don't have.
- **Volume-node thickness at trigger (Bellafiore)** — single source; volume-at-price concentration is RVOL in new clothes without careful shape-normalization; weakest power.
- **Overhead same-day resistance distance (Neill)** — single source, and the book itself flags naive levels as unreliable "until confirmed by the action of the stock."
- **Bid-erosion early exit (Bellafiore)** — single source; 15s sampling too coarse for "held bid"; largely restates mark-to-market drawdown; folded conceptually into H1/H3.
- **Impulse decay half-life adaptive stops (Hasbrouck)** — 5 noisy data points per fire can't support a fitted decay rate; strictly a refinement of H1 if H1 shows signal.
- **Trigger-bar close-location drive gate (Dalton) / trigger-bar volume conviction (Neill)** — too close to the refuted close-above-trigger confirmation family; their live content (shape/participation quality) is already carried by H2 and H4 with better mechanism backing.
- **Everything from Cetin-Danilova** — existence-proof mathematics over unobservable signed order flow; zero computable features.
## Data upgrades that would unlock more
1. **Verify whether bid_size/ask_size are in the 15s quote snapshots** — if yes, the CKS order-flow-imbalance family (the highest-R² predictor on this shelf) becomes partially computable; costs one grep.
2. Tighten quote snapshot cadence to ~5s for watched names in the 5 minutes around triggers — halves the aliasing that weakens H3.
3. Log NBBO at the exact trigger/fill instant in the entry receipts (not nearest-15s) — removes the largest measurement error in every spread hypothesis.
4. Per-minute uptick/downtick trade counts (available from SIP aggregates on some feeds) would upgrade H2's crude bar-sign proxy to a real tick-rule imbalance.
5. Tick-by-tick trades remain the true unlock for trade-sign imbalance and held-bid detection — everything above is the best available proxy without them.
+59
View File
@@ -0,0 +1,59 @@
# The regime gate: what it is, and why "it fails everything" is the right question
**Owner's question (2026-07-18):** "I don't know if we need the regime thing if
everything just fails the regime as well. What's the whole point of that?"
Honest answer, in three parts.
## What it actually is
The regime gate scores each session `P(our ORB makes money today)` from free
pre-open features (premarket breadth, SPY momentum/vol, the bot's own recent
trailing results), using a small walk-forward model. Below the threshold, it
sits the whole day out. Config: `TRADING_REGIME_GATE_ENABLED=1`,
`TRADING_REGIME_GATE_P_MIN=0.5`. It is the **one** research lever out of ~13
that survived out-of-sample testing.
## Why it "fails everything" right now — and why that's honest, not broken
Recent sessions genuinely scored bad (Friday 07-17 = −0.9R realized). The gate
isn't malfunctioning when it sits out — it's correctly reporting that the
recent regime is one the ORB loses in. Over the full 266-session backtest it
did NOT sit out everything: it traded ~32% of days and took the strategy from
−0.11R/session (bleeding) to roughly flat. So "always no" is a recent-weeks
artifact of a genuinely bad stretch, not the gate's normal behavior.
## The real limitation you're pointing at (and you're right)
**A gate that only ever says "no" is indistinguishable from turning the bot
off.** That is the honest ceiling: the regime gate is **damage control, not an
edge source.** It cannot create profit that isn't there — it can only allocate
the strategy's existing (slightly negative) edge onto the least-bad days. Its
best possible outcome on a losing base strategy is **flat**. Never gain.
So its value is entirely **conditional**:
- **If the base entry has no positive edge** (where we are now — 12 conversion
levers refuted, ~20% win rate), the regime gate is just a sophisticated "don't
trade much." Flat-vs-bleeding is a real but unexciting win, and yes — at the
limit it's a kill switch with extra steps. Your instinct is correct.
- **If a positive entry edge is found** (the whole point of the plan-017 L2
depth work), the regime gate flips from damage-control to **force
multiplier**: trade the good NAMES on the good DAYS, and the two signals
compound instead of one merely limiting losses.
## Big-picture relevance
The regime gate is not the thing that makes money. It's the thing that stops
the bot bleeding **while** we search for the thing that makes money. Keep it on
because flat beats bleeding for free — but judge it correctly: it is a seatbelt,
not an engine. The engine, if it exists, is an entry signal that separates
winners at the trigger (plan 017 / order-book depth). **The day an entry edge
clears the bar, the regime gate becomes genuinely valuable. Until then, it's a
holding pattern — and a holding pattern is the correct posture for a strategy
still looking for its edge.**
If plan-017 Phase 0 shows the order book separates winners, the regime gate
earns its keep. If not, the honest move is the owner-selector fork, and the
regime gate reverts to a simple "trade fewer days" toggle you can leave on or
off with little consequence.
+55
View File
@@ -0,0 +1,55 @@
# Regime-detection research notes — distilled for the momentum bot
**Date:** 2026-07-15
**Method:** 6 books in `books/market_regime_state_detection/` distilled by parallel readers (Ang & Timmermann NBER survey; Kritzman *Regime Shifts*; Zucchini/MacDonald/Langrock *HMM for Time Series*; Chan *Hands-On AI Trading* & *Quantitative Trading*; Kakkar et al *Trading Decoded*). Hamilton 1989 is a scanned image (no extractable text); its Markov-switching method is covered by the others.
**Problem this feeds:** the bot is ~breakeven with no entry edge after honest fills; the one thing untried is **regime awareness** — it did well 07-08/09 (a runner regime) and sits out / bleeds otherwise. Goal: a FREE-data daily regime signal that gates/sizes ORB entries, harness-backtestable over ~260 sessions.
---
## The one insight that reframes everything (Chan's metalabeling)
> "The key to successfully apply AI/ML to finance is to focus on **metalabeling** — finding the probability of profit of *your own* simple trading strategy, and **not** to use it to predict the market directly. Your own strategy's past track record is private; no one else is trying to predict it." — Chan, *Quantitative Trading*
Don't detect abstract "market regime." **Predict `P(our ORB makes money today)`** from free pre-open features, using our OWN per-session expR (which the plan-006 harness already computes) as the label. This targets exactly what matters (does *our* edge work today), sidesteps the "high-vol lumps runner-days with crash-days" problem, reuses the harness, is free, and is a *private* signal nobody arbs away. Every practical book (Chan CPO, Trading Decoded RF) converges here; the academic survey (Ang-Timmermann) calls the same thing "regime-conditioned position sizing."
## The method stack (what to build, in escalation order — ponytail)
1. **Threshold baseline first** (Kritzman, Trading Decoded both insist): a trivial rule — e.g. `SPY 5d momentum > 0 AND SPY 20d vol < median`, or Kritzman **Financial Turbulence** (`sqrt((r−μ)Σ⁻¹(r−μ)')` on ~11 sector-ETF returns) below its 75th percentile. If the fancy model can't beat this net-of-costs out-of-sample, ship the rule.
2. **Metalabel classifier** (the primary): `GradientBoostingClassifier` / `RandomForest` (free sklearn — the books' `predictnow.ai`/`gbdt` replaced) predicting `P(profitable ORB session)` from ~5–8 free features. Gate/size on the probability.
3. **HMM regime as a FEATURE** (not the whole model): a 2-state Markov-switching state on SPY (`statsmodels MarkovRegression(spy_returns, k_regimes=2, switching_variance=True)`) or on the turbulence series — one column feeding the classifier, or a standalone gate to compare.
## Free features (all pre-open, all we have or can ingest free)
| Feature | Source | Note |
|---|---|---|
| SPY 5d/20d momentum + 20d realized vol (VIX proxy) | already ingesting SPY (plan 009) | the level-1 input every book uses |
| IWM (small-cap index) + sector-ETF momentum | Alpaca/yfinance, **free** | small-caps specifically; sector rotation |
| Kritzman turbulence (Mahalanobis on ~11 sector ETFs) | sector-ETF daily returns, **free** | "prices behaving uncharacteristically" |
| Breadth = % of the gapper universe green / held-VWAP yesterday | our own stored data | the SIGNED feature that breaks vol-symmetry |
| **Bot's own trailing-N-day realized expR + hit-rate** | our harness | the most on-target feature — a hot/cold-streak detector on our OWN edge |
| VIX level + 1d change | CBOE/yfinance, **free** | the vol axis directly |
Keep to **≤5–8 features** — only ~260 sessions, ~a third fewer "runner days." More features = curve-fit.
## The load-bearing discipline (or the whole thing is a lie)
- **Filtered / walk-forward / purged only.** Only a regime label computed from data **≤ today** is tradeable. Smoothed HMM probs and full-sample fits **peek at the future**. Sanity check every book demanded: *if a smoothed-gated backtest crushes the filtered-gated one, your edge is look-ahead.* Use purged/embargoed walk-forward CV (drop the boundary day) — the same leakage-safe split used in `plans/008`.
- **Label is same-day-outcome, features are strictly pre-open.** The session's expR (label) uses open-knowable exits; every feature is lagged to the prior close.
- **Out-of-sample split is mandatory.** Fit on the older half, apply to the recent half; confirm the favorable-regime expR advantage survives. "In-sample regimes need not persist out-of-sample" (Ang-Timmermann) — if the edge evaporates OOS, the regimes were in-sample noise.
- **Acceptance = NET expR improves across EVERY walk-forward fold**, not AUC. A 52.8% classifier can still be net-negative (Chan Hands-On's own OOS accuracy was 52.77%).
- **Let the DATA pick the sign.** Ang-Timmermann finds low-vol = high-Sharpe for a *long-index* investor — but a long-*dispersion* small-cap momentum strategy likely inverts this (runners need volatility). Measure conditional expR per regime; never assume high-vol = bad.
- **Small-sample humility.** 2 states (don't LR-test the count), shallow trees (`max_depth ≤ 3`), report bootstrap/posterior bands on the regime probs, deflate any OOS Sharpe by the number of tweaks tried (Deflated Sharpe Ratio).
- **Regimes lag.** A daily gate enters the runner regime a day or two late and exits late — structural. It sits out *some* good days and takes *some* bad ones. The bar is net improvement, not perfection.
## Honest prior
This is the most promising lever of the session — it targets the bot's own edge, uses free data, reuses the harness, and directly explains the 07-08/09-vs-today swing. **But 260 sessions is a small sample**, so the real risk is curve-fitting a regime that doesn't persist. The threshold-baseline-first rule, purged walk-forward, and OOS split are the guards. If, after all that, the gated expR doesn't clearly beat trade-always out-of-sample → the honest finding is "no exploitable regime signal in free data at this sample size," and we've learned it cheaply.
## Sources (in `books/market_regime_state_detection/`)
- Ang & Timmermann (2011), *Regime Changes and Financial Markets*, NBER WP 17182 — the survey + the sizing template + every caveat.
- Kritzman et al, *Regime Shifts: Implications for Dynamic Strategies* — Financial Turbulence + absorption ratio.
- Zucchini, MacDonald, Langrock, *Hidden Markov Models for Time Series* — the estimation recipe (forward-filter, BIC state selection, why filtered ≠ smoothed).
- Chan et al, *Hands-On AI Trading* — the exact `statsmodels MarkovRegression` daily-gate recipe (Example 4) + the 4-year-lookback calibration.
- Chan, *Quantitative Trading* — Conditional Parameter Optimization + the metalabeling principle.
- Kakkar, Sahni, Shanmugamani, *Trading Decoded* — the RandomForest regime-as-feature recipe + purged walk-forward CV + the "beat the trivial rule or ship the rule" discipline.
+155
View File
@@ -0,0 +1,155 @@
<!-- Distilled 2026-07-17 from books/supply_side/ (15 sources). PARKED BY OWNER DIRECTION:
no new experiments from this note unless the owner asks — see memory: owner-intent-keep-it-simple.
Filed for reference; its M1/M2 mechanism (filings FOLLOW strength; supply hits at the 424B
pricing, not registration) explains dilution_veto_v1's backfire. -->
# Supply-Side Mechanics Shelf — Final Distillation Note
**System:** small-cap ORB momentum bot (gap ≥10%, $2–20, ≥300k day vol, long-only, minutes–hours holds, ~2k logged fires, pre-registered counterfactual harness).
**Framing.** Twelve conversion levers are dead, and the one supply-side gate already tried — the 90-day any-EDGAR-filing keep-or-zero veto — failed *backwards* (flagged fires outperformed, n=64). This shelf's 15 sources converge on a single explanation for that backfire, and it is the most useful thing in the entire pile: **a registration filing is a lagging marker of price strength, not a leading marker of supply pressure.** Firms file and sell *because* the stock ran (Greenwood §V; Billett/Floros/Garfinkel's sell-into-strength coefficient; WTD's baby-shelf capacity that mechanically grows with the 60-day high). So the naive dilution-trap story was not just noisy — it was sign-inverted by construction, and the ledger's n=64 result is what the mechanism predicts.
The squeeze literature warnings hold up under the primary sources here: nothing in this shelf supports any squeeze-long construction. Fabozzi/Mann, Lamont, Staley, both gamma papers, the meme-frenzy paper, and the SEC GME report collectively add zero computable squeeze-timing signal and several independent reasons the free proxies (SI, short-vol%, FTDs) are equilibrium quantities or noise. Those are all rejected below, explicitly.
What survives is narrow but real: **the timing axis inside the EDGAR feed itself** (registration vs. pricing), **an exit-shape prediction** (priced supply dampens continuation, not just price), and **two accrual features** (forward dilution propensity; halt-reopen discontinuity). None require new paid data. Two are not gates at all, which matters under the hardened random-gate null.
---
## 2. Convergent mechanisms
**M1 — Registration ≠ sale: supply hits the tape at the pricing/takedown event, not the filing.** An S-1/S-3 creates legal *capacity*; float actually changes at the 424B pricing (or ATM takedown), which is the US analog of Greenwood's pay-date and Hong–Scheinkman–Xiong's lock-up expiration. Six sources land here independently: Greenwood (pay-date, not announcement, carries the negative return), Harris (shelf vs. pricing are mechanically distinct moments with opposite information content), Asness/Fabozzi (lending-supply mechanics fire only at execution), HSX (price drops at the supply event even though the date is public), ATM/JFQA and WTD (program existence is EDGAR-visible; sale dates never are). The failed veto collapsed these two stages into one flag. This is the shelf's load-bearing mechanism.
**M2 — Issuance follows strength (the veto-backfire mechanism).** Greenwood §V (issuance/insider selling *after* the float-driven rise, even matched on prior returns), ATM/JFQA (same-quarter abnormal return is the only significant takedown predictor; prior quarter is not), WTD (baby-shelf legal capacity is computed off the trailing 60-day *high*, so capacity peaks exactly on gap days). Consequence: any filing-recency feature is partially a momentum-strength proxy, which is *positive-EV* in this system. Every hypothesis below must be checked net of gap%/rvol/quality for this reason.
**M3 — ATM supply is invisible at daily granularity and decays with program age.** Sale dates are never disclosed (quarterly aggregates only, 8-K usually backward-looking — ATM/JFQA Table 1); only ~1/3 of programs ever complete, and takedown pace decays monotonically with quarters elapsed. Two implications: (a) no EDGAR-derived feature can ever be a same-day supply flag, so expect attenuated signal and design for it; (b) a flat N-day cutoff is the wrong functional form — features should age/decay.
**M4 — Priced supply changes the *path*, not only the level: turnover and volatility compress.** HSX's model predicts price, turnover, and volatility all fall together when float is released, with a *nonlinear* effect that is largest exactly in the thin-float regime this system trades. Asness/Fabozzi adds the lending-side version: a priced secondary relieves borrow specialness, removing short-covering fuel for continuation legs. Both point at the same tradeable consequence: fires shortly after a priced takedown should show weaker *continuation* (lower MFE, shorter legs) — an exit-config claim, not an entry veto.
**M5 — Sign of the post-pricing effect is contested; the separation is not.** Greenwood + HSX predict 424B-recent fires are *worse* (supply pressure at the release event); Harris + Asness/Fabozzi predict they're *fine-to-better* (overhang resolved, borrow relieved) while *shelf-only* names carry the drag. All four agree the two buckets should behave *differently* — which is precisely the shape the hardened random-gate null demands. Register the split; let the harness pick the sign.
**M6 — Halt reopen is a regime switch, not a pause.** Harris: a halt swaps continuous discriminatory pricing for a single-price call auction; a stop resting through a halt fills at the discontinuous clearing price, not its level. This is a *different event class* from ordinary recrossing (the refuted trigger-erasure lever fired on continuous-market recrosses, which are normal here). Thin gappers are the maximally halt-prone population.
---
## 3. Ranked hypotheses (top 4)
### H1 — Filing-stage split of the dilution flag: 424B-recent vs. shelf-only vs. clean
*The highest-convergence item on the shelf (M1+M2+M5; proposed independently by 5 of 15 sources). Re-cuts the failed veto along the mechanism's own joint, and predicts which fires get BETTER, not just which to skip.*
**Statement.** The failed 90-day veto averaged two mechanically opposite populations. Split every fire three ways at fire time: **shelf-only** (S-1 or S-3 accession in trailing 90d, no 424B in trailing 15 trading days), **424B-recent** (any 424B accession in trailing 15 trading days), **clean** (neither). Prediction: the buckets separate. Primary direction (Greenwood pay-date + HSX): 424B-recent underperforms clean by ≥0.05R; shelf-only performs at/above clean (consistent with the observed n=64 outperformance, since shelf-only dominates the flagged population by base rate). Registered alternative (Harris/Asness lending-relief): the drag sits on shelf-only instead. Either outcome is a usable gate; *no separation* kills it.
**Data needed:** existing (EDGAR dilution_events: form type + accession date; verify the feed actually preserves 424B subtypes — filter out debt-only/boilerplate 424B3s where identifiable).
**Computable features:** `days_since_last_424B`, `days_since_last_S1S3`, categorical `filing_stage ∈ {424b_recent, shelf_only, clean}`; secondary graded feature per Greenwood magnitude-scaling: offering size / trailing 20d dollar volume where parseable.
**DRAFT PRE-REGISTRATION**
- population: `{min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}`
- intervention: keep-or-zero gate on the bucket the split identifies as the drag bucket (primary registration: zero `424b_recent`, keep `shelf_only` and `clean`); both non-gated buckets reported so the adjudicator sees separation, not exposure-cut
- bars: `{expR_delta_min: 0.05, min_n: 300}` — random-gate null applies: gated bucket must be worse than `flagged_share × |mean baseline| + 0.05R`, AND the kept flagged bucket (shelf_only) must be ≥ baseline (that's the separation claim)
- fidelity: `{atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}`
**Expected direction:** 424B-recent expR ≤ baseline − 0.05R; shelf-only ≥ baseline; clean = baseline.
**Kill criteria:** (a) 424B-recent and shelf-only statistically indistinguishable → mechanism dead, do not re-cut a third time; (b) separation exists but vanishes after conditioning on gap%/rvol/quality → M2 confound, it was momentum all along; (c) 424B-recent bucket still < n=300 after 3 months of accrual → park as accrual feature, do not adjudicate thin.
**Confound guards:** condition on gap%/rvol/quality (M2: filing stage proxies strength); exclude/segment fires already tagged `is_dilutive` by the LLM classifier so this isolates the EDGAR-timing signal from news content; EDGAR nightly refresh lags one session — use accession dates strictly < fire date.
**Coverage-design note (dilution-v1 lesson):** a fire whose symbol has *no EDGAR coverage at all* (no CIK match, feed gap) must be **skipped from the test population, not defaulted to "clean/keep"** — the prior veto's cousin failure (regime lever) shipped inert because missing data silently read as pass. Log coverage % per bucket in the probe.
---
### H2 — Recent-priced-supply continuation dampener (exit-config variant, not a gate)
*M4 (HSX volatility/turnover compression + Asness/Fabozzi covering-fuel removal + WTD ATM-absorbs-the-pop). Structurally immune to the random-gate null because it gates nothing — it's a paired counterfactual on the same fires.*
**Statement.** Fires in names with a 424B priced within the trailing 10 trading days have structurally weaker continuation (lower MFE, faster fade after trigger) regardless of entry EV — fresh float dampens the path. A tighter exit (earlier partial profit-take / tighter trail) beats the standard exit config *on this subset only*, while the standard config stays default everywhere else.
**Data needed:** existing (EDGAR 424B dates + 1-min OHLCV for MFE/MAE path replay in the harness).
**Computable features:** `recent_424B_flag` (≤10 trading days), per-fire MFE/MAE/realized-range under current vs. tightened exit config; optional interaction with `is_60d_high` (WTD baby-shelf capacity peaks there — flag it, don't gate it).
**DRAFT PRE-REGISTRATION**
- population: `{min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}`, restricted to `recent_424B_flag = 1` fires
- intervention: exit-config variant (tighter trail + earlier partial) replayed against standard exit on the flagged subset; standard exit unchanged on all other fires
- bars: `{expR_delta_min: 0.05, min_n: 300}` on the subset (random-gate null formally applies but is trivially satisfied — no fires are zeroed; the comparison is config-vs-config on identical fires)
- fidelity: `{atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}`
**Expected direction:** tighter exit wins by ≥0.05R on 424B-recent fires; loses or ties on clean fires (predicted asymmetry is the mechanism check — if the tight exit wins everywhere, that's a generic exit finding, not a supply finding, and it collides with the refuted fast-exit ledger).
**Kill criteria:** (a) tight exit wins on clean fires too → not supply-specific, discard as trigger-erasure-adjacent; (b) MFE distribution of 424B-recent fires indistinguishable from clean after gap/rvol conditioning → M4 doesn't bite at this horizon; (c) subset accrues < n=300 in 4 months → keep logging, don't test.
**Confound guards:** 424B pricing clusters with news-heavy days — compare against a news-matched clean control, not raw baseline; reverse causality (firms price after weakness) — check the flagged subset's prior-5d return distribution.
**Coverage-design note:** same skip-not-keep rule as H1 — no EDGAR coverage → excluded from both arms.
---
### H3 — Post-pump dilution propensity → same-name re-entry derate (accrual-first)
*M2 turned from a bug into a feature (Greenwood §V + ATM sell-into-strength). The only hypothesis here that uses the lagging direction of causality correctly.*
**Statement.** Big fires *cause* filings, not vice versa. Squeeze-tier fires (gap ≥150% & rvol ≥5) should show materially elevated (>2×) incidence of new dilution accessions in T+1..T+10 vs. ordinary-tier fires. Once accrued, the tradeable form is a **second-fire** rule: a re-fire in the same name within 10 sessions of a squeeze-tier fire, *with an intervening dilution accession*, is a genuinely distinct population predicted to underperform — the pump happened, the supply response is now filed, and the M2 strength-proxy value of the filing is already spent on fire #1.
**Data needed:** existing (fire tier features + EDGAR feed).
**Computable features:** `any_dilution_accession_T1_T10` per fire; tier-bucketed incidence table; `refire_after_squeeze_with_intervening_filing` flag on subsequent fires.
**DRAFT PRE-REGISTRATION** (accrual phase now; gate registration deferred)
- population: `{min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}`
- intervention (deferred): keep-or-zero on `refire_after_squeeze_with_intervening_filing = 1`; all first fires untouched
- bars: `{expR_delta_min: 0.05, min_n: 300}`, random-gate null applies — the kept re-fires (no intervening filing) must match baseline for separation to count
- fidelity: `{atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}`
**Expected direction:** incidence >2× after squeeze-tier fires; flagged re-fires expR ≤ baseline − 0.05R; unflagged re-fires ≈ baseline.
**Kill criteria:** incidence ratio <1.5× after 2 months of accrual → mechanism absent in this universe, drop before ever gating; flagged-refire n hopelessly thin (<50 in 3 months) → park.
**Confound guards:** must never be applied to the *triggering* fire (that is exactly the §V mistake the failed veto made); EDGAR nightly lag means T+1 incidence is undercounted by up to one session — measure T+2..T+10 as the robust window.
---
### H4 — Halt-discontinuity forced exit at reopen (accrual-first, tail-risk)
*M6 (Harris). Explicitly distinct from the refuted trigger-erasure lever: fires on a regime-switch event (halt→auction), not on continuous-market recrossing.*
**Statement.** A stop resting through a mid-hold halt fills at the reopen's single clearing price, not its level. Forcing exit at/immediately after the reopening print on currently-held names reduces tail losses vs. letting the stop ride the gap. This is a variance/tail claim on the halted subset, not a mean-edge claim.
**Data needed:** existing (1-min bar timestamp gaps for halt detection + 15s quote snapshots for size-collapse confirmation). Validate the halt flag against a real halt source before trusting (a feed hiccup also makes a bar gap) — the existing bar-gap halt guard's detection can be reused (rung 2: this codebase already detects bar gaps for the entry-side halt guard; extend, don't rebuild).
**Computable features:** `mid_hold_halt_flag` (intraday bar-gap >1min on a held name, quote-size collapse corroboration), `reopen_gap_pct`.
**DRAFT PRE-REGISTRATION** (accrual phase: log halt incidence + reopen-gap distribution on held fires)
- population: `{min_entry_gap_pct: 0.10, min_day_volume: 300000, price_min: 2.0, price_max: 20.0}`, restricted to fires with `mid_hold_halt_flag = 1`
- intervention: exit-config variant — forced exit at first reopen print vs. current stop-through-halt behavior, paired replay
- bars: `{expR_delta_min: 0.05, min_n: 300}` — honestly unlikely to reach n=300 soon; adjudicate only when it does, report tail metrics (P5 of fire R) alongside expR
- fidelity: `{atr_frac: 0.25, spread_mult: 3, adverse_ticks: 1}` — reopen fills should additionally assume the adverse side of the reopen print
**Expected direction:** forced reopen-exit improves expR and truncates the left tail on halted fires; no effect elsewhere by construction.
**Kill criteria:** halt incidence on held fires <1/month → not worth harness slots, keep as passive logging; reopen-gap distribution symmetric around zero → discontinuity isn't adverse-biased, no edge in forcing.
**Confound guards:** halt-up vs. halt-down must be separated (halt-up reopens may favor holding — Harris's gravitational effect); data-gap false positives excluded via quote-snapshot corroboration.
---
## 4. Rejected during synthesis
- **Any squeeze-long timing feature** (Staley float-scarcity, Barbon–Buraschi gamma, Baig meme-gamma, 0DTE paper) — requires options/borrow data we don't have; SEC GME report found no gamma squeeze even in the canonical case; literature-refuted on base rates.
- **Prior-day short-sale-volume% gate** (Harris-derived) — Lamont + Asness/Fabozzi show short-flow/SI is an equilibrium quantity, sign-ambiguous at exactly the extremes; ATS-estimate noise on micro-caps; shape is exposure-shrink with no credible "these fires get better" leg. Fails the hardened bar by design.
- **FTD-based flags** — SEC report: non-persistent, conflates long/short fails; predicts negative returns at monthly staleness; nothing to time with.
- **Bi-monthly SI / days-to-cover anything** — doubly dead: stale (brief's own warning) and structurally confounded (Lamont's zero-SI-at-max-constraint case).
- **Flat filing-recency re-test with a different window** — already failed at 90d; M3 says the functional form is wrong, not the window length. Age-decay is folded into H1's features instead of standing alone.
- **S-1 vs. S-3 issuer-quality split as a standalone** (WTD) — plausible but n=64-scale before splitting; folded into H1 as a secondary cut if H1's buckets separate.
- **60d-high × ATM-capacity entry veto** (WTD) — collides head-on with new-high momentum being positive in this system; demoted to a logged interaction flag inside H2.
- **Sub-$5 margin-lending borrow-pool constraint** (Asness/Fabozzi) — re-litigates refuted price/float/liquidity gating.
- **Borrow-scarcity spread/liquidity gates** (Fabozzi/Mann) — refuted liquidity-floor lever wearing a lending-market hat.
- **Opening-auction imbalance fade** (Harris) — already captured by the static open-gap population variable; nothing incremental.
- **Anti-shorting-firm / loan-recall event features** (Lamont) — event class not identifiable from any available feed; monthly-year horizon mismatch.
- **A-/B-share float-speculation transfer** (Mei et al.) — its generalizable claim is the already-refuted low-float lever; market institutions non-analogous.
## 5. Data acquisition order
1. **Nothing new for H1/H2/H3** — first job is verifying the existing dilution_events feed preserves 424B form subtypes and accession dates cleanly, plus per-symbol CIK coverage stats (the skip-not-keep prerequisite).
2. **OpenBB/yfinance float shares** — second, and only to magnitude-grade H1 (offering $/float per Greenwood scaling); never as a gate input (float gating is refuted). Accept staleness on actively-diluting names.
3. **Do not wire** stockgrid short-vol%, FINRA SI, or SEC FTD files — every source that examined them here disqualified them; wiring them invites exactly the re-litigation this ledger exists to prevent.
+96 -24
View File
@@ -23,6 +23,12 @@ from dotenv import load_dotenv
sys.path.insert(0, ".")
load_dotenv()
# config parity with the live loop: DB-backed knobs (config_overrides) beat
# compose env — a bare env would replay under DEFAULT exits (the 4.5x-baseline
# landmine, orderflow-shelf-verdicts 2026-07-17)
from runtime.env_overrides import load as _load_cfg_overrides # noqa: E402
_load_cfg_overrides()
os.environ.setdefault("STRATEGY_SETUPS", "opening_range_break")
from alpaca_paper.client import AlpacaPaperClient
@@ -32,6 +38,7 @@ from research.ingestion.market_data import (
classify_session,
ingest_daily_history,
ingest_live_minute_bars,
upsert_minute_bars,
)
from research.ingestion.signals import scan_gappers
from research.multi_schema import open_research_db
@@ -39,19 +46,22 @@ from research.query import query_avg_daily_volume, query_minute_bars, query_prev
from strategy.backtest.engine import BacktestEngine
from strategy.exits import ExitConfig, TRAIL_PRIOR_LOW, parse_profit_tiers
from strategy.models import SetupCriteria
from strategy.population import PopulationConfig, population_reject
# the live verified config
PRICE_MIN = float(os.environ.get("WATCHER_PRICE_MIN", "1"))
PRICE_MAX = float(os.environ.get("WATCHER_PRICE_MAX", "5"))
GAP_MIN = float(os.environ.get("TRIGGER_GAP_MIN", "3"))
GAP_MAX = float(os.environ.get("TRIGGER_GAP_MAX", "60"))
RVOL_MIN = float(os.environ.get("TRIGGER_RVOL_MIN", "2"))
# Population is defined ONCE, in strategy/population.py — the SAME gate the live
# loop trades ($2-20 band + gap floor + 35-150% dead band + >=150% squeeze). This
# replaces the stale $1-5 / 3-60% look-ahead screen. Prod thresholds are passed
# explicitly (a bare from_env() shell would disable the gap gate).
_POP = PopulationConfig(min_entry_gap_pct=0.10)
RISK_PCT = float(os.environ.get("TRADING_RISK_PER_TRADE_PCT", "0.01"))
def _daily_gappers(con, sess, universe):
"""Identify today's qualifying gappers from DAILY bars (cheap) so we only
minute-backfill the handful that matter, not all 100 names."""
minute-backfill the handful that matter, not all 100 names. Screens through
the SHARED live population gate (strategy/population.py) on the OPEN price and
STATIC open gap — both open-knowable; the old screen used t_close (look-ahead)
and a hard 60% cap that dropped the >=150% squeezes production re-admits."""
out = []
for sym in universe:
# Bars ON OR BEFORE the session date, newest first — so rows[0] is the
@@ -65,17 +75,53 @@ def _daily_gappers(con, sess, universe):
"AND trade_date <= ? ORDER BY trade_date DESC LIMIT 25", [sym, sess]).fetchall()
if len(rows) < 2 or str(rows[0][0]) != str(sess):
continue
t_open, t_close, t_vol = float(rows[0][1]), float(rows[0][2]), float(rows[0][3])
t_open, t_vol = float(rows[0][1]), float(rows[0][3])
prev_close = float(rows[1][2])
prior = [float(r[3]) for r in rows[1:21]]
adv = sum(prior) / len(prior) if prior else 0.0
gap = (t_open / prev_close - 1) * 100 if prev_close else 0.0
gap = (t_open / prev_close - 1) if prev_close else 0.0 # FRACTION (population contract)
rvol = (t_vol / adv) if adv else 0.0
if PRICE_MIN <= t_close <= PRICE_MAX and GAP_MIN <= gap <= GAP_MAX and rvol >= RVOL_MIN:
# OPEN price + open gap are open-knowable; rvol re-admits real >=150% squeezes.
if population_reject(_POP, gap_pct=gap, price=t_open, volume=None, rvol=rvol) is None:
out.append(sym)
return out
def _session_universe(con, client, sess) -> list[str]:
"""Candidate names for a session, from three sources union-ed:
(a) what the LIVE scanner discovered that day (events) — the top-100
absolute-volume list misses small-cap gappers entirely: 2026-07-28 the
replay found 1 gapper on a day the live loop signaled 8 names, because
MUZ/DAMD/BIYA-class names never crack the most-actives list;
(b) historical_gappers — the year-scale gapper record, covers dates before
the live loop existed (backfill path);
(c) today's most-actives — ONLY for a same-day replay (the API returns the
CURRENT list; unioning it into a past date would be look-ahead).
The shared population screen still runs after this — the union only widens
candidates, it never bypasses the gate."""
syms: set[str] = set()
try:
rows = con.execute(
"SELECT DISTINCT payload_json::json->>'symbol' FROM events "
"WHERE event_type = 'symbol_discovered' AND timestamp::date = ?",
[str(sess)]).fetchall()
syms |= {r[0] for r in rows if r and r[0]}
except Exception as exc: # noqa: BLE001 — events table may not exist in research-only DBs
print(f"[universe] discovered-symbols lookup skipped: {exc}")
try:
rows = con.execute(
"SELECT DISTINCT symbol FROM historical_gappers WHERE session_date = ?",
[str(sess)]).fetchall()
syms |= {r[0] for r in rows if r and r[0]}
except Exception as exc: # noqa: BLE001
print(f"[universe] historical_gappers lookup skipped: {exc}")
if sess == datetime.now(timezone.utc).date():
actives = client.get_most_actives(top=100, by="volume")
syms |= {a["symbol"] for a in actives if a.get("symbol")}
return sorted(syms)
def _exit_cfg() -> ExitConfig:
return ExitConfig(
target_r=float(os.environ.get("TRADING_REWARD_MULTIPLE", "10")),
@@ -94,31 +140,56 @@ def main(argv=None):
con = open_research_db("market")
client = AlpacaPaperClient(AlpacaPaperSettings.from_env())
# 1) full active universe (vs the ~20 the live loop watched)
actives = client.get_most_actives(top=100, by="volume")
universe = sorted({a["symbol"] for a in actives if a.get("symbol")})
# 1) session universe: scanner-discovered + historical gapper record +
# (same-day only) most-actives — see _session_universe
universe = _session_universe(con, client, sess)
print(f"=== EOD REPLAY {sess} ===")
print(f"universe: {len(universe)} most-active names (live loop watches ~20)")
print(f"universe: {len(universe)} candidates (scanner-discovered + "
f"historical_gappers + same-day most-actives)")
# 2) cheap DAILY backfill -> screen gappers from daily bars (no need to pull
# minute bars for all 100; only the gappers need them)
ingest_daily_history(con, client, universe, days=30)
gapper_syms = _daily_gappers(con, sess, universe)
print(f"daily-screen gappers (${PRICE_MIN:g}-{PRICE_MAX:g}, gap {GAP_MIN:g}-{GAP_MAX:g}%, "
f"rvol>={RVOL_MIN:g}): {len(gapper_syms)}")
print(f"daily-screen gappers (shared population gate: ${_POP.price_min:g}-{_POP.price_max:g}, "
f"gap >={_POP.min_entry_gap_pct:.0%} ex {_POP.gap_dead_band_lo:.0%}-{_POP.gap_dead_band_hi:.0%} "
f"dead-band, squeeze >={_POP.gap_dead_band_hi:.0%}+rvol{_POP.squeeze_rvol_min:g}): "
f"{len(gapper_syms)}")
# 3) minute-backfill ONLY the gappers (fast), then rank them, and grow the
# dataset. Use the SIP (full-volume) feed so rvol/$-volume in the replay
# reflect REAL volume — the live IEX feed sees only ~2-5% of it, which was
# silently corrupting every volume-based backtest conclusion.
ing = ingest_live_minute_bars(con, client, gapper_syms, lookback_minutes=440,
feed=client.settings.backfill_feed)
print(f"backfilled {ing.minute_rows} minute rows ({client.settings.backfill_feed}) "
f"across {len(ing.symbols)} gappers\n")
if sess == datetime.now(timezone.utc).date():
ing = ingest_live_minute_bars(con, client, gapper_syms, lookback_minutes=440,
feed=client.settings.backfill_feed)
n_rows, n_syms = ing.minute_rows, len(ing.symbols)
else:
# historical session: a from-now lookback window would fetch the WRONG
# day — fetch that session's 04:00-20:00 ET explicitly (premarket incl.)
from zoneinfo import ZoneInfo
et = ZoneInfo("America/New_York")
start = datetime(sess.year, sess.month, sess.day, 4, 0, tzinfo=et)
end = datetime(sess.year, sess.month, sess.day, 20, 0, tzinfo=et)
by_sym = client.get_minute_bars(
gapper_syms, start_iso=start.isoformat(), end_iso=end.isoformat(),
feed=client.settings.backfill_feed) if gapper_syms else {}
n_rows, n_syms = 0, 0
for sym, bars in by_sym.items():
if bars:
n_rows += upsert_minute_bars(con, sym, bars,
source_provider="alpaca_sip_backfill")
n_syms += 1
print(f"backfilled {n_rows} minute rows ({client.settings.backfill_feed}) "
f"across {n_syms} gappers\n")
keep = set(gapper_syms)
gappers = [g for g in scan_gappers(con, sess, min_gap_pct=GAP_MIN, min_relative_volume=RVOL_MIN,
price_min=PRICE_MIN, price_max=PRICE_MAX, limit=200)
if g.gap_pct <= GAP_MAX and g.symbol in keep]
# Population membership was already decided by the shared gate in
# _daily_gappers; scan_gappers here only computes gap/rvol/vwap for ranking, so
# pass permissive bounds and let `keep` be the sole authority (this also keeps
# the >=150% squeeze names the old `gap_pct <= GAP_MAX` cap wrongly dropped).
gappers = [g for g in scan_gappers(con, sess, min_gap_pct=0.0, min_relative_volume=0.0,
price_min=0.0, price_max=1e9, limit=200)
if g.symbol in keep]
if not gappers:
print("no qualifying gappers today.")
return
@@ -127,7 +198,8 @@ def main(argv=None):
cfg = _exit_cfg()
eng = BacktestEngine(Config(), target_r=cfg.target_r, warmup_bars=3, eval_every=1,
ready_score_pct=55, min_bars=3, exit_config=cfg,
criteria=SetupCriteria(gap_pct_min=0.05, relative_volume_min=RVOL_MIN))
# intrabar ORB SETUP criteria (not population) — preserved
criteria=SetupCriteria(gap_pct_min=0.05, relative_volume_min=2.0))
equity = float(client.get_account().get("equity") or 0) or 100000.0
risk_dollars = equity * RISK_PCT
-11
View File
@@ -1,11 +0,0 @@
#!/usr/bin/env bash
# End-of-day whole-market replay: what today's logic would have done + grow the
# stored dataset for the tuner/verification. Schedule right after the close,
# before nightly_tune (which then tunes on the freshly-grown data).
# crontab -e -> 5 16 * * 1-5 /mnt/c/code/momentum/eod_replay.sh
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_PY="/home/philip/.venvs/momentum/bin/python"
cd "$ROOT" || exit 1
mkdir -p "$ROOT/data"
echo "=== eod_replay $(date) ===" >> "$ROOT/data/eod_replay.log"
PYTHONPATH="$ROOT" "$VENV_PY" eod_replay.py >> "$ROOT/data/eod_replay.log" 2>&1
+9
View File
@@ -0,0 +1,9 @@
{
"id": "asym_exit_v1",
"hypothesis": "An asymmetric exit (volatility-fractional stop with a spread floor, no profit target, trail to EOD) captures the published fat-right-tail edge better than the current managed exit.",
"population": {"min_entry_gap_pct": 0.10, "min_day_volume": 300000, "price_min": 2.0, "price_max": 20.0},
"intervention": {"kind": "asym_exit", "atr_frac": 0.25, "no_target": true, "trail": "eod"},
"bars": {"expR_delta_min": 0.10, "min_n": 300},
"futility": {"min_n": 500, "expR_delta_max": -0.02},
"fidelity": {"atr_frac": 0.25, "spread_mult": 3, "adverse_ticks": 1}
}
+8
View File
@@ -0,0 +1,8 @@
{
"id": "depth_overhang_broad_v1",
"hypothesis": "Statistically-powered version of depth_overhang_v1 over the BROAD $2-20 v1 ORB population (drops the gap>=10% live-band filter to reach n). Same overhang gate (keep>=0.034) under the REAL deployed asymmetric exit. This is the powered number; depth_overhang_v1 (gap>=10%) is the smaller live-population directional check.",
"population": {"min_entry_gap_pct": 0.0, "min_day_volume": 0, "price_min": 2.0, "price_max": 20.0},
"intervention": {"kind": "depth_overhang_gate", "min_overhang": 0.034},
"bars": {"expR_delta_min": 0.05, "min_n": 300},
"fidelity": {"atr_frac": 0.25, "spread_mult": 3, "adverse_ticks": 1}
}
+8
View File
@@ -0,0 +1,8 @@
{
"id": "depth_overhang_v1",
"hypothesis": "L2 order-book overhang (plan 017): resting ask size within 1% ABOVE the ORB break, normalized to bid depth, from Databento depth. Sniff+reweighted analysis (n=602, 21 sessions) cleared the bar AND the random-gate null: keep fires with overhang>=0.034, zero the rest, lifts win rate 23%->28%. Counter-intuitive direction (more overhang above = MORE likely to win = absorption/fuel). This runs it under the REAL deployed asymmetric exit (simulate_exit, winners trail to +3R) which should exceed the conservative +1/-1 estimate. Coverage: only the 602 depth-backfilled fires have data; the rest SKIP (variant None, excluded from n). Random-gate null applies.",
"population": {"min_entry_gap_pct": 0.10, "min_day_volume": 300000, "price_min": 2.0, "price_max": 20.0},
"intervention": {"kind": "depth_overhang_gate", "min_overhang": 0.034},
"bars": {"expR_delta_min": 0.05, "min_n": 300},
"fidelity": {"atr_frac": 0.25, "spread_mult": 3, "adverse_ticks": 1}
}
+8
View File
@@ -0,0 +1,8 @@
{
"id": "dilution_veto_v1",
"hypothesis": "Zeroing ORB fires on names with an S-1/S-3/424B filing in the trailing 90 days raises whole-population expR — the dilution-trap gaps fade as new shares hit.",
"population": {"min_entry_gap_pct": 0.10, "min_day_volume": 300000, "price_min": 2.0, "price_max": 20.0},
"intervention": {"kind": "dilution_gate", "lookback_days": 90},
"bars": {"expR_delta_min": 0.05, "min_n": 300},
"fidelity": {"atr_frac": 0.25, "spread_mult": 3, "adverse_ticks": 1}
}
Loaded 100 of 390 files, more files were not shown because too many files have changed in this diff. Show more