GO_LIVE flags `spread_pct` as a not-yet-real field. The reason it was null is
structural: OHLCV bars carry no NBBO, so `minute_bars.spread_pct` can never be
populated from the bars endpoint. The execution-relevant figure is the spread at
the moment a setup is evaluated — a wide book quietly eats R on the thin sub-$20
names this system trades.
This makes that metric real, shadow-first (measure before gating), mirroring how
the catalyst score was introduced:
- `strategy/evaluation/liquidity.py`: pure `compute_spread_pct(bid, ask)` →
(ask-bid)/mid, or None for missing/crossed/non-positive quotes (None never
collapses to a real 0.0). Unit-tested.
- `AlpacaPaperClient.get_latest_quotes`: batched latest-NBBO fetch, mirrors
`get_latest_trades`.
- `Watcher`: optional injected `spread_provider` (same shape as
`catalyst_score_provider`); default None == prior behavior byte-for-byte.
Computed spread is logged onto the criteria_evaluated and signal_ready events.
Observe-only — it does NOT gate ready/blocked.
- `run_live_paper`: TTL-cached spread provider off the client; degrades to None
on any fault, SPREAD_PCT_ENABLED=0 to silence.
The dead `scanner.spread_pct_max` knob is intentionally left for a later
promotion, once the logged data shows spread separates outcomes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019psgRHdtTXz6TCnnqPiqGw
Trade journal was blind: exits emit no event, so closed_trades / win-rate /
avg-R / the trade list all read 0 even on days with booked round-trips.
Now AlpacaPaperSync.sync_all reconciles closes by diffing the positions
snapshot: a symbol that was open and is now gone/flat has closed. This one
signal catches every exit path — bracket STOP/TARGET legs fill broker-side, EOD
flatten closes via market orders — and reads the exit fill (price + reason) and
the bracket stop level from the orders snapshot, emitting a PositionClosedEvent
with entry/exit/stop/side/realized so query_session_pnl can compute the
R-multiple. Headline realized $ stays broker-authoritative (the broker-delta
branch is unchanged); these events feed the wins/losses/avg-R/trade-list journal.
Robustness: the close baseline seeds from the last PERSISTED snapshot so a close
during downtime is still journaled once; reconcile is wrapped so it can never
break sync; PositionClosedEvent gains optional entry_price/stop_loss_price/side/
quantity fields (older emitters/tests unaffected).
Dashboard is view-only: removed the manual Exit button + exitPos handler from
Open Positions (kept the approval-queue approve/reject — that's the intended
human-in-the-loop control).
Tests: stop->loss, target->win, still-open->no-emit, no-fill->last-mark
fallback, restart-seed-from-persisted. 49 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The only news source was RSS (slow, regex ticker-scraping). Alpaca exposes
Benzinga news with a stable id + authoritative symbols list. Add it as a second
source landing into the SAME raw_news_items table, so recent_news_map (catalyst
prioritisation) and the Ollama enrichment (PR #2) pick it up with no other changes.
- AlpacaPaperClient.get_news(symbols, limit): GET /v1beta1/news (works on the
existing data keys; verified live).
- research/ingestion/alpaca_news.py: ingest_alpaca_news lands new items, deduped
by the Alpaca news id; never raises (news is an enhancement, not a dependency).
- run_live_paper step_news on the scheduler (ALPACA_NEWS_* env, default on, 120s),
scoped to the tracked universe.
Verified live: passes landed ~34 then deduped; recent_news_map went from RSS-only
to 370 ticker->headline catalysts. tests/unit: 165 passed (+5).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two more visualizations + the bug building them exposed:
- Equity-curve sparkline in the P&L strip: query_equity_curve returns the day's
total_equity series + prior-close baseline (dashed); rendered as an inline SVG
(green/red vs baseline) so the day's P&L shape reads at a glance.
- Per-position R bar in Open Positions: shows current open profit/loss in R units
(-1R = the stop, 0R = breakeven), green right / red left, with a per-row ⚠ when
a position has no resting protective stop. Falls back to % when no stop known.
Also fixed the P/L column to compute from entry/last (was reading a None field).
- BUG: sync_orders dropped stop_price (kept only limit_price), so the synced
snapshot showed stop=None on real stop orders — breaking the R bar and spark
stop-line. Now captured. (The naked alarm was unaffected — it counts qty
coverage, not the level.)
Verified live: equity curve renders the -1,886 day (94,124 -> 92,238); R bars
render (fall back to % until the loop re-syncs stop_price). Tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
From the multi-agent audit (ship-now, low-overfit-risk cluster):
INGESTION RESILIENCY
- client._request: 429/5xx now retry with Retry-After + exponential backoff
(previously hard-failed, feeding silent-stale ingest).
- SIP backfill feed (settings.backfill_feed='sip'): live realtime stays IEX, but
backtest/backfill ingestion uses the full consolidated tape — IEX sees only
~2-5% of true volume (verified: AAPL 22,797 vs 452,704), which was silently
corrupting every rvol/$-volume decision and backtest. eod_replay wired to it.
- step_ingest freshness watchdog: emits a STALE event + board note when the
newest bar is older than INGEST_STALE_SECONDS during RTH (45% of passes
historically returned 0 rows with no error). Optional hard arm-block
(INGEST_STALE_BLOCK, off by default).
RISK
- Portfolio gross-notional cap (TRADING_MAX_GROSS_NOTIONAL_PCT=0.60): both sizing
paths now cap each entry to the remaining gross budget, so 6 x 16% can't pile
the book into ~96% gross of correlated small-caps. Blocks/shrinks late entries.
- Fixed the duplicate TRADING_MAX_CONCURRENT_POSITIONS in .env.example.
MONITORING (UI)
- Naked/unprotected-position alarm: _unprotected_positions joins the latest
position + order snapshots (order sync now flattens legs) and the TUI board
shows a red UNPROTECTED banner the instant a held long lacks stop coverage —
the exact blind spot when APWC/NOWL went naked. Verified it flags today's
3 overnight positions.
Tests: gross-notional cap blocks when budget full; 50 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Today's-trades reconstruction showed normally-exited positions as falsely
"open" and understated realized P&L. Two causes:
1. sync_orders recorded only TOP-LEVEL orders. A position closed by its bracket
STOP or TAKE-PROFIT leg has that exit fill as a child LEG, so every stop/TP
exit was invisible — QTEX/BBD/UWMC/etc. looked open when they were flat, and
their realized P&L was dropped. Now flatten parent + filled legs (legs inherit
the parent symbol).
2. get_orders only fetched the most-recent page; Alpaca's `limit` truncates by
recency (legs count toward the cap), so a busy day could drop the morning's
fills entirely. Unscoped get_orders now pages backwards with `until` (dedup by
id) to capture the full day; the symbol-scoped path stays single-page.
Verified live: journal open set now matches the broker exactly
(ATPC/GRAB/LNKS/NOWL/WPRT) and every reconstructed round-trip ties to a real
broker fill. NB: the realized total is FIFO-lot based, so while positions are
partially open it won't equal the broker's average-cost day P&L — the broker
figure remains the authoritative headline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Alpaca's orders `limit` keeps only the most-RECENT orders and counts child
legs toward the cap. On a busy day the order list runs to hundreds (every
backed-out entry is an order), so the exit manager's limit=100 snapshot
silently dropped the stop legs of older OPEN positions. A stop-leg lookup then
read "no stop" on a fully-protected position -> broken trailing/profit-lock for
that name (and false-naked alarms in monitoring).
Fix: get_orders gains a `symbols` filter; the exit manager scopes its snapshot
to the handful of held names (no truncation possible). Verified live: an
unscoped limit=100 query reported PLUG stop=None while the scoped query
correctly surfaced PLUG's stop @2.65.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The get_positions TTL cache (added for the speed fix) silently broke the
naked-stop guard: _broker_held_symbols read 2.5s-stale positions, so a
just-filled entry wasn't seen as held, the guard thought it was unfilled, and
cancelled its bracket on a price dip -> NAKED position (APWC bled -12.8%
unprotected today). Fixes:
- client.get_positions(fresh=True) bypasses the cache; the guard uses it.
- entry_grace_seconds (default 5): never cancel an entry within N seconds of
arming, so a marketable fill can confirm before the bracket could be stripped.
Regression test added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- momentum watch: replace 'open positions' with TODAY'S trade history (FIFO
round-trips from the broker orders snapshot: time/symbol/qty/entry/exit/P&L +
open lots), resets each day. Headline day P&L is broker-accurate (equity -
last_equity, now carried on AccountSummaryUpdatedEvent), shown in the header
and the panel title. sync_orders limit 50->500 so the log captures the whole
day across restarts.
- SYMBOL_WEIGHTS: optional manual per-symbol priority nudge applied to the
armed-trigger rank score (small; doesn't override gap/volume logic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 2.5s value-cache + fewer calls only halved the slowness; the real cause was
flaky WSL DNS (~5s on ~1 in 5 resolves) and urllib re-resolving on EVERY
request. Cache successful DNS lookups process-wide (5min TTL) so each Alpaca
host is resolved at most once per window. Measured: get_orders went from
intermittent 5s spikes to a steady ~200ms. Opt out via ALPACA_DNS_CACHE=0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The single-threaded passes (and the trigger thread) ballooned to 60-120s.
Cause: flaky WSL DNS makes ~1 in 5 Alpaca calls take ~5s to resolve, and each
pass fired ~20 calls (breaker, sync, guard, exit manager all re-reading
account/positions/orders). Fixes:
- client: 2.5s TTL cache for get_account/get_positions (invalidated on any
order mutation) — collapses the redundant per-pass reads to one.
- exit manager: fetch the (heavy) status=all orders snapshot ONCE per pass and
reuse it for every position's stop-leg + entry-time lookup (was 2 calls/pos).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A transient WSL DNS outage made 3 consecutive equity reads fail, and the
fail-CLOSED breaker then PERMANENTLY halted new entries for the whole session
even after DNS recovered seconds later. Fixes:
- split the breaker into a permanent _halted (real daily-loss breach) and a
recoverable _data_halt (equity unreadable) that clears on the next good read;
raise the pause threshold to 5 (TRADING_EQUITY_FAIL_LIMIT).
- AlpacaPaperClient._request retries transient URLError/gaierror (not HTTP
4xx/5xx) a couple times, so brief blips don't even register as failures.
- sync logs network failures as one-line warnings, not full tracebacks (the
log was spamming stacks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the static -1R/+2R bracket with active, parameterised exit management
shared by the backtest and live (strategy/exits.py): move to breakeven, trail
under prior-bar lows or by %, scale out, or first-red exit.
Swept all variants over 187 sessions with a held-out train/test split
(sweep_exits.py), then an adversarial 3-judge workflow picked the robust winner
OUT-OF-SAMPLE: trailLow_a1 (trail under the prior bar's low after +1R) — test
avgR 0.128 vs the old static_2R 0.067 (~1.9x), win 49% vs 38%. The naive
in-sample winner (trailPct10) was correctly rejected as overfit
(train 0.684 -> test 0.085).
- strategy/exits.py: ExitConfig + simulate_exit (backtest) + manage_live (live)
- strategy/backtest/engine.py: simulate the managed exit on entry; Trade.entry_index
- runtime/exit_manager.py: live manager — ratchets the broker stop UP via
REPLACE/PATCH (never cancel = never naked), scales out, first-red exit
- alpaca_paper/client.py: replace_order (PATCH) + partial close_position
- run_live_paper.py: 'exits' step (12s); TRIGGER_MAX_ARMED 6 -> 10
- tests: exits sim/live + manager ratchet-never-cancel
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HIGH:
- Circuit breaker fails CLOSED: a repeated unreadable equity (API 401/429/5xx or
missing field) now halts new entries after 3 consecutive failures instead of
silently reporting 'no breach' and continuing to trade through a drawdown.
- execute(): map order statuses. Terminal-fail (rejected/expired/...) -> ok=False
(never armed as a resting entry); record ANY fill (partial too), not only an
exact 'filled'; require a real broker order id. approve_order only arms
genuinely OPEN statuses.
- sync: account/positions/orders failures now emit a BrokerHealthChangedEvent
(degraded) and step_sync surfaces 'DEGRADED', instead of a success-shaped None
that left risk gates reading stale snapshots as current.
MEDIUM:
- _flatten_all records a position closed only on a CONFIRMED close; the breaker
event reports 'halted_flatten_incomplete' + the unclosed symbols when a close
fails, instead of claiming the book is flat.
- submit_exit_order prefers close_position (handles short sign + fractional qty;
a plain sell of abs(qty) would double a short).
LOW:
- get_orders(nested=true) so bracket child legs aren't dropped from the snapshot.
+ tests/unit/test_execution_safety.py (4 regressions). 75/75 green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The live 'symbols' list only ever grew (discover appends every 5 min, dashboard
injects add more) and was comma-joined into ONE bars request; once >~100 symbols
Alpaca 400s the whole batch -> ALL minute-bar ingestion silently stops mid-session.
- client.get_minute_bars / get_daily_bars / get_latest_trades batch symbols in
chunks of 100 (cap-safe) and merge; get_daily_bars also paginates now.
- run_live_paper bounds the watchlist (WATCHER_MAX_UNIVERSE, default 80) so it
rotates toward current movers instead of growing all day.
Found by the silent-failure audit workflow. Verified _chunked(250)->[100,100,50],
live calls intact. 71/71 tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Snapshot before the DuckDB->PostgreSQL migration. data/, archive/, the 232MB
rebuild zip, the Windows .venv, and .env are excluded via .gitignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
