Review sweep: money-path fixes + security + delete 805MB dead weight
Consolidated 4-tool review (2 correctness agents + Codex + over-eng audit). Money-path (regression-tested): - _flatten_all: moved the fresh-broker close from close_session INTO the shared function + added a broker-held guard on entry cancellation, so the daily-loss CIRCUIT BREAKER (which called _flatten_all raw) no longer strips a just-filled bracket's legs or leaves a sync-gap fill naked. One guard, all callers. - restart state rebuild: _fresh_entries cap, _requested_symbols dedup, and losing benches now rebuild from today's events on boot — a mid-session restart no longer resets the cap to 0 or re-fires a resting order into a duplicate (the 2026-07-06 incident). - POST /orders no longer retries a lost-response URLError (was re-sending duplicate live orders); GET/DELETE still retry. - expire_stale_entries now @_locked (raced the trigger thread). - _session_open: ET conversion instead of a hardcoded 13:30-13:33Z window that only matched EDT — the fast path armed nothing all winter. Security: - admin panel fail-CLOSED on empty ADMIN_TOKEN (was fail-open on a panel fronting the docker socket); header-only token, constant-time compare. - admin API responses redact secret override VALUES (live keys would have echoed after a paper->live switch). - dashboard: removed wildcard CORS (cross-origin drive-by could drive trade actions); optional DASHBOARD_TOKEN gate on POST. Deletions (~805MB + dead code shipped in the prod image): momentum- rebuilt.zip, archive venv/htmlcov, and schwab/ core/ telemetry/ models/ runtime/providers/ storage/replay.py controlroom/ + three dead evaluation stubs + package.json — all verified zero-importer. +4 regression tests. 99 money-path/trigger/exit tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
momentum devandClaude Opus 4.8 committed
1 parent
cad9051a96
commit
d54a2a930c
28 files changed
+210
-1789
No files matched your search
@@ -0,0 +1,664 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Momentum Trading System - Architecture Diagram</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
|
||||
background: linear-gradient(135deg, #1e1e2e 0%, #2d2d44 100%);
|
||||
color: #e0e0e0;
|
||||
padding: 40px 20px;
|
||||
min-height: 100vh;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
h1 {
|
||||
text-align: center;
|
||||
margin-bottom: 40px;
|
||||
font-size: 2.5em;
|
||||
background: linear-gradient(135deg, #00d4ff, #00ffaa);
|
||||
-webkit-background-clip: text;
|
||||
-webkit-text-fill-color: transparent;
|
||||
background-clip: text;
|
||||
}
|
||||
|
||||
.diagram {
|
||||
background: #2d2d44;
|
||||
border: 2px solid #00d4ff;
|
||||
border-radius: 12px;
|
||||
padding: 40px;
|
||||
margin-bottom: 40px;
|
||||
position: relative;
|
||||
overflow-x: auto;
|
||||
}
|
||||
|
||||
svg {
|
||||
min-width: 100%;
|
||||
height: auto;
|
||||
}
|
||||
|
||||
.legend {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
|
||||
gap: 20px;
|
||||
margin-top: 30px;
|
||||
padding: 20px;
|
||||
background: rgba(0, 212, 255, 0.1);
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.legend-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.legend-box {
|
||||
width: 30px;
|
||||
height: 30px;
|
||||
border-radius: 4px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.color-data { background: #ff6b6b; }
|
||||
.color-process { background: #4ecdc4; }
|
||||
.color-event { background: #ffd93d; }
|
||||
.color-storage { background: #6c5ce7; }
|
||||
.color-output { background: #00d4ff; }
|
||||
.color-external { background: #95a5a6; }
|
||||
|
||||
.section {
|
||||
margin-bottom: 40px;
|
||||
}
|
||||
|
||||
.section-title {
|
||||
font-size: 1.3em;
|
||||
font-weight: 600;
|
||||
color: #00ffaa;
|
||||
margin-bottom: 15px;
|
||||
padding-bottom: 10px;
|
||||
border-bottom: 2px solid #00ffaa;
|
||||
}
|
||||
|
||||
.flow-box {
|
||||
background: #3a3a52;
|
||||
border: 1px solid #00d4ff;
|
||||
border-radius: 8px;
|
||||
padding: 15px;
|
||||
margin-bottom: 10px;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.step-number {
|
||||
display: inline-block;
|
||||
background: #00d4ff;
|
||||
color: #1e1e2e;
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
border-radius: 50%;
|
||||
text-align: center;
|
||||
line-height: 28px;
|
||||
font-weight: bold;
|
||||
margin-right: 10px;
|
||||
}
|
||||
|
||||
code {
|
||||
background: #1e1e2e;
|
||||
padding: 2px 6px;
|
||||
border-radius: 4px;
|
||||
font-family: "Monaco", "Menlo", monospace;
|
||||
color: #00ffaa;
|
||||
}
|
||||
|
||||
.arrow {
|
||||
color: #00d4ff;
|
||||
margin: 0 5px;
|
||||
}
|
||||
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
margin-top: 20px;
|
||||
background: #3a3a52;
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
th, td {
|
||||
padding: 12px;
|
||||
text-align: left;
|
||||
border-bottom: 1px solid #505070;
|
||||
}
|
||||
|
||||
th {
|
||||
background: #505070;
|
||||
color: #00ffaa;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
tr:hover {
|
||||
background: rgba(0, 212, 255, 0.1);
|
||||
}
|
||||
|
||||
.file-list {
|
||||
background: #3a3a52;
|
||||
border-radius: 8px;
|
||||
padding: 15px;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
.file-list li {
|
||||
margin-left: 20px;
|
||||
margin-bottom: 5px;
|
||||
list-style: disc;
|
||||
}
|
||||
|
||||
.file-list code {
|
||||
background: #1e1e2e;
|
||||
padding: 2px 6px;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
h1 {
|
||||
font-size: 1.8em;
|
||||
}
|
||||
|
||||
.diagram {
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.legend {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h1>⚡ Momentum Trading System Architecture</h1>
|
||||
|
||||
<div class="diagram">
|
||||
<svg viewBox="0 0 1200 800" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Define styles -->
|
||||
<defs>
|
||||
<style>
|
||||
.node-label { font-size: 14px; font-weight: bold; fill: white; }
|
||||
.arrow { stroke: #00d4ff; stroke-width: 2; fill: none; marker-end: url(#arrowhead); }
|
||||
.arrow-event { stroke: #ffd93d; stroke-width: 2; fill: none; marker-end: url(#arrowhead-yellow); }
|
||||
.arrow-data { stroke: #ff6b6b; stroke-width: 2; fill: none; marker-end: url(#arrowhead-red); }
|
||||
</style>
|
||||
<marker id="arrowhead" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
|
||||
<polygon points="0 0, 10 3, 0 6" fill="#00d4ff" />
|
||||
</marker>
|
||||
<marker id="arrowhead-yellow" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
|
||||
<polygon points="0 0, 10 3, 0 6" fill="#ffd93d" />
|
||||
</marker>
|
||||
<marker id="arrowhead-red" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
|
||||
<polygon points="0 0, 10 3, 0 6" fill="#ff6b6b" />
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<!-- DATA SOURCES -->
|
||||
<rect x="50" y="20" width="140" height="60" fill="#ff6b6b" rx="5"/>
|
||||
<text x="120" y="55" class="node-label" text-anchor="middle">Alpaca IEX Bars</text>
|
||||
|
||||
<!-- INGESTION LAYER -->
|
||||
<rect x="280" y="20" width="160" height="60" fill="#4ecdc4" rx="5"/>
|
||||
<text x="360" y="55" class="node-label" text-anchor="middle">Ingestion Module</text>
|
||||
|
||||
<!-- RESEARCH DB -->
|
||||
<rect x="530" y="20" width="140" height="60" fill="#6c5ce7" rx="5"/>
|
||||
<text x="600" y="50" class="node-label" text-anchor="middle">Research DB</text>
|
||||
<text x="600" y="70" style="font-size: 12px; fill: #ccc; text-anchor: middle;">(market_data)</text>
|
||||
|
||||
<!-- EVENT STORE (CENTER) -->
|
||||
<rect x="480" y="180" width="240" height="80" fill="#6c5ce7" rx="5" stroke="#ffd93d" stroke-width="3"/>
|
||||
<text x="600" y="210" class="node-label" text-anchor="middle">EVENT STORE</text>
|
||||
<text x="600" y="230" style="font-size: 12px; fill: #ccc; text-anchor: middle;">(momentum_events.duckdb)</text>
|
||||
<text x="600" y="250" style="font-size: 11px; fill: #ffd93d; text-anchor: middle;">append-only canonical events</text>
|
||||
|
||||
<!-- WATCHER -->
|
||||
<rect x="50" y="180" width="140" height="80" fill="#4ecdc4" rx="5"/>
|
||||
<text x="120" y="210" class="node-label" text-anchor="middle">Watcher</text>
|
||||
<text x="120" y="235" style="font-size: 12px; fill: white; text-anchor: middle;">symbol evaluation</text>
|
||||
<text x="120" y="255" style="font-size: 12px; fill: white; text-anchor: middle;">state machine</text>
|
||||
|
||||
<!-- EXECUTION -->
|
||||
<rect x="50" y="340" width="140" height="80" fill="#4ecdc4" rx="5"/>
|
||||
<text x="120" y="370" class="node-label" text-anchor: "middle">Execution Service</text>
|
||||
<text x="120" y="395" style="font-size: 12px; fill: white; text-anchor: middle;">sizing + approval</text>
|
||||
<text x="120" y="415" style="font-size: 12px; fill: white; text-anchor: middle;">order submission</text>
|
||||
|
||||
<!-- ALPACA PAPER SYNC -->
|
||||
<rect x="880" y="180" width="140" height="80" fill="#4ecdc4" rx="5"/>
|
||||
<text x="950" y="210" class="node-label" text-anchor="middle">Alpaca Sync</text>
|
||||
<text x="950" y="235" style="font-size: 12px; fill: white; text-anchor: middle;">positions, orders,</text>
|
||||
<text x="950" y="255" style="font-size: 12px; fill: white; text-anchor: middle;">account state</text>
|
||||
|
||||
<!-- ALPACA PAPER BROKER -->
|
||||
<rect x="880" y="340" width="140" height="80" fill="#95a5a6" rx="5"/>
|
||||
<text x="950" y="370" class="node-label" text-anchor="middle">Alpaca Paper</text>
|
||||
<text x="950" y="395" style="font-size: 12px; fill: white; text-anchor: middle;">order submission</text>
|
||||
<text x="950" y="415" style="font-size: 12px; fill: white; text-anchor: middle;">& fills</text>
|
||||
|
||||
<!-- DASHBOARD -->
|
||||
<rect x="800" y="500" width="220" height="80" fill="#00d4ff" rx="5"/>
|
||||
<text x="910" y="530" class="node-label" text-anchor="middle" fill="#1e1e2e">Dashboard API & UI</text>
|
||||
<text x="910" y="555" style="font-size: 12px; fill: #1e1e2e; text-anchor: middle;">http://127.0.0.1:8010</text>
|
||||
<text x="910" y="575" style="font-size: 12px; fill: #1e1e2e; text-anchor: middle;">projections + SSE streaming</text>
|
||||
|
||||
<!-- WATCHLIST PROVIDER -->
|
||||
<rect x="280" y="500" width="160" height="60" fill="#ff6b6b" rx="5"/>
|
||||
<text x="360" y="535" class="node-label" text-anchor="middle">Watchlist</text>
|
||||
|
||||
<!-- ARROWS -->
|
||||
<!-- Alpaca → Ingestion -->
|
||||
<path d="M 190 50 L 280 50" class="arrow-data"/>
|
||||
|
||||
<!-- Ingestion → Research DB -->
|
||||
<path d="M 440 50 L 530 50" class="arrow-data"/>
|
||||
|
||||
<!-- Research DB → Watcher -->
|
||||
<path d="M 530 80 L 300 220" class="arrow-data"/>
|
||||
|
||||
<!-- Watcher → Event Store -->
|
||||
<path d="M 190 220 L 480 220" class="arrow-event"/>
|
||||
|
||||
<!-- Event Store → Alpaca Sync (feedback) -->
|
||||
<path d="M 720 220 L 880 220" class="arrow-event"/>
|
||||
|
||||
<!-- Alpaca Sync → Alpaca Paper Broker -->
|
||||
<path d="M 950 260 L 950 340" class="arrow"/>
|
||||
|
||||
<!-- Alpaca Paper Broker → Alpaca Sync (fills feedback) -->
|
||||
<path d="M 920 340 L 920 260" class="arrow-data" stroke-dasharray="5,5"/>
|
||||
|
||||
<!-- Watcher also submits execution requests → Execution Service -->
|
||||
<path d="M 120 260 L 120 340" class="arrow-event"/>
|
||||
|
||||
<!-- Execution → Alpaca Paper Broker -->
|
||||
<path d="M 190 380 L 880 380" class="arrow"/>
|
||||
|
||||
<!-- Event Store → Dashboard -->
|
||||
<path d="M 600 260 L 910 500" class="arrow-event"/>
|
||||
|
||||
<!-- Watchlist → Watcher -->
|
||||
<path d="M 360 500 L 200 260" class="arrow-data" stroke-dasharray="5,5"/>
|
||||
</svg>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<div class="legend-item">
|
||||
<div class="legend-box color-data"></div>
|
||||
<span><strong>Data Flow:</strong> Market data, bars, watchlist</span>
|
||||
</div>
|
||||
<div class="legend-item">
|
||||
<div class="legend-box color-process"></div>
|
||||
<span><strong>Process:</strong> Computation & evaluation</span>
|
||||
</div>
|
||||
<div class="legend-item">
|
||||
<div class="legend-box color-event"></div>
|
||||
<span><strong>Events:</strong> Canonical state changes</span>
|
||||
</div>
|
||||
<div class="legend-item">
|
||||
<div class="legend-box color-storage"></div>
|
||||
<span><strong>Storage:</strong> Persistent data</span>
|
||||
</div>
|
||||
<div class="legend-item">
|
||||
<div class="legend-box color-output"></div>
|
||||
<span><strong>Output:</strong> User-facing interface</span>
|
||||
</div>
|
||||
<div class="legend-item">
|
||||
<div class="legend-box color-external"></div>
|
||||
<span><strong>External:</strong> Broker APIs</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- DETAILED FLOW SECTION -->
|
||||
<div class="section">
|
||||
<div class="section-title">📊 Complete Processing Flow (Every Tick)</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">1</span>
|
||||
<strong>INGEST</strong> <span class="arrow">→</span>
|
||||
<code>ingest_live_minute_bars()</code> +
|
||||
<code>ingest_daily_history()</code>
|
||||
<br/><small>Latest 1-minute bars from Alpaca IEX into research/market.duckdb + daily backfill for gap% and RVOL calculations</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">2</span>
|
||||
<strong>DISCOVER</strong> <span class="arrow">→</span>
|
||||
<code>ResearchWatchlistProvider.get_candidates()</code>
|
||||
<br/><small>Pull symbols from watchlist (manual, screener, or most-actives from Alpaca)</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">3</span>
|
||||
<strong>WATCH</strong> <span class="arrow">→</span>
|
||||
<code>Watcher.tick()</code>
|
||||
<br/><small>For each symbol: fetch bars from research DB, evaluate 9 setup criteria, emit symbol_discovered / state_changed events</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">4</span>
|
||||
<strong>EVALUATE</strong> <span class="arrow">→</span>
|
||||
<code>evaluate_setup()</code>
|
||||
<br/><small>Pure strategy logic: gap%, RVOL, bull-flag structure, price action quality. Emit criteria_evaluated events.</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">5</span>
|
||||
<strong>SIGNAL</strong> <span class="arrow">→</span>
|
||||
<code>Watcher</code> emits <code>signal_ready</code>
|
||||
<br/><small>Symbol passes ≥60% of criteria (tunable). Debounced (once per session unless it falls back to blocked).</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">6</span>
|
||||
<strong>EXECUTE</strong> <span class="arrow">→</span>
|
||||
<code>TradingExecutionService.tick()</code>
|
||||
<br/><small>On signal_ready: position sizing → risk checks (max concurrent, daily loss circuit breaker) → order_approval_requested event</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">7</span>
|
||||
<strong>APPROVE</strong> <span class="arrow">→</span>
|
||||
<code>Dashboard</code> or <code>--auto-approve</code>
|
||||
<br/><small>Manual: click Approve in dashboard. Auto: env flag TRADING_AUTO_APPROVE=1. Emit order_approved event.</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">8</span>
|
||||
<strong>SUBMIT</strong> <span class="arrow">→</span>
|
||||
<code>AlpacaPaperExecutor.submit()</code>
|
||||
<br/><small>Bracket order: entry (limit or market), profit target (entry + reward_multiple × risk), stop loss. Emit order_submitted event.</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">9</span>
|
||||
<strong>SYNC</strong> <span class="arrow">→</span>
|
||||
<code>AlpacaPaperSync.sync_account()</code>
|
||||
<br/><small>Poll Alpaca: account balance, open positions, pending orders. Emit account_summary_updated, account_positions_updated, account_orders_updated events.</small>
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<span class="step-number">10</span>
|
||||
<strong>PROJECT</strong> <span class="arrow">→</span>
|
||||
<code>Dashboard (SSE stream)</code>
|
||||
<br/><small>Read-only projections over event store: P&L, open positions, approval queue, activity feed, sparkline charts. Fall back to polling if SSE drops.</small>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- KEY MODULES -->
|
||||
<div class="section">
|
||||
<div class="section-title">🔧 Key Modules & Responsibilities</div>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Module</th>
|
||||
<th>Responsibility</th>
|
||||
<th>Key Files</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>research/ingestion</strong></td>
|
||||
<td>Ingest market data (bars, daily history), discover candidates, classify market sessions</td>
|
||||
<td>market_data.py, watcher_task.py, scheduler.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>research/</strong></td>
|
||||
<td>Query and project research databases (minute/daily bars, gappers, RSS news)</td>
|
||||
<td>query.py, schema.py, multi_schema.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>runtime/watcher</strong></td>
|
||||
<td>Symbol state machine: discovered → watching → ready | blocked. Emit all watcher events.</td>
|
||||
<td>watcher.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>strategy/evaluation</strong></td>
|
||||
<td>Pure setup evaluation: 9 criteria (gap%, RVOL, bull-flag, structure, quality, etc.)</td>
|
||||
<td>setup_evaluator.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>strategy/risk</strong></td>
|
||||
<td>Position sizing, risk per trade, reward multiple, daily loss circuit breaker</td>
|
||||
<td>position_sizing.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>strategy/backtest</strong></td>
|
||||
<td>Intraday backtest engine: replay bars, simulate entry/exit, calculate P&L</td>
|
||||
<td>engine.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>storage/event_store</strong></td>
|
||||
<td>Append-only event log (canonical source of truth). Correlate by session_id.</td>
|
||||
<td>event_store.py, event_schema.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>storage/projections</strong></td>
|
||||
<td>Query event store for current state: approval queue, open positions, P&L, etc.</td>
|
||||
<td>projections.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>alpaca_paper/</strong></td>
|
||||
<td>Alpaca paper broker integration: order submission, position sync, account state</td>
|
||||
<td>client.py, execution.py, sync.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>schwab/</strong></td>
|
||||
<td>Schwab live integration (OAuth, market data, positions, orders, health)</td>
|
||||
<td>oauth.py, market_data.py, positions.py, health.py</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>api/</strong></td>
|
||||
<td>Dashboard JSON API + SSE streaming. Read-only projections served as REST + events.</td>
|
||||
<td>main.py, static/</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>trading_execution.py</strong></td>
|
||||
<td>Signal → sizing → approval → order pipeline. Manages armed entries & timeouts.</td>
|
||||
<td>trading_execution.py</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- EVENT FLOW -->
|
||||
<div class="section">
|
||||
<div class="section-title">📨 Key Event Types (Event-Sourced)</div>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Event</th>
|
||||
<th>Emitted By</th>
|
||||
<th>Purpose</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>symbol_discovered</code></td>
|
||||
<td>Watcher</td>
|
||||
<td>New symbol added to watchlist</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>symbol_state_changed</code></td>
|
||||
<td>Watcher</td>
|
||||
<td>Symbol transitions: discovered → watching → ready | blocked</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>criteria_evaluated</code></td>
|
||||
<td>Watcher (via evaluate_setup)</td>
|
||||
<td>9 setup criteria evaluated for symbol (pass/fail/not-evaluated)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>signal_ready</code></td>
|
||||
<td>Watcher</td>
|
||||
<td>Symbol ready: ≥60% criteria passed. Debounced once per session.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>signal_blocked</code></td>
|
||||
<td>Watcher</td>
|
||||
<td>Symbol failed criteria check. Resets debounce.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>order_approval_requested</code></td>
|
||||
<td>TradingExecutionService</td>
|
||||
<td>Ready signal → sizing + risk checks passed. Awaiting approval or auto-submit.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>order_approved</code></td>
|
||||
<td>Dashboard (manual) or TradingExecutionService (auto)</td>
|
||||
<td>Approval given. Next: submit to broker.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>order_rejected</code></td>
|
||||
<td>Dashboard (manual) or TradingExecutionService (risk rule)</td>
|
||||
<td>Approval denied or risk rule triggered. Signal dequeued.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>order_submitted</code></td>
|
||||
<td>AlpacaPaperExecutor</td>
|
||||
<td>Bracket order sent to Alpaca (entry + target + stop).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>account_positions_updated</code></td>
|
||||
<td>AlpacaPaperSync</td>
|
||||
<td>Open positions snapshot from Alpaca. Includes entry, qty, P&L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>account_orders_updated</code></td>
|
||||
<td>AlpacaPaperSync</td>
|
||||
<td>Pending orders snapshot from Alpaca. Tracks fills, cancellations, expirations.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>account_summary_updated</code></td>
|
||||
<td>AlpacaPaperSync</td>
|
||||
<td>Account balance, buying power, margin, NAV, realized P&L.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- ENTRY POINT -->
|
||||
<div class="section">
|
||||
<div class="section-title">🚀 Main Entry Point: run_live_paper.py</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<strong>Overall orchestrator that ties everything together:</strong>
|
||||
<ol style="margin-left: 20px; margin-top: 10px;">
|
||||
<li>Parse CLI args (--once, --symbols, --auto-approve, --no-dashboard)</li>
|
||||
<li>Load .env and configure Alpaca client + settings</li>
|
||||
<li>Initialize EventStore (momentum_events.duckdb)</li>
|
||||
<li>Open ResearchConnection (market_data.duckdb) + ResearchWatchlistProvider</li>
|
||||
<li>Create Watcher + TradingExecutionService + AlpacaPaperSync + AlpacaPaperExecutor</li>
|
||||
<li>Optional: start DashboardAPI server (SSE + JSON endpoints) in background thread</li>
|
||||
<li>Main loop (or single pass with --once):
|
||||
<ul style="margin-top: 5px;">
|
||||
<li>Ingest latest minute bars + daily history</li>
|
||||
<li>Watcher.tick() → symbol evaluation + state changes + signal emissions</li>
|
||||
<li>TradingExecutionService.tick() → approval requests + submissions</li>
|
||||
<li>AlpacaPaperSync.sync_account() → positions + orders + account state</li>
|
||||
<li>Sleep (configurable interval, e.g., 60 seconds)</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ol>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- TYPICAL ISSUE DEBUGGING CHECKLIST -->
|
||||
<div class="section">
|
||||
<div class="section-title">🔍 Debugging: Where to Look When Something Breaks</div>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Symptom</th>
|
||||
<th>Likely Cause</th>
|
||||
<th>Check</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>No symbols appearing in Watcher</td>
|
||||
<td>Watchlist provider not returning candidates</td>
|
||||
<td>ResearchWatchlistProvider.get_candidates() → is research DB populated? Check data/research/market.duckdb</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Symbols discovered but never go ready</td>
|
||||
<td>Criteria evaluation failing or threshold too high</td>
|
||||
<td>Watcher → criteria_evaluated events. Check setup_evaluator.py scores. Lower ready_score_pct or min_quality config.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Signal ready but no approval request</td>
|
||||
<td>TradingExecutionService.tick() not running or risk rule triggered</td>
|
||||
<td>Check trading_execution.py: max_concurrent_positions, max_daily_loss_pct. Emit risk_rule_triggered? Check event store.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Approval request but order never submitted</td>
|
||||
<td>Dashboard approval not clicked OR auto-approve not set</td>
|
||||
<td>Dashboard approval queue visible? Or set TRADING_AUTO_APPROVE=1 in .env</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Order submitted but no fills from Alpaca</td>
|
||||
<td>API keys missing or Alpaca API down</td>
|
||||
<td>Check ALPACA_API_KEY / ALPACA_SECRET_KEY in .env. Run --once to test without loop.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Dashboard not updating (SSE fails)</td>
|
||||
<td>Network drop or event stream blocked</td>
|
||||
<td>Dashboard falls back to polling. Check http://127.0.0.1:8010/api/events (should return JSON). Check browser console.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Backtests showing different P&L than live</td>
|
||||
<td>Bar data mismatch or backtest engine using different fills</td>
|
||||
<td>Backtest engine simulates limit fills at exact prices. Live uses actual Alpaca fills. Compare bars in research DB vs Alpaca API.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- DATA STORAGE LOCATIONS -->
|
||||
<div class="section">
|
||||
<div class="section-title">💾 Data Storage Locations</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<strong>Event Store (Canonical Source of Truth)</strong><br/>
|
||||
<code>data/momentum_events.duckdb</code> — append-only events, indexed by timestamp + session_id
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<strong>Research Database (Market Data)</strong><br/>
|
||||
<code>data/research/market.duckdb</code> — minute bars + daily bars ingested from Alpaca
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<strong>Schwab Integration</strong><br/>
|
||||
<code>data/schwab_tokens.json</code> — OAuth tokens (git-ignored, 0600 perms)
|
||||
</div>
|
||||
|
||||
<div class="flow-box">
|
||||
<strong>Configuration</strong><br/>
|
||||
<code>.env</code> — API keys, trading settings (git-ignored)
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,268 @@
|
||||
================================================================================
|
||||
MOMENTUM SYSTEM - QUICK REFERENCE
|
||||
================================================================================
|
||||
|
||||
PHASE 1 (✅ COMPLETE): Core Infrastructure
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
What was built:
|
||||
✓ core/contracts.py Module I/O contracts (5 modules × 2 types each)
|
||||
✓ core/telemetry.py Metric collection + event emission
|
||||
✓ core/validation.py Data validation (schema, ranges, checksums, rules)
|
||||
✓ 3 new event types (ModuleTickEvent, DataValidationEvent, SymbolEvaluationDetailEvent)
|
||||
|
||||
Why it matters:
|
||||
BEFORE: ingest_live_minute_bars() → [silence] → "did it work?"
|
||||
AFTER: "500 bars collected, 0 validation failures, completed in 245ms"
|
||||
|
||||
Key classes:
|
||||
TelemetryCollector Record what's happening during execution
|
||||
TelemetryEmitter Emit events to event store + callbacks
|
||||
ModuleMetrics duration_ms, input_count, output_count, custom_metrics
|
||||
SchemaValidator Check required fields and types
|
||||
BarDataValidator Pre-built OHLCV validation
|
||||
|
||||
Quick import:
|
||||
from core.telemetry import TelemetryCollector, emit_module_tick
|
||||
from core.validation import SchemaValidator, BarDataValidator
|
||||
|
||||
Quick usage:
|
||||
collector = TelemetryCollector("evaluation")
|
||||
collector.record_input(5)
|
||||
collector.record_metric("gap_pct", 2.5)
|
||||
metrics = collector.finalize()
|
||||
emit_module_tick("evaluation", "completed", metrics)
|
||||
|
||||
================================================================================
|
||||
|
||||
PHASE 2 (📍 NEXT): Module Instrumentation (~4 hours)
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
What you'll do:
|
||||
1. Wrap ingest_live_minute_bars() with TelemetryCollector
|
||||
2. Wrap evaluate_setup() with telemetry + SymbolEvaluationDetailEvent
|
||||
3. Wrap research, execution, broker modules
|
||||
4. Update run_live_paper.py to initialize emitter
|
||||
5. Test and verify events appear in event store
|
||||
|
||||
Each module will emit:
|
||||
✓ ModuleTickEvent at completion (duration, counts, metrics)
|
||||
✓ DataValidationEvent when data validated
|
||||
✓ SymbolEvaluationDetailEvent for each evaluation stage (evaluation module only)
|
||||
|
||||
See: PHASE2_PREVIEW.md for exact code examples
|
||||
|
||||
================================================================================
|
||||
|
||||
PHASE 3 (🔲 TODO): Inspection Dashboard API (~6 hours)
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Build: api/inspection.py with REST endpoints
|
||||
|
||||
Endpoints:
|
||||
GET /api/inspection/modules module metrics table
|
||||
GET /api/inspection/pipeline pipeline flow diagram data
|
||||
GET /api/inspection/symbol/:symbol eval timeline for symbol
|
||||
GET /api/inspection/data-quality validation report
|
||||
GET /api/inspection/events SSE stream of events
|
||||
GET /api/inspection/module/:m/history last N executions
|
||||
|
||||
Projections to build:
|
||||
query_module_performance()
|
||||
query_symbol_evaluation_detail(symbol)
|
||||
query_data_quality_snapshot()
|
||||
query_pipeline_stats(session_id)
|
||||
|
||||
================================================================================
|
||||
|
||||
PHASE 4 (🔲 TODO): React Dashboard (~8 hours)
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Build: api/inspection/ directory with React SPA
|
||||
|
||||
Features:
|
||||
□ Pipeline diagram (Mermaid auto-layout)
|
||||
□ Module performance grid (5x5 table with metrics)
|
||||
□ Symbol inspector (search + expandable stages)
|
||||
□ Sparkline charts (OHLCV with entry/stop lines)
|
||||
□ Event timeline (live feed with filtering)
|
||||
□ Data quality report (heatmap + validation stats)
|
||||
□ Live SSE updates + polling fallback
|
||||
□ Mobile responsive
|
||||
|
||||
Port: http://127.0.0.1:8011 (separate from execution dashboard :8010)
|
||||
|
||||
================================================================================
|
||||
|
||||
PHASES 5-8 (OPTIONAL): Refactoring & Testing
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Phase 5: Module refactoring into explicit contracts
|
||||
modules/ingestion.py, research.py, evaluation.py, execution.py, broker.py
|
||||
|
||||
Phase 6: Module-level testing
|
||||
tests/modules/test_*.py (one per module)
|
||||
Fixtures for synthetic data (bull_flag_bars, fading_bars, etc.)
|
||||
|
||||
Phase 7: Dry-run mode + synthetic data
|
||||
python run_live_paper.py --dry-run --bars-provider synthetic:bull_flag
|
||||
|
||||
Phase 8: Rich TUI (terminal UI)
|
||||
python run_live_paper.py --tui
|
||||
(useful for SSH, CI/CD, no browser needed)
|
||||
|
||||
================================================================================
|
||||
|
||||
CURRENT FILES CREATED
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Core:
|
||||
core/__init__.py
|
||||
core/contracts.py 5 module contracts
|
||||
core/telemetry.py Metric collection + emission
|
||||
core/validation.py Multi-layer validation
|
||||
|
||||
Extended:
|
||||
storage/event_schema.py +3 event types
|
||||
|
||||
Documentation:
|
||||
README_ARCHITECTURE_REDESIGN.md Start here! Big picture
|
||||
BUILD_SUMMARY.md All 8 phases + timeline
|
||||
PHASE1_IMPLEMENTATION.md How Phase 1 works
|
||||
PHASE2_PREVIEW.md Exact Phase 2 code examples
|
||||
INSPECTION_ROADMAP.md What you'll see after Phase 4
|
||||
QUICK_REFERENCE.txt This file
|
||||
|
||||
Old (no longer needed):
|
||||
ARCHITECTURE_VISUAL.html Old static visualization
|
||||
SYSTEM_FLOW.txt Old text flow diagram
|
||||
DATA_TRANSFORMATIONS.md Old transformation doc
|
||||
|
||||
================================================================================
|
||||
|
||||
TIMELINE
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Week 1:
|
||||
Phase 1: Core infrastructure ✅ DONE (~2 hours)
|
||||
Phase 2: Module instrumentation 📍 START (~4 hours)
|
||||
|
||||
Week 2:
|
||||
Phase 3: Inspection API backend (~6 hours)
|
||||
Phase 4: React dashboard frontend (~8 hours)
|
||||
|
||||
Week 3:
|
||||
Phase 5: Module refactoring (~6 hours)
|
||||
Phase 6: Testing framework (~5 hours)
|
||||
|
||||
Week 4:
|
||||
Phase 7: Dry-run mode (~4 hours)
|
||||
Phase 8: TUI (optional) (~3 hours)
|
||||
|
||||
Total: ~4 weeks for full implementation, or ~2 weeks for core (1-4)
|
||||
|
||||
================================================================================
|
||||
|
||||
SUCCESS METRICS
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
After Phase 4, you'll be able to:
|
||||
|
||||
✓ Open http://127.0.0.1:8011 (inspection dashboard)
|
||||
✓ See real-time module pipeline (ingestion → research → eval → exec → broker)
|
||||
✓ Search for a symbol → see complete evaluation journey
|
||||
✓ Understand why AAPL is blocked (exact stage, metric, threshold)
|
||||
✓ View timestamps, metrics, and raw data for each stage
|
||||
✓ See live event stream
|
||||
✓ Access data quality report
|
||||
|
||||
Time to answer "why isn't AAPL signaling?" → 10 seconds (was 30 minutes)
|
||||
|
||||
================================================================================
|
||||
|
||||
KEY INSIGHTS
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
The System Used to Work Like:
|
||||
Alpaca API → [silence] → Signal ready → [silence] → Order submitted
|
||||
❓ Were bars validated?
|
||||
❓ Which criterion failed?
|
||||
❓ How long did evaluation take?
|
||||
❓ What was the gap%?
|
||||
|
||||
The System Will Work Like:
|
||||
Alpaca API → ModuleTickEvent(500 rows in, 245ms) → Signal ready
|
||||
→ SymbolEvaluationDetailEvent(gap_checked: 2.1% ✓)
|
||||
→ SymbolEvaluationDetailEvent(rvol_calculated: 1.2x ✗)
|
||||
→ SymbolEvaluationDetailEvent(score: 48% < 60%)
|
||||
→ Order rejected (criteria not met)
|
||||
|
||||
Dashboard shows all of this in real-time
|
||||
|
||||
================================================================================
|
||||
|
||||
NEXT IMMEDIATE STEPS
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
1. Read README_ARCHITECTURE_REDESIGN.md (10 minutes)
|
||||
2. Read BUILD_SUMMARY.md (20 minutes)
|
||||
3. Review PHASE1_IMPLEMENTATION.md (15 minutes)
|
||||
4. Skim PHASE2_PREVIEW.md for code examples (10 minutes)
|
||||
5. Decide: Start Phase 2 or want clarifications?
|
||||
|
||||
If starting Phase 2:
|
||||
Use PHASE2_PREVIEW.md as your code template
|
||||
Follow task list in BUILD_SUMMARY.md
|
||||
Test after each module (verify events in DB)
|
||||
Expected time: ~4 hours
|
||||
|
||||
================================================================================
|
||||
|
||||
QUESTIONS?
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Common questions answered in:
|
||||
|
||||
"How do I use TelemetryCollector?"
|
||||
→ PHASE1_IMPLEMENTATION.md, "Usage Pattern" section
|
||||
|
||||
"What does Phase 2 code look like?"
|
||||
→ PHASE2_PREVIEW.md, Examples 1-4
|
||||
|
||||
"What will the dashboard show me?"
|
||||
→ INSPECTION_ROADMAP.md, "Real-Time Module Dashboard" section
|
||||
|
||||
"Can I test modules independently?"
|
||||
→ Phase 6, coming soon (BUILD_SUMMARY.md)
|
||||
|
||||
"How do I run without API keys?"
|
||||
→ Phase 7, --dry-run mode (BUILD_SUMMARY.md)
|
||||
|
||||
"Where do I look if ingestion fails?"
|
||||
→ Inspection dashboard, Module Grid, click "ingestion", see error
|
||||
|
||||
"How do I know bars are being collected?"
|
||||
→ ModuleTickEvent shows input_count after Phase 2
|
||||
|
||||
"How do I verify research data?"
|
||||
→ DataValidationEvent after Phase 2, dashboard after Phase 4
|
||||
|
||||
================================================================================
|
||||
|
||||
READY TO START?
|
||||
────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Phase 1 is complete. You have the foundation.
|
||||
|
||||
Phase 2 is ~4 hours of work:
|
||||
- Wrap 5 modules with TelemetryCollector
|
||||
- Add validation checkpoints
|
||||
- Emit completion events
|
||||
- Test
|
||||
|
||||
After Phase 2, you'll start seeing telemetry events in your event store.
|
||||
After Phase 4, you'll have a fully functional inspection dashboard.
|
||||
|
||||
Let's go!
|
||||
|
||||
================================================================================
|
||||
@@ -0,0 +1,405 @@
|
||||
================================================================================
|
||||
MOMENTUM TRADING SYSTEM - COMPLETE LOGICAL FLOW
|
||||
================================================================================
|
||||
|
||||
ENTRY POINT: python run_live_paper.py [--once] [--symbols AAPL,SNDL] [--auto-approve]
|
||||
|
||||
================================================================================
|
||||
STARTUP PHASE
|
||||
================================================================================
|
||||
|
||||
1. INITIALIZE CONFIGURATION
|
||||
├─ Load .env (ALPACA_API_KEY, ALPACA_SECRET_KEY, etc.)
|
||||
├─ Parse CLI args
|
||||
└─ Create settings objects: AlpacaPaperSettings, ExecutionSettings, TradingModeSettings
|
||||
|
||||
2. INITIALIZE STORAGE & DATABASES
|
||||
├─ EventStore("data/momentum_events.duckdb")
|
||||
│ └─ Append-only log of all events (canonical source of truth)
|
||||
├─ ResearchConnection("data/research/market.duckdb")
|
||||
│ └─ Minute bars + daily bars for all symbols
|
||||
└─ AlpacaPaperClient (if API keys present)
|
||||
└─ Connection to Alpaca paper trading account
|
||||
|
||||
3. INITIALIZE RUNTIME COMPONENTS
|
||||
├─ Watcher (state machine for symbol discovery/evaluation)
|
||||
│ └─ Provider: ResearchWatchlistProvider (pulls symbols from research DB)
|
||||
├─ TradingExecutionService (signal → approval → order pipeline)
|
||||
├─ AlpacaPaperSync (poll Alpaca account/positions/orders)
|
||||
└─ DashboardAPI (optional HTTP server on port 8010)
|
||||
└─ Serves JSON API + static UI + SSE event stream
|
||||
|
||||
================================================================================
|
||||
MAIN LOOP (runs every tick, or once with --once)
|
||||
================================================================================
|
||||
|
||||
TICK = 1 iteration (typically 60 seconds)
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 1: INGEST LATEST MARKET DATA │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
ingest_live_minute_bars()
|
||||
├─ Call Alpaca REST API: get latest 1-minute bars
|
||||
├─ Insert into research/market.duckdb (bars table)
|
||||
└─ Return: list of symbols with new data
|
||||
|
||||
ingest_daily_history()
|
||||
├─ On startup or when needed:
|
||||
│ ├─ Alpaca REST API: get daily bars (past 30 days)
|
||||
│ ├─ Insert into research/market.duckdb (daily_bars table)
|
||||
│ └─ Used by: gap%, RVOL calculations
|
||||
└─ Return: daily bar snapshots
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 2: DISCOVER & UPDATE WATCHLIST │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
ResearchWatchlistProvider.get_candidates(session_date)
|
||||
├─ Check what symbols are in the watchlist for today
|
||||
│ ├─ Manual symbols (from --symbols or WATCHER_SYMBOLS env)
|
||||
│ ├─ Screener results (most-actives from Alpaca if --discover)
|
||||
│ └─ RSS feed candidates (if configured)
|
||||
├─ Return list of WatchCandidate objects (symbol, last_price, avg_volume)
|
||||
└─ Watcher._states updated with new symbols
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 3: EVALUATE SYMBOLS (CORE STRATEGY LOGIC) │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
Watcher.tick(bar_data, candidates)
|
||||
│
|
||||
└─ For each symbol in candidates:
|
||||
│
|
||||
├─ 1. FETCH BARS
|
||||
│ └─ ResearchWatchlistProvider.get_bars(symbol, session_date)
|
||||
│ └─ Query research DB for all minute bars from 9:30 AM to now
|
||||
│
|
||||
├─ 2. EMIT SYMBOL DISCOVERED (if new)
|
||||
│ └─ EventStore.emit(SymbolDiscoveredEvent(symbol, source, ...))
|
||||
│ └─ Event recorded with timestamp, session_id, payload
|
||||
│
|
||||
├─ 3. EVALUATE SETUP (if ≥10 bars available)
|
||||
│ │
|
||||
│ └─ evaluate_setup(bars, symbol) ← PURE STRATEGY ENGINE
|
||||
│ │
|
||||
│ ├─ Gap Check
|
||||
│ │ └─ (today_open - prev_close) / prev_close ≥ 2% ?
|
||||
│ │
|
||||
│ ├─ Relative Volume (RVOL)
|
||||
│ │ └─ (current 5-min avg vol / 20-day avg) ≥ 1.5 ?
|
||||
│ │
|
||||
│ ├─ Bull Flag Structure
|
||||
│ │ └─ Pullback from high + ascending support pattern?
|
||||
│ │
|
||||
│ ├─ Price Action Quality
|
||||
│ │ └─ Higher highs + higher lows? Clean bars?
|
||||
│ │
|
||||
│ ├─ 5 Other Criteria (tunable)
|
||||
│ │ └─ Momentum, volatility, trend confirmation, etc.
|
||||
│ │
|
||||
│ ├─ Calculate SCORE
|
||||
│ │ └─ (criteria_passed / total_criteria) * 100 = X%
|
||||
│ │
|
||||
│ └─ Return: {score, details, setup_data}
|
||||
│
|
||||
├─ 4. EMIT CRITERIA EVALUATED
|
||||
│ └─ EventStore.emit(CriteriaEvaluatedEvent(symbol, criteria=[...], score))
|
||||
│ └─ Each criterion: {name, passed, reason}
|
||||
│
|
||||
├─ 5. CHECK SIGNAL READY THRESHOLD
|
||||
│ └─ IF score ≥ ready_score_pct (default 60%) AND NOT already_ready_this_session:
|
||||
│ │
|
||||
│ ├─ Emit SIGNAL READY
|
||||
│ │ └─ EventStore.emit(SignalReadyEvent(symbol, entry, stop, target, ...))
|
||||
│ │
|
||||
│ ├─ Update symbol state: watching → ready
|
||||
│ │ └─ EventStore.emit(SymbolStateChangedEvent(symbol, "watching" → "ready"))
|
||||
│ │
|
||||
│ └─ Mark symbol as already signaled (no duplicate signals this session)
|
||||
│
|
||||
├─ 6. CHECK SIGNAL BLOCKED
|
||||
│ └─ ELSE IF score < ready_score_pct OR quality < min_quality:
|
||||
│ │
|
||||
│ ├─ Emit SIGNAL BLOCKED
|
||||
│ │ └─ EventStore.emit(SignalBlockedEvent(symbol, reason, ...))
|
||||
│ │
|
||||
│ ├─ Update symbol state: watching → blocked
|
||||
│ │ └─ EventStore.emit(SymbolStateChangedEvent(symbol, "watching" → "blocked"))
|
||||
│ │
|
||||
│ └─ RESET: allow signal_ready again if it recovers above threshold
|
||||
│
|
||||
└─ END for each symbol
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 4: EXECUTION PIPELINE (signal_ready → approval → order) │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
TradingExecutionService.tick()
|
||||
│
|
||||
└─ Query event store for new signal_ready events (not yet processed)
|
||||
│
|
||||
└─ For each signal_ready:
|
||||
│
|
||||
├─ 1. CHECK RISK RULES
|
||||
│ │
|
||||
│ ├─ How many positions are open?
|
||||
│ │ └─ IF open_positions ≥ max_concurrent_positions (e.g., 3):
|
||||
│ │ └─ RiskRuleTriggeredEvent("max concurrent positions")
|
||||
│ │ └─ SKIP this signal
|
||||
│ │
|
||||
│ ├─ Check daily loss circuit breaker
|
||||
│ │ └─ IF realized_pnl_today ≤ -max_daily_loss_pct * account_equity:
|
||||
│ │ └─ RiskRuleTriggeredEvent("daily loss exceeded")
|
||||
│ │ └─ SKIP this signal (halt trading for the day)
|
||||
│ │
|
||||
│ └─ Max orders per tick
|
||||
│ └─ Only submit max_orders_per_tick (e.g., 1) per tick to avoid order queue floods
|
||||
│
|
||||
├─ 2. CALCULATE POSITION SIZE
|
||||
│ │
|
||||
│ └─ calculate_position_size(entry, stop, account_equity, risk_per_trade_pct)
|
||||
│ │
|
||||
│ ├─ Risk per trade = account_equity * risk_per_trade_pct (e.g., 1%)
|
||||
│ ├─ Risk per share = entry - stop
|
||||
│ ├─ Shares = risk_per_trade / risk_per_share
|
||||
│ └─ Return: {shares, entry, stop, target}
|
||||
│
|
||||
├─ 3. EMIT ORDER APPROVAL REQUESTED
|
||||
│ └─ EventStore.emit(OrderApprovalRequestedEvent(
|
||||
│ symbol, entry, stop, target, shares, risk_dollar, notional, ...
|
||||
│ ))
|
||||
│
|
||||
├─ 4. AUTO-APPROVE OR QUEUE FOR MANUAL APPROVAL
|
||||
│ │
|
||||
│ ├─ IF --auto-approve or TRADING_AUTO_APPROVE=1:
|
||||
│ │ │
|
||||
│ │ ├─ Emit OrderApprovedEvent immediately
|
||||
│ │ │ └─ EventStore.emit(OrderApprovedEvent(order_id, ...))
|
||||
│ │ │
|
||||
│ │ └─ Jump to Phase 5 (SUBMIT ORDER)
|
||||
│ │
|
||||
│ └─ ELSE (manual approval):
|
||||
│ │
|
||||
│ └─ Add to approval_queue in dashboard
|
||||
│ └─ User sees: symbol, entry, stop, target, risk, share count, sparkline chart
|
||||
│ └─ User clicks Approve or Reject
|
||||
│ └─ Dashboard emits OrderApprovedEvent or OrderRejectedEvent
|
||||
│
|
||||
├─ 5. ON APPROVAL (if approved)
|
||||
│ │
|
||||
│ └─ TradingExecutionService detects OrderApprovedEvent
|
||||
│ │
|
||||
│ ├─ Create ExecutionRequest
|
||||
│ │ └─ {symbol, entry, stop, target, shares, order_type: "limit"/"market", ...}
|
||||
│ │
|
||||
│ └─ Call AlpacaPaperExecutor.submit(request)
|
||||
│ │
|
||||
│ └─ AlpacaPaperExecutor.submit() [PHASE 5]
|
||||
│
|
||||
└─ END for each signal_ready
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 5: BROKER ORDER SUBMISSION │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
AlpacaPaperExecutor.submit(execution_request)
|
||||
│
|
||||
├─ Create bracket order:
|
||||
│ │
|
||||
│ ├─ ENTRY ORDER
|
||||
│ │ └─ Type: limit @ entry_price (restful, will fill on touch, can be cancelled)
|
||||
│ │ OR market (fills immediately at market)
|
||||
│ │
|
||||
│ ├─ PROFIT TARGET
|
||||
│ │ └─ Type: limit @ target_price
|
||||
│ │ (target = entry + (entry - stop) * reward_multiple)
|
||||
│ │
|
||||
│ └─ STOP LOSS
|
||||
│ └─ Type: stop @ stop_price (becomes market order when stop is hit)
|
||||
│
|
||||
├─ Submit to Alpaca REST API
|
||||
│ └─ Returns: Alpaca order_id
|
||||
│
|
||||
├─ Emit OrderSubmittedEvent
|
||||
│ └─ EventStore.emit(OrderSubmittedEvent(symbol, order_id, entry, stop, target, ...))
|
||||
│
|
||||
├─ Track armed entry in TradingExecutionService._armed dict
|
||||
│ └─ _armed[order_id] = {symbol, entry_price, broker_order_id, armed_at, checks}
|
||||
│
|
||||
└─ Monitor for:
|
||||
│
|
||||
├─ Entry Fill
|
||||
│ └─ If entry fills → position open → wait for target or stop
|
||||
│
|
||||
├─ Entry Timeout (if not filled after entry_timeout_bars minutes)
|
||||
│ └─ Call Alpaca to cancel unfilled entry order
|
||||
│
|
||||
├─ Price Break Invalidation
|
||||
│ └─ If current price < (entry - entry_invalidate_pct * entry):
|
||||
│ └─ Cancel entry order (price broke back below entry)
|
||||
│
|
||||
└─ Full Fill (target or stop hit)
|
||||
└─ Position closes → emit OrderFilledEvent
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 6: ACCOUNT & POSITION SYNC │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
AlpacaPaperSync.sync_account()
|
||||
│
|
||||
├─ 1. GET ACCOUNT SUMMARY
|
||||
│ └─ Alpaca REST: /account
|
||||
│ └─ Returns: {equity, buying_power, multiplier, long_market_value, ...}
|
||||
│
|
||||
├─ 2. EMIT ACCOUNT SUMMARY UPDATED
|
||||
│ └─ EventStore.emit(AccountSummaryUpdatedEvent(account_data))
|
||||
│
|
||||
├─ 3. GET OPEN POSITIONS
|
||||
│ └─ Alpaca REST: /positions
|
||||
│ └─ Returns: [{symbol, qty, avg_fill_price, current_price, unrealized_pl, ...}]
|
||||
│
|
||||
├─ 4. EMIT ACCOUNT POSITIONS UPDATED
|
||||
│ └─ EventStore.emit(AccountPositionsUpdatedEvent(positions))
|
||||
│
|
||||
├─ 5. GET PENDING ORDERS
|
||||
│ └─ Alpaca REST: /orders?status=open
|
||||
│ └─ Returns: [{id, symbol, qty, type, status, filled_qty, ...}]
|
||||
│
|
||||
├─ 6. EMIT ACCOUNT ORDERS UPDATED
|
||||
│ └─ EventStore.emit(AccountOrdersUpdatedEvent(orders))
|
||||
│
|
||||
└─ 7. CALCULATE SESSION P&L
|
||||
└─ Query event store for all fills this session
|
||||
├─ For each closed trade:
|
||||
│ └─ realized_pnl = (exit_price - entry_price) * qty
|
||||
├─ Aggregate:
|
||||
│ └─ total_realized_pnl, total_unrealized_pnl, win_rate, avg_r, trade_count
|
||||
└─ Used by dashboard P&L strip
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 7: DASHBOARD UPDATES (READ-ONLY PROJECTIONS) │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
DashboardAPI (runs in background thread)
|
||||
│
|
||||
├─ SSE Stream Handler (/api/events)
|
||||
│ └─ Pushes new events to all connected clients in real-time
|
||||
│ └─ Falls back to polling if connection drops
|
||||
│
|
||||
├─ JSON Endpoints:
|
||||
│ │
|
||||
│ ├─ GET /api/approval-queue
|
||||
│ │ └─ query_approval_queue(event_store)
|
||||
│ │ └─ Returns: [{symbol, entry, stop, target, shares, risk_pct, notional, ...}]
|
||||
│ │
|
||||
│ ├─ GET /api/positions
|
||||
│ │ └─ query_account_positions_snapshot(event_store)
|
||||
│ │ └─ Returns: [{symbol, qty, entry, current, unrealized_pl, ...}]
|
||||
│ │
|
||||
│ ├─ GET /api/pnl
|
||||
│ │ └─ query_pnl_snapshot(event_store)
|
||||
│ │ └─ Returns: {realized, unrealized, total, win_rate, avg_r, open_count, closed_count}
|
||||
│ │
|
||||
│ ├─ GET /api/activity-feed
|
||||
│ │ └─ query_activity_feed(event_store, limit=100)
|
||||
│ │ └─ Returns: [{timestamp, event_type, message, symbol, ...}]
|
||||
│ │
|
||||
│ ├─ GET /api/symbol/:symbol/bars
|
||||
│ │ └─ query_minute_bars(research_db, symbol, session_date)
|
||||
│ │ └─ Returns: [{timestamp, open, high, low, close, volume}]
|
||||
│ │ (for sparkline chart rendering)
|
||||
│ │
|
||||
│ └─ POST /api/approval/:order_id/approve|reject
|
||||
│ └─ Emits OrderApprovedEvent or OrderRejectedEvent
|
||||
│
|
||||
└─ UI Features:
|
||||
├─ P&L Strip (realized, unrealized, total, win rate, R multiple, counts)
|
||||
├─ Approval Queue (cards with entry/stop/target preview, Approve/Reject buttons)
|
||||
├─ Open Positions (list with live sparkline charts, P&L per position)
|
||||
├─ Activity Feed (chronological log of all events)
|
||||
├─ Symbol Details (expand to see 9 criteria: pass/fail/not-evaluated)
|
||||
└─ Keyboard Shortcuts: A=Approve, R=Reject, S=Sound Toggle, /=Add Symbol, ?=Help
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 8: CLEANUP & REPEAT │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
After all phases complete:
|
||||
├─ Log tick summary (discovered, ready, blocked, evaluated, errors)
|
||||
├─ IF --once: exit
|
||||
└─ ELSE: sleep(60 seconds) and go back to PHASE 1
|
||||
|
||||
================================================================================
|
||||
KEY CONCEPTS & INVARIANTS
|
||||
================================================================================
|
||||
|
||||
EVENT SOURCING
|
||||
• All state changes are recorded as immutable events in the append-only event store
|
||||
• The current state is reconstructed by replaying events (projections)
|
||||
• Every read operation queries the event store; no other state is authoritative
|
||||
• Correlation ID = session_id (all events from one trading session have the same correlation_id)
|
||||
|
||||
SIGNAL DEBOUNCING
|
||||
• Once a symbol emits signal_ready, it cannot emit another signal_ready until:
|
||||
→ It falls back to signal_blocked (criteria fail), which resets the debounce
|
||||
• Prevents duplicate signals from the same symbol in one session
|
||||
|
||||
STATE MACHINE (Watcher)
|
||||
• discovered → watching → { ready | blocked } (can toggle between ready ↔ blocked)
|
||||
• States are per-session; new session = fresh symbol states
|
||||
|
||||
BRACKET ORDER
|
||||
• Entry (limit) + Profit Target (limit) + Stop Loss (stop) submitted as a single order
|
||||
• If entry fills:
|
||||
→ Two child orders activate (target and stop)
|
||||
→ First one to fill = exit (other cancels automatically)
|
||||
• If entry never fills:
|
||||
→ Entry can timeout (entry_timeout_bars) or be cancelled on price break
|
||||
|
||||
RISK CONTROLS
|
||||
• Max concurrent positions: once N positions open, no more entry orders until one closes
|
||||
• Daily loss circuit breaker: once losses hit threshold, halt trading for the session
|
||||
• Entry timeout: if entry rests >N minutes, cancel to avoid stale orders
|
||||
• Price break invalidation: if price falls back >X% below entry, cancel entry (setup failed)
|
||||
|
||||
APPROVAL WORKFLOW
|
||||
• Manual: user clicks Approve in dashboard → OrderApprovedEvent
|
||||
• Auto: --auto-approve flag or TRADING_AUTO_APPROVE=1 env var → immediate approval
|
||||
• Rejection: user clicks Reject or risk rule triggers → OrderRejectedEvent (queued signal removed)
|
||||
|
||||
================================================================================
|
||||
TROUBLESHOOTING BY PHASE
|
||||
================================================================================
|
||||
|
||||
Phase 1 (Ingest):
|
||||
• No new bars? Check if Alpaca API keys are valid. Run --once in dry mode.
|
||||
• Bars not in research DB? Check data/research/market.duckdb exists and is writable.
|
||||
|
||||
Phase 2 (Discover):
|
||||
• No symbols in watchlist? Check --symbols / WATCHER_SYMBOLS env / most-actives screener.
|
||||
|
||||
Phase 3 (Evaluate):
|
||||
• Symbols stuck in "watching"? Criteria too strict. Lower ready_score_pct or min_quality.
|
||||
• Signals appear then disappear? They're being blocked. Check criteria_evaluated events.
|
||||
• Check event store: SELECT * FROM events WHERE event_type = 'criteria_evaluated'
|
||||
|
||||
Phase 4 (Execution):
|
||||
• Signal ready but no approval request? Risk rule triggered. Check RiskRuleTriggeredEvent.
|
||||
• Approval request visible but user forgets to click? Use --auto-approve for testing.
|
||||
|
||||
Phase 5 (Broker):
|
||||
• Order submitted but no fill? Entry price may be stale. Use market orders (entry_order_type).
|
||||
• Entry stuck for >N minutes? Timeout will cancel it. Check entry_timeout_bars setting.
|
||||
|
||||
Phase 6 (Sync):
|
||||
• Account sync fails? Alpaca API likely down. Dry mode still works.
|
||||
• Positions don't match Alpaca? Event store is source of truth. Refresh browser.
|
||||
|
||||
Phase 7 (Dashboard):
|
||||
• Dashboard not updating? Check browser console for errors. Try hard-refresh (Ctrl+Shift+R).
|
||||
• SSE stream failing? Falls back to polling (/api/approval-queue, etc.). Check /api/events.
|
||||
|
||||
================================================================================
|
||||
END
|
||||
================================================================================
|
||||
Reference in new issue
Block a user