Baseline before Postgres migration (WS0)

Snapshot before the DuckDB->PostgreSQL migration. data/, archive/, the 232MB
rebuild zip, the Windows .venv, and .env are excluded via .gitignore.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
momentum devandClaude Opus 4.8 committed 2026-06-15 23:39:29 -04:00
commit 007cd95c79
133 files changed
+17321

No files matched your search

+120
View File
@@ -0,0 +1,120 @@
# Comprehensive Backtest Guide
This guide will help you run a multi-week backtest to assess the profitability and risk profile of the momentum strategy before trading live.
## Quick Start (Windows Terminal)
Run a 1-week backtest on small-cap momentum names (the strategy's sweet spot):
```powershell
uv run comprehensive_backtest.py --start 2026-06-05 --end 2026-06-12
```
This analyzes every trading day in that range, pulling minute bars for the specified symbols and reporting:
- **Daily P&L** — win/loss pattern by day
- **Symbol breakdown** — which tickers traded well/poorly
- **Trade-by-trade details** — entry/exit, profit/loss, reason for exit
- **Statistical summary** — win rate, average R-multiple, profit factor, risk metrics
- **Loss analysis** — patterns in losses (gap-throughs, wide stops, etc.)
## What the Output Tells You
### Summary Metrics
- **Win Rate**: % of trades that were profitable. 40%+ is acceptable; 50%+ is strong.
- **Profit Factor**: Total wins ÷ absolute value of total losses.
- < 1.3: marginal edge or slippage issues (investigate before live)
- 1.3–2.0: decent edge
- > 2.0: strong edge
- **Avg R-Multiple**: Average return per unit of risk. 1.5+ suggests the strategy rewards you for the risk taken.
- **Max Loss**: Largest single loss. If it's 2–3× worse than your average loss, stops may be too wide or slippage is severe.
### Red Flags to Watch For
1. **Profit factor < 1.3** — your losses are eating most of your wins
2. **Win rate < 40%** — even with 2:1 R ratio, this struggles to be profitable
3. **Max loss >> average loss** — suggests gap-throughs or wide stops on losers
4. **Inconsistent daily results** — if one bad day wipes out a week's gains, volatility is too high
## Advanced Options
```powershell
# Test a different date range (e.g., last 2 weeks)
uv run comprehensive_backtest.py --start 2026-05-27 --end 2026-06-12
# Test on your watched symbols instead of the defaults
uv run comprehensive_backtest.py --start 2026-06-05 --end 2026-06-12 --symbols SNDL,MARA,RIOT
# Test a full month (if available)
uv run comprehensive_backtest.py --start 2026-05-13 --end 2026-06-12
```
## Important Caveats
1. **This is NOT the live execution path** — the backtest engine shows signal→entry→exit on historical bars, but it does NOT exercise:
- The new auto-arm resting-limit logic
- The tick-by-tick entry invalidation (live prices)
- The approval queue or manual trading decisions
All of those features only run in `uv run run_live_paper.py --auto-approve`. This backtest proves the signal-generation and position-management logic works; it's not a full replay of the live system.
2. **IEX feed gaps** — free tier only sees IEX trades (a subset of consolidated). On illiquid small-caps there may be minutes with no prints, causing simulated fills to happen on stale data. The paid SIP feed (ALPACA_DATA_FEED=sip) would see more ticks.
3. **Minute-bar vs tick-by-tick** — the backtest assumes fills happen at the bar open of the next bar. Live, entries/exits are much more granular (actual filled prices as they happen). This is usually conservative (against you) but can help on some exits.
4. **Slippage assumptions** — the engine includes configurable slippage (base_spread_pct in config). The default assumes ~0.05% slippage; real fills vary by size, liquidity, and market conditions.
## Interpretation Guide
### Strong Strategy (Should Trade)
- Win rate ≥ 45%
- Profit factor ≥ 1.8
- Avg R ≥ 1.5R
- Max loss ≤ 2× average loss
- Consistent week-to-week results
### Marginal Strategy (Investigate Further)
- Win rate 40–45%
- Profit factor 1.3–1.8
- Avg R ≈ 1.0–1.5R
- Max loss 2–3× average loss
- High daily variance
### Weak Strategy (Do Not Trade)
- Win rate < 40%
- Profit factor < 1.3
- Avg R < 0.5R
- Max loss > 3× average loss
- Losing weeks common
## Next Steps
1. **Run the backtest** for the most recent 2–4 weeks using small-cap symbols (SNDL, MARA, RIOT, BBAI, SOUN, PLUG).
2. **Review the loss analysis** — click "Loss Analysis" section and understand why you're losing. Is it:
- Entries are wrong (enter on fakeouts)? → Tighten entry criteria
- Stops are too wide (gap-throughs)? → Use a tighter stop or accept wider slippage
- Exits are leaving money? → Adjust target or exit rules
3. **Check the daily pattern** — if one symbol dominates losses, exclude it (maybe illiquid on IEX).
4. **If strong** → uv run run_live_paper.py --auto-approve on the next trading day.
5. **If marginal/weak** → dig into the top losers to understand the pattern before risking real capital.
## Debugging a Bad Backtest
If you see 0 trades despite seeing signals:
```powershell
# First, make sure minute bars were fetched
uv run fetch_minute_bars.py SNDL MARA RIOT --lookback 5000
# Then try a single-symbol, single-day backtest to see detailed output
uv run backtest_cli.py --symbol SNDL --date 2026-06-12
```
If you get "no symbols with minute bars," the date may not have tradeable data (check it's a weekday) or the lookback was too short.
## Questions?
- The backtest engine: `strategy/backtest/engine.py`
- Setup evaluation (what defines a "signal"): `strategy/evaluation/setup_evaluator.py`
- Risk/position sizing: `strategy/risk/position_sizing.py`
- Config defaults: `config.py` (BacktestConfig section)