Accuracy: news_enrichment now sends a JSON schema to Ollama's `format`
(enum-locked catalyst_type, typed fields) so decoding is constrained at
generation time, and build_prompt carries 3 few-shot examples (offering,
FDA, earnings) — the biggest accuracy lever for a small local model.
classify_headline gains use_schema (default True; False = plain json).
Validation:
- research/ingestion/catalyst_eval.py — hand-labeled fixture + pure
score_predictions (type accuracy, is_dilutive precision/recall, sentiment
sign). catalyst_eval.py CLI A/Bs models (--model) against the same set.
- validate_catalysts.py — joins enriched labels to each stored gapper's
open->close / open->high outcome; pure outcome_correlation splits by
dilution flag and bullish-score bucket. Reports coverage honestly (news is
live-only) and writes a CSV. Run before flipping the Phase 2 flags.
Tests: schema-in-request assertions, pure eval/aggregator coverage. 212 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M5HTMkaadGDnAPtXe2hWwg
Leverage a local Ollama model to interpret news catalysts for the small-cap
momentum strategy — distinguishing bullish catalysts (FDA approval, M&A) from
dilutive offerings the technicals alone can't see. Advisory-first, then automate.
Phase 1 (advisory; no change to trade behavior):
- research/ingestion/news_enrichment.py: stdlib urllib call to Ollama with
structured JSON output, classifying each headline into a catalyst type,
sentiment, conviction and dilution flag. Graceful degradation (None on any
failure) so trading is never blocked.
- news_catalyst_cache table + populate the previously-dead news_events.sentiment.
- Runs on the existing interval Scheduler (off the hot path), gated by
OllamaConfig; deduped by the raw_news_items payload hash.
- Surface the advisory on the dashboard approval queue (projection + api + UI).
Phase 2 (automation; ships gated OFF):
- Optional catalyst_score blended into calculate_setup_quality (None reproduces
the exact legacy weights); pure pass-through via evaluate_setup.
- Dilution veto in TradingExecutionService (slow + fast paths) emitting a
catalyst_dilution_veto risk event, behind NEWS_DILUTION_VETO_ENABLED.
Config (OllamaConfig.from_env), .env.example docs, and tests for parsing,
persistence/dedupe, the projection, the score blend, the veto, and an
end-to-end enrich->advisory flow (Ollama mocked). Default model moves to
qwen2.5:7b-instruct (best small-model JSON, fits a 12GB GPU).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M5HTMkaadGDnAPtXe2hWwg
The duckdb library is already gone (not a dependency, not importable, no
`import duckdb` anywhere) — these were leftover text references that misled.
- Delete storage/schema.py: 418 lines of DuckDB DDL with zero importers. The
Postgres schema is created by db_pg._apply_schema + event_store._ensure_schema.
- Neutralise stale default db_path strings (./data/momentum*.duckdb) in
event_store, config, run_live_paper, dashboard_api — the connection layer
ignores the path (DATABASE_URL is the datastore), so these phantom file paths
only created confusion. Dropped the now-pointless os.makedirs for them.
- research/multi_schema: the "three DuckDB files" are one Postgres DB now (all
resolve to the same connection); doc + constants updated to match reality.
- Fix stale "DuckDB handle" comments in api/main.
Kept the DuckDB mentions in storage/db_pg.py + storage/db.py — they document
why the Postgres facade mimics DuckDB's ? placeholders and :memory: semantics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
News (you were right it's relevant): recent_news_map() pulls fresh catalysts per
ticker from the ingested RSS feeds; step_arm tags each armed candidate and BOOSTS
catalyst-backed gappers up the rank (NEWS_RANK_BOOST) — a fresh PR/FDA/offering
is why a small-cap runs, so it surfaces the hotter name earlier and higher. The
board shows a 📰 marker + the headline. Feeds verified live (PR Newswire /
Business Wire / GlobeNewswire) — pulling real catalysts (FDA approvals, offerings).
Forward-tested only (news isn't OOS-backtestable).
eod_replay.sh + crontab: replay at 16:05 (grow dataset), tune at 17:30.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- discover_active_symbols: distinguish AUTH/entitlement failures (HTTP 401/403,
logged as ERROR, 'not no-movers') from transient/empty in the screener except.
- run_live_paper: the boot watchlist event records an explicit error when it
falls back (screener empty at boot) instead of looking like 'market quiet'.
- discovery.news_candidates: log DB faults instead of swallowing to [].
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- discover_active_symbols logs the screener error instead of swallowing silently.
- run_discovery records an explicit error when the universe is empty (surfaces in
the discovery event + Grafana).
- momentum doctor now FAILs when the latest discovery universe is 0 (the morning's
dead screener would have shown red within a minute).
- tests/unit/research/test_discovery.py: pins get_most_actives top<=100 and the
ETF/price-band filtering.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- discover_active_symbols requested get_most_actives(top*3); once screen_universe
used top*2 headroom for the ETF filter (top=20 -> 40 -> get_most_actives(120)),
this exceeded Alpaca's hard cap of 100 -> HTTP 400, swallowed by the bare
except -> discovery returned EMPTY every cycle. The live loop therefore traded
the static fallback list (non-gappers) instead of real movers all morning.
Fix: cap at min(top*3, 100). Verified: screen_universe now returns 20 real
sub-$20 names, ETF-filtered.
- Add ./momentum launcher (project venv) so 'momentum watch' works without the
full venv path ('python' on the box is the Windows pyenv shim, no deps).
67/67 tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- grafana/: docker-compose + provisioned Postgres datasource + 'Momentum — Live'
dashboard (evaluation board with status colours, ready signals, risk events,
bars + equity timeseries) reading the event store via JSONB extraction.
Run: cd grafana && docker compose up -d -> http://localhost:3000.
- RSS news-catalyst discovery wired: extract_tickers also matches $CASHTAGS;
run_discovery pulls tickers from recent news into the universe when NEWS_FEEDS
is set (no-op / safe otherwise).
- docs: .env.example NEWS_FEEDS; GO_LIVE monitoring adds 'momentum watch' + Grafana.
67/67 tests green. Live board (momentum watch) verified rendering the evaluation
board with real determinations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- trading_execution: implement the daily-loss circuit breaker (account equity vs
prior close; latches once tripped; fail-open on a data hiccup) — was
advertised in the docstring/settings but never implemented.
- discovery: filter leveraged ETFs (SOXS/TZA/TSLL/NVD/BITO/...) out of the
universe; retry the screener on a transient empty (rate-limit) result.
- run_live_paper: fallback watchlist is now liquid sub-$20 names, not mega-caps
(mega-caps just get filtered by the watcher band -> nothing to trade).
Verified live: forced ready signal -> auto-approve -> real Alpaca bracket order
(pending_new, then cancelled); breaker trips at -3% (simulated). 67/67 tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- research/ingestion/discovery.py: screen sub-$20 most-actives, backfill daily,
rank gappers, persist scanner snapshot, and EMIT a discovery event.
- run_live_paper: watchlist defaults to the screener universe (was a hardcoded
mega-cap list); add a periodic 'discover' task; emit bars-collected
(per-symbol counts) and discovery events every pass.
- IngestionResult.per_symbol: per-symbol row counts (incl 0) so 'are bars
collected?' is answerable per symbol, distinguishing market-closed from broken.
Verified live vs Postgres: watches SOXS/RGNT/GRAB/SOFI/ACHR/... (sub-$20),
820 daily bars backfilled, discovery+ingestion events land in the event store.
67/67 tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- storage/db_pg.py: thin psycopg2 connection shim (?->%s translation,
autocommit, .df(), and per-test isolated schema for ':memory:').
- storage/db.py: delegate get_connection/get_memory_connection to db_pg.
- storage/pg_schema.sql: Postgres schema, applied on connect.
- research/query.py runs UNMODIFIED on Postgres via the shim.
- ingestion upserts: INSERT OR REPLACE -> INSERT ... ON CONFLICT DO UPDATE.
- event_store: drop invalid btree-on-JSON-text index.
Full logic suite: 67/67 green against Postgres (was DuckDB).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Snapshot before the DuckDB->PostgreSQL migration. data/, archive/, the 232MB
rebuild zip, the Windows .venv, and .env are excluded via .gitignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude