momentum-system/plans/002-dead-weight-deletion.md

Plan 002: Delete the confirmed-dead surface — every module and table names its reader or dies

Executor instructions: Follow this plan step by step. Run every verification command and confirm the expected result before moving to the next step. If anything in the "STOP conditions" section occurs, stop and report — do not improvise. When done, update the status row for this plan in plans/README.md.

Drift check (run first): git diff --stat d54a2a9..HEAD -- schwab/ grafana/ telemetry/ comprehensive_backtest.py strategy/ storage/pg_schema.sql .github/workflows/docker-publish.yml uv.lock milestone3_verify.py docker-compose.yml .env.example Diffs introduced by plan 001 (db-backup service, workflow test job) are EXPECTED — see plans/README.md "Expected drift". If any in-scope file changed since this plan was written, compare the "Current state" excerpts against the live code before proceeding; on a mismatch, treat it as a STOP condition.

Status

• Priority: P2
• Effort: S
• Risk: LOW (deletion of verified-dead code; every deletion is grep-gated)
• Depends on: 001 (CI gate should exist so the post-deletion image is test-verified)
• Category: tech-debt
• Planned at: commit d54a2a9, 2026-07-13

Why this matters

Commit d54a2a9's message claims "delete 805MB dead weight" including schwab/ — but git ls-files schwab/ | wc -l returns 23 tracked files at HEAD; the commit deleted core/, models/, controlroom/ but not schwab/. Beyond that, the repo carries: a backtest runner that crashes on import, a grafana stack pointing at the decommissioned pre-migration database with committed plaintext dev credentials, an empty telemetry/ husk, a dead CI workflow that fires on a nonexistent branch, a stale uv.lock from the DuckDB era, and write-only tables collected nightly and read by nothing. Every dead artifact taxes every future session with "which one is real?"; the grafana credentials are a standing security smell.

Current state (verified at HEAD, each with its evidence)

Target	Evidence
schwab/ (23 files) + tests/unit/schwab/ + milestone3_verify.py	`git ls-files schwab/
comprehensive_backtest.py	line 12: from research.connection import ResearchConnection — research/connection.py does not exist; the script crashes on import.
strategy/universe.py, strategy/backtest/outcomes.py	zero references outside their own files (verify with the step-1 grep). NOTE: recon also listed strategy/regime.py — already deleted at HEAD; do not look for it.
ignition detector	partially dead only: detect_momentum_ignition in strategy/evaluation/ignition.py has no consumers, but _last_higher_low from the same file IS imported by strategy/evaluation/runner.py:33, premarket_setup.py:32, and liquid_breakout.py:44 (live via research/labeler). Disposition: delete only detect_momentum_ignition (+ its tests) after moving _last_higher_low into a shared evaluation helper — or leave the whole file and log it. Do NOT git rm ignition.py wholesale.
grafana/	grafana/provisioning/datasources/momentum.yaml hardcodes 192.168.1.5:5432 (the OLD pre-migration instance) with committed plaintext credentials (user/password in the YAML — reference only, do not copy values anywhere). The compose stack has its own optional Grafana; this standalone stack is a pre-compose leftover.
telemetry/	contains only __pycache__ (stale .pyc): ls telemetry/ → __pycache__.
.github/workflows/docker-publish.yml	triggers on main (branch does not exist — only master) and the stale artifact branch claude/kubernetes-image-package-kkdkom (exists only as a dormant origin remote), plus v* tags (none cut). Pushes image name momentum-system referenced only by the parked helm chart.
uv.lock	stale DuckDB-era lockfile contradicting pyproject.toml; Docker installs from requirements.txt (the real manifest).
tests/unit/test_alpaca_client.py	0-byte file squatting the name of the biggest untested money-path module (plan 003 fills it — here, just do not delete it; leave for 003).
Write-only tables	quote_snapshots (fed 24/7 incl. weekends, no retention, no readers), float_cache + dilution_events (written nightly by the scheduler's cache refresh, zero readers). ~12 zero-reference tables total in storage/pg_schema.sql (DuckDB grand-plan residue) — the authoritative list is derived by the step-4 grep procedure, not assumed.
Dead projections	6 zero-consumer functions in storage/projections.py (one with a latent wrong-answer bug per recon) — identify by the step-4 grep procedure.

Commands you will need

Purpose	Command	Expected on success
Tests	DATABASE_URL=<pg url> python -m pytest -q	all pass
Reference check	`grep -rn "" --include=".py" --include=".yml" --include=".sh" --include=".html" .	grep -v `
Import sanity	python -c "import run_live_paper" (needs env; alternatively python -m pytest -q covers it)	exit 0

Scope

In scope:

• Deletions listed above (schwab/, tests/unit/schwab/, milestone3_verify.py, comprehensive_backtest.py, strategy/universe.py, strategy/backtest/outcomes.py, detect_momentum_ignition only per the evidence table, grafana/, telemetry/, .github/workflows/docker-publish.yml, uv.lock)
• docker-compose.yml (grafana dashboards bind-mount path only) + deploy/compose/grafana/dashboards/ (moved provisioning files)
• storage/pg_schema.sql (remove dead CREATE TABLEs), storage/projections.py (remove dead functions)
• README.md (remove schwab/grafana rows from the repo map; fix "~130 tests" → actual count)
• .env.example (remove the SCHWAB_* keys that die with schwab/)
• pyproject.toml (remove deps only schwab/deleted code used — verify by grep)
• A new short plans/002-deletions.log listing every deleted path + the grep that cleared it (executor-written receipt)
• plans/README.md (status row only)

Out of scope:

• DROPping tables in the production database — this plan only removes DDL from pg_schema.sql and code references. Actual DROP TABLE statements go into a "for the operator" section of the deletions log for the owner to run after a backup exists (plan 001) and a one-session soak.
• tests/unit/test_alpaca_client.py (empty file — plan 003 fills it)
• alpaca_paper/ — live money path
• The optional Grafana service inside docker-compose.yml (only the standalone grafana/ directory dies; its dashboards bind-mount is re-pointed in step 2, nothing else in the service changes)

Git workflow

• Branch: advisor/002-dead-weight-deletion
• One commit per target group (schwab, strategy-dead, grafana+telemetry, workflows+lock, schema+projections) so any single group can be reverted.
• Do NOT push or open a PR unless the operator instructed it.

Steps

Step 1: Grep-gate and delete the module-level dead weight

For EACH of: schwab, milestone3_verify, comprehensive_backtest, strategy.universe / strategy/universe, backtest.outcomes / backtest/outcomes, the ignition detector module name:

grep -rn "<name>" --include="*.py" --include="*.yml" --include="*.sh" . | grep -v "<the file itself>" | grep -v "^plans/" | grep -v "^docs/"

If the grep is empty (or only hits tests that test the dead module itself, or README prose), delete the target (and its tests). If it hits LIVE code (anything imported by run_live_paper.py, trading_execution.py, api/, research/ runtime paths), STOP for that target and record it in the deletions log as "kept — live reference at file:line". Record every grep result in plans/002-deletions.log.

Known wrinkle: schwab appears in README.md, .env.example, and possibly momentum_cli.py (milestone3_verify walk-through). CLI subcommands that only invoke deleted modules get removed too — grep momentum_cli.py for the module names.

Verify: DATABASE_URL=<pg url> python -m pytest -q → suite passes with schwab tests gone (count drops from 376; record the new count).

Step 2: Delete grafana/, telemetry/, docker-publish.yml, uv.lock

First, un-break the live bind-mount: docker-compose.yml:178 mounts ./grafana/provisioning/dashboards into the in-compose grafana service this plan keeps. Move that directory to deploy/compose/grafana/dashboards/ (next to the datasources dir already mounted at line 177) and update line 178 to the new path. Then:

git rm -r grafana/ telemetry/ .github/workflows/docker-publish.yml uv.lock

The credentials inside grafana/provisioning/datasources/momentum.yaml were committed — note in the deletions log: "grafana dev credentials were committed to history; owner should rotate that Postgres user's password if the old instance is still running" (do NOT write the credential values anywhere).

Verify: grep -n "\./grafana/" docker-compose.yml → empty. git ls-files grafana/ telemetry/ | wc -l → 0. ls .github/workflows/ → only docker.yml (+ any unrelated workflows present at HEAD). ls deploy/compose/grafana/dashboards/ → the moved provisioning files.

Step 3: Prune README + pyproject

• README repo map: remove the schwab/ row (find with grep -n "^schwab/" README.md, currently line 183); update the tests line (grep -n "~130 tests" README.md, currently line 113 → the count from step 1).
• pyproject.toml: remove dependencies used ONLY by deleted code (grep each candidate dep name across remaining *.py first; keep anything still imported). Do not touch requirements.txt unless a removed dep is listed there AND unused elsewhere (same grep gate).

Verify: DATABASE_URL=<pg url> python -m pytest -q → passes; grep -n "schwab" README.md → no repo-map row (prose mentions in history/docs sections are fine to leave).

Step 4: Dead DDL and dead projections

Procedure (mechanical, grep-driven):

1. List tables: grep -oE "CREATE TABLE IF NOT EXISTS [a-z_]+" storage/pg_schema.sql | awk '{print $NF}' | sort -u
2. For each table name: grep -rn "<table>" --include="*.py" . | grep -v pg_schema — zero hits → dead. Remove its CREATE TABLE (and its indexes) from pg_schema.sql; append DROP TABLE IF EXISTS <table>; to the "operator section" of plans/002-deletions.log.
3. quote_snapshots, float_cache, dilution_events are written but not read. These are explicitly NOT deleted here — plan 006 wires float_cache/dilution_events into the experiment features, and quote_snapshots gets retention there. Note: only dilution_events has DDL in pg_schema.sql (leave it); quote_snapshots and float_cache are created by code (run_live_paper.py, research/ingestion/float_cache.py) — there is nothing to leave in the schema file, just log all three as "write-only, disposition in plan 006".
4. Projections: for each def query_* / public function in storage/projections.py: grep -rn "<fn>(" --include="*.py" --include="*.html" . | grep -v projections.py — zero hits → delete the function. Record each in the log.

Verify: DATABASE_URL=<pg url> python -m pytest -q → passes. python -c "import storage.projections" → exit 0.

Test plan

No new tests — this plan is deletion. The gate is the existing suite staying green after every step, plus the grep receipts in plans/002-deletions.log.

Done criteria

• git ls-files schwab/ | wc -l → 0
• ls grafana/ telemetry/ 2>&1 → both "No such file or directory"
• test -f comprehensive_backtest.py || echo gone → gone (same for strategy/universe.py, strategy/backtest/outcomes.py, milestone3_verify.py, uv.lock, .github/workflows/docker-publish.yml)
• DATABASE_URL=… python -m pytest -q exits 0
• plans/002-deletions.log exists with one line per deletion (grep receipt) + the operator DROP section
• git status clean outside in-scope paths
• plans/README.md status row updated

STOP conditions

• Any step-1 grep shows a LIVE reference to a deletion target — keep that target, log it, continue with the rest.
• The suite fails after a deletion and the failure is not obviously "test of deleted module" — revert that group's commit and report.
• pg_schema.sql table names don't match the grep-derived list shape (schema file drifted) — re-derive, and if >20 tables look dead, stop and report (that's a sign the grep is wrong, not the schema).

Maintenance notes

• The standing rule this plan establishes: every table names its reader or dies — enforceable at review time by requiring any new CREATE TABLE PR to name its consumer.
• The operator DROP section in the deletions log should be run only after plan 001's backup lands and one trading session soaks.
• If the human-vs-bot comparison (the one non-delete rationale for schwab/) is ever revived, it returns as a fresh integration against current Schwab APIs — the deleted code was dormant scaffolding, not a working head start.