Files
momentum-system/deploy/docker
momentum devandClaude Opus 4.8 092d6072e8 Regime gate (plan 011): deploy-wire the shadow pipeline + on-demand admin trigger + prod backtest
The session-level metalabel P(our ORB profits today) — the one lever that
survives out-of-sample. Backtested on 266 prod sessions (walk-forward):

  trade-always (trained subset): -0.111R/session, 36% hit
  meta_v1 GATED (trades best ~32% of days): -0.004R/session, 48% hit
  delta +0.107R in-sample, +0.128R OOS (recent 80 sessions), improves across folds

It does NOT make the bot profitable — it takes a bleeding strategy to roughly
flat by sitting out the worst ~2/3 of sessions. threshold_v1 rule gates too
little to matter; meta_v1 (walk-forward GBM) is the signal. Leaky full-fit
(+0.61R) >> walk-forward confirms the discipline was load-bearing; the OOS
holding is the evidence the +0.078R is real, not look-ahead.

Changes:
- research/regime.py: batch minute-bar load per session (64k per-fire round-trips
  -> ~266 queries; fixed a >5min build to ~3min) + incremental label pass (skip
  already-labeled sessions; --rebuild forces full). Nightly cost now ~one new day.
- docker-compose.yml + truenas-compose.yml: scheduler runs build-features + score
  (both models) before `experiments run`, nightly. ALL-SHADOW — does NOT gate live
  trading (that stays owner-gated Step 5).
- admin_api.py + admin.html: POST /api/regime/run (detached) + GET /api/regime/status
  + a Run/Status button — the panel had no way to trigger this.
- plans/011 + docs/research/regime_notes.md (6-book distillation) + plans 008/009 docs.

Prod state (shadow, rollback = DROP regime_features/regime_scores + DELETE
experiments WHERE id LIKE 'regime%'): 266 feature rows, 532 scores, 2 experiments
registered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 14:05:34 -04:00
..

Running the Momentum stack in Docker Compose

One image, six services: postgres, loop, dashboard (:8765), qabot (Telegram), transcripts, scheduler (20:00 ET weekday housekeeping), plus on-demand jobs and an optional monitoring (Grafana) profile.

docker compose build
docker compose up -d
docker compose ps            # healthchecks: loop heartbeat, dashboard HTTP, pg_isready
docker compose logs -f loop

Secrets live in .env (gitignored, never in the image). The compose file overrides DATABASE_URL to the in-stack postgres so the stack is self-contained.

Migrating the production DB into the stack

The live DB today is native Postgres 18 on 192.168.1.5. To move it into the compose volume:

# 1. dump the live DB (~4 GB as of 2026-07; events retention keeps it bounded)
pg_dump -h 192.168.1.5 -U admin -Fc momentum > momentum.dump

# 2. start ONLY postgres, restore into it
docker compose up -d postgres
docker compose exec -T postgres pg_restore -U momentum -d momentum --no-owner < momentum.dump

# 3. bring up the rest
docker compose up -d

Until then you can run the app services against the LAN DB (transition mode): comment out the DATABASE_URL line in the x-app-env anchor and .env's value (192.168.1.5) flows through.

Where to host it

Recommended: the DB box (192.168.1.5) or any always-on Linux host — NOT the WSL machine. The week of 2026-07-06 produced three trading-hours outages from WSL restarts and networking-mode changes (see memory: wsl-virtioproxy-networking). A dedicated Linux host removes that whole class. WSL + Docker works for development/validation but inherits WSL's fragility.

Cutover checklist (from the systemd stack):

  1. docker compose build && docker compose up -d on the target host
  2. Verify: dashboard on :8765, docker compose ps all healthy, loop log shows ingest/watch/execute cycles, Telegram bot answers
  3. Diff Alpaca open orders — a crash-restart on cutover can resubmit resting entries (memory: momentum-live-loop-architecture)
  4. Disable the WSL systemd units: systemctl --user disable --now momentum-loop momentum-dashboard momentum-qabot momentum-transcripts momentum-premarket-eval.timer
  5. Keep momentum-testpg on the dev machine (tests only, not production)

Notes:

  • TZ inside containers is America/New_York (market-clock logic is ET).
  • data/ (logs, learned_params.json, transcripts) lives in the app-data named volume, shared by all app services.
  • Ollama stays external (192.168.1.9, OLLAMA_* in .env); enrichment fails safe when it's down.
  • Do the cutover OUTSIDE market hours.

Config layering: what goes where

.env.example is the committed, sanitized inventory of every variable (regenerated 2026-07-09). Three layers:

  1. Secrets — keep in .env in a protected space (root-owned chmod 600, or a Docker secret): ALPACA_API_KEY/ALPACA_SECRET_KEY, TELEGRAM_BOT_TOKEN, YOUTUBE_API_KEY, POSTGRES_PASSWORD/DATABASE_URL, Discord webhooks, Schwab keys. The compose file reads them via env_file; they are never baked into the image.
  2. Strategy knobs — fine to promote into the compose environment: block if you want them versioned with the deployment: TRADING_*, TRIGGER_* (gap band, squeeze exception, quality floor, caps), WATCHER_*, LIVE_*. environment: overrides env_file, so promoted values win.
  3. Infra addresses: OLLAMA_HOST (external box, see below), ALPACA_DATA_FEED=sip, dashboard host/port — either layer works.

External Ollama (news enrichment)

Ollama is the ONE external dependency: OLLAMA_HOST=http://192.168.1.9:11434 reaches the LAN box from inside containers via normal outbound NAT — no compose changes needed. If you ever run Ollama on the docker HOST itself, use http://host.docker.internal:11434 and add extra_hosts: ["host.docker.internal:host-gateway"] to the app anchor. Enrichment is fully fail-safe: if Ollama is down, trading is unaffected (news ranking just loses the catalyst-type boost).

NOTE (2026-07-09): the live .env contains two commented-out PowerShell lines embedding the real Alpaca credentials — harmless there, but they nearly leaked into the committed .env.example (caught by the regeneration leak check). Consider deleting them from .env.

TrueNAS gotchas (each one caused a real outage — read before editing)

  1. Anchors get expanded. TrueNAS renders x-app:/<<: *app into inline per-service blocks. After the first save, the x-app block is decorative — env/volume edits there reach nothing. Edit each service's own block.
  2. Verify env landed with docker exec <svc> sh -c 'echo $VAR' — not by reading the YAML, and not via the admin panel (it reports its own container).
  3. Paths are case-sensitive and Docker auto-creates missing bind sources: /mnt/Volume vs /mnt/volume silently forked the database into two worlds. After any data move, verify by size at destination (du -sh), never ls.
  4. Volume changes need container recreation — Stop/Start or Update the app; a container restart keeps the old mounts (and a running postgres keeps writing to deleted files if you move its dir underneath it).
  5. First deploy initializes postgres — POSTGRES_PASSWORD only applies to an empty pgdata. A failed first attempt leaves a cluster with the old password; fix with ALTER USER via docker exec ... psql -U momentum or wipe before the real deploy.
  6. uid 10001: the app containers run non-root; chown -R 10001:10001 the host data dir or every file-writer (scheduler TSVs, admin profiles, transcripts) hits PermissionError.