momentum devandClaude Opus 4.8
5ac2cc7b89
- 10 era documents (phase plans, build summaries, redesign writeups, old runbooks, completed consolidation plan) moved to docs/history/ with a do-not-follow README. - Root README: current validated population ($2-20, 10-35% static gap + squeeze exception, SIP), Deployment section, automated-maintenance reality, corrected repo map (admin panel, youtube_claims, deploy/). - RUNBOOK rewritten for the running system: daily rhythm, health checks per deployment, and the production failure-mode table earned this week (env drift, pgdata mount traps, uid 10001, zombie approvals, PG_SKIP_ SCHEMA, timezone artifacts). - deploy/README.md is now the three-mode index: compose (canonical), systemd (local dev — unit templates now committed), helm (refresh staged as issue #8). Old k8s doc moved to deploy/helm/README.md. - docker/README.md: six TrueNAS gotchas, each from a real outage. - Admin panel improvements staged as GitHub issues #4-#7. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
118 lines
5.6 KiB
Markdown
118 lines
5.6 KiB
Markdown
# Running the Momentum stack in Docker Compose
|
|
|
|
One image, six services: `postgres`, `loop`, `dashboard` (:8765), `qabot`
|
|
(Telegram), `transcripts`, `scheduler` (20:00 ET weekday housekeeping), plus
|
|
on-demand `jobs` and an optional `monitoring` (Grafana) profile.
|
|
|
|
```bash
|
|
docker compose build
|
|
docker compose up -d
|
|
docker compose ps # healthchecks: loop heartbeat, dashboard HTTP, pg_isready
|
|
docker compose logs -f loop
|
|
```
|
|
|
|
Secrets live in `.env` (gitignored, never in the image). The compose file
|
|
overrides `DATABASE_URL` to the in-stack postgres so the stack is
|
|
self-contained.
|
|
|
|
## Migrating the production DB into the stack
|
|
|
|
The live DB today is native Postgres 18 on 192.168.1.5. To move it into the
|
|
compose volume:
|
|
|
|
```bash
|
|
# 1. dump the live DB (~4 GB as of 2026-07; events retention keeps it bounded)
|
|
pg_dump -h 192.168.1.5 -U admin -Fc momentum > momentum.dump
|
|
|
|
# 2. start ONLY postgres, restore into it
|
|
docker compose up -d postgres
|
|
docker compose exec -T postgres pg_restore -U momentum -d momentum --no-owner < momentum.dump
|
|
|
|
# 3. bring up the rest
|
|
docker compose up -d
|
|
```
|
|
|
|
Until then you can run the app services against the LAN DB (transition mode):
|
|
comment out the `DATABASE_URL` line in the `x-app-env` anchor and `.env`'s
|
|
value (192.168.1.5) flows through.
|
|
|
|
## Where to host it
|
|
|
|
Recommended: **the DB box (192.168.1.5) or any always-on Linux host** — NOT
|
|
the WSL machine. The week of 2026-07-06 produced three trading-hours outages
|
|
from WSL restarts and networking-mode changes (see memory:
|
|
wsl-virtioproxy-networking). A dedicated Linux host removes that whole class.
|
|
WSL + Docker works for development/validation but inherits WSL's fragility.
|
|
|
|
Cutover checklist (from the systemd stack):
|
|
1. `docker compose build && docker compose up -d` on the target host
|
|
2. Verify: dashboard on :8765, `docker compose ps` all healthy, loop log shows
|
|
ingest/watch/execute cycles, Telegram bot answers
|
|
3. Diff Alpaca open orders — a crash-restart on cutover can resubmit resting
|
|
entries (memory: momentum-live-loop-architecture)
|
|
4. Disable the WSL systemd units: `systemctl --user disable --now
|
|
momentum-loop momentum-dashboard momentum-qabot momentum-transcripts
|
|
momentum-premarket-eval.timer`
|
|
5. Keep `momentum-testpg` on the dev machine (tests only, not production)
|
|
|
|
Notes:
|
|
- TZ inside containers is America/New_York (market-clock logic is ET).
|
|
- `data/` (logs, learned_params.json, transcripts) lives in the `app-data`
|
|
named volume, shared by all app services.
|
|
- Ollama stays external (192.168.1.9, `OLLAMA_*` in .env); enrichment
|
|
fails safe when it's down.
|
|
- Do the cutover OUTSIDE market hours.
|
|
|
|
## Config layering: what goes where
|
|
|
|
`.env.example` is the committed, sanitized inventory of every variable
|
|
(regenerated 2026-07-09). Three layers:
|
|
|
|
1. **Secrets — keep in `.env` in a protected space** (root-owned `chmod 600`,
|
|
or a Docker secret): `ALPACA_API_KEY`/`ALPACA_SECRET_KEY`,
|
|
`TELEGRAM_BOT_TOKEN`, `YOUTUBE_API_KEY`, `POSTGRES_PASSWORD`/`DATABASE_URL`,
|
|
Discord webhooks, Schwab keys. The compose file reads them via `env_file`;
|
|
they are never baked into the image.
|
|
2. **Strategy knobs — fine to promote into the compose `environment:` block**
|
|
if you want them versioned with the deployment: `TRADING_*`, `TRIGGER_*`
|
|
(gap band, squeeze exception, quality floor, caps), `WATCHER_*`, `LIVE_*`.
|
|
`environment:` overrides `env_file`, so promoted values win.
|
|
3. **Infra addresses**: `OLLAMA_HOST` (external box, see below),
|
|
`ALPACA_DATA_FEED=sip`, dashboard host/port — either layer works.
|
|
|
|
## External Ollama (news enrichment)
|
|
|
|
Ollama is the ONE external dependency: `OLLAMA_HOST=http://192.168.1.9:11434`
|
|
reaches the LAN box from inside containers via normal outbound NAT — no
|
|
compose changes needed. If you ever run Ollama on the docker HOST itself, use
|
|
`http://host.docker.internal:11434` and add
|
|
`extra_hosts: ["host.docker.internal:host-gateway"]` to the app anchor.
|
|
Enrichment is fully fail-safe: if Ollama is down, trading is unaffected
|
|
(news ranking just loses the catalyst-type boost).
|
|
|
|
NOTE (2026-07-09): the live `.env` contains two commented-out PowerShell lines
|
|
embedding the real Alpaca credentials — harmless there, but they nearly leaked
|
|
into the committed `.env.example` (caught by the regeneration leak check).
|
|
Consider deleting them from `.env`.
|
|
|
|
## TrueNAS gotchas (each one caused a real outage — read before editing)
|
|
|
|
1. **Anchors get expanded.** TrueNAS renders `x-app:`/`<<: *app` into inline
|
|
per-service blocks. After the first save, the `x-app` block is decorative —
|
|
env/volume edits there reach nothing. Edit each service's own block.
|
|
2. **Verify env landed** with `docker exec <svc> sh -c 'echo $VAR'` — not by
|
|
reading the YAML, and not via the admin panel (it reports its own container).
|
|
3. **Paths are case-sensitive** and Docker auto-creates missing bind sources:
|
|
`/mnt/Volume` vs `/mnt/volume` silently forked the database into two worlds.
|
|
After any data move, verify by **size at destination** (`du -sh`), never `ls`.
|
|
4. **Volume changes need container recreation** — Stop/Start or Update the app;
|
|
a container restart keeps the old mounts (and a running postgres keeps
|
|
writing to *deleted* files if you move its dir underneath it).
|
|
5. **First deploy initializes postgres** — `POSTGRES_PASSWORD` only applies to
|
|
an empty pgdata. A failed first attempt leaves a cluster with the old
|
|
password; fix with `ALTER USER` via `docker exec ... psql -U momentum` or
|
|
wipe before the real deploy.
|
|
6. **uid 10001**: the app containers run non-root; `chown -R 10001:10001` the
|
|
host data dir or every file-writer (scheduler TSVs, admin profiles,
|
|
transcripts) hits `PermissionError`.
|