Post-migration DB health: healthcheck DDL storm + broken-pipe fixes

The loop healthcheck opened a fresh connection every 60s with schema
auto-apply enabled; the CREATE-IF-NOT-EXISTS lock bursts cancelled
minute_bars autovacuum every single cycle (visible as 'canceling
autovacuum task' each minute post-migration), so the 19M-row table
never got analyzed, snapshot queries crawled, browsers timed out, and
the dashboard log filled with double BrokenPipeError tracebacks.

- healthchecks now set PG_SKIP_SCHEMA=1 (both compose files)
- api/main.py _send_json swallows BrokenPipe/ConnectionReset (client
  hung up; nothing to tell it)
- live DB: ALTER SYSTEM max_wal_size=4GB, shared_buffers=1GB,
  effective_cache_size=3GB, checkpoint_completion_target=0.9 (the
  checkpoint-every-24s warning under migration load on NAS storage)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

api/main.py

@@ -293,12 +293,17 @@ class DashboardHandler(BaseHTTPRequestHandler):
 
     def _send_json(self, payload, status: int = 200) -> None:
         body = _dump(payload)
-        self.send_response(status)
-        self.send_header("Content-Type", "application/json")
-        self.send_header("Content-Length", str(len(body)))
-        self.send_header("Access-Control-Allow-Origin", "*")
-        self.end_headers()
-        self.wfile.write(body)
+        try:
+            self.send_response(status)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.send_header("Access-Control-Allow-Origin", "*")
+            self.end_headers()
+            self.wfile.write(body)
+        except (BrokenPipeError, ConnectionResetError):
+            # the client hung up while we were rendering (slow query + browser
+            # timeout) — nothing to send it; don't double-traceback the log
+            pass
 
     def _send_html(self, path: Path) -> None:
         if not path.exists():

deploy/docker/truenas-compose.yml

@@ -78,7 +78,10 @@ services:
     <<: *app
     command: ["python", "-u", "run_live_paper.py", "--no-dashboard"]
     healthcheck:
-      test: ["CMD", "python", "-c", "import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)"]
+      # PG_SKIP_SCHEMA=1: a healthcheck must NOT re-apply schema DDL — the 60s
+      # CREATE-IF-NOT-EXISTS lock burst cancelled minute_bars autovacuum forever
+      # (2026-07-09 post-migration: stats rotted, dashboard queries crawled)
+      test: ["CMD", "python", "-c", "import os; os.environ['PG_SKIP_SCHEMA']='1'; import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)"]
       interval: 60s
       timeout: 20s
       retries: 5

docker-compose.yml

@@ -62,7 +62,10 @@ services:
     healthcheck:
       # heartbeat: the loop must have written SOME event in the last 5 minutes
       # (module ticks flow around the clock, not just market hours)
-      test: ["CMD", "python", "-c", "import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)"]
+      # PG_SKIP_SCHEMA=1: a healthcheck must NOT re-apply schema DDL — the 60s
+      # CREATE-IF-NOT-EXISTS lock burst cancelled minute_bars autovacuum forever
+      # (2026-07-09 post-migration: stats rotted, dashboard queries crawled)
+      test: ["CMD", "python", "-c", "import os; os.environ['PG_SKIP_SCHEMA']='1'; import datetime,sys; from research.multi_schema import open_research_db; r=open_research_db('market').execute('SELECT max(timestamp) FROM events').fetchall()[0][0]; sys.exit(0 if r and (datetime.datetime.now()-r).total_seconds()<300 else 1)"]
       interval: 60s
       timeout: 20s
       retries: 5