perf: stop the watcher scoring names that can never trade

criteria_evaluated + signal_blocked held 3.1 GB of a 7.6 GB events table.
On 2026-09-10 alone the watcher wrote 191,458 criteria receipts across 57
symbols -- ~3,359 per symbol per day -- and 154,260 of them (80.6%) were
for names whose static open gap was outside the tradeable band. NOK, NIO,
PATH, PLUG, SOFI, AAL and F all sit inside the $2-20 price filter, so
WATCHER_PRICE_MIN/MAX admits them; only the gap floor excludes them, and
that was checked per-tick instead of once up front.

Apply the SHARED population gate (strategy/population.population_reject --
the same function, same config the entry path already enforces) before the
evaluation and its receipts. This is behaviour-preserving for trading: those
names were being scored, emitted, and then rejected downstream anyway --
429 "outside tradeable population" risk events across 221 symbols, led by
exactly the names that top the receipt counts. The decision just moves
upstream of the write.

Details that matter:

  - The verdict is recomputed each tick, never cached. A >150% gapper reads
    "squeeze unverified" on thin volume and is admitted once rvol confirms;
    a cached rejection would bench it for the session.
  - _transition only emits on CHANGE, so a rejected symbol costs one
    state event per session and is then silent.
  - Fail-open throughout: an unreadable open, a missing previous close or
    absent volume returns None, so a data gap can never bench a symbol.
  - Gap is the STATIC OPEN gap (session open vs previous close), fixed for
    the day, not the live price -- the verdict must not wander intraday.

Off by default (population=None) so every existing caller and test keeps
its current behaviour; the live loop opts in, and WATCHER_POPULATION_GATE=0
restores the old path.

Note for diagnostics: momentum_cli's criteria views now show only names
inside the population. That is the intended trade -- the excluded rows
described symbols the bot could never have taken.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
This commit is contained in:
momentum devandClaude Opus 5 committed 2026-09-13 14:27:06 -05:00
1 parent 9034839661
commit 97a3a1b29d
4 files changed
+236 -3

No files matched your search

+58
View File
@@ -29,6 +29,7 @@ from storage.event_schema import (
)
from storage.event_store import EventStore
from strategy.evaluation.setup_evaluator import evaluate_setup
from strategy.population import PopulationConfig, population_reject
logger = logging.getLogger(__name__)
@@ -60,6 +61,15 @@ class WatcherConfig:
min_bars: int = 10
ready_score_pct: float = 60.0
min_quality: float = 0.30
# Shared tradeable-population gate, applied BEFORE the per-tick evaluation.
# None = off (the historical behaviour every existing caller and test relies
# on). The live loop passes PopulationConfig.from_env() so the watcher stops
# scoring names the entry path can never accept: on 2026-09-10 the watcher
# wrote 191,458 criteria receipts across 57 symbols and 154,260 of them
# (80.6%) were for gaps outside the tradeable band — large caps like NOK, F
# and SOFI that sit inside the $2-20 price filter but can never pass the gap
# floor. Those two event types were 3.1 GB of a 7.6 GB events table.
population: "PopulationConfig | None" = None
@dataclass
@@ -69,6 +79,7 @@ class WatcherTickResult:
ready: list[str] = field(default_factory=list)
blocked: list[str] = field(default_factory=list)
discovered: list[str] = field(default_factory=list)
skipped: int = 0 # rejected by the population gate before evaluation
errors: list[str] = field(default_factory=list)
@@ -122,6 +133,39 @@ class Watcher:
)
)
def _population_reject(self, candidate: WatchCandidate, bars) -> str | None:
"""Reject reason if this name is outside the tradeable population, else None.
Uses the STATIC OPEN gap (session open vs previous close) — the same
open-knowable quantity the live entry gate uses, and fixed for the day —
never the live price, which would make the verdict wander intraday.
Fail-open throughout: anything unreadable returns None so a data gap can
never silently bench a symbol.
"""
try:
open_px = float(bars["open"].iloc[0])
except Exception: # noqa: BLE001
return None
prev_close = candidate.previous_close
gap = None
if prev_close:
try:
gap = (open_px - float(prev_close)) / float(prev_close)
except (TypeError, ValueError, ZeroDivisionError):
gap = None
volume = rvol = None
try:
volume = float(bars["volume"].sum())
adv = candidate.avg_daily_volume
if adv and float(adv) > 0:
rvol = volume / float(adv)
except Exception: # noqa: BLE001
volume = rvol = None
return population_reject(
self.config.population,
gap_pct=gap, price=open_px, volume=volume, rvol=rvol, as_of="open",
)
def _discover(self, candidate: WatchCandidate) -> None:
if candidate.symbol in self._states:
return
@@ -172,6 +216,20 @@ class Watcher:
)
continue
# POPULATION PRE-FILTER — skip the expensive evaluation (and its
# two fat receipts) for a name the entry path could never accept.
# Recomputed every tick rather than cached: a >150% gapper starts
# "squeeze unverified" on thin volume and is admitted later once
# rvol confirms, so a cached rejection would bench it for the
# session. _transition only emits on CHANGE, so a rejected symbol
# costs exactly one event and then goes silent.
if self.config.population is not None:
reason = self._population_reject(candidate, bars)
if reason is not None:
self._transition(symbol, "out_of_population", reason)
result.skipped += 1
continue
# Evaluate the setup as of the latest bar's timestamp, not the
# wall clock. Minute bars are stored UTC-naive, but the entry
# cutoff reasons in US/Eastern wall-clock, so convert. This