perf: stop the watcher scoring names that can never trade
criteria_evaluated + signal_blocked held 3.1 GB of a 7.6 GB events table.
On 2026-09-10 alone the watcher wrote 191,458 criteria receipts across 57
symbols -- ~3,359 per symbol per day -- and 154,260 of them (80.6%) were
for names whose static open gap was outside the tradeable band. NOK, NIO,
PATH, PLUG, SOFI, AAL and F all sit inside the $2-20 price filter, so
WATCHER_PRICE_MIN/MAX admits them; only the gap floor excludes them, and
that was checked per-tick instead of once up front.
Apply the SHARED population gate (strategy/population.population_reject --
the same function, same config the entry path already enforces) before the
evaluation and its receipts. This is behaviour-preserving for trading: those
names were being scored, emitted, and then rejected downstream anyway --
429 "outside tradeable population" risk events across 221 symbols, led by
exactly the names that top the receipt counts. The decision just moves
upstream of the write.
Details that matter:
- The verdict is recomputed each tick, never cached. A >150% gapper reads
"squeeze unverified" on thin volume and is admitted once rvol confirms;
a cached rejection would bench it for the session.
- _transition only emits on CHANGE, so a rejected symbol costs one
state event per session and is then silent.
- Fail-open throughout: an unreadable open, a missing previous close or
absent volume returns None, so a data gap can never bench a symbol.
- Gap is the STATIC OPEN gap (session open vs previous close), fixed for
the day, not the live price -- the verdict must not wander intraday.
Off by default (population=None) so every existing caller and test keeps
its current behaviour; the live loop opts in, and WATCHER_POPULATION_GATE=0
restores the old path.
Note for diagnostics: momentum_cli's criteria views now show only names
inside the population. That is the intended trade -- the excluded rows
described symbols the bot could never have taken.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2LepdJfPCfK5Q8uznjFBM
This commit is contained in:
1 parent
9034839661
commit
97a3a1b29d
4 files changed
+236
-3
No files matched your search
@@ -29,6 +29,7 @@ from storage.event_schema import (
|
||||
)
|
||||
from storage.event_store import EventStore
|
||||
from strategy.evaluation.setup_evaluator import evaluate_setup
|
||||
from strategy.population import PopulationConfig, population_reject
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -60,6 +61,15 @@ class WatcherConfig:
|
||||
min_bars: int = 10
|
||||
ready_score_pct: float = 60.0
|
||||
min_quality: float = 0.30
|
||||
# Shared tradeable-population gate, applied BEFORE the per-tick evaluation.
|
||||
# None = off (the historical behaviour every existing caller and test relies
|
||||
# on). The live loop passes PopulationConfig.from_env() so the watcher stops
|
||||
# scoring names the entry path can never accept: on 2026-09-10 the watcher
|
||||
# wrote 191,458 criteria receipts across 57 symbols and 154,260 of them
|
||||
# (80.6%) were for gaps outside the tradeable band — large caps like NOK, F
|
||||
# and SOFI that sit inside the $2-20 price filter but can never pass the gap
|
||||
# floor. Those two event types were 3.1 GB of a 7.6 GB events table.
|
||||
population: "PopulationConfig | None" = None
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -69,6 +79,7 @@ class WatcherTickResult:
|
||||
ready: list[str] = field(default_factory=list)
|
||||
blocked: list[str] = field(default_factory=list)
|
||||
discovered: list[str] = field(default_factory=list)
|
||||
skipped: int = 0 # rejected by the population gate before evaluation
|
||||
errors: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
@@ -122,6 +133,39 @@ class Watcher:
|
||||
)
|
||||
)
|
||||
|
||||
def _population_reject(self, candidate: WatchCandidate, bars) -> str | None:
|
||||
"""Reject reason if this name is outside the tradeable population, else None.
|
||||
|
||||
Uses the STATIC OPEN gap (session open vs previous close) — the same
|
||||
open-knowable quantity the live entry gate uses, and fixed for the day —
|
||||
never the live price, which would make the verdict wander intraday.
|
||||
Fail-open throughout: anything unreadable returns None so a data gap can
|
||||
never silently bench a symbol.
|
||||
"""
|
||||
try:
|
||||
open_px = float(bars["open"].iloc[0])
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
prev_close = candidate.previous_close
|
||||
gap = None
|
||||
if prev_close:
|
||||
try:
|
||||
gap = (open_px - float(prev_close)) / float(prev_close)
|
||||
except (TypeError, ValueError, ZeroDivisionError):
|
||||
gap = None
|
||||
volume = rvol = None
|
||||
try:
|
||||
volume = float(bars["volume"].sum())
|
||||
adv = candidate.avg_daily_volume
|
||||
if adv and float(adv) > 0:
|
||||
rvol = volume / float(adv)
|
||||
except Exception: # noqa: BLE001
|
||||
volume = rvol = None
|
||||
return population_reject(
|
||||
self.config.population,
|
||||
gap_pct=gap, price=open_px, volume=volume, rvol=rvol, as_of="open",
|
||||
)
|
||||
|
||||
def _discover(self, candidate: WatchCandidate) -> None:
|
||||
if candidate.symbol in self._states:
|
||||
return
|
||||
@@ -172,6 +216,20 @@ class Watcher:
|
||||
)
|
||||
continue
|
||||
|
||||
# POPULATION PRE-FILTER — skip the expensive evaluation (and its
|
||||
# two fat receipts) for a name the entry path could never accept.
|
||||
# Recomputed every tick rather than cached: a >150% gapper starts
|
||||
# "squeeze unverified" on thin volume and is admitted later once
|
||||
# rvol confirms, so a cached rejection would bench it for the
|
||||
# session. _transition only emits on CHANGE, so a rejected symbol
|
||||
# costs exactly one event and then goes silent.
|
||||
if self.config.population is not None:
|
||||
reason = self._population_reject(candidate, bars)
|
||||
if reason is not None:
|
||||
self._transition(symbol, "out_of_population", reason)
|
||||
result.skipped += 1
|
||||
continue
|
||||
|
||||
# Evaluate the setup as of the latest bar's timestamp, not the
|
||||
# wall clock. Minute bars are stored UTC-naive, but the entry
|
||||
# cutoff reasons in US/Eastern wall-clock, so convert. This
|
||||
|
||||
Reference in new issue
Block a user