Commit Graph
48 Commits
Author SHA1 Message Date
KoalaTrapLord b6222cbc7c fix(#176): capture life-at-timeout for draw games + add DrawGameAnalyzer
- forge_runner._run_process_blocking: parse [DRAW] PARSE-SUMMARY lines
  and record life totals per seat into state.draw_game_snapshots
- routes/lab.py get_result(): attach draw_game_snapshots to result JSON
  under 'drawGames' so the dashboard and analysis scripts can query them
- New services/draw_game_analyzer.py: report which deck held the highest
  average life at timeout; used by /api/lab/result and scripts/
- Also lowers the default batch --clock from 6000 → 3000 per game
  (30s per-game wall limit for batch mode, down from 60s).
  Interactive games can override via req.clock which defaults stay at 6000.

Closes #176
2026-04-01 14:38:53 -04:00
KoalaTrapLord 5411fd6bf6 fix(#175): alternate win condition parser & winType tagging (#179)
Adds win_condition_parser.py, GameResult.win_type field, and winType stamping on both the Java/Forge and DeepSeek batch paths. Closes #175.
2026-04-01 14:28:10 -04:00
KoalaTrapLord 5b5bac42fb fix: use vars(row) instead of dict(row) in _build_collection_from_bulk
dict() on a types.SimpleNamespace raises TypeError: object is not iterable.
vars() correctly extracts the __dict__ attribute map. Fixes
TestBuildCollectionFailureTracking::test_high_failure_rate_raises_runtime_error.
2026-03-30 10:39:49 -04:00
KoalaTrapLord 882e2370c1 fix(#127): persist built_at in ChromaDB collection metadata as sidecar fallback
- _write_build_meta: after writing build_meta.json, also stamps built_at
  onto the collection via col.modify() (best-effort, swallows exceptions)
- _collection_is_current: if sidecar built_at is missing/zero, falls back
  to collection.metadata["built_at"] so a deleted sidecar doesn't force
  an unnecessary full rebuild when the ChromaDB data is still valid
2026-03-29 04:29:35 -04:00
KoalaTrapLord bbca0b7985 fix(#126): add batch insert retry logic with exponential backoff
- Add BATCH_RETRY_LIMIT and BATCH_RETRY_DELAY constants after DEFAULT_BATCH_SIZE
- Add _insert_batch_with_retry() helper with exponential backoff
- Replace both bare try/except insert blocks in _build_collection_from_bulk
  with calls to the retry helper so transient ChromaDB errors are retried
  up to BATCH_RETRY_LIMIT times before a batch is counted as failed
2026-03-29 04:24:58 -04:00
KoalaTrapLord aefd0cace6 feat(rag): Phase 1 — rewrite build_card_text to natural-language format (#137)
* feat(rag): Phase 1 — rewrite build_card_text to natural-language format

* feat(rag): Phase 2 — Forge-enriched ChromaDB RAG store (2a–2e)

Phase 2a: Forge lazy import guard (_try_import_forge_lookup)
Phase 2b: _build_document appends Forge trigger/replacement/static/keyword tags
Phase 2c: _build_metadata adds has_trigger, has_replacement, has_static, trigger_modes fields
Phase 2d: query_cards extended with require_trigger, require_replacement, require_static params + ChromaDB where clause support
Phase 2e: get_rag_stats reports forge_enriched and forge_coverage; _write_build_meta tracks coverage; build loop counts enriched cards

* feat(rag): Phase 3 — add mtgrules second ChromaDB collection

Adds build_rules_index() and query_rules() operating on a separate
'mtgrules' ChromaDB collection. Chunks MagicCompRules.txt by rule-number
boundaries, embeds each chunk, and exposes semantic search. Completely
independent from the mtg_oracle card collection.

* feat(coach): Phase 4 — behavior-aware RAG + rules context in coach_service

Phase 4a: run_coaching_session uses dual RAG query
  - Primary: commander synergy (n=10)
  - Secondary: require_replacement=True for graveyard/protection cards (n=5)
  - Deduplicates by name before merging
Phase 4b: query_rules() fetches 3 Commander rules chunks to ground the prompt
  - Passed as rules_context kwarg to build_user_prompt()

* feat(rag): Phase 5 — add /api/rag/build-rules and /api/rag/query-rules endpoints
2026-03-29 02:11:38 -04:00
KoalaTrapLord 1c491bc2b0 fix: add Land role detection to _detect_card_roles (fixes basics showing as Unknown) 2026-03-27 20:28:35 -04:00
KoalaTrapLord 0010b0f394 refactor: RAG improvements — shared card_text, Chroma fallback, staleness monitor (fixes #133, #134, #135) (#136)
* refactor: add shared build_card_text() utility

* refactor: unify _build_document via shared card_text + add is_chroma_available/check_staleness (fixes #133, #134, #135)

* refactor: delegate embeddings._build_card_text to shared utility

* feat: add background RAG staleness monitor to lab_api lifespan (fixes #135)
2026-03-27 16:46:33 -04:00
KoalaTrapLord bcf558418a fix(rag): squash all RAG/ChromaDB bugs #123–#130
Fixes critical API contract mismatch, batch_size wiring, lifespan init, failure tracking, build_meta persistence, mana_cost in embeddings, pyproject.toml deps, and adds full test suite.

Closes #123, #124, #125, #126, #127, #128, #129, #130
2026-03-27 13:24:41 -04:00
KoalaTrapLord 4b0cd34eda feat(rag): Phase 2 — ChromaDB vector index over bulk card DB (#120)
* feat(rag): Phase 2 — ChromaDB vector index over bulk card DB

* feat(rag): add routes/rag.py with build, search, status endpoints

* Update lab_api.pyfeat(rag): register rag_router in lab_api.py, remove old inline rag_status

* Update requirements.txtfeat(rag): add chromadb>=0.5.0 to requirements.txtfeat(rag): add chromadb to requirements.txt
2026-03-27 08:39:02 -04:00
KoalaTrapLord 8df5900ff9 feat(rag): Phase 1 — Scryfall bulk offline card database (SQLite FTS5) (#117)
* feat(rag): add scryfall_bulk.py — offline SQLite FTS5 card database (RAG Phase 1)

* feat(rag): wire ensure_bulk_db() into startup and add /api/rag/status endpoint (RAG Phase 1)

* fix: migrate on_event to lifespan handler, fix /api/rag/status route ordering
2026-03-27 07:16:55 -04:00
KoalaTrapLord 6c03e56794 fix(#116): dual card_records fallback JOIN, card_type/card_category fields, simulator land detection
- deck_service._compute_deck_analysis: add LEFT JOIN card_records with COALESCE on all fields
- deck_service: add _classify_card_category() with oracle-text keyword matching
- deckbuilder.GET /api/decks/{id}/cards: add card_records fallback JOIN + COALESCE, emit card_type/card_category per card
- deepseek._fetch_deck_card_data: add card_records fallback JOIN so lands get correct type_line before enrich_card()

Closes #116
2026-03-27 04:10:17 -04:00
KoalaTrapLord af2a71d1a5 fix: eliminate event-loop-blocking sync calls in async FastAPI routes (#106)
* feat: add async DB wrapper and eliminate event-loop blocking

* fix: rewrite llm_client.py with native async httpx - removes urlopen and run_in_executor

* fix: rewrite routes/coach.py - replace all sync DB/HTTP with async (aiosqlite + httpx)

* fix: add async DB shutdown hook to lab_api.py

* feat: add aiosqlite>=0.19.0 to requirements
2026-03-25 19:17:41 -04:00
KoalaTrapLord e7ffccf3e8 feat: persist coach sessions to SQLite with disk fallback 2026-03-24 23:44:20 -04:00
KoalaTrapLord 015b9da1e9 rename: update forge_runner.py file prefix from batch- to ml-decision- 2026-03-23 23:06:33 -04:00
87404f3702 refactor: move logging out of routes/shared.py into services/logging.py (#99)
Create services/logging.py as the canonical home for setup_logging(),
all log format constants, named loggers, and _ml_logging_enabled.

Update all route modules and lab_api.py to import logging symbols from
services.logging and all other symbols from their canonical sources
(models.state, models.requests, models.responses, services.database,
services.deck_service, services.scryfall, services.import_helpers,
services.import_service, services.precon_service, services.forge_runner).

Delete routes/shared.py — the re-export hub is no longer needed.

Closes #61

Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 21:04:25 -04:00
KoalaTrapLord b37968e3c2 fix: always pass --ml-log flag to Forge Java batch runner 2026-03-23 17:22:58 -04:00
KoalaTrapLord 49fc817187 fix(#92): bump BRAIN_TIMEOUT fallback from 120s to 300s in forge_runner.py 2026-03-22 23:40:28 -04:00
ClaudeandClaude Opus 4.6 978fb12969 fix: early return in _load_deck_cards_by_name when deck not found in DB
Fixes UnboundLocalError crash when deck_row is None by returning an
empty list instead of falling through to a query with an unbound deck_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 07:25:39 +00:00
KoalaTrapLord 11978736a1 fix: raise BRAIN_TIMEOUT default from 30s to 120s (#71) 2026-03-21 19:39:29 -04:00
KoalaTrapLord 06f007e605 fix: invert guard in _load_deck_cards_by_name — was if not deck_row (crash on None[0])
The guard condition was inverted: `if not deck_row:` entered the query
block when no deck was found, causing `deck_row[0]` to crash with
'NoneType' object is not subscriptable. Flipped to `if deck_row:`
so the query runs only when the deck exists.
2026-03-21 15:11:08 -04:00
KoalaTrapLord a7ab0e43a5 feat: Phase 6 — make primary brain env-configurable (BRAIN_* vars)
Replace hardcoded DEEPSEEK_API_BASE/DEEPSEEK_MODEL env vars in
_get_deepseek_brain() with new BRAIN_* environment variables:
- BRAIN_API_BASE (default: http://localhost:11434)
- BRAIN_MODEL (default: gpt-oss:20b)
- BRAIN_TEMPERATURE (default: 0.3)
- BRAIN_MAX_TOKENS (default: 1024)
- BRAIN_TIMEOUT (default: 30.0)

All params now passed directly to DeepSeekConfig constructor.
2026-03-21 13:21:58 -04:00
KoalaTrapLord d86109c5c0 Fix duplicate WHERE clause in _load_deck_cards_by_name SQL query 2026-03-19 17:54:34 -04:00
KoalaTrapLord d9dd098ca3 fix: add power, toughness, keywords to _load_deck_cards_by_name SQL query 2026-03-19 17:43:57 -04:00
KoalaTrapLord b00df14f30 fix: use card_name key from _load_deck_cards_by_name (KeyError: 'name') 2026-03-19 16:51:27 -04:00
KoalaTrapLord 3542efd982 fix: align build_java_command with LabCli picocli args (--forge-jar, --forge-dir, --deck1..4) 2026-03-19 16:42:03 -04:00
KoalaTrapLord 0e5d7ba7d3 fix: harden _find_java17 to search multiple vendors, JAVA_HOME, and warn on wrong version 2026-03-19 15:23:44 -04:00
KoalaTrapLord b4d8c2f50e Fix indentation error on line 47 of forge_runner.py 2026-03-19 06:54:16 -04:00
KoalaTrapLord 9db1ed112f Increase _STALL_LIMIT from 300s to 600s (10 min) to prevent premature batch kills 2026-03-19 06:30:03 -04:00
KoalaTrapLord d834247885 Fix forge_runner.py: restore full file, fix sim mode arg
- Restored _run_deepseek_batch_thread and _get_deepseek_brain
  (accidentally removed in prior commit)
- Changed '-sim' to 'sim' — Forge expects positional mode, not flag
2026-03-19 10:13:11 +00:00
rgriffith 310a408509 feat: polish precon picker — card art, color pips, grid layout
- Add full CSS for precon grid: responsive card grid, hover effects,
  install button states, color pips, dark theme styling
- Build commander art_crop image URLs from set_code/number during
  precon download (Scryfall direct image redirect, no API call needed)
- Batch-fetch color_identity from Scryfall /cards/collection endpoint
  (2 requests for ~145 unique commanders) and store in precon index
- Widen precon modal to 900px for grid layout
- Auto-invalidate precon cache when enrichment fields are missing
- JS renderer now shows commander art crop + card body layout
- Bump cache-busting version to v=4
2026-03-19 10:10:40 +00:00
KoalaTrapLord ef7bde841f Fix forge_runner.py: sim mode arg + indentation corruption
- Changed '-sim' to 'sim' (Forge expects positional mode, not flag)
- Fixed indentation: build_java_command, _run_process_blocking,
  run_batch_subprocess were incorrectly nested inside get_java17()
- Fixed last_activity assignment stuck inside except block
- Added docstrings
2026-03-19 10:09:59 +00:00
rgriffith 8eba39cca7 fix: precon picker empty — mutate PRECON_INDEX in-place
The PRECON_INDEX list was imported by value in routes/shared.py.
When download_precon_database() or load_precon_index() reassigned it
with 'PRECON_INDEX = ...', the new list object was only visible in
precon_service module scope — shared.PRECON_INDEX (and thus the
/api/lab/precons endpoint) still pointed to the original empty [].

Fix: use PRECON_INDEX.clear() + .extend() instead of reassignment
so all modules sharing the same list reference see the updated data.
2026-03-19 10:01:51 +00:00
Computer c6c3bfabf8 refactor: replace per-batch watchdog threads with single global watchdog 2026-03-19 06:56:56 +00:00
KoalaTrapLord 83a8f288b7 Fix scrambled deck_service.py: close SQL query in _load_deck_cards_by_name, remove orphaned lines 2026-03-19 06:24:13 +00:00
KoalaTrapLord e3f0787c73 Optimize deckgen: push colour-identity filtering into SQL with json_each
- Replace SELECT * + Python-side color loop in _get_collection_for_colors()
  with projected SELECT + NOT EXISTS json_each() sub-query
- Apply same pattern to _build_collection_summary(), removing card_fits_identity()
- Add partial index idx_ce_quantity on collection_entries(quantity) WHERE qty > 0
- Handle NULL and empty-string color_identity via CASE guard
- Fix colorless commander bug: old code skipped color check entirely for []
- Add scripts/bench_deckgen.py benchmark (3-5x speedup on filtered queries)
- Add tests/test_deckgen_color_filter.py with 7 regression tests
2026-03-19 06:18:32 +00:00
KoalaTrapLord 40a166188a fix: unify DB connection management — init_collection_db() uses _get_db_conn()
Previously, init_collection_db() opened a separate sqlite3.connect() call,
ran migrations, then closed that connection — entirely outside the
thread-local pool that route handlers use via _get_db_conn().

This meant:
- The migration connection and the request-serving connection were
  independent objects with no ordering guarantee under concurrent startup.
- The warm-up connection was thrown away immediately, so every handler
  thread still had to open its own connection on first use.

Fix:
- init_collection_db() now calls _get_db_conn() directly, so schema
  creation and migrations run on the same thread-local connection that
  will serve subsequent requests.
- Added _close_db_conn() helper for explicit teardown on thread exit
  (e.g. FastAPI lifespan shutdown or worker thread cleanup).
- Removed the redundant standalone conn.close() that discarded the
  init connection without ever making it available to route handlers.
2026-03-19 01:24:24 -04:00
KoalaTrapLord b0d0b75dbb refactor: add _build_dck_lines() and _write_dck_file() to services/deck_service.py 2026-03-19 01:06:29 -04:00
Commander AI Lab Agent fbc3bdb6b5 fix: add _resolve_precon_dir() auto-detection so CFG.precon_dir is not empty
The precon deck list was missing after the monolith refactor because
PRECON_DIR was hardcoded in services/precon_service.py with no CLI arg,
env var, or auto-resolve fallback — unlike forge_jar, forge_dir, etc.

Changes:
- Add precon_dir field to Config class (models/state.py)
- Add --precon-dir CLI arg and PRECON_DIR env var support
- Add _resolve_precon_dir() with project-root, parent-dir, and
  forge_dir-relative detection; creates dir as fallback for downloads
- Refactor precon_service.py to use _get_precon_dir() helper that reads
  CFG.precon_dir at runtime instead of a hardcoded module-level Path
- Update routes/precon.py to use _get_precon_dir() for install endpoints
- Wire _on_startup to resolve precon_dir for uvicorn-direct launches
- Remove unused PRECON_DIR/PRECON_INDEX imports from routes/lab.py
- Audit confirms all other CFG path fields already have fallbacks
2026-03-18 04:56:03 +00:00
ClaudeandClaude Opus 4.6 7bc4aac0aa fix: repair run_batch_subprocess signature and _run_process_blocking state attrs in forge_runner.py
The port of run_batch_subprocess from lab_api_backup.py to
services/forge_runner.py truncated the function signature to 8 params +
**kwargs, but the route handler in routes/lab.py passes 13 positional
arguments, causing TypeError at runtime when POST /api/lab/start fires.

Restore all 13 explicit parameters (policy_style, policy_greedy,
ai_simplified, ai_think_time_ms, max_queue_depth) to match the caller
and pass them as keyword args to build_java_command. Also compute
policy_server from CFG.port and add error handling, matching the
original backup implementation.

Additionally, _run_process_blocking was writing to wrong BatchState
attribute names (state.status, state.games_done, state.games_total,
state.throughput, state.log, state.started_at, state.ended_at) instead
of the correct ones (state.running, state.completed_games,
state.total_games, state.sims_per_sec, state.log_lines, state.elapsed_ms).
This caused the /api/lab/status endpoint to never see progress updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 04:38:40 +00:00
ClaudeandClaude Opus 4.6 70ad16fc79 fix: add missing _get_deepseek_brain and _run_deepseek_batch_thread to forge_runner.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 04:31:14 +00:00
ClaudeandClaude Opus 4.6 6298f83abd fix: correct indentation errors in services/*.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 04:22:17 +00:00
Claude d1422d6f64 fix: correct indentation errors in services/database.py 2026-03-18 04:15:06 +00:00
KoalaTrapLord 554b166787 Phase 2d/2e/2f: Extract forge_runner, precon_service, import_service/helpers into services/ 2026-03-17 15:22:19 -04:00
KoalaTrapLord 874561f423 Phase 2c: Extract card analysis to services/card_analysis.py (#12) 2026-03-17 05:14:35 -04:00
KoalaTrapLord c7309d1ace Phase 2b: Extract deck service to services/deck_service.py (#11) 2026-03-17 05:05:52 -04:00
KoalaTrapLord 5aee93dfae Phase 1c: Extract datPhase 2a: Extract Scryfall client to services/scryfall.py (#10)abase service to services/database.py (#9) 2026-03-17 04:54:36 -04:00
KoalaTrapLord e14a6f9500 Phase 1c: Extract database service to services/database.py (#9) 2026-03-17 04:40:44 -04:00