Commit Graph
54 Commits
Author SHA1 Message Date
rgriffithandClaude Opus 4.6 9eaa41a4ae feat: deck dropdown, user-decks pipeline, deckgen fixes, gpt-oss removal
- Add user-decks/ directory and USER_DECKS_DIR config for exported decks
- Update export-to-sim to write to user-decks/ with static mount in lab_api
- Replace import/paste UI with API-backed "My Decks" dropdown in LAN client
  and AI Lab, fetching from GET /api/lab/decks with source grouping
- Fix GET /api/lab/decks to JOIN deck_cards for totalCards count
- Fix deckgenerator: add missing DOM elements (dg-strategy-bar, dg-stats-bar,
  dg-card-groups) inside #dg-results, null guards on render functions,
  fix setLoading always hiding results
- Replace gpt-oss:20b with claude-opus-4-20250514 across all files
- Add user-decks/ to _BUILTIN_DECK_DIRS in deck_service.py
- Enrich missing type_line from Forge card cache in DB deck path
- Collapse import bar behind <details> toggle in AI Lab

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 01:17:30 -04:00
KoalaTrapLord 1fc99a5d4a fix: auto-discover precon-decks/ and sample-decks/ when CFG paths unset
_find_dck_file only searched CFG.precon_dir / CFG.forge_decks_dir, both
of which are empty when the server is started without CLI args. This
caused all precon deck lookups to return 0 cards.

Fix: add _REPO_ROOT and _BUILTIN_DECK_DIRS constants, then append
precon-decks/, sample-decks/, imported-decks/ to the search list
whenever they exist on disk, regardless of whether CFG paths are set.
2026-04-03 04:50:49 -04:00
KoalaTrapLord 7a40b5deb8 fix: enrich type_line/oracle from Forge cache in .dck fallback loader
_load_deck_cards_from_dck was hardcoding type_line="" for every card,
causing Card.is_land() to return False for all lands when a deck is
loaded via the .dck file path (i.e. not in the SQLite DB). This meant
the engine never recognised lands as lands, breaking mana production.

Fix: call lookup_forge_card() per card and populate type_line,
oracle_text, cmc, mana_cost, power, toughness, and keywords from the
Forge cardsfolder cache. The cache is lazy-loaded and module-level so
there is no per-game overhead after the first deck load.
2026-04-03 04:01:59 -04:00
KoalaTrapLord af8970f3a8 fix: _load_deck_cards_by_name falls back to .dck file when deck not in DB
Precon decks (Elven_Empire, Blood_Rites, etc.) are stored as .dck files
on disk but not inserted into the SQLite decks/deck_cards tables.
The DeepSeek batch engine calls _load_deck_cards_by_name() which
previously returned [] for any deck not in the DB, causing
'No decks could be loaded.' errors.

Adds _find_dck_file() and _load_deck_cards_from_dck() helpers.
_find_dck_file searches CFG.precon_dir, CFG.forge_decks_dir, and
FORGE_DIR/res/decks/commander with exact + fuzzy matching.
_load_deck_cards_by_name now tries DB first, then .dck fallback.
2026-04-02 22:59:01 -04:00
KoalaTrapLord a81772c68a fix: set cwd=CFG.forge_dir on Popen so Forge finds res/ (#184) 2026-04-01 19:41:52 -04:00
KoalaTrapLord 395ad06cdd fix: resolve deck names to full .dck paths in build_java_command (#183) 2026-04-01 14:57:57 -04:00
KoalaTrapLord e4a6ce644d fix(#176): draw-game life capture, DrawGameAnalyzer, clock 6000→3000 (#181)
Captures life-at-timeout snapshots for every draw game via [DRAW] [PARSE-SUMMARY] log lines. Adds services/draw_game_analyzer.py with DrawGameSnapshot + summarize_draw_games(). Wires drawGames summary into GET /api/lab/result response. Lowers batch clock default from 6000 to 3000 (30s per-game). Closes #176.
2026-04-01 14:40:29 -04:00
KoalaTrapLord 5411fd6bf6 fix(#175): alternate win condition parser & winType tagging (#179)
Adds win_condition_parser.py, GameResult.win_type field, and winType stamping on both the Java/Forge and DeepSeek batch paths. Closes #175.
2026-04-01 14:28:10 -04:00
KoalaTrapLord 5b5bac42fb fix: use vars(row) instead of dict(row) in _build_collection_from_bulk
dict() on a types.SimpleNamespace raises TypeError: object is not iterable.
vars() correctly extracts the __dict__ attribute map. Fixes
TestBuildCollectionFailureTracking::test_high_failure_rate_raises_runtime_error.
2026-03-30 10:39:49 -04:00
KoalaTrapLord 882e2370c1 fix(#127): persist built_at in ChromaDB collection metadata as sidecar fallback
- _write_build_meta: after writing build_meta.json, also stamps built_at
  onto the collection via col.modify() (best-effort, swallows exceptions)
- _collection_is_current: if sidecar built_at is missing/zero, falls back
  to collection.metadata["built_at"] so a deleted sidecar doesn't force
  an unnecessary full rebuild when the ChromaDB data is still valid
2026-03-29 04:29:35 -04:00
KoalaTrapLord bbca0b7985 fix(#126): add batch insert retry logic with exponential backoff
- Add BATCH_RETRY_LIMIT and BATCH_RETRY_DELAY constants after DEFAULT_BATCH_SIZE
- Add _insert_batch_with_retry() helper with exponential backoff
- Replace both bare try/except insert blocks in _build_collection_from_bulk
  with calls to the retry helper so transient ChromaDB errors are retried
  up to BATCH_RETRY_LIMIT times before a batch is counted as failed
2026-03-29 04:24:58 -04:00
KoalaTrapLord aefd0cace6 feat(rag): Phase 1 — rewrite build_card_text to natural-language format (#137)
* feat(rag): Phase 1 — rewrite build_card_text to natural-language format

* feat(rag): Phase 2 — Forge-enriched ChromaDB RAG store (2a–2e)

Phase 2a: Forge lazy import guard (_try_import_forge_lookup)
Phase 2b: _build_document appends Forge trigger/replacement/static/keyword tags
Phase 2c: _build_metadata adds has_trigger, has_replacement, has_static, trigger_modes fields
Phase 2d: query_cards extended with require_trigger, require_replacement, require_static params + ChromaDB where clause support
Phase 2e: get_rag_stats reports forge_enriched and forge_coverage; _write_build_meta tracks coverage; build loop counts enriched cards

* feat(rag): Phase 3 — add mtgrules second ChromaDB collection

Adds build_rules_index() and query_rules() operating on a separate
'mtgrules' ChromaDB collection. Chunks MagicCompRules.txt by rule-number
boundaries, embeds each chunk, and exposes semantic search. Completely
independent from the mtg_oracle card collection.

* feat(coach): Phase 4 — behavior-aware RAG + rules context in coach_service

Phase 4a: run_coaching_session uses dual RAG query
  - Primary: commander synergy (n=10)
  - Secondary: require_replacement=True for graveyard/protection cards (n=5)
  - Deduplicates by name before merging
Phase 4b: query_rules() fetches 3 Commander rules chunks to ground the prompt
  - Passed as rules_context kwarg to build_user_prompt()

* feat(rag): Phase 5 — add /api/rag/build-rules and /api/rag/query-rules endpoints
2026-03-29 02:11:38 -04:00
KoalaTrapLord 1c491bc2b0 fix: add Land role detection to _detect_card_roles (fixes basics showing as Unknown) 2026-03-27 20:28:35 -04:00
KoalaTrapLord 0010b0f394 refactor: RAG improvements — shared card_text, Chroma fallback, staleness monitor (fixes #133, #134, #135) (#136)
* refactor: add shared build_card_text() utility

* refactor: unify _build_document via shared card_text + add is_chroma_available/check_staleness (fixes #133, #134, #135)

* refactor: delegate embeddings._build_card_text to shared utility

* feat: add background RAG staleness monitor to lab_api lifespan (fixes #135)
2026-03-27 16:46:33 -04:00
KoalaTrapLord bcf558418a fix(rag): squash all RAG/ChromaDB bugs #123–#130
Fixes critical API contract mismatch, batch_size wiring, lifespan init, failure tracking, build_meta persistence, mana_cost in embeddings, pyproject.toml deps, and adds full test suite.

Closes #123, #124, #125, #126, #127, #128, #129, #130
2026-03-27 13:24:41 -04:00
KoalaTrapLord 4b0cd34eda feat(rag): Phase 2 — ChromaDB vector index over bulk card DB (#120)
* feat(rag): Phase 2 — ChromaDB vector index over bulk card DB

* feat(rag): add routes/rag.py with build, search, status endpoints

* Update lab_api.pyfeat(rag): register rag_router in lab_api.py, remove old inline rag_status

* Update requirements.txtfeat(rag): add chromadb>=0.5.0 to requirements.txtfeat(rag): add chromadb to requirements.txt
2026-03-27 08:39:02 -04:00
KoalaTrapLord 8df5900ff9 feat(rag): Phase 1 — Scryfall bulk offline card database (SQLite FTS5) (#117)
* feat(rag): add scryfall_bulk.py — offline SQLite FTS5 card database (RAG Phase 1)

* feat(rag): wire ensure_bulk_db() into startup and add /api/rag/status endpoint (RAG Phase 1)

* fix: migrate on_event to lifespan handler, fix /api/rag/status route ordering
2026-03-27 07:16:55 -04:00
KoalaTrapLord 6c03e56794 fix(#116): dual card_records fallback JOIN, card_type/card_category fields, simulator land detection
- deck_service._compute_deck_analysis: add LEFT JOIN card_records with COALESCE on all fields
- deck_service: add _classify_card_category() with oracle-text keyword matching
- deckbuilder.GET /api/decks/{id}/cards: add card_records fallback JOIN + COALESCE, emit card_type/card_category per card
- deepseek._fetch_deck_card_data: add card_records fallback JOIN so lands get correct type_line before enrich_card()

Closes #116
2026-03-27 04:10:17 -04:00
KoalaTrapLord af2a71d1a5 fix: eliminate event-loop-blocking sync calls in async FastAPI routes (#106)
* feat: add async DB wrapper and eliminate event-loop blocking

* fix: rewrite llm_client.py with native async httpx - removes urlopen and run_in_executor

* fix: rewrite routes/coach.py - replace all sync DB/HTTP with async (aiosqlite + httpx)

* fix: add async DB shutdown hook to lab_api.py

* feat: add aiosqlite>=0.19.0 to requirements
2026-03-25 19:17:41 -04:00
KoalaTrapLord e7ffccf3e8 feat: persist coach sessions to SQLite with disk fallback 2026-03-24 23:44:20 -04:00
KoalaTrapLord 015b9da1e9 rename: update forge_runner.py file prefix from batch- to ml-decision- 2026-03-23 23:06:33 -04:00
87404f3702 refactor: move logging out of routes/shared.py into services/logging.py (#99)
Create services/logging.py as the canonical home for setup_logging(),
all log format constants, named loggers, and _ml_logging_enabled.

Update all route modules and lab_api.py to import logging symbols from
services.logging and all other symbols from their canonical sources
(models.state, models.requests, models.responses, services.database,
services.deck_service, services.scryfall, services.import_helpers,
services.import_service, services.precon_service, services.forge_runner).

Delete routes/shared.py — the re-export hub is no longer needed.

Closes #61

Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 21:04:25 -04:00
KoalaTrapLord b37968e3c2 fix: always pass --ml-log flag to Forge Java batch runner 2026-03-23 17:22:58 -04:00
KoalaTrapLord 49fc817187 fix(#92): bump BRAIN_TIMEOUT fallback from 120s to 300s in forge_runner.py 2026-03-22 23:40:28 -04:00
ClaudeandClaude Opus 4.6 978fb12969 fix: early return in _load_deck_cards_by_name when deck not found in DB
Fixes UnboundLocalError crash when deck_row is None by returning an
empty list instead of falling through to a query with an unbound deck_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 07:25:39 +00:00
KoalaTrapLord 11978736a1 fix: raise BRAIN_TIMEOUT default from 30s to 120s (#71) 2026-03-21 19:39:29 -04:00
KoalaTrapLord 06f007e605 fix: invert guard in _load_deck_cards_by_name — was if not deck_row (crash on None[0])
The guard condition was inverted: `if not deck_row:` entered the query
block when no deck was found, causing `deck_row[0]` to crash with
'NoneType' object is not subscriptable. Flipped to `if deck_row:`
so the query runs only when the deck exists.
2026-03-21 15:11:08 -04:00
KoalaTrapLord a7ab0e43a5 feat: Phase 6 — make primary brain env-configurable (BRAIN_* vars)
Replace hardcoded DEEPSEEK_API_BASE/DEEPSEEK_MODEL env vars in
_get_deepseek_brain() with new BRAIN_* environment variables:
- BRAIN_API_BASE (default: http://localhost:11434)
- BRAIN_MODEL (default: gpt-oss:20b)
- BRAIN_TEMPERATURE (default: 0.3)
- BRAIN_MAX_TOKENS (default: 1024)
- BRAIN_TIMEOUT (default: 30.0)

All params now passed directly to DeepSeekConfig constructor.
2026-03-21 13:21:58 -04:00
KoalaTrapLord d86109c5c0 Fix duplicate WHERE clause in _load_deck_cards_by_name SQL query 2026-03-19 17:54:34 -04:00
KoalaTrapLord d9dd098ca3 fix: add power, toughness, keywords to _load_deck_cards_by_name SQL query 2026-03-19 17:43:57 -04:00
KoalaTrapLord b00df14f30 fix: use card_name key from _load_deck_cards_by_name (KeyError: 'name') 2026-03-19 16:51:27 -04:00
KoalaTrapLord 3542efd982 fix: align build_java_command with LabCli picocli args (--forge-jar, --forge-dir, --deck1..4) 2026-03-19 16:42:03 -04:00
KoalaTrapLord 0e5d7ba7d3 fix: harden _find_java17 to search multiple vendors, JAVA_HOME, and warn on wrong version 2026-03-19 15:23:44 -04:00
KoalaTrapLord b4d8c2f50e Fix indentation error on line 47 of forge_runner.py 2026-03-19 06:54:16 -04:00
KoalaTrapLord 9db1ed112f Increase _STALL_LIMIT from 300s to 600s (10 min) to prevent premature batch kills 2026-03-19 06:30:03 -04:00
KoalaTrapLord d834247885 Fix forge_runner.py: restore full file, fix sim mode arg
- Restored _run_deepseek_batch_thread and _get_deepseek_brain
  (accidentally removed in prior commit)
- Changed '-sim' to 'sim' — Forge expects positional mode, not flag
2026-03-19 10:13:11 +00:00
rgriffith 310a408509 feat: polish precon picker — card art, color pips, grid layout
- Add full CSS for precon grid: responsive card grid, hover effects,
  install button states, color pips, dark theme styling
- Build commander art_crop image URLs from set_code/number during
  precon download (Scryfall direct image redirect, no API call needed)
- Batch-fetch color_identity from Scryfall /cards/collection endpoint
  (2 requests for ~145 unique commanders) and store in precon index
- Widen precon modal to 900px for grid layout
- Auto-invalidate precon cache when enrichment fields are missing
- JS renderer now shows commander art crop + card body layout
- Bump cache-busting version to v=4
2026-03-19 10:10:40 +00:00
KoalaTrapLord ef7bde841f Fix forge_runner.py: sim mode arg + indentation corruption
- Changed '-sim' to 'sim' (Forge expects positional mode, not flag)
- Fixed indentation: build_java_command, _run_process_blocking,
  run_batch_subprocess were incorrectly nested inside get_java17()
- Fixed last_activity assignment stuck inside except block
- Added docstrings
2026-03-19 10:09:59 +00:00
rgriffith 8eba39cca7 fix: precon picker empty — mutate PRECON_INDEX in-place
The PRECON_INDEX list was imported by value in routes/shared.py.
When download_precon_database() or load_precon_index() reassigned it
with 'PRECON_INDEX = ...', the new list object was only visible in
precon_service module scope — shared.PRECON_INDEX (and thus the
/api/lab/precons endpoint) still pointed to the original empty [].

Fix: use PRECON_INDEX.clear() + .extend() instead of reassignment
so all modules sharing the same list reference see the updated data.
2026-03-19 10:01:51 +00:00
Computer c6c3bfabf8 refactor: replace per-batch watchdog threads with single global watchdog 2026-03-19 06:56:56 +00:00
KoalaTrapLord 83a8f288b7 Fix scrambled deck_service.py: close SQL query in _load_deck_cards_by_name, remove orphaned lines 2026-03-19 06:24:13 +00:00
KoalaTrapLord e3f0787c73 Optimize deckgen: push colour-identity filtering into SQL with json_each
- Replace SELECT * + Python-side color loop in _get_collection_for_colors()
  with projected SELECT + NOT EXISTS json_each() sub-query
- Apply same pattern to _build_collection_summary(), removing card_fits_identity()
- Add partial index idx_ce_quantity on collection_entries(quantity) WHERE qty > 0
- Handle NULL and empty-string color_identity via CASE guard
- Fix colorless commander bug: old code skipped color check entirely for []
- Add scripts/bench_deckgen.py benchmark (3-5x speedup on filtered queries)
- Add tests/test_deckgen_color_filter.py with 7 regression tests
2026-03-19 06:18:32 +00:00
KoalaTrapLord 40a166188a fix: unify DB connection management — init_collection_db() uses _get_db_conn()
Previously, init_collection_db() opened a separate sqlite3.connect() call,
ran migrations, then closed that connection — entirely outside the
thread-local pool that route handlers use via _get_db_conn().

This meant:
- The migration connection and the request-serving connection were
  independent objects with no ordering guarantee under concurrent startup.
- The warm-up connection was thrown away immediately, so every handler
  thread still had to open its own connection on first use.

Fix:
- init_collection_db() now calls _get_db_conn() directly, so schema
  creation and migrations run on the same thread-local connection that
  will serve subsequent requests.
- Added _close_db_conn() helper for explicit teardown on thread exit
  (e.g. FastAPI lifespan shutdown or worker thread cleanup).
- Removed the redundant standalone conn.close() that discarded the
  init connection without ever making it available to route handlers.
2026-03-19 01:24:24 -04:00
KoalaTrapLord b0d0b75dbb refactor: add _build_dck_lines() and _write_dck_file() to services/deck_service.py 2026-03-19 01:06:29 -04:00
Commander AI Lab Agent fbc3bdb6b5 fix: add _resolve_precon_dir() auto-detection so CFG.precon_dir is not empty
The precon deck list was missing after the monolith refactor because
PRECON_DIR was hardcoded in services/precon_service.py with no CLI arg,
env var, or auto-resolve fallback — unlike forge_jar, forge_dir, etc.

Changes:
- Add precon_dir field to Config class (models/state.py)
- Add --precon-dir CLI arg and PRECON_DIR env var support
- Add _resolve_precon_dir() with project-root, parent-dir, and
  forge_dir-relative detection; creates dir as fallback for downloads
- Refactor precon_service.py to use _get_precon_dir() helper that reads
  CFG.precon_dir at runtime instead of a hardcoded module-level Path
- Update routes/precon.py to use _get_precon_dir() for install endpoints
- Wire _on_startup to resolve precon_dir for uvicorn-direct launches
- Remove unused PRECON_DIR/PRECON_INDEX imports from routes/lab.py
- Audit confirms all other CFG path fields already have fallbacks
2026-03-18 04:56:03 +00:00
ClaudeandClaude Opus 4.6 7bc4aac0aa fix: repair run_batch_subprocess signature and _run_process_blocking state attrs in forge_runner.py
The port of run_batch_subprocess from lab_api_backup.py to
services/forge_runner.py truncated the function signature to 8 params +
**kwargs, but the route handler in routes/lab.py passes 13 positional
arguments, causing TypeError at runtime when POST /api/lab/start fires.

Restore all 13 explicit parameters (policy_style, policy_greedy,
ai_simplified, ai_think_time_ms, max_queue_depth) to match the caller
and pass them as keyword args to build_java_command. Also compute
policy_server from CFG.port and add error handling, matching the
original backup implementation.

Additionally, _run_process_blocking was writing to wrong BatchState
attribute names (state.status, state.games_done, state.games_total,
state.throughput, state.log, state.started_at, state.ended_at) instead
of the correct ones (state.running, state.completed_games,
state.total_games, state.sims_per_sec, state.log_lines, state.elapsed_ms).
This caused the /api/lab/status endpoint to never see progress updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 04:38:40 +00:00
ClaudeandClaude Opus 4.6 70ad16fc79 fix: add missing _get_deepseek_brain and _run_deepseek_batch_thread to forge_runner.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 04:31:14 +00:00
ClaudeandClaude Opus 4.6 6298f83abd fix: correct indentation errors in services/*.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 04:22:17 +00:00
Claude d1422d6f64 fix: correct indentation errors in services/database.py 2026-03-18 04:15:06 +00:00
KoalaTrapLord 554b166787 Phase 2d/2e/2f: Extract forge_runner, precon_service, import_service/helpers into services/ 2026-03-17 15:22:19 -04:00