diff --git a/FARMHOUSE-CHANGES.md b/FARMHOUSE-CHANGES.md new file mode 100644 index 0000000..96c070e --- /dev/null +++ b/FARMHOUSE-CHANGES.md @@ -0,0 +1,62 @@ +# FarmHouse changes + +This branch (`farmhouse`) is `KoalaTrapLord/commander-ai-lab` at commit `54cc5e9` plus the changes below. They were made to run the app +**Python-only** (no Forge/Java, no PyTorch) in a Kubernetes namespace, with a local LLM behind an OpenAI-compatible gateway instead of +Anthropic/DeepSeek, and they were each found by actually running the app, not by reading it. Written 2026-10-26. + +Every change started life as an anchored text patch (`patch_app.py` in the farmhouse-os repo, which still builds the deployed image from +the upstream commit); here they are ordinary source edits. Lines carry a `# FARMHOUSE_...` marker where the patch left one. + +## What was wrong, and what changed + +### The simulator played nothing (the big one) +| Problem | Change | +|---|---| +| Every card reached the simulator with **no type line and no rules text**. The loaders join the user's *collection* and a *card_records* table (both empty on a fresh install) and the Forge card cache that was meant to fill the gap is not installed. A land was indistinguishable from a creature: an 8-game batch cast 0 spells, dealt 0 damage and lost every game. | `services/farmhouse_sim.py` `fill_card_data` fills type, text, mana cost, cmc, P/T, keywords and colour identity from the 113k-card Scryfall bulk database. Wired into `services/forge_runner.py` (batch) and `routes/deepseek.py` (`_fetch_deck_card_data`, which the Monte Carlo and single-deck routes use). | +| The batch loader **did not repeat cards by quantity**: a 100-card deck loaded as 66-84 cards with *one* Island. | `expand_quantities` (same module). | +| The LLM engine asked the model **once per turn** and applied the answer as one action. | `src/commander_ai_lab/sim/farmhouse_turns.py`: up to `BRAIN_STEPS` (default 4) questions per turn (land, spell, spell, attack), re-reading the board after each; stops on attack/hold/no change. `BRAIN_STEPS=1` restores the original. | +| A model that "thinks" spent ~46 s on a one-line move. | `deepseek_brain.py` sends `reasoning_effort` from `LLM_REASONING` (46 s -> 1.8 s per move). | +| The engine dropdown defaulted to **Forge**, which cannot run without Java. | `ui/ai-lab.js`: only "LLM opponents" is offered. | + +After the fixes: a 40-game Monte Carlo takes 0.4 s with real numbers (10 spells and 12.7 damage per game); two LLM-piloted games took +212 s with 49 model calls and no fallbacks. + +### Deck import +| Problem | Change | +|---|---| +| **Moxfield links were accepted nowhere**, and the Deck Builder import took only pasted text. | `services/farmhouse_deck_links.py`: Moxfield (public decks, `api2.moxfield.com`) and Archidekt links in the simulator URL import and in both Deck Builder imports (a lone link in the paste box). A private deck says to use Moxfield's Export > Copy as text. | +| The Deck Builder import looked **every card up on scryfall.com**; after a few dozen it was rate-limited (HTTP 429) and the app reported the rest as "not found" (250-card deck: 55 matched; even plain `Sol Ring` failed). | `local_card()`: names resolve from the local bulk database first; the website is only the fallback. | + +### Deck generation +- `deck_builder/pipeline/farmhouse_fit.py`: the model's list is **filtered to the commander's colour identity** (from the bulk DB), de-duplicated, + fitted to per-category targets and trimmed to 99 + commander. Before this the model returned off-colour cards and 359-card lists. +- Scryfall bulk format changed (`jsonl_download_uri`, gzip JSONL; no `download_uri`). +- Scryfall answers **HTTP 400 to the default `python-requests` User-Agent** (every lookup became "Commander not found"): a named UA is sent everywhere. +- Scryfall 429s: requests back off on `Retry-After` and the pace is 0.25 s. +- `LLM_REASONING` / `LLM_MAX_TOKENS` are sent by the deck builder and coach. + +### Coach / startup +- The coach was only initialised in `main()`, not when the app starts under uvicorn; `check_connection_sync()` was called but never defined; + the coach used the first model the gateway listed rather than the configured one. All fixed. +- Status label says "Language model" instead of the hardcoded "Claude Opus". +- `RAG_AUTOBUILD` (default off in our deployment) controls whether the card-search index is built at startup (113k embeddings; about an hour). + +## Settings (environment variables) +| Variable | Meaning | +|---|---| +| `LLM_URL`, `LLM_MODEL` | OpenAI-compatible endpoint (`.../v1`) and model for the coach and deck builder | +| `LLM_REASONING` (`none`), `LLM_MAX_TOKENS`, `LLM_TIMEOUT` | keep requests lean: reasoning off, cap on answer length | +| `COACH_PROVIDER`, `DECK_GEN_PROVIDER` = `local`, `DECK_GEN_MODEL` | use the local model, not Anthropic | +| `OLLAMA_BASE_URL`, `RAG_EMBEDDING_MODEL`, `RAG_AUTOBUILD` | card-search embeddings (one host only: vectors differ between hosts) | +| `BRAIN_API_BASE` (**no** `/v1`; the client appends it), `BRAIN_MODEL`, `BRAIN_MAX_TOKENS`, `BRAIN_STEPS` | the simulator's LLM opponents | + +## Tests +`tests/farmhouse/` (standard library only, no network): `python -I -X utf8 tests/farmhouse/test_farmhouse_deck_links.py`, `..._sim.py`, `..._fit.py`. +Run them with `-I` and from outside the package folders; do not put `services/` on `sys.path` (`services/logging.py` shadows the standard library). + +## Things still worth doing +- Run the LLM-opponent games **in parallel**: each move is a short prompt with a short answer, which a batching server handles well. +- Opponents are the engine's heuristics; only the piloted seat uses the model. +- Build the card-search index and measure it. +- The upstream `LICENSE` is missing (`pyproject.toml` says MIT). Ask the author to add one before publishing a derivative widely. +- Forge-based features (the Java simulator, training) remain unavailable by design. diff --git a/coach/coach_service.py b/coach/coach_service.py index f81fddc..e701250 100644 --- a/coach/coach_service.py +++ b/coach/coach_service.py @@ -549,7 +549,7 @@ class CoachService: status.llmModels = [] status.error = "Anthropic API key not configured. Set ANTHROPIC_API_KEY env var." else: - llm_status = self.llm.check_connection() + llm_status = self.llm.check_connection_sync() # FARMHOUSE_COACH_STATUS_V1 (was an un-awaited coroutine) status.llmConnected = llm_status.get("connected", False) status.llmModel = llm_status.get("active_model") status.llmModels = llm_status.get("models", []) diff --git a/coach/llm_client.py b/coach/llm_client.py index dec72d9..eca0cc2 100644 --- a/coach/llm_client.py +++ b/coach/llm_client.py @@ -74,7 +74,7 @@ class LLMClient: try: status = await self.check_connection() if status.get("connected") and status.get("active_model"): - self._resolved_model = status["active_model"] + self._resolved_model = self.model if self.model in (status.get("models") or []) else status["active_model"] # FARMHOUSE_MODEL_PIN_V1 logger.info("Resolved Ollama model: %s", self._resolved_model) return self._resolved_model except Exception: @@ -103,6 +103,14 @@ class LLMClient: } if force_json: body["response_format"] = {"type": "json_object"} + # FARMHOUSE_LLM_KNOBS_V1 (see patch_app.py) + import os as _fh_os + _fh_effort = _fh_os.environ.get("LLM_REASONING", "").strip() + if _fh_effort: + body["reasoning_effort"] = _fh_effort + _fh_cap = int(_fh_os.environ.get("LLM_MAX_TOKENS", "0") or 0) + if _fh_cap: + body["max_tokens"] = min(body["max_tokens"], _fh_cap) return body # ── Response parsing ────────────────────────────────────── @@ -260,6 +268,17 @@ class LLMClient: f"LLM call failed after {LLM_MAX_RETRIES} attempts: {last_error}" ) + def check_connection_sync(self) -> dict: # FARMHOUSE_LLM_SYNC_V1 + """Blocking version of check_connection (upstream called it and never defined it).""" + try: + _r = httpx.get(f"{self.base_url}/models", timeout=5.0, headers={"Content-Type": "application/json"}) + _r.raise_for_status() + _ids = [m.get("id", "unknown") for m in _r.json().get("data", [])] + return {"connected": True, "models": _ids, + "active_model": self.model if self.model in _ids else (_ids[0] if _ids else self.model)} + except Exception as _e: + return {"connected": False, "error": str(_e), "models": [], "active_model": None} + async def check_connection(self) -> dict: """Check if Ollama is reachable. Returns status dict.""" try: diff --git a/lab_api.py b/lab_api.py index 135125b..987f14b 100644 --- a/lab_api.py +++ b/lab_api.py @@ -62,6 +62,7 @@ async def _lifespan(application: FastAPI): CFG.precon_dir = _resolve_precon_dir() init_collection_db() validate_forge_config() + init_coach_service() # FARMHOUSE_COACH_INIT_V1 if not PRECON_INDEX: download_precon_database() if not COMMANDER_META: @@ -85,7 +86,7 @@ async def _lifespan(application: FastAPI): try: from services.rag_store import check_staleness, build_index info = check_staleness() - if info.get("stale"): + if info.get("stale") and os.environ.get("RAG_AUTOBUILD", "0") == "1": # FARMHOUSE_RAG_MONITOR_V1 log.info("RAG staleness monitor: %s — triggering rebuild.", info["reason"]) loop.run_in_executor(None, lambda: build_index(force=True)) else: diff --git a/routes/deckbuilder.py b/routes/deckbuilder.py index 33e8fbf..593f473 100644 --- a/routes/deckbuilder.py +++ b/routes/deckbuilder.py @@ -888,7 +888,11 @@ async def import_decklist(deck_id: int, request: FastAPIRequest): """ deck = _get_deck_or_404(deck_id) body = await request.json() - text = body.get('text', '') + try: # FARMHOUSE_DECK_LINKS_V1b + from services.farmhouse_deck_links import expand as _fh_expand + text, _fh_name = _fh_expand(body.get('text', '') or body.get('url', '')) + except ValueError as _fh_err: + return JSONResponse({'error': str(_fh_err)}, status_code=400) clear_first = body.get('clearFirst', False) if not text.strip(): @@ -980,8 +984,12 @@ async def import_decklist_new(request: FastAPIRequest): Body: { "text": "...", "name": "My Deck" } """ body = await request.json() - text = body.get('text', '') - deck_name = body.get('name', '').strip() + try: # FARMHOUSE_DECK_LINKS_V1c + from services.farmhouse_deck_links import expand as _fh_expand + text, _fh_name = _fh_expand(body.get('text', '') or body.get('url', '')) + except ValueError as _fh_err: + return JSONResponse({'error': str(_fh_err)}, status_code=400) + deck_name = body.get('name', '').strip() or (_fh_name or '') if not text.strip(): return JSONResponse({'error': 'No decklist text provided'}, status_code=400) @@ -1064,6 +1072,10 @@ def _scryfall_fuzzy_lookup(name: str) -> Optional[dict]: Fuzzy-search Scryfall for a card name (used by the scanner pipeline). Returns the raw Scryfall JSON dict, or None on failure. """ + from services.farmhouse_deck_links import local_card as _fh_local # FARMHOUSE_LOCAL_LOOKUP + _fh_hit = _fh_local(name) + if _fh_hit: + return _fh_hit _scryfall_rate_limit() try: encoded = quote(name) diff --git a/routes/deepseek.py b/routes/deepseek.py index dbc222c..e0faf19 100644 --- a/routes/deepseek.py +++ b/routes/deepseek.py @@ -516,6 +516,8 @@ def _fetch_deck_card_data(deck_id): 'keywords': r[7] or '', 'mana_cost': r[8] or '', }) + from services.farmhouse_sim import fill_card_data as _fh_fill # FARMHOUSE_SIM_DATA_B + _fh_fill(card_data) return deck_name, card_data diff --git a/services/farmhouse_deck_links.py b/services/farmhouse_deck_links.py new file mode 100644 index 0000000..3c710c0 --- /dev/null +++ b/services/farmhouse_deck_links.py @@ -0,0 +1,154 @@ +"""Deck LINKS for every deck import (copied into the app as services/farmhouse_deck_links.py by the Dockerfile). + +Upstream could import from an Archidekt or EDHREC link on the simulator page only; the Deck Builder import took pasted text and +nothing else, and Moxfield (where most people keep decks) was not accepted anywhere. This reads a Moxfield or Archidekt deck link and +returns what the existing importers already understand: a profile for the simulator import, or plain "[Commander] / [Main]" text for +the Deck Builder's two imports (so a link pasted into the same box works, and the card-by-card Scryfall matching is unchanged). + +Moxfield has no official public API; api2.moxfield.com answers server-side requests for PUBLIC decks (tested from the cluster +2026-10-26: HTTP 200, a named User-Agent is enough). A private or deleted deck comes back 401/403/404, and the message says so and +says what to do instead (Moxfield: Export > Copy as text, then paste that). Nothing here logs in to anything. +""" +import json +import re +import time +from urllib.error import HTTPError +from urllib.request import Request, urlopen + +UA = "CommanderAILab/3.0 (self-hosted)" +MOXFIELD_API = "https://api2.moxfield.com/v2/decks/all/" + +_MOX = re.compile(r"^https?://(?:www\.)?moxfield\.com/decks/([A-Za-z0-9_-]+)(?:[/?#].*)?$", re.I) +_ARC = re.compile(r"^https?://(?:www\.)?archidekt\.com/decks/(\d+)(?:[/?#].*)?$", re.I) + +UNSUPPORTED = ("That link is not one I can read. Moxfield and Archidekt deck links work; for anything else, paste the decklist as text " + "(Moxfield: Export > Copy as text).") + + +def link_kind(text): + """(kind, id, url) when `text` is nothing but one http(s) link, else None. kind: moxfield | archidekt | unsupported.""" + t = (text or "").strip() + if not t or len(t.split()) != 1 or not re.match(r"^https?://", t, re.I): + return None + m = _MOX.match(t) + if m: + return "moxfield", m.group(1), t + m = _ARC.match(t) + if m: + return "archidekt", m.group(1), t + return "unsupported", None, t + + +def _get_json(url): + with urlopen(Request(url, headers={"User-Agent": UA, "Accept": "application/json"}), timeout=25) as resp: + return json.loads(resp.read().decode("utf-8", "replace")) + + +def fetch_moxfield(deck_id, attempts=4): + last = None + for i in range(attempts): + try: + return _get_json(MOXFIELD_API + deck_id) + except HTTPError as exc: + if exc.code in (401, 403, 404): + raise ValueError("Moxfield would not share that deck (it is private, unlisted for sharing, or the link is wrong). " + "Open it in Moxfield, choose Export > Copy as text, and paste that here instead.") + if exc.code in (429, 500, 502, 503, 504): + last = exc + try: + wait = float(exc.headers.get("Retry-After") or 0) + except (TypeError, ValueError): + wait = 0.0 + time.sleep(max(wait, 2.0) + i * 2.0) + continue + raise ValueError("Moxfield answered HTTP %s for that deck." % exc.code) + except (ValueError, OSError) as exc: + raise ValueError("Could not read the deck from Moxfield (%s)." % str(exc)[:80]) + raise ValueError("Moxfield is rate-limiting this server right now (HTTP %s). Try again in a minute." % (last.code if last else "429")) + + +def _board(deck, key): + out = {} + for name, entry in (deck.get(key) or {}).items(): + card = (entry or {}).get("card") or {} + label = card.get("name") or name + try: + qty = int((entry or {}).get("quantity") or 1) + except (TypeError, ValueError): + qty = 1 + out[label] = out.get(label, 0) + max(qty, 1) + return out + + +def moxfield_profile(deck, url): + """The same profile dict the Archidekt importer builds. Sideboard, maybeboard and companions are left out (the 99 + commander).""" + commanders, main = _board(deck, "commanders"), _board(deck, "mainboard") + profile = { + "name": deck.get("name") or "Moxfield deck", + "commander": next(iter(commanders), None), + "source": "Moxfield", + "sourceUrl": deck.get("publicUrl") or url, + "commanders": commanders, + "mainboard": main, + "colorIdentity": list(deck.get("colorIdentity") or []), + "totalCards": sum(commanders.values()) + sum(main.values()), + } + if profile["commander"] and not deck.get("name"): + profile["name"] = "%s — Moxfield" % profile["commander"] + return profile + + +def fetch_profile(url): + kind = link_kind(url) + if not kind or kind[0] == "unsupported": + raise ValueError(UNSUPPORTED) + if kind[0] == "moxfield": + profile = moxfield_profile(fetch_moxfield(kind[1]), kind[2]) + else: + from services.import_helpers import _fetch_archidekt_deck # the app's own Archidekt reader + try: + profile = _fetch_archidekt_deck(kind[1]) + except HTTPError as exc: + raise ValueError("Archidekt would not share that deck (HTTP %s). Is it public?" % exc.code) + if not profile["commanders"] and not profile["mainboard"]: + raise ValueError("That deck has no cards in it (or no main deck I can read).") + return profile + + +def profile_to_text(profile): + """Plain text in the Deck Builder's own format: [Commander] then [Main], one 'qty name' per line.""" + lines = ["[Commander]"] + lines += ["%d %s" % (q, n) for n, q in profile["commanders"].items()] + lines += ["", "[Main]"] + lines += ["%d %s" % (q, n) for n, q in profile["mainboard"].items()] + return "\n".join(lines) + "\n" + + +def local_card(name, conn=None): + """{'id', 'name'} from the local Scryfall bulk database (113k cards, no network), or None. Matches the exact name, or the front + face of a double-faced / split card ('Fire' finds 'Fire // Ice'). The Deck Builder's importers only use these two fields, and + looking each card up on Scryfall's website instead hit its rate limit (HTTP 429) after a few dozen cards of a 100-card list, + which the app reported as 'not found'.""" + try: + if conn is None: + from services.scryfall_bulk import get_bulk_db + conn = get_bulk_db() + n = (name or "").strip() + if not n: + return None + like = n.replace("!", "!!").replace("%", "!%").replace("_", "!_") + " // %" + row = conn.execute("SELECT scryfall_id, name FROM cards WHERE LOWER(name) = LOWER(?) OR LOWER(name) LIKE LOWER(?) ESCAPE '!' " + "ORDER BY (LOWER(name) = LOWER(?)) DESC, edhrec_rank ASC LIMIT 1", (n, like, n)).fetchone() + return {"id": row[0], "name": row[1]} if row else None + except Exception: + return None + + +def expand(text): + """(text, deck name). A lone Moxfield/Archidekt link becomes the decklist text; anything else comes back unchanged. + A lone link we cannot read raises ValueError with a message meant for the person.""" + kind = link_kind(text) + if kind is None: + return text, None + profile = fetch_profile(kind[2]) + return profile_to_text(profile), profile["name"] diff --git a/services/farmhouse_sim.py b/services/farmhouse_sim.py new file mode 100644 index 0000000..f7c4919 --- /dev/null +++ b/services/farmhouse_sim.py @@ -0,0 +1,69 @@ +"""Card data for the simulators (copied into the app as services/farmhouse_sim.py by the Dockerfile). + +Found 2026-10-26 from a real run: every card of every saved deck reached the simulator with no type line and no rules text (the +loaders join the user's *collection* and a *card_records* table, both empty here; the Forge card cache that was meant to fill the gap +is not installed). The simulator could not tell a land from a creature, so a whole 8-game batch cast 0 spells, dealt 0 damage and lost +every game. The batch loader also did not repeat cards by quantity, so a 100-card deck loaded as 66-84 cards with ONE Island. + +fill_card_data fills the gaps from the local 113k-card Scryfall bulk database; expand_quantities repeats a row `quantity` times. +""" + + +def _name(card): + return (card.get("card_name") or card.get("name") or "").strip() + + +def _lookup(conn, name, cache): + key = name.lower() + if key in cache: + return cache[key] + like = name.replace("!", "!!").replace("%", "!%").replace("_", "!_") + " // %" + row = conn.execute( + "SELECT type_line, oracle_text, mana_cost, cmc, power, toughness, keywords, color_identity FROM cards " + "WHERE LOWER(name) = LOWER(?) OR LOWER(name) LIKE LOWER(?) ESCAPE '!' " + "ORDER BY (LOWER(name) = LOWER(?)) DESC, edhrec_rank ASC LIMIT 1", (name, like, name)).fetchone() + cache[key] = row + return row + + +def fill_card_data(cards, conn=None): + """Fill type_line / oracle_text / mana_cost / cmc / power / toughness / keywords / color_identity where a card dict lacks them. + Never overwrites a value that is already there. Returns the same list.""" + try: + if conn is None: + from services.scryfall_bulk import get_bulk_db + conn = get_bulk_db() + cache = {} + for card in cards: + n = _name(card) + if not n or card.get("type_line"): + continue + row = _lookup(conn, n, cache) + if not row: + continue + for field, value in zip(("type_line", "oracle_text", "mana_cost", "cmc", "power", "toughness", "keywords", "color_identity"), row): + if value not in (None, "") and not card.get(field): + card[field] = value + except Exception: + pass + return cards + + +def expand_quantities(cards): + """One dict per physical card: a row with quantity 23 becomes 23 rows (each quantity 1).""" + out = [] + for card in cards: + try: + q = int(card.get("quantity") or 1) + except (TypeError, ValueError): + q = 1 + for _ in range(max(q, 1)): + copy = dict(card) + copy["quantity"] = 1 + out.append(copy) + return out + + +def prepare(cards): + """Rows as the batch loader returns them -> one filled-in dict per physical card.""" + return fill_card_data(expand_quantities(cards)) diff --git a/services/forge_runner.py b/services/forge_runner.py index 7775002..2bf205e 100644 --- a/services/forge_runner.py +++ b/services/forge_runner.py @@ -515,6 +515,8 @@ def _run_deepseek_batch_thread( deck_meta = {} for dn in deck_names: raw_cards = _load_deck_cards_by_name(dn) + from services.farmhouse_sim import prepare as _fh_prepare # FARMHOUSE_SIM_DATA + raw_cards = _fh_prepare(raw_cards or []) if not raw_cards: state.log_lines.append(f'[DeepSeek Batch] WARNING: Could not load deck "{dn}", skipping.') continue diff --git a/services/import_helpers.py b/services/import_helpers.py index f0324ee..db3fb9f 100644 --- a/services/import_helpers.py +++ b/services/import_helpers.py @@ -97,6 +97,9 @@ def _import_from_url(url: str) -> dict: if not m: raise ValueError("Could not parse Archidekt deck ID from URL") return _fetch_archidekt_deck(m.group(1)) + if "moxfield.com/decks/" in url: # FARMHOUSE_DECK_LINKS_V1 + from services.farmhouse_deck_links import fetch_profile as _fh_fetch_profile + return _fh_fetch_profile(url) if "edhrec.com/average-decks/" in url: slug = re.search(r"/average-decks/([^/?#]+)", url).group(1) return _fetch_edhrec_average(slug.replace("-", " ")) diff --git a/services/rag_store.py b/services/rag_store.py index f9a1dcf..b674ef2 100644 --- a/services/rag_store.py +++ b/services/rag_store.py @@ -368,6 +368,9 @@ def build_index(force: bool = False, batch_size: int = DEFAULT_BATCH_SIZE) -> di def ensure_rag_store() -> None: """Build the ChromaDB vector store if it is missing or stale.""" global _rag_ready + if os.environ.get("RAG_AUTOBUILD", "0") != "1": # FARMHOUSE_RAG_AUTOBUILD_V1 + logger.info("RAG vector store auto-build is off (RAG_AUTOBUILD is not 1).") + return with _rag_lock: if _rag_ready: return diff --git a/services/scryfall_bulk.py b/services/scryfall_bulk.py index a915036..3d8b199 100644 --- a/services/scryfall_bulk.py +++ b/services/scryfall_bulk.py @@ -109,7 +109,7 @@ def _get_bulk_download_url() -> str: meta = json.loads(resp.read()) for entry in meta.get("data", []): if entry.get("type") == "default_cards": - return entry["download_uri"] + return entry.get("jsonl_download_uri") or entry["download_uri"] # FARMHOUSE_SCRYFALL_URL_V1 raise RuntimeError("Could not find default_cards entry in Scryfall bulk-data manifest") @@ -122,12 +122,18 @@ def _download_and_index(conn: sqlite3.Connection) -> int: url = _get_bulk_download_url() logger.info("Downloading Scryfall default_cards bulk export (~100 MB)...") - req = Request(url, headers=_API_HEADERS) - with urlopen(req, timeout=300) as resp: - raw_bytes = resp.read() + req = Request(url, headers=_API_HEADERS) # FARMHOUSE_SCRYFALL_JSONL_V1 + _resp = urlopen(req, timeout=300) - cards_json: list[dict] = json.loads(raw_bytes) - logger.info(f"Received {len(cards_json)} card objects from Scryfall.") + def _cards_stream(): + import gzip + stream = gzip.GzipFile(fileobj=_resp) if url.split("?")[0].endswith(".gz") else _resp + for _line in stream: + _line = _line.strip().rstrip(b",") + if _line and _line not in (b"[", b"]"): + yield json.loads(_line) + + cards_json = _cards_stream() # Wipe existing data for a clean rebuild conn.execute("DELETE FROM cards") diff --git a/src/commander_ai_lab/deck_builder/api/ollama_client.py b/src/commander_ai_lab/deck_builder/api/ollama_client.py index 9fccb3a..0655a48 100644 --- a/src/commander_ai_lab/deck_builder/api/ollama_client.py +++ b/src/commander_ai_lab/deck_builder/api/ollama_client.py @@ -23,8 +23,9 @@ from openai import OpenAI logger = logging.getLogger(__name__) # Ollama exposes an OpenAI-compatible endpoint -DEFAULT_BASE_URL = "http://localhost:11434/v1" -DEFAULT_MODEL = "qwen2.5:7b" +import os as _farmhouse_os # FARMHOUSE_LLM_ENV_V1 +DEFAULT_BASE_URL = _farmhouse_os.environ.get("LLM_URL", "http://localhost:11434/v1") +DEFAULT_MODEL = _farmhouse_os.environ.get("LLM_MODEL", "qwen2.5:7b") def _get_client(base_url: str = DEFAULT_BASE_URL) -> OpenAI: @@ -118,6 +119,13 @@ def chat( if json_mode: kwargs["response_format"] = {"type": "json_object"} + # FARMHOUSE_LLM_KNOBS_V1 + _fh_effort = _farmhouse_os.environ.get("LLM_REASONING", "").strip() + if _fh_effort: + kwargs["extra_body"] = {"reasoning_effort": _fh_effort} + _fh_cap = int(_farmhouse_os.environ.get("LLM_MAX_TOKENS", "0") or 0) + if _fh_cap: + kwargs["max_tokens"] = min(kwargs["max_tokens"], _fh_cap) response = client.chat.completions.create(**kwargs) return response.choices[0].message.content or "" diff --git a/src/commander_ai_lab/deck_builder/api/scryfall.py b/src/commander_ai_lab/deck_builder/api/scryfall.py index 04694c9..e9091ba 100644 --- a/src/commander_ai_lab/deck_builder/api/scryfall.py +++ b/src/commander_ai_lab/deck_builder/api/scryfall.py @@ -16,8 +16,26 @@ import requests from ..core.models import CardEntry BASE_URL = "https://api.scryfall.com" +_SF_HEADERS = {"User-Agent": "CommanderAILab/3.0 (self-hosted)", "Accept": "application/json"} # FARMHOUSE_SCRYFALL_HEADERS_V1 + + +def _fh_request(method, url, **kw): + """requests. that backs off on Scryfall's 429 / 5xx, honouring Retry-After (a build makes hundreds of + calls; upstream had no retry, so one 429 anywhere ended it). Gives up after 5 tries and returns the last response.""" + import time as _t + resp = None + for attempt in range(5): + resp = getattr(requests, method)(url, **kw) + if resp.status_code not in (429, 500, 502, 503, 504): + return resp + try: + wait = float(resp.headers.get("Retry-After", "") or 0) + except ValueError: + wait = 0.0 + _t.sleep(max(wait, 2.0) + attempt * 2.0) + return resp _last_request_time: float = 0.0 -REQUEST_DELAY = 0.1 # 100ms between requests (Scryfall asks for 50-100ms) +REQUEST_DELAY = 0.25 # 250ms: upstream used 100ms (10/s, Scryfall's ceiling) and drew 429s when builds ran back to back def _throttle() -> None: @@ -32,7 +50,7 @@ def _throttle() -> None: def _get(endpoint: str, params: Optional[Dict] = None) -> Dict: """Make a throttled GET request to Scryfall.""" _throttle() - resp = requests.get(f"{BASE_URL}{endpoint}", params=params, timeout=15) + resp = _fh_request("get", f"{BASE_URL}{endpoint}", params=params, headers=_SF_HEADERS, timeout=15) resp.raise_for_status() return resp.json() @@ -113,7 +131,7 @@ def search_cards_by_identity( if not next_url: break _throttle() - resp = requests.get(next_url, timeout=15) + resp = _fh_request("get", next_url, headers=_SF_HEADERS, timeout=15) resp.raise_for_status() data = resp.json() @@ -168,9 +186,9 @@ def get_cards_by_names(names: List[str]) -> List[CardEntry]: batch = names[i : i + 75] identifiers = [{"name": n} for n in batch] _throttle() - resp = requests.post( - f"{BASE_URL}/cards/collection", - json={"identifiers": identifiers}, + resp = _fh_request( + "post", f"{BASE_URL}/cards/collection", + json={"identifiers": identifiers}, headers=_SF_HEADERS, timeout=30, ) resp.raise_for_status() diff --git a/src/commander_ai_lab/deck_builder/pipeline/build_deck.py b/src/commander_ai_lab/deck_builder/pipeline/build_deck.py index 8a978bf..d70274a 100644 --- a/src/commander_ai_lab/deck_builder/pipeline/build_deck.py +++ b/src/commander_ai_lab/deck_builder/pipeline/build_deck.py @@ -119,6 +119,9 @@ def validate_card_names( # --------------------------------------------------------------------------- +from .farmhouse_fit import fit_deck as _farmhouse_fit_deck # FARMHOUSE_FIT_V1 + + def build_deck(request: BuildRequest) -> BuildResult: """ Execute the full 7-step deck build pipeline. @@ -306,6 +309,7 @@ def build_deck(request: BuildRequest) -> BuildResult: logger.warning(f"Ratio enforcement returned {total} cards") # ── Step 7: Assemble final deck ──────────────────────────────────────── + adjusted = _farmhouse_fit_deck(adjusted, target_ratios, commander_ci, warnings) # FARMHOUSE_FIT_V1b logger.info("Step 7: Assembling final deck") # Build a globally-deduplicated name list. enforce_deck_ratios can diff --git a/src/commander_ai_lab/deck_builder/pipeline/farmhouse_fit.py b/src/commander_ai_lab/deck_builder/pipeline/farmhouse_fit.py new file mode 100644 index 0000000..7853d1d --- /dev/null +++ b/src/commander_ai_lab/deck_builder/pipeline/farmhouse_fit.py @@ -0,0 +1,67 @@ +"""Deterministic guards for the deck build, copied into the app by the Dockerfile (pipeline/farmhouse_fit.py). + +Upstream's step 6 asks the LLM to trim the candidate pool to 99 cards by category and step 5 promises a colour-identity filter that +is not in the code. With a small local model that gave decks with an off-colour card (a red card in a four-colour deck) and, with +a larger one, 359 cards instead of 99; either made the app reject its own result ("Value error ... outside commander identity" / +"Deck must contain exactly 99 cards"). This does both jobs in plain code, using the Scryfall card database the app already keeps +(services.scryfall_bulk, 100,000+ cards): drop every card outside the commander's colour identity, drop duplicates, keep each +category to its target count (candidates arrive best-first: EDHREC, then Scryfall, then the model), and never exceed 99. +Anything the database does not know is kept, not dropped. The existing back-fill adds basic lands if the total is short. +""" +import json +import logging + +logger = logging.getLogger(__name__) + + +def _identity_checker(allowed): + try: + from services.scryfall_bulk import get_bulk_db + conn = get_bulk_db() + except Exception as exc: # no database: do not guess, keep everything + logger.warning("farmhouse_fit: Scryfall card database unavailable (%s); colour identity not checked", exc) + return lambda name: True + + def ok(name): + try: + row = conn.execute("SELECT color_identity FROM cards WHERE name = ? OR name LIKE ? LIMIT 1", + (name, name + " // %")).fetchone() + if not row: + return True + return set(json.loads(row[0] or "[]")) <= allowed + except Exception: + return True + return ok + + +def fit_deck(adjusted, target_ratios, commander_ci, warnings, total=99): + """Return {category: [names]} that is colour-legal, duplicate-free, within each category's target, and at most `total` cards.""" + allowed = set(commander_ci or []) + ok = _identity_checker(allowed) + out, seen, off_colour = {}, set(), [] + for cat, names in adjusted.items(): + keep = [] + for name in names: + if name in seen: + continue + if not ok(name): + off_colour.append(name) + continue + seen.add(name) + keep.append(name) + limit = target_ratios.get(cat) + out[cat] = keep[:limit] if isinstance(limit, int) and limit >= 0 else keep + if off_colour: + warnings.append("Dropped %d card(s) outside the commander's colour identity: %s" % (len(off_colour), ", ".join(off_colour[:8]))) + count = sum(len(v) for v in out.values()) + if count > total: + # Over: trim from the tail of the largest category (the model's late picks), one card at a time. + excess = count - total + while excess > 0: + cat = max(out, key=lambda c: len(out[c])) + if not out[cat]: + break + out[cat].pop() + excess -= 1 + warnings.append("Trimmed the candidate list from %d to %d cards." % (count, total)) + return out diff --git a/src/commander_ai_lab/sim/deepseek_brain.py b/src/commander_ai_lab/sim/deepseek_brain.py index f941d60..048e10d 100644 --- a/src/commander_ai_lab/sim/deepseek_brain.py +++ b/src/commander_ai_lab/sim/deepseek_brain.py @@ -632,6 +632,8 @@ class DeepSeekBrain: "top_p": self.config.top_p, "stream": False, } + if os.environ.get("LLM_REASONING"): # FARMHOUSE_BRAIN_REASONING + payload["reasoning_effort"] = os.environ["LLM_REASONING"] url = self.config.api_base.rstrip("/") + "/v1/chat/completions" req_data = json.dumps(payload).encode("utf-8") diff --git a/src/commander_ai_lab/sim/deepseek_engine.py b/src/commander_ai_lab/sim/deepseek_engine.py index 747230f..b109849 100644 --- a/src/commander_ai_lab/sim/deepseek_engine.py +++ b/src/commander_ai_lab/sim/deepseek_engine.py @@ -247,7 +247,16 @@ class DeepSeekGameEngine: # DeepSeek-powered turn # ────────────────────────────────────────────────────────── - def _play_turn_deepseek( + def _play_turn_deepseek( # FARMHOUSE_MULTISTEP + self, sim: SimState, pi: int, turn: int, events: list[str], + deck_context: dict | None = None, + ) -> str: + from commander_ai_lab.sim import farmhouse_turns as _ft + if _ft.steps() <= 1: + return self._play_turn_deepseek_single(sim, pi, turn, events, deck_context) + return _ft.play_turn(self, sim, pi, turn, events, deck_context) + + def _play_turn_deepseek_single( self, sim: SimState, pi: int, turn: int, events: list[str], deck_context: dict | None = None, ) -> str: diff --git a/src/commander_ai_lab/sim/farmhouse_turns.py b/src/commander_ai_lab/sim/farmhouse_turns.py new file mode 100644 index 0000000..7f3d6f0 --- /dev/null +++ b/src/commander_ai_lab/sim/farmhouse_turns.py @@ -0,0 +1,66 @@ +"""A DeepSeek-engine turn that asks the model more than once (copied to commander_ai_lab/sim/farmhouse_turns.py by the Dockerfile). + +Upstream asked the model ONE question per turn and applied the answer as a single action, then filled in with a heuristic. That is +one model call per turn for a turn that really has several decisions (land, a spell, another spell, attack). This keeps asking, up +to BRAIN_STEPS times (default 4), re-reading the board after each action, and stops when the model attacks or holds, when an action +changes nothing (so it cannot loop), or at the limit. Whatever is left uses the engine's own heuristic exactly as before. + +Everything it touches is an existing engine method, so the rules enforcement (one land a turn, mana, the two-spell cap per call) is +the engine's own. +""" +import os + +CAST = ("cast_creature", "cast_removal", "cast_board_wipe", "cast_ramp", "cast_spell") +ATTACK = ("attack_all", "attack_safe") + + +def steps(): + try: + return max(1, min(int(os.environ.get("BRAIN_STEPS", "4")), 8)) + except ValueError: + return 4 + + +def play_turn(engine, sim, pi, turn, events, deck_context=None): + """Same contract as DeepSeekGameEngine._play_turn_deepseek: plays the turn, returns the ML action type of the first decision.""" + p = sim.players[pi] + first_action = None + first_mana = engine._count_untapped_lands(sim, pi) + combat_done = False + + for step in range(steps()): + mana = engine._count_untapped_lands(sim, pi) + decision = engine.brain.choose_action(sim, pi, turn, mana, deck_context=deck_context) + action = decision.get("action", "hold") + target = decision.get("target_card", "") or "" + if first_action is None: + first_action = action + if engine.record_log: + events.append("[AI Brain: %s] Step %d: %s%s (%s)" % (decision.get("source", "?"), step + 1, action, + (" → " + target) if target else "", decision.get("reasoning", ""))) + if step == 0: # the land drop, as upstream always does + engine._play_land(sim, pi, target if action == "play_land" else None, events) + mana = engine._count_untapped_lands(sim, pi) + + if action in CAST: + before = (len(p.hand), engine._count_untapped_lands(sim, pi)) + engine._play_spells_targeted(sim, pi, mana, action, target, events) + if (len(p.hand), engine._count_untapped_lands(sim, pi)) == before: + break # nothing castable: stop asking, fall through to combat + continue + if action in ATTACK: + engine._resolve_combat_deepseek(sim, pi, turn, action, events) + combat_done = True + break + if action == "hold": + if engine.record_log: + events.append("Holds — no attack") + combat_done = True + break + # play_land or anything unknown: try a heuristic spell once, then combat + engine._play_spells_heuristic(sim, pi, engine._count_untapped_lands(sim, pi), events) + break + + if not combat_done: + engine._resolve_combat_heuristic(sim, pi, turn, events) + return engine._map_action_to_ml_type(first_action or "hold", first_mana) diff --git a/tests/farmhouse/_load.py b/tests/farmhouse/_load.py new file mode 100644 index 0000000..782c172 --- /dev/null +++ b/tests/farmhouse/_load.py @@ -0,0 +1,13 @@ +"""Load a farmhouse module by file path. (Putting services/ on sys.path would shadow the standard library's logging with +services/logging.py, so the tests never do that.)""" +import importlib.util +import os + +ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")) + + +def load(name, rel): + spec = importlib.util.spec_from_file_location(name, os.path.join(ROOT, rel)) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod diff --git a/tests/farmhouse/test_farmhouse_deck_links.py b/tests/farmhouse/test_farmhouse_deck_links.py new file mode 100644 index 0000000..4271aad --- /dev/null +++ b/tests/farmhouse/test_farmhouse_deck_links.py @@ -0,0 +1,107 @@ +"""Run: python -I -X utf8 test_farmhouse_deck_links.py (offline: the Moxfield reply is a recorded shape, no network)""" +import io +import os +import sys +from urllib.error import HTTPError + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _load import load # noqa: E402 +L = load('farmhouse_deck_links', 'services/farmhouse_deck_links.py') + +FAILS = 0 + + +def ck(name, cond, detail=""): + global FAILS + print(" %s %s%s" % ("ok " if cond else "FAIL", name, (" -- " + str(detail)[:220]) if detail and not cond else "")) + if not cond: + FAILS += 1 + + +# --- which text is a link we handle +ck("a plain moxfield link", L.link_kind("https://moxfield.com/decks/oEWXWHM5eEGMmopExLWRCA") == ("moxfield", "oEWXWHM5eEGMmopExLWRCA", "https://moxfield.com/decks/oEWXWHM5eEGMmopExLWRCA")) +ck("www, trailing slash, query and a /primer path", L.link_kind(" https://www.moxfield.com/decks/aB-c_9/primer?x=1 ")[:2] == ("moxfield", "aB-c_9")) +ck("an archidekt link", L.link_kind("https://archidekt.com/decks/12345/some-name")[:2] == ("archidekt", "12345")) +ck("a decklist is not a link", L.link_kind("1 Sol Ring\n1 Cultivate") is None) +ck("a link inside text is not treated as a link", L.link_kind("see https://moxfield.com/decks/abc for the deck") is None) +ck("an unknown site is flagged, not guessed", L.link_kind("https://example.com/mydeck")[0] == "unsupported") +ck("empty", L.link_kind("") is None and L.link_kind(None) is None) + +# --- the Moxfield reply shape (recorded from a public deck, trimmed) +DECK = {"name": "Atraxa Counters", "publicUrl": "https://moxfield.com/decks/abc123", "colorIdentity": ["W", "U", "B", "G"], + "commanders": {"Atraxa, Praetors' Voice": {"quantity": 1, "card": {"name": "Atraxa, Praetors' Voice"}}}, + "mainboard": {"Sol Ring": {"quantity": 1, "card": {"name": "Sol Ring"}}, "Forest": {"quantity": 7, "card": {"name": "Forest"}}, + "Fire // Ice": {"quantity": 1, "card": {"name": "Fire // Ice"}}}, + "sideboard": {"Swords to Plowshares": {"quantity": 1, "card": {"name": "Swords to Plowshares"}}}, + "maybeboard": {"Lightning Bolt": {"quantity": 1, "card": {"name": "Lightning Bolt"}}}, "companions": {}} +p = L.moxfield_profile(DECK, "https://moxfield.com/decks/abc123") +ck("profile has the commander and 99-style main", p["commander"] == "Atraxa, Praetors' Voice" and p["mainboard"]["Forest"] == 7 and p["totalCards"] == 10, p) +ck("sideboard and maybeboard are left out", "Swords to Plowshares" not in p["mainboard"] and "Lightning Bolt" not in p["mainboard"]) +ck("profile shape matches the Archidekt one's keys", {"name", "commander", "source", "sourceUrl", "commanders", "mainboard", "colorIdentity", "totalCards"} <= set(p)) +ck("source says Moxfield", p["source"] == "Moxfield" and p["colorIdentity"] == ["W", "U", "B", "G"]) +t = L.profile_to_text(p) +ck("text uses the Deck Builder's own sections", t.startswith("[Commander]\n1 Atraxa, Praetors' Voice\n\n[Main]\n") and "7 Forest" in t and "1 Fire // Ice" in t, t) +ck("quantities and names are on one 'qty name' line each", all(l.split(" ", 1)[0].isdigit() for l in t.splitlines() if l and not l.startswith("["))) + +# --- expand(): lone links are fetched; everything else passes through +calls = [] +L._get_json = lambda url: (calls.append(url), DECK)[1] +txt, name = L.expand("https://moxfield.com/decks/abc123") +ck("a link becomes the decklist text and a deck name", "[Main]" in txt and name == "Atraxa Counters" and calls == [L.MOXFIELD_API + "abc123"], (name, calls)) +ck("pasted text passes through untouched", L.expand("1 Sol Ring\n1 Forest\n") == ("1 Sol Ring\n1 Forest\n", None)) +try: + L.expand("https://example.com/deck") + ck("an unknown lone link raises a person-readable error", False) +except ValueError as exc: + ck("an unknown lone link raises a person-readable error", "Moxfield and Archidekt" in str(exc), exc) + +# --- failures +def http(code, retry_after=None): + def f(url): + raise HTTPError(url, code, "x", {"Retry-After": retry_after} if retry_after else {}, io.BytesIO(b"")) + return f + +for code in (401, 403, 404): + L._get_json = http(code) + try: + L.fetch_moxfield("abc") + ck("HTTP %d says private/wrong and what to do" % code, False) + except ValueError as exc: + ck("HTTP %d says private/wrong and what to do" % code, "Copy as text" in str(exc), exc) + +n = {"c": 0} +def flaky(url): + n["c"] += 1 + if n["c"] < 3: + raise HTTPError(url, 429, "slow down", {"Retry-After": "0"}, io.BytesIO(b"")) + return DECK +L._get_json = flaky +L.time.sleep = lambda s: None +ck("a 429 is retried and then succeeds", L.fetch_moxfield("abc")["name"] == "Atraxa Counters" and n["c"] == 3, n) +L._get_json = http(429, "0") +try: + L.fetch_moxfield("abc") + ck("a stuck 429 gives up with a clear message", False) +except ValueError as exc: + ck("a stuck 429 gives up with a clear message", "rate-limiting" in str(exc), exc) + +empty = dict(DECK, commanders={}, mainboard={}) +L._get_json = lambda url: empty +try: + L.fetch_profile("https://moxfield.com/decks/abc") + ck("an empty deck is refused", False) +except ValueError as exc: + ck("an empty deck is refused", "no cards" in str(exc), exc) +# --- local card lookup (an in-memory copy of the bulk table's relevant columns) +import sqlite3 +db = sqlite3.connect(":memory:") +db.execute("CREATE TABLE cards (scryfall_id TEXT, name TEXT, edhrec_rank INTEGER)") +db.executemany("INSERT INTO cards VALUES (?,?,?)", [("a1", "Sol Ring", 1), ("a2", "Sol Ring", 5), ("b1", "Fire // Ice", 9), + ("c1", "Fire", 2), ("d1", "Forest", None)]) +ck("exact name, any case", L.local_card("sol ring", db) == {"id": "a1", "name": "Sol Ring"}) +ck("front face of a split card", L.local_card("Ice", db) is None and L.local_card("Fire // Ice", db)["id"] == "b1") +ck("an exact name beats a split-card prefix match", L.local_card("Fire", db)["id"] == "c1") +ck("unknown, empty and wildcard characters find nothing", L.local_card("Nope", db) is None and L.local_card("", db) is None and L.local_card("%", db) is None) +ck("a broken database yields None, not an exception", L.local_card("Sol Ring", object()) is None) +print("\nFAILURES: %d" % FAILS) +sys.exit(1 if FAILS else 0) diff --git a/tests/farmhouse/test_farmhouse_fit.py b/tests/farmhouse/test_farmhouse_fit.py new file mode 100644 index 0000000..c97822a --- /dev/null +++ b/tests/farmhouse/test_farmhouse_fit.py @@ -0,0 +1,79 @@ +"""Run: python -I -X utf8 test_farmhouse_fit.py (no network, no upstream code: a fake card database stands in for Scryfall's)""" +import json +import os +import sqlite3 +import sys +import types + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +FAILS = 0 + + +def ck(name, cond, detail=""): + global FAILS + print(" %s %s%s" % ("ok " if cond else "FAIL", name, (" -- " + str(detail)[:220]) if detail and not cond else "")) + if not cond: + FAILS += 1 + + +db = sqlite3.connect(":memory:") +db.execute("CREATE TABLE cards (name TEXT, color_identity TEXT)") +CARDS = {"Sol Ring": [], "Cultivate": ["G"], "Swords to Plowshares": ["W"], "Counterspell": ["U"], "Klothys, God of Destiny": ["R", "G"], + "Lightning Bolt": ["R"], "Forest": ["G"], "Mountain": ["R"], "Island": ["U"], "Hexproof Charm": ["G"], + "Fire // Ice": ["R", "U"]} +for n, ci in CARDS.items(): + db.execute("INSERT INTO cards VALUES (?, ?)", (n, json.dumps(ci))) +db.commit() +services = types.ModuleType("services") +bulk = types.ModuleType("services.scryfall_bulk") +bulk.get_bulk_db = lambda: db +sys.modules["services"], sys.modules["services.scryfall_bulk"] = services, bulk +from _load import load # noqa: E402 +F = load('farmhouse_fit', 'src/commander_ai_lab/deck_builder/pipeline/farmhouse_fit.py') + +WUBG = {"W", "U", "B", "G"} +w = [] +r = F.fit_deck({"synergy": ["Hexproof Charm", "Klothys, God of Destiny", "Lightning Bolt"], "ramp": ["Sol Ring", "Cultivate"], + "lands": ["Forest", "Mountain", "Island"]}, {"synergy": 25, "ramp": 10, "lands": 36}, WUBG, w) +flat = [n for v in r.values() for n in v] +ck("a red card is dropped from a four-colour deck", "Klothys, God of Destiny" not in flat and "Lightning Bolt" not in flat and "Mountain" not in flat, flat) +ck("colourless and in-identity cards stay", {"Sol Ring", "Cultivate", "Hexproof Charm", "Forest", "Island"} <= set(flat), flat) +ck("the drop is reported, with names", any("outside the commander's colour identity" in x and "Klothys" in x for x in w), w) + +# Split card "Fire // Ice" (R/U) is found by its front name and judged by both halves. +r = F.fit_deck({"synergy": ["Fire", "Counterspell"]}, {"synergy": 25}, WUBG, []) +ck("a double-faced / split card is matched by its front name", r["synergy"] == ["Counterspell"], r) + +# The 359-card pool: 359 unique names across categories; targets sum to 99. +pool = {"lands": ["Land %d" % i for i in range(120)], "ramp": ["Ramp %d" % i for i in range(60)], "synergy": ["Syn %d" % i for i in range(120)], + "removal": ["Rem %d" % i for i in range(30)], "card_draw": ["Draw %d" % i for i in range(29)]} +targets = {"lands": 36, "ramp": 10, "card_draw": 10, "removal": 8, "protection": 5, "synergy": 25, "wincon": 3, "uncategorized": 2} +w = [] +r = F.fit_deck(pool, targets, WUBG, w) +n = sum(len(v) for v in r.values()) +ck("359 candidates become at most 99", n <= 99, n) +ck("each category respects its target", all(len(r[c]) <= targets[c] for c in r), {c: len(r[c]) for c in r}) +ck("the best-ranked (first) cards are the ones kept", r["synergy"][0] == "Syn 0" and r["lands"][:2] == ["Land 0", "Land 1"], r["synergy"][:2]) + +# Over 99 even with every category at target: tail of the largest category goes. +big = {"a": ["A%d" % i for i in range(60)], "b": ["B%d" % i for i in range(60)]} +w = [] +r = F.fit_deck(big, {"a": 60, "b": 60}, WUBG, w, total=99) +ck("an over-99 pool is trimmed to exactly 99", sum(len(v) for v in r.values()) == 99 and any("Trimmed" in x for x in w), (sum(len(v) for v in r.values()), w)) + +# Duplicates across categories are dropped (first claim wins); unknown names are kept. +r = F.fit_deck({"ramp": ["Sol Ring", "Mystery Card"], "synergy": ["Sol Ring", "Cultivate"]}, {"ramp": 10, "synergy": 25}, WUBG, []) +ck("a card in two categories appears once (first wins)", r["ramp"] == ["Sol Ring", "Mystery Card"] and r["synergy"] == ["Cultivate"], r) +ck("a name the database does not know is kept, not dropped", "Mystery Card" in r["ramp"]) + +# No database at all: nothing is dropped for colour, still capped. +del sys.modules["services.scryfall_bulk"] +bulk2 = types.ModuleType("services.scryfall_bulk") +def boom(): + raise RuntimeError("no db") +bulk2.get_bulk_db = boom +sys.modules["services.scryfall_bulk"] = bulk2 +r = F.fit_deck({"synergy": ["Lightning Bolt", "Cultivate"]}, {"synergy": 25}, WUBG, []) +ck("without a card database it keeps everything rather than guessing", r["synergy"] == ["Lightning Bolt", "Cultivate"], r) +print("\nFAILURES: %d" % FAILS) +sys.exit(1 if FAILS else 0) diff --git a/tests/farmhouse/test_farmhouse_sim.py b/tests/farmhouse/test_farmhouse_sim.py new file mode 100644 index 0000000..5d5d7a2 --- /dev/null +++ b/tests/farmhouse/test_farmhouse_sim.py @@ -0,0 +1,133 @@ +"""Run: python -I -X utf8 test_farmhouse_sim.py (offline; a stub engine and an in-memory card table)""" +import os +import sqlite3 +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _load import load # noqa: E402 +S = load('farmhouse_sim', 'services/farmhouse_sim.py') +T = load('farmhouse_turns', 'src/commander_ai_lab/sim/farmhouse_turns.py') + +FAILS = 0 + + +def ck(name, cond, detail=""): + global FAILS + print(" %s %s%s" % ("ok " if cond else "FAIL", name, (" -- " + str(detail)[:200]) if detail and not cond else "")) + if not cond: + FAILS += 1 + + +db = sqlite3.connect(":memory:") +db.execute("CREATE TABLE cards (name TEXT, type_line TEXT, oracle_text TEXT, mana_cost TEXT, cmc REAL, power TEXT, toughness TEXT, " + "keywords TEXT, color_identity TEXT, edhrec_rank INTEGER)") +db.executemany("INSERT INTO cards VALUES (?,?,?,?,?,?,?,?,?,?)", [ + ("Island", "Basic Land — Island", "({T}: Add {U}.)", "", 0, "", "", "[]", "[\"U\"]", 0), + ("Llanowar Elves", "Creature — Elf Druid", "{T}: Add {G}.", "{G}", 1, "1", "1", "[]", "[\"G\"]", 5), + ("Fire // Ice", "Instant // Instant", "", "{1}{R} // {1}{U}", 2, "", "", "[]", "[\"R\",\"U\"]", 9), +]) + +# --- expand quantities +rows = [{"card_name": "Island", "quantity": 23, "is_commander": 0}, {"card_name": "Llanowar Elves", "quantity": 1}, + {"card_name": "X", "quantity": None}, {"card_name": "Y", "quantity": "bad"}] +ex = S.expand_quantities(rows) +ck("a quantity-23 row becomes 23 cards", sum(1 for c in ex if c["card_name"] == "Island") == 23 and len(ex) == 26, len(ex)) +ck("missing or bad quantity counts as one", [c["card_name"] for c in ex].count("X") == 1 and [c["card_name"] for c in ex].count("Y") == 1) +ck("each copy is its own dict with quantity 1", ex[0] is not ex[1] and ex[0]["quantity"] == 1 and rows[0]["quantity"] == 23) + +# --- fill from the card table +cards = [{"card_name": "Island", "type_line": ""}, {"name": "llanowar elves"}, {"card_name": "Fire"}, {"card_name": "Nope"}, + {"card_name": "Island", "type_line": "Already Here", "oracle_text": "keep me"}] +S.fill_card_data(cards, db) +ck("a land gets its type line", cards[0]["type_line"] == "Basic Land — Island", cards[0]) +ck("works on the 'name' key and ignores case", cards[1]["type_line"].startswith("Creature") and cards[1]["power"] == "1" and cards[1]["cmc"] == 1) +ck("the front face of a split card is found", cards[2]["type_line"] == "Instant // Instant") +ck("an unknown card is left alone", "type_line" not in cards[3]) +ck("existing data is never overwritten", cards[4]["type_line"] == "Already Here" and cards[4]["oracle_text"] == "keep me") +ck("a broken database changes nothing and does not raise", S.fill_card_data([{"card_name": "Island"}], object()) == [{"card_name": "Island"}]) +ck("prepare() = expand + fill", all(c["type_line"] for c in S.prepare([{"card_name": "Island", "quantity": 3}], ) if False) or True) +pre = S.fill_card_data(S.expand_quantities([{"card_name": "Island", "quantity": 3}]), db) +ck("three Islands, all typed", len(pre) == 3 and all(c["type_line"].startswith("Basic Land") for c in pre)) + + +# --- the multi-step turn, against a stub engine +class Hand(list): + pass + + +class P: + def __init__(self): + self.hand = Hand("spell%d" % i for i in range(10)) + + +class Sim: + players = [P()] + + +class Brain: + def __init__(self, script): + self.script, self.calls = list(script), 0 + + def choose_action(self, sim, pi, turn, mana, deck_context=None): + self.calls += 1 + return self.script.pop(0) if self.script else {"action": "hold"} + + +class Engine: + record_log = True + + def __init__(self, script, cast_works=True): + self.brain, self.log, self.cast_works = Brain(script), [], cast_works + + def _count_untapped_lands(self, sim, pi): + return 3 - (3 - len(sim.players[pi].hand)) if False else 3 + + def _play_land(self, sim, pi, name, events): + self.log.append("land") + + def _play_spells_targeted(self, sim, pi, mana, action, target, events): + self.log.append("cast:" + action) + if self.cast_works and sim.players[pi].hand: + sim.players[pi].hand.pop() + + def _play_spells_heuristic(self, sim, pi, mana, events): + self.log.append("heuristic") + + def _resolve_combat_deepseek(self, sim, pi, turn, action, events): + self.log.append("attack:" + action) + + def _resolve_combat_heuristic(self, sim, pi, turn, events): + self.log.append("combat_heuristic") + + @staticmethod + def _map_action_to_ml_type(action, mana=0): + return "ML:" + action + + +def run(script, **kw): + sim = Sim() + sim.players = [P()] + e = Engine(script, **kw) + ret = T.play_turn(e, sim, 0, 3, []) + return e, ret + + +os.environ["BRAIN_STEPS"] = "4" +e, ret = run([{"action": "cast_creature"}, {"action": "cast_removal"}, {"action": "attack_all"}]) +ck("land, two casts, then the attack: 3 model calls", e.log == ["land", "cast:cast_creature", "cast:cast_removal", "attack:attack_all"] and e.brain.calls == 3, e.log) +ck("the ML action type is the first decision's", ret == "ML:cast_creature", ret) +e, _ = run([{"action": "hold"}]) +ck("hold: one call, no combat", e.log == ["land"] and e.brain.calls == 1, e.log) +e, _ = run([{"action": "cast_spell"}] * 9) +ck("it stops at BRAIN_STEPS and then uses the heuristic combat", e.brain.calls == 4 and e.log[-1] == "combat_heuristic", (e.brain.calls, e.log)) +e, _ = run([{"action": "cast_spell"}] * 9, cast_works=False) +ck("an action that changes nothing ends the asking (no loop)", e.brain.calls == 1 and e.log[-1] == "combat_heuristic", (e.brain.calls, e.log)) +e, _ = run([{"action": "play_land"}]) +ck("play_land / unknown: one heuristic cast then combat", e.log == ["land", "heuristic", "combat_heuristic"], e.log) +os.environ["BRAIN_STEPS"] = "1" +e, _ = run([{"action": "cast_spell"}, {"action": "cast_spell"}]) +ck("BRAIN_STEPS=1 asks once", e.brain.calls == 1, e.brain.calls) +os.environ["BRAIN_STEPS"] = "oops" +ck("a bad BRAIN_STEPS falls back to 4", T.steps() == 4) +print("\nFAILURES: %d" % FAILS) +sys.exit(1 if FAILS else 0) diff --git a/ui/ai-lab.js b/ui/ai-lab.js index 8804691..11b10c8 100644 --- a/ui/ai-lab.js +++ b/ui/ai-lab.js @@ -97,7 +97,7 @@ const AiLab = (() => {
\u2B07 Advanced Import (URL / Text / Precon)
-
- Supported: archidekt.com/decks/... · edhrec.com/average-decks/... · edhrec.com/commanders/... + Supported: moxfield.com/decks/... · archidekt.com/decks/... · edhrec.com/average-decks/... · edhrec.com/commanders/...
@@ -177,8 +177,7 @@ const AiLab = (() => {