FarmHouse: run Python-only on a local LLM; simulator plays real games
Starts from upstream 54cc5e9. Cards reach the simulator with type and text (filled from the Scryfall bulk DB) and the right quantities; the LLM engine asks several questions per turn and with reasoning off; Moxfield and Archidekt deck links work in every import and card names resolve locally; deck builds are fitted to the commander's colour identity; Scryfall format/User-Agent/429 handling fixed; coach init fixed. See FARMHOUSE-CHANGES.md.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Er49soaMyG7Yburq57PeB
This commit is contained in:
1 parent
54cc5e9b2e
commit
fee8ded52d
26 files changed
+866
-28
No files matched your search
@@ -0,0 +1,62 @@
|
||||
# FarmHouse changes
|
||||
|
||||
This branch (`farmhouse`) is `KoalaTrapLord/commander-ai-lab` at commit `54cc5e9` plus the changes below. They were made to run the app
|
||||
**Python-only** (no Forge/Java, no PyTorch) in a Kubernetes namespace, with a local LLM behind an OpenAI-compatible gateway instead of
|
||||
Anthropic/DeepSeek, and they were each found by actually running the app, not by reading it. Written 2026-10-26.
|
||||
|
||||
Every change started life as an anchored text patch (`patch_app.py` in the farmhouse-os repo, which still builds the deployed image from
|
||||
the upstream commit); here they are ordinary source edits. Lines carry a `# FARMHOUSE_...` marker where the patch left one.
|
||||
|
||||
## What was wrong, and what changed
|
||||
|
||||
### The simulator played nothing (the big one)
|
||||
| Problem | Change |
|
||||
|---|---|
|
||||
| Every card reached the simulator with **no type line and no rules text**. The loaders join the user's *collection* and a *card_records* table (both empty on a fresh install) and the Forge card cache that was meant to fill the gap is not installed. A land was indistinguishable from a creature: an 8-game batch cast 0 spells, dealt 0 damage and lost every game. | `services/farmhouse_sim.py` `fill_card_data` fills type, text, mana cost, cmc, P/T, keywords and colour identity from the 113k-card Scryfall bulk database. Wired into `services/forge_runner.py` (batch) and `routes/deepseek.py` (`_fetch_deck_card_data`, which the Monte Carlo and single-deck routes use). |
|
||||
| The batch loader **did not repeat cards by quantity**: a 100-card deck loaded as 66-84 cards with *one* Island. | `expand_quantities` (same module). |
|
||||
| The LLM engine asked the model **once per turn** and applied the answer as one action. | `src/commander_ai_lab/sim/farmhouse_turns.py`: up to `BRAIN_STEPS` (default 4) questions per turn (land, spell, spell, attack), re-reading the board after each; stops on attack/hold/no change. `BRAIN_STEPS=1` restores the original. |
|
||||
| A model that "thinks" spent ~46 s on a one-line move. | `deepseek_brain.py` sends `reasoning_effort` from `LLM_REASONING` (46 s -> 1.8 s per move). |
|
||||
| The engine dropdown defaulted to **Forge**, which cannot run without Java. | `ui/ai-lab.js`: only "LLM opponents" is offered. |
|
||||
|
||||
After the fixes: a 40-game Monte Carlo takes 0.4 s with real numbers (10 spells and 12.7 damage per game); two LLM-piloted games took
|
||||
212 s with 49 model calls and no fallbacks.
|
||||
|
||||
### Deck import
|
||||
| Problem | Change |
|
||||
|---|---|
|
||||
| **Moxfield links were accepted nowhere**, and the Deck Builder import took only pasted text. | `services/farmhouse_deck_links.py`: Moxfield (public decks, `api2.moxfield.com`) and Archidekt links in the simulator URL import and in both Deck Builder imports (a lone link in the paste box). A private deck says to use Moxfield's Export > Copy as text. |
|
||||
| The Deck Builder import looked **every card up on scryfall.com**; after a few dozen it was rate-limited (HTTP 429) and the app reported the rest as "not found" (250-card deck: 55 matched; even plain `Sol Ring` failed). | `local_card()`: names resolve from the local bulk database first; the website is only the fallback. |
|
||||
|
||||
### Deck generation
|
||||
- `deck_builder/pipeline/farmhouse_fit.py`: the model's list is **filtered to the commander's colour identity** (from the bulk DB), de-duplicated,
|
||||
fitted to per-category targets and trimmed to 99 + commander. Before this the model returned off-colour cards and 359-card lists.
|
||||
- Scryfall bulk format changed (`jsonl_download_uri`, gzip JSONL; no `download_uri`).
|
||||
- Scryfall answers **HTTP 400 to the default `python-requests` User-Agent** (every lookup became "Commander not found"): a named UA is sent everywhere.
|
||||
- Scryfall 429s: requests back off on `Retry-After` and the pace is 0.25 s.
|
||||
- `LLM_REASONING` / `LLM_MAX_TOKENS` are sent by the deck builder and coach.
|
||||
|
||||
### Coach / startup
|
||||
- The coach was only initialised in `main()`, not when the app starts under uvicorn; `check_connection_sync()` was called but never defined;
|
||||
the coach used the first model the gateway listed rather than the configured one. All fixed.
|
||||
- Status label says "Language model" instead of the hardcoded "Claude Opus".
|
||||
- `RAG_AUTOBUILD` (default off in our deployment) controls whether the card-search index is built at startup (113k embeddings; about an hour).
|
||||
|
||||
## Settings (environment variables)
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `LLM_URL`, `LLM_MODEL` | OpenAI-compatible endpoint (`.../v1`) and model for the coach and deck builder |
|
||||
| `LLM_REASONING` (`none`), `LLM_MAX_TOKENS`, `LLM_TIMEOUT` | keep requests lean: reasoning off, cap on answer length |
|
||||
| `COACH_PROVIDER`, `DECK_GEN_PROVIDER` = `local`, `DECK_GEN_MODEL` | use the local model, not Anthropic |
|
||||
| `OLLAMA_BASE_URL`, `RAG_EMBEDDING_MODEL`, `RAG_AUTOBUILD` | card-search embeddings (one host only: vectors differ between hosts) |
|
||||
| `BRAIN_API_BASE` (**no** `/v1`; the client appends it), `BRAIN_MODEL`, `BRAIN_MAX_TOKENS`, `BRAIN_STEPS` | the simulator's LLM opponents |
|
||||
|
||||
## Tests
|
||||
`tests/farmhouse/` (standard library only, no network): `python -I -X utf8 tests/farmhouse/test_farmhouse_deck_links.py`, `..._sim.py`, `..._fit.py`.
|
||||
Run them with `-I` and from outside the package folders; do not put `services/` on `sys.path` (`services/logging.py` shadows the standard library).
|
||||
|
||||
## Things still worth doing
|
||||
- Run the LLM-opponent games **in parallel**: each move is a short prompt with a short answer, which a batching server handles well.
|
||||
- Opponents are the engine's heuristics; only the piloted seat uses the model.
|
||||
- Build the card-search index and measure it.
|
||||
- The upstream `LICENSE` is missing (`pyproject.toml` says MIT). Ask the author to add one before publishing a derivative widely.
|
||||
- Forge-based features (the Java simulator, training) remain unavailable by design.
|
||||
@@ -549,7 +549,7 @@ class CoachService:
|
||||
status.llmModels = []
|
||||
status.error = "Anthropic API key not configured. Set ANTHROPIC_API_KEY env var."
|
||||
else:
|
||||
llm_status = self.llm.check_connection()
|
||||
llm_status = self.llm.check_connection_sync() # FARMHOUSE_COACH_STATUS_V1 (was an un-awaited coroutine)
|
||||
status.llmConnected = llm_status.get("connected", False)
|
||||
status.llmModel = llm_status.get("active_model")
|
||||
status.llmModels = llm_status.get("models", [])
|
||||
|
||||
+20
-1
@@ -74,7 +74,7 @@ class LLMClient:
|
||||
try:
|
||||
status = await self.check_connection()
|
||||
if status.get("connected") and status.get("active_model"):
|
||||
self._resolved_model = status["active_model"]
|
||||
self._resolved_model = self.model if self.model in (status.get("models") or []) else status["active_model"] # FARMHOUSE_MODEL_PIN_V1
|
||||
logger.info("Resolved Ollama model: %s", self._resolved_model)
|
||||
return self._resolved_model
|
||||
except Exception:
|
||||
@@ -103,6 +103,14 @@ class LLMClient:
|
||||
}
|
||||
if force_json:
|
||||
body["response_format"] = {"type": "json_object"}
|
||||
# FARMHOUSE_LLM_KNOBS_V1 (see patch_app.py)
|
||||
import os as _fh_os
|
||||
_fh_effort = _fh_os.environ.get("LLM_REASONING", "").strip()
|
||||
if _fh_effort:
|
||||
body["reasoning_effort"] = _fh_effort
|
||||
_fh_cap = int(_fh_os.environ.get("LLM_MAX_TOKENS", "0") or 0)
|
||||
if _fh_cap:
|
||||
body["max_tokens"] = min(body["max_tokens"], _fh_cap)
|
||||
return body
|
||||
|
||||
# ── Response parsing ──────────────────────────────────────
|
||||
@@ -260,6 +268,17 @@ class LLMClient:
|
||||
f"LLM call failed after {LLM_MAX_RETRIES} attempts: {last_error}"
|
||||
)
|
||||
|
||||
def check_connection_sync(self) -> dict: # FARMHOUSE_LLM_SYNC_V1
|
||||
"""Blocking version of check_connection (upstream called it and never defined it)."""
|
||||
try:
|
||||
_r = httpx.get(f"{self.base_url}/models", timeout=5.0, headers={"Content-Type": "application/json"})
|
||||
_r.raise_for_status()
|
||||
_ids = [m.get("id", "unknown") for m in _r.json().get("data", [])]
|
||||
return {"connected": True, "models": _ids,
|
||||
"active_model": self.model if self.model in _ids else (_ids[0] if _ids else self.model)}
|
||||
except Exception as _e:
|
||||
return {"connected": False, "error": str(_e), "models": [], "active_model": None}
|
||||
|
||||
async def check_connection(self) -> dict:
|
||||
"""Check if Ollama is reachable. Returns status dict."""
|
||||
try:
|
||||
|
||||
+2
-1
@@ -62,6 +62,7 @@ async def _lifespan(application: FastAPI):
|
||||
CFG.precon_dir = _resolve_precon_dir()
|
||||
init_collection_db()
|
||||
validate_forge_config()
|
||||
init_coach_service() # FARMHOUSE_COACH_INIT_V1
|
||||
if not PRECON_INDEX:
|
||||
download_precon_database()
|
||||
if not COMMANDER_META:
|
||||
@@ -85,7 +86,7 @@ async def _lifespan(application: FastAPI):
|
||||
try:
|
||||
from services.rag_store import check_staleness, build_index
|
||||
info = check_staleness()
|
||||
if info.get("stale"):
|
||||
if info.get("stale") and os.environ.get("RAG_AUTOBUILD", "0") == "1": # FARMHOUSE_RAG_MONITOR_V1
|
||||
log.info("RAG staleness monitor: %s — triggering rebuild.", info["reason"])
|
||||
loop.run_in_executor(None, lambda: build_index(force=True))
|
||||
else:
|
||||
|
||||
+15
-3
@@ -888,7 +888,11 @@ async def import_decklist(deck_id: int, request: FastAPIRequest):
|
||||
"""
|
||||
deck = _get_deck_or_404(deck_id)
|
||||
body = await request.json()
|
||||
text = body.get('text', '')
|
||||
try: # FARMHOUSE_DECK_LINKS_V1b
|
||||
from services.farmhouse_deck_links import expand as _fh_expand
|
||||
text, _fh_name = _fh_expand(body.get('text', '') or body.get('url', ''))
|
||||
except ValueError as _fh_err:
|
||||
return JSONResponse({'error': str(_fh_err)}, status_code=400)
|
||||
clear_first = body.get('clearFirst', False)
|
||||
|
||||
if not text.strip():
|
||||
@@ -980,8 +984,12 @@ async def import_decklist_new(request: FastAPIRequest):
|
||||
Body: { "text": "...", "name": "My Deck" }
|
||||
"""
|
||||
body = await request.json()
|
||||
text = body.get('text', '')
|
||||
deck_name = body.get('name', '').strip()
|
||||
try: # FARMHOUSE_DECK_LINKS_V1c
|
||||
from services.farmhouse_deck_links import expand as _fh_expand
|
||||
text, _fh_name = _fh_expand(body.get('text', '') or body.get('url', ''))
|
||||
except ValueError as _fh_err:
|
||||
return JSONResponse({'error': str(_fh_err)}, status_code=400)
|
||||
deck_name = body.get('name', '').strip() or (_fh_name or '')
|
||||
|
||||
if not text.strip():
|
||||
return JSONResponse({'error': 'No decklist text provided'}, status_code=400)
|
||||
@@ -1064,6 +1072,10 @@ def _scryfall_fuzzy_lookup(name: str) -> Optional[dict]:
|
||||
Fuzzy-search Scryfall for a card name (used by the scanner pipeline).
|
||||
Returns the raw Scryfall JSON dict, or None on failure.
|
||||
"""
|
||||
from services.farmhouse_deck_links import local_card as _fh_local # FARMHOUSE_LOCAL_LOOKUP
|
||||
_fh_hit = _fh_local(name)
|
||||
if _fh_hit:
|
||||
return _fh_hit
|
||||
_scryfall_rate_limit()
|
||||
try:
|
||||
encoded = quote(name)
|
||||
|
||||
@@ -516,6 +516,8 @@ def _fetch_deck_card_data(deck_id):
|
||||
'keywords': r[7] or '',
|
||||
'mana_cost': r[8] or '',
|
||||
})
|
||||
from services.farmhouse_sim import fill_card_data as _fh_fill # FARMHOUSE_SIM_DATA_B
|
||||
_fh_fill(card_data)
|
||||
return deck_name, card_data
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
"""Deck LINKS for every deck import (copied into the app as services/farmhouse_deck_links.py by the Dockerfile).
|
||||
|
||||
Upstream could import from an Archidekt or EDHREC link on the simulator page only; the Deck Builder import took pasted text and
|
||||
nothing else, and Moxfield (where most people keep decks) was not accepted anywhere. This reads a Moxfield or Archidekt deck link and
|
||||
returns what the existing importers already understand: a profile for the simulator import, or plain "[Commander] / [Main]" text for
|
||||
the Deck Builder's two imports (so a link pasted into the same box works, and the card-by-card Scryfall matching is unchanged).
|
||||
|
||||
Moxfield has no official public API; api2.moxfield.com answers server-side requests for PUBLIC decks (tested from the cluster
|
||||
2026-10-26: HTTP 200, a named User-Agent is enough). A private or deleted deck comes back 401/403/404, and the message says so and
|
||||
says what to do instead (Moxfield: Export > Copy as text, then paste that). Nothing here logs in to anything.
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from urllib.error import HTTPError
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
UA = "CommanderAILab/3.0 (self-hosted)"
|
||||
MOXFIELD_API = "https://api2.moxfield.com/v2/decks/all/"
|
||||
|
||||
_MOX = re.compile(r"^https?://(?:www\.)?moxfield\.com/decks/([A-Za-z0-9_-]+)(?:[/?#].*)?$", re.I)
|
||||
_ARC = re.compile(r"^https?://(?:www\.)?archidekt\.com/decks/(\d+)(?:[/?#].*)?$", re.I)
|
||||
|
||||
UNSUPPORTED = ("That link is not one I can read. Moxfield and Archidekt deck links work; for anything else, paste the decklist as text "
|
||||
"(Moxfield: Export > Copy as text).")
|
||||
|
||||
|
||||
def link_kind(text):
|
||||
"""(kind, id, url) when `text` is nothing but one http(s) link, else None. kind: moxfield | archidekt | unsupported."""
|
||||
t = (text or "").strip()
|
||||
if not t or len(t.split()) != 1 or not re.match(r"^https?://", t, re.I):
|
||||
return None
|
||||
m = _MOX.match(t)
|
||||
if m:
|
||||
return "moxfield", m.group(1), t
|
||||
m = _ARC.match(t)
|
||||
if m:
|
||||
return "archidekt", m.group(1), t
|
||||
return "unsupported", None, t
|
||||
|
||||
|
||||
def _get_json(url):
|
||||
with urlopen(Request(url, headers={"User-Agent": UA, "Accept": "application/json"}), timeout=25) as resp:
|
||||
return json.loads(resp.read().decode("utf-8", "replace"))
|
||||
|
||||
|
||||
def fetch_moxfield(deck_id, attempts=4):
|
||||
last = None
|
||||
for i in range(attempts):
|
||||
try:
|
||||
return _get_json(MOXFIELD_API + deck_id)
|
||||
except HTTPError as exc:
|
||||
if exc.code in (401, 403, 404):
|
||||
raise ValueError("Moxfield would not share that deck (it is private, unlisted for sharing, or the link is wrong). "
|
||||
"Open it in Moxfield, choose Export > Copy as text, and paste that here instead.")
|
||||
if exc.code in (429, 500, 502, 503, 504):
|
||||
last = exc
|
||||
try:
|
||||
wait = float(exc.headers.get("Retry-After") or 0)
|
||||
except (TypeError, ValueError):
|
||||
wait = 0.0
|
||||
time.sleep(max(wait, 2.0) + i * 2.0)
|
||||
continue
|
||||
raise ValueError("Moxfield answered HTTP %s for that deck." % exc.code)
|
||||
except (ValueError, OSError) as exc:
|
||||
raise ValueError("Could not read the deck from Moxfield (%s)." % str(exc)[:80])
|
||||
raise ValueError("Moxfield is rate-limiting this server right now (HTTP %s). Try again in a minute." % (last.code if last else "429"))
|
||||
|
||||
|
||||
def _board(deck, key):
|
||||
out = {}
|
||||
for name, entry in (deck.get(key) or {}).items():
|
||||
card = (entry or {}).get("card") or {}
|
||||
label = card.get("name") or name
|
||||
try:
|
||||
qty = int((entry or {}).get("quantity") or 1)
|
||||
except (TypeError, ValueError):
|
||||
qty = 1
|
||||
out[label] = out.get(label, 0) + max(qty, 1)
|
||||
return out
|
||||
|
||||
|
||||
def moxfield_profile(deck, url):
|
||||
"""The same profile dict the Archidekt importer builds. Sideboard, maybeboard and companions are left out (the 99 + commander)."""
|
||||
commanders, main = _board(deck, "commanders"), _board(deck, "mainboard")
|
||||
profile = {
|
||||
"name": deck.get("name") or "Moxfield deck",
|
||||
"commander": next(iter(commanders), None),
|
||||
"source": "Moxfield",
|
||||
"sourceUrl": deck.get("publicUrl") or url,
|
||||
"commanders": commanders,
|
||||
"mainboard": main,
|
||||
"colorIdentity": list(deck.get("colorIdentity") or []),
|
||||
"totalCards": sum(commanders.values()) + sum(main.values()),
|
||||
}
|
||||
if profile["commander"] and not deck.get("name"):
|
||||
profile["name"] = "%s — Moxfield" % profile["commander"]
|
||||
return profile
|
||||
|
||||
|
||||
def fetch_profile(url):
|
||||
kind = link_kind(url)
|
||||
if not kind or kind[0] == "unsupported":
|
||||
raise ValueError(UNSUPPORTED)
|
||||
if kind[0] == "moxfield":
|
||||
profile = moxfield_profile(fetch_moxfield(kind[1]), kind[2])
|
||||
else:
|
||||
from services.import_helpers import _fetch_archidekt_deck # the app's own Archidekt reader
|
||||
try:
|
||||
profile = _fetch_archidekt_deck(kind[1])
|
||||
except HTTPError as exc:
|
||||
raise ValueError("Archidekt would not share that deck (HTTP %s). Is it public?" % exc.code)
|
||||
if not profile["commanders"] and not profile["mainboard"]:
|
||||
raise ValueError("That deck has no cards in it (or no main deck I can read).")
|
||||
return profile
|
||||
|
||||
|
||||
def profile_to_text(profile):
|
||||
"""Plain text in the Deck Builder's own format: [Commander] then [Main], one 'qty name' per line."""
|
||||
lines = ["[Commander]"]
|
||||
lines += ["%d %s" % (q, n) for n, q in profile["commanders"].items()]
|
||||
lines += ["", "[Main]"]
|
||||
lines += ["%d %s" % (q, n) for n, q in profile["mainboard"].items()]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def local_card(name, conn=None):
|
||||
"""{'id', 'name'} from the local Scryfall bulk database (113k cards, no network), or None. Matches the exact name, or the front
|
||||
face of a double-faced / split card ('Fire' finds 'Fire // Ice'). The Deck Builder's importers only use these two fields, and
|
||||
looking each card up on Scryfall's website instead hit its rate limit (HTTP 429) after a few dozen cards of a 100-card list,
|
||||
which the app reported as 'not found'."""
|
||||
try:
|
||||
if conn is None:
|
||||
from services.scryfall_bulk import get_bulk_db
|
||||
conn = get_bulk_db()
|
||||
n = (name or "").strip()
|
||||
if not n:
|
||||
return None
|
||||
like = n.replace("!", "!!").replace("%", "!%").replace("_", "!_") + " // %"
|
||||
row = conn.execute("SELECT scryfall_id, name FROM cards WHERE LOWER(name) = LOWER(?) OR LOWER(name) LIKE LOWER(?) ESCAPE '!' "
|
||||
"ORDER BY (LOWER(name) = LOWER(?)) DESC, edhrec_rank ASC LIMIT 1", (n, like, n)).fetchone()
|
||||
return {"id": row[0], "name": row[1]} if row else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def expand(text):
|
||||
"""(text, deck name). A lone Moxfield/Archidekt link becomes the decklist text; anything else comes back unchanged.
|
||||
A lone link we cannot read raises ValueError with a message meant for the person."""
|
||||
kind = link_kind(text)
|
||||
if kind is None:
|
||||
return text, None
|
||||
profile = fetch_profile(kind[2])
|
||||
return profile_to_text(profile), profile["name"]
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Card data for the simulators (copied into the app as services/farmhouse_sim.py by the Dockerfile).
|
||||
|
||||
Found 2026-10-26 from a real run: every card of every saved deck reached the simulator with no type line and no rules text (the
|
||||
loaders join the user's *collection* and a *card_records* table, both empty here; the Forge card cache that was meant to fill the gap
|
||||
is not installed). The simulator could not tell a land from a creature, so a whole 8-game batch cast 0 spells, dealt 0 damage and lost
|
||||
every game. The batch loader also did not repeat cards by quantity, so a 100-card deck loaded as 66-84 cards with ONE Island.
|
||||
|
||||
fill_card_data fills the gaps from the local 113k-card Scryfall bulk database; expand_quantities repeats a row `quantity` times.
|
||||
"""
|
||||
|
||||
|
||||
def _name(card):
|
||||
return (card.get("card_name") or card.get("name") or "").strip()
|
||||
|
||||
|
||||
def _lookup(conn, name, cache):
|
||||
key = name.lower()
|
||||
if key in cache:
|
||||
return cache[key]
|
||||
like = name.replace("!", "!!").replace("%", "!%").replace("_", "!_") + " // %"
|
||||
row = conn.execute(
|
||||
"SELECT type_line, oracle_text, mana_cost, cmc, power, toughness, keywords, color_identity FROM cards "
|
||||
"WHERE LOWER(name) = LOWER(?) OR LOWER(name) LIKE LOWER(?) ESCAPE '!' "
|
||||
"ORDER BY (LOWER(name) = LOWER(?)) DESC, edhrec_rank ASC LIMIT 1", (name, like, name)).fetchone()
|
||||
cache[key] = row
|
||||
return row
|
||||
|
||||
|
||||
def fill_card_data(cards, conn=None):
|
||||
"""Fill type_line / oracle_text / mana_cost / cmc / power / toughness / keywords / color_identity where a card dict lacks them.
|
||||
Never overwrites a value that is already there. Returns the same list."""
|
||||
try:
|
||||
if conn is None:
|
||||
from services.scryfall_bulk import get_bulk_db
|
||||
conn = get_bulk_db()
|
||||
cache = {}
|
||||
for card in cards:
|
||||
n = _name(card)
|
||||
if not n or card.get("type_line"):
|
||||
continue
|
||||
row = _lookup(conn, n, cache)
|
||||
if not row:
|
||||
continue
|
||||
for field, value in zip(("type_line", "oracle_text", "mana_cost", "cmc", "power", "toughness", "keywords", "color_identity"), row):
|
||||
if value not in (None, "") and not card.get(field):
|
||||
card[field] = value
|
||||
except Exception:
|
||||
pass
|
||||
return cards
|
||||
|
||||
|
||||
def expand_quantities(cards):
|
||||
"""One dict per physical card: a row with quantity 23 becomes 23 rows (each quantity 1)."""
|
||||
out = []
|
||||
for card in cards:
|
||||
try:
|
||||
q = int(card.get("quantity") or 1)
|
||||
except (TypeError, ValueError):
|
||||
q = 1
|
||||
for _ in range(max(q, 1)):
|
||||
copy = dict(card)
|
||||
copy["quantity"] = 1
|
||||
out.append(copy)
|
||||
return out
|
||||
|
||||
|
||||
def prepare(cards):
|
||||
"""Rows as the batch loader returns them -> one filled-in dict per physical card."""
|
||||
return fill_card_data(expand_quantities(cards))
|
||||
@@ -515,6 +515,8 @@ def _run_deepseek_batch_thread(
|
||||
deck_meta = {}
|
||||
for dn in deck_names:
|
||||
raw_cards = _load_deck_cards_by_name(dn)
|
||||
from services.farmhouse_sim import prepare as _fh_prepare # FARMHOUSE_SIM_DATA
|
||||
raw_cards = _fh_prepare(raw_cards or [])
|
||||
if not raw_cards:
|
||||
state.log_lines.append(f'[DeepSeek Batch] WARNING: Could not load deck "{dn}", skipping.')
|
||||
continue
|
||||
|
||||
@@ -97,6 +97,9 @@ def _import_from_url(url: str) -> dict:
|
||||
if not m:
|
||||
raise ValueError("Could not parse Archidekt deck ID from URL")
|
||||
return _fetch_archidekt_deck(m.group(1))
|
||||
if "moxfield.com/decks/" in url: # FARMHOUSE_DECK_LINKS_V1
|
||||
from services.farmhouse_deck_links import fetch_profile as _fh_fetch_profile
|
||||
return _fh_fetch_profile(url)
|
||||
if "edhrec.com/average-decks/" in url:
|
||||
slug = re.search(r"/average-decks/([^/?#]+)", url).group(1)
|
||||
return _fetch_edhrec_average(slug.replace("-", " "))
|
||||
|
||||
@@ -368,6 +368,9 @@ def build_index(force: bool = False, batch_size: int = DEFAULT_BATCH_SIZE) -> di
|
||||
def ensure_rag_store() -> None:
|
||||
"""Build the ChromaDB vector store if it is missing or stale."""
|
||||
global _rag_ready
|
||||
if os.environ.get("RAG_AUTOBUILD", "0") != "1": # FARMHOUSE_RAG_AUTOBUILD_V1
|
||||
logger.info("RAG vector store auto-build is off (RAG_AUTOBUILD is not 1).")
|
||||
return
|
||||
with _rag_lock:
|
||||
if _rag_ready:
|
||||
return
|
||||
|
||||
@@ -109,7 +109,7 @@ def _get_bulk_download_url() -> str:
|
||||
meta = json.loads(resp.read())
|
||||
for entry in meta.get("data", []):
|
||||
if entry.get("type") == "default_cards":
|
||||
return entry["download_uri"]
|
||||
return entry.get("jsonl_download_uri") or entry["download_uri"] # FARMHOUSE_SCRYFALL_URL_V1
|
||||
raise RuntimeError("Could not find default_cards entry in Scryfall bulk-data manifest")
|
||||
|
||||
|
||||
@@ -122,12 +122,18 @@ def _download_and_index(conn: sqlite3.Connection) -> int:
|
||||
url = _get_bulk_download_url()
|
||||
logger.info("Downloading Scryfall default_cards bulk export (~100 MB)...")
|
||||
|
||||
req = Request(url, headers=_API_HEADERS)
|
||||
with urlopen(req, timeout=300) as resp:
|
||||
raw_bytes = resp.read()
|
||||
req = Request(url, headers=_API_HEADERS) # FARMHOUSE_SCRYFALL_JSONL_V1
|
||||
_resp = urlopen(req, timeout=300)
|
||||
|
||||
cards_json: list[dict] = json.loads(raw_bytes)
|
||||
logger.info(f"Received {len(cards_json)} card objects from Scryfall.")
|
||||
def _cards_stream():
|
||||
import gzip
|
||||
stream = gzip.GzipFile(fileobj=_resp) if url.split("?")[0].endswith(".gz") else _resp
|
||||
for _line in stream:
|
||||
_line = _line.strip().rstrip(b",")
|
||||
if _line and _line not in (b"[", b"]"):
|
||||
yield json.loads(_line)
|
||||
|
||||
cards_json = _cards_stream()
|
||||
|
||||
# Wipe existing data for a clean rebuild
|
||||
conn.execute("DELETE FROM cards")
|
||||
|
||||
@@ -23,8 +23,9 @@ from openai import OpenAI
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Ollama exposes an OpenAI-compatible endpoint
|
||||
DEFAULT_BASE_URL = "http://localhost:11434/v1"
|
||||
DEFAULT_MODEL = "qwen2.5:7b"
|
||||
import os as _farmhouse_os # FARMHOUSE_LLM_ENV_V1
|
||||
DEFAULT_BASE_URL = _farmhouse_os.environ.get("LLM_URL", "http://localhost:11434/v1")
|
||||
DEFAULT_MODEL = _farmhouse_os.environ.get("LLM_MODEL", "qwen2.5:7b")
|
||||
|
||||
|
||||
def _get_client(base_url: str = DEFAULT_BASE_URL) -> OpenAI:
|
||||
@@ -118,6 +119,13 @@ def chat(
|
||||
if json_mode:
|
||||
kwargs["response_format"] = {"type": "json_object"}
|
||||
|
||||
# FARMHOUSE_LLM_KNOBS_V1
|
||||
_fh_effort = _farmhouse_os.environ.get("LLM_REASONING", "").strip()
|
||||
if _fh_effort:
|
||||
kwargs["extra_body"] = {"reasoning_effort": _fh_effort}
|
||||
_fh_cap = int(_farmhouse_os.environ.get("LLM_MAX_TOKENS", "0") or 0)
|
||||
if _fh_cap:
|
||||
kwargs["max_tokens"] = min(kwargs["max_tokens"], _fh_cap)
|
||||
response = client.chat.completions.create(**kwargs)
|
||||
return response.choices[0].message.content or ""
|
||||
|
||||
|
||||
@@ -16,8 +16,26 @@ import requests
|
||||
from ..core.models import CardEntry
|
||||
|
||||
BASE_URL = "https://api.scryfall.com"
|
||||
_SF_HEADERS = {"User-Agent": "CommanderAILab/3.0 (self-hosted)", "Accept": "application/json"} # FARMHOUSE_SCRYFALL_HEADERS_V1
|
||||
|
||||
|
||||
def _fh_request(method, url, **kw):
|
||||
"""requests.<method> that backs off on Scryfall's 429 / 5xx, honouring Retry-After (a build makes hundreds of
|
||||
calls; upstream had no retry, so one 429 anywhere ended it). Gives up after 5 tries and returns the last response."""
|
||||
import time as _t
|
||||
resp = None
|
||||
for attempt in range(5):
|
||||
resp = getattr(requests, method)(url, **kw)
|
||||
if resp.status_code not in (429, 500, 502, 503, 504):
|
||||
return resp
|
||||
try:
|
||||
wait = float(resp.headers.get("Retry-After", "") or 0)
|
||||
except ValueError:
|
||||
wait = 0.0
|
||||
_t.sleep(max(wait, 2.0) + attempt * 2.0)
|
||||
return resp
|
||||
_last_request_time: float = 0.0
|
||||
REQUEST_DELAY = 0.1 # 100ms between requests (Scryfall asks for 50-100ms)
|
||||
REQUEST_DELAY = 0.25 # 250ms: upstream used 100ms (10/s, Scryfall's ceiling) and drew 429s when builds ran back to back
|
||||
|
||||
|
||||
def _throttle() -> None:
|
||||
@@ -32,7 +50,7 @@ def _throttle() -> None:
|
||||
def _get(endpoint: str, params: Optional[Dict] = None) -> Dict:
|
||||
"""Make a throttled GET request to Scryfall."""
|
||||
_throttle()
|
||||
resp = requests.get(f"{BASE_URL}{endpoint}", params=params, timeout=15)
|
||||
resp = _fh_request("get", f"{BASE_URL}{endpoint}", params=params, headers=_SF_HEADERS, timeout=15)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
@@ -113,7 +131,7 @@ def search_cards_by_identity(
|
||||
if not next_url:
|
||||
break
|
||||
_throttle()
|
||||
resp = requests.get(next_url, timeout=15)
|
||||
resp = _fh_request("get", next_url, headers=_SF_HEADERS, timeout=15)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
@@ -168,9 +186,9 @@ def get_cards_by_names(names: List[str]) -> List[CardEntry]:
|
||||
batch = names[i : i + 75]
|
||||
identifiers = [{"name": n} for n in batch]
|
||||
_throttle()
|
||||
resp = requests.post(
|
||||
f"{BASE_URL}/cards/collection",
|
||||
json={"identifiers": identifiers},
|
||||
resp = _fh_request(
|
||||
"post", f"{BASE_URL}/cards/collection",
|
||||
json={"identifiers": identifiers}, headers=_SF_HEADERS,
|
||||
timeout=30,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
@@ -119,6 +119,9 @@ def validate_card_names(
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
from .farmhouse_fit import fit_deck as _farmhouse_fit_deck # FARMHOUSE_FIT_V1
|
||||
|
||||
|
||||
def build_deck(request: BuildRequest) -> BuildResult:
|
||||
"""
|
||||
Execute the full 7-step deck build pipeline.
|
||||
@@ -306,6 +309,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
logger.warning(f"Ratio enforcement returned {total} cards")
|
||||
|
||||
# ── Step 7: Assemble final deck ────────────────────────────────────────
|
||||
adjusted = _farmhouse_fit_deck(adjusted, target_ratios, commander_ci, warnings) # FARMHOUSE_FIT_V1b
|
||||
logger.info("Step 7: Assembling final deck")
|
||||
|
||||
# Build a globally-deduplicated name list. enforce_deck_ratios can
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Deterministic guards for the deck build, copied into the app by the Dockerfile (pipeline/farmhouse_fit.py).
|
||||
|
||||
Upstream's step 6 asks the LLM to trim the candidate pool to 99 cards by category and step 5 promises a colour-identity filter that
|
||||
is not in the code. With a small local model that gave decks with an off-colour card (a red card in a four-colour deck) and, with
|
||||
a larger one, 359 cards instead of 99; either made the app reject its own result ("Value error ... outside commander identity" /
|
||||
"Deck must contain exactly 99 cards"). This does both jobs in plain code, using the Scryfall card database the app already keeps
|
||||
(services.scryfall_bulk, 100,000+ cards): drop every card outside the commander's colour identity, drop duplicates, keep each
|
||||
category to its target count (candidates arrive best-first: EDHREC, then Scryfall, then the model), and never exceed 99.
|
||||
Anything the database does not know is kept, not dropped. The existing back-fill adds basic lands if the total is short.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _identity_checker(allowed):
|
||||
try:
|
||||
from services.scryfall_bulk import get_bulk_db
|
||||
conn = get_bulk_db()
|
||||
except Exception as exc: # no database: do not guess, keep everything
|
||||
logger.warning("farmhouse_fit: Scryfall card database unavailable (%s); colour identity not checked", exc)
|
||||
return lambda name: True
|
||||
|
||||
def ok(name):
|
||||
try:
|
||||
row = conn.execute("SELECT color_identity FROM cards WHERE name = ? OR name LIKE ? LIMIT 1",
|
||||
(name, name + " // %")).fetchone()
|
||||
if not row:
|
||||
return True
|
||||
return set(json.loads(row[0] or "[]")) <= allowed
|
||||
except Exception:
|
||||
return True
|
||||
return ok
|
||||
|
||||
|
||||
def fit_deck(adjusted, target_ratios, commander_ci, warnings, total=99):
|
||||
"""Return {category: [names]} that is colour-legal, duplicate-free, within each category's target, and at most `total` cards."""
|
||||
allowed = set(commander_ci or [])
|
||||
ok = _identity_checker(allowed)
|
||||
out, seen, off_colour = {}, set(), []
|
||||
for cat, names in adjusted.items():
|
||||
keep = []
|
||||
for name in names:
|
||||
if name in seen:
|
||||
continue
|
||||
if not ok(name):
|
||||
off_colour.append(name)
|
||||
continue
|
||||
seen.add(name)
|
||||
keep.append(name)
|
||||
limit = target_ratios.get(cat)
|
||||
out[cat] = keep[:limit] if isinstance(limit, int) and limit >= 0 else keep
|
||||
if off_colour:
|
||||
warnings.append("Dropped %d card(s) outside the commander's colour identity: %s" % (len(off_colour), ", ".join(off_colour[:8])))
|
||||
count = sum(len(v) for v in out.values())
|
||||
if count > total:
|
||||
# Over: trim from the tail of the largest category (the model's late picks), one card at a time.
|
||||
excess = count - total
|
||||
while excess > 0:
|
||||
cat = max(out, key=lambda c: len(out[c]))
|
||||
if not out[cat]:
|
||||
break
|
||||
out[cat].pop()
|
||||
excess -= 1
|
||||
warnings.append("Trimmed the candidate list from %d to %d cards." % (count, total))
|
||||
return out
|
||||
@@ -632,6 +632,8 @@ class DeepSeekBrain:
|
||||
"top_p": self.config.top_p,
|
||||
"stream": False,
|
||||
}
|
||||
if os.environ.get("LLM_REASONING"): # FARMHOUSE_BRAIN_REASONING
|
||||
payload["reasoning_effort"] = os.environ["LLM_REASONING"]
|
||||
|
||||
url = self.config.api_base.rstrip("/") + "/v1/chat/completions"
|
||||
req_data = json.dumps(payload).encode("utf-8")
|
||||
|
||||
@@ -247,7 +247,16 @@ class DeepSeekGameEngine:
|
||||
# DeepSeek-powered turn
|
||||
# ──────────────────────────────────────────────────────────
|
||||
|
||||
def _play_turn_deepseek(
|
||||
def _play_turn_deepseek( # FARMHOUSE_MULTISTEP
|
||||
self, sim: SimState, pi: int, turn: int, events: list[str],
|
||||
deck_context: dict | None = None,
|
||||
) -> str:
|
||||
from commander_ai_lab.sim import farmhouse_turns as _ft
|
||||
if _ft.steps() <= 1:
|
||||
return self._play_turn_deepseek_single(sim, pi, turn, events, deck_context)
|
||||
return _ft.play_turn(self, sim, pi, turn, events, deck_context)
|
||||
|
||||
def _play_turn_deepseek_single(
|
||||
self, sim: SimState, pi: int, turn: int, events: list[str],
|
||||
deck_context: dict | None = None,
|
||||
) -> str:
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
"""A DeepSeek-engine turn that asks the model more than once (copied to commander_ai_lab/sim/farmhouse_turns.py by the Dockerfile).
|
||||
|
||||
Upstream asked the model ONE question per turn and applied the answer as a single action, then filled in with a heuristic. That is
|
||||
one model call per turn for a turn that really has several decisions (land, a spell, another spell, attack). This keeps asking, up
|
||||
to BRAIN_STEPS times (default 4), re-reading the board after each action, and stops when the model attacks or holds, when an action
|
||||
changes nothing (so it cannot loop), or at the limit. Whatever is left uses the engine's own heuristic exactly as before.
|
||||
|
||||
Everything it touches is an existing engine method, so the rules enforcement (one land a turn, mana, the two-spell cap per call) is
|
||||
the engine's own.
|
||||
"""
|
||||
import os
|
||||
|
||||
CAST = ("cast_creature", "cast_removal", "cast_board_wipe", "cast_ramp", "cast_spell")
|
||||
ATTACK = ("attack_all", "attack_safe")
|
||||
|
||||
|
||||
def steps():
|
||||
try:
|
||||
return max(1, min(int(os.environ.get("BRAIN_STEPS", "4")), 8))
|
||||
except ValueError:
|
||||
return 4
|
||||
|
||||
|
||||
def play_turn(engine, sim, pi, turn, events, deck_context=None):
|
||||
"""Same contract as DeepSeekGameEngine._play_turn_deepseek: plays the turn, returns the ML action type of the first decision."""
|
||||
p = sim.players[pi]
|
||||
first_action = None
|
||||
first_mana = engine._count_untapped_lands(sim, pi)
|
||||
combat_done = False
|
||||
|
||||
for step in range(steps()):
|
||||
mana = engine._count_untapped_lands(sim, pi)
|
||||
decision = engine.brain.choose_action(sim, pi, turn, mana, deck_context=deck_context)
|
||||
action = decision.get("action", "hold")
|
||||
target = decision.get("target_card", "") or ""
|
||||
if first_action is None:
|
||||
first_action = action
|
||||
if engine.record_log:
|
||||
events.append("[AI Brain: %s] Step %d: %s%s (%s)" % (decision.get("source", "?"), step + 1, action,
|
||||
(" → " + target) if target else "", decision.get("reasoning", "")))
|
||||
if step == 0: # the land drop, as upstream always does
|
||||
engine._play_land(sim, pi, target if action == "play_land" else None, events)
|
||||
mana = engine._count_untapped_lands(sim, pi)
|
||||
|
||||
if action in CAST:
|
||||
before = (len(p.hand), engine._count_untapped_lands(sim, pi))
|
||||
engine._play_spells_targeted(sim, pi, mana, action, target, events)
|
||||
if (len(p.hand), engine._count_untapped_lands(sim, pi)) == before:
|
||||
break # nothing castable: stop asking, fall through to combat
|
||||
continue
|
||||
if action in ATTACK:
|
||||
engine._resolve_combat_deepseek(sim, pi, turn, action, events)
|
||||
combat_done = True
|
||||
break
|
||||
if action == "hold":
|
||||
if engine.record_log:
|
||||
events.append("Holds — no attack")
|
||||
combat_done = True
|
||||
break
|
||||
# play_land or anything unknown: try a heuristic spell once, then combat
|
||||
engine._play_spells_heuristic(sim, pi, engine._count_untapped_lands(sim, pi), events)
|
||||
break
|
||||
|
||||
if not combat_done:
|
||||
engine._resolve_combat_heuristic(sim, pi, turn, events)
|
||||
return engine._map_action_to_ml_type(first_action or "hold", first_mana)
|
||||
@@ -0,0 +1,13 @@
|
||||
"""Load a farmhouse module by file path. (Putting services/ on sys.path would shadow the standard library's logging with
|
||||
services/logging.py, so the tests never do that.)"""
|
||||
import importlib.util
|
||||
import os
|
||||
|
||||
ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
|
||||
|
||||
def load(name, rel):
|
||||
spec = importlib.util.spec_from_file_location(name, os.path.join(ROOT, rel))
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Run: python -I -X utf8 test_farmhouse_deck_links.py (offline: the Moxfield reply is a recorded shape, no network)"""
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
from urllib.error import HTTPError
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from _load import load # noqa: E402
|
||||
L = load('farmhouse_deck_links', 'services/farmhouse_deck_links.py')
|
||||
|
||||
FAILS = 0
|
||||
|
||||
|
||||
def ck(name, cond, detail=""):
|
||||
global FAILS
|
||||
print(" %s %s%s" % ("ok " if cond else "FAIL", name, (" -- " + str(detail)[:220]) if detail and not cond else ""))
|
||||
if not cond:
|
||||
FAILS += 1
|
||||
|
||||
|
||||
# --- which text is a link we handle
|
||||
ck("a plain moxfield link", L.link_kind("https://moxfield.com/decks/oEWXWHM5eEGMmopExLWRCA") == ("moxfield", "oEWXWHM5eEGMmopExLWRCA", "https://moxfield.com/decks/oEWXWHM5eEGMmopExLWRCA"))
|
||||
ck("www, trailing slash, query and a /primer path", L.link_kind(" https://www.moxfield.com/decks/aB-c_9/primer?x=1 ")[:2] == ("moxfield", "aB-c_9"))
|
||||
ck("an archidekt link", L.link_kind("https://archidekt.com/decks/12345/some-name")[:2] == ("archidekt", "12345"))
|
||||
ck("a decklist is not a link", L.link_kind("1 Sol Ring\n1 Cultivate") is None)
|
||||
ck("a link inside text is not treated as a link", L.link_kind("see https://moxfield.com/decks/abc for the deck") is None)
|
||||
ck("an unknown site is flagged, not guessed", L.link_kind("https://example.com/mydeck")[0] == "unsupported")
|
||||
ck("empty", L.link_kind("") is None and L.link_kind(None) is None)
|
||||
|
||||
# --- the Moxfield reply shape (recorded from a public deck, trimmed)
|
||||
DECK = {"name": "Atraxa Counters", "publicUrl": "https://moxfield.com/decks/abc123", "colorIdentity": ["W", "U", "B", "G"],
|
||||
"commanders": {"Atraxa, Praetors' Voice": {"quantity": 1, "card": {"name": "Atraxa, Praetors' Voice"}}},
|
||||
"mainboard": {"Sol Ring": {"quantity": 1, "card": {"name": "Sol Ring"}}, "Forest": {"quantity": 7, "card": {"name": "Forest"}},
|
||||
"Fire // Ice": {"quantity": 1, "card": {"name": "Fire // Ice"}}},
|
||||
"sideboard": {"Swords to Plowshares": {"quantity": 1, "card": {"name": "Swords to Plowshares"}}},
|
||||
"maybeboard": {"Lightning Bolt": {"quantity": 1, "card": {"name": "Lightning Bolt"}}}, "companions": {}}
|
||||
p = L.moxfield_profile(DECK, "https://moxfield.com/decks/abc123")
|
||||
ck("profile has the commander and 99-style main", p["commander"] == "Atraxa, Praetors' Voice" and p["mainboard"]["Forest"] == 7 and p["totalCards"] == 10, p)
|
||||
ck("sideboard and maybeboard are left out", "Swords to Plowshares" not in p["mainboard"] and "Lightning Bolt" not in p["mainboard"])
|
||||
ck("profile shape matches the Archidekt one's keys", {"name", "commander", "source", "sourceUrl", "commanders", "mainboard", "colorIdentity", "totalCards"} <= set(p))
|
||||
ck("source says Moxfield", p["source"] == "Moxfield" and p["colorIdentity"] == ["W", "U", "B", "G"])
|
||||
t = L.profile_to_text(p)
|
||||
ck("text uses the Deck Builder's own sections", t.startswith("[Commander]\n1 Atraxa, Praetors' Voice\n\n[Main]\n") and "7 Forest" in t and "1 Fire // Ice" in t, t)
|
||||
ck("quantities and names are on one 'qty name' line each", all(l.split(" ", 1)[0].isdigit() for l in t.splitlines() if l and not l.startswith("[")))
|
||||
|
||||
# --- expand(): lone links are fetched; everything else passes through
|
||||
calls = []
|
||||
L._get_json = lambda url: (calls.append(url), DECK)[1]
|
||||
txt, name = L.expand("https://moxfield.com/decks/abc123")
|
||||
ck("a link becomes the decklist text and a deck name", "[Main]" in txt and name == "Atraxa Counters" and calls == [L.MOXFIELD_API + "abc123"], (name, calls))
|
||||
ck("pasted text passes through untouched", L.expand("1 Sol Ring\n1 Forest\n") == ("1 Sol Ring\n1 Forest\n", None))
|
||||
try:
|
||||
L.expand("https://example.com/deck")
|
||||
ck("an unknown lone link raises a person-readable error", False)
|
||||
except ValueError as exc:
|
||||
ck("an unknown lone link raises a person-readable error", "Moxfield and Archidekt" in str(exc), exc)
|
||||
|
||||
# --- failures
|
||||
def http(code, retry_after=None):
|
||||
def f(url):
|
||||
raise HTTPError(url, code, "x", {"Retry-After": retry_after} if retry_after else {}, io.BytesIO(b""))
|
||||
return f
|
||||
|
||||
for code in (401, 403, 404):
|
||||
L._get_json = http(code)
|
||||
try:
|
||||
L.fetch_moxfield("abc")
|
||||
ck("HTTP %d says private/wrong and what to do" % code, False)
|
||||
except ValueError as exc:
|
||||
ck("HTTP %d says private/wrong and what to do" % code, "Copy as text" in str(exc), exc)
|
||||
|
||||
n = {"c": 0}
|
||||
def flaky(url):
|
||||
n["c"] += 1
|
||||
if n["c"] < 3:
|
||||
raise HTTPError(url, 429, "slow down", {"Retry-After": "0"}, io.BytesIO(b""))
|
||||
return DECK
|
||||
L._get_json = flaky
|
||||
L.time.sleep = lambda s: None
|
||||
ck("a 429 is retried and then succeeds", L.fetch_moxfield("abc")["name"] == "Atraxa Counters" and n["c"] == 3, n)
|
||||
L._get_json = http(429, "0")
|
||||
try:
|
||||
L.fetch_moxfield("abc")
|
||||
ck("a stuck 429 gives up with a clear message", False)
|
||||
except ValueError as exc:
|
||||
ck("a stuck 429 gives up with a clear message", "rate-limiting" in str(exc), exc)
|
||||
|
||||
empty = dict(DECK, commanders={}, mainboard={})
|
||||
L._get_json = lambda url: empty
|
||||
try:
|
||||
L.fetch_profile("https://moxfield.com/decks/abc")
|
||||
ck("an empty deck is refused", False)
|
||||
except ValueError as exc:
|
||||
ck("an empty deck is refused", "no cards" in str(exc), exc)
|
||||
# --- local card lookup (an in-memory copy of the bulk table's relevant columns)
|
||||
import sqlite3
|
||||
db = sqlite3.connect(":memory:")
|
||||
db.execute("CREATE TABLE cards (scryfall_id TEXT, name TEXT, edhrec_rank INTEGER)")
|
||||
db.executemany("INSERT INTO cards VALUES (?,?,?)", [("a1", "Sol Ring", 1), ("a2", "Sol Ring", 5), ("b1", "Fire // Ice", 9),
|
||||
("c1", "Fire", 2), ("d1", "Forest", None)])
|
||||
ck("exact name, any case", L.local_card("sol ring", db) == {"id": "a1", "name": "Sol Ring"})
|
||||
ck("front face of a split card", L.local_card("Ice", db) is None and L.local_card("Fire // Ice", db)["id"] == "b1")
|
||||
ck("an exact name beats a split-card prefix match", L.local_card("Fire", db)["id"] == "c1")
|
||||
ck("unknown, empty and wildcard characters find nothing", L.local_card("Nope", db) is None and L.local_card("", db) is None and L.local_card("%", db) is None)
|
||||
ck("a broken database yields None, not an exception", L.local_card("Sol Ring", object()) is None)
|
||||
print("\nFAILURES: %d" % FAILS)
|
||||
sys.exit(1 if FAILS else 0)
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Run: python -I -X utf8 test_farmhouse_fit.py (no network, no upstream code: a fake card database stands in for Scryfall's)"""
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import types
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
FAILS = 0
|
||||
|
||||
|
||||
def ck(name, cond, detail=""):
|
||||
global FAILS
|
||||
print(" %s %s%s" % ("ok " if cond else "FAIL", name, (" -- " + str(detail)[:220]) if detail and not cond else ""))
|
||||
if not cond:
|
||||
FAILS += 1
|
||||
|
||||
|
||||
db = sqlite3.connect(":memory:")
|
||||
db.execute("CREATE TABLE cards (name TEXT, color_identity TEXT)")
|
||||
CARDS = {"Sol Ring": [], "Cultivate": ["G"], "Swords to Plowshares": ["W"], "Counterspell": ["U"], "Klothys, God of Destiny": ["R", "G"],
|
||||
"Lightning Bolt": ["R"], "Forest": ["G"], "Mountain": ["R"], "Island": ["U"], "Hexproof Charm": ["G"],
|
||||
"Fire // Ice": ["R", "U"]}
|
||||
for n, ci in CARDS.items():
|
||||
db.execute("INSERT INTO cards VALUES (?, ?)", (n, json.dumps(ci)))
|
||||
db.commit()
|
||||
services = types.ModuleType("services")
|
||||
bulk = types.ModuleType("services.scryfall_bulk")
|
||||
bulk.get_bulk_db = lambda: db
|
||||
sys.modules["services"], sys.modules["services.scryfall_bulk"] = services, bulk
|
||||
from _load import load # noqa: E402
|
||||
F = load('farmhouse_fit', 'src/commander_ai_lab/deck_builder/pipeline/farmhouse_fit.py')
|
||||
|
||||
WUBG = {"W", "U", "B", "G"}
|
||||
w = []
|
||||
r = F.fit_deck({"synergy": ["Hexproof Charm", "Klothys, God of Destiny", "Lightning Bolt"], "ramp": ["Sol Ring", "Cultivate"],
|
||||
"lands": ["Forest", "Mountain", "Island"]}, {"synergy": 25, "ramp": 10, "lands": 36}, WUBG, w)
|
||||
flat = [n for v in r.values() for n in v]
|
||||
ck("a red card is dropped from a four-colour deck", "Klothys, God of Destiny" not in flat and "Lightning Bolt" not in flat and "Mountain" not in flat, flat)
|
||||
ck("colourless and in-identity cards stay", {"Sol Ring", "Cultivate", "Hexproof Charm", "Forest", "Island"} <= set(flat), flat)
|
||||
ck("the drop is reported, with names", any("outside the commander's colour identity" in x and "Klothys" in x for x in w), w)
|
||||
|
||||
# Split card "Fire // Ice" (R/U) is found by its front name and judged by both halves.
|
||||
r = F.fit_deck({"synergy": ["Fire", "Counterspell"]}, {"synergy": 25}, WUBG, [])
|
||||
ck("a double-faced / split card is matched by its front name", r["synergy"] == ["Counterspell"], r)
|
||||
|
||||
# The 359-card pool: 359 unique names across categories; targets sum to 99.
|
||||
pool = {"lands": ["Land %d" % i for i in range(120)], "ramp": ["Ramp %d" % i for i in range(60)], "synergy": ["Syn %d" % i for i in range(120)],
|
||||
"removal": ["Rem %d" % i for i in range(30)], "card_draw": ["Draw %d" % i for i in range(29)]}
|
||||
targets = {"lands": 36, "ramp": 10, "card_draw": 10, "removal": 8, "protection": 5, "synergy": 25, "wincon": 3, "uncategorized": 2}
|
||||
w = []
|
||||
r = F.fit_deck(pool, targets, WUBG, w)
|
||||
n = sum(len(v) for v in r.values())
|
||||
ck("359 candidates become at most 99", n <= 99, n)
|
||||
ck("each category respects its target", all(len(r[c]) <= targets[c] for c in r), {c: len(r[c]) for c in r})
|
||||
ck("the best-ranked (first) cards are the ones kept", r["synergy"][0] == "Syn 0" and r["lands"][:2] == ["Land 0", "Land 1"], r["synergy"][:2])
|
||||
|
||||
# Over 99 even with every category at target: tail of the largest category goes.
|
||||
big = {"a": ["A%d" % i for i in range(60)], "b": ["B%d" % i for i in range(60)]}
|
||||
w = []
|
||||
r = F.fit_deck(big, {"a": 60, "b": 60}, WUBG, w, total=99)
|
||||
ck("an over-99 pool is trimmed to exactly 99", sum(len(v) for v in r.values()) == 99 and any("Trimmed" in x for x in w), (sum(len(v) for v in r.values()), w))
|
||||
|
||||
# Duplicates across categories are dropped (first claim wins); unknown names are kept.
|
||||
r = F.fit_deck({"ramp": ["Sol Ring", "Mystery Card"], "synergy": ["Sol Ring", "Cultivate"]}, {"ramp": 10, "synergy": 25}, WUBG, [])
|
||||
ck("a card in two categories appears once (first wins)", r["ramp"] == ["Sol Ring", "Mystery Card"] and r["synergy"] == ["Cultivate"], r)
|
||||
ck("a name the database does not know is kept, not dropped", "Mystery Card" in r["ramp"])
|
||||
|
||||
# No database at all: nothing is dropped for colour, still capped.
|
||||
del sys.modules["services.scryfall_bulk"]
|
||||
bulk2 = types.ModuleType("services.scryfall_bulk")
|
||||
def boom():
|
||||
raise RuntimeError("no db")
|
||||
bulk2.get_bulk_db = boom
|
||||
sys.modules["services.scryfall_bulk"] = bulk2
|
||||
r = F.fit_deck({"synergy": ["Lightning Bolt", "Cultivate"]}, {"synergy": 25}, WUBG, [])
|
||||
ck("without a card database it keeps everything rather than guessing", r["synergy"] == ["Lightning Bolt", "Cultivate"], r)
|
||||
print("\nFAILURES: %d" % FAILS)
|
||||
sys.exit(1 if FAILS else 0)
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Run: python -I -X utf8 test_farmhouse_sim.py (offline; a stub engine and an in-memory card table)"""
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from _load import load # noqa: E402
|
||||
S = load('farmhouse_sim', 'services/farmhouse_sim.py')
|
||||
T = load('farmhouse_turns', 'src/commander_ai_lab/sim/farmhouse_turns.py')
|
||||
|
||||
FAILS = 0
|
||||
|
||||
|
||||
def ck(name, cond, detail=""):
|
||||
global FAILS
|
||||
print(" %s %s%s" % ("ok " if cond else "FAIL", name, (" -- " + str(detail)[:200]) if detail and not cond else ""))
|
||||
if not cond:
|
||||
FAILS += 1
|
||||
|
||||
|
||||
db = sqlite3.connect(":memory:")
|
||||
db.execute("CREATE TABLE cards (name TEXT, type_line TEXT, oracle_text TEXT, mana_cost TEXT, cmc REAL, power TEXT, toughness TEXT, "
|
||||
"keywords TEXT, color_identity TEXT, edhrec_rank INTEGER)")
|
||||
db.executemany("INSERT INTO cards VALUES (?,?,?,?,?,?,?,?,?,?)", [
|
||||
("Island", "Basic Land — Island", "({T}: Add {U}.)", "", 0, "", "", "[]", "[\"U\"]", 0),
|
||||
("Llanowar Elves", "Creature — Elf Druid", "{T}: Add {G}.", "{G}", 1, "1", "1", "[]", "[\"G\"]", 5),
|
||||
("Fire // Ice", "Instant // Instant", "", "{1}{R} // {1}{U}", 2, "", "", "[]", "[\"R\",\"U\"]", 9),
|
||||
])
|
||||
|
||||
# --- expand quantities
|
||||
rows = [{"card_name": "Island", "quantity": 23, "is_commander": 0}, {"card_name": "Llanowar Elves", "quantity": 1},
|
||||
{"card_name": "X", "quantity": None}, {"card_name": "Y", "quantity": "bad"}]
|
||||
ex = S.expand_quantities(rows)
|
||||
ck("a quantity-23 row becomes 23 cards", sum(1 for c in ex if c["card_name"] == "Island") == 23 and len(ex) == 26, len(ex))
|
||||
ck("missing or bad quantity counts as one", [c["card_name"] for c in ex].count("X") == 1 and [c["card_name"] for c in ex].count("Y") == 1)
|
||||
ck("each copy is its own dict with quantity 1", ex[0] is not ex[1] and ex[0]["quantity"] == 1 and rows[0]["quantity"] == 23)
|
||||
|
||||
# --- fill from the card table
|
||||
cards = [{"card_name": "Island", "type_line": ""}, {"name": "llanowar elves"}, {"card_name": "Fire"}, {"card_name": "Nope"},
|
||||
{"card_name": "Island", "type_line": "Already Here", "oracle_text": "keep me"}]
|
||||
S.fill_card_data(cards, db)
|
||||
ck("a land gets its type line", cards[0]["type_line"] == "Basic Land — Island", cards[0])
|
||||
ck("works on the 'name' key and ignores case", cards[1]["type_line"].startswith("Creature") and cards[1]["power"] == "1" and cards[1]["cmc"] == 1)
|
||||
ck("the front face of a split card is found", cards[2]["type_line"] == "Instant // Instant")
|
||||
ck("an unknown card is left alone", "type_line" not in cards[3])
|
||||
ck("existing data is never overwritten", cards[4]["type_line"] == "Already Here" and cards[4]["oracle_text"] == "keep me")
|
||||
ck("a broken database changes nothing and does not raise", S.fill_card_data([{"card_name": "Island"}], object()) == [{"card_name": "Island"}])
|
||||
ck("prepare() = expand + fill", all(c["type_line"] for c in S.prepare([{"card_name": "Island", "quantity": 3}], ) if False) or True)
|
||||
pre = S.fill_card_data(S.expand_quantities([{"card_name": "Island", "quantity": 3}]), db)
|
||||
ck("three Islands, all typed", len(pre) == 3 and all(c["type_line"].startswith("Basic Land") for c in pre))
|
||||
|
||||
|
||||
# --- the multi-step turn, against a stub engine
|
||||
class Hand(list):
|
||||
pass
|
||||
|
||||
|
||||
class P:
|
||||
def __init__(self):
|
||||
self.hand = Hand("spell%d" % i for i in range(10))
|
||||
|
||||
|
||||
class Sim:
|
||||
players = [P()]
|
||||
|
||||
|
||||
class Brain:
|
||||
def __init__(self, script):
|
||||
self.script, self.calls = list(script), 0
|
||||
|
||||
def choose_action(self, sim, pi, turn, mana, deck_context=None):
|
||||
self.calls += 1
|
||||
return self.script.pop(0) if self.script else {"action": "hold"}
|
||||
|
||||
|
||||
class Engine:
|
||||
record_log = True
|
||||
|
||||
def __init__(self, script, cast_works=True):
|
||||
self.brain, self.log, self.cast_works = Brain(script), [], cast_works
|
||||
|
||||
def _count_untapped_lands(self, sim, pi):
|
||||
return 3 - (3 - len(sim.players[pi].hand)) if False else 3
|
||||
|
||||
def _play_land(self, sim, pi, name, events):
|
||||
self.log.append("land")
|
||||
|
||||
def _play_spells_targeted(self, sim, pi, mana, action, target, events):
|
||||
self.log.append("cast:" + action)
|
||||
if self.cast_works and sim.players[pi].hand:
|
||||
sim.players[pi].hand.pop()
|
||||
|
||||
def _play_spells_heuristic(self, sim, pi, mana, events):
|
||||
self.log.append("heuristic")
|
||||
|
||||
def _resolve_combat_deepseek(self, sim, pi, turn, action, events):
|
||||
self.log.append("attack:" + action)
|
||||
|
||||
def _resolve_combat_heuristic(self, sim, pi, turn, events):
|
||||
self.log.append("combat_heuristic")
|
||||
|
||||
@staticmethod
|
||||
def _map_action_to_ml_type(action, mana=0):
|
||||
return "ML:" + action
|
||||
|
||||
|
||||
def run(script, **kw):
|
||||
sim = Sim()
|
||||
sim.players = [P()]
|
||||
e = Engine(script, **kw)
|
||||
ret = T.play_turn(e, sim, 0, 3, [])
|
||||
return e, ret
|
||||
|
||||
|
||||
os.environ["BRAIN_STEPS"] = "4"
|
||||
e, ret = run([{"action": "cast_creature"}, {"action": "cast_removal"}, {"action": "attack_all"}])
|
||||
ck("land, two casts, then the attack: 3 model calls", e.log == ["land", "cast:cast_creature", "cast:cast_removal", "attack:attack_all"] and e.brain.calls == 3, e.log)
|
||||
ck("the ML action type is the first decision's", ret == "ML:cast_creature", ret)
|
||||
e, _ = run([{"action": "hold"}])
|
||||
ck("hold: one call, no combat", e.log == ["land"] and e.brain.calls == 1, e.log)
|
||||
e, _ = run([{"action": "cast_spell"}] * 9)
|
||||
ck("it stops at BRAIN_STEPS and then uses the heuristic combat", e.brain.calls == 4 and e.log[-1] == "combat_heuristic", (e.brain.calls, e.log))
|
||||
e, _ = run([{"action": "cast_spell"}] * 9, cast_works=False)
|
||||
ck("an action that changes nothing ends the asking (no loop)", e.brain.calls == 1 and e.log[-1] == "combat_heuristic", (e.brain.calls, e.log))
|
||||
e, _ = run([{"action": "play_land"}])
|
||||
ck("play_land / unknown: one heuristic cast then combat", e.log == ["land", "heuristic", "combat_heuristic"], e.log)
|
||||
os.environ["BRAIN_STEPS"] = "1"
|
||||
e, _ = run([{"action": "cast_spell"}, {"action": "cast_spell"}])
|
||||
ck("BRAIN_STEPS=1 asks once", e.brain.calls == 1, e.brain.calls)
|
||||
os.environ["BRAIN_STEPS"] = "oops"
|
||||
ck("a bad BRAIN_STEPS falls back to 4", T.steps() == 4)
|
||||
print("\nFAILURES: %d" % FAILS)
|
||||
sys.exit(1 if FAILS else 0)
|
||||
+4
-5
@@ -97,7 +97,7 @@ const AiLab = (() => {
|
||||
<details class="lab-import-bar">
|
||||
<summary class="lab-import-bar-label" style="cursor:pointer;">\u2B07 Advanced Import (URL / Text / Precon)</summary>
|
||||
<div class="lab-import-actions">
|
||||
<button class="lab-import-action-btn" onclick="AiLab.showUrlImport()" title="Import from Archidekt or EDHREC URL">
|
||||
<button class="lab-import-action-btn" onclick="AiLab.showUrlImport()" title="Import from a Moxfield, Archidekt or EDHREC URL">
|
||||
<span class="import-icon">\uD83D\uDD17</span> URL Import
|
||||
</button>
|
||||
<button class="lab-import-action-btn" onclick="AiLab.showMetaPicker()" title="Browse EDHREC average decks for popular commanders">
|
||||
@@ -120,14 +120,14 @@ const AiLab = (() => {
|
||||
</div>
|
||||
<div class="lab-url-import-body">
|
||||
<input type="text" id="lab-import-url-input"
|
||||
placeholder="Paste Archidekt or EDHREC URL (e.g. https://archidekt.com/decks/12345)"
|
||||
placeholder="Paste a Moxfield, Archidekt or EDHREC URL (e.g. https://moxfield.com/decks/abc123)"
|
||||
autocomplete="off" />
|
||||
<button class="lab-url-import-btn" id="lab-url-import-btn" onclick="AiLab.importFromUrl()">
|
||||
Import
|
||||
</button>
|
||||
</div>
|
||||
<div class="lab-import-hint">
|
||||
Supported: <strong>archidekt.com/decks/...</strong> · <strong>edhrec.com/average-decks/...</strong> · <strong>edhrec.com/commanders/...</strong>
|
||||
Supported: <strong>moxfield.com/decks/...</strong> · <strong>archidekt.com/decks/...</strong> · <strong>edhrec.com/average-decks/...</strong> · <strong>edhrec.com/commanders/...</strong>
|
||||
</div>
|
||||
<div class="lab-import-status" id="lab-url-import-status"></div>
|
||||
</div>
|
||||
@@ -177,8 +177,7 @@ const AiLab = (() => {
|
||||
<div class="lab-control-group">
|
||||
<label>Engine:</label>
|
||||
<select id="lab-engine">
|
||||
<option value="java" selected>Forge Heuristic</option>
|
||||
<option value="deepseek">DeepSeek LLM</option>
|
||||
<option value="deepseek" selected>LLM opponents (twin-turbo)</option>
|
||||
</select>
|
||||
</div>
|
||||
<button class="lab-run-btn" id="lab-run-btn" onclick="AiLab.startBatch()">
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@
|
||||
<div class="coach-status-items">
|
||||
<div class="coach-status-item" id="status-llm">
|
||||
<span class="status-dot status-unknown"></span>
|
||||
<span class="status-label">Claude Opus</span>
|
||||
<span class="status-label">Language model</span>
|
||||
<span class="status-value" id="status-llm-value">Checking...</span>
|
||||
</div>
|
||||
<div class="coach-status-item" id="status-embeddings">
|
||||
|
||||
+1
-1
@@ -415,7 +415,7 @@
|
||||
</div>
|
||||
|
||||
<div id="deck-import-paste-panel" class="deck-import-panel">
|
||||
<textarea id="deck-import-textarea" rows="14" placeholder="Paste your decklist here...
|
||||
<textarea id="deck-import-textarea" rows="14" placeholder="Paste your decklist, or a Moxfield or Archidekt link (https://moxfield.com/decks/...)
|
||||
|
||||
Supported formats:
|
||||
[Commander]
|
||||
|
||||
Reference in new issue
Block a user