Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
01b28a0025 | ||
|
|
3f6d71ecbc | ||
|
|
4246555228 | ||
|
|
8dd9623417 | ||
|
|
817b71f69d | ||
|
|
e71cac9dd1 |
No files matched your search
@@ -1,381 +1,180 @@
|
||||
# Commander AI Lab
|
||||
|
||||
Headless batch simulation environment for 3-AI MTG Commander pods.
|
||||
Run 100+ games, analyze win rates, and stress-test your decks — all from a web UI or CLI.
|
||||
A full-stack MTG Commander AI research platform. Simulate hundreds of games overnight, train ML models on the results, and let simulation data automatically shape the AI deck builder — all from a web UI or CLI.
|
||||
|
||||
## What It Does
|
||||
|
||||
| Layer | What it is |
|
||||
|---|---|
|
||||
| **Monte Carlo Engine** | Pure-Python headless sim — runs thousands of games in minutes, no JVM needed |
|
||||
| **Forge Engine** | Java/Forge rules-complete sim for ground-truth validation |
|
||||
| **Overnight Runner** | Unattended batch runner — runs sims, updates sim weights, trains neural net |
|
||||
| **ML Pipeline** | Outcome-weighted RL nudges card scoring weights; neural net trains on decision logs |
|
||||
| **Deck Builder** | 7-step LLM pipeline (EDHrec → Scryfall → Ollama) now injected with sim-learned insights |
|
||||
| **AI Coach** | LLM-powered coaching sessions (Anthropic/Ollama) with streaming support |
|
||||
| **Web UI** | React + FastAPI dashboard for decks, sims, training, and deck building |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Web Browser (ai-lab.js + ai-lab.css) │
|
||||
│ Select decks → Start batch → Poll progress → View stats │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│ HTTP (REST)
|
||||
┌─────────────────────┴───────────────────────────────────────┐
|
||||
│ Python FastAPI Server (lab_api.py, port 8080) │
|
||||
│ /api/lab/start — spawn Java batch runner subprocess │
|
||||
│ /api/lab/status — poll progress (parses stdout) │
|
||||
│ /api/lab/result — return completed JSON results │
|
||||
│ /api/lab/decks — list decks from Forge user data │
|
||||
│ /api/lab/history— browse past batch runs │
|
||||
│ /api/lab/log — stream live console output │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│ subprocess (java -jar)
|
||||
┌─────────────────────┴───────────────────────────────────────┐
|
||||
│ Java CLI (LabCli.java + BatchRunner + MultiThreadRunner) │
|
||||
│ Parses Forge sim output → GameResult objects → JSON │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│ subprocess per game
|
||||
┌─────────────────────┴───────────────────────────────────────┐
|
||||
│ Card Forge Engine (forge-gui-desktop sim mode) │
|
||||
│ java -jar forge.jar sim -d "deck1" "deck2" "deck3" │
|
||||
│ -f commander -n 1 -q │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Web UI (React/TypeScript — frontend/) │
|
||||
│ Deck Builder · Sim Dashboard · Training · Coach │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ HTTP/REST + WebSocket
|
||||
┌────────────────────────┴────────────────────────────────────────┐
|
||||
│ FastAPI Server (lab_api.py / routes/ — port 8080) │
|
||||
│ /api/lab/* /api/ml/* /api/coach/* /api/deck-builder/* │
|
||||
└────────┬──────────────────┬──────────────────┬──────────────────┘
|
||||
│ │ │
|
||||
Java subprocess Python sim engine Ollama (local LLM)
|
||||
(Forge rules) (Monte Carlo) qwen2.5:7b + others
|
||||
│ │
|
||||
┌────────┴───────┐ ┌───────┴────────────────────────────────────┐
|
||||
│ Forge Engine │ │ src/commander_ai_lab/ │
|
||||
│ (rules- │ │ sim/ — engine, models, rules │
|
||||
│ complete) │ │ deck_builder/ — 7-step LLM pipeline │
|
||||
└────────────────┘ │ coach/ — streaming LLM coach │
|
||||
└────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Quick Start (Windows)
|
||||
|
||||
### Prerequisites
|
||||
- **Java 17+** (`java -version`)
|
||||
- **Python 3.10+** (`python --version`)
|
||||
- **Maven 3.8+** (`mvn --version`)
|
||||
- **Forge repo** cloned and built at `D:\ForgeCommander\forge-repo`
|
||||
**Prerequisites:** Java 17+, Python 3.10+, Maven 3.8+, [Forge](https://github.com/Card-Forge/forge) built at `D:\ForgeCommander\forge-repo`
|
||||
|
||||
### 1. Build Forge (one-time)
|
||||
```powershell
|
||||
# 1. Build Forge (one-time)
|
||||
cd D:\ForgeCommander\forge-repo
|
||||
mvn package -pl forge-gui-desktop "-DskipTests" "-Drevision=2.0.12-SNAPSHOT" -am
|
||||
```
|
||||
|
||||
### 2. Build Commander AI Lab
|
||||
```powershell
|
||||
# 2. Build Commander AI Lab
|
||||
cd D:\ForgeCommander\commander-ai-lab
|
||||
mvn package "-DskipTests"
|
||||
```
|
||||
This produces `target/commander-ai-lab-1.0.0-SNAPSHOT.jar` (shaded fat JAR).
|
||||
|
||||
### 3. Install Python dependencies
|
||||
```powershell
|
||||
pip install fastapi uvicorn
|
||||
```
|
||||
# 3. Install Python deps
|
||||
pip install -e .
|
||||
|
||||
### 4. Launch the Web UI
|
||||
**Option A — One-click launcher:**
|
||||
```powershell
|
||||
# 4. Launch
|
||||
.\start-lab.bat
|
||||
# → open http://localhost:8080
|
||||
```
|
||||
Edit the paths at the top of `start-lab.bat` if needed.
|
||||
|
||||
**Option B — Manual launch:**
|
||||
## Overnight AFK Runner
|
||||
|
||||
Runs batch sims unattended, then updates card weights and trains the neural net:
|
||||
|
||||
```powershell
|
||||
python lab_api.py `
|
||||
--forge-jar "D:\ForgeCommander\forge-repo\forge-gui-desktop\target\forge-gui-desktop-2.0.12-SNAPSHOT-jar-with-dependencies.jar" `
|
||||
--forge-dir "D:\ForgeCommander\forge-repo\forge-gui" `
|
||||
--port 8080
|
||||
python overnight-run.py # default: 11.5h DeepSeek sims
|
||||
python overnight-run.py --batches 20 --games 50 # fixed batch count
|
||||
python overnight-run.py --no-update-weights # skip weight update
|
||||
python overnight-run.py --no-train # skip neural net training
|
||||
```
|
||||
|
||||
Open `http://localhost:8080` — select 3 decks, set game count, click **Run AI Sims**.
|
||||
**What happens after sims complete:**
|
||||
1. `update_weights.py` — nudges `learned_weights.json` from decision logs (feeds deck builder)
|
||||
2. `/api/ml/train` — trains the neural net on the full decision dataset
|
||||
|
||||
## Sim → Deck Builder Feedback Loop
|
||||
|
||||
Simulation outcomes automatically improve deck recommendations:
|
||||
|
||||
```
|
||||
Overnight sims → decisions_*.jsonl
|
||||
↓
|
||||
update_weights.py → learned_weights.json
|
||||
↓
|
||||
sim_insights.py → natural-language insight block
|
||||
↓
|
||||
Ollama prompts (suggest_cards + enforce_deck_ratios)
|
||||
↓
|
||||
Better card recommendations
|
||||
```
|
||||
|
||||
When active, `BuildResult.sources_consulted` includes `"sim_weights"`. No-op until real sim data exists.
|
||||
|
||||
## Deck Builder Pipeline
|
||||
|
||||
The `POST /api/deck-builder/build` endpoint runs 7 steps:
|
||||
|
||||
1. Resolve commander via Scryfall
|
||||
2. Fetch EDHrec recommendations
|
||||
3. Fetch Scryfall candidates by category (ramp, removal, draw, lands)
|
||||
4. Ollama suggests synergy/protection/wincon cards ← **sim insights injected here**
|
||||
5. Filter by color identity, collection, ban list
|
||||
6. Ollama enforces ratios (trim/expand to 99) ← **sim insights injected here**
|
||||
7. Assemble final deck
|
||||
|
||||
## ML Pipeline
|
||||
|
||||
### 5. CLI-only usage (no web server)
|
||||
```powershell
|
||||
cd D:\ForgeCommander\forge-repo\forge-gui
|
||||
# Update sim card weights from decision logs (runs automatically in overnight-run.py)
|
||||
python -m ml.scripts.update_weights --min-games 10 --lr 0.05 --dry-run
|
||||
|
||||
java -jar "D:\ForgeCommander\commander-ai-lab\target\commander-ai-lab-1.0.0-SNAPSHOT.jar" `
|
||||
--forge-jar "..\forge-gui-desktop\target\forge-gui-desktop-2.0.12-SNAPSHOT-jar-with-dependencies.jar" `
|
||||
--forge-dir "." `
|
||||
--deck1 "Edgar Markov" `
|
||||
--deck2 "Grimgrin" `
|
||||
--deck3 "Xyris" `
|
||||
--games 100 `
|
||||
--threads 4 `
|
||||
--output results/batch-100.json
|
||||
# Reset weights to defaults
|
||||
python -m ml.scripts.update_weights --reset
|
||||
|
||||
# Generate synthetic training data
|
||||
python ml/scripts/generate_synthetic_data.py
|
||||
```
|
||||
|
||||
## Scaling Tests
|
||||
Learned weights are clamped to `[-20.0, 20.0]` and stored in `src/commander_ai_lab/sim/learned_weights.json`.
|
||||
|
||||
### Single-threaded baseline (10 games)
|
||||
```powershell
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar `
|
||||
--forge-jar "..." --forge-dir "..." `
|
||||
--deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" `
|
||||
--games 10 --threads 1 --output results/baseline-10.json
|
||||
## Python Simulator API
|
||||
|
||||
```python
|
||||
from commander_ai_lab.lab.experiments import build_deck, run_simulation
|
||||
|
||||
deck = build_deck(["Sol Ring", "Cultivate", "Lightning Bolt"] + ["Forest"] * 36)
|
||||
stats = run_simulation(deck, num_games=100, name_a="My Deck")
|
||||
print(f"Win rate: {stats['win_rate']:.1%}, Avg turns: {stats['avg_turns']:.1f}")
|
||||
```
|
||||
|
||||
### Multi-threaded (10 games, 2 threads)
|
||||
```powershell
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar `
|
||||
--forge-jar "..." --forge-dir "..." `
|
||||
--deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" `
|
||||
--games 10 --threads 2 --output results/mt-10-t2.json
|
||||
```
|
||||
|
||||
### Scale to 100 games (4 threads)
|
||||
```powershell
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar `
|
||||
--forge-jar "..." --forge-dir "..." `
|
||||
--deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" `
|
||||
--games 100 --threads 4 --output results/scale-100-t4.json
|
||||
```
|
||||
|
||||
### Maximum throughput test (100 games, 8 threads)
|
||||
```powershell
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar `
|
||||
--forge-jar "..." --forge-dir "..." `
|
||||
--deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" `
|
||||
--games 100 --threads 8 --seed 42 --output results/scale-100-t8-seed42.json
|
||||
```
|
||||
|
||||
### Performance comparison script
|
||||
Run different thread counts and compare sims/sec:
|
||||
```powershell
|
||||
# Baseline: 1 thread
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar --forge-jar "..." --forge-dir "..." --deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" --games 20 --threads 1 --seed 42 --output results/perf-t1.json
|
||||
|
||||
# 2 threads
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar --forge-jar "..." --forge-dir "..." --deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" --games 20 --threads 2 --seed 42 --output results/perf-t2.json
|
||||
|
||||
# 4 threads
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar --forge-jar "..." --forge-dir "..." --deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" --games 20 --threads 4 --seed 42 --output results/perf-t4.json
|
||||
|
||||
# 8 threads
|
||||
java -jar target\commander-ai-lab-1.0.0-SNAPSHOT.jar --forge-jar "..." --forge-dir "..." --deck1 "Edgar Markov" --deck2 "Grimgrin" --deck3 "Xyris" --games 20 --threads 8 --seed 42 --output results/perf-t8.json
|
||||
```
|
||||
|
||||
Then compare the `simsPerSecond` values in each output JSON to see scaling behavior.
|
||||
The Monte Carlo engine models: combat keywords (flying, trample, deathtouch, lifelink, menace, reach, haste), summoning sickness, tapped/untap state, commander zone, multiplayer attack targeting, and board wipes. It does **not** model the stack, instants, or continuous type-changing effects.
|
||||
|
||||
## API Reference
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|---|---|---|
|
||||
| `/api/lab/decks` | GET | List available Commander decks |
|
||||
| `/api/lab/start` | POST | Start a batch run (returns batchId) |
|
||||
| `/api/lab/status?batchId=X` | GET | Poll progress for a running batch |
|
||||
| `/api/lab/result?batchId=X` | GET | Get completed batch result JSON |
|
||||
| `/api/lab/log?batchId=X` | GET | Get live log lines from running batch |
|
||||
| `/api/lab/history` | GET | List past batch result files |
|
||||
|
||||
### POST /api/lab/start
|
||||
```json
|
||||
{
|
||||
"decks": ["Edgar Markov", "Grimgrin", "Xyris"],
|
||||
"numGames": 100,
|
||||
"threads": 4,
|
||||
"seed": null,
|
||||
"clock": 120
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/lab/status response
|
||||
```json
|
||||
{
|
||||
"batchId": "abc123",
|
||||
"running": true,
|
||||
"completed": 42,
|
||||
"total": 100,
|
||||
"threads": 4,
|
||||
"elapsedMs": 156000,
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
## Python Simulator (Monte Carlo Engine)
|
||||
|
||||
A pure-Python deck simulator for rapid Monte Carlo analysis — no Forge JVM required.
|
||||
Ported from the [mtg-commander-lan](https://github.com/) Node.js simulator.
|
||||
|
||||
### Location
|
||||
|
||||
```
|
||||
src/commander_ai_lab/
|
||||
├── __init__.py
|
||||
├── sim/ # Core simulation engine
|
||||
│ ├── __init__.py # Public exports
|
||||
│ ├── models.py # Card, Player, SimState, GameResult, PlayerStats
|
||||
│ ├── engine.py # GameEngine — turn loop, combat (flying/trample/deathtouch/lifelink/menace/reach)
|
||||
│ └── rules.py # AI_DEFAULT_WEIGHTS, enrich_card(), score_card(), parse_decklist()
|
||||
└── lab/ # High-level experiment helpers
|
||||
├── __init__.py
|
||||
└── experiments.py # build_deck(), run_single_game(), run_simulation(), _cli_main()
|
||||
```
|
||||
|
||||
### Install as editable package
|
||||
|
||||
```powershell
|
||||
cd D:\ForgeCommander\commander-ai-lab
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
This installs `commander_ai_lab` as an importable package and registers the `commander-sim` CLI command.
|
||||
|
||||
### Quick smoke test
|
||||
|
||||
```powershell
|
||||
python -m commander_ai_lab.lab.experiments
|
||||
```
|
||||
|
||||
Expected output — a 10-game Monte Carlo run of a sample "Korvold" deck:
|
||||
|
||||
```
|
||||
══════════════════════════════════════════
|
||||
Commander AI Lab — Experiment Runner
|
||||
══════════════════════════════════════════
|
||||
|
||||
--- Demo: Korvold Aristocrats (10 games) ---
|
||||
|
||||
Win rate: XX.X%
|
||||
Avg turns: XX.X
|
||||
Avg damage dealt: XX.X
|
||||
Avg spells cast: XX.X
|
||||
Avg creatures: XX.X
|
||||
Elapsed: 0.02s
|
||||
|
||||
Done.
|
||||
```
|
||||
|
||||
### Python API
|
||||
|
||||
```python
|
||||
from commander_ai_lab.sim.models import Card
|
||||
from commander_ai_lab.sim.engine import GameEngine
|
||||
from commander_ai_lab.lab.experiments import build_deck, run_simulation
|
||||
|
||||
# Build a deck from card names
|
||||
deck = build_deck(["Sol Ring", "Cultivate", "Lightning Bolt", "Forest"] + ["Forest"] * 36)
|
||||
|
||||
# Monte Carlo simulation (100 games)
|
||||
stats = run_simulation(deck, num_games=100, name_a="My Deck")
|
||||
print(f"Win rate: {stats['win_rate']:.1%}")
|
||||
print(f"Avg turns: {stats['avg_turns']:.1f}")
|
||||
```
|
||||
|
||||
Key functions:
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `build_deck(card_names)` | Create enriched `Card` list from names |
|
||||
| `build_deck_from_text(decklist)` | Parse `"1 Sol Ring\n37 Forest"` format |
|
||||
| `run_single_game(deck_a, deck_b)` | One game, returns winner/turns/stats |
|
||||
| `run_simulation(deck_a, deck_b, n)` | Monte Carlo over `n` games, aggregated stats |
|
||||
|
||||
### CLI entry point
|
||||
|
||||
After `pip install -e .`:
|
||||
|
||||
```powershell
|
||||
commander-sim
|
||||
```
|
||||
|
||||
Runs the same demo as `python -m commander_ai_lab.lab.experiments`.
|
||||
|
||||
### How the engine works
|
||||
|
||||
1. **Deck enrichment** — `enrich_card()` infers type, CMC, keywords, and power/toughness from card name heuristics when Scryfall data is unavailable.
|
||||
2. **AI scoring** — `score_card()` uses weighted categories (removal, ramp, draw, aggro, etc.) to pick the best play each turn.
|
||||
3. **Game loop** — Each turn: untap → draw → play lands → cast spells (AI scored) → combat (attack/block decisions) → end step. Full keyword support for flying, trample, deathtouch, lifelink, menace, and reach.
|
||||
4. **Monte Carlo** — `run_simulation()` runs N independent games and aggregates win rate, avg turns, damage dealt/received, spells cast, creatures played, removal used, ramp played, cards drawn, and max board size.
|
||||
| `/api/lab/start` | POST | Start Forge batch (returns batchId) |
|
||||
| `/api/lab/start-deepseek` | POST | Start Python/DeepSeek batch |
|
||||
| `/api/lab/status?batchId=X` | GET | Poll batch progress |
|
||||
| `/api/lab/result?batchId=X` | GET | Get completed batch JSON |
|
||||
| `/api/lab/history` | GET | List past batch runs |
|
||||
| `/api/ml/train` | POST | Start neural net training |
|
||||
| `/api/ml/train/status` | GET | Poll training progress |
|
||||
| `/api/coach/chat` | POST | LLM coaching session (streaming) |
|
||||
| `/api/deck-builder/build` | POST | Build a 99-card Commander deck |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
commander-ai-lab/
|
||||
├── lab_api.py # Python FastAPI server (web UI backend)
|
||||
├── overnight-run.py # AFK batch runner
|
||||
├── start-lab.bat # Windows one-click launcher
|
||||
├── pom.xml # Maven build (Java 17, shaded JAR)
|
||||
├── pyproject.toml # Python package config (src/ layout)
|
||||
├── ui/
|
||||
│ ├── index.html # Web UI entry point
|
||||
│ ├── ai-lab.js # UI controller (deck selection, progress, results)
|
||||
│ ├── ai-lab.css # Styles (dark theme, responsive)
|
||||
│ ├── collection.html # Card collection manager
|
||||
│ ├── collection.js # Collection controller
|
||||
│ ├── collection.css # Collection styles
|
||||
│ ├── deckbuilder.html # Deck builder + EDH advisor
|
||||
│ ├── deckbuilder.js # Deck builder controller
|
||||
│ └── deckbuilder.css # Deck builder styles
|
||||
├── src/commander_ai_lab/ # Python simulator package
|
||||
│ ├── __init__.py
|
||||
│ ├── sim/ # Core engine (models, engine, rules)
|
||||
│ └── lab/ # Experiment helpers
|
||||
├── src/main/java/commanderailab/ # Java batch runner (Forge integration)
|
||||
│ ├── cli/LabCli.java # CLI entry point (picocli)
|
||||
│ ├── batch/
|
||||
│ │ ├── BatchRunner.java # Single-thread runner (parses Forge output)
|
||||
│ │ └── MultiThreadBatchRunner.java # Multi-thread runner (ExecutorService)
|
||||
│ ├── bridge/WebApiBridge.java # Java-side web API bridge
|
||||
│ ├── schema/
|
||||
│ │ ├── BatchResult.java # Data model POJOs
|
||||
│ │ └── JsonExporter.java # JSON serialization + validation
|
||||
│ ├── stats/StatsAggregator.java # Win rates, turn counts, breakdowns
|
||||
│ └── ai/
|
||||
│ ├── AiPolicy.java # Policy interface
|
||||
│ └── ForgeBuiltinPolicy.java # v1 Forge AI wrapper
|
||||
├── scanner/ # Card scanner (OCR pipeline)
|
||||
│ ├── __init__.py
|
||||
│ ├── ocr.py # Tesseract OCR wrapper
|
||||
│ ├── pipeline.py # Image preprocessing pipeline
|
||||
│ └── preprocess.py # Image preprocessing helpers
|
||||
├── docs/
|
||||
│ ├── PROJECT_VISION.md
|
||||
│ ├── batch-result-schema.json
|
||||
│ └── game-log-schema.json
|
||||
├── sample-decks/ # 3 sample .dck files
|
||||
├── precon-decks/ # Preconstructed decks + index
|
||||
├── results/ # Batch output JSON files (gitignored)
|
||||
└── src/test/java/ # Unit tests
|
||||
├── lab_api.py # FastAPI server entry point
|
||||
├── frontend/ # React + TypeScript web UI
|
||||
├── ui/ # Legacy static HTML UI
|
||||
├── src/commander_ai_lab/
|
||||
│ ├── sim/ # Monte Carlo engine (engine, models, rules)
|
||||
│ ├── deck_builder/ # LLM deck builder pipeline
|
||||
│ │ ├── api/ # Scryfall, EDHrec, Ollama, sim_insights
|
||||
│ │ ├── core/ # Models, rules engine, assembler
|
||||
│ │ └── pipeline/ # build_deck.py (7-step orchestrator)
|
||||
│ └── coach/ # LLM coaching (Anthropic + Ollama)
|
||||
├── ml/
|
||||
│ ├── scripts/ # update_weights.py, ml_cli.py
|
||||
│ ├── training/ # Neural net trainer
|
||||
│ ├── encoder/ # Feature encoding
|
||||
│ └── models/ # Saved model checkpoints
|
||||
├── scanner/ # Card OCR pipeline
|
||||
├── logs/decisions/ # ML decision logs (gitignored)
|
||||
└── src/main/java/commanderailab/ # Java/Forge batch runner
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
Each batch run produces JSON conforming to `docs/batch-result-schema.json`:
|
||||
```json
|
||||
{
|
||||
"metadata": {
|
||||
"schemaVersion": "1.0.0",
|
||||
"batchId": "uuid",
|
||||
"timestamp": "2026-03-06T...",
|
||||
"totalGames": 100,
|
||||
"completedGames": 100,
|
||||
"format": "commander",
|
||||
"podSize": 3,
|
||||
"engineVersion": "forge-2.0.12-SNAPSHOT",
|
||||
"masterSeed": 42,
|
||||
"threads": 4,
|
||||
"elapsedMs": 260000
|
||||
},
|
||||
"decks": [...],
|
||||
"games": [...],
|
||||
"summary": {
|
||||
"perDeck": [
|
||||
{ "deckName": "Edgar Markov", "winRate": 0.60, "wins": 60, ... }
|
||||
],
|
||||
"avgGameTurns": 12.3,
|
||||
"simsPerSecond": 0.38
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Frontend Development
|
||||
|
||||
The UI is a React + TypeScript SPA in the `frontend/` directory.
|
||||
|
||||
### Dev server (with hot reload)
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
Vite proxies `/api` and `/ws` to `localhost:8000`, so start the FastAPI backend first.
|
||||
|
||||
### Production build
|
||||
```bash
|
||||
cd frontend
|
||||
npm run build
|
||||
```
|
||||
FastAPI serves the built assets from `frontend/dist/`.
|
||||
|
||||
## Known Limitations
|
||||
- Winner's life total not captured (Forge quiet mode only reports losers)
|
||||
- Win condition classification is heuristic-based on Forge loss reason strings
|
||||
- Each game spawns a new JVM process (~2-3s overhead per game)
|
||||
- Max throughput ~0.04 sims/sec single-threaded (Forge sim is CPU-heavy)
|
||||
- Multithreading spawns parallel subprocesses — throughput scales linearly with threads up to CPU core count
|
||||
|
||||
- **Monte Carlo engine** — no stack, no instants, no continuous type changes (animated lands don't attack)
|
||||
- **Forge engine** — each game spawns a new JVM (~2-3s overhead), ~0.04 sims/sec single-threaded
|
||||
- **Forge winner life total** not captured (quiet mode only reports losers)
|
||||
- **Deck builder** — Ollama suggestions are only as good as the local model; `qwen2.5:7b` works well
|
||||
@@ -4,6 +4,9 @@ Commander AI Lab — Anthropic Claude Structured Output Client
|
||||
Uses the anthropic SDK to call Claude's messages endpoint
|
||||
with structured JSON output enforced via system prompt.
|
||||
Supports both sync and native async operation.
|
||||
|
||||
All substantive calls use .messages.stream() to avoid the SDK's
|
||||
10-minute non-streaming timeout restriction on large max_tokens.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
@@ -53,6 +56,8 @@ class AnthropicClient:
|
||||
Anthropic Claude API client.
|
||||
|
||||
Drop-in replacement for PerplexityClient.
|
||||
All substantive calls use .messages.stream() so the SDK never
|
||||
enforces the 10-minute non-streaming timeout.
|
||||
|
||||
Usage:
|
||||
client = AnthropicClient(api_key="sk-ant-...")
|
||||
@@ -101,37 +106,24 @@ class AnthropicClient:
|
||||
) -> AnthropicResponse:
|
||||
"""
|
||||
Send a chat request expecting JSON output.
|
||||
|
||||
The system_prompt should instruct the model to return JSON matching
|
||||
the expected schema. Claude Opus follows JSON instructions reliably.
|
||||
|
||||
Args:
|
||||
system_prompt: System message (should instruct JSON output)
|
||||
user_prompt: User message
|
||||
json_schema: (optional) Kept for call-site compatibility; appended
|
||||
as a schema hint in the system prompt when provided
|
||||
schema_name: (optional) Label used in the schema hint
|
||||
model: Override model
|
||||
temperature: Sampling temperature (0.0-1.0)
|
||||
max_tokens: Max output tokens
|
||||
|
||||
Returns:
|
||||
AnthropicResponse with parsed_json extracted from model output
|
||||
Uses streaming to avoid the Anthropic SDK 10-minute timeout restriction.
|
||||
"""
|
||||
use_model = model or self.model
|
||||
effective_system = self._build_system(system_prompt, json_schema, schema_name)
|
||||
|
||||
try:
|
||||
response = self._client.messages.create(
|
||||
with self._client.messages.stream(
|
||||
model=use_model,
|
||||
system=effective_system,
|
||||
messages=[{"role": "user", "content": user_prompt}],
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
)
|
||||
return self._parse_response(response, use_model)
|
||||
) as stream:
|
||||
content = stream.get_final_text()
|
||||
final_msg = stream.get_final_message()
|
||||
return self._parse_streamed(content, final_msg, use_model)
|
||||
except Exception as e:
|
||||
logger.error("[anthropic] API call failed: %s", e)
|
||||
logger.error("[anthropic] chat_structured failed: %s", e)
|
||||
raise
|
||||
|
||||
def chat_plain(
|
||||
@@ -144,21 +136,23 @@ class AnthropicClient:
|
||||
) -> AnthropicResponse:
|
||||
"""
|
||||
Send a plain chat request without strict structured output.
|
||||
Used for substitution fallback queries.
|
||||
Uses streaming to avoid the Anthropic SDK 10-minute timeout restriction.
|
||||
"""
|
||||
use_model = model or self.model
|
||||
|
||||
try:
|
||||
response = self._client.messages.create(
|
||||
with self._client.messages.stream(
|
||||
model=use_model,
|
||||
system=system_prompt,
|
||||
messages=[{"role": "user", "content": user_prompt}],
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
)
|
||||
return self._parse_response(response, use_model)
|
||||
) as stream:
|
||||
content = stream.get_final_text()
|
||||
final_msg = stream.get_final_message()
|
||||
return self._parse_streamed(content, final_msg, use_model)
|
||||
except Exception as e:
|
||||
logger.error("[anthropic] Plain chat failed: %s", e)
|
||||
logger.error("[anthropic] chat_plain failed: %s", e)
|
||||
raise
|
||||
|
||||
async def achat_structured(
|
||||
@@ -171,21 +165,23 @@ class AnthropicClient:
|
||||
temperature: float = 0.2,
|
||||
max_tokens: int = 8192,
|
||||
) -> AnthropicResponse:
|
||||
"""Native async structured chat — no run_in_executor needed."""
|
||||
"""Native async structured chat using streaming."""
|
||||
use_model = model or self.model
|
||||
effective_system = self._build_system(system_prompt, json_schema, schema_name)
|
||||
|
||||
try:
|
||||
response = await self._async_client.messages.create(
|
||||
async with self._async_client.messages.stream(
|
||||
model=use_model,
|
||||
system=effective_system,
|
||||
messages=[{"role": "user", "content": user_prompt}],
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
)
|
||||
return self._parse_response(response, use_model)
|
||||
) as stream:
|
||||
content = await stream.get_final_text()
|
||||
final_msg = await stream.get_final_message()
|
||||
return self._parse_streamed(content, final_msg, use_model)
|
||||
except Exception as e:
|
||||
logger.error("[anthropic] Async structured chat failed: %s", e)
|
||||
logger.error("[anthropic] achat_structured failed: %s", e)
|
||||
raise
|
||||
|
||||
async def achat_plain(
|
||||
@@ -196,24 +192,29 @@ class AnthropicClient:
|
||||
temperature: float = 0.2,
|
||||
max_tokens: int = 4096,
|
||||
) -> AnthropicResponse:
|
||||
"""Native async plain chat."""
|
||||
"""Native async plain chat using streaming."""
|
||||
use_model = model or self.model
|
||||
|
||||
try:
|
||||
response = await self._async_client.messages.create(
|
||||
async with self._async_client.messages.stream(
|
||||
model=use_model,
|
||||
system=system_prompt,
|
||||
messages=[{"role": "user", "content": user_prompt}],
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
)
|
||||
return self._parse_response(response, use_model)
|
||||
) as stream:
|
||||
content = await stream.get_final_text()
|
||||
final_msg = await stream.get_final_message()
|
||||
return self._parse_streamed(content, final_msg, use_model)
|
||||
except Exception as e:
|
||||
logger.error("[anthropic] Async plain chat failed: %s", e)
|
||||
logger.error("[anthropic] achat_plain failed: %s", e)
|
||||
raise
|
||||
|
||||
def check_status(self) -> dict:
|
||||
"""Quick health check — try a minimal API call."""
|
||||
"""
|
||||
Quick health check — tiny max_tokens=10 call, safe to use non-streaming
|
||||
since it will never approach the 10-minute threshold.
|
||||
"""
|
||||
try:
|
||||
response = self._client.messages.create(
|
||||
model=self.model,
|
||||
@@ -249,13 +250,9 @@ class AnthropicClient:
|
||||
f"{schema_str}"
|
||||
)
|
||||
|
||||
def _parse_response(self, response, use_model: str) -> AnthropicResponse:
|
||||
"""Extract content and attempt JSON parsing from an Anthropic response."""
|
||||
content = ""
|
||||
if response.content:
|
||||
content = response.content[0].text or ""
|
||||
|
||||
usage = response.usage
|
||||
def _parse_streamed(self, content: str, final_msg, use_model: str) -> AnthropicResponse:
|
||||
"""Build an AnthropicResponse from streamed text + final message metadata."""
|
||||
usage = getattr(final_msg, "usage", None)
|
||||
prompt_tokens = usage.input_tokens if usage else 0
|
||||
completion_tokens = usage.output_tokens if usage else 0
|
||||
|
||||
|
||||
+20
-41
@@ -65,10 +65,8 @@ class CoachService:
|
||||
|
||||
def load_deck_report(self, deck_id: str) -> Optional[DeckReport]:
|
||||
"""Load a deck report from disk."""
|
||||
# Try exact filename first, then case-insensitive search
|
||||
report_path = DECK_REPORTS_DIR / f"{deck_id}.json"
|
||||
if not report_path.exists():
|
||||
# Try case-insensitive match
|
||||
for f in DECK_REPORTS_DIR.glob("*.json"):
|
||||
if re.sub(r'[^a-z0-9]', '', f.stem.lower()) == re.sub(r'[^a-z0-9]', '', deck_id.lower()):
|
||||
report_path = f
|
||||
@@ -96,12 +94,10 @@ class CoachService:
|
||||
|
||||
def save_session(self, session: CoachSession) -> Path:
|
||||
"""Persist a coaching session to disk and SQLite."""
|
||||
# 1. Write JSON file (backward compat)
|
||||
COACH_SESSIONS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
path = COACH_SESSIONS_DIR / f"{session.sessionId}.json"
|
||||
with open(path, "w") as f:
|
||||
f.write(session.model_dump_json(indent=2))
|
||||
# 2. Write to SQLite
|
||||
if _DB_AVAILABLE:
|
||||
try:
|
||||
_db_save_session({
|
||||
@@ -124,10 +120,8 @@ class CoachService:
|
||||
logger.warning("Failed to persist session to SQLite: %s", e)
|
||||
return path
|
||||
|
||||
|
||||
def load_session(self, session_id: str) -> Optional[CoachSession]:
|
||||
"""Load a coaching session from SQLite, falling back to disk."""
|
||||
# Try SQLite first
|
||||
if _DB_AVAILABLE:
|
||||
try:
|
||||
row = _db_load_session(session_id)
|
||||
@@ -150,7 +144,6 @@ class CoachService:
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("SQLite load failed for %s: %s", session_id, e)
|
||||
# Fallback to disk
|
||||
path = COACH_SESSIONS_DIR / f"{session_id}.json"
|
||||
if not path.exists():
|
||||
return None
|
||||
@@ -162,10 +155,8 @@ class CoachService:
|
||||
logger.error("Failed to load session %s: %s", session_id, e)
|
||||
return None
|
||||
|
||||
|
||||
def list_sessions(self, deck_id: str = None) -> List[dict]:
|
||||
"""List coaching sessions from SQLite, falling back to disk."""
|
||||
# Try SQLite first
|
||||
if _DB_AVAILABLE:
|
||||
try:
|
||||
rows = _db_list_sessions(deck_id)
|
||||
@@ -173,7 +164,6 @@ class CoachService:
|
||||
return rows
|
||||
except Exception as e:
|
||||
logger.warning("SQLite list_sessions failed: %s", e)
|
||||
# Fallback to disk
|
||||
if not COACH_SESSIONS_DIR.exists():
|
||||
return []
|
||||
sessions = []
|
||||
@@ -195,20 +185,21 @@ class CoachService:
|
||||
continue
|
||||
return sessions
|
||||
|
||||
|
||||
# ── Main Coaching Pipeline ─────────────────────────────────
|
||||
|
||||
async def run_coaching_session(self, deck_id: str,
|
||||
goals: Optional[CoachGoals] = None,
|
||||
fallback_report: Optional[DeckReport] = None,
|
||||
) -> CoachSession:
|
||||
async def run_coaching_session(
|
||||
self,
|
||||
deck_id: str,
|
||||
goals: Optional[CoachGoals] = None,
|
||||
fallback_report: Optional[DeckReport] = None,
|
||||
) -> CoachSession:
|
||||
"""
|
||||
Full coaching pipeline:
|
||||
1. Load deck report (or use fallback from DB card list)
|
||||
2. Identify underperformers
|
||||
3. Find replacement candidates via embeddings
|
||||
4. Build prompt
|
||||
5. Call LLM
|
||||
5. Call LLM (streaming for Anthropic to avoid 10-min SDK timeout)
|
||||
6. Parse response
|
||||
7. Persist session
|
||||
"""
|
||||
@@ -224,7 +215,7 @@ class CoachService:
|
||||
except Exception as e:
|
||||
logger.warning("Embeddings auto-load failed: %s", e)
|
||||
|
||||
# 1. Load deck report — try simulation report first, then fallback
|
||||
# 1. Load deck report
|
||||
report = self.load_deck_report(deck_id)
|
||||
if report is None and fallback_report is not None:
|
||||
report = fallback_report
|
||||
@@ -237,20 +228,15 @@ class CoachService:
|
||||
|
||||
# 3. Find replacement candidates for underperformers
|
||||
candidates: Dict[str, List[dict]] = {}
|
||||
|
||||
underperformers = report.underperformers[:MAX_UNDERPERFORMERS]
|
||||
if not underperformers:
|
||||
# If no explicit underperformers, pick lowest impact cards
|
||||
sorted_cards = sorted(report.cards, key=lambda c: c.impactScore)
|
||||
underperformers = [
|
||||
c.name for c in sorted_cards[:MAX_UNDERPERFORMERS]
|
||||
if c.impactScore < UNDERPERFORMER_IMPACT_THRESHOLD
|
||||
]
|
||||
|
||||
# If still no underperformers and we have cards, pick bottom N by impact
|
||||
if not underperformers and report.cards:
|
||||
sorted_cards = sorted(report.cards, key=lambda c: c.impactScore)
|
||||
# Take the bottom 5 non-land cards as candidates for cuts
|
||||
underperformers = [
|
||||
c.name for c in sorted_cards[:5]
|
||||
if not any(t in (c.tags or []) for t in [])
|
||||
@@ -273,29 +259,33 @@ class CoachService:
|
||||
system_prompt = build_system_prompt(report, goals)
|
||||
user_prompt = build_user_prompt(report, candidates)
|
||||
|
||||
# 5. Call LLM (Anthropic or local Ollama)
|
||||
# 5. Call LLM
|
||||
logger.info("Calling LLM for deck: %s (provider=%s)", deck_id, COACH_PROVIDER)
|
||||
if COACH_PROVIDER == "anthropic":
|
||||
import anthropic
|
||||
import anthropic as _anthropic
|
||||
anthropic_key = ANTHROPIC_API_KEY
|
||||
if not anthropic_key:
|
||||
raise ConnectionError("Anthropic API key not configured. Set ANTHROPIC_API_KEY env var.")
|
||||
aclient = anthropic.AsyncAnthropic(api_key=anthropic_key)
|
||||
resp = await aclient.messages.create(
|
||||
|
||||
# Use streaming to avoid the Anthropic SDK 10-minute non-streaming timeout.
|
||||
aclient = _anthropic.AsyncAnthropic(api_key=anthropic_key)
|
||||
async with aclient.messages.stream(
|
||||
model=ANTHROPIC_MODEL,
|
||||
system=system_prompt,
|
||||
messages=[{"role": "user", "content": user_prompt}],
|
||||
max_tokens=DEFAULT_MAX_TOKENS,
|
||||
temperature=0.7,
|
||||
)
|
||||
content = resp.content[0].text if resp.content else ""
|
||||
usage = resp.usage
|
||||
) as stream:
|
||||
content = await stream.get_final_text()
|
||||
final_msg = await stream.get_final_message()
|
||||
|
||||
usage = getattr(final_msg, "usage", None)
|
||||
|
||||
# Build an LLMResponse-compatible object
|
||||
from types import SimpleNamespace
|
||||
llm_response = SimpleNamespace(
|
||||
content=content,
|
||||
model=resp.model,
|
||||
model=final_msg.model,
|
||||
prompt_tokens=usage.input_tokens if usage else 0,
|
||||
completion_tokens=usage.output_tokens if usage else 0,
|
||||
parsed_json=None,
|
||||
@@ -305,7 +295,6 @@ class CoachService:
|
||||
cleaned = re.sub(r'\s*```$', '', cleaned.strip())
|
||||
llm_response.parsed_json = json.loads(cleaned)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
# Try to find JSON object in the text
|
||||
match = re.search(r'\{[\s\S]*\}', content)
|
||||
if match:
|
||||
try:
|
||||
@@ -317,7 +306,6 @@ class CoachService:
|
||||
|
||||
# 6. Parse response into CoachSession
|
||||
session_id = f"sess-{datetime.now(timezone.utc).strftime('%Y%m%d-%H%M%S')}-{uuid.uuid4().hex[:6]}"
|
||||
|
||||
session = CoachSession(
|
||||
sessionId=session_id,
|
||||
deckId=deck_id,
|
||||
@@ -331,14 +319,12 @@ class CoachService:
|
||||
if llm_response.parsed_json:
|
||||
self._populate_session_from_json(session, llm_response.parsed_json)
|
||||
else:
|
||||
# Fallback: store raw text
|
||||
session.rawTextExplanation = llm_response.content
|
||||
session.summary = "LLM returned non-JSON response. See raw explanation."
|
||||
|
||||
# 7. Persist
|
||||
self.save_session(session)
|
||||
logger.info("Coaching session saved: %s", session_id)
|
||||
|
||||
return session
|
||||
|
||||
def _populate_session_from_json(self, session: CoachSession, data: dict):
|
||||
@@ -347,11 +333,9 @@ class CoachService:
|
||||
session.rawTextExplanation = data.get("rawTextExplanation", "")
|
||||
session.manaBaseAdvice = data.get("manaBaseAdvice")
|
||||
|
||||
# Heuristic hints
|
||||
hints = data.get("heuristicHints", [])
|
||||
session.heuristicHints = hints if isinstance(hints, list) else [str(hints)]
|
||||
|
||||
# Suggested cuts
|
||||
for cut_data in data.get("suggestedCuts", []):
|
||||
if isinstance(cut_data, dict):
|
||||
session.suggestedCuts.append(SuggestedCut(
|
||||
@@ -360,7 +344,6 @@ class CoachService:
|
||||
replacementOptions=cut_data.get("replacementOptions", []),
|
||||
))
|
||||
|
||||
# Suggested adds
|
||||
for add_data in data.get("suggestedAdds", []):
|
||||
if isinstance(add_data, dict):
|
||||
session.suggestedAdds.append(SuggestedAdd(
|
||||
@@ -376,7 +359,6 @@ class CoachService:
|
||||
"""Check the health of all coach subsystems."""
|
||||
status = CoachStatus()
|
||||
|
||||
# Check LLM / Anthropic provider
|
||||
if COACH_PROVIDER == "anthropic":
|
||||
anthropic_key = ANTHROPIC_API_KEY
|
||||
if anthropic_key:
|
||||
@@ -396,11 +378,8 @@ class CoachService:
|
||||
if not status.llmConnected:
|
||||
status.error = llm_status.get("error", "Local LLM not reachable")
|
||||
|
||||
# Check embeddings
|
||||
status.embeddingsLoaded = self.embeddings.loaded
|
||||
status.embeddingCards = self.embeddings.card_count
|
||||
|
||||
# Check deck reports
|
||||
status.deckReportsAvailable = len(self.list_deck_reports())
|
||||
|
||||
return status
|
||||
+75
-4
@@ -25,6 +25,7 @@ import json
|
||||
import logging
|
||||
import logging.handlers
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
@@ -42,6 +43,9 @@ DEFAULT_CONFIG = {
|
||||
"time_limit_hours": 11.5, # stop after this many hours (leave buffer for training)
|
||||
"pause_between_batches": 10, # seconds to pause between batches
|
||||
"auto_train": True, # trigger ML training after all batches
|
||||
"auto_update_weights": True, # run update_weights.py after sims
|
||||
"weights_min_games": 10, # minimum games required for weight update
|
||||
"weights_lr": 0.05, # learning rate for weight update
|
||||
"train_epochs": 100, # training epochs
|
||||
"train_lr": 0.001, # learning rate
|
||||
"train_batch_size": 256, # training batch size
|
||||
@@ -229,6 +233,52 @@ def wait_for_batch(base_url, batch_id, engine="deepseek", deadline=None):
|
||||
time.sleep(poll_interval)
|
||||
|
||||
|
||||
def run_update_weights(cfg):
|
||||
"""
|
||||
Run ml.scripts.update_weights to nudge learned_weights.json from sim
|
||||
decision logs. This feeds the deck builder's sim_insights injection.
|
||||
|
||||
Returns True on success, False on failure (non-fatal — overnight run
|
||||
continues regardless).
|
||||
"""
|
||||
log.info("")
|
||||
log.info("── Auto-update sim weights ─────────────────────────────────")
|
||||
log.info("Running update_weights.py to refresh learned_weights.json...")
|
||||
|
||||
cmd = [
|
||||
sys.executable, "-m", "ml.scripts.update_weights",
|
||||
"--min-games", str(cfg["weights_min_games"]),
|
||||
"--lr", str(cfg["weights_lr"]),
|
||||
]
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120, # 2 min ceiling — this should complete in seconds
|
||||
)
|
||||
if result.returncode == 0:
|
||||
# Print the last ~800 chars of the weight report to the log
|
||||
output = (result.stdout or "").strip()
|
||||
if output:
|
||||
for line in output.splitlines()[-20:]:
|
||||
log.info(f" {line}")
|
||||
log.info("Sim weights updated — deck builder will use new insights on next build.")
|
||||
return True
|
||||
else:
|
||||
log.warning(f"update_weights.py exited with code {result.returncode}")
|
||||
if result.stderr:
|
||||
log.warning(result.stderr.strip()[-300:])
|
||||
return False
|
||||
except subprocess.TimeoutExpired:
|
||||
log.warning("update_weights.py timed out after 120s — skipping")
|
||||
return False
|
||||
except Exception as e:
|
||||
log.warning(f"update_weights.py failed: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def start_training(base_url, epochs, lr, batch_size, patience):
|
||||
"""Trigger ML training and return immediately."""
|
||||
resp = api_post(base_url, "/api/ml/train", {
|
||||
@@ -364,6 +414,7 @@ def run_overnight(cfg):
|
||||
else:
|
||||
log.info(f" Mode: Run for {cfg['time_limit_hours']}h, {games_per_batch} games/batch")
|
||||
log.info(f" Engine: {engine}")
|
||||
log.info(f" Auto-update weights: {'Yes' if cfg['auto_update_weights'] else 'No'}")
|
||||
log.info(f" Auto-train after: {'Yes' if cfg['auto_train'] else 'No'}")
|
||||
log.info("")
|
||||
|
||||
@@ -440,6 +491,17 @@ def run_overnight(cfg):
|
||||
log.info(f" Elapsed time: {str(timedelta(seconds=int(total_elapsed)))}")
|
||||
log.info("")
|
||||
|
||||
# ── Auto-update sim weights ───────────────────────────────
|
||||
# Runs update_weights.py to nudge learned_weights.json from the
|
||||
# decision logs produced during this run. This feeds sim_insights
|
||||
# into the deck builder's Ollama prompts on the next deck build.
|
||||
# Runs before neural net training so both pipelines benefit from
|
||||
# the freshest possible data.
|
||||
if cfg["auto_update_weights"]:
|
||||
run_update_weights(cfg)
|
||||
else:
|
||||
log.info("Skipping sim weight update (--no-update-weights flag set)")
|
||||
|
||||
# ── Auto-train ────────────────────────────────────────────
|
||||
if cfg["auto_train"]:
|
||||
if decisions_after < 100:
|
||||
@@ -482,17 +544,17 @@ def main():
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
# Run DeepSeek sims for 11.5 hours, then train:
|
||||
# Run DeepSeek sims for 11.5 hours, then update weights + train:
|
||||
python overnight-run.py
|
||||
|
||||
# Run 20 batches of 50 games each, then train:
|
||||
# Run 20 batches of 50 games each, then update weights + train:
|
||||
python overnight-run.py --batches 20 --games 50
|
||||
|
||||
# Run for 6 hours with Java engine:
|
||||
python overnight-run.py --engine java --hours 6 --decks "Deck A" "Deck B" "Deck C"
|
||||
|
||||
# Just run sims, no training:
|
||||
python overnight-run.py --no-train
|
||||
# Just run sims, skip both weight update and training:
|
||||
python overnight-run.py --no-train --no-update-weights
|
||||
|
||||
# Custom training settings:
|
||||
python overnight-run.py --epochs 200 --lr 0.0005 --patience 20
|
||||
@@ -514,6 +576,12 @@ Examples:
|
||||
help="Deck names (auto-detected if not specified)")
|
||||
parser.add_argument("--no-train", action="store_true",
|
||||
help="Skip auto-training after sims")
|
||||
parser.add_argument("--no-update-weights", action="store_true",
|
||||
help="Skip updating learned_weights.json after sims")
|
||||
parser.add_argument("--weights-min-games", type=int, default=10,
|
||||
help="Minimum games required for weight update (default: 10)")
|
||||
parser.add_argument("--weights-lr", type=float, default=0.05,
|
||||
help="Learning rate for sim weight update (default: 0.05)")
|
||||
parser.add_argument("--epochs", type=int, default=100,
|
||||
help="Training epochs (default: 100)")
|
||||
parser.add_argument("--lr", type=float, default=0.001,
|
||||
@@ -536,6 +604,9 @@ Examples:
|
||||
cfg["time_limit_hours"] = args.hours
|
||||
cfg["pause_between_batches"] = args.pause
|
||||
cfg["auto_train"] = not args.no_train
|
||||
cfg["auto_update_weights"] = not args.no_update_weights
|
||||
cfg["weights_min_games"] = args.weights_min_games
|
||||
cfg["weights_lr"] = args.weights_lr
|
||||
cfg["train_epochs"] = args.epochs
|
||||
cfg["train_lr"] = args.lr
|
||||
cfg["train_batch_size"] = args.batch_size
|
||||
|
||||
@@ -102,7 +102,7 @@ def chat(
|
||||
return response.choices[0].message.content or ""
|
||||
|
||||
|
||||
# ── 1. Analyze synergies ─────────────────────────────────────────
|
||||
# ── 1. Analyze synergies ─────────────────────────────────────────────
|
||||
|
||||
def analyze_synergies(
|
||||
commander_name: str,
|
||||
@@ -145,7 +145,7 @@ Cards to evaluate:
|
||||
return []
|
||||
|
||||
|
||||
# ── 2. Suggest cards for a category ──────────────────────────────
|
||||
# ── 2. Suggest cards for a category ────────────────────────────────
|
||||
|
||||
def suggest_cards(
|
||||
commander_name: str,
|
||||
@@ -154,13 +154,23 @@ def suggest_cards(
|
||||
count: int = 10,
|
||||
exclude: Optional[List[str]] = None,
|
||||
strategy_notes: Optional[str] = None,
|
||||
sim_context: str = "",
|
||||
) -> List[str]:
|
||||
"""
|
||||
Ask the model to suggest cards for a specific deck category.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
sim_context:
|
||||
Optional natural-language insight string from get_sim_insights().
|
||||
When non-empty, appended to the prompt so the LLM can factor in
|
||||
statistically successful card traits from simulation data.
|
||||
|
||||
Returns a list of card names.
|
||||
"""
|
||||
exclude_str = f"\nDo NOT suggest these cards: {json.dumps(exclude)}" if exclude else ""
|
||||
strategy = f"\nStrategy notes: {strategy_notes}" if strategy_notes else ""
|
||||
sim_block = f"\n\n{sim_context}" if sim_context else ""
|
||||
|
||||
prompt = f"""You are a Magic: The Gathering Commander deck-building expert.
|
||||
|
||||
@@ -168,7 +178,7 @@ Commander: {commander_name}
|
||||
Color identity: {', '.join(color_identity)}
|
||||
Category: {category}
|
||||
{strategy}
|
||||
{exclude_str}
|
||||
{exclude_str}{sim_block}
|
||||
|
||||
Suggest exactly {count} cards for the {category} category that work well with this commander.
|
||||
Only suggest cards that are legal in Commander and match the color identity.
|
||||
@@ -200,7 +210,7 @@ Return ONLY a JSON array of card name strings, like: ["Card A", "Card B", ...]""
|
||||
return []
|
||||
|
||||
|
||||
# ── 3. Filter by color identity ──────────────────────────────────
|
||||
# ── 3. Filter by color identity ────────────────────────────────────
|
||||
|
||||
def filter_color_identity(
|
||||
card_names: List[str],
|
||||
@@ -242,17 +252,28 @@ Cards:
|
||||
return card_names
|
||||
|
||||
|
||||
# ── 4. Enforce deck ratios ───────────────────────────────────────
|
||||
# ── 4. Enforce deck ratios ───────────────────────────────────────────
|
||||
|
||||
def enforce_deck_ratios(
|
||||
cards_by_category: Dict[str, List[str]],
|
||||
target_ratios: Dict[str, int],
|
||||
commander_name: str,
|
||||
sim_context: str = "",
|
||||
) -> Dict[str, List[str]]:
|
||||
"""
|
||||
Ask the model to trim/expand categories to hit target counts.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
sim_context:
|
||||
Optional natural-language insight string from get_sim_insights().
|
||||
When non-empty, appended to the prompt so the LLM preferentially
|
||||
retains cards matching statistically high-value traits when trimming.
|
||||
|
||||
Returns adjusted dict of category -> card names.
|
||||
"""
|
||||
sim_block = f"\n\n{sim_context}" if sim_context else ""
|
||||
|
||||
prompt = f"""You are a Magic: The Gathering Commander deck-building expert.
|
||||
|
||||
Commander: {commander_name}
|
||||
@@ -265,7 +286,7 @@ Target card counts per category:
|
||||
|
||||
Adjust each category to match the target count. Remove the weakest cards from
|
||||
over-filled categories. For under-filled categories, suggest additional cards.
|
||||
The total across all categories must be exactly 99.
|
||||
The total across all categories must be exactly 99.{sim_block}
|
||||
|
||||
Return a JSON object with the same category keys, each mapping to an array of card name strings."""
|
||||
|
||||
@@ -284,7 +305,7 @@ Return a JSON object with the same category keys, each mapping to an array of ca
|
||||
return cards_by_category
|
||||
|
||||
|
||||
# ── 5. Assemble final deck JSON ──────────────────────────────────
|
||||
# ── 5. Assemble final deck JSON ─────────────────────────────────────────
|
||||
|
||||
def assemble_deck_json(
|
||||
commander_name: str,
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
"""
|
||||
sim_insights.py
|
||||
===============
|
||||
Translates learned simulation weights (learned_weights.json) into concise
|
||||
natural-language insight strings that can be injected into Ollama prompts.
|
||||
|
||||
This bridges the gap between the Monte Carlo engine's outcome-weighted
|
||||
reinforcement learning and the LLM-driven deck builder: keywords that
|
||||
statistically correlate with wins are surfaced to Ollama so it can
|
||||
prioritize those card traits when suggesting and trimming cards.
|
||||
|
||||
Usage::
|
||||
|
||||
from commander_ai_lab.deck_builder.api.sim_insights import get_sim_insights
|
||||
|
||||
context = get_sim_insights() # returns "" if no meaningful learning yet
|
||||
if context:
|
||||
prompt += f"\n\n{context}"
|
||||
|
||||
The function is intentionally safe to call when learned_weights.json does
|
||||
not yet exist (load_weights() falls back to AI_DEFAULT_WEIGHTS silently),
|
||||
and returns an empty string when no weight has drifted far enough from the
|
||||
default to be worth mentioning — so existing prompt behaviour is unchanged
|
||||
until there is real simulation data.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from commander_ai_lab.sim.rules import AI_DEFAULT_WEIGHTS, load_weights
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Maps weight keys → human-readable MTG concepts shown to the LLM.
|
||||
# Only keys the LLM can meaningfully act on are included.
|
||||
_KEY_TO_CONCEPT: dict[str, str] = {
|
||||
"kw_flying": "flying (evasion)",
|
||||
"kw_haste": "haste",
|
||||
"kw_deathtouch": "deathtouch",
|
||||
"kw_indestructible": "indestructible",
|
||||
"kw_doubleStrike": "double strike",
|
||||
"kw_hexproof": "hexproof",
|
||||
"kw_trample": "trample",
|
||||
"kw_lifelink": "lifelink",
|
||||
"kw_menace": "menace",
|
||||
"kw_ward": "ward",
|
||||
"kw_annihilator": "annihilator",
|
||||
"kw_cascade": "cascade",
|
||||
"spell_ramp": "mana ramp spells",
|
||||
"spell_destroy": "targeted removal",
|
||||
"spell_draw": "card draw",
|
||||
"spell_counter": "counterspells and board control",
|
||||
"trig_etb": "enters-the-battlefield triggers",
|
||||
"trig_dies": "death triggers",
|
||||
"trig_attack": "attack triggers",
|
||||
"card_ptBonus": "high power/toughness creatures",
|
||||
}
|
||||
|
||||
|
||||
def get_sim_insights(
|
||||
top_n: int = 6,
|
||||
threshold_delta: float = 0.05,
|
||||
weights_path: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Return a concise insight paragraph for injecting into LLM prompts.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
top_n:
|
||||
Maximum number of concepts to mention (avoids bloating the prompt).
|
||||
threshold_delta:
|
||||
Minimum absolute drift from the default weight before a concept
|
||||
is considered meaningful enough to report. 0.05 means a weight
|
||||
must have moved by at least 5% of its default before it matters.
|
||||
weights_path:
|
||||
Optional explicit path to learned_weights.json. If None, the
|
||||
default location (``src/.../sim/learned_weights.json``) is used.
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
A multi-line insight string ready to append to an LLM prompt, or
|
||||
an empty string if no meaningful learning is available yet.
|
||||
"""
|
||||
try:
|
||||
learned = load_weights(weights_path)
|
||||
except Exception as exc:
|
||||
logger.warning("sim_insights: could not load weights: %s", exc)
|
||||
return ""
|
||||
|
||||
deltas: dict[str, float] = {}
|
||||
for key, concept in _KEY_TO_CONCEPT.items():
|
||||
default_val = AI_DEFAULT_WEIGHTS.get(key, 0.0)
|
||||
learned_val = learned.get(key, default_val)
|
||||
delta = learned_val - default_val
|
||||
if abs(delta) >= threshold_delta:
|
||||
deltas[concept] = delta
|
||||
|
||||
if not deltas:
|
||||
# No meaningful drift yet — don't inject noise into prompts
|
||||
return ""
|
||||
|
||||
# Sort by magnitude, take top_n
|
||||
sorted_items = sorted(deltas.items(), key=lambda x: -abs(x[1]))[:top_n]
|
||||
boosted = [c for c, d in sorted_items if d > 0]
|
||||
penalized = [c for c, d in sorted_items if d < 0]
|
||||
|
||||
lines = [
|
||||
"[Simulation Insights] Data from thousands of Commander games in this lab suggests:",
|
||||
]
|
||||
if boosted:
|
||||
lines.append(f" - PRIORITIZE cards with: {', '.join(boosted)}")
|
||||
if penalized:
|
||||
lines.append(f" - DEPRIORITIZE cards with: {', '.join(penalized)}")
|
||||
lines.append(
|
||||
"Use this to guide your selections where multiple options are otherwise comparable."
|
||||
)
|
||||
|
||||
result = "\n".join(lines)
|
||||
logger.debug("sim_insights: injecting %d concept(s) into prompt", len(sorted_items))
|
||||
return result
|
||||
@@ -13,6 +13,7 @@ from typing import Dict, List, Optional
|
||||
|
||||
from ..api import edhrec, scryfall
|
||||
from ..api import ollama_client as ollama
|
||||
from ..api.sim_insights import get_sim_insights
|
||||
from ..core.collection_filter import filter_names_by_collection
|
||||
from ..core.models import BuildRequest, BuildResult, CardEntry, CommanderDeck, DeckRatios
|
||||
from ..core.rules_engine import check_ban_list, filter_by_color_identity, validate_deck
|
||||
@@ -37,7 +38,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
warnings: List[str] = []
|
||||
sources: List[str] = []
|
||||
|
||||
# ── Step 1: Resolve commander ────────────────────────────────
|
||||
# ── Step 1: Resolve commander ────────────────────────────────────────────
|
||||
logger.info(f"Step 1: Resolving commander '{request.commander_name}'")
|
||||
commander = scryfall.search_commander(request.commander_name)
|
||||
if not commander:
|
||||
@@ -50,7 +51,17 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
logger.info(f"Commander: {commander.name}, CI: {ci_list}")
|
||||
sources.append("scryfall")
|
||||
|
||||
# ── Step 2: Fetch EDHrec recommendations ─────────────────────
|
||||
# Load simulation insights once; injected into Ollama prompts in Steps 4 & 6.
|
||||
# Returns "" when learned_weights.json doesn't exist or has no meaningful drift
|
||||
# yet — so behaviour is identical to before until real sim data is available.
|
||||
sim_context = get_sim_insights()
|
||||
if sim_context:
|
||||
logger.info("Sim insights available — injecting into Ollama prompts")
|
||||
sources.append("sim_weights")
|
||||
else:
|
||||
logger.debug("No sim insights available (run update_weights.py after overnight sims)")
|
||||
|
||||
# ── Step 2: Fetch EDHrec recommendations ─────────────────────────────
|
||||
logger.info(f"Step 2: Fetching EDHrec data for '{commander.name}'")
|
||||
edhrec_cards = edhrec.get_recommended_cards(commander.name)
|
||||
if edhrec_cards:
|
||||
@@ -59,7 +70,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
else:
|
||||
warnings.append("EDHrec returned no data — relying on Scryfall + Ollama")
|
||||
|
||||
# ── Step 3: Fetch Scryfall candidates by category ────────────
|
||||
# ── Step 3: Fetch Scryfall candidates by category ────────────────────────
|
||||
logger.info("Step 3: Fetching Scryfall candidates by category")
|
||||
scryfall_candidates: Dict[str, List[CardEntry]] = {
|
||||
"ramp": scryfall.search_ramp(commander_ci),
|
||||
@@ -70,7 +81,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
for cat, cards in scryfall_candidates.items():
|
||||
logger.info(f" Scryfall {cat}: {len(cards)} candidates")
|
||||
|
||||
# ── Step 4: Ollama suggestions to fill gaps ──────────────────
|
||||
# ── Step 4: Ollama suggestions to fill gaps ────────────────────────────
|
||||
logger.info("Step 4: Ollama suggesting additional cards")
|
||||
|
||||
# Collect all names we already have
|
||||
@@ -79,7 +90,9 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
all_names.extend(c.name for c in cards)
|
||||
all_names = list(set(all_names))
|
||||
|
||||
# Ask Ollama to suggest cards for categories that need more
|
||||
# Ask Ollama to suggest cards for categories that need more.
|
||||
# sim_context is forwarded so the LLM factors in what traits have
|
||||
# been statistically successful in our own game simulations.
|
||||
categories_to_fill = ["synergy", "protection", "wincon"]
|
||||
ollama_suggestions: Dict[str, List[str]] = {}
|
||||
|
||||
@@ -91,6 +104,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
count=15,
|
||||
exclude=all_names,
|
||||
strategy_notes=request.strategy_notes,
|
||||
sim_context=sim_context,
|
||||
)
|
||||
ollama_suggestions[category] = suggested
|
||||
all_names.extend(suggested)
|
||||
@@ -98,7 +112,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
|
||||
sources.append("ollama")
|
||||
|
||||
# ── Step 5: Filter (color identity, collection, ban list) ────
|
||||
# ── Step 5: Filter (color identity, collection, ban list) ────────────────
|
||||
logger.info("Step 5: Filtering candidates")
|
||||
|
||||
# Build a unified name pool per category
|
||||
@@ -140,7 +154,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
n for n in cards_by_category[cat] if n not in BANNED_CARDS
|
||||
]
|
||||
|
||||
# ── Step 6: Ollama enforce ratios ────────────────────────────
|
||||
# ── Step 6: Ollama enforce ratios ────────────────────────────────────
|
||||
logger.info("Step 6: Ollama enforcing deck ratios")
|
||||
target_ratios = {
|
||||
"lands": request.ratios.lands,
|
||||
@@ -153,10 +167,13 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
"uncategorized": request.ratios.uncategorized,
|
||||
}
|
||||
|
||||
# sim_context is forwarded so Ollama preferentially retains cards
|
||||
# matching high-value sim traits when trimming over-filled categories.
|
||||
adjusted = ollama.enforce_deck_ratios(
|
||||
cards_by_category=cards_by_category,
|
||||
target_ratios=target_ratios,
|
||||
commander_name=commander.name,
|
||||
sim_context=sim_context,
|
||||
)
|
||||
|
||||
# Verify total is 99
|
||||
@@ -165,7 +182,7 @@ def build_deck(request: BuildRequest) -> BuildResult:
|
||||
warnings.append(f"Ollama returned {total} cards instead of 99 — adjusting")
|
||||
logger.warning(f"Ratio enforcement returned {total} cards")
|
||||
|
||||
# ── Step 7: Assemble final deck ──────────────────────────────
|
||||
# ── Step 7: Assemble final deck ────────────────────────────────────────
|
||||
logger.info("Step 7: Assembling final deck")
|
||||
|
||||
# Build a globally-deduplicated name list. enforce_deck_ratios can
|
||||
|
||||
+23
-8
@@ -58,7 +58,10 @@
|
||||
|
||||
<!-- Power Level Slider -->
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="coach-power-level">Target Power Level</label>
|
||||
<label class="form-label" for="coach-power-level">
|
||||
Target Power Level
|
||||
<span class="tip-icon tip-below" data-tip="Guides the LLM on how competitive suggestions should be. 1-3 = casual jank; 4-6 = upgraded precon; 7-8 = high power; 9-10 = cEDH with fast mana and infinite combos.">?</span>
|
||||
</label>
|
||||
<div class="coach-slider-row">
|
||||
<input type="range" id="coach-power-level" min="1" max="10" value="7" />
|
||||
<span class="coach-slider-value" id="coach-power-value">7</span>
|
||||
@@ -68,7 +71,10 @@
|
||||
|
||||
<!-- Meta Focus -->
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="coach-meta-focus">Meta Focus</label>
|
||||
<label class="form-label" for="coach-meta-focus">
|
||||
Meta Focus
|
||||
<span class="tip-icon tip-below" data-tip="The game plan the coach optimizes for. Aggro = fast damage; Control = answers and card advantage; Combo = tutors and win-con protection; Stax = slow everyone down.">?</span>
|
||||
</label>
|
||||
<select id="coach-meta-focus" class="form-select">
|
||||
<option value="">Any / Balanced</option>
|
||||
<option value="aggro">Aggro</option>
|
||||
@@ -81,7 +87,10 @@
|
||||
|
||||
<!-- Budget -->
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="coach-budget">Budget</label>
|
||||
<label class="form-label" for="coach-budget">
|
||||
Budget
|
||||
<span class="tip-icon tip-below" data-tip="Restricts suggestions to cards within a price range. Budget caps cards at ~$2 each; Medium at ~$10. No Limit allows any card including Reserved List staples.">?</span>
|
||||
</label>
|
||||
<select id="coach-budget" class="form-select">
|
||||
<option value="">No Limit</option>
|
||||
<option value="budget">Budget (< $2 per card)</option>
|
||||
@@ -92,7 +101,10 @@
|
||||
|
||||
<!-- Focus Areas -->
|
||||
<div class="form-group coach-field-wide">
|
||||
<label class="form-label">Focus Areas (optional)</label>
|
||||
<label class="form-label">
|
||||
Focus Areas (optional)
|
||||
<span class="tip-icon tip-below" data-tip="Narrows the coach to specific deck categories. Leaving all unchecked triggers a full-deck general analysis across all categories.">?</span>
|
||||
</label>
|
||||
<div class="coach-focus-tags">
|
||||
<label class="coach-tag"><input type="checkbox" value="ramp" /> Ramp</label>
|
||||
<label class="coach-tag"><input type="checkbox" value="card draw" /> Card Draw</label>
|
||||
@@ -121,7 +133,7 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ═══ Multi-turn Chat Panel ═══ -->
|
||||
<!-- Multi-turn Chat Panel -->
|
||||
<section class="panel coach-chat-section" id="coach-chat-section">
|
||||
<div class="panel-header coach-chat-header">
|
||||
<h2>Chat with Coach</h2>
|
||||
@@ -143,7 +155,7 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ═══ Cards-Like Search Panel ═══ -->
|
||||
<!-- Cards-Like Search Panel -->
|
||||
<section class="panel coach-cardslike-section" id="coach-cardslike-section">
|
||||
<div class="panel-header coach-panel-collapsible" id="cardslike-header">
|
||||
<h2>Find Similar Cards</h2>
|
||||
@@ -163,7 +175,10 @@
|
||||
<button class="coach-color-pip coach-pip-G" data-color="G" title="Green">G</button>
|
||||
</div>
|
||||
<div class="form-group coach-cardslike-topn">
|
||||
<label class="form-label">Results</label>
|
||||
<label class="form-label">
|
||||
Results
|
||||
<span class="tip-icon tip-below" data-tip="How many similar cards to return. Uses vector embeddings to find cards with the most semantically similar oracle text and mechanics to your query card.">?</span>
|
||||
</label>
|
||||
<input type="number" id="cardslike-topn" class="form-input" value="10" min="1" max="50" />
|
||||
</div>
|
||||
<button id="cardslike-search-btn" class="btn btn-accent btn-sm">Search</button>
|
||||
@@ -173,7 +188,7 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ═══ One-Shot Results Section ═══ -->
|
||||
<!-- One-Shot Results Section -->
|
||||
<section id="coach-results" class="coach-results" style="display:none;">
|
||||
|
||||
<!-- Apply Suggestions Bar -->
|
||||
|
||||
+33
-49
@@ -62,9 +62,13 @@
|
||||
class="dg-input"
|
||||
id="dg-strategy"
|
||||
placeholder="e.g. zombie tokens, voltron, spellslinger..."
|
||||
title="Describe your deck strategy — the AI uses this to pick thematically aligned cards"
|
||||
/>
|
||||
<div class="dg-bracket-row">
|
||||
<label class="dg-label">Target Bracket</label>
|
||||
<label class="dg-label">
|
||||
Target Bracket
|
||||
<span class="tip-icon" data-tip="B1 = precon-level casual. B2 = upgraded with synergies. B3 = high power with strong combos. B4 = cEDH with maximum optimization, fast mana, and instant-speed interaction.">?</span>
|
||||
</label>
|
||||
<div class="dg-bracket-btns" id="dg-bracket-btns">
|
||||
<button class="dg-bracket-btn" data-bracket="1" title="Casual — precon level">B1</button>
|
||||
<button class="dg-bracket-btn" data-bracket="2" title="Upgraded Casual">B2</button>
|
||||
@@ -105,7 +109,7 @@
|
||||
min="0"
|
||||
step="25"
|
||||
/>
|
||||
<select class="dg-select" id="dg-budget-mode">
|
||||
<select class="dg-select" id="dg-budget-mode" title="Total Deck: caps the combined price of all 100 cards. Per Card Max: caps each individual card price.">
|
||||
<option value="total">Total Deck $</option>
|
||||
<option value="per_card">Per Card Max $</option>
|
||||
</select>
|
||||
@@ -120,25 +124,34 @@
|
||||
</div>
|
||||
<div class="dg-panel-body">
|
||||
<label class="dg-toggle-row">
|
||||
<span class="dg-toggle-label">Use my collection</span>
|
||||
<span class="dg-toggle-label">
|
||||
Use my collection
|
||||
<span class="tip-icon" data-tip="Prioritizes cards you already own before pulling from the full card pool. Saves money and ensures the deck is ready to play. Requires an imported collection.">?</span>
|
||||
</span>
|
||||
<input type="checkbox" class="dg-toggle-check" id="dg-use-collection" checked />
|
||||
<span class="dg-toggle-track"><span class="dg-toggle-thumb"></span></span>
|
||||
</label>
|
||||
<label class="dg-toggle-row">
|
||||
<span class="dg-toggle-label">Smart Substitution</span>
|
||||
<span class="dg-toggle-label">
|
||||
Smart Substitution
|
||||
<span class="tip-icon" data-tip="When a suggested card isn't in your collection, automatically finds the closest functional replacement you do own. Keeps the deck playable without buying extra cards.">?</span>
|
||||
</span>
|
||||
<input type="checkbox" class="dg-toggle-check" id="dg-substitution" checked />
|
||||
<span class="dg-toggle-track"><span class="dg-toggle-thumb"></span></span>
|
||||
</label>
|
||||
<div class="dg-model-row">
|
||||
<label class="dg-label">AI Model</label>
|
||||
<label class="dg-label">
|
||||
AI Model
|
||||
<span class="tip-icon" data-tip="The local LLM used to generate the deck list. Larger models produce better card selections but are slower. Requires Ollama running locally with the selected model pulled.">?</span>
|
||||
</label>
|
||||
<select class="dg-select" id="dg-model">
|
||||
<option value="gpt-oss:20b" selected>gpt-oss:20b (Local Ollama)</option>
|
||||
<option value="gpt-oss:20b" selected>gpt-oss:20b (Local Ollama)</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Omit Cards (chips input) -->
|
||||
<!-- Omit Cards (chips input) -->
|
||||
<div class="dg-panel">
|
||||
<div class="dg-panel-header">
|
||||
<span class="dg-panel-title">Omit Cards</span>
|
||||
@@ -160,12 +173,12 @@
|
||||
<!-- Action Buttons -->
|
||||
<div class="dg-actions">
|
||||
<button class="dg-btn dg-btn-preview" id="dg-preview-btn" disabled>
|
||||
<span class="dg-btn-icon">⚡</span> Generate Deck
|
||||
<span class="dg-btn-icon">⚡</span> Generate Deck
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<!-- Conversation Log -->
|
||||
<div id="dg-conversation-log" class="dg-conversation-log">
|
||||
<!-- Conversation Log -->
|
||||
<div id="dg-conversation-log" class="dg-conversation-log">
|
||||
<div class="dg-log-entry dg-log-info"><span class="dg-log-ts">[Ready]</span> Waiting for deck generation...</div>
|
||||
</div>
|
||||
|
||||
@@ -175,7 +188,7 @@
|
||||
<section class="dg-col-results">
|
||||
<!-- Empty State -->
|
||||
<div class="dg-empty-state" id="dg-empty-state">
|
||||
<div class="dg-empty-icon">⚡</div>
|
||||
<div class="dg-empty-icon">⚡</div>
|
||||
<div class="dg-empty-title">V3 AI Deck Generator</div>
|
||||
<div class="dg-empty-text">Select a commander, set your bracket and strategy, then click "Generate Deck" to build a complete 100-card deck using Claude Opus with structured output.</div>
|
||||
</div>
|
||||
@@ -185,47 +198,18 @@
|
||||
<div class="dg-spinner"></div>
|
||||
<div class="dg-loading-text" id="dg-loading-text">Generating deck via Local AI...</div>
|
||||
</div>
|
||||
<div class="dg-progress-wrap" id="dg-progress-wrap" style="display:none">
|
||||
<div class="dg-progress-bar">
|
||||
<div class="dg-progress-fill" id="dg-progress-fill"></div>
|
||||
</div>
|
||||
<div class="dg-progress-meta">
|
||||
<span class="dg-progress-phase" id="dg-progress-phase"></span>
|
||||
<span class="dg-progress-elapsed" id="dg-progress-elapsed">0s elapsed</span>
|
||||
</div>
|
||||
<div class="dg-progress-wrap" id="dg-progress-wrap" style="display:none">
|
||||
<div class="dg-progress-bar">
|
||||
<div class="dg-progress-fill" id="dg-progress-fill"></div>
|
||||
</div>
|
||||
|
||||
<!-- Results -->
|
||||
<div class="dg-results" id="dg-results" style="display:none">
|
||||
|
||||
<!-- Strategy & Bracket Summary -->
|
||||
<div class="dg-strategy-bar" id="dg-strategy-bar"></div>
|
||||
|
||||
<!-- Stats Bar -->
|
||||
<div class="dg-stats-bar" id="dg-stats-bar"></div>
|
||||
|
||||
<!-- Substitution Summary -->
|
||||
<div class="dg-sub-summary" id="dg-sub-summary" style="display:none"></div>
|
||||
|
||||
<!-- Card Groups -->
|
||||
<div class="dg-card-groups" id="dg-card-groups"></div>
|
||||
|
||||
<!-- Export & Save Bar -->
|
||||
<div class="dg-save-bar" id="dg-save-bar">
|
||||
<button class="dg-btn dg-btn-save" id="dg-save-btn">
|
||||
<span class="dg-btn-icon">💾</span> Save to Deck Builder
|
||||
</button>
|
||||
<button class="dg-btn dg-btn-ghost" id="dg-regenerate-btn">
|
||||
<span class="dg-btn-icon">🔄</span> Regenerate
|
||||
</button>
|
||||
<div class="dg-export-group">
|
||||
<button class="dg-btn dg-btn-export" id="dg-export-csv">📊 CSV</button>
|
||||
<button class="dg-btn dg-btn-export" id="dg-export-dck">🎮 Forge .dck</button>
|
||||
<button class="dg-btn dg-btn-export" id="dg-export-moxfield">📋 Moxfield</button>
|
||||
<button class="dg-btn dg-btn-export" id="dg-export-shopping">🛒 Shopping List</button>
|
||||
</div>
|
||||
<div class="dg-progress-meta">
|
||||
<span class="dg-progress-phase" id="dg-progress-phase"></span>
|
||||
<span class="dg-progress-elapsed" id="dg-progress-elapsed">0s elapsed</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Results placeholder — rest generated dynamically by deckgenerator.js -->
|
||||
<div class="dg-results" id="dg-results" style="display:none"></div>
|
||||
</section>
|
||||
|
||||
</main>
|
||||
|
||||
+44
-12
@@ -16,7 +16,7 @@
|
||||
<div class="page-header" style="display:flex;align-items:center;justify-content:space-between;">
|
||||
<div>
|
||||
<h1>Closed-Loop Distillation</h1>
|
||||
<p>Iterative supervised → PPO → quality gate → merge pipeline</p>
|
||||
<p>Iterative supervised → PPO → quality gate → merge pipeline</p>
|
||||
</div>
|
||||
<button id="btn-refresh" class="btn btn-secondary" onclick="refreshAll()" title="Refresh status">
|
||||
<span class="btn-icon" id="refresh-icon">↻</span> Refresh
|
||||
@@ -59,43 +59,70 @@
|
||||
<div class="train-controls">
|
||||
<div class="param-grid form-row">
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-iterations">Max Generations</label>
|
||||
<label class="form-label" for="param-iterations">
|
||||
Max Generations
|
||||
<span class="tip-icon" data-tip="How many distillation cycles to run. Each cycle: supervised train → PPO fine-tune → quality gate. Stops early if convergence is detected.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-iterations" class="form-input" value="10" min="1" max="100" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-conv-window">Convergence Window</label>
|
||||
<label class="form-label" for="param-conv-window">
|
||||
Convergence Window
|
||||
<span class="tip-icon" data-tip="Number of recent generations to check for plateau. If ELO gain stays below threshold across this many gens, training stops early to save time.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-conv-window" class="form-input" value="3" min="2" max="10" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-sup-epochs">Supervised Epochs</label>
|
||||
<label class="form-label" for="param-sup-epochs">
|
||||
Supervised Epochs
|
||||
<span class="tip-icon" data-tip="Training epochs for the supervised step each generation. Fewer is fine since each cycle trains on incrementally better data. 30 is a solid starting point.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-sup-epochs" class="form-input" value="30" min="5" max="200" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-sup-lr">Supervised LR</label>
|
||||
<label class="form-label" for="param-sup-lr">
|
||||
Supervised LR
|
||||
<span class="tip-icon" data-tip="Learning rate for the supervised phase. 0.001 is a solid default. Lower it in later generations if the model is already strong.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-sup-lr" class="form-input" value="0.001" step="0.0001" min="0.00001" max="0.1" />
|
||||
</div>
|
||||
</div>
|
||||
<div class="param-grid form-row">
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-ppo-iters">PPO Iterations</label>
|
||||
<label class="form-label" for="param-ppo-iters">
|
||||
PPO Iterations
|
||||
<span class="tip-icon" data-tip="Number of PPO update steps per generation. Each step samples game episodes, computes advantages, and updates policy weights.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-ppo-iters" class="form-input" value="50" min="10" max="500" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-ppo-episodes">PPO Episodes/Iter</label>
|
||||
<label class="form-label" for="param-ppo-episodes">
|
||||
PPO Episodes/Iter
|
||||
<span class="tip-icon" data-tip="Game simulations per PPO update step. More episodes = more stable gradients but slower iterations. 64 balances speed and stability well.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-ppo-episodes" class="form-input" value="64" min="8" max="512" step="8" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-ppo-lr">PPO Learning Rate</label>
|
||||
<label class="form-label" for="param-ppo-lr">
|
||||
PPO Learning Rate
|
||||
<span class="tip-icon" data-tip="PPO step size. Keep this lower than the supervised LR. 0.0003 is the standard RL default and works well here.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-ppo-lr" class="form-input" value="0.0003" step="0.0001" min="0.00001" max="0.01" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-min-wr">Min Win Rate Gate</label>
|
||||
<label class="form-label" for="param-min-wr">
|
||||
Min Win Rate Gate
|
||||
<span class="tip-icon" data-tip="Quality gate threshold. A generation must beat the heuristic opponent at this win rate to be accepted and merged into the next cycle. 0.30 is a lenient starting gate.">?</span>
|
||||
</label>
|
||||
<input type="number" id="param-min-wr" class="form-input" value="0.30" step="0.05" min="0.1" max="0.9" />
|
||||
</div>
|
||||
</div>
|
||||
<div class="param-grid form-row distill-bottom-params">
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-opponent">Opponent</label>
|
||||
<label class="form-label" for="param-opponent">
|
||||
Opponent
|
||||
<span class="tip-icon" data-tip="Who the agent trains against during PPO. Heuristic is recommended for a consistent challenge. Self-play can become unstable without a strong supervised baseline first.">?</span>
|
||||
</label>
|
||||
<select id="param-opponent" class="form-select">
|
||||
<option value="heuristic">Heuristic</option>
|
||||
<option value="random">Random</option>
|
||||
@@ -103,7 +130,10 @@
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label" for="param-playstyle">Playstyle</label>
|
||||
<label class="form-label" for="param-playstyle">
|
||||
Playstyle
|
||||
<span class="tip-icon" data-tip="Deck archetype used for simulations in this distillation run. Midrange is the most balanced test bed. Match this to the archetype you want the model to master.">?</span>
|
||||
</label>
|
||||
<select id="param-playstyle" class="form-select">
|
||||
<option value="midrange">Midrange</option>
|
||||
<option value="aggro">Aggro</option>
|
||||
@@ -164,7 +194,9 @@
|
||||
<!-- ELO Rating Chart -->
|
||||
<section class="panel" id="elo-panel">
|
||||
<div class="panel-header elo-panel-header">
|
||||
<h2>ELO Ratings</h2>
|
||||
<h2>ELO Ratings
|
||||
<span class="tip-icon tip-below" data-tip="ELO scores measure relative strength. Baseline is 1200. Scores above that mean the model wins more than it loses. Tournament runs the top 10 most recent checkpoints head-to-head.">?</span>
|
||||
</h2>
|
||||
<button id="btn-run-elo" class="btn btn-sm btn-secondary" onclick="runEloTournament()">
|
||||
<span class="btn-icon">🏆</span> Run Tournament
|
||||
</button>
|
||||
|
||||
@@ -864,3 +864,92 @@ body {
|
||||
.form-input:focus, .form-select:focus, .form-textarea:focus {
|
||||
box-shadow: 0 0 0 3px var(--accent-dim), inset 0 1px 3px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
|
||||
/* ══════════════════════════════════════════════════════════════
|
||||
Tooltip System (.tip-icon + data-tip)
|
||||
Shared across all pages — do not duplicate in page-specific CSS
|
||||
══════════════════════════════════════════════════════════════ */
|
||||
|
||||
.tip-icon {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 15px;
|
||||
height: 15px;
|
||||
border-radius: 50%;
|
||||
background: var(--surface-3);
|
||||
color: var(--text-secondary);
|
||||
font-size: 10px;
|
||||
font-weight: 700;
|
||||
font-style: normal;
|
||||
cursor: default;
|
||||
position: relative;
|
||||
vertical-align: middle;
|
||||
margin-left: 4px;
|
||||
flex-shrink: 0;
|
||||
transition: background 0.15s, color 0.15s;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.tip-icon:hover {
|
||||
background: var(--accent-dim);
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
.tip-icon::after {
|
||||
content: attr(data-tip);
|
||||
position: absolute;
|
||||
bottom: calc(100% + 8px);
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
background: var(--surface-0, #1a1a2e);
|
||||
color: var(--text, #e0e0e0);
|
||||
border: 1px solid var(--border, #333);
|
||||
border-radius: var(--radius-md, 6px);
|
||||
padding: 8px 12px;
|
||||
font-size: 12px;
|
||||
font-weight: 400;
|
||||
line-height: 1.5;
|
||||
white-space: normal;
|
||||
width: 240px;
|
||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.4);
|
||||
pointer-events: none;
|
||||
opacity: 0;
|
||||
transition: opacity 0.15s ease;
|
||||
z-index: 1000;
|
||||
text-align: left;
|
||||
text-transform: none;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
.tip-icon::before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
bottom: calc(100% + 3px);
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
border: 5px solid transparent;
|
||||
border-top-color: var(--border, #333);
|
||||
pointer-events: none;
|
||||
opacity: 0;
|
||||
transition: opacity 0.15s ease;
|
||||
z-index: 1001;
|
||||
}
|
||||
|
||||
.tip-icon:hover::after,
|
||||
.tip-icon:hover::before {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.tip-icon.tip-below::after {
|
||||
bottom: auto;
|
||||
top: calc(100% + 8px);
|
||||
}
|
||||
|
||||
.tip-icon.tip-below::before {
|
||||
bottom: auto;
|
||||
top: calc(100% + 3px);
|
||||
border-top-color: transparent;
|
||||
border-bottom-color: var(--border, #333);
|
||||
}
|
||||
+25
-6
@@ -28,20 +28,27 @@
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group sim-field-auto">
|
||||
<label class="form-label" for="sim-opponent-select">Opponent</label>
|
||||
<label class="form-label" for="sim-opponent-select">
|
||||
Opponent
|
||||
<span class="tip-icon tip-below" data-tip="Heuristic AI uses rule-based logic — fast and deterministic. DeepSeek AI routes decisions through your local Ollama LLM for smarter but slower play.">?</span>
|
||||
</label>
|
||||
<select id="sim-opponent-select" class="form-select">
|
||||
<option value="heuristic">Heuristic AI</option>
|
||||
<option value="deepseek">DeepSeek AI</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group sim-field-auto">
|
||||
<label class="form-label" for="sim-num-games">Games</label>
|
||||
<label class="form-label" for="sim-num-games">
|
||||
Games
|
||||
<span class="tip-icon tip-below" data-tip="Number of games to simulate. 10 is quick for a gut-check; 100+ gives statistically meaningful win rates. Each game is fully simulated end-to-end.">?</span>
|
||||
</label>
|
||||
<input type="number" id="sim-num-games" class="form-input sim-num-input" value="10" min="1" max="1000" />
|
||||
</div>
|
||||
<div class="form-group sim-field-auto sim-field-end">
|
||||
<label class="form-checkbox">
|
||||
<input type="checkbox" id="sim-record-logs" checked />
|
||||
Record turn logs
|
||||
<span class="tip-icon tip-below" data-tip="Saves per-turn decision snapshots to .jsonl files. Required for ML training data collection. Slightly slower — disable if you only need win rate stats.">?</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="form-group sim-field-auto sim-field-end">
|
||||
@@ -61,19 +68,31 @@
|
||||
</div>
|
||||
<div class="deepseek-config-row">
|
||||
<div class="form-group sim-field-grow">
|
||||
<label class="form-label" for="ds-api-base">API Endpoint</label>
|
||||
<label class="form-label" for="ds-api-base">
|
||||
API Endpoint
|
||||
<span class="tip-icon tip-below" data-tip="Base URL of your Ollama server. Default is localhost:11434. Change this if Ollama is running on a different machine or WSL2 network address.">?</span>
|
||||
</label>
|
||||
<input type="text" id="ds-api-base" class="form-input" value="http://localhost:11434" placeholder="http://localhost:11434" />
|
||||
</div>
|
||||
<div class="form-group sim-field-auto">
|
||||
<label class="form-label" for="ds-model">Model</label>
|
||||
<label class="form-label" for="ds-model">
|
||||
Model
|
||||
<span class="tip-icon tip-below" data-tip="Ollama model tag to use for AI decisions. Leave blank to auto-detect the best available model. Examples: deepseek-r1:7b, llama3:8b, mistral.">?</span>
|
||||
</label>
|
||||
<input type="text" id="ds-model" class="form-input" value="" placeholder="Auto-detect" style="width:200px" />
|
||||
</div>
|
||||
<div class="form-group sim-field-auto">
|
||||
<label class="form-label" for="ds-temp">Temp</label>
|
||||
<label class="form-label" for="ds-temp">
|
||||
Temp
|
||||
<span class="tip-icon tip-below" data-tip="LLM sampling temperature. Lower = more deterministic decisions. 0.3 is a good balance — fully greedy (0.0) can get stuck in repetitive play patterns.">?</span>
|
||||
</label>
|
||||
<input type="number" id="ds-temp" class="form-input sim-num-input" value="0.3" min="0" max="2" step="0.1" />
|
||||
</div>
|
||||
<div class="form-group sim-field-auto">
|
||||
<label class="form-label" for="ds-timeout">Timeout (s)</label>
|
||||
<label class="form-label" for="ds-timeout">
|
||||
Timeout (s)
|
||||
<span class="tip-icon tip-below" data-tip="Max seconds to wait for an LLM response per decision. If exceeded, falls back to heuristic AI automatically. Increase if your GPU is slow on cold inference.">?</span>
|
||||
</label>
|
||||
<input type="number" id="ds-timeout" class="form-input sim-num-input" value="10" min="1" max="60" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Reference in new issue
Block a user