# Commander-AI-Lab GUI + LLM Opponents Roadmap

## Project Summary

Build a playable Magic: The Gathering Commander interface where human players face AI opponents powered by a local LLM. The existing Python rules engine handles game legality and state management, while the LLM acts as the decision-making and narration brain for each AI opponent. A Pygame (or web-based) GUI renders the battlefield, zones, and card interactions. Each AI opponent gets a unique personality prompt to simulate different playstyles (aggro, control, combo, politics).

**Local LLM:** GPT-OSS 20B (Q4_K_M) via Ollama or LM Studio

---

## GUI Strategy: Two-Phase Frontend Plan

### Phase A (Prototype) — Pygame
Use **Pygame** for Phases 1–3 of development. Pure Python means the rules engine and AI opponent code plug in directly with zero bridging. Fast to iterate — a working battlefield renderer can be built in a day. Handles game loop, input events, and card rendering naturally. Ideal for testing AI decisions visually before investing in a polished UI.

### Phase B (Production) — Flask/FastAPI + Web Frontend
Once the core AI gameplay is working, migrate the UI to a **Flask or FastAPI backend + HTML/JS frontend** (vanilla JS or React). Benefits:
- Far more polished look for a Commander game
- Card art from Scryfall API renders natively in browser
- Easier to add multiplayer later via WebSockets
- Python backend stays pure — frontend calls it via REST API
- No Python/JS bridge complexity; clean separation of concerns

### GUI Framework Comparison

| Option | Language | Curve | Best For | Weakness |
|---|---|---|---|---|
| **Pygame** ✅ Phase A | Python | Low | Quick prototype, game loops | Dated UI, manual widgets |
| **Flask + HTML/JS** ✅ Phase B | Python + JS | Medium | Polished UI, browser-based | JS context switch |
| PyQt6 / PySide6 | Python | Medium | Desktop app widgets | Verbose, overkill for a game |
| Godot | GDScript | Medium | Full game engine, animations | Separate codebase from engine |
| Electron + Python | JS + Python | High | Desktop with web UI | Heavy, complex bridge |
| Tkinter | Python | Low | Simple UI | Very dated look |

---

## Architecture Overview

```
Game State (Python dict/dataclass)
        ↓
Legal Moves Generator (rules engine)
        ↓
LLM Prompt: "Given this state and these legal moves, what do you do?"
        ↓ (GPT-OSS 20B Q4_K_M)
LLM Response → parsed action
        ↓
Rules Engine executes + validates
        ↓
GUI updates board state (Pygame → Web)
```

---

## Task List

### Phase 1 — Game State Serialization
- [ ] Define a compressed game state schema (Python dataclass or dict) covering: hand, battlefield, graveyard, exile, stack, life totals, commander zone, mana pool
- [ ] Write a `state_to_prompt()` function that converts game state to a compact LLM-readable string
- [ ] Write a `legal_moves_to_prompt()` function that lists valid actions in numbered format for LLM selection
- [ ] Add a state snapshot logger for debugging AI decisions

### Phase 2 — LLM AI Opponent Integration
- [ ] Load GPT-OSS 20B (Q4_K_M) in Ollama or LM Studio; verify inference speed on RTX 5070 Ti
- [ ] Benchmark VRAM usage for GPT-OSS 20B Q4_K_M (expected ~12–14 GB VRAM at Q4_K_M quantization)
- [ ] Build `AIOpponent` class with fields: name, personality_prompt, model_endpoint, deck
- [ ] Implement `decide_action(game_state, legal_moves)` method — sends prompt to GPT-OSS 20B, parses LLM response, returns validated action index
- [ ] Add fallback logic: if LLM returns invalid/unparseable action, default to highest-priority heuristic move
- [ ] Write personality prompt templates (Aggro Timmy, Control Spike, Combo Johnny, Political Negotiator)
- [ ] Add narration method: `narrate_play(action)` — GPT-OSS 20B generates in-character flavor text for each play
- [ ] Tune system prompt length to stay within GPT-OSS 20B context window limits

### Phase 3 — Multi-Player Turn Management
- [ ] Implement turn queue: Player → AI1 → AI2 → AI3 → repeat
- [ ] Run AI decisions asynchronously (threading or asyncio) so GUI stays responsive during LLM inference
- [ ] Add priority-passing logic for instants/triggers during other players' turns
- [ ] Implement basic AI threat assessment: each AI scores opponents by board state danger level

### Phase 4 — Pygame Prototype GUI
- [ ] Design zone layout: hand, battlefield (creature/non-creature split), graveyard, exile, command zone, stack
- [ ] Build card rendering: card art placeholder, name, P/T or loyalty, tapped/untapped state
- [ ] Add click-to-play card interaction for human player
- [ ] Display AI narration/flavor text in a side panel
- [ ] Add life total trackers and commander damage matrix (4-player)
- [ ] Build phase/step indicator (Main 1, Combat, Main 2, etc.)
- [ ] Add end-of-game screen with win condition display

### Phase 5 — Web Frontend Migration *(Flask/FastAPI + HTML/JS)*
- [ ] Set up Flask or FastAPI backend exposing game state as REST API
- [ ] Build HTML/JS board layout mirroring Pygame zone design
- [ ] Integrate Scryfall API for card art rendering
- [ ] Add WebSocket support for real-time game state updates
- [ ] Add drag-and-drop card interaction
- [ ] Port AI narration panel to web chat-style UI
- [ ] Test full game loop end-to-end on web frontend

### Phase 6 — Politics & Table Talk *(Stretch Goal)*
- [ ] Implement inter-AI negotiation: before attacking, AI queries GPT-OSS 20B "should I propose a deal?"
- [ ] Build deal proposal UI: AI proposes, human can accept/decline, other AIs respond
- [ ] Add threat-based targeting memory: AIs remember who attacked them last turn

### Phase 7 — Testing & Tuning
- [ ] Run 10 full simulated games (4 AI vs AI) and log decision quality
- [ ] Benchmark GPT-OSS 20B Q4_K_M inference time per decision on RTX 5070 Ti
- [ ] Tune context window compression to stay within model token limits
- [ ] Profile Pygame and web render loops for stability during AI thinking phases
- [ ] Add unit tests for `state_to_prompt()` and `decide_action()` parsing

### Phase 8 — Multiplayer (Human vs Human + AI)
- [ ] **Lobby system** — create/join game rooms with configurable seat count (2–4 players); assign each seat as Human or AI
- [ ] **WebSocket game server** — upgrade Flask/FastAPI backend to handle persistent connections for real-time state sync across multiple browser clients
- [ ] **Player session management** — unique session tokens per human player; reconnect support if connection drops mid-game
- [ ] **Private hand views** — each human player sees only their own hand; other hands shown as face-down card backs
- [ ] **Turn notification system** — alert active player when it's their turn (browser notification or in-page indicator)
- [ ] **Action broadcast** — all players see each action played in real-time (card played, attack declared, ability activated) via WebSocket push
- [ ] **Mixed lobby support** — allow any combination of human and AI seats (e.g., 2 humans + 2 AIs, or 1 human + 3 AIs)
- [ ] **Spectator mode** — allow observers to watch a game without participating; full board visible, no hand info
- [ ] **Local LAN play** — host game server on local network so friends can connect via browser on the same Wi-Fi (no internet required)
- [ ] **Optional: Remote play** — expose server via ngrok or Tailscale for play over the internet without full deployment
- [ ] **Multiplayer AI pacing** — add configurable AI decision delay (e.g., 1–3 sec) so human players can follow AI turns without them resolving instantly
- [ ] **Chat panel** — in-game text chat between human players; AI opponents can optionally post narration lines into the same channel
- [ ] **End-of-game summary** — show full game recap (turns played, damage dealt, commanders cast count) visible to all players after win condition met

---

## Tech Stack

| Component | Choice | Notes |
|---|---|---|
| GUI (Prototype) | **Pygame** | Pure Python, fast iteration |
| GUI (Production) | **Flask/FastAPI + HTML/JS** | Polished, browser-based |
| Multiplayer | **WebSockets (Flask-SocketIO)** | Real-time state sync across clients |
| LLM Backend | Ollama or LM Studio | Serves GPT-OSS 20B locally |
| Model | **GPT-OSS 20B (Q4_K_M)** | ~12–14 GB VRAM, strong reasoning |
| Game State | Python dataclass | Structured, easy to serialize |
| Async | asyncio / threading | Keep GUI responsive during inference |
| Card Data | Scryfall API | Free MTG card data + art |
| LAN Hosting | Local network / ngrok | No deployment needed for local play |

### GPT-OSS 20B Q4_K_M Notes
- Q4_K_M quantization balances quality and speed well for a 20B model
- Expected VRAM: ~12–14 GB on RTX 5070 Ti (16 GB VRAM) — fits comfortably
- Stronger reasoning than 7B models — better for multi-step MTG decision making
- Slower inference than smaller models — async handling is essential so GUI doesn't block
- System prompt + game state should be kept under ~3000 tokens to maintain fast response times