Commander-AI-Lab GUI + LLM Opponents Roadmap

Project Summary

Build a playable Magic: The Gathering Commander interface where human players face AI opponents powered by a local LLM. The existing Python rules engine handles game legality and state management, while the LLM acts as the decision-making and narration brain for each AI opponent. A Pygame (or web-based) GUI renders the battlefield, zones, and card interactions. Each AI opponent gets a unique personality prompt to simulate different playstyles (aggro, control, combo, politics).

Local LLM: GPT-OSS 20B (Q4_K_M) via Ollama or LM Studio

Architecture Overview

Game State (Python dict/dataclass)
        ↓
Legal Moves Generator (rules engine)
        ↓
LLM Prompt: "Given this state and these legal moves, what do you do?"
        ↓ (GPT-OSS 20B Q4_K_M)
LLM Response → parsed action
        ↓
Rules Engine executes + validates
        ↓
GUI updates board state

Task List

Phase 1 — Game State Serialization

Define a compressed game state schema (Python dataclass or dict) covering: hand, battlefield, graveyard, exile, stack, life totals, commander zone, mana pool
Write a state_to_prompt() function that converts game state to a compact LLM-readable string
Write a legal_moves_to_prompt() function that lists valid actions in numbered format for LLM selection
Add a state snapshot logger for debugging AI decisions

Phase 2 — LLM AI Opponent Integration

Load GPT-OSS 20B (Q4_K_M) in Ollama or LM Studio; verify inference speed on RTX 5070 Ti
Benchmark VRAM usage for GPT-OSS 20B Q4_K_M (expected ~12–14 GB VRAM at Q4_K_M quantization)
Build AIOpponent class with fields: name, personality_prompt, model_endpoint, deck
Implement decide_action(game_state, legal_moves) method — sends prompt to GPT-OSS 20B, parses LLM response, returns validated action index
Add fallback logic: if LLM returns invalid/unparseable action, default to highest-priority heuristic move
Write personality prompt templates (Aggro Timmy, Control Spike, Combo Johnny, Political Negotiator)
Add narration method: narrate_play(action) — GPT-OSS 20B generates in-character flavor text for each play
Tune system prompt length to stay within GPT-OSS 20B context window limits

Phase 3 — Multi-Player Turn Management

Implement turn queue: Player → AI1 → AI2 → AI3 → repeat
Run AI decisions asynchronously (threading or asyncio) so GUI stays responsive during LLM inference
Add priority-passing logic for instants/triggers during other players' turns
Implement basic AI threat assessment: each AI scores opponents by board state danger level

Phase 4 — GUI (Pygame or Web Frontend)

Design zone layout: hand, battlefield (creature/non-creature split), graveyard, exile, command zone, stack
Build card rendering: card art placeholder, name, P/T or loyalty, tapped/untapped state
Add drag-and-drop (or click-to-play) card interaction for human player
Display AI narration/flavor text in a chat-style side panel
Add life total trackers and commander damage matrix (4-player)
Build phase/step indicator (Main 1, Combat, Main 2, etc.)
Add end-of-game screen with win condition display

Phase 5 — Politics & Table Talk (Stretch Goal)

Implement inter-AI negotiation: before attacking, AI queries GPT-OSS 20B "should I propose a deal?"
Build deal proposal UI: AI proposes, human can accept/decline, other AIs respond
Add threat-based targeting memory: AIs remember who attacked them last turn

Phase 6 — Testing & Tuning

Run 10 full simulated games (4 AI vs AI) and log decision quality
Benchmark GPT-OSS 20B Q4_K_M inference time per decision on RTX 5070 Ti
Tune context window compression to stay within model token limits
Profile Pygame render loop for frame rate stability during AI thinking phases
Add unit tests for state_to_prompt() and decide_action() parsing

Tech Stack

Component	Choice	Notes
GUI	Pygame	Fast to prototype, stays in Python
LLM Backend	Ollama or LM Studio	Serves GPT-OSS 20B locally
Model	GPT-OSS 20B (Q4_K_M)	~12–14 GB VRAM, strong reasoning
Game State	Python dataclass	Structured, easy to serialize
Async	asyncio / threading	Keep GUI responsive during inference
Card Data	Scryfall API	Free MTG card data + art

GPT-OSS 20B Q4_K_M Notes

Q4_K_M quantization balances quality and speed well for a 20B model
Expected VRAM: ~12–14 GB on RTX 5070 Ti (16 GB VRAM) — fits comfortably
Stronger reasoning than 7B models — better for multi-step MTG decision making
Slower inference than smaller models — async handling is essential so GUI doesn't block
System prompt + game state should be kept under ~3000 tokens to maintain fast response times

4.9 KiB Raw Blame History Unescape Escape