fix: AI land-only bug — enforce 1-land/turn, populate legal moves, tune policy

Addresses the three root causes in commander-ai-lab-land-only-bug-analysis.md
plus several collateral fixes discovered along the way.

P0 — Land-per-turn enforcement (engine.py, deepseek_engine.py, turn_manager.py)
  * Guard _play_land() against double plays by tracking
    _lands_played_this_turn vs _land_plays_this_turn_limit on each player
  * Reset per-turn counters at the start of each player's turn
  * Ramp cards that grant additional plays can bump the limit

P0 — Legal move pipeline (new legal_move_generator.py, game_state.py)
  * New generate_legal_moves() emits the full macro-action set
    (play_land, cast_creature, cast_removal, cast_ramp, cast_draw,
    cast_commander, attack, block, pass_priority) based on phase,
    active player, land-drop state, mana, and commander tax
  * get_legal_moves() delegates to the new generator instead of the
    pass-only stub that blocked the turn_manager/policy-server path
  * apply_move() gains handlers for play_land, cast_spell,
    cast_commander, and declare_attacker — spells resolve to
    battlefield or graveyard, stats update correctly

P1 — Policy exploration (LearnedPolicy.java, scope.py)
  * Default greedy=false so the neural policy samples its
    distribution instead of always taking argmax
  * Raise ppo_entropy_coeff 0.01 → 0.04 to keep the action
    distribution flat during PPO self-play and prevent the
    pass/land collapse

Collateral fixes pulled in during the same session:
  * lan-client/precon-loader.js: await enrichAllZonesWithScryfall()
    so opening-hand cards have typeLine/cmc before the AI's first turn
  * lan-client/ui.js: closeZoneModal auto-shuffles the library
    whenever the zone modal closes
  * routes/game.py: BatchState() was missing total_games/threads,
    crashing POST /api/game/start
  * routes/deckgen.py: dead code after raise left headers undefined,
    breaking the Anthropic API path

All sim/phase/integration tests pass (650+).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
rgriffithandClaude Opus 4.6 committed 2026-04-05 10:34:37 -04:00
1 parent 44123f7622
commit d308e8f413
11 files changed
+494 -55

No files matched your search

+12 -2
View File
@@ -50,8 +50,18 @@ async function startGameWithDecks() {
if (typeof renderPlayerPanels === 'function') renderPlayerPanels();
if (typeof updateActivePlayerHighlight === 'function') updateActivePlayerHighlight();
// Kick off image enrichment for all players in the background (non-blocking)
enrichAllZonesWithScryfall();
// Block on Scryfall enrichment BEFORE the game becomes interactive. This is
// critical: the opening 7 cards are dealt via player.zones.hand.push(lib.shift())
// directly (bypassing drawCard()), so they don't get _enrichCardFromCache'd at
// draw time. Without awaiting here, the AI's first turn fires with empty
// typeLine/cmc/oracleText/isRamp/isRemoval flags and can only play lands.
addLogEntry('\u23F3 Loading card data from Scryfall...');
try {
await enrichAllZonesWithScryfall();
} catch (e) {
console.warn('[precon-loader] Enrichment failed, continuing anyway:', e);
}
addLogEntry('\u2705 Card data loaded. Game ready.');
}
+14
View File
@@ -843,6 +843,20 @@ function openZoneModal(playerId, zone) {
}
function closeZoneModal() {
// Auto-shuffle the library when closing, and ONLY the library
if (zoneModalPlayerId !== null && gameState) {
const player = gameState.players[zoneModalPlayerId];
if (player && player.zones.library.length > 1) {
const lib = player.zones.library;
for (let i = lib.length - 1; i > 0; i--) {
const j = Math.floor(Math.random() * (i + 1));
[lib[i], lib[j]] = [lib[j], lib[i]];
}
addLogEntry('&#x1F500; <strong>' + player.name + "'s</strong> library shuffled.");
updateZoneCountsForPlayer(zoneModalPlayerId);
}
}
const modal = document.getElementById('zone-modal');
if (modal) modal.classList.remove('visible');
zoneModalPlayerId = null;
+9 -1
View File
@@ -292,7 +292,15 @@ class DistillationDefaults:
ppo_episodes_per_iter: int = 64
ppo_lr: float = 3e-4
ppo_clip_epsilon: float = 0.2
ppo_entropy_coeff: float = 0.01
# Increased from 0.01 to 0.04 to prevent premature convergence on the
# "pass"/"land" macro-actions. The policy-collapse bug (see
# commander-ai-lab-land-only-bug-analysis.md) is exacerbated by a low
# entropy bonus: PPO settled on argmax very early in training, so the
# model never learned to cast creatures / removal / draw spells even
# when they were affordable. A higher entropy bonus keeps the action
# distribution flat enough during self-play for those alternatives to
# accumulate reward signal.
ppo_entropy_coeff: float = 0.04
ppo_eval_episodes: int = 100
# Dataset mixing — Forge-only by default (Phase 5 follow-up decision)
+6 -5
View File
@@ -861,12 +861,13 @@ async def _call_anthropic_api(messages: list[dict], max_tokens: int = 4096, temp
else:
if not CFG.anthropic_api_key:
raise HTTPException(400, 'Anthropic Claude API key not configured. Set anthropic_api_key env var or --anthropic-key.')
base_url = 'https://api.anthropic.com'
model = ANTHROPIC_MODEL
headers = {
base_url = 'https://api.anthropic.com'
model = ANTHROPIC_MODEL
headers = {
'Content-Type': 'application/json',
'x-api-key': CFG.anthropic_api_key, 'anthropic-version': '2023-06-01',
}
'x-api-key': CFG.anthropic_api_key,
'anthropic-version': '2023-06-01',
}
# Separate system messages for Anthropic API format
system_msgs = [m['content'] for m in messages if m.get('role') == 'system']
user_msgs = [m for m in messages if m.get('role') != 'system']
+1 -1
View File
@@ -147,7 +147,7 @@ async def start_game(req: StartGameRequest):
# Build a BatchState so the forge_runner watchdog and progress tracking
# work the same way they do for overnight batch runs.
state = BatchState(batch_id=game_id)
state = BatchState(batch_id=game_id, total_games=1, threads=1)
_sessions[game_id] = {
"game_id": game_id,
+19 -1
View File
@@ -148,6 +148,11 @@ class DeepSeekGameEngine:
p.stats.turns_alive += 1
phase_events: list[str] = []
# ── Reset per-turn counters (land drop, etc.) ──
p._lands_played_this_turn = 0
if not hasattr(p, '_land_plays_this_turn_limit'):
p._land_plays_this_turn_limit = 1
# ── Draw ──
if p.library:
drawn = p.library[-1]
@@ -357,8 +362,20 @@ class DeepSeekGameEngine:
self, sim: SimState, pi: int,
preferred_name: str | None, events: list[str] | None
):
"""Play a land from hand. If preferred_name given, try that first."""
"""Play a land from hand. If preferred_name given, try that first.
Enforces the one-land-per-turn rule (MTG CR 305.2). Cards that grant
additional land plays (e.g. Exploration, Oracle of Mul Daya) may bump
``_land_plays_this_turn_limit`` on the player to a higher value.
"""
p = sim.players[pi]
# ── LAND-PER-TURN ENFORCEMENT ──
max_land_plays = getattr(p, '_land_plays_this_turn_limit', 1)
lands_this_turn = getattr(p, '_lands_played_this_turn', 0)
if lands_this_turn >= max_land_plays:
return # Already used land drop(s) this turn
land_idx = -1
if preferred_name:
@@ -385,6 +402,7 @@ class DeepSeekGameEngine:
sim.next_card_id += 1
sim.add_to_battlefield(pi, land_card)
p.stats.lands_played += 1
p._lands_played_this_turn = lands_this_turn + 1
if events is not None and self.record_log:
events.append(f"Played land: {land_card.name}")
+19 -1
View File
@@ -135,6 +135,11 @@ class GameEngine:
sim.active_player_index = pi
phase_events: list[str] = []
# ── Reset per-turn counters (land drop, etc.) ──
p._lands_played_this_turn = 0
if not hasattr(p, '_land_plays_this_turn_limit'):
p._land_plays_this_turn_limit = 1
# ── Capture ML snapshot BEFORE turn actions ──
pre_snapshot = None
if self.ml_log:
@@ -454,8 +459,20 @@ class GameEngine:
return sim
def _play_land(self, sim: SimState, pi: int) -> None:
"""Play the first land from hand onto the battlefield."""
"""Play the first land from hand onto the battlefield.
Enforces the one-land-per-turn rule (MTG CR 305.2). Cards that grant
additional land plays may bump ``_land_plays_this_turn_limit`` on the
player to a higher value.
"""
p = sim.players[pi]
# ── LAND-PER-TURN ENFORCEMENT ──
max_land_plays = getattr(p, '_land_plays_this_turn_limit', 1)
lands_this_turn = getattr(p, '_lands_played_this_turn', 0)
if lands_this_turn >= max_land_plays:
return # Already used land drop(s) this turn
land_idx = -1
for i, card in enumerate(p.hand):
if card.is_land():
@@ -470,6 +487,7 @@ class GameEngine:
sim.next_card_id += 1
sim.add_to_battlefield(pi, land_card)
p.stats.lands_played += 1
p._lands_played_this_turn = lands_this_turn + 1
@staticmethod
def _send_to_graveyard(sim: SimState, card, owner_idx: int) -> None:
+173 -43
View File
@@ -229,61 +229,54 @@ class CommanderGameState:
def stack_is_empty(self) -> bool:
return len(self.stack) == 0
# Bug 9 fix: stub get_legal_moves() and apply_move() so turn_manager
# doesn't crash. These delegate to the rules engine once implemented.
def get_legal_moves(self, seat: int) -> list[dict]:
"""Return legal moves for a seat.
"""Return legal moves for ``seat``.
During DECLARE_BLOCKERS phase, emits block moves for defending
creatures against each attacker in sim_state.combat.
Otherwise returns pass-only (stub for remaining rules engine).
Delegates to :mod:`legal_move_generator` which produces macro
actions (play_land, cast_creature, cast_removal, cast_ramp,
cast_draw, cast_commander, attack, block, pass_priority).
"""
moves: list[dict] = [{"id": 0, "category": "pass_priority", "description": "Pass"}]
combat = self.sim_state.combat
# Local import to avoid a circular reference at module load
from commander_ai_lab.sim.legal_move_generator import generate_legal_moves
return generate_legal_moves(self, seat)
# Emit block moves during declare_blockers phase
if (
combat
and combat.active
and self.current_phase in (Phase.DECLARE_BLOCKERS, "declare_blockers")
and seat != self.active_player_seat
):
move_id = 1000
my_creatures = [
c for c in self.sim_state.get_battlefield(seat)
if c.is_creature() and not c.tapped
]
for atk_id in combat.attackers:
atk = next(
(c for bf in self.sim_state.battlefields for c in bf if c.id == atk_id),
None,
)
if atk is None:
continue
for blocker in my_creatures:
moves.append({
"id": move_id,
"category": "block",
"blocker_id": blocker.id,
"attacker_id": atk_id,
"description": f"Block {atk.name} with {blocker.name}",
})
move_id += 1
return moves
def apply_move(self, seat: int, move_id: int) -> None:
"""Apply a chosen move.
"""Apply a chosen move to the game state.
Handles 'block' moves by updating sim_state.combat.blockers.
Other categories are no-op stubs until the rules engine is wired.
Supports all move categories produced by
:mod:`legal_move_generator`. For categories that require full
rules simulation (spell resolution, combat damage), this is
still a thin layer — it mutates the game state and delegates
final resolution to the engine/turn manager.
"""
# Find the move dict
all_moves = self.get_legal_moves(seat)
move = next((m for m in all_moves if m["id"] == move_id), None)
if move is None:
return
if move.get("category") == "block":
category = move.get("category")
if category == "pass_priority":
return
if category == "play_land":
self._apply_play_land(seat, move)
return
if category in ("cast_creature", "cast_removal", "cast_ramp",
"cast_draw", "cast_spell"):
self._apply_cast_spell(seat, move)
return
if category == "cast_commander":
self._apply_cast_commander(seat, move)
return
if category == "attack":
self._apply_declare_attacker(seat, move)
return
if category == "block":
combat = self.sim_state.combat
if combat and combat.active:
atk_id = move["attacker_id"]
@@ -291,6 +284,143 @@ class CommanderGameState:
if atk_id not in combat.blockers:
combat.blockers[atk_id] = []
combat.blockers[atk_id].append(blocker_id)
return
# ── Move execution helpers ───────────────────────────────
def _apply_play_land(self, seat: int, move: dict) -> None:
"""Remove the chosen land from hand, place on battlefield, bump counter."""
if not (0 <= seat < len(self.commander_players)):
return
player = self.commander_players[seat]
# Enforce land-per-turn at apply-time as a second safety check
if player.lands_played_this_turn >= player.land_plays_allowed:
return
card_name = move.get("card_name")
idx = next(
(i for i, c in enumerate(player.hand)
if c.name == card_name and c.is_land()),
-1,
)
if idx < 0:
return
land = player.hand.pop(idx)
land.owner_id = seat
land.tapped = False
land.turn_played = self.turn
land.id = self.sim_state.next_card_id
self.sim_state.next_card_id += 1
self.sim_state.add_to_battlefield(seat, land)
player.lands_played_this_turn += 1
self.land_drop_used = True
if hasattr(player, "base") and hasattr(player.base, "stats"):
player.base.stats.lands_played += 1
def _apply_cast_spell(self, seat: int, move: dict) -> None:
"""Remove the spell from hand, place it on the battlefield.
This is a simplified resolution (no stack, no targeting) sufficient
for macro-action training and live games until the full stack
engine is wired.
"""
if not (0 <= seat < len(self.commander_players)):
return
player = self.commander_players[seat]
card_name = move.get("card_name")
idx = next(
(i for i, c in enumerate(player.hand) if c.name == card_name),
-1,
)
if idx < 0:
return
card = player.hand.pop(idx)
card.owner_id = seat
card.turn_played = self.turn
card.id = self.sim_state.next_card_id
self.sim_state.next_card_id += 1
# Creatures and permanents land on the battlefield; instants and
# sorceries go straight to the graveyard after resolving.
type_line = (card.type_line or "").lower()
if "instant" in type_line or "sorcery" in type_line:
player.graveyard.append(card)
else:
card.tapped = False
self.sim_state.add_to_battlefield(seat, card)
if hasattr(player, "base") and hasattr(player.base, "stats"):
player.base.stats.spells_cast += 1
if card.is_creature():
player.base.stats.creatures_played += 1
def _apply_cast_commander(self, seat: int, move: dict) -> None:
"""Cast the commander from the command zone (increments tax)."""
if not (0 <= seat < len(self.commander_players)):
return
player = self.commander_players[seat]
card_name = move.get("card_name")
idx = next(
(i for i, c in enumerate(player.commander_zone)
if c.name == card_name),
-1,
)
if idx < 0 and player.commander_zone:
idx = 0
if idx < 0:
return
cmd = player.commander_zone.pop(idx)
cmd.owner_id = seat
cmd.turn_played = self.turn
cmd.tapped = False
cmd.is_commander = True
cmd.id = self.sim_state.next_card_id
self.sim_state.next_card_id += 1
self.sim_state.add_to_battlefield(seat, cmd)
player.commander_cast_count += 1
if hasattr(player, "base") and hasattr(player.base, "stats"):
player.base.stats.spells_cast += 1
if cmd.is_creature():
player.base.stats.creatures_played += 1
def _apply_declare_attacker(self, seat: int, move: dict) -> None:
"""Mark a creature as attacking in the current combat state.
``combat.attackers`` is a ``dict[int, int]`` mapping attacker
card id to defending seat. If no defending seat has been
chosen yet, fall back to the first non-active, non-eliminated
opponent.
"""
combat = self.sim_state.combat
if combat is None:
return
atk_id = move.get("attacker_id")
if atk_id is None:
return
# Tap the attacker unless it has vigilance
for card in self.sim_state.get_battlefield(seat):
if card.id == atk_id:
has_vigilance = (
card.has_keyword("vigilance")
if hasattr(card, "has_keyword") else False
)
if not has_vigilance:
card.tapped = True
break
# Determine defending seat
defending_seat = combat.defending_seat
if defending_seat < 0 or defending_seat == seat:
for i, p in enumerate(self.commander_players):
if i != seat and not p.eliminated:
defending_seat = i
break
if defending_seat >= 0:
combat.defending_seat = defending_seat
if defending_seat >= 0:
combat.attackers[atk_id] = defending_seat
combat.active = True
def living_players(self) -> list[CommanderPlayer]:
return [p for p in self.commander_players if not p.eliminated]
@@ -0,0 +1,230 @@
"""
Legal Move Generator
====================
Generates the set of legal moves a player can take from a
:class:`CommanderGameState`. This replaces the "pass-only" stub in
``CommanderGameState.get_legal_moves`` so the turn_manager/policy-server
path can drive real games instead of just passing priority every turn.
The emitted moves are intentionally coarse (macro-actions: play_land,
cast_creature, cast_removal, cast_ramp, cast_draw, cast_commander,
attack, block, pass_priority) so that both the neural policy and
heuristic/LLM brains can pick from the same list.
Long-term, when the Forge IPC path is fully wired, Forge's own legal
action list will replace the output of this module. Until then, this
is the authoritative move source for the Python rules layer.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from commander_ai_lab.sim.models import Card, Phase
if TYPE_CHECKING:
from commander_ai_lab.sim.game_state import CommanderGameState
# Map from card enrichment flags to macro-action categories the AI brain
# recognizes. Order matters: the first True flag wins.
_SPELL_FLAG_TO_CATEGORY: tuple[tuple[str, str], ...] = (
("is_ramp", "cast_ramp"),
("is_removal", "cast_removal"),
("is_board_wipe", "cast_removal"),
)
def _classify_spell(card: Card) -> str:
"""Map a Card to a macro-action category string.
Matches the 8-action macro-action space used by the policy network
(``scope.py``): cast_creature, cast_removal, cast_draw, cast_ramp,
cast_commander, attack_opponent, hold_mana, pass. Cards that don't
fit any flag fall back to ``cast_spell`` so downstream code can
still execute them.
"""
for flag, category in _SPELL_FLAG_TO_CATEGORY:
if getattr(card, flag, False):
return category
# Heuristic: draw spells often mention "draw" in oracle text
ot = (getattr(card, "oracle_text", "") or "").lower()
if "draw" in ot and ("card" in ot or "cards" in ot):
return "cast_draw"
if card.is_creature():
return "cast_creature"
return "cast_spell"
def _count_available_mana(game_state: "CommanderGameState", seat: int) -> int:
"""Rough available mana = floating pool + untapped lands on battlefield.
This is intentionally permissive. A production rules engine would
also consider mana dorks, mana rocks, and color requirements, but
for macro-action gating CMC-vs-total-mana is sufficient.
"""
player = game_state.players[seat]
mana = 0
# Floating mana pool (if ManaPool has ``total``)
pool = getattr(player, "mana_pool", None)
if pool is not None:
total_fn = getattr(pool, "total", None)
if callable(total_fn):
try:
mana += int(total_fn())
except Exception:
pass
# Untapped lands on battlefield (each produces ~1 mana)
for card in game_state.sim_state.get_battlefield(seat):
if card.is_land() and not getattr(card, "tapped", False):
mana += 1
return mana
def _phase_is(current: object, target: Phase) -> bool:
"""Safe phase comparison that works with Phase enum or raw string."""
if current == target:
return True
if isinstance(current, str) and current == target.value:
return True
return False
def generate_legal_moves(game_state: "CommanderGameState", seat: int) -> list[dict]:
"""Produce the list of legal moves for ``seat`` given ``game_state``.
Always includes a pass-priority move (id 0) as a safe fallback.
Additional moves are emitted based on phase, turn ownership, and
available resources.
"""
moves: list[dict] = [
{"id": 0, "category": "pass_priority", "description": "Pass"}
]
if not (0 <= seat < len(game_state.players)):
return moves
player = game_state.players[seat]
if getattr(player, "eliminated", False):
return moves
current_phase = game_state.current_phase
is_active = (seat == game_state.active_player_seat)
move_id = 1
# ── Sorcery-speed actions (main phases, active player only) ──
if is_active and (
_phase_is(current_phase, Phase.MAIN1)
or _phase_is(current_phase, Phase.MAIN2)
):
# Land drop — respect one-land-per-turn unless modified
lands_played = getattr(player, "lands_played_this_turn", 0)
land_limit = getattr(player, "land_plays_allowed", 1)
if lands_played < land_limit:
for card in player.hand:
if card.is_land():
moves.append({
"id": move_id,
"category": "play_land",
"card_name": card.name,
"description": f"Play {card.name}",
})
move_id += 1
# Castable spells from hand
available_mana = _count_available_mana(game_state, seat)
for card in player.hand:
if card.is_land():
continue
cost = int(getattr(card, "cmc", 0) or 0)
if cost <= available_mana:
category = _classify_spell(card)
moves.append({
"id": move_id,
"category": category,
"card_name": card.name,
"cmc": cost,
"description": f"Cast {card.name} ({cost} mana)",
})
move_id += 1
# Commander from the command zone (respect commander tax)
tax = 0
if hasattr(player, "commander_tax"):
try:
tax = int(player.commander_tax())
except Exception:
tax = 0
for cmd in getattr(player, "commander_zone", []):
base_cost = int(getattr(cmd, "cmc", 0) or 0)
total_cost = base_cost + tax
if total_cost <= available_mana:
moves.append({
"id": move_id,
"category": "cast_commander",
"card_name": cmd.name,
"cmc": total_cost,
"description": (
f"Cast {cmd.name} from command zone "
f"(base={base_cost}, tax={tax})"
),
})
move_id += 1
# ── Declare attackers — one attack move per legal attacker ──
if is_active and _phase_is(current_phase, Phase.DECLARE_ATTACKERS):
bf = game_state.sim_state.get_battlefield(seat)
for creature in bf:
if not creature.is_creature():
continue
if getattr(creature, "tapped", False):
continue
# Summoning sickness: played this turn and no haste
turn_played = getattr(creature, "turn_played", -1)
has_haste = creature.has_keyword("haste") if hasattr(creature, "has_keyword") else False
if turn_played == game_state.turn and not has_haste:
continue
moves.append({
"id": move_id,
"category": "attack",
"attacker_id": creature.id,
"card_name": creature.name,
"description": f"Attack with {creature.name}",
})
move_id += 1
# ── Declare blockers — emit block pairings for defending player ──
if _phase_is(current_phase, Phase.DECLARE_BLOCKERS) and not is_active:
combat = game_state.sim_state.combat
if combat and getattr(combat, "active", False):
defender_creatures = [
c for c in game_state.sim_state.get_battlefield(seat)
if c.is_creature() and not getattr(c, "tapped", False)
]
for atk_id in getattr(combat, "attackers", []):
atk = None
for bf in game_state.sim_state.battlefields:
for c in bf:
if c.id == atk_id:
atk = c
break
if atk is not None:
break
if atk is None:
continue
for blocker in defender_creatures:
moves.append({
"id": move_id,
"category": "block",
"blocker_id": blocker.id,
"attacker_id": atk_id,
"description": f"Block {atk.name} with {blocker.name}",
})
move_id += 1
return moves
+4
View File
@@ -267,6 +267,10 @@ class CommanderTurnManager:
player = self.gs.players[seat]
logger.debug("Turn %d — %s's turn begins", turn_num, player.name)
# ── Reset per-turn counters at the start of each turn ──
# Enforces MTG CR 305.2 (one land per turn unless modified).
player.lands_played_this_turn = 0
for phase in PHASES:
if self.game_over:
return
@@ -59,7 +59,13 @@ public class LearnedPolicy implements AiPolicy {
public LearnedPolicy(String serverUrl) {
this.client = new PolicyServerClient(serverUrl);
this.playstyle = "midrange";
this.greedy = true;
// Default to stochastic sampling (greedy=false) so the policy
// network actually explores its action distribution during live
// games. With greedy=true the AI always picks argmax, which
// combined with a pass/land-biased policy causes the "AI only
// plays lands" collapse. Callers that want deterministic
// replay can still override via the 3-arg constructor.
this.greedy = false;
this.policyServerAvailable = false;
}