LabCli passed relative 'results' to enableMlLogging(), which resolved
to Forge-Repo/forge-gui/results/ because the Java subprocess CWD is
forgeWorkDir. Now uses outputPath's parent directory (absolute) so
JSONL training data goes to commander-ai-lab/results/ where the
Python ML pipeline and Training page expect it.
Forge verbose log uses different verb forms and prefixes than the
patterns expected:
- Turn: 'Turn: Turn 5 (Ai(1)' not 'Turn 5 (Ai(1)'
- Cast: 'cast' (no s) not 'casts', 'Add To Stack:' prefix, no period
- Land: 'played' not 'plays', 'Land:' prefix, card ID suffix '(76)'
- Draw: 'drew' variant, card ID suffix
Updated all four patterns to match both old and new Forge output
formats. This is the final fix for the training data pipeline —
DecisionExtractor will now find cast/land/draw/attack events in
the Forge game log and emit JSONL training snapshots.
Forge's sim mode looks up decks by the Name= field inside the .dck
file metadata header, NOT by filename. For example:
Filename: Counter_Intelligence.dck
Header: Name=Counter Intelligence
Forge expects: 'Counter Intelligence' (with space)
BatchRunner now reads the Name= field from the .dck file and passes
that to Forge. Falls back to filename with underscores→spaces if
the Name= field can't be read.
Forge's sim mode looks up decks by NAME in its own
%APPDATA%/Forge/decks/commander/ directory. Passing a full path like
D:\...\precon-decks\Counterpunch.dck causes 'Could not load deck'.
Now strips the path and .dck extension so Forge sees just the bare
name (e.g. 'Counterpunch') and finds it in its deck folder.
Precon .dck files must be copied to %APPDATA%/Forge/decks/commander/
for Forge to find them.
The condition 'mlLoggingEnabled && !quiet && mlDecisionLogger != null'
silently skipped ML training data extraction whenever quiet mode was
active. DecisionExtractor works with the same log lines regardless of
quiet mode, so the guard was unnecessary and prevented JSONL generation.
Now: mlLoggingEnabled && mlDecisionLogger != null — ML logging runs in
all modes as long as --ml-log is passed.
The Java DecisionLogger was writing files as ml-decisions-{batchId}.jsonl
but the Python dataset_builder.py scans for ml-decisions-sim-*.jsonl to
categorize them as 'forge' source training data.
This one-line fix adds the 'sim-' prefix so the Forge batch heuristic
path now feeds directly into the ML training pipeline:
BatchRunner → DecisionExtractor → DecisionLogger
→ results/ml-decisions-sim-{batchId}.jsonl
→ dataset_builder.py discovers as 'forge' source
→ encode → train
The full chain was already wired (Issue #83 Phase 1 infrastructure),
this naming mismatch was the only thing preventing it from working.
Refs: Issue #83 Phase 1
## New File
- PolicyControllerAi.java (840 lines): Extends Forge's PlayerControllerAi
to intercept AI decisions at chooseSpellAbilityToPlay() and route them
through the Python policy server. Falls back to Forge built-in AI on
any failure. Inner PolicyAiController overrides AiController with full
legal action gathering via ComputerUtilAbility.
## Wiring
- GameSession.java: Import + Javadoc for PolicyControllerAi decision flow.
Logs decision pipeline when policy server connects. End-of-game reward
submission via PolicyControllerAi lifecycle.
- BatchRunner.java: New setPolicyServer(url) method. Appends
--policy midrange --policyServer <url> to Forge subprocess command when
policy server URL is configured.
## Path Updates (Forge-Repo rename)
- README.md: Forge link → KoalaTrapLord/Forge-Repo, build path updated
- .env.example: FORGE_JAR_PATH and FORGE_WORKING_DIR → Forge-Repo
- lab_api.py: Auto-detect paths try Forge-Repo first, forge-repo as legacy
## MacroActionExecutor
- Added public isRemovalCard(), isDrawCard(), isRampCard() wrappers so
PolicyControllerAi can consult the same card-role databases.
Refs: Issue #83 Phase 2.4
Addresses the three root causes in commander-ai-lab-land-only-bug-analysis.md
plus several collateral fixes discovered along the way.
P0 — Land-per-turn enforcement (engine.py, deepseek_engine.py, turn_manager.py)
* Guard _play_land() against double plays by tracking
_lands_played_this_turn vs _land_plays_this_turn_limit on each player
* Reset per-turn counters at the start of each player's turn
* Ramp cards that grant additional plays can bump the limit
P0 — Legal move pipeline (new legal_move_generator.py, game_state.py)
* New generate_legal_moves() emits the full macro-action set
(play_land, cast_creature, cast_removal, cast_ramp, cast_draw,
cast_commander, attack, block, pass_priority) based on phase,
active player, land-drop state, mana, and commander tax
* get_legal_moves() delegates to the new generator instead of the
pass-only stub that blocked the turn_manager/policy-server path
* apply_move() gains handlers for play_land, cast_spell,
cast_commander, and declare_attacker — spells resolve to
battlefield or graveyard, stats update correctly
P1 — Policy exploration (LearnedPolicy.java, scope.py)
* Default greedy=false so the neural policy samples its
distribution instead of always taking argmax
* Raise ppo_entropy_coeff 0.01 → 0.04 to keep the action
distribution flat during PPO self-play and prevent the
pass/land collapse
Collateral fixes pulled in during the same session:
* lan-client/precon-loader.js: await enrichAllZonesWithScryfall()
so opening-hand cards have typeLine/cmc before the AI's first turn
* lan-client/ui.js: closeZoneModal auto-shuffles the library
whenever the zone modal closes
* routes/game.py: BatchState() was missing total_games/threads,
crashing POST /api/game/start
* routes/deckgen.py: dead code after raise left headers undefined,
breaking the Anthropic API path
All sim/phase/integration tests pass (650+).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Forge's sim mode does not honour --policy/--policyServer CLI flags, so
the Python brain was never called during live games. AI only played lands
because Forge ran its built-in AI for every decision.
Fix: when parseForgeLine() detects a Priority event for an AI seat, the
session now immediately calls policyClient.decide() with the current
state snapshot and POSTs the result to /api/policy/override before Forge
reads the next line. This mirrors what handleHumanTurn() already does
for seat 0. The synthetic fallback path is unchanged.
- start-lab.bat: Set FORGE_DECKS_DIR to D:\ForgeCommander\commander-ai-lab\precon-decks
- BatchRunner.java: resolveForgeDecksPath() now checks PRECON_DECKS_DIR env var
first, then falls back to the hardcoded precon-decks path, before trying
%APPDATA%\Forge\decks\commander. Also pass full absolute path to Forge -d flag.
- .env.example: document PRECON_DECKS_DIR variable
Replaces synthetic stub loop with a live Forge process:
- Spawns Forge with --policy + --policyServer flags so Forge calls
back to /api/policy/decide at each AI decision point
- Reads Forge stdout line-by-line and parses zone/phase/life updates
via the same regex patterns used in BatchRunner
- Syncs full board state (battlefield, hand, graveyard, exile,
command zone, stack) on every Forge state-change line
- Routes human input from humanActionQueue via a side-channel
HTTP POST to /api/policy/override on the policy server
- Handles activePlayer + priorityPlayer correctly for 3-4 player pods
- Falls back to synthetic stub loop if Forge subprocess fails to start
- Watchdog thread kills stalled Forge process after 10 min
Adds a buildStateVector(int seat) method to GameSession that produces
a float[29] global scalar vector matching STATE_DIMS layout in
ml/config/scope.py:
Per-player block (14 features × 2 players = 28):
[0] life_total / 40.0
[1] commander_damage / 21.0 (stub 0.0 until real Forge tracks it)
[2] mana_available / 20.0 (stub 0.0 until mana pool wired)
[3] commander_tax / 10.0
[4] cards_in_hand / 15.0
[5] cards_in_graveyard / 100.0
[6] creatures_on_battlefield / 30.0
[7] total_power_on_board / 100.0 (stub = creature count until P/T data)
[8] is_active_player (0 or 1)
[9] phase one-hot[0] main1
[10] phase one-hot[1] combat
[11] phase one-hot[2] main2
[12] phase one-hot[3] end
[13] (padding / reserved)
Global:
[28] turn_number / max_turns (25)
buildStateSnapshot() now includes "state_vector": float[] so that
PolicyClient.decide(JsonObject) sends a machine-readable vector the
Python /api/policy/decide endpoint can pass directly to the encoder
without re-parsing the nested human-readable fields.
Stub fields (mana_available, commander_damage, total_power_on_board)
are zero until real Forge board state is wired in Step 4.
GameSession.java references these APIs which were missing from
PolicyClient, causing 3 compilation errors. Added:
- getBaseUrl() getter
- PolicyDecision record (action, confidence)
- decide(String, int) overload that delegates to decide(JsonObject)
DecisionExtractor requires verbose Forge output to reconstruct board
states. When quiet=true, Forge runs with -q which strips the verbose
log lines, causing 0 decisions to be extracted. This change passes
quiet=false to BatchRunner when mlLogEnabled is true.
* feat(83): add PolicyClient for live Forge-to-policy-server IPC (#83)
Phase 2.1: Bridge between Forge decision points and Python policy
server for online training. Calls /api/policy/decide endpoint,
collects (state, action, reward) tuples in real-time, and flushes
them to the server for online PPO updates.
Falls back to Forge built-in AI if server is unavailable.
* feat(83): add /api/policy/* routes for live Forge IPC (#83)
Phase 2.2: Policy decision routes for real-time Forge IPC.
- POST /api/policy/decide — full game state inference with log_prob/value
- POST /api/policy/collect — online learning tuple batch collection
- POST /api/policy/reward — end-of-game reward propagation
- GET /api/policy/health — readiness check for Java PolicyClient
- GET /api/policy/stats — session statistics
Includes Forge phase mapping and PlayerZoneState with full zone contents.
* feat(83): integrate PolicyClient into GameSession for bidirectional sync (#83)
- CAST_PATTERN: Add optional (?:Add To Stack:\s*)? prefix to match Forge's verbose log format
- LAND_PLAY_PATTERN: Add optional (?:Land:\s*)? prefix to match Forge's verbose log format
- Add [PARSE] diagnostic logging after combat stats parsing to confirm spells/lands/cmdr stats per seat
Root cause: Forge verbose output prefixes cast lines with "Add To Stack: " and land lines with "Land: "
but the regexes expected Ai(N) at the start of the line, causing 0 matches for all combat stats.
Issue #1 — Winner's life total now captured:
- Parse verbose Forge log for life-total-change events
- Track last-known life per seat via 3 regex patterns
- Winner's final life set from log data (not left at default 40)
- Losers who died to mill/poison retain their actual life total
Issue #2 — Win condition classification formalized:
- New WinCondition enum (schema/WinCondition.java) with 8 categories
- Centralized Forge loss-reason-string → enum mapping (regex-based)
- UNKNOWN fallback with logging for unrecognized loss reasons
- All classification now goes through WinCondition.classify()
Issue #3 — JVM startup overhead reduced:
- Added -XX:+UseSerialGC (lighter GC for batch processes)
- Added -XX:TieredStopAtLevel=1 (faster JIT, less compilation)
- ForgeWorker stub class for future persistent batch mode
- Estimated reduction: ~2-3s → ~1-1.5s per game startup
Issue #4 — Single-threaded throughput optimization:
- New CLI flags: --ai-simplified, --ai-think-time, --benchmark
- JVM flags passed through to Forge subprocess
- Benchmark mode: run 1 game before batch, report wall/Forge/overhead times
- API support: aiSimplified, aiThinkTimeMs in StartRequest
Issue #5 — Threading, progress, and backpressure:
- --threads 0 or -1 auto-detects optimal count from CPU topology
- detectOptimalThreads() estimates physical cores (excludes HT)
- ProgressCallback interface for real-time sims/sec reporting
- Per-game progress emitted as [PROGRESS] lines, parsed by lab_api.py
- --max-queue flag for graceful backpressure (limits concurrent JVMs)
- Final throughput summary printed at batch completion
- StatusResponse now includes simsPerSec field
Also: .gitignore restored, lab_api.py updated with all new API fields