From b779987802e06c75694f52bf66d93884644ec735 Mon Sep 17 00:00:00 2001 From: fez5587 Date: Mon, 26 Oct 2026 16:53:23 -0500 Subject: [PATCH] FARMHOUSE-CHANGES: notes on making LLM-opponent turns faster Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_016Er49soaMyG7Yburq57PeB --- FARMHOUSE-CHANGES.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/FARMHOUSE-CHANGES.md b/FARMHOUSE-CHANGES.md index 96c070e..e21f988 100644 --- a/FARMHOUSE-CHANGES.md +++ b/FARMHOUSE-CHANGES.md @@ -55,7 +55,14 @@ After the fixes: a 40-game Monte Carlo takes 0.4 s with real numbers (10 spells Run them with `-I` and from outside the package folders; do not put `services/` on `sys.path` (`services/logging.py` shadows the standard library). ## Things still worth doing -- Run the LLM-opponent games **in parallel**: each move is a short prompt with a short answer, which a batching server handles well. +- **Make the LLM-opponent turns faster** (a piloted move averages ~7.5 s: about 2.2k tokens in, ~30 out, so reading the prompt dominates). + These are ideas, not measured; none is implemented: + - *Shared prefix:* about 1.2k tokens of every prompt (the system prompt) are identical each time; a llama.cpp server can reuse them + if the calls share the prefix, so only the board state is read each move. + - *Smaller prompt:* send a compact board state instead of the indented JSON the engine builds now (`json.dumps(snapshot, indent=2)` + in `deepseek_brain.py`). + - *Parallel games:* run several games at once, which would also fill the GPU while one game waits on a move. Caveat: for long prompts + two requests at once gave no extra throughput on a V100 (reading the prompt is compute-bound), so measure before relying on it. - Opponents are the engine's heuristics; only the piloted seat uses the model. - Build the card-search index and measure it. - The upstream `LICENSE` is missing (`pyproject.toml` says MIT). Ask the author to add one before publishing a derivative widely.