fix: walk back round-1/2 conclusions — the cause was think=false all along
Seth asked "was this with think=false?" Yes — and that was the only question that mattered. Everything I concluded in round 1 and round 2 was wrong. Actual cause, isolated in round 3: - At identical message state, gemma4:26b with think=false returns eval=4 (silent stop); with think unset or think=true, returns eval=165 and emits the correct tool call. - Original round-1 write_file harness + think unset: 26B passes in 8 iters, 20s. No mitigations needed. - 31B dense and qwen3-coder:30b tolerate think=false; 26B MoE does not. Red herrings (kept on-record in the bakeoff doc, not silently erased): - Round 1: "write_file tool-call argument size" — wrong - Round 2a: refuted the arg-size theory but for the wrong reason (still failed because think=false was still set) - Round 2b: "cumulative tool-response context size" — truncating did make 26B pass, but by coincidence. Shorter context at the decision turn dodged the think=false side effect. Why the existing "always think:false" guidance was misleading: it was derived from AI_Visualizer (single-turn JSON pipelines) where thinking tokens do eat num_predict invisibly. In multi-turn tool-calling agents the channels are separate and the flag has a different effect — catastrophic on 26B specifically. Doc updates: - GOTCHAS: replaced the 26B entry with the actual cause; scoped the original "Thinking Mode Eats Context" entry to single-turn pipelines - SYNTHESIS: split the "Mandatory Ollama Settings" block into single-turn vs multi-turn variants; updated anti-patterns and quick-start checklist - CORPUS_cli_coding_agent.md: revised pointer and config template - docs/reference/bakeoff-2026-04-18.md: added Round 3 section with the correction notice at the top of the file and full diagnostic methodology New artifacts: harness_no_think_flag.py, harness_write_no_think.py, and 4 new log files demonstrating all three models pass when think is left at default.
This commit is contained in:
+11
-11
@@ -6,16 +6,17 @@
|
||||
> `IMPLEMENTATIONS.md` chat-agent patterns (Simon) and pipeline patterns
|
||||
> (AI_Visualizer).
|
||||
|
||||
> **Empirical follow-up:** `docs/reference/bakeoff-2026-04-18.md` — 2 rounds of
|
||||
> **Empirical follow-up:** `docs/reference/bakeoff-2026-04-18.md` — 3 rounds of
|
||||
> runs against a custom minimal CLI-agent harness on a fix-the-median-bug task.
|
||||
> **Round 1:** 31B clean (8 iters), Qwen3-Coder correct but chatty (15 iters),
|
||||
> 26B silently quits with zero edits. **Round 2 (diagnostic):** the 26B failure
|
||||
> is NOT about edit-tool-argument size — it's about **cumulative tool-response
|
||||
> context shape**. Capping tool responses ≤1200 chars makes 26B pass cleanly
|
||||
> *and* in the fastest wall time of any run (8.4s). Most production CLI agents
|
||||
> already truncate tool responses, so the issue may be invisible in them.
|
||||
> Read when: scoping which model to point an agent at, hitting an unexpected
|
||||
> tool-call halt, or writing a custom harness targeting the 26B MoE.
|
||||
> **Bottom line after the full investigation:** all three models (gemma4:26b,
|
||||
> gemma4:31b-it-q4_K_M, qwen3-coder:30b) pass the task cleanly in 8-14
|
||||
> iterations. The only real gotcha is **`think: false` in the Ollama payload
|
||||
> silently breaks 26B** in multi-turn tool-calling loops — contradicts the
|
||||
> older "always think:false" guidance which was derived from single-turn
|
||||
> pipelines. Round 1 and Round 2 pursued wrong hypotheses (edit-tool size,
|
||||
> tool-response size) before Round 3 isolated the actual cause. Read when:
|
||||
> configuring an agent payload, debugging a silent halt, or fact-checking
|
||||
> older think-flag guidance.
|
||||
|
||||
## TL;DR
|
||||
|
||||
@@ -90,7 +91,6 @@ The baseline settings from `SYNTHESIS.md` still apply. CLI coding agent-specific
|
||||
```json
|
||||
{
|
||||
"model": "gemma4:26b",
|
||||
"think": false,
|
||||
"keep_alive": "4h",
|
||||
"options": {
|
||||
"num_ctx": 32768,
|
||||
@@ -103,7 +103,7 @@ The baseline settings from `SYNTHESIS.md` still apply. CLI coding agent-specific
|
||||
- `num_ctx: 32768` is the working minimum for repo-scale work. Agents interleave file reads, bash output, and edits; 4K will truncate the second `read_file`.
|
||||
- `num_predict: 4096` — single edits are short but the agent may emit a bash invocation + reasoning + tool call in one turn.
|
||||
- `temperature: 0.3` — per `SYNTHESIS.md` temperature table, "structured extraction" tier. Coding edits want low variance.
|
||||
- `think: false` — critical. `GOTCHAS.md` documents that Ollama 0.20+ thinking silently eats `num_predict` and drops tool calls. If an agent somehow injects `think: true`, you'll see empty responses.
|
||||
- **Do NOT set `think: false`.** Leave it unset (Ollama default). Verified 2026-04-18 that setting `think: false` silent-stops `gemma4:26b` at tool-decision turns in multi-turn loops. The older single-turn-pipeline guidance ("always think: false") does not apply here — see `GOTCHAS.md` § "`think: false` Kills Gemma 4 26B in Multi-Turn Tool-Calling Loops".
|
||||
- `keep_alive: 4h` — agent sessions have think pauses; avoid reload penalty.
|
||||
|
||||
### Streaming
|
||||
|
||||
Reference in New Issue
Block a user