Coordinate-whitened GeGLU late pre-edit commit gate across 13 high-signal layers (dose 1.20)
When
Recorded date: 2026-11-12
Ledger date only; exact weight-build/change timestamp is not recorded here. Not a live training event.
Exact tensor-change timestamp: unavailable.
Why change it?
Across the 210 `b001` episodes, a second major failure bottleneck—completely untouched by all seven queued thought-shortening edits (`0007`--`0013`)—is **indecisive `read_file` spiralling into the 30-tool-call hard cap**: - Solved episodes (`n=108`) average `5.48` `read_file` calls and make their first file edit on call `10.3` (`108/108` edit at least once). - Unsolved episodes average `8.90`--`11.20` `read_file` calls and delay their first edit to call `15.0` (or never edit). In **5 of the 10 within-task discordant `b001` tasks** (`kayak__pypika`, `conda__conda-6131`, `mozilla__bleach`, `automl__smac3-1069`), failing arms still had up to…
Which part changed?
Coordinate-whitened GeGLU late pre-edit commit gate across 13 high-signal layers (dose 1.20)
The source describes adapter targets and mathematical changes. No before/after tensor comparison is available in this viewer.
How was it changed?
Computed directly by author-written code and packaged as a low-rank adapter over the frozen base; not gradient-trained. base: champion generation 0 Designed and pre-tested on generation 0 (unedited checkpoint); entry contains only this change. Completely orthogonal in mechanism and target positions to all queued thought-ending edits (`0007`--`0013`), so it stacks cleanly if any thought-ending edit is promoted to champion generation 1. size: whole network 1.055181% over 39 matrices, largest matrix 9.659172% control size: whole network 1.055154% over 39 matrices, largest matrix 9.659172% Data and task split: `31` fit tasks and `10` strictly held-out check tasks from `wsp31_screen_v1` (`tools/data/probe_calls_b001.jsonl` plus exact per-call prefixes from…
Did it improve?
- · No task-level result recorded in this snapshot.
Evaluation set: wsp31_screen_v1 · Batch: -. Local screening and historical task scores are not competition scores unless explicitly identified in the source. Different evaluation sets cannot be compared as a gain.
Original evidence
- research/wsp31/ledger/0017-gemini-late-preedit-commit-gate.md
- research/wsp31/paper/tables/leaderboard.md
Literal source excerpts follow. They may contain original predictions and superseded counts; use the measured-outcome summary and corrections above for the current result.
Hypothesis
Across the 210 `b001` episodes, a second major failure bottleneck—completely untouched by all seven queued thought-shortening edits (`0007`--`0013`)—is **indecisive `read_file` spiralling into the 30-tool-call hard cap**:
- Solved episodes (`n=108`) average `5.48` `read_file` calls and make their first file edit on call `10.3` (`108/108` edit at least once).
- Unsolved episodes average `8.90`--`11.20` `read_file` calls and delay their first edit to call `15.0` (or never edit). In **5 of the 10 within-task discordant `b001` tasks** (`kayak__pypika`, `conda__conda-6131`, `mozilla__bleach`, `automl__smac3-1069`), failing arms still had up to `210s` of wall-clock left (`max_think = 787`--`1270`, so deep-thought gates never fire!) yet burned 25--30 tool calls on 20-line `read_file` slices until `BudgetExceeded` at call 30 blocked `edit_file` and forced an empty `submit_patch`.
Why this differs from `PRIOR_E4B.md` ("always-on edit sooner: 12 vs 28") and `0007`--`0013`:
- Always-on "edit sooner" failed on E4B because pushing `edit_file` on turns 1--5 forces blind edits before reading the code and perturbs tokens inside tool arguments.
- Here we compile a **two-stage decomposed, coordinate-whitened, leak-free GeGLU gate** into one dormant MLP neuron per layer across the 13 highest-signal layers (`L30..39` and `L57..59`, `39` MLP matrices; skipping the lower-AUC middle band `L40..56`) that opens when (a) the current position is the `:` of `<|tool_call>call:` (`held-out AUC > 0.99`), (b) **zero** `edit_file`/`write_file` calls have occurred so far, AND (c) **8 or more** tool calls have already been spent exploring. Inside thoughts, turns, and 3 & 9 tokens later inside `edit_file` arguments (`neg_arg`), pre-activations sit below `-4.5` (`gelu_tanh <= -0.00002`, zero GeLU negative leak), leaving code generation and verification unperturbed.Exact recipe
base: champion generation 0
Designed and pre-tested on generation 0 (unedited checkpoint); entry contains only this change. Completely orthogonal in mechanism and target positions to all queued thought-ending edits (`0007`--`0013`), so it stacks cleanly if any thought-ending edit is promoted to champion generation 1.
size: whole network 1.055181% over 39 matrices, largest matrix 9.659172%
control size: whole network 1.055154% over 39 matrices, largest matrix 9.659172%
Data and task split: `31` fit tasks and `10` strictly held-out check tasks from `wsp31_screen_v1` (`tools/data/probe_calls_b001.jsonl` plus exact per-call prefixes from `results/wsp31/batches/b001/{base,aa}/requests`, partitioned deterministically by `int(sha256(instance_id)[:8], 16) % 4 == 0` for check and `!= 0` for fit; `98` balanced 4-stratum calls, zero task-outcome labels used in any direction or gate, no trained adapter touched).
Construction (`code/fit_and_build.py`, family `late_commit_gate`, 1 dormant neuron replaced via `mode="set"` in each of the 13 layers `30..39` and `57..59` = 39 MLP matrices, adapter rank 1):
1. Dormant neuron selection: in each layer `L in {30..39, 57..59}`, select `j_L = argmin_j mean(|mlp_act_L[j]|)` on fit passes (`mean |act| ~ 0`).
2. Coordinate-whitened two-stage Woodbury ridge gate on `mlp_in[L]` (fit tasks only): standardize coordinates by fit-set RMS `c_rms` (neutralizing `pre_feedforward_layernorm` outlier gains per `ARCH.md` #4) and fold `1 / c_rms` back into linear weights. Synthesize unit constant reader `u_z . z ~ 1.0` (`lambda = 0.50 d`). Fit Stage 1 `w_tool` (`lambda = 0.10 d`) separating `<|tool_call>call:` from all thought/turn/argument tokens (`held-out AUC ~ 0.999`), zero-centered on tool points (`w_tool - mu_tool u_z`) so it strongly suppresses non-tool tokens (`99.7%` below `-4.5`) without shifting within-tool scores. Fit Stage 2 `w_reg` (`lambda = 1.0 d`) purely on tool-call `:` points separating `pos_late_pre` (`n_edits == 0, n_tools >= 8`) from `neg_early_pre` (`n_tools <= 5`) and `neg_post_edit` (`n_edits >= 1`).
3. Leak-free GeGLU calibration: set `up_row = u_z / c_rms` and `gate_row = (k_t (w_tool - mu_tool u_z) + k_r (w_reg - tau_r u_z)) / c_rms` mapping the 78th percentile of negative tool calls to `-1.8`, the 70th percentile of `pos_late_pre` to `+2.0`, and `99.7%` of non-tool/argument points below `-4.5` (`gelu_tanh <= -2e-5`).
4. Write vector `down_col` (`gate_dose = 1.20`): unit direction combining fit-task `mlp_raw` tool-action contrast `unit(mean(mlp_raw[edit/write]) - mean(mlp_raw[read]))` (`100%` in layers `30..39`; blended `30%/70%` in final layers `57..59` with `clipdiv((E[edit] + 0.15 E[write] - 0.85 E[read] - 0.30 E[submit]) * final_norm, post_ff_norm)`), scaled so an active firing writes `0.10 * gate_dose` of `mlp_raw_rms[L]` per layer (capped at `0.055 * ||W_down[L]||_F * gate_dose` so final layer 59 never dominates).
Matched control (`ctrl/`): Replaces the exact same dormant neurons `j_L` with the **identical `gate_row` and `up_row`** (firing at the exact same late pre-edit tool-call positions and strengths), while replacing the write direction with a seeded coordinate-sign-flipped vector orthogonalized to `d_col` (`seed = 603000 + L`) matched to the exact per-matrix `|dW|_F`. This tests the `edit - read` write direction against an identical-norm, identical-gate perturbation at those exact decision points.
Approximations: bf16 storage; 5% `clipdiv` floor on `post_ff_norm`; ridge constant-reader `u_const . x ~ 1`; teacher-forced pre-test evaluation.
Held-out check-task gate diagnostics (mean over 13 layers `30..39, 57..59` on `10` unseen check tasks):
- Linear probe AUC (`pos_late_pre` vs negative buckets): vs `early_pre` `0.7316`, vs `post_edit` `0.7265`, vs `non-tool` `0.9617`.
- Active firing rate (`act > 0.1`): `pos_late_pre = 0.308` vs `neg_early_pre = 0.1026`, `neg_post_edit = 0.1196`, `neg_arg = 0.0245`, `neg_other = 0.0019`.
- Full-model pre-test note: On the 99 held-out causal decision points, `edit/` is `ACTIVE` with `61x` larger KL at `pos_late_pre` (`0.00061`) than `ctrl/` (`0.00001`), zero thought/close drift, `P(call:write)` `+0.001` across `tool` points, and `d_lp_read = -0.0801` / `d_lp_sub = +0.2558` at `pos_late_pre`. However, on the `n=5` held-out `pos_late_pre` calls (all 5 having base `p(edit) ~ 0` in the deep tail), non-firing calls sitting in GeLU's `[-1.8, 0]` negative trough (`gelu_tanh ~ -0.15`) pulled mean tail `d_lp_edit` negative (`-1.04`), a subtle single-neuron GeLU-trough caveat explicitly flagged for rollout verification and 2-neuron ReLU cancellation in Turn 3.
Rebuild: `bash build.sh [OUT_DIR]` executes `code/fit_and_build.py build` from saved `code/data/gemini_directions.pt` (fitted by `code/fit_and_build.py fit` from `code/capture_gemini_stats.py`). Verified byte-for-byte in an isolated scratch directory.
- Directions SHA256: `daa9e2b99a7d98fe9a8a52fe396b77ff5d7b0a19ba2b58f151949707336d04ee`
- Edit weights SHA256: `59603dbdc552e28d2c3cf2a65c3da189417fb843fdbadd5c50d2705d348909a4`
- Control weights SHA256: `ce0fcd07171aecf9968d139af8a7ff4dd9f7a3e5cdeda6572a3b04fa3548e5fe`
Checker output:
```
OK edit: 39 modules changed (of 39 present), r=1, max |dW|_F/|W|_F 0.09659 at 38.gate_proj
OK ctrl: 39 modules changed (of 39 present), r=1, max |dW|_F/|W|_F 0.09659 at 38.gate_proj
```
### edit: causal decision-point pre-test (offline `g31.G31`, verdict thresholds NOT VERIFIED YET in `TOOLS.md`)
Held-out check tasks (`probe_b001.jsonl` + exact per-call contexts, unseen during fitting): `99` decision points; verdict `ACTIVE` (mean KL 0.0002 (max 0.003), top-1 changed at 0.0% of points, mean d_logp(next) -0.0002).
| kind | n | mean d_logp(logged) | p(logged) base -> arm | KL(base to arm) | max KL | top-1 changed |
|---|---:|---:|---:|---:|---:|---:|
| all | 99 | -0.00022 | 0.9789 -> 0.9787 | 0.00023 | 0.0030 | 0.0000 |
| turn | 20 | -0.00001 | 0.9728 -> 0.9728 | 0.00022 | 0.0008 | 0.0000 |
| step | 15 | -0.00124 | 0.9586 -> 0.9575 | 0.00030 | 0.0015 | 0.0000 |
| think | 32 | -0.00158 | 0.9852 -> 0.9838 | 0.00025 | 0.0027 | 0.0000 |
| tool | 32 | +0.00151 | 0.9860 -> 0.9873 | 0.00019 | 0.0030 | 0.0000 |
Tool-token probability mass (`tool` points): `call:run` 0.444->0.444, `call:read` 0.408->0.408, `call:edit` 0.094->0.094, `call:write` 0.023->0.024, `call:get` 0.000->0.000, `call:submit` 0.031->0.031.
### edit: exact per-call `closeprobe` thought-end pre-test (`closeprobe.py`, NOT VERIFIED YET in `TOOLS.md`)
Exact generation contexts (`prompt_token_ids + completion_token_ids`, never re-rendered history prompts) on held-out check tasks:
| kind | depth band | n | base log p(close) | arm log p(close) | delta log p(close) | top=close base -> arm | d_logp(logged) |
|---|---|---:|---:|---:|---:|---:|---:|
| close | all | 16 | -0.259 | -0.259 | -0.000 | 0.938 -> 0.938 | -0.0003 |
| close | depth<200 | 8 | -0.327 | -0.329 | -0.002 | 0.875 -> 0.875 | -0.0016 |
| close | 200..799 | 5 | -0.076 | -0.075 | +0.001 | 1.000 -> 1.000 | +0.0008 |
| close | 800+ | 3 | -0.383 | -0.382 | +0.001 | 1.000 -> 1.000 | +0.0013 |
| cont | all | 92 | -19.372 | -19.380 | -0.008 | 0.000 -> 0.000 | +0.0003 |
| cont | depth<200 | 58 | -19.077 | -19.071 | +0.006 | 0.000 -> 0.000 | -0.0002 |
| cont | 200..799 | 31 | -20.159 | -20.192 | -0.033 | 0.000 -> 0.000 | -0.0002 |
| cont | 800+ | 3 | -16.928 | -16.955 | -0.028 | 0.000 -> 0.000 | +0.0141 |
### edit: held-out exact tool-selection & collateral-specificity pre-test (offline `g31.G31`, NOT VERIFIED YET against rollouts)
| held-out stratum | n | KL(base to arm) | top-1 flip | d_logp(edit) | d_logp(read) | d_logp(submit) | d_margin(edit-read) | p(edit)>p(read) base -> arm |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| pos_late_pre | 5 | 0.00061 | 0.0000 | -1.0426 | -0.0801 | +0.2558 | -0.9625 | 0.000 -> 0.000 |
| neg_early_pre | 6 | 0.00026 | 0.0000 | -0.0077 | +0.5825 | +0.3829 | -0.5902 | 0.333 -> 0.333 |
| neg_post_edit | 5 | 0.00013 | 0.0000 | -0.2816 | +0.3246 | +0.2176 | -0.6062 | 0.400 -> 0.400 |
| neg_arg (tool args) | 31 | 0.00026 | 0.0000 | - | - | - | - | - |
### ctrl: causal decision-point pre-test (offline `g31.G31`, verdict thresholds NOT VERIFIED YET in `TOOLS.md`)
Held-out check tasks (`probe_b001.jsonl` + exact per-call contexts, unseen during fitting): `99` decision points; verdict `ACTIVE` (mean KL 0.0002 (max 0.003), top-1 changed at 0.0% of points, mean d_logp(next) -0.0007).
| kind | n | mean d_logp(logged) | p(logged) base -> arm | KL(base to arm) | max KL | top-1 changed |
|---|---:|---:|---:|---:|---:|---:|
| all | 99 | -0.00068 | 0.9789 -> 0.9783 | 0.00021 | 0.0027 | 0.0000 |
| turn | 20 | -0.00039 | 0.9728 -> 0.9724 | 0.00025 | 0.0011 | 0.0000 |
| step | 15 | -0.00517 | 0.9586 -> 0.9542 | 0.00050 | 0.0027 | 0.0000 |
| think | 32 | -0.00082 | 0.9852 -> 0.9845 | 0.00020 | 0.0022 | 0.0000 |
| tool | 32 | +0.00140 | 0.9860 -> 0.9871 | 0.00007 | 0.0015 | 0.0000 |
Tool-token probability mass (`tool` points): `call:run` 0.444->0.444, `call:read` 0.408->0.407, `call:edit` 0.094->0.094, `call:write` 0.023->0.024, `call:get` 0.000->0.000, `call:submit` 0.031->0.031.
### ctrl: exact per-call `closeprobe` thought-end pre-test (`closeprobe.py`, NOT VERIFIED YET in `TOOLS.md`)
Exact generation contexts (`prompt_token_ids + completion_token_ids`, never re-rendered history prompts) on held-out check tasks:
| kind | depth band | n | base log p(close) | arm log p(close) | delta log p(close) | top=close base -> arm | d_logp(logged) |
|---|---|---:|---:|---:|---:|---:|---:|
| close | all | 16 | -0.259 | -0.274 | -0.014 | 0.938 -> 0.938 | -0.0144 |
| close | depth<200 | 8 | -0.327 | -0.345 | -0.018 | 0.875 -> 0.875 | -0.0180 |
| close | 200..799 | 5 | -0.076 | -0.075 | +0.001 | 1.000 -> 1.000 | +0.0006 |
| close | 800+ | 3 | -0.383 | -0.413 | -0.030 | 1.000 -> 1.000 | -0.0296 |
| cont | all | 92 | -19.372 | -19.360 | +0.012 | 0.000 -> 0.000 | +0.0004 |
| cont | depth<200 | 58 | -19.077 | -19.057 | +0.020 | 0.000 -> 0.000 | -0.0001 |
| cont | 200..799 | 31 | -20.159 | -20.162 | -0.002 | 0.000 -> 0.000 | +0.0012 |
| cont | 800+ | 3 | -16.928 | -16.926 | +0.002 | 0.000 -> 0.000 | +0.0017 |
### ctrl: held-out exact tool-selection & collateral-specificity pre-test (offline `g31.G31`, NOT VERIFIED YET against rollouts)
| held-out stratum | n | KL(base to arm) | top-1 flip | d_logp(edit) | d_logp(read) | d_logp(submit) | d_margin(edit-read) | p(edit)>p(read) base -> arm |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| pos_late_pre | 5 | 0.00001 | 0.0000 | -0.0079 | -0.0173 | +0.0327 | +0.0094 | 0.000 -> 0.000 |
| neg_early_pre | 6 | 0.00025 | 0.0000 | -0.0200 | -0.1095 | -0.0064 | +0.0895 | 0.333 -> 0.333 |
| neg_post_edit | 5 | 0.00000 | 0.0000 | -0.0500 | -0.0250 | -0.0430 | -0.0250 | 0.400 -> 0.400 |
| neg_arg (tool args) | 31 | 0.00005 | 0.0000 | - | - | - | - | - |
Runner facts (2026-11-12 23:52:41 EST): source `wsp31/authors/gemini/queue/late-preedit-commit-gate/`; served copy `wsp31/queue/frozen/gemini__late-preedit-commit-gate/`; sha256 over files, edit `ef389b25347bad41d30815980754ab697a2489408796d37a88367d3457c6ed62`, control `d0145a1db3e72233ceaf00d015a71e91b86a63363c814e685a3617e50135becb`; both passed the checker.Prediction (written before the run)
predicted_gain_120: -3 to 10 Behaviour that should move (stage 1, 120 screening tasks vs same-batch champion): - `read_file` calls per episode on multi-read tasks: down by 1.0 to 2.5 calls (shifting marginal calls 9..25 from repetitive `read_file` slices to `edit_file`); - share of episodes reaching `tool_calls == 30` or submitting with an empty patch (`patch_chars == 0`): down by 20% to 45% relative to reference; - mean call index of first `edit_file` among episodes with >=8 calls: earlier by 1.5 to 3.5 calls, while episodes that already edit on calls 1..7 and inside-argument code tokens remain essentially unperturbed. Matched control (`ctrl/`): fires only at the same late pre-edit tool colons with an orthogonal write, leaving `edit - read` log-odds within +/-0.15 nats, empty-patch rate within +/-3 percentage points, and solved delta within `-4 to +4`. What would prove the hypothesis wrong: (1) episodes where the gate triggers (`call >= 8`, no prior edit) emit malformed `edit_file` arguments or higher `FileEditError` rates that lower solve rate by >=4 tasks vs reference (showing the model lacked enough file context when nudged to edit); or (2) empty-patch / 30-call exhaustion count moves by less than 2 episodes out of 120.
Result
Pending.
Verdict and notes
Parked 2026-11-13 09:12:02 EST, not screened: rounds began on 13 Nov 2026 and each author keeps two waiting entries. The author can put it back in a later round (an empty file `RESUBMIT` in `wsp31/authors/gemini/queue/late-preedit-commit-gate/`); it then counts as one of that round's entries and is served from the frozen copy, unchanged.