GEMMA DEVELOPER AGENTSAVED EVIDENCE

← Weight change

Direct mathematical edit / wsp31-0036

Source-grounding instruction contrast across all 60 layers at dose 1

Gemma 4 31B, frozen four-bit baseAuthor: gptState: doneVerdict: null

When

Recorded date: 2026-11-14

Ledger date only; exact weight-build/change timestamp is not recorded here. Not a live training event.

Exact tensor-change timestamp: unavailable.

Why change it?

The last two code-style writer edits, 0025 and 0026, were null (-1 and 0 tasks against the reference mean). Close-writer amplification in 0011 and 0012 harmed syntax. This entry instead tests whether the model can use current source text and tool errors more reliably when its own response to a source-grounding instruction is compiled into its weights. In the 208 b007 reference episodes from the 104 tasks shared by both stage-1 lists, 333 edit calls had 244 successful replies, 32 error replies and 57 missing replies. Among known replies the edit-error rate was 32/276. Missing replies were not treated as successes. A borgbackup trace…

Which part changed?

Source-grounding instruction contrast across all 60 layers at dose 1

The source describes adapter targets and mathematical changes. No before/after tensor comparison is available in this viewer.

How was it changed?

Computed directly by author-written code and packaged as a low-rank adapter over the frozen base; not gradient-trained. base: champion generation 0 size: whole network 0.653101% over 180 matrices, largest matrix 1.586776% control size: whole network 0.653100% over 180 matrices, largest matrix 1.586776% Rank 1 in every changed matrix, alpha 2, bf16 factors. The 180 changed matrices are gate_proj, up_proj and down_proj at layers 0 through 59. No champion edit existed at design time. The adapter contains only this change. If the runner rebases it after a promotion, overlap with another constant-neuron edit could interfere; no additive benefit is assumed. This entry is scratch candidate G10, beta = 1. The size includes the replacement of 60 gate/up rows. It is…

Did it improve?

null · Stage 1: -3.50 (x003) tasks against reference arms. Stage 2: not yet measured. Matched control, stage 2: not yet measured. Verdict: null.

No reliable improvement: recorded verdict is null.

Evaluation set: wsp31_screen_v2 · Batch: x003. Local screening and historical task scores are not competition scores unless explicitly identified in the source. Different evaluation sets cannot be compared as a gain.

Original evidence

Literal source excerpts follow. They may contain original predictions and superseded counts; use the measured-outcome summary and corrections above for the current result.

Hypothesis
The last two code-style writer edits, 0025 and 0026, were null (-1 and 0 tasks against the reference mean). Close-writer amplification in 0011 and 0012 harmed syntax. This entry instead tests whether the model can use current source text and tool errors more reliably when its own response to a source-grounding instruction is compiled into its weights.

In the 208 b007 reference episodes from the 104 tasks shared by both stage-1 lists, 333 edit calls had 244 successful replies, 32 error replies and 57 missing replies. Among known replies the edit-error rate was 32/276. Missing replies were not treated as successes. A borgbackup trace reconstructed a function in its thought, failed to match old_string, and then read the file again. Exact repeated failed calls were rare: 9 immediately repeated calls after 488 error replies. Preventing bad edits is therefore the main proposed benefit; retry suppression alone has a small ceiling. Audit code and counts: code/turn7/recovery_audit.py, repeat_audit.py and scratch/turn7/*audit.json.

The contrast is the model's response to the source-specific paragraph minus its response to a generic coding-task paragraph. It uses no solve labels, thought-close direction, depth labels or code-versus-prose contrast. Subtracting the generic paragraph removes some generic instruction effects; it does not prove that the remaining direction represents only source fidelity.

Prior E4B prompt-rule compilation was null. This test differs in model size, the source/error-specific paragraph, the generic-paragraph subtraction, incremental per-layer shifts, calibrated constant reads, and checks on later calls. Claude's 0031/0032 use the related constant-neuron compiler for brevity; this entry changes the instruction, contrast, fit captures and control. These differences justify a test, not an expected confirmed gain.
Exact recipe
base: champion generation 0

size: whole network 0.653101% over 180 matrices, largest matrix 1.586776%

control size: whole network 0.653100% over 180 matrices, largest matrix 1.586776%

Rank 1 in every changed matrix, alpha 2, bf16 factors. The 180 changed matrices are gate_proj, up_proj and down_proj at layers 0 through 59. No champion edit existed at design time. The adapter contains only this change. If the runner rebases it after a promotion, overlap with another constant-neuron edit could interfere; no additive benefit is assumed.

This entry is scratch candidate G10, beta = 1. The size includes the replacement of 60 gate/up rows. It is not a direct measure of the functional write. Stored squared-update norm shares by matrix type: {"down_proj": 0.0883287549580875, "gate_proj": 0.5639568923327042, "up_proj": 0.34771435270920753}. Raw new write relative to the natural MLP output, min/median/max: 0.00258, 0.01084, 0.07778. Do not equate this geometry with a trained update.

Data: exact per-call token sequences in tools/data/probe_calls_b001_full.jsonl. Keep tasks in both stage1_ids.txt and stage1_ids_v2.txt and in neither stage-2 list; calls after the first, with at least 24 completion tokens, a closed thought and at most 4600 total tokens. One middle call per task; sort task names, reserve indices with index modulo 4 equal to 1 for checks, shuffle the rest with seed 6. CPU fallback: sort each side of that split by total token length, take the shortest six fit calls and three check calls, and stop captures after the 1200-second capture budget (finishing the call in progress). The actual 6 are listed below. Sample at most 160 evenly spaced completion positions, 48 of them for the constant-read fit, plus 40 prompt positions and BOS. No task outcome enters selection or fitting.

Fit episodes: python-markdown__markdown-743 (aa, call 14); r1chardj0n3s__parse.30da9e4f.func_basic__83wal9im (zero, call 17); django-money__django-money.835c1ab8.lm_rewrite__v5cndhkm (aa, call 1); rubik__radon.54b88e58.func_basic__sn48uojb (base, call 1); jd__tenacity.0d40e76f.func_basic__85q611gl (aa, call 21); rasbt__mlxtend-1087 (zero, call 22).

Check pool, excluded by task from fitting: mewwts__addict.75284f95.combine_file__3fttj8ti (call 1); r1chardj0n3s__parse.30da9e4f.func_pm_remove_cond__k4df18dk (call 1); ipython__traitlets-517 (call 28). The table gives the number actually evaluated for each arm. Checks were reused to select dose, so they are not an independent performance test. Stage 1 remains design data; stage 2 is the first unseen-task outcome test.

Positive paragraph (I1):
> Important: ground every change in current source text and observed tool results. Read the relevant implementation before editing. Copy old_string exactly from the latest file contents. After a tool error, use its message to correct the next action instead of repeating the failed call. Check the resulting diff against the requested behavior before submitting.

Generic paragraph (I0):
> Important: work on the user's coding task with the tools provided in this session. The repository contains source files and supporting files. Tool calls take arguments and return results. Follow the task description while deciding what action to take next. Continue working on the task using the available context and provide your final response when finished.

Insert a blank line and the paragraph at the end of the first user message, before its turn-end token. The insertions contain 64 and 65 tokens before padding; append one newline token 107 to I1 so both have 65 tokens. Run the same logged continuation with each insertion and without either insertion. The runtime harness prompt is unchanged.

For each layer L, d_L is the task-weighted mean of resid_post(I1) minus resid_post(I0) at matched continuation positions. Set delta_L = d_L - layer_scalar_L * d_(L-1), with d_(-1)=0. Split-half direction cosine, min/median/max over layers: -0.001, 0.478, 0.914.

Choose the neuron with the third-smallest mean absolute activation in each layer on these captures. Fit w so w dot mlp_in is about 1: compare the mean-input read with a dual ridge solve (regularizer N times the mean squared input coordinate); use ridge only if its standard deviation on calibration calls is below 0.7 of the mean read's and its calibration mean is between 0.9 and 1.1. Calibration calls have capture index modulo 3 equal to zero; the others fit w. If the absolute BOS score exceeds 3, remove that BOS direction and rescale. Set the gate row to 8w and the up row to w. The new activation is approximately 8(w dot x)^2, not a true constant. Its calibration coefficient of variation has min/median/max 0.057, 0.116, 0.582.

The new down column is clipdiv(beta * delta_L, post_feedforward_norm_weight) times mean_raw_MLP_RMS / (mean_new_activation * layer_scalar_L). clipdiv zeroes the lowest 5% of absolute post-norm weights before inversion. Save row/column differences against the unedited weights. This is a first-order write approximation: it ignores the new column's change to RMS and the response of later layers. It also averages away the position-dependent part of the instruction and does not reproduce its attention patterns. It acts at prompt positions too. Bf16 factors add rounding.

Control: identical gate/up changes, neuron, layers, rank and dose. First flip the signs of the intended NEW down column c with seed 710000 + L. Let u be the unit old column, a = (c dot u)u, and z be the randomized column projected off u. Set the new control column to a + z * norm(c-a)/norm(z), then subtract the old column to form the update. This preserves the norm of the new write and its projection onto the old column, so the total column-update norm also matches before rounding. The remaining direction is random. The largest stored per-matrix norm mismatch is 0.0007. Stored new-column raw-norm ratio (control/edit), min/median/max: [0.9987460364303841, 1.0000832447387193, 1.0024976224637483]. The corresponding post-norm-weighted ratio: [0.9985130507274579, 1.000219579087168, 1.0336821808790708]. This controls update direction while holding the read edits fixed. An initial whole-update sign control was discarded before edited replay because it could leave a large old-column write under the new gate; CONTROL_AMENDMENT.md records the correction. Those discarded controls are not this entry's ctrl/.

Rebuild: bash build.sh OUT_DIR invokes code/turn7/build_ground.py from frozen code/turn7/ground_stats.pt. ground_cpu.py captured and solved the statistics on CPU with 16 threads and dense-weight caching. ground_job.py supplies the compiler used for rebuilds. Both use a frozen copy of the documented tools in code/turn7/tools. The compiler adapts claude's turn-6 icv_job.py; ground_job.py and the small thinklib.py contain the complete changed code path. No optimizer, gradients applied to weights, trained-adapter direction, or new rollout was used.

Both official check_adapter.py runs passed. train_size.py supplied the size lines. A separate rebuild matched adapter_config.json, adapter_model.safetensors and edit.json byte for byte in both arms; the saved weight bytes also match the scratch arms used in the pre-tests. See rebuild_receipt.json and source_hashes.json. Statistics SHA256: 20bf2b231c9e84420c73c63346685b6e3a3436076f59b85717299d48e94a0ed8.

### Teacher-forced pre-tests

Capture device: CPU. Replay device: cpu. Documented g31 forward pass, mirror=True, exact prefixes. The GPU forward pass is verified against serving. The CPU backend is NOT VERIFIED against an edited served model; baseline agreement below does not establish edited agreement. This new capture/evaluation script is not independently verified. The dose choice is adaptive. Counts are top-token choices on logged paths, not generated episode outcomes. The unvalidated interp/flipcount.sh was read but not used; its validation failure makes it unsuitable as sole evidence. We count flips directly from full-vocabulary forward-pass log probabilities.

| Arm | Calls / positions | Top-token changes | KL from base | Change in log p(close) at true closes | Closes kept | Wrong delimiter choices |
|---|---:|---:|---:|---:|---:|---:|
| P_I1 | 2 / 153 | 3 | 0.005449 | -0.23401 | 2/2 | 0/4 |
| P_I0 | 2 / 153 | 3 | 0.004589 | -0.21125 | 2/2 | 0/4 |
| G10 | 2 / 153 | 1 | 0.000804 | -0.00180 | 2/2 | 0/4 |
| G30 | 2 / 153 | 1 | 0.005292 | -0.00670 | 2/2 | 0/4 |
| G90 | 2 / 153 | 7 | 0.048505 | -0.79879 | 1/2 | 0/4 |
| G10c | 2 / 153 | 1 | 0.000474 | +0.00084 | 2/2 | 0/4 |
| G30c | 2 / 153 | 3 | 0.002122 | +0.00197 | 2/2 | 0/4 |
| G90c | 2 / 153 | 4 | 0.015557 | +0.00176 | 2/2 | 0/4 |

For this entry, agreement between the unedited forward pass and the logged next token was 99.3464%. The edit changes a top token on 1 check tasks. Positive-prompt log-prob shift correlation is 0.1512, versus 0.1446 for control; the fraction of KL to the positive-prompt distribution removed is -0.9503, versus -1.1611. That comparison is to I1 alone; the fitted direction is I1 minus I0, so it is not an exact reconstruction target. Negative KL-reconstruction scores mean the adapter moved farther from I1 in this metric. Full per-arm counts and limits: eval_report.json.

### Independent cheap check and generation limits

The official tools/closeprobe.py was run with --device cpu on a named exact check context, outside the fit tasks. Its CPU edited-serving agreement is NOT VERIFIED. G10: 2 points, change in log p(close) at true close -2.1456871763803065e-06, continuation -0.0628976821899414. G10c: 2 points, change in log p(close) at true close -5.006600986234844e-06, continuation 0.000186920166015625. Full outputs are in the official_close JSON files and logs. This is an implementation cross-check on one context, not an independent outcome test.

Served generation was NOT RUN: GPU 0 remained locked by another job. GPU 1 was not used. Consequently there is no measured free-generation syntax rate or long-loop rate, and no verdict from toolcall_check.py on generated calls. Logged-path delimiter choices are in the table; they cannot rule out failures after the edit changes the path. No task rollout or tool execution was launched.

Selection and any declined doses:

Tested write doses were 1, 3 and 9. Doses 1 and 3 each changed 1/153 sampled top tokens on two reserved tasks, with zero early-close top choices, 2/2 logged closes retained, and 0/4 wrong sampled delimiters. Their controls changed 1/153 and 3/153 tokens. The dose-3 control had one early-close top choice, despite retaining both logged closes; this is a control-side behavior difference, not proof of source grounding.

Dose 9 changed 7/153 tokens but retained only 1/2 logged closes, versus 2/2 for its control. It was declined before READY. Its two-position official close probe on the first task passed, so that narrow check did not detect the failure on the second task. The declined draft and receipts are under scratch/turn7/declined/source-grounding-icv-b9, outside queue/.

Select dose 3 as the largest tested dose that preserves both sampled closes, and dose 1 as its one-third write-dose comparison. Gate/up replacements are held fixed, so their whole-network norms do not differ threefold. Both edits changed only one sampled token: this is weak behavioral reach, not demonstrated source fidelity. Their prompt-KL reconstruction scores were negative, and dose 3's prompt-shift correlation was barely above control. The pre-tests therefore do not establish instruction reproduction or task benefit.

The task split, fit captures and randomized controls were fixed before edited replay. The initial whole-update sign control was corrected before any edited replay; only corrected controls appear in these results. Both selected doses and controls completed the documented CPU closeprobe. No larger dose than 9 was tested. The optional third task replay was stopped after the two shared checks completed to release CPU capacity; no third-task res

Runner facts (2026-11-14 00:18:26 EST): source `wsp31/authors/gpt/queue/source-grounding-icv-b1/`; served copy `wsp31/queue/frozen/gpt__source-grounding-icv-b1/`; sha256 over files, edit `21e94f8fbbee4c444609021cb75093a4218ce958978c731abd7f60b0210c1f49`, control `f9ba919fcdfaa250d7c87872505c5d4d1d31a5159784775706696fae70a1c9ee`; both passed the checker.
Prediction (written before the run)
predicted_gain_120: -8 to 10

This is an 80% interval over the champion reference in the same stage-1 batch; median 1 task. The breadth reflects prior null instruction-compilation results, weak prompt-reconstruction evidence, absent free-generation checks, and single-run noise. The live runner uses the 10000-token episode clock and zero-adapter references; old wall-clock results are not directly comparable.

Expected behavior: at least 15% lower edit-error rate among edit calls with recorded tool replies than the pooled reference rate. The audit reference is 32/276 (11.6%); the comparison must use this new batch's reference rate, not that historic value. The share of immediate identical retries after errors should not rise; its historic count is only 9/488, so it is a secondary measure. Reading before an edit may rise. No thought-length reduction is required. Syntax errors should stay below 3% of completed tool calls, and the edit should not add long repetitive thoughts.

Control: no consistent source-error reduction; gain -6 to +6 tasks and error rates within the reference-arm spread. With both doses screened, the larger dose should lower the edit-error rate more, provided syntax remains intact.

The mechanism fails if the edit-error rate falls by less than 10%, or if the control lowers it as much. If errors fall but solved tasks do not rise, the proposed solve-rate benefit fails. A solve-rate gain without the predicted error change is not support for this mechanism. New syntax failures or long loops reject the dose. Screening is the first test of task outcomes; this record claims none.
Result
**Stage 1** (batch `x003`, 4 arms on one server, concurrency 4 per arm, finished 2026-11-14 05:39:03 EST; `results/wsp31/batches/x003/`). Reference arm: unedited. Both reference arms are the unedited model served through an all-zero adapter, the same serving path as an edit (ledger record 0033). Box batch: run on rented boxes as 4 shards by task, every arm of a shard on one server; its edits are compared only with its own reference arms. Episode budget of every arm: 20,000 generated tokens, 60 tool calls, 80 turns (budget `x2`, harness `harness_x2`).

- unedited: solved 81 of 120, agent errors 21, at session limit 21, at tool-call cap 6, empty patch 8, median tool calls 21.0, mean 398.7 s, median 23.6 completion tokens/s of model time.
- edit: solved 78 of 120, agent errors 31, at session limit 31, at tool-call cap 4, empty patch 7, median tool calls 24.0, mean 423.6 s, median 24.2 completion tokens/s of model time; vs base on 120 common tasks: -3.0 tasks (95% interval -9.0 to +3.0), base-only 7, arm-only 4, exact p 0.5488.
- second unedited arm (identical weights): solved 82 against 81; 3 tasks solved only by the first, 4 only by the second (7 tasks differ; record 0003 measured 11.3 per identical pair); median 23.3 completion tokens/s. The gain used by the rule is against the mean of the two reference arms per task.
- edit against the mean of the two reference arms: -3.5 tasks of 120 (95% interval -10.0 to +3.0).
- Predicted gain -8 to +10; measured -3.5: inside the interval.
Verdict and notes
null by the fixed rule: stage-1 gain -3.5, bar +3, harmful at -6 or below.