What 1,012 edited neurons do to a sentence

A neuron edit changes how strongly some of a model's MLP neurons fire and leaves every weight alone. We made one on Qwen3-4B-Instruct, taught by a plain-writing instruction, and then measured what it actually does: whether it can be shipped as an ordinary adapter, what each changed neuron does to the text, and how much behaviour it buys.

It exports exactly

Scaling neuron n in layer l by a gain g is the same as adding (g − 1) times that neuron's output column to the layer's down projection, and that is a low-rank update: one row of a one-hot matrix A and one column of B per edited neuron. So 1,012 edited neurons become a LoRA on down_proj with rank equal to the most neurons edited in any one layer, here 49 across 36 layers. Loaded back with PEFT, the exported adapter picks the same top token as the runtime hooks at 100% of positions, and the largest logit difference between them is 0.56, against 11.0 for the edit itself. That gap is bf16 rounding, and it means a neuron edit serves on any multi-LoRA server without special support.

Where the edits sit

The edits cluster in layers 9 to 23, the same band where a steering vector for this behaviour worked best in earlier experiments, and they split almost evenly between neurons turned up (478) and turned down (534), with the lowest gain at 0.20, so no neuron was switched off or flipped.

What each change does to the text

The chart below is a trace of the 24 strongest changes across the tokens the edited model wrote for one prompt. In the default view each cell is a measured causal effect: the log-probability of that token with the full edit, minus its log-probability with only that one neuron put back to normal. Hover any cell for the numbers, pick a neuron to read what it writes, and switch prompts at the top.

Moving to a new city is one of the most exciting—and sometimes overwhelming—decisions life offers. It’s a fresh start, a chance to leave behind the familiar and step into the unknown. The first few weeks are often a blur of packing, unpacking, and navigating unfamiliar streets, public transport, or even the local grocery store. But slowly, a new rhythm begins to form.

One of the most beautiful parts of moving is discovering the city

Reading a neuron's output column straight through the vocabulary, the usual "logit lens" view of what a neuron writes, says little for most of these, because they sit in the middle of the network and act through later layers. That's why the causal view is the default, and in it no single neuron carries the behaviour. The most important change, layer 16 neuron 6440, moves the whole output distribution by about 1.2e-3 nats when undone, and per-token contributions are small next to the total change. The behaviour comes from hundreds of small pushes in the same direction.

How much behaviour it buys

Measured as AI-writing tics per thousand words on 40 held-out prompts, the base model scored 121.3 and the model given the instruction as a system prompt scored 61.9. The gain edit, taught to imitate that prompted model, reached 103.7 at a drift of 0.010, which is about 30% of the prompt's effect. A LoRA trained on rewritten examples reached 10.2, at six times the drift.

Learning a new output direction for each of the same located neurons, instead of only rescaling it, got to 65.5, which is 94% of the prompt's effect, with 2.6 million trainable parameters against 33 million for a rank-16 LoRA, at a drift of 0.017. That's the version Weightship's neuron edits now use. For teaching hard rules, though, an ordinary LoRA trained with amplified distillation did better, which is the subject of the next post.