Training a model on your rules beats putting them in the prompt

Somebody who sells a model on Weightship wants it to behave the same way for every customer. The quick way to get there is a system prompt: write the rules down and send them with every request. We wanted to know whether training the rules into the weights does better, and by how much, so we measured it on Qwen3-4B-Instruct.

With no system prompt, the trained model kept all five of our style rules on 97.5% of 120 requests it had never seen. The untrained model with the rules in its system prompt kept them on 88.3%. Then we added a customer's own system prompt in front, the way the model would actually be deployed, and the prompted model fell to 65% while the trained one stayed at 92.5%.

The test

We used two rule sets that a buyer might plausibly want. The first is a house style for an everyday assistant: start with the answer instead of "Sure!" or "Here's", no markdown, at most 120 words, no em dashes, and no closing offer such as "Let me know if you have any other questions." The second is a format: reply with a single JSON object holding an answer and a confidence, and nothing else.

Every rule is a function that checks the reply, so nothing here depends on a judge model's opinion. Training used 124 requests covering how-tos, explanations, advice, short writing tasks and code. Scores come from a different set of 120 requests on other subjects: 40 at first, and 80 more written afterwards to make sure the winners hadn't been chosen by luck. Each model also answered a 30-question quiz of arithmetic, facts and small logic puzzles, so that a model which follows rules by getting worse at answering would show up, and we measured drift as the KL divergence from the base model on ordinary human-written text. Every trained model is a rank-16 LoRA, two epochs, about fifteen minutes on a laptop GPU.

How the training data is made

None of the training data is written by hand. The base model is the teacher, so we give it the rules as a system prompt and ask it each training request four times at temperature 0.8. That model already follows the rules most of the time, which is why the prompt baseline is high, but it still slips, most often with an em dash, so we keep only the answers that pass every check, which is rejection sampling with the buyer's own checks. They are the same checks the benchmark uses, so on Weightship a project's tests also decide which of the teacher's answers it is trained on.

We also change what the student is trained to imitate. Plain distillation teaches it to match the teacher's next-token distribution, which caps it at the teacher's own rate of slips, so instead we train it to match the teacher pushed further in the direction the instruction moves it:

target = softmax( ℓ_with_rules + α · (ℓ_with_rules − ℓ_without_rules) )

Here ℓ is the model's logits for the same answer tokens with and without the rules in its context. Wherever the rules make a token less likely, an em dash for example, the target makes it less likely still, and wherever the rules leave a token alone the difference is close to zero and nothing changes. This is the same idea as classifier-free guidance at decoding time, applied once during training, so the finished model needs neither the prompt nor a second forward pass. We used α = 1, because at α = 2 the JSON format generalised further but drift on ordinary text roughly doubled and the style score didn't improve.

Why the customer's system prompt matters

On a marketplace the person who trained the model is rarely the person who deploys it. The customer writes their own system prompt, which here was a friendly assistant for an online home-goods shop, and the creator's rules either get appended after it or get lost. With the rules appended after the persona, the prompted model kept all five rules on 65% of requests, down from 90% on the same 40 without the persona, and the em dashes and sign-offs came back. The trained models lost less because there is no instruction left to dilute: the one Weightship uses went from 97.5% to 92.5%.

A rule learned on ordinary requests didn't cover unusual ones

Every trained JSON model matched the prompt on all 120 held-out requests. The quiz showed the gap: asked "What is 17 + 26? Answer with just the number", models trained on our everyday requests usually answered "43" instead of a JSON object, because nothing in their training looked like that question. Amplification raised the share in JSON from about a third to over half, and what closed the rest was letting the model write its own practice requests in the shapes our list lacked, such as one-word questions, greetings, puzzles and demands for a particular format, with anything resembling a quiz question filtered out. With 160 of those added, the format held on 100% of quiz questions, and the answers inside the JSON were still 100% correct.

What didn't help

Rewriting the output directions of 1,024 located MLP neurons, which on an earlier writing-style task captured 94% of what the instruction did, only matched the prompt here (85.8% on the style rules) and carried the JSON format to unusual questions worst of all, at higher drift than the LoRA. We also tried keeping the teacher's failed answers and using the checks to locate the exact tokens that broke a rule, then training those specific tokens down. It tied the simpler method at 117 of 120 and moved the rest of the model more, so we left it out.

Limits

This is one 4B model, two rule sets and 120 held-out requests, decoded greedily. Rules here are the kind a function can check; a rule like "be warmer with upset customers" would need a judge model in the filter, which Weightship's benchmarks support but we haven't measured. The quiz is small enough that it would catch a model that broke but could miss one that got slightly worse. A two-proportion test puts the chance of a gap as large as 97.5% against 88.3%over 120 requests below 1 in 100, and the gap under a customer's prompt is larger again.

In the product

On Weightship this is what training from a project does. You write the rules, Weightship turns them into an instruction and checks, writes extra practice requests, keeps the teacher's answers that pass, and trains a LoRA with amplified distillation. The job page shows how many answers passed and were used, and the project's benchmark runs on the result next to the base model, so the numbers above are the kind you see for your own model before you publish it.