Uncensored and Abliterated Models: What Changes

These models are a large and visible part of the open-weight ecosystem, and the interesting question about them is not the political one. It is that a modification aimed at one narrow behaviour reaches other behaviours too, and the model cards almost never say which.

What the words mean

Three distinct techniques travel under overlapping labels, and they have different effects and different failure modes.

• Fine-tuned to comply. Additional training on a dataset of compliant responses. Changes many weights, can teach new behaviour as well as remove old, and inherits every risk of any community fine-tune — including whatever else was in the dataset.
• Abliterated or orthogonalised. A surgical weight edit that removes the model’s ability to express refusal, without further training. Very cheap to produce, which is why there are so many of these.
• A base model with no instruction tuning at all. Not “uncensored” so much as un-aligned in every direction: it also does not follow instructions, does not hold a chat format, and completes text rather than answering. Frequently mislabelled.

It is also worth being precise about what none of them do. Removing a refusal does not add knowledge. A model that now answers a question it used to decline is not thereby more likely to answer it correctly — it has lost the ability to say no, not gained the ability to say something true.

How abliteration works

The technique rests on an empirical finding about how refusal is represented. Run a set of harmful prompts and a set of harmless ones through the model, take the mean hidden-state activation for each set at some layer, and subtract. The resulting vector — a “refusal direction” — turns out to be strikingly consistent: adding it to activations induces refusal on ordinary requests, and removing its component makes refusals stop.

Abliteration makes that removal permanent by editing the weights. Every matrix that writes into the residual stream is projected so that it can no longer output any component along that direction:

r = normalise(mean_act(harmful) - mean_act(harmless))   # refusal direction

for W in every matrix writing to the residual stream:
    W <- W - r (r^T W)        # strip the component along r

# no training, no gradients, no data beyond the two prompt sets.
# the result is an ordinary checkpoint that cannot express refusal.

Two things follow. It is remarkably cheap — minutes on modest hardware, no training run — which explains the volume of these uploads. And it is blunt: the direction was estimated from two prompt sets, and whatever else happens to lie along it is removed as well.

What it costs beyond the refusals

This is the section that is missing from the model cards. The edit removes a direction in activation space, and directions in these models are not cleanly labelled with one behaviour each.

• Caution goes with the refusals. Hedging, uncertainty expressions, safety caveats and “I am not sure” are often entangled with the same representation. A model that never declines frequently also never doubts, which is worse for accuracy than the refusal behaviour ever was.
• Instruction adherence can degrade. A model tuned to follow instructions was tuned to follow all of them, including the ones that produced refusals. Editing that capability affects the general one.
• Compliance is not selective. The model also loses the ability to refuse a badly framed, mistaken or impossible request. It will attempt what it should have pushed back on, which in an agentic setting means acting on an instruction that should have been questioned.
• Fine-tuned variants carry their dataset. Where the technique was training rather than editing, everything else in that dataset came along — style, biases, format habits, and any quality problems in it.
• Quantised repackagings compound. Most of these circulate as GGUF quantisations of a modification of a fine-tune of a base. Each step is a place where quality was lost and nobody measured it.

A paired test worth running

The whole question is empirical and the test is the same paired design used elsewhere in this cluster: base versus modified, identical prompts, identical settings, differences attributable to the modification alone.

• Use the exact base the modification was made from, at the same quantisation. Comparing an abliterated four-bit build against a sixteen-bit base measures two changes at once.
• Temperature 0, same seed, same chat template for both sides.
• Three prompt sets. Your real workload; a general instruction-following set; and a calibration set of questions with known answers where the model should express uncertainty. The third is where the collateral damage shows, and it is the one people omit.
• Score mechanically first. Format validity, task success, and how often the model asserts a confident answer to something unanswerable.
• Read the difference, not the modified output alone. An output that looks fine in isolation may still be worse than what the base produced for the same prompt.

If the modified model is materially worse on your real work, the honest conclusion is that the refusals were not the binding constraint — which is a common result, because prompt framing and a good system prompt resolve a great many refusals without touching weights.

Legitimate uses, and what you still own

There are real reasons to want a model that does not decline. Security research and red-teaming, where the model must discuss attacks to help defend against them. Fiction with dark subject matter. Clinical, legal and harm-reduction contexts where a general-purpose refusal policy misfires against a professional need. Content moderation research, which requires generating the thing being moderated. And a genuinely frequent, mundane case: false positives on non-English or domain-specific text where the refusal was simply a mistake.

What does not change is where responsibility sits. You are the operator of whatever you deploy, and the output is yours. The base model’s licence and any acceptable-use policy it incorporates continue to bind you — modifying weights does not remove a licence term, and several licence families attach use policies precisely so that they survive derivation. If you serve a modified model to other people, you have taken on the entire content-policy problem that the original tuning was doing for you, and you now need your own filtering, logging and abuse handling. That is a real engineering cost, and it belongs in the decision.