Heretic changes language model behavior without training, refusal removal by default
An AGPL-3.0 Python tool pairs ablation modifiers with an Optuna optimizer to change model behavior such as refusals, slop or sycophancy without expensive training.
What it does
Heretic is a Python tool for modifying, rather than merely running, language models. It does not perform expensive training; according to the README it uses a collection of mathematical techniques to steer behavior based on contrasting example prompts and matching scorers. Its default action is refusal suppression, which the project calls censorship removal. Other bundled configurations reduce slop (config.noslop.toml), humor (config.nohumor.toml) or sycophancy (config.nosyco.toml), or alter emotional alignment (config.emotions.toml).
The README says Heretic supports most dense models, including many multimodal models, several MoE architectures and many modern hybrid models. Optional on-the-fly 4-bit quantization with bitsandbytes (set the quantization option to bnb_4bit) can reduce the VRAM needed to roughly what a typical quantized version of the model uses, at a slight accuracy cost.
The repository is licensed under AGPL-3.0 and was created on 2025-09-21. The GitHub trending snapshot showed 38682 stars, 699 of them gained in the last day, and the README says the community has published thousands of models made with it on Hugging Face.
Running it
The README asks for a Python 3.10+ environment with PyTorch 2.6+ and then:
pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507
Configuration is read from config.toml in the working directory, or from the path in the HERETIC_CONFIG_FILE environment variable. Configuration files made for Heretic 1.x are not compatible with Heretic 2.x, and a migration guide lists the changes. At startup the tool benchmarks the system to pick a batch size. The README gives about 30-40 minutes for Qwen3-4B-Instruct-2507 on an RTX 3090 with the default configuration. Afterwards the user can save the model, upload it to Hugging Face, chat with it, run standard benchmarks on it, or any combination of those.
Scorers, modifiers and the optimizer
Heretic steers a model by optimizing one or more scores computed by scorer plugins. By default it scores refusal rate and KL divergence. Any number of scorers can be co-optimized, each set to be minimized or maximized, and a custom scorer only has to implement get_score from the Scorer base class, without changes to the package code.
The parameters come from a TPE-based hyperparameter optimizer powered by Optuna, which feeds a pluggable modifier. Two modifiers ship with the tool: Arbitrary-Rank Ablation (ARA), the default, developed specifically for Heretic, and directional ablation, also known as abliteration, with support for projected abliteration and norm preservation (MPOA). Only a single modifier can be used per processing run.
Reproducible models
When uploading to Hugging Face, Heretic can include a reproduce folder with instructions and a reproduce.json manifest of around 9 kilobytes. Running heretic --reproduce path/to/reproduce.json recreates the model without redoing the original multi-hour computation, checks the environment for mismatches and compares the hashes of the weight files; whether they match depends on how closely the system resembles the original. The collection of such manifests is called the Heretic Grimoire, and heretic --collect-reproducibles path/to/target/folder/ keeps a local, append-only copy.
Source: p-e-w/heretic on GitHub