skulldc

SteerDistill lets a full-precision teacher correct a quantized agent while it works

SteerDistill is a quantization-aware distillation method in which a full-precision teacher steers a quantized student agent through its failed turns, and the authors report results within one point of BF16.

The problem

Low-bit quantization makes LLM agents cheaper to run, but it can cost task success. SteerDistill, a paper published on arXiv on 2026-11-10 and listed on Hugging Face daily papers on 2026-11-09, starts from a close look at where quantized agents go wrong.

According to the abstract, the authors examined failed rollouts and found that quantized agents overlook task requirements or environmental conditions. The result is a sequence of environment transitions that drifts away from the successful runs of the full-precision model used as a reference.

Fixing that drift is hard for two reasons the paper names. Offline training data may not contain the situations where the quantized student actually fails. The student’s own rollouts do contain those situations, but in them it may produce few correct responses to learn from.

How the method works

SteerDistill is a quantization-aware distillation method built around a full-precision teacher that stays in the loop. When the student produces a failed response, the teacher guides it to revise that response. The teacher then refines its guidance iteratively, using feedback from how the student’s revised actions play out in execution.

The authors call this closed-loop teacher steering. Its output is a set of behavioral corrections that are verified locally, in the contexts where the student actually failed, rather than taken from a fixed dataset.

The guidance cannot be present at inference time, so the corrections have to be internalized. The student is trained without guidance to match two targets: the steered student’s predictions at the turns that were corrected, and the full-precision teacher’s predictions at all other turns. After training, the quantized agent runs unassisted.

Reported results

All figures below are the authors’ own, as given in the abstract.

The evaluation covers five agentic benchmarks and three models. Under W4A16 quantization, SteerDistill exceeds the strongest quantized baseline by 1.67 to 2.18 points in average score, and it stays within one point of BF16.

On τ^2-Bench the authors compare directly against QeRL. SteerDistill outperforms it by 1.84 percentage points in success rate while using 67% fewer training GPU hours. The abstract also reports that SteerDistill produces shorter successful trajectories than the quantized baselines.

What the abstract leaves open

The abstract does not name the three models or list the five benchmarks, and it does not give absolute scores, so the size of the remaining gap to BF16 on each task cannot be read from it. W4A16 is the only quantization setting it mentions. It also does not say how many steering iterations the teacher needs per failure or what the teacher’s share of the training cost is. The authors list a code repository for SteerDistill on GitHub.


Source: SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering (arXiv 2611.13410)