Tree-Ising repairs a looping language model by flipping bits in its BF16 weights
Tree-Ising turns a gradient-free model repair into per-row QUBO problems, and the authors report it beats an LLM agent's edit on MATH-500 using four forward-pass requests.
The problem
Weight-space programming, introduced in recent MIT work, lets a frontier LLM agent read a network’s weights and write a repair directly, without gradients. In that work, the GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B. According to the Tree-Ising abstract, the agent needed 158 agent turns, 89.7 million tokens and 50 forward-pass requests to get there.
Tree-Ising, published on arXiv on 2026-11-10 and listed on Hugging Face daily papers on 2026-11-09, asks whether the same kind of repair can be done as an optimization problem instead of an agent session.
How the method works
Tree-Ising casts the repair as a sparse discrete optimization problem over the model’s own BF16 weights. Forward statistics define the objective, which has three parts: weaken a loop direction, strengthen a concluding direction, and preserve the layer’s output on productive tokens.
The allowed edits are deliberately narrow. The method works within a state-tree representation, in which lossless BF16 storage acts as a special case, and the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. The repair is therefore a set of bit-level changes to weights the model already has, kept inside its storage format.
The authors decompose this objective exactly into independent QUBO (Ising) problems, one per output row. Each of those problems is solved on a GPU, using either simulated annealing or quantum-inspired simulated bifurcation.
Reported results
All figures below are the authors’ own, as given in the abstract.
On the agent’s two layers, Tree-Ising reaches 54.6% on MATH-500, against 51.7% for the agent’s edit under the same evaluation protocol. The base model scores 7.9% as reported. The simulated bifurcation solver gives a repair at 55.1%, which the authors describe as statistically indistinguishable. Both solvers outperform the agent edit across all five math benchmarks, and the improvements on MATH-500 and GSM8K are statistically significant after correcting for multiple comparisons.
The edit is also smaller. Tree-Ising modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. The cost gap is the largest difference: the method requires only four forward-pass requests and zero LLM tokens, and the annealing recipe costs about $0.90 in GPU time.
The authors also apply the recipe to Qwen3.5-2B, with layers and dose selected using only the base model. There, greedy GSM8K accuracy goes from 21.6% to 69.4%. As a proof of concept, they execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.
What the abstract leaves open
The abstract names MATH-500 and GSM8K but not the remaining math benchmarks. It does not define “dose”, give wall-clock solve times, or say whether the repaired models were evaluated on anything outside math. The Qwen3.5-2B result is reported without an agent edit to compare against, so the head-to-head comparison rests on the smaller model only. The QAOA run is described as a proof of concept on a subproblem, not as a full repair. The authors list a code repository for Tree-Ising on GitHub.
Source: Tree-Ising: Repairing Language Models with Bit-Level Ising Edits (arXiv 2611.14210)