skulldc

PLatour tests whether a small model really uses a larger model's prompt representations

A paper on prefill-to-autoregressive cross-model transfer releases an open framework and finds that some reported gains appear even when the sender's signal carries no information.

What PACT is

Prefill-to-Autoregressive Cross-model Transfer (PACT) is a form of latent communication between language models. A larger model processes the prompt, and its prompt representations are combined with a smaller model that handles generation efficiently. The appeal is a balance between capability and inference cost: the large model reads, the small model writes.

The paper On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer (arXiv 2611.06976) lists 11 authors and the organization King’s College London. It was published on arXiv on 2026-11-04 and listed on Hugging Face daily papers on the same day.

The authors identify two gaps. Existing systems offer limited support for independently evaluating large-small model collaboration, and better task performance alone does not show that the receiver benefits from the sender’s information.

The PLatour framework

To address the first gap, the authors introduce PLatour, an open-source PACT framework. It connects frozen models through lightweight, zero-initialized linear bridges that are trained directly with task rewards. It also supports hybrid-attention architectures.

According to the abstract, PLatour significantly outperforms open-source baselines on generation tasks. On shared mathematics and coding benchmarks it surpasses Mostik’s reported results while using a substantially smaller sender. These are the authors’ own figures and comparisons from the abstract.

The code is available at github.com/zhenyi4/platour.

The catch: gains without communication

The more interesting part of the paper is a set of controlled experiments aimed at the second gap. The authors find that on some tasks the gains can be reproduced by uninformative sender signals, by self-communication, or by training the receiver alone.

Their reading is that trainable bridges can behave as parameter-efficient adaptation mechanisms. In other words, the bridge may improve the small model without exploiting any useful information from the larger one, so a higher score does not prove that cross-model transfer is happening.

To separate the two effects, the authors propose PACT-Check, a measure of how much of a trained PACT system’s gain genuinely comes from communication. The paper closes with a discussion of the opportunities and challenges of PACT systems.

What the abstract leaves open

The abstract does not name the sender and receiver models or give their sizes, and it gives no numbers for the improvements over baselines or over Mostik. It does not say which tasks showed gains that could be reproduced without communication and which did not, or how PACT-Check is computed. Inference cost savings are the motivation for PACT, but the abstract reports no latency or cost measurements.


Source: On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer (arXiv 2611.06976)