Reviso trains a video captioning agent to choose which visual evidence to inspect
An agentic video captioner plans what to look at from a low-frame-rate preview, revisits key segments with tools, and is trained with GRPO and cost-aware rewards.
The problem
Multimodal large language models (MLLMs) can already generate detailed video captions, but the authors argue that they may overemphasize minor details while overlooking essential context and key events. Their starting point is that not all visual evidence deserves equal attention, because the importance of visual information depends on the content of each video.
Reviso, a paper published on arXiv and listed on Hugging Face daily papers on 2026-11-02, is the authors’ answer: an agentic video captioning framework that learns what to observe and how to gather the relevant visual evidence.
How Reviso works
Reviso starts from a low-frame-rate global preview of the video. From that preview it builds a hierarchical Context-Focus-Points (CFP) observation plan. The plan identifies the video context, prioritizes key aspects, and defines concrete observation targets together with the tools to use on them.
Guided by this plan, the model performs what the authors call goal-directed active perception: it selectively revisits temporal segments and combines tools for fine-grained inspection. It interprets the evidence it acquires, updates its plan as needed, and integrates global and local information to generate the final caption.
Training
To learn this observation strategy, the authors develop a verification-guided pipeline that constructs training trajectories through factual verification and teacher-guided refinement. They then train Reviso with Group Relative Policy Optimization (GRPO), using these signals:
- a key-fact reward for caption quality;
- a Plan Alignment Reward (PAR) for identifying important observation targets;
- an Unproductive Tool-Call Penalty (UTCP) that adjusts observation-cost penalties based on caption quality.
The penalty is the cost side of the design: the training objective carries an observation cost that is adjusted by caption quality, rather than rewarding caption quality alone.
Results as reported
According to the abstract, experiments across four video captioning benchmarks show that Reviso outperforms the evaluated open-source baselines, and its generated captions better support downstream video question answering. These are the authors’ own claims, and the abstract gives no figures for them.
What the abstract leaves open
The abstract does not name the four benchmarks or the baselines, and it reports no scores. It also does not say which base model Reviso is built on, which tools it can call, or how “unproductive” is decided for the tool-call penalty. No code repository is listed with the paper. The full text on arXiv (2611.03602) is where those details would have to come from.
Source: Reviso: Not All Visual Evidence Deserves Equal Attention in Video Captioning (arXiv 2611.03602)