EventLens: Event-Structure Reinforcement Learning for Video Understanding
Abstract
Recent multimodal large language models (MLLMs) achieve strong results on video question answering, yet often fail to recover the temporal structure that makes a video coherent. They may merge neighboring events, miss single-event progression, or infer implausible relations across events. We study structured temporal understanding as the ability to recover event structure from visual evidence, including event boundaries, single-event progression, and multi-event relations. We argue that this gap arises from a mismatch between current post-training objectives and temporal reasoning, where caption-style supervision allows models to rely on language shortcuts rather than visual temporal grounding. To address this, we propose EventLens, a vision-centric reinforcement learning framework that learns video understanding through event-structure recovery. Instead of reconstructing captions, EventLens trains models to recover temporal structure under controlled perturbations, such as altered temporal granularity, reversed progression, and shuffled event order. We instantiate this framework with a three-level temporal ontology and derive three verifiable task families: event segmentation, progression discrimination, and multi-event relation reasoning. These tasks admit deterministic rewards, enabling task-specialized GRPO training without human preference labels or learned reward models. We further introduce multi-teacher on-policy distillation (MT-OPD) to consolidate specialized policies into a unified model. Experiments on Qwen3-VL backbones show consistent improvements on temporal grounding and event-sensitive reasoning benchmarks, while preserving general video-QA performance. Learning curves further demonstrate that EventLens achieves comparable downstream transfer with significantly reduced training cost compared to direct mixed RL optimization.
Community
Excited to share EventLens (NeurIPS 2026)! ๐
Can video MLLMs truly understand how events unfold over time?
We introduce Event-Structure Reinforcement Learning, a vision-centric framework that learns temporal structure through verifiable rewards rather than caption-style supervision. By explicitly modeling event boundaries, progression, and inter-event relations, EventLens improves temporal grounding and event-sensitive reasoning while preserving general video understanding capabilities.
๐ Project: https://lostgreen.github.io/EventLens/
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning (2026)
- Video-MOPD: Multi-Teacher On-Policy Distillation for Video Understanding (2026)
- Agentic Video Understanding: A Survey (2026)
- OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction (2026)
- STSG-VQA: Evidence-Grounded Temporal Question Answering from Surgical Spatio-Temporal Scene Graphs (2026)
- VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction (2026)
- Less from More: Reinforcing Sparse Video Reasoning from Dense References (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.16172 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper