UniX-Lab/StreamOPD-Data

StreamOPD: Training Data for Streaming Video Post-Training

StreamOPD-Data provides video question-answer annotations, local video time intervals, and optional teacher-side visual cues for on-policy distillation and related controls. Video files, model weights, training logs and credentials are not included. The QA annotations are derived from ThinkStream. Video references resolve to LLaVA-Video-178K and the Kinetics-700 component of Tarsier2-Recap-585K.

Configurations

Configuration	Train	Validation	Training rows with cues	Purpose
current (default)	25,106	326	24,245	Current shared OPD/CueGate pool after known input and source-overlap exclusions
legacy_25118	25,118	326	24,257	Earlier QA pool with the explicit cue-instruction teacher prompt
grpo_25114	25,114	326	0	Readability-filtered GRPO data with the actual direct-answer prompt suffixes

These are overlapping versions of the same data family, not independent datasets to concatenate. Validation questions, answers and intervals are shared; the GRPO version has its own direct-answer prompt wording. Each configuration's original row order is retained. The legacy_25118 configuration retains the known problematic rows for historical reproduction; use current unless reproducing that specific configuration.

Two source questions contained email text among answer choices. This publication replaces those strings with [REDACTED_EMAIL] in every affected prompt, without changing answer letters. privacy_redacted marks the two affected training rows in each configuration. Therefore this release is a privacy-sanitized representation, not a byte-identical copy of private training inputs. See data processing.

Load annotations

from datasets import load_dataset

data = load_dataset("UniX-Lab/StreamOPD-Data", "current")
print(data["train"][0]["prompt"])
print(data["train"][0]["videos"])

videos contains plain relative-path metadata, not embedded video bytes or automatic downloads. Obtain the upstream videos using video_sources.md.

Download this small annotation release with one worker:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="UniX-Lab/StreamOPD-Data",
    repo_type="dataset",
    local_dir="StreamOPD-Data",
    max_workers=1,
)

Resolve paths into local trainer inputs after obtaining the videos:

python StreamOPD-Data/scripts/prepare_local_data.py \
  --config current --video-root /path/to/videos \
  --output-dir ./streamopd-local

python StreamOPD-Data/scripts/validate_data.py

The preparation tool requires all video paths to resolve, preserves timestamps and annotation content, and writes train.parquet and validation.parquet to a new local directory. It supports the extra data/ alias used in some historical LLaVA paths. It checks file presence without decoding videos; the known legacy interval failures remain documented rather than silently filtered.

Fields

Field	Meaning
sample_id	Stable ID, including split; shared across configurations for the same underlying source row
source_row_id	Original zero-based row number in the base train pool or validation split
privacy_redacted	Whether contact text was replaced in this row for publication
prompt	Student chat messages, with <video> placeholder and question/choices
videos	List of {video, video_start, video_end}; paths relative to the video root, times in seconds
video_source	LLaVA-Video-178K or Kinetics-700
ground_truth	Verifiable reference answer as a string
response_format	Multiple Choice, Binary, or Counting
data_source	Preserved training reward-routing tag; not the video's provenance
reward_model	Reference-answer struct expected by the original training interface
teacher_prompt	Cue-augmented teacher messages where available; otherwise identical to the student prompt
teacher_reference_raw	Original student messages used as a no-cue reference
teacher_reference_empty	Same cue instruction wrapper with the cue removed; or original messages on fallback rows
cue_text, cue_available	Optional generated pointer and availability flag

Teacher-reference fields are stored for convenience in every configuration. Only current used both stored reference forms in the reference-template preparation; the corresponding fields in legacy_25118 are reconstructed from the unchanged cue wrapper. All teacher fields in grpo_25114 equal its direct-answer student prompt. Plain OPD must use the student prompt without injecting teacher_prompt as a cue.

Current training split statistics

Answer format	Rows
Multiple choice	17,485
Binary	6,364
Counting	1,257

19,814 training records reference LLaVA-Video-178K and 5,292 reference Kinetics-700. There are 22,848 unique video files in the current training split. Intervals range from 0.4 to 30 seconds. 861 training rows have no accepted cue and use the original prompt as their teacher input. Validation has no cue annotations.

Scope and limitations

• current excludes four confirmed unreadable video intervals and eight training records tied to seven confirmed overlapping source videos. This is not an exhaustive disjointness guarantee against every benchmark.
• An existing audit identified 252 current MCQs without an explicit option block. Their mapping is unresolved; this release preserves them and publishes stable IDs in metadata/known_issues.json.
• Cues are generated annotations. Automated screening is not a guarantee that a cue is correct or free of answer-bearing information.
• Historical cue generation could inspect the full source clip, including content after a query timestamp. These are not certified prefix-only cues.
• Student input intervals end at the recorded query timestamp. This does not certify that every answer or cue is recoverable from the sampled student frames.
• Training interval sampling and four-frame streaming deployment are distinct. No rendered clips or precomputed frame cache are distributed here.
• This dataset release does not establish new benchmark scores or independent training-seed uncertainty.

Provenance, terms and integrity

See video sources, construction and preprocessing, and component terms. ThinkStream's MIT notice is included under licenses/. Source videos remain subject to their original distribution conditions; the dataset repository does not grant rights to those videos.

metadata/release_manifest.json identifies the release files and upstream revisions. checksums.sha256 enables local integrity checks. metadata/video_manifest.parquet lists unique source video paths without internal machine paths.

If you use these annotations, cite StreamOPD and acknowledge the upstream ThinkStream, LLaVA-Video and Tarsier2/Kinetics sources. Use the repository revision when reporting the precise data version.

Citation

@article{wu2026streamopd,
  title={StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding},
  author={Wu, Keming and Wang, Baoyi and Zhang, Kaichen and An, Xiang and Yang, Zuhao and Wang, Sudong and Zhu, Haowei and Huang, Tingxuan and Gao, Hongcheng and Wang, Bin},
  journal={arXiv preprint arXiv:2608.16320},
  year={2026}
}