Frame-wise autoregressive video diffusion

Context-Matched Distillation:
Teacher Causality for
Autoregressive Video Distillation

Hmrishav Bandyopadhyay1,2 Xuanchi Ren1 Zijian Huang1 Jay Zhangjie Wu1 Tianshi Cao1 Ruilong Li1 Bryan Chu1 Sanja Fidler1 Yi-Zhe Song2 Zian Wang1

1 NVIDIA 2 SketchX, CVSSP, University of Surrey

Paper arXiv Code soon BibTeX
Generated rollout
TL;DR

We distill frame-wise autoregressive video generation with a causal teacher that scores each frame from the same prefix and controls available to the student—improving quality, conditional control, and long-horizon stability while enabling interaction at every frame.

01 / 04 latent frames per generation step
Matched teacher–student
information context
Every frame available for
interactive control
Long horizon rollout context
supervision

01 / Results

Generation, one frame at a time.

Videos play automatically as they enter view. Select any clip to pause or resume it.

01.1

Short Video Generation

Few-step, frame-wise rollouts preserve visual quality while removing the block boundaries that constrain interactive generation.

Chunk 1

Chunk 4

01.2

Camera Control

Matching the teacher’s prefix and control context improves conditional alignment across distinct camera trajectories.

Chunk 1

Chunk 4

01.3

Long Video Generation

Rollout context matching lets the student learn from extended causal trajectories without being restricted to the fixed windows of a bidirectional teacher.

Chunk 1

Chunk 4

02 / Method

Score what the student can see.

Prefix Scoring aligns the teacher’s information with the student’s causal generation process.

Context-Matched Distillation method figure showing on-policy rollout and cached prefixes, block-masked prefix and target scoring, and context-matched DMD gradients.
Core principle

Teacher score context = Student generation context

03 / Paper

Abstract

Interactive autoregressive video generation demands both low-latency rollouts and precise online control. Few-step distillation accelerates generation by reducing denoising steps, while online control imposes a causal constraint: frames and blocks should depend on history and controls available during generation. Existing video distribution matching distillation (DMD) pipelines, however, often supervise causal few-step students using bidirectional teachers that score complete clips. The score for a target can therefore depend on future frames and controls that were unavailable when the student generated it, misaligning teacher supervision with the student's causal information set.

We introduce Context-Matched Distillation (CMD), a causal DMD framework that aligns teacher supervision with the information available when each target is generated. CMD replaces bidirectional full-clip scoring with a causal teacher that evaluates each target without access to future frames or controls. The same causal teacher initializes the few-step student, establishing a consistent causal formulation across teacher training, student distillation, and inference. Beyond aligning the temporal information boundary, Prefix Scoring matches supervision to the student's realized rollout context by evaluating each target under the cached student-generated prefix that produced it.

Prefix Corruption further stabilizes training by perturbing unreliable prefixes produced early in training while preserving this target-context alignment. With a simple causal formulation, CMD naturally extends to frame-wise and chunk-wise generation, long video distillation, and camera-conditioned distillation. Experiments demonstrate state-of-the-art aggregate performance among autoregressive methods on both short- and long-video benchmarks, together with substantially improved adherence to time-varying camera controls.

04 / Cite

BibTeX

@article{bandyopadhyay2026context,
  title   = {Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation},
  author  = {Bandyopadhyay, Hmrishav and Ren, Xuanchi and Huang, Zijian
             and Wu, Jay Zhangjie and Cao, Tianshi and Li, Ruilong
             and Chu, Bryan and Fidler, Sanja and Song, Yi-Zhe
             and Wang, Zian},
  journal = {arXiv preprint arXiv:2608.13391},
  year    = {2026},
  eprint  = {2608.13391},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url     = {https://arxiv.org/abs/2608.13391}
}