Temporal Accumulation Errors from Temporal Autoregressive Model

[MinghanLi]

The temporal autoregressive approach introduces temporal accumulation errors, leading to issues such as blurriness and darker tones in videos, especially for longer sequences. Additional artifacts may also appear over time. Below are two examples of 16-frame video generation at 384p resolution: the first video shows results from MMDIT, while the second video shows results from MiniFlux.

A_car_is_moving_on_the_road_of_the_forest._The_car_is_moving_to_the_camera.mp4

A_car_is_moving_on_the_road_of_the_forest._The_car_is_moving_to_the_camera.mp4

A possible solution to address this issue is to truncate the historical frame information, such as using only the previous 8 frames for autoregression. Have there been any plans or considerations for implementing this solution in future work?

[feifeiobama]

Yes, temporal error accumulation is an important problem we will look into in the future work. We recommend reading CausVid as a highly relevant work.