discriminability of the evaluation metrics

[HanLiii]

@ziqihuangg Thanks for the great work👏! I noticed that some metrics, e.g. subject consistency, background consistency, motion smoothness, have relatively high score (close to 1) on the Vbench Leaderboard. Is there any method that can increase the discriminability of the evaluation metrics?

[ziqihuangg]

Thanks for this question! Indeed, some dimensions have become gradually saturated, and that's largely because recent models have already addressed these aspects quite well (e.g., achieving good frame consistency and smooth motion). We are working on new solutions to increase discriminability and evaluate more challenging aspects in video generation (no promises yet, but stay tuned for updates in our future releases 👀).