Skip to content
This repository has been archived by the owner on Jan 1, 2025. It is now read-only.

Explanation on why off-the-self MAE models do not need proportional attention in ToMeAttention.forward()? #49

Open
Lopa07 opened this issue Nov 22, 2024 · 0 comments

Comments

@Lopa07
Copy link

Lopa07 commented Nov 22, 2024

For MAE off-the-self models, self._tome_info['prop_attn'] should be False as mentioned here. Why for MAE off-the-self models, in ToMeAttention.forward() proportional attention is not needed?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant