Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix default attention mask of generate in MoshiForConditionalGeneration #36171

Merged
merged 3 commits into from
Feb 20, 2025

Conversation

cyan-channel-io
Copy link
Contributor

@cyan-channel-io cyan-channel-io commented Feb 13, 2025

What does this PR do?

Fixes #36160, which is caused by the absence of default attention mask initialization when attention_mask is not given.

_prepare_attention_mask_for_generation is mostly borrowed from MusicGen implementation.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ylacombe, @eustlb

@SeungyounShin
Copy link

Hi everyone,

I just wanted to bring this PR to your attention, as it addresses a critical issue where generation in MoshiForConditionalGeneration doesn't work at all due to the absence of a default attention mask.

I understand you're all busy, but if you could find some time to review this, it would be greatly appreciated. This fix was primarily adapted from the MusicGen implementation to ensure consistency and reliability.

Thanks in advance for your time and consideration!

cc. @ArthurZucker, @amyeroberts, @younesbelkada, @ylacombe, @eustlb

@Rocketknight1
Copy link
Member

cc @eustlb and @gante for generation

Copy link
Member

@gante gante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for identifying the issue and fixing it 🤗

@gante
Copy link
Member

gante commented Feb 20, 2025

@eustlb I've confirmed that this fixes the root issue, including the script in this comment, so I'm taking the liberty of merging it 🤗

@cyan-channel-io @SeungyounShin Thank you for making the library better for everyone 🙌

@gante gante merged commit e18f233 into huggingface:main Feb 20, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Moshi Generation Does Not Work as Expected
4 participants