Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Mamba2] Fix caching, slow path, and multi-gpu #35154

Merged
merged 16 commits into from
Dec 20, 2024
Merged

[Mamba2] Fix caching, slow path, and multi-gpu #35154

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
1fc0151
fixup mamba2 - caching and several other small fixes
vasqu Dec 8, 2024
d96c752
fixup cached forward
vasqu Dec 8, 2024
b13d6ee
correct fix this time
vasqu Dec 8, 2024
1da890c
fixup cache - we do not need to extend the attn mask it's handled by …
vasqu Dec 8, 2024
eb64f1e
remove unnecessary (un)squeeze
vasqu Dec 8, 2024
16ca5d8
fixup cache position
vasqu Dec 8, 2024
f95eb09
simplify a few things
vasqu Dec 9, 2024
f2bdce3
[run-slow] mamba2
vasqu Dec 9, 2024
f56a805
multi gpu attempt two
vasqu Dec 9, 2024
dcf8de1
[run-slow] mamba2
vasqu Dec 9, 2024
faee40b
[run-slow] mamba2
vasqu Dec 9, 2024
720baf6
Merge remote-tracking branch 'upstream/main' into fix-mamba2-caching
vasqu Dec 9, 2024
0da9454
[run-slow] mamba2
vasqu Dec 9, 2024
4160edf
[run-slow] mamba2
vasqu Dec 9, 2024
4f62ae8
add newer slow path fix
vasqu Dec 10, 2024
3739f8a
[run-slow] mamba2
vasqu Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading