Export recent MHA changes from fbcode

triton_splitk extensively expanded.
Includes some paged attention, merge_attentions

ghstack-source-id: 2e01b95df689395cb64844eb6db5e5638058e599
Pull Request resolved: fairinternal/xformers#1031

__original_commit__ = fairinternal/xformers@19b9c21

CHANGELOG.md

@@ -6,8 +6,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [0.0.25] - TBD
 ### Added
+- New merge_attentions function
 ### Improved
 - fMHA: Updated Flash-Attention to v2.5.2: this has a performance improvement for multiquery.
+- fMHA: triton_splitk changed and expanded. Now amalgamates using LSE. Can autotune, supports causal with a small number of queries - not just 1. Experimental support for paged attention.
 ### Removed
 
 ## [0.0.24] - 2024-01-31