This repository is not active. moe-recipes is now actively being developed in other repository.
To install the package, run the following command:
pip install -r requirements.txtIf you want to use the library in multi-nodes, you need to install the below packages:
module load openmpi/4.x.x
pip install mpi4pyTo install the FlashAttention, run the following command: (GPU is required)
pip install ninja packaging wheel
pip install flash-attn --no-build-isolation