Looking for mlx-community/llama-3.1-405b-Instruct with bf16 Precision

[quebic-source]

Hi! I'm looking for an existing version of mlx-community/llama-3.1-405b-Instruct with bf16 precision, or any guidance on how to convert the model to bf16. Any help would be appreciated!

[awni]

For b16 you can just use the original model meta-llama/Llama-3.1-405B-Instruct. I'm not sure what your plan is, but just be-warned that model in bf16 doesn't run on any one machine.

[scouzi1966]

Inspired by this project, https://github.com/exo-explore/exo, it would be nice to see a native mlx clustered solution using thunderbolt 4/5 bridge to run larger models too big to fit on any single available SKU. Apple can breakout here with its Apple Silicon