Using CUDA

[domi413]

Hey,

I noticed an option under Ollama Instance to set CUDA_VISIBLE_DEVICES. I assume this is for running the AI models with CUDA instead of the CPU. Is there any documentation or instructions on how to use Alpaca with CUDA instead of the CPU?

[Jeffser]

Hi, the entry for that option has a button to go to Ollama's documentation
https://github.com/ollama/ollama/blob/main/docs/gpu.md#gpu-selection

I know it isn't really clear but I just wanted to expose those environment variables for people that needed them