On ARM by default turn on GPU acceleration

On x86 we don't want to do this without a GPU because x86 integrated graphics have very limited access to VRAM and it's normally not worth it. But ARM SoC's share memory between CPU and GPU, meaning it's worth it generally. And we care mostly about Apple Silicon where we want this on in podman machine. Signed-off-by: Eric Curtin <[email protected]>
containers · Jan 10, 2025 · 4e98c92 · 4e98c92
1 parent 181c871
commit 4e98c92
Showing 1 changed file with 6 additions and 2 deletions.
diff --git a/ramalama/model.py b/ramalama/model.py
@@ -1,6 +1,7 @@
 import os
 import sys
 import atexit
+import platform
 
 from ramalama.common import (
     container_manager,
@@ -223,9 +224,12 @@ def gpu_args(self):
             # any additional arguments.
             pass
         elif sys.platform == "linux" and (
-            os.getenv("HIP_VISIBLE_DEVICES") or os.getenv("ASAHI_VISIBLE_DEVICES") or os.getenv("CUDA_VISIBLE_DEVICES")
+            platform.machine() == "aarch64"
+            or os.getenv("HIP_VISIBLE_DEVICES")
+            or os.getenv("ASAHI_VISIBLE_DEVICES")
+            or os.getenv("CUDA_VISIBLE_DEVICES")
         ):
-            gpu_args = ["-ngl", "99"]
+            gpu_args = ["-ngl", "999"]
         else:
             print("GPU offload was requested but is not available on this system")