GPU Sizing

[tangopoint]

Is there a way to estimate the amount of GPU Memory required to use a GPU index? It would be helpful to know for all GPU index types. I am currently unable to load a GPU_IVF_PQ index onto a GPU with 24GB available.

Can I assume the "Loading Memory" value in the sizing tool for an IVF_Flat is an estimation of the GPU memory needed of an GPU_IVF_FLAT index?

[yhmo]

The sizing tool is mainly for CPU index.

N vectors with D dim, the original data size is ND4Byte.

The size of GPU_CAGRA/GPU_IVF_FLAT index is a bit larger than the original size.
The size of GPU_BRUTE_FORCE index is equal to the original size.
The size of GPU_IVF_PQ is about 25%~30% of the original size.

This doc tells how to deploy a standalone GPU mode milvus and how to configure the GPU-related configurations.
https://milvus.io/docs/install_standalone-docker-compose-gpu.md

Set the gpu.initMemSize and gpu.maxMemSize to a proper value in the milvus.yaml, according to your gpu index size.
For example, if the GPU index size is about 10GB, we can set 16384.
...
gpu:
initMemSize: 16384 # unit: MB
maxMemSize: 16384 # unit: MB
...