Using NPU inference model, how to set up a single process

[Bofuser]

When using NPU to inference the MiniCPM_2_6 model, it was found that it divided python into four parallel processes for inferencing, which took up a lot of memory.

The use of GPU inference is single-process inference,

in the demo coding did not find the relevant setting of single-process code, and how to set it?

[Oscilloscope98]

Hi @Bofuser,

For MiniCPM-V_2_6, we currently only supports running on NPU with multiple processes. We will update here for any updates if we further support this model running on a single process :)

[Bofuser]

Got it, thanks for your reply!