MediaPipe LLMInference Task Crash

[tribhuwan-kandpal]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Android 15

MediaPipe Tasks SDK version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

LLMInference

Programming Language and version (e.g. C++, Python, Java)

Java

Describe the actual behavior

I compiled LLMInference task from MediaPipe source code and tried to use in place of the one available from play service maven repo and it crashed while trying to use gemma-2b-it-gpu-int8.

Describe the expected behaviour

LLMInference task should not crash.

Standalone code/steps you may have used to try to get what you need

Compile mediapipe for Android using docker. In Android Studio project , remove "implementation 'com.google.mediapipe:tasks-genai:0.10.14'" and add dependency on local built aar i.e. tasks_genai.aar. Run app it crashed. Remove local aar dependency and use maven repo dependency for mediapipe lib, it works well.

Other info / Complete Logs

com.google.mediapipe.framework.MediaPipeException: internal: Failed to initialize engine: %sFailed to create engine: UNKNOWN: Unable to open zip archive. [MediaPipeTasksStatus='104']
                                                                                                    	at com.google.mediapipe.tasks.core.LlmTaskRunner.nativeCreateEngine(Native Method)
                                                                                                    	at com.google.mediapipe.tasks.core.LlmTaskRunner.<init>(LlmTaskRunner.java:168)
                                                                                                    	at com.google.mediapipe.tasks.genai.llminference.LlmInference.<init>(LlmInference.java:93)
                                                                                                    	at com.google.mediapipe.tasks.genai.llminference.LlmInference.createFromOptions(LlmInference.java:59)

[tribhuwan-kandpal]

I added some logs to find out where it is crashing.
In native it is failing when it is trying to extract content from zip file .What contents it is looking for I am not aware but its not null data so I am suspecting some kind of data type mismatch.

In Native side,
CreateTfliteLlmCpuEngine -> ModelAssetBundleResources::ExtractFilesFromExternalFileProto() -> ExtractFilesfromZipFile() -> Fail/Crash

Can someone check and confirm if some other kind of file needs to be passed for model ? And where can I get information about file format of model that mediapipe supports ?

[kuaashish]

Hi @tribhuwan-kandpal,

Could you please provide the complete steps you are following from our documentation, or share the command you are using along with the full error log? This will help us better understand the issue and reproduce it on our end.

Thank you!!

[tribhuwan-kandpal]

1. I am using LLMInference APIs and gemma models as mentioned in docs
   https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android
2. Works well for gemma models if I use precompiled maven dependencies with the steps mentioned
   https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android#dependencies
3. Now I compiled llminference task aar for Android from MediaPipe source code.
   Guides :
   https://ai.google.dev/edge/mediapipe/framework/getting_started/install.md#installing_using_docker
   Command to compile llm task aar lib :
   bazel build -c opt --config=android_arm64 //mediapipe/tasks/java/com/google/mediapipe/tasks/genai:tasks_genai.aar
4. Used the locally generated aar file in place of the one from google in maven repo.
5. Device I am using is Android 15 , Google Pixel Tablet
6. I was able to run gemma models with the library from maven repo (.bin files for int4 and int8, gpu and cpu)
7. When I tried to run same models with locally compiled library(aar), it crashed with the call stack in java that I provided earlier.

[tribhuwan-kandpal]

For the logs, I could not find the way to get logs on Android. I am using my custom logging that probably wont help much.
For logging details I opened other ticket so that I know how to provide you logs. So plz help me get the logs.
#5827

[kuaashish]

Hi @tribhuwan-kandpal,

We believe the issue may be occurring because you are trying to generate the app using the AAR method. However, we typically recommend using the Maven package, as we are unsure about the AAR method. Please allow us some time to discuss this with our team, and we will provide you with more information soon.

Thank you!!

[tribhuwan-kandpal]

Sure, I will wait for response.
Reason of using aar is, I want to understand how LLMInference is working in mediapipe and secondly if I want to use my own tflite delegate for llminferencing , will it be possible or not. With precompiled library I can not get the information and do all these things.