OPEA docker images take a lot of disk space

[xiguiw]

OPEA takes a lot of disk space.

"To deploy them on ITAC using medium or large Xeon VMs, 64GB isn’t sufficient and the next tier is 2TB which is much more expensive."

OPEA consumes lots of space: Both memory and disk space.

It is helpful to reduce the OPEA size.

[erojaso]

I've had similar problems
Creating an image for embedding-tei-server used to take 5 min now it takes 20 min and 4.6 gb

REPOSITORY TAG IMAGE ID CREATED SIZE
acraksexplore.azurecr.io/embedding-tei 0.0.3 881d61f97080 10 hours ago 4.6GB
opea/embedding-tei 0.0.3 881d61f97080 10 hours ago 4.6GB
ghcr.io/huggingface/text-embeddings-inference cpu-1.5 64f8afb02d72 4 months ago 138MB

[eero-t]

OPEA consumes lots of space: Both memory and disk space.

RAM usage depends almost completely on the specified inferencing model, and data type. Suitable model depends on the needs of the user, but on newer Xeons, memory usage can be halved by using BF16 data type instead of FP32, see: opea-project/GenAIExamples#330

I've had similar problems Creating an image for embedding-tei-server used to take 5 min now it takes 20 min and 4.6 gb

For which OPEA version and application this is? E.g. v1.1 ChatQnA works fine without the TEI/TGI wrapper services.

---

#265 includes examples for significant reductions in OPEA Comps image sizes, and https://github.com/opea-project/GenAIExamples/pull/1031PR is step 1/2 of reducing image sizes for the apps using them.