added docker-multi-stage builds

[rudiservo]

Added multi-stage dockerfile builds, improved total build time to under 2 hours, added Vulkan and Full-intel.

Updated rocm dockerfile.

Hopefully it will all work without any problems.

[ngxson]

Please also fix coding style as reported by editorconfig workflow

[ngxson]

I think it's safe to switch the image ro runtime here, as we don't gonna build anything from this point on

Suggested change

	FROM ${BASE_CUDA_DEV_CONTAINER} AS full
	FROM ${BASE_CUDA_RUN_CONTAINER} AS full

[rudiservo]

if that is so, it is safe to not install cmake or any other build packages?
This will make the image and build time smaller by a tiny bit.

[ngxson]

yes, the runtime doesn't need to have cmake and build-essentials

[ngxson]

same here

[ngxson]

Will the old full tag become full-cpu? If yes, this will be a breaking change for downstream projects, I don't think we should let that happen, because the full tag will not be deleted people using it in an automated way won't receive any updates

[rudiservo]

issue is that "full" only tag came without any specification, and we have many types of builds for given targets, the possible workaround will be a hack but I think it's doable.
We have the cuda, rocm, intel, vulkan but cpu has no spec, it was just full, when you do multi-stage with the current tag system it is a bit tricky, but I will give it a try.

[ngxson]

we should keep the tag as-is, because many people depends on that and we currently don't have a way to communicate with them about breaking changes in tag name.

changing full, light and especially server to something else will make someone already using it never receive updates in the future. and worst, because there is no error or warning when they run it, they wouldn't know there is a breaking change.

[rudiservo]

@ngxson the tags are as-is, just a little issue of caching layers, it's going to be a few more pushes, sorry.

[rudiservo]

@ngxson I am really sorry about the spam, starting to use act.
Just a slight issue with caching, will report when fixed.

[rudiservo]

@ngxson ok figured out the cache, it will be using github cache, still a bit experimental but it works best at this point in time.

In the dockerfiles there is a

cp build/bin/* .

can this be changed to a mv? It would same some space.

Or I can just copy this from the cpu dockerfile

COPY --from=build /app/build/bin/ /app/
COPY --from=build /applib/ /app/
COPY --from=build /app/convert_hf_to_gguf.py /app/
COPY --from=build /app/gguf-py /app/gguf-py

instead of copying the full /app in the full versions.

[ngxson]

Suggested change

	COPY requirements.txt requirements.txt
	COPY requirements.txt .

[ngxson]

Just wondering, can we do this a bit more simple by using sed to remove -cpu from tag name? so we don't have to specify FULLTAGS, LIGHTTAGS, SERVERTAGS

Another approach (with less code) could be:

BASE_IMAGE="ghcr.io/${REPO_OWNER}/${REPO_NAME}"
FULLTAGS="${BASE_IMAGE}:full,${BASE_IMAGE}:full-cpu"
SERVERTAGS="${BASE_IMAGE}:server,${BASE_IMAGE}:server-cpu"

[rudiservo]

I could try something with sed, like:

echo '${{ matrix.config.tag }}' | sed 's/-cpu//g'

It would remove the if statement, but in my opinion the if statement is more "readable" or expressive at first glance.

I can simplify this, still I would need at least 3 vars to make this work and keep the project tags has they are.
The PREFIX, POSTFIX and one extra TYPE for the "matrix tag"

if [[ "${{ matrix.config.tag }}" == "cpu" ]]; then
    TYPE=""
else
    TYPE="-${{ matrix.config.tag }}"
fi
PREFIX="ghcr.io/${REPO_OWNER}/${REPO_NAME}:"
POSTFIX="-${TAG_POSTFIX}"

And

echo "prefix=$PREFIX" >> $GITHUB_OUTPUT
echo "type=$TYPE" >> $GITHUB_OUTPUT
echo "postfix=$POSTFIX" >> $GITHUB_OUTPUT

with in each build-push-action for the there it would be

tags: ${{ steps.tag.outputs.prefix }}full${{ steps.tag.outputs.type }},${{ steps.tag.outputs.output_tags }}full${{ steps.tag.outputs.type }}${{ steps.tag.outputs.postfix }}

Or

FULLTAGS="${PREFIX}full${TYPE},${PREFIX}full${TYPE}${POSTFIX}"
LIGHTTAGS="${PREFIX}light${TYPE},${PREFIX}light${TYPEl${POSTFIX}"
SERVERTAGS="${PREFIX}server${TYPE},${PREFIX}server${TYPE}${POSTFIX}"
echo "full_output_tags=$FULLTAGS" >> $GITHUB_OUTPUT
echo "light_output_tags=$LIGHTTAGS" >> $GITHUB_OUTPUT
echo "server_output_tags=$SERVERTAGS" >> $GITHUB_OUTPUT

What do you think?

[rudiservo]

Pushed new code, normalized all docker images, hopefully optimized docker layers.

[ngxson]

In the dockerfiles there is a

cp build/bin/* .

can this be changed to a mv? It would same some space.

Yes I think we can, I don't see any problem with this as we never run cmake after that (cc @slaren too, in case I'm missing something)

I'll test on cpu + CUDA in the next few days when I come back from vacation

[slaren]

Yes, I don't see why the cp couldn't be replaced with a mv. Same for the find -exec cp that is used to copy the .so files. Although all of this should probably be replaced with cmake --install in the future.

[rudiservo]

@slaren agree, but I would be happy to know what do I need to copy for the full image to work instead of the complete /app folder just to try and keep these images has small has possible for now.

From what I can tell it's the python scripts, the .so and the build/bin.
I will make the appropriate changes if that is ok?

[slaren]

I already tried to copy as little as possible for the full image in the full.Dockerfile image. The same should work for the other backend images.

[rudiservo]

I normalized all docker images, improved the code a bit, I think everything is ok.
Total build time for 15 images is now at ~1h50.

[rudiservo]

It's pushing multiple untaged images, checking.

[rudiservo]

Fixed, added provenance to build-and-push.