Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 fix: Cache the runner image to reduce build time #143

Merged
merged 26 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
546e074
✨ feat: Add CI for container
wanjohiryan Dec 8, 2024
af36efd
Merge branch 'main' into feat/ci
wanjohiryan Dec 8, 2024
c1d86b2
🐛 fix: Rename `USER_PASSWORD` to `USER_PWD` to avoid fix CI job
wanjohiryan Dec 8, 2024
db84d11
Merge branch 'feat/ci' of ssh://github.com/nestriness/nestri into fea…
wanjohiryan Dec 8, 2024
dea1c4f
fix: `COPY` instead of clone
wanjohiryan Dec 8, 2024
0f9b830
fix: Copy the whole repo
wanjohiryan Dec 8, 2024
46c1a6f
migrate `Container*` to `containers/` subdirectory
wanjohiryan Dec 8, 2024
ebe2c10
✨ feat: Run CI job for
wanjohiryan Dec 8, 2024
46fc7a0
fix: Add ` gst-plugin-rswebrtc`
wanjohiryan Dec 9, 2024
157cdff
feat: Add CI job for main
wanjohiryan Dec 9, 2024
307f5b2
feat: Rename to `Push to main`
wanjohiryan Dec 9, 2024
e8c1942
fix: Add some caching to runner image
wanjohiryan Dec 10, 2024
9c4e77b
Merge branch 'main' into fix/caching
wanjohiryan Dec 10, 2024
ed4b90b
fix: Remove repo caching
wanjohiryan Dec 10, 2024
561f89d
fix: Fix working directory directory
wanjohiryan Dec 10, 2024
b244338
fix: Copy contents to /artifacts
wanjohiryan Dec 10, 2024
f65385c
fix: Fixup the blank spaces
wanjohiryan Dec 10, 2024
102367b
fix: Copy the whole include
wanjohiryan Dec 10, 2024
db5aea0
fix: Use /plugin
wanjohiryan Dec 10, 2024
fdd5384
fix: Rename build -> builder
wanjohiryan Dec 10, 2024
f820e0f
fix: Fix some more stuff
wanjohiryan Dec 10, 2024
f1c2e30
fix: Try adding some cargo-c cache
wanjohiryan Dec 10, 2024
7a41a11
fix: Add `/root/.cargo/bin/`
wanjohiryan Dec 10, 2024
ed83abb
fix: Clean Up
wanjohiryan Dec 10, 2024
e65bae6
fix: Type bind
wanjohiryan Dec 10, 2024
a73c0c1
fix: Add read/write functionality
wanjohiryan Dec 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion .github/workflows/runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,14 @@ jobs:
-
name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
-
name: Build Docker image
uses: docker/build-push-action@v5
Expand All @@ -49,7 +57,14 @@ jobs:
context: ./
push: false
load: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
tags: nestri:runner
-
name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache

build-docker-main:
name: Build image on main
Expand Down Expand Up @@ -83,12 +98,27 @@ jobs:
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
-
name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
-
name: Build Docker image
uses: docker/build-push-action@v5
with:
file: containers/runner.Containerfile
context: ./
push: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
labels: ${{ steps.meta.outputs.labels }}
-
name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
49 changes: 34 additions & 15 deletions containers/runner.Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,19 @@ RUN pacman -Syu --noconfirm meson pkgconf cmake git gcc make rustup \

# Setup stable rust toolchain #
RUN rustup default stable
# # Clone nestri source #

#Copy the whole repo inside the build container
COPY ./ /builder/nestri/
# COPY ./ /builder/nestri/

RUN mkdir -p /artifacts

RUN cd /builder/nestri/packages/server/ && \
cargo build --release
RUN --mount=type=bind,target=/builder/nestri/,rw \
--mount=type=cache,target=/builder/nestri/target/ \
--mount=type=cache,target=/usr/local/cargo/git/db \
--mount=type=cache,target=/usr/local/cargo/registry/ \
cd /builder/nestri/packages/server/ \
&& cargo build --release \
&& cp /builder/nestri/target/release/nestri-server /artifacts/

#******************************************************************************
# gstwayland-builder
Expand All @@ -33,15 +40,28 @@ RUN pacman -Syu --noconfirm meson pkgconf cmake git gcc make rustup \
# Setup stable rust toolchain #
RUN rustup default stable
# Build required cargo-c package #
RUN cargo install cargo-c
RUN --mount=type=cache,target=/usr/local/cargo/git/db \
--mount=type=cache,target=/usr/local/cargo/registry/ \
--mount=type=cache,target=/root/.cargo/bin/ \
cargo install cargo-c

# Clone gst plugin source #
RUN git clone https://github.com/games-on-whales/gst-wayland-display.git

# Build gst plugin #
RUN mkdir plugin && \
cd gst-wayland-display && \
cargo cinstall --prefix=/builder/plugin/
RUN mkdir plugin

RUN mkdir -p /artifacts

WORKDIR /builder/gst-wayland-display

RUN --mount=type=cache,target=/builder/gst-wayland-display/target/ \
--mount=type=cache,target=/root/.cargo/bin/ \
--mount=type=cache,target=/builder/plugin/ \
--mount=type=cache,target=/usr/local/cargo/git/db \
--mount=type=cache,target=/usr/local/cargo/registry/ \
cargo cinstall --prefix=/builder/plugin/ \
&& cp -r /builder/plugin/ /artifacts/

#******************************************************************************
# runtime
Expand All @@ -63,7 +83,6 @@ RUN pacman -Syu --noconfirm --needed \
# Clean up pacman cache
paccache -rk1


## User ##
# Create and setup user #
ENV USER="nestri" \
Expand All @@ -90,13 +109,13 @@ RUN usermod -aG input root && usermod -aG input ${USER} && \
## Copy files from builders ##
# this is done here at end to not trigger full rebuild on changes to builder
# nestri
COPY --from=gst-builder /builder/nestri/target/release/nestri-server /usr/bin/nestri-server
COPY --from=gst-builder /artifacts/nestri-server /usr/bin/nestri-server
# gstwayland
COPY --from=gstwayland-builder /builder/plugin/include/libgstwaylanddisplay /usr/include/
COPY --from=gstwayland-builder /builder/plugin/lib/*libgstwayland* /usr/lib/
COPY --from=gstwayland-builder /builder/plugin/lib/gstreamer-1.0/libgstwayland* /usr/lib/gstreamer-1.0/
COPY --from=gstwayland-builder /builder/plugin/lib/pkgconfig/gstwayland* /usr/lib/pkgconfig/
COPY --from=gstwayland-builder /builder/plugin/lib/pkgconfig/libgstwayland* /usr/lib/pkgconfig/
COPY --from=gstwayland-builder /artifacts/plugin/include/libgstwaylanddisplay /usr/include/
COPY --from=gstwayland-builder /artifacts/plugin/lib/*libgstwayland* /usr/lib/
COPY --from=gstwayland-builder /artifacts/plugin/lib/gstreamer-1.0/libgstwayland* /usr/lib/gstreamer-1.0/
COPY --from=gstwayland-builder /artifacts/plugin/lib/pkgconfig/gstwayland* /usr/lib/pkgconfig/
COPY --from=gstwayland-builder /artifacts/plugin/lib/pkgconfig/libgstwayland* /usr/lib/pkgconfig/

## Copy scripts ##
COPY packages/scripts/ /etc/nestri/
Expand Down
Loading