Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub CI #2

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# waf
.lock-waf*
.waf*/
build/
__pycache__/
/waf-tools/*.pyc

# cmake
cmake-build*/

# IDE
*.code-workspace
.vscode/
.idea/
.cache/

# test
test/out/
test/.nfs*

# TeX
/docs/*.aux
/docs/*.log
/docs/*.out
/docs/*.synctex.gz
/docs/*.toc

# Misc
tmp/
82 changes: 82 additions & 0 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
name: Docker Image CI

on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# In this step, this action saves a list of existing images,
# the cache is created without them in the post run.
# It also restores the cache if it exists.
- name: Enable Docker Layer Caching
uses: jpribyl/[email protected]
# Ignore the failure of a step and avoid terminating the job.
continue-on-error: true

# sdpb image
- name: Build and export sdpb image
uses: docker/build-push-action@v5
with:
context: .
tags: sdpb
outputs: type=docker,dest=/tmp/sdpb.tar
- name: Upload sdpb artifact
uses: actions/upload-artifact@v3
with:
name: sdpb
path: /tmp/sdpb.tar

# sdpb-test image
- name: Build and export sdpb-test image
uses: docker/build-push-action@v5
with:
context: .
tags: sdpb-test
target: test
outputs: type=docker,dest=/tmp/sdpb-test.tar
- name: Upload sdpb-test artifact
uses: actions/upload-artifact@v3
with:
name: sdpb-test
path: /tmp/sdpb-test.tar

test:
runs-on: ubuntu-latest
needs: build
steps:
- name: Download sdpb artifact
uses: actions/download-artifact@v3
with:
name: sdpb
path: /tmp
- name: Download sdpb-test artifact
uses: actions/download-artifact@v3
with:
name: sdpb-test
path: /tmp

- name: Load images
run: |
docker load --input /tmp/sdpb.tar
docker load --input /tmp/sdpb-test.tar
docker image ls -a

- name: Run sdpb --help
run: docker run sdpb sdpb --help

- name: Run all tests
run: docker run sdpb-test ./test/run_all_tests.sh mpirun --oversubscribe


101 changes: 101 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# syntax=docker/dockerfile:1

# This file contains three stages: 'build', 'install', 'test'
# https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#use-multi-stage-builds
#
# 'build' builds sdpb binaries. This image is heavy, because it contains all sources etc.
# 'install' contains only sdpb binaries (in /usr/local/bin/) + necessary dynamic libraries
# 'test' is based on 'install', but also copies sdpb/build/ and sdpb/test/ folders to /home/testuser/sdpb/
# - this allows to in order to run unit and integration tests by calling sdpb/test/run_all_tests.sh
# Final (lightweight) image is made from 'install' target.
#
# How to check that SDPB image works:
#
# docker build . -t sdpb
# docker run sdpb mpirun --allow-run-as-root sdpb --help
#
# How to run unit/integration tests to check that SDPB works correctly:
#
# docker build . -t sdpb-test --target test
# docker run sdpb-test ./test/run_all_tests.sh mpirun --oversubscribe
#
# Note: 'mpirun --oversubscribe' is necessary only if your environment has less than 6 CPUs available

# latest alpine release
FROM alpine:3.18 AS build

RUN apk add \
cmake \
g++ \
git \
make \
python3 \
unzip \
boost-dev \
gmp-dev \
libarchive-dev \
libxml2-dev \
mpfr-dev \
openblas-dev \
openmpi \
openmpi-dev \
rapidjson-dev
WORKDIR /usr/local/src
# Build Elemental
RUN git clone https://gitlab.com/bootstrapcollaboration/elemental.git && \
cd elemental && \
mkdir -p build && \
cd build && \
cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_INSTALL_PREFIX=/usr/local/ && \
make && \
make install
# Build SDPB from current sources
COPY . sdpb
RUN cd sdpb && \
./waf configure --elemental-dir=/usr/local --prefix=/usr/local && \
python3 ./waf && \
python3 ./waf install

# Take only sdpb binaries + load necessary dynamic libraries
FROM alpine:3.18 as install
RUN apk add \
boost1.82-date_time \
boost1.82-filesystem \
boost1.82-iostreams \
boost1.82-program_options \
boost1.82-serialization \
boost1.82-system \
gmp \
libarchive \
libgmpxx \
libstdc++ \
libxml2 \
mpfr \
openblas \
openmpi \
openssh
COPY --from=build /usr/local/bin /usr/local/bin
COPY --from=build /usr/local/lib /usr/local/lib

# Separate test target, see
# https://docs.docker.com/language/java/run-tests/#multi-stage-dockerfile-for-testing
# Contains /home/testuser/sdpb/build and /home/testuser/sdpb/test folders,
# which is sufficient to run tests as shown below:
#
# docker build -t sdpb-test --target test
# docker run sdpb-test ./test/run_all_tests.sh mpirun --oversubscribe
FROM install as test
# Create testuser to run Docker non-root (and avoid 'mpirun --allow-run-as-root' warning)
RUN addgroup --gid 10000 testgroup && \
adduser --disabled-password --uid 10000 --ingroup testgroup --shell /bin/sh testuser
WORKDIR /home/testuser/sdpb
COPY --from=build /usr/local/src/sdpb/build build
COPY --from=build /usr/local/src/sdpb/test test
RUN chown -R testuser test
USER testuser:testgroup

# Resulting image
FROM install
# TODO best practices suggest to run containter as non-root.
# https://github.com/dnaprawa/dockerfile-best-practices#run-as-a-non-root-user
# But this requires some extra work with permissions when mounting folders for docker run.
2 changes: 1 addition & 1 deletion test/run_all_tests.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/bin/sh

if [[ $1 == "--help" ]]; then
echo "Pass custom mpirun command line as arguments to the testing script, e.g.:"
Expand Down