Skip to content

Commit

Permalink
Merge pull request #18 from tst-race/ci-builds
Browse files Browse the repository at this point in the history 
Ci builds
  • Loading branch information
@plvines
plvines authored Dec 28, 2024
2 parents fe9fddd + c283aa4 commit d70d902
Show file tree
Hide file tree
Showing 56 changed files with 434 additions and 303 deletions.
255 changes: 203 additions & 52 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,87 +9,238 @@ on:

permissions:
contents: write
packages: read
packages: write

jobs:
build:
builder-image:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Log in to Container Registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: 'arm64,arm'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Extract Docker Metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ghcr.io/${{ github.repository }}/raceboat-builder
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{raw}}
type=semver,pattern=v{{major}}.{{minor}}
type=semver,pattern=v{{major}}
- name: Configure Docker Caching
id: cache
uses: int128/docker-build-cache-config-action@v1
with:
image: ghcr.io/${{ github.repository }}/raceboat-builder/cache

- name: Build and Push raceboat-builder Docker Image
uses: docker/build-push-action@v4
with:
context: raceboat-builder-image
file: raceboat-builder-image/Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: ${{ steps.cache.outputs.cache-from }}
cache-to: ${{ steps.cache.outputs.cache-to }}

build-framework:
needs: [builder-image]
runs-on: ubuntu-latest
container:
image: ghcr.io/tst-race/race-images/race-compile:main
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
image: ghcr.io/tst-race/raceboat/raceboat-builder:${{ github.head_ref || github.ref_name }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Build
run: ./build_it_all.sh

- name: Build-pt
run: ./pluggable-transport/build.sh -p=build/LINUX_x86_64/language-shims/source/include/src/core
run: ./build.sh

- name: Create Build Artifact
# NOTE: _commsPluginBindings.so not in racesdk/package/
run: "tar cvf ${{ github.event.repository.name }}.tar.gz -C racesdk/package/ ."
run: "tar cvf ${{ github.event.repository.name }}.tar.gz -C /__w/raceboat/raceboat/ ./racesdk"

- name: Upload Build Artifact
uses: actions/upload-artifact@v3
with:
name: "${{ github.event.repository.name }}-linux-x86_64.tar.gz"
path: "${{ github.event.repository.name }}-linux-x86_64.tar.gz"
name: "${{ github.event.repository.name }}.tar.gz"
path: "${{ github.event.repository.name }}.tar.gz"
retention-days: 10

# todo base these on the raceboat image
test-unit:
# needs: build
compile-image:
needs: [build-framework]
runs-on: ubuntu-latest
container:
image: ghcr.io/tst-race/race-images/race-compile:main
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: unit test
- name: Checkout
uses: actions/checkout@v3

- name: Build and Run Unit Tests
# shuffle the order of the tests to potentially uncover weirdness
# note that you can reproduce the test results by finding the seed in the log output and
# setting the environment variable GTEST_RANDOM_SEED to that value.
env:
GTEST_SHUFFLE: 1
run: |
./build_it_all.sh
cmake --build --preset=LINUX_x86_64 --target run_tests -j
test-coverage:
# needs: build
- name: Download Framework Artifacts
uses: actions/download-artifact@v3
with:
name: "${{ github.event.repository.name }}.tar.gz"

- name: debug ls
run: ls -R .

- name: untar
run: tar xf ${{ github.event.repository.name }}.tar.gz

- name: debug ls
run: ls -R racesdk

- name: debu pwdg
run: pwd

- name: env
run: env

- name: Log in to Container Registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: 'arm64,arm'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Extract Docker Metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ghcr.io/${{ github.repository }}/raceboat-compile
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{raw}}
type=semver,pattern=v{{major}}.{{minor}}
type=semver,pattern=v{{major}}
- name: Configure Docker Caching
id: cache
uses: int128/docker-build-cache-config-action@v1
with:
image: ghcr.io/${{ github.repository }}/raceboat-compile/cache

- name: Build and Push raceboat-compile Docker Image
uses: docker/build-push-action@v4
with:
context: /home/runner/work/raceboat/raceboat
file: raceboat-compile-image/Dockerfile
platforms: linux/amd64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: ${{ steps.cache.outputs.cache-from }}
cache-to: ${{ steps.cache.outputs.cache-to }}

run-tests:
needs: [compile-image]
runs-on: ubuntu-latest
container:
image: ghcr.io/tst-race/race-images/race-compile:main
image: ghcr.io/tst-race/raceboat/raceboat-compile:${{ github.head_ref || github.ref_name }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Build and Run Unit Tests
# shuffle the order of the tests to potentially uncover weirdness
# note that you can reproduce the test results by finding the seed in the log output and
# setting the environment variable GTEST_RANDOM_SEED to that value.
env:
GTEST_SHUFFLE: 1
run: |
cmake --preset=coverage
cmake --build --preset=coverage --target coverage
test-format:
# needs: build
- name: Test
run: ./test.sh

runtime-image:
needs: [compile-image]
runs-on: ubuntu-latest
container:
image: ghcr.io/tst-race/race-images/race-compile:main
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Check Format
run: |
./build_it_all.sh
cmake --build --preset=LINUX_x86_64 --target check_format -j
- name: Download Framework Artifacts
uses: actions/download-artifact@v3
with:
name: "${{ github.event.repository.name }}.tar.gz"

- name: debug ls
run: ls -R .

- name: untar
run: tar xf ${{ github.event.repository.name }}.tar.gz

- name: debug ls
run: ls -R racesdk

- name: debu pwdg
run: pwd

- name: env
run: env

- name: Log in to Container Registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: 'arm64,arm'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Extract Docker Metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ghcr.io/${{ github.repository }}/raceboat-runtime
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{raw}}
type=semver,pattern=v{{major}}.{{minor}}
type=semver,pattern=v{{major}}
- name: Configure Docker Caching
id: cache
uses: int128/docker-build-cache-config-action@v1
with:
image: ghcr.io/${{ github.repository }}/raceboat-runtime/cache

- name: Build and Push raceboat-runtime Docker Image
uses: docker/build-push-action@v4
with:
context: /home/runner/work/raceboat/raceboat
file: raceboat-runtime-image/Dockerfile
platforms: linux/amd64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: ${{ steps.cache.outputs.cache-from }}
cache-to: ${{ steps.cache.outputs.cache-to }}
build-contexts: ghcr.io/tst-race/raceboat/raceboat-compile=docker-image://ghcr.io/tst-race/raceboat/raceboat-compile:${{ steps.meta.outputs.version }}
11 changes: 9 additions & 2 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ include(race/test-targets)
include(race/valgrind)
include(race/warnings)

# setup_project_test_targets(raceSdkCommon UNIT INTEGRATION)
# setup_project_format_targets(raceSdkCommon)
setup_project_test_targets(raceSdkCommon UNIT INTEGRATION)
setup_project_format_targets(raceSdkCommon)

add_subdirectory(language-shims)
add_subdirectory(source)
Expand All @@ -54,6 +54,12 @@ if(NOT ANDROID)
add_subdirectory(test/common/DecomposedTestStub EXCLUDE_FROM_ALL)
add_subdirectory(test/common/plugins/DecomposedTestImplementation EXCLUDE_FROM_ALL)
add_subdirectory(test/source EXCLUDE_FROM_ALL)
install(
TARGETS
race-cli
bridge-distro
RUNTIME DESTINATION app
)
endif()

# Create this after all subdirectories have been added
Expand All @@ -63,6 +69,7 @@ install(
TARGETS
raceSdkCommon
EXPORT raceSdkCommon
RUNTIME DESTINATION app
LIBRARY DESTINATION lib
PUBLIC_HEADER DESTINATION include
)
Expand Down
21 changes: 19 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,32 @@ Raceboat provides two different interfaces for censorship circumvention channels


## **Building**
Raceboat uses a docker-based build process, running the below may take some time to download the compilation image. This build command also creates a `raceboat` docker image with the shared objects and executable built into it for easy testing and execution. To build a x86_64 image:
Raceboat uses a docker-based build process, running the below may take some time to download the compilation image. This build command also creates a `raceboat` runtumie docker image with the shared objects and executable built into it for easy testing and execution. **These are all built in GitHub CI and available from the `ghcr.io/tst-race/raceboat` namespace.

- `ghcr.io/tst-race/raceboat/raceboat-builder:main`: an image for building the raceboat framework itself (i.e. the code in this repository)

- `ghcr.io/tst-race/raceboat/raceboat-compile:main`: extension of the raceboat-builder with prebuilt framework binaries, used to compile plugins that provide channels or components for raceboat to use.

- `ghcr.io/tst-race/raceboat/raceboat:main`: runtime image which contains both prebuilt framework binaries and runtime dependencies.

Use the following commands to build a local set of x86_64 images:

```bash
pushd raceboat-builder-image && \
./build_image.sh -n ghcr.io/tst-race/raceboat --platform-x86_64 && \
popd && \
docker run -it --rm --name=build-pt \
-e MAKEFLAGS="-j" \
-v $(pwd)/:/code/ \
-w /code \
raceboat-builder:latest \
./build.sh && docker-image/build_image.sh
./build.sh && \
pushd raceboat-compile-image && \
./build_image.sh -n ghcr.io/tst-race/raceboat --platform-x86_64 && \
popd && \
pushd runtime-image && \
./build_image.sh -n ghcr.io/tst-race/raceboat --platform-x86_64 && \
popd
```

To build an aarch64 raceboat image locally:
Expand Down
Loading

0 comments on commit d70d902

Please sign in to comment.