Skip to content

Commit

Permalink
Add workflow for comparing schemas for changes
Browse files Browse the repository at this point in the history
  • Loading branch information
JeremyMcCormick committed Dec 5, 2024
1 parent afec504 commit 6347c59
Show file tree
Hide file tree
Showing 2 changed files with 99 additions and 0 deletions.
95 changes: 95 additions & 0 deletions .github/workflows/compare.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: Compare Schemas for Changes

on:
push:
branches:
- main
tags:
- "*"
pull_request:

jobs:
compare:
runs-on: ubuntu-latest

steps:
- name: Check out repo
uses: actions/checkout@v4
with:
path: current-ref

- name: Fetch origin main
working-directory: current-ref
run: git fetch origin main

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"

- name: Install dependencies
working-directory: current-ref
run: |
python -m pip install --upgrade pip uv
uv pip install --system -r requirements.txt
- name: Set relative path to schema directory
run: |
echo "SCHEMA_DIR=python/lsst/sdm_schemas/schemas" >> $GITHUB_ENV
- name: Get list of changed YAML files
working-directory: current-ref
run: |
CHANGED_FILES=$(git diff --name-only origin/main..HEAD -- ${{ env.SCHEMA_DIR }} | sed "s|^${{ env.SCHEMA_DIR }}/||")
echo "Changed YAML files: $CHANGED_FILES"
echo "CHANGED_FILES=$CHANGED_FILES" >> $GITHUB_ENV
- name: Check out main branch
uses: actions/checkout@v4
with:
ref: main
path: main-branch

- name: Run schema comparison with deepdiff
run: |
for file in ${{ env.CHANGED_FILES }}; do
echo "Comparing $file:"
MAIN_FILE=main-branch/${{ env.SCHEMA_DIR }}/$file
CURRENT_FILE=current-ref/${{ env.SCHEMA_DIR }}/$file
felis --log-level ERROR diff -c deepdiff $MAIN_FILE $CURRENT_FILE
done
- name: Run schema comparison with alembic
run: |
for file in ${{ env.CHANGED_FILES }}; do
echo "Comparing $file:"
MAIN_FILE=main-branch/${{ env.SCHEMA_DIR }}/$file
CURRENT_FILE=current-ref/${{ env.SCHEMA_DIR }}/$file
felis --log-level ERROR diff -c alembic $MAIN_FILE $CURRENT_FILE
done
- name: Run schema comparison on deployed schemas
run: |
set -e
set -x
ERROR_FLAG=0
DEPLOYED_SCHEMAS=$(cat current-ref/yml/DEPLOYED.txt)
echo "Checking deployed schemas: $DEPLOYED_SCHEMAS"
for file in ${{ env.CHANGED_FILES }}; do
if echo "$DEPLOYED_SCHEMAS" | grep -q "^$file$"; then
echo "Comparing $file:"
MAIN_FILE=main-branch/${{ env.SCHEMA_DIR }}/$file
CURRENT_FILE=current-ref/${{ env.SCHEMA_DIR }}/$file
if ! felis --log-level ERROR diff -E -c alembic $MAIN_FILE $CURRENT_FILE; then
echo "Error comparing $file"
ERROR_FLAG=1
fi
else
echo "Skipping $file (not in DEPLOYED.txt)"
fi
done
if [ $ERROR_FLAG -ne 0 ]; then
echo "One or more schemas was changed."
exit 1
fi
4 changes: 4 additions & 0 deletions python/lsst/sdm_schemas/schemas/DEPLOYED.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
dp02_dc2.yaml
dp02_obscore.yaml
dp03_10yr.yaml
dp03_1yr.yaml

0 comments on commit 6347c59

Please sign in to comment.