Skip to content

Commit

Permalink
Adopt sqlx migrate (#1221)
Browse files Browse the repository at this point in the history
We now use `sqlx` ([1]) and its CLI ([2]) to manage our SQL schema and
migrate it to new versions. The schema is now represented as a series of
SQL scripts in `db/`, along with teardown scripts to enable rollbacks.
The objective here is to enable easy migrations, namely:

 - adding new tables
 - adding indices to tables
 - adding columns with default values to tables

More complex migrations which transform or delete data may require more
sophisticated tools than what's here, and even these "easy" migrations
require some planning and operational tools that aren't present in this
repository.

This commit contains these changes:

 - the `publish-sql-schema` workflow now publishes all migration scripts
   in `db` to a Google Cloud Storage bucket to later be consumed by
   Divvi Up ops tools
 - the interop test containers use `sqlx::migrate::Migrator` instead of
   `janus_cli` to write the schema into their database
 - `janus_aggregator_core::datastore::test_util::ephemeral_dataastore`
   uses `sqlx::migrate::Migrator` to apply migrations to ephemeral
   databases
 - `db/schema.sql` is copied to a `sqlx` migration script, and a
   corresponding `down` script is added which deletes tables, indices,
   etc.

[1]: https://crates.io/crates/sqlx
[2]: https://crates.io/crates/sqlx-cli

Part of #30
  • Loading branch information
tgeoghegan authored Apr 10, 2023
1 parent a109d25 commit c32b581
Show file tree
Hide file tree
Showing 14 changed files with 570 additions and 46 deletions.
5 changes: 4 additions & 1 deletion .github/workflows/publish-sql-schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,7 @@ jobs:
id: get_version
run: echo VERSION=${GITHUB_REF/refs\/tags\//} >> $GITHUB_OUTPUT
- name: "Upload schema file(s)"
run: gcloud alpha storage cp db/schema.sql gs://janus-artifacts-sql-schemas/${{ steps.get_version.outputs.VERSION }}/schema.sql
run: |-
gcloud alpha storage cp --recursive \
db \
gs://janus-artifacts-sql-schemas/${{ steps.get_version.outputs.VERSION }}/
Loading

0 comments on commit c32b581

Please sign in to comment.