-
Notifications
You must be signed in to change notification settings - Fork 1
Home
Many applications maintain persistent stored data in some kind of database. To be useful to the application, that data is kept in a certain, (explicit or implicit), structure. As the application evolves, the structure of its stored data needs to evolve with it. In addition to the structures used to hold data, many data stores hold "stored code definitions"— the definitions of executable objects such as views, functions and stored procedures. dotMigrator is a tool to help manage these data structure changes and stored objects in an automated way during application development, testing, and release. dotMigrator is extensible to be able to manage all kinds of data stores and migrations written in all kinds of forms.
Data structure migrations and stored code definitions should be kept in source control along with the application's source code. Migrations and stored code definitions can be written as scripts (such as SQL scripts) or executable code.
Migrations often need to run while the application is offline, but sometimes migrations can be written to work while the application stays online. dotMigrator will integrate into your release process and tools to safely manage both types.
Stored code definitions can be dropped and re-created or replaced without impact to the stored data. dotMigrator can track these definitions and run scripts or code to revise these objects when their definitions have changed.
For development and testing, dotMigrator can also run a sequence of scripts or code to populate the application's database with test data. These test data scripts can be kept in source control with the application code, migrations, and stored code definitions and dotMigrator can be instructed to run them when re-creating a database.
Each target data store that will be managed with dotMigrator is expected to have an associated journal of migrations that have been applied. This journal is consulted when dotMigrator runs to determine which migrations and stored code definitions, if any, need to be applied to bring the data structure up to the version matching the application being deployed. By tracking the migrations that have already been applied to a given data store, dotMigrator will also detect and prevent an attempt to deploy an incompatible version.
dotMigrator can determine if an offline migration is necessary before making any changes so that your deployment script can bring the application down and then back up.
When dotMigrator runs offline, it will first check that all prior migrations have completed, then run the offline migrations one at a time in their defined order, followed by applying the changed stored object definitions. If there are any online migrations, they must follow all of the offline migrations and will not be run at this point.
If an offline migration fails, dotMigrator will not continue. The database is assumed to be broken and will need to be repaired before dotMigrator will run again.
It might be repaired by restoring from a backup, or by manual intervention. If it is corrected by manual intervention, the associated migration journal will need to be updated to reflect that the the migration was either backed-out, or completed. This way, dotMigrator will know what it needs to apply when it is run again.
Sometimes a data migration can be written such that it can run while the application remains online. It is especially useful to write migrations that can run online when they take a long time to complete.
An online migration must be "resumable" meaning that if it fails or is interrupted dotMigrator can restart it, allowing it to recover itself and then complete. The migration must be written carefully to anticipate a potential failure or interruption at any point and be able to continue from where it left off when it restarts.
When dotMigrator runs online, it will first check that all offline migrations have completed, then run the online migrations one at a time in their defined order.
These stored code objects are logically part of your application's executable code, so replacing them is only done when dotMigrator runs offline.
dotMigrator will compare the current definition with the previously-run definition to determine if it needs to be replaced.
dotMigrator will try to prevent you from migrating from an incompatible version of your database. If dotMigrator detects that the set of previously-completed migrations does not match the list of known migrations, it will not run any migrations. This could easily happen when deploying builds from different source code branches to the same target environment. A migration that is included in one branch may not be included in the other and vice-versa. dotMigrator cannot "know" that the deployment would be safe, nor can it know the correct order to run them until the migrations are merged into a single branch.
Even if there are not any migrations that need to run, dotMigrator will inform you if the target database has already been upgraded to a potentially incompatible version.
dotMigrator will also try to ensure that whether you build up a database from scratch or migrate from an older version, you'll end up with the same database structure. If dotMigrator detects that a previously-completed migration's code has been changed, it will not run any migrations.
With inspiration from:
Can also be compared with
- RoundHousE
- Several other similar tools