Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Introduce a target version up to which migrations should be applied. #1544

Merged
merged 4 commits into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions docs/modules/ROOT/pages/usage.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,20 @@ You can spot these cases beforehand by validating your chain of migrations (see
If you absolutely must however, you can use `withOutOfOrderAllowed` on the config object or the corresponding property in either the Spring Boot starter or the Quarkus extension.
Setting this to true will integrate out-of-order migrations into the chain.

By default, all migrations will be applied.
However, you can configure a target version to stop at.
The target version will always be an *inclusive* stop.
This is especially important for repeatable migrations: If the target is repeatable and has changed since the last application, it will be applied again.
The target version must be an exact version number as given with the file- or classname (such as `V1_2_3`, `R010`, `V100` or whatever scheme you use), specifying the description again is not necessary (See ref:concepts.adoc#concepts_naming-conventions[naming conventions]).
Migrations does check whether the target version can be resolved and will abort if there is no such version, either in pending or applied state).
You can use `V010?` (with a question mark) to skip this check.
Migrations will then apply every version that is sorted below that version.
Additionally, three special values are supported as well (These names are case-insensitive):

`current`:: Designates the current version of the schema.
`latest`:: The latest version of the schema, as defined by the migration with the highest version.
`next`:: The next version of the schema, as defined by the first pending migration.

[[usage_common_repair]]
=== Repair

Expand Down Expand Up @@ -683,12 +697,23 @@ A configurable delay that will be applied in between applying two migrations.
- Type: `java.time.Duration`
- Default: `false`

`org.neo4j.migrations.version-sort-order`::
How versions should be sorted. The default of `LEXICOGRAPHIC` will change to `SEMANTIC` in a future version of this library.

- Type: `ac.simons.neo4j.migrations.core.MigrationsConfig#VersionSortOrder`
- Default: `LEXICOGRAPHIC`

`org.neo4j.migrations.out-of-order`::
A flag to enable out-of-order migrations.

- Type: `java.lang.Boolean`
- Default: `false`

`org.neo4j.migrations.target`::
Configures the target version up to which migrations should be considered. This must be a valid migration version, or one of the special values `current`, `latest` or `next`.

- Type: `java.lang.String`
- Default: `null`

NOTE: Migrations can be disabled by setting `org.neo4j.migrations.enabled` to `false`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-annotation-processing</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
</parent>

<artifactId>neo4j-migrations-annotation-catalog</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion extensions/neo4j-migrations-annotation-processing/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-annotation-processing</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
</parent>

<artifactId>neo4j-migrations-annotation-processor-api</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-annotation-processing</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
</parent>

<artifactId>neo4j-migrations-annotation-processor</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion extensions/neo4j-migrations-formats-adoc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion extensions/neo4j-migrations-formats-csv/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion extensions/neo4j-migrations-formats-markdown/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion neo4j-migrations-bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
</parent>

<artifactId>neo4j-migrations-bom</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion neo4j-migrations-cli/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
</parent>

<artifactId>neo4j-migrations-cli</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,12 @@ public static void main(String... args) {
)
private boolean outOfOrder;

@Option(
names = {"--target"},
description = "Use this option to specify a valid target version up to which migrations should be considered. Can also be one of current, latest or next."
)
private String target;

@Spec
private CommandSpec commandSpec;

Expand Down Expand Up @@ -306,6 +312,7 @@ MigrationsConfig getConfig(boolean forceSilence) {
.withDelayBetweenMigrations(delayBetweenMigrations)
.withVersionSortOrder(versionSortOrder)
.withOutOfOrderAllowed(outOfOrder)
.withTarget(target)
.build();

if (!forceSilence) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,26 @@ void outOfOrderShouldBeApplied() {
assertThat(cli.getConfig().isOutOfOrder()).isTrue();
}

@Test // GH-1536
void targetShouldBeNullByDefault() {

MigrationsCli cli = new MigrationsCli();
CommandLine commandLine = new CommandLine(cli);
commandLine.parseArgs();

assertThat(cli.getConfig().getTarget()).isNull();
}

@Test // GH-1536
void targetShouldBeApplied() {

MigrationsCli cli = new MigrationsCli();
CommandLine commandLine = new CommandLine(cli);
commandLine.parseArgs("--target=current");

assertThat(cli.getConfig().getTarget()).isEqualTo("current");
}

@Test
void shouldRequire2Connections() {

Expand Down
2 changes: 1 addition & 1 deletion neo4j-migrations-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-parent</artifactId>
<version>2.14.2-SNAPSHOT</version>
<version>2.15.0-SNAPSHOT</version>
</parent>

<artifactId>neo4j-migrations</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,34 @@ public Collection<Element> getElements() {
return this.elements.values();
}

@Override
public Optional<MigrationVersion> findTargetVersion(MigrationVersion.TargetVersion targetVersion) {

if (this.elements.isEmpty()) {
return Optional.empty();
}

return switch (targetVersion) {
case CURRENT -> getLastAppliedVersion();
case LATEST -> this.elements.keySet().stream().skip(this.elements.size() - 1L).findFirst();
case NEXT ->
this.elements.entrySet().stream().dropWhile(e -> e.getValue().getState() == MigrationState.APPLIED)
.map(Map.Entry::getKey)
.findFirst();
};
}

@Override
public Optional<MigrationVersion> getLastAppliedVersion() {
Iterator<MigrationVersion> it = this.elements.keySet().iterator();
Iterator<Map.Entry<MigrationVersion, Element>> it = this.elements.entrySet().iterator();
MigrationVersion version = null;
while (it.hasNext()) {
version = it.next();
var next = it.next();
if (next.getValue().getState() == MigrationState.APPLIED) {
version = next.getKey();
} else {
break;
}
}
return Optional.ofNullable(version);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,13 @@
package ac.simons.neo4j.migrations.core;

import java.time.Duration;
import java.util.Comparator;
import java.util.Iterator;
import java.util.List;
import java.util.NoSuchElementException;
import java.util.logging.Level;

import ac.simons.neo4j.migrations.core.MigrationVersion.StopVersion;

/**
* A helper class that can be used to delay the iteration of migrations by a configurable amount of time.
Expand All @@ -31,42 +36,85 @@ final class IterableMigrations implements Iterable<Migration> {

private final List<Migration> migrations;

IterableMigrations(MigrationsConfig config, List<Migration> migrations) {
private final MigrationVersion optionalStop;

static IterableMigrations of(MigrationsConfig config, List<Migration> migrations) {
return of(config, migrations, null);
}

static IterableMigrations of(MigrationsConfig config, List<Migration> migrations, StopVersion stopVersion) {
MigrationVersion optionalStop;
if (stopVersion == null) {
optionalStop = null;
} else {
optionalStop = stopVersion.version();
if (!stopVersion.optional() && migrations.stream().filter(m -> m.getVersion().equals(optionalStop)).findFirst().isEmpty()) {
throw new MigrationsException("Target version %s is not available".formatted(optionalStop.getValue()));
}
}
return new IterableMigrations(config, migrations, optionalStop);
}

private IterableMigrations(MigrationsConfig config, List<Migration> migrations, MigrationVersion optionalStop) {
this.config = config;
this.migrations = migrations;
this.optionalStop = optionalStop;
}

@Override
public Iterator<Migration> iterator() {
var iterator = migrations.iterator();
return config.getOptionalDelayBetweenMigrations().map(d -> (Iterator<Migration>) new DelayingIterator(iterator, d))
.orElse(iterator);
if (optionalStop != null) {
Migrations.LOGGER.log(Level.INFO, "Will stop at target version {0}", optionalStop);
}
return new DelayingIterator(iterator, config.getOptionalDelayBetweenMigrations().orElse(null), config.getVersionComparator(), optionalStop);
}

private static final class DelayingIterator implements Iterator<Migration> {

private final Iterator<Migration> delegate;

private final Duration delay;
private final Duration optionalDelay;

private final Comparator<MigrationVersion> comparator;

private final MigrationVersion optionalStop;

private Migration next;

DelayingIterator(Iterator<Migration> delegate, Duration delay) {
DelayingIterator(Iterator<Migration> delegate, Duration optionalDelay, Comparator<MigrationVersion> comparator, MigrationVersion optionalStop) {
this.delegate = delegate;
this.delay = delay;
this.optionalDelay = optionalDelay;
this.comparator = comparator;
this.optionalStop = optionalStop;
}

@Override
public boolean hasNext() {
return delegate.hasNext();
var hasNext = delegate.hasNext();
if (hasNext) {
this.next = delegate.next();
hasNext = this.optionalStop == null || comparator.compare(next.getVersion(), optionalStop) <= 0;
} else {
this.next = null;
}
return hasNext;
}

@Override
public Migration next() {
try {
Thread.sleep(delay.toMillis());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
public Migration next() throws NoSuchElementException {
if (next == null) {
throw new NoSuchElementException();
}

if (optionalDelay != null) {
try {
Thread.sleep(optionalDelay.toMillis());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
return delegate.next();
return next;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
import java.util.Map;
import java.util.Optional;

import ac.simons.neo4j.migrations.core.MigrationVersion.TargetVersion;

/**
* Public information about an applied migration. All migrations (applied and pending) form a chain of transformations
* to a database. The chain starts implicit with a baseline version. The baseline version is not contained in this chain.
Expand Down Expand Up @@ -114,6 +116,15 @@ default Optional<MigrationVersion> getLastAppliedVersion() {
return Optional.empty();
}

/**
* Translates a target version into a concrete version
*
* @param targetVersion the target version to translate into a concrete version
* @return the concrete version if any
* @since 2.15.0
*/
Optional<MigrationVersion> findTargetVersion(TargetVersion targetVersion);

/**
* A chain element describing a pending or applied migration.
*/
Expand Down
Loading
Loading