Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scylladb #8002

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ body:
- QuestDB
- RabbitMQ
- Redpanda
- ScyllaDB
- Selenium
- Solace
- Solr
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/enhancement.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ body:
- QuestDB
- RabbitMQ
- Redpanda
- ScyllaDB
- Selenium
- Solace
- Solr
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ body:
- Pulsar
- RabbitMQ
- Redpanda
- ScyllaDB
- Selenium
- Solace
- Solr
Expand Down
5 changes: 5 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,11 @@ updates:
schedule:
interval: "weekly"
open-pull-requests-limit: 10
- package-ecosystem: "gradle"
directory: "/modules/scylladb"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
- package-ecosystem: "gradle"
directory: "/modules/selenium"
schedule:
Expand Down
4 changes: 4 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,10 @@
- changed-files:
- any-glob-to-any-file:
- modules/redpanda/**/*
"modules/scylladb":
- changed-files:
- any-glob-to-any-file:
- modules/scylladb/**/*
"modules/selenium":
- changed-files:
- any-glob-to-any-file:
Expand Down
50 changes: 50 additions & 0 deletions docs/modules/databases/scylladb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# ScyllaDB

Testcontainers module for [ScyllaDB](https://hub.docker.com/r/scylladb/scylla)

## ScyllaDB's usage examples

You can start a ScyllaDB container instance from any Java application by using:

<!--codeinclude-->
[Create container](../../../modules/scylladb/src/test/java/org/testcontainers/scylladb/ScyllaDBContainerTest.java) inside_block:container
<!--/codeinclude-->

### Building CqlSession

<!--codeinclude-->
[Using CQL port](../../../modules/scylladb/src/test/java/org/testcontainers/scylladb/ScyllaDBContainerTest.java) inside_block:session
<!--/codeinclude-->

<!--codeinclude-->
[Using Shard Awareness port](../../../modules/scylladb/src/test/java/org/testcontainers/scylladb/ScyllaDBContainerTest.java) inside_block:shardAwarenessSession
<!--/codeinclude-->

### Alternator

<!--codeinclude-->
[Enabling Alternator](../../../modules/scylladb/src/test/java/org/testcontainers/scylladb/ScyllaDBContainerTest.java) inside_block:alternator
<!--/codeinclude-->

<!--codeinclude-->
[DynamoDbClient with Alternator](../../../modules/scylladb/src/test/java/org/testcontainers/scylladb/ScyllaDBContainerTest.java) inside_block:dynamodDbClient
<!--/codeinclude-->

## Adding this module to your project dependencies

Add the following dependency to your `pom.xml`/`build.gradle` file:

=== "Gradle"
```groovy
testImplementation "org.testcontainers:scylladb:{{latest_version}}"
```

=== "Maven"
```xml
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>scylladb</artifactId>
<version>{{latest_version}}</version>
<scope>test</scope>
</dependency>
```
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ nav:
- modules/databases/postgres.md
- modules/databases/presto.md
- modules/databases/questdb.md
- modules/databases/scylladb.md
- modules/databases/tidb.md
- modules/databases/timeplus.md
- modules/databases/trino.md
Expand Down
9 changes: 9 additions & 0 deletions modules/scylladb/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
description = "Testcontainers :: ScyllaDB"

dependencies {
api project(":database-commons")
api "com.scylladb:java-driver-core:4.15.0.0"

testImplementation 'org.assertj:assertj-core:3.24.2'
testImplementation 'software.amazon.awssdk:dynamodb:2.28.6'
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package org.testcontainers.scylladb;

import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.utility.DockerImageName;

import java.net.InetSocketAddress;

/**
* Testcontainers implementation for ScyllaDB.
* <p>
* Supported image: {@code scylladb/scylla}
* <p>
* Exposed ports:
* <ul>
* <li>CQL Port: 9042</li>
* <li>Shard Aware Port: 19042</li>
* <li>Alternator Port: 8000</li>
* </ul>
*/
public class ScyllaDBContainer extends GenericContainer<ScyllaDBContainer> {

private static final DockerImageName DEFAULT_IMAGE_NAME = DockerImageName.parse("scylladb/scylla");

private static final Integer CQL_PORT = 9042;

private static final Integer SHARD_AWARE_PORT = 19042;

private static final Integer ALTERNATOR_PORT = 8000;

private static final String COMMAND = "--developer-mode=1 --overprovisioned=1";

private boolean alternatorEnabled = false;

public ScyllaDBContainer(String dockerImageName) {
this(DockerImageName.parse(dockerImageName));
}

public ScyllaDBContainer(DockerImageName dockerImageName) {
super(dockerImageName);
dockerImageName.assertCompatibleWith(DEFAULT_IMAGE_NAME);

withExposedPorts(CQL_PORT, SHARD_AWARE_PORT);

withCommand(COMMAND);
waitingFor(Wait.forLogMessage(".*initialization completed..*", 1));
}

@Override
protected void configure() {
if (this.alternatorEnabled) {
addExposedPort(8000);
String newCommand =
COMMAND + " --alternator-port=" + ALTERNATOR_PORT + " --alternator-write-isolation=always";
withCommand(newCommand);
}
}

public ScyllaDBContainer withAlternator() {
this.alternatorEnabled = true;
return this;
}

/**
* Retrieve an {@link InetSocketAddress} for connecting to the ScyllaDB container via the driver.
*
* @return A InetSocketAddress representation of this ScyllaDB container's host and port.
*/
public InetSocketAddress getContactPoint() {
return new InetSocketAddress(getHost(), getMappedPort(CQL_PORT));
}

public InetSocketAddress getShardAwareContactPoint() {
return new InetSocketAddress(getHost(), getMappedPort(SHARD_AWARE_PORT));
}

public String getAlternatorEndpoint() {
if (!this.alternatorEnabled) {
throw new IllegalStateException("Alternator is not enabled");
}
return "http://" + getHost() + ":" + getMappedPort(ALTERNATOR_PORT);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
package org.testcontainers.scylladb;

import com.datastax.oss.driver.api.core.CqlSession;
import com.datastax.oss.driver.api.core.cql.ResultSet;
import org.junit.Test;
import org.testcontainers.utility.DockerImageName;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.AttributeDefinition;
import software.amazon.awssdk.services.dynamodb.model.BillingMode;
import software.amazon.awssdk.services.dynamodb.model.CreateTableRequest;
import software.amazon.awssdk.services.dynamodb.model.KeySchemaElement;
import software.amazon.awssdk.services.dynamodb.model.KeyType;
import software.amazon.awssdk.services.dynamodb.model.ScalarAttributeType;

import java.net.URI;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;

public class ScyllaDBContainerTest {

private static final DockerImageName SCYLLADB_IMAGE = DockerImageName.parse("scylladb/scylla:5.2.9");

private static final String BASIC_QUERY = "SELECT release_version FROM system.local";

@Test
public void testSimple() {
try ( // container {
ScyllaDBContainer scylladb = new ScyllaDBContainer("scylladb/scylla:5.2.9")
// }
) {
scylladb.start();
// session {
CqlSession session = CqlSession
.builder()
.addContactPoint(scylladb.getContactPoint())
.withLocalDatacenter("datacenter1")
.build();
// }
ResultSet resultSet = session.execute(BASIC_QUERY);
assertThat(resultSet.wasApplied()).isTrue();
assertThat(resultSet.one().getString(0)).isNotNull();
assertThat(session.getMetadata().getNodes().values()).hasSize(1);
}
}

@Test
public void testShardAwareness() {
try (ScyllaDBContainer scylladb = new ScyllaDBContainer(SCYLLADB_IMAGE)) {
scylladb.start();
// shardAwarenessSession {
CqlSession session = CqlSession
.builder()
.addContactPoint(scylladb.getShardAwareContactPoint())
.withLocalDatacenter("datacenter1")
.build();
// }
ResultSet resultSet = session.execute("SELECT driver_name FROM system.clients");
assertThat(resultSet.one().getString(0)).isNotNull();
assertThat(session.getMetadata().getNodes().values()).hasSize(1);
}
}

@Test
public void testAlternator() {
try ( // alternator {
ScyllaDBContainer scylladb = new ScyllaDBContainer(SCYLLADB_IMAGE).withAlternator()
// }
) {
scylladb.start();

// dynamodDbClient {
DynamoDbClient client = DynamoDbClient
.builder()
.endpointOverride(URI.create(scylladb.getAlternatorEndpoint()))
.credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("test", "test")))
.region(Region.US_EAST_1)
.build();
// }
client.createTable(
CreateTableRequest
.builder()
.tableName("demo_table")
.keySchema(KeySchemaElement.builder().attributeName("id").keyType(KeyType.HASH).build())
.attributeDefinitions(
AttributeDefinition.builder().attributeName("id").attributeType(ScalarAttributeType.S).build()
)
.billingMode(BillingMode.PAY_PER_REQUEST)
.build()
);
assertThat(client.listTables().tableNames()).containsExactly(("demo_table"));
}
}

@Test
public void throwExceptionWhenAlternatorDisabled() {
try (ScyllaDBContainer scylladb = new ScyllaDBContainer(SCYLLADB_IMAGE)) {
scylladb.start();
assertThatThrownBy(scylladb::getAlternatorEndpoint)
.isInstanceOf(IllegalStateException.class)
.hasMessageContaining("Alternator is not enabled");
}
}
}
7 changes: 7 additions & 0 deletions modules/scylladb/src/test/resources/initial.cql
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
CREATE KEYSPACE keySpaceTest WITH replication = {'class': 'SimpleStrategy', 'replication_factor' : 1};

USE keySpaceTest;

CREATE TABLE catalog_category (id bigint primary key, name text);

INSERT INTO catalog_category (id, name) VALUES (1, 'test_category');
16 changes: 16 additions & 0 deletions modules/scylladb/src/test/resources/logback-test.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<configuration>

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} %-5level %logger - %msg%n</pattern>
</encoder>
</appender>

<root level="INFO">
<appender-ref ref="STDOUT"/>
</root>

<logger name="org.testcontainers" level="INFO"/>
</configuration>
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# ScyllaDB storage config YAML

# NOTE:
# See https://opensource.docs.scylladb.com/stable/operating-scylla/admin.html for
# full explanations of configuration directives
# /NOTE

# The name of the cluster. This is mainly used to prevent machines in
# one logical cluster from joining another.
cluster_name: 'Test Cluster Integration Test'

# This defines the number of tokens randomly assigned to this node on the ring
# The more tokens, relative to other nodes, the larger the proportion of data
# that this node will store. You probably want all nodes to have the same number
# of tokens assuming they have equal hardware capability.
num_tokens: 256

# Directory where Scylla should store data on disk.
data_file_directories:
- /var/lib/scylla/data

# commit log. when running on magnetic HDD, this should be a
# separate spindle than the data directories.
commitlog_directory: /var/lib/scylla/commitlog

# schema commit log. A special commitlog instance
# used for schema and system tables.
# When running on magnetic HDD, this should be a
# separate spindle than the data directories.
# schema_commitlog_directory: /var/lib/scylla/commitlog/schema

# seed_provider class_name is saved for future use.
# A seed address is mandatory.
seed_provider:
# The addresses of hosts that will serve as contact points for the joining node.
# It allows the node to discover the cluster ring topology on startup (when
# joining the cluster).
# Once the node has joined the cluster, the seed list has no function.
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
# In a new cluster, provide the address of the first node.
# In an existing cluster, specify the address of at least one existing node.
# If you specify addresses of more than one node, use a comma to separate them.
# For example: "<IP1>,<IP2>,<IP3>"
- seeds: "127.0.0.1"

# Address or interface to bind to and tell other Scylla nodes to connect to.
# You _must_ change this if you want multiple nodes to be able to communicate!
#
# Setting listen_address to 0.0.0.0 is always wrong.
listen_address: localhost

# Address to broadcast to other Scylla nodes
# Leaving this blank will set it to the same value as listen_address
# broadcast_address: 1.2.3.4

# port for the CQL native transport to listen for clients on
# For security reasons, you should not expose this port to the internet. Firewall it if needed.
native_transport_port: 9042

# Uncomment to enable experimental features
# experimental: true
Loading