Skip to content

Commit

Permalink
Merge pull request #38 from SolaceDev/add-releasing
Browse files Browse the repository at this point in the history
* Add tasks to release the connector to Maven Central
* Upgrade log4j to 2.16.0 in tests to fix CVE-2021-45046 risks
  • Loading branch information
Nephery authored Dec 17, 2021
2 parents 45bfdf2 + 9458724 commit 2c6aabe
Show file tree
Hide file tree
Showing 9 changed files with 541 additions and 26 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
/build/*

# Jave related
target/**
/target/
*.jar
*.war
*.ear
Expand Down
59 changes: 39 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,43 +319,62 @@ Kerberos has some very specific requirements to operate correctly. Some addition

JDK 8 or higher is required for this project.

First, clone this GitHub repo:
```shell
git clone https://github.com/SolaceProducts/pubsubplus-connector-kafka-source.git
cd pubsubplus-connector-kafka-source
```

Then run the build script:
```shell
gradlew clean build
```
1. First, clone this GitHub repo:
```shell
git clone https://github.com/SolaceProducts/pubsubplus-connector-kafka-source.git
cd pubsubplus-connector-kafka-source
```
2. Install the test support module:
```shell
git submodule update --init --recursive
cd solace-integration-test-support
./mvnw clean install -DskipTests
cd ..
```
3. Then run the build script:
```shell
gradlew clean build
```

This script creates artifacts in the `build` directory, including the deployable packaged PubSub+ Source Connector archives under `build\distributions`.

### Test the Project

An integration test suite is also included, which spins up a Docker-based deployment environment that includes a PubSub+ event broker, Zookeeper, Kafka broker, Kafka Connect. It deploys the connector to Kafka Connect and runs end-to-end tests.

1. Install the test support module:
```shell
git submodule update --init --recursive
cd solace-integration-test-support
./mvnw clean install -DskipTests
cd ..
```
2. Run the tests:
1. Run the tests:
```shell
./gradlew clean test integrationTest
```

### Build a New Message Processor

The processing of the Solace message to create a Kafka source record is handled by an interface defined in [`SolaceMessageProcessorIF.java`](/src/main/java/com/solace/connector/kafka/connect/source/SolMessageProcessorIF.java). This is a simple interface that creates the Kafka source records from the PubSub+ messages. This project includes two examples of classes that implement this interface:
The processing of the Solace message to create a Kafka source record is handled by [`SolaceMessageProcessorIF`](/src/main/java/com/solace/connector/kafka/connect/source/SolMessageProcessorIF.java). This is a simple interface that creates the Kafka source records from the PubSub+ messages.

To get started, import the following dependency into your project:

**Maven**
```xml
<dependency>
<groupId>com.solace.connector.kafka.connect</groupId>
<artifactId>pubsubplus-connector-kafka-source</artifactId>
<version>2.1.0</version>
</dependency>
```

**Gradle**
```groovy
compile "com.solace.connector.kafka.connect:pubsubplus-connector-kafka-source:2.1.0"
```

Now you can implement your custom `SolaceMessageProcessorIF`.

For reference, this project includes two examples which you can use as starting points for implementing your own custom message processors:

* [SolSampleSimpleMessageProcessor](/src/main/java/com/solace/connector/kafka/connect/source/msgprocessors/SolSampleSimpleMessageProcessor.java)
* [SolaceSampleKeyedMessageProcessor](/src/main/java/com/solace/connector/kafka/connect/source/msgprocessors/SolaceSampleKeyedMessageProcessor.java)

You can use these examples as starting points for implementing your own custom message processors.
Once you've built the jar file for your custom message processor project, place it into the same directory as this connector, and update the connector's `sol.message_processor_class` config to point to the class of your new message processor.

More information on Kafka source connector development can be found here:
- [Apache Kafka Connect](https://kafka.apache.org/documentation/)
Expand Down
103 changes: 100 additions & 3 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
import com.github.spotbugs.snom.SpotBugsTask
import io.github.gradlenexus.publishplugin.InitializeNexusStagingRepository

plugins {
id 'java'
id 'distribution'
id 'jacoco'
id 'maven-publish'
id 'pmd'
id 'signing'
id 'com.github.spotbugs' version '4.7.6'
id 'io.github.gradle-nexus.publish-plugin' version '1.1.0'
id 'org.unbroken-dome.test-sets' version '2.2.1'
}

ext {
kafkaVersion = '2.8.1'
solaceJavaAPIVersion = '10.12.0'
isSnapshot = project.version.endsWith('-SNAPSHOT')
}

repositories {
Expand All @@ -38,9 +43,9 @@ dependencies {
integrationTestImplementation 'org.testcontainers:testcontainers:1.16.0'
integrationTestImplementation 'org.testcontainers:junit-jupiter:1.16.0'
integrationTestImplementation 'org.testcontainers:kafka:1.16.0'
integrationTestImplementation 'com.solace.test.integration:pubsubplus-junit-jupiter:0.5.0'
integrationTestImplementation 'com.solace.test.integration:pubsubplus-junit-jupiter:0.7.2'
integrationTestImplementation 'org.slf4j:slf4j-api:1.7.32'
integrationTestImplementation 'org.apache.logging.log4j:log4j-slf4j-impl:2.14.1'
integrationTestImplementation 'org.apache.logging.log4j:log4j-slf4j-impl:2.16.0'
integrationTestImplementation 'org.apache.commons:commons-configuration2:2.6'
integrationTestImplementation 'commons-beanutils:commons-beanutils:1.9.4'
integrationTestImplementation 'com.google.code.gson:gson:2.3.1'
Expand All @@ -49,7 +54,7 @@ dependencies {
integrationTestImplementation "org.apache.kafka:kafka-clients:$kafkaVersion"
testImplementation 'org.junit.jupiter:junit-jupiter:5.8.1'
testImplementation 'org.hamcrest:hamcrest-all:1.3'
testImplementation 'org.apache.logging.log4j:log4j-slf4j-impl:2.14.1'
testImplementation 'org.apache.logging.log4j:log4j-slf4j-impl:2.16.0'
compile "org.apache.kafka:connect-api:$kafkaVersion"
compile "com.solacesystems:sol-jcsmp:$solaceJavaAPIVersion"
}
Expand Down Expand Up @@ -96,6 +101,7 @@ project.integrationTest {
useJUnitPlatform()
outputs.upToDateWhen { false }
dependsOn prepDistForIntegrationTesting
shouldRunAfter test
afterSuite { desc, result ->
if (!desc.parent)
println("${result.resultType} " +
Expand Down Expand Up @@ -181,6 +187,11 @@ project.compileJava {
dependsOn generateJava
}

java {
withJavadocJar()
withSourcesJar()
}

distributions {
main {
contents {
Expand All @@ -197,3 +208,89 @@ distributions {
}
}
}

publishing {
publications {
maven(MavenPublication) {
from components.java
pom {
name = "Solace PubSub+ Connector for Kafka: Source"
description = "The PubSub+ Kafka Source Connector consumes PubSub+ event broker real-time queue or topic data events and streams them to a Kafka topic as Source Records."
url = "https://github.com/SolaceProducts/pubsubplus-connector-kafka-source"
packaging = "jar"
licenses {
license {
name = "Apache License, Version 2.0"
url = "https://github.com/SolaceProducts/pubsubplus-connector-kafka-source/blob/master/LICENSE"
distribution = "repo"
}
}
organization {
name = "Solace"
url = "https://www.solace.com"
}
developers {
developer {
name = "Support for Solace"
email = "[email protected]"
organization = "Solace"
organizationUrl = "http://solace.community"
}
}
scm {
connection = "scm:git:git://github.com/SolaceProducts/pubsubplus-connector-kafka-source.git"
developerConnection = "scm:git:[email protected]:SolaceProducts/pubsubplus-connector-kafka-source.git"
url = "https://github.com/SolaceProducts/pubsubplus-connector-kafka-source.git"
}
}
}
}
repositories {
maven {
def releasesUrl = uri('http://apps-jenkins:9090/nexus/content/repositories/releases')
def snapshotRepositoryUrl = uri('http://apps-jenkins:9090/nexus/content/repositories/snapshots')
url = isSnapshot ? snapshotRepositoryUrl : releasesUrl
name = 'internal'
credentials {
username = project.properties[name + "Username"]
password = project.properties[name + "Password"]
}
}
}
}

nexusPublishing {
repositories {
sonatype {
nexusUrl = uri('https://oss.sonatype.org/service/local/')
snapshotRepositoryUrl = uri('https://oss.sonatype.org/content/repositories/snapshots')
// gets credentials from project.properties["sonatypeUsername"] project.properties["sonatypePassword"]
}
}
}

signing {
required {
!isSnapshot
}
useGpgCmd()
sign publishing.publications.maven
}

tasks.withType(Sign) {
onlyIf {
gradle.taskGraph.allTasks.any {task ->
task.name.startsWith("publish") && task.name.contains('Sonatype')
}
}
shouldRunAfter test, integrationTest
}

tasks.withType(InitializeNexusStagingRepository).configureEach {
dependsOn test, integrationTest
shouldRunAfter tasks.withType(Sign)
}

tasks.withType(PublishToMavenRepository).configureEach {
dependsOn test, integrationTest
}
Loading

0 comments on commit 2c6aabe

Please sign in to comment.