Skip to content
This repository has been archived by the owner on May 3, 2024. It is now read-only.

Commit

Permalink
Switch from maven to gradle (#323)
Browse files Browse the repository at this point in the history
* gradle build

jackson fix

some more jackson testing

testFixtures, version to project properties

test build

maven-publishing conventions

remove jackson bom (not supported by old versions)

pom polishing

zstd

dependency linting

compose-provided support

cleanup files

jackson bla

documentation

remove jdbc + redis

fix e2e-test

simplify e2e test properties

run full github workflow on gradle

convention plugin cleanup

drop jackson 2.6.x support, and test with latest library versions

Jackson 2.6.x support was broken anyway, because fahrschein-typeresolver didn't compile.
We shouldn't state we support a certain library version if only parts of our
tool work with it.

use provided gradle

play with dependency reasoning

README fix

publishing setup

from guide https://docs.gradle.org/current/userguide/publishing_maven.html

publishing test

polishing

clean up maven pom.xml

documentation and compatibility testing

dependency report

test coverage reporting

CVE scanning

don't discriminate eclipse

* use xy.version property naming

* Update buildSrc/src/main/groovy/fahrschein.maven-publishing-conventions.gradle

Co-authored-by: Christoph Berg <[email protected]>

* Update buildSrc/src/main/groovy/fahrschein.maven-publishing-conventions.gradle

Co-authored-by: Christoph Berg <[email protected]>

* Code review changes

* fahrschein-http-jdk11/build.gradle: avoid escape for single quote
* fahrschein-http-spring/build.gradle: follow naming pattern for module
  name
* fahrschein/build.gradle b/fahrschein/build.gradle: add jackson-core as
  implementation dependency
* gradle.properties: add newline
* gradle/wrapper/gradle-wrapper.properties: add distributionSha256Sum

* adds newline

* add note about connection pool size

* adds another newline

* Document dependency baselines

* make tasks.withType evaluation lazy

* formatting, fixing testcontainers version string

cleanup

* obsolete dependency

* use java toolchains to ensure JDK11

Co-authored-by: Christoph Berg <[email protected]>
  • Loading branch information
otrosien and cberg-zalando authored May 30, 2022
1 parent 4d8f658 commit 68f1625
Show file tree
Hide file tree
Showing 50 changed files with 832 additions and 1,253 deletions.
14 changes: 8 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@ jobs:
with:
distribution: 'adopt-hotspot'
java-version: '11'
cache: 'maven'
- uses: gradle/gradle-build-action@v2
with:
gradle-version: 7.4.1
- name: Test Default
run: mvn clean verify
- name: Test Ancient Jackson
run: mvn -Dversion.jackson=2.6.7 -pl !fahrschein-typeresolver,!fahrschein-example,!fahrschein-e2e-test clean verify
- name: Test Oldest Jackson (with full feature support)
run: mvn -Dversion.jackson=2.8.0 clean verify
run: gradle build
- name: Test oldest supported library versions
run: gradle build -Pjackson.version=2.8.0 -Pspring.version=4.3.0.RELEASE -Pokhttp.version=3.3.0 -Papachehttp.version=4.3
- name: Test with latest library (micro) updates
run: gradle build -Pjackson.version=2.13.+ -Pspring.version=5.3.+ -Pokhttp.version=4.9.+ -Papachehttp.version=4.5.+
18 changes: 18 additions & 0 deletions .github/workflows/cve-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: CVE scan
on:
schedule:
- cron: "0 0 * * *"

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-java@v2
with:
distribution: 'adopt-hotspot'
java-version: '11'
- uses: gradle/gradle-build-action@v2
with:
gradle-version: 7.4.1
arguments: dependencyCheckAggregate
22 changes: 11 additions & 11 deletions .github/workflows/maven-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,25 @@ name: Publish package to the Maven Central Repository and GitHub Packages
on:
release:
types: [created]

jobs:
publish:
runs-on: ubuntu-latest
steps:
- name: Check out Git repository
uses: actions/checkout@v2

- name: Set up Java for publishing to Maven Central Repository
- name: Set up Java
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'

- name: Release Maven package
uses: samuelmeuli/action-maven-publish@v1
- name: Publish package
uses: gradle/gradle-build-action@v2
with:
gpg_private_key: ${{ secrets.GPG_KEY }}
gpg_passphrase: ${{ secrets.GPG_PASSPHRASE }}
nexus_username: ${{ secrets.OSSRH_USERNAME }}
nexus_password: ${{ secrets.OSSRH_TOKEN }}
maven_profiles: release
maven_args: -DskipITs=true
gradle-version: 7.4.1
arguments: publish -Prelease
env:
NEXUS_USERNAME: ${{ secrets.OSSRH_USERNAME }}
NEXUS_PASSWORD: ${{ secrets.OSSRH_TOKEN }}
GPG_KEY: ${{ secrets.GPG_KEY }}
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@ target/
.settings/
.vscode
bin/

#gradle
.gradle/
build/
71 changes: 64 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,8 +209,8 @@ final CloseableHttpClient httpClient = HttpClients.custom()
.disableAutomaticRetries()
.setDefaultRequestConfig(config)
.disableRedirectHandling()
.setMaxConnTotal(8)
.setMaxConnPerRoute(2)
.setMaxConnTotal(20)
.setMaxConnPerRoute(20)
.build();

final RequestFactory requestFactory = new HttpComponentsRequestFactory(httpClient, ContentEncoding.GZIP);
Expand Down Expand Up @@ -245,6 +245,21 @@ final NakadiClient nakadiClient = NakadiClient.builder(NAKADI_URI, requestFactor

**Note:** The implementations from the Spring framework don't handle closing of streams as expected. They will try to consume remaining data, which will usually time out when nakadi does not receive a commit.

**Note:** Regarding sizing and reuse of HTTP client connection pools, make sure to have a connection pool
size bigger than the number of subscriptions you're making, because subscriptions use long-polling to
retrieve events, each effectively blocking one connection.

## Dependency compatibility

Although Fahrschein is using fixed dependency versions, it is integration-tested against the following dependency matrix. We will inform in the release notes in case we bump the compatibility baseline.

| Dependency | Baseline | Latest |
| ---- | ---- |
| Jackson | 2.8.0 | 2.13.+ |
| Spring Core | 4.3.0.RELEASE | 5.3.+ |
| okHttp | 3.3.0 | 4.9.+ |
| Apache HttpClient | 4.3 | 4.5.+ |

## Content-Compression

Fahrschein handles content compression transparently to the API consumer, and mostly independently of the actual HTTP
Expand Down Expand Up @@ -284,21 +299,63 @@ If you have questions, concerns, bug reports, etc, please file an issue in this

## Local development

For local development, Fahrschein requires:
Fahrschein is a gradle-based project. For local development, Fahrschein requires:

* A local installation of JDK8
* Any recent version of Maven
* A local installation of JDK11
* A local Docker installation for running integration tests

When developing, make sure to run unit and integration tests with `mvn verify`.
When developing, make sure to run unit and integration tests with `./gradlew test`.

### Understanding the build

We use Gradle [convention plugins](https://docs.gradle.org/current/samples/sample_convention_plugins.html) to share common build logic across multiple subprojects. These are [fahrschein.java-conventions.gradle](./buildSrc/src/main/groovy/fahrschein.java-conventions.gradle) for common java properties, and [fahrschein.maven-publishing-conventions.gradle](./buildSrc/src/main/groovy/fahrschein.maven-publishing-conventions.gradle) for subprojects that are released as maven artefacts.

### Bumping dependency versions

Most dependencies are defined on a per-subproject level, only the versions for the most-used shared dependencies are controlled centrally, in the [gradle.properties](./gradle.properties) file. This also allows you testing your build with a different version by specifying the property on the command-line.

```sh
./gradlew test -Pjackson.version=2.9.0
```

The [integration tests](.github/workflows/ci.yaml) include running the build with our supported baseline dependency as well as the latest micro release of Jackson, Apache HttpClient and Spring. Please update the section in the README when bumping dependency baselines, and add this to the release notes.

### End-to-End tests

The `fahrschein-e2e-test` module has end-to-end tests using `docker-compose`. If you wish to leave the Docker containers running between tests, you can also bring up docker-compose manually using its [docker-compose.yaml](fahrschein-e2e-test/src/test/resources/docker-compose.yaml), before running the tests. In either case, you will need the port `8080` to be available for Nakadi to run.

```sh
docker-compose -f fahrschein-e2e-test/src/test/resources/docker-compose.yaml up -d
mvn verify -DCOMPOSE_PROVIDED
./gradlew :fahrschein-e2e:test -Pe2e.composeProvided
```

If you want to skip end-to-end tests completely, run
```sh
./gradlew test -Pe2e.skip
```

### CVE scanning

The project integrates CVE scanning to check for vulnerable dependencies. In case of build failure, this can be caused by a high-risk vulnerability in a dependency being identified. You can run the reporting locally:

```
./gradlew dependencyCheckAggregate
open build/reports/dependency-check-report.html
```

### Releasing

Fahrschein uses Github Workflows to [build](.github/workflows/ci.yaml) and [publish releases](.github/workflows/maven-publish.yaml). This happens automatically whenever a new release is created in Github. After creating a release, please bump the `project.version` property in [gradle.properties](./gradle.properties).

If needed, you can preview the signed release artifacts in your local maven repository.

```sh
env "GPG_KEY=$(cat ~/.gnupg/private-key.pgp)" \
"GPG_PASSPHRASE=$(read -s password ; echo $password)" \
./gradlew publishToMavenLocal
```



## Getting involved

Expand Down
14 changes: 14 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
plugins {
id 'eclipse'
id 'idea'
id 'com.github.ben-manes.versions' version '0.42.0'
id 'org.owasp.dependencycheck' version '7.1.0.1'
}


// CVE vulnerability scanning
// run: ./gradlew dependencyCheckAggregate

dependencyCheck {
failBuildOnCVSS = '9'
}
7 changes: 7 additions & 0 deletions buildSrc/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
plugins {
id 'groovy-gradle-plugin'
}

repositories {
gradlePluginPortal()
}
63 changes: 63 additions & 0 deletions buildSrc/src/main/groovy/fahrschein.java-conventions.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
plugins {
id 'java-library'
id 'jacoco'
id 'org.owasp.dependencycheck'
}

repositories {
mavenCentral()
}

group = 'org.zalando'
version = project.property('project.version') + (project.hasProperty('release') ? '' : '-SNAPSHOT')

java {
toolchain {
languageVersion = JavaLanguageVersion.of(11)
}
}

tasks.withType(JavaCompile).configureEach {
options.compilerArgs << '-parameters'
options.compilerArgs << '-Xlint:all'
options.encoding = 'UTF-8'
options.release = 8
}

dependencies {
implementation("org.slf4j:slf4j-api:${property('slf4j.version')}") {
because "we want slf4j-api to be available everywhere"
}
testRuntimeOnly("org.slf4j:slf4j-simple:${property('slf4j.version')}") {
because "we want the slf4j-simple logger implementation available at test runtime"
}
testImplementation("org.mockito:mockito-core:${property('mockito.version')}") {
because "we want mockito to be used in tests"
}
testImplementation("junit:junit:${property('junit.version')}") {
because "we want a common version JUnit across all projects"
}
testImplementation('org.hamcrest:hamcrest:2.2') {
because "we want to have hamcrest available for all testing"
}
compileOnly('com.google.code.findbugs:jsr305:2.0.1') {
because "we want to have annotations like @Deprecated available at compilation time"
}
}

configurations.all {
resolutionStrategy.dependencySubstitution {
substitute(platform(module('commons-logging:commons-logging'))).
using module("org.slf4j:jcl-over-slf4j:${property('slf4j.version')}")
substitute(platform(module('org.hamcrest:hamcrest-core'))).
using module("org.hamcrest:hamcrest:2.2")
}
}

test {
finalizedBy jacocoTestReport // report is always generated after tests run
}
jacocoTestReport {
dependsOn test // tests are required to run before generating the report
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
plugins {
id 'maven-publish'
id 'signing'
}

java {
withJavadocJar()
withSourcesJar()
}

publishing {
publications {
maven(MavenPublication) {
from(components.java)
versionMapping {
usage('java-api') {
fromResolutionOf('runtimeClasspath')
}
usage('java-runtime') {
fromResolutionResult()
}
}
pom {
url = 'https://github.com/zalando-nakadi/fahrschein'
inceptionYear = '2006'
licenses {
license {
name = 'Apache License, Version 2.0'
url = 'https://www.apache.org/licenses/LICENSE-2.0.txt'
}
}
developers {
developer {
name = 'Othon Crelier'
email = '[email protected]'
organization = 'Zalando SE'
organizationUrl = 'https://tech.zalando.com/'
}
developer {
name = 'Malte Pickhan'
email = '[email protected]'
organization = 'Zalando Payments GmbH'
organizationUrl = 'https://tech.zalando.com/'
}
developer {
name = 'Oliver Trosien'
email = '[email protected]'
organization = 'Zalando SE'
organizationUrl = 'https://tech.zalando.com/'
}
}
scm {
connection = 'scm:git:[email protected]:zalando-nakadi/fahrschein.git'
developerConnection = 'scm:git:[email protected]:zalando-nakadi/fahrschein.git'
url = 'https://github.com/zalando-nakadi/fahrschein'
}
}
repositories {
maven {
def snapshotsRepo = 'https://oss.sonatype.org/content/repositories/snapshots/'
def releasesRepo ='https://oss.sonatype.org/service/local/staging/deploy/maven2/'
url = version.endsWith('SNAPSHOT') ? snapshotsRepo : releasesRepo
credentials {
username = System.getenv("NEXUS_USERNAME")
password = System.getenv("NEXUS_PASSWORD")
}
}
}
}
}
}


signing {
def signingKey = System.getenv("GPG_KEY")
def signingPassword = System.getenv("GPG_PASSPHRASE")
useInMemoryPgpKeys(signingKey, signingPassword)
sign publishing.publications.maven
}

javadoc {
if(JavaVersion.current().isJava9Compatible()) {
options.addBooleanOption('html5', true)
}
}
18 changes: 0 additions & 18 deletions docs/RELEASING.md

This file was deleted.

Loading

0 comments on commit 68f1625

Please sign in to comment.