Skip to content
This repository has been archived by the owner on Feb 22, 2018. It is now read-only.

Commit

Permalink
Merge pull request #64 from ysb33r/development
Browse files Browse the repository at this point in the history
FIXED #49
FIXED #51 
FIXED #53 
FIXED #55 
FIXED #56 
FIXED #57
FIXED #59
FIXED #60
FIXED #61 
FIXED #62 
FIXED #63 
FIXED #67
  • Loading branch information
ysb33r authored May 25, 2017
2 parents 51ec49d + 4aead96 commit 51ac576
Show file tree
Hide file tree
Showing 68 changed files with 1,147 additions and 72 deletions.
6 changes: 3 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@

language: groovy

sudo: false
sudo: required

before_script:
- ./gradlew -S clean --console=plain

script:
- ./gradlew -S -i --console=plain assemble
- ./gradlew -S -i --console=plain --debug test
- ./gradlew -S -i --console=plain test
- ./gradlew -S -i --console=plain integrationTest compatibilityTest

jdk:
Expand All @@ -34,4 +34,4 @@ branches:
only:
- master
- development
# - release
# - release
20 changes: 15 additions & 5 deletions CHANGELOG.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ for a specific release, please visit the http://ysb33r.github.io/gradleTest[plug
* https://github.com/ysb33r/gradleTest/issues/5[#5] - Allow tests to be executed in parallel
* https://github.com/ysb33r/gradleTest/issues/52[#52] - Allow for expected failure cases
* https://github.com/ysb33r/gradleTest/issues/54[#54] - Added base plugin `org.ysb33r.gradletest.base`.
* https://github.com/ysb33r/gradleTest/issues/62[#62] - Propagate `--rerun-tasks` down to test instances.
* https://github.com/ysb33r/gradleTest/issues/67[#67] - `org.ysb33r.gradlerunner` plugin provides `gradleRunner` task which can interleave different steps with independent Gradle executions.

=== Bugs

Expand All @@ -30,32 +32,40 @@ for a specific release, please visit the http://ysb33r.github.io/gradleTest[plug
* https://github.com/ysb33r/gradleTest/issues/42[#42] - `GradleTest` HTML reports should not overwrite over Test
task reports.
* https://github.com/ysb33r/gradleTest/issues/45[#45] - When there is no `src/gradleTest` the build fails.
* https://github.com/ysb33r/gradleTest/issues/46[#46] - When folders below `src/gradleTest` non-word characters,
* https://github.com/ysb33r/gradleTest/issues/46[#46] - When folders below `src/gradleTest` contain non-word characters,
compilation of test classes fails.
* https://github.com/ysb33r/gradleTest/issues/47[#47] - Classpath not correctly resolved under test.
* https://github.com/ysb33r/gradleTest/issues/48[#48] - When copying gradleTest folders maintain same permissions
* https://github.com/ysb33r/gradleTest/issues/49[#49] - When folders below `gradleTest` are removed, generated code
should be removed too.
* https://github.com/ysb33r/gradleTest/issues/53[#53] - Skip test generation if `src/gradleTest` does not exist.
* https://github.com/ysb33r/gradleTest/issues/57[#57] - Transitive dependencies for pluginsare not injected into classpath.
* https://github.com/ysb33r/gradleTest/issues/59[#59] - GradleTest does not detect new folders if previous task completed successfully

=== Other

* https://github.com/ysb33r/gradleTest/issues/12[#12] - Fix Javadoc errors.
* https://github.com/ysb33r/gradleTest/issues/12[#12] - Fixed Javadoc errors.
* https://github.com/ysb33r/gradleTest/issues/36[#36] - Integration test for `GradleTest` failure under M$ Windows.
* https://github.com/ysb33r/gradleTest/issues/37[#37] - Integration test for `legacy20.GeadleTest` failure under
* https://github.com/ysb33r/gradleTest/issues/37[#37] - Integration test for `legacy20.GradleTest` failure under
M$ Windows.
* https://github.com/ysb33r/gradleTest/issues/40[#40] - Add gh-pages based docs
* https://github.com/ysb33r/gradleTest/issues/40[#40] - Added `gh-pages`-based .
* https://github.com/ysb33r/gradleTest/issues/44[#44] - If license plugin is applied, exclude GradleTest sourcesets
from check.
* https://github.com/ysb33r/gradleTest/issues/50[#50] - Removed the use of deprecated `TaskInputs.source(Object)`
when running under Gradle 3.0+.
* https://github.com/ysb33r/gradleTest/issues/51[#51] - Compatibility test for multi-project scenarios.
* https://github.com/ysb33r/gradleTest/issues/55[#55] - Memory: Split compatiblity tests into smaller runs on Appveyor.
* https://github.com/ysb33r/gradleTest/issues/55[#56] - Memory: Use virtual image rather than container on TravisCI.
* https://github.com/ysb33r/gradleTest/issues/60[#60] - Mention alternative approaches to `GradleTest`.
* https://github.com/ysb33r/gradleTest/issues/61[#61] - Validate support for Gradle 3.3.
* https://github.com/ysb33r/gradleTest/issues/63[#63] - Documented `--offline` behaviour.

// end::changelog[]

== Contributors

// tag::contributors[]
*

// end::contributors[]

== v0.5.5
Expand Down
10 changes: 8 additions & 2 deletions appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@ build:
verbosity: detailed

build_script:
- gradlew.bat assemble --info --no-daemon -PbintrayUserName -PbintrayApiKey
- gradlew.bat assemble --info --no-daemon

test_script:
- gradlew.bat test integrationTest compatibilityTest --info --no-daemon -PbintrayUserName -PbintrayApiKey
- gradlew.bat test integrationTest --info --no-daemon
- gradlew.bat test compatibilityTest --info --no-daemon -DgradleTest.versions=2.0,2.1,2.2,2.3 -DnonLegacyOnlyGradleTest.versions=2.13
- gradlew.bat test compatibilityTest --info --no-daemon -DgradleTest.versions=2.4,2.5,2.6,2.7 -DnonLegacyOnlyGradleTest.versions=2.14.1
- gradlew.bat test compatibilityTest --info --no-daemon -DgradleTest.versions=2.8,2.9,2.10,2.11 -DnonLegacyOnlyGradleTest.versions=3.0
- gradlew.bat test compatibilityTest --info --no-daemon -DgradleTest.versions=2.12,2.13,2.14.1 -DnonLegacyOnlyGradleTest.versions=3.1
- gradlew.bat test compatibilityTest --info --no-daemon -DgradleTest.versions=3.0,3.1,3.2.1 -DnonLegacyOnlyGradleTest.versions=3.2.1
- gradlew.bat test compatibilityTest --info --no-daemon -DgradleTest.versions=3.3,3.4.1,3.5 -DnonLegacyOnlyGradleTest.versions=3.3,3.4.1,3.5

branches:
only:
Expand Down
9 changes: 5 additions & 4 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

plugins {
id 'com.gradle.build-scan' version '1.0'
id 'com.gradle.build-scan' version '1.6'
id 'groovy'
id 'maven'
id 'org.ysb33r.bintray' version '1.5'
Expand All @@ -14,7 +14,7 @@ import org.asciidoctor.gradle.AsciidoctorTask

group = 'org.ysb33r.gradle'
archivesBaseName = 'gradletest'
version = '1.0-beta7'
version = '1.0-beta9'

apply from : 'gradle/integration-tests.gradle'
apply from : 'gradle/compatibility-tests.gradle'
Expand Down Expand Up @@ -64,6 +64,7 @@ ext {

dependencies {
compile gradleApi()
compile gradleTestKit()
compile localGroovy()
compile depCommonsIO

Expand Down Expand Up @@ -229,8 +230,8 @@ license {
header = rootProject.file('config/HEADER')
strictCheck = true
ignoreFailures = false
ext.year = '2015 - 2016'
excludes(['**/*.ad', '**/*.asciidoc', '**/*.adoc', '**/*.md','**/*.properties'])
ext.year = '2015 - 2017'
excludes(['**/*.ad', '**/*.asciidoc', '**/*.adoc', '**/*.md','**/*.properties','**/*.groovy.template'])
mapping 'gradle', 'SLASHSTAR_STYLE'
}

Expand Down
24 changes: 20 additions & 4 deletions compatibility/compatibility.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
// build directory.
// - Any test dpenedencie/Users/schalkc/Projects/GradleTest/compatibility/compatibility.gradles are picked up from the test dependencies of the
// calling project.
// - We need to run this project with the GradleTest jat that was created by
// - We need to run this project with the GradleTest jar that was created by
// the calling project
// - We also need to tell the gradleTestGenerator where to find the GradleTest
// jar. (This is not something a consumer of this plugin would normally do.)
Expand Down Expand Up @@ -68,12 +68,15 @@ task sanityCheck {
}


// *IMPORTANT* If you update this list also update .travil.yml & appveyor.yml
// tag::applyplugin2[]
gradleTest {
// end::applyplugin2[]
dependsOn sanityCheck
systemProperties = [:]
// tag::applyplugin2[]
versions '3.2.1', '3.1', '3.0'
versions '3.5', '3.4.1'
versions '3.3', '3.2.1', '3.1', '3.0'
versions '2.14.1'
versions '2.13'
versions '2.12' ,'2.11','2.10','2.9','2.8'
Expand All @@ -94,20 +97,32 @@ appveyor {
'-XX:+UseConcMarkSweepGC'
}

maxParallelForks = 4
maxParallelForks = 2
minHeapSize = "256m"
maxHeapSize = "768m"
}
}

travisci {
gradleTest {
maxParallelForks = 2
minHeapSize = "256m"
maxHeapSize = "768m"
}
}


test.enabled = false

// *IMPORTANT* If you update this list also update .travis.yml & appveyor.yml
if(GradleVersion.current() >= GradleVersion.version('2.13')) {
// tag::addTestSets[]
additionalGradleTestSet 'nonLegacyOnly' // <1>

nonLegacyOnlyGradleTest { // <2>
versions '3.5', '3.4.1', '3.3'
versions '3.2', '3.1', '3.0' // <3>
versions '2.14','2.13'
versions '2.14.1'
}

nonLegacyOnlyGradleTestGenerator { // <4>
Expand All @@ -117,6 +132,7 @@ if(GradleVersion.current() >= GradleVersion.version('2.13')) {

nonLegacyOnlyGradleTest {
dependsOn sanityCheck
systemProperties = [:]

beforeTest {
println " ${it.name}"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// This buildscript block is not required in normal build scripts. It is purely here to get the classpath
// correct during compatibility testing.
buildscript {
dependencies {
classpath gradleTestKit()
}
}

apply plugin : 'org.ysb33r.gradlerunner'

gradleRunner {
step 'gradleTasks', 'tasks', '--all'
}

task runGradleTest {
dependsOn gradleRunner
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// See https://github.com/ysb33r/gradleTest/issues/51

task runGradleTest {
dependsOn ':pluginA:runGradleTest'
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apply plugin : 'groovy'

dependencies {
compile localGroovy()
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package multiproject

class MyLib {
final static String NAME = 'MyLib'
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
apply plugin : 'groovy'
apply plugin : 'org.ysb33r.gradletest'

repositories {
jcenter()
}

dependencies {
compile project(':myLib')
compile localGroovy()
compile gradleApi()
}

gradleTest {
versions '2.13','3.0'
}

// This is nice for debugging, but can only be done on later versions
if(GradleVersion.current() > GradleVersion.version('2.13')) {
gradleTest {
beforeTest {
println '-'.multiply(60)
println "| [${project.name}] ${it.name}"
println "| ${configurations.gradleTestCompile.asPath}"
println '-'.multiply(60)
}
}
}

task runGradleTest( dependsOn : ['assemble','gradleTest'] )
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apply plugin : multiproject.MyLibPlugin

task runGradleTest {
dependsOn myLibRunner
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package multiproject

import org.gradle.api.Plugin
import org.gradle.api.Project

class MyLibPlugin implements Plugin<Project> {
void apply(Project project) {
project.tasks.create ('myLibRunner') {
doLast {
println MyLib.NAME
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
include 'myLib'
include 'pluginA'
3 changes: 2 additions & 1 deletion gradle/compatibility-tests.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ task compatibilityTest( type : GradleBuild ) {
dependsOn 'integrationTestRepo', 'jar'
mustRunAfter 'test', 'integrationTest'

tasks = ['gradleTest','nonLegacyOnlyGradleTest']
// tasks = ['gradleTest','nonLegacyOnlyGradleTest']
tasks = ['nonLegacyOnlyGradleTest']
dir = file("${projectDir}/compatibility")
buildFile = 'compatibility/compatibility.gradle'
startParameter.projectProperties.compatibilityVersion = version
Expand Down
30 changes: 30 additions & 0 deletions src/docs/asciidoc/parts/alternatives.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
= Alternative Approaches to GradleTest

== Gradle TestKit

`GradleTestKit` is the core API that ships with Gradle since Gradle 2.6. which is meant for plugin developers to test compatibility across Gradle versions. Easrly versions could only tests against the same version of Gradle that was running, but it has been enhanced over different Gradle releases. Today it is powereful allowing multiple versions to be tested and a number of different environments to be set up.

It is also a core compoenent of `GradleTest` when the version of Gradle is 2.13 or later. (When the current build version is older `GradleTest` falls back to legacy mode, simply because the `GradleTestKit` API either is non-existent or not mature enough.

`GradleTestKit` works well for integration testing when writing plugins if a select small group of Gradle versions are used or even if it only targets the current Gradle version. In this way one can quickly determine issues in a plugin which cannot be discovered via normal testing. `GradleTest` used `GradleTestKit` directly for its integration tests for exactly this reason.

Testing multiple versions of Gradle with GradleTestKit soon bcomes a burden for developers as the API differs and classpath setups can be tricky. Older versions of Gradle give the most issues. In addition `GradleTestKit` is essentially a library it focuses on the developer rather than the build script user.

In this regard `GradleTest` steps in as it allow tests to be written as if a normal buildscript would be written. it also manages differences (or non-existance) in API from Gradle 2.0 onwards in a completely transparent way to the plugin author.

Overall the approach should be to write integration tests with `GradleTestKit` using a limited or even just one Gradle version. This is then complimented by some `GradleTest` scripts to test a wider range of Gradle versions.

== Stutter

https://github.com/ajoberstar/gradle-stutter[Stutter] is another library by https://github.com/ajoberstar[Andy Oberstar] to aid with plugin development. It focuses on the cases where you might have to run the same tests against all supported versions, but don't want to deal with making each test loop over those versions (or `@Unroll` with a `where` block in Spock). `Stutter` generates one task per supported version for you and runs the full set of tests against it.

It definitely removes some of the complexity of setting up the `GradleTestKit` environment, but beware that it works from Gradle 2.13 onwards. The latter is not a restriction, but essentialy the same reason why `GradleTest` reverts to legacy mode if the build version of Gradle is older than 2.13: a lack of maturity in the `GradleTestKit` API in those releases.

It adds a `compatTest` source set, which will not clash with the `gradleTest` source set from `GradleTest`.

`Stutter` is essentialy developer-centric whereas `GradleTest` is script author-centric. This distinction is important as it clarifies different levels of validation:

- If you want to create good samples and make sure they build, GradleTest is a clear win. Not having to write the extra test code (or touch the TestKit API) is a great benefit.
- If you need to validate what the build did, you are back in `GradleTestKit` land and then `Stutter` will make it easier for you.

It can be an easy way to manage some more complex plugin integration tests. There is no reason to not use it alongside `GradleTest` as part of plugin development.
11 changes: 8 additions & 3 deletions src/docs/asciidoc/parts/bootstrap.adoc
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
== Bootstrap
= Bootstrap

GradleTest is available in the https://plugins.gradle.org/plugin/org.ysb33r.gradletest[Gradle Plugins] repository.
To use it, add the following snippet into your build script.
To use it, add the appropriate plugin to the `plugins` block.

[source,groovy,subs="attributes"]
----
plugins {
id 'org.ysb33r.gradletest' version '{revnumber}'
id 'org.ysb33r.gradletest' version '{revnumber}' // <1>
id 'org.ysb33r.gradletest.base' version '{revnumber}' // <2>
id 'org.ysb33r.gradlerunner' version '{revnumber}' // <3>
}
----
<1> The common use case for compatibility testing
<2> Use the base plugin when you do not require the default `gradleTest` tasks (and related source sets)
<3> Use the `gradlerunner` to easily execute independent steps of `GradleRunner`.

When your setup is more complex and the plugins block does not work OR if you are using Gradle 2.0 use the following instead

Expand Down
6 changes: 6 additions & 0 deletions src/docs/asciidoc/parts/cmdline.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
== Command-line Awareness

GradleTest is aware of certain command-line parameters and will pass them on to running tests.

* `--rerun-tasks` - All tests will be re-run even if they were previsouly successful.
* `--offline` - No dependency checks will be performed and if plugins implements any offline behaviour this will automatically be enabled.
26 changes: 26 additions & 0 deletions src/docs/asciidoc/parts/gradle-runner.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
= GradleRunner (Gradle 2.14.1+)

The `org.ysb33r.gradlerunner` plugin adds a default task called `gradleRunner` which is of type `GradleRunnerSteps`. This task allows people to run specific Gradle steps interlevaed via other actions such as copying files. This is specifically useful for people who need to document development steps where Gradle is involved.

Steps are added via the `step` keyword. Each step has a name which can later be used to extract information after execution.

[source,groovy]
----
gradleRunner {
step 'a simple closure', { // <1>
new File(workingDir,'build.gradle').text = '/* empty build file */'
new File(reportsDir,'foo.txt').txt = 'output to report directory' // <2>
}
step 'running gradle', 'tasks', '--all' // <3>
failingStep 'this gradle will fail', 'non-existing-task' // <4>
}
----
<1> A step can be a closure or an `Action`.
<2> Every step will have an unique directory for dropping files,
<3> A step can be a Gradle execution. Supply all command-line parameters after the name. A Gradle execution will drop `out.txt` and `err.txt` in the report directory.
<4> A step can also be a Gradle execution that is known to fail.

After execution use `gradleRunner.getStepReportDir(STEP_NAME)` to obtain the reporting directory for a step.
Loading

0 comments on commit 51ac576

Please sign in to comment.