0.19.3

new features and improvements

• Introduce scenarioExecutionParallelismFactor configuration (#385) @agourlay
  • the default behaviour is unchanged

bug fixes

• Do not remove Content-Type but only override it (#388) @agourlay
• Harden before/after feature hooks (#384) @agourlay
• Fix a MatchError in cornichon-http-mock server start (4f7e039) @agourlay

dependency updates

• Update caffeine to 2.8.5 (#382) @scala-steward
• Update http4s-blaze-client, ... to 0.21.6 (#381) @scala-steward
• Update sbt to 1.3.13 (#380) @scala-steward
• Update pure-config to 0.13.0 (6bc5982) @agourlay

0.19.2

new features and improvements

• Use a more precise Scalatest dependency in cornichon-scalatest 74fbf80
• Various performance improvements:
  • save_body_path parses session value only once
  • optimise DataTable rendering
  • reduce allocations across the board

bug fixes

• Do not override Content-Type by default #377

dependency updates

• Update http4s-blaze-client, ... to 0.21.5 (#378) @scala-steward
• Update scalatest to 3.2.0 (#376) @scala-steward
• Update sangria to 2.0.0 (#375) @scala-steward
• Update monix-execution, monix-reactive to 3.2.2 (#373) @scala-steward
• Update sbt to 1.3.12 (#372) @scala-steward
• Update scala to 2.13.3

0.19.1

new features and improvements

• Add containsExactly matcher for arrays (#370) @agourlay
• Add support for HTTP redirect @agourlay
  • the new configuration field followRedirect is false by default
  • see reference configuration https://agourlay.github.io/cornichon/reference-configuration.html

bug fixes

• Fix typos in documentation (#361) @lauraluiz
• Fix error reporting for isNull (#356) @agourlay

dependency updates

• Update monix-execution, monix-reactive to 3.2.1 (#364) @scala-steward
• Update http4s to 0.21.4 (#363) @scala-steward
• Update sangria to 2.0.0-RC2 (#362) @scala-steward
• Update kafka-clients to 2.5.0 (#351) @scala-steward
• Update decline to 1.2.0 (#350) @scala-steward
• Update parboiled to 2.2.0 (#341) @scala-steward
• Update pureconfig to 0.12.3 (#340) @scala-steward
• Update cats-core, cats-macros to 2.1.1 (#339) @scala-steward

0.19.0

breaking changes

• the cornichon-check module has been moved into cornichon-core

  • no need to mix with CheckDsl anymore
  • generators moved to com.github.agourlay.cornichon.core

• RandomContext does not expose its scala.util.Random object anymore.

new features and improvements

• cross compiled to scala 2.13

• main runner for cornichon-test-framwork to execute tests without SBT

Usage: cornichon-test-framework --packageToScan <string> [--featureParallelism <integer>] [--seed <integer>] [--scenarioNameFilter <string>]

Run your cornichon features without SBT.

Options and flags:
    --help
        Display this help text.
    --packageToScan <string>
        Package containing the feature files.
    --reportsOutputDir <string>
       Output directory for junit.xml files (default to current).
    --featureParallelism <integer>
        Number of feature running in parallel (default=1).
    --seed <integer>
        Seed to use for starting random processes.
    --scenarioNameFilter <string>
        Filter scenario to run by name.

• introduce <scenario-unique-number> and <global-unique-number> placeholder generators.

• improve error reporting for Matcher pointing to nothing

• show initial request in case of status error assertion

• gql.withVariables can use different json encoders for each variable #278 by @yanns

dependency updates

• cats 2.1.0
• http4s 0.21.0
• circe 0.13.0
• monix 3.1.0
• pure-config 0.12.2

0.18.1

This release contains minor change around the global configuration file.

e.g for a file /src/test/resources/reference.conf

cornichon {
  executeScenariosInParallel = false
  traceRequests = false 
  warnOnDuplicateHeaders = true
  failOnDuplicateHeaders = false
  disableCertificateVerification = true
}

breaking change

• baseUrl has been renamed to globalBaseUrl for more clarity
  • baseUrl can still be defined at the Feature level otherwise it falls back to globalBaseUrl

improvement

• introduce a new configuration field named disableCertificateVerification to interact with servers with malformed certificate.

  • false by default.

• fix addAcceptGzipByDefault configuration field which was not working properly

  • true by default

dependency updates

• Monix 3.0.0-RC5

0.18.0

There are quite a few breaking changes in this release, however it should only impact users who are:

• defining custom steps
• using the cornichon-kafka module

If you are only using the built-in DSL you should be fine 🤞

Deterministic tests 🎲

The goal of this release is to have deterministic runs via a fixed seed.

class DeterministicFeature extends CornichonFeature {

  override lazy val seed: Option[Long] = Some(1L) 

  def feature =
    Feature("demo deterministic feature") {

      Scenario("always the same") {

        Given I save("my-uuid" -> "<random-uuid>")

        Then assert session_value("my-uuid").is("bb1ad573-19b8-9cd8-68fb-0e6f684df992")

    }
  }
}     

This test will be valid as long as the fixed seed stays the same.

It is also possible to pass the seed via a command argument and not change the code (only in cornichon-test-framework).

testOnly *DeterministicFeature -- "always the same" "--seed=1"

On a failure the initial seed will be provided in the error reporting, enabling you to replay the exact same test even if it contains a source of randomness such as:

• randomized placeholders (random-uuid, random-string, random-boolean etc.)
• cornichon-check value generators & transitions
• custom steps using ScenarioContext.randomContext
• RandomMapper as extractor

breaking changes

• EffectStep and Assertion steps now consume a ScenarioContext instead of a Session.

  • example for the effect field:

    effect: Session         ⇒ Either[CornichonError, Session]
    effect: ScenarioContext ⇒ Either[CornichonError, Session]

  • ScenarioContext is a new concept encapsulating all the states of a given scenario.

    trait ScenarioContext {
      val randomContext: RandomContext
      val session: Session
    
      // uses randomContext to resolve randomized placeholders
      def fillPlaceholders(input: String): Either[CornichonError, String]
    }

  • disclaimer: mutation still exists within RandomContext - it will be tackled in a future release.

• placeholderResolver is now gone, to resolve placeholders use fillPlaceholders in the ScenarioContext .

• the underlying Step contract was changed to accommodate the propagation of state.

  trait Step {
    def run(engine: Engine)(initialRunState: RunState): Task[(RunState, FailedStep Either Done)]
  }

  to

  trait Step {
    val stateUpdate: StateT[Task, RunState, FailedStep Either Done]
    def runStep(runState: RunState): Task[(RunState, FailedStep Either Done)] =
      stateUpdate.run(runState)
  }

• CornichonJson.parseJsonUnsafe has been renamed to CornichonJson.parseDslJsonUnsafe .

  • aDslJson String can either represent:

    • a JSON object
    • a JSON array
    • a JSON string
    • a datatable

  • If you want to parse regular JSON use CornichonJson.parseString .

• RandomContext was moved to com.github.agourlay.cornichon.core.RandomContext .

• various changes to the cornichon-kafka DSL #244 .

• Gzip is now enabled by default.

• http4s client is now the default http client, replacing akka-http client.

  • The configuration line useExperimentalHttp4sClient must be removed if you were using it as experimental client.
  • the akka dependency has been removed completely.

• check_model does not take a seed as an input anymore.

improvements

• deterministic run via fixed seed on the feature and command line argument.

• new matcher on JSON body to check that a field is present and not null.

  body.path("myKey").isNotNull

• the noisy stack trace appearing when shutting down servers in cornichon-http-mock is now gone.

bug fixes

• thread safe date matchers #259 by @michalmela

dependency updates

• scala 2.12.9
• http4s 0.20.10
• pureConfig 0.11.1
• monix 3.0.0-RC3
• scalatest 3.0.8
• caffeine 2.780
• parboiled 2.1.8
• akka-http has been removed

0.17.1

breaking changes

• header assertions are now case-insensitive on the field name

The following assertions are now equivalent

headers.name("X-Served-Config").isPresent
headers.name("x-served-config").isPresent

• ResourceStep renamed to ScenarioResourceStep to clarify that resources are released at the end of a scenario

improvements

• andThen on Step to offer a simpler chain API

def chain(others: Session ⇒ List[Step]): Step
def andThen(others: List[Step]): Step
def andThen(other: Step): Step

bug fixes

• http4s error with debug level #228

dependency updates

• http4s 0.20.0
• scalatest 3.0.7
• akka-http 10.1.8
• caffeine 2.7.0
• parboiled 2.1.6

0.17.0

Property based testing support

This is the big new feature of that release, a brand-new module to enable property based testing of HTTP API

It offers two flavors of testing, the classical forAll approach and a way to explore API interactions randomly.

For more information please refer to the documentation of the cornichon-check module: http://agourlay.github.io/cornichon/modules/module-check.html

The release also contains the usual batch of breaking changes and improvements.

Breaking changes

• DebugStep now requires an explicit title

• ResourceStep was moved to com.github.agourlay.cornichon.steps.wrapped.ResourceStep

• RepeatConcurrently has an additional parallelism parameter to work as a stream processor.

In the following example, the nested block will be executed 500 times by groups of 10 in a streaming fashion.

RepeatConcurrently(times = 500, parallelism = 10, maxTime = 10.seconds) {
  Given I get("https://github.com/agourlay/cornichon")
  Then assert status.is(200)
}

• JsonPath.run now returns an Option

  • None instead of JNull if the path is not present
  • use JsonPath.runStrict to return an error if absent

• The json body assertions are not conflating a null field with an absent field anymore.

  given the following response body
  { “data” : null }

  in 0.16.x

  body.path(“data”).isAbsent is correct
  body.path(“data”).isPresent is incorrect

  in 0.17.x
body.path(“data”).isAbsent is incorrect
  body.path(“data”).isPresent is correct

  and additional matcher has been added for that case
  body.path(“data”).isNull

• AttachStep/AttachAsStep now requires Session ⇒ List[Step] instead of a List[Step]

  • it enables FlatMap-like behavior.

• EffectStep.chainEffects is gone because it promoted the direct usage of the effect function which breaks encapsulation and error reporting.

  • use AttachStep instead to bind steps together.

Improvements

• new ConcurrentlyStep to perform a series of steps in parallel

Concurrently(maxTime = 1.seconds) {
  Given I get("https://github.com/agourlay/cornichon")
  Given I get("https://github.com/agourlay/cornichon")
}

• new assertions for the size of JSON arrays

body.asArray.size.isLesserThan(x)
body.asArray.size.isGreaterThan(x)
body.asArray.size.isBetween(x, y)

• improve error reporting and logging for cornichon-test-framework

  • the final message reported now contains the full log
  • ignored features will not create noise in the logs

• new cats.EffectStep which is not using Future but interop with the cats-effects Effect instead

import com.github.agourlay.cornichon.steps.cats.EffectStep

val myTaskEffect = EffectStep("identity task", s => Task.now(Right(s)))

• add oscillation detection to Eventually in order to fail if the final successful state is not reached with oscillation of errors.

Given I send_async_command_updating_search_index
Eventually(maxDuration = 1.seconds, interval = 100.ms, oscillationAllowed = false) {
  Given I search_for_new_state
  Then assert status.is(200)
}

Here, an oscillation could be seen in the status with something like 404 -> 500 -> 404 -> 200.
The oscillation detector would fail the test because of the 404 oscillation.

This flag can come in handy when dealing with eventually consistent system that should converge to a final state without oscillation.

Dependency updates

• monix 3.0.0-RC2
• cats 1.6.0
• circe 0.11.1
• http4s 0.20.0-M5
• moved from ficus to PureConfig to manage configuration

0.16.3

bug fixes

• fix scenario filtering for cornichon-test-framework #185

improvements

• when ignoring scenarios or features, the reason passed as parameter will now be printed in the logs to ease debugging

• a new function ignoredIfDefined is available on Scenario and Feature to enable dynamic toggling based on configuration.

val config = loadMyConfig()

val toggle: Option[String] =
  if (config.ignoreThatFeature) Some("ignored due to local configuration") else None  

def feat = Feature("important feature).ignoredIfDefined(toggle) {...}

dependency updates

• cats 1.4.0
• circe 0.10.0

0.16.2

breaking changes

• support for scala 2.11 is discontinued as it caused more issues than I am willing to handle
• the custom extractor SimpleMapper lost its factory method fromFct. (use the case class constructor instead)
• remove unecessary type parameter on ResourceStep

improvements

• eventually will now show in the logs all distinct errors encountered to ease debugging
• new custom extractor RandomMapper(generator: Random ⇒ String) to generate random data in placeholders
• internal caches are now done properly by using caffeine
• limit parallelism of scenario execution in cornichon-test-framework in function of the number of cores
• cornichon-kafka updated to support Kafka 2.0
• sbt-updates is not pushed as a dependency anymore
• introduction of new documentation sub-sections