JUnit formatting overwrites previous runs when verifying multiple pacts

[gregtyler]

Original Slack thread

When running with --format RspecJunitFormatter --out /out/pact.xml, the verifier outputs a JUnit XML file containing the results. However, if the verifier is run against multiple consumers (e.g. we use --consumer-version-tag main to pick up the latest pact from each of 4 consumers) then the XML file is recreated/overwritten for each one. This means you only end up with the results for the last consumer run.

This can be very confusing because the verifier can fail but produce JUnit output suggesting everything was successful (if the last consumer passes but at least one other fails).

Would it be possible to either output one JUnit file for all consumers (e.g. --outdir /out), or to allow --out to contain a variable so that it can produce a predictable file for each consumer (e.g. --out /out/pact-{consumer}.xml)?

(NB: we can't use the new verifier because it doesn't yet support JUnit output)