Provide an example or documentation showing use with scala.concurrent.Future

[OndrejSpanel]

I was lead to this library by searching on Web how can I use ScalaCheck with async tests, in particular from typelevel/scalacheck#214. The documentation says this is possible , saying:

but any effect F[_] with an instance of MonadError[F, Throwable] can be used, including scala.concurrent.Future.

I have experience with Scala Futures, ScalaTest and ScalaCheck, but no experience at all with Cats. I have no idea how can I provide what is necessary to use Futures with this library. Could perhaps some example or documentation be added here, on StackOverflow, or on any discoverable location?

[armanbilge]

Are you using munit? Here's a rewrite of the example from the README.

import munit.{FunSuite, ScalaCheckEffectSuite}
import org.scalacheck.effect.PropF
import scala.concurrent.Future
import scala.concurrent.ExcutionContext.Implicits.global

class ExampleSuite extends FunSuite with ScalaCheckEffectSuite {
  test("first PropF test") {
    PropF.forAllF { (x: Int) =>
      Future(x).map(res => assert(res == x))
    }
  }
}

[OndrejSpanel]

I am using ScalaTest - AsyncFlatSpec. I have tried adapting the code you have provided as:

  "Number" should "equal to itself" in {
    PropF.forAllF { (x: Int) =>
      Future(x).map(res => assert(res == x))
    }
  }

I get an error:

polymorphic expression cannot be instantiated to expected type;
found : [F[_]]org.scalacheck.effect.PropF[F]
required: scala.concurrent.Future[org.scalatest.compatible.Assertion]
PropF.forAllF { (x: Int) =>

I really have no clue how to work with this.

[armanbilge]

It looks to me like you need an example of how to use scalacheck-effect with ScalaTest. I don't think the problem is related to Future.

[armanbilge]

Ok, found an example.
https://github.com/circe/circe-fs2/blob/717ab29cdcc31405ce2d2164fb541923514b25ab/fs2/src/test/scala/io/circe/fs2/Fs2Suite.scala#L103-L115

I think you need to add .check().map(r => assert(r.passed)).

[OndrejSpanel]

I have tried:

    PropF.forAllF { (x: Int) =>
      Future(x).map(res => assert(res == x))
    }.check().map(r => assert(r.passed))

The error is now:

No implicit view available from scala.concurrent.Future[org.scalatest.Assertion] => org.scalacheck.effect.PropF[F].

[armanbilge]

https://github.com/circe/circe-fs2/blob/717ab29cdcc31405ce2d2164fb541923514b25ab/fs2/src/test/scala/io/circe/fs2/Fs2Suite.scala#L232-L237

[OndrejSpanel]

Still the same error (adding import cats.effect.IO was be needed)..

[armanbilge]

🙂 right, you want to implement a version of that based on Future, not IO. It's just an example of how to implement the implicit conversion.

[OndrejSpanel]

Ok. I will try that tomorrow, thanks.

[OndrejSpanel]

I am afraid cats, IO and similar concepts are really alien for me. I feel like typing more or less randomly, without really knowing what am I doing or what I am supposed to do.

My current attempt is:

  private implicit def assertionToProp: Future[Assertion] => PropF[Future] = { assertion =>
    IO.fromFuture(IO(assertion))
  }

This (unsurprisingly) does not work. I feel a bit stupid, but I am unable to find almost any examples, tutorials or documentation on Cats IO cooperation with Futures.

[armanbilge]

You don't need IO at all, you should be able to write it 100% using Future.

[armanbilge]

If it helps: the goal of that function is to convert a ScalaTest Future[Assertion] to a ScalaCheck Effect PropF[Future]. There are three important types of Props: Prop.True, Prop.Undecided, and Prop.Exception.

[mpilquist]

Here's a minimal example that uses scalacheck-effect with scalatest:

https://scastie.scala-lang.org/xg0H49nTQOShsniANwynlg

import org.scalatest.Assertion
import org.scalatest.funsuite.AnyFunSuite
import org.scalacheck.Prop
import org.scalacheck.effect.PropF
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

class ExampleTest extends AnyFunSuite {
  
  test("example of using scalacheck-effect with scalatest") {
    // Create a PropF[Future]
    val p: PropF[Future] = PropF.forAllF { (x: Int) =>
      val myAssertion: Future[Assertion] = Future(x).map(res => assert(res == x))
      // forAllF supports Future[Unit] by default, so we need to throw away the Assertion value
      // This is okay b/c ScalaTest throws if the assertion fails, so if we get a value, it passed
      myAssertion.map(_ => ())
    }
    // Check the property and get the result as a Future[PropF.Result]
    val result: Future[org.scalacheck.Test.Result] = p.check()

    // Convert the result to a Future[Assertion]; a better implementation
    // would pass along more information about the test result
    result.map(r => assert(r.passed))
  }
}

Note that this is a bare minimal integration. Ideally, you'd create a separate trait that contains all boilerplate needed for integration, similar to what scalacheck-effect-munit provides for munit.

[OndrejSpanel]

I think this was the critical part I was missing:

forAllF supports Future[Unit] by default, so we need to throw away the Assertion value

I can compile now. Thanks a lot.

[OndrejSpanel]

There is still one thing needed to be implemented: when the test fails, currently a new assertion is thrown by assert(r.passed).

One needs to somehow propagate the original exception or the test result instead. I will try to find this on my own and post the code here, but if it is obvious to you or someone else, it will help me a lot.

[OndrejSpanel]

I have found the code handling Test.Result in https://github.com/scalatest/scalatestplus-scalacheck/blob/main/scalatestPlusScalaCheck/src/main/scala/org/scalatestplus/scalacheck/CheckerAsserting.scala - the CheckerAsserting.check seems to do all the work,

Unfortunately the conversion functionality itself does not seem to be exposed in any accessible way, the function calls check on its own, therefore I am not sure if there is a way to use it with PropF.forAllF.

[OndrejSpanel]

I was able to (ab)use the Prop.apply function so that I can execute check on a Prop.Result, which in turn I create from ScalaCheck Effects Test.Result by matching. See https://github.com/OndrejSpanel/ScalaTestCheck/blob/master/src/test/scala/MainTest.scala:

    PropF.forAllF(Gen.oneOf(0 until 10)) { i  =>
      Future {
        assert(i < 5) // intentional fail - we want to see how the failure is reported
      }.map(_ => ()) // forAllF supports Future[Unit] by default, so we need to throw away the Assertion value
    }.check().map { result =>
      val propResult = result.status match {
        case _: Test.Proved =>
          Prop.Result(Prop.Proof)
        case Test.Exhausted =>
          Prop.Result(Prop.Undecided)
        case Test.Failed(scalaCheckArgs, scalaCheckLabels) =>
          Prop.Result(status = Prop.False, args = scalaCheckArgs, labels = scalaCheckLabels)
        case Test.PropException(scalaCheckArgs, e, scalaCheckLabels) =>
          Prop.Result(status = Prop.Exception(e), args = scalaCheckArgs, labels = scalaCheckLabels)
        case _ if result.passed =>
          Prop.Result(Prop.True)
        case _ =>
          Prop.Result(Prop.False)
      }

      Checkers.check(Prop(_ => propResult))
    }

[mpilquist]

@OndrejSpanel For a more full featured example, see this gist: https://gist.github.com/mpilquist/7dd30a44ca2a7fe0cd494d9b04e4f661#file-eff-scala

Note it relies on a copy/pasted version of CheckerAsserting.scala with the type renamed to CheckerAsserting2.scala and with the check method split in to check and convertTestResult. The main idea is to reuse all the logic from ScalaTest that converts a org.scalacheck.Test.Result to an output that matches the ScalaTest style.

This example could get further cleaned up with support for ScalaTest style property config (e.g. forAllF(maxSize = 200)((x: Int) => ...)). I'll leave that as an exercise for the reader. ;)

[OndrejSpanel]

Are you sure you want to copy CheckerAsserting code, instead of calling it through Checkers.check(Prop(_ => propResult))? What are the benefits of doing so?

[mpilquist]

Perhaps nothing? I thought going through Checkers.check(Prop(_ => propResult)) might yield worse error messages & reporting but maybe not!