How to use?

[patriknw]

Are there any instructions of how to setup a project with the Fuzz tester? I can see that the branches contain tests for various projects, but it is hard to see what I would need to do to setup a new project.

[colin-scott]

Yeah, the documentation (other than comments in the code itself) is basically non-existent at this point ;-)

If you're interested in using it, I'd be more than happy to write up documentation, and help you iron out any issues you run into. FWIW, at some point in the future, I think it would be fun to give a tech talk at Typesafe. For now though, the tool itself obviously isn't easily understandable.

For now, a whirwind tour:

• the demi repository contains the tool itself
• an application (this repository has a set of examples) pulls in demi by having it as a git subtree, located at interposition/
• in the application's Build.scala (or build.sbt), we need to include AspectJ, through the sbt-aspectj plugin. Here is an example: https://github.com/NetSys/demi-applications/blob/raft-45/project/Build.scala#L3. When we run sbt build from the application's top-level directory, AspectJ interposition will be automatically weaved in. The AspectJ code itself is here. It mostly weaves interfaces within Akka, and should hopefully work on most versions of higher than 2.0.
• In the application's Main method, we need to configure demi. For now, here is very verbose example. That example contains a whole bunch of cruft that is needed for minimization (+ other auxiliary stuff like recording the execution, writing it to disk, and replaying it later), but is not needed for simple fuzz testing. For just fuzz testing, you should be able to do it in a few lines of code. I'd be glad to come up with an example if you're interested.

[patriknw]

Thanks for the description. After reading the raft blog post I think the tool looks interesting and I was thinking I could try it on Akka Distributed Data. I can't promise that I will do it, but if I do and run into questions I will ask you.

[colin-scott]

That looks interesting! I wasn't aware of that library.

We're actually looking for another system to evaluate on for our research paper, and that use case looks interesting and different from our previous two use cases (Spark, akka-raft).

If you'd be willing to walk us through an example use case for that library, we'd be glad to set up the fuzzer. (Might be somewhat delayed though -- I'm busy for the next 3 weeks or so)

[colin-scott]

One concern though: we need well-defined safety conditions to check. I'm no expert, but it seems that CDRTs are specified as a liveness condition ("eventually, all updates will be reflected on all nodes, and all values will be the same"). Are there safety conditions we can also check?

I guess we could also convert the liveness condition into a safety condition ("if all nodes have received a message from all other nodes, the CDRT value should be the same across all nodes")

[colin-scott]

We might also check safety conditions for each consistency level ("If a Replicator.UpdateSuccess is sent, then at least N nodes must have that value replicated")

[patriknw]

That sounds awesome. As a first inspiration you can look at this test: https://github.com/akka/akka/blob/master/akka-distributed-data/src/multi-jvm/scala/akka/cluster/ddata/JepsenInspiredInsertSpec.scala

We should probably use the ORSet CRDT since that is the most advanced.

Starting out with a PNCounter might also be interesting, since that is "easy" to understand.

[colin-scott]

Hey Patrik,

A few questions for you:

• Are there any documented (possibly already fixed) bugs in Distributed Data? I found this one. Any others?
• Is Distributed Data distinct than Jonas' CRDT library?

Thanks!

[colin-scott]

@patriknw One more questions [also see above]:

• Is it possible to run multiple Replicator actors within a single JVM?

Thanks!

[patriknw]

Are there any documented (possibly already fixed) bugs in Distributed Data? I found this one. Any others?

That bug is actually not in Distributed Data. It's in Cluster Sharding's usage of Distributed Data, so it's not intereresting for you.

Issues are marked with label t:distributed-data, but non of these are bugs as far as I know.

Is Distributed Data distinct than Jonas' CRDT library?

Jonas' was an early prototype, which got me inspired. Distributed Data still have a few code lines that originates from Jonas' library. There is also my patriknw/akka-data-replication, which is a predecessor of Akka Distributed Data. Both these libraries are superseeded by Akka Distributed Data.

Is it possible to run multiple Replicator actors within a single JVM?

Yes, you can start several ActorSystem instances in the same JVM, each corresponds to an Akka Cluster node, and each can host a Replicator actor (or more Replicator actors, but that's not interesting to test). Each must bind to a unique remoting port. Here is an example of how to do that: https://github.com/akka/akka/blob/master/akka-samples/akka-sample-cluster-scala/src/main/scala/sample/cluster/simple/SimpleClusterApp.scala

[colin-scott]

Awesome, thanks!

[colin-scott]

Hey @patriknw, sorry for the delay on this.

I just bumped across a tool that may help you achieve the same kind of fuzz testing without having to deal with the vagaries of research prototypes. Check it out here:

https://github.com/osrg/namazu/blob/master/README.md

[AkihiroSuda]

@colin-scott Wow, thank you a lot for mentioning our tool 👍

[patriknw]

thanks for the link