This project provides a suite of benchmarks for C++ Actor Framework. All benchamarks focus on latency performances while controlling throughput. The main goal is to measure CAF reactivity at high message rates as well as at very low message rates.
In this benchmark actors send messages to each other with configurable payloads in order to measure performances of CAF framework.
- Build (eventually multiple) pipeline actors chain(s) by calling topology builder.
- Starts the benchmark sending messages to the head of chain(s) or at the rate generator.
- Sends results and statistics to the
result_collector_actor.
- Sends messages at a given rate to the head of the pipeline.
- If there are multiple chains, it sends messages with the specified policy to the heads.
- Spawns the requested number of benchmark actors (one of them may be detached).
- Returns the head of the chain to the
executor_actor.
- A
benchmark_actorknows its previous and its next pipeline item and sends messages forth and back to them (depending on benchmark type). - For latency tracking type benchmark, the last actor of the pipeline take the stop time in order to compute the time between the first message dispatch and the stop time.
- For cross-times tracking type benchmark, each benchmark actor adds an hop to the cross-times istance in the message.
- Collects statistics on system resources usage.
- Writes statistics collected from the
executor_actorto file.
In order to run the executable needs the caf_latency_benchmarks.ini file in a folder named config.
It will output results in a semicolon-separated CSV file configurable with the result_path, extension and output_file parameters of .ini file.
Run executable with --help to see available command line options.
See the provided .ini file to see al available parameters.
In addition to the benchmark-specific options provided in the .ini file, CAF specific options can be added both either to the .ini file or to the command line, for istance in order to change CAF scheduler policies.
Using the benchmark from command line may look like the following:
> ./caf_latency_benchmarks --caf#work-stealing.moderate-sleep-duration=1 --caf#work-stealing.relaxed-steal-interval=5 --caf#work-stealing.relaxed-sleep-duration=200 --benchmark.pipeline_actors=12 --parallel.rate=1000 --benchmark.duration=100
Tha may output:
Total average latency (ns): 19096
While the output result CSV file may look like:
> cat trace//benchmark_result_PARALLEL.txt
number of actors;sender message rate (msg/s);percentage sender error;all actor latency (us);pipeline actor latency (us);receiver message rate (msg/s);percentage receiver error
12;999.999;5.3666e-05;1.591;0.946;999.999;0.00010734
-P, --benchmark.payload_type = [NONE|COMPLEX]
CAF Benchmark support two message type:
NONE: a weightless messages (pure latency measure)COMPLEX: message composed by a std::string and a C struct with some fixed data and a std::vector<char>
The string lenght and vector size configuration parameters:
--payload.string_length, --payload.vector_size
In addition, it is possilbe to configure an actor computing time in microseconds:
--payload.actor_computing_time=t
This will cause every benchmark benchmark_actor to do some computation for t microseconds before sending a message to the next pipeline actor.
-N, --pipeline_actors
Linear topology with a fixed number of actors.
--benchmark.detach = [0 |...| N-1]
An actor in the pipeline can be spawned as detached.
--parallel.chains_number
-t, –benchmark.benchmark_type = [SERIAL|PARALLEL]
SERIAL: Sends message to head when the previous one arrived ad the end of the chainPARALLEL: Sends messages to head at a given rate
Parallel message rate can be specified with
-r, --parallel.rate
--benchmark.forth_and_back = [0|1]
Specifies if message must run from head to tail and then back to head again.
--parallel.sending_order = [0|1]
Sending order policy to the head of the chains(s):
0: Random1: Round-Robin
In addition it is possible to specify a batch size when sending messages from the rate_generator to the chain heads. This will cause messages to be sent in a square wave fashion:
--payload.batch_size=N
-K, --benchmark.tracking_type = [LATENCY|XTIMES]
LATENCY:track latencies between pipeline head and tail onlyXTIMES:track idividual actor latencies (cross-times)
--benchmark.result_path
Folder path where to store output files.
--benchmark.output_file
File name where to store the results (_serial or _parallel will be added depending on the benchmark type).
--benchmark.extension
Extension of the output file.
Depending on the benchmark type (serial / parallel), the output filename will be constructed as:
result_path + benchmark_type + output_file + extension
| Field | Description |
|---|---|
| number of actors | Number of actors in the pipeline |
| net latency (us) | Average time to dispatch a message from the head to the tail of the chain / number of actors |
| gross latency (us) | Computed as: benchmark time / (number of sent messages * number of actors) |
| message rate (msg/sec) | Messages per second received by the executor_actor |
| cpu usage ave | Cpu usage average |
| ram usage ave (MB) | Memory usage average |
| Field | Description |
|---|---|
| number of actors | Number of actors in the pipeline |
| sender message rate (msg/s) | Message rate generated by rate_generator |
| percentage sender error | Percentage error between requested and generated rate |
| actor latency (us) | average time to dispatch a message from the head to the tail of the chain / number of actors |
| receiver message rate (msg/s) | Message rate received by executor_actor |
| percentage receiver error | Percentage error between requested and received rate |
| cpu usage ave | Cpu usage average |
| ram usage ave (MB) | Memory usage average |
For cross-times tracking type, three additional field will be output:
| Field | Description |
|---|---|
| detached_send | Average latency needed to send a message from a detached actor |
| detached_receive | Average latency needed to receive a message from a detached actor |
| not_detached | Average latency needed to send or receive a message between non detached actors. |