Fake/deceiving benchmark numbers?

[ishtms]

Tested a hello world actix-web application and a hello world uwebsockets.js example (on a 16 core ubuntu ec2 instance).
I copy pasted the hello world example from the repo's example directory.

Here are the results

Actix web
Requests per second - 586,472
Average latency - 1.19ms
50% latency - 333us (microseconds)

uWebSockets.js
Requests per second - 112,258
Average latency -2.29ms
50% latency -2.23ms

The benchmarks of uWebsockets claim it's 2.5x faster than actix-web, in reality it's ~5x slower.

[ghost]

Actix-web is a multithreaded server utilizing all 16 CPU cores by default. uWS is a single threaded server utilizing 1 CPU core by default. That's why the graph you refer to display "CPU-time normalized" results.

I compiled the latest actix-web using latest Rust and with release optimizations and setting workers to 1 (so that it doesn't utilize 16x the CPU-time compared with uWS). Here is the result run on a MacBook Air M1, with verified CPU-time being the same.

actix-web:
root@5d057ec32e43:/wrk# ./wrk http://localhost:8080
Running 10s test @ http://localhost:8080
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 102.44us 131.82us 6.87ms 98.93%
Req/Sec 46.95k 2.22k 49.65k 86.63%
943130 requests in 10.10s, 77.35MB read
Socket errors: connect 0, read 0, write 0, timeout 10
Requests/sec: 93381.46
Transfer/sec: 7.66MB

uWS:
root@5d057ec32e43:/wrk# ./wrk http://localhost:3000
Running 10s test @ http://localhost:3000
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 68.23us 88.26us 6.09ms 97.49%
Req/Sec 69.22k 2.64k 73.23k 81.59%
1381923 requests in 10.10s, 89.62MB read
Requests/sec: 136834.61
Transfer/sec: 8.87MB
root@5d057ec32e43:/wrk#

[ghost]

Of course, you can also scale uWS to use more than one CPU-core, but easiest to benchmark is to limit actix-web to use only 1 (so that your wrk load test can run on its own CPU and not interfere any more than necessary with the server being tested)