v20.25.0

[uNetworkingAB]

Alien tech 👾

This release introduces a new mode of integration; no integration. Setting the environment variable ALIEN_UWS=1 makes uWS run entirely alien ("disjoint") to Node.js, on its own separate event-loop. This mode increases performance of uWS.js given the same CPU-time budget, by avoiding Node.js's heavy weight way of delivering events to the script.

This kind of integration is a prerequisite for the future potential swap to io_uring. So please test this mode. Test it in all kinds of debug modes and weird set ups. Try to break it.

---

This discussion was created from the release v20.25.0.

[uNetworkingAB]

This is an experimental mode that does not honor the AsyncHooks feature of Node.js, for uWS callbacks. That's where all the performance gain comes from - by not having to pay for the Node.js "always pay, regardless if used" way of implementing said AsyncHooks.

Everything in Node.js pays for this overhead, always, regardless if used or not. You see, Node.js is optimized to have a fast slow path and a slow fast path, rather than a fast fast path and a slow slow path. It makes sense, when you don't make sense.

Similar to EXPERIMENTAL_FASTCALL (now removed) but not as broken.

[Zubnix]

Are there any breaking API changes to consider or is it enough to set ALIEN_UWS=1 ?

[uNetworkingAB]

If you write code that relies on a particular ordering of async completions (in other words; if you write broken code) it can fail. Otherwise, should be fine (famous last words)

[kartikk221]

Hey, I added support for ALIEN_UWS=1 in my webserver hyper-express and was able to get it to work but with the following observations:

• This mode only works on linux and darwin (so far tested) based platforms only, Windows simply hangs.
• I used to be able to directly convert and push the incoming ArrayBuffer from the uWS.HttpResponse.onData() callback into a Readable as a Buffer stream which would then be instantly consumed by the internal chunk handling method to copy the raw Buffer onto the pre-allocated body buffer for the request body.

I am assuming it is because pushing to the Readable and then the Readable eventually emitting the data event to the consumer(s) had too many iterations in it to the point where the event loops become out of sync and the Buffer gets de-allocated by uWS before the stream can even emit the buffer.

The reason I was doing this in the first place was because I would actually pre-allocate an empty Buffer for the whole body and then simply copy the incoming onData() provided buffers onto the body buffer. Since I did not have to do Buffer.concat() to copy and hold the buffer in memory for each buffer, I could take advantage of uWS instantly deallocating the body chunk buffers for me instead of leaving them in memory for the garbage collector to eventually clean up.

But regardless, everything else seems to work fine with all tests passing and I think the performance gains should be worth it even though the new implementation always copying buffers into memory will result in higher memory usage and more work for the garbage collector over time.

[uNetworkingAB]

I think the performance gains should be worth it even though the new implementation always copying buffers into memory will result in higher memory usage and more work for the garbage collector over time.

You definitely won't see any gains by doing this. Garbage collection and copying is going to dominate your CPU-time. This alien mode really only makes sense if you already do everything to avoid GC and copying.

That means, no Node.js abstractions such as Readable stream since everything provided by Node.js is of poor performance.

[uasan]

Why do you need Readable streams, for such a simple task, we use this async method:

const readBody = (request, response) =>
  new Promise((resolve, reject) => {
    let offset = 0;
    const buffer = new Uint8Array(+request.getHeader('content-length'));

    response.onAborted = reject;
    response.onData((chunk, done) => {
      buffer.set(chunk, offset);
      if (done) {
        resolve(buffer);
      } else {
        offset += chunk.byteLength;
      }
    });
  });

[kartikk221]

It's popular and still uses Readable because of compatibility. HyperExpress is easy / almost drop-in for the average Node.js developer to use due to the API familiarity with Express. Request being a Readable and Response being a Writable are non-negotiable as everything is built on top of this in the Node world and thus needed for compatibility.

Furthermore because uWS binding of onData() in the first synchronous cycle of the request, I have to begin downloading the body immediately without knowing ahead of time whether the user will use a optimized method like Request.json()/Request.text()/Request.buffer() or the user will pipe the Readable into their own consumption resource which may be slower and cause back-pressure in queue.

Overall, the only sacrifices made were the ones absolutely necessary for compatibility, everything else is geared for performance.

[kartikk221]

Why do you need Readable streams, for such a simple task, we use this async method:

const readBody = (request, response) =>

  new Promise((resolve, reject) => {

    let offset = 0;

    const buffer = new Uint8Array(+request.getHeader('content-length'));



    response.onAborted = reject;

    response.onData((chunk, done) => {

      buffer.set(chunk, offset);

      if (done) {

        resolve(buffer);

      } else {

        offset += chunk.byteLength;

      }

    });

  });

Because HyperExpress has to support Request/Response as Readable/Writable streams and anytime you work with streams with uWS, you have to copy each buffer because a stream can be piped into another Transform or Writable both of which can have back-pressure and thus the chunks may be queued in the Readable for more than synchronous cycle. And if you recall, uWS deallocates each body buffer after the first synchronous execution of the handler hence the buffers have to be copied if they are not immediately being consumed as is the case with Streams. The approach you have above is the one taken for the native methods like buffer(),text(),json() because we know these methods will immediately consume the chunk with no back-pressure ever.

[uasan]

HyperExpress has to support Request/Response

This pattern, by the way, involves at least two reading methods

1. My example method for - Request.json(), Request.text(), Request.arrayBuffer(), Request.blob()
2. You stream method only for - Request.body.pipeTo()

So your pattern "Request/Response" should have told you that these are different methods of reading the body )

[porsager]

Oh boy.. uWebSockets.js is getting closer to raw uWebSockets huh 🎉

I don't know if you wanted tests on macOS yet, but I'm seeing considerable lower performance with ALIEN_UWS=1 on macOS.

This is a simple test with node v20.2 and uWebSockets.js v20.25.0

node uws.js:              164,664 req/s
ALIEN_UWS=1 node uws.js:  137,956 req/s

// uws.js
import uWS from 'uWebSockets.js'

uWS.App()
   .get('/*', (res, req) => res.end('OK'))
   .listen(9000, () => console.log('listening'))

I'm also seeing

Assertion failed: (err == 0), function uv_loop_delete, file ../deps/uv/src/uv-common.c, line 845.

when exiting the node process using process.exit().

[uasan]

I didn't pay for it either, it was given to me at work )

So the new mode makes sense if the application has a lot of asynchronous tasks between reading the request and the final response, so there is no noticeable performance gain in the synchronous Hello World benchmarks?

[Mupli]

@uNetworkingAB - Hm I have same results on linux mint. :(. Is this working? (Alien slower than .23)

Btw. there is some crashes when pipelining higher than 1. (in .23 was ok)
autocannon -c 50 -d 5 -p 10 http://localhost:3000/test

[uNetworkingAB]

Well if you think critically in steps:

1. AutoCannon is written in JavaScript as a single threaded Node.js "load tester".
2. The entire fundamental idea of uWS is that Node.js is bloated and slow as a snail.
3. How on earth are you then going to possibly stress uWS using a Node.js scripted tool - it's a contradiction from the start
4. It makes no sense, and AutoCannon is inherently a failure since it really can only be used to test other Node.js scripts, it can not be used to test uWS reliably.
5. We have http_load_test written in C with uSockets to test uWS for this reason

[uNetworkingAB]

When I read about AutoCannon I just want to hold and pet the guy who wrote it, on his head like you do a cat, from pity. They (the Node.js core developers) are inherently lost and confused and can't really seem to get a break when it comes to performance. They have done it for 10+ years now and they still don't get it. All they do, every single time, like an alzheimers patient will repeat time after time, is that they just say "oh, we need to write this parser in Assembly". That's their entire game play, they always look at some "parser" and say, "this C++ parser can be written in liquid Assembly instructions" as if that would fix, anything, in their heap of dynamic allocations and dynamic function calling and dynamic object creation. It's as if they don't understand the concept of "low hanging fruit" but just spend all their time on something that, at most, will do a 0.000000003% change - for the worse.

A parser will make no change to performance at all, unless you wrote the first one in the most idiotic way imaginable. Their problem is that everything about Node.js is rotten to the core, the entire systems architecture is written in such a way that a single parser will never even move the needle. And yet they still relentlessly spend all their time on nothing but a parser:

Node.js 20 includes specific changes, such as the updated version 2.0 of Ada, a fast and spec-compliant URL parser written in C++.

They fundamentally do not get any of it, how to make a performant cohesive whole, but rather just look at any one screw and say "this screw needs to be sharper", yeah that surely will make the cohesive whole any better....

So relying on their genius projects to try and measure uWS or any other optimised server is like asking a blind-deaf man for directions

[Mupli]

Yeah, got it. So this is the limitation of the benchmark here. Well, I never planned to test it with hyper-precision. Anyway, thanks for the tip. Btw Do you use some framework for testing ?

[Marocco2]

I keep getting this error if I use ALIEN_UWS=1

DEFLT [22/05/23, 18:29:02,628] /layers/paketo-buildpacks_npm-install/modules/node_modules/engine.io/build/userver.js:157
DEFLT [22/05/23, 18:29:02,628]             }, req.getHeader("sec-websocket-key"), req.getHeader("sec-websocket-protocol"), req.getHeader("sec-websocket-extensions"), context);
DEFLT [22/05/23, 18:29:02,628]                    ^
ERROR [22/05/23, 18:29:02,629] Error: Using uWS.HttpRequest / uWS.Http3Request past its request handler return is forbidden (it is stack allocated).
    at callback (/layers/paketo-buildpacks_npm-install/modules/node_modules/engine.io/build/userver.js:157:20)
DEFLT [22/05/23, 18:29:02,629] node: ../deps/uv/src/uv-common.c:843: uv_loop_delete: Assertion `err == 0' failed.

I use socket.io with uWS integration.

[uNetworkingAB]

Two points to be made very explicit:

1. You are definitely not (in the fullest sense of the word; of highest degree) going to see any gains, whatsoever, in any shape or form, from this mode for what you are trying to do. This mode is for thin usages directly atop uWS.js without any third party wrapper crap.
2. Socket.IO is not a product we encourage, supply, or control. Any failure of some unofficial frankenstein heap of shit that happens to somehow run atop uWS.js before, is not, has never been, and will never be, supported in any shape or form. It is deeply discouraged (and a fundamental misunderstanding of the whole point of uWS.js) to run Socket.IO with uWS.js. You are, essentially, mixing oil and water; two inherently contradictory elements that should never be mixed.

[Marocco2]

Very clear, thanks for answering 👍