Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebSocket server massively over-allocates memory in a "hello world" ping/pong example #27925

Open
josephrocca opened this issue Feb 1, 2025 · 0 comments

Comments

@josephrocca
Copy link
Contributor

josephrocca commented Feb 1, 2025

  • Version: Deno 2.1.9 (Ubuntu 22.04)
  • Context: Debugging a weird case where a websocket server under high load was growing to 20GB+ of RAM.

In this example, the client and the server "take turns" in sending messages, so it shouldn't need to allocate 1GB worth of RAM. And in fact, for some reason we can prevent it from doing that by adding setInterval(() => console.log("hello"), 1000) to the top of the server script. If we do that, the RSS memory doesn't go past 200MB.

Server:

// deno run --allow-net server.js
Deno.serve({port:3086}, (req) => {
  if(req.headers.get("upgrade") != "websocket") {
    return new Response(null, { status: 501 });
  }
  const { socket, response } = Deno.upgradeWebSocket(req);
  socket.addEventListener("message", (event) => {
    if(event.data === "ping") {
      socket.send("pong");
    }
  });
  return response;
});

Client:

// deno run --allow-net client.js
function connect() {
  const socket = new WebSocket("ws://localhost:3086");
  socket.onopen = () => socket.send("ping");
  socket.onerror = () => socket.close();
  socket.onclose = () => setTimeout(connect, 1000);
  socket.onmessage = (e) => {
    if(e.data === "pong") {

      // First send some extra messages that don't trigger a "pong" response (necessary to reproduce the issue):
      for(let i = 0; i < 50; i++) {
        socket.send("foo");
      }

      // Then trigger another response:
      socket.send("ping");
    }
  };
}

for(let i = 0; i < 1000; i++) {
  connect();
}

Notes:

  • I'm also guessing that it's expected behavior that even after all clients disconnect, the memory isn't freed. I.e. the buffer never shrinks. I can deal with that, though it's definitely not ideal, especially with bugs like this one (i.e. spike in traffic causes allocations, which then "permanently" hog RAM).
  • On the same server that I was debugging this issue on, I also got an issue like this:
    • WebSocket close event is not emitted after the socket is closed on the server #27924
    • I.e. sockets not closing properly, even after I'd completely disconnected all clients. They were in a CLOSING state, but never triggered the close event. And note that the above-linked issue uses websocat, whereas in my case I was just using Deno as the client, like in the above example. I'm unsure if this is related to this issue. It seems separate.
  • Possibly related:
    • WebSocket memory leaks #27156
    • Although the reason I title this issue "over-allocated" instead of "leak" is because if I terminate all the clients, then start them again, the memory doesn't go to 2GB, it just stays at 1GB. So this indicates that it's just allocating too much space, and not actually leaking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant