Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

socket: Use a default listen backlog of -1 on Linux #1387

Merged
merged 1 commit into from
Aug 19, 2024

Conversation

ac000
Copy link
Member

@ac000 ac000 commented Aug 14, 2024

On FreeBSD, OpenBSD & macOS we use a default listen(2) backlog of -1 which means use the OS's default value.

On Linux (and others) we used a hard coded value of 511, presumably due to this comment

  /* Linux, Solaris, and NetBSD treat negative value as 0. */

On Linux (at least since 2.4), this is wrong, Linux treats -1 (and so on) as use the OS's default (net.core.somaxconn). See this code in net/socket.c::__sys_listen()

                if ((unsigned int)backlog > somaxconn)
                        backlog = somaxconn;

On Linux prior to 5.4 somaxconn defaulted to 128, since 5.4 it defaults to 4096.

We've had complaints that a listen backlog of 511 is too small. This would help in those cases.

Unless they are on an old Kernel, in which case it's worse, but then the plan is to also make this configurable. This would effect RHEL 8, which is based on 4.10, however they seem to set somaxconn to 2048, so that's fine.

Another advantage of using -1 is that we will automatically keep up to date with the kernels default value.

Before this change

  $ ss -tunxlp | grep unit
  Netid State  Recv-Q Send-Q                                                  Local Address:Port     Peer Address:Port           Process
  u_str LISTEN 0      511                                   /opt/unit/control.unit.sock.tmp 4302333            * 0    users:(("unitd",pid=18290,fd=6),("unitd",pid=18289,fd=6),("unitd",pid=18287,fd=6))
  tcp   LISTEN 0      511                                                         127.0.0.1:8080         0.0.0.0:*    users:(("unitd",pid=18290,fd=12))
  tcp   LISTEN 0      511                                                             [::1]:8080            [::]:*    users:(("unitd",pid=18290,fd=11))

After

  $ ss -tunxlp | grep unit
  Netid State  Recv-Q Send-Q                                                  Local Address:Port     Peer Address:Port           Process
  u_str LISTEN 0      4096                                  /opt/unit/control.unit.sock.tmp 5408464            * 0    users:(("unitd",pid=132442,fd=6),("unitd",pid=132441,fd=6),("unitd",pid=132439,fd=6))
  tcp   LISTEN 0      4096                                                        127.0.0.1:8080         0.0.0.0:*    users:(("unitd",pid=132442,fd=12))
  tcp   LISTEN 0      4096                                                            [::1]:8080            [::]:*    users:(("unitd",pid=132442,fd=11))

@ac000
Copy link
Member Author

ac000 commented Aug 15, 2024

Add link to related GH issue...

$ git range-diff 46334aa2...5819e3ea
1:  46334aa2 ! 1:  5819e3ea socket: Use a default listen backlog of -1 on Linux
    @@ Commit message
           tcp   LISTEN 0      4096                                                        127.0.0.1:8080         0.0.0.0:*    users:(("unitd",pid=132442,fd=12))
           tcp   LISTEN 0      4096                                                            [::1]:8080            [::]:*    users:(("unitd",pid=132442,fd=11))
     
    +    Link: <https://github.com/nginx/unit/issues/1384>
         Link: <https://lore.kernel.org/netdev/[email protected]/>
         Signed-off-by: Andrew Clayton <[email protected]>
     

@callahad callahad added this to the 1.33 milestone Aug 19, 2024
On FreeBSD, OpenBSD & macOS we use a default listen(2) backlog of -1
which means use the OS's default value.

On Linux (and others) we used a hard coded value of 511, presumably due
to this comment

  /* Linux, Solaris, and NetBSD treat negative value as 0. */

On Linux (at least since 2.4), this is wrong, Linux treats -1 (and so
on) as use the OS's default (net.core.somaxconn). See this code in
net/socket.c::__sys_listen()

                if ((unsigned int)backlog > somaxconn)
                        backlog = somaxconn;

On Linux prior to 5.4 somaxconn defaulted to 128, since 5.4 it defaults
to 4096.

We've had complaints that a listen backlog of 511 is too small. This
would help in those cases.

Unless they are on an old Kernel, in which case it's worse, but then the
plan is to also make this configurable. This would effect RHEL 8, which
is based on 4.10, however they seem to set somaxconn to 2048, so that's
fine.

Another advantage of using -1 is that we will automatically keep up to
date with the kernels default value.

Before this change

  $ ss -tunxlp | grep unit
  Netid State  Recv-Q Send-Q                                                  Local Address:Port     Peer Address:Port           Process
  u_str LISTEN 0      511                                   /opt/unit/control.unit.sock.tmp 4302333            * 0    users:(("unitd",pid=18290,fd=6),("unitd",pid=18289,fd=6),("unitd",pid=18287,fd=6))
  tcp   LISTEN 0      511                                                         127.0.0.1:8080         0.0.0.0:*    users:(("unitd",pid=18290,fd=12))
  tcp   LISTEN 0      511                                                             [::1]:8080            [::]:*    users:(("unitd",pid=18290,fd=11))

After

  $ ss -tunxlp | grep unit
  Netid State  Recv-Q Send-Q                                                  Local Address:Port     Peer Address:Port           Process
  u_str LISTEN 0      4096                                  /opt/unit/control.unit.sock.tmp 5408464            * 0    users:(("unitd",pid=132442,fd=6),("unitd",pid=132441,fd=6),("unitd",pid=132439,fd=6))
  tcp   LISTEN 0      4096                                                        127.0.0.1:8080         0.0.0.0:*    users:(("unitd",pid=132442,fd=12))
  tcp   LISTEN 0      4096                                                            [::1]:8080            [::]:*    users:(("unitd",pid=132442,fd=11))

Link: <nginx#1384>
Link: <https://lore.kernel.org/netdev/[email protected]/>
Signed-off-by: Andrew Clayton <[email protected]>
@ac000
Copy link
Member Author

ac000 commented Aug 19, 2024

Rebased with master

@ac000 ac000 merged commit 97c15fa into nginx:master Aug 19, 2024
24 checks passed
@ac000 ac000 deleted the def-listen-backlog branch August 19, 2024 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants