[Windows 10] A message sent on a datagram socket was larger than the internal message buffer

[ichixia]

I get same error in windows 10 by use go-ping. can pro-bing fix this?

go-ping/ping#168

[asiffer]

Same error but only on Windows Server 2022 (github actions runner windows-2022). I have not this error either on Windows Server 2019 (github actions runner windows-2019) or on self-hosted Windows 10 runner.

[martylamb]

getting this same error for what should be a successful ipv6 ping as well.

[KA-Ryzhkov]

I have the same problem on Windows 11 and it seems to have been around for a long time.

panic: read ip4 0.0.0.0: wsarecvfrom: A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself.

[w570955342]

Is anyone here? It's really urgent. Whether you can solve this problem or not, I hope to get a reply. I hope to know your plan. Please.

[ken-schneider]

Hi folks, I've been looking into this issue myself and largely agree with the suggestions made by @d-Rickyy-b on the original issue linked above.

I was able to reproduce this myself and did some digging into the actual packets returned to this code. First, I modified the library to have a large read buffer. Finally, I modified the library to output both the ICMP packet written and the data we received.

See the results below:

Sent Data

Raw ICMP Echo
2024/06/02 19:14:27 INFO: Writing ICMP echo: [8 0 195 251 90 246 0 2 23 213 82 250 23 130 168 156 167 165 143 116 65 233 69 226 178 136 86 1 35 100 195 73]

Break Down
ICMP Echo Request Header: 8 0 195 251 90 246 0 2

ICMP Echo Data: 23 213 82 250 23 130 168 156 167 165 143 116 65 233 69 226 178 136 86 1 35 100 195 73

Received Data

Raw Destination Unreachable
2024/06/02 19:14:27 INFO: Raw Packet: [3 1 131 140 0 0 0 0 69 0 0 52 145 69 0 0 64 1 0 0 10 211 55 8 31 25 2 3 8 0 195 251 90 246 0 2 23 213 82 250 23 130 168 156 167 165 143 116 65 233 69 226 178 136 86 1 35 100 195 73]

Break Down
ICMP Destination Unreachable Header: 3 1 131 140 0 0 0 0

Raw Request IPv4 Header: 69 0 0 52 145 69 0 0 64 1 0 0 10 211 55 8 31 25 2 3

Parsed Request IPv4 Header: ver=4 hdrlen=20 tos=0x0 totallen=52 id=0x9145 flags=0x0 fragoff=0x0 ttl=64 proto=1 cksum=0x0 src=10.211.55.8 dst=31.25.2.3

ICMP Destination Unreachable Data: 8 0 195 251 90 246 0 2 23 213 82 250 23 130 168 156 167 165 143 116 65 233 69 226 178 136 86 1 35 100 195 73

^Note that this is the same as the full ICMP echo request

Conclusion

Based on this, what we see on this system is that the response is in line with RFC 1812. We get the ICMP header which identifies the message as Destination Unreachable, the IPv4 header of the original echo request, and the entirety of the original ICMP Echo Request.

In the case of this configuration of pro-bing, this adds up to 60 bytes. Let’s break that down.

For my testing, I did not modify p.Size thus the default value of 24 was used.

So, our original ICMP Echo Request was composed of:

• ICMP Echo Request Header (8 bytes)

• ICMP Data Section (p.Size = 24)

Thus, the total size of the original ICMP Echo Request was:

8 (ICMP Header) + 24 (p.Size) = 32

All that remains then is the ICMP header of the error response and the IP header of the Echo Request:

32 (original ICMP Echo Request) + 20 (original IP header) + 8 (ICMP Response header) = 60 bytes

However, the library doesn’t account for this and instead creates a buffer:

24 (p.Size) + 20 (ipv4.HeaderLen) + 8 (ICMP Response header) = 52

Solution

Based on what was observed above, the library hasn't accounted for the header of both the ICMP echo request and the ICMP error response.

The least heavy handed approach then is to simply modify the calculation:

// Returns the length of an ICMP error response
// Calculated as:
// len(ICMP response header) + len(original IP header)
// + len(ICMP request header) + len(ICMP request data)
func (p *Pinger) getMessageLength() int {
	if p.ipv4 {
		return 8 + ipv4.HeaderLen + 8 + p.Size
	}
	return 8 + ipv6.HeaderLen + 8 + p.Size
}

Ideally, this approach would allow us to correctly size the buffer every time. However, I'm not a network engineer and don't know if we can count on routers to always behave in line with the RFCs.

What if, for example, a router replies with the entire datagram, but pads the message with 0s up to the RFC 1812's suggested maximum of 576? In that case, this would break and there’s no reason it couldn’t pad it with even more 0s.

Another option would be to set the buffer size to 576. This also doesn’t work as one Github user pointed out in the go-ping issue. If p.Size is set >547, a router could truncate to 576, but it could also choose to send the entire datagram.

As pointed out in the go-ping Github issue, there are many ping implementations that use arbitrary large sizes such as 1024, 2048, and even 4096.

Final Suggestion

If we want a constant value, I suggest either 576 to be consistent with the suggested max size of an ICMP message or 2048 as this is a sufficiently large buffer consistent with RFC 1812’s suggestions for the MTU.

Alternatively, we could be more conservative and dynamically size the buffer using math.Max like so:

const (
  minimumSize = 576 // or 2048
)
// Returns the length of an ICMP error response
// Calculated as:
// len(ICMP response header) + len(original IP header)
// + len(ICMP request header) + len(ICMP request data)
func (p *Pinger) getMessageLength() int {
	if p.ipv4 {
		calculatedSize := 8 + ipv4.HeaderLen + 8 + p.Size
		return int(math.Max(float64(calculatedSize), float64(minimumSize))
	}
	calculatedSize := 8 + ipv6.HeaderLen + 8 + p.Size
	return int(math.Max(float64(calculatedSize), float64(minimumSize))
}

I'm going to open a pull request with last suggest I've made shortly, but I'm open to further discussion/feedback.

[ken-schneider]

Ran a few more tests, looks like the internal buffer needs to be large enough to hold the IP header of the response as well. I've updated my PR to reflect this:

calculatedSize := p.Size + (ipv4.HeaderLen + 8) * 2