Unicode encoding (table) problem on engine level leading to CRS false positives (U+062F and U+D8AF resolving to slash)

[dune73]

Description

Both ModSecurity 2, ModSecurity 3 as well as Coraza are translating U+062F and U+D8AF to slash leading to a false positive with the CRS path traversal rule 930110.

Link to ModSec Issue: owasp-modsecurity/ModSecurity#3294

Steps to reproduce

$ echo "Payload U+062F"; curl  http://example.com/ -s -d "test=$(printf '\x06\x2f').."

$ echo "Payload U+D8AF"; curl  http://example.com/ -s -d "test=$(printf '\xd8\xaf').."

Expected result

No alert.

Actual result

Both these requests trigger CRS 930110 PL1.

Playground

POST / HTTP/1.1
Host: playground.coraza.io
Content-Type: application/x-www-form-urlencoded
Content-Length: 9

test=د..

--->

930110	Path Traversal Attack (/../) or (/.../)
949110	Inbound Anomaly Score Exceeded (Total Score: 10)

Notice how the single alert leads to a score of 10.

Reason

Given ModSec exhibits the same wrong behavior, this may have to do with a problem in the coding table.

[dune73]

On the ModSecurity front, Marc Stern found out, this is pretty much bogus.

owasp-modsecurity/ModSecurity#3294 (comment)

[fzipi]

Just in case, did you tried using the proper URL encoding?

[fzipi]

When trying this test in go-ftw, using the coraza engine:

func (s *localEngineTestSuite) TestCrsCall() {
	s.Require().NotNil(s.engine)

	// simple payload, no matches
	matchedRules := s.engine.CrsCall("this is a test")
	s.Require().Empty(matchedRules)

	// this payload will match a few rules
	matchedRules = s.engine.CrsCall("' OR 1 = 1")
	s.Require().NotEmpty(matchedRules)

	matchedRules = s.engine.CrsCall("د..test")
	s.Require().NotEmpty(matchedRules)
...

I get:

=== RUN   TestLocalEngineTestSuite/TestCrsCall
{"level":"debug","time":"2024-11-21T20:27:31-03:00","message":"Using paranoia level: 1"}
{"level":"trace","time":"2024-11-21T20:27:31-03:00","message":"Encoded payload: /get?uri_payload=this+is+a+test"}
{"level":"trace","time":"2024-11-21T20:27:31-03:00","message":"Encoded payload: /get?uri_payload=%27+OR+1+%3D+1"}
{"level":"trace","time":"2024-11-21T20:27:31-03:00","message":"Encoded payload: /get?uri_payload=%D8%AF..test"}
    local_engine_test.go:75: 
        	Error Trace:	go-ftw/internal/quantitative/local_engine_test.go:75
        	Error:      	Should NOT be empty, but was map[]
        	Test:       	TestLocalEngineTestSuite/TestCrsCall

As you can see is it properly encoded.