ArrayIndexOutOfBoundsException from UTF32Reader.read on invalid input

[emilyselwood]

We are doing some fuzz testing on internal projects and found some input that causes an unexpected exception from JsonParser. While it is throwing an exception and stopping on this, it should probably be some kind of IOException rather than an ArrayIndexOutOfBoundsException.

In our case we are catching the three exception types listed in ObjectMapper.readTree (JsonParseException | JsonProcessingException | IOException) and this managed to escape.

Simple test case follows

public void testInvalidInput() throws IOException {

        byte[] data = {
                0x00,
                0x00,
                0x00,
                0x20,
                (byte) 0xFE,
                (byte) 0xFF,
                0x00,
                0x01,
                (byte) 0xFB
        };

        JsonFactory FACTORY = new JsonFactory();
        JsonParser parser = FACTORY.createParser(data);

        parser.nextToken();


    }

stack trace:

java.lang.ArrayIndexOutOfBoundsException: 9

	at com.fasterxml.jackson.core.io.UTF32Reader.read(UTF32Reader.java:138)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._loadMore(ReaderBasedJsonParser.java:243)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:2331)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:646)
	at com.fasterxml.jackson.core.io.TestUTF32Reader.testInvalidInput(TestUTF32Reader.java:32)

[cowtowncoder]

Thank you for reporting this. It probably doesn't come as a big surprise, but usage for UTF-32 is rather low, and as such we don't often get bug reports, compared to UTF-8. And existing testing is bit limited (there is some but only for simplest use cases).

[cowtowncoder]

Does indeed look like input is invalid. But I agree it should be handled in proper manner as an IOException or subtypes (I'll see how other Unicode decoding problems are reported).

[emilyselwood]

Indeed I'm not surprised. Most of our normal valid input is UTF-8. As I said this one popped out of a fuzz test. Thought I would provide a patch as it seemed reasonably simple to deal with using the existing unexpected end of file functionality.

[cowtowncoder]

@wselwood yes, much appreciated!

[cowtowncoder]

@wselwood Thank you for reporting this, providing fix -- I ended up merging manually, using test, partly for backporting, and partly as I wanted to change code in related parts. Fix will be in 2.8.9, 2.9.0(.pr4)

[emilyselwood]

Fair enough @cowtowncoder . You did a much better job of the fix and tests than I did. Thanks for the quick work.