Not able to deserialize Enum with default typing after upgrading 2.15.4 -> 2.17.1 #4849

lenrok258 · 2024-12-11T13:50:00Z

Search before asking

I searched in the issues and found nothing similar.

Describe the bug

After bumping up the version to 2.17.1 I'm not longer able to deserialize values serialized with the version 2.15.4.
Moreover, values serialized with the new version (2.17.1) give error when deserializing.

Version Information

2.17.1

Reproduction

package com.kze.test;

import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.jsontype.BasicPolymorphicTypeValidator;
import com.fasterxml.jackson.databind.jsontype.PolymorphicTypeValidator;
import com.fasterxml.jackson.databind.jsontype.impl.StdTypeResolverBuilder;

import java.io.IOException;
import java.util.EnumSet;

import static com.fasterxml.jackson.annotation.JsonTypeInfo.As.PROPERTY;
import static com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping.NON_FINAL;
import static com.kze.test.TestClazz.TestEnum.TEST_ENUM_VALUE;

public class TestClazz {

    public enum TestEnum {
        TEST_ENUM_VALUE
    }

    public static void main(String[] args) throws IOException {

        ObjectMapper mapper = new ObjectMapper();

        final PolymorphicTypeValidator validator = BasicPolymorphicTypeValidator.builder()
                .allowIfSubType("com.kze.test")
                .allowIfSubType("java")
                .build();

        ObjectMapper.DefaultTypeResolverBuilder resolverBuilder
                = new ObjectMapper.DefaultTypeResolverBuilder(NON_FINAL, validator) {
            @Override
            public boolean useForType(JavaType t) {
                return true;
            }
        };

        StdTypeResolverBuilder stdTypeResolverBuilder = resolverBuilder
                .init(JsonTypeInfo.Id.CLASS, null)
                .inclusion(PROPERTY);

        mapper.setDefaultTyping(stdTypeResolverBuilder);
        mapper.configure(JsonParser.Feature.INCLUDE_SOURCE_IN_LOCATION, true);

        EnumSet<TestEnum> input = EnumSet.of(TEST_ENUM_VALUE);
        String inputJson = mapper.writeValueAsString(input);
        System.out.println(inputJson);

        Object inputDeserialized = mapper.readValue(inputJson, Object.class);
        System.out.println(inputDeserialized);
    }

}

Version 2.15.4 output

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]
[TEST_ENUM_VALUE]

Version 2.17.1 error:

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]
Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_STRING), expected START_ARRAY: need Array value to contain `As.WRAPPER_ARRAY` type information for class com.kze.test.TestClazz$TestEnum
 at [Source: (String)"["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]"; line: 1, column: 56] (through reference chain: java.util.RegularEnumSet[0])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1913)
	at com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1699)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._locateTypeId(AsArrayTypeDeserializer.java:141)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:96)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromScalar(AsArrayTypeDeserializer.java:66)
	at com.fasterxml.jackson.databind.deser.std.StdScalarDeserializer.deserializeWithType(StdScalarDeserializer.java:66)
	at com.fasterxml.jackson.databind.deser.std.EnumSetDeserializer._deserialize(EnumSetDeserializer.java:281)
	at com.fasterxml.jackson.databind.deser.std.EnumSetDeserializer.deserialize(EnumSetDeserializer.java:246)
	at com.fasterxml.jackson.databind.deser.std.EnumSetDeserializer.deserialize(EnumSetDeserializer.java:24)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:120)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromArray(AsArrayTypeDeserializer.java:53)
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromAny(AsPropertyTypeDeserializer.java:238)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR.deserializeWithType(UntypedObjectDeserializerNR.java:112)
	at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:74)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4905)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3848)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3816)
	at com.kze.test.TestClazz.main(TestClazz.java:52)

Expected behavior

Should be able to deserialized values

Additional context

No response

The text was updated successfully, but these errors were encountered:

JooHyukKim · 2024-12-11T14:23:18Z

Seems similar to the one I am working on ryt now FasterXML/jackson-modules-java8#335

lenrok258 · 2024-12-11T15:00:06Z

I'm not sure the route cause is the same.

In my case serializing Enum does not preceded it with a class name as a first array element

["TEST_ENUM_VALUE"]

From some reason older version is able to read it. The newer one throws an error, expecting class name at index 0.

As a proof if you proceed a serialized Enum with a class name at the index "0" it works perfectly fine:

["com.kze.test.TestClazz$TestEnum", "TEST_ENUM_VALUE"]

[edit]
So the bottom line is:
newer Jackson serialize Enum without explicitly putting class name as a first value in the array (as in the all other cases) and throws an error it trying to deserialize it

JooHyukKim · 2024-12-11T15:17:41Z

There is ${rootDir}/release-notes to check what's changed from 2.15.4 -> later version. Could you try out with 2.16.0 as well? It would help track down which version

lenrok258 · 2024-12-11T15:19:35Z

Just as a note, using recommended in docs activateDefaultTyping() instead of deprecated setDefaultTyping() doesn't help.

        ObjectMapper mapper = new ObjectMapper();
        PolymorphicTypeValidator polymorphicTypeValidator =  BasicPolymorphicTypeValidator.builder()
                .allowIfSubType("com.kze.test")
                .allowIfSubType("java")
                .build();
        mapper.activateDefaultTypingAsProperty(polymorphicTypeValidator, EVERYTHING, "type");
        mapper.registerModule(new JavaTimeModule());

lenrok258 · 2024-12-11T17:31:00Z

There is ${rootDir}/release-notes to check what's changed from 2.15.4 -> later version. Could you try out with 2.16.0 as well? It would help track down which version

Works fine with 2.16.0.

I've checked 2.17.3 and it crashes exactly as the 2.17.1.

cowtowncoder · 2024-12-11T22:56:32Z

While it probably makes no difference, testing should try the latest patch -- 2.18.2 -- to eliminate possibility of the issue being fixed.

I don't think any change in-between versions would change behavior purposefully, but this is pretty complicated case so probably a side-effect of a fix.

Now: difference between:

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]

and

[TEST_ENUM_VALUE]

is that latter has no (polymorphic) type id, just EnumSet value; and former includes type id using so-called "Wrapper array"; that is, JSON Array where [0] is type id and [1] is the actual value).

And deserialization expects one or the other, depending on whether polymorphic handling is indicated. Due to default typing being enabled, presumably second structure should be serialized.

It is also possible that the discrepancy between expecting inclusion (or not) of Type Id, via default Typing, differs between read- and writer-side due to Java Type Erasure -- serializing polymorphic root values, as is done here, is always riskier than serializing values referred indirectly (because in latter case full type information is available, unlike in root value case).

lenrok258 · 2024-12-12T14:35:49Z

While it probably makes no difference, testing should try the latest patch -- 2.18.2 -- to eliminate possibility of the issue being fixed.

I've tested it with the 2.18.2 and it's still not working.

What I found out is the problematic bit is the EnumSet. If I replace it with regular Set it works perfectly fine with all the versions!
In my example attached to this ticket if you change the line

EnumSet<TestEnum> input = EnumSet.of(TEST_ENUM_VALUE);

to

Set<TestEnum> input = Set.of(TEST_ENUM_VALUE);

mapper is able to deserialize json, no exception is thrown and the output is:

["java.util.ImmutableCollections$Set12",[["com.kze.test.TestClazz$TestEnum","TEST_ENUM_VALUE"]]]
[TEST_ENUM_VALUE]

So I guess it is rather rare case as it needs to have EnumSet with enum values serialized and deserialized with default typing enabled.

Nevertheless there is a decadency between serializer (decides not to include type as a index 0) and deserializer which, in such case, requires type at the beginning of the array.

[edit]
Not having type for enum in EnumSet seems sensible as there's already an enum type as a generic type in EnumSet.

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]

My assumption is that deserializer logic is too strict thus throwing Exception even if the type could be inferred from EnumSet

cowtowncoder · 2024-12-12T17:08:30Z

@lenrok258 good analysis. Yes, not super common problem but definitely something that'd be good to fix.

EnumSet is quite special since type id serialized is the only type Jackson adds generic type info on, because otherwise it is impossible to resolve actual type (must use Enum class in constructor).

lenrok258 · 2024-12-13T09:00:31Z

@cowtowncoder
Any ideas for a workaround?
As values are in a production db I need to be able to deserialize them using newer Jackson.

Reading

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]

will always end up with an Exception (_com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_STRING), expected START_ARRAY: _).

I've tried to add @JsonTypeInfo

@JsonTypeInfo(use = JsonTypeInfo.Id.NONE)
public EnumSet<TestEnum> testEnumCollection = EnumSet.of(TEST_ENUM_VALUE);

on the EnumSet property but with no luck. Seems like deserializer doesn't care and still expects array wrapper on the enum value itself (no only on an EnumSet).

JooHyukKim · 2024-12-13T09:03:38Z

For urgent case, u may want tp update the varchar "j.u.EnumSet" to Set then modify code to deserialize into Set not EnumSet.
But... even better create a clone column and leave the existing data as is.

lenrok258 · 2024-12-13T09:11:20Z

For urgent case, u may want tp update the varchar "j.u.EnumSet" to Set then modify code to deserialize into Set not EnumSet. But... even better create a clone column and leave the existing data as is.

Unformtunately as the json in db has the java.util.EnumSet<com.kze.test.TestClazz$TestEnum> as the type I'm not able to deserialize it into Set.

I think my only option is to write a migration code which takes all the values from the db and saves them again with EnumSet->Set migrated.

lenrok258 · 2024-12-13T10:57:21Z

Just to let you know, applying @JsonTypeInfo(use = NONE) on the enum declaration instead of on EnumSet property seems to be working fine.

@JsonTypeInfo(use = NONE)
public enum TestEnum {
        TEST_ENUM_VALUE
    }

Reading

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]

doesn't throw Exception in newer versions of Jackson so it could be treated as a good workaround for now.

JooHyukKim · 2024-12-16T14:45:04Z

Thank you for sharing workaround @lenrok258! 🙏🏼🙏🏼
Hopefully I get to find solution tonight (taking a shot at it) 🥲

JooHyukKim · 2024-12-16T16:14:36Z

@lenrok258 Btw, are you able to upgrade to 2.18 version and not 2.17? ~~2.17 is no longer maintained and only fatal security issues would be back-ported.~~

EDIT : I remembered wrong, Tatu reminded me that backporting 2.17 is still possible, tho not for all issues I guess?

cowtowncoder · 2024-12-16T16:22:30Z

Quick note: 2.17 branch is still open not just for security fixes (unlike 1.16). Although each patch will be evaluated for risk so backporting may or may not be done for any particular one.

…

On Mon, Dec 16, 2024 at 11:15 AM Kim, Joo Hyuk ***@***.***> wrote: @lenrok258 <https://github.com/lenrok258> Btw, are you able to upgrade to 2.18 version and not 2.17? 2.17 is no longer maintained and only fatal security issues would be back-ported. — Reply to this email directly, view it on GitHub <#4849 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANOGMIHYW5HJGM3IYSQI32F34AHAVCNFSM6AAAAABTNRAOAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBWGA2TEMBUHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

lenrok258 · 2024-12-16T16:29:46Z

@lenrok258 Btw, are you able to upgrade to 2.18 version and not 2.17? 2.17 is no longer maintained and only fatal security issues would be back-ported.

Yes, that's fine. Any, even the newest version would be fine.

JooHyukKim · 2024-12-17T00:40:08Z

@cowtowncoder right, I thought 2.17 version is closed for some reason 🙉. Let's talk versions when you get back! 👍🏼

cowtowncoder · 2024-12-17T05:46:22Z

Ok, I think this was caused by fix to #4214 which enabled use of (polymorphic) type id on deserialization (see EnumSetDeserializer) but NOT on serialization (see EnumSetSerializer). It is curious that such a change resolved that particular problem.

Anyway: I think the fix needed is to add handling in EnumSetSerializer; I'll see if I can modify #4857 with that idea.

lenrok258 added the to-evaluate Issue that has been received but not yet evaluated label Dec 11, 2024

cowtowncoder removed the to-evaluate Issue that has been received but not yet evaluated label Dec 11, 2024

JooHyukKim mentioned this issue Dec 16, 2024

[databind#4849] Allow standard defaultTyping with EnumSet<E> #4857

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Not able to deserialize Enum with default typing after upgrading 2.15.4 -> 2.17.1 #4849

Not able to deserialize Enum with default typing after upgrading 2.15.4 -> 2.17.1 #4849

lenrok258 commented Dec 11, 2024 •

edited

Loading

JooHyukKim commented Dec 11, 2024

lenrok258 commented Dec 11, 2024 •

edited

Loading

JooHyukKim commented Dec 11, 2024

lenrok258 commented Dec 11, 2024

lenrok258 commented Dec 11, 2024 •

edited

Loading

cowtowncoder commented Dec 11, 2024 •

edited

Loading

lenrok258 commented Dec 12, 2024 •

edited

Loading

cowtowncoder commented Dec 12, 2024

lenrok258 commented Dec 13, 2024 •

edited

Loading

JooHyukKim commented Dec 13, 2024

lenrok258 commented Dec 13, 2024

lenrok258 commented Dec 13, 2024 •

edited

Loading

JooHyukKim commented Dec 16, 2024

JooHyukKim commented Dec 16, 2024 •

edited

Loading

cowtowncoder commented Dec 16, 2024 via email

lenrok258 commented Dec 16, 2024

JooHyukKim commented Dec 17, 2024

cowtowncoder commented Dec 17, 2024

Not able to deserialize Enum with default typing after upgrading 2.15.4 -> 2.17.1 #4849

Not able to deserialize Enum with default typing after upgrading 2.15.4 -> 2.17.1 #4849

Comments

lenrok258 commented Dec 11, 2024 • edited Loading

Search before asking

Describe the bug

Version Information

Reproduction

Expected behavior

Additional context

JooHyukKim commented Dec 11, 2024

lenrok258 commented Dec 11, 2024 • edited Loading

JooHyukKim commented Dec 11, 2024

lenrok258 commented Dec 11, 2024

lenrok258 commented Dec 11, 2024 • edited Loading

cowtowncoder commented Dec 11, 2024 • edited Loading

lenrok258 commented Dec 12, 2024 • edited Loading

cowtowncoder commented Dec 12, 2024

lenrok258 commented Dec 13, 2024 • edited Loading

JooHyukKim commented Dec 13, 2024

lenrok258 commented Dec 13, 2024

lenrok258 commented Dec 13, 2024 • edited Loading

JooHyukKim commented Dec 16, 2024

JooHyukKim commented Dec 16, 2024 • edited Loading

cowtowncoder commented Dec 16, 2024 via email

lenrok258 commented Dec 16, 2024

JooHyukKim commented Dec 17, 2024

cowtowncoder commented Dec 17, 2024

lenrok258 commented Dec 11, 2024 •

edited

Loading

lenrok258 commented Dec 11, 2024 •

edited

Loading

lenrok258 commented Dec 11, 2024 •

edited

Loading

cowtowncoder commented Dec 11, 2024 •

edited

Loading

lenrok258 commented Dec 12, 2024 •

edited

Loading

lenrok258 commented Dec 13, 2024 •

edited

Loading

lenrok258 commented Dec 13, 2024 •

edited

Loading

JooHyukKim commented Dec 16, 2024 •

edited

Loading