[SPARK-29483][BUILD] Bump Jackson to 2.10.0 #627
Conversation
Release blog: https://medium.com/cowtowncoder/jackson-2-10-features-cd880674d8a2 Fixes the following CVE's: https://www.cvedetails.com/cve/CVE-2019-16942/ https://www.cvedetails.com/cve/CVE-2019-16943/ Looking back, there were 3 major goals for this minor release: - Resolve the growing problem of “endless CVE patches”, a stream of fixes for reported CVEs related to “Polymorphic Deserialization” problem (described in “On Jackson CVEs… ”) that resulted in security tools forcing Jackson upgrades. 2.10 now includes “Safe Default Typing” that is hoped to resolve this problem. - Evolve 2.x API towards 3.0, based on changes that were done in master, within limits of 2.x API backwards-compatibility requirements. - Add JDK support for versions beyond Java 8: specifically add“module-info.class” for JDK9+, defining proper module definitions for Jackson components Full changelog: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10 Improved Scala 2.13 support: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10#scala Patches CVE's reported by the vulnerability scanner. No Ran `mvn clean install -DskipTests` locally. Closes apache#26131 from Fokko/SPARK-29483. Authored-by: Fokko Driesprong <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
|
Cherry-pick of apache#26131 @sjrand for consultation on whether or not this conflicts with our Hadoop's Jackson dependencies. @robert3005 for review. |
|
I'm not sure how we would determine whether there's a conflict with hadoop's jackson (2.7.8 in the version of hadoop we're using here). If I had to bet on it, I would say that there will be problems, just because it feels like there always are. I would say just try it and see what breaks. |
|
Also, stuff like this is one of the many reasons we should do #583 -- I think the shaded client JAR in hadoop3 makes things like this much less scary. |
|
@sjrand I agree with you that those shaded clients are nice from dependencies pov since hadoop has some old crufty libraries. Fortunately jackson didn't change that much and there's pretty good code coverage in spark tests from previous 2.6 -> 2.9 bump. The only thing that really broke between jackson 2.9 and 2.10 is serialization of java duration (revertable via config) but those are never used here. |
| jackson-xc-1.9.13.jar | ||
| jakarta.activation-api-1.2.1.jar |
There was a problem hiding this comment.
jakarta.activation-api-1.2.1.jar is a new version of javax.annotation-api-1.2.jar - we should tell maven to replace one with the other
There was a problem hiding this comment.
this is probably benign but could lead to some head scratching later on
| @@ -179,7 +179,7 @@ | |||
| <!-- for now, not running scalafmt as part of default verify pipeline --> | |||
| <scalafmt.skip>true</scalafmt.skip> | |||
| <codehaus.jackson.version>1.9.13</codehaus.jackson.version> | |||
| <fasterxml.jackson.version>2.9.7</fasterxml.jackson.version> | |||
| <fasterxml.jackson.version>2.10.0</fasterxml.jackson.version> | |||
|
ping? |
|
👍 since this is a cherry-pick |
Release blog: https://medium.com/cowtowncoder/jackson-2-10-features-cd880674d8a2
Fixes the following CVE's:
https://www.cvedetails.com/cve/CVE-2019-16942/
https://www.cvedetails.com/cve/CVE-2019-16943/
Looking back, there were 3 major goals for this minor release:
Full changelog: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10
Improved Scala 2.13 support: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10#scala
Patches CVE's reported by the vulnerability scanner.
Authored-by: Fokko Driesprong [email protected]
Signed-off-by: Dongjoon Hyun [email protected]