Problems with File::Unpack

[coolo]

ldig@legaldb:~/cavil> ./script/cavil minion job -f 28673440
T: 445 files ...
Deep recursion on subroutine "File::Unpack::unpack" at /usr/lib/perl5/vendor_perl/5.18.2/File/Unpack.pm line 1170.
unpack('/data/auto-co/legal-bot/gcc46/5c4638b8b35ffd2d07223f6844e9f64e/.unpacked/gcc-4.6.2-20111212/libgo/go/archive/zip/testdata/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r','/data/auto-co/legal-bot/gcc46/5c4638b8b35ffd2d07223f6844e9f64e/.unpacked/gcc-4.6.2-20111212/libgo/go/archive/zip/testdata/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r/r'): recursion limit 200 at /home/ldig/cavil/script/../lib/Cavil/Checkout.pm line 154.
[2019-07-04 06:52:47.58874] [8318] [info] [27411
] Unpacked /data/auto-co/legal-bot/gcc46/5c4638b8b35ffd2d07223f6844e9f64e

[kraih]

At some point we'll just have to write our own unpacker.

[kraih]

We just had another issue like that. jnweiger/perl-File-Unpack#12 And since PRs are not merged, the openSUSE package now gets 7 or so custom patches applied.

[kraih]

Changed the title so we can just collect File::Unpack problems in one place. In case we actually find the time to write a replacement.

[kraih]

And we have another issue, this time via the dependency File::LibMagic.

libmagic TrueType font collection data, 1.0, 162 fonts, at 0x294 can't allocate 18446744073709551264 bytes (Cannot allocate memory) at /usr/lib/perl5/vendor_perl/5.26.1/x86_64-linux-thread-multi/File/LibMagic.pm line 206.

Which caused an iosevka-fonts update to get stuck in legal review.

[coolo]

That's a lot of bytes :)

[coolo]

But if that bug is in libmagic, we're talking CVE level here

[coolo]

hmm, so it's hard to classify this as DoS as the number of bytes are so insane that every machine gives up straight away :)

iosevka-aile.ttc:       TrueType font collection data, 1.0, 27 fonts, at 0x78 TrueType Font data, 17 tables, 1st "GDEF"
iosevka-curly-slab.ttc: ERROR: TrueType font collection data, 1.0, 162 fonts, at 0x294 can't allocate 18446744073709551264 bytes (Cannot allocate memory)
iosevka-curly.ttc:      ERROR: TrueType font collection data, 1.0, 162 fonts, at 0x294 can't allocate 18446744073709551264 bytes (Cannot allocate memory)

[kraih]

We recently ran into an issue unpacking the onedrive tarball.

.../.unpacked/onedrive-2.4.15/tests/State-of-the-art, challenges, and open issues in the integration of Internet of Things and Cloud Computing/State-of-the-art, challenges, and open issues in the integration of Internet of Things and Cloud Computing.txt path escaped....

Looks like there's a test case with a broken archive that trips up File::Unpack.

[kraih]

There's been another problematic test case in the buildah tarball, which results in an untar that runs endlessly.

buildah-1.27.0/tests/conformance/testdata/add/dir-not-dir/test.tar

[kraih]

Since it doesn't look like we will be rewriting the unpacker anytime soon, i've added an exclude file feature as a temporary workaround. 6f44ecb

[kraih]

I think it is time to fork File::Unpack, so we don't have to deal with all those custom patches anymore: https://github.com/openSUSE/perl-File-Unpack2