Refactor/Replace Compression & Decompression Filters #195

gsteel · 2024-11-08T00:38:01Z

Q	A
Documentation	not yet
BC Break	yes

Description

The old Compress and Decompress filters were a mess. Different adapters had different or unknown capabilities. Disregarding the now removed formats such as rar and lzf, the Tar and Zip adapters cannot simply compress arbitrary strings but can of course be used to compress and decompress files/archives, and the BZ2 and GZ adapters aren't suited to working with archives but are capable of compressing arbitary strings.

Instead of attempting to do all of these things with all the adapters, I've split the adapters into 2 interfaces:

ArchiveAdapterInterface
StringCompressionAdapterInterface

and replaced the Compress and Decompress filters with

CompressString
CompressToArchive
DecompressString
DecompressArchive

The file related filter will now accept paths, directories, PSR uploads and PHP $_FILES arrays.

The tools added here will also be useful for the other file related filters.

I've introduced the concept of an ArchiveAdapterResolver, shipping concrete implementations by default, that first scan the mime-type of the file to see if we have a matching adapter and then guess the filetype from the filename extension.

TODO:

Documentation and migration
A PR for the deprecations

gsteel · 2024-11-08T09:12:33Z

@froschdesign - I'd appreciate a review on the general direction of this patch when you get the chance, before I do more work on the outstanding items.

Reviews welcome from anyone else too :)

cc @alexmerlin, @ramchale, @Xerkus, @samsonasik

src/Compress/Bz2.php

ramchale

Looks really good

Added a couple of suggestions that I think mitigate the need to add the dev dependency for laminas/laminas-diactoros, but it depends how much mocking you want to use.

test/File/FileInformationTest.php

test/CompressToArchiveTest.php

Adds a new interface for string compression adapters with implementations for GZ and BZ2 along with dedicated string compression and decompression filters. Further refactoring of related filters will include more adapters and filters for dealing with files explicitly Signed-off-by: George Steel <[email protected]>

…erface Signed-off-by: George Steel <[email protected]>

…or php uploads Signed-off-by: George Steel <[email protected]>

Signed-off-by: George Steel <[email protected]>

Also enables removal of the legacy `Compression Algorithm` interface and abstract Signed-off-by: George Steel <[email protected]>

Signed-off-by: George Steel <[email protected]>

- Bz2 -> Bz2Adapter - Gz -> GzAdapter - Tar -> TarAdapter - Zip -> ZipAdapter Signed-off-by: George Steel <[email protected]>

Signed-off-by: George Steel <[email protected]>

This is so that matchers have more information to use to determine an appropriate adapter Signed-off-by: George Steel <[email protected]>

In order to recognise the type of archive when it's an uploaded file, and does not have a file name extension, the mime type matcher can be used to guess the mime type and match to the appropriate adapter. The aggregate matcher allows chaining of matchers so that multiple strategies can be attempted. Signed-off-by: George Steel <[email protected]>

…re methods are in use. Signed-off-by: George Steel <[email protected]>

Signed-off-by: George Steel <[email protected]>

.gitignore

froschdesign

Great improvements!

Signed-off-by: George Steel <[email protected]>

gsteel

Adding some notes to the PR for posterity!

gsteel · 2024-12-04T21:42:02Z

src/Compress/AggregateArchiveAdapterResolver.php

+use function array_values;
+use function sprintf;
+
+final class AggregateArchiveAdapterResolver implements ArchiveAdapterResolverInterface


This "Aggregate Adapter Resolver" can be used to compose a chain of concrete adapter resolvers. The first successful resolver wins.

gsteel · 2024-12-04T21:44:32Z

src/Compress/Bz2Adapter.php

+ *     blocksize?: int<1, 9>|null,
+ * }
+ */
+final class Bz2Adapter implements StringCompressionAdapterInterface


The BZ2 adapter now only supports string content.

In future, it will be possible to introduce a FileCompressionInterface that this adapter could also implement so that we can filter an arbitrary string, or a single un-compressed file to somefile.bz2

gsteel · 2024-12-04T21:46:01Z

src/Compress/FileExtensionArchiveAdapterResolver.php

+
+use const PATHINFO_EXTENSION;
+
+final class FileExtensionArchiveAdapterResolver implements ArchiveAdapterResolverInterface


Simply matches whatever.zip to use the Zip archive adapter based on filename extension. By default it is composed last in the chain, preferring mime-magic based mime detection.

gsteel · 2024-12-04T21:46:59Z

src/Compress/GzAdapter.php

+ *     mode?: 'deflate'|'compress',
+ * }
+ */
+final class GzAdapter implements StringCompressionAdapterInterface


As per the BZ2 adapter, the GZ adapter now only supports string content.

In future, it will be possible to introduce a FileCompressionInterface that this adapter could also implement so that we can filter an arbitrary string, or a single un-compressed file to somefile.gz

gsteel · 2024-12-04T21:48:11Z

src/Compress/MimeTypeArchiveAdapterResolver.php

+
+use function sprintf;
+
+final class MimeTypeArchiveAdapterResolver implements ArchiveAdapterResolverInterface


Map the detected mime-type of a file to an archive adapter - the default method of adapter selection

gsteel · 2024-12-04T21:50:51Z

src/File/FileInformation.php

+
+use const FILEINFO_MIME_TYPE;
+
+final class FileInformation


General purpose file information utility. This would have been marked as @internal, but we use it to match unknown files to archive adapters in the interface. Marking it as @internal would have been a pain in the ass for anyone who wants to implement a different detection strategy

gsteel added BC Break Documentation Needed Enhancement Feature Removal labels Nov 8, 2024

gsteel added this to the 3.0.0 milestone Nov 8, 2024

gsteel force-pushed the de-compress-refactor branch from c940406 to b954f28 Compare November 8, 2024 00:47

gsteel marked this pull request as draft November 8, 2024 09:07

froschdesign reviewed Nov 8, 2024

View reviewed changes

src/Compress/Bz2.php Outdated Show resolved Hide resolved

ramchale reviewed Nov 8, 2024

View reviewed changes

test/File/FileInformationTest.php Show resolved Hide resolved

test/CompressToArchiveTest.php Show resolved Hide resolved

gsteel added 20 commits November 19, 2024 21:50

Re-work the Tar and Zip adapters to implement a new File specific int…

4a3d996

…erface Signed-off-by: George Steel <[email protected]>

Introduce FileInformation to abstract file handling for paths, psr …

7f7f379

…or php uploads Signed-off-by: George Steel <[email protected]>

Extract tmp directory cleanup to utility

0490f22

Signed-off-by: George Steel <[email protected]>

Add compress to archive filter

235e233

Signed-off-by: George Steel <[email protected]>

Remove legacy Compress and Decompress filters

469999b

Also enables removal of the legacy `Compression Algorithm` interface and abstract Signed-off-by: George Steel <[email protected]>

Minor SA fixes

59aa232

Signed-off-by: George Steel <[email protected]>

Remove unused Exception

90efe75

Signed-off-by: George Steel <[email protected]>

Remove various unused fixtures

ada4977

Signed-off-by: George Steel <[email protected]>

Require minimum recent pear version for compat with 8.2+

e4b81ca

Signed-off-by: George Steel <[email protected]>

Rename compression adapters

4a460c4

- Bz2 -> Bz2Adapter - Gz -> GzAdapter - Tar -> TarAdapter - Zip -> ZipAdapter Signed-off-by: George Steel <[email protected]>

Add file-name matcher utility to determine the correct adapter to use

4014315

Signed-off-by: George Steel <[email protected]>

Use Archive_Tar's automatic archive format detection

ee45d8f

Signed-off-by: George Steel <[email protected]>

Also accept bz2 compressed tar archives

464616b

Signed-off-by: George Steel <[email protected]>

Ensure tests fail on PHPUnit deprecations

5382108

Signed-off-by: George Steel <[email protected]>

Initial filter for decompressing archives

b875c12

Signed-off-by: George Steel <[email protected]>

Test coverage improvements

2c7bf59

Signed-off-by: George Steel <[email protected]>

Add mime-type detection capabilities to FileInformation

9482a67

Signed-off-by: George Steel <[email protected]>

Refactor the matcher to accept a FileInformation instance.

7e1da0c

This is so that matchers have more information to use to determine an appropriate adapter Signed-off-by: George Steel <[email protected]>

gsteel added 3 commits November 19, 2024 21:58

Remove immutable annotation - it's not actually true and lots of impu…

10ece1d

…re methods are in use. Signed-off-by: George Steel <[email protected]>

Suppress unused method issues at config file level

9f8089f

Signed-off-by: George Steel <[email protected]>

Minor CS fix

5d5ddf8

Signed-off-by: George Steel <[email protected]>

gsteel force-pushed the de-compress-refactor branch from 7589e98 to 5d5ddf8 Compare November 19, 2024 21:59

gsteel added 5 commits November 19, 2024 22:01

Baseline non-use of AbstractFilter::getOptions

e166e73

Signed-off-by: George Steel <[email protected]>

Improve naming of archive adapters

3409872

Signed-off-by: George Steel <[email protected]>

Cover mismatched adapters during string de/compression

c6b74d2

Signed-off-by: George Steel <[email protected]>

Update documentation with new compression related filters

d134bfe

Signed-off-by: George Steel <[email protected]>

Ignore documentation related tooling and files

fb97552

Signed-off-by: George Steel <[email protected]>

gsteel removed the Documentation Needed label Dec 4, 2024

gsteel marked this pull request as ready for review December 4, 2024 10:27

gsteel requested a review from froschdesign December 4, 2024 10:27

gsteel force-pushed the de-compress-refactor branch from 1f0b00f to 4e70264 Compare December 4, 2024 10:29

froschdesign reviewed Dec 4, 2024

View reviewed changes

.gitignore Show resolved Hide resolved

froschdesign approved these changes Dec 4, 2024

View reviewed changes

Minor linting fix

5d3e4e3

Signed-off-by: George Steel <[email protected]>

gsteel force-pushed the de-compress-refactor branch from 4e70264 to 5d3e4e3 Compare December 4, 2024 21:35

Add DecompressArchive filter to the plugin manager

a818449

Signed-off-by: George Steel <[email protected]>

gsteel commented Dec 4, 2024

View reviewed changes

gsteel mentioned this pull request Dec 4, 2024

Deprecate legacy compression related adapters and filters #203

Merged

gsteel self-assigned this Dec 4, 2024

gsteel merged commit 46b5d30 into laminas:3.0.x Dec 4, 2024
16 of 17 checks passed

gsteel deleted the de-compress-refactor branch December 4, 2024 22:14

This was referenced Dec 4, 2024

Laminas\Filter v3 TODO #177

Open

V3 Migration Guide TODO #196

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor/Replace Compression & Decompression Filters #195

Refactor/Replace Compression & Decompression Filters #195

gsteel commented Nov 8, 2024 •

edited

Loading

gsteel commented Nov 8, 2024

ramchale left a comment

froschdesign left a comment

gsteel left a comment

gsteel Dec 4, 2024

gsteel Dec 4, 2024

gsteel Dec 4, 2024

gsteel Dec 4, 2024

gsteel Dec 4, 2024

gsteel Dec 4, 2024


		use const PATHINFO_EXTENSION;

		final class FileExtensionArchiveAdapterResolver implements ArchiveAdapterResolverInterface


		use function sprintf;

		final class MimeTypeArchiveAdapterResolver implements ArchiveAdapterResolverInterface

Refactor/Replace Compression & Decompression Filters #195

Refactor/Replace Compression & Decompression Filters #195

Conversation

gsteel commented Nov 8, 2024 • edited Loading

Description

gsteel commented Nov 8, 2024

ramchale left a comment

Choose a reason for hiding this comment

froschdesign left a comment

Choose a reason for hiding this comment

gsteel left a comment

Choose a reason for hiding this comment

gsteel Dec 4, 2024

Choose a reason for hiding this comment

gsteel Dec 4, 2024

Choose a reason for hiding this comment

gsteel Dec 4, 2024

Choose a reason for hiding this comment

gsteel Dec 4, 2024

Choose a reason for hiding this comment

gsteel Dec 4, 2024

Choose a reason for hiding this comment

gsteel Dec 4, 2024

Choose a reason for hiding this comment

gsteel commented Nov 8, 2024 •

edited

Loading