Can't mount .tar, .tar.bz2, .tar.gz #43

tuxArg · 2024-05-27T01:18:40Z

Hi, openning new issue here and closing #42 .

Basically I am unable to mount any .tar (compressed or not) file but it works using directly the command line.

I've tried this that works ok:

mkdir -p ~/testFilesArchive/extracted
cd ~/testFilesArchive/
cp -a /var/www/nextcloud/apps/files_archive/vendor/ .
cat <<'EOF' > test-tar-bz2.php
<?php
require (__DIR__.'/./vendor/autoload.php');
use \wapmorgan\UnifiedArchive\UnifiedArchive;
$archive = UnifiedArchive::open('test.tar.bz2');
$extracted = $archive->extract('./extracted');
echo 'Number of extracted files: ' . $extracted.PHP_EOL;
EOF
tar cfj test.tar.bz2 test-tar-bz2.php
php test-tar-bz2.php

When trying mounting in nextcloud, it results in a 500 http code.

zip files work ok and can be mounted.
.tar.xz files don't work for me, even using UnifiedArchive directly.

The text was updated successfully, but these errors were encountered:

rotdrop · 2024-05-27T07:20:27Z

So ... I suppose then that it is something with wrong or better unexpected mime-types.

I would suggest to start with uncompressed tar archives, although both -- .tar.[ADDITIONAL EXTENSION] and .tar should work. I have attached two screenshots with the details view of a compressed and an uncompressed tar archive.

First: do you have that archive information in the details tab at all? If so, what is the mime type and archive format, and which archive backend is in use? Please note that .tar.xz is not a "mime-extension", this is just a compressed format .xz which is "correct".

If there is no archive tab in the details view, then the issue is that the mime type is not accepted by files_archive. Then it would probably good to augment the settings sections by the supported mime types and archive backends to ease debugging (would probably a good idea anyway).

tuxArg · 2024-05-27T11:49:09Z

Here's the screenshot of a zip file that works:

And a tar file that doesn't:

When I click on the Archive tab there's a 400 http code response.
URL: /apps/files_archive/archive/info/%252FDocuments%252FnoBackup%252FtestFiles%252Ftest.tar
Here's the output:

{
    "messages": [
        "Error: Entry xt does not exist"
    ],
    "archiveStatus": 0,
    "archiveInfo": {
        "defaultMountPoint": "test",
        "defaultTargetBaseName": "test"
    }
}

rotdrop · 2024-05-27T11:56:32Z

Well, this test.tar, does it contain an archive member name xt?

tuxArg · 2024-05-27T12:00:16Z

Well, this test.tar, does it contain an archive member name xt?

No, Here I attach it. (unfortunately github doesn't allow tar files, so it's zipped, inside there's the test.tar file)
test.tar.zip

rotdrop · 2024-05-27T12:04:43Z

Perhaps the xt is some spurious remnant from .txt ... anyhow. I will now first inspect the tar file and then we will see. Are there any other apps which use pear/archive_tar? Could perhaps be checked with the Unix find command in the apps folder ...

tuxArg · 2024-05-27T12:19:54Z

Perhaps the xt is some spurious remnant from .txt ... anyhow.

Indeed! I've tried with a test.php.tar, and the message was "Entry hp does not exist".

Are there any other apps which use pear/archive_tar? Could perhaps be checked with the Unix find command in the apps folder ...

I don't think so:

me@nextcloud:/var/www/nextcloud/apps$ find ./ -name archive_tar 
./files_archive/vendor/pear/archive_tar

rotdrop · 2024-05-27T12:32:43Z

I am trying to find the piece of code stating 'Entry xt does not exist' but am not able to find it. The exception should also be logged to the Nextcloud log, can you please look it up? I am not sure were this exception is thrown . Thanks.

tuxArg · 2024-05-27T12:42:19Z

I am trying to find the piece of code stating 'Entry xt does not exist' but am not able to find it. The exception should also be logged to the Nextcloud log, can you please look it up? I am not sure were this exception is thrown . Thanks.

Please see my previous message, the xt is a substring of test.txt.

Here it is in the logs:
(note that /var/www/nextcloud/apps is a symlink to /mnt/nc_data/apps)

{
  "reqId": "zkpBqJn5SeSA73NdhFCF",
  "level": 3,
  "time": "2024-05-27T12:15:27 00:00",
  "remoteAddr": "10.8.0.100",
  "user": "testUser",
  "app": "files_archive",
  "method": "POST",
  "url": "/index.phps_archive/archive/info/%2FDocuments%2FnoBackup%2FtestFiles%2Ftest.php.tar",
  "message": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php:157: OCA\\FilesArchive\\Controller\\ArchiveController::info: Caught an Exception",
  "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
  "version": "29.0.1.1",
  "exception": {
    "Exception": "BadMethodCallException",
    "Message": "Entry hp does not exist",
    "Code": 0,
    "Trace": [
      {
        "file": "/mnt/nc_data/apps/files_archive/vendor/wapmorgan/unified-archive/src/Drivers/TarByPhar.php",
        "line": 181,
        "function": "offsetGet",
        "class": "PharData",
        "type": "->"
      },
      {
        "file": "/mnt/nc_data/apps/files_archive/vendor/wapmorgan/unified-archive/src/UnifiedArchive.php",
        "line": 498,
        "function": "getFileData",
        "class": "wapmorgan\\UnifiedArchive\\Drivers\\TarByPhar",
        "type": "->"
      },
      {
        "file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
        "line": 378,
        "function": "getFileData",
        "class": "wapmorgan\\UnifiedArchive\\UnifiedArchive",
        "type": "->"
      },
      {
        "file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
        "line": 365,
        "function": "getFiles",
        "class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
        "type": "->"
      },
      {
        "file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
        "line": 329,
        "function": "getCommonDirectoryPrefix",
        "class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
        "type": "->"
      },
      {
        "file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
        "line": 286,
        "function": "getArchiveInfo",
        "class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
        "type": "->"
      },
      {
        "file": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php",
        "line": 146,
        "function": "open",
        "class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 232,
        "function": "info",
        "class": "OCA\\FilesArchive\\Controller\\ArchiveController",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 138,
        "function": "executeController",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/AppFramework/App.php",
        "line": 184,
        "function": "dispatch",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Route/Router.php",
        "line": 338,
        "function": "main",
        "class": "OC\\AppFramework\\App",
        "type": "::"
      },
      {
        "file": "/var/www/nextcloud/lib/base.php",
        "line": 1050,
        "function": "match",
        "class": "OC\\Route\\Router",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/index.php",
        "line": 49,
        "function": "handleRequest",
        "class": "OC",
        "type": "::"
      }
    ],
    "File": "/mnt/nc_data/apps/files_archive/vendor/wapmorgan/unified-archive/src/Drivers/TarByPhar.php",
    "Line": 181,
    "message": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php:157: OCA\\FilesArchive\\Controller\\ArchiveController::info: Caught an Exception",
    "exception": {},
    "CustomMessage": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php:157: OCA\\FilesArchive\\Controller\\ArchiveController::info: Caught an Exception"
  }
}

rotdrop · 2024-05-27T13:04:54Z

Oh well. It is TarByPhar, not TarByPear (so the requested find of tar_archive was unnecessary, sorry for that).

However, this means the error messages is generated by the phar extension.

Ok. The way I would go would be to edit that TarByPhar.php and throw out log messages in order to check where things break. E.g. in line 181 it would be interesting to know whether the there mentioned parametere $fileName is already truncated. Same for ArchiveService::getFiles() and so on.

rotdrop · 2024-05-27T13:09:06Z

Is the data directory also a symlink which maybe confuses the phar extension?

rotdrop · 2024-05-27T13:13:58Z

Yes. That is the problem. Or the cause of the manifestation of whatever problem we really have. I have just now replaced my data directory by a symlink like this:

mv nextcloud-git-29 nextcloud-git-29-blah
ln -s nextcloud-git-29-blah nextcloud-git-29

Then everything else still works, but the phar-extension does not seem to like this change. The error message is slightly different:

Entry .tar/test.txt does not exist

rotdrop · 2024-05-27T13:17:49Z

You could try to replace your symlink by a bind mount, just to check if this changes anything. The phar-extension problems could perhaps be cured by using realpath before feeding the archive file into UnifiedArchive. Still it would be good to inspect the situation further and decide whether to escalate this problem to e.g. the authors of the phar extension.

rotdrop · 2024-05-27T13:30:08Z

The realpath() approach would consist of the following: replace

nextcloud-app-files-archive/lib/Toolkit/Service/ArchiveService.php

Lines 231 to 234 in 44baceb

    
           private static function getLocalPath(File $fileNode):string 
        
           { 
        
             return $fileNode->getStorage()->getLocalFile($fileNode->getInternalPath()); 
        
           }

by

private static function getLocalPath(File $fileNode):string 
{ 
   return realpath($fileNode->getStorage()->getLocalFile($fileNode->getInternalPath())); 
}

tuxArg · 2024-05-27T13:34:28Z

You could try to replace your symlink by a bind mount,

I was already doing that! And it works now!!!! Thanks a lot! I hope there's no side effect.

Now, I can't mount [.tar].xz and [.tar].gz files, should I open a new issue?

rotdrop · 2024-05-27T13:47:06Z

You could try to replace your symlink by a bind mount,

I was already doing that! And it works now!!!! Thanks a lot! I hope there's no side effect.

Now, I can't mount [.tar].xz and [.tar].gz files, should I open a new issue?

Oops. Still? Neither can I on my laptop test setup, but it works @home in another instance. The actual backend driver in use is then indeed TarByPear, no longer TarByPhar. Please leave the issue open for the moment; the topic is correct although different things may cause the problems.

rotdrop · 2024-05-27T13:59:22Z

Ok, @home I can mount .tar.xz, .tar.bz2, but .tar.gz is not recognized. @work with my notebook neither of the archives works, and .tar.gz is not recognized. This means I have one system where I can reproduce the problem locally, so it should not be too difficult to solve. However, not now. I'll come back to it later.

rotdrop · 2024-05-27T20:08:20Z

Nextcloud (at least my instance) likes to classify .gz as application/gzip while the app -- or UnifiedArchive -- expects application/x-gzip. More funny is .tgz which is classified as application/x-compressed. Strange enough, but of course, simple thing of a translation between Nextcloud mime-type flavour and UnifiedArchive mime-type flavour.

This does not address the problem that on some systems .tar.WHATEVER does not work. Will probably have to wait until tomorrow.

tuxArg · 2024-05-27T20:30:59Z

Well, according to https://www.rfc-editor.org/rfc/rfc9110.html#name-gzip-coding "x-gzip" and "gzip" are equivalent.

rotdrop · 2024-05-28T09:52:40Z

Well, according to https://www.rfc-editor.org/rfc/rfc9110.html#name-gzip-coding "x-gzip" and "gzip" are equivalent.

Yes "of course", but that does not help if Nextcloud returns one flavour and your app expects the other. However, I think that the latest pre-release fixes the archive-not-detected problems (.tar.gz, .tgz). I did not yet address the problem that on my laptop I still cannot open those archives.

Oops. I missed that realpath() change in the latest pre-release.

rotdrop · 2024-05-28T12:54:46Z

Ok, about .tar.bz2: on my Laptop UA likes to use its AlchemyZippy backend which has a bug; I'll shortly will place an issue about it. On my desktop PC @home UA likes to use the PHAR backend (where the PHP extension is buggy, but can be worked around using realpath()).

rotdrop · 2024-05-28T14:15:30Z

The alchemy/zippy bug got its work around in 761b4bf until wapmorgan/UnifiedArchive#44 has been merged "upstream". Can you please re-check whether perhaps now it works for you also? Why UA chooses AlchemyZippy on my laptop for .tar.bz and ext-phar on my desktop remains an open question -- but maybe I do not want to search for the answer ;)

tuxArg · 2024-05-28T15:02:33Z

Hi, .tar.bz2 are working fine using TarByPhar for me after bind mounting.
Do you mean .tar.gz?

rotdrop · 2024-05-28T15:05:09Z

Mmh. Ok then: are there any problems remaining? (restricting ourselves to the problems you mentioned in this issue ;) )

tuxArg · 2024-05-28T15:12:51Z

Of the file formats I've tried: .tar.gz and .tar.xz still fail. The .xz one has the Archive tab with all fields "unknown" and the .gz one hasn't. But you can reproduce that, right?

rotdrop · 2024-05-29T08:49:18Z

No, the xz stuff just also works. But please let me double check this later.

rotdrop · 2024-06-04T12:33:29Z

No, the xz stuff just also works. But please let me double check this later.

Ok. At home I have lzma installed, not on the laptop. If we look at the support matrix locally (no on the web, I must filed a bug report wapmorgan/UnifiedArchive#45) we have the following output:

claus@anaxagoras /var/www/dev2/htdocs/nextcloud-git-29/apps/files_archive $ php8.3  vendor/bin/cam system:formats
+----------------------+------------------------+------------+------+------+------+------+--------+--------+--------+-----------+-----+-----+-----+------+-----+-----+-----+-----+-----+-----+-----+------+-------+
| driver / format      | driver type            | zip        | rar  | gz   | bz2  | xz   | tar    | tgz    | tbz2   | 7z        | cab | iso | arj | uefi | gpt | mbr | msi | dmg | rpm | deb | udf | txz  | tar.z |
+----------------------+------------------------+------------+------+------+------+------+--------+--------+--------+-----------+-----+-----+-----+------+-----+-----+-----+-----+-----+-----+-----+------+-------+
| Zip                  | php extension          | oOtxsadTcC |      |      |      |      |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| Rar                  | php extension          |            | oOxs |      |      |      |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| Gzip                 | php extension          |            |      | oxsc |      |      |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| Bzip                 | php extension          |            |      |      | oxsc |      |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| Lzma                 | php extension          |            |      |      |      | oxsc |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| TarByPhar            | php extension          | oxsadc     |      |      |      |      | oxsadc | oxsadc | oxsadc |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| SevenZip             | utilities + php bridge | oOxadcC    |      |      |      |      | oxadc  |        |        | oOtxadTcC | ox  | ox  | ox  | ox   | ox  | ox  | ox  | ox  | ox  | ox  | ox  |      |       |
| AlchemyZippy         | utilities + php bridge | oxadc      |      |      |      |      | oxadc  | oxadc  | oxadc  |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| SplitbrainPhpArchive | pure php               |            |      |      |      |      |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| NelexaZip            | pure php               | oOtxadTcCS |      |      |      |      |        |        |        |           |     |     |     |      |     |     |     |     |     |     |     |      |       |
| TarByPear            | pure php               |            |      |      |      |      | oxac   | oxac   | oxac   |           |     |     |     |      |     |     |     |     |     |     |     | oxac | oxac  |
| Iso                  | pure php               |            |      |      |      |      |        |        |        |           |     | ox  |     |      |     |     |     |     |     |     |     |      |       |
| Cab                  | pure php               |            |      |      |      |      |        |        |        |           | ox  |     |     |      |     |     |     |     |     |     |     |      |       |
+----------------------+------------------------+------------+------+------+------+------+--------+--------+--------+-----------+-----+-----+-----+------+-----+-----+-----+-----+-----+-----+-----+------+-------+
o - open
O - open (+password)
t - get comment
x - extract
s - stream
a - append
d - delete
T - set comment
c - create
C - create (+password)
S - create (as string)

Actually going back to you initial issue #42 it would perhaps be good to either document the use of cam a bit better or point the users more "drastically" to wapmorgen/UnifiedArchive. In principle the installation suggestions are listed by

heinecj@nbnmh110:/var/www/html/nextcloud-git-29/apps/files_archive$ vendor/bin/cam system:drivers
driver               type                   description / installation                                                        
Zip                  php extension          adapter for ext-zip (1.7.3)                                                       
Rar                  php extension          install [rar] extension.                                                          
                                            Can be installed with pecl: pecl install rar                                      
Gzip                 php extension          adapter for ext-zlib (1.2.11)                                                     
Bzip                 php extension          install [bz2] php extension                                                       
Lzma                 php extension          install [xz] extension                                                            
                                            For 5.x: https://github.com/payden/php-xz                                         
                                            For 7.x/8.x: https://github.com/codemasher/php-ext-xz                             
TarByPhar            php extension          adapter for ext-phar                                                              
SevenZip             utilities + php bridge php-library and console program (16.02)                                           
AlchemyZippy         utilities + php bridge php-library and console programs                                                  
SplitbrainPhpArchive pure php               install library [splitbrain/php-archive]: composer require splitbrain/php-archive 
NelexaZip            pure php               nelexa/zip driver                                                                 
TarByPear            pure php               php-library for tar                                                               
Iso                  pure php               iso archives reader                                                               
Cab                  pure php               php-library

Installing the lzma php extension seems to be the only option at the moment.

rotdrop · 2024-06-04T12:46:05Z

... and yes, there must be a test and a suitable error message for this case ...

rotdrop · 2024-06-04T14:45:05Z

6d4d4c1 is an attempt to have a less unfriendly error handling. The error text is displayed as toast and also contains some installation instructions. Perhaps one could also provide the summary generated by the system:drivers and system:formats command on the admin settings page.

tuxArg · 2024-06-04T15:25:42Z

Ok, installing xz extension from https://github.com/codemasher/php-ext-xz made it work!

How all this "mounting" works. Is it really extracted on demand or it's extracted on a temp dir until it's unmounted?

tuxArg · 2024-06-04T15:27:55Z

Perhaps one could also provide the summary generated by the system:drivers and system:formats command on the admin settings page.

I couldn't run php vendor/bin/cam system:formats in my box. It complained about symfony/console being missing.

rotdrop · 2024-06-05T10:01:22Z

Perhaps one could also provide the summary generated by the system:drivers and system:formats command on the admin settings page.

I couldn't run php vendor/bin/cam system:formats in my box. It complained about symfony/console being missing.

composer install or require could have helped for that. Anyhow, the current state of the affair is that I have hopefully improved the error message:

and in the admin settings there is the support matrix and driver overview (by executing the console commands without console, Symfony supports this via its BufferedOutput class, https://symfony.com/doc/current/console/command_in_controller.html:

hampoelz mentioned this issue May 28, 2024

Can't mount large .zip #44

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't mount .tar, .tar.bz2, .tar.gz #43

Can't mount .tar, .tar.bz2, .tar.gz #43

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024 •

edited

Loading

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 28, 2024

rotdrop commented May 28, 2024

rotdrop commented May 28, 2024

tuxArg commented May 28, 2024

rotdrop commented May 28, 2024

tuxArg commented May 28, 2024 •

edited

Loading

rotdrop commented May 29, 2024

rotdrop commented Jun 4, 2024

rotdrop commented Jun 4, 2024

rotdrop commented Jun 4, 2024

tuxArg commented Jun 4, 2024

tuxArg commented Jun 4, 2024

rotdrop commented Jun 5, 2024

Can't mount .tar, .tar.bz2, .tar.gz #43

Can't mount .tar, .tar.bz2, .tar.gz #43

Comments

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024 • edited Loading

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

rotdrop commented May 27, 2024

tuxArg commented May 27, 2024

rotdrop commented May 28, 2024

rotdrop commented May 28, 2024

rotdrop commented May 28, 2024

tuxArg commented May 28, 2024

rotdrop commented May 28, 2024

tuxArg commented May 28, 2024 • edited Loading

rotdrop commented May 29, 2024

rotdrop commented Jun 4, 2024

rotdrop commented Jun 4, 2024

rotdrop commented Jun 4, 2024

tuxArg commented Jun 4, 2024

tuxArg commented Jun 4, 2024

rotdrop commented Jun 5, 2024

tuxArg commented May 27, 2024 •

edited

Loading

tuxArg commented May 28, 2024 •

edited

Loading