Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] request takes too long #150

Closed
vntrungld opened this issue May 17, 2020 · 8 comments
Closed

[BUG] request takes too long #150

vntrungld opened this issue May 17, 2020 · 8 comments
Labels

Comments

@vntrungld
Copy link

What version of Laravel are you using? (nothing lower than v6).

v6

is it an issue related to installation or usage ?

Usage

what is the error you are getting ?

extreme slow when you try to get files

Extra Info "attaching a screen shot would be very helpful"

When I try to get 34 files from a folder and it takes 2.4 min
image

When I try to debug it by this
image

And the result:
image
It's still slow but faster than normal

But when I tried to dive deeper, I saw that the function iterator_to_array makes the request slow extra 2min for just transforming from PHP Object to Array. Do we have a different function to use instead of iterator_to_array?

Do You Know A Solution For this Issue ?

Not yet

@vntrungld vntrungld added the bug label May 17, 2020
@ctf0
Copy link
Owner

ctf0 commented May 17, 2020

i believe its part of https://github.com/jhofm/flysystem-iterator which was suppose to make things faster.

also what driver r u using, aws, google, public ?

@ctf0 ctf0 changed the title [BUG] [BUG] request take too long May 17, 2020
@vntrungld vntrungld changed the title [BUG] request take too long [BUG] request takes too long May 17, 2020
@vntrungld
Copy link
Author

Thanks for your reply. I'm using digital ocean spaces

@ctf0
Copy link
Owner

ctf0 commented May 17, 2020

can u make a repo for me to test with ?

@ctf0
Copy link
Owner

ctf0 commented Jun 1, 2020

#144

@kmarkovych
Copy link

I have a similar issue.
Using DO S3 and Laravel 8.
In my folder structure, I have /media/files folder which contains 16 000+ files.
Set 'get_folder_info' => false and got some performance improvement.
The main issue is in \Vendor\Mediamanager\GetContent::getFolderContent, it asks 'mimetype', 'visibility', 'timestamp', 'size' by separate HTTP queries, in my case it's 'mimetype' and 'visibility' so a have 32 000 HTTP queries.

@ctf0
Copy link
Owner

ctf0 commented Apr 5, 2021

@kvmarkovich this is not true for 2 reasons

  1. we use listWith which get all the relevant info in the same request "at least thats why we use it in the first place"

    return $this->storageDisk->listWith(
    ['mimetype', 'visibility', 'timestamp', 'size'],
    $folder,
    $rec
    );

  2. all the files at the same level will be fetched in the same request, so if the 16,000 files are at the same level then we will make only one request to list all of them except that it will take some time for the request to finish.

however there is a downside with the current solution due to pagination which we do over the whole collection instead of the original listing which is not originally available in the filesystem plugin , thats why its highly recommended to use the cache adaptor.

u can test the prev points using a local disk on ur machine with ex.100 file at the same level and check the network tab.

@kmarkovych
Copy link

I wish you know better code you work with...
You can check how it works one more time. If you won't have any success I'll explain to you how it works.

@ctf0
Copy link
Owner

ctf0 commented Apr 8, 2021

am probably missing ur point, so yeah explain to me how it works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants