Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid return value in RemotePathIterator when batch_size=1 or a directory only contains a single file #4

Open
asgersvenning opened this issue Dec 15, 2023 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@asgersvenning
Copy link
Owner

When trying to iterate over a directory with a single file (which matches pattern, i.e. RemotePathIterator.remote_paths has length 1) or batch_size = 1, the IOHandler.download branch for downloading a single file is triggered, which returns "/" as the local path, instead of the path to the file.

@asgersvenning asgersvenning added the bug Something isn't working label Dec 15, 2023
@asgersvenning asgersvenning self-assigned this Dec 15, 2023
@asgersvenning
Copy link
Owner Author

A quick-fix is just to disable the single-download branch in IOHandler.download, but the correct solution is to review why this happens. I think the error ultimately originates from line 584:

     local_result = self.pget(remote_path, local_destination, blocking, **kwargs)

If the return value of IOHandler.pget in this case for some reason is "/" then this would explain the issue. This would of course suggest that the true error is in IOHandler.pget.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant