Skip to content

Commit

Permalink
Speed up parallelized SFTP reads when read size goes beyond end of file
Browse files Browse the repository at this point in the history
This commit speeds up SFTP read calls when the requested read size is
much larger than the size of the file. Instead of continuing to read
past the end, the parallel read algorithm now stops issuing new read
requests as soon as an earlier read completes with an EOF error. While
calling read() with no length is still the best option if you want to
read a whole file, this optimization may be helpful in some cases.

Thanks go to Maximilian Knespel for reporting this issue and doing
performance measurements on the code before and after the change!
  • Loading branch information
ronf committed Sep 25, 2024
1 parent a3a3b26 commit 8fdce8a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion asyncssh/sftp.py
Original file line number Diff line number Diff line change
Expand Up @@ -670,7 +670,7 @@ async def iter(self) -> AsyncIterator[Tuple[int, _T]]:
self._pending.add(asyncio.ensure_future(
self._start_task(offset+count, size-count)))
except SFTPEOFError:
pass
self._bytes_left = 0
except (OSError, SFTPError) as exc:
exceptions.append(exc)

Expand Down

0 comments on commit 8fdce8a

Please sign in to comment.