Feature: Async

[lostmsu]

As in the title - it would be nice to modernize the interface to be asynchronous, and perhaps also switch to Memory<byte> or Span<byte> for buffers.

A few other performance improvements like iterative file search as mentioned by @LTRData would be welcome too.

[LTRData]

I am working on that in a fork here:
https://github.com/LTRData/dokan-dotnet/

The goal is to eventually merge it here as a pull request/branch in one way or another too. But there are lots of breaking changes and changes to implementation interfaces, so it requires some work to migrate existing implementations to it. The performance improvements are so big, I really think it is worth the breaking changes though. We could probably provide some kind of compatibility layer to make migration easier too.

[LTRData]

But there is really not anything that needs to be changed to support async as I see it. There should be no issues with calling async code from file system implementation routines. I have done that in lots of implementations without any issues.

[Liryna]

@LTRData I totally approve what you said #354 (comment) Feel free to make the change when you feel comfortable. I agree that keeping the compatibility would be nice.

[d2dyno1]

The performance improvements are so big, I really think it is worth the breaking changes though.

@LTRData Do you have any performance benchmarks for the changes? 🙂

[LTRData]

The benchmarks were a bit complicated to make, because they needed two applications. One that made file requests and one Dokan implementation that was monitored by the benchmark. I am thinking of recreating it in a way that I could share more easily.

[LTRData]

Some basic performance profiling. Mounted Mirror sample and copied a directory with lots of subdirectories and about 6,000 files out of that mirrored drive to another place.

First, current version. There are lots of allocations of temporary objects that GC collects and frees over time, but it consumes lots of CPU and memory just to do these tasks.

Then, current version with -unsafe. This avoids some byte arrays and we can clearly see that the pressure on GC and memory consumed is lower.

Then, my new design without creation of string objects on each request, without allocating byte arrays, with DokanFileInfo as a struct instead of object etc:

My idea now is to create subdirectory in the library and put my modified files there and let the existing files provide the existing API for compatibility. There are some changes needed to implementations for consuming this redesigned API, so we cannot simply replace the old API.

[lostmsu]

@LTRData was there any change in practical performance though? As in time to copy.

[LTRData]

@LTRData was there any change in practical performance though? As in time to copy.

Not that much, really. It gets a bit more responsive when opening and working with files on Dokan file systems.

But the most notable performance boost come from the lower memory and CPU pressure in scenarios where that is critical. For example, I am working on an application that is used on VDI servers with lots of simultaneous user sessions where each user mounts several different Dokan file systems for accessing cloud file storage etc. There can be hundreds of Dokan file systems on such servers and optimization in ways like this have great impact.

But, please suggest more ideas for measuring performance differences! File copying is not really the most interesting thing to try for all use cases!