Fetch: Support HTTP Range Requests

[james-pre]

@jvilk:

For large files (100MB+), downloading and storing the entire file in-memory is prohibitively expensive, and may not > be needed if only small parts of the file are needed.

It would be great to support HTTP range requests in both synchronous and asynchronous modes.

Some important questions:

• How large should the chunks be? Should that be configurable?
• Should we prefetch chunks? What prefetch policies are useful?
• What configuration knobs should we expose to the user?
• What eviction policy should we use? FIFO? How much data should we keep in-memory?
• Do browsers cache data retrieved via HTTP range requests? This should make re-acquiring previously evicted blocks much cheaper / avoid draining the server over a long session.

I don't anticipate being able to work on this anytime soon, but I'm recording my thoughts here for now.

jvilk/BrowserFS#219

[anthonyhoegberg]

What's needed for this to work? i might try it but im not quite familiar with the library yet so it might take time

[james-pre]

@anthonyhoegberg,

This requires some changes to the Fetch backend to support the underlying partial reading and writing of files and configuration for the option. Other than that, it is just a matter of sending the right HTTP headers. If you want to take a look at implementing the feature, I can help with any questions or comments you may have.