Hello! Reaching out to inquire about potentially collaborating

[emilniklas]

Hey Paul! 🙂

Earlier today I was speaking with a few folks from Dfinity re: an idea of mine, and they directed me to this project, which seems very similar to what I'm looking to build. (In fact, I just created the https://github.com/ic-fs org not knowing it collided with your naming of this project...)

I'd love to speak to you and see if our respective visions align, to potentially collaborate on it! If you'd like, I can lay out my thoughts here. Or if you prefer we can take the discussion someplace else.

Consider this my virtual hand reaching out! ✋

[paulyoung]

Hey, thanks for getting in touch!

Feel free to respond here or we can discuss things further wherever you feel most comfortable.

My main motivation for doing this was to be able to leverage existing crates that rely on certain traits to be implemented.

In practice I’ve only done that with fatfs. It works well (see the example) but I would like to provide a familiar API and remove some boilerplate (see #23)

I also made a start on ext4 (#18) but didn’t get very far. I think I got blocked and decided to come back to it. That crate may also be unmaintained, I’m not sure. I was mainly trying to demonstrate that this is flexible enough to support multiple implementations.

I think the other issues speak for themselves. I had planned to let things stabilize and then try and contribute the low level stuff back to the Rust CDK (#25)

Beyond all that, I have some use cases in mind that I’m excited about.

[emilniklas]

Oh, cool! Yeah, looking at the code this was kind of what I gathered. I think our ideas are very different, but could potentially align a little bit.

So the main thought that I've been having is that it's not strictly necessary for the developer (controller) of a particular canister to also be the controller of the canister where the user's data is stored. Obviously, as we're all used to the status quo, we think about canisters kind of like containers to be deployed in a service mesh, right? But, I'm starting to think one of the major paradigm shifts that the IC allows is having non-developers (i.e. the users themselves) own and control canisters, but doing so through dapps, to which they delegate permission to do so.

So, just like the regular certified assets canister the dfx sdk uses to upload FE assets, you can imagine a canister specification (e.g. precompiled .wasm and/or a "protocol" candid schema), that gets deployed on behalf of a user.

I'm seeing a lot of this pattern: HashMap<Principal, UserData>. But for many applications there is no particular use-case for cross-user querying of data, and therefore no reason why those UserData should be stored in the same data structure. Instead, there's a "multi-canister architecture" emerging, right? I think I heard that the OpenChat team uses a canister per user. But they still have to control/manage/upgrade all of those canisters themselves.

So the connection to file systems is that what I'm proposing is pretty much exactly how personal computing worked before the internet, right? The developers of a word processor or image editor weren't responsible for storing your text documents or images. You used files on your hard-drive, and their file type, as an implicit interoperability layer between applications.

I'm rambling a little bit now, I can tell 😄

Bottom line is, I've built a PoC Dropbox/Google Drive-style file manager app, but that deploys/reads/writes to the user's own canisters (currently just using the normal assets canisters from the sdk that I mentioned above).

To take it further, I want to build a canister .wasm that supports all manner of file-system-like operations. Here's a few off the top of my head:

• Stuff that we'd get mostly for free using your approach, I'm guessing:
  • Create file
  • Stat file
  • Read chunks of bytes from file
  • Write chunks of bytes to file
  • List/search files
  • Directories
  • Move/Copy
  • Delete files, incl. recursively
• Manage permissions to the above on a per-principal basis (maybe reimplementing unix-style permissions)
• Choosing to expose/hide files from access using the HTTP request API
• File encryption
• Content encoding/compression

Then I plan to basically rebuild my PoC but a little more "for real", making use of all of those capabilities listed above.

So, do you think that there is an overlap here where we can collaborate?