Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Rust's ndarray library more efficiently in the interpreter #8

Open
gussmith23 opened this issue Sep 22, 2020 · 4 comments
Open
Assignees
Labels
enhancement New feature or request interpreter Related to Glenside's interpreter

Comments

@gussmith23
Copy link
Owner

We use ndarray as our tensor representation, but we don't use it very efficiently. Specifically, we copy data everywhere, for nearly every operation. The ndarray library is smart; it can do many of the operations we need to do without copying data, if only we understood how to use it well. There are also opportunities to contribute back to ndarray, if we find something we'd like to be able to do that they haven't yet implemented. From what little I've dug in their source code already, it seems like these opportunities might be abundant, and contributing shouldn't be too hard!

@gussmith23 gussmith23 added enhancement New feature or request good first issue Good for newcomers labels Sep 22, 2020
@gussmith23 gussmith23 added this to the 2021isca milestone Sep 28, 2020
@gussmith23
Copy link
Owner Author

@hypercubestart This one mostly involves taking a look at the ndarray library and using it more efficiently. I don't have a lot of good guidance here; we can talk in more detail later. Basically, this task involves getting to understand the library and how it works, and using it "more correctly" in Glenside. Specifically, avoiding data copies wherever possible!

@gussmith23
Copy link
Owner Author

@hypercubestart Please add your notes on this whenever you can! Would like to document all of the issues we had before we move on to something else.

@hypercubestart
Copy link
Collaborator

crossposting from #59:

on hold for now...because of difficulty of getting lifetimes to work out. Ideally, we could use ArrayViews that reference owned arrays already in the hashmap, but Rust lifetimes don't permit this

  • CowArray (doesn't work because view variant has lifetimes, so if we insert array back into hashmap, we end up borrowing the hashmap mutably + immutably)
  • Wrap with RC and see if that works (same error as cowarray)
  • ArcArray (shared references, but same benchmarks as naive copying)
  • Try out data structure idea (see whether there’s a better datatype for what we’re trying to do, better than hashmap, i.e an insert-only datastructure that guarantees that lifetimes do not change when we insert) (not quite sure if this is expressible in rust?)
  • Adding a enum wrapper over ndarrray that holds owned data or views/shared data ArrayBase<MyCustomRepr…, >
  • unsafe

@gussmith23
Copy link
Owner Author

Thanks, sorry I didn't see that comment!! Many open issues and PRs haha

@gussmith23 gussmith23 added interpreter Related to Glenside's interpreter and removed good first issue Good for newcomers labels Oct 26, 2020
@gussmith23 gussmith23 removed this from the 2021isca milestone Oct 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request interpreter Related to Glenside's interpreter
Projects
None yet
Development

No branches or pull requests

2 participants