Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes for the engine for using it in a pyodide context #2

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

stuartlynn
Copy link
Contributor

Adding some changes to make the engine work in wasm

  • moves data.py to cache.py ( I was having some conflicts with the old data folder)
  • Adding a check in the cache to see if pyodide_js is available and if so changing the cache to use pyodide versions of the models
  • Adding a check to see if the models are bundled as Model or as a tuple and work appropriately (I was having issues pickling the Model without using the demoland library. Bundling as a tuple of (W, model) seems more portable?)
  • Adding a build script for generating the wasm wheel

TODO

  • Need to allow passing through of the BASE_URL for the pooch cache so that when this is hosted we can grab the files from the root URL

stuartlynn and others added 6 commits November 6, 2023 12:13
- moves data.py to cache.py ( I was having some conflicts with the old data folder)
- Adding a check in the cache to see if pyodide_js is avaliable and if so changing the cache to use pyodide versions of the models
- Adding a check to see if the models are bundled as Model or as a tuple and work appropriatly (I was having issues pickling the Model without using the demoland library. Bundling as a tuple of (W, model) seems more portable?)
- Adding a build script for generating the wasm wheel

TODO
- Need to allow passing through of the BASE_URL for the pooch cache so that when this is hosted we can grab the files from the root URL
@martinfleis
Copy link
Collaborator

what is the state of this? some commented-out bits feel WIP?

@stuartlynn
Copy link
Contributor Author

It needs a bit of a tidy but I wanted to get your eyes on it before we wrapped it up. Especially for the changes to model loading (tuple of weights and model vs the Model class)

If it looks ok to you I can tidy up the code and we can merge it in

@martinfleis
Copy link
Collaborator

Go for it. We can revise the model loading with the national model again as discussed before.

@stuartlynn
Copy link
Contributor Author

Sounds good. Will try and do this today or tomorrow!

@yongrenjie
Copy link
Contributor

@stuartlynn, are you ok if I merge this and reopen a new the PR from the wasm branch of this repo?

I'd like to spend a little bit of time unifying all the different versions (fastAPI / Azure / WASM) into the same branch, otherwise it feels like the web app needs three different submodules which is pretty ugly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants