Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: migrate repository to uv & simplify Pyodide build #899

Merged
merged 13 commits into from
Nov 17, 2024

Conversation

peter-gy
Copy link
Collaborator

@peter-gy peter-gy commented Nov 10, 2024

How to test this PR?

git checkout this branch, then execute make. Everything should work without any further action (linting, tests, typechecks, doc-gen).

Running all the make targets still takes a while, because mypy and pytype are really slow and generating the Jupyter Book output also requires patience, given that we execute all notebooks on the fly to generate the HTML bundle.

Note

Please make sure you have uv installed and available on your PATH.
You can refer to the installation instructions here: https://docs.astral.sh/uv/getting-started/installation/

CHANGELOG

  • Drop Poetry in favor of uv
  • Move draco module source under src/
  • Use Python 3.12.7
  • Update .github/workflows/test.yml CI flow
  • Make devcontainer Dockerfile slimmer (~2x image build performance improvement)
  • Define [project.optional-dependencies] in pyproject.toml, allowing pip install draco to pull only core dependencies
    • pip install 'draco[server]': will include FastAPI deps
    • pip install 'draco[renderer]': will include deps to render chart specs
    • pip install 'draco[learn]': will include scikit-learn enabling draco.learn.train_model
  • Stop maintaining custom Pyodide distribution, since clingo and vega-datasets are now members of the official Pyodide distribution (available through https://cdn.jsdelivr.net/pyodide//full/)
  • Build JupyterLite into docs via jupyterlite-sphinx.

Pyodide Build Simplifications

The complexity of the current Pyodide build pipeline stems from the fact that clingo is an impure Python package (depends on cffi) without an official recipe for it in the Pyodide repo. In parallel to this PR I am opening PRs against https://github.com/pyodide/pyodide to include packages we need into their official build. This way, we can dramatically reduce the code we maintain in this repo responsible for WASM builds.

Foreign PRs

Closes

Closes #895
Closes #884

@peter-gy peter-gy changed the title refactor: migrate repository to uv refactor: migrate repository to uv & simplify Pyodide build Nov 10, 2024
@peter-gy peter-gy marked this pull request as draft November 11, 2024 13:07
@@ -5,7 +5,7 @@
"appUrl": "./lab",
"litePluginSettings": {
"@jupyterlite/pyodide-kernel-extension:kernel": {
"pyodideUrl": "/static/pyodide/pyodide.js"
"pyodideUrl": "https://cdn.jsdelivr.net/pyodide/dev/full/pyodide.js"
Copy link
Collaborator Author

@peter-gy peter-gy Nov 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now we must use the dev version (rolling release pointing to pyodide@main) because it is the only distro over a public CDN which already includes pyodide/pyodide#5183 and pyodide/pyodide#5184.

As soon as we get a Pyodide release with clingo and vega-datasets included, we'll pin that version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good

@peter-gy peter-gy marked this pull request as ready for review November 17, 2024 14:58
Copy link
Member

@domoritz domoritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@peter-gy peter-gy merged commit 493757c into main Nov 17, 2024
11 checks passed
@peter-gy peter-gy deleted the refactor/move-to-uv branch November 17, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider switching to UV Using AltairRenderer in pyodide enviornment
2 participants