From 316963d4154c51081b9545d3eb584da5afb6ea06 Mon Sep 17 00:00:00 2001 From: Stefan Ulbrich <6009224+StefanUlbrich@users.noreply.github.com> Date: Mon, 21 Aug 2023 12:09:41 +0200 Subject: [PATCH] Update Readme --- Readme.md | 87 ++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 33 deletions(-) diff --git a/Readme.md b/Readme.md index 41239ef..b038a3b 100644 --- a/Readme.md +++ b/Readme.md @@ -46,50 +46,71 @@ efficiency and simplicity (both in terms of implementation and comprehensibility). Plus, it's my favorite and I'm doing active research with it. -* Numerical python extensions written in Rust -* Seamless integration via [PyO3](https://pyo3.rs/v0.16.1/) and [rust-numpy](https://docs.rs/numpy/0.7.0/numpy/) -* High-performance via [ndarray](https://github.com/rust-ndarray/ndarray)/[rayon](https://docs.rs/ndarray/0.13.1/ndarray/parallel/index.html), [tch-rs](https://github.com/LaurentMazare/tch-rs) ([PyTorch](https://pytorch.org/) bindings) -* Dependency management / publishing with [Poetry](https://python-poetry.org/docs/) and [Maturin](https://github.com/PyO3/maturin) -* [Monorepo](https://en.wikipedia.org/wiki/Monorepo) with [Cargo workspaces][(]()) -* Technical documentation / GitHub page with [Sphinx](https://www.sphinx-doc.org/en/master/) and [MyST](https://myst-parser.readthedocs.io/en/latest/sphinx/intro.html) -* Eventually, distributed computation (e.g., [actor model](https://en.wikipedia.org/wiki/Actor_model) or [timely - dataflow](https://timelydataflow.github.io/timely-dataflow/)) -* Hopefully, GUI application using [Tauri](https://tauri.studio/), [Angular](https://angular.io/) and [ThreeJS](https://threejs.org/) - -Feel free use as a basis for your own projects (MIT licensed). +The development explores: -## Development - -For the monorepo, there is a top-level python project defined using poetry. This project is -mainly for dependency locking, integration testing and the overall project's documentation. -The python module -build with Maturin is in a nested folder and it's project file is also created with poetry. -This is necessary as the top-level project can only add local packages that are either -created with Poetry or contain a `setup.py` file. +* Numerical python extensions written in Rust: Seamless integration via [PyO3](https://pyo3.rs/v0.16.1/) and [rust-numpy](https://docs.rs/numpy/0.7.0/numpy/). Dependency management / publishing with [Poetry](https://python-poetry.org/docs/) and [Maturin](https://github.com/PyO3/maturin) +* High-performance via [ndarray](https://github.com/rust-ndarray/ndarray)/[rayon](https://docs.rs/ndarray/0.13.1/ndarray/parallel/index.html). -While this setup supports a monorepo setup (and should support integration -testing on the imported local packages), there is another caveat. Building the python extension with -`pip` creates a temp directory which does not copy the local rust dependencies. Newer versions of `pip` -build within the tree, so this limitation can be avoided easily. Use the following commands to -setup the environment. +## The cerebral and potpourri Python packages -> :warning: We want to avoid creating a virtual environment for the nested packages. Work in -> a top-level shell instead. +The`potpourri` and `citrate` sub folders contain the Python bindings for +the two crates in `potpourri-rs` and `citrate-rs`. You can build them in a +similar fashion. ```sh -cd self-organization -pyenv shell 3.10.2 -poetry env use 3.10.2 -poetry run pip install -U pip # only required until pip>=22.0 becomes the default +cd citrate +# skip the next line if you are not using pyenv +pyenv shell 3.10.2 # replace with desired/installed version +# skip if you only have one version of python installed +poetry env use 3.10.2 # replace with desired/installed version +# Install dependencies poetry install -# to debug / develop the extension +# Or optionally, with jupyter +poetry install --with jupyter +# activate the virtual environment poetry shell -cd pysom +# install into the current environment maturin develop +# with optimizations (recommended) +maturin develop --release +# build an installable package +maturin build --release ``` To install the virtual environment as a kernel for jupyter: ```sh -python -m ipykernel install --user --name py310_selforganization --display-name "Python3.10 (self-organization)" +poetry add ipykernel +# Adjust names as suitable +python -m ipykernel install --user --name py320_citrate --display-name "Python3.10 (Citrate)" ``` + +## Development + +* Bulding documentation. We need extra headers to have formulas in the rustdoc + + ```sh + RUSTDOCFLAGS="--html-in-header ./static/header.html" cargo doc -p cerebral --no-deps + RUSTDOCFLAGS="--html-in-header ./static/header.html" cargo doc -F ndarray -p potpourri --no-deps + # Open (e.g. with Firefox) + firefox target/doc/citrate/index.html + firefox target/doc/potpourri/index.html + ``` + +* Benchmarks (currently only in Potpourri) + + ```sh + cd potpourri-rs + cargo bench -F ndarray + ``` + +* Running individual tests + + ```sh + cd potpourri-rs + cargo test -F ndarray -p potpourri test_multi_pass + ``` + +## License + +Published under the MIT license.