Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Examples #268

Merged
merged 13 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML linguist-generated=true
1 change: 1 addition & 0 deletions .github/workflows/deploy-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
- name: Build book
working-directory: ${{runner.workspace}}/routee-compass/docs/
run: |
python examples/_convert_examples_to_notebooks.py
jupyter-book build .

# Push the book's HTML to github-pages
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/python-build-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,15 @@ jobs:
pip install ".[all]"

- name: Run mypy
run: mypy --strict ./python/nrel
run: mypy .

- name: Run linter
run: |
ruff check python
ruff check

- name: Run formatter
run: |
ruff format python
ruff format

- name: Run tests
run: |
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Cargo.lock

docs/examples/*.ipynb

# onnx runtimes get downloaded here:
onnxruntime/

Expand Down Expand Up @@ -119,3 +121,7 @@ ENV/

# mypy
.mypy_cache/

# pixi environments
.pixi
*.egg-info
12 changes: 0 additions & 12 deletions docs/README.md

This file was deleted.

29 changes: 18 additions & 11 deletions docs/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,29 @@
# Learn more at https://jupyterbook.org/customize/toc.html

format: jb-book
root: intro
root: intro
parts:
- caption: Getting Started
chapters:
- file: installation
- file: running
- file: installation
- file: running

- caption: Examples
chapters:
- file: examples/open_street_maps_example
- file: examples/different_powertrains_example
- file: examples/time_energy_tradeoff_example

- caption: User Reference
chapters:
- file: notebooks/open_street_maps_example
- file: query
- file: config
- file: python_api_docs
- file: rust_api_docs
- file: query
- file: config
- file: python_api_docs
- file: rust_api_docs

- caption: Developers
- caption: Developers
chapters:
- file: developers/build_the_docs
- file: developers/contributing
- file: developers/build_the_docs
- file: developers/CONTRIBUTING
- file: developers/rust_code_style

2 changes: 1 addition & 1 deletion docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Each `CompassApp` instance is defined by a configuration toml file.
The configuration file specifies things like "Which traversal model should I use, and what are its parameters?" and "Which data sources should I use?".

If you follow the [open street maps example](notebooks/open_street_maps_example.ipynb), the code will produce a few configuration files in the `golden_co/` folder. Let's take a look at the `osm_default_energy.toml` file.
If you follow the [open street maps example](examples/open_street_maps_example), the code will produce a few configuration files in the `golden_co/` folder. Let's take a look at the `osm_default_energy.toml` file.
We added some annotations to describe the different sections:

```toml
Expand Down
7 changes: 4 additions & 3 deletions docs/developers/build_the_docs.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
# Build The Docs

The main documentation is built using jupyter-book which you can install with pip:
The main documentation is built using jupyter-book which you can install with dev dependencies:

```bash
pip install jupyter-book
pip install ".[dev]"
```

Then, to build the docs, run the following command from the root of the repository:
Then, to build the docs, run the following commands from the root of the repository:

```bash
python docs/examples/_convert_examples_to_notebooks.py
jupyter-book build docs/
```

Expand Down
22 changes: 10 additions & 12 deletions docs/developers/rust_code_style.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
## Rust Code Style
# Rust Code Style

### Prefer `Box<[T]>` (boxed slices) over `Vec<T>` for large collections
## Prefer `Box<[T]>` (boxed slices) over `Vec<T>` for large collections

while a `Vec<T>` is more common, it lacks the ability to match capacity exactly with length unless it is set beforehand. in many cases this is not possible. when dealing with large persistent datasets such as `Graph` and `TraversalModel` data, we want to store that data in a `Box<[T]>`. when constructing the dataset, the `Vec` can be used, and then converted to a `Box<[T]>` afterward via `into_boxed_slice`:

```ignore
```rust
let path: Path = todo!();
let data: Vec<T> = read_utils::from_csv(&file_path, True, None)?;
let output: Box<[T]> = data.into_boxed_slice();
```

`Vec` is preferred for smaller or non-persistent datasets, or when the additional features of `Vec` are used, such as iteration or dynamic allocation.

### The Builder, Service, Model Convention in RouteE Compass
## The Builder, Service, Model Convention in RouteE Compass

in RouteE Compass, large Traversal and Frontier objects are loaded once and shared across threads.
in RouteE Compass, large Traversal and Frontier objects are loaded once and shared across threads.
they are built in two phases: once at application initialization, and once again for each query, so that query-specific parameters can be applied.
these phases are represented by the following types:

phase | description | lifetime | Frontier | Traversal
--- | --- | --- | --- | ---
**builder** | an empty struct with a `build` method that creates a **service** | app initialization only | `FrontierModelBuilder` | `TraversalModelBuilder`
**service** | struct with a `build` method that creates a **model** | entire program lifetime (same as CompassApp instance) | `FrontierModelService` | `TraversalModelService`
**model** | object used by the search algorithm | duration of a single query | `FrontierModel` | `TraversalModel`
| phase | description | lifetime | Frontier | Traversal |
| ----------- | ---------------------------------------------------------------- | ----------------------------------------------------- | ---------------------- | ----------------------- |
| **builder** | an empty struct with a `build` method that creates a **service** | app initialization only | `FrontierModelBuilder` | `TraversalModelBuilder` |
| **service** | struct with a `build` method that creates a **model** | entire program lifetime (same as CompassApp instance) | `FrontierModelService` | `TraversalModelService` |
| **model** | object used by the search algorithm | duration of a single query | `FrontierModel` | `TraversalModel` |

when we apply the `build` methods, we get these results (using the travel time `TraversalModel` as an example):

Expand All @@ -35,5 +35,3 @@ let model: Arc<dyn TraversalModel> = service.build(&query);
```

the **builder** object instances are wrapped in a `Box` referenced by the `CompassAppBuilder` and used when creating `CompassApp` instances. once we build a **service** from the **builder**, the app requires that they are wrapped in an `Arc`, which is a thread-safe pointer. this way, the **service** can be shared across threads so we can build a **model** for a specific user query from within a query thread.


55 changes: 55 additions & 0 deletions docs/examples/_convert_examples_to_notebooks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
from pathlib import Path
import nbformat


def script_to_notebook(script_path: Path, notebook_path: Path) -> None:
# Read the script
with open(script_path, "r") as script_file:
lines = script_file.readlines()

notebook = nbformat.v4.new_notebook()
current_code_block: list[str] = []
current_markdown_block: list[str] = []

def add_code_cell(block: list[str]) -> None:
if block:
notebook.cells.append(nbformat.v4.new_code_cell("".join(block)))

def add_markdown_cell(block: list[str]) -> None:
if block:
notebook.cells.append(nbformat.v4.new_markdown_cell("".join(block).strip()))

# markdown cells will be enclosed in triple quotes
# the remaining cells will be code cells
in_markdown = False
for line in lines:
# strip any ipython code blocks
if line.strip().startswith("# %%"):
continue
if line.strip().startswith('"""'):
in_markdown = not in_markdown
if in_markdown:
add_code_cell(current_code_block)
current_code_block = []
else:
add_markdown_cell(current_markdown_block)
current_markdown_block = []
elif in_markdown:
current_markdown_block.append(line)
else:
current_code_block.append(line)

add_code_cell(current_code_block)
add_markdown_cell(current_markdown_block)

with open(notebook_path, "w") as notebook_file:
nbformat.write(notebook, notebook_file)


if __name__ == "__main__":
here = Path(__file__).parent
examples = here.glob("*example.py")
for example in examples:
notebook = example.with_suffix(".ipynb")
script_to_notebook(example, notebook)
print(f"Converted {example} to {notebook}")
Loading
Loading