Skip to content

Commit

Permalink
fix: expand benchmarking readme
Browse files Browse the repository at this point in the history
  • Loading branch information
lvjonok committed Sep 17, 2024
1 parent 8423247 commit 128d72c
Show file tree
Hide file tree
Showing 6 changed files with 107 additions and 9 deletions.
56 changes: 56 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Contributing to JaxADi

We're thrilled that you're interested in contributing to JaxADi! This document outlines the process for contributing to this project.

## How to Contribute

There are many ways to contribute to JaxADi:

1. Reporting bugs
2. Suggesting enhancements
3. Writing documentation
4. Submitting code changes

### Reporting Bugs

1. Check the [issue tracker](https://github.com/based-robotics/jaxadi/issues) to see if the bug has already been reported.
2. If not, create a new issue. Provide a clear title and description, as much relevant information as possible, and a code sample or executable test case demonstrating the bug.

### Suggesting Enhancements

1. Check the [issue tracker](https://github.com/based-robotics/jaxadi/issues) to see if the enhancement has already been suggested.
2. If not, create a new issue. Clearly describe the enhancement, why it would be useful, and any potential drawbacks.

### Writing Documentation

Good documentation is crucial. If you notice any part of our documentation that could be improved or expanded, please let us know or submit a pull request with your suggested changes.

### Submitting Code Changes

1. Fork the repository.
2. Create a new branch for your changes.
3. Make your changes in your branch.
4. Add or update tests as necessary.
5. Ensure the test suite passes.
6. Update the documentation as needed.
7. Push your branch and submit a pull request.

## Pull Request Process

1. Ensure your code follows the project's style guidelines.
2. Update the README.md or relevant documentation with details of changes, if applicable.
3. Add tests for your changes and ensure all tests pass.
4. Your pull request will be reviewed by the maintainers. They may suggest changes or improvements.
5. Once approved, your pull request will be merged.

<!-- TODO:
## Style Guidelines
We use Ruff to enforce our code style and linting rules....
```bash
pip3 install pre-commit
```
```bash
pre-commit install
``` -->
27 changes: 24 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ JAXADI can be particularly useful in scenarios involving:
- Robotics simulations
- Optimal control problems
- Machine learning models with complex dynamics
- Large-scale numerical optimizations

## Installation

Expand Down Expand Up @@ -65,6 +64,12 @@ output = jax_fn(input_x, input_y)

```

<strong>Note:</strong> For now translation does not support functions with very
large number of operations, due to the translation implementation. Secret component of
translation is work-tree expansion, which might lead to large overhead in number of
symbols. We are working on finding the compromise in both speed and extensive
functions support.

## Examples

JAXADI comes with several examples to help you get started:
Expand All @@ -85,11 +90,27 @@ JAXADI comes with several examples to help you get started:
## Performance Benchmarks

![speedup](https://github.com/based-robotics/jaxadi/blob/master/docs/static/images/speedup_ratio.png?raw=true)

The process of benchmarking and evaluating the performance of Jaxadi is described in the [benchmarks](benchmarks/README.md) directory.

<!-- ## Contributing
## Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for more details.

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for more details. -->
## Citation

If you use JaxADi in your research, please cite it as follows:

```bibtex
@misc{jaxadi2024,
title = {JaxADi: Bridging CasADi and JAX for Efficient Numerical Computing},
author = {Alentev, Igor and Kozlov, Lev and Nedelchev, Simeon},
year = {2024},
url = {https://github.com/based-robotics/jaxadi},
note = {Accessed: [Insert Access Date]}
}
```

## Acknowledgements

Expand Down
20 changes: 20 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,23 @@ Adding the complexity of setup of benchmarks of [cusadi](https://github.com/se-h
Due limitations we cover only the functions with less than 1e3 operations. All of them are located in the [data](data) directory.

One may run the benchmarks in the colab environment and get the [cusadi results](cuda_benchmark_results.npz) and [jaxadi results](jax_benchmark_results.npz) for comparison.

#### Overview

This analysis compares the performance of two computational tools, **Jaxadi** and **Cusadi**, using a range of batch sizes for different operations. The tools are evaluated based on average computation times and their relative speedup across various workloads.

![1e1](https://github.com/based-robotics/jaxadi/blob/master/docs/static/images/compare_1e1_bar.png?raw=true)

![1e2](https://github.com/based-robotics/jaxadi/blob/master/docs/static/images/compare_1e2_bar.png?raw=true)

![Speedup](https://github.com/based-robotics/jaxadi/blob/master/docs/static/images/speedup_ratio.png?raw=true)

#### Key Findings

- **Jaxadi's Consistent Advantage:** Across all tested scenarios, Jaxadi consistently shows better performance than Cusadi. This is evident from the lower average computation times for Jaxadi, particularly as the workload increases.

- **Scalability Differences:** As batch sizes grow, the gap between the two tools becomes more pronounced. Cusadi's computation time increases rapidly with larger batch sizes, especially beyond \(2^{12}\). In contrast, Jaxadi maintains a more stable performance with a much slower increase in computation time, indicating better batch scalability.

#### Conclusion

Overall, Jaxadi offers a clear performance advantage over Cusadi, particularly for small functions which can be translated propely.
13 changes: 7 additions & 6 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">JAXADI</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<a href="https://github.com/lvjonok">Lev Kozlov</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://me.m8pie.xyz">Igor Alentev</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://github.com/lvjonok">Lev Kozlov</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://simeon-ned.com/">Simeon Nedelchev</a><sup>2</sup>,
</span>
Expand Down Expand Up @@ -207,11 +207,12 @@ <h2 class="title is-3">Related Links</h2>
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code> @misc{jaxadi_2024,
author = {Kozlov, Lev and Alentev, Igor and Nedelchev, Simeon},
title = {jaxadi},
<pre><code>@misc{jaxadi2024,
title = {JaxADi: Bridging CasADi and JAX for Efficient Numerical Computing},
author = {Alentev, Igor and Kozlov, Lev and Nedelchev, Simeon},
year = {2024},
howpublished = {\url{https://github.com/based-robotics/jaxadi}},
url = {https://github.com/based-robotics/jaxadi},
note = {Accessed: [Insert Access Date]}
}
</code></pre>
</div>
Expand Down
Binary file removed docs/static/images/interpolate_end.jpg
Binary file not shown.
Binary file removed docs/static/images/interpolate_start.jpg
Binary file not shown.

0 comments on commit 128d72c

Please sign in to comment.