Skip to content

Commit

Permalink
feat: add notebook from colab
Browse files Browse the repository at this point in the history
  • Loading branch information
@lvjonok
lvjonok committed Sep 17, 2024
1 parent 7fb8680 commit c47a88d
Show file tree
Hide file tree
Showing 2 changed files with 242 additions and 0 deletions.
5 changes: 5 additions & 0 deletions benchmarks/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Benchmarking

This directory contains a set of benchmarks that we use to evaluate the performance of the `jaxadi` in comparison to `cusadi`.

Due to the difficulty of installation and `cuda` dependencies, we were able to reproduce the tests in the `colab` environment only.
237 changes: 237 additions & 0 deletions benchmarks/jaxadi_vs_cusadi.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"gpuType": "T4"
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "code",
"source": [
"%%capture\n",
"\n",
"!git clone https://github.com/se-hwan/cusadi\n",
"!git clone -b feature/benchmarking https://github.com/based-robotics/jaxadi\n",
"# copy the content back to the current directory\n",
"!cp -r cusadi/** .\n",
"!pip install -e .\n",
"!cd jaxadi && pip install .\n",
"!cp jaxadi/benchmarks/run_benchmark.py ."
],
"metadata": {
"id": "6wCJ13jopz9v"
},
"execution_count": 1,
"outputs": []
},
{
"cell_type": "code",
"source": [
"from run_benchmark import main\n",
"main()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "UQRLUvYeHJxH",
"outputId": "ab357d37-412c-4f86-ea26-c73e70901604"
},
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Generating CUDA code for CasADi function: fn_1e1\n",
"CUDA codegen complete for CasADi function: fn_1e1\n",
"Generating CUDA code for CasADi function: fn_1e2\n",
"CUDA codegen complete for CasADi function: fn_1e2\n",
"fn_1e1\n",
"fn_1e2\n",
"Running CUDA benchmark for 1 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d4592d0>\n",
"Running CUDA benchmark for 2 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459a80>\n",
"Running CUDA benchmark for 4 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459870>\n",
"Running CUDA benchmark for 8 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459ab0>\n",
"Running CUDA benchmark for 16 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459e70>\n",
"Running CUDA benchmark for 32 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459840>\n",
"Running CUDA benchmark for 64 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459d80>\n",
"Running CUDA benchmark for 128 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459960>\n",
"Running CUDA benchmark for 256 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459930>\n",
"Running CUDA benchmark for 512 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d4597b0>\n",
"Running CUDA benchmark for 1024 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459990>\n",
"Running CUDA benchmark for 2048 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459db0>\n",
"Running CUDA benchmark for 4096 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459f30>\n",
"Running CUDA benchmark for 8192 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d45a0b0>\n",
"Running CUDA benchmark for 16384 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459810>\n",
"Running CUDA benchmark for 32768 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d4599f0>\n",
"Running CUDA benchmark for 65536 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d45a0e0>\n",
"Running CUDA benchmark for 131072 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d459f90>\n",
"Running CUDA benchmark for 262144 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d45a440>\n",
"Running CUDA benchmark for 524288 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b3d45a290>\n",
"Running CUDA benchmark for 1048576 environments with function fn_1e1...\n",
"Loaded CasADi function: fn_1e1:(i0[4],i1[2],i2[2],i3)->(o0[2],o1[3x3],o2[3]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e1.so', handle 5bb692f4d500 at 0x7e0b406f7d00>\n",
"Running CUDA benchmark for 1 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a3e0>\n",
"Running CUDA benchmark for 2 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45ae90>\n",
"Running CUDA benchmark for 4 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a800>\n",
"Running CUDA benchmark for 8 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d459ae0>\n",
"Running CUDA benchmark for 16 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a980>\n",
"Running CUDA benchmark for 32 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a5f0>\n",
"Running CUDA benchmark for 64 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45ada0>\n",
"Running CUDA benchmark for 128 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a680>\n",
"Running CUDA benchmark for 256 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45aef0>\n",
"Running CUDA benchmark for 512 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a920>\n",
"Running CUDA benchmark for 1024 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b160>\n",
"Running CUDA benchmark for 2048 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b190>\n",
"Running CUDA benchmark for 4096 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a890>\n",
"Running CUDA benchmark for 8192 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b3d0>\n",
"Running CUDA benchmark for 16384 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b010>\n",
"Running CUDA benchmark for 32768 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b400>\n",
"Running CUDA benchmark for 65536 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45a8c0>\n",
"Running CUDA benchmark for 131072 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b640>\n",
"Running CUDA benchmark for 262144 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b340>\n",
"Running CUDA benchmark for 524288 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b700>\n",
"Running CUDA benchmark for 1048576 environments with function fn_1e2...\n",
"Loaded CasADi function: fn_1e2:(i0[9],i1[3],i2[6],i3[2])->(o0[3],o1[5x5,23nz],o2[5]) SXFunction\n",
"Loaded library: <CDLL '/content/build/libfn_1e2.so', handle 5bb69380ab70 at 0x7e0b3d45b520>\n",
"Running Jaxadi benchmark for 1 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 2 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 4 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 8 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 16 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 32 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 64 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 128 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 256 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 512 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 1024 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 2048 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 4096 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 8192 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 16384 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 32768 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 65536 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 131072 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 262144 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 524288 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 1048576 environments with function fn_1e1...\n",
"Running Jaxadi benchmark for 1 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 2 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 4 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 8 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 16 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 32 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 64 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 128 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 256 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 512 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 1024 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 2048 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 4096 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 8192 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 16384 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 32768 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 65536 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 131072 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 262144 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 524288 environments with function fn_1e2...\n",
"Running Jaxadi benchmark for 1048576 environments with function fn_1e2...\n",
"Benchmark results saved.\n"
]
}
]
}
]
}

0 comments on commit c47a88d

Please sign in to comment.