Skip to content

Commit

Permalink
Merge pull request #43 from ayasyrev:fix_doc
Browse files Browse the repository at this point in the history
docs fix
  • Loading branch information
ayasyrev authored Feb 10, 2024
2 parents f5cd9ad + bb21a2c commit daceb1b
Show file tree
Hide file tree
Showing 4 changed files with 108 additions and 272 deletions.
130 changes: 35 additions & 95 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# Benchmark utils

Utils for benchmark - wrapper over python timeit.
Expand All @@ -6,7 +7,7 @@ Utils for benchmark - wrapper over python timeit.
[![PyPI Status](https://badge.fury.io/py/benchmark-utils.svg)](https://badge.fury.io/py/benchmark-utils)
[![Tests](https://github.com/ayasyrev/benchmark_utils/workflows/Tests/badge.svg)](https://github.com/ayasyrev/benchmark_utils/actions?workflow=Tests) [![Codecov](https://codecov.io/gh/ayasyrev/benchmark_utils/branch/main/graph/badge.svg)](https://codecov.io/gh/ayasyrev/benchmark_utils)

Tested on python 3.7 - 3.12
Tested on python 3.8 - 3.12

## Install

Expand Down Expand Up @@ -55,37 +56,23 @@ bench = Benchmark(
```python
bench
```
<details open> <summary>output</summary>
<pre>
Benchmark(func_to_test_1, func_to_test_2)
</pre>
<details open> <summary>output</summary>
<pre>Benchmark(func_to_test_1, func_to_test_2)</pre>
</details>



Now we can benchmark that functions.


```python
# we can run bench.run() or just:
bench()
```
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.10</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.0</span>%
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_2: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-8.9</span>%
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_2: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-9.1</span>%
</pre>

</details>


Expand All @@ -95,33 +82,25 @@ We can run it again, all functions, some of it, exclude some and change number o
```python
bench.run(num_repeats=10)
```
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.10</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.0</span>%
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_2: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-9.3</span>%
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_2: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-9.1</span>%
</pre>

</details>


After run, we can prunt results - sorted or not, reversed, compare results with best or not.
After run, we can print results - sorted or not, reversed, compare results with best or not.


```python
bench.print_results(reverse=True)
```
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run


<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
</pre>


Expand All @@ -133,10 +112,9 @@ bench.print_results(reverse=True)



<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.10</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">10.2</span>%
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.10</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">10.0</span>%
</pre>

</details>


We can add functions to benchmark as list of functions (or partial) or as dictionary: `{"name": function}`.
Expand All @@ -157,43 +135,21 @@ bench = Benchmark(
bench
```
<details open> <summary>output</summary>
<pre>
Benchmark(func_to_test_1, func_to_test_1(0.12), func_to_test_1(sleep_time=0.11))
</pre>
<pre>Benchmark(func_to_test_1, func_to_test_1(0.12), func_to_test_1(sleep_time=0.11))</pre>
</details>




```python
bench.run()
```
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Sec <span style="color: #800080; text-decoration-color: #800080">/</span> run
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.10</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.0</span>%
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="color: #800080; text-decoration-color: #800080; font-weight: bold">func_to_test_1</span><span style="font-weight: bold">(</span><span style="color: #808000; text-decoration-color: #808000">sleep_time</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span><span style="font-weight: bold">)</span>: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-8.8</span>%
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="color: #800080; text-decoration-color: #800080; font-weight: bold">func_to_test_1</span><span style="font-weight: bold">(</span><span style="color: #808000; text-decoration-color: #808000">sleep_time</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span><span style="font-weight: bold">)</span>: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.11</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-9.1</span>%
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="color: #800080; text-decoration-color: #800080; font-weight: bold">func_to_test_1</span><span style="font-weight: bold">(</span><span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.12</span><span style="font-weight: bold">)</span>: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.12</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-16.4</span>%
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="color: #800080; text-decoration-color: #800080; font-weight: bold">func_to_test_1</span><span style="font-weight: bold">(</span><span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.12</span><span style="font-weight: bold">)</span>: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0.12</span> <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">-16.7</span>%
</pre>

</details>



```python
bench = Benchmark(
{
Expand All @@ -208,18 +164,16 @@ bench = Benchmark(
bench
```
<details open> <summary>output</summary>
<pre>
Benchmark(func_1, func_2)
</pre>
<pre>Benchmark(func_1, func_2)</pre>
</details>



When we run benchmark script in terminal, we got pretty progress thanks to rich. Lets run example_1.py from example folder:

![exaple_1](images/run_example_1.gif)
![example_1](images/run_example_1.gif)

# BenchmarkIter
## BenchmarkIter

With BenchmarkIter we can benchmark functions over iterables, for example read list of files or run functions with different arguments.

Expand Down Expand Up @@ -252,26 +206,21 @@ bench = BenchmarkIter(
```python
bench()
```
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Items/sec
</pre>

</details>







<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Items/sec
</pre>

<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">97.93</span>
</pre>

<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_2: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">65.25</span>
</pre>

We can run it again, all functions, some of it, exclude some and change number of repeats.
And we can limit number of items with `num_samples` argument:
`bench.run(num_samples=5)`

# Multiprocessing
## Multiprocessing

By default we tun functions in one thread.
But we can use multiprocessing with `multiprocessing=True` argument:
Expand All @@ -284,29 +233,20 @@ And we can use `num_workers` argument to limit used cpu cores:
```python
bench.run(multiprocessing=True, num_workers=2)
```
<details open> <summary>output</summary>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Items/sec
</pre>

</details>



<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Func name | Items/sec
</pre>




<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_1: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">173.20</span>
</pre>




```python
0.1022316165981465 / 0.09319195459829643
```
<details open> <summary>output</summary>
<pre>
1.0970004550158379
</pre>
</details>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">func_to_test_2: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">120.80</span>
</pre>


Loading

0 comments on commit daceb1b

Please sign in to comment.