Skip to content

Commit

Permalink
Merge branch 'deephaven:main' into release-0.29.0-redo
Browse files Browse the repository at this point in the history
  • Loading branch information
stanbrub authored Dec 7, 2023
2 parents 699a72c + 5e61556 commit 7e9db51
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 1 deletion.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

[Summary of Latest Successful Nightly Benchmarks](docs/NightlySummary.md)
![Operation Rate Change Tracking By Release](https://storage.googleapis.com/deephaven-benchmark/nightly/benchmark-summary.svg?)
([See Other Deephaven Summaries Below](#other-deephaven-summaries))

The Benchmark framework provides support for gathering performance measurements and statistics for operations on tabular data. It uses the JUnit
framework as a runner and works from popular IDEs or from the command line. It is geared towards scale testing interfaces capable of ingesting
Expand Down Expand Up @@ -71,4 +72,7 @@ or fetched from the remote Deephaven instance where the test is running as is do
[StandardTestRunner](src/it/java/io/deephaven/benchmark/tests/standard/StandardTestRunner.java)
used for nightly Deephaven benchmarks. Either way the submission of the result to the Benchmark framework is under the test-writer's control.

## Other Deephaven Summaries

[Summary of Comparison Benchmarks](docs/ComparisonSummary.md)
![Operation Rate Product Comparison](https://storage.googleapis.com/deephaven-benchmark/compare/benchmark-summary.svg?)
19 changes: 19 additions & 0 deletions docs/ComparisonSummary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Comparison Benchmark Summary

![Operation Rate Comparison](https://storage.googleapis.com/deephaven-benchmark/compare/benchmark-summary.svg?)

## Comparison Table Organization

- Each row shows the benchmark in rows per sec for equivalent operations
- The versions displayed show the latest stable release of each product
- The Benchmark Date shows the day when the benchmarks where collected

## Basic Benchmark Methodology

- All products are run on the same hardware in the same environment
- For each operation (row), all products use the same single parquet file
- All products are benchmarked "out of the box" with no tuning
- Parquet read is included in the benchmark
- Some products group or index on parquet read
- Deephaven runs as a service while the other products run in-process

2 changes: 1 addition & 1 deletion docs/NightlySummary.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

- Common operations are shown first followed by less common operations
- Benchmarks are taken for each operation twice; Static and Ticking
- [Static](https://deephaven.io/core/docs/how-to-guides/data-import-export/parquet-flat): Parquet data is read into memory and made
- [Static](https://deephaven.io/core/docs/how-to-guides/data-import-export/parquet-single/): Parquet data is read into memory and made
available to the operation as a whole
- [Ticking](https://deephaven.io/core/docs/conceptual/deephaven-overview/): Data is released incrementally each cycle
- The Benchmark Date shows the day when the benchmarks where collected, which is the latest successful run
Expand Down

0 comments on commit 7e9db51

Please sign in to comment.