Skip to content

Commit

Permalink
Add links to our new paper to the documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
PhilippWendler committed Sep 6, 2017
1 parent b340042 commit 584c64b
Show file tree
Hide file tree
Showing 5 changed files with 10 additions and 10 deletions.
12 changes: 5 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,13 @@
[![PyPI version](https://img.shields.io/pypi/v/BenchExec.svg)](https://pypi.python.org/pypi/BenchExec)

**News**:
- An extended version of our paper on BenchExec and its background gets published in [STTT](http://sttt.cs.uni-dortmund.de/index.html),
you can now read the preprint of [Reliable Benchmarking: Requirements and Solutions](https://www.sosy-lab.org/~dbeyer/Publications/2017-STTT.Reliable_Benchmarking_Requirements_and_Solutions.pdf).
In addition to the [previous version from SPIN'15](https://www.sosy-lab.org/~dbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf),
it also describes the container mode and how to present result data.
- BenchExec 1.9 adds a [container mode](https://github.com/sosy-lab/benchexec/blob/master/doc/container.md)
that isolates each run from the host system and from other runs
(disabled by now, will become default in BenchExec 2.0).
- We have published a paper titled
[Benchmarking and Resource Measurement](https://www.sosy-lab.org/~dbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf)
on BenchExec and its background
at [SPIN 2015](http://www.spin2015.org/).
It also contains a list of rules that you should always follow when doing benchmarking
(and which BenchExec handles for you).

BenchExec provides three major features:

Expand Down Expand Up @@ -69,7 +67,7 @@ at the [Software Systems Lab](https://www.sosy-lab.org) at the [Ludwig-Maximilia
- [BenchExec GitHub Repository](https://github.com/sosy-lab/benchexec),
use this for [reporting issues and asking questions](https://github.com/sosy-lab/benchexec/issues)
- [BenchExec at PyPI](https://pypi.python.org/pypi/BenchExec)
- Paper [Benchmarking and Resource Measurement](https://www.sosy-lab.org/~dbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf) about BenchExec ([supplementary webpage](https://www.sosy-lab.org/~dbeyer/benchmarking/))
- *New*: Paper [Reliable Benchmarking: Requirements and Solutions](https://www.sosy-lab.org/~dbeyer/Publications/2017-STTT.Reliable_Benchmarking_Requirements_and_Solutions.pdf) about BenchExec ([supplementary webpage](https://www.sosy-lab.org/~dbeyer/benchmarking/))

### Authors
Maintainer: [Philipp Wendler](https://www.philippwendler.de)
Expand Down
2 changes: 1 addition & 1 deletion doc/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The documentation for BenchExec is available in the following files:
- [Resource handling](resources.md) for measuring and limiting resources like time and memory

More on the background of BenchExec can also be found in our paper
[Benchmarking and Resource Measurement](https://www.sosy-lab.org/~dbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf).
[Reliable Benchmarking: Requirements and Solutions](https://www.sosy-lab.org/~dbeyer/Publications/2017-STTT.Reliable_Benchmarking_Requirements_and_Solutions.pdf).

Additional resources such as helper scripts can be found in the directory [contrib](../contrib),
for example [files for generating plots](../contrib/plots/README.md).
Expand Down
2 changes: 1 addition & 1 deletion doc/benchexec.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ One can use this to create a reliable archive of experimental results.
in the best possible way.
More information on what should be considered when allocating hardware resources such as CPU cores
can be found in our paper
[Benchmarking and Resource Measurement](https://www.sosy-lab.org/~dbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf).
[Reliable Benchmarking: Requirements and Solutions](https://www.sosy-lab.org/~dbeyer/Publications/2017-STTT.Reliable_Benchmarking_Requirements_and_Solutions.pdf).
Some additional technical information is also present in the documentation on [resource handling](resources.md).


Expand Down
2 changes: 1 addition & 1 deletion doc/benchmarking.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Please check the following guidelines for advice
on how to get reliable benchmark results when using BenchExec.
For more general guidelines about benchmarking,
e.g., what to do when implementing a benchmarking tool,
please read our paper [Benchmarking and Resource Measurement](https://www.sosy-lab.org/%7Edbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf).
please read our paper [Reliable Benchmarking: Requirements and Solutions](https://www.sosy-lab.org/~dbeyer/Publications/2017-STTT.Reliable_Benchmarking_Requirements_and_Solutions.pdf).

### Use the latest version of BenchExec
New versions of BenchExec might contain features or bug fixes
Expand Down
2 changes: 2 additions & 0 deletions doc/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Contrary to using a VM, the application is still executed with native performanc
because it is run directly on the host kernel, without any additional layers.
By using an overlay filesystem (if available), the benchmarked process can still read from the host filesystem,
but not modify any files except where specifically allowed.
You can read more about it in our paper
[Reliable Benchmarking: Requirements and Solutions](https://www.sosy-lab.org/~dbeyer/Publications/2017-STTT.Reliable_Benchmarking_Requirements_and_Solutions.pdf).

The features of container mode are:

Expand Down

0 comments on commit 584c64b

Please sign in to comment.