-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
34 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# Tests | ||
|
||
## Seeding | ||
|
||
Some testcases involve random numbers, including: | ||
- All neural network models including DARTS, Bayesian DARTS | ||
- All Tree models including BMS. | ||
|
||
In order to avoid the testcases from running correctly sometimes, and incorrectly other times, we seed all the relevant random number generators for those testcases. To accomplish this, add the following pytest fixture to the test file and include it as required in the test functions: | ||
|
||
```python | ||
import random | ||
import pytest | ||
import torch | ||
|
||
@pytest.fixture | ||
def seed(): | ||
""" | ||
Ensures that the results are the same each time the tests are run. | ||
""" | ||
random.seed(180) # required for models which use the python `random` module, e.g. BMS | ||
torch.manual_seed(180) # required for PyTorch models, e.g. DARTS | ||
return | ||
|
||
|
||
def test_foo(seed): | ||
""" Test something. """ | ||
|
||
# No need to use `seed` in the function body – adding it as an argument is sufficient | ||
|
||
... # Run tests | ||
``` | ||
|
||
The seed value should be consistent but not tuned to produce correct results. The integer `180` is used in many tests, inspired "180 George St., Providence, RI, USA", the office address for the Center for Computation and Visualization at Brown University, whose staff supported the development of the AutoRA package. Sensible alternatives are `42`, `31415926` and `2654435769`. See [https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number](https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number) for more inspiration. |