Auto-resolution and experiment versions

Important changes

Migrated to orion on PyPi

We finally have orion namespace: pypi/warehouse#4189!!! 🎉 🎊

Auto-resolution and experiment versions

The modification of experiments (code change, search space modification, etc) will now be automatically resolved and will no longer lead to the (confusing) branch resolver prompt. When there is any modification that leads to a branching, the version of the experiment will be incremented (starting at 1). The unique index of the experiments are now (name, version) instead of (name, user).

No more user index

The user name is no longer part of the experiment's index. This means that someone with a username A in one environment may retrieve the same experiments with a username B in another environment without any problem. Previously this would only be possible using the option --user B to override the default system's username. This was due to the fact that queries on the database was done using the username and it is no longer the case.

Improved ASHA

Interface

The new prior fidelity(low, high, base) now makes it more convenient to define the fidelity dimension. The different budget for each rungs can now be scaled between low and high according to a base logarithm.

Efficiency

The algorithm will now stop registering trials in the low rungs once the higher rungs are filled. This reduces the waste of resources spend on trials in low rungs that cannot be promoted to higher rungs since the latter will be completed shortly.

Detailed list of changes

New features

• Auto-resolution (#244)
• Add versioning to experiments (#240, #242, #243, #245, #246, #247, #248, #259, #268, #272, #279, #281 )
• Add to algo the ability to opt out (#266)
• Add env-vars to consumer (#280)

Breaking changes

• Remove user as an index for Experiment (#264, #273 )
• Add low, high, base to Fidelity (#269)

Bug Fixes

• Fix Trial ID mismatch with higher dimension (#237)
• Make Space Sorted instead of Ordered (#257, #263 )
• Fix bug in conda test (#258)
• Handle status --name when empty (#270)
• Fix ASHA termination condition and branching with fidelity() (#274 )
• Avoid branching in info (#277)

Other improvements

• Add a global configuration object orion.core.config (#239, #251, #265)
• Remove direct access to database backend in favor of storage backend (#241)
• Remove generic queries in favor of specific queries in storage backend (#250)
• Add best trial id to info command (#260)
• Make ASHA opt out when last rungs are filled (#267)
• Migrate to orion on PyPi (#271 )

Documentation improvements

• Fix invalid section names (#252)
• Make tutorial a bit cleaner (#262)
• Improve documentation of pool-size and fix experiment.API (#278)
• Document script integration with templates (#280 & #282)