Merge pull request #296 from Epistimio/release-v0.1.7rc2

Release v0.1.7rc2

.travis.yml

@@ -37,7 +37,13 @@ jobs:
         dist: xenial
         sudo: true
       - <<: *test
-        env: TOXENV=demo_random
+        env: TOXENV=demo-random
+        python: 3.6
+      - <<: *test
+        env: TOXENV=backward-compatibility, ORION_DB_TYPE=mongodb
+        python: 3.6
+      - <<: *test
+        env: TOXENV=backward-compatibility, ORION_DB_TYPE=pickleddb
         python: 3.6
       - stage: packaging
         env: TOXENV=packaging

ROADMAP.md

@@ -1,9 +1,9 @@
 # Roadmap
-Last update August 26th, 2019
+Last update October 8th, 2019
 
 ## Next releases - Short-Term
 
-### v0.1.7
+### v0.1.8
 
 #### Preliminary Python API
 Library API to simplify usage of algorithms without Oríon's worker.

docs/src/code/core/cli.rst

@@ -12,6 +12,5 @@ Command lines
    cli/hunt
    cli/insert
    cli/init_only
-   cli/setup
-   cli/test_db
+   cli/db
    cli/evc

docs/src/code/core/cli/db.rst

@@ -0,0 +1,12 @@
+db commands
+===========
+
+.. automodule:: orion.core.cli.db_main
+   :members:
+
+.. toctree::
+   :maxdepth: 1
+   :caption: DB command line modules
+
+   db/test
+   db/setup

docs/src/code/core/cli/setup.rst → docs/src/code/core/cli/db/setup.rst

@@ -1,5 +1,5 @@
 setup command
 =============
 
-# .. automodule:: orion.core.cli.setup
+# .. automodule:: orion.core.cli.db.setup
 #    :members:

docs/src/code/core/cli/db/test.rst

@@ -0,0 +1,5 @@
+db test command
+===============
+
+.. automodule:: orion.core.cli.db.test
+   :members:

docs/src/install/core.rst

@@ -2,6 +2,9 @@
 Installation of Orion's core
 ****************************
 
+Oríon should work on most Linux distributions and Mac OS X. It is tested on Ubuntu 16.04 LTS and Mac
+OS X 10.13. We do not support Windows and there is no short term plan to do so.
+
 Via PyPI
 ========
 

docs/src/install/database.rst

@@ -91,7 +91,7 @@ Configuring Oríon's Database
 
 There are different ways that database backend attributes can be configured.
 The first one is by using a global configuration file, which can easily be done
-using the command ``orion setup``. This will create a yaml file
+using the command ``orion db setup``. This will create a yaml file
 of the following format.
 
    .. code-block:: yaml
@@ -199,11 +199,11 @@ EphemeralDB has no arguments.
 Test connection
 ===============
 
-You can use the command ``orion test-db`` to test the setup of your database backend.
+You can use the command ``orion db test`` to test the setup of your database backend.
 
 .. code-block:: sh
 
-   $ orion test-db
+   $ orion db test
 
    Check for a configuration inside the default paths...
        {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost'}
@@ -227,7 +227,7 @@ stage. Here's an example including all three configuration methods.
 
 .. code-block:: sh
 
-   $ ORION_DB_PORT=27018 orion test_db --config local.yaml
+   $ ORION_DB_PORT=27018 orion db test --config local.yaml
 
    Check for a configuration inside the global paths...
        {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost'}
@@ -244,7 +244,7 @@ that will be used and then prints the instance created to confirm the database t
 
 .. code-block:: sh
 
-   $ orion test-db
+   $ orion db test
 
    [...]
 
@@ -257,11 +257,30 @@ tests fail because of insufficient user access rights on the database.
 
 .. code-block:: sh
 
-   $ orion test-db
+   $ orion db test
 
    [...]
 
    Check if database supports write operation... Success
    Check if database supports read operation... Success
    Check if database supports count operation... Success
    Check if database supports delete operation... Success
+
+
+Upgrade Database
+================
+
+Database scheme may change from one version of Oríon to another. If such change happens, you will
+get the following error after upgrading Oríon.
+
+.. code-block:: sh
+
+   The database is outdated. You can upgrade it with the command `orion db upgrade`.
+
+Make sure to create a backup of your database before upgrading it. You should also make sure that no
+process writes to the database during the upgrade otherwise the latter could fail. When ready,
+simply run the upgrade command.
+
+.. code-block:: sh
+
+   orion db upgrade

docs/src/user/algorithms.rst

@@ -157,9 +157,8 @@ Configuration
 .. code-block:: yaml
 
      algorithms:
-        bayesopt:
+        BayesianOptimizer:
            seed: null
-           strategy: cl_min
            n_initial_points: 10
            acq_func: gp_hedge
            alpha: 1e-10
@@ -169,13 +168,6 @@ Configuration
 
 ``seed``
 
-
-``strategy``
-
-Method to use to sample multiple points.
-Supported options are `"cl_min"`, `"cl_mean"` or `"cl_max"`.
-Check skopt docs for details.
-
 ``n_initial_points``
 
 Number of evaluations of ``func`` with initialization points

src/orion/core/cli/db/__init__.py

@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+:mod:`orion.core.cli.db` -- Functions that define console scripts for database
+==============================================================================
+
+.. module:: cli
+   :platform: Unix
+   :synopsis: Commands to maintain database
+
+"""

src/orion/core/cli/db/setup.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+:mod:`orion.core.cli.db.setup` -- Module running the setup command
+===============================================================
+
+.. module:: setup
+   :platform: Unix
+   :synopsis: Creates a configurarion file for the database.
+"""
+
+import logging
+import os
+
+import yaml
+
+import orion.core
+
+log = logging.getLogger(__name__)
+
+
+def add_subparser(parser):
+    """Return the parser that needs to be used for this command"""
+    setup_parser = parser.add_parser('setup', help='setup help')
+
+    setup_parser.set_defaults(func=main)
+
+    return setup_parser
+
+
+def ask_question(question, default=None):
+    """Ask a question to the user and receive an answer.
+
+    Parameters
+    ----------
+    question: str
+        The question to be asked.
+    default: str
+        The default value to use if the user enters nothing.
+
+    Returns
+    -------
+    str
+        The answer provided by the user.
+
+    """
+    if default is not None:
+        question = question + " (default: {}) ".format(default)
+
+    answer = input(question)
+
+    if answer.strip() == "":
+        return default
+
+    return answer
+
+
+# pylint: disable = unused-argument
+def main(*args):
+    """Build a configuration file."""
+    default_file = orion.core.DEF_CONFIG_FILES_PATHS[-1]
+
+    if os.path.exists(default_file):
+        cancel = ''
+        while cancel.strip().lower() not in ['y', 'n']:
+            cancel = ask_question(
+                "This will overwrite {}, do you want to proceed? (y/n) ".format(default_file), "n")
+
+        if cancel.strip().lower() == 'n':
+            return
+
+    _type = ask_question("Enter the database type: ", "mongodb")
+    name = ask_question("Enter the database name: ", "test")
+    host = ask_question("Enter the database host: ", "localhost")
+
+    config = {'database': {'type': _type, 'name': name, 'host': host}}
+
+    print("Default configuration file will be saved at: ")
+    print(default_file)
+
+    dirs = '/'.join(default_file.split('/')[:-1])
+    os.makedirs(dirs, exist_ok=True)
+
+    with open(default_file, 'w') as output:
+        yaml.dump(config, output, default_flow_style=False)

src/orion/core/cli/db/test.py

@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+:mod:`orion.core.cli.db.test` -- Module to check if the DB worrks
+=================================================================
+
+.. module:: test_db
+   :platform: Unix
+   :synopsis: Runs multiple checks to see if the database was correctly setup.
+
+"""
+import argparse
+import logging
+
+from orion.core.cli.checks.creation import CreationStage
+from orion.core.cli.checks.operations import OperationsStage
+from orion.core.cli.checks.presence import PresenceStage
+from orion.core.io.experiment_builder import ExperimentBuilder
+from orion.core.utils.exceptions import CheckError
+
+log = logging.getLogger(__name__)
+
+
+def add_subparser(parser):
+    """Add the subparser that needs to be used for this command"""
+    test_db_parser = parser.add_parser('test', help='test_db help')
+
+    test_db_parser.add_argument('-c', '--config', type=argparse.FileType('r'),
+                                metavar='path-to-config', help="user provided "
+                                "orion configuration file")
+
+    test_db_parser.set_defaults(func=main)
+
+    return test_db_parser
+
+
+def main(args):
+    """Run through all checks for database."""
+    experiment_builder = ExperimentBuilder()
+    presence_stage = PresenceStage(experiment_builder, args)
+    creation_stage = CreationStage(presence_stage)
+    operations_stage = OperationsStage(creation_stage)
+    stages = [presence_stage, creation_stage, operations_stage]
+
+    for stage in stages:
+        for check in stage.checks():
+            print(check.__doc__, end='.. ')
+            try:
+                status, msg = check()
+                print(status)
+                if status == "Skipping":
+                    print(msg)
+            except CheckError as ex:
+                print("Failure")
+                print(ex)
+
+        stage.post_stage()