Add dev documentation on everest vs ert data models (#9820)

add differences everest vs. ert data models in dev. docs

docs/everest/development.rst

@@ -18,6 +18,8 @@ client component.
     :width: 700px
     :alt: Everest architecture
 
+    Everest architecture
+
 Every time an optimization instance is ran by a user, the client component of the
 application spawns an instance of the server component, which is started either on a
 cluster node using LSF (when the `queue_system` is defined to be *lsf*) or on the
@@ -52,3 +54,53 @@ long as the optimization process is running.
    * - POST
      - '/stop'
      - Signal everest optimization run termination. It will be called by the client when the optimization needs to be terminated in the middle of the run
+
+
+EVEREST vs. ERT data models
+===========================
+EVEREST uses ERT for running an `experiment`. An `experiment` contains several `batches` (i.e., `ensembles`).
+An `ensemble` / `batch` contains inputs & computed outputs for several `realizations`, denoted as `simulations` in EVEREST.
+EVEREST `controls` are mapped to ERT `parameters` and serve as input to each forward model evaluation.
+ERT generates `responses` for each forward model evaluation in the `batch`.
+ERT writes these `responses` to storage in the `simulation_results` folder (per `batch` and per `realization`).
+These `responses` are mapped to `objectives` and `constraints` in EVEREST and forwarded to `ropt` (i.e., the optimizer).
+To summarize, every forward model evaluation for a single set of inputs/parameters (`controls`) and generated outputs/responses (`objectives` / `constraints`)
+constitutes an ERT `realization` (`simulation` in EVEREST).
+
+For `controls` in EVEREST, there is a distinction between `unperturbed controls` (i.e., `controls` of the current best solution
+yielding the best objective function value(s), see also :ref:`opt-process-label`) and
+`perturbed controls` (i.e., required to calculate the `gradient`). Furthermore, when performing robust optimization a `batch` contains
+multiple `model_realizations` (denoted by `<MODEL_ID>`). A `model_realization` is a set of `static` variables (i.e., not changing
+during the optimization) which affect the response of the underlying optimized model (NOTE: `model_realization` is not the same as an
+ERT `realization`; see also :ref:`robust-optimization-label`). Each `model_realization` can contain several `simulations`
+(i.e., forward model runs). This is the key difference between the hierarchical data model of EVEREST and ERT (Fig 3).
+NOTE: `<MODEL_ID>` (or `<GEO_ID>` due to legacy reasons) is inserted (and substituted) in the `run_path` for each `model_realization`.
+
+.. figure:: images/Everest_vs_Ert_01.png
+    :align: center
+    :width: 700px
+    :alt: EVEREST vs. ERT data models
+
+    Difference between `ensemble` in ERT and `batch` in EVEREST.
+
+.. figure:: images/Everest_vs_Ert_02.png
+    :align: center
+    :width: 700px
+    :alt: Additional explanation of Fig 3
+
+    Different meaning of `realization` and `simulation`.
+
+The mapping from data models in EVEREST to ERT is done in EVEREST, meaning `realization` (ERT) to `<MODEL_ID>` and `pertubation`-number (EVEREST).
+`Batches` in EVEREST can contain several different configurations depending on the algorithm used. Gradient-based algorithms can have a single function
+evaluation (`unperturbed controls`) per `<MODEL_ID>`, a set of `perturbed controls` per `<MODEL_ID>` to evaluate the gradient, or both.
+Derivative-free methods can have several function evaluations per `<MODEL_ID>` and no `perturbed controls`.
+**NOTE:** the optimizer may decide that some `<MODEL_ID>` are not needed, these are then skipped and the output from `ropt`
+will reflect this (i.e., less `<MODEL_ID>`s in the `batch` results than expected).
+
+.. figure:: images/Everest_vs_Ert_03.png
+    :align: center
+    :width: 700px
+    :alt: Other `batch` configurations EVEREST
+
+    Three other possible configurations of EVEREST `batches` in the context of gradient-based
+    and gradient-free optimization algorithms.

docs/everest/theory.rst

@@ -22,6 +22,8 @@ Other factors often have to be considered. Does the reservoir have pressure supp
 For this simple example of 3 wells the number of possibilities and scenarios to be considered is equal to 3 factorial, i.e. 6 different combinations. Thus it is possible to "manually" test these 6 different drilling combinations in order to find the best possible solution. However what if you are tasked with planning 6 potential wells. This would entail 6 factorial (i.e. 720) combinations of the drilling order to be tested. Can this task of testing 720 different possible combinations be performed manually? What if you have to plan 10 wells or more? As the number of wells increase the potential number of possible combinations increases exponentially.
 Finding or obtaining the optimal order for the wells to be drilled is a tedious task. Solutions to such problems can be obtained using computationally efficient optimization techniques. So what is Optimization and these Optimization techniques?
 
+.. _opt-process-label:
+
 The optimization process
 ------------------------
 **Optimization** is the process of finding the best possible solution for a particular problem. For petroleum/reservoir engineering, such problems would primarily imply
@@ -90,6 +92,8 @@ Note that a larger number of blue dots will increase the quality of the gradient
 Traditionally many studies have used 10-50 perturbed controls during the optimization process to estimate the gradient.
 These studies have used a single model realization (i.e. a single geological representation) of the reservoir which may not always be an accurate representation of the real reservoir system.
 
+.. _robust-optimization-label:
+
 Robust Optimization: Stochastic Simplex Gradients (StoSAG)
 ==========================================================
 In reality, geological and reservoir modeling process is fraught with uncertainties since a reservoir is modeled using uncertain interpretations based on uncertain data sources such as seismic, well logs etc.