Skip to content

Commit

Permalink
more review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
StephanDeHoop committed Feb 4, 2025
1 parent ba5d7e0 commit 3685354
Showing 1 changed file with 13 additions and 22 deletions.
35 changes: 13 additions & 22 deletions docs/everest/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -58,29 +58,20 @@ long as the optimization process is running.

EVEREST vs. ERT data models
===========================
EVEREST uses ERT for running an experiment. Whenver an EVEREST `experiment` is ran, `everserver` creates an `EverestRunModel` based on the
`ErtConfig` (obtained from `EverestConfig`). The `ErtConfig` requires an `EnsembleConfig` where EVEREST `objectives`
and `constraints` are mapped to ERT `responses` using `GenDataConfig` (stored in `EnsembleConfig.response_config`).
In `everest_to_ert_config` all the EVEREST `controls` are mapped to ERT `parameters` using `ExtParamConfig`
(stored in `EnsembleConfig.parameter_configs`). Then in `EverestRunModel._forward_model_evaluator` (used as evaluator
by the external optimization library `ropt`) a local storage is created for the `batch` (i.e., `ensemble`),
linked to the current `experiment`, and setup up `simulations` for all sets of `controls`.

All the forward models in the `batch` are then evaluated in ERT using `EverestRunModel._evaluate_and_postprocess`
(and in turn `BaseRunModel.run_workflows`) which uses the `EnsembleEvaluator` for execution. After ERT writes
to `simulation_results` and stores results in the `LocalEnsemble` (i.e., storage)
these results are then gathered in the forward model evaluator (called by `ropt`) and feedforward into the optimizer using
`EverestRunModel._gather_simulation_results` (i.e., reading the `.parquet` files generated by ERT). Every set of
forward model executions for a single set of inputs (`controls`) and outputs (`objectives` / `constraints`)
constitutes an ERT `realization` in the ERT queue system; denoted in EVEREST as a `simulation`.

There is a distinction between `unperturbed controls` (i.e., current `objective function` value) and
EVEREST uses ERT for running an `experiment`. An `experiment` contains several `batches` (i.e., `ensembles`).
EVEREST `controls` are mapped to ERT `parameters` and serve as input to each forward model evaluation.
ERT generates `responses` for each forward model evaluation in the `batch`.
ERT writes these `responses` to storage in the `simulation_results` folder (per `batch` and per `realization`, as defined in the `runpath`).
These `responses` are mapped to `objectives` and `constraints` in EVEREST and read by `ropt` (i.e., the optimizer).
To summarize, every forward model evaluation for a single set of inputs/parameters (`controls`) and generated outputs/responses (`objectives` / `constraints`)
constitutes an ERT `realization` which is denoted in EVEREST as a `simulation`.

For `controls` in EVEREST, there is a distinction between `unperturbed controls` (i.e., current `objective function` value) and
`perturbed controls` (i.e., required to calculate the `gradient`).
Furthermore, when performing robust optimization (i.e., multiple static `realizations`) a `batch` contains a
certain number of `realizations` (denoted by `<GEO_ID>`) and each `realization` contains a number of `simulations`
(i.e., forward model runs). These `simulations` are forward model runs for either `unperturbed controls` and/or
`perturbed controls`. This is the key differences between the hierarchical data model of EVEREST and ERT (Fig 3).
NOTE: `<GEO_ID>` is inserted in `everest_to_ert._extract_environment` and substituted in `ert.substitutions.substitute_real_iter` for all of the `run_paths`.
Furthermore, when performing robust optimization (i.e., having multiple static `geo_realizations` / `model_realizations`, NOTE: not the same as an ERT `realization`) a `batch` contains
multiple `geo_realizations` (denoted by `<GEO_ID>`) and each `geo_realization` can contain several `simulations`
(i.e., forward model run). This is the key differences between the hierarchical data model of EVEREST and ERT (Fig 3).
NOTE: `<GEO_ID>` is inserted (and substituted) in the `run_path` for each `geo_realization`.

.. figure:: images/Everest_vs_Ert_01.png
:align: center
Expand Down

0 comments on commit 3685354

Please sign in to comment.