Documentation source update detected and pushed compilation build dir…

…ectory for Github Pages

docs/compilation/main/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: c6c831a774d6803ae0644b5bcc441bf4
+config: 6ec10aa988f56f8653db3d2767195dda
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/compilation/main/_sources/pages/deep-reinforcement-learning.rst.txt

@@ -24,30 +24,31 @@ Current algorithms checked by *Sinergym* are:
 | TD3       |    NO    |     YES    | OffPolicyAlgorithm |
 +-----------+----------+------------+--------------------+
 
-``Type`` column has been specified due to its importance about 
-*Stable Baselines callback* functionality.
-
-****************
-DRL Logger
-****************
-
+For that purpose, we are going to refine and develop 
 `Callbacks <https://stable-baselines3.readthedocs.io/en/master/guide/callbacks.html>`__ 
-are a set of functions that will be called at given **stages of the training procedure**. 
+which are a set of functions that will be called at given **stages of the training procedure**. 
 You can use callbacks to access internal state of the RL model **during training**. 
 It allows one to do monitoring, auto saving, model manipulation, progress bars, ...
+Our callbacks inherit from Stable Baselines 3 and are available in 
+`sinergym/sinergym/utils/callbacks.py <https://github.com/ugr-sail/sinergym/blob/main/sinergym/utils/callbacks.py>`__.
+
+``Type`` column has been specified due to its importance about 
+*Stable Baselines callback* functionality.
+
+********************
+DRL Callback Logger
+********************
 
-This structure allows to custom our own logger for DRL executions. Our objective 
-is to **log all information about our custom environment** specifically.
-Therefore, `sinergym/sinergym/utils/callbacks.py <https://github.com/ugr-sail/sinergym/blob/main/sinergym/utils/callbacks.py>`__ 
-has been created with this proposal. Each algorithm has its own differences 
-about how information is extracted which is why its implementation. ``LoggerCallback``
-can deal with those subtleties.
+A callback allows to custom our own logger for DRL Sinergym executions. Our objective 
+is to **log all information about our custom environment** specifically in real-time.
+Each algorithm has its own differences 
+about how information is extracted which is why its implementation.
 
 .. note:: You can specify if you want Sinergym logger (see :ref:`Logger`) to record 
           simulation interactions during training at the same time using 
           ``sinergym_logger`` attribute in constructor. 
 
-This callback derives ``BaseCallback`` from Stable Baselines 3 while ``BaseCallBack`` 
+``LoggerCallback`` inherits from Stable Baselines 3 ``BaseCallback`` and 
 uses `Tensorboard <https://www.tensorflow.org/tensorboard?hl=es-419>`__ on the 
 background at the same time. With *Tensorboard*, it's possible to visualize all DRL 
 training in real time and compare between different executions. This is an example: 
@@ -69,9 +70,33 @@ It is important the difference between ``OnPolicyAlgorithms`` and ``OffPolicyAlg
   Thus, Off Policy Algorithms record a **mean value** of whole episode values instead 
   of values steps by steps (see ``LoggerCallback`` class implementation).
 
-~~~~~~~~~~~~~~~~~~~~~~
+********************
+Evaluation Callback
+********************
+
+A callback has also been refined for the evaluation of the model versions obtained during 
+the training process with Sinergym, so that it stores the best model obtained (not the one resulting 
+at the end of the training).
+
+Its name is ``LoggerEvalCallback`` and it inherits from Stable Baselines 3 ``EvalCallback``. 
+The main feature added is that the model evaluation is logged in a particular section in 
+Tensorboard too for the concrete metrics of the building model.
+
+We have to define in ``LoggerEvalCallback`` construction how many training episodes we want 
+the evaluation process to take place. On the other hand, we have to define how many episodes 
+are going to occupy each of the evaluations to be performed. 
+
+The more episodes, the more accurate the averages of the reward-based indicators will be, and, 
+therefore, the more faithful it will be to reality in terms of how good the current model is 
+turning out to be. However, it will take more time.
+
+It calculates timestep and episode average for power consumption, comfort penalty and power penalty.
+On the other hand, it calculates too comfort violation percentage in episodes too.
+Currently, only mean reward is taken into account to decide when a model is better.
+
+***********************
 Tensorboard structure
-~~~~~~~~~~~~~~~~~~~~~~
+***********************
 
 The main structure for *Sinergym* with *Tensorboard* is:
 

docs/compilation/main/_sources/pages/gcloudAPI.rst.txt

@@ -342,7 +342,7 @@ The list of parameter is pretty large. Let's see it:
 - ``--tensorboard`` or ``-tens``: This parameter will contain a 
   **path-file** or **path-remote-bucket** to allocate tensorboard 
   training logs. If it isn't specified this log will be deactivate 
-  (see :ref:`DRL Logger`).
+  (see :ref:`Tensorboard structure`).
 
 - ``--evaluation`` or ``-eval``: If it is specified, evaluation 
   callback will be activate, else model evaluation will be deactivate 

docs/compilation/main/pages/deep-reinforcement-learning.html

@@ -103,12 +103,11 @@
 <li class="toctree-l1"><a class="reference internal" href="extra-configuration.html">7. Extra Configuration in Sinergym simulations</a></li>
 <li class="toctree-l1"><a class="reference internal" href="output.html">8. Output format</a></li>
 <li class="toctree-l1 current"><a class="current reference internal" href="#">9. Deep Reinforcement Learning Integration</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="#drl-logger">9.1. DRL Logger</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#tensorboard-structure">9.1.1. Tensorboard structure</a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="#how-use">9.2. How use</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#mlflow">9.3. Mlflow</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#drl-callback-logger">9.1. DRL Callback Logger</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#evaluation-callback">9.2. Evaluation Callback</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#tensorboard-structure">9.3. Tensorboard structure</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#how-use">9.4. How use</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#mlflow">9.5. Mlflow</a></li>
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="gcloudAPI.html">10. Sinergym with Google Cloud</a></li>
@@ -267,27 +266,28 @@ <h1><span class="section-number">9. </span>Deep Reinforcement Learning Integrati
 </tr>
 </tbody>
 </table>
+<p>For that purpose, we are going to refine and develop
+<a class="reference external" href="https://stable-baselines3.readthedocs.io/en/master/guide/callbacks.html">Callbacks</a>
+which are a set of functions that will be called at given <strong>stages of the training procedure</strong>.
+You can use callbacks to access internal state of the RL model <strong>during training</strong>.
+It allows one to do monitoring, auto saving, model manipulation, progress bars, …
+Our callbacks inherit from Stable Baselines 3 and are available in
+<a class="reference external" href="https://github.com/ugr-sail/sinergym/blob/main/sinergym/utils/callbacks.py">sinergym/sinergym/utils/callbacks.py</a>.</p>
 <p><code class="docutils literal notranslate"><span class="pre">Type</span></code> column has been specified due to its importance about
 <em>Stable Baselines callback</em> functionality.</p>
-<section id="drl-logger">
-<h2><span class="section-number">9.1. </span>DRL Logger<a class="headerlink" href="#drl-logger" title="Permalink to this heading">¶</a></h2>
-<p><a class="reference external" href="https://stable-baselines3.readthedocs.io/en/master/guide/callbacks.html">Callbacks</a>
-are a set of functions that will be called at given <strong>stages of the training procedure</strong>.
-You can use callbacks to access internal state of the RL model <strong>during training</strong>.
-It allows one to do monitoring, auto saving, model manipulation, progress bars, …</p>
-<p>This structure allows to custom our own logger for DRL executions. Our objective
-is to <strong>log all information about our custom environment</strong> specifically.
-Therefore, <a class="reference external" href="https://github.com/ugr-sail/sinergym/blob/main/sinergym/utils/callbacks.py">sinergym/sinergym/utils/callbacks.py</a>
-has been created with this proposal. Each algorithm has its own differences
-about how information is extracted which is why its implementation. <code class="docutils literal notranslate"><span class="pre">LoggerCallback</span></code>
-can deal with those subtleties.</p>
+<section id="drl-callback-logger">
+<h2><span class="section-number">9.1. </span>DRL Callback Logger<a class="headerlink" href="#drl-callback-logger" title="Permalink to this heading">¶</a></h2>
+<p>A callback allows to custom our own logger for DRL Sinergym executions. Our objective
+is to <strong>log all information about our custom environment</strong> specifically in real-time.
+Each algorithm has its own differences
+about how information is extracted which is why its implementation.</p>
 <div class="admonition note">
 <p class="admonition-title">Note</p>
 <p>You can specify if you want Sinergym logger (see <a class="reference internal" href="output.html#logger"><span class="std std-ref">Logger</span></a>) to record
 simulation interactions during training at the same time using
 <code class="docutils literal notranslate"><span class="pre">sinergym_logger</span></code> attribute in constructor.</p>
 </div>
-<p>This callback derives <code class="docutils literal notranslate"><span class="pre">BaseCallback</span></code> from Stable Baselines 3 while <code class="docutils literal notranslate"><span class="pre">BaseCallBack</span></code>
+<p><code class="docutils literal notranslate"><span class="pre">LoggerCallback</span></code> inherits from Stable Baselines 3 <code class="docutils literal notranslate"><span class="pre">BaseCallback</span></code> and
 uses <a class="reference external" href="https://www.tensorflow.org/tensorboard?hl=es-419">Tensorboard</a> on the
 background at the same time. With <em>Tensorboard</em>, it’s possible to visualize all DRL
 training in real time and compare between different executions. This is an example:</p>
@@ -303,8 +303,27 @@ <h2><span class="section-number">9.1. </span>DRL Logger<a class="headerlink" hre
 Thus, Off Policy Algorithms record a <strong>mean value</strong> of whole episode values instead
 of values steps by steps (see <code class="docutils literal notranslate"><span class="pre">LoggerCallback</span></code> class implementation).</p></li>
 </ul>
+</section>
+<section id="evaluation-callback">
+<h2><span class="section-number">9.2. </span>Evaluation Callback<a class="headerlink" href="#evaluation-callback" title="Permalink to this heading">¶</a></h2>
+<p>A callback has also been refined for the evaluation of the model versions obtained during
+the training process with Sinergym, so that it stores the best model obtained (not the one resulting
+at the end of the training).</p>
+<p>Its name is <code class="docutils literal notranslate"><span class="pre">LoggerEvalCallback</span></code> and it inherits from Stable Baselines 3 <code class="docutils literal notranslate"><span class="pre">EvalCallback</span></code>.
+The main feature added is that the model evaluation is logged in a particular section in
+Tensorboard too for the concrete metrics of the building model.</p>
+<p>We have to define in <code class="docutils literal notranslate"><span class="pre">LoggerEvalCallback</span></code> construction how many training episodes we want
+the evaluation process to take place. On the other hand, we have to define how many episodes
+are going to occupy each of the evaluations to be performed.</p>
+<p>The more episodes, the more accurate the averages of the reward-based indicators will be, and,
+therefore, the more faithful it will be to reality in terms of how good the current model is
+turning out to be. However, it will take more time.</p>
+<p>It calculates timestep and episode average for power consumption, comfort penalty and power penalty.
+On the other hand, it calculates too comfort violation percentage in episodes too.
+Currently, only mean reward is taken into account to decide when a model is better.</p>
+</section>
 <section id="tensorboard-structure">
-<h3><span class="section-number">9.1.1. </span>Tensorboard structure<a class="headerlink" href="#tensorboard-structure" title="Permalink to this heading">¶</a></h3>
+<h2><span class="section-number">9.3. </span>Tensorboard structure<a class="headerlink" href="#tensorboard-structure" title="Permalink to this heading">¶</a></h2>
 <p>The main structure for <em>Sinergym</em> with <em>Tensorboard</em> is:</p>
 <ul>
 <li><p><strong>action</strong>: This section has action values during training. When algorithm
@@ -352,9 +371,8 @@ <h3><span class="section-number">9.1.1. </span>Tensorboard structure<a class="he
 to model learn method.</p>
 </div>
 </section>
-</section>
 <section id="how-use">
-<h2><span class="section-number">9.2. </span>How use<a class="headerlink" href="#how-use" title="Permalink to this heading">¶</a></h2>
+<h2><span class="section-number">9.4. </span>How use<a class="headerlink" href="#how-use" title="Permalink to this heading">¶</a></h2>
 <p>You can try your own experiments and benefit from this functionality.
 <a class="reference external" href="https://github.com/ugr-sail/sinergym/blob/main/scripts/DRL_battery.py">sinergym/scripts/DRL_battery.py</a>
 is a example code to use it. You can use <code class="docutils literal notranslate"><span class="pre">DRL_battery.py</span></code> directly from
@@ -380,7 +398,7 @@ <h2><span class="section-number">9.2. </span>How use<a class="headerlink" href="
 </ul>
 </section>
 <section id="mlflow">
-<h2><span class="section-number">9.3. </span>Mlflow<a class="headerlink" href="#mlflow" title="Permalink to this heading">¶</a></h2>
+<h2><span class="section-number">9.5. </span>Mlflow<a class="headerlink" href="#mlflow" title="Permalink to this heading">¶</a></h2>
 <p>Our scripts to run DRL with <em>Sinergym</em> environments are using
 <a class="reference external" href="https://mlflow.org/">Mlflow</a>, in order to <strong>tracking experiments</strong>
 and recorded them methodically. It is recommended to use it.

docs/compilation/main/pages/gcloudAPI.html

@@ -1133,7 +1133,7 @@ <h2><span class="section-number">10.2. </span>Executing experiments in remote co
 <li><p><code class="docutils literal notranslate"><span class="pre">--tensorboard</span></code> or <code class="docutils literal notranslate"><span class="pre">-tens</span></code>: This parameter will contain a
 <strong>path-file</strong> or <strong>path-remote-bucket</strong> to allocate tensorboard
 training logs. If it isn’t specified this log will be deactivate
-(see <a class="reference internal" href="deep-reinforcement-learning.html#drl-logger"><span class="std std-ref">DRL Logger</span></a>).</p></li>
+(see <a class="reference internal" href="deep-reinforcement-learning.html#tensorboard-structure"><span class="std std-ref">Tensorboard structure</span></a>).</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">--evaluation</span></code> or <code class="docutils literal notranslate"><span class="pre">-eval</span></code>: If it is specified, evaluation
 callback will be activate, else model evaluation will be deactivate
 during training (see <a class="reference internal" href="deep-reinforcement-learning.html#deep-reinforcement-learning-integration"><span class="std std-ref">Deep Reinforcement Learning Integration</span></a>).</p></li>

docs/compilation/v1.4.0/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 468e916dfbcce0b9adea7ae2d1adc39a
+config: 2bc733e232670197528c0a11ccbd4a2b
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/compilation/v1.6.0/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 9f4bbfcb8ab598acd5fb0277d311de00
+config: 67f5b1d34df4d267f79e4e889cd744cf
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/compilation/v1.7.0/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 3d4aa0353970258c6c9e5d91c66832af
+config: 1f1d1b2fdb9d117324d2eaa5535c046e
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/compilation/v1.7.2/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: d2108f6a345cbbb131e958b49ad426a3
+config: 01cc9c771ab743c7c48e467a324aafbb
 tags: 645f666f9bcd5a90fca523b33c5a78b7