From 734b24d6461b0c54f5fa0e020b70a663e4138252 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Feb 2019 09:44:26 +0800
Subject: [PATCH 01/46] Update README.md

---
 keras/README.md | 64 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/keras/README.md b/keras/README.md
index 34c4df3..67cc57a 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -1 +1,65 @@
+# Tutorials for running distribued Keras (v1.2.2) on Analytics Zoo
 Tutorials for running _**distribued Keras (v1.2.2) on Analytics Zoo**_. These tutorials are ported from François Chollet's [Jupyter notebooks](https://github.com/fchollet/deep-learning-with-python-notebooks) for the book [Deep Learning with Python (Manning Publications)](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff)
+
+This repository is built to describe how to write Keras-style code and directly run it in analytics-zoo so that we could run the original Keras code in distributed mode via limited modification. We make this repository based on notebook https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python, which is the sample code repository of book "Deep Learning with Python" based on Keras 2.0.8 and we pick some basic chapters from it. 
+
+Currently analytics-zoo builds its Keras-style code based on Keras 1.2.2. Thus, this repository contains the code of implementation of original Keras 2.0.8 code based on Keras 1.2.2, and how to modify the code in order to run it in analytics-zoo.
+
+To make it simple, we omit the description of plenty concepts here and you could find them in original notebook above [(link)](https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python). Besides, we directly post Keras 1.2.2 code here and the replacements needed from Keras 2.0.8 to Keras 1.2.2 are noted in `Keras_2-to-1.md`.
+
+This repository use Python 3.5, Keras 1.2.2 (Keras code), Analytics-zoo 0.4.0 (zoo code). We post the summary of Keras-to-zoo code convertion as well as the table of contents.
+
+## Summary of Keras-to-zoo code convertion
+We summarize the modification we need to make here, attached with the link to the example code chapter.
+
+#### First of all
+Make sure you have analytics-zoo installed, see install guide [here](https://analytics-zoo.github.io/master/#PythonUserGuide/install/). Then set the environment variables like following
+  
+    PYSPARK_PYTHON=/path_to_your_python
+    PYSPARK_DRIVER_PYTHON=/path_to_your_python
+    # To avoid version conflict, in my ubuntu, this path is /usr/bin/python3.5
+
+    SPARK_DRIVER_MEMORY=4g
+    # If you encounter heap space exception, you could simply increase this variable to 8g, 16g, etc. 
+    # If no more memory is available on your machine, you could consider reduce the data size.
+
+Then, make sure you have following code at the beginning of your zoo code.
+  
+    from zoo.common.nncontext import *
+    sc = init_nncontext(init_spark_conf().setMaster("local[4]"))
+    
+we set core number to 4 above, you can also set it with another number. But we still recommend 4 because analytics-zoo need the core number could divide the batch size of learning, which is normal set as powers of 4, e.g. 16, 32, 128. Error would be raised if this requirement is not satisfied.
+
+#### Accuracy checkout
+Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via tensorboard, see [Chapter 3.5]()
+
+#### Evaluate return
+The return of `evaluate` method is an `EvaluationResult` object, which is different from Keras, see [Chapter 2.1]()
+
+#### Predict result
+The return of `predict` method is RDD, so you need to call `collect` method to collect them, see [Chapter 3.5]()
+
+#### Parameters not supported
+Analytics-zoo does not support following parameters currently
+
+* validation_split:
+* verbose:
+
+## Table of contents
+
+The main purpose of this repository is to decribe the convertion from Keras to analytics-zoo so that we rename the notebooks to make it more experienced-user oriented. We keep the chapter index so you could still make a reference to the original notebook [here](https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python). We only keep some key description of the original notebook.
+
+* Chapter 2:
+    * [2.1: MNIST]()
+* Chapter 3:
+    * [3.5: Binary classification]()
+    * [3.6: Multi-class classification]()
+    * [3.7: Regression]()
+* Chapter 4:
+    * [4.4: Regularization and Dropout]()
+* Chapter 5:
+    * [5.1: CNN]()
+* Chapter 6:
+    * [6.2: RNN]()
+
+    

From 9006edb606d9171c23622bda286d214b2a080cef Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Feb 2019 09:51:16 +0800
Subject: [PATCH 02/46] Add files via upload

---
 keras/2.1-mnist.ipynb                      | 144 ++++++++++++++++
 keras/3.5-binary-classification.ipynb      | 190 +++++++++++++++++++++
 keras/3.6-multi-class-classification.ipynb | 130 ++++++++++++++
 keras/3.7-regression.ipynb                 | 174 +++++++++++++++++++
 keras/4.4-regularization-and-dropout.ipynb |  53 ++++++
 keras/5.1-cnn.ipynb                        | 126 ++++++++++++++
 keras/6.2-rnn.ipynb                        | 171 +++++++++++++++++++
 7 files changed, 988 insertions(+)
 create mode 100644 keras/2.1-mnist.ipynb
 create mode 100644 keras/3.5-binary-classification.ipynb
 create mode 100644 keras/3.6-multi-class-classification.ipynb
 create mode 100644 keras/3.7-regression.ipynb
 create mode 100644 keras/4.4-regularization-and-dropout.ipynb
 create mode 100644 keras/5.1-cnn.ipynb
 create mode 100644 keras/6.2-rnn.ipynb

diff --git a/keras/2.1-mnist.ipynb b/keras/2.1-mnist.ipynb
new file mode 100644
index 0000000..d971f5d
--- /dev/null
+++ b/keras/2.1-mnist.ipynb
@@ -0,0 +1,144 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# MNIST\n",
+    "\n",
+    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset.\n",
+    "\n",
+    "The MNIST dataset comes pre-loaded in Keras, in the form of a set of four Numpy arrays."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import mnist\n",
+    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Import the modules we need to build the network. In Keras it is:\n",
+    "\n",
+    "    from keras import models\n",
+    "    from keras import layers\n",
+    "Just replace it with following in order to use analytics-zoo:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Build the network, compile and fit:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network = models.Sequential()\n",
+    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
+    "network.add(layers.Dense(10, activation='softmax'))\n",
+    "\n",
+    "network.compile(optimizer='rmsprop',\n",
+    "                loss='categorical_crossentropy',\n",
+    "                metrics=['accuracy'])\n",
+    "\n",
+    "train_images = train_images.reshape((60000, 28 * 28))\n",
+    "train_images = train_images.astype('float32') / 255\n",
+    "\n",
+    "test_images = test_images.reshape((10000, 28 * 28))\n",
+    "test_images = test_images.astype('float32') / 255\n",
+    "\n",
+    "\n",
+    "from keras.utils.np_utils import to_categorical\n",
+    "train_labels = to_categorical(train_labels)\n",
+    "#test_labels = to_categorical(test_labels)\n",
+    "\n",
+    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Evaluate return\n",
+    "Check our result on test set. In Keras it is:\n",
+    "\n",
+    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
+    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
+    "print('test_acc:', test_result[0].result)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/keras/3.5-binary-classification.ipynb b/keras/3.5-binary-classification.ipynb
new file mode 100644
index 0000000..929a731
--- /dev/null
+++ b/keras/3.5-binary-classification.ipynb
@@ -0,0 +1,190 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Binary classification\n",
+    "We'll be working with \"IMDB dataset\", a set of 50,000 highly-polarized reviews from the Internet Movie Database. They are split into 25,000 reviews for training and 25,000 reviews for testing, each set consisting in 50% negative and 50% positive reviews.\n",
+    "\n",
+    "Just like the MNIST dataset, the IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words) have been turned into sequences of integers, where each integer stands for a specific word in a dictionary.\n",
+    "\n",
+    "The following code will load the dataset (when you run it for the first time, about 80MB of data will be downloaded to your machine).\n",
+    "\n",
+    "Then we vectorize the sequences to prepare the data we are going to feed to the model. We also separate part of the data for validation."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import imdb\n",
+    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)\n",
+    "\n",
+    "import numpy as np\n",
+    "def vectorize_sequences(sequences, dimension=10000):\n",
+    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
+    "    results = np.zeros((len(sequences), dimension))\n",
+    "    for i, sequence in enumerate(sequences):\n",
+    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
+    "    return results\n",
+    "\n",
+    "y_train = np.asarray(train_labels).astype('float32')\n",
+    "y_test = np.asarray(test_labels).astype('float32')\n",
+    "\n",
+    "x_val = x_train[:10000]\n",
+    "partial_x_train = x_train[10000:]\n",
+    "y_val = y_train[:10000]\n",
+    "partial_y_train = y_train[10000:]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Build the network, then compile:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "model = models.Sequential()\n",
+    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "model.add(layers.Dense(16, activation='relu'))\n",
+    "model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='binary_crossentropy',\n",
+    "              metrics=['accuracy'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Accuracy checkout\n",
+    "To checkout the behavior of this model in Keras, the original code use `matplotlib` library to draw the following `history` object\n",
+    "    \n",
+    "    history = model.fit(partial_x_train,\n",
+    "                    partial_y_train,\n",
+    "                    nb_epoch=5,\n",
+    "                    batch_size=512,\n",
+    "                    validation_data=(x_val, y_val)\n",
+    "                    )\n",
+    "After `fit` method finishes, the results are stored in `history` and thus could be visualized.\n",
+    "\n",
+    "Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via tensorboard. Code above need to be replaced with following:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.set_tensorboard('./', '3-5_summary')\n",
+    "model.fit(partial_x_train,\n",
+    "          partial_y_train,\n",
+    "          nb_epoch=5,\n",
+    "          batch_size=512,\n",
+    "          validation_data=(x_val, y_val)\n",
+    "          )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then you could see result in tensorboard by command in terminal `tensorboard --logdir ./`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Check the result on test data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "results = model.evaluate(x_test, y_test)\n",
+    "print('test_acc:', results[0].result)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Predict result\n",
+    "Predict the result on test data, in Keras, it is easy to just call following code to get the result\n",
+    "\n",
+    "    model.predict(x_test)\n",
+    "In analytics-zoo, the return of `predict` is RDD, so you need to call `collect` method to get the result:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prediction = model.predict(x_test)\n",
+    "result = prediction.collect()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/keras/3.6-multi-class-classification.ipynb b/keras/3.6-multi-class-classification.ipynb
new file mode 100644
index 0000000..7e8c9ff
--- /dev/null
+++ b/keras/3.6-multi-class-classification.ipynb
@@ -0,0 +1,130 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Multi-class classification\n",
+    "In this section, we will build a network to classify Reuters newswires into 46 different mutually-exclusive topics. Since we have many classes, this problem is an instance of \"multi-class classification\", and since each data point should be classified into only one category, the problem is more specifically an instance of \"single-label, multi-class classification\".\n",
+    "\n",
+    "We will be working with the Reuters dataset, a set of short newswires and their topics, published by Reuters in 1986. It's a very simple, widely used toy dataset for text classification. There are 46 different topics; some topics are more represented than others, but each topic has at least 10 examples in the training set.\n",
+    "\n",
+    "Like IMDB and MNIST, the Reuters dataset comes packaged as part of Keras. Let's take a look right away:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import reuters\n",
+    "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(nb_words=10000)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then we do some preprocessing like Chapter 3.5"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "word_index = reuters.get_word_index()\n",
+    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
+    "\n",
+    "import numpy as np\n",
+    "def vectorize_sequences(sequences, dimension=10000):\n",
+    "    results = np.zeros((len(sequences), dimension))\n",
+    "    for i, sequence in enumerate(sequences):\n",
+    "        results[i, sequence] = 1.\n",
+    "    return results\n",
+    "\n",
+    "x_train = vectorize_sequences(train_data)\n",
+    "x_test = vectorize_sequences(test_data)\n",
+    "# this part pending to modify, one-hot or integer issue\n",
+    "\n",
+    "x_val = x_train[:1000]\n",
+    "partial_x_train = x_train[1000:]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Build the model, compile, train and evaluate"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = models.Sequential()\n",
+    "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
+    "model.add(layers.Dense(64, activation='relu'))\n",
+    "model.add(layers.Dense(46, activation='softmax'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='categorical_crossentropy',\n",
+    "              metrics=['accuracy'])\n",
+    "model.set_tensorboard('./', '3-6_summary')\n",
+    "model.fit(partial_x_train,\n",
+    "          partial_y_train,\n",
+    "          nb_epoch=20,\n",
+    "          batch_size=512,\n",
+    "          validation_data=(x_val, y_val))\n",
+    "\n",
+    "results = model.evaluate(x_test, one_hot_test_labels)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/keras/3.7-regression.ipynb b/keras/3.7-regression.ipynb
new file mode 100644
index 0000000..b2547a2
--- /dev/null
+++ b/keras/3.7-regression.ipynb
@@ -0,0 +1,174 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This dataset is packaged in Keras 2.0.8 but not in Keras 1.2.2, so that we need to use following code to get the data, then we also apply normalization on these data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.utils.data_utils import get_file\n",
+    "def load_data(path='boston_housing.npz', test_split=0.2, seed=113):\n",
+    "    \"\"\"Loads the Boston Housing dataset.\n",
+    "    # Arguments\n",
+    "        path: path where to cache the dataset locally\n",
+    "            (relative to ~/.zoo.pipeline.api.keras/datasets).\n",
+    "        test_split: fraction of the data to reserve as test set.\n",
+    "        seed: Random seed for shuffling the data\n",
+    "            before computing the test split.\n",
+    "    # Returns\n",
+    "        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.\n",
+    "    \"\"\"\n",
+    "    assert 0 <= test_split < 1\n",
+    "    path = get_file(\n",
+    "        path,\n",
+    "        origin='https://s3.amazonaws.com/zoo.pipeline.api.keras-datasets/boston_housing.npz'\n",
+    "        )\n",
+    "    with np.load(path) as f:\n",
+    "        x = f['x']\n",
+    "        y = f['y']\n",
+    "\n",
+    "    np.random.seed(seed)\n",
+    "    indices = np.arange(len(x))\n",
+    "    np.random.shuffle(indices)\n",
+    "    x = x[indices]\n",
+    "    y = y[indices]\n",
+    "\n",
+    "    x_train = np.array(x[:int(len(x) * (1 - test_split))])\n",
+    "    y_train = np.array(y[:int(len(x) * (1 - test_split))])\n",
+    "    x_test = np.array(x[int(len(x) * (1 - test_split)):])\n",
+    "    y_test = np.array(y[int(len(x) * (1 - test_split)):])\n",
+    "    return (x_train, y_train), (x_test, y_test)\n",
+    "\n",
+    "(train_data, train_targets), (test_data, test_targets) = load_data()\n",
+    "\n",
+    "mean = train_data.mean(axis=0)\n",
+    "train_data -= mean\n",
+    "std = train_data.std(axis=0)\n",
+    "train_data /= std\n",
+    "\n",
+    "test_data -= mean\n",
+    "test_data /= std"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "k = 4\n",
+    "num_val_samples = len(train_data) // k\n",
+    "num_nb_epoch = 50\n",
+    "all_scores = []\n",
+    "for i in range(k):\n",
+    "    print('processing fold #', i)\n",
+    "    # Prepare the validation data: data from partition # k\n",
+    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "\n",
+    "    # Prepare the training data: data from all other partitions\n",
+    "    partial_train_data = np.concatenate(\n",
+    "        [train_data[:i * num_val_samples],\n",
+    "         train_data[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "    partial_train_targets = np.concatenate(\n",
+    "        [train_targets[:i * num_val_samples],\n",
+    "         train_targets[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "\n",
+    "    # Build the Keras model (already compiled)\n",
+    "    model = build_model()\n",
+    "    # Train the model (in silent mode, verbose=0)\n",
+    "    #model.fit(partial_train_data, partial_train_targets,\n",
+    "    #          nb_epoch=num_nb_epoch, batch_size=1, verbose=0)\n",
+    "    model.fit(partial_train_data, partial_train_targets,\n",
+    "              nb_epoch=num_nb_epoch, batch_size=16)\n",
+    "\n",
+    "    # Evaluate the model on the validation data\n",
+    "    #val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
+    "    val_mae = model.evaluate(val_data, val_targets)\n",
+    "    all_scores.append(val_mae[0].result)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = build_model()\n",
+    "# Train it on the entirety of the data.\n",
+    "model.fit(train_data, train_targets,\n",
+    "          nb_epoch=80, batch_size=16)\n",
+    "test_result = model.evaluate(test_data, test_targets)\n",
+    "\n",
+    "print('test result:', test_result[0].result)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/keras/4.4-regularization-and-dropout.ipynb b/keras/4.4-regularization-and-dropout.ipynb
new file mode 100644
index 0000000..3c926d5
--- /dev/null
+++ b/keras/4.4-regularization-and-dropout.ipynb
@@ -0,0 +1,53 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/keras/5.1-cnn.ipynb b/keras/5.1-cnn.ipynb
new file mode 100644
index 0000000..d36c4aa
--- /dev/null
+++ b/keras/5.1-cnn.ipynb
@@ -0,0 +1,126 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First, let's take a practical look at a very simple convnet example. We will use our convnet to classify MNIST digits, a task that you've already been through in Chapter 2.\n",
+    "\n",
+    "First load the MNIST dataset:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import mnist\n",
+    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### CNN input shape\n",
+    "Once we get the dataset, we need to reshape the images. In keras the shape of the dataset is `(sample_size, height, width, channel)`, like the Keras code below:\n",
+    "    \n",
+    "    train_images = train_images.reshape((60000, 28, 28, 1))\n",
+    "In analytics-zoo, you need to use a little different order, `(sample_size, channel, height, width)`, like following:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "train_images = train_images.reshape((60000, 1, 28, 28))\n",
+    "train_images = train_images.astype('float32') / 255\n",
+    "\n",
+    "test_images = test_images.reshape((10000, 1, 28, 28))\n",
+    "test_images = test_images.astype('float32') / 255\n",
+    "\n",
+    "from keras.utils.np_utils import to_categorical\n",
+    "train_labels = to_categorical(train_labels)\n",
+    "test_labels = to_categorical(test_labels)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then build the model, compile, train and evaluate:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import layers\n",
+    "from zoo.pipeline.api.keras import models\n",
+    "\n",
+    "model = models.Sequential()\n",
+    "model.add(layers.Conv2D(32, nb_col=3, nb_row=3, activation='relu', input_shape=(1,28,28)))\n",
+    "model.add(layers.MaxPooling2D((2, 2)))\n",
+    "model.add(layers.Conv2D(64, nb_col=3, nb_row=3, activation='relu'))\n",
+    "model.add(layers.MaxPooling2D((2, 2)))\n",
+    "model.add(layers.Conv2D(64, nb_col=3, nb_row=3, activation='relu'))\n",
+    "\n",
+    "model.summary()\n",
+    "\n",
+    "model.add(layers.Flatten())\n",
+    "model.add(layers.Dense(64, activation='relu'))\n",
+    "model.add(layers.Dense(10, activation='softmax'))\n",
+    "\n",
+    "test_result = model.evaluate(test_images, test_labels)\n",
+    "print('test_acc:', test_result[0].result)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/keras/6.2-rnn.ipynb b/keras/6.2-rnn.ipynb
new file mode 100644
index 0000000..16817b9
--- /dev/null
+++ b/keras/6.2-rnn.ipynb
@@ -0,0 +1,171 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# RNN\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.models import Sequential\n",
+    "from zoo.pipeline.api.keras.layers import Embedding, SimpleRNN"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Specify input shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = Sequential()\n",
+    "model.add(Embedding(10000, 32, input_shape=(500,)))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32))  # This last layer only returns the last outputs.\n",
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import imdb\n",
+    "from keras.preprocessing import sequence\n",
+    "\n",
+    "max_features = 10000  # number of words to consider as features\n",
+    "maxlen = 500  # cut texts after this number of words (among top max_features most common words)\n",
+    "batch_size = 32\n",
+    "\n",
+    "(input_train, y_train), (input_test, y_test) = imdb.load_data(nb_words=max_features)\n",
+    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
+    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
+    "print('input_train shape:', input_train.shape)\n",
+    "print('input_test shape:', input_test.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.layers import Dense\n",
+    "\n",
+    "model = Sequential()\n",
+    "model.add(Embedding(max_features, 32, input_shape=(500,)))\n",
+    "model.add(SimpleRNN(32))\n",
+    "model.add(Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
+    "history = model.fit(input_train, y_train,\n",
+    "                    nb_epoch=10,\n",
+    "                    batch_size=128,\n",
+    "                    #validation_split=0.2\n",
+    "                    )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.layers import LSTM\n",
+    "\n",
+    "model = Sequential()\n",
+    "model.add(Embedding(max_features, 32, input_shape=(500,)))\n",
+    "model.add(LSTM(32))\n",
+    "model.add(Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='binary_crossentropy',\n",
+    "              metrics=['acc'])\n",
+    "model.set_tensorboard('./', '6-2_summary')\n",
+    "history = model.fit(input_train, y_train,\n",
+    "                    nb_epoch=10,\n",
+    "                    batch_size=128,\n",
+    "                    #validation_split=0.2\n",
+    "                    )"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From e4549ba85b51ee31ebfb934061425ff536da7abf Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Feb 2019 09:55:16 +0800
Subject: [PATCH 03/46] Update README.md

---
 keras/README.md | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/keras/README.md b/keras/README.md
index 67cc57a..b6a6da1 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -1,11 +1,9 @@
 # Tutorials for running distribued Keras (v1.2.2) on Analytics Zoo
 Tutorials for running _**distribued Keras (v1.2.2) on Analytics Zoo**_. These tutorials are ported from François Chollet's [Jupyter notebooks](https://github.com/fchollet/deep-learning-with-python-notebooks) for the book [Deep Learning with Python (Manning Publications)](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff)
 
-This repository is built to describe how to write Keras-style code and directly run it in analytics-zoo so that we could run the original Keras code in distributed mode via limited modification. We make this repository based on notebook https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python, which is the sample code repository of book "Deep Learning with Python" based on Keras 2.0.8 and we pick some basic chapters from it. 
+This repository is built to describe how to write Keras-style code and directly run it in analytics-zoo so that we could run the original Keras code in distributed mode via limited modification. The original notebook is based on Keras 2.0.8. Currently analytics-zoo builds its Keras-style code based on Keras 1.2.2. Thus, this repository contains the code of implementation of original Keras 2.0.8 code based on Keras 1.2.2, and how to modify the code in order to run it in analytics-zoo.
 
-Currently analytics-zoo builds its Keras-style code based on Keras 1.2.2. Thus, this repository contains the code of implementation of original Keras 2.0.8 code based on Keras 1.2.2, and how to modify the code in order to run it in analytics-zoo.
-
-To make it simple, we omit the description of plenty concepts here and you could find them in original notebook above [(link)](https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python). Besides, we directly post Keras 1.2.2 code here and the replacements needed from Keras 2.0.8 to Keras 1.2.2 are noted in `Keras_2-to-1.md`.
+To make it simple, we omit the description of plenty concepts here and you could find them in original notebook above [(link)](https://github.com/fchollet/deep-learning-with-python-notebooks). Besides, we directly post Keras 1.2.2 code here and the replacements needed from Keras 2.0.8 to Keras 1.2.2 are noted in `Keras_2-to-1.md`.
 
 This repository use Python 3.5, Keras 1.2.2 (Keras code), Analytics-zoo 0.4.0 (zoo code). We post the summary of Keras-to-zoo code convertion as well as the table of contents.
 

From f229f22f1f4b5f0c8075001e65ef279f796031f8 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Feb 2019 15:56:10 +0800
Subject: [PATCH 04/46] Update README.md

---
 keras/README.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/keras/README.md b/keras/README.md
index b6a6da1..0ca5e8a 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -38,10 +38,10 @@ The return of `evaluate` method is an `EvaluationResult` object, which is differ
 The return of `predict` method is RDD, so you need to call `collect` method to collect them, see [Chapter 3.5]()
 
 #### Parameters not supported
-Analytics-zoo does not support following parameters currently
+Analytics-zoo does not support following parameters currently, so we just comment these code in this notebook.
 
-* validation_split:
-* verbose:
+* validation_split: To split the training data into two parts, another part is used for validation
+* verbose: To control the information showed during training
 
 ## Table of contents
 

From 29897ec0374bd3073bf40df630f168faaa78c695 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Feb 2019 15:58:00 +0800
Subject: [PATCH 05/46] Update README.md

---
 keras/README.md | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/keras/README.md b/keras/README.md
index 0ca5e8a..a70481b 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -48,16 +48,16 @@ Analytics-zoo does not support following parameters currently, so we just commen
 The main purpose of this repository is to decribe the convertion from Keras to analytics-zoo so that we rename the notebooks to make it more experienced-user oriented. We keep the chapter index so you could still make a reference to the original notebook [here](https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python). We only keep some key description of the original notebook.
 
 * Chapter 2:
-    * [2.1: MNIST]()
+    * [2.1: MNIST](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/2.1-mnist.ipynb)
 * Chapter 3:
-    * [3.5: Binary classification]()
-    * [3.6: Multi-class classification]()
-    * [3.7: Regression]()
+    * [3.5: Binary classification](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/3.5-binary-classification.ipynb)
+    * [3.6: Multi-class classification](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/3.6-multi-class-classification.ipynb)
+    * [3.7: Regression](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/3.7-regression.ipynb)
 * Chapter 4:
-    * [4.4: Regularization and Dropout]()
+    * [4.4: Regularization and Dropout](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/4.4-regularization-and-dropout.ipynb)
 * Chapter 5:
-    * [5.1: CNN]()
+    * [5.1: CNN](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/5.1-cnn.ipynb)
 * Chapter 6:
-    * [6.2: RNN]()
+    * [6.2: RNN](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/6.2-rnn.ipynb)
 
     

From 6fad60123452328cc61b02cc147690adb8735c77 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Feb 2019 15:59:19 +0800
Subject: [PATCH 06/46] Add files via upload

---
 keras/3.7-regression.ipynb                 | 36 +++++----
 keras/4.4-regularization-and-dropout.ipynb | 92 +++++++++++++++++++++-
 keras/5.1-cnn.ipynb                        |  5 +-
 keras/6.2-rnn.ipynb                        | 31 +++++---
 4 files changed, 134 insertions(+), 30 deletions(-)

diff --git a/keras/3.7-regression.ipynb b/keras/3.7-regression.ipynb
index b2547a2..98c69d9 100644
--- a/keras/3.7-regression.ipynb
+++ b/keras/3.7-regression.ipynb
@@ -21,6 +21,16 @@
     "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Regression\n",
+    "We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
+    "\n",
+    "The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, others between 0 and 100..."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -79,11 +89,13 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "In this example we have so few data points, the validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot depending on which data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a high variance with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
+    "\n",
+    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions (typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining partition. The validation score for the model used would then be the average of the K validation scores obtained."
+   ]
   },
   {
    "cell_type": "code",
@@ -128,11 +140,11 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "Then we could check our K-fold training result:"
+   ]
   },
   {
    "cell_type": "code",
@@ -140,13 +152,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "model = build_model()\n",
-    "# Train it on the entirety of the data.\n",
-    "model.fit(train_data, train_targets,\n",
-    "          nb_epoch=80, batch_size=16)\n",
-    "test_result = model.evaluate(test_data, test_targets)\n",
-    "\n",
-    "print('test result:', test_result[0].result)"
+    "all_scores"
    ]
   }
  ],
diff --git a/keras/4.4-regularization-and-dropout.ipynb b/keras/4.4-regularization-and-dropout.ipynb
index 3c926d5..c79d8e1 100644
--- a/keras/4.4-regularization-and-dropout.ipynb
+++ b/keras/4.4-regularization-and-dropout.ipynb
@@ -21,12 +21,102 @@
     "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Regularization and dropout\n",
+    "Let's review some of the most common techniques to prevent overfitting, and let's apply them in practice to improve our movie classification model from the previous chapter. First let's prepare the data using the code from Chapter 3.5 and preprocess the data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import imdb\n",
+    "import numpy as np\n",
+    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)\n",
+    "\n",
+    "def vectorize_sequences(sequences, dimension=10000):\n",
+    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
+    "    results = np.zeros((len(sequences), dimension))\n",
+    "    for i, sequence in enumerate(sequences):\n",
+    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
+    "    return results\n",
+    "\n",
+    "x_train = vectorize_sequences(train_data)\n",
+    "x_test = vectorize_sequences(test_data)\n",
+    "\n",
+    "y_train = np.asarray(train_labels).astype('float32')\n",
+    "y_test = np.asarray(test_labels).astype('float32')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's add L2 weight regularization to our movie review classification network:"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "from zoo.pipeline.api.keras import regularizers\n",
+    "\n",
+    "l2_model = models.Sequential()\n",
+    "l2_model.add(layers.Dense(16, W_regularizer=regularizers.l2(0.001),\n",
+    "                          activation='relu', input_shape=(10000,)))\n",
+    "l2_model.add(layers.Dense(16, W_regularizer=regularizers.l2(0.001),\n",
+    "                          activation='relu'))\n",
+    "l2_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "l2_model.compile(optimizer='rmsprop',\n",
+    "                 loss='binary_crossentropy',\n",
+    "                 metrics=['acc'])\n",
+    "\n",
+    "l2_model_hist = l2_model.fit(x_train, y_train,\n",
+    "                             nb_epoch=20,\n",
+    "                             batch_size=512,\n",
+    "                             validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's add two Dropout layers in our IMDB network to see how well they do at reducing overfitting:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dpt_model = models.Sequential()\n",
+    "dpt_model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "dpt_model.add(layers.Dropout(0.5))\n",
+    "dpt_model.add(layers.Dense(16, activation='relu'))\n",
+    "dpt_model.add(layers.Dropout(0.5))\n",
+    "dpt_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "dpt_model.compile(optimizer='rmsprop',\n",
+    "                  loss='binary_crossentropy',\n",
+    "                  metrics=['acc'])\n",
+    "\n",
+    "dpt_model_hist = dpt_model.fit(x_train, y_train,\n",
+    "                               nb_epoch=20,\n",
+    "                               batch_size=512,\n",
+    "                               validation_data=(x_test, y_test))"
+   ]
   }
  ],
  "metadata": {
diff --git a/keras/5.1-cnn.ipynb b/keras/5.1-cnn.ipynb
index d36c4aa..161499e 100644
--- a/keras/5.1-cnn.ipynb
+++ b/keras/5.1-cnn.ipynb
@@ -25,6 +25,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "# CNN\n",
     "First, let's take a practical look at a very simple convnet example. We will use our convnet to classify MNIST digits, a task that you've already been through in Chapter 2.\n",
     "\n",
     "First load the MNIST dataset:"
@@ -48,7 +49,9 @@
     "Once we get the dataset, we need to reshape the images. In keras the shape of the dataset is `(sample_size, height, width, channel)`, like the Keras code below:\n",
     "    \n",
     "    train_images = train_images.reshape((60000, 28, 28, 1))\n",
-    "In analytics-zoo, you need to use a little different order, `(sample_size, channel, height, width)`, like following:"
+    "In analytics-zoo, the default order is theano-style NCHW `(sample_size, channel, height, width)`, so you can process data like following:\n",
+    "\n",
+    "Alternatively, you can also use tensorflow-style NHWC as Keras default just by setting `Convolution2D(dim_ordering=\"tf\")`"
    ]
   },
   {
diff --git a/keras/6.2-rnn.ipynb b/keras/6.2-rnn.ipynb
index 16817b9..5c68e6d 100644
--- a/keras/6.2-rnn.ipynb
+++ b/keras/6.2-rnn.ipynb
@@ -42,7 +42,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Specify input shape"
+    "#### Specify input shape\n",
+    "We could add an embedding layer as our first layer in Keras as following:\n",
+    "    \n",
+    "    model = Sequential()\n",
+    "    model.add(Embedding(10000, 32))\n",
+    "In analytics-zoo, you need to specify the input shape of first layer, in this example, the sequence length is 500, so we could build our model as following:"
    ]
   },
   {
@@ -61,11 +66,11 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "Now let's try to use such a model on the IMDB movie review classification problem. First, let's preprocess the data:"
+   ]
   },
   {
    "cell_type": "code",
@@ -88,11 +93,11 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "Let's train a simple recurrent network using an `Embedding` layer and a `SimpleRNN` layer:"
+   ]
   },
   {
    "cell_type": "code",
@@ -116,11 +121,11 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "Now let's switch to more practical concerns: we will set up a model using a LSTM layer and train it on the IMDB data."
+   ]
   },
   {
    "cell_type": "code",

From cdcf30b5a97eb0ec76e57ad738a8d0a117047d28 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 15:01:23 +0800
Subject: [PATCH 07/46] Update README.md

---
 keras/README.md | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/keras/README.md b/keras/README.md
index a70481b..99282e6 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -5,6 +5,8 @@ This repository is built to describe how to write Keras-style code and directly
 
 To make it simple, we omit the description of plenty concepts here and you could find them in original notebook above [(link)](https://github.com/fchollet/deep-learning-with-python-notebooks). Besides, we directly post Keras 1.2.2 code here and the replacements needed from Keras 2.0.8 to Keras 1.2.2 are noted in `Keras_2-to-1.md`.
 
+The training log of each epoch is stored in INFO, you could see these information though configuring python output level, or store these information in a file and check afterwards, these may includes plenty lines of information. For better expression here, we run the code in Pycharm IDE and **we only paste the last info of last epoch of each training function in this notebook**.
+
 This repository use Python 3.5, Keras 1.2.2 (Keras code), Analytics-zoo 0.4.0 (zoo code). We post the summary of Keras-to-zoo code convertion as well as the table of contents.
 
 ## Summary of Keras-to-zoo code convertion
@@ -37,6 +39,12 @@ The return of `evaluate` method is an `EvaluationResult` object, which is differ
 #### Predict result
 The return of `predict` method is RDD, so you need to call `collect` method to collect them, see [Chapter 3.5]()
 
+#### CNN input shape
+In analytics-zoo, the default order is theano-style NCHW, you can also use tensorflow-style NHWC as Keras default just by setting `Convolution2D(dim_ordering="tf")`, see [Chapter 5.1]()
+
+#### Specify input shape
+In analytics-zoo, you need to specify the input shape of first layer, see [Chapter 6.2]()
+
 #### Parameters not supported
 Analytics-zoo does not support following parameters currently, so we just comment these code in this notebook.
 

From a0b235b52207b3f118e9a5b4d2c69bb733ef5d9e Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 15:03:11 +0800
Subject: [PATCH 08/46] Update README.md

---
 keras/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/README.md b/keras/README.md
index 99282e6..73d00d8 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -46,7 +46,7 @@ In analytics-zoo, the default order is theano-style NCHW, you can also use tenso
 In analytics-zoo, you need to specify the input shape of first layer, see [Chapter 6.2]()
 
 #### Parameters not supported
-Analytics-zoo does not support following parameters currently, so we just comment these code in this notebook.
+Analytics-zoo does not support following parameters currently, so we do not use these code in this notebook if they exist in original notebook.
 
 * validation_split: To split the training data into two parts, another part is used for validation
 * verbose: To control the information showed during training

From 6bfa0ab7359b854621308e0e48115c16d230497a Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 15:04:12 +0800
Subject: [PATCH 09/46] Add files via upload

---
 keras/2.1-mnist.ipynb                      | 21 +++--
 keras/3.5-binary-classification.ipynb      | 14 +++-
 keras/3.6-multi-class-classification.ipynb | 39 +++++++--
 keras/3.7-regression.ipynb                 | 56 ++++++++++++-
 keras/4.4-regularization-and-dropout.ipynb | 70 +++++++++++++---
 keras/5.1-cnn.ipynb                        | 33 ++++++--
 keras/6.2-rnn.ipynb                        | 92 +++++++++++++---------
 7 files changed, 258 insertions(+), 67 deletions(-)

diff --git a/keras/2.1-mnist.ipynb b/keras/2.1-mnist.ipynb
index d971f5d..cbf64b7 100644
--- a/keras/2.1-mnist.ipynb
+++ b/keras/2.1-mnist.ipynb
@@ -81,7 +81,7 @@
     "network.add(layers.Dense(10, activation='softmax'))\n",
     "\n",
     "network.compile(optimizer='rmsprop',\n",
-    "                loss='categorical_crossentropy',\n",
+    "                loss='sparse_categorical_crossentropy',\n",
     "                metrics=['accuracy'])\n",
     "\n",
     "train_images = train_images.reshape((60000, 28 * 28))\n",
@@ -90,14 +90,16 @@
     "test_images = test_images.reshape((10000, 28 * 28))\n",
     "test_images = test_images.astype('float32') / 255\n",
     "\n",
-    "\n",
-    "from keras.utils.np_utils import to_categorical\n",
-    "train_labels = to_categorical(train_labels)\n",
-    "#test_labels = to_categorical(test_labels)\n",
-    "\n",
     "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -118,6 +120,13 @@
     "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
     "print('test_acc:', test_result[0].result)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.9783999919891357"
+   ]
   }
  ],
  "metadata": {
diff --git a/keras/3.5-binary-classification.ipynb b/keras/3.5-binary-classification.ipynb
index 929a731..79a17cc 100644
--- a/keras/3.5-binary-classification.ipynb
+++ b/keras/3.5-binary-classification.ipynb
@@ -102,7 +102,9 @@
     "                    )\n",
     "After `fit` method finishes, the results are stored in `history` and thus could be visualized.\n",
     "\n",
-    "Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via tensorboard. Code above need to be replaced with following:"
+    "Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via tensorboard. Code above need to be replaced with following:\n",
+    "\n",
+    "Then you could see result in tensorboard by command in terminal `tensorboard --logdir ./` starting tensorboard service and go to `localhost:port_number` as shown in your terminal."
    ]
   },
   {
@@ -124,7 +126,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Then you could see result in tensorboard by command in terminal `tensorboard --logdir ./`"
+    "Trained 512 records in 0.022409147 seconds. Throughput is 22847.812 records/second. Loss is 0.21351601.\n",
+    "Top1Accuracy is Accuracy(correct: 8606, count: 10000, accuracy: 0.8606)"
    ]
   },
   {
@@ -144,6 +147,13 @@
     "print('test_acc:', results[0].result)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.8624799847602844"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
diff --git a/keras/3.6-multi-class-classification.ipynb b/keras/3.6-multi-class-classification.ipynb
index 7e8c9ff..0655565 100644
--- a/keras/3.6-multi-class-classification.ipynb
+++ b/keras/3.6-multi-class-classification.ipynb
@@ -71,7 +71,11 @@
     "# this part pending to modify, one-hot or integer issue\n",
     "\n",
     "x_val = x_train[:1000]\n",
-    "partial_x_train = x_train[1000:]"
+    "partial_x_train = x_train[1000:]\n",
+    "\n",
+    "y_val = train_labels[:1000]\n",
+    "partial_y_train = train_labels[1000:]    # this line would return list\n",
+    "partial_y_train = np.array(partial_y_train)    # convert list to ndarray"
    ]
   },
   {
@@ -93,16 +97,39 @@
     "model.add(layers.Dense(46, activation='softmax'))\n",
     "\n",
     "model.compile(optimizer='rmsprop',\n",
-    "              loss='categorical_crossentropy',\n",
+    "              loss='sparse_categorical_crossentropy',\n",
     "              metrics=['accuracy'])\n",
-    "model.set_tensorboard('./', '3-6_summary')\n",
+    "\n",
     "model.fit(partial_x_train,\n",
     "          partial_y_train,\n",
     "          nb_epoch=20,\n",
     "          batch_size=512,\n",
-    "          validation_data=(x_val, y_val))\n",
-    "\n",
-    "results = model.evaluate(x_test, one_hot_test_labels)"
+    "          validation_data=(x_val, y_val))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 512 records in 0.03322949 seconds. Throughput is 15408.001 records/second. Loss is 0.36856997.\n",
+    "Top1Accuracy is Accuracy(correct: 808, count: 1000, accuracy: 0.808)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_results = model.evaluate(x_test, test_labels)\n",
+    "print (test_results[0].result)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.7885128855705261"
    ]
   }
  ],
diff --git a/keras/3.7-regression.ipynb b/keras/3.7-regression.ipynb
index 98c69d9..769e343 100644
--- a/keras/3.7-regression.ipynb
+++ b/keras/3.7-regression.ipynb
@@ -94,7 +94,38 @@
    "source": [
     "In this example we have so few data points, the validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot depending on which data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a high variance with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
     "\n",
-    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions (typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining partition. The validation score for the model used would then be the average of the K validation scores obtained."
+    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions (typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining partition. The validation score for the model used would then be the average of the K validation scores obtained.\n",
+    "\n",
+    "Since we are using K-fold so that we have to build the model multiple times, we use following function to build our model:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "def build_model():\n",
+    "    # Because we will need to instantiate\n",
+    "    # the same model multiple times,\n",
+    "    # we use a function to construct it.\n",
+    "    model = models.Sequential()\n",
+    "    model.add(layers.Dense(64, activation='relu',\n",
+    "                           input_shape=(train_data.shape[1],)))\n",
+    "    model.add(layers.Dense(64, activation='relu'))\n",
+    "    model.add(layers.Dense(1))\n",
+    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
+    "    return model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then let's start our training:"
    ]
   },
   {
@@ -139,6 +170,20 @@
     "    all_scores.append(val_mae[0].result)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "processing fold # 0\n",
+    "Trained 16 records in 0.011235845 seconds. Throughput is 1424.0139 records/second. Loss is 8.708786.\n",
+    "processing fold # 1\n",
+    "Trained 16 records in 0.009535034 seconds. Throughput is 1678.0223 records/second. Loss is 5.3613434.\n",
+    "processing fold # 2\n",
+    "Trained 16 records in 0.008636178 seconds. Throughput is 1852.6713 records/second. Loss is 18.106756.\n",
+    "processing fold # 3\n",
+    "Trained 16 records in 0.009207628 seconds. Throughput is 1737.6897 records/second. Loss is 7.0931993."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -152,7 +197,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "all_scores"
+    "print (all_scores)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "[20.572654724121094, 19.606250762939453, 21.224998474121094, 22.60078239440918]"
    ]
   }
  ],
diff --git a/keras/4.4-regularization-and-dropout.ipynb b/keras/4.4-regularization-and-dropout.ipynb
index c79d8e1..5ed40ab 100644
--- a/keras/4.4-regularization-and-dropout.ipynb
+++ b/keras/4.4-regularization-and-dropout.ipynb
@@ -57,7 +57,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let's add L2 weight regularization to our movie review classification network:"
+    "Let's first try an easy model and try to optimize it afterwards:"
    ]
   },
   {
@@ -69,6 +69,42 @@
     "from zoo.pipeline.api.keras import models\n",
     "from zoo.pipeline.api.keras import layers\n",
     "\n",
+    "original_model = models.Sequential()\n",
+    "original_model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "original_model.add(layers.Dense(16, activation='relu'))\n",
+    "original_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "original_model.compile(optimizer='rmsprop',\n",
+    "                       loss='binary_crossentropy',\n",
+    "                       metrics=['acc'])\n",
+    "\n",
+    "original_model.fit(x_train, y_train,\n",
+    "                   epochs=20,\n",
+    "                   batch_size=512,\n",
+    "                   validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 512 records in 0.024455326 seconds. Throughput is 20936.135 records/second. Loss is 0.01585226.\n",
+    "Top1Accuracy is Accuracy(correct: 21341, count: 25000, accuracy: 0.85364)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's add L2 weight regularization to our movie review classification network:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "from zoo.pipeline.api.keras import regularizers\n",
     "\n",
     "l2_model = models.Sequential()\n",
@@ -82,10 +118,18 @@
     "                 loss='binary_crossentropy',\n",
     "                 metrics=['acc'])\n",
     "\n",
-    "l2_model_hist = l2_model.fit(x_train, y_train,\n",
-    "                             nb_epoch=20,\n",
-    "                             batch_size=512,\n",
-    "                             validation_data=(x_test, y_test))"
+    "l2_model.fit(x_train, y_train,\n",
+    "             nb_epoch=20,\n",
+    "             batch_size=512,\n",
+    "             validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 512 records in 0.024366594 seconds. Throughput is 21012.373 records/second. Loss is 0.13651785.\n",
+    "Top1Accuracy is Accuracy(correct: 21684, count: 25000, accuracy: 0.86736)"
    ]
   },
   {
@@ -112,10 +156,18 @@
     "                  loss='binary_crossentropy',\n",
     "                  metrics=['acc'])\n",
     "\n",
-    "dpt_model_hist = dpt_model.fit(x_train, y_train,\n",
-    "                               nb_epoch=20,\n",
-    "                               batch_size=512,\n",
-    "                               validation_data=(x_test, y_test))"
+    "dpt_model.fit(x_train, y_train,\n",
+    "              nb_epoch=20,\n",
+    "              batch_size=512,\n",
+    "              validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 512 records in 0.017992654 seconds. Throughput is 28456.057 records/second. Loss is 0.112769656. \n",
+    "Top1Accuracy is Accuracy(correct: 21871, count: 25000, accuracy: 0.87484)"
    ]
   }
  ],
diff --git a/keras/5.1-cnn.ipynb b/keras/5.1-cnn.ipynb
index 161499e..72786d2 100644
--- a/keras/5.1-cnn.ipynb
+++ b/keras/5.1-cnn.ipynb
@@ -64,11 +64,7 @@
     "train_images = train_images.astype('float32') / 255\n",
     "\n",
     "test_images = test_images.reshape((10000, 1, 28, 28))\n",
-    "test_images = test_images.astype('float32') / 255\n",
-    "\n",
-    "from keras.utils.np_utils import to_categorical\n",
-    "train_labels = to_categorical(train_labels)\n",
-    "test_labels = to_categorical(test_labels)"
+    "test_images = test_images.astype('float32') / 255"
    ]
   },
   {
@@ -100,9 +96,36 @@
     "model.add(layers.Dense(64, activation='relu'))\n",
     "model.add(layers.Dense(10, activation='softmax'))\n",
     "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='sparse_categorical_crossentropy',\n",
+    "              metrics=['acc'])\n",
+    "\n",
+    "model.fit(train_images, train_labels, nb_epoch=5, batch_size=64)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 64 records in 0.03212866 seconds. Throughput is 1991.9911 records/second. Loss is 0.0023578003."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "test_result = model.evaluate(test_images, test_labels)\n",
     "print('test_acc:', test_result[0].result)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.9915000200271606"
+   ]
   }
  ],
  "metadata": {
diff --git a/keras/6.2-rnn.ipynb b/keras/6.2-rnn.ipynb
index 5c68e6d..ef1146f 100644
--- a/keras/6.2-rnn.ipynb
+++ b/keras/6.2-rnn.ipynb
@@ -42,12 +42,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Specify input shape\n",
-    "We could add an embedding layer as our first layer in Keras as following:\n",
-    "    \n",
-    "    model = Sequential()\n",
-    "    model.add(Embedding(10000, 32))\n",
-    "In analytics-zoo, you need to specify the input shape of first layer, in this example, the sequence length is 500, so we could build our model as following:"
+    "Now let's try to use such a model on the IMDB movie review classification problem. First, let's preprocess the data:"
    ]
   },
   {
@@ -56,20 +51,38 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "model = Sequential()\n",
-    "model.add(Embedding(10000, 32, input_shape=(500,)))\n",
-    "model.add(SimpleRNN(32, return_sequences=True))\n",
-    "model.add(SimpleRNN(32, return_sequences=True))\n",
-    "model.add(SimpleRNN(32, return_sequences=True))\n",
-    "model.add(SimpleRNN(32))  # This last layer only returns the last outputs.\n",
-    "model.summary()"
+    "from keras.datasets import imdb\n",
+    "from keras.preprocessing import sequence\n",
+    "\n",
+    "max_features = 10000  # number of words to consider as features\n",
+    "maxlen = 500  # cut texts after this number of words (among top max_features most common words)\n",
+    "batch_size = 32\n",
+    "\n",
+    "(input_train, y_train), (input_test, y_test) = imdb.load_data(nb_words=max_features)\n",
+    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
+    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
+    "print('input_train shape:', input_train.shape)\n",
+    "print('input_test shape:', input_test.shape)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now let's try to use such a model on the IMDB movie review classification problem. First, let's preprocess the data:"
+    "input_train shape: (25000, 500)\n",
+    "input_test shape: (25000, 500)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Specify input shape\n",
+    "We could add an embedding layer as our first layer in Keras as following:\n",
+    "    \n",
+    "    model = Sequential()\n",
+    "    model.add(Embedding(10000, 32))\n",
+    "In analytics-zoo, you need to specify the input shape of first layer, in this example, the sequence length is 500, so we could build our model as following:"
    ]
   },
   {
@@ -78,18 +91,13 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from keras.datasets import imdb\n",
-    "from keras.preprocessing import sequence\n",
-    "\n",
-    "max_features = 10000  # number of words to consider as features\n",
-    "maxlen = 500  # cut texts after this number of words (among top max_features most common words)\n",
-    "batch_size = 32\n",
-    "\n",
-    "(input_train, y_train), (input_test, y_test) = imdb.load_data(nb_words=max_features)\n",
-    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
-    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
-    "print('input_train shape:', input_train.shape)\n",
-    "print('input_test shape:', input_test.shape)"
+    "model = Sequential()\n",
+    "model.add(Embedding(10000, 32, input_shape=(500,)))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32))  # This last layer only returns the last outputs.\n",
+    "model.summary()"
    ]
   },
   {
@@ -113,11 +121,16 @@
     "model.add(Dense(1, activation='sigmoid'))\n",
     "\n",
     "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
-    "history = model.fit(input_train, y_train,\n",
-    "                    nb_epoch=10,\n",
-    "                    batch_size=128,\n",
-    "                    #validation_split=0.2\n",
-    "                    )"
+    "model.fit(input_train, y_train,\n",
+    "          nb_epoch=10,\n",
+    "          batch_size=128)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 128 records in 0.046239497 seconds. Throughput is 2768.1963 records/second. Loss is 0.16970885."
    ]
   },
   {
@@ -143,12 +156,17 @@
     "model.compile(optimizer='rmsprop',\n",
     "              loss='binary_crossentropy',\n",
     "              metrics=['acc'])\n",
-    "model.set_tensorboard('./', '6-2_summary')\n",
-    "history = model.fit(input_train, y_train,\n",
-    "                    nb_epoch=10,\n",
-    "                    batch_size=128,\n",
-    "                    #validation_split=0.2\n",
-    "                    )"
+    "\n",
+    "model.fit(input_train, y_train,\n",
+    "          nb_epoch=10,\n",
+    "          batch_size=128)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 128 records in 0.335889472 seconds. Throughput is 381.07776 records/second. Loss is 0.14791179."
    ]
   }
  ],

From a6abad4cff2d7110a3fc53450f3b4aa864a978e4 Mon Sep 17 00:00:00 2001
From: Jiaming <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 17:06:10 +0800
Subject: [PATCH 10/46] delete

---
 keras/2.1-mnist.ipynb                      | 153 --------------
 keras/3.5-binary-classification.ipynb      | 200 ------------------
 keras/3.6-multi-class-classification.ipynb | 157 --------------
 keras/3.7-regression.ipynb                 | 232 ---------------------
 keras/4.4-regularization-and-dropout.ipynb | 195 -----------------
 keras/5.1-cnn.ipynb                        | 152 --------------
 keras/6.2-rnn.ipynb                        | 194 -----------------
 7 files changed, 1283 deletions(-)
 delete mode 100644 keras/2.1-mnist.ipynb
 delete mode 100644 keras/3.5-binary-classification.ipynb
 delete mode 100644 keras/3.6-multi-class-classification.ipynb
 delete mode 100644 keras/3.7-regression.ipynb
 delete mode 100644 keras/4.4-regularization-and-dropout.ipynb
 delete mode 100644 keras/5.1-cnn.ipynb
 delete mode 100644 keras/6.2-rnn.ipynb

diff --git a/keras/2.1-mnist.ipynb b/keras/2.1-mnist.ipynb
deleted file mode 100644
index cbf64b7..0000000
--- a/keras/2.1-mnist.ipynb
+++ /dev/null
@@ -1,153 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# MNIST\n",
-    "\n",
-    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset.\n",
-    "\n",
-    "The MNIST dataset comes pre-loaded in Keras, in the form of a set of four Numpy arrays."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import mnist\n",
-    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Import the modules we need to build the network. In Keras it is:\n",
-    "\n",
-    "    from keras import models\n",
-    "    from keras import layers\n",
-    "Just replace it with following in order to use analytics-zoo:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Build the network, compile and fit:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network = models.Sequential()\n",
-    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
-    "network.add(layers.Dense(10, activation='softmax'))\n",
-    "\n",
-    "network.compile(optimizer='rmsprop',\n",
-    "                loss='sparse_categorical_crossentropy',\n",
-    "                metrics=['accuracy'])\n",
-    "\n",
-    "train_images = train_images.reshape((60000, 28 * 28))\n",
-    "train_images = train_images.astype('float32') / 255\n",
-    "\n",
-    "test_images = test_images.reshape((10000, 28 * 28))\n",
-    "test_images = test_images.astype('float32') / 255\n",
-    "\n",
-    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Evaluate return\n",
-    "Check our result on test set. In Keras it is:\n",
-    "\n",
-    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
-    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
-    "print('test_acc:', test_result[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.9783999919891357"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/keras/3.5-binary-classification.ipynb b/keras/3.5-binary-classification.ipynb
deleted file mode 100644
index 79a17cc..0000000
--- a/keras/3.5-binary-classification.ipynb
+++ /dev/null
@@ -1,200 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Binary classification\n",
-    "We'll be working with \"IMDB dataset\", a set of 50,000 highly-polarized reviews from the Internet Movie Database. They are split into 25,000 reviews for training and 25,000 reviews for testing, each set consisting in 50% negative and 50% positive reviews.\n",
-    "\n",
-    "Just like the MNIST dataset, the IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words) have been turned into sequences of integers, where each integer stands for a specific word in a dictionary.\n",
-    "\n",
-    "The following code will load the dataset (when you run it for the first time, about 80MB of data will be downloaded to your machine).\n",
-    "\n",
-    "Then we vectorize the sequences to prepare the data we are going to feed to the model. We also separate part of the data for validation."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import imdb\n",
-    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)\n",
-    "\n",
-    "import numpy as np\n",
-    "def vectorize_sequences(sequences, dimension=10000):\n",
-    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
-    "    results = np.zeros((len(sequences), dimension))\n",
-    "    for i, sequence in enumerate(sequences):\n",
-    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
-    "    return results\n",
-    "\n",
-    "y_train = np.asarray(train_labels).astype('float32')\n",
-    "y_test = np.asarray(test_labels).astype('float32')\n",
-    "\n",
-    "x_val = x_train[:10000]\n",
-    "partial_x_train = x_train[10000:]\n",
-    "y_val = y_train[:10000]\n",
-    "partial_y_train = y_train[10000:]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Build the network, then compile:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers\n",
-    "\n",
-    "model = models.Sequential()\n",
-    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
-    "model.add(layers.Dense(16, activation='relu'))\n",
-    "model.add(layers.Dense(1, activation='sigmoid'))\n",
-    "\n",
-    "model.compile(optimizer='rmsprop',\n",
-    "              loss='binary_crossentropy',\n",
-    "              metrics=['accuracy'])"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Accuracy checkout\n",
-    "To checkout the behavior of this model in Keras, the original code use `matplotlib` library to draw the following `history` object\n",
-    "    \n",
-    "    history = model.fit(partial_x_train,\n",
-    "                    partial_y_train,\n",
-    "                    nb_epoch=5,\n",
-    "                    batch_size=512,\n",
-    "                    validation_data=(x_val, y_val)\n",
-    "                    )\n",
-    "After `fit` method finishes, the results are stored in `history` and thus could be visualized.\n",
-    "\n",
-    "Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via tensorboard. Code above need to be replaced with following:\n",
-    "\n",
-    "Then you could see result in tensorboard by command in terminal `tensorboard --logdir ./` starting tensorboard service and go to `localhost:port_number` as shown in your terminal."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "model.set_tensorboard('./', '3-5_summary')\n",
-    "model.fit(partial_x_train,\n",
-    "          partial_y_train,\n",
-    "          nb_epoch=5,\n",
-    "          batch_size=512,\n",
-    "          validation_data=(x_val, y_val)\n",
-    "          )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 512 records in 0.022409147 seconds. Throughput is 22847.812 records/second. Loss is 0.21351601.\n",
-    "Top1Accuracy is Accuracy(correct: 8606, count: 10000, accuracy: 0.8606)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Check the result on test data:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "results = model.evaluate(x_test, y_test)\n",
-    "print('test_acc:', results[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.8624799847602844"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Predict result\n",
-    "Predict the result on test data, in Keras, it is easy to just call following code to get the result\n",
-    "\n",
-    "    model.predict(x_test)\n",
-    "In analytics-zoo, the return of `predict` is RDD, so you need to call `collect` method to get the result:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "prediction = model.predict(x_test)\n",
-    "result = prediction.collect()"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/keras/3.6-multi-class-classification.ipynb b/keras/3.6-multi-class-classification.ipynb
deleted file mode 100644
index 0655565..0000000
--- a/keras/3.6-multi-class-classification.ipynb
+++ /dev/null
@@ -1,157 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Multi-class classification\n",
-    "In this section, we will build a network to classify Reuters newswires into 46 different mutually-exclusive topics. Since we have many classes, this problem is an instance of \"multi-class classification\", and since each data point should be classified into only one category, the problem is more specifically an instance of \"single-label, multi-class classification\".\n",
-    "\n",
-    "We will be working with the Reuters dataset, a set of short newswires and their topics, published by Reuters in 1986. It's a very simple, widely used toy dataset for text classification. There are 46 different topics; some topics are more represented than others, but each topic has at least 10 examples in the training set.\n",
-    "\n",
-    "Like IMDB and MNIST, the Reuters dataset comes packaged as part of Keras. Let's take a look right away:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import reuters\n",
-    "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(nb_words=10000)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Then we do some preprocessing like Chapter 3.5"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "word_index = reuters.get_word_index()\n",
-    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
-    "\n",
-    "import numpy as np\n",
-    "def vectorize_sequences(sequences, dimension=10000):\n",
-    "    results = np.zeros((len(sequences), dimension))\n",
-    "    for i, sequence in enumerate(sequences):\n",
-    "        results[i, sequence] = 1.\n",
-    "    return results\n",
-    "\n",
-    "x_train = vectorize_sequences(train_data)\n",
-    "x_test = vectorize_sequences(test_data)\n",
-    "# this part pending to modify, one-hot or integer issue\n",
-    "\n",
-    "x_val = x_train[:1000]\n",
-    "partial_x_train = x_train[1000:]\n",
-    "\n",
-    "y_val = train_labels[:1000]\n",
-    "partial_y_train = train_labels[1000:]    # this line would return list\n",
-    "partial_y_train = np.array(partial_y_train)    # convert list to ndarray"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Build the model, compile, train and evaluate"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "model = models.Sequential()\n",
-    "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
-    "model.add(layers.Dense(64, activation='relu'))\n",
-    "model.add(layers.Dense(46, activation='softmax'))\n",
-    "\n",
-    "model.compile(optimizer='rmsprop',\n",
-    "              loss='sparse_categorical_crossentropy',\n",
-    "              metrics=['accuracy'])\n",
-    "\n",
-    "model.fit(partial_x_train,\n",
-    "          partial_y_train,\n",
-    "          nb_epoch=20,\n",
-    "          batch_size=512,\n",
-    "          validation_data=(x_val, y_val))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 512 records in 0.03322949 seconds. Throughput is 15408.001 records/second. Loss is 0.36856997.\n",
-    "Top1Accuracy is Accuracy(correct: 808, count: 1000, accuracy: 0.808)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "test_results = model.evaluate(x_test, test_labels)\n",
-    "print (test_results[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.7885128855705261"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/keras/3.7-regression.ipynb b/keras/3.7-regression.ipynb
deleted file mode 100644
index 769e343..0000000
--- a/keras/3.7-regression.ipynb
+++ /dev/null
@@ -1,232 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Regression\n",
-    "We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
-    "\n",
-    "The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, others between 0 and 100..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This dataset is packaged in Keras 2.0.8 but not in Keras 1.2.2, so that we need to use following code to get the data, then we also apply normalization on these data:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.utils.data_utils import get_file\n",
-    "def load_data(path='boston_housing.npz', test_split=0.2, seed=113):\n",
-    "    \"\"\"Loads the Boston Housing dataset.\n",
-    "    # Arguments\n",
-    "        path: path where to cache the dataset locally\n",
-    "            (relative to ~/.zoo.pipeline.api.keras/datasets).\n",
-    "        test_split: fraction of the data to reserve as test set.\n",
-    "        seed: Random seed for shuffling the data\n",
-    "            before computing the test split.\n",
-    "    # Returns\n",
-    "        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.\n",
-    "    \"\"\"\n",
-    "    assert 0 <= test_split < 1\n",
-    "    path = get_file(\n",
-    "        path,\n",
-    "        origin='https://s3.amazonaws.com/zoo.pipeline.api.keras-datasets/boston_housing.npz'\n",
-    "        )\n",
-    "    with np.load(path) as f:\n",
-    "        x = f['x']\n",
-    "        y = f['y']\n",
-    "\n",
-    "    np.random.seed(seed)\n",
-    "    indices = np.arange(len(x))\n",
-    "    np.random.shuffle(indices)\n",
-    "    x = x[indices]\n",
-    "    y = y[indices]\n",
-    "\n",
-    "    x_train = np.array(x[:int(len(x) * (1 - test_split))])\n",
-    "    y_train = np.array(y[:int(len(x) * (1 - test_split))])\n",
-    "    x_test = np.array(x[int(len(x) * (1 - test_split)):])\n",
-    "    y_test = np.array(y[int(len(x) * (1 - test_split)):])\n",
-    "    return (x_train, y_train), (x_test, y_test)\n",
-    "\n",
-    "(train_data, train_targets), (test_data, test_targets) = load_data()\n",
-    "\n",
-    "mean = train_data.mean(axis=0)\n",
-    "train_data -= mean\n",
-    "std = train_data.std(axis=0)\n",
-    "train_data /= std\n",
-    "\n",
-    "test_data -= mean\n",
-    "test_data /= std"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "In this example we have so few data points, the validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot depending on which data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a high variance with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
-    "\n",
-    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions (typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining partition. The validation score for the model used would then be the average of the K validation scores obtained.\n",
-    "\n",
-    "Since we are using K-fold so that we have to build the model multiple times, we use following function to build our model:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers\n",
-    "\n",
-    "def build_model():\n",
-    "    # Because we will need to instantiate\n",
-    "    # the same model multiple times,\n",
-    "    # we use a function to construct it.\n",
-    "    model = models.Sequential()\n",
-    "    model.add(layers.Dense(64, activation='relu',\n",
-    "                           input_shape=(train_data.shape[1],)))\n",
-    "    model.add(layers.Dense(64, activation='relu'))\n",
-    "    model.add(layers.Dense(1))\n",
-    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
-    "    return model"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Then let's start our training:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import numpy as np\n",
-    "\n",
-    "k = 4\n",
-    "num_val_samples = len(train_data) // k\n",
-    "num_nb_epoch = 50\n",
-    "all_scores = []\n",
-    "for i in range(k):\n",
-    "    print('processing fold #', i)\n",
-    "    # Prepare the validation data: data from partition # k\n",
-    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
-    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
-    "\n",
-    "    # Prepare the training data: data from all other partitions\n",
-    "    partial_train_data = np.concatenate(\n",
-    "        [train_data[:i * num_val_samples],\n",
-    "         train_data[(i + 1) * num_val_samples:]],\n",
-    "        axis=0)\n",
-    "    partial_train_targets = np.concatenate(\n",
-    "        [train_targets[:i * num_val_samples],\n",
-    "         train_targets[(i + 1) * num_val_samples:]],\n",
-    "        axis=0)\n",
-    "\n",
-    "    # Build the Keras model (already compiled)\n",
-    "    model = build_model()\n",
-    "    # Train the model (in silent mode, verbose=0)\n",
-    "    #model.fit(partial_train_data, partial_train_targets,\n",
-    "    #          nb_epoch=num_nb_epoch, batch_size=1, verbose=0)\n",
-    "    model.fit(partial_train_data, partial_train_targets,\n",
-    "              nb_epoch=num_nb_epoch, batch_size=16)\n",
-    "\n",
-    "    # Evaluate the model on the validation data\n",
-    "    #val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
-    "    val_mae = model.evaluate(val_data, val_targets)\n",
-    "    all_scores.append(val_mae[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "processing fold # 0\n",
-    "Trained 16 records in 0.011235845 seconds. Throughput is 1424.0139 records/second. Loss is 8.708786.\n",
-    "processing fold # 1\n",
-    "Trained 16 records in 0.009535034 seconds. Throughput is 1678.0223 records/second. Loss is 5.3613434.\n",
-    "processing fold # 2\n",
-    "Trained 16 records in 0.008636178 seconds. Throughput is 1852.6713 records/second. Loss is 18.106756.\n",
-    "processing fold # 3\n",
-    "Trained 16 records in 0.009207628 seconds. Throughput is 1737.6897 records/second. Loss is 7.0931993."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Then we could check our K-fold training result:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print (all_scores)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[20.572654724121094, 19.606250762939453, 21.224998474121094, 22.60078239440918]"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/keras/4.4-regularization-and-dropout.ipynb b/keras/4.4-regularization-and-dropout.ipynb
deleted file mode 100644
index 5ed40ab..0000000
--- a/keras/4.4-regularization-and-dropout.ipynb
+++ /dev/null
@@ -1,195 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Regularization and dropout\n",
-    "Let's review some of the most common techniques to prevent overfitting, and let's apply them in practice to improve our movie classification model from the previous chapter. First let's prepare the data using the code from Chapter 3.5 and preprocess the data"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import imdb\n",
-    "import numpy as np\n",
-    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)\n",
-    "\n",
-    "def vectorize_sequences(sequences, dimension=10000):\n",
-    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
-    "    results = np.zeros((len(sequences), dimension))\n",
-    "    for i, sequence in enumerate(sequences):\n",
-    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
-    "    return results\n",
-    "\n",
-    "x_train = vectorize_sequences(train_data)\n",
-    "x_test = vectorize_sequences(test_data)\n",
-    "\n",
-    "y_train = np.asarray(train_labels).astype('float32')\n",
-    "y_test = np.asarray(test_labels).astype('float32')"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's first try an easy model and try to optimize it afterwards:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers\n",
-    "\n",
-    "original_model = models.Sequential()\n",
-    "original_model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
-    "original_model.add(layers.Dense(16, activation='relu'))\n",
-    "original_model.add(layers.Dense(1, activation='sigmoid'))\n",
-    "\n",
-    "original_model.compile(optimizer='rmsprop',\n",
-    "                       loss='binary_crossentropy',\n",
-    "                       metrics=['acc'])\n",
-    "\n",
-    "original_model.fit(x_train, y_train,\n",
-    "                   epochs=20,\n",
-    "                   batch_size=512,\n",
-    "                   validation_data=(x_test, y_test))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 512 records in 0.024455326 seconds. Throughput is 20936.135 records/second. Loss is 0.01585226.\n",
-    "Top1Accuracy is Accuracy(correct: 21341, count: 25000, accuracy: 0.85364)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's add L2 weight regularization to our movie review classification network:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import regularizers\n",
-    "\n",
-    "l2_model = models.Sequential()\n",
-    "l2_model.add(layers.Dense(16, W_regularizer=regularizers.l2(0.001),\n",
-    "                          activation='relu', input_shape=(10000,)))\n",
-    "l2_model.add(layers.Dense(16, W_regularizer=regularizers.l2(0.001),\n",
-    "                          activation='relu'))\n",
-    "l2_model.add(layers.Dense(1, activation='sigmoid'))\n",
-    "\n",
-    "l2_model.compile(optimizer='rmsprop',\n",
-    "                 loss='binary_crossentropy',\n",
-    "                 metrics=['acc'])\n",
-    "\n",
-    "l2_model.fit(x_train, y_train,\n",
-    "             nb_epoch=20,\n",
-    "             batch_size=512,\n",
-    "             validation_data=(x_test, y_test))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 512 records in 0.024366594 seconds. Throughput is 21012.373 records/second. Loss is 0.13651785.\n",
-    "Top1Accuracy is Accuracy(correct: 21684, count: 25000, accuracy: 0.86736)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's add two Dropout layers in our IMDB network to see how well they do at reducing overfitting:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "dpt_model = models.Sequential()\n",
-    "dpt_model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
-    "dpt_model.add(layers.Dropout(0.5))\n",
-    "dpt_model.add(layers.Dense(16, activation='relu'))\n",
-    "dpt_model.add(layers.Dropout(0.5))\n",
-    "dpt_model.add(layers.Dense(1, activation='sigmoid'))\n",
-    "\n",
-    "dpt_model.compile(optimizer='rmsprop',\n",
-    "                  loss='binary_crossentropy',\n",
-    "                  metrics=['acc'])\n",
-    "\n",
-    "dpt_model.fit(x_train, y_train,\n",
-    "              nb_epoch=20,\n",
-    "              batch_size=512,\n",
-    "              validation_data=(x_test, y_test))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 512 records in 0.017992654 seconds. Throughput is 28456.057 records/second. Loss is 0.112769656. \n",
-    "Top1Accuracy is Accuracy(correct: 21871, count: 25000, accuracy: 0.87484)"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/keras/5.1-cnn.ipynb b/keras/5.1-cnn.ipynb
deleted file mode 100644
index 72786d2..0000000
--- a/keras/5.1-cnn.ipynb
+++ /dev/null
@@ -1,152 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# CNN\n",
-    "First, let's take a practical look at a very simple convnet example. We will use our convnet to classify MNIST digits, a task that you've already been through in Chapter 2.\n",
-    "\n",
-    "First load the MNIST dataset:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import mnist\n",
-    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### CNN input shape\n",
-    "Once we get the dataset, we need to reshape the images. In keras the shape of the dataset is `(sample_size, height, width, channel)`, like the Keras code below:\n",
-    "    \n",
-    "    train_images = train_images.reshape((60000, 28, 28, 1))\n",
-    "In analytics-zoo, the default order is theano-style NCHW `(sample_size, channel, height, width)`, so you can process data like following:\n",
-    "\n",
-    "Alternatively, you can also use tensorflow-style NHWC as Keras default just by setting `Convolution2D(dim_ordering=\"tf\")`"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "train_images = train_images.reshape((60000, 1, 28, 28))\n",
-    "train_images = train_images.astype('float32') / 255\n",
-    "\n",
-    "test_images = test_images.reshape((10000, 1, 28, 28))\n",
-    "test_images = test_images.astype('float32') / 255"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Then build the model, compile, train and evaluate:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import layers\n",
-    "from zoo.pipeline.api.keras import models\n",
-    "\n",
-    "model = models.Sequential()\n",
-    "model.add(layers.Conv2D(32, nb_col=3, nb_row=3, activation='relu', input_shape=(1,28,28)))\n",
-    "model.add(layers.MaxPooling2D((2, 2)))\n",
-    "model.add(layers.Conv2D(64, nb_col=3, nb_row=3, activation='relu'))\n",
-    "model.add(layers.MaxPooling2D((2, 2)))\n",
-    "model.add(layers.Conv2D(64, nb_col=3, nb_row=3, activation='relu'))\n",
-    "\n",
-    "model.summary()\n",
-    "\n",
-    "model.add(layers.Flatten())\n",
-    "model.add(layers.Dense(64, activation='relu'))\n",
-    "model.add(layers.Dense(10, activation='softmax'))\n",
-    "\n",
-    "model.compile(optimizer='rmsprop',\n",
-    "              loss='sparse_categorical_crossentropy',\n",
-    "              metrics=['acc'])\n",
-    "\n",
-    "model.fit(train_images, train_labels, nb_epoch=5, batch_size=64)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 64 records in 0.03212866 seconds. Throughput is 1991.9911 records/second. Loss is 0.0023578003."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "test_result = model.evaluate(test_images, test_labels)\n",
-    "print('test_acc:', test_result[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.9915000200271606"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/keras/6.2-rnn.ipynb b/keras/6.2-rnn.ipynb
deleted file mode 100644
index ef1146f..0000000
--- a/keras/6.2-rnn.ipynb
+++ /dev/null
@@ -1,194 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# RNN\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras.models import Sequential\n",
-    "from zoo.pipeline.api.keras.layers import Embedding, SimpleRNN"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Now let's try to use such a model on the IMDB movie review classification problem. First, let's preprocess the data:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import imdb\n",
-    "from keras.preprocessing import sequence\n",
-    "\n",
-    "max_features = 10000  # number of words to consider as features\n",
-    "maxlen = 500  # cut texts after this number of words (among top max_features most common words)\n",
-    "batch_size = 32\n",
-    "\n",
-    "(input_train, y_train), (input_test, y_test) = imdb.load_data(nb_words=max_features)\n",
-    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
-    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
-    "print('input_train shape:', input_train.shape)\n",
-    "print('input_test shape:', input_test.shape)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "input_train shape: (25000, 500)\n",
-    "input_test shape: (25000, 500)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Specify input shape\n",
-    "We could add an embedding layer as our first layer in Keras as following:\n",
-    "    \n",
-    "    model = Sequential()\n",
-    "    model.add(Embedding(10000, 32))\n",
-    "In analytics-zoo, you need to specify the input shape of first layer, in this example, the sequence length is 500, so we could build our model as following:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "model = Sequential()\n",
-    "model.add(Embedding(10000, 32, input_shape=(500,)))\n",
-    "model.add(SimpleRNN(32, return_sequences=True))\n",
-    "model.add(SimpleRNN(32, return_sequences=True))\n",
-    "model.add(SimpleRNN(32, return_sequences=True))\n",
-    "model.add(SimpleRNN(32))  # This last layer only returns the last outputs.\n",
-    "model.summary()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's train a simple recurrent network using an `Embedding` layer and a `SimpleRNN` layer:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras.layers import Dense\n",
-    "\n",
-    "model = Sequential()\n",
-    "model.add(Embedding(max_features, 32, input_shape=(500,)))\n",
-    "model.add(SimpleRNN(32))\n",
-    "model.add(Dense(1, activation='sigmoid'))\n",
-    "\n",
-    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
-    "model.fit(input_train, y_train,\n",
-    "          nb_epoch=10,\n",
-    "          batch_size=128)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 128 records in 0.046239497 seconds. Throughput is 2768.1963 records/second. Loss is 0.16970885."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Now let's switch to more practical concerns: we will set up a model using a LSTM layer and train it on the IMDB data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras.layers import LSTM\n",
-    "\n",
-    "model = Sequential()\n",
-    "model.add(Embedding(max_features, 32, input_shape=(500,)))\n",
-    "model.add(LSTM(32))\n",
-    "model.add(Dense(1, activation='sigmoid'))\n",
-    "\n",
-    "model.compile(optimizer='rmsprop',\n",
-    "              loss='binary_crossentropy',\n",
-    "              metrics=['acc'])\n",
-    "\n",
-    "model.fit(input_train, y_train,\n",
-    "          nb_epoch=10,\n",
-    "          batch_size=128)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 128 records in 0.335889472 seconds. Throughput is 381.07776 records/second. Loss is 0.14791179."
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

From bf37983161a5a028552c6477003885099b426d95 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 17:11:13 +0800
Subject: [PATCH 11/46] Update README.md

---
 keras/README.md | 70 -------------------------------------------------
 1 file changed, 70 deletions(-)

diff --git a/keras/README.md b/keras/README.md
index 73d00d8..34c4df3 100644
--- a/keras/README.md
+++ b/keras/README.md
@@ -1,71 +1 @@
-# Tutorials for running distribued Keras (v1.2.2) on Analytics Zoo
 Tutorials for running _**distribued Keras (v1.2.2) on Analytics Zoo**_. These tutorials are ported from François Chollet's [Jupyter notebooks](https://github.com/fchollet/deep-learning-with-python-notebooks) for the book [Deep Learning with Python (Manning Publications)](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff)
-
-This repository is built to describe how to write Keras-style code and directly run it in analytics-zoo so that we could run the original Keras code in distributed mode via limited modification. The original notebook is based on Keras 2.0.8. Currently analytics-zoo builds its Keras-style code based on Keras 1.2.2. Thus, this repository contains the code of implementation of original Keras 2.0.8 code based on Keras 1.2.2, and how to modify the code in order to run it in analytics-zoo.
-
-To make it simple, we omit the description of plenty concepts here and you could find them in original notebook above [(link)](https://github.com/fchollet/deep-learning-with-python-notebooks). Besides, we directly post Keras 1.2.2 code here and the replacements needed from Keras 2.0.8 to Keras 1.2.2 are noted in `Keras_2-to-1.md`.
-
-The training log of each epoch is stored in INFO, you could see these information though configuring python output level, or store these information in a file and check afterwards, these may includes plenty lines of information. For better expression here, we run the code in Pycharm IDE and **we only paste the last info of last epoch of each training function in this notebook**.
-
-This repository use Python 3.5, Keras 1.2.2 (Keras code), Analytics-zoo 0.4.0 (zoo code). We post the summary of Keras-to-zoo code convertion as well as the table of contents.
-
-## Summary of Keras-to-zoo code convertion
-We summarize the modification we need to make here, attached with the link to the example code chapter.
-
-#### First of all
-Make sure you have analytics-zoo installed, see install guide [here](https://analytics-zoo.github.io/master/#PythonUserGuide/install/). Then set the environment variables like following
-  
-    PYSPARK_PYTHON=/path_to_your_python
-    PYSPARK_DRIVER_PYTHON=/path_to_your_python
-    # To avoid version conflict, in my ubuntu, this path is /usr/bin/python3.5
-
-    SPARK_DRIVER_MEMORY=4g
-    # If you encounter heap space exception, you could simply increase this variable to 8g, 16g, etc. 
-    # If no more memory is available on your machine, you could consider reduce the data size.
-
-Then, make sure you have following code at the beginning of your zoo code.
-  
-    from zoo.common.nncontext import *
-    sc = init_nncontext(init_spark_conf().setMaster("local[4]"))
-    
-we set core number to 4 above, you can also set it with another number. But we still recommend 4 because analytics-zoo need the core number could divide the batch size of learning, which is normal set as powers of 4, e.g. 16, 32, 128. Error would be raised if this requirement is not satisfied.
-
-#### Accuracy checkout
-Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via tensorboard, see [Chapter 3.5]()
-
-#### Evaluate return
-The return of `evaluate` method is an `EvaluationResult` object, which is different from Keras, see [Chapter 2.1]()
-
-#### Predict result
-The return of `predict` method is RDD, so you need to call `collect` method to collect them, see [Chapter 3.5]()
-
-#### CNN input shape
-In analytics-zoo, the default order is theano-style NCHW, you can also use tensorflow-style NHWC as Keras default just by setting `Convolution2D(dim_ordering="tf")`, see [Chapter 5.1]()
-
-#### Specify input shape
-In analytics-zoo, you need to specify the input shape of first layer, see [Chapter 6.2]()
-
-#### Parameters not supported
-Analytics-zoo does not support following parameters currently, so we do not use these code in this notebook if they exist in original notebook.
-
-* validation_split: To split the training data into two parts, another part is used for validation
-* verbose: To control the information showed during training
-
-## Table of contents
-
-The main purpose of this repository is to decribe the convertion from Keras to analytics-zoo so that we rename the notebooks to make it more experienced-user oriented. We keep the chapter index so you could still make a reference to the original notebook [here](https://github.com/fchollet/deep-learning-with-python-notebooks#companion-jupyter-notebooks-for-the-book-deep-learning-with-python). We only keep some key description of the original notebook.
-
-* Chapter 2:
-    * [2.1: MNIST](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/2.1-mnist.ipynb)
-* Chapter 3:
-    * [3.5: Binary classification](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/3.5-binary-classification.ipynb)
-    * [3.6: Multi-class classification](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/3.6-multi-class-classification.ipynb)
-    * [3.7: Regression](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/3.7-regression.ipynb)
-* Chapter 4:
-    * [4.4: Regularization and Dropout](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/4.4-regularization-and-dropout.ipynb)
-* Chapter 5:
-    * [5.1: CNN](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/5.1-cnn.ipynb)
-* Chapter 6:
-    * [6.2: RNN](https://github.com/Litchilitchy/zoo-tutorials/blob/master/keras/6.2-rnn.ipynb)
-
-    

From d201267f4758803ccb70cbc0fce7ea24ffd8adf4 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 18:17:12 +0800
Subject: [PATCH 12/46] Add files via upload

---
 2.1-mnist.ipynb | 239 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 239 insertions(+)
 create mode 100644 2.1-mnist.ipynb

diff --git a/2.1-mnist.ipynb b/2.1-mnist.ipynb
new file mode 100644
index 0000000..80dd3df
--- /dev/null
+++ b/2.1-mnist.ipynb
@@ -0,0 +1,239 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**First of all, set environment variables and initialize spark context:**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# A first look at a neural network\n",
+    "This notebook is imported from Chapter 2, Section 1 of [Deep Learning with Python Notebook]().\n",
+    "\n",
+    "We will now take a look at a first concrete example of a neural network, which makes use of analytics-zoo Keras module to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
+    "\n",
+    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
+    "\n",
+    "The MNIST dataset comes pre-loaded in Keras, in the form of a set of four Numpy arrays:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.datasets import mnist\n",
+    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "train_images and train_labels form the \"training set\", the data that the model will learn from. The model will then be tested on the \"test set\", test_images and test_labels. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging from 0 to 9. There is a one-to-one correspondence between the images and the labels."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our workflow will be as follow: first we will present our neural network with the training data, train_images and train_labels. The network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for test_images, and we will verify if these predictions match the labels from test_labels.\n",
+    "\n",
+    "Let's build our network -- again, remember that you aren't supposed to understand everything about this example just yet."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Module import (for previous Keras user only)\n",
+    "Import the modules we need to build the network. In Keras it is:\n",
+    "\n",
+    "    from keras import models\n",
+    "    from keras import layers\n",
+    "Just replace it with following in order to use analytics-zoo:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network = models.Sequential()\n",
+    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
+    "network.add(layers.Dense(10, activation='softmax'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The core building block of neural networks is the \"layer\", a data-processing module which you can conceive as a \"filter\" for data. Some \n",
+    "data comes in, and comes out in a more useful form. Precisely, layers extract _representations_ out of the data fed into them -- hopefully \n",
+    "representations that are more meaningful for the problem at hand. Most of deep learning really consists of chaining together simple layers \n",
+    "which will implement a form of progressive \"data distillation\". A deep learning model is like a sieve for data processing, made of a \n",
+    "succession of increasingly refined data filters -- the \"layers\".\n",
+    "\n",
+    "Here our network consists of a sequence of two `Dense` layers, which are densely-connected (also called \"fully-connected\") neural layers. \n",
+    "The second (and last) layer is a 10-way \"softmax\" layer, which means it will return an array of 10 probability scores (summing to 1). Each \n",
+    "score will be the probability that the current digit image belongs to one of our 10 digit classes.\n",
+    "\n",
+    "To make our network ready for training, we need to pick three more things, as part of \"compilation\" step:\n",
+    "\n",
+    "* A loss function: the is how the network will be able to measure how good a job it is doing on its training data, and thus how it will be \n",
+    "able to steer itself in the right direction.\n",
+    "* An optimizer: this is the mechanism through which the network will update itself based on the data it sees and its loss function.\n",
+    "* Metrics to monitor during training and testing. Here we will only care about accuracy (the fraction of the images that were correctly \n",
+    "classified).\n",
+    "\n",
+    "The exact purpose of the loss function and the optimizer will be made clear throughout the next two chapters."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network.compile(optimizer='rmsprop',\n",
+    "                loss='sparse_categorical_crossentropy',\n",
+    "                metrics=['accuracy'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Before training, we will preprocess our data by reshaping it into the shape that the network expects, and scaling it so that all values are in \n",
+    "the `[0, 1]` interval. Previously, our training images for instance were stored in an array of shape `(60000, 28, 28)` of type `uint8` with \n",
+    "values in the `[0, 255]` interval. We transform it into a `float32` array of shape `(60000, 28 * 28)` with values between 0 and 1."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "train_images = train_images.reshape((60000, 28 * 28))\n",
+    "train_images = train_images.astype('float32') / 255\n",
+    "\n",
+    "test_images = test_images.reshape((10000, 28 * 28))\n",
+    "test_images = test_images.astype('float32') / 255"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are now ready to train our network, which in Keras is done via a call to the `fit` method of the network: \n",
+    "we \"fit\" the model to its training data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Evaluate return (for previous Keras user only)\n",
+    "Check our result on test set. In Keras it is:\n",
+    "\n",
+    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
+    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
+    "print('test_acc:', test_result[0].result)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.9783999919891357"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This concludes our very first example -- you just saw how we could build and a train a neural network to classify handwritten digits, in \n",
+    "less than 20 lines of Python code. In the next chapter, we will go in detail over every moving piece we just previewed, and clarify what is really \n",
+    "going on behind the scenes. You will learn about \"tensors\", the data-storing objects going into the network, about tensor operations, which \n",
+    "layers are made of, and about gradient descent, which allows our network to learn from its training examples."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From 8c1f67e080002037d231456213a4e3cd16bfe01c Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 28 Feb 2019 18:17:55 +0800
Subject: [PATCH 13/46] Rename 2.1-mnist.ipynb to
 2.1-a-first-look-at-a-neural-network.ipynb

---
 2.1-mnist.ipynb => 2.1-a-first-look-at-a-neural-network.ipynb | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename 2.1-mnist.ipynb => 2.1-a-first-look-at-a-neural-network.ipynb (100%)

diff --git a/2.1-mnist.ipynb b/2.1-a-first-look-at-a-neural-network.ipynb
similarity index 100%
rename from 2.1-mnist.ipynb
rename to 2.1-a-first-look-at-a-neural-network.ipynb

From bf9ca63189d0dbcaa9a7f8491bfdad1066c5a243 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Fri, 1 Mar 2019 15:12:00 +0800
Subject: [PATCH 14/46] Add files via upload

---
 keras/2.1-mnist.ipynb | 241 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 241 insertions(+)
 create mode 100644 keras/2.1-mnist.ipynb

diff --git a/keras/2.1-mnist.ipynb b/keras/2.1-mnist.ipynb
new file mode 100644
index 0000000..073b04a
--- /dev/null
+++ b/keras/2.1-mnist.ipynb
@@ -0,0 +1,241 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**First of all, set environment variables and initialize spark context:**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# A first look at a neural network\n",
+    "This notebook is imported from Chapter 2, Section 1 of [Deep Learning with Python Notebook]().\n",
+    "\n",
+    "----\n",
+    "\n",
+    "We will now take a look at a first concrete example of a neural network, which makes use of analytics-zoo Keras module to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
+    "\n",
+    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
+    "\n",
+    "The MNIST dataset comes pre-loaded in analytics-zoo Keras module, in the form of a set of four Numpy arrays:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import mnist\n",
+    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "train_images and train_labels form the \"training set\", the data that the model will learn from. The model will then be tested on the \"test set\", test_images and test_labels. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging from 0 to 9. There is a one-to-one correspondence between the images and the labels."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our workflow will be as follow: first we will present our neural network with the training data, train_images and train_labels. The network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for test_images, and we will verify if these predictions match the labels from test_labels.\n",
+    "\n",
+    "Let's build our network -- again, remember that you aren't supposed to understand everything about this example just yet."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Module import (for previous Keras user)\n",
+    "Import the modules we need to build the network. In Keras it is:\n",
+    "\n",
+    "    from keras import models\n",
+    "    from keras import layers\n",
+    "Just replace it with following in order to use analytics-zoo:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network = models.Sequential()\n",
+    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
+    "network.add(layers.Dense(10, activation='softmax'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The core building block of neural networks is the \"layer\", a data-processing module which you can conceive as a \"filter\" for data. Some \n",
+    "data comes in, and comes out in a more useful form. Precisely, layers extract _representations_ out of the data fed into them -- hopefully \n",
+    "representations that are more meaningful for the problem at hand. Most of deep learning really consists of chaining together simple layers \n",
+    "which will implement a form of progressive \"data distillation\". A deep learning model is like a sieve for data processing, made of a \n",
+    "succession of increasingly refined data filters -- the \"layers\".\n",
+    "\n",
+    "Here our network consists of a sequence of two `Dense` layers, which are densely-connected (also called \"fully-connected\") neural layers. \n",
+    "The second (and last) layer is a 10-way \"softmax\" layer, which means it will return an array of 10 probability scores (summing to 1). Each \n",
+    "score will be the probability that the current digit image belongs to one of our 10 digit classes.\n",
+    "\n",
+    "To make our network ready for training, we need to pick three more things, as part of \"compilation\" step:\n",
+    "\n",
+    "* A loss function: the is how the network will be able to measure how good a job it is doing on its training data, and thus how it will be \n",
+    "able to steer itself in the right direction.\n",
+    "* An optimizer: this is the mechanism through which the network will update itself based on the data it sees and its loss function.\n",
+    "* Metrics to monitor during training and testing. Here we will only care about accuracy (the fraction of the images that were correctly \n",
+    "classified).\n",
+    "\n",
+    "The exact purpose of the loss function and the optimizer will be made clear throughout the next two chapters."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network.compile(optimizer='rmsprop',\n",
+    "                loss='sparse_categorical_crossentropy',\n",
+    "                metrics=['accuracy'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Before training, we will preprocess our data by reshaping it into the shape that the network expects, and scaling it so that all values are in \n",
+    "the `[0, 1]` interval. Previously, our training images for instance were stored in an array of shape `(60000, 28, 28)` of type `uint8` with \n",
+    "values in the `[0, 255]` interval. We transform it into a `float32` array of shape `(60000, 28 * 28)` with values between 0 and 1."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "train_images = train_images.reshape((60000, 28 * 28))\n",
+    "train_images = train_images.astype('float32') / 255\n",
+    "\n",
+    "test_images = test_images.reshape((10000, 28 * 28))\n",
+    "test_images = test_images.astype('float32') / 255"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are now ready to train our network, which in analytics-zoo Keras module is done via a call to the `fit` method of the network: \n",
+    "we \"fit\" the model to its training data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Evaluate return (for previous Keras user)\n",
+    "Check our result on test set. In Keras it is:\n",
+    "\n",
+    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
+    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
+    "print('test_acc:', test_result[0].result)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.9783999919891357"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This concludes our very first example -- you just saw how we could build and a train a neural network to classify handwritten digits, in \n",
+    "less than 20 lines of Python code. In the next chapter, we will go in detail over every moving piece we just previewed, and clarify what is really \n",
+    "going on behind the scenes. You will learn about \"tensors\", the data-storing objects going into the network, about tensor operations, which \n",
+    "layers are made of, and about gradient descent, which allows our network to learn from its training examples."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From fa23ec80ca133abafd61c32dd39ba2e28838c8f5 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Fri, 1 Mar 2019 15:12:13 +0800
Subject: [PATCH 15/46] Delete 2.1-a-first-look-at-a-neural-network.ipynb

---
 2.1-a-first-look-at-a-neural-network.ipynb | 239 ---------------------
 1 file changed, 239 deletions(-)
 delete mode 100644 2.1-a-first-look-at-a-neural-network.ipynb

diff --git a/2.1-a-first-look-at-a-neural-network.ipynb b/2.1-a-first-look-at-a-neural-network.ipynb
deleted file mode 100644
index 80dd3df..0000000
--- a/2.1-a-first-look-at-a-neural-network.ipynb
+++ /dev/null
@@ -1,239 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**First of all, set environment variables and initialize spark context:**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# A first look at a neural network\n",
-    "This notebook is imported from Chapter 2, Section 1 of [Deep Learning with Python Notebook]().\n",
-    "\n",
-    "We will now take a look at a first concrete example of a neural network, which makes use of analytics-zoo Keras module to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
-    "\n",
-    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
-    "\n",
-    "The MNIST dataset comes pre-loaded in Keras, in the form of a set of four Numpy arrays:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from keras.datasets import mnist\n",
-    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "train_images and train_labels form the \"training set\", the data that the model will learn from. The model will then be tested on the \"test set\", test_images and test_labels. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging from 0 to 9. There is a one-to-one correspondence between the images and the labels."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Our workflow will be as follow: first we will present our neural network with the training data, train_images and train_labels. The network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for test_images, and we will verify if these predictions match the labels from test_labels.\n",
-    "\n",
-    "Let's build our network -- again, remember that you aren't supposed to understand everything about this example just yet."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Module import (for previous Keras user only)\n",
-    "Import the modules we need to build the network. In Keras it is:\n",
-    "\n",
-    "    from keras import models\n",
-    "    from keras import layers\n",
-    "Just replace it with following in order to use analytics-zoo:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network = models.Sequential()\n",
-    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
-    "network.add(layers.Dense(10, activation='softmax'))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The core building block of neural networks is the \"layer\", a data-processing module which you can conceive as a \"filter\" for data. Some \n",
-    "data comes in, and comes out in a more useful form. Precisely, layers extract _representations_ out of the data fed into them -- hopefully \n",
-    "representations that are more meaningful for the problem at hand. Most of deep learning really consists of chaining together simple layers \n",
-    "which will implement a form of progressive \"data distillation\". A deep learning model is like a sieve for data processing, made of a \n",
-    "succession of increasingly refined data filters -- the \"layers\".\n",
-    "\n",
-    "Here our network consists of a sequence of two `Dense` layers, which are densely-connected (also called \"fully-connected\") neural layers. \n",
-    "The second (and last) layer is a 10-way \"softmax\" layer, which means it will return an array of 10 probability scores (summing to 1). Each \n",
-    "score will be the probability that the current digit image belongs to one of our 10 digit classes.\n",
-    "\n",
-    "To make our network ready for training, we need to pick three more things, as part of \"compilation\" step:\n",
-    "\n",
-    "* A loss function: the is how the network will be able to measure how good a job it is doing on its training data, and thus how it will be \n",
-    "able to steer itself in the right direction.\n",
-    "* An optimizer: this is the mechanism through which the network will update itself based on the data it sees and its loss function.\n",
-    "* Metrics to monitor during training and testing. Here we will only care about accuracy (the fraction of the images that were correctly \n",
-    "classified).\n",
-    "\n",
-    "The exact purpose of the loss function and the optimizer will be made clear throughout the next two chapters."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network.compile(optimizer='rmsprop',\n",
-    "                loss='sparse_categorical_crossentropy',\n",
-    "                metrics=['accuracy'])"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Before training, we will preprocess our data by reshaping it into the shape that the network expects, and scaling it so that all values are in \n",
-    "the `[0, 1]` interval. Previously, our training images for instance were stored in an array of shape `(60000, 28, 28)` of type `uint8` with \n",
-    "values in the `[0, 255]` interval. We transform it into a `float32` array of shape `(60000, 28 * 28)` with values between 0 and 1."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "train_images = train_images.reshape((60000, 28 * 28))\n",
-    "train_images = train_images.astype('float32') / 255\n",
-    "\n",
-    "test_images = test_images.reshape((10000, 28 * 28))\n",
-    "test_images = test_images.astype('float32') / 255"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We are now ready to train our network, which in Keras is done via a call to the `fit` method of the network: \n",
-    "we \"fit\" the model to its training data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Evaluate return (for previous Keras user only)\n",
-    "Check our result on test set. In Keras it is:\n",
-    "\n",
-    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
-    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
-    "print('test_acc:', test_result[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.9783999919891357"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This concludes our very first example -- you just saw how we could build and a train a neural network to classify handwritten digits, in \n",
-    "less than 20 lines of Python code. In the next chapter, we will go in detail over every moving piece we just previewed, and clarify what is really \n",
-    "going on behind the scenes. You will learn about \"tensors\", the data-storing objects going into the network, about tensor operations, which \n",
-    "layers are made of, and about gradient descent, which allows our network to learn from its training examples."
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

From b19c4427fe5d743f5493856b51ec01d8b6a2f51b Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Fri, 1 Mar 2019 15:12:54 +0800
Subject: [PATCH 16/46] Add files via upload

---
 ...2.1-a-first-look-at-a-neural-network.ipynb | 241 ++++++++++++++++++
 1 file changed, 241 insertions(+)
 create mode 100644 keras/2.1-a-first-look-at-a-neural-network.ipynb

diff --git a/keras/2.1-a-first-look-at-a-neural-network.ipynb b/keras/2.1-a-first-look-at-a-neural-network.ipynb
new file mode 100644
index 0000000..073b04a
--- /dev/null
+++ b/keras/2.1-a-first-look-at-a-neural-network.ipynb
@@ -0,0 +1,241 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**First of all, set environment variables and initialize spark context:**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# A first look at a neural network\n",
+    "This notebook is imported from Chapter 2, Section 1 of [Deep Learning with Python Notebook]().\n",
+    "\n",
+    "----\n",
+    "\n",
+    "We will now take a look at a first concrete example of a neural network, which makes use of analytics-zoo Keras module to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
+    "\n",
+    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
+    "\n",
+    "The MNIST dataset comes pre-loaded in analytics-zoo Keras module, in the form of a set of four Numpy arrays:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import mnist\n",
+    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "train_images and train_labels form the \"training set\", the data that the model will learn from. The model will then be tested on the \"test set\", test_images and test_labels. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging from 0 to 9. There is a one-to-one correspondence between the images and the labels."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our workflow will be as follow: first we will present our neural network with the training data, train_images and train_labels. The network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for test_images, and we will verify if these predictions match the labels from test_labels.\n",
+    "\n",
+    "Let's build our network -- again, remember that you aren't supposed to understand everything about this example just yet."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Module import (for previous Keras user)\n",
+    "Import the modules we need to build the network. In Keras it is:\n",
+    "\n",
+    "    from keras import models\n",
+    "    from keras import layers\n",
+    "Just replace it with following in order to use analytics-zoo:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network = models.Sequential()\n",
+    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
+    "network.add(layers.Dense(10, activation='softmax'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The core building block of neural networks is the \"layer\", a data-processing module which you can conceive as a \"filter\" for data. Some \n",
+    "data comes in, and comes out in a more useful form. Precisely, layers extract _representations_ out of the data fed into them -- hopefully \n",
+    "representations that are more meaningful for the problem at hand. Most of deep learning really consists of chaining together simple layers \n",
+    "which will implement a form of progressive \"data distillation\". A deep learning model is like a sieve for data processing, made of a \n",
+    "succession of increasingly refined data filters -- the \"layers\".\n",
+    "\n",
+    "Here our network consists of a sequence of two `Dense` layers, which are densely-connected (also called \"fully-connected\") neural layers. \n",
+    "The second (and last) layer is a 10-way \"softmax\" layer, which means it will return an array of 10 probability scores (summing to 1). Each \n",
+    "score will be the probability that the current digit image belongs to one of our 10 digit classes.\n",
+    "\n",
+    "To make our network ready for training, we need to pick three more things, as part of \"compilation\" step:\n",
+    "\n",
+    "* A loss function: the is how the network will be able to measure how good a job it is doing on its training data, and thus how it will be \n",
+    "able to steer itself in the right direction.\n",
+    "* An optimizer: this is the mechanism through which the network will update itself based on the data it sees and its loss function.\n",
+    "* Metrics to monitor during training and testing. Here we will only care about accuracy (the fraction of the images that were correctly \n",
+    "classified).\n",
+    "\n",
+    "The exact purpose of the loss function and the optimizer will be made clear throughout the next two chapters."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network.compile(optimizer='rmsprop',\n",
+    "                loss='sparse_categorical_crossentropy',\n",
+    "                metrics=['accuracy'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Before training, we will preprocess our data by reshaping it into the shape that the network expects, and scaling it so that all values are in \n",
+    "the `[0, 1]` interval. Previously, our training images for instance were stored in an array of shape `(60000, 28, 28)` of type `uint8` with \n",
+    "values in the `[0, 255]` interval. We transform it into a `float32` array of shape `(60000, 28 * 28)` with values between 0 and 1."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "train_images = train_images.reshape((60000, 28 * 28))\n",
+    "train_images = train_images.astype('float32') / 255\n",
+    "\n",
+    "test_images = test_images.reshape((10000, 28 * 28))\n",
+    "test_images = test_images.astype('float32') / 255"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are now ready to train our network, which in analytics-zoo Keras module is done via a call to the `fit` method of the network: \n",
+    "we \"fit\" the model to its training data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Evaluate return (for previous Keras user)\n",
+    "Check our result on test set. In Keras it is:\n",
+    "\n",
+    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
+    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
+    "print('test_acc:', test_result[0].result)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "test_acc: 0.9783999919891357"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This concludes our very first example -- you just saw how we could build and a train a neural network to classify handwritten digits, in \n",
+    "less than 20 lines of Python code. In the next chapter, we will go in detail over every moving piece we just previewed, and clarify what is really \n",
+    "going on behind the scenes. You will learn about \"tensors\", the data-storing objects going into the network, about tensor operations, which \n",
+    "layers are made of, and about gradient descent, which allows our network to learn from its training examples."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From 5022ebf2c2cc2e1b7079fb09533d4b15d2622dc3 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Fri, 1 Mar 2019 15:13:09 +0800
Subject: [PATCH 17/46] Delete 2.1-mnist.ipynb

---
 keras/2.1-mnist.ipynb | 241 ------------------------------------------
 1 file changed, 241 deletions(-)
 delete mode 100644 keras/2.1-mnist.ipynb

diff --git a/keras/2.1-mnist.ipynb b/keras/2.1-mnist.ipynb
deleted file mode 100644
index 073b04a..0000000
--- a/keras/2.1-mnist.ipynb
+++ /dev/null
@@ -1,241 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**First of all, set environment variables and initialize spark context:**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# A first look at a neural network\n",
-    "This notebook is imported from Chapter 2, Section 1 of [Deep Learning with Python Notebook]().\n",
-    "\n",
-    "----\n",
-    "\n",
-    "We will now take a look at a first concrete example of a neural network, which makes use of analytics-zoo Keras module to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
-    "\n",
-    "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
-    "\n",
-    "The MNIST dataset comes pre-loaded in analytics-zoo Keras module, in the form of a set of four Numpy arrays:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras.datasets import mnist\n",
-    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "train_images and train_labels form the \"training set\", the data that the model will learn from. The model will then be tested on the \"test set\", test_images and test_labels. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging from 0 to 9. There is a one-to-one correspondence between the images and the labels."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Our workflow will be as follow: first we will present our neural network with the training data, train_images and train_labels. The network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for test_images, and we will verify if these predictions match the labels from test_labels.\n",
-    "\n",
-    "Let's build our network -- again, remember that you aren't supposed to understand everything about this example just yet."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Module import (for previous Keras user)\n",
-    "Import the modules we need to build the network. In Keras it is:\n",
-    "\n",
-    "    from keras import models\n",
-    "    from keras import layers\n",
-    "Just replace it with following in order to use analytics-zoo:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network = models.Sequential()\n",
-    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
-    "network.add(layers.Dense(10, activation='softmax'))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The core building block of neural networks is the \"layer\", a data-processing module which you can conceive as a \"filter\" for data. Some \n",
-    "data comes in, and comes out in a more useful form. Precisely, layers extract _representations_ out of the data fed into them -- hopefully \n",
-    "representations that are more meaningful for the problem at hand. Most of deep learning really consists of chaining together simple layers \n",
-    "which will implement a form of progressive \"data distillation\". A deep learning model is like a sieve for data processing, made of a \n",
-    "succession of increasingly refined data filters -- the \"layers\".\n",
-    "\n",
-    "Here our network consists of a sequence of two `Dense` layers, which are densely-connected (also called \"fully-connected\") neural layers. \n",
-    "The second (and last) layer is a 10-way \"softmax\" layer, which means it will return an array of 10 probability scores (summing to 1). Each \n",
-    "score will be the probability that the current digit image belongs to one of our 10 digit classes.\n",
-    "\n",
-    "To make our network ready for training, we need to pick three more things, as part of \"compilation\" step:\n",
-    "\n",
-    "* A loss function: the is how the network will be able to measure how good a job it is doing on its training data, and thus how it will be \n",
-    "able to steer itself in the right direction.\n",
-    "* An optimizer: this is the mechanism through which the network will update itself based on the data it sees and its loss function.\n",
-    "* Metrics to monitor during training and testing. Here we will only care about accuracy (the fraction of the images that were correctly \n",
-    "classified).\n",
-    "\n",
-    "The exact purpose of the loss function and the optimizer will be made clear throughout the next two chapters."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network.compile(optimizer='rmsprop',\n",
-    "                loss='sparse_categorical_crossentropy',\n",
-    "                metrics=['accuracy'])"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Before training, we will preprocess our data by reshaping it into the shape that the network expects, and scaling it so that all values are in \n",
-    "the `[0, 1]` interval. Previously, our training images for instance were stored in an array of shape `(60000, 28, 28)` of type `uint8` with \n",
-    "values in the `[0, 255]` interval. We transform it into a `float32` array of shape `(60000, 28 * 28)` with values between 0 and 1."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "train_images = train_images.reshape((60000, 28 * 28))\n",
-    "train_images = train_images.astype('float32') / 255\n",
-    "\n",
-    "test_images = test_images.reshape((10000, 28 * 28))\n",
-    "test_images = test_images.astype('float32') / 255"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We are now ready to train our network, which in analytics-zoo Keras module is done via a call to the `fit` method of the network: \n",
-    "we \"fit\" the model to its training data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Evaluate return (for previous Keras user)\n",
-    "Check our result on test set. In Keras it is:\n",
-    "\n",
-    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
-    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
-    "print('test_acc:', test_result[0].result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.9783999919891357"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This concludes our very first example -- you just saw how we could build and a train a neural network to classify handwritten digits, in \n",
-    "less than 20 lines of Python code. In the next chapter, we will go in detail over every moving piece we just previewed, and clarify what is really \n",
-    "going on behind the scenes. You will learn about \"tensors\", the data-storing objects going into the network, about tensor operations, which \n",
-    "layers are made of, and about gradient descent, which allows our network to learn from its training examples."
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

From 81073f4a0806ec51020d4fd51f138b1fe4116ca0 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Fri, 8 Mar 2019 15:13:29 +0800
Subject: [PATCH 18/46] Add files via upload

---
 ...2.1-a-first-look-at-a-neural-network.ipynb | 250 +++++++++++++++---
 1 file changed, 219 insertions(+), 31 deletions(-)

diff --git a/keras/2.1-a-first-look-at-a-neural-network.ipynb b/keras/2.1-a-first-look-at-a-neural-network.ipynb
index 073b04a..c281ede 100644
--- a/keras/2.1-a-first-look-at-a-neural-network.ipynb
+++ b/keras/2.1-a-first-look-at-a-neural-network.ipynb
@@ -9,11 +9,19 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
    "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
     "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
     "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
     "\n",
@@ -26,22 +34,43 @@
    "metadata": {},
    "source": [
     "# A first look at a neural network\n",
-    "This notebook is imported from Chapter 2, Section 1 of [Deep Learning with Python Notebook]().\n",
     "\n",
     "----\n",
     "\n",
-    "We will now take a look at a first concrete example of a neural network, which makes use of analytics-zoo Keras module to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
+    "We will now take a look at a first concrete example of a neural network, which makes use of Keras (v1.2.2) API in [Analytics Zoo](https://github.com/intel-analytics/analytics-zoo) to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
     "\n",
     "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
     "\n",
-    "The MNIST dataset comes pre-loaded in analytics-zoo Keras module, in the form of a set of four Numpy arrays:"
+    "The MNIST dataset  comes pre-loaded in the Keras API of Analytics Zoo, in the form of a set of four Numpy arrays:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Datasets import\n",
+    "<font color='red'>In Keras you could use following code to import the datasets:</font>\n",
+    "\n",
+    "    from keras.datasets import mnist\n",
+    "<font color='red'>Just replace it with following in analytics-zoo:</font>"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting /tmp/.zoo/dataset/mnist/train-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/.zoo/dataset/mnist/train-labels-idx1-ubyte.gz\n",
+      "Extracting /tmp/.zoo/dataset/mnist/t10k-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/.zoo/dataset/mnist/t10k-labels-idx1-ubyte.gz\n"
+     ]
+    }
+   ],
    "source": [
     "from zoo.pipeline.api.keras.datasets import mnist\n",
     "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
@@ -51,14 +80,140 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "train_images and train_labels form the \"training set\", the data that the model will learn from. The model will then be tested on the \"test set\", test_images and test_labels. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging from 0 to 9. There is a one-to-one correspondence between the images and the labels."
+    "`train_images` and `train_labels` form the \"training set\", the data that the model will learn from. The model will then be tested on the \n",
+    "\"test set\", `test_images` and `test_labels`. Our images are encoded as Numpy arrays, and the labels are simply an array of digits, ranging \n",
+    "from 0 to 9. There is a one-to-one correspondence between the images and the labels.\n",
+    "\n",
+    "Let's have a look at the training data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(60000, 28, 28, 1)"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "train_images.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "60000"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(train_labels)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "train_labels"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(10000, 28, 28, 1)"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_images.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "10000"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(test_labels)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([7, 2, 1, ..., 4, 5, 6], dtype=uint8)"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_labels"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Our workflow will be as follow: first we will present our neural network with the training data, train_images and train_labels. The network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for test_images, and we will verify if these predictions match the labels from test_labels.\n",
+    "Our workflow will be as follow: first we will present our neural network with the training data, `train_images` and `train_labels`. The \n",
+    "network will then learn to associate images and labels. Finally, we will ask the network to produce predictions for `test_images`, and we \n",
+    "will verify if these predictions match the labels from `test_labels`.\n",
     "\n",
     "Let's build our network -- again, remember that you aren't supposed to understand everything about this example just yet."
    ]
@@ -67,17 +222,17 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Module import (for previous Keras user)\n",
-    "Import the modules we need to build the network. In Keras it is:\n",
+    "#### Module import\n",
+    "<font color='red'>In Keras you could use following code to import the modules we need to build the network:</font>\n",
     "\n",
     "    from keras import models\n",
     "    from keras import layers\n",
-    "Just replace it with following in order to use analytics-zoo:"
+    "<font color='red'>Just replace it with following in analytics-zoo:</font>"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 9,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -87,9 +242,29 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 10,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<zoo.pipeline.api.keras.models.Sequential at 0x7f113870cdd8>"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
    "source": [
     "network = models.Sequential()\n",
     "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
@@ -123,9 +298,19 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 11,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createRMSprop\n",
+      "creating: createZooKerasSparseCategoricalCrossEntropy\n",
+      "creating: createZooKerasAccuracy\n"
+     ]
+    }
+   ],
    "source": [
     "network.compile(optimizer='rmsprop',\n",
     "                loss='sparse_categorical_crossentropy',\n",
@@ -143,7 +328,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 12,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -164,7 +349,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 13,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -175,14 +360,16 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556."
+    "Blue messages below is the last INFO of training, you can find full training process in INFO, which outputs in your terminal or IDE\n",
+    "\n",
+    "<font color='blue'>INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556.</font>"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Evaluate return (for previous Keras user)\n",
+    "#### Evaluate return (for previous Keras user) pending for code\n",
     "Check our result on test set. In Keras it is:\n",
     "\n",
     "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
@@ -191,21 +378,22 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 14,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "test_acc: 0.9771000146865845\n"
+     ]
+    }
+   ],
    "source": [
     "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
     "print('test_acc:', test_result[0].result)"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "test_acc: 0.9783999919891357"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},

From 390d1125c4db4257f0f239e2a6ba7d46fdf0b848 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 12 Mar 2019 16:07:17 +0800
Subject: [PATCH 19/46] Add files via upload

---
 keras/2.1-a-first-look-at-a-neural-network.ipynb | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/keras/2.1-a-first-look-at-a-neural-network.ipynb b/keras/2.1-a-first-look-at-a-neural-network.ipynb
index c281ede..41bcf51 100644
--- a/keras/2.1-a-first-look-at-a-neural-network.ipynb
+++ b/keras/2.1-a-first-look-at-a-neural-network.ipynb
@@ -257,7 +257,7 @@
     {
      "data": {
       "text/plain": [
-       "<zoo.pipeline.api.keras.models.Sequential at 0x7f113870cdd8>"
+       "<zoo.pipeline.api.keras.models.Sequential at 0x7fa294f49d30>"
       ]
      },
      "execution_count": 10,
@@ -369,11 +369,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Evaluate return (for previous Keras user) pending for code\n",
-    "Check our result on test set. In Keras it is:\n",
-    "\n",
-    "    test_loss, test_acc = network.evaluate(test_images, test_labels)\n",
-    "In analytics-zoo, the return of `evaluate` method is an `EvaluationResult` object, which is different from Keras. We use following code to check:"
+    "We quickly reach an accuracy of 0.989 (i.e. 98.9%) on the training data. Now let's check that our model performs well on the test set too:"
    ]
   },
   {
@@ -385,13 +381,13 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "test_acc: 0.9771000146865845\n"
+      "test_acc: 0.9797000288963318\n"
      ]
     }
    ],
    "source": [
-    "test_result = network.evaluate(test_images, test_labels, batch_size=32)\n",
-    "print('test_acc:', test_result[0].result)"
+    "test_loss, test_acc = network.evaluate(test_images, test_labels, batch_size=32)\n",
+    "print('test_acc:', test_acc)"
    ]
   },
   {

From e3b1b5cd30dbfd15ae37f4be8394770d9eb91ad2 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 12 Mar 2019 16:11:29 +0800
Subject: [PATCH 20/46] Add files via upload

---
 keras/2.1-a-first-look-at-a-neural-network.ipynb | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/keras/2.1-a-first-look-at-a-neural-network.ipynb b/keras/2.1-a-first-look-at-a-neural-network.ipynb
index 41bcf51..81f35ef 100644
--- a/keras/2.1-a-first-look-at-a-neural-network.ipynb
+++ b/keras/2.1-a-first-look-at-a-neural-network.ipynb
@@ -49,10 +49,10 @@
    "metadata": {},
    "source": [
     "#### Datasets import\n",
-    "<font color='red'>In Keras you could use following code to import the datasets:</font>\n",
+    "_In Keras one could use following code to import the datasets:_\n",
     "\n",
     "    from keras.datasets import mnist\n",
-    "<font color='red'>Just replace it with following in analytics-zoo:</font>"
+    "_Just replace it with following in analytics-zoo:_"
    ]
   },
   {
@@ -223,11 +223,11 @@
    "metadata": {},
    "source": [
     "#### Module import\n",
-    "<font color='red'>In Keras you could use following code to import the modules we need to build the network:</font>\n",
+    "_In Keras one could use following code to import the modules we need to build the network:_\n",
     "\n",
     "    from keras import models\n",
     "    from keras import layers\n",
-    "<font color='red'>Just replace it with following in analytics-zoo:</font>"
+    "_Just replace it with following in analytics-zoo:_"
    ]
   },
   {

From 201c4abadf45846673adead985252848355f0513 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 12 Mar 2019 16:15:26 +0800
Subject: [PATCH 21/46] Add files via upload

---
 keras/2.1-a-first-look-at-a-neural-network.ipynb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/keras/2.1-a-first-look-at-a-neural-network.ipynb b/keras/2.1-a-first-look-at-a-neural-network.ipynb
index 81f35ef..d022670 100644
--- a/keras/2.1-a-first-look-at-a-neural-network.ipynb
+++ b/keras/2.1-a-first-look-at-a-neural-network.ipynb
@@ -360,9 +360,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Blue messages below is the last INFO of training, you can find full training process in INFO, which outputs in your terminal or IDE\n",
+    "Messages below is the last INFO of training, you can find full training process in INFO, which outputs in your terminal or IDE (not the output of the program)\n",
     "\n",
-    "<font color='blue'>INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556.</font>"
+    "_INFO - Trained 128 records in 0.018066358 seconds. Throughput is 7084.992 records/second. Loss is 0.012087556._"
    ]
   },
   {

From 09f6bc7053c5de375d30371443cbb2db48022b64 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 10:42:27 +0800
Subject: [PATCH 22/46] Add files via upload

---
 keras/3.5-binary-classification.ipynb | 1686 +++++++++++++++++++++++++
 1 file changed, 1686 insertions(+)
 create mode 100644 keras/3.5-binary-classification.ipynb

diff --git a/keras/3.5-binary-classification.ipynb b/keras/3.5-binary-classification.ipynb
new file mode 100644
index 0000000..93ac869
--- /dev/null
+++ b/keras/3.5-binary-classification.ipynb
@@ -0,0 +1,1686 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**First of all, set environment variables and initialize spark context:**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=32g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=32g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Classifying movie reviews: a binary classification example\n",
+    "This notebook is imported from Chapter 3, Section 5 of [Deep Learning with Python Notebook]().\n",
+    "\n",
+    "----\n",
+    "\n",
+    "Two-class classification, or binary classification, may be the most widely applied kind of machine learning problem. In this example, we \n",
+    "will learn to classify movie reviews into \"positive\" reviews and \"negative\" reviews, just based on the text content of the reviews."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The IMDB dataset\n",
+    "We'll be working with \"IMDB dataset\", a set of 50,000 highly-polarized reviews from the Internet Movie Database. They are split into 25,000 \n",
+    "reviews for training and 25,000 reviews for testing, each set consisting in 50% negative and 50% positive reviews.\n",
+    "\n",
+    "Why do we have these two separate training and test sets? You should never test a machine learning model on the same data that you used to \n",
+    "train it! Just because a model performs well on its training data doesn't mean that it will perform well on data it has never seen, and \n",
+    "what you actually care about is your model's performance on new data (since you already know the labels of your training data -- obviously \n",
+    "you don't need your model to predict those). For instance, it is possible that your model could end up merely _memorizing_ a mapping between \n",
+    "your training samples and their targets -- which would be completely useless for the task of predicting targets for data never seen before. \n",
+    "We will go over this point in much more detail in the next chapter.\n",
+    "\n",
+    "Just like the MNIST dataset, the IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words) \n",
+    "have been turned into sequences of integers, where each integer stands for a specific word in a dictionary.\n",
+    "\n",
+    "The following code will load the dataset (when you run it for the first time, about 80MB of data will be downloaded to your machine):"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Using TensorFlow backend.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from keras.datasets import imdb\n",
+    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The argument `nb_words=10000` means that we will only keep the top 10,000 most frequently occurring words in the training data. Rare words \n",
+    "will be discarded. This allows us to work with vector data of manageable size.\n",
+    "\n",
+    "The variables `train_data` and `test_data` are lists of reviews, each review being a list of word indices (encoding a sequence of words). \n",
+    "`train_labels` and `test_labels` are lists of 0s and 1s, where 0 stands for \"negative\" and 1 stands for \"positive\":"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Since we restricted ourselves to the top 10,000 most frequent words, no word index will exceed 10,000:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "9999"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "max([max(sequence) for sequence in train_data])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For kicks, here's how you can quickly decode one of these reviews back to English words:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# word_index is a dictionary mapping words to an integer index\n",
+    "word_index = imdb.get_word_index()\n",
+    "# We reverse it, mapping integer indices to words\n",
+    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
+    "# We decode the review; note that our indices were offset by 3\n",
+    "# because 0, 1 and 2 are reserved indices for \"padding\", \"start of sequence\", and \"unknown\".\n",
+    "decoded_review = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])\n",
+    "decoded_review"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We cannot feed lists of integers into a neural network. We have to turn our lists into tensors. There are two ways we could do that:\n",
+    "\n",
+    "* We could pad our lists so that they all have the same length, and turn them into an integer tensor of shape `(samples, word_indices)`, \n",
+    "then use as first layer in our network a layer capable of handling such integer tensors (the `Embedding` layer, which we will cover in \n",
+    "detail later in the book).\n",
+    "* We could one-hot-encode our lists to turn them into vectors of 0s and 1s. Concretely, this would mean for instance turning the sequence \n",
+    "`[3, 5]` into a 10,000-dimensional vector that would be all-zeros except for indices 3 and 5, which would be ones. Then we could use as \n",
+    "first layer in our network a `Dense` layer, capable of handling floating point vector data.\n",
+    "\n",
+    "We will go with the latter solution. Let's vectorize our data, which we will do manually for maximum clarity:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "def vectorize_sequences(sequences, dimension=10000):\n",
+    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
+    "    results = np.zeros((len(sequences), dimension))\n",
+    "    for i, sequence in enumerate(sequences):\n",
+    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
+    "    return results\n",
+    "\n",
+    "# Our vectorized training data\n",
+    "x_train = vectorize_sequences(train_data)\n",
+    "# Our vectorized test data\n",
+    "x_test = vectorize_sequences(test_data)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here's what our samples look like now:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([0., 1., 1., ..., 0., 0., 0.])"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "x_train[0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We should also vectorize our labels, which is straightforward:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "y_train = np.asarray(train_labels).astype('float32')\n",
+    "y_test = np.asarray(test_labels).astype('float32')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now our data is ready to be fed into a neural network."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Building our network\n",
+    "\n",
+    "\n",
+    "Our input data is simply vectors, and our labels are scalars (1s and 0s): this is the easiest setup you will ever encounter. A type of \n",
+    "network that performs well on such a problem would be a simple stack of fully-connected (`Dense`) layers with `relu` activations: `Dense(16, activation='relu')`\n",
+    "\n",
+    "The argument being passed to each `Dense` layer (16) is the number of \"hidden units\" of the layer. What's a hidden unit? It's a dimension \n",
+    "in the representation space of the layer. You may remember from the previous chapter that each such `Dense` layer with a `relu` activation implements \n",
+    "the following chain of tensor operations:\n",
+    "\n",
+    "`output = relu(dot(W, input) + b)`\n",
+    "\n",
+    "Having 16 hidden units means that the weight matrix `W` will have shape `(input_dimension, 16)`, i.e. the dot product with `W` will project the \n",
+    "input data onto a 16-dimensional representation space (and then we would add the bias vector `b` and apply the `relu` operation). You can \n",
+    "intuitively understand the dimensionality of your representation space as \"how much freedom you are allowing the network to have when \n",
+    "learning internal representations\". Having more hidden units (a higher-dimensional representation space) allows your network to learn more \n",
+    "complex representations, but it makes your network more computationally expensive and may lead to learning unwanted patterns (patterns that \n",
+    "will improve performance on the training data but not on the test data).\n",
+    "\n",
+    "There are two key architecture decisions to be made about such stack of dense layers:\n",
+    "\n",
+    "* How many layers to use.\n",
+    "* How many \"hidden units\" to chose for each layer.\n",
+    "\n",
+    "In the next chapter, you will learn formal principles to guide you in making these choices. \n",
+    "For the time being, you will have to trust us with the following architecture choice: \n",
+    "two intermediate layers with 16 hidden units each, \n",
+    "and a third layer which will output the scalar prediction regarding the sentiment of the current review. \n",
+    "The intermediate layers will use `relu` as their \"activation function\", \n",
+    "and the final layer will use a sigmoid activation so as to output a probability \n",
+    "(a score between 0 and 1, indicating how likely the sample is to have the target \"1\", i.e. how likely the review is to be positive). \n",
+    "A `relu` (rectified linear unit) is a function meant to zero-out negative values, \n",
+    "while a sigmoid \"squashes\" arbitrary values into the `[0, 1]` interval, thus outputting something that can be interpreted as a probability."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here's what our network looks like:\n",
+    "\n",
+    "![3-layer network](https://s3.amazonaws.com/book.keras.io/img/ch3/3_layer_network.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "And here's the analytics-zoo implementation, very similar to the MNIST example you saw previously:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<zoo.pipeline.api.keras.models.Sequential at 0x7f50e074cfd0>"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "model = models.Sequential()\n",
+    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "model.add(layers.Dense(16, activation='relu'))\n",
+    "model.add(layers.Dense(1, activation='sigmoid'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Lastly, we need to pick a loss function and an optimizer. Since we are facing a binary classification problem and the output of our network \n",
+    "is a probability (we end our network with a single-unit layer with a sigmoid activation), is it best to use the `binary_crossentropy` loss. \n",
+    "It isn't the only viable choice: you could use, for instance, `mean_squared_error`. But crossentropy is usually the best choice when you \n",
+    "are dealing with models that output probabilities. Crossentropy is a quantity from the field of Information Theory, that measures the \"distance\" \n",
+    "between probability distributions, or in our case, between the ground-truth distribution and our predictions.\n",
+    "\n",
+    "Here's the step where we configure our model with the `rmsprop` optimizer and the `binary_crossentropy` loss function. Note that we will \n",
+    "also monitor accuracy during training."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='binary_crossentropy',\n",
+    "              metrics=['accuracy'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Validating our approach\n",
+    "\n",
+    "In order to monitor during training the accuracy of the model on data that it has never seen before, we will create a \"validation set\" by \n",
+    "setting apart 10,000 samples from the original training data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "x_val = x_train[:10000]\n",
+    "partial_x_train = x_train[10000:]\n",
+    "y_val = y_train[:10000]\n",
+    "partial_y_train = y_train[10000:]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We will now train our model for 20 epochs (20 iterations over all samples in the `x_train` and `y_train` tensors), in mini-batches of 512 \n",
+    "samples. At this same time we will monitor loss and accuracy on the 10,000 samples that we set apart. This is done by passing the \n",
+    "validation data as the `validation_data` argument:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Accuracy checkout\n",
+    "_To checkout the behavior of this model in Keras, one could use following code accompanied with `matplotlib` library to draw the following `history` object_\n",
+    "    \n",
+    "    history = model.fit(partial_x_train,\n",
+    "                    partial_y_train,\n",
+    "                    nb_epoch=5,\n",
+    "                    batch_size=512,\n",
+    "                    validation_data=(x_val, y_val)\n",
+    "                    )\n",
+    "_After `fit` method finishes, the results are stored in `history` and thus could be visualized. Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via setting tensorboard._"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To do training visualization, you can configure tensorboard in the model. The code of setting tensorboard and train is following, note that `set_tesnsorboard` need to be called before `fit` method:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.set_tensorboard('./', '3-5_summary')\n",
+    "model.fit(partial_x_train,\n",
+    "          partial_y_train,\n",
+    "          nb_epoch=20,\n",
+    "          batch_size=512,\n",
+    "          validation_data=(x_val, y_val))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then result could be visualized in either of following ways: \n",
+    "\n",
+    "* Start tensorboard web interface in terminal by `tensorboard --logdir ./` and go to web browser url `localhost:port_number` as shown in your terminal.\n",
+    "* Use analytics-zoo built-in method `get_scalar_from_summary` with parameter `Loss` or `Validation` to get the array of scalar, then visualize via `matplotlib`.\n",
+    "\n",
+    "We use the second approach here in order to directly show the result in this notebook."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd4VNXWwOHfSiP0jnQCivQeitKbUhRFuqKCCoIitqtiASuC5VOuiihcsaCAgFIkVOkgAqH3HiChhQAhlJC2vz9mMkySSTIJM5mU9T4Pj3PO2efMCpFZs7sYY1BKKaUAvDwdgFJKqexDk4JSSikbTQpKKaVsNCkopZSy0aSglFLKRpOCUkopG00KyqVExFtEropIZVeW9SQRuUtEXD52W0Q6iUiI3fFBEWntTNlMvNf/ROStzN6fxnM/EpGfXP1c5Tk+ng5AeZaIXLU7LADcBOKtx88aY37LyPOMMfFAIVeXzQuMMTVc8RwReQYYaIxpZ/fsZ1zxbJX7aVLI44wxtg9l6zfRZ4wxf6dWXkR8jDFxWRGbUirrafORSpO1eeB3EZkhIlHAQBG5R0T+FZHLInJGRL4SEV9reR8RMSISYD3+1Xp9sYhEichGEama0bLW611F5JCIRIrI1yKyQUQGpRK3MzE+KyJHROSSiHxld6+3iHwpIhEicgzoksbfz9siMjPZuYki8oX19TMist/68xy1fotP7VmhItLO+rqAiEyzxrYXaJKs7Dsicsz63L0i0sN6vh7wDdDa2jR3we7v9j27+4dZf/YIEZknIuWc+btJj4j0tMZzWURWikgNu2tvichpEbkiIgfsftYWIrLNev6ciHzm7PspNzDG6B/9gzEGIATolOzcR0AM8CCWLxH5gaZAcyw1zWrAIWCEtbwPYIAA6/GvwAUgEPAFfgd+zUTZMkAU8JD12itALDAolZ/FmRjnA0WBAOBi4s8OjAD2AhWBksBayz8Vh+9TDbgKFLR79nkg0Hr8oLWMAB2AG0B967VOQIjds0KBdtbXnwOrgeJAFWBfsrJ9gXLW38mj1hjusF57BlidLM5fgfesr++zxtgQ8Ae+BVY683fj4Of/CPjJ+rqWNY4O1t/RW8BB6+s6wAmgrLVsVaCa9fUWYID1dWGguaf/LeTlP1pTUM5Yb4z5yxiTYIy5YYzZYozZZIyJM8YcAyYDbdO4f44xJtgYEwv8huXDKKNlHwB2GGPmW699iSWBOORkjOOMMZHGmBAsH8CJ79UX+NIYE2qMiQDGp/E+x4A9WJIVQGfgkjEm2Hr9L2PMMWOxElgBOOxMTqYv8JEx5pIx5gSWb//27zvLGHPG+juZjiWhBzrxXIDHgP8ZY3YYY6KBUUBbEaloVya1v5u09AcWGGNWWn9H47EkluZAHJYEVMfaBHnc+ncHluReXURKGmOijDGbnPw5lBtoUlDOOGV/ICI1RSRIRM6KyBXgA6BUGveftXt9nbQ7l1MrW94+DmOMwfLN2iEnY3TqvbB8w03LdGCA9fWj1uPEOB4QkU0iclFELmP5lp7W31WicmnFICKDRGSntZnmMlDTyeeC5eezPc8YcwW4BFSwK5OR31lqz03A8juqYIw5CLyK5fdw3tocWdZadDBQGzgoIptFpJuTP4dyA00KyhnJh2N+j+Xb8V3GmCLAGCzNI+50BktzDgAiIiT9EEvudmI8A1SyO05vyOwsoJOIVMBSY5hujTE/MAcYh6VppxiwzMk4zqYWg4hUAyYBw4GS1ucesHtuesNnT2Npkkp8XmEszVRhTsSVked6YfmdhQEYY341xrTE0nTkjeXvBWPMQWNMfyxNhP8H/CEi/rcZi8okTQoqMwoDkcA1EakFPJsF77kQaCwiD4qID/AiUNpNMc4CXhKRCiJSEngjrcLGmLPAeuAn4KAx5rD1Uj7ADwgH4kXkAaBjBmJ4S0SKiWUexwi7a4WwfPCHY8mPQ7DUFBKdAyomdqw7MAN4WkTqi0g+LB/O64wxqda8MhBzDxFpZ33v17D0A20SkVoi0t76fjesfxKw/ACPi0gpa80i0vqzJdxmLCqTNCmozHgVeBLLP/jvsXQIu5Ux5hzQD/gCiADuBLZjmVfh6hgnYWn7342lE3SOE/dMx9JxbGs6MsZcBl4G5mLprO2NJbk5410sNZYQYDHwi91zdwFfA5utZWoA9u3wy4HDwDkRsW8GSrx/CZZmnLnW+ytj6We4LcaYvVj+zidhSVhdgB7W/oV8wKdY+oHOYqmZvG29tRuwXyyj2z4H+hljYm43HpU5YmmaVSpnERFvLM0VvY0x6zwdj1K5hdYUVI4hIl2szSn5gNFYRq1s9nBYSuUqmhRUTtIKOIalaeJ+oKcxJrXmI6VUJmjzkVJKKRutKSillLLJcQvilSpVygQEBHg6DKWUylG2bt16wRiT1jBuIAcmhYCAAIKDgz0dhlJK5Sgikt7MfECbj5RSStnRpKCUUspGk4JSSimbHNenoHKH2NhYQkNDiY6O9nQoKg3+/v5UrFgRX9/UllFSuY0mBeURoaGhFC5cmICAACwLnqrsxhhDREQEoaGhVK1aNf0bVK6gzUfKI6KjoylZsqQmhGxMRChZsqTW5vIYtyYF61o1B617vY5ycL2yiKwSke0isks318hbNCFkf/o7ynvclhSsq1hOBLpi2VVpgIjUTlbsHWCWMaYRlq38vnVXPEop5UkXb1yk0y+dyO5LC7mzptAMOGLdnzYGmMmtfWwTGaCI9XVRLEshK+V2ly9f5ttvM/cdpFu3bly+fNnp8u+99x6ff/55pt5L5R4jF49kxfEVBB0OytT9a0+s5Wac+9d/dGdSqEDSPWZDSbl94nvAQBEJBRYBLzh6kIgMFZFgEQkODw93R6wqj0krKcTFxaV576JFiyhWrJg7wlK5WHRcdJL/ZsSpyFO0/aktj/75qKvDSsHTHc0DgJ+MMRWx7L40zbqvaxLGmMnGmEBjTGDp0uku3aFUukaNGsXRo0dp2LAhr732GqtXr6Z169b06NGD2rUtrZwPP/wwTZo0oU6dOkyePNl2b0BAABcuXCAkJIRatWoxZMgQ6tSpw3333ceNGzfSfN8dO3bQokUL6tevT8+ePbl06RIAX331FbVr16Z+/fr0798fgDVr1tCwYUMaNmxIo0aNiIqKctPfhsruIm5EAHD04lG3v5c7h6SGkXTjcdsG3naexrJlH8aYjdbNuksB590Yl8pm3v9rL/tOX3HpM2uXL8K7D9ZJ9fr48ePZs2cPO3bsAGD16tVs27aNPXv22IZfTp06lRIlSnDjxg2aNm1Kr169KFmyZJLnHD58mBkzZjBlyhT69u3LH3/8wcCBA1N93yeeeIKvv/6atm3bMmbMGN5//30mTJjA+PHjOX78OPny5bM1TX3++edMnDiRli1bcvXqVfz9dS975X7urClsAaqLSFUR8cPSkbwgWZmTWDcyt26u7o9lAxWlslyzZs2SjMf/6quvaNCgAS1atODUqVMcPnw4xT1Vq1alYcOGADRp0oSQkJBUnx8ZGcnly5dp27YtAE8++SRr164FoH79+jz22GP8+uuv+PhYvqu1bNmSV155ha+++orLly/bzivlTm77v8wYEyciI4ClgDcw1RizV0Q+AIKNMQuwbK4+RURextLpPMhk96555XJpfaPPSgULFrS9Xr16NX///TcbN26kQIECtGvXzuF4/Xz58tlee3t7p9t8lJqgoCDWrl3LX3/9xdixY9m9ezejRo2ie/fuLFq0iJYtW7J06VJq1qyZqecr5Sy3fvUwxizC0oFsf26M3et9QEt3xqCUI4ULF06zjT4yMpLixYtToEABDhw4wL///nvb71m0aFGKFy/OunXraN26NdOmTaNt27YkJCRw6tQp2rdvT6tWrZg5cyZXr14lIiKCevXqUa9ePbZs2cKBAwc0KSi30/qoypNKlixJy5YtqVu3Ll27dqV79+5Jrnfp0oXvvvuOWrVqUaNGDVq0aOGS9/35558ZNmwY169fp1q1avz444/Ex8czcOBAIiMjMcYwcuRIihUrxujRo1m1ahVeXl7UqVOHrl27uiQGpdKS4/ZoDgwMNLrJTs63f/9+atWq5ekwlBP0d+UavWf15o/9fzC7z2x61+6doXt3nN1Bo+8b0eCOBuwYtiNT7y8iW40xgemV8/SQVKWUUtmIJgWllFI2mhSUUkrZaFJQSillo0lBKaWUjSYFpZRSNpoUlHJSoUKFADh9+jS9ezseUtiuXTvSGzI9YcIErl+/bjvO6FLcqdElupUraFJQKoPKly/PnDlzMn1/8qSgS3Gr7ESTgsqTRo0axcSJE23Hid+yr169SseOHWncuDH16tVj/vz5Ke4NCQmhbt26ANy4cYP+/ftTq1YtevbsmWTto+HDhxMYGEidOnV49913Acsie6dPn6Z9+/a0b98euLUUN8AXX3xB3bp1qVu3LhMmTLC9ny7RrbKKLnOhPO6lJS+x42zmZmmmpmHZhkzoMiHV6/369eOll17i+eefB2DWrFksXboUf39/5s6dS5EiRbhw4QItWrSgR48eqe5VPGnSJAoUKMD+/fvZtWsXjRs3tl0bO3YsJUqUID4+no4dO7Jr1y5GjhzJF198wapVqyhVqlSSZ23dupUff/yRTZs2YYyhefPmtG3bluLFi+sS3SrLaE1B5UmNGjXi/PnznD59mp07d1K8eHEqVaqEMYa33nqL+vXr06lTJ8LCwjh37lyqz1m7dq3tw7l+/frUr1/fdm3WrFk0btyYRo0asXfvXvbt25dmTOvXr6dnz54ULFiQQoUK8cgjj7Bu3TpAl+hWWUd/+8rj0vpG7059+vRhzpw5nD17ln79+gHw22+/ER4eztatW/H19SUgIMDhktnpOX78OJ9//jlbtmyhePHiDBo0KFPPSaRLdKusojUFlWf169ePmTNnMmfOHPr06QNYvmWXKVMGX19fVq1axYkTJ9J8Rps2bZg+fToAe/bsYdeuXQBcuXKFggULUrRoUc6dO8fixYtt96S2bHfr1q2ZN28e169f59q1a8ydO5fWrVtn+OeyX6IbcLhE9yeffEJkZCRXr17l6NGj1KtXjzfeeIOmTZty4MCBDL+ncq/DEZYNnny83P89XmsKKs+qU6cOUVFRVKhQgXLlygHw2GOP8eCDD1KvXj0CAwPT/cY8fPhwBg8eTK1atahVqxZNmjQBoEGDBjRq1IiaNWtSqVIlWra8tW3I0KFD6dKlC+XLl2fVqlW2840bN2bQoEE0a9YMgGeeeYZGjRql2VSUGl2iO/c4E3WG/n9YBgU8XPNht7+fW5fOFpEuwH+x7Lz2P2PM+GTXvwTaWw8LAGWMMWmOzdOls3MHXY4559DflWtkZuns6Lho2v3Ujk1hmwCY1H0SwwKHZer9Pb50toh4AxOBrkBtYICI1LYvY4x52RjT0BjTEPga+NNd8SilVE5ijGHYwmFsCtvExG4T07/BRdzZp9AMOGKMOWaMiQFmAg+lUX4AMMON8SilVI7x5b9f8vPOn3mv7Xs8UuuRLHtfdyaFCsApu+NQ67kURKQKUBVYmcr1oSISLCLB4eHhmQrmw4X7CBgVRGx8QqbuV66X03b9y4v0d+QZS48s5bXlr9GrVi9Gtx2dpe+dXUYf9QfmGGPiHV00xkw2xgQaYwJLly6dqTf4Yf1xAI5fuJbpIJXr+Pv7ExERoR862ZgxhoiICJ3MlsUORRyi35x+1CtTj58f/hkvydqPaXeOPgoDKtkdV7Sec6Q/8LwbY7H5fOlBJj+Rbl+LcrOKFSsSGhpKZmt+Kmv4+/tTsWJFT4eRZ1yOvkyPGT3w9fZlfv/5FPQrmOUxuDMpbAGqi0hVLMmgP/Bo8kIiUhMoDmx0Yywsf7kNnb9cy7J9qc9OVVnH19eXqlWrejoMpbKN+IR4BvwxgKOXjrLiiRVUKVbFI3G4rV5ijIkDRgBLgf3ALGPMXhH5QER62BXtD8w0bm5HuKtMIXc+XimlbsubK95kyZElTOw2kTZV2ngsDrdOXjPGLAIWJTs3Jtnxe+6MIZH9gmYXr8VQoqBfVrytUkoBcPbqWQASTMrBLtN2TuOzfz7jucDnGNpkaFaHlkR26WjOUjdiHfZnK6WUW4RdCWPDqQ2ApSPZ3uawzQz5awjtAtp5bB0we3kyKbQcv5LAj5Z7OgylVB5wKvIUbX9qazu2rymcjjrNwzMfpnzh8szuMxtfb1+HzyjgW4Dnmz5PndJ13B5vnlr7qHThfIRH3QTgwtUYD0ejlMrtTlw+Qfuf2xNxIyLFtei4aHr+3pOomCiWPb6MUgVKOXiCRZF8Rfim2zfuDNUmT9UUEhOCUkq52/FLx2n7U1su3rjI8seTtkwYYxj611A2h23m156/UrdMXQ9FmVKeqikopVRWOHrxKB1+6UDUzShWPLGCJuWbJLn+fxv/j2m7pvFh+w95qGZaq/9kvTyVFIrm9yXyRqynw1BK5WKHIw7T4ZcOXI+9zoonVtCoXKMk1xcfWcym0E30qd2Ht1u/7aEoU5enmo8CShbwdAhKqVzs4IWDtPu5HdFx0ax8YmWKhADwb+i/NCjbgB8f+jHVvb89KU8lhV+eau7pEJRSudT+8P20+7kdsfGxrHpyFQ3KNki1rKeWsHBGnkoKRQs4Hu6llFK3Y+/5vbT7uR3GGFYPWu2w47hCYcsi0S82f5HKRStndYhOy1NJQSmlbocxhviEpJNfd53bRbuf2+Et3qwetJrapWs7vLdFxRYAtKrcyu1x3g5NCkop5aSSn5bE58Nb43O2n9lO+5/bk887H2sGraFmqbT39M4JNCkopfKUcevGMX339Ezdeyn6ku311tNb6fhLRwr6FmTNoDVUL1ndVSF6VJ4akqqUUm+tfAuAR+ulWMk/TVdjrtpebw7bzH3T7qOYfzFWPbmKqsVzzzLwebqmcD0mztMhKKVygH3h+2g2pZntuPO0zpTIX4LVg1Y7nRA6Vu0IQL0y9dwSo6vk6aRw4GyUp0NQSmVzv+36jaZTmiZZv6h0gdKsHrSagGIBTj9neNPhmHcNNUrVcEOUruPWpCAiXUTkoIgcEZFRqZTpKyL7RGSviGSuoS8DmlUtYXt9+bouiqeUciw6LpphC4cxcO5AmpRrwvZnt9uurR60OlsPK70dbutTEBFvYCLQGQgFtojIAmPMPrsy1YE3gZbGmEsiUsZd8SS6GXdr2drQSzfc/XZKqRzo2KVj9Jndh21ntvFGyzf4qMNH+Hj5ULVYVY5fPk7FIrl332p3djQ3A44YY44BiMhM4CFgn12ZIcBEY8wlAGPMeTfGA0CLqiXYeeoyAFdvap+CUiqp+Qfm8+S8JxER5vefT48at3YPPvbiMQ9GljXc2XxUAThldxxqPWfvbuBuEdkgIv+KSBc3xgNAnQpFba+vaVJQSlnFxsfy+vLXefj3h7mrxF1sG7otSULIKzzd0ewDVAfaAQOAKSJSLHkhERkqIsEiEhweHn5bb1jQz9v2euKqo7f1LKVU7nA66jQdfunAZ/98xvDA4ax/an2uGmaaEe5MCmFAJbvjitZz9kKBBcaYWGPMceAQliSRhDFmsjEm0BgTWLp06dsKqoBf0hazaN2vWak8Y/3J9SnOrTi2gkbfN2LbmW389shvfNv9W/x9/D0QXfbgzqSwBaguIlVFxA/oDyxIVmYelloCIlIKS3OSWxvtCubzTnJcc/QSd76dUiqbmLJ1Ch1+7gBY5gokmAQ+WvsRnad1pmT+kmwZsiXDE9pyI7d1NBtj4kRkBLAU8AamGmP2isgHQLAxZoH12n0isg+IB14zxqTczNSFktcUlFK5W2x8LC8vfZmJWybS5a4u7Dm/h5IFStJ9eneWHFnCY/Ue47sHvqOQXyFPh5otuPUT0hizCFiU7NwYu9cGeMX6J0skrykopXKv8Gvh9JndhzUn1vDava8xruM4Gn7fkNUhq/Hz9mNS90k82+TZbLnZjafkua/NWlNQKm/YeXYnD818iLNXzzKt5zQG1h8IQHH/4lQtVpXZfWan2DtZ5cmkoDUFpXK7P/b9wRPznqCYfzHWDV5H0wpNbdcWProQP2+/PN2ZnBZPD0nNcr7eee5HVirPSDAJjFk1ht6ze1P/jvoEDwlOkhAAiuQrogkhDfoJqZTKUYwx7A/fT2x8bJLzUTej6DWrFx+u/ZDBDQez+snVlCtczkNR5lx5MimUL6rfEpTKaUKvhDJ27ViqfVWN2t/WpsfMW7ONj148yj0/3MNfB/9iwv0T+KHHD+TzyefBaHOuPNenALD8lbbUeXep7fh6TJx2QCuVDd2Mu8mCgwuYumMqS48sxWC4o+AdACw5YpljtOLYCvrO6YsxhiUDl9CpWidPhpzj5cmaQsF8SRPAzlORHopEKeXIjrM7GLl4JOW/KE/fOX3Ze34v77R5h6MjjxL0aJCt3FebvuL+X++nXKFybBmyRROCC+jXY6VUtnDxxkWm757O1O1T2X52O37efvSs2ZOnGj1Fx6od8fayjBzcemOr7Z4Xl7zIQzUeYlrPaRTOV9hToecqmhSUUh4TnxDPiuMrmLp9KnMPzCUmPobG5RrzTddvGFBvACXyl0jz/tFtRvNeu/fwkjzZ6OEWmhSUUlnuRuwNxq8fz487fuTUlVOUyF+CYU2GMbjRYBqWbZjmvUX9by1//0H7D9wdap6jSQEYMOVfjo/rplPdlcoiK46v4IO1H9A+oD3/d9//0aNGD6dHCyXWCqoWy5tLW7tbnq1zVSiWP8nxhiMRXLqmezYr5U6HIg7x8bqPeXDGgwC8du9r9KnTJ0PDR8sXLg/AM42fcUuMeV2erSlsGNWBgFG3RjF8t+Yo649cIGhkK+qUL5rGnUqpjNgfvp85++Ywe99sdp/fneTauWvnMvw8fx9/zLvGVeGpZPJsTSG59UcuALDt5GUPR6JU1rkee50rN6+49JnGGPac38N7q9+j7rd1qf1tbd5d/S5F/Ysy4f4JnHzpJE82eNKl76lcJ8/WFFIzet4eHm9RxdNhKJUles3qxYpjK+hUrRN9avfhoZoPpTvixxFjDLvP72b23tnM2T+HAxcOIAhtqrTh665f80itR2zNPip706SgVB4Wfi2ckgVKsv/Cfp5a8BQ+C33oWLUjvWv35uGaD1OqQKlU7zXGsOPsDmbvm82cfXM4fPEwXuJFu4B2jGw2kp61elK2UNks/GmUK7g1KYhIF+C/WHZe+58xZnyy64OAz7i1d/M3xpj/uTMmpVRSTco14a8Bf7H1zFZb2/+Qv4YwbOEw2ldtT5/afehZsyelCybdH33CvxN4ZdkreIs3Hap24D/3/oeHaz5MmYJlPPSTKFdwW1IQEW9gItAZCAW2iMgCY8y+ZEV/N8aMcFccSqn0iQiB5QMJLB/IuI7jbDWA2ftm8+zCZxkeNJx2Ae3oXas3j9R6hDsK3cGpK6fI75Ofky+fTLNGoXIWd9YUmgFHjDHHAERkJvAQkDwpKKWyERGhUblGNCrXiLEdxrLr3C5bDeK5Rc/x/KLnaVOlDRtDNxITH6MJIZdx5+ijCsApu+NQ67nkeonILhGZIyKVHD1IRIaKSLCIBIeHh7sswFnP3uOyZymVG4kIDco24MMOH7L/+f3sHr6b0W1GE349nJh4ndeTG3l6SOpfQIAxpj6wHPjZUSFjzGRjTKAxJrB06dKOimTKnaULuuxZSuV2IkLdMnV5v/377H1ur6fDUW7izqQQBth/86/IrQ5lAIwxEcaYm9bD/wFZuot28iW0lVJZo2WllgDUv6O+hyNRybkzKWwBqotIVRHxA/oDC+wLiIj9Xnk9gP1ujCeFfD6erigplTcNaTKE2NGxNC7X2NOhqGTc9lXZGBMnIiOApViGpE41xuwVkQ+AYGPMAmCkiPQA4oCLwCB3xeOILoCnlOf4eGlNPTty62/FGLMIWJTs3Bi7128Cb7ozhswIGBVEyPjung5DKaWynLafKKWUstH6m1J5VMT1CLae2Zp+QZWnaE1BqTzoz/1/UufbOrf1jNfufc1F0ajsJM8nhbvvKOTpEJTKMuevnafv7L70mtXrtlct/bTzp7qvQS6U55uP/HRYqsoDjDHM3DOTFxa/QFRMFB93+Jj/3Psfvvz3S12mQiWR55NCPh9vh+f/3neOTrXvyOJolHK9M1FnGB40nPkH59O8QnOmPjSV2qVrA/B6y9c9HJ3KbvJ8UvDzdlxTeOaXYMoX9Wflf9rh7+s4cSiVnRlj+GXnL7y09CWi46L5rPNnvNziZby99P9nlbo833YysmP1VK+djozm539Csi4YpTIoJj4GeV94fXnSb/ynIk/RfXp3Bs0fRN0yddk5bCf/ufc/mhBUuvJ8UrjnzpJpTlQ7eDYqC6NRKmMuR1v2FP9px0+ApXYwZesU6nxbhzUn1vDfLv9lzaA13F3ybg9GqXKSPJ8U0vPn9rD0CynlIZvDNtteh1wOofO0zgxdOJTA8oHsHr6bkc1H4iX6z1w5z6k+BRG5Ewg1xtwUkXZAfeAXY8xldwanlHJs17ldvLXiLYIOBwFw5eYV6n5bFxFhUvdJDG0yVJOByhRn/6/5A4gXkbuAyViWxJ7utqiymYBRQVy+rhuKKM87dukYA/8cSMPvGrLh1AZGtxkNwM34m7Ss3JK9z+1lWOAwTQgq05wdfZRgXfW0J/C1MeZrEdnuzsCym+/WHGNU15qeDkNlU/MOzGPtibV8cf8Xbnn+2atn+WjtR0zeOhkfLx/eaPkGr7d8nWL+xbgac5V6ZeoxqOEgXflX3TZnk0KsiAwAngQetJ7zdU9I2c95v7F8vy+GUV2DPR2KyoaCDgXRZ3YfvMTL5UkhMjqSz/75jC///ZKbcTcZ0ngIo9uOTjIb2V2JSOVNziaFwcAwYKwx5riIVAWmuS+s7OWG90ZCrno6CpUdrTuxjt6zexOXEIeft5/Lnnsj9gYTt0xk3PpxXLxxkf51+/NBuw+oXjL1IdRKuYJTDY/GmH3GmJHGmBkiUhwobIz5xM2xZTuJw/9embWDluNXejga5WnbzmzjgRkPEFAsgKcbPX1bz4pPiAcgLiGOH7b9wN3f3M1ry1+jWYWRmwhQAAAgAElEQVRmbBu6jRm9ZmhCUFnCqaQgIqtFpIiIlAC2AVNEJN06q4h0EZGDInJEREalUa6XiBgRCXQ+9Kw3bOEwjDH8uS2MsMs3PB2O8qCDFw7S5dcuFPMvxrKByyhdoHSGn3Et5ho/bv+RllNb4vOhD19u/JK639blmb+eoWKRiqx6chWLH1tMo3KN3PATKOWYs81HRY0xV0TkGSxDUd8VkV1p3SAi3sBEoDMQCmwRkQXGmH3JyhUGXgQ2ZTz8rPX73t+57877AF0TKS87GXmSztM6IyIsf3w5lYpWytD9W09v5X/b/sf0PdO5cvOK7fwry16hdunazOs3jx41eminsfIIZ8et+YhIOaAvsNDJe5oBR4wxx4wxMcBM4CEH5T4EPgGinXyuW9WtUCTVax2qduCFxS8QK6FZGJHKTs5fO0/naZ25cvMKSwcutc0U3nx6MzHxqQ9bjoyOZNKWSTT+vjGBUwL5aedPPFzzYdYOWkvv2r0BaFi2IbuG7eKhmg9pQlAe42xS+ABYChw1xmwRkWrA4XTuqQCcsjsOtZ6zEZHGQCVjTFBaDxKRoSISLCLB4eHhToacOSUL5kv12tstJuLr5U+436cYYt0ah8p+IqMj6fJrF05FnmLhowtpWLah7drK4yn7mIwxbDi5gUHzBlHu/8rx3KLnSDAJfNP1G868eoafH/6Z1lVaU65QOQAGNxysaxMpj3Oq+cgYMxuYbXd8DOh1O28sIl7AF8AgJ95/MpZJcwQGBrp1V48hraux5pDjxPPU1KP4eT1PZL4PueT7E/CwO0NR2ciN2Bv0mNmD3ed3s6D/AlpVbpVq2QvXL/DLzl/437b/sf/Cfgr5FeLx+o8zpMkQmpRrorUAla05u8xFReBroKX11DrgRWNMWu0oYVhmPieqaD2XqDBQF1ht/UdSFlggIj2MMR6bEBCXkJDm9QIJzSkc9yBRPvNZfHgxXat3zaLIlKfExsfSZ3Yf1p1Yx4xeM9L8nfef05+5B+YSEx9Di4ot+KHHD/St05dCfrrDn8oZnG0++hFYAJS3/vnLei4tW4DqIlJVRPyA/tZnAGCMiTTGlDLGBBhjAoB/AY8lhI41ywAQG59+RaR47GB8EwJ4ct6TnL161t2hKQ9KMAkMmj+IoMNBTOo+iX51+6VZfvmx5QwPHM7u4bvZ+PRGnmr0lCYElaM4mxRKG2N+NMbEWf/8BKQ5Bs8YEweMwNIXsR+YZYzZKyIfiEiP24raDXytm+3ExqddUwAQ/CgV8zpXY67yxNwnSDDp36Pc759T/1Djmxr8se8PlzzPGMMLi15g+u7pjOs4jmcDn033nrBXwpjQZQJ1y9R1SQxKZTVnk0KEiAwUEW/rn4FARHo3GWMWGWPuNsbcaYwZaz03xhizwEHZdp5sNnr8nioANKxULMU1//hG+CXUSHLOz1RmQpcJLD+2nC826jIDnjZ562Ta/dSOQxGH2H9hv0ueOWbVGL4N/pbX732dUa1SnWaThL+Pf4bfJ/GezNyrlKs5mxSewjIc9SxwBuiNEx3EOUnLu0oRMr475YvlT3GteOxgSsQMS3F+SOMh9KrVizdXvMmWsC1ZEWauFxsfy7Kjy5yufd2Mu8nQv4by7MJnaRvQ1mVxfLHxCz5a9xFDGg9hfKfxLnuuI++1e482Vdrc9qxopVzB2WUuThhjehhjShtjyhhjHuY2Rx/lJH6mGvlMyiUGRIQpD06hXKFyDPhjAFE3dZe222GMYdjCYdz/6/0En06/0ng66jTtf27PlG1TeLPVmywc4OwUmrRN3T6VV5e9Sp/afZjUfZLbRwsV8C3AmkFrdDiqyhZuZ9H1V1wWRQ5WPH9xfnvkN45fPs6IxSM8HU6O9vG6j5m6YypgqQGk5Z9T/9BkchN2ndvF7D6z+bjjx7f9oTp562T6zu7LkL+GcP+d9/PrI7/qB7XKc5xd5sIRHWxt1bpKa0a3Gc37a97nvmr38Vj9xzwdUo4zY/cM3ln1DjVL1eTAhQNplp28dTIjFo2gctHKLH98eaY6deMT4jlw4QCbwzZb/pzezLYz2wC4t9K9/NH3jwytejqv3zw+3/h5huNQKru5naTg1klkOc07bd5hxfEVDA8aTouKLbizxJ2eDinHWHdiHYPmD6JNlTa80fINuk/v7rDczbibvLD4BaZsm8L9d97PjF4zKJ6/eLrPN8YQeiU0SQIIPh3M1RjLeuhF8hWhafmmtvILByykoF/BDP0MD9V8iIdqOlrFRamcJc2kICJROP7wFyBlj2weNmjqVo4ffQrvYq/w6J+Psn7weny988w+RJl2KOIQD//+MFWLVWVuv7lsP+N4Q7/TUafpPas3G0M38marN/mw/YcpmnYSl58evWo0LSq2YHPYZjaFbWJz2GbbfBJfL18alm3Ikw2epFmFZjSr0Iy7S96Nl3gh71sqv84kGqVyqzSTgjGmcFYFkhPFxSfgY53fsP7IBXwow5QHp9Bndh/GrBrDuE7jPBxh9nbh+gW6T++Ol3gR9GgQJfKXcFjun1P/0GtWL6JuRjG7z2zbAnLJxSbcWo+q87TOANQoWYPO1TrbEkCDOxqQzyf19a2Uyutup/koz+s16R/mPtcSL69b3Su9a/dmaOOhfLLhEzpV60THah09GGH2FR0XzUMzH+JU5ClWPbkq1ea2jPQf+Hjd+t95+ePLCSwfSDH/lPNOlFKpu53RR7nek9YJbanZGRrJ+3/tTXH+yy5fUrNUTR6f+zjh19y7qmtOlGASGDRvEP+c+odpPadxT6V7UpS5GX9r/kHHah3ZMmRLuh3KXmL537lUgVJ0qtZJE4JSmaBJwYF6FYrSoWYZ/P3SH47488YTBIxKuvJ3Ad8CzOg1g4s3LjJ4/mCM0T55e2+veJvf9/7OJ50+oU+dPg7LdJ7WOcn8A2fa+f28/Xir1VtseGqDq0NWKs/Q5iMH/nrBsizyH1szv5lOg7IN+KzzZ4xcMpJinxQjclSkq8LL0aZsncL4DeN5tsmzvHbvaymuX7h+wfY6rf6D1IztOPa2Y1QqL9OaQhoeaVwh/UJpGNHMMpntys0rtuGPedmyo8sYHjScLnd14Ztu3zicKbzz3E4AXr3n1QwnhNsV9GgQfz/+d5a+p1LZjSaFNGRmeYOAUUGMnLE9xf0Xb1x0WVw50e5zu+k9qzd1ytTh996/J+kUdiS1kUju1K16Nx0YoPI8TQpusGDnafafuZJ+wTzidNRpuk/vTuF8hQl6NIgi+VLfB1sp5Vnap+AmT0zdzLNtqtmOC/pmbIZsbnE15ioPzniQizcusm7wOioWqejpkJRSaXBrTUFEuojIQRE5IiIpFqQXkWEisltEdojIehGp7c54slJMXALjFt9aw2f0yvfy3PDU+IR4BvwxgB1ndzCrzywalWvk6ZCUUulwW1IQEW9gItAVqA0McPChP90YU88Y0xD4FMg1u9VE3oglPuHWUNRJW7+hyoQqvLDoBUIuh2RJDOmtNOouxy4dY+fZnby05CUWHlrIN12/oVv1bh6JRSmVMe5sPmoGHDHGHAMQkZnAQ8C+xALGGPuG94LkwkX2xPhjJJq1T27kp51T+H7r90wKnsSAegN4/d7XqXdHPZe8T0x8DLvO7WJT6CY2n97MptBNHIw4yOQHJjOkyRCXvEdawq6EMWvvLGbuncnmsM2286/e8yrDmw53+/srpVzDnUmhAnDK7jgUaJ68kIg8j2VvBj+gg6MHichQYChA5cqVXR6osx5qWJ75O05n6J4Ssc8R4fcFjco2oXVAC6rmG8Sn679g7v65/LrrV7pX786oVqNoVbmVw/tPXD5BlWJJZ1YbYzh26ZhtwbdNYZvYfmY7N+MtNYMyBcvQvEJzDkYc5NSVU44e6xLh18L5Y/8fzNgzg3Un1mEwNC7X2Hb9kVqP8GnnTzP0zKFNhjJu/TgGNRzk4miVUs7weEezMWYiMFFEHgXeAZ50UGYyMBkgMDDQY7WJdjVKZzgpFIrvQKEbHWzDU9fuT6BE3DP8/ugElp34hf9u+i+tf2xNy0otGdVqFN2qd7Mt17Dw0EIenPEg33T9hjtL3GmrBWwO22yb5JXfJz9NyjdhRLMRNK/QnGYVmlG5aGVEBK/3M946aIwh5HIIjb5vRONyjVn55Mok1yOjI5l7YC4z98zk72N/E2/iqVmqJu+1e49+dfpRo1QN22qj03pOs/0szgooFoB5N9dVGJXKMdyZFMKASnbHFa3nUjMTmOTGeLKVYv4lGN12NK/c8wpTt0/l842f8+CMB6lbpi5vtHyDfnX68fcxy0SqxB3dBKF26dr0uLsHzSo0o3nF5tQtUzfdMf9pSTAJ7Avfx7oT61h7ci3rTqwjLMrya1oVsgqAazHXWHhoITP3zmTR4UXExMdQtVhVXm/5Ov3r9qdemXoO53QU8C2Q6biUUp7hzqSwBaguIlWxJIP+wKP2BUSkujHmsPWwO3CYPKagX0FeaP4CwwKH8fve3xm/fjyPz32cd1a+w4nIE7ZyK59YSZPyTW57jH9sfCxbz2xl3Yl1rDu5jg2nNtgm1pUrVI7WVVrTpnIb3lzxJlExUTz6x6MsOLiAa7HXKFeoHM8FPkf/uv1pVqGZ2/cuVkplPbclBWNMnIiMAJYC3sBUY8xeEfkACDbGLABGiEgnIBa4hIOmo+yka91yvPz7TsY8UJsPFu5L/wY7xy9c45eNIale9/X2ZWD9gTxa71EWHV7EuPXjbEnh8fqP075q+wzHazBcuH6BFcdWsO6kJQn8G/ov12OvA1C9RHUervEwrau0pnXl1lQrXs32QT/v4Dz+PvY3y44uY2D9gQyoO4BWlVs5tWfx1B5TGbpwaIbjVUp5nuS0FTwDAwNNcHBwlr3fuSvRNP94BQAh4y3bREZcvUmTjzK3Rk6Jgn5cvBbD0pfaUKNs2nsYrT+5no/XfcycvnMy1RST2LYPlqanBmUb0Lpya9pUaUOryq0oW6hsqvcevHCQk5EnaRfQTneQUyoXEJGtxpjA9Mp5vKM5u7ujiH+KcyUL5ePXp5sz8IdNGX5ebHyC02VbVW7FoscWZfg97BXyK8Ss3rO4t9K9FPUv6vR9NUrVoEapGrf13kqpnEeTQi6mo3iUUhmlC+J5yLPTgvlt04n0CyqlVBbSpOAhIRHXeXvuHk+HoZRSSWhSyGJR0XGeDkEppVKlSSGTapd3zZ4A3685yrK9Z13yLKWUul3a0eyEzW93TLLiKViGlrqC/fLaAK/dX4Pn2t1pmy9w4OwVukxYx8yhLWhRraRL3lMppVKjScEJZQqnHJbqLp8tPcidpQvh7+vFoB+32M4v2n1Gk4JSyu00KWRDw37dmuJcRuY3KKVUZmmfQg4RE6dzDpRS7qdJIYfQmoJSKitoUsghNCkopbKCJoXb8N3AJgxvdydVSrp/34DFe85Se8wSt7+PUipv06RwG7rULcsbXWqy5rWML2udGddj4p0q992aowSMCnJzNEqp3EiTQhYpVcg18xqW7T3LjxuOp1lmfLK5D0op5SwdkppFvn+8CeFRNxn267bbes7QaZbhqoNbVnVFWEoplYRbawoi0kVEDorIEREZ5eD6KyKyT0R2icgKEanizniyUsj47vw1opXtuLC/LwGlCrrs+eeuRBMwKoivVuS5HUyVUm7ktqQgIt7ARKArUBsYICK1kxXbDgQaY+oDc4BP3RWPu33Sq16Kc/Uq3trUplA+H/x90t/K0lnztocBsP7wBZc9Uyml3Nl81Aw4Yow5BiAiM4GHANvmxsaYVXbl/wUGujEet+obWIkzkdE80qgilR2MRirk78MNJzuKnZG4ZtLmkIucvxJNGQc7xCmlVEa5s/moAnDK7jjUei41TwOLHV0QkaEiEiwiweHh4S4M0XVEhJc63e0wIQAU8ffF39d1NQV7zT5eQZMPl2fq3kvXYvh0yQHidB6EUopsMvpIRAYCgcBnjq4bYyYbYwKNMYGlS5fO2uBcKL+bkgJAxLUYZmw+SXRsxmojHwXt59vVR1m275ybIlNK5STuTAphQCW744rWc0mISCfgbaCHMeamG+PxOF9vcevz3/xzNxuPRtiOA0YFsWL/OU5fvpHqPcZY1lRKPgciYFSQznVQKg9yZ5/CFqC6iFTFkgz6A4/aFxCRRsD3QBdjzHk3xpItJO6R4E6Df9qS5Pjpn4OpWqog80e0ZOLKI3y/9hgAL3S4i1fvq0E+X8v3gptxruvvUErlXG6rKRhj4oARwFJgPzDLGLNXRD4QkR7WYp8BhYDZIrJDRBa4K5687PiFawyautmWEAC+XnkEgHzWEVExcdqnoJRy8+Q1Y8wiYFGyc2PsXndy5/tnR8+0qsr/1juekZzPx4ubbvpwPnnRcRNSPp/EmkICe8Ii8ff14q4yhd0Sg1Iq+9MZzW5WoVh+3uha03b8ZrdaPNO6GldvxtLpi7VJyvq5MSk4armKTzC3kkJsAg98vR6Aec+3dEsMSqnsL1uMPsrNNozqQI8G5W3H3l5C2aL+FMnvm6KsfTlXC49K2Yc/+Kct+Pmk7FPYeepyms+6GRfP1Ztxrg1QKZUtaFLwkKJ2SaF7vXIA3HNnSX5+qlmWxbD2UDiXr8cCpFlDiYqOJWBUEMcvXAOgxjtLqPvuUrfEtP/MFbc8VynlHE0KHpIvlSUvAqsUT/febvXKuiyOkxevAzB/x63Rwu8u2JukzOLdZwH4dtWRVJ/z57ZQHrQ2PwWHXOS531LuM52eGZtP0vW/61h5QOdMKOUpmhSymYL5fOgbWDHNMkUdND1lVj7rhLoLV2PSLTt7a2iKc/EJhldm7eCVWTvZHRYJQL/J/7LImkgy4sj5qwAcC7+W4XuVUq6hHc0e9PvQFpQp4m9b3K5SccsSGekth1HQz3W/tsSO5rQcCb9qe33M7nXAqCCebVuNP7clnZMYn2BcFp9SKmtpUvCg5tVKAvBSp+p0qVuWWuWKAFDNbontWuWKcOVGLGF2s5IL5HPdry296XT9vt/IpuMXbccd/m9NkutL9mS8RqCUyr60+SgbEBFbQgB44p4A2+v7at/BghFJh4gWyue6NZQcNQnZs08IzhgxPfObCP2QyvwNpVTW0aSQDXl5CffXucN2XLJQviTXC7qwpuBqC3edSXIcdvmGS5cMV0q5lyaFbKpm2SJJju3nMBTKxkkhuZbjV/L4D5scXvtxw3ECRgVx1K6fAuCXjScy/X77Tl/RJKTUbdCkkEO0vduyZPgjjSo47GiePqR5VoeUrp//CQEg+MQlwLKF6HdrjhIXn8Dm4xf595hlRdfD56KS3Jc4TDaj4uIT6PbVOnp+uyHzQSuVx+Wcr5x5TK1yhZP8t0Lx/ABULlmAtjVKU6lEfk7ZrWd0752lCChZgJCIzH2guoP9fIeQC9do9/lqABbuOs2esCtUL1PIdv2zpQdu+/3irKOejl3QIa1KZZbWFLKpLnXL8c+oDnSpa5nt3KJaSaY93YwXO1bH19uLFa+045HGSTey+/O57LtmUWJCANgTZpm1fN5u6Y2Jq45mdUhKKQc0KWRj5YvlT3Lcunpp254Mfj5efNG3YZLr7tzZLTUnbqNmEnnDssTGsF8dj1iKvG5ZXuOUk81J7pwfkaBzL1QeoUkhF8nv581fI1qlWy4glX2ks5OEBMM869IbU9YdS3H96s04Gn+4nPWHL9jOvfT7jnSfezMunk3HItItZ+/g2SiqvbWIHzfokFmV+2lSyGUqlUhau/j2sca0q5Hz9rX+ddOtEUgHzkYRMCqIYdNurae0NyySi9dimPD3Idu55U7sMz1u0QH6Tf43QwvvHT5v6QgPDrnE5uMXmRV8yul7lcpp3JoURKSLiBwUkSMiMsrB9TYisk1E4kSktztjya2ebVONKnbf/H28k/5Ku9Urx1cDGjH9mew3OiktXyy/9WG/2TqBbsnes7T9bBUAUdGWpbsdLUGelsRO6LNXogHLUh21xyxx+v6+32/k9Tm7MvSeSuUkbksKIuINTAS6ArWBASJSO1mxk8AgYLq74sjt3uxWizWvtbcdO1q2ooi/L/feVcp2nLi20hd9G6T5bD9vz1UkL1+PTbFaK1j6MBISDBHXLJ3Uhf0tA+iux6S/v8PNuHi2WBPMjZh42wJ813VeQ5aKTzBsCcnYTHmVddz5r74ZcMQYc8wYEwPMBB6yL2CMCTHG7AJ0g2AXcWaBuwJ+lqRQJZ2+hRc63OWSmFyt5pglvPHHbgDm7zjN1hMXCb10a3hu8v2mY+ISOHg2ildm7eRGrCUBvD13N52+SLqO0+0IuXCN6FhNLs74dtUR+ny30TZPRWUv7kwKFQD7xtdQ67kME5GhIhIsIsHh4eEuCS638vH24vi4bk6XDxnfPcW5ne/eB8CglgHMHNoCgGYBJVwToAsk/9DvNWkj93251mHZvacjufudxdw/YW2SHeUuWTcXcoaXdcRXXILj7y6XrsXQ7vPVfLhwn9PPzMsSV90Nu+R433DlWTmio9kYM9kYE2iMCSxdOud1mmY1cbQhcwYUze9LyPjuFPb3pUW1kix7uQ1P3FvFRdFljcTtR4fbDXcNTeNDaNvJS6lee3uupVaS2hak+89aOq0Pno1yeD07237yEgfOpt/pHh0bjzGuGZY7f8dpAGLjtYEgO3JnUggDKtkdV7SeUx7yZb8GLH6xtW2BvSL+KTtpa5YtnOLc3XcUpqt1El1O0XTs39yMi3d67sbR81dTnDt/JZqwyzdstYrEzu3U7r3LboZ2WgJGBREwKsipsu7W89t/6DJhXZplTkRco+boJemuqJtRsTr3I1tyZ1LYAlQXkaoi4gf0Bxa48f2UncQOWHs9G1WkVrkiTOjXkPd71KH6HSkTwIhU+hG8vYQfBzd1eZzuVOOdJeT3cy4pbDhyge/WHOVm3K1+gWYfr6Dl+JW2412hkQ7vPX7BMrlu5pZbraWLd5+h5fiVxGXw2/C1m3G2GklcfEKKpjJP2HfaUpNYsd+126TGZoOfTaXktqRgjIkDRgBLgf3ALGPMXhH5QER6AIhIUxEJBfoA34tIyuEmKlOCXmjNO91rObxWMJ8PT94bcKvsyFb8/UpbBjSrRPd6qdcIyhbxB2BU15qplmlRLfv0PQDssOtHSMu8HacZv/gAE1emvg+1vcRv+hNXHWGqg0lt7y7YS9jlG2luc/rstOAU5+q8u5R67y0F4K63F3P3O4udisedoq2JMr0dATMqtT4a5VluXRDPGLMIWJTs3Bi711uwNCspF6tcsgDPtK7mVNk65YsCMO6R+mmWq1WuCKv/044qJQswfnHKBewK+nnToWYZ/j1mGW5Ytoi/bT5ATvHVyiMcvXCNf45cSL8w8NnSgw7PF8rnw/mom1y9GQv4OyyzdG/Sb95rD1kGUTjbdD9ryylmbDnJ3EyueeVMLeZo+FVe/n0nAP4+rk0K127qaK3sKEd0NKvsI6BUQUSEla+25dHmlZNc61jrjiS7xs169h5e7Fg9iyO8fUG7zjg1OslRv8CMzScByzLhAFetH3x/7Tyd7izqJ6ZuzlCcr/+xi+0nU68JnY+KTnOY7JR16S/bYb+sub+vaz8u/rvisEufB5blUXRo8O3RpKAypVrpQnzcs57tePnLbfisT33bhLceDcpTuWQBXu58N+Meqceika3Z98H9HPiwi+2eu+9wrmM2J3nzT8tIpWvWCXET/j5EXHwCL8zYzoNfr3f4gfXegr18szLtD8iEBMPEVUe4dC2GhATD6Hl70o2l2dgV3DNuBQARV28yZe0xTl++QaQ14Z3LYC3O1c1H7vDJkgPUHL1EE8Nt0P0UlEvYd1oHv9OJonbLTwxoVtnRLZQo6Jfq897qVpOPF93+HguetvpgOOusi/bFJRiG/7o1RZmfrJsRpSZgVBA/Dm7KZ0sPcuT8VTYejUjSLHclOtbhSDK4NR/jnXl7WLznLGMX7adM4XxsfruTUx/yy+zWk8qXA5LCzxtDAPeumJvbaU1B3ZbRD9Smld0SGgClCuXD14klMh6022I0NenNus6OkjcrDf5pi+31kWRbj+4JczyiKXkH8+AfLc+Yuz0sRT/Ns7+kTDTJJdh1VCTuY+FMc9Cf226NInd185E7RMda+km8bnOuTl6W/X/LKlt7ulVVfs3gYnvPtq3G5Meb8FjzKux8974kM7CXvdyGFa+2pURBy1yKAc0qp+iXKOpgEbxKJfITMr67U0uHe1LyZPnA1+sdlsvIUNSNxyIIGBVEs7F/O7weG5/A2kMpO87t53A4MzHNmY7mK9GxSTqwZwWfImBUkG2Y7T9HnevAz40uXovhkW83uGwSoLtoUlBZ7s2utbivTlnA8gFvPwP77jsKc2fpQvRqXIHX7q/B0NbVeLnz3fzyVDNbmYaVilG8gC9zn7uX2cPuSfLsehWLZs0PkUnHwt23VWhiDeB6TBy/2S09HvjR37Y1n+zZNx/Fxqf/QbX1xCVuxsVzJTr1Tvj67y1Lsq/Fp0ssTYBR1nt+3BCS7vu4woS/D9Hqk5X8z8FeHJmRkGAyPOckuff/2su2k5dZsf+8S2JyF00KKlsSEZ5vfxdeXpaE0ebu0qx7vb3t9fYx99GocnHuKJxyuGfr6qVSnEvrfG4yfvEBao9Zyttzb3VEJ+5wZ2/+jrAkq9Da70Vx0Lp/xaqDST+8gnafocY7S6j/3jIafbAs1aVBFu46Y3udOE/DxytrP2q+X3uM0Es3+Chov0ueV+2tRdz1tuM5IzM3n0y1GdBeYu0vJpsv76EdzSpbmPZ0MwrmS/t/x0olCqRYwK9iccumQm91vTVRb9rTt5qzEtv35z/fkpj4BE5EXGfRi62p++5SV4WerXy3xrm9rl+cmXSXuqDdp+levxyR12PZfNyyeulXaQwZvXQ9lke+/YfvBja27SNu78j5q04v+2Hv9OUbxCcYKpXIPgzGkLcAAA7JSURBVH1JaW3FmpBgGGUdcXbs4262LzE5mdYUVLbQunppGlcunuH7vLyEkPHd6ZrGTGwvgQaVitE0oARrX29PIbvkc2Rs10zF2y+wUvqFcpBFu88yee1RGnywjNHzLTWItOZAJLLfX3vc4lvfyjt9sYaFu06ne3/AqCBb0xLAveNX0vrTVbZjYww/rD/OlehYzkdFs/e042/kw6Ztddt6UsEnUl8s0d71XDIMVpOCytV+GtyUf0Z1TPW6j7dXkiSRXOKSHs+0qpqklvJUq6pJypUt4s/QNs7NIM+ufspke3/iN+nv1yRtvx8xfbvtddOxf5OQYBxumTpvu+N1MietPkp765Lk9d9bRrOxK+j+leOO+aX7zmYqdmck7xg+GXHdYf/C1VQWTMxpNCmoXK1djTKULZqy3+H9HnWY9rSl83rb6M6288m3LW1W1bKWU90KaXdglyjox5tdayaZnGevXoWitmdlV6cjM7ckSbW3Fjn1Lb3aW4tSvTZz88kUz/hkyQFCIq6nKOvMLnuOuGLUz+XrMbT5bBXPT9+W4pplSZNb4uITOHI+ikmrj/L3vnO2pdWX7j3L5eupr4mVXGJtKeLqzdsL3knap6DyJPsFAf18vOherxxBu88Qce3WP9bENuLNb3WkdGHLENmFL7Ti+IVbI4gCShagV+OKPNq8MiKCv683ZQrn43zUTQ6P7croeXuYueUUvw1pzlM/3pqvkNynvepzR1F/nszgUhe5RWK7fKKHJ25ItWztMUtZMKIlc7eH8UD98tQuVyTd9aKqvRlE8QJ+DG1TjYNno/hzexhBI1vR/av1LH+5DdXvKMzVm3HEJ5gUQ57j7PoULlsnAyZftwpSLq3+2pxdzHVQC5q/4zRrD4Wzfcx9aQdttSXkEh8u3Mf2k5f45tHGTt1zOzQpKAU0qFSUoN1nKODnzcyhLUhIMLZOwzJFbtU06lYoSt0KRTl18brt2gvJ5lGsfb09Jy9ex9fbi/G96jO+l2WhwadaVU21fbpvU0sfxYj2d3EtJo53H6yT5rfvaqULunV4a1ZK7MOwl97qtj2+sSSNHzeEOLWXeIKBiGsxjLNbyPH1ObsAmLMtlDe71qLNp6u4eC2Gv19pQ4ViBcjv552iyStxbatEP6y/tX5U8hVxD51LfdOl9NbWen3OToJPXGLlq+04fdmyOdTCXWcY1jYy3Vrr7dKkoBTwTKtq1CxbhNbVSzm1c12lEgV4s2tN+jVN2eHs7+vN3Q72quhWrxwh47sTHHKR3t9tpF2N0oztWS/Jvtr/ub+G7fXIjtWJvB7DzxtvzTloXrUEre4qxQsdqzNu0X6+X2tpx7/3zpJMH9KCu99Z7HDiW05csdZZ6Q3xPHLe8Ydz4of292uOcV/tsly01hI7fWHZ2nXBiJYMmPyvbR0rwPb3nWjsolud60N+CXa4vW1qIm/E2molN2Li8fEWfL29OH35BrOCb21olPgFBOCNP3YRNLK10++RGZLdZ9clFxgYaIKDU65Dr1RuFRxykT+2hVG7fBEeb5F0W9TE2sShj7ri5+PFg1+vZ3dYJEfGdrWNq0/8oHJU8yhewDfFt9YxD9TmA+t+04X/v717j5GqPOM4/v2xC8udBaF0cVGwIEKiyy3CCrFKlaKoiRarpKkYsdSojSaaFqpFa2y1tfHWVNNqrdaopVgvW2K13OxFlJvcQQTrctkqCwiIKLAsT/847x6GYWAX6szs2X0+yWTOec87m/eBs/vM+55z3rd1IZeW9eD5+RuP+Gxzd3JxG6p2Hr7E660X9KV/SUe+/2z9U4+8fOM5DOjRkWkLNzE19JZm/GDkYU+5r713DP3ufD3e71/Skb/dcmJJQdJiMxtaX72s9hQkjQEeAQqAJ83s/rTjRcAfgSHAduAqM6vMZpucS5qhvbowtFfmi9TPXHc2ry6polXobUy/oZwtn+6lsKAF911xJmWlxUd8pvL+sazbspu2RYV0a1/Ej19ewQPjzmLovdE0GdeN7E1BC3FXxSpuH92PCef0YunGnawOU3/3L+l42DTgc277Ok/868N4aGVASUd+dWUZFz96aJnPc0/vFq8X0VSkJwSAh2c1fDrwyx+bd0RZ+rQnqQkhV7LWU5BUALwPXAhsJlqec7yZrU6pcyNwlpndIOlq4HIzu+pYP9d7Cs6dmLnvVdOzSxv6fOXIoa1MFm/YwaCexfG1lSUbd7Cu+jMuK+vBri9qGPbz2fTu2o65t58Xf6am9iAFEi1aiC/219J/avRH7a3Jo/jzwk3U1B7ksTcPPWD37MSzOfPkThS3bcUz8yq5q2IVhS3EtwaXMm3RJo7l0rIe/HXZoWch2hcVxnMsNVUj+pzEc9cPP6HPNrSnkM2kUA7cbWbfDPtTAMzsvpQ6b4Q6b0sqBD4GutkxGuVJwbnk2LPvALPfq+aytBlxZ63eQuX2PcdcHbDuz8Ce/bXsramla/si9tbUcsZPokRTef9Yag8aNbUH2bp7X3yH2H+27uG0bu3ieq1btohnT63TuW1LDtQay+8ezfLNu/h8fy3jn3inwXFdN6J3xmVYs+3uSwdw7Yje9VfMoDEkhXHAGDO7Pux/FxhmZjen1FkZ6mwO+x+EOtvSftYkYBLAKaecMmTDhg0451wuvbKkisICcUH/7odNJvjZvgO0aVlAC8GG7Z+z5qNP4yfsl23aSU3tQT7Zs58LB3SnaucXdGjdkp2f7+f5BRu5YlAp/b7agZVVu3hw5vuMPbOE26ZHy5/efH4f1lXvjm9/nTd5FCWdWjfoRohMmlRSSOU9BeecO34NTQrZfKK5Cki9X680lGWsE4aPOhFdcHbOOZcH2UwKC4G+knpLagVcDVSk1akAJoTtccCcY11PcM45l11ZuyXVzA5Iuhl4g+iW1KfMbJWke4BFZlYB/B54VtJ64BOixOGccy5Psvqcgpm9BryWVjY1ZXsvcGU22+Ccc67hfJZU55xzMU8KzjnnYp4UnHPOxTwpOOeciyVullRJW4ETfaS5K3DUB+MSpqnE0lTiAI+lsfJYIqeaWbf6KiUuKfw/JC1qyBN9SdBUYmkqcYDH0lh5LMfHh4+cc87FPCk455yLNbek8Lt8N+BL1FRiaSpxgMfSWHksx6FZXVNwzjl3bM2tp+Ccc+4YPCk455yLNYukIGmMpLWS1kuanO/21EfSU5KqwyJEdWVdJM2UtC68dw7lkvRoiG25pMH5a/mRJPWUNFfSakmrJN0SyhMXj6TWkhZIWhZi+Wko7y1pfmjztDBVPJKKwv76cLxXPtufTlKBpCWSZoT9pMZRKWmFpKWSFoWyxJ1fAJKKJb0o6T1JaySV5zqWJp8UJBUAvwEuAgYA4yUNyG+r6vU0MCatbDIw28z6ArPDPkRx9Q2vScDjOWpjQx0AbjOzAcBw4Kbw75/EePYBo8ysDBgIjJE0HPgF8JCZ9QF2ABND/YnAjlD+UKjXmNwCrEnZT2ocAOeb2cCUe/iTeH4BPAK8bmZnAGVE/z+5jcXMmvQLKAfeSNmfAkzJd7sa0O5ewMqU/bVASdguAdaG7d8C4zPVa4wv4FXgwqTHA7QF3gWGET1hWph+vhGtJVIetgtDPeW77aE9pUR/YEYBMwAlMY7Qpkqga1pZ4s4vopUnP0z/t811LE2+pwCcDGxK2d8cypKmu5l9FLY/BrqH7cTEF4YdBgHzSWg8YchlKVANzAQ+AHaa2YFQJbW9cSzh+C7gpNy2+KgeBn4IHAz7J5HMOAAM+LukxZImhbIknl+9ga3AH8Kw3pOS2pHjWJpDUmhyLPpakKh7iSW1B/4C3Gpmn6YeS1I8ZlZrZgOJvmmfDZyR5yYdN0mXANVmtjjfbfmSjDSzwUTDKTdJOjf1YILOr0JgMPC4mQ0C9nBoqAjITSzNISlUAT1T9ktDWdJskVQCEN6rQ3mjj09SS6KE8JyZvRSKExsPgJntBOYSDbMUS6pbxTC1vXEs4XgnYHuOm5rJCOAySZXAn4iGkB4heXEAYGZV4b0aeJkoWSfx/NoMbDaz+WH/RaIkkdNYmkNSWAj0DXdWtCJaB7oiz206ERXAhLA9gWhsvq78mnAnwnBgV0pXM+8kiWgt7jVm9mDKocTFI6mbpOKw3Ybo2sgaouQwLlRLj6UuxnHAnPBNL6/MbIqZlZpZL6Lfhzlm9h0SFgeApHaSOtRtA6OBlSTw/DKzj4FNkvqFom8Aq8l1LPm+uJKjCzgXA+8Tjf/eke/2NKC9LwAfATVE3x4mEo3hzgbWAbOALqGuiO6u+gBYAQzNd/vTYhlJ1N1dDiwNr4uTGA9wFrAkxLISmBrKTwMWAOuB6UBRKG8d9teH46flO4YMMZ0HzEhqHKHNy8JrVd3vdxLPr9C+gcCicI69AnTOdSw+zYVzzrlYcxg+cs4510CeFJxzzsU8KTjnnIt5UnDOORfzpOCccy7mScG5o5B0R5gNdXmYgXOYpFsltc1325zLFr8l1bkMJJUDDwLnmdk+SV2BVsA8ovvBt+W1gc5lifcUnMusBNhmZvsAQhIYB/QA5kqaCyBptKS3Jb0raXqY46lujv9fhnn+F0jqE8qvlLRS0ZoM/8xPaM4dnfcUnMsg/HH/N9EU2bOAaWb2jzBf0FAz2xZ6Dy8BF5nZHkk/InoK+J5Q7wkz+5mka4Bvm9klklYAY8ysSlKxRXMoOddoeE/BuQzM7DNgCNHiJVuBaZKuTas2nGjhprfCdNoTgFNTjr+Q8l4ett8Cnpb0PaAgO6137sQV1l/FuebJzGqBN4E3wzf8CWlVBMw0s/FH+xHp22Z2g6RhwFhgsaQhZtZoZhx1znsKzmUgqZ+kvilFA4ENwG6gQyh7BxiRcr2gnaTTUz5zVcr726HO18xsvplNJeqBpE597FzeeU/BuczaA78OU2UfIJohdBIwHnhd0n/N7PwwpPSCpKLwuTuJZuQF6CxpOdHaznW9iQdCshHRzJfLchKNcw3kF5qdy4LUC9L5botzx8OHj5xzzsW8p+Cccy7mPQXnnHMxTwrOOedinhScc87FPCk455yLeVJwzjkX+x+DURa1CBCVtgAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "train_loss = np.array(model.get_train_summary('Loss'))\n",
+    "val_loss = np.array(model.get_validation_summary('Loss'))\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "plt.plot(train_loss[:,0],train_loss[:,1],label='train loss')\n",
+    "plt.plot(val_loss[:,0],val_loss[:,1],label='validation loss',color='green')\n",
+    "plt.title('Training and validation loss')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 512 records in 0.020173091 seconds. Throughput is 25380.344 records/second. Loss is 0.0092472015.\n",
+    "Top1Accuracy is Accuracy(correct: 8707, count: 10000, accuracy: 0.8707)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The blue line is the training loss, while the green line is the validation loss. Note that your own results may vary \n",
+    "slightly due to a different random initialization of your network."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the training loss decreases with every epoch. That's what you would \n",
+    "expect when running gradient descent optimization -- the quantity you are trying to minimize should get lower with every iteration. But that \n",
+    "isn't the case for the validation loss: it seems to be optimized at about 1/5 of the total training epochs, which is 20/5 = 4. This is an example of what we were warning \n",
+    "against earlier: a model that performs better on the training data isn't necessarily a model that will do better on data it has never seen \n",
+    "before. In precise terms, what you are seeing is \"overfitting\": after the second epoch, we are over-optimizing on the training data, and we \n",
+    "ended up learning representations that are specific to the training data and do not generalize to data outside of the training set.\n",
+    "\n",
+    "In this case, to prevent overfitting, we could simply stop training after three epochs. In general, there is a range of techniques you can \n",
+    "leverage to mitigate overfitting, which we will cover in the next chapter.\n",
+    "\n",
+    "Let's train a new network from scratch for 4 epochs, then evaluate it on our test data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = models.Sequential()\n",
+    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "model.add(layers.Dense(16, activation='relu'))\n",
+    "model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='binary_crossentropy',\n",
+    "              metrics=['accuracy'])\n",
+    "\n",
+    "model.fit(x_train, y_train, nb_epoch=4, batch_size=512)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 512 records in 0.023978868 seconds. Throughput is 21352.133 records/second. Loss is 0.108611815._"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[0.3026159405708313, 0.8824800252914429]"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "results = model.evaluate(x_test, y_test)\n",
+    "results"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our fairly naive approach achieves an accuracy of 88%. With state-of-the-art approaches, one should be able to get close to 95%."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using a trained network to generate predictions on new data\n",
+    "\n",
+    "After having trained a network, you will want to use it in a practical setting. You can generate the likelihood of reviews being positive \n",
+    "by using the `predict` method:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Predict result\n",
+    "_In Keras, one could just call following code to predict the test data_\n",
+    "\n",
+    "    model.predict(x_test)\n",
+    "_In analytics-zoo, the return of `predict` is RDD, so you need to call `collect` method to get the result:_"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[array([0.9291455], dtype=float32),\n",
+       " array([0.96646625], dtype=float32),\n",
+       " array([0.9992888], dtype=float32),\n",
+       " array([0.6787984], dtype=float32),\n",
+       " array([0.8021081], dtype=float32),\n",
+       " array([0.21841279], dtype=float32),\n",
+       " array([0.00925558], dtype=float32),\n",
+       " array([0.03703438], dtype=float32),\n",
+       " array([0.9931043], dtype=float32),\n",
+       " array([0.77527326], dtype=float32),\n",
+       " array([0.57820714], dtype=float32),\n",
+       " array([0.9590181], dtype=float32),\n",
+       " array([0.23954743], dtype=float32),\n",
+       " array([0.99956185], dtype=float32),\n",
+       " array([0.99986804], dtype=float32),\n",
+       " array([0.89252], dtype=float32),\n",
+       " array([0.6362871], dtype=float32),\n",
+       " array([0.00453862], dtype=float32),\n",
+       " array([0.00022487], dtype=float32),\n",
+       " array([0.5878058], dtype=float32),\n",
+       " array([0.79077387], dtype=float32),\n",
+       " array([0.951337], dtype=float32),\n",
+       " array([0.02377308], dtype=float32),\n",
+       " array([0.01233012], dtype=float32),\n",
+       " array([2.1225938e-05], dtype=float32),\n",
+       " array([0.92153585], dtype=float32),\n",
+       " array([0.27290833], dtype=float32),\n",
+       " array([0.9678427], dtype=float32),\n",
+       " array([0.98325783], dtype=float32),\n",
+       " array([0.01623936], dtype=float32),\n",
+       " array([0.9855356], dtype=float32),\n",
+       " array([0.00476582], dtype=float32),\n",
+       " array([0.99998844], dtype=float32),\n",
+       " array([0.37596926], dtype=float32),\n",
+       " array([0.99066436], dtype=float32),\n",
+       " array([0.995574], dtype=float32),\n",
+       " array([0.92657614], dtype=float32),\n",
+       " array([0.00201875], dtype=float32),\n",
+       " array([0.04801914], dtype=float32),\n",
+       " array([0.9741373], dtype=float32),\n",
+       " array([0.11522112], dtype=float32),\n",
+       " array([0.26702783], dtype=float32),\n",
+       " array([0.9866962], dtype=float32),\n",
+       " array([0.9840716], dtype=float32),\n",
+       " array([0.7866682], dtype=float32),\n",
+       " array([0.8413062], dtype=float32),\n",
+       " array([0.5110062], dtype=float32),\n",
+       " array([0.9998983], dtype=float32),\n",
+       " array([0.80300266], dtype=float32),\n",
+       " array([0.914983], dtype=float32),\n",
+       " array([0.9802167], dtype=float32),\n",
+       " array([0.99499327], dtype=float32),\n",
+       " array([0.6802777], dtype=float32),\n",
+       " array([0.94399655], dtype=float32),\n",
+       " array([0.00192062], dtype=float32),\n",
+       " array([0.01228456], dtype=float32),\n",
+       " array([0.0108898], dtype=float32),\n",
+       " array([0.08844584], dtype=float32),\n",
+       " array([0.00843766], dtype=float32),\n",
+       " array([0.975319], dtype=float32),\n",
+       " array([0.9284045], dtype=float32),\n",
+       " array([0.28974563], dtype=float32),\n",
+       " array([0.00184639], dtype=float32),\n",
+       " array([0.7453014], dtype=float32),\n",
+       " array([0.08012109], dtype=float32),\n",
+       " array([0.00030172], dtype=float32),\n",
+       " array([0.9998647], dtype=float32),\n",
+       " array([0.9998778], dtype=float32),\n",
+       " array([0.99604577], dtype=float32),\n",
+       " array([0.9999883], dtype=float32),\n",
+       " array([0.03597537], dtype=float32),\n",
+       " array([0.9160098], dtype=float32),\n",
+       " array([0.1100359], dtype=float32),\n",
+       " array([0.21295448], dtype=float32),\n",
+       " array([0.45663244], dtype=float32),\n",
+       " array([0.9961888], dtype=float32),\n",
+       " array([0.3244737], dtype=float32),\n",
+       " array([0.99997187], dtype=float32),\n",
+       " array([0.12168489], dtype=float32),\n",
+       " array([0.04461992], dtype=float32),\n",
+       " array([0.05755902], dtype=float32),\n",
+       " array([0.16763222], dtype=float32),\n",
+       " array([0.87495023], dtype=float32),\n",
+       " array([0.21862818], dtype=float32),\n",
+       " array([0.01208456], dtype=float32),\n",
+       " array([0.10023356], dtype=float32),\n",
+       " array([0.29649988], dtype=float32),\n",
+       " array([0.99824023], dtype=float32),\n",
+       " array([0.39383507], dtype=float32),\n",
+       " array([0.3296326], dtype=float32),\n",
+       " array([0.68079424], dtype=float32),\n",
+       " array([0.7821922], dtype=float32),\n",
+       " array([0.99843687], dtype=float32),\n",
+       " array([0.998406], dtype=float32),\n",
+       " array([0.09952371], dtype=float32),\n",
+       " array([0.20677786], dtype=float32),\n",
+       " array([0.07357709], dtype=float32),\n",
+       " array([0.99855345], dtype=float32),\n",
+       " array([0.00219191], dtype=float32),\n",
+       " array([0.9438781], dtype=float32),\n",
+       " array([0.11632669], dtype=float32),\n",
+       " array([0.31134847], dtype=float32),\n",
+       " array([0.12542696], dtype=float32),\n",
+       " array([0.9763289], dtype=float32),\n",
+       " array([0.9640528], dtype=float32),\n",
+       " array([0.6855567], dtype=float32),\n",
+       " array([0.9791134], dtype=float32),\n",
+       " array([0.47091678], dtype=float32),\n",
+       " array([0.01279368], dtype=float32),\n",
+       " array([0.33637416], dtype=float32),\n",
+       " array([0.92876065], dtype=float32),\n",
+       " array([0.02191273], dtype=float32),\n",
+       " array([0.02534392], dtype=float32),\n",
+       " array([0.27622753], dtype=float32),\n",
+       " array([0.03425935], dtype=float32),\n",
+       " array([0.11640935], dtype=float32),\n",
+       " array([0.99640983], dtype=float32),\n",
+       " array([0.99434376], dtype=float32),\n",
+       " array([0.02413097], dtype=float32),\n",
+       " array([0.36645678], dtype=float32),\n",
+       " array([0.01748311], dtype=float32),\n",
+       " array([0.18354651], dtype=float32),\n",
+       " array([0.06130786], dtype=float32),\n",
+       " array([0.21773124], dtype=float32),\n",
+       " array([0.95380867], dtype=float32),\n",
+       " array([0.5504796], dtype=float32),\n",
+       " array([0.0219801], dtype=float32),\n",
+       " array([0.01981366], dtype=float32),\n",
+       " array([0.00031499], dtype=float32),\n",
+       " array([0.13779135], dtype=float32),\n",
+       " array([0.9984407], dtype=float32),\n",
+       " array([0.40540016], dtype=float32),\n",
+       " array([0.77313596], dtype=float32),\n",
+       " array([0.01747493], dtype=float32),\n",
+       " array([0.0557996], dtype=float32),\n",
+       " array([0.06081589], dtype=float32),\n",
+       " array([0.04389222], dtype=float32),\n",
+       " array([0.9974957], dtype=float32),\n",
+       " array([0.5977306], dtype=float32),\n",
+       " array([0.02096312], dtype=float32),\n",
+       " array([0.8821718], dtype=float32),\n",
+       " array([0.00831421], dtype=float32),\n",
+       " array([0.85648173], dtype=float32),\n",
+       " array([0.61687416], dtype=float32),\n",
+       " array([0.00555464], dtype=float32),\n",
+       " array([0.9578813], dtype=float32),\n",
+       " array([0.16184929], dtype=float32),\n",
+       " array([0.8980497], dtype=float32),\n",
+       " array([0.99223125], dtype=float32),\n",
+       " array([0.0132063], dtype=float32),\n",
+       " array([0.92253935], dtype=float32),\n",
+       " array([0.06267989], dtype=float32),\n",
+       " array([0.9891216], dtype=float32),\n",
+       " array([0.9419726], dtype=float32),\n",
+       " array([0.00210088], dtype=float32),\n",
+       " array([0.99873346], dtype=float32),\n",
+       " array([0.01406829], dtype=float32),\n",
+       " array([0.08238389], dtype=float32),\n",
+       " array([0.7304331], dtype=float32),\n",
+       " array([0.07515999], dtype=float32),\n",
+       " array([0.00137386], dtype=float32),\n",
+       " array([0.999446], dtype=float32),\n",
+       " array([0.06388262], dtype=float32),\n",
+       " array([0.1658486], dtype=float32),\n",
+       " array([0.99999976], dtype=float32),\n",
+       " array([0.9154651], dtype=float32),\n",
+       " array([0.9564062], dtype=float32),\n",
+       " array([0.9038421], dtype=float32),\n",
+       " array([0.9414884], dtype=float32),\n",
+       " array([0.023891], dtype=float32),\n",
+       " array([0.27174172], dtype=float32),\n",
+       " array([0.9541309], dtype=float32),\n",
+       " array([0.06518997], dtype=float32),\n",
+       " array([0.01072453], dtype=float32),\n",
+       " array([0.99960166], dtype=float32),\n",
+       " array([0.0525161], dtype=float32),\n",
+       " array([0.00074362], dtype=float32),\n",
+       " array([0.84470475], dtype=float32),\n",
+       " array([0.68433523], dtype=float32),\n",
+       " array([0.73134536], dtype=float32),\n",
+       " array([0.02615881], dtype=float32),\n",
+       " array([0.9435008], dtype=float32),\n",
+       " array([0.9924217], dtype=float32),\n",
+       " array([0.81417906], dtype=float32),\n",
+       " array([0.99532396], dtype=float32),\n",
+       " array([0.11175746], dtype=float32),\n",
+       " array([0.01164351], dtype=float32),\n",
+       " array([0.99615234], dtype=float32),\n",
+       " array([0.99891615], dtype=float32),\n",
+       " array([0.85309887], dtype=float32),\n",
+       " array([0.3838549], dtype=float32),\n",
+       " array([0.08728907], dtype=float32),\n",
+       " array([0.99386746], dtype=float32),\n",
+       " array([0.99560165], dtype=float32),\n",
+       " array([0.01668872], dtype=float32),\n",
+       " array([0.865859], dtype=float32),\n",
+       " array([0.00344433], dtype=float32),\n",
+       " array([0.10099232], dtype=float32),\n",
+       " array([0.18755046], dtype=float32),\n",
+       " array([0.17657793], dtype=float32),\n",
+       " array([0.99963737], dtype=float32),\n",
+       " array([0.1608229], dtype=float32),\n",
+       " array([0.99993265], dtype=float32),\n",
+       " array([0.9839023], dtype=float32),\n",
+       " array([0.8809537], dtype=float32),\n",
+       " array([0.16208851], dtype=float32),\n",
+       " array([0.9696871], dtype=float32),\n",
+       " array([0.9999682], dtype=float32),\n",
+       " array([0.06924216], dtype=float32),\n",
+       " array([0.03222934], dtype=float32),\n",
+       " array([0.01602055], dtype=float32),\n",
+       " array([0.16013445], dtype=float32),\n",
+       " array([0.13429435], dtype=float32),\n",
+       " array([0.9997141], dtype=float32),\n",
+       " array([0.5197483], dtype=float32),\n",
+       " array([0.02680757], dtype=float32),\n",
+       " array([0.9984849], dtype=float32),\n",
+       " array([0.00670389], dtype=float32),\n",
+       " array([0.04960306], dtype=float32),\n",
+       " array([0.05909725], dtype=float32),\n",
+       " array([0.07385926], dtype=float32),\n",
+       " array([0.01410465], dtype=float32),\n",
+       " array([0.4758584], dtype=float32),\n",
+       " array([0.9994578], dtype=float32),\n",
+       " array([0.00207514], dtype=float32),\n",
+       " array([0.98792577], dtype=float32),\n",
+       " array([0.99999547], dtype=float32),\n",
+       " array([3.411692e-06], dtype=float32),\n",
+       " array([0.7747846], dtype=float32),\n",
+       " array([0.91780055], dtype=float32),\n",
+       " array([0.9927326], dtype=float32),\n",
+       " array([0.2352484], dtype=float32),\n",
+       " array([0.00142602], dtype=float32),\n",
+       " array([0.32317147], dtype=float32),\n",
+       " array([0.00565691], dtype=float32),\n",
+       " array([0.53445995], dtype=float32),\n",
+       " array([0.8927338], dtype=float32),\n",
+       " array([0.13075478], dtype=float32),\n",
+       " array([0.92551], dtype=float32),\n",
+       " array([0.06454863], dtype=float32),\n",
+       " array([0.945902], dtype=float32),\n",
+       " array([0.98765355], dtype=float32),\n",
+       " array([0.00029585], dtype=float32),\n",
+       " array([0.02011549], dtype=float32),\n",
+       " array([0.03295863], dtype=float32),\n",
+       " array([0.00324995], dtype=float32),\n",
+       " array([0.01008756], dtype=float32),\n",
+       " array([0.9823131], dtype=float32),\n",
+       " array([0.27388793], dtype=float32),\n",
+       " array([0.5470663], dtype=float32),\n",
+       " array([0.00781587], dtype=float32),\n",
+       " array([0.005428], dtype=float32),\n",
+       " array([0.9992046], dtype=float32),\n",
+       " array([0.11337327], dtype=float32),\n",
+       " array([0.0242104], dtype=float32),\n",
+       " array([0.06808829], dtype=float32),\n",
+       " array([0.9719501], dtype=float32),\n",
+       " array([0.960842], dtype=float32),\n",
+       " array([0.05452807], dtype=float32),\n",
+       " array([0.9993693], dtype=float32),\n",
+       " array([0.7771042], dtype=float32),\n",
+       " array([0.99957746], dtype=float32),\n",
+       " array([0.05997226], dtype=float32),\n",
+       " array([0.903902], dtype=float32),\n",
+       " array([0.86632144], dtype=float32),\n",
+       " array([0.99996936], dtype=float32),\n",
+       " array([0.69629955], dtype=float32),\n",
+       " array([0.8225713], dtype=float32),\n",
+       " array([0.00246663], dtype=float32),\n",
+       " array([0.0023386], dtype=float32),\n",
+       " array([0.9990294], dtype=float32),\n",
+       " array([0.9977755], dtype=float32),\n",
+       " array([0.9861735], dtype=float32),\n",
+       " array([0.03454849], dtype=float32),\n",
+       " array([0.08657297], dtype=float32),\n",
+       " array([0.9999199], dtype=float32),\n",
+       " array([0.9969818], dtype=float32),\n",
+       " array([0.05295636], dtype=float32),\n",
+       " array([0.99947375], dtype=float32),\n",
+       " array([0.01684208], dtype=float32),\n",
+       " array([0.00564773], dtype=float32),\n",
+       " array([0.00795649], dtype=float32),\n",
+       " array([0.9999298], dtype=float32),\n",
+       " array([0.06059966], dtype=float32),\n",
+       " array([0.9730349], dtype=float32),\n",
+       " array([0.3703475], dtype=float32),\n",
+       " array([0.00036947], dtype=float32),\n",
+       " array([0.3769037], dtype=float32),\n",
+       " array([0.00783887], dtype=float32),\n",
+       " array([0.0283057], dtype=float32),\n",
+       " array([0.04452141], dtype=float32),\n",
+       " array([0.47198933], dtype=float32),\n",
+       " array([0.99648005], dtype=float32),\n",
+       " array([0.9991768], dtype=float32),\n",
+       " array([0.99995506], dtype=float32),\n",
+       " array([0.1129663], dtype=float32),\n",
+       " array([0.01932632], dtype=float32),\n",
+       " array([0.01605185], dtype=float32),\n",
+       " array([0.9423293], dtype=float32),\n",
+       " array([0.06175272], dtype=float32),\n",
+       " array([0.99719644], dtype=float32),\n",
+       " array([0.10867236], dtype=float32),\n",
+       " array([0.02944934], dtype=float32),\n",
+       " array([0.9235288], dtype=float32),\n",
+       " array([0.47749949], dtype=float32),\n",
+       " array([0.88871026], dtype=float32),\n",
+       " array([0.11335868], dtype=float32),\n",
+       " array([0.9990363], dtype=float32),\n",
+       " array([0.03595558], dtype=float32),\n",
+       " array([0.19236687], dtype=float32),\n",
+       " array([0.99891937], dtype=float32),\n",
+       " array([0.28199562], dtype=float32),\n",
+       " array([0.01422782], dtype=float32),\n",
+       " array([0.0095924], dtype=float32),\n",
+       " array([0.05586888], dtype=float32),\n",
+       " array([0.90418166], dtype=float32),\n",
+       " array([0.06067238], dtype=float32),\n",
+       " array([0.12407589], dtype=float32),\n",
+       " array([0.00962292], dtype=float32),\n",
+       " array([0.01531602], dtype=float32),\n",
+       " array([0.00537257], dtype=float32),\n",
+       " array([0.3383139], dtype=float32),\n",
+       " array([0.9790156], dtype=float32),\n",
+       " array([0.99629223], dtype=float32),\n",
+       " array([0.999041], dtype=float32),\n",
+       " array([0.4543444], dtype=float32),\n",
+       " array([0.9975923], dtype=float32),\n",
+       " array([0.994676], dtype=float32),\n",
+       " array([0.9955705], dtype=float32),\n",
+       " array([0.00852212], dtype=float32),\n",
+       " array([0.00965995], dtype=float32),\n",
+       " array([0.04810032], dtype=float32),\n",
+       " array([0.03414589], dtype=float32),\n",
+       " array([0.19549885], dtype=float32),\n",
+       " array([0.04221633], dtype=float32),\n",
+       " array([0.999081], dtype=float32),\n",
+       " array([0.95819485], dtype=float32),\n",
+       " array([0.02422899], dtype=float32),\n",
+       " array([0.00078607], dtype=float32),\n",
+       " array([0.01256537], dtype=float32),\n",
+       " array([0.573112], dtype=float32),\n",
+       " array([0.97446126], dtype=float32),\n",
+       " array([0.01481443], dtype=float32),\n",
+       " array([0.91800165], dtype=float32),\n",
+       " array([0.04669483], dtype=float32),\n",
+       " array([0.12667258], dtype=float32),\n",
+       " array([0.9989146], dtype=float32),\n",
+       " array([0.11938414], dtype=float32),\n",
+       " array([0.8915277], dtype=float32),\n",
+       " array([0.01737923], dtype=float32),\n",
+       " array([0.9999982], dtype=float32),\n",
+       " array([0.96499765], dtype=float32),\n",
+       " array([0.02043628], dtype=float32),\n",
+       " array([0.6315207], dtype=float32),\n",
+       " array([0.9999362], dtype=float32),\n",
+       " array([0.34459296], dtype=float32),\n",
+       " array([0.98566204], dtype=float32),\n",
+       " array([0.97014564], dtype=float32),\n",
+       " array([0.99786866], dtype=float32),\n",
+       " array([0.01015446], dtype=float32),\n",
+       " array([0.8746796], dtype=float32),\n",
+       " array([0.9308818], dtype=float32),\n",
+       " array([0.00047523], dtype=float32),\n",
+       " array([0.99945456], dtype=float32),\n",
+       " array([0.00871587], dtype=float32),\n",
+       " array([0.87762976], dtype=float32),\n",
+       " array([0.00176486], dtype=float32),\n",
+       " array([0.9776403], dtype=float32),\n",
+       " array([0.00964555], dtype=float32),\n",
+       " array([0.38256386], dtype=float32),\n",
+       " array([0.9978903], dtype=float32),\n",
+       " array([0.48501348], dtype=float32),\n",
+       " array([0.9758839], dtype=float32),\n",
+       " array([0.76296157], dtype=float32),\n",
+       " array([0.00493866], dtype=float32),\n",
+       " array([0.05346973], dtype=float32),\n",
+       " array([0.9999949], dtype=float32),\n",
+       " array([0.00160436], dtype=float32),\n",
+       " array([0.00270788], dtype=float32),\n",
+       " array([0.2500689], dtype=float32),\n",
+       " array([0.01582536], dtype=float32),\n",
+       " array([0.8722655], dtype=float32),\n",
+       " array([0.95772576], dtype=float32),\n",
+       " array([0.9999635], dtype=float32),\n",
+       " array([0.86149096], dtype=float32),\n",
+       " array([0.12169719], dtype=float32),\n",
+       " array([0.28068733], dtype=float32),\n",
+       " array([0.00027394], dtype=float32),\n",
+       " array([0.00176786], dtype=float32),\n",
+       " array([0.00266076], dtype=float32),\n",
+       " array([0.00955428], dtype=float32),\n",
+       " array([0.06166862], dtype=float32),\n",
+       " array([0.96516556], dtype=float32),\n",
+       " array([0.99725515], dtype=float32),\n",
+       " array([0.86680585], dtype=float32),\n",
+       " array([0.7473102], dtype=float32),\n",
+       " array([0.09695191], dtype=float32),\n",
+       " array([0.00296136], dtype=float32),\n",
+       " array([0.00260568], dtype=float32),\n",
+       " array([0.9957995], dtype=float32),\n",
+       " array([0.99882144], dtype=float32),\n",
+       " array([0.00024917], dtype=float32),\n",
+       " array([0.9272133], dtype=float32),\n",
+       " array([0.00036355], dtype=float32),\n",
+       " array([0.9843275], dtype=float32),\n",
+       " array([0.02331734], dtype=float32),\n",
+       " array([0.10120519], dtype=float32),\n",
+       " array([0.47101545], dtype=float32),\n",
+       " array([0.02735368], dtype=float32),\n",
+       " array([0.9993734], dtype=float32),\n",
+       " array([0.9975063], dtype=float32),\n",
+       " array([0.9815989], dtype=float32),\n",
+       " array([0.9998317], dtype=float32),\n",
+       " array([0.08344828], dtype=float32),\n",
+       " array([0.9794001], dtype=float32),\n",
+       " array([0.9987134], dtype=float32),\n",
+       " array([0.00018663], dtype=float32),\n",
+       " array([0.8894404], dtype=float32),\n",
+       " array([0.04193362], dtype=float32),\n",
+       " array([0.99497104], dtype=float32),\n",
+       " array([0.04068065], dtype=float32),\n",
+       " array([0.00536961], dtype=float32),\n",
+       " array([0.06312026], dtype=float32),\n",
+       " array([0.05857188], dtype=float32),\n",
+       " array([0.68636453], dtype=float32),\n",
+       " array([0.9688917], dtype=float32),\n",
+       " array([0.19696496], dtype=float32),\n",
+       " array([0.06663571], dtype=float32),\n",
+       " array([0.03183109], dtype=float32),\n",
+       " array([0.96671313], dtype=float32),\n",
+       " array([0.05208089], dtype=float32),\n",
+       " array([0.09314187], dtype=float32),\n",
+       " array([0.9926622], dtype=float32),\n",
+       " array([0.92366344], dtype=float32),\n",
+       " array([0.9675213], dtype=float32),\n",
+       " array([0.01860829], dtype=float32),\n",
+       " array([0.9165048], dtype=float32),\n",
+       " array([0.94261616], dtype=float32),\n",
+       " array([0.0835857], dtype=float32),\n",
+       " array([0.00041597], dtype=float32),\n",
+       " array([0.00907566], dtype=float32),\n",
+       " array([0.94010156], dtype=float32),\n",
+       " array([0.9840152], dtype=float32),\n",
+       " array([0.00041544], dtype=float32),\n",
+       " array([0.8775654], dtype=float32),\n",
+       " array([0.32026634], dtype=float32),\n",
+       " array([0.01421119], dtype=float32),\n",
+       " array([0.0130173], dtype=float32),\n",
+       " array([0.9884545], dtype=float32),\n",
+       " array([0.04571154], dtype=float32),\n",
+       " array([1.6011174e-05], dtype=float32),\n",
+       " array([0.00115296], dtype=float32),\n",
+       " array([0.9790429], dtype=float32),\n",
+       " array([0.64808387], dtype=float32),\n",
+       " array([4.6414338e-05], dtype=float32),\n",
+       " array([0.5914479], dtype=float32),\n",
+       " array([0.00711486], dtype=float32),\n",
+       " array([0.82269937], dtype=float32),\n",
+       " array([0.66748506], dtype=float32),\n",
+       " array([0.9879972], dtype=float32),\n",
+       " array([6.153964e-05], dtype=float32),\n",
+       " array([0.99550027], dtype=float32),\n",
+       " array([0.00570012], dtype=float32),\n",
+       " array([0.46884078], dtype=float32),\n",
+       " array([0.00010328], dtype=float32),\n",
+       " array([0.03197822], dtype=float32),\n",
+       " array([0.9972145], dtype=float32),\n",
+       " array([0.05284659], dtype=float32),\n",
+       " array([0.9037368], dtype=float32),\n",
+       " array([0.04048614], dtype=float32),\n",
+       " array([0.998744], dtype=float32),\n",
+       " array([0.99049884], dtype=float32),\n",
+       " array([0.01359443], dtype=float32),\n",
+       " array([0.9997179], dtype=float32),\n",
+       " array([0.99999964], dtype=float32),\n",
+       " array([0.98721075], dtype=float32),\n",
+       " array([0.00063402], dtype=float32),\n",
+       " array([0.820883], dtype=float32),\n",
+       " array([0.4547376], dtype=float32),\n",
+       " array([0.891108], dtype=float32),\n",
+       " array([0.16223074], dtype=float32),\n",
+       " array([0.9726654], dtype=float32),\n",
+       " array([0.9003827], dtype=float32),\n",
+       " array([0.99944574], dtype=float32),\n",
+       " array([0.7704998], dtype=float32),\n",
+       " array([0.95534146], dtype=float32),\n",
+       " array([0.0062368], dtype=float32),\n",
+       " array([0.9787254], dtype=float32),\n",
+       " array([0.00126028], dtype=float32),\n",
+       " array([0.7004171], dtype=float32),\n",
+       " array([0.09580212], dtype=float32),\n",
+       " array([0.97376186], dtype=float32),\n",
+       " array([0.9920665], dtype=float32),\n",
+       " array([0.12573221], dtype=float32),\n",
+       " array([0.96389884], dtype=float32),\n",
+       " array([0.9980578], dtype=float32),\n",
+       " array([0.00578512], dtype=float32),\n",
+       " array([0.01206519], dtype=float32),\n",
+       " array([0.9992673], dtype=float32),\n",
+       " array([0.07898365], dtype=float32),\n",
+       " array([0.00214792], dtype=float32),\n",
+       " array([0.05026222], dtype=float32),\n",
+       " array([0.99995553], dtype=float32),\n",
+       " array([0.99840695], dtype=float32),\n",
+       " array([0.99762005], dtype=float32),\n",
+       " array([1.], dtype=float32),\n",
+       " array([0.7520437], dtype=float32),\n",
+       " array([0.03680907], dtype=float32),\n",
+       " array([0.5764651], dtype=float32),\n",
+       " array([0.99977905], dtype=float32),\n",
+       " array([0.20490777], dtype=float32),\n",
+       " array([0.00173326], dtype=float32),\n",
+       " array([0.9939528], dtype=float32),\n",
+       " array([1.9042971e-05], dtype=float32),\n",
+       " array([0.95325047], dtype=float32),\n",
+       " array([0.9958734], dtype=float32),\n",
+       " array([0.03266709], dtype=float32),\n",
+       " array([0.02785043], dtype=float32),\n",
+       " array([0.05891791], dtype=float32),\n",
+       " array([0.985227], dtype=float32),\n",
+       " array([0.00210726], dtype=float32),\n",
+       " array([0.9943926], dtype=float32),\n",
+       " array([0.02131884], dtype=float32),\n",
+       " array([0.99827754], dtype=float32),\n",
+       " array([0.8846837], dtype=float32),\n",
+       " array([0.99997437], dtype=float32),\n",
+       " array([0.9946067], dtype=float32),\n",
+       " array([0.99978703], dtype=float32),\n",
+       " array([0.00300169], dtype=float32),\n",
+       " array([0.00031111], dtype=float32),\n",
+       " array([0.9819504], dtype=float32),\n",
+       " array([0.00375891], dtype=float32),\n",
+       " array([0.63086605], dtype=float32),\n",
+       " array([0.83123654], dtype=float32),\n",
+       " array([0.63774806], dtype=float32),\n",
+       " array([0.6908987], dtype=float32),\n",
+       " array([0.15456767], dtype=float32),\n",
+       " array([0.4055819], dtype=float32),\n",
+       " array([0.0910763], dtype=float32),\n",
+       " array([0.24731727], dtype=float32),\n",
+       " array([0.994842], dtype=float32),\n",
+       " array([0.38033506], dtype=float32),\n",
+       " array([0.98958546], dtype=float32),\n",
+       " array([0.9998734], dtype=float32),\n",
+       " array([0.99897206], dtype=float32),\n",
+       " array([0.8665204], dtype=float32),\n",
+       " array([0.00639246], dtype=float32),\n",
+       " array([0.9556338], dtype=float32),\n",
+       " array([0.9666423], dtype=float32),\n",
+       " array([0.9984849], dtype=float32),\n",
+       " array([0.02892273], dtype=float32),\n",
+       " array([0.995031], dtype=float32),\n",
+       " array([0.72732645], dtype=float32),\n",
+       " array([0.998869], dtype=float32),\n",
+       " array([0.01101623], dtype=float32),\n",
+       " array([0.9236663], dtype=float32),\n",
+       " array([0.0053416], dtype=float32),\n",
+       " array([0.9376339], dtype=float32),\n",
+       " array([0.9097744], dtype=float32),\n",
+       " array([0.32959577], dtype=float32),\n",
+       " array([0.69777864], dtype=float32),\n",
+       " array([0.02821315], dtype=float32),\n",
+       " array([0.8764768], dtype=float32),\n",
+       " array([0.01698065], dtype=float32),\n",
+       " array([0.05337453], dtype=float32),\n",
+       " array([0.00699902], dtype=float32),\n",
+       " array([0.01098163], dtype=float32),\n",
+       " array([0.02664173], dtype=float32),\n",
+       " array([0.19115667], dtype=float32),\n",
+       " array([0.01039254], dtype=float32),\n",
+       " array([0.7853336], dtype=float32),\n",
+       " array([0.13310696], dtype=float32),\n",
+       " array([0.12221986], dtype=float32),\n",
+       " array([0.99144626], dtype=float32),\n",
+       " array([0.12488245], dtype=float32),\n",
+       " array([0.10422938], dtype=float32),\n",
+       " array([0.03960704], dtype=float32),\n",
+       " array([0.96108264], dtype=float32),\n",
+       " array([0.00321886], dtype=float32),\n",
+       " array([0.9612626], dtype=float32),\n",
+       " array([0.77753425], dtype=float32),\n",
+       " array([0.992634], dtype=float32),\n",
+       " array([0.08653396], dtype=float32),\n",
+       " array([0.573064], dtype=float32),\n",
+       " array([0.994193], dtype=float32),\n",
+       " array([0.9994568], dtype=float32),\n",
+       " array([0.00164113], dtype=float32),\n",
+       " array([0.02974954], dtype=float32),\n",
+       " array([0.00094306], dtype=float32),\n",
+       " array([0.01469964], dtype=float32),\n",
+       " array([0.01007712], dtype=float32),\n",
+       " array([0.5073839], dtype=float32),\n",
+       " array([0.00891581], dtype=float32),\n",
+       " array([0.01340619], dtype=float32),\n",
+       " array([0.19153547], dtype=float32),\n",
+       " array([0.00785635], dtype=float32),\n",
+       " array([0.00529725], dtype=float32),\n",
+       " array([0.9981645], dtype=float32),\n",
+       " array([0.89561176], dtype=float32),\n",
+       " array([0.44029003], dtype=float32),\n",
+       " array([0.9998586], dtype=float32),\n",
+       " array([0.09905386], dtype=float32),\n",
+       " array([0.9964311], dtype=float32),\n",
+       " array([0.9990119], dtype=float32),\n",
+       " array([0.85216373], dtype=float32),\n",
+       " array([0.9999999], dtype=float32),\n",
+       " array([0.9584791], dtype=float32),\n",
+       " array([0.00405316], dtype=float32),\n",
+       " array([0.9539604], dtype=float32),\n",
+       " array([0.9477786], dtype=float32),\n",
+       " array([0.99440527], dtype=float32),\n",
+       " array([0.9769205], dtype=float32),\n",
+       " array([0.89980936], dtype=float32),\n",
+       " array([0.00249994], dtype=float32),\n",
+       " array([0.8010222], dtype=float32),\n",
+       " array([0.20035101], dtype=float32),\n",
+       " array([0.00624598], dtype=float32),\n",
+       " array([0.80348605], dtype=float32),\n",
+       " array([0.19786465], dtype=float32),\n",
+       " array([0.0849141], dtype=float32),\n",
+       " array([0.9986082], dtype=float32),\n",
+       " array([0.13313013], dtype=float32),\n",
+       " array([0.16162278], dtype=float32),\n",
+       " array([0.993683], dtype=float32),\n",
+       " array([0.55489], dtype=float32),\n",
+       " array([0.07838932], dtype=float32),\n",
+       " array([0.008188], dtype=float32),\n",
+       " array([0.00667501], dtype=float32),\n",
+       " array([0.16437183], dtype=float32),\n",
+       " array([0.96623826], dtype=float32),\n",
+       " array([0.8830661], dtype=float32),\n",
+       " array([0.9948244], dtype=float32),\n",
+       " array([0.98182225], dtype=float32),\n",
+       " array([0.98158526], dtype=float32),\n",
+       " array([0.03434094], dtype=float32),\n",
+       " array([0.3056716], dtype=float32),\n",
+       " array([0.98550564], dtype=float32),\n",
+       " array([0.03481789], dtype=float32),\n",
+       " array([0.99663454], dtype=float32),\n",
+       " array([0.9985311], dtype=float32),\n",
+       " array([0.9939167], dtype=float32),\n",
+       " array([0.10510859], dtype=float32),\n",
+       " array([0.00508491], dtype=float32),\n",
+       " array([0.00165993], dtype=float32),\n",
+       " array([0.8245149], dtype=float32),\n",
+       " array([0.9556251], dtype=float32),\n",
+       " array([0.9887638], dtype=float32),\n",
+       " array([0.17581154], dtype=float32),\n",
+       " array([0.00011383], dtype=float32),\n",
+       " array([0.00106234], dtype=float32),\n",
+       " array([0.01681961], dtype=float32),\n",
+       " array([0.00525573], dtype=float32),\n",
+       " array([0.9957818], dtype=float32),\n",
+       " array([0.8603696], dtype=float32),\n",
+       " array([0.00199909], dtype=float32),\n",
+       " array([0.04100141], dtype=float32),\n",
+       " array([0.00017335], dtype=float32),\n",
+       " array([0.9992545], dtype=float32),\n",
+       " array([0.9659639], dtype=float32),\n",
+       " array([0.8995427], dtype=float32),\n",
+       " array([0.12103864], dtype=float32),\n",
+       " array([0.710389], dtype=float32),\n",
+       " array([0.69011146], dtype=float32),\n",
+       " array([0.05472863], dtype=float32),\n",
+       " array([0.01922707], dtype=float32),\n",
+       " array([0.8451995], dtype=float32),\n",
+       " array([0.99947244], dtype=float32),\n",
+       " array([0.09135215], dtype=float32),\n",
+       " array([0.00159073], dtype=float32),\n",
+       " array([0.05302845], dtype=float32),\n",
+       " array([0.98199064], dtype=float32),\n",
+       " array([0.50679266], dtype=float32),\n",
+       " array([0.17369047], dtype=float32),\n",
+       " array([0.9998796], dtype=float32),\n",
+       " array([0.6899049], dtype=float32),\n",
+       " array([0.00706529], dtype=float32),\n",
+       " array([0.00957006], dtype=float32),\n",
+       " array([0.78653455], dtype=float32),\n",
+       " array([0.00385365], dtype=float32),\n",
+       " array([0.3887317], dtype=float32),\n",
+       " array([0.03332283], dtype=float32),\n",
+       " array([0.04271935], dtype=float32),\n",
+       " array([0.00691515], dtype=float32),\n",
+       " array([0.51850504], dtype=float32),\n",
+       " array([0.00992748], dtype=float32),\n",
+       " array([0.29605916], dtype=float32),\n",
+       " array([0.00028048], dtype=float32),\n",
+       " array([0.9928925], dtype=float32),\n",
+       " array([0.00124131], dtype=float32),\n",
+       " array([0.9764488], dtype=float32),\n",
+       " array([0.7932969], dtype=float32),\n",
+       " array([0.97071785], dtype=float32),\n",
+       " array([0.98584795], dtype=float32),\n",
+       " array([0.07081781], dtype=float32),\n",
+       " array([0.04931527], dtype=float32),\n",
+       " array([0.15695621], dtype=float32),\n",
+       " array([0.9994659], dtype=float32),\n",
+       " array([0.5096887], dtype=float32),\n",
+       " array([0.9917842], dtype=float32),\n",
+       " array([0.9981895], dtype=float32),\n",
+       " array([0.12094765], dtype=float32),\n",
+       " array([0.11894753], dtype=float32),\n",
+       " array([0.90818393], dtype=float32),\n",
+       " array([0.98540974], dtype=float32),\n",
+       " array([0.94457304], dtype=float32),\n",
+       " array([0.9998313], dtype=float32),\n",
+       " array([0.9907136], dtype=float32),\n",
+       " array([0.72061336], dtype=float32),\n",
+       " array([0.99645764], dtype=float32),\n",
+       " array([0.8892745], dtype=float32),\n",
+       " array([0.999595], dtype=float32),\n",
+       " array([0.4836757], dtype=float32),\n",
+       " array([0.97150517], dtype=float32),\n",
+       " array([0.95611787], dtype=float32),\n",
+       " array([0.01308193], dtype=float32),\n",
+       " array([0.00030093], dtype=float32),\n",
+       " array([0.99999774], dtype=float32),\n",
+       " array([0.00015023], dtype=float32),\n",
+       " array([0.00129508], dtype=float32),\n",
+       " array([0.02929328], dtype=float32),\n",
+       " array([0.15967111], dtype=float32),\n",
+       " array([0.04437197], dtype=float32),\n",
+       " array([0.00077572], dtype=float32),\n",
+       " array([0.00117108], dtype=float32),\n",
+       " array([0.02031642], dtype=float32),\n",
+       " array([0.78561157], dtype=float32),\n",
+       " array([0.6640868], dtype=float32),\n",
+       " array([0.96112865], dtype=float32),\n",
+       " array([0.9977581], dtype=float32),\n",
+       " array([0.24247183], dtype=float32),\n",
+       " array([0.3876404], dtype=float32),\n",
+       " array([0.00950658], dtype=float32),\n",
+       " array([0.12216215], dtype=float32),\n",
+       " array([0.00086611], dtype=float32),\n",
+       " array([0.99396956], dtype=float32),\n",
+       " array([0.77168643], dtype=float32),\n",
+       " array([0.00386494], dtype=float32),\n",
+       " array([0.9824516], dtype=float32),\n",
+       " array([0.9933523], dtype=float32),\n",
+       " array([0.95625204], dtype=float32),\n",
+       " array([0.00056513], dtype=float32),\n",
+       " array([0.949406], dtype=float32),\n",
+       " array([0.91392314], dtype=float32),\n",
+       " array([0.998437], dtype=float32),\n",
+       " array([0.00321832], dtype=float32),\n",
+       " array([0.3081614], dtype=float32),\n",
+       " array([0.18779686], dtype=float32),\n",
+       " array([0.86671174], dtype=float32),\n",
+       " array([0.99736756], dtype=float32),\n",
+       " array([0.37101072], dtype=float32),\n",
+       " array([0.97278196], dtype=float32),\n",
+       " array([0.03783982], dtype=float32),\n",
+       " array([0.98899263], dtype=float32),\n",
+       " array([0.99997747], dtype=float32),\n",
+       " array([0.33761773], dtype=float32),\n",
+       " array([0.9922051], dtype=float32),\n",
+       " array([0.9986929], dtype=float32),\n",
+       " array([0.9734451], dtype=float32),\n",
+       " array([0.00104312], dtype=float32),\n",
+       " array([0.00905259], dtype=float32),\n",
+       " array([0.9999858], dtype=float32),\n",
+       " array([0.2685942], dtype=float32),\n",
+       " array([8.498155e-07], dtype=float32),\n",
+       " array([0.00194448], dtype=float32),\n",
+       " array([0.9610404], dtype=float32),\n",
+       " array([0.06272461], dtype=float32),\n",
+       " array([0.9734326], dtype=float32),\n",
+       " array([0.9998591], dtype=float32),\n",
+       " array([0.02384088], dtype=float32),\n",
+       " array([0.00458063], dtype=float32),\n",
+       " array([0.8619814], dtype=float32),\n",
+       " array([0.3280481], dtype=float32),\n",
+       " array([0.58994853], dtype=float32),\n",
+       " array([0.00738818], dtype=float32),\n",
+       " array([0.9968213], dtype=float32),\n",
+       " array([0.94588715], dtype=float32),\n",
+       " array([0.89741385], dtype=float32),\n",
+       " array([0.0001792], dtype=float32),\n",
+       " array([7.942238e-05], dtype=float32),\n",
+       " array([0.2487981], dtype=float32),\n",
+       " array([0.99818295], dtype=float32),\n",
+       " array([0.06495794], dtype=float32),\n",
+       " array([0.61300725], dtype=float32),\n",
+       " array([0.00142293], dtype=float32),\n",
+       " array([0.7782267], dtype=float32),\n",
+       " array([0.70798534], dtype=float32),\n",
+       " array([0.15175731], dtype=float32),\n",
+       " array([0.99284136], dtype=float32),\n",
+       " array([0.9841339], dtype=float32),\n",
+       " array([0.00554728], dtype=float32),\n",
+       " array([0.0500682], dtype=float32),\n",
+       " array([1.7751601e-06], dtype=float32),\n",
+       " array([0.12731266], dtype=float32),\n",
+       " array([0.01886535], dtype=float32),\n",
+       " array([0.9990376], dtype=float32),\n",
+       " array([0.27495182], dtype=float32),\n",
+       " array([0.90534323], dtype=float32),\n",
+       " array([0.8381721], dtype=float32),\n",
+       " array([0.12258686], dtype=float32),\n",
+       " array([0.23695664], dtype=float32),\n",
+       " array([0.04559099], dtype=float32),\n",
+       " array([0.798738], dtype=float32),\n",
+       " array([0.9249577], dtype=float32),\n",
+       " array([0.5790399], dtype=float32),\n",
+       " array([0.7356898], dtype=float32),\n",
+       " array([0.9420959], dtype=float32),\n",
+       " array([0.80315626], dtype=float32),\n",
+       " array([0.907965], dtype=float32),\n",
+       " array([0.18890426], dtype=float32),\n",
+       " array([0.04044292], dtype=float32),\n",
+       " array([0.00435959], dtype=float32),\n",
+       " array([0.01255109], dtype=float32),\n",
+       " array([0.973041], dtype=float32),\n",
+       " array([0.89595586], dtype=float32),\n",
+       " array([0.15041849], dtype=float32),\n",
+       " array([0.7386434], dtype=float32),\n",
+       " array([0.01395628], dtype=float32),\n",
+       " array([0.00037464], dtype=float32),\n",
+       " array([0.30354175], dtype=float32),\n",
+       " array([0.92193896], dtype=float32),\n",
+       " array([0.95892274], dtype=float32),\n",
+       " array([1.066259e-06], dtype=float32),\n",
+       " array([0.96353555], dtype=float32),\n",
+       " array([0.14788437], dtype=float32),\n",
+       " array([0.9997639], dtype=float32),\n",
+       " array([0.01777537], dtype=float32),\n",
+       " array([0.9861092], dtype=float32),\n",
+       " array([0.13082978], dtype=float32),\n",
+       " array([0.0002504], dtype=float32),\n",
+       " array([0.8804745], dtype=float32),\n",
+       " array([0.9967051], dtype=float32),\n",
+       " array([0.56104803], dtype=float32),\n",
+       " array([0.36787862], dtype=float32),\n",
+       " array([0.8360154], dtype=float32),\n",
+       " array([0.9998766], dtype=float32),\n",
+       " array([0.00568995], dtype=float32),\n",
+       " array([0.00194393], dtype=float32),\n",
+       " array([0.00631262], dtype=float32),\n",
+       " array([0.03533027], dtype=float32),\n",
+       " array([0.9103368], dtype=float32),\n",
+       " array([0.9982439], dtype=float32),\n",
+       " array([0.97841996], dtype=float32),\n",
+       " array([0.00286406], dtype=float32),\n",
+       " array([0.0708506], dtype=float32),\n",
+       " array([0.9432028], dtype=float32),\n",
+       " array([0.9654381], dtype=float32),\n",
+       " array([0.05079986], dtype=float32),\n",
+       " array([0.8743878], dtype=float32),\n",
+       " array([0.00240675], dtype=float32),\n",
+       " array([0.98993146], dtype=float32),\n",
+       " array([0.07532773], dtype=float32),\n",
+       " array([0.22899462], dtype=float32),\n",
+       " array([0.00091621], dtype=float32),\n",
+       " array([0.9989504], dtype=float32),\n",
+       " array([0.39317238], dtype=float32),\n",
+       " array([0.3326581], dtype=float32),\n",
+       " array([0.01213577], dtype=float32),\n",
+       " array([0.99774724], dtype=float32),\n",
+       " array([0.9886003], dtype=float32),\n",
+       " array([0.79621345], dtype=float32),\n",
+       " array([0.79079646], dtype=float32),\n",
+       " array([0.93861336], dtype=float32),\n",
+       " array([0.07021908], dtype=float32),\n",
+       " array([0.7411332], dtype=float32),\n",
+       " array([0.969042], dtype=float32),\n",
+       " array([0.9099184], dtype=float32),\n",
+       " array([0.02733893], dtype=float32),\n",
+       " array([0.9999924], dtype=float32),\n",
+       " array([0.9897418], dtype=float32),\n",
+       " array([0.03869773], dtype=float32),\n",
+       " array([0.97638786], dtype=float32),\n",
+       " array([0.08542448], dtype=float32),\n",
+       " array([0.05407662], dtype=float32),\n",
+       " array([0.9999993], dtype=float32),\n",
+       " array([0.14986295], dtype=float32),\n",
+       " array([0.999286], dtype=float32),\n",
+       " array([0.24805169], dtype=float32),\n",
+       " array([0.01673634], dtype=float32),\n",
+       " array([0.01463007], dtype=float32),\n",
+       " array([0.3670616], dtype=float32),\n",
+       " array([0.9926224], dtype=float32),\n",
+       " array([0.6253009], dtype=float32),\n",
+       " array([0.03401245], dtype=float32),\n",
+       " array([0.00030278], dtype=float32),\n",
+       " array([0.96080303], dtype=float32),\n",
+       " array([0.04573576], dtype=float32),\n",
+       " array([0.04926103], dtype=float32),\n",
+       " array([0.5770354], dtype=float32),\n",
+       " array([0.02184597], dtype=float32),\n",
+       " array([0.9933947], dtype=float32),\n",
+       " array([0.00422782], dtype=float32),\n",
+       " array([0.7942074], dtype=float32),\n",
+       " array([0.14047284], dtype=float32),\n",
+       " array([0.90892816], dtype=float32),\n",
+       " array([0.79335517], dtype=float32),\n",
+       " array([0.02081179], dtype=float32),\n",
+       " array([0.03224256], dtype=float32),\n",
+       " array([0.00269347], dtype=float32),\n",
+       " array([0.7325373], dtype=float32),\n",
+       " array([0.86657375], dtype=float32),\n",
+       " array([0.9994041], dtype=float32),\n",
+       " array([0.99819297], dtype=float32),\n",
+       " array([0.306308], dtype=float32),\n",
+       " array([0.9358532], dtype=float32),\n",
+       " array([0.00968082], dtype=float32),\n",
+       " array([0.22723815], dtype=float32),\n",
+       " array([0.88686043], dtype=float32),\n",
+       " array([0.00376564], dtype=float32),\n",
+       " array([0.9558993], dtype=float32),\n",
+       " array([0.03709094], dtype=float32),\n",
+       " array([0.9284992], dtype=float32),\n",
+       " array([0.01156035], dtype=float32),\n",
+       " array([0.6904194], dtype=float32),\n",
+       " array([0.7789368], dtype=float32),\n",
+       " array([0.06749155], dtype=float32),\n",
+       " array([0.83822256], dtype=float32),\n",
+       " array([0.00499537], dtype=float32),\n",
+       " array([0.96375054], dtype=float32),\n",
+       " array([0.99763095], dtype=float32),\n",
+       " array([0.00083689], dtype=float32),\n",
+       " array([0.1384925], dtype=float32),\n",
+       " array([0.99911016], dtype=float32),\n",
+       " array([0.18213369], dtype=float32),\n",
+       " array([0.01104294], dtype=float32),\n",
+       " array([0.9997731], dtype=float32),\n",
+       " array([0.00157826], dtype=float32),\n",
+       " array([0.45021382], dtype=float32),\n",
+       " array([0.70889956], dtype=float32),\n",
+       " array([0.99980146], dtype=float32),\n",
+       " array([0.14717786], dtype=float32),\n",
+       " array([0.9981312], dtype=float32),\n",
+       " array([0.99910754], dtype=float32),\n",
+       " array([0.00473733], dtype=float32),\n",
+       " array([0.00330126], dtype=float32),\n",
+       " array([0.17611578], dtype=float32),\n",
+       " array([0.69635725], dtype=float32),\n",
+       " array([0.39411786], dtype=float32),\n",
+       " array([0.26741236], dtype=float32),\n",
+       " array([0.56975543], dtype=float32),\n",
+       " array([0.06516983], dtype=float32),\n",
+       " array([0.70290774], dtype=float32),\n",
+       " array([0.1079508], dtype=float32),\n",
+       " array([0.9905323], dtype=float32),\n",
+       " array([0.07408904], dtype=float32),\n",
+       " array([0.99945086], dtype=float32),\n",
+       " array([0.08830733], dtype=float32),\n",
+       " array([0.47597456], dtype=float32),\n",
+       " array([0.08325432], dtype=float32),\n",
+       " array([0.9963791], dtype=float32),\n",
+       " array([0.99327046], dtype=float32),\n",
+       " array([0.987528], dtype=float32),\n",
+       " array([0.2695155], dtype=float32),\n",
+       " array([0.01687575], dtype=float32),\n",
+       " array([0.0887219], dtype=float32),\n",
+       " array([0.00404755], dtype=float32),\n",
+       " array([0.8474386], dtype=float32),\n",
+       " array([0.02510497], dtype=float32),\n",
+       " array([0.00147101], dtype=float32),\n",
+       " array([0.00696711], dtype=float32),\n",
+       " array([0.01805459], dtype=float32),\n",
+       " array([0.37892923], dtype=float32),\n",
+       " array([0.32513785], dtype=float32),\n",
+       " array([0.00713208], dtype=float32),\n",
+       " array([0.05214171], dtype=float32),\n",
+       " array([0.9894679], dtype=float32),\n",
+       " array([0.74764496], dtype=float32),\n",
+       " array([0.0094498], dtype=float32),\n",
+       " array([0.05753988], dtype=float32),\n",
+       " array([0.9815139], dtype=float32),\n",
+       " array([0.994449], dtype=float32),\n",
+       " array([0.0733721], dtype=float32),\n",
+       " array([0.03602724], dtype=float32),\n",
+       " array([0.99997675], dtype=float32),\n",
+       " array([0.6763087], dtype=float32),\n",
+       " array([0.9927671], dtype=float32),\n",
+       " array([0.02451441], dtype=float32),\n",
+       " array([0.86146873], dtype=float32),\n",
+       " array([0.04389035], dtype=float32),\n",
+       " array([0.9999443], dtype=float32),\n",
+       " array([0.809564], dtype=float32),\n",
+       " array([0.99578035], dtype=float32),\n",
+       " array([0.4989446], dtype=float32),\n",
+       " array([0.02612785], dtype=float32),\n",
+       " array([0.87981015], dtype=float32),\n",
+       " array([0.6465501], dtype=float32),\n",
+       " array([0.576932], dtype=float32),\n",
+       " array([0.03007537], dtype=float32),\n",
+       " array([0.00870073], dtype=float32),\n",
+       " array([0.9998024], dtype=float32),\n",
+       " array([0.08114275], dtype=float32),\n",
+       " array([0.68397623], dtype=float32),\n",
+       " array([0.9999337], dtype=float32),\n",
+       " array([0.0099621], dtype=float32),\n",
+       " array([0.99060285], dtype=float32),\n",
+       " array([0.00027312], dtype=float32),\n",
+       " array([0.9289166], dtype=float32),\n",
+       " array([0.9932289], dtype=float32),\n",
+       " array([0.02628781], dtype=float32),\n",
+       " array([0.99826354], dtype=float32),\n",
+       " array([0.6789669], dtype=float32),\n",
+       " ...]"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "prediction = model.predict(x_test)\n",
+    "result = prediction.collect()\n",
+    "result"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Further experiments\n",
+    "\n",
+    "\n",
+    "* We were using 2 hidden layers. Try to use 1 or 3 hidden layers and see how it affects validation and test accuracy.\n",
+    "* Try to use layers with more hidden units or less hidden units: 32 units, 64 units...\n",
+    "* Try to use the `mse` loss function instead of `binary_crossentropy`.\n",
+    "* Try to use the `tanh` activation (an activation that was popular in the early days of neural networks) instead of `relu`.\n",
+    "\n",
+    "These experiments will help convince you that the architecture choices we have made are all fairly reasonable, although they can still be \n",
+    "improved!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusions\n",
+    "\n",
+    "\n",
+    "Here's what you should take away from this example:\n",
+    "\n",
+    "* There's usually quite a bit of preprocessing you need to do on your raw data in order to be able to feed it -- as tensors -- into a neural \n",
+    "network. In the case of sequences of words, they can be encoded as binary vectors -- but there are other encoding options too.\n",
+    "* Stacks of `Dense` layers with `relu` activations can solve a wide range of problems (including sentiment classification), and you will \n",
+    "likely use them frequently.\n",
+    "* In a binary classification problem (two output classes), your network should end with a `Dense` layer with 1 unit and a `sigmoid` activation, \n",
+    "i.e. the output of your network should be a scalar between 0 and 1, encoding a probability.\n",
+    "* With such a scalar sigmoid output, on a binary classification problem, the loss function you should use is `binary_crossentropy`.\n",
+    "* The `rmsprop` optimizer is generally a good enough choice of optimizer, whatever your problem. That's one less thing for you to worry \n",
+    "about.\n",
+    "* As they get better on their training data, neural networks eventually start _overfitting_ and end up obtaining increasingly worse results on data \n",
+    "never-seen-before. Make sure to always monitor performance on data that is outside of the training set.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From cd838f21e4e47c07ee40b79ff8ec76533599f86f Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 10:47:47 +0800
Subject: [PATCH 23/46] Rename 3.5-binary-classification.ipynb to
 3.5-classifying-movie-reviews.ipynb

---
 ...y-classification.ipynb => 3.5-classifying-movie-reviews.ipynb} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename keras/{3.5-binary-classification.ipynb => 3.5-classifying-movie-reviews.ipynb} (100%)

diff --git a/keras/3.5-binary-classification.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
similarity index 100%
rename from keras/3.5-binary-classification.ipynb
rename to keras/3.5-classifying-movie-reviews.ipynb

From 696832558850d60012e9084f2b1904bc76dcbaba Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 10:50:49 +0800
Subject: [PATCH 24/46] Add files via upload

---
 keras/3.5-classifying-movie-reviews.ipynb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 93ac869..43a3971 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -59,7 +59,7 @@
     "your training samples and their targets -- which would be completely useless for the task of predicting targets for data never seen before. \n",
     "We will go over this point in much more detail in the next chapter.\n",
     "\n",
-    "Just like the MNIST dataset, the IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words) \n",
+    "Just like the MNIST dataset, the IMDB dataset comes packaged with the Keras API of Analytics Zoo. It has already been preprocessed: the reviews (sequences of words) \n",
     "have been turned into sequences of integers, where each integer stands for a specific word in a dictionary.\n",
     "\n",
     "The following code will load the dataset (when you run it for the first time, about 80MB of data will be downloaded to your machine):"
@@ -79,7 +79,7 @@
     }
    ],
    "source": [
-    "from keras.datasets import imdb\n",
+    "from zoo.pipeline.api.keras.datasets import imdb\n",
     "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)"
    ]
   },

From d0ff83eea7ecc41117309ae0d76c00a5d4ad8b29 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 11:09:00 +0800
Subject: [PATCH 25/46] Add files via upload

---
 keras/3.5-classifying-movie-reviews.ipynb | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 43a3971..0ca5992 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -436,6 +436,14 @@
     "          validation_data=(x_val, y_val))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 512 records in 0.020173091 seconds. Throughput is 25380.344 records/second. Loss is 0.0092472015.\n",
+    "Top1Accuracy is Accuracy(correct: 8707, count: 10000, accuracy: 0.8707)_"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -480,14 +488,6 @@
     "plt.show()"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Trained 512 records in 0.020173091 seconds. Throughput is 25380.344 records/second. Loss is 0.0092472015.\n",
-    "Top1Accuracy is Accuracy(correct: 8707, count: 10000, accuracy: 0.8707)"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},

From 1c25881314d16ba049f7c15604c32454bed869c5 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 13:47:32 +0800
Subject: [PATCH 26/46] Add files via upload

---
 keras/2.1-a-first-look-at-a-neural-network.ipynb | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/keras/2.1-a-first-look-at-a-neural-network.ipynb b/keras/2.1-a-first-look-at-a-neural-network.ipynb
index d022670..c211e3f 100644
--- a/keras/2.1-a-first-look-at-a-neural-network.ipynb
+++ b/keras/2.1-a-first-look-at-a-neural-network.ipynb
@@ -37,7 +37,7 @@
     "\n",
     "----\n",
     "\n",
-    "We will now take a look at a first concrete example of a neural network, which makes use of Keras (v1.2.2) API in [Analytics Zoo](https://github.com/intel-analytics/analytics-zoo) to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed analytics-zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
+    "We will now take a look at a first concrete example of a neural network, which makes use of Keras (v1.2.2) API in [Analytics Zoo](https://github.com/intel-analytics/analytics-zoo) to learn to classify hand-written digits. Unless you already have experience with Keras or similar libraries, you will not understand everything about this first example right away. You probably haven't even installed Analytics zoo yet. Don't worry, that is perfectly fine. In the next chapter, we will review each element in our example and explain them in detail. So don't worry if some steps seem arbitrary or look like magic to you! We've got to start somewhere.\n",
     "\n",
     "The problem we are trying to solve here is to classify grayscale images of handwritten digits (28 pixels by 28 pixels), into their 10 categories (0 to 9). The dataset we will use is the MNIST dataset, a classic dataset in the machine learning community, which has been around for almost as long as the field itself and has been very intensively studied. It's a set of 60,000 training images, plus 10,000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \"solving\" MNIST as the \"Hello World\" of deep learning -- it's what you do to verify that your algorithms are working as expected. As you become a machine learning practitioner, you will see MNIST come up over and over again, in scientific papers, blog posts, and so on.\n",
     "\n",
@@ -52,7 +52,7 @@
     "_In Keras one could use following code to import the datasets:_\n",
     "\n",
     "    from keras.datasets import mnist\n",
-    "_Just replace it with following in analytics-zoo:_"
+    "_Just replace it with following in Analytics zoo:_"
    ]
   },
   {
@@ -227,7 +227,7 @@
     "\n",
     "    from keras import models\n",
     "    from keras import layers\n",
-    "_Just replace it with following in analytics-zoo:_"
+    "_Just replace it with following in Analytics zoo:_"
    ]
   },
   {
@@ -343,7 +343,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We are now ready to train our network, which in analytics-zoo Keras module is done via a call to the `fit` method of the network: \n",
+    "We are now ready to train our network, which in Keras API of Analytics Zoo is done via a call to the `fit` method of the network: \n",
     "we \"fit\" the model to its training data."
    ]
   },

From 81d6031ddcfeaf7408d2eff2e7b75a8a8f3182b4 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 13:47:50 +0800
Subject: [PATCH 27/46] Add files via upload

---
 keras/3.5-classifying-movie-reviews.ipynb | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 0ca5992..40822fb 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -294,7 +294,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "And here's the analytics-zoo implementation, very similar to the MNIST example you saw previously:"
+    "And here's the Analytics zoo implementation, very similar to the MNIST example you saw previously:"
    ]
   },
   {
@@ -412,7 +412,7 @@
     "                    batch_size=512,\n",
     "                    validation_data=(x_val, y_val)\n",
     "                    )\n",
-    "_After `fit` method finishes, the results are stored in `history` and thus could be visualized. Currently in analytics-zoo, `fit` method does not have any return. Results can only be checked via setting tensorboard._"
+    "_After `fit` method finishes, the results are stored in `history` and thus could be visualized. Currently in Analytics zoo, `fit` method does not have any return. Results can only be checked via setting tensorboard._"
    ]
   },
   {
@@ -451,7 +451,7 @@
     "Then result could be visualized in either of following ways: \n",
     "\n",
     "* Start tensorboard web interface in terminal by `tensorboard --logdir ./` and go to web browser url `localhost:port_number` as shown in your terminal.\n",
-    "* Use analytics-zoo built-in method `get_scalar_from_summary` with parameter `Loss` or `Validation` to get the array of scalar, then visualize via `matplotlib`.\n",
+    "* Use Analytics zoo built-in method `get_scalar_from_summary` with parameter `Loss` or `Validation` to get the array of scalar, then visualize via `matplotlib`.\n",
     "\n",
     "We use the second approach here in order to directly show the result in this notebook."
    ]
@@ -598,7 +598,7 @@
     "_In Keras, one could just call following code to predict the test data_\n",
     "\n",
     "    model.predict(x_test)\n",
-    "_In analytics-zoo, the return of `predict` is RDD, so you need to call `collect` method to get the result:_"
+    "_In Analytics zoo, the return of `predict` is RDD, so you need to call `collect` method to get the result:_"
    ]
   },
   {

From d55c0b7e562bd9cb1d7f65cab4804abe5ace3c2f Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 16:41:13 +0800
Subject: [PATCH 28/46] Add files via upload

---
 keras/3.5-classifying-movie-reviews.ipynb | 2017 ++++++++++-----------
 1 file changed, 1005 insertions(+), 1012 deletions(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 40822fb..9c38c15 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -36,7 +36,6 @@
    "metadata": {},
    "source": [
     "# Classifying movie reviews: a binary classification example\n",
-    "This notebook is imported from Chapter 3, Section 5 of [Deep Learning with Python Notebook]().\n",
     "\n",
     "----\n",
     "\n",
@@ -69,15 +68,7 @@
    "cell_type": "code",
    "execution_count": 2,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Using TensorFlow backend.\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from zoo.pipeline.api.keras.datasets import imdb\n",
     "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)"
@@ -136,7 +127,7 @@
     {
      "data": {
       "text/plain": [
-       "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
+       "\"distracting ? a evil entertainment ? ? might ? might an films ? tries who because truly tries talent too br she man ? steven how determination will ? looks world's which can ? it screen that in have way gonna of of least ? want take toxic even paint ? similar ? japanese that ? would ? ? charles cover movie even ? moment ear ? ? not wanted involved ? ? ? ? quality ? ? ? point and sequences will ? bruckheimer how actually he way kinds ? genre fact fine l a either her ? ? movie ? cover and minute ? ending ? ? favorite ? of of private ? spoiler down remember and while ? ? having 200 while movie ? prove charisma pretty ? chuck and perspective seriously if a bed movie in ? cover was most five springer for to free film been but woods was showing director movie this ? ? display sinister much details to and ? ? many no there if i which explore is will ? paramount the we without on most ? eddie urban just the ? harold came even like cares charged ? be most comparison buck hollywood or mixed well 1 or have doesn't comes and point more ? meant scenes you'd for work doesn't school is pants ? was ? mask of of example is ? to friend flying making br any or ? seems as sending and you it this ? mcintire ? is style it role think parts guy was most feeling ? and awful if is close and down meets and shoot more lies ? anything question making for or ? try finally the way plot way car of of if assistant ? as a first is victim ? ? most corrupt be ? does few in a ? former teeth completely and top ? pets ? 50 score seems as will scenes ? plot done independent to ? waste the ? all john is talent just am ? for reading ? we ? this writing guy maintain jokes has on think team ? nudity been its film guy is 3 ? ? as and ? let's body from ground film was it terrified voice throughout distracting ? ? there and ? negative of of ? bland began been his most yourself was most enjoy the across cried ? luis for br been his it restaurant or better but shows ? very an off and comments in\""
       ]
      },
      "execution_count": 4,
@@ -315,7 +306,7 @@
     {
      "data": {
       "text/plain": [
-       "<zoo.pipeline.api.keras.models.Sequential at 0x7f50e074cfd0>"
+       "<zoo.pipeline.api.keras.models.Sequential at 0x7f779381f4a8>"
       ]
      },
      "execution_count": 8,
@@ -428,7 +419,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "model.set_tensorboard('./', '3-5_summary')\n",
+    "import time\n",
+    "dir_name = '3-5 ' + str(time.ctime())\n",
+    "model.set_tensorboard('./', dir_name)\n",
     "model.fit(partial_x_train,\n",
     "          partial_y_train,\n",
     "          nb_epoch=20,\n",
@@ -463,7 +456,7 @@
    "outputs": [
     {
      "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd4VNXWwOHfSiP0jnQCivQeitKbUhRFuqKCCoIitqtiASuC5VOuiihcsaCAgFIkVOkgAqH3HiChhQAhlJC2vz9mMkySSTIJM5mU9T4Pj3PO2efMCpFZs7sYY1BKKaUAvDwdgFJKqexDk4JSSikbTQpKKaVsNCkopZSy0aSglFLKRpOCUkopG00KyqVExFtEropIZVeW9SQRuUtEXD52W0Q6iUiI3fFBEWntTNlMvNf/ROStzN6fxnM/EpGfXP1c5Tk+ng5AeZaIXLU7LADcBOKtx88aY37LyPOMMfFAIVeXzQuMMTVc8RwReQYYaIxpZ/fsZ1zxbJX7aVLI44wxtg9l6zfRZ4wxf6dWXkR8jDFxWRGbUirrafORSpO1eeB3EZkhIlHAQBG5R0T+FZHLInJGRL4SEV9reR8RMSISYD3+1Xp9sYhEichGEama0bLW611F5JCIRIrI1yKyQUQGpRK3MzE+KyJHROSSiHxld6+3iHwpIhEicgzoksbfz9siMjPZuYki8oX19TMist/68xy1fotP7VmhItLO+rqAiEyzxrYXaJKs7Dsicsz63L0i0sN6vh7wDdDa2jR3we7v9j27+4dZf/YIEZknIuWc+btJj4j0tMZzWURWikgNu2tvichpEbkiIgfsftYWIrLNev6ciHzm7PspNzDG6B/9gzEGIATolOzcR0AM8CCWLxH5gaZAcyw1zWrAIWCEtbwPYIAA6/GvwAUgEPAFfgd+zUTZMkAU8JD12itALDAolZ/FmRjnA0WBAOBi4s8OjAD2AhWBksBayz8Vh+9TDbgKFLR79nkg0Hr8oLWMAB2AG0B967VOQIjds0KBdtbXnwOrgeJAFWBfsrJ9gXLW38mj1hjusF57BlidLM5fgfesr++zxtgQ8Ae+BVY683fj4Of/CPjJ+rqWNY4O1t/RW8BB6+s6wAmgrLVsVaCa9fUWYID1dWGguaf/LeTlP1pTUM5Yb4z5yxiTYIy5YYzZYozZZIyJM8YcAyYDbdO4f44xJtgYEwv8huXDKKNlHwB2GGPmW699iSWBOORkjOOMMZHGmBAsH8CJ79UX+NIYE2qMiQDGp/E+x4A9WJIVQGfgkjEm2Hr9L2PMMWOxElgBOOxMTqYv8JEx5pIx5gSWb//27zvLGHPG+juZjiWhBzrxXIDHgP8ZY3YYY6KBUUBbEaloVya1v5u09AcWGGNWWn9H47EkluZAHJYEVMfaBHnc+ncHluReXURKGmOijDGbnPw5lBtoUlDOOGV/ICI1RSRIRM6KyBXgA6BUGveftXt9nbQ7l1MrW94+DmOMwfLN2iEnY3TqvbB8w03LdGCA9fWj1uPEOB4QkU0iclFELmP5lp7W31WicmnFICKDRGSntZnmMlDTyeeC5eezPc8YcwW4BFSwK5OR31lqz03A8juqYIw5CLyK5fdw3tocWdZadDBQGzgoIptFpJuTP4dyA00KyhnJh2N+j+Xb8V3GmCLAGCzNI+50BktzDgAiIiT9EEvudmI8A1SyO05vyOwsoJOIVMBSY5hujTE/MAcYh6VppxiwzMk4zqYWg4hUAyYBw4GS1ucesHtuesNnT2Npkkp8XmEszVRhTsSVked6YfmdhQEYY341xrTE0nTkjeXvBWPMQWNMfyxNhP8H/CEi/rcZi8okTQoqMwoDkcA1EakFPJsF77kQaCwiD4qID/AiUNpNMc4CXhKRCiJSEngjrcLGmLPAeuAn4KAx5rD1Uj7ADwgH4kXkAaBjBmJ4S0SKiWUexwi7a4WwfPCHY8mPQ7DUFBKdAyomdqw7MAN4WkTqi0g+LB/O64wxqda8MhBzDxFpZ33v17D0A20SkVoi0t76fjesfxKw/ACPi0gpa80i0vqzJdxmLCqTNCmozHgVeBLLP/jvsXQIu5Ux5hzQD/gCiADuBLZjmVfh6hgnYWn7342lE3SOE/dMx9JxbGs6MsZcBl4G5mLprO2NJbk5410sNZYQYDHwi91zdwFfA5utZWoA9u3wy4HDwDkRsW8GSrx/CZZmnLnW+ytj6We4LcaYvVj+zidhSVhdgB7W/oV8wKdY+oHOYqmZvG29tRuwXyyj2z4H+hljYm43HpU5YmmaVSpnERFvLM0VvY0x6zwdj1K5hdYUVI4hIl2szSn5gNFYRq1s9nBYSuUqmhRUTtIKOIalaeJ+oKcxJrXmI6VUJmjzkVJKKRutKSillLLJcQvilSpVygQEBHg6DKWUylG2bt16wRiT1jBuIAcmhYCAAIKDgz0dhlJK5Sgikt7MfECbj5RSStnRpKCUUspGk4JSSimbHNenoHKH2NhYQkNDiY6O9nQoKg3+/v5UrFgRX9/UllFSuY0mBeURoaGhFC5cmICAACwLnqrsxhhDREQEoaGhVK1aNf0bVK6gzUfKI6KjoylZsqQmhGxMRChZsqTW5vIYtyYF61o1B617vY5ycL2yiKwSke0isks318hbNCFkf/o7ynvclhSsq1hOBLpi2VVpgIjUTlbsHWCWMaYRlq38vnVXPEop5UkXb1yk0y+dyO5LC7mzptAMOGLdnzYGmMmtfWwTGaCI9XVRLEshK+V2ly9f5ttvM/cdpFu3bly+fNnp8u+99x6ff/55pt5L5R4jF49kxfEVBB0OytT9a0+s5Wac+9d/dGdSqEDSPWZDSbl94nvAQBEJBRYBLzh6kIgMFZFgEQkODw93R6wqj0krKcTFxaV576JFiyhWrJg7wlK5WHRcdJL/ZsSpyFO0/aktj/75qKvDSsHTHc0DgJ+MMRWx7L40zbqvaxLGmMnGmEBjTGDp0uku3aFUukaNGsXRo0dp2LAhr732GqtXr6Z169b06NGD2rUtrZwPP/wwTZo0oU6dOkyePNl2b0BAABcuXCAkJIRatWoxZMgQ6tSpw3333ceNGzfSfN8dO3bQokUL6tevT8+ePbl06RIAX331FbVr16Z+/fr0798fgDVr1tCwYUMaNmxIo0aNiIqKctPfhsruIm5EAHD04lG3v5c7h6SGkXTjcdsG3naexrJlH8aYjdbNuksB590Yl8pm3v9rL/tOX3HpM2uXL8K7D9ZJ9fr48ePZs2cPO3bsAGD16tVs27aNPXv22IZfTp06lRIlSnDjxg2aNm1Kr169KFmyZJLnHD58mBkzZjBlyhT69u3LH3/8wcCBA1N93yeeeIKvv/6atm3bMmbMGN5//30mTJjA+PHjOX78OPny5bM1TX3++edMnDiRli1bcvXqVfz9dS975X7urClsAaqLSFUR8cPSkbwgWZmTWDcyt26u7o9lAxWlslyzZs2SjMf/6quvaNCgAS1atODUqVMcPnw4xT1Vq1alYcOGADRp0oSQkJBUnx8ZGcnly5dp27YtAE8++SRr164FoH79+jz22GP8+uuv+PhYvqu1bNmSV155ha+++orLly/bzivlTm77v8wYEyciI4ClgDcw1RizV0Q+AIKNMQuwbK4+RURextLpPMhk96555XJpfaPPSgULFrS9Xr16NX///TcbN26kQIECtGvXzuF4/Xz58tlee3t7p9t8lJqgoCDWrl3LX3/9xdixY9m9ezejRo2ie/fuLFq0iJYtW7J06VJq1qyZqecr5Sy3fvUwxizC0oFsf26M3et9QEt3xqCUI4ULF06zjT4yMpLixYtToEABDhw4wL///nvb71m0aFGKFy/OunXraN26NdOmTaNt27YkJCRw6tQp2rdvT6tWrZg5cyZXr14lIiKCevXqUa9ePbZs2cKBAwc0KSi30/qoypNKlixJy5YtqVu3Ll27dqV79+5Jrnfp0oXvvvuOWrVqUaNGDVq0aOGS9/35558ZNmwY169fp1q1avz444/Ex8czcOBAIiMjMcYwcuRIihUrxujRo1m1ahVeXl7UqVOHrl27uiQGpdKS4/ZoDgwMNLrJTs63f/9+atWq5ekwlBP0d+UavWf15o/9fzC7z2x61+6doXt3nN1Bo+8b0eCOBuwYtiNT7y8iW40xgemV8/SQVKWUUtmIJgWllFI2mhSUUkrZaFJQSillo0lBKaWUjSYFpZRSNpoUlHJSoUKFADh9+jS9ezseUtiuXTvSGzI9YcIErl+/bjvO6FLcqdElupUraFJQKoPKly/PnDlzMn1/8qSgS3Gr7ESTgsqTRo0axcSJE23Hid+yr169SseOHWncuDH16tVj/vz5Ke4NCQmhbt26ANy4cYP+/ftTq1YtevbsmWTto+HDhxMYGEidOnV49913Acsie6dPn6Z9+/a0b98euLUUN8AXX3xB3bp1qVu3LhMmTLC9ny7RrbKKLnOhPO6lJS+x42zmZmmmpmHZhkzoMiHV6/369eOll17i+eefB2DWrFksXboUf39/5s6dS5EiRbhw4QItWrSgR48eqe5VPGnSJAoUKMD+/fvZtWsXjRs3tl0bO3YsJUqUID4+no4dO7Jr1y5GjhzJF198wapVqyhVqlSSZ23dupUff/yRTZs2YYyhefPmtG3bluLFi+sS3SrLaE1B5UmNGjXi/PnznD59mp07d1K8eHEqVaqEMYa33nqL+vXr06lTJ8LCwjh37lyqz1m7dq3tw7l+/frUr1/fdm3WrFk0btyYRo0asXfvXvbt25dmTOvXr6dnz54ULFiQQoUK8cgjj7Bu3TpAl+hWWUd/+8rj0vpG7059+vRhzpw5nD17ln79+gHw22+/ER4eztatW/H19SUgIMDhktnpOX78OJ9//jlbtmyhePHiDBo0KFPPSaRLdKusojUFlWf169ePmTNnMmfOHPr06QNYvmWXKVMGX19fVq1axYkTJ9J8Rps2bZg+fToAe/bsYdeuXQBcuXKFggULUrRoUc6dO8fixYtt96S2bHfr1q2ZN28e169f59q1a8ydO5fWrVtn+OeyX6IbcLhE9yeffEJkZCRXr17l6NGj1KtXjzfeeIOmTZty4MCBDL+ncq/DEZYNnny83P89XmsKKs+qU6cOUVFRVKhQgXLlygHw2GOP8eCDD1KvXj0CAwPT/cY8fPhwBg8eTK1atahVqxZNmjQBoEGDBjRq1IiaNWtSqVIlWra8tW3I0KFD6dKlC+XLl2fVqlW2840bN2bQoEE0a9YMgGeeeYZGjRql2VSUGl2iO/c4E3WG/n9YBgU8XPNht7+fW5fOFpEuwH+x7Lz2P2PM+GTXvwTaWw8LAGWMMWmOzdOls3MHXY4559DflWtkZuns6Lho2v3Ujk1hmwCY1H0SwwKHZer9Pb50toh4AxOBrkBtYICI1LYvY4x52RjT0BjTEPga+NNd8SilVE5ijGHYwmFsCtvExG4T07/BRdzZp9AMOGKMOWaMiQFmAg+lUX4AMMON8SilVI7x5b9f8vPOn3mv7Xs8UuuRLHtfdyaFCsApu+NQ67kURKQKUBVYmcr1oSISLCLB4eHhmQrmw4X7CBgVRGx8QqbuV66X03b9y4v0d+QZS48s5bXlr9GrVi9Gtx2dpe+dXUYf9QfmGGPiHV00xkw2xgQaYwJLly6dqTf4Yf1xAI5fuJbpIJXr+Pv7ExERoR862ZgxhoiICJ3MlsUORRyi35x+1CtTj58f/hkvydqPaXeOPgoDKtkdV7Sec6Q/8LwbY7H5fOlBJj+Rbl+LcrOKFSsSGhpKZmt+Kmv4+/tTsWJFT4eRZ1yOvkyPGT3w9fZlfv/5FPQrmOUxuDMpbAGqi0hVLMmgP/Bo8kIiUhMoDmx0Yywsf7kNnb9cy7J9qc9OVVnH19eXqlWrejoMpbKN+IR4BvwxgKOXjrLiiRVUKVbFI3G4rV5ijIkDRgBLgf3ALGPMXhH5QER62BXtD8w0bm5HuKtMIXc+XimlbsubK95kyZElTOw2kTZV2ngsDrdOXjPGLAIWJTs3Jtnxe+6MIZH9gmYXr8VQoqBfVrytUkoBcPbqWQASTMrBLtN2TuOzfz7jucDnGNpkaFaHlkR26WjOUjdiHfZnK6WUW4RdCWPDqQ2ApSPZ3uawzQz5awjtAtp5bB0we3kyKbQcv5LAj5Z7OgylVB5wKvIUbX9qazu2rymcjjrNwzMfpnzh8szuMxtfb1+HzyjgW4Dnmz5PndJ13B5vnlr7qHThfIRH3QTgwtUYD0ejlMrtTlw+Qfuf2xNxIyLFtei4aHr+3pOomCiWPb6MUgVKOXiCRZF8Rfim2zfuDNUmT9UUEhOCUkq52/FLx2n7U1su3rjI8seTtkwYYxj611A2h23m156/UrdMXQ9FmVKeqikopVRWOHrxKB1+6UDUzShWPLGCJuWbJLn+fxv/j2m7pvFh+w95qGZaq/9kvTyVFIrm9yXyRqynw1BK5WKHIw7T4ZcOXI+9zoonVtCoXKMk1xcfWcym0E30qd2Ht1u/7aEoU5enmo8CShbwdAhKqVzs4IWDtPu5HdFx0ax8YmWKhADwb+i/NCjbgB8f+jHVvb89KU8lhV+eau7pEJRSudT+8P20+7kdsfGxrHpyFQ3KNki1rKeWsHBGnkoKRQs4Hu6llFK3Y+/5vbT7uR3GGFYPWu2w47hCYcsi0S82f5HKRStndYhOy1NJQSmlbocxhviEpJNfd53bRbuf2+Et3qwetJrapWs7vLdFxRYAtKrcyu1x3g5NCkop5aSSn5bE58Nb43O2n9lO+5/bk887H2sGraFmqbT39M4JNCkopfKUcevGMX339Ezdeyn6ku311tNb6fhLRwr6FmTNoDVUL1ndVSF6VJ4akqqUUm+tfAuAR+ulWMk/TVdjrtpebw7bzH3T7qOYfzFWPbmKqsVzzzLwebqmcD0mztMhKKVygH3h+2g2pZntuPO0zpTIX4LVg1Y7nRA6Vu0IQL0y9dwSo6vk6aRw4GyUp0NQSmVzv+36jaZTmiZZv6h0gdKsHrSagGIBTj9neNPhmHcNNUrVcEOUruPWpCAiXUTkoIgcEZFRqZTpKyL7RGSviGSuoS8DmlUtYXt9+bouiqeUciw6LpphC4cxcO5AmpRrwvZnt9uurR60OlsPK70dbutTEBFvYCLQGQgFtojIAmPMPrsy1YE3gZbGmEsiUsZd8SS6GXdr2drQSzfc/XZKqRzo2KVj9Jndh21ntvFGyzf4qMNH+Hj5ULVYVY5fPk7FIrl332p3djQ3A44YY44BiMhM4CFgn12ZIcBEY8wlAGPMeTfGA0CLqiXYeeoyAFdvap+CUiqp+Qfm8+S8JxER5vefT48at3YPPvbiMQ9GljXc2XxUAThldxxqPWfvbuBuEdkgIv+KSBc3xgNAnQpFba+vaVJQSlnFxsfy+vLXefj3h7mrxF1sG7otSULIKzzd0ewDVAfaAQOAKSJSLHkhERkqIsEiEhweHn5bb1jQz9v2euKqo7f1LKVU7nA66jQdfunAZ/98xvDA4ax/an2uGmaaEe5MCmFAJbvjitZz9kKBBcaYWGPMceAQliSRhDFmsjEm0BgTWLp06dsKqoBf0hazaN2vWak8Y/3J9SnOrTi2gkbfN2LbmW389shvfNv9W/x9/D0QXfbgzqSwBaguIlVFxA/oDyxIVmYelloCIlIKS3OSWxvtCubzTnJcc/QSd76dUiqbmLJ1Ch1+7gBY5gokmAQ+WvsRnad1pmT+kmwZsiXDE9pyI7d1NBtj4kRkBLAU8AamGmP2isgHQLAxZoH12n0isg+IB14zxqTczNSFktcUlFK5W2x8LC8vfZmJWybS5a4u7Dm/h5IFStJ9eneWHFnCY/Ue47sHvqOQXyFPh5otuPUT0hizCFiU7NwYu9cGeMX6J0skrykopXKv8Gvh9JndhzUn1vDava8xruM4Gn7fkNUhq/Hz9mNS90k82+TZbLnZjafkua/NWlNQKm/YeXYnD818iLNXzzKt5zQG1h8IQHH/4lQtVpXZfWan2DtZ5cmkoDUFpXK7P/b9wRPznqCYfzHWDV5H0wpNbdcWProQP2+/PN2ZnBZPD0nNcr7eee5HVirPSDAJjFk1ht6ze1P/jvoEDwlOkhAAiuQrogkhDfoJqZTKUYwx7A/fT2x8bJLzUTej6DWrFx+u/ZDBDQez+snVlCtczkNR5lx5MimUL6rfEpTKaUKvhDJ27ViqfVWN2t/WpsfMW7ONj148yj0/3MNfB/9iwv0T+KHHD+TzyefBaHOuPNenALD8lbbUeXep7fh6TJx2QCuVDd2Mu8mCgwuYumMqS48sxWC4o+AdACw5YpljtOLYCvrO6YsxhiUDl9CpWidPhpzj5cmaQsF8SRPAzlORHopEKeXIjrM7GLl4JOW/KE/fOX3Ze34v77R5h6MjjxL0aJCt3FebvuL+X++nXKFybBmyRROCC+jXY6VUtnDxxkWm757O1O1T2X52O37efvSs2ZOnGj1Fx6od8fayjBzcemOr7Z4Xl7zIQzUeYlrPaRTOV9hToecqmhSUUh4TnxDPiuMrmLp9KnMPzCUmPobG5RrzTddvGFBvACXyl0jz/tFtRvNeu/fwkjzZ6OEWmhSUUlnuRuwNxq8fz487fuTUlVOUyF+CYU2GMbjRYBqWbZjmvUX9by1//0H7D9wdap6jSQEYMOVfjo/rplPdlcoiK46v4IO1H9A+oD3/d9//0aNGD6dHCyXWCqoWy5tLW7tbnq1zVSiWP8nxhiMRXLqmezYr5U6HIg7x8bqPeXDGgwC8du9r9KnTJ0PDR8sXLg/AM42fcUuMeV2erSlsGNWBgFG3RjF8t+Yo649cIGhkK+qUL5rGnUqpjNgfvp85++Ywe99sdp/fneTauWvnMvw8fx9/zLvGVeGpZPJsTSG59UcuALDt5GUPR6JU1rkee50rN6+49JnGGPac38N7q9+j7rd1qf1tbd5d/S5F/Ysy4f4JnHzpJE82eNKl76lcJ8/WFFIzet4eHm9RxdNhKJUles3qxYpjK+hUrRN9avfhoZoPpTvixxFjDLvP72b23tnM2T+HAxcOIAhtqrTh665f80itR2zNPip706SgVB4Wfi2ckgVKsv/Cfp5a8BQ+C33oWLUjvWv35uGaD1OqQKlU7zXGsOPsDmbvm82cfXM4fPEwXuJFu4B2jGw2kp61elK2UNks/GmUK7g1KYhIF+C/WHZe+58xZnyy64OAz7i1d/M3xpj/uTMmpVRSTco14a8Bf7H1zFZb2/+Qv4YwbOEw2ldtT5/afehZsyelCybdH33CvxN4ZdkreIs3Hap24D/3/oeHaz5MmYJlPPSTKFdwW1IQEW9gItAZCAW2iMgCY8y+ZEV/N8aMcFccSqn0iQiB5QMJLB/IuI7jbDWA2ftm8+zCZxkeNJx2Ae3oXas3j9R6hDsK3cGpK6fI75Ofky+fTLNGoXIWd9YUmgFHjDHHAERkJvAQkDwpKKWyERGhUblGNCrXiLEdxrLr3C5bDeK5Rc/x/KLnaVOlDRtDNxITH6MJIZdx5+ijCsApu+NQ67nkeonILhGZIyKVHD1IRIaKSLCIBIeHh7sswFnP3uOyZymVG4kIDco24MMOH7L/+f3sHr6b0W1GE349nJh4ndeTG3l6SOpfQIAxpj6wHPjZUSFjzGRjTKAxJrB06dKOimTKnaULuuxZSuV2IkLdMnV5v/377H1ur6fDUW7izqQQBth/86/IrQ5lAIwxEcaYm9bD/wFZuot28iW0lVJZo2WllgDUv6O+hyNRybkzKWwBqotIVRHxA/oDC+wLiIj9Xnk9gP1ujCeFfD6erigplTcNaTKE2NGxNC7X2NOhqGTc9lXZGBMnIiOApViGpE41xuwVkQ+AYGPMAmCkiPQA4oCLwCB3xeOILoCnlOf4eGlNPTty62/FGLMIWJTs3Bi7128Cb7ozhswIGBVEyPjung5DKaWynLafKKWUstH6m1J5VMT1CLae2Zp+QZWnaE1BqTzoz/1/UufbOrf1jNfufc1F0ajsJM8nhbvvKOTpEJTKMuevnafv7L70mtXrtlct/bTzp7qvQS6U55uP/HRYqsoDjDHM3DOTFxa/QFRMFB93+Jj/3Psfvvz3S12mQiWR55NCPh9vh+f/3neOTrXvyOJolHK9M1FnGB40nPkH59O8QnOmPjSV2qVrA/B6y9c9HJ3KbvJ8UvDzdlxTeOaXYMoX9Wflf9rh7+s4cSiVnRlj+GXnL7y09CWi46L5rPNnvNziZby99P9nlbo833YysmP1VK+djozm539Csi4YpTIoJj4GeV94fXnSb/ynIk/RfXp3Bs0fRN0yddk5bCf/ufc/mhBUuvJ8UrjnzpJpTlQ7eDYqC6NRKmMuR1v2FP9px0+ApXYwZesU6nxbhzUn1vDfLv9lzaA13F3ybg9GqXKSPJ8U0vPn9rD0CynlIZvDNtteh1wOofO0zgxdOJTA8oHsHr6bkc1H4iX6z1w5z6k+BRG5Ewg1xtwUkXZAfeAXY8xldwanlHJs17ldvLXiLYIOBwFw5eYV6n5bFxFhUvdJDG0yVJOByhRn/6/5A4gXkbuAyViWxJ7utqiymYBRQVy+rhuKKM87dukYA/8cSMPvGrLh1AZGtxkNwM34m7Ss3JK9z+1lWOAwTQgq05wdfZRgXfW0J/C1MeZrEdnuzsCym+/WHGNU15qeDkNlU/MOzGPtibV8cf8Xbnn+2atn+WjtR0zeOhkfLx/eaPkGr7d8nWL+xbgac5V6ZeoxqOEgXflX3TZnk0KsiAwAngQetJ7zdU9I2c95v7F8vy+GUV2DPR2KyoaCDgXRZ3YfvMTL5UkhMjqSz/75jC///ZKbcTcZ0ngIo9uOTjIb2V2JSOVNziaFwcAwYKwx5riIVAWmuS+s7OWG90ZCrno6CpUdrTuxjt6zexOXEIeft5/Lnnsj9gYTt0xk3PpxXLxxkf51+/NBuw+oXjL1IdRKuYJTDY/GmH3GmJHGmBkiUhwobIz5xM2xZTuJw/9embWDluNXejga5WnbzmzjgRkPEFAsgKcbPX1bz4pPiAcgLiGOH7b9wN3f3M1ry1+jWYWRmwhQAAAgAElEQVRmbBu6jRm9ZmhCUFnCqaQgIqtFpIiIlAC2AVNEJN06q4h0EZGDInJEREalUa6XiBgRCXQ+9Kw3bOEwjDH8uS2MsMs3PB2O8qCDFw7S5dcuFPMvxrKByyhdoHSGn3Et5ho/bv+RllNb4vOhD19u/JK639blmb+eoWKRiqx6chWLH1tMo3KN3PATKOWYs81HRY0xV0TkGSxDUd8VkV1p3SAi3sBEoDMQCmwRkQXGmH3JyhUGXgQ2ZTz8rPX73t+57877AF0TKS87GXmSztM6IyIsf3w5lYpWytD9W09v5X/b/sf0PdO5cvOK7fwry16hdunazOs3jx41eminsfIIZ8et+YhIOaAvsNDJe5oBR4wxx4wxMcBM4CEH5T4EPgGinXyuW9WtUCTVax2qduCFxS8QK6FZGJHKTs5fO0/naZ25cvMKSwcutc0U3nx6MzHxqQ9bjoyOZNKWSTT+vjGBUwL5aedPPFzzYdYOWkvv2r0BaFi2IbuG7eKhmg9pQlAe42xS+ABYChw1xmwRkWrA4XTuqQCcsjsOtZ6zEZHGQCVjTFBaDxKRoSISLCLB4eHhToacOSUL5kv12tstJuLr5U+436cYYt0ah8p+IqMj6fJrF05FnmLhowtpWLah7drK4yn7mIwxbDi5gUHzBlHu/8rx3KLnSDAJfNP1G868eoafH/6Z1lVaU65QOQAGNxysaxMpj3Oq+cgYMxuYbXd8DOh1O28sIl7AF8AgJ95/MpZJcwQGBrp1V48hraux5pDjxPPU1KP4eT1PZL4PueT7E/CwO0NR2ciN2Bv0mNmD3ed3s6D/AlpVbpVq2QvXL/DLzl/437b/sf/Cfgr5FeLx+o8zpMkQmpRrorUAla05u8xFReBroKX11DrgRWNMWu0oYVhmPieqaD2XqDBQF1ht/UdSFlggIj2MMR6bEBCXkJDm9QIJzSkc9yBRPvNZfHgxXat3zaLIlKfExsfSZ3Yf1p1Yx4xeM9L8nfef05+5B+YSEx9Di4ot+KHHD/St05dCfrrDn8oZnG0++hFYAJS3/vnLei4tW4DqIlJVRPyA/tZnAGCMiTTGlDLGBBhjAoB/AY8lhI41ywAQG59+RaR47GB8EwJ4ct6TnL161t2hKQ9KMAkMmj+IoMNBTOo+iX51+6VZfvmx5QwPHM7u4bvZ+PRGnmr0lCYElaM4mxRKG2N+NMbEWf/8BKQ5Bs8YEweMwNIXsR+YZYzZKyIfiEiP24raDXytm+3ExqddUwAQ/CgV8zpXY67yxNwnSDDp36Pc759T/1Djmxr8se8PlzzPGMMLi15g+u7pjOs4jmcDn033nrBXwpjQZQJ1y9R1SQxKZTVnk0KEiAwUEW/rn4FARHo3GWMWGWPuNsbcaYwZaz03xhizwEHZdp5sNnr8nioANKxULMU1//hG+CXUSHLOz1RmQpcJLD+2nC826jIDnjZ562Ta/dSOQxGH2H9hv0ueOWbVGL4N/pbX732dUa1SnWaThL+Pf4bfJ/GezNyrlKs5mxSewjIc9SxwBuiNEx3EOUnLu0oRMr475YvlT3GteOxgSsQMS3F+SOMh9KrVizdXvMmWsC1ZEWauFxsfy7Kjy5yufd2Mu8nQv4by7MJnaRvQ1mVxfLHxCz5a9xFDGg9hfKfxLnuuI++1e482Vdrc9qxopVzB2WUuThhjehhjShtjyhhjHuY2Rx/lJH6mGvlMyiUGRIQpD06hXKFyDPhjAFE3dZe222GMYdjCYdz/6/0En06/0ng66jTtf27PlG1TeLPVmywc4OwUmrRN3T6VV5e9Sp/afZjUfZLbRwsV8C3AmkFrdDiqyhZuZ9H1V1wWRQ5WPH9xfnvkN45fPs6IxSM8HU6O9vG6j5m6YypgqQGk5Z9T/9BkchN2ndvF7D6z+bjjx7f9oTp562T6zu7LkL+GcP+d9/PrI7/qB7XKc5xd5sIRHWxt1bpKa0a3Gc37a97nvmr38Vj9xzwdUo4zY/cM3ln1DjVL1eTAhQNplp28dTIjFo2gctHKLH98eaY6deMT4jlw4QCbwzZb/pzezLYz2wC4t9K9/NH3jwytejqv3zw+3/h5huNQKru5naTg1klkOc07bd5hxfEVDA8aTouKLbizxJ2eDinHWHdiHYPmD6JNlTa80fINuk/v7rDczbibvLD4BaZsm8L9d97PjF4zKJ6/eLrPN8YQeiU0SQIIPh3M1RjLeuhF8hWhafmmtvILByykoF/BDP0MD9V8iIdqOlrFRamcJc2kICJROP7wFyBlj2weNmjqVo4ffQrvYq/w6J+Psn7weny988w+RJl2KOIQD//+MFWLVWVuv7lsP+N4Q7/TUafpPas3G0M38marN/mw/YcpmnYSl58evWo0LSq2YHPYZjaFbWJz2GbbfBJfL18alm3Ikw2epFmFZjSr0Iy7S96Nl3gh71sqv84kGqVyqzSTgjGmcFYFkhPFxSfgY53fsP7IBXwow5QHp9Bndh/GrBrDuE7jPBxh9nbh+gW6T++Ol3gR9GgQJfKXcFjun1P/0GtWL6JuRjG7z2zbAnLJxSbcWo+q87TOANQoWYPO1TrbEkCDOxqQzyf19a2Uyutup/koz+s16R/mPtcSL69b3Su9a/dmaOOhfLLhEzpV60THah09GGH2FR0XzUMzH+JU5ClWPbkq1ea2jPQf+Hjd+t95+ePLCSwfSDH/lPNOlFKpu53RR7nek9YJbanZGRrJ+3/tTXH+yy5fUrNUTR6f+zjh19y7qmtOlGASGDRvEP+c+odpPadxT6V7UpS5GX9r/kHHah3ZMmRLuh3KXmL537lUgVJ0qtZJE4JSmaBJwYF6FYrSoWYZ/P3SH47488YTBIxKuvJ3Ad8CzOg1g4s3LjJ4/mCM0T55e2+veJvf9/7OJ50+oU+dPg7LdJ7WOcn8A2fa+f28/Xir1VtseGqDq0NWKs/Q5iMH/nrBsizyH1szv5lOg7IN+KzzZ4xcMpJinxQjclSkq8LL0aZsncL4DeN5tsmzvHbvaymuX7h+wfY6rf6D1IztOPa2Y1QqL9OaQhoeaVwh/UJpGNHMMpntys0rtuGPedmyo8sYHjScLnd14Ztu3zicKbzz3E4AXr3n1QwnhNsV9GgQfz/+d5a+p1LZjSaFNGRmeYOAUUGMnLE9xf0Xb1x0WVw50e5zu+k9qzd1ytTh996/J+kUdiS1kUju1K16Nx0YoPI8TQpusGDnafafuZJ+wTzidNRpuk/vTuF8hQl6NIgi+VLfB1sp5Vnap+AmT0zdzLNtqtmOC/pmbIZsbnE15ioPzniQizcusm7wOioWqejpkJRSaXBrTUFEuojIQRE5IiIpFqQXkWEisltEdojIehGp7c54slJMXALjFt9aw2f0yvfy3PDU+IR4BvwxgB1ndzCrzywalWvk6ZCUUulwW1IQEW9gItAVqA0McPChP90YU88Y0xD4FMg1u9VE3oglPuHWUNRJW7+hyoQqvLDoBUIuh2RJDOmtNOouxy4dY+fZnby05CUWHlrIN12/oVv1bh6JRSmVMe5sPmoGHDHGHAMQkZnAQ8C+xALGGPuG94LkwkX2xPhjJJq1T27kp51T+H7r90wKnsSAegN4/d7XqXdHPZe8T0x8DLvO7WJT6CY2n97MptBNHIw4yOQHJjOkyRCXvEdawq6EMWvvLGbuncnmsM2286/e8yrDmw53+/srpVzDnUmhAnDK7jgUaJ68kIg8j2VvBj+gg6MHichQYChA5cqVXR6osx5qWJ75O05n6J4Ssc8R4fcFjco2oXVAC6rmG8Sn679g7v65/LrrV7pX786oVqNoVbmVw/tPXD5BlWJJZ1YbYzh26ZhtwbdNYZvYfmY7N+MtNYMyBcvQvEJzDkYc5NSVU44e6xLh18L5Y/8fzNgzg3Un1mEwNC7X2Hb9kVqP8GnnTzP0zKFNhjJu/TgGNRzk4miVUs7weEezMWYiMFFEHgXeAZ50UGYyMBkgMDDQY7WJdjVKZzgpFIrvQKEbHWzDU9fuT6BE3DP8/ugElp34hf9u+i+tf2xNy0otGdVqFN2qd7Mt17Dw0EIenPEg33T9hjtL3GmrBWwO22yb5JXfJz9NyjdhRLMRNK/QnGYVmlG5aGVEBK/3M946aIwh5HIIjb5vRONyjVn55Mok1yOjI5l7YC4z98zk72N/E2/iqVmqJu+1e49+dfpRo1QN22qj03pOs/0szgooFoB5N9dVGJXKMdyZFMKASnbHFa3nUjMTmOTGeLKVYv4lGN12NK/c8wpTt0/l842f8+CMB6lbpi5vtHyDfnX68fcxy0SqxB3dBKF26dr0uLsHzSo0o3nF5tQtUzfdMf9pSTAJ7Avfx7oT61h7ci3rTqwjLMrya1oVsgqAazHXWHhoITP3zmTR4UXExMdQtVhVXm/5Ov3r9qdemXoO53QU8C2Q6biUUp7hzqSwBaguIlWxJIP+wKP2BUSkujHmsPWwO3CYPKagX0FeaP4CwwKH8fve3xm/fjyPz32cd1a+w4nIE7ZyK59YSZPyTW57jH9sfCxbz2xl3Yl1rDu5jg2nNtgm1pUrVI7WVVrTpnIb3lzxJlExUTz6x6MsOLiAa7HXKFeoHM8FPkf/uv1pVqGZ2/cuVkplPbclBWNMnIiMAJYC3sBUY8xeEfkACDbGLABGiEgnIBa4hIOmo+yka91yvPz7TsY8UJsPFu5L/wY7xy9c45eNIale9/X2ZWD9gTxa71EWHV7EuPXjbEnh8fqP075q+wzHazBcuH6BFcdWsO6kJQn8G/ov12OvA1C9RHUervEwrau0pnXl1lQrXs32QT/v4Dz+PvY3y44uY2D9gQyoO4BWlVs5tWfx1B5TGbpwaIbjVUp5nuS0FTwDAwNNcHBwlr3fuSvRNP94BQAh4y3bREZcvUmTjzK3Rk6Jgn5cvBbD0pfaUKNs2nsYrT+5no/XfcycvnMy1RST2LYPlqanBmUb0Lpya9pUaUOryq0oW6hsqvcevHCQk5EnaRfQTneQUyoXEJGtxpjA9Mp5vKM5u7ujiH+KcyUL5ePXp5sz8IdNGX5ebHyC02VbVW7FoscWZfg97BXyK8Ss3rO4t9K9FPUv6vR9NUrVoEapGrf13kqpnEeTQi6mo3iUUhmlC+J5yLPTgvlt04n0CyqlVBbSpOAhIRHXeXvuHk+HoZRSSWhSyGJR0XGeDkEppVKlSSGTapd3zZ4A3685yrK9Z13yLKWUul3a0eyEzW93TLLiKViGlrqC/fLaAK/dX4Pn2t1pmy9w4OwVukxYx8yhLWhRraRL3lMppVKjScEJZQqnHJbqLp8tPcidpQvh7+vFoB+32M4v2n1Gk4JSyu00KWRDw37dmuJcRuY3KKVUZmmfQg4RE6dzDpRS7qdJIYfQmoJSKitoUsghNCkopbKCJoXb8N3AJgxvdydVSrp/34DFe85Se8wSt7+PUipv06RwG7rULcsbXWqy5rWML2udGddj4p0q992aowSMCnJzNEqp3EiTQhYpVcg18xqW7T3LjxuOp1lmfLK5D0op5SwdkppFvn+8CeFRNxn267bbes7QaZbhqoNbVnVFWEoplYRbawoi0kVEDorIEREZ5eD6KyKyT0R2icgKEanizniyUsj47vw1opXtuLC/LwGlCrrs+eeuRBMwKoivVuS5HUyVUm7ktqQgIt7ARKArUBsYICK1kxXbDgQaY+oDc4BP3RWPu33Sq16Kc/Uq3trUplA+H/x90t/K0lnztocBsP7wBZc9Uyml3Nl81Aw4Yow5BiAiM4GHANvmxsaYVXbl/wUGujEet+obWIkzkdE80qgilR2MRirk78MNJzuKnZG4ZtLmkIucvxJNGQc7xCmlVEa5s/moAnDK7jjUei41TwOLHV0QkaEiEiwiweHh4S4M0XVEhJc63e0wIQAU8ffF39d1NQV7zT5eQZMPl2fq3kvXYvh0yQHidB6EUopsMvpIRAYCgcBnjq4bYyYbYwKNMYGlS5fO2uBcKL+bkgJAxLUYZmw+SXRsxmojHwXt59vVR1m275ybIlNK5STuTAphQCW744rWc0mISCfgbaCHMeamG+PxOF9vcevz3/xzNxuPRtiOA0YFsWL/OU5fvpHqPcZY1lRKPgciYFSQznVQKg9yZ5/CFqC6iFTFkgz6A4/aFxCRRsD3QBdjzHk3xpItJO6R4E6Df9qS5Pjpn4OpWqog80e0ZOLKI3y/9hgAL3S4i1fvq0E+X8v3gptxruvvUErlXG6rKRhj4oARwFJgPzDLGLNXRD4QkR7WYp8BhYDZIrJDRBa4K5687PiFawyautmWEAC+XnkEgHzWEVExcdqnoJRy8+Q1Y8wiYFGyc2PsXndy5/tnR8+0qsr/1juekZzPx4ubbvpwPnnRcRNSPp/EmkICe8Ii8ff14q4yhd0Sg1Iq+9MZzW5WoVh+3uha03b8ZrdaPNO6GldvxtLpi7VJyvq5MSk4armKTzC3kkJsAg98vR6Aec+3dEsMSqnsL1uMPsrNNozqQI8G5W3H3l5C2aL+FMnvm6KsfTlXC49K2Yc/+Kct+Pmk7FPYeepyms+6GRfP1Ztxrg1QKZUtaFLwkKJ2SaF7vXIA3HNnSX5+qlmWxbD2UDiXr8cCpFlDiYqOJWBUEMcvXAOgxjtLqPvuUrfEtP/MFbc8VynlHE0KHpIvlSUvAqsUT/febvXKuiyOkxevAzB/x63Rwu8u2JukzOLdZwH4dtWRVJ/z57ZQHrQ2PwWHXOS531LuM52eGZtP0vW/61h5QOdMKOUpmhSymYL5fOgbWDHNMkUdND1lVj7rhLoLV2PSLTt7a2iKc/EJhldm7eCVWTvZHRYJQL/J/7LImkgy4sj5qwAcC7+W4XuVUq6hHc0e9PvQFpQp4m9b3K5SccsSGekth1HQz3W/tsSO5rQcCb9qe33M7nXAqCCebVuNP7clnZMYn2BcFp9SKmtpUvCg5tVKAvBSp+p0qVuWWuWKAFDNbontWuWKcOVGLGF2s5IL5HPdry296XT9vt/IpuMXbccd/m9NkutL9mS8RqCUyr60+SgbEBFbQgB44p4A2+v7at/BghFJh4gWyue6NZQcNQnZs08IzhgxPfObCP2QyvwNpVTW0aSQDXl5CffXucN2XLJQviTXC7qwpuBqC3edSXIcdvmGS5cMV0q5lyaFbKpm2SJJju3nMBTKxkkhuZbjV/L4D5scXvtxw3ECRgVx1K6fAuCXjScy/X77Tl/RJKTUbdCkkEO0vduyZPgjjSo47GiePqR5VoeUrp//CQEg+MQlwLKF6HdrjhIXn8Dm4xf595hlRdfD56KS3Jc4TDaj4uIT6PbVOnp+uyHzQSuVx+Wcr5x5TK1yhZP8t0Lx/ABULlmAtjVKU6lEfk7ZrWd0752lCChZgJCIzH2guoP9fIeQC9do9/lqABbuOs2esCtUL1PIdv2zpQdu+/3irKOejl3QIa1KZZbWFLKpLnXL8c+oDnSpa5nt3KJaSaY93YwXO1bH19uLFa+045HGSTey+/O57LtmUWJCANgTZpm1fN5u6Y2Jq45mdUhKKQc0KWRj5YvlT3Lcunpp254Mfj5efNG3YZLr7tzZLTUnbqNmEnnDssTGsF8dj1iKvG5ZXuOUk81J7pwfkaBzL1QeoUkhF8nv581fI1qlWy4glX2ks5OEBMM869IbU9YdS3H96s04Gn+4nPWHL9jOvfT7jnSfezMunk3HItItZ+/g2SiqvbWIHzfokFmV+2lSyGUqlUhau/j2sca0q5Hz9rX+ddOtEUgHzkYRMCqIYdNurae0NyySi9dimPD3Idu55U7sMz1u0QH6Tf43QwvvHT5v6QgPDrnE5uMXmRV8yul7lcpp3JoURKSLiBwUkSMiMsrB9TYisk1E4kSktztjya2ebVONKnbf/H28k/5Ku9Urx1cDGjH9mew3OiktXyy/9WG/2TqBbsnes7T9bBUAUdGWpbsdLUGelsRO6LNXogHLUh21xyxx+v6+32/k9Tm7MvSeSuUkbksKIuINTAS6ArWBASJSO1mxk8AgYLq74sjt3uxWizWvtbcdO1q2ooi/L/feVcp2nLi20hd9G6T5bD9vz1UkL1+PTbFaK1j6MBISDBHXLJ3Uhf0tA+iux6S/v8PNuHi2WBPMjZh42wJ813VeQ5aKTzBsCcnYTHmVddz5r74ZcMQYc8wYEwPMBB6yL2CMCTHG7AJ0g2AXcWaBuwJ+lqRQJZ2+hRc63OWSmFyt5pglvPHHbgDm7zjN1hMXCb10a3hu8v2mY+ISOHg2ildm7eRGrCUBvD13N52+SLqO0+0IuXCN6FhNLs74dtUR+ny30TZPRWUv7kwKFQD7xtdQ67kME5GhIhIsIsHh4eEuCS638vH24vi4bk6XDxnfPcW5ne/eB8CglgHMHNoCgGYBJVwToAsk/9DvNWkj93251mHZvacjufudxdw/YW2SHeUuWTcXcoaXdcRXXILj7y6XrsXQ7vPVfLhwn9PPzMsSV90Nu+R433DlWTmio9kYM9kYE2iMCSxdOud1mmY1cbQhcwYUze9LyPjuFPb3pUW1kix7uQ1P3FvFRdFljcTtR4fbDXcNTeNDaNvJS6lee3uupVaS2hak+89aOq0Pno1yeD07237yEgfOpt/pHh0bjzGuGZY7f8dpAGLjtYEgO3JnUggDKtkdV7SeUx7yZb8GLH6xtW2BvSL+KTtpa5YtnOLc3XcUpqt1El1O0XTs39yMi3d67sbR81dTnDt/JZqwyzdstYrEzu3U7r3LboZ2WgJGBREwKsipsu7W89t/6DJhXZplTkRco+boJemuqJtRsTr3I1tyZ1LYAlQXkaoi4gf0Bxa48f2UncQOWHs9G1WkVrkiTOjXkPd71KH6HSkTwIhU+hG8vYQfBzd1eZzuVOOdJeT3cy4pbDhyge/WHOVm3K1+gWYfr6Dl+JW2412hkQ7vPX7BMrlu5pZbraWLd5+h5fiVxGXw2/C1m3G2GklcfEKKpjJP2HfaUpNYsd+126TGZoOfTaXktqRgjIkDRgBLgf3ALGPMXhH5QER6AIhIUxEJBfoA34tIyuEmKlOCXmjNO91rObxWMJ8PT94bcKvsyFb8/UpbBjSrRPd6qdcIyhbxB2BU15qplmlRLfv0PQDssOtHSMu8HacZv/gAE1emvg+1vcRv+hNXHWGqg0lt7y7YS9jlG2luc/rstOAU5+q8u5R67y0F4K63F3P3O4udisedoq2JMr0dATMqtT4a5VluXRDPGLMIWJTs3Bi711uwNCspF6tcsgDPtK7mVNk65YsCMO6R+mmWq1WuCKv/044qJQswfnHKBewK+nnToWYZ/j1mGW5Ytoi/bT5ATvHVyiMcvXCNf45cSL8w8NnSgw7PF8rnw/mom1y9GQv4OyyzdG/Sb95rD1kGUTjbdD9ryylmbDnJ3EyueeVMLeZo+FVe/n0nAP4+rk0K127qaK3sKEd0NKvsI6BUQUSEla+25dHmlZNc61jrjiS7xs169h5e7Fg9iyO8fUG7zjg1OslRv8CMzScByzLhAFetH3x/7Tyd7izqJ6ZuzlCcr/+xi+0nU68JnY+KTnOY7JR16S/bYb+sub+vaz8u/rvisEufB5blUXRo8O3RpKAypVrpQnzcs57tePnLbfisT33bhLceDcpTuWQBXu58N+Meqceika3Z98H9HPiwi+2eu+9wrmM2J3nzT8tIpWvWCXET/j5EXHwCL8zYzoNfr3f4gfXegr18szLtD8iEBMPEVUe4dC2GhATD6Hl70o2l2dgV3DNuBQARV28yZe0xTl++QaQ14Z3LYC3O1c1H7vDJkgPUHL1EE8Nt0P0UlEvYd1oHv9OJonbLTwxoVtnRLZQo6Jfq897qVpOPF93+HguetvpgOOusi/bFJRiG/7o1RZmfrJsRpSZgVBA/Dm7KZ0sPcuT8VTYejUjSLHclOtbhSDK4NR/jnXl7WLznLGMX7adM4XxsfruTUx/yy+zWk8qXA5LCzxtDAPeumJvbaU1B3ZbRD9Smld0SGgClCuXD14klMh6022I0NenNus6OkjcrDf5pi+31kWRbj+4JczyiKXkH8+AfLc+Yuz0sRT/Ns7+kTDTJJdh1VCTuY+FMc9Cf226NInd185E7RMda+km8bnOuTl6W/X/LKlt7ulVVfs3gYnvPtq3G5Meb8FjzKux8974kM7CXvdyGFa+2pURBy1yKAc0qp+iXKOpgEbxKJfITMr67U0uHe1LyZPnA1+sdlsvIUNSNxyIIGBVEs7F/O7weG5/A2kMpO87t53A4MzHNmY7mK9GxSTqwZwWfImBUkG2Y7T9HnevAz40uXovhkW83uGwSoLtoUlBZ7s2utbivTlnA8gFvPwP77jsKc2fpQvRqXIHX7q/B0NbVeLnz3fzyVDNbmYaVilG8gC9zn7uX2cPuSfLsehWLZs0PkUnHwt23VWhiDeB6TBy/2S09HvjR37Y1n+zZNx/Fxqf/QbX1xCVuxsVzJTr1Tvj67y1Lsq/Fp0ssTYBR1nt+3BCS7vu4woS/D9Hqk5X8z8FeHJmRkGAyPOckuff/2su2k5dZsf+8S2JyF00KKlsSEZ5vfxdeXpaE0ebu0qx7vb3t9fYx99GocnHuKJxyuGfr6qVSnEvrfG4yfvEBao9Zyttzb3VEJ+5wZ2/+jrAkq9Da70Vx0Lp/xaqDST+8gnafocY7S6j/3jIafbAs1aVBFu46Y3udOE/DxytrP2q+X3uM0Es3+Chov0ueV+2tRdz1tuM5IzM3n0y1GdBeYu0vJpsv76EdzSpbmPZ0MwrmS/t/x0olCqRYwK9iccumQm91vTVRb9rTt5qzEtv35z/fkpj4BE5EXGfRi62p++5SV4WerXy3xrm9rl+cmXSXuqDdp+levxyR12PZfNyyeulXaQwZvXQ9lke+/YfvBja27SNu78j5q04v+2Hv9OUbxCcYKpXIPgzGkLcAAA7JSURBVH1JaW3FmpBgGGUdcXbs4262LzE5mdYUVLbQunppGlcunuH7vLyEkPHd6ZrGTGwvgQaVitE0oARrX29PIbvkc2Rs10zF2y+wUvqFcpBFu88yee1RGnywjNHzLTWItOZAJLLfX3vc4lvfyjt9sYaFu06ne3/AqCBb0xLAveNX0vrTVbZjYww/rD/OlehYzkdFs/e042/kw6Ztddt6UsEnUl8s0d71XDIMVpOCytV+GtyUf0Z1TPW6j7dXkiSRXOKSHs+0qpqklvJUq6pJypUt4s/QNs7NIM+ufspke3/iN+nv1yRtvx8xfbvtddOxf5OQYBxumTpvu+N1MietPkp765Lk9d9bRrOxK+j+leOO+aX7zmYqdmck7xg+GXHdYf/C1VQWTMxpNCmoXK1djTKULZqy3+H9HnWY9rSl83rb6M6288m3LW1W1bKWU90KaXdglyjox5tdayaZnGevXoWitmdlV6cjM7ckSbW3Fjn1Lb3aW4tSvTZz88kUz/hkyQFCIq6nKOvMLnuOuGLUz+XrMbT5bBXPT9+W4pplSZNb4uITOHI+ikmrj/L3vnO2pdWX7j3L5eupr4mVXGJtKeLqzdsL3knap6DyJPsFAf18vOherxxBu88Qce3WP9bENuLNb3WkdGHLENmFL7Ti+IVbI4gCShagV+OKPNq8MiKCv683ZQrn43zUTQ6P7croeXuYueUUvw1pzlM/3pqvkNynvepzR1F/nszgUhe5RWK7fKKHJ25ItWztMUtZMKIlc7eH8UD98tQuVyTd9aKqvRlE8QJ+DG1TjYNno/hzexhBI1vR/av1LH+5DdXvKMzVm3HEJ5gUQ57j7PoULlsnAyZftwpSLq3+2pxdzHVQC5q/4zRrD4Wzfcx9aQdttSXkEh8u3Mf2k5f45tHGTt1zOzQpKAU0qFSUoN1nKODnzcyhLUhIMLZOwzJFbtU06lYoSt0KRTl18brt2gvJ5lGsfb09Jy9ex9fbi/G96jO+l2WhwadaVU21fbpvU0sfxYj2d3EtJo53H6yT5rfvaqULunV4a1ZK7MOwl97qtj2+sSSNHzeEOLWXeIKBiGsxjLNbyPH1ObsAmLMtlDe71qLNp6u4eC2Gv19pQ4ViBcjv552iyStxbatEP6y/tX5U8hVxD51LfdOl9NbWen3OToJPXGLlq+04fdmyOdTCXWcY1jYy3Vrr7dKkoBTwTKtq1CxbhNbVSzm1c12lEgV4s2tN+jVN2eHs7+vN3Q72quhWrxwh47sTHHKR3t9tpF2N0oztWS/Jvtr/ub+G7fXIjtWJvB7DzxtvzTloXrUEre4qxQsdqzNu0X6+X2tpx7/3zpJMH9KCu99Z7HDiW05csdZZ6Q3xPHLe8Ydz4of292uOcV/tsly01hI7fWHZ2nXBiJYMmPyvbR0rwPb3nWjsolud60N+CXa4vW1qIm/E2molN2Li8fEWfL29OH35BrOCb21olPgFBOCNP3YRNLK10++RGZLdZ9clFxgYaIKDU65Dr1RuFRxykT+2hVG7fBEeb5F0W9TE2sShj7ri5+PFg1+vZ3dYJEfGdrWNq0/8oHJU8yhewDfFt9YxD9TmA+t+04X/v717j5GqPOM4/v2xC8udBaF0cVGwIEKiyy3CCrFKlaKoiRarpKkYsdSojSaaFqpFa2y1tfHWVNNqrdaopVgvW2K13OxFlJvcQQTrctkqCwiIKLAsT/847x6GYWAX6szs2X0+yWTOec87m/eBs/vM+55z3rd1IZeW9eD5+RuP+Gxzd3JxG6p2Hr7E660X9KV/SUe+/2z9U4+8fOM5DOjRkWkLNzE19JZm/GDkYU+5r713DP3ufD3e71/Skb/dcmJJQdJiMxtaX72s9hQkjQEeAQqAJ83s/rTjRcAfgSHAduAqM6vMZpucS5qhvbowtFfmi9TPXHc2ry6polXobUy/oZwtn+6lsKAF911xJmWlxUd8pvL+sazbspu2RYV0a1/Ej19ewQPjzmLovdE0GdeN7E1BC3FXxSpuH92PCef0YunGnawOU3/3L+l42DTgc277Ok/868N4aGVASUd+dWUZFz96aJnPc0/vFq8X0VSkJwSAh2c1fDrwyx+bd0RZ+rQnqQkhV7LWU5BUALwPXAhsJlqec7yZrU6pcyNwlpndIOlq4HIzu+pYP9d7Cs6dmLnvVdOzSxv6fOXIoa1MFm/YwaCexfG1lSUbd7Cu+jMuK+vBri9qGPbz2fTu2o65t58Xf6am9iAFEi1aiC/219J/avRH7a3Jo/jzwk3U1B7ksTcPPWD37MSzOfPkThS3bcUz8yq5q2IVhS3EtwaXMm3RJo7l0rIe/HXZoWch2hcVxnMsNVUj+pzEc9cPP6HPNrSnkM2kUA7cbWbfDPtTAMzsvpQ6b4Q6b0sqBD4GutkxGuVJwbnk2LPvALPfq+aytBlxZ63eQuX2PcdcHbDuz8Ce/bXsramla/si9tbUcsZPokRTef9Yag8aNbUH2bp7X3yH2H+27uG0bu3ieq1btohnT63TuW1LDtQay+8ezfLNu/h8fy3jn3inwXFdN6J3xmVYs+3uSwdw7Yje9VfMoDEkhXHAGDO7Pux/FxhmZjen1FkZ6mwO+x+EOtvSftYkYBLAKaecMmTDhg0451wuvbKkisICcUH/7odNJvjZvgO0aVlAC8GG7Z+z5qNP4yfsl23aSU3tQT7Zs58LB3SnaucXdGjdkp2f7+f5BRu5YlAp/b7agZVVu3hw5vuMPbOE26ZHy5/efH4f1lXvjm9/nTd5FCWdWjfoRohMmlRSSOU9BeecO34NTQrZfKK5Cki9X680lGWsE4aPOhFdcHbOOZcH2UwKC4G+knpLagVcDVSk1akAJoTtccCcY11PcM45l11ZuyXVzA5Iuhl4g+iW1KfMbJWke4BFZlYB/B54VtJ64BOixOGccy5Psvqcgpm9BryWVjY1ZXsvcGU22+Ccc67hfJZU55xzMU8KzjnnYp4UnHPOxTwpOOeciyVullRJW4ETfaS5K3DUB+MSpqnE0lTiAI+lsfJYIqeaWbf6KiUuKfw/JC1qyBN9SdBUYmkqcYDH0lh5LMfHh4+cc87FPCk455yLNbek8Lt8N+BL1FRiaSpxgMfSWHksx6FZXVNwzjl3bM2tp+Ccc+4YPCk455yLNYukIGmMpLWS1kuanO/21EfSU5KqwyJEdWVdJM2UtC68dw7lkvRoiG25pMH5a/mRJPWUNFfSakmrJN0SyhMXj6TWkhZIWhZi+Wko7y1pfmjztDBVPJKKwv76cLxXPtufTlKBpCWSZoT9pMZRKWmFpKWSFoWyxJ1fAJKKJb0o6T1JaySV5zqWJp8UJBUAvwEuAgYA4yUNyG+r6vU0MCatbDIw28z6ArPDPkRx9Q2vScDjOWpjQx0AbjOzAcBw4Kbw75/EePYBo8ysDBgIjJE0HPgF8JCZ9QF2ABND/YnAjlD+UKjXmNwCrEnZT2ocAOeb2cCUe/iTeH4BPAK8bmZnAGVE/z+5jcXMmvQLKAfeSNmfAkzJd7sa0O5ewMqU/bVASdguAdaG7d8C4zPVa4wv4FXgwqTHA7QF3gWGET1hWph+vhGtJVIetgtDPeW77aE9pUR/YEYBMwAlMY7Qpkqga1pZ4s4vopUnP0z/t811LE2+pwCcDGxK2d8cypKmu5l9FLY/BrqH7cTEF4YdBgHzSWg8YchlKVANzAQ+AHaa2YFQJbW9cSzh+C7gpNy2+KgeBn4IHAz7J5HMOAAM+LukxZImhbIknl+9ga3AH8Kw3pOS2pHjWJpDUmhyLPpakKh7iSW1B/4C3Gpmn6YeS1I8ZlZrZgOJvmmfDZyR5yYdN0mXANVmtjjfbfmSjDSzwUTDKTdJOjf1YILOr0JgMPC4mQ0C9nBoqAjITSzNISlUAT1T9ktDWdJskVQCEN6rQ3mjj09SS6KE8JyZvRSKExsPgJntBOYSDbMUS6pbxTC1vXEs4XgnYHuOm5rJCOAySZXAn4iGkB4heXEAYGZV4b0aeJkoWSfx/NoMbDaz+WH/RaIkkdNYmkNSWAj0DXdWtCJaB7oiz206ERXAhLA9gWhsvq78mnAnwnBgV0pXM+8kiWgt7jVm9mDKocTFI6mbpOKw3Ybo2sgaouQwLlRLj6UuxnHAnPBNL6/MbIqZlZpZL6Lfhzlm9h0SFgeApHaSOtRtA6OBlSTw/DKzj4FNkvqFom8Aq8l1LPm+uJKjCzgXA+8Tjf/eke/2NKC9LwAfATVE3x4mEo3hzgbWAbOALqGuiO6u+gBYAQzNd/vTYhlJ1N1dDiwNr4uTGA9wFrAkxLISmBrKTwMWAOuB6UBRKG8d9teH46flO4YMMZ0HzEhqHKHNy8JrVd3vdxLPr9C+gcCicI69AnTOdSw+zYVzzrlYcxg+cs4510CeFJxzzsU8KTjnnIt5UnDOORfzpOCccy7mScG5o5B0R5gNdXmYgXOYpFsltc1325zLFr8l1bkMJJUDDwLnmdk+SV2BVsA8ovvBt+W1gc5lifcUnMusBNhmZvsAQhIYB/QA5kqaCyBptKS3Jb0raXqY46lujv9fhnn+F0jqE8qvlLRS0ZoM/8xPaM4dnfcUnMsg/HH/N9EU2bOAaWb2jzBf0FAz2xZ6Dy8BF5nZHkk/InoK+J5Q7wkz+5mka4Bvm9klklYAY8ysSlKxRXMoOddoeE/BuQzM7DNgCNHiJVuBaZKuTas2nGjhprfCdNoTgFNTjr+Q8l4ett8Cnpb0PaAgO6137sQV1l/FuebJzGqBN4E3wzf8CWlVBMw0s/FH+xHp22Z2g6RhwFhgsaQhZtZoZhx1znsKzmUgqZ+kvilFA4ENwG6gQyh7BxiRcr2gnaTTUz5zVcr726HO18xsvplNJeqBpE597FzeeU/BuczaA78OU2UfIJohdBIwHnhd0n/N7PwwpPSCpKLwuTuJZuQF6CxpOdHaznW9iQdCshHRzJfLchKNcw3kF5qdy4LUC9L5botzx8OHj5xzzsW8p+Cccy7mPQXnnHMxTwrOOedinhScc87FPCk455yLeVJwzjkX+x+DURa1CBCVtgAAAABJRU5ErkJggg==\n",
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd4lGXWwOHfyaSRAgldCBBAlBp6UVBAiiD2AiiiuCKuq6t86K7YEF3BunYUG7oWBBSx0qugCAFFeg0tIARCek/m+f6YkkkyKUAmk3Lu6+Ji3pl33jmTwJx52nnEGINSSikF4OPtAJRSSlUemhSUUko5aVJQSinlpElBKaWUkyYFpZRSTpoUlFJKOWlSUOVKRCwikioizcvzXG8SkQtFpNznbovIYBE55HK8R0QuK8u55/BaH4rI4+f6/BKu+5yIfFLe11Xe4+vtAJR3iUiqy2EQkAXk2Y/vNcZ8cTbXM8bkASHlfW5NYIy5uDyuIyLjgduNMQNcrj2+PK6tqj9NCjWcMcb5oWz/JjreGLO8uPNFxNcYk1sRsSmlKp52H6kS2bsH5orIlyKSAtwuIpeIyG8ikigif4nImyLiZz/fV0SMiETajz+3P75IRFJEZL2ItDzbc+2PDxeRvSKSJCJvicgvIjKumLjLEuO9IrJfRBJE5E2X51pE5DURiReRGGBYCT+fJ0RkTqH7ZojIq/bb40Vkl/39HLB/iy/uWrEiMsB+O0hEPrPHtgPoXujcJ0Ukxn7dHSJyrf3+TsDbwGX2rrnTLj/bqS7P/7v9vceLyLcickFZfjalEZEb7PEkishKEbnY5bHHReS4iCSLyG6X99pHRH63339SRF4u6+spDzDG6B/9gzEG4BAwuNB9zwHZwDXYvkTUAnoCvbG1NFsBe4EH7Of7AgaItB9/DpwGegB+wFzg83M4tyGQAlxnf2wSkAOMK+a9lCXG74A6QCRwxvHegQeAHUAEUA/42fZfxe3rtAJSgWCXa8cBPezH19jPEeAKIAOIsj82GDjkcq1YYID99ivAaiAcaAHsLHTuSOAC++/kNnsMjeyPjQdWF4rzc2Cq/fZQe4xdgEDgHWBlWX42bt7/c8An9tvt7HFcYf8dPQ7ssd/uABwGGtvPbQm0st+OBm613w4Fenv7/0JN/qMtBVUW64wxPxhjrMaYDGNMtDFmgzEm1xgTA7wP9C/h+V8bYzYZY3KAL7B9GJ3tuVcDW4wx39kfew1bAnGrjDE+b4xJMsYcwvYB7HitkcBrxphYY0w88EIJrxMDbMeWrACGAAnGmE32x38wxsQYm5XACsDtYHIhI4HnjDEJxpjD2L79u77uPGPMX/bfyWxsCb1HGa4LMAb40BizxRiTCUwG+otIhMs5xf1sSjIa+N4Ys9L+O3oBW2LpDeRiS0Ad7F2QB+0/O7Al9zYiUs8Yk2KM2VDG96E8QJOCKoujrgci0lZEfhKREyKSDDwL1C/h+SdcbqdT8uBycec2cY3DGGOwfbN2q4wxlum1sH3DLcls4Fb77dvsx444rhaRDSJyRkQSsX1LL+ln5XBBSTGIyDgR+dPeTZMItC3jdcH2/pzXM8YkAwlAU5dzzuZ3Vtx1rdh+R02NMXuAh7H9HuLs3ZGN7afeBbQH9ojIRhG5qozvQ3mAJgVVFoWnY76H7dvxhcaY2sAUbN0jnvQXtu4cAEREKPghVtj5xPgX0MzluLQps/OAwSLSFFuLYbY9xlrA18Dz2Lp2woClZYzjRHExiEgr4F3gPqCe/bq7Xa5b2vTZ49i6pBzXC8XWTXWsDHGdzXV9sP3OjgEYYz43xvTF1nVkwfZzwRizxxgzGlsX4X+B+SISeJ6xqHOkSUGdi1AgCUgTkXbAvRXwmj8C3UTkGhHxBR4CGngoxnnARBFpKiL1gEdLOtkYcwJYB3wC7DHG7LM/FAD4A6eAPBG5Ghh0FjE8LiJhYlvH8YDLYyHYPvhPYcuP92BrKTicBCIcA+tufAncLSJRIhKA7cN5rTGm2JbXWcR8rYgMsL/2v7CNA20QkXYiMtD+ehn2P1Zsb2CsiNS3tyyS7O/Nep6xqHOkSUGdi4eBO7H9h38P24CwRxljTgKjgFeBeKA18Ae2dRXlHeO72Pr+t2EbBP26DM+ZjW3g2Nl1ZIxJBP4PWIBtsPZmbMmtLJ7G1mI5BCwCPnW57lbgLWCj/ZyLAdd++GXAPuCkiLh2AzmevxhbN84C+/ObYxtnOC/GmB3YfubvYktYw4Br7eMLAcBL2MaBTmBrmTxhf+pVwC6xzW57BRhljMk+33jUuRFb16xSVYuIWLB1V9xsjFnr7XiUqi60paCqDBEZZu9OCQCewjZrZaOXw1KqWtGkoKqSfkAMtq6JK4EbjDHFdR8ppc6Bdh8ppZRy8mhLwd7c32NfLj/ZzePNRWSViPwhIlt1frJSSnmXx1oK9oHAvdhWeMaSv5R9p8s57wN/GGPeFZH2wEJjTGRJ161fv76JjCzxFKWUUoVs3rz5tDGmpGncgGerpPYC9juWstuLhl2HrYaLgwFq22/XwTabpESRkZFs2rSpnENVSqnqTURKW5kPeLb7qCkFl+nHUnQF6lRsVTdjgYXAP91dSEQmiMgmEdl06tQpT8SqlFIK788+uhVbhcUIbAtYPrMvjS/AGPO+MaaHMaZHgwaltn6UUkqdI08mhWMUrN3irIHi4m5sS+MxxqzHVkWxrEW9lFJKlTNPJoVobOVwW4qIP/ayuoXOOYK9Foy9Pk0gtjnoSimlvMBjScHYtmx8AFgC7ALmGWN2iMizjl2isNWnuUdE/sRWpGuc0YUTSinlNR7do9kYsxDbALLrfVNcbu8E+noyBqWUUmXn7YFmpZRSlYgmBaWUquTSc9L597J/czixTEsNzosmBaWUqsTWHFpD1LtRvPzryyzct7D0J5wnTQpKKVUJpWan8sDCBxjwvwEYDKvuXMV9Pe/z+Ot6dKBZKaXU2Vses5x7friHw4mHeaj3Q0y7YhrB/sEV8tqaFJRSqpJIykziX8v+xQe/f8BF9S5i7V1r6du8YidoalJQSqlKYPH+xdzzwz0cTznOI5c8wrMDn6WWX60Kj0OTglJKeVFCRgKTlk7iky2f0L5Be76+5Wt6R/T2WjyaFJRSykt+2PMD9/54L3FpcTze73Gm9J9CgG+AV2PSpKCUUhUsPj2eBxc/yOxts4lqFMWPt/1Itwu6eTssQJOCUkpVqPk75/OPhf/gTMYZpvafymOXPYa/xd/bYTlpUlBKqQpwKPEQ/172b77a+RXdLujGsrHLiGoU5e2witCkoJRSHrTz1E5eWPcCs7fNxuJjYdoV0/jXpf/Cz+Ln7dDc0qSglFIesPHYRp5f9zzf7v6WIL8g/tnrn0y6ZBLN6jQr/clepElBKaXKiTGGlQdXMn3ddFYeXElYYBhPXf4UD/Z+kPpBVWNTSU0KSil1nqzGyvd7vmf62ulEH4+mcUhjXh7yMvd2v5fQgFBvh3dWNCkopdQ5ysnL4cvtX/LiLy+y89ROWoa1ZOaImdzZ5U4CfQO9Hd458WhSEJFhwBuABfjQGPNCocdfAwbaD4OAhsaYME/GpJRS5ysjJ4NZf8zi5V9f5nDSYTo17MQXN37ByA4j8fWp2t+1PRa9iFiAGcAQIBaIFpHv7VtwAmCM+T+X8/8JdPVUPMcTM9h5PJnB7Rt56iWUUtVcSlYKM6Jn8NpvrxGXFsclEZfw9lVvM6LNCETE2+GVC0/up9AL2G+MiTHGZANzgOtKOP9W4EtPBfPdluOM/3QTyZk5nnoJpVQ1diL1BP0+7sdjKx6ja+OurL5zNb/87ReuvujqapMQwLPdR02Boy7HsYDbKk8i0gJoCaws5vEJwASA5s2bn1MwbRqGALA/LpVuzcPP6RpKqZrpcOJhBn82mOMpx1k0ZhHDLhzm7ZA8prLsvDYa+NoYk+fuQWPM+8aYHsaYHg0aNDinF2jTyJ4UTqaec5BKqZpnz+k9XPbxZZxOP83yscurdUIAz7YUjgGuqzQi7Pe5Mxq434OxEBEeRICvD/viUjz5MkqpamTLiS0M/WwoIsLqO1fTuXFnb4fkcZ5sKUQDbUSkpYj4Y/vg/77wSSLSFggH1nswFiw+QusGIezVloJSqgx+OfILAz4ZQKBvIGvvWlsjEgJ4MCkYY3KBB4AlwC5gnjFmh4g8KyLXupw6GphjjDGeisWhTaMQ9sdpUlBKlWzpgaUM/XwojUIase5v67io3kXeDqnCeHRCrTFmIbCw0H1TCh1P9WQMrto0DOG7LcdJycwhNLByFqNSSnnX/J3zuXX+rbRv0J4lty+hUUjNmsZeWQaaK0THpnUA2HYsycuRKKUqo0+2fMLIr0fSs2lPVo9bXeMSAtSwpNA5wrZYesvRRC9HopSqbN747Q3u+u4uBrUcxNLblxIWWDOLK9SopBAe7E+9YH9iEzK8HYpSqpIwxvDsmmeZuGQiN7a7kR9u/YFg/2Bvh+U1VbtIxzkIDfQlLSvX22EopSoBYwyPLH2EV397lXFdxvHBNR9U+dpF56vGvfuQQF9SMjUpKFXT5VnzuPfHe/noj494sNeDvDbsNXykRnWeuFXzkkKAL6maFJSq0bLzsrn9m9v5audXTLl8ClMHTK1W9YvORw1MCn4cS9QxBaVqqvScdG6adxOL9y/mv0P/y6RLJnk7pEqlxiWF0EBfUrO0UqpSNY0xhkX7F/H4isfZenIrH1zzAeO7jfd2WJVOjUsK2n2kVM3iSAZTV08l+ng0kWGRLBi1gOvallTJv+aqeUkh0JfUrFyMMdqHqFQ15i4ZfHjNh9zR+Q78LFrRoDg1LykE+JKTZ8jKtRLoZ/F2OEqpcqbJ4PzUuKRQu5btH0VyRo4mBaWqEU0G5aPGJYWGoQEAnEzOomHtQC9Ho5Q6X5oMyleNSwqN7IngZHImnajj5WiUUufKGMPi/YuZumYqG49tJDIskg+u+YA7Ot+Bv8Xf2+FVWTUuKTR2JIWUTC9HopQ6V0v2L2HK6imaDDygxiWF+iH+iNi6j5RSVUtyVjIPLX6IT7Z8osnAQzxa6ENEhonIHhHZLyKTizlnpIjsFJEdIjLbk/EA+Fp8qBfszyltKShVpfxy5Be6zOzCp39+ylOXP8WeB/Ywvtt4TQjlzGMtBRGxADOAIUAsEC0i3xtjdrqc0wZ4DOhrjEkQkYaeisdVnVp+JGXoqmalqoKcvByeXfMs09dNp0WdFqy9ay2XNrvU22FVW57sPuoF7DfGxACIyBzgOmCnyzn3ADOMMQkAxpg4D8bjpElBqaphb/xebv/mdqKPR3NXl7t4Y9gbhAaEejusas2T3UdNgaMux7H2+1xdBFwkIr+IyG8iMsyD8ThpUlCqcjPG8P7m9+n6XlcOJBzgq1u+YtZ1szQhVABvDzT7Am2AAUAE8LOIdDLGFNgvU0QmABMAmjdvft4vWqeWH/viUs/7Okqp8heXFsf478fzw94fGNJqCB9f9zFNaxf+Pqk8xZMthWNAM5fjCPt9rmKB740xOcaYg8BebEmiAGPM+8aYHsaYHg0aNDjvwMKCbFtyPrFg23lfSylVfn7a+xOd3u3E0gNLef3K11l8+2JNCBXMk0khGmgjIi1FxB8YDXxf6JxvsbUSEJH62LqTYjwYE4BzO84vNhzx9EsppcogPSedf/z0D67+8moahzRm04RNPNTnId0JzQs81n1kjMkVkQeAJYAFmGWM2SEizwKbjDHf2x8bKiI7gTzgX8aYeE/F5JCVawXAR4ukKuV1m49vZsw3Y9gTv4eHL3mYaVdMI8A3wNth1VgeHVMwxiwEFha6b4rLbQNMsv+pME+MaMf3fx6ndYOQinxZpZSLPGseL/3yElNWT6FRcCOWj13OoFaDvB1WjeftgWavaFQ7kNE9m7Fid4XMgFVKFXI06ShjvhnD2iNrGdlhJO+OeJe6tep6OyxFDU0KAHWD/UlIy9bNdpSqYIv2LeL2BbeTk5fDp9d/yu1Rt+v/wUqkxo7i1A32J9dqSMnSrTmVqgi51lyeWPEEV82+iojaEWyasImxncdqQqhkamxLoYFjX4WkTGoHas11pTzpr5S/uO2b21h9aDXju47nzeFvUsuvlrfDUm7U2KTQrG4QAEfOpNOmka6SVMpTVh1cxa3zbyU5K5lPrvuEO7vc6e2QVAlqbPdRC5ekoJQqf1ZjZdrP0xj82WDCa4Wz8Z6NmhCqgBrbUqgb7E+wv4XD8ZoUlCpvp9NPM3bBWBbvX8xtnW7jvavfI8Rfp4BXBTU2KYgITcJqcSJJ91VQqjz9evRXRn09iri0OGaOmMmE7hN0MLkKqbHdRwCN6wRyIlmTglLlwRjDq+tfpf8n/fG3+LP+7vXc2+NeTQhVTI1tKQA0DA3kQNxpb4ehVJWXmJnIXd/dxbe7v+WGtjcw67pZhAWGeTssdQ5qdFJoVDuAuJQsrFaDjxZCUuqcbD6+mVu+uoWjyUd5deirTOwzUVsHVViN7z7KtRri07K9HYpSVY4xhpmbZnLprEvJsebw87if+b9L/k8TQhVXo1sKDUMDATiZnOlczKaUKt22k9t4dPmjLNq/iOEXDufTGz6lflB9b4elykGNbik0qm1LBFe/tY7TqVlejkapyu9o0lHu+u4uOs/szPrY9bw69FV+vO1HTQjVSI1uKTSuE+i8feRMOvVDtLWglDuJmYm8sO4F3tjwBlZjZdIlk3j8sse1smk1VKOTgmsSyM0zXoxEqcopKzeLd6Lf4bm1z5GQkcCYqDH8Z+B/iAyL9HZoykNqdFLws+T3nqVm5XgxEqUqF6uxMmf7HJ5Y+QSHEg8xpNUQXhz8Il0v6Ort0JSHeXRMQUSGicgeEdkvIpPdPD5ORE6JyBb7n/GejMedx4a3BSAlU0toKwWwImYFPT/oyZhvxhAWGMbS25eydOxSTQg1hMeSgohYgBnAcKA9cKuItHdz6lxjTBf7nw89FU9xruvSFIC0rLyKfmmlKpU/T/zJsM+HMfizwcSnx/PZDZ+xecJmhrQe4u3QVAXyZPdRL2C/MSYGQETmANcBOz34mmctJND2I9DuI1VTHUk6wlOrnuKzPz8jLDCMV4a8wv297ifQN7D0J6tqx5NJoSlw1OU4Fujt5rybRORyYC/wf8aYo4VPEJEJwASA5s2bl2uQQX4WRODPo0mcSMosMCNJqeosPSed535+jlfXvwrAvy79F5P7TSa8VriXI1Pe5O11Cj8AkcaYKGAZ8D93Jxlj3jfG9DDG9GjQoEG5BuDjI4T4+/LTtr/o8/yKcr22UpXVipgVRL0bxfPrnmdkh5Hs/edeXhzyoiYE5dGkcAxo5nIcYb/PyRgTb4xxrBr7EOjuwXiK5ehCUqq6i0+PZ9y34xj82WB8xIdVd67i0xs+pXmd8m2Bq6rLk5+G0UAbEWmJLRmMBm5zPUFELjDG/GU/vBbY5cF4ihUa6MtfSd54ZaUqhjGGL7d/ycTFE0nITODxfo/z5OVP6j7JqgiPJQVjTK6IPAAsASzALGPMDhF5FthkjPkeeFBErgVygTPAOE/FU5Igf20pqOrrUOIh7vvpPhbvX0yvpr1Yfs1yohpFeTssVUl59NPQGLMQWFjovikutx8DHvNkDGXhZ9Gqjqr6ybPm8eaGN3ly1ZMIwhvD3uD+nvdj8bF4OzRVielXZMCieymoaubPE38y/ofxbDq+iRFtRvDOiHd03ECViSYFwNfH25OwlCofGTkZPLPmGV759RXqBdVjzk1zGNlhpO5xoMpMkwLormuqWlgRs4J7f7yXAwkH+FuXv/Hy0Je1iqk6a5oUgLpBfs7bujWnqmri0+N5ZNkjfLLlEy6seyEr71jJwJYDvR2WqqI0KQBTr+3A4h0nyMyxkpVrpZa/DsSpyi8xM5EZG2fw2m+vkZSVxGP9HuOpy5/SaabqvGhSAMKC/Jk8rC1Tf9hJenauJgVVqcWlxfH6b68zI3oGyVnJjGgzgumDpus0U1UuNCnYOdYqZORotVRVOR1JOsIrv77CB79/QFZuFrd0uIXH+j1Gl8ZdvB2aqkY0Kdg5WgdaQltVNnvj9/Liuhf5dOunAIyNGsujfR/l4voXezkyVR1pUrCLrBcMwL64FC5uHOrlaJSyrTWYvm46X+34igDfAO7rcR+PXPqIV9Yb5OTkEBsbS2ZmZoW/tjo7gYGBRERE4OfnV/rJbmhSsLuocQh+FmHbsSSujmri7XBUDfbr0V+ZvnY6P+37iVD/UB7t+ygT+0ykUUgjr8UUGxtLaGgokZGRuuahEjPGEB8fT2xsLC1btjyna2hSsAvwtdCyfjD7T6ayfOdJBrVrqP/4VYUxxrA8ZjnT1k5jzeE11KtVj+cGPsf9ve4nLDDM2+GRmZmpCaEKEBHq1avHqVOnzvkamhRc1Knlx4rdcazYHcd7Y7tzZYfG3g5JVWPGGLae3MrSA0uZu2Mum//aTNPQprx25Wvc0+0egv2DvR1iAZoQqobz/T1pUnARHJD/4ziRpH2nqvydTD3JsphlLD2wlKUHlnIy7SQAUY2ieP/q97mj8x0E+AZ4OcrKJzExkdmzZ/OPf/zjrJ971VVXMXv2bMLCytbimjp1KiEhITzyyCNn/VrVgSYFF65JIddqvBiJqi4yczP55cgvLD2wlCUHlvDnyT8BqB9Un6GthzK01VCGtB5Ck1AdxypJYmIi77zzjtukkJubi69v8R9lCxcuLPYxVZQmBRchLvsqWDUpqHNgjGHX6V3OlsDqQ6vJyM3Az8ePvs378vyg5xnaeihdGnfBR7QQY1lNnjyZAwcO0KVLF4YMGcKIESN46qmnCA8PZ/fu3ezdu5frr7+eo0ePkpmZyUMPPcSECRMAiIyMZNOmTaSmpjJ8+HD69evHr7/+StOmTfnuu++oVav4FeBbtmzh73//O+np6bRu3ZpZs2YRHh7Om2++ycyZM/H19aV9+/bMmTOHNWvW8NBDDwG2Lpyff/6Z0NCqN5NRk4IL1205HS2FzYfPYDXQM1ILi6nibYjdwPub32dpzFJik2MBuLjexYzvNp4rW19J/8j+hPiHeDnK8vHMDzvYeTy5XK/Zvkltnr6mQ7GPv/DCC2zfvp0tW7YAsHr1an7//Xe2b9/unGUza9Ys6tatS0ZGBj179uSmm26iXr16Ba6zb98+vvzySz744ANGjhzJ/Pnzuf3224t93TvuuIO33nqL/v37M2XKFJ555hlef/11XnjhBQ4ePEhAQACJiYkAvPLKK8yYMYO+ffuSmppKYGDg+f5YvKJMSUFEWgOxxpgsERkARAGfGmMSPRlcRSvQfZRnBeCmd9cDcOiFEV6JSVVux1OOM3n5ZD7b+hl1AuowpPUQhrYaytDWQ2kR1sLb4VVrvXr1KjDt8s0332TBggUAHD16lH379hVJCi1btqRLF9sK8O7du3Po0KFir5+UlERiYiL9+/cH4M477+SWW24BICoqijFjxnD99ddz/fXXA9C3b18mTZrEmDFjuPHGG4mIiCi391qRytpSmA/0EJELgfeB74DZwFUlPUlEhgFvYNuO80NjzAvFnHcT8DXQ0xizqYwxlbuQgPyaR+la7kKVIDM3k9fWv8a0tdPIsebwWL/HeKzfY4QGVL3ugrNV0jf6ihQcnD87a/Xq1Sxfvpz169cTFBTEgAED3C60CwjIH8S3WCxkZGSc02v/9NNP/Pzzz/zwww9MmzaNbdu2MXnyZEaMGMHChQvp27cvS5YsoW3btud0fW8qa1Kw2vdcvgF4yxjzloj8UdITRMQCzACGALFAtIh8b4zZWei8UOAhYMPZh1++XFsKqZm5BR7LybPiZ9E+4JrOGMO3u7/l4aUPczDxINe3vZ7/Dv0vrcJbeTu0ai00NJSUlJRiH09KSiI8PJygoCB2797Nb7/9dt6vWadOHcLDw1m7di2XXXYZn332Gf3798dqtXL06FEGDhxIv379mDNnDqmpqcTHx9OpUyc6depEdHQ0u3fvrtZJIUdEbgXuBK6x31faGupewH5jTAyAiMwBrgN2FjrvP8CLwL/KGIvHhLgkhbSsgkkhLiWLpmFakrgm2x63nYmLJ7Li4Ao6NOjAsrHLGNxqsLfDqhHq1atH37596dixI8OHD2fEiILducOGDWPmzJm0a9eOiy++mD59+pTL6/7vf/9zDjS3atWKjz/+mLy8PG6//XaSkpIwxvDggw8SFhbGU089xapVq/Dx8aFDhw4MHz68XGKoaGJM6bNsRKQ98HdgvTHmSxFpCYw0xrxYwnNuBoYZY8bbj8cCvY0xD7ic0w14whhzk4isBh5x130kIhOACQDNmzfvfvjw4bN5j2W2Zu8p7py1EYAh7RvxwR09iJz8E1ZSee22SG6K6u2R11WV25mMM0xZNYV3N71LnYA6/Gfgf7i3x734+tSceRq7du2iXbt23g5DlZG735eIbDbG9CjtuWX6V23v8nnQfuFwILSkhFAWIuIDvAqMK8Prv49tLIMePXp4bK5o07D82QIHT6eRZzUYDCcDnubhFXB1+z91YVENkmvN5b1N7zFl9RQSMxO5r8d9PDPgGeoF1Sv9yUpVUWXqJBeR1SJSW0TqAr8DH4jIq6U87RjQzOU4wn6fQyjQEVgtIoeAPsD3IlJqJvOUC+rkdw/tj0slNiEdQQjLuZXDyXuYvna6t0JTFWzlwZV0fa8rDyx6gC6Nu7Dl3i28fdXbmhBUtVfWkdM6xphk4EZsU1F7A6V1pkYDbUSkpYj4A6OB7x0PGmOSjDH1jTGRxphI4DfgWm/OPnIdaAbYctQ247aWtQcDm93E8+ueZ9vJbd4ITVWQmIQYbpx7I4M+HURadhoLRi1g+djldGrUyduhKVUhypoUfEXkAmAk8GNZnmCMyQUeAJYAu4B5xpgdIvKsiFx7TtFWsLjkLOftv3WcSlhgGHd/fzd5Vp2uWt0cOHOASUsm0X5Ge5YeWMr0K6az8/6dXN/2ei0Ep2qUso6UPYvtw/0XY0y0iLQC9pX2JGPMQmBhofumFHPugDLG4lF/PDWEQ/Fp3PDOr7yzer/z/iDfMN4c/ia3zr+VNza8waRLJnkxSlUe8qx5LN6/mBnRM1i8fzEWHwucgIfZAAAgAElEQVRjOo1h+qDpWotI1VhlHWj+CvjK5TgGuMlTQXlTeLA/mbm2lkBCeo7z/tw8w6gOo5i9bTZPrnyS69ter3PTq6j49Hg++uMjZm6aycHEg1wQcgFP93+ae7rfo8lA1XhlHWiOEJEFIhJn/zNfRKrmGu4ycF2vcFtv29aH2XlWRIR3RryDn8WPe364h7JM51WVR/SxaMZ9O46mrzbl0eWP0rxOc+bdPI/DEw/z9ICnNSFUMyEhtlpTx48f5+abb3Z7zoABA9i0qeRhzNdff5309HTn8VVXXeWsd3Q+pk6dyiuvvHLe1ylvZR1T+BjbIHET+58f7PdVS8Eu1VK7RNhqsDtqIUXUjuClwS+x8uBKZv0xyyvxqbLLyMngky2f0OuDXvT6sBfzd83nb13/xrb7trF63Gpu6XALfpZz28tWVQ1NmjTh66+/PufnF04KCxcuLPPeDFVRWZNCA2PMx8aYXPufT4AGHozLq3x88gcWL2psq2Xjur/CPd3voX+L/jy89GGOpxyv8PhU6Q4mHOTfy/5NxGsR3PXdXaRmp/L28Lc5NukY74x4h44NO3o7RHUWJk+ezIwZM5zHjm/ZqampDBo0iG7dutGpUye+++67Is89dOgQHTvaft8ZGRmMHj2adu3accMNNxSofXTffffRo0cPOnTowNNPPw3YiuwdP36cgQMHMnDgQMBWivv06dMAvPrqq3Ts2JGOHTvy+uuvO1+vXbt23HPPPXTo0IGhQ4eWWmNpy5Yt9OnTh6ioKG644QYSEhKcr9++fXuioqIYPXo0AGvWrKFLly506dKFrl27llj+41yUdaA5XkRuB760H98KxJdrJJVUZL0gALJyrcyLPsp1XZsQ4Gvhg2s+IGpmFA8sfIBvRn3j5SgV2OoSLT2wlLc2vsXCfQvxER+ub3s99/e8nwGRA3QWUTmZuHgiW05sKddrdmnchdeHvV7s46NGjWLixIncf//9AMybN48lS5YQGBjIggULqF27NqdPn6ZPnz5ce+21xf6u3333XYKCgti1axdbt26lW7duzsemTZtG3bp1ycvLY9CgQWzdupUHH3yQV199lVWrVlG/fv0C19q8eTMff/wxGzZswBhD79696d+/P+Hh4VW6RHdZWwp/wzYd9QTwF3AzZViJXB04xhdW7o7j3/O38uKiPQC0qdeGZwY8w4LdC5i/c743Q1TA6kOr6TurL8O+GMam45t48vInOTTxEF+P/JqBLQdqQqjiunbtSlxcHMePH+fPP/8kPDycZs2aYYzh8ccfJyoqisGDB3Ps2DFOnjxZ7HV+/vln54dzVFQUUVFRzsfmzZtHt27d6Nq1Kzt27GDnzsJl2gpat24dN9xwA8HBwYSEhHDjjTeydu1a4PxLdP/888/OGMeMGcPnn3/u3F3OUaL7zTffJDExscRd585FWWcfHQYKrC0QkYlA8am9mrDYu5Ks9kHl348kOB+bdMkk5u6Yy/0L72dgy4HUraUb8VS06GPRPLHyCZbFLKNpaFPeu/o9xnUZh7/F39uhVVslfaP3pFtuuYWvv/6aEydOMGrUKAC++OILTp06xebNm/Hz8yMyMtJtyezSHDx4kFdeeYXo6GjCw8MZN27cOV3HoSqX6D6fWtDVeqL+53f3Zs6EPogIfhYhI9s2TfXomXSOnknHGIOvjy8fXfsRp9NP8/DSh70ccc2yI24HN869kV4f9uKPE3/w36H/Zd8/9zGh+wRNCNXUqFGjmDNnDl9//bVzs5ukpCQaNmyIn58fq1atorRimZdffjmzZ88GYPv27WzduhWA5ORkgoODqVOnDidPnmTRokXO5xRXtvuyyy7j22+/JT09nbS0NBYsWMBll1121u/LtUQ34LZE94svvkhSUhKpqakcOHCATp068eijj9KzZ09279591q9ZkvNpd1Tr9ni/Nvn9hzl5ht0nbP8o4tOyueylVTx9TXvu6tuSLo278GjfR5m+bjq3dbyNIa2HeCvkGiEmIYapq6fy+dbPCQ0I5dkBzzKxz8QasblNTdehQwdSUlJo2rQpF1xwAQBjxozhmmuuoVOnTvTo0aPUb8z33Xcfd911F+3ataNdu3Z0794dgM6dO9O1a1fatm1Ls2bN6Nu3r/M5EyZMYNiwYTRp0oRVq1Y57+/WrRvjxo2jV69eAIwfP56uXbuW2FVUnMpUortMpbPdPlHkiDGmeblGUwY9evQwpc0rLm+Rk38qct/gdo348E5b7b7M3Ey6zOxCVl4W2+7bVm324q1MjiUf47mfn+PDPz7Ez8ePf/b6J//u+28tUFdBtHR21eKx0tkikgK4yxoC1OgdZ3Ls6xYAAn0D+fDaD7ns48t4cuWTXutzrY5Op5/mxXUv8nb02+RZ85jQbQJPXP6ELjRTykNKTArGGG2TFyPXai1w3K95P+7veT9vbniT0R1H0yeifHZ+qqmSs5J5bf1r/Hf9f0nLSWNs1Fie7v80LcNblv5kpdQ5002Hz1FObtEG1PODnieidgR3f383WblZbp6lSnM85Tgv//Iyrd5oxdQ1Uxnaeijb7tvGJ9d/oglBqQpQc/YTLGc5hVoKAKEBocy8eiYjZo9g+trpPDPwGS9EVrVk52Xz69FfWbx/MYv2L2LrSdtskCtbX8lzVzxHjyZe23NJFWKM0fUeVcD51mTTpHCOcvPc/+CvanMVYzqN4fl1z3Nz+5t1cxY3jiQdcSaBFTErSMlOwdfHl37N+/Hi4BcZfuFw/blVMoGBgcTHx1OvXj1NDJWYMYb4+PjzWuWsSeEcuQ40F/b6sNdZcmAJd39/N+vvXo/Fx1KBkVU+WblZrD2ylkX7FrH4wGJ2nrKtFG1epzm3dbqNYRcOY1DLQTqttBKLiIggNjaWU6dOeTsUVYrAwEAiIs69iLUmhXNUUlKoH1Sft4a/VaM35IlJiHEmgZUHV5Kek46/xZ/+Lfpzd9e7GX7hcNrWb6vfOqsIPz8/WrbUMZ2awKNJQUSGAW8AFuBDY8wLhR7/O3A/kAekAhOMMSUXHKkkcq2mxD7WUR1G8cW2L6r9hjzpOensPLWTrSe3su3kNrbFbWPrya2cSrd9o2wV3oq7utzF8AuHMyByAMH+wV6OWClVknNevFbqhUUswF5gCBALRAO3un7oi0htY0yy/fa1wD+MMcNKuq43F69NHNyG15fbdiG1+Ah5VsOrIztzYzf3TbXY5Fjaz2hPy/CW9GvWD4uPBR/xwSL2v8twHOgbyJBWQ7i4/sUV9n7dsRorMQkxRT7895/Zj7EvZanlW4uODTvSqWEnul3QjaGth9KmXhuvxq2UsimXxWvnqRew3751JyIyB7gOcCYFR0KwC8b9Qjmv87MIOXmGG7tGOJNCnn1/hU9+PVRsUoioHcH717zPw0sfZu6OuViNlTyTZ/vbavvb9b6SdG7UmVEdRjGq4yiPtzpyrblEH4tm47GNzg//Had2kJ5j22hEEFrXbU1Uoyhu63QbUY2i6NSwE63CW9X48ROlqjpPthRuBoYZY8bbj8cCvY0xDxQ6735sxfX8gSuMMfvcXGsCMAGgefPm3UsrelXeUjJzsBrAQOdnlxZ4rGX9YD68swct6gbhazm/ZR/OJGFPGHkmj/j0eL7d/S1zd8xlfex6AHo26cmoDqMY2WEkzeo0O6/XBNuMhb3xe1kWs4zlMctZdWgVyVm2fF0/qD6dGnZyfvBHNYqifYP22g2kVBVT1paC15OCy/m3AVcaY+4s6bre6D5ysFoNrR5f6Paxey5ryRMj2pOZk0dSRg6NapfvxhcAhxMPM2/HPObumMvmvzYD0LdZX0Z1GMUtHW6hcUjjMl8rLi2O5THLnX+OJh8FIDIskiGthjCk1RD6Ne9H45DGOhisVDVQGZLCJcBUY8yV9uPHAIwxzxdzvg+QYIypU9J1vZkUwH1xPIAOTWrz04OXMfr99fwWc4ZDL4zwaBz7z+xn7va5zN0xl21x2xCE/pH9Gd1hNDe1v4n6QQV3iUrPSWft4bUsi1nGsphlzkVi4YHhXNHyCoa0GsLgVoNpXbe1R+NWSnlHZUgKvtgGmgcBx7ANNN9mjNnhck4bR3eRiFwDPF1a0JU1KQA8Nrwtzy+y1Tb3dFJwtevULubumMuc7XPYE78Hi1gY1GoQN7e7mVPpp1ges5xfjv5Cdl42/hZ/+jbr60wC3S7opuMAStUAXk8K9iCuwrY7mwWYZYyZJiLPApuMMd+LyBvAYCAHSAAecE0a7lTmpOBq/7Th5z3GcLaMMWw9uZW5O2wtiJiEGMA2SO1IApe1uIwgv6AKjUsp5X2VIil4QlVJChMHt+HqqCZc2NA7eysYY9hxagcNgxvSMLihV2JQSlUeZU0KWiX1LI3sEUHPyHC2Th3K+seuKPa815fv4+aZv1ZgZAWJCB0bdtSEoJQ6K1rm4iy9dHNn5+3agX4lnpuYnnPOr5OVm8eHaw9yz2Wt8PfV3K2Uqhj6aVOOQgPKL8d++uthXl6yh1m/HCy3ayqlVGk0KZSj2rVKbjmcjcycPMC2cE4ppSqKJoVylJ6dW27XslhsC8ZyrVVrIoBSqmrTpFBOLD5CwnmMIRTm62NLCnnFbOajlFKeoEnhPHVvEQ7AgelXUTfYv8jjuSXsu1ASi4/tV6MtBaVURdLZR+fpy3v6kG3/4J9/36XM3nCYD9bmDw4npOeQk2elbrA/gX5lXznsbCloUlBKVSBtKZwnf18fQuyzjlrWD+aazk0KPH4mLZtLX1jJHR9tBOBEUiYz1xwodXNti4+OKSilKp62FMpZ4TUFJ5IzAdh46AwAD8z+nU2HE2hcO5AvNhzm1ZFdaFa3aNkJi7OlcG7dT0opdS60pVDOAnwLdhEdPZPuvH0iKZPD9uM50UeIPpTA9IW73F7H0ZDQloJSqiJpUihnhVsKR1ySwu9HEsjItq0/+C3G1nJYtP0EM1btJzMnj9kbjji7lRwtBB1TUEpVJO0+Kmf+hSqjHonPTwrbjiWRlZtX5DkvL9nDmbRsPlp3kIahAQxu34gc+1TUsrYUNh8+Q/sL6lDLX8tgK6XOnbYUylmAX/6P1EfgZEqm8/hkUqbzw95Vi3pBHEvIAODhr/7kpcW7eXnJHsC221tp/krK4KZ31/PEgm3nG75SqobTlkI5C/bP/5GGB/nzx5FE5/HB+DQABl7cgDG9W9AjMpy/f76ZnDzjbEEkZeTwzuoDzueUpaUQn5oNwK4TKeXyHpRSNZe2FMqZxUe4t38rWjUILtCVUzvQl5hTtqQwqmdzBrdvRFiQP6GBfqRl5TrXOhRWljGFlExbeY1afvrrVEqdH/0U8YDHhrdj5cMDcF2K0DQ8iKQMWxmM+iH5K5+D/S2kZ+eRleM+KeSUYUX0mTRbS0HHE5RS58ujSUFEhonIHhHZLyKT3Tw+SUR2ishWEVkhIi08GU9Fc/1AbxoW6LxdLyTAeTsowJf07Fyyct1/+MecSnPOWCrOmXRbUgj01aSglDo/HksKImIBZgDDgfbArSLSvtBpfwA9jDFRwNfAS56Kxxtcu34uqFMLsHUvNQzNTwrB/hbi07JJy3JfYfVYYgabDp8p8XXO2McUdDMepdT58uSnSC9gvzEmxhiTDcwBrnM9wRizyhjjmLP5GxDhwXgqnKOlMO/eS5z3XdG2IcEum/EE+ftiDMScTiv2OumltBQS7C2FsnQ1KaVUSTyZFJoCR12OY+33FeduYJG7B0RkgohsEpFNp06dKscQPcvRUqhTy48ekbZqqo8MvbjAOWX5du/YcAcgLjmzSN0kx+PLd8UxN/oIVqshcvJPvGKf1qqUUmVVKfobROR2oAfwsrvHjTHvG2N6GGN6NGjQoGKDOw85Lknhui5N2TdtOBc3Di1wzqmUrFKv4xiE3n0imV7TV/DFhiMFHs92GY+Yte6QcybT+z/HnFf8Sqmax5NJ4RjQzOU4wn5fASIyGHgCuNYYU/onZBUSYG8F1LFv0+lnKfrjDg4ofXA4w94ScKyOXr0nrsDjWS7dRvVC/J2D1iLnELRSqkbzZFKIBtqISEsR8QdGA9+7niAiXYH3sCWEODfXqNLmTriEiYPbEFjC+oF/XtGmSGmMwjJz8vj8t8POb/6ZhaavurYU6gb7O48dlVaVUqqsPLai2RiTKyIPAEsACzDLGLNDRJ4FNhljvsfWXRQCfCW2r7VHjDHXeiqmita+SW3aN6ld4jmBfhaGtG/ET9v+KvachPQcnl+023nsOsYABZNCWJCfs/vIR5sKSqmz5NEyF8aYhcDCQvdNcbk92JOvX1Vc0rpeiUlh5poDBY4zc4tPCnnW/ONzyQlJGTk8v3AXT13dvsAsKaVUzVApBpprujG9m7Pu0YFlPr9I91Gelb4X1iMivBZZuXnOpFBcSyEhLZtPfjlYYBZTntVgtRpmrjnAnOijjHp/Pd/+UWQISClVzWlSqAREhIjworuvFcfRfXTodBrxqVnsOZGCv8WHAF8fsnKsbscUrFbjTAL/nr+VqT/sZPuxZOfj7acs5uq31jlLc2w/lszEuVvO960ppaoYTQqVyPJJ/ct0XmZOHgdOpTLgldV0f245qVm5+Pv6EOBr4adtf3E0wTZLyZETxn60gVaPL+TR+VsBOJ1qm+SVnZffDZWVa2XnX8mUNOa9/VgSP++tOutElFJnT5NCJXJhwxCa1a1V6nlZOVZnxVUHf18Ladm2Uhn/+OJ3wNYCMcawdt9pAOZtii3wHOOmAKulhIGIq99axx2zNpYan1Kq6tKkUMnUDfIv9ZzM3LwitZJ8fYTcQhv4WETcburjkJFTtHzGmyv3lzFSpVR1pEmhkgkPzk8KTcPyWw2rHxngvJ2TZ0gplBRy8qxFpqr6CG4rrDpaCGlZtsfKsrubUqpm0KRQydS1J4WXbopi3aMDGdK+Eb0i6xJZP7jAeSeTMgsc5+RZi3zzFxG3rQGH9Oxcftx6nFaPLyz2HLDNTMp1U2wvNSuX9lMWs3L3yRKfr5SqOnQieiUztk8Lvvn9GL1b1UVE+OCOHm7Pe3tVwW6enDxTJAFYfNwnBUe7IC07j6U7Sv9AH/raGo6eyXAe51kNFh9h5e440rPzeG3ZPq5o26jU6yilKj9NCpVM1+bhHHphxFk/LyfPWmTg2F33UZbLwrf0rNwyLXA7UGhQ+9cDp2nVIIQHv/wD0H0clKpO9H9zNZHtZuc2HzfdRwlpOc4xhLTsvHNa9Tz2o43Ep+bXLiytdpNSqurQ/83VRE6elZ72PRscfHykyODzuv2nOXLGto5hzsYjLNx2osTrFt67wcF14x93LYXdJ5J5bdneYp+vlKqcNClUQe6qrubkGT65q1eB+3wEDpxKBeDJEe0AeOSrP0nKyAEgrtBeDu+M6VbkunnFzExyXAPcJ4UnFmznjRX72PVXSklvBSg+8SilKp4mhSokKqIOvj7uB5+v6XxBkQJ2eVbDlO92ABARXvqiuNDAokNMxa1zcE0ovvYWyTVvrePX/baFci3q2sp2rNpTekX07s8t5++fbS71PKWU5+lAcxXy7T/6AjjLWDh8eEcPBrVrWOR81wHipmGl11YK9Cu64Y+7sQqAWJcYsnKtRB86w7ZjSby0ZA/fXljfmWB2/ZXs9vkOqVm5nEnLZvGOkruxlFIVQ1sKVYiPj+DjI7SoF8ymJ/OrjjeuE4gUGjG+tHW9AseN6wSWev1A36JJ4XhShpsz4VhC/v3p2blsPpwAQOeIOkB+Jdf9canFvl5KZg7/p0X3lKpUNClUUfVDApy3w4L8ijxeuCsopAx7I4QHF73O8DfWuj33WGIGtQN9GXBxA9Kz84hPzQbytxx1TH3dfSKF7ceS3F5jxqoDLNtpWyfRqHaA23PcWbjtL15fvrfM5yulys6jSUFEhonIHhHZLyKT3Tx+uYj8LiK5InKzJ2OpzsLc1EsKCbB9wDcMDeDTv/Wiln/pe0G7JprivH1bVwBiEzKoE+RHkL+F9Ow85ywnxwZAmTlWguyv+dWmo26v5TpAfTZrHf7xxe+8vnxfmc9XSpWdx5KCiFiAGcBwoD1wq4i0L3TaEWAcMNtTcdQEwS4f+L1b1gUgwD5DqXerelx+UQMA/CwlL0oI9LMQ7qbV4ap1gxAATqVkUaeWH0H+vhyJT3dOUV237zTJmTlk5ebRukEILesHE5+WXeAaSRk5fLHhMGG18l/LUYfpfGyLTSK1UE0opdTZ8WRLoRew3xgTY4zJBuYA17meYIw5ZIzZCrgfzVRl4jqe8PFdPVn774Hk2WcNtWkY4nwswGXMwKeY/PD7U0O4pXuE28cCfH1oXDt/bKJOLT+ahQeRnWd1bid6KD6d8f/bRGaOlUA/H+oG+/Pj1r+46o21zkHrp77dzhMLtrM+Jh6ATk3rOKu+zo0+wje/x1KcrFz3ySM2IZ1r3l7HtJ92FftcpVTpPJkUmgKu/Qax9vvOmohMEJFNIrLp1Cnd5MVh9SMD+PKePgXuC/L3pVndIOJSbAXzXKeiBrh00ViKyQoi4nYWEkBooB/hwf7ORFOnlh8PXHFhkfM2HjxDZm4eAb4W6tkL/O38K5mTybaYHH+fSctGBAa1a0hWrpXcPCuPzt/GpHm2tRSuayEc7vjI/X4OC363bR2akln0OUqpsqsSA83GmPeNMT2MMT0aNGjg7XAqjcj6wVxSaJaRw8lk2zoC1/LbJSWFCZe3cnveiE4XOAeta9v/7to8zH7sh8VHnB/8rv44kkignw/1QvIfG/vRBr75PdZZWuNMWja1/CzOQfA0l1XSD8/bQudnlnIsMX+WU1JGDhsOnnEeOxa9pWXl8pZ9H4hmdcu+ralSqihPJoVjQDOX4wj7faoCOFoKTV1aChc2CnXeLrzD2uNXtXPebtkgv0z3XX0jefs220rnUPsYgKOMt699jML1g99Vdp5xlgIHW9fSY99sw8f+2scSM6jlZ3EuujuVkl8OfPku26K3P44kOO+LLbQ+I9tezjsuJct5u7h1FdGHznDTu78W2/2klLLxZFKIBtqISEsR8QdGA9978PWUiyHtbaWsXccA3hrd1Xnb8Y26a/Mwpt3QscBzb+vVnI/u7MHB56+iR2RdZwE9R0uhUajtmkkZtnGAesHuZy0lZ+TQqHbB9RHtm9R2thSyc60E+lmcs5T2x6UVvgRTv9/BC4t2269nez3HGow3lu/jZHKmc89pxzXd+ddXf7L5cAKHTqe7fdyd9OxcXli0m6T0iumSysmz8vlvh93uXaFURfFYUjDG5AIPAEuAXcA8Y8wOEXlWRK4FEJGeIhIL3AK8JyI7PBVPTfOf6zqyZcoQfF0qmNYJ8qP9BbUBmHptB165pTPf3HcpY3q3KPBcEWFQu0bOAexkez997UBbS8HRMnAki7rFtBSSM3Ocs5UcMrLzCnxw1/K3OK+7083q59Op2cxccwDIr7fULNyW0N5ZfYBB/13D0TP5H/TukoIxxtmdlppV9g/415btZeaaA3y7pWIauP/79RBPfrud2RuPFLh/+sJdjHxvvc6sUhXCo2MKxpiFxpiLjDGtjTHT7PdNMcZ8b78dbYyJMMYEG2PqGWM6eDKemsTX4uN2/YKjklFIgC83d48oshLanUta1aOWn4V7+9vGHfpdWJ97+7fiqattM4wbFFrf8PFdPQHbN/vCSeF0alaBAWSr1TiTzJLtxZe6yMrN48/YRADqh+a/r9SsXCbN+xOwFQrMdvMte+aaGGcJ8ZveXU9ienaRc1zFJWdy/Yxf+GDtQed1K8IZ+9Rd15ZJXHIm7/8cw8aDZ1hdhjpSZfXo11v5aN3Bcrueqj6qxECzKj8TB7cBKLK9Z0ka1g5k13+GERVhG2D2tfjw2PB2ztIZTcIKdhG1rm9LBCmZOUVWKp9Ozebg6fxuopjTac5Fc3tOptCxaW23MQx/fS3vrra1GNwtshOxdZW5GzP4aF1MgeMjZ0ruQvp8wxG2HE10HqdkVsw3dEfCds3TR13KiaSVY0th7qaj/OfHneV2PVV9aFKoYa7s0JhDL4woU9mLsipcbK9JWCBNw2ox7YZOiAhbpw7l/oGtubKDbZwjJ8/w+qguzvNdB6PbNMwfDH/xpk4MuNg22yzGJZHUdTPbqV5wAEH+vkW6jyIn/8Tp1IItg/jUklsKhT98HUkhJ89aZH+KH7ceJ8H+Df9sS4Cv3H2SyMk/EWMvb+54umvrLdmlVVV4F71zVV7XUdWTJgV13hrXKfjN3dfiwy+Tr+Bm+yK42oF+/OvKttzSPX8y2rCOjZ23A/0shNqTVIPQ/GuN6tmcj+7s6XzM9fzCWtQLws8iLN8Vx/j/RTN5/lbnmAfAyzdHOW871kkUp7ikMPK99bR9arHz/qNn0nlg9h9MmreF9OxcWj62kM/WHyrx2q5+/NO24O/3I7ZWibG3FVxbCsku6y4ycspnAPpYYtkH21XNo0lBnbcI+8DvyB4RJe4vHVE3f3qs44P9kla2mUQB9uMGIQG8eFMnXrixE2BbT/F/Qy4qcB13dZJa1Asiy95KWL4rjjnRR50bDAFc1CiU/1xnG7JyDDq/tWIfA19Z7fymvyEmns2Hz7Byd37fvY/kL4j7w/7h/dyPO4mc/BOXvbQKgL0nU/llv2119pxo93We3PGxrxWx2psIjpaCa3eV6/hL4VbKuYpNcF/5VinQ/RRUOWhUO5Af/9mvyKByYa4L6QD2PDcMXx/bB3ztQF9Op2bRIDSA67sWXPjuOq31pZui3O4J3aJuMFuOJBa4b+Oh/IVurRuG0LlZGK8v38fJlEysVsN/l9kqrW6JTWTLkUTeWFG0yF6TsFpFxhQ+dDNA+5u9ZMeFDYv/GWw8eIYL6gQ6pwM71g86drdz/P3u6gNcYq9Z5eg+Eim/pOCY5dWkDOXUq5sj8ekEBVjKVPyxptKWgioXHZvWKbUSa6h96mkte6sgwNfiXFndqkHBBdZAk5sAABVZSURBVHGuQuzrI65o25CRPZu5bSm0aRTibCk4/Gr/9v7iTZ2cYygNawcSl5xJp6lLnOe9vHiP24QAtu6s1KzcAjOWChcWTEzPZu9J27ajqSUMSo98b72zdQE4F/E5uqtcP/Q32lduJ2fmEujnQ+1Av3JLCtH2aweV47hSVXH5y6u49PmV3g6jUqt5/yqUV82/71K3G/5MvbYDVgOXtSlaxqR3y7qM7dOCf9rrLLlrKbRtHFokKTiK9LlOzW1UO8C5WtqhpPn/tQP92HAwni7PLnPe1zA0sED5jbTsPNbus21DeiY9m0lzt2DxEe4b0JqkjBwiwoO49IUVzvM3xMTTu1V+eRJHayDdZQDY4iMcPZPO5sMJ1KnlhyDOabXnIys3z5lwaurqbnfTllU+TQqqQnVvEe72/ojwIGaN6+n2sUA/C/+5Pn/VtfsxhWCyXT7kujQLc04rDfbP/2feMLRot0G8y4pogP9c14Gn7HtbD27fiDV7CxZhPOEyUO3rI+S6DGjHJmQ4xx5EYN6mWC5uFFpgr+uPfzlE71b1nLWekjJy+C0mnnX2/a0BDpxKdbYqLmoUQnau1TnQvO9kCkcT0rmibaMCcRljyLOaAgsWC9t0KIG07DwahgaQVU4D11VFnvXsZofVVNp9pKqsixqFEP3EYH54oB8WH8kfaJ7Un39febHzvOCA/G4t1ymr397fl9BA3wKF+B4ZehFjL4l0Ht/Wqzmdm4UVeF3XDxfXvulafhZOpeQnmPn2yq177F1LDo5WhmMAOykjh9Hv/1bguT9u/ct5u0lYLQL9LM7uoyGv/czfPtlUpBzGcz/t4sInFjlnXW2LTeK3mHgS07OdGx0djrfNPOraPKxIyyq5mleY1Qq6ZaNJQVU5jimbreqH0CA0gE72faEd3QK1A31p41L8L9il77yuvU7TV3+/hC7NwooMODrGPebfdylzJvTB4iN8fnevIpVgHceu1WZHRF3gvH1BnUC330w7Nq3tTAonkmwtjl8PxJf4fvtdWN+ZFM64bFi0/1TB/a8dK5RPJGeSk2flmrfXMfr93/j755v519db2XMixTmbqWFoIEkZOdz2wW8AfLg2hqipS9nlptTI2dh+LInnftzpXLPR8eklPPtDyYvkjsSnFyl26AkJFVTDqqrTpKCqnLaNa/PKLZ15+ZaoAvc7pnSGBPpS36UeU5DLAPgjV17Ee2O70zPStkOdY3Gcg2NAunuLcPrY+/1DA/2KtBYa2mdEuSaFe13Kjxe+rkP35uGcSctm8fa/2H3C1oKIs7cQGtcuOtbSvUU4f+vbklr2pOC6GnujSxlxV5e+sJJ7Pt3kPP79sK076/Xle1m8/S/8LT7UsVe8dSSk/y61zcTadOiMc1A9J8/Ky0t2F+leK8nYjzbw4bqDnEnLxmo1pGblMuuXkstpXP7yKvq9uKrEc8pDQinlTZSNJgVVJd3cPcL5rd6htX0GUy0/S4FVwa5jCkH+vlzZIX/h3O19ChYDzCtmVfIz13agQ5P8EhyO8h2u21K4DqCPu7QlAP/f3p3HR1VlCRz/nayEJCTEhAAJewIIyK6AIi2ICri1igjtR20/IuN0q632R0dGZVptR7t7bAVb7XaYdmkVsd1Ax2kFxAVB9i2AIGiAJGIISwiBhCxn/nivHlVFJSQ2WSqc7+eTT+q9uknuCUVO3XvfO3d0rzTvD/CrtwzjnG5Oonn0gy2IwL1+01zzbj+Phff8JODqpr4d2xARIcTFRHK0ooo9xUe9n/v0wm/IyS9mxrwccvKLA/r76dbj6yC+EdT/5exhfV4xbeKiA/bM8PfQvE0MfGQBu/cfYe2ugzy7eAfjZ35R5/l431Rc0eFjITdJ8ldZVX1Cv3025hX/6IXw/r/5iJkh9vD2rylV37vPG9qmgmI25oX+XTQ2SwqmxXhj2gjemDbcSwizbxzKyKxU2sTVvO9096AaUDWV3u6U0pr/vfN8716Ltu4VTRER4t25ndgq2tvKtEdaPAvuHsXMKYP49cXOzXdnZSZ5myLlHzzKLed1Y2RWqvcz2raOIatdAoM7H1+M/9cLegBOUb6c/EPc9uoaAF66+RxUlckvfMUry3Zyx5y1dfkVAZAUFxVw1/Rlz3xxwpVNK3P3e3/UC0vKmTEvp9a7tauqlWueX+r9/gpLythXWvsIY8QTn3DZM0u8Y9+aydY9JVz+pyVMfH5ZnWPyOVZZzaGySp5auO2E5/xLrB9pZqU+Lp21hMv/tOTkDRuBJQXTYqQlxnpTPuBcOfTq1GE1bj0KTp2hpfePYcHdo7hmcOYJN84FaxvvJBjfDXXndE3hd9f05+tHxwHw+NVneSXLs9MTadMqmhuGd+G7xyeQFBdNSnwM1w/rzMV90rlzbDYd/IoJ+q6qeuDSM4mJjGDB3aPokOQkId9d4z7nZ6fymyv6epfT+hcZ9Pn5uV2JChF7Ulx0QBLIyT9xHeHDjXsCpqBeW76Lh+ZtYtKfQ/+hXp93kNU7j2+ItLek/ISaU/62F5YELKzD8UVw3/rCxhpGEaEs3V7EPW+uq3WKyP8y4h9bhry0vJJzH1/Ex5tqrugb7uySVHPa6+i++39y0oCTtr1yQAZb95QwKjuVy/p3IDs9gcgIITLCWbcIVbI8uDz5Y1ed5T1OiDnxv2D/zGS2PTY+4Jz/1NWEs9ojIlwxoCNV1Urx0QoeDrGYe/2wzsTHRvLs4h3e8WvLd1F8tCLkO+UIcabaqlRZuOWHkPGvyN3Py0tzeX99AU+5RQ33lx4LmK4C5w/6u2uP70MxZ8UuLunbniXbi4gQZ+8InwGdklm/+yCLvv6BtMRYbnn5eDIqPlrhTb/5K6uo4qrnlnLfJb0Y3bsdP5u9HIDL+3f02kx/ZwMjs9KIjIDOKfEB5T1CbQBVF+t2H6SguIyH5uVwsd80JEDBwaMs+aaISWd3quGrQ6vwu4rsyLFKWod4TTQmSwrG1MOto7oz9fxuddqHoi4iahnF+PONgGbfOJSx7q56IsLVgzPZte8ID7+/mdSEmIB3551SWnP32J48u3gHI7NSufm8bry2fBd9OiaFrJT6/h0j6ZmeyLV/XhZQOjzYf364hfLKambMy+Hzb4pCrje8+GVuwPH0dzYy/Z2NIb/ftUMyaRUVwdyVu9kRtPvePXPXsejrQiYNzeT3E52kvWPvYS588jMA7n1rAyv+/UKvfWAi2s2cFbvd3xUM73Z8FPl9cVnAFWq1qayqpqJKiYuJ9LaHDY65pKyCc59w7pQe2yc9ZCXfmviuQgMoOFgWUCrlcHklR49VBRSKbGiWFIypp1OVEHwevPRMb/e5mqS3aVVjscFOKXHcOSaLCf07UFmldD6jNa2iIr3pqKX3j6FNXDQJsVF8cd9o2rSKPmF3t5tGdKFvR+fS3huGd6k1Kfjub1gcNDoIZXDnZK8KbLDuqfF8W1RKVrsEJp/TibvnrvemkHwWucUJ31yVx11je3L97OUBU2VFh8sZ+MjH3vH89QUhf5YqfFN4mGHdUlj+3f56FQV8+P3N/O2rnTxyZV/+y71Kq+jwMXbsPezV+/Jt9ASQf+BovZKC/6XFuw8cCUgK1zy3lK0/lNRaaPJUk4ZchReRccBMIBKYrapPBD0fC7wCDAH2Adepam5t33Po0KG6atWq2poYY06iqlpZvfMA8bGRxEVH0j2omGFpeSVXPfclufuOMKFfe95b5/yxTU2IpehwOf0y2nhrEQvvGUX7pDjGPvkZew6VERkhVFUrr08dxrlZqZRXVvHil7neXts+MycPpG/HJLLaJVBWURVQlhxg+vjePLt4O4fcelLtEmO9y3fT28RydteUgJv8RmalBtwVHsofJvbn3rc2kJYYS/dUZ0opKlK4dkgmC7cU0iMtgXH92pOWGEtaYizFRyqYMOuLgO9x9aAM3l2Xz8isVHqkJXDVoAwm/WWZlyyz2jnn4mMi6ZeRRL+MJGIiI3h/QwEdk+M4u2sKZRVVXqXg+9/e4FXX/ZefdGf6+DOprlbeW5fvJZu1D11E23okmlBEZLWqDj1pu4ZKCiISCWwDLgLygJXAFFXd7NfmF0B/Vb1NRCYDV6nqdbV9X0sKxjSOY5XVKEpsVCSqiohQWl7JfW9v4BcX9ODSWc7VMtt+O56YqAg2Fxzio0176JramgfezWH1gxcFFEl8ZVkuM+Zt4prBmby9Jo+504YH1IB6d20eu/YdZV9pOaXlVd4aT1lFFZc8/XnAKGLm5IFc0rc9t7++hlbRkZzdNYWfDsxggN+owcdXiqRvxzbMmTacoY8urHf9o6x2CYw9M52e6Qlc1CedqS+vYnkN94nUx4DMJDZ/f4hrBmd6ieH87FQGdW7LLL8ijT8b1pl2ibHcNbZnTd/qpJpDUhgB/EZVL3GPpwOo6uN+bT5y2ywTkShgD5CmtXTKkoIxzcP2whI2FRziyoG1X7Hlr6SsgrjoSL76dj8js1NP/gWugoNHeXt1Hhlt4/jpwIwa12LeXLmbfhlJTJj1Bednp3LHmGx6pMUTHxvlvTNflbufiirlnG4pbPn+EHtLynlxaS6je6XRq30i767JJ7l1NCnxzqjogw0FPHf9YIZ0SfF+zrId+3jwvY1MPb87ZRVVfLu3lPvH96bvfzjVd6MihPQ2raisrvb276hJz/QEXr91OJNf+IrthYdrbfvizWczule7Ov/e/DWHpDARGKeqU93jG4Bhqnq7X5sct02ee7zDbVMU9L2mAdMAOnfuPGTnzp0N0mdjTMuwv/QY8bGRxEbVXs79VMs/eJStew4xulc7b+2ppKyCbT+U0D8zmc+37aV3hzbERkWwc98RUhNiyEiOIyoyggOlx9h94AjV6iw+79h7mC5ntCY5LoalO4pYtfMAd43N5twedU+m/uqaFMJioVlVXwBeAGek0MTdMcY0c/VZ6D2VMpLjTthMKrFVtDfKuPDM45Vtg+tutY2POb5uEHRVa31GVf+shrx5LZ/A0DLdcyHbuNNHSTgLzsYYY5pAQyaFlUC2iHQTkRhgMjA/qM184Cb38UTgk9rWE4wxxjSsBps+UtVKEbkd+AjnktS/quomEXkEWKWq84H/Af4mItuB/TiJwxhjTBNp0DUFVf0Q+DDo3Ay/x2XAtQ3ZB2OMMXVnBfGMMcZ4LCkYY4zxWFIwxhjjsaRgjDHG06AF8RqCiOwFfuwtzalA7RWzwkdLiaWlxAEWS3NlsTi6qGrozcP9hF1S+GeIyKq63OYdDlpKLC0lDrBYmiuLpX5s+sgYY4zHkoIxxhjP6ZYUXmjqDpxCLSWWlhIHWCzNlcVSD6fVmoIxxpjanW4jBWOMMbWwpGCMMcZzWiQFERknIltFZLuI3N/U/TkZEfmriBS6O9P5zqWIyAIR+cb93NY9LyIyy41tg4gMbrqen0hEOonIYhHZLCKbRORX7vmwi0dEWonIChFZ78bysHu+m4gsd/s81y0Vj4jEusfb3ee7NmX/g4lIpIisFZEP3ONwjSNXRDaKyDoRWeWeC7vXF4CIJIvIWyLytYhsEZERjR1Li08KIhIJPAuMB/oAU0SkT9P26qReAsYFnbsfWKSq2cAi9xicuLLdj2nA843Ux7qqBH6tqn2A4cAv3d9/OMZTDoxR1QHAQGCciAwHfgc8papZwAHgFrf9LcAB9/xTbrvm5FfAFr/jcI0DYLSqDvS7hj8cX18AM4F/qGpvYADOv0/jxqKqLfoDGAF85Hc8HZje1P2qQ7+7Ajl+x1uBDu7jDsBW9/FfgCmh2jXHD2AecFG4xwO0BtYAw3DuMI0Kfr3h7CUywn0c5baTpu67259MnD8wY4APAAnHONw+5QKpQefC7vWFs/Pkd8G/28aOpcWPFIAMYLffcZ57Ltykq+r37uM9gG+z17CJz512GAQsJ0zjcadc1gGFwAJgB3BQVSvdJv799WJxny8GzmjcHtfoaeA+oNo9PoPwjANAgY9FZLWITHPPhePrqxuwF3jRndabLSLxNHIsp0NSaHHUeVsQVtcSi0gC8DZwl6oe8n8unOJR1SpVHYjzTvscoHcTd6neROQyoFBVVzd1X06Rkao6GGc65ZciMsr/yTB6fUUBg4HnVXUQUMrxqSKgcWI5HZJCPtDJ7zjTPRdufhCRDgDu50L3fLOPT0SicRLCa6r6jns6bOMBUNWDwGKcaZZkEfHtYujfXy8W9/kkYF8jdzWU84ArRCQXeANnCmkm4RcHAKqa734uBN7FSdbh+PrKA/JUdbl7/BZOkmjUWE6HpLASyHavrIjB2Qd6fhP36ceYD9zkPr4JZ27ed/5G90qE4UCx31CzyYmI4OzFvUVV/+j3VNjFIyJpIpLsPo7DWRvZgpMcJrrNgmPxxTgR+MR9p9ekVHW6qmaqalec/w+fqOr1hFkcACISLyKJvsfAxUAOYfj6UtU9wG4R6eWeuhDYTGPH0tSLK420gDMB2IYz//tAU/enDv2dA3wPVOC8e7gFZw53EfANsBBIcdsKztVVO4CNwNCm7n9QLCNxhrsbgHXux4RwjAfoD6x1Y8kBZrjnuwMrgO3A34FY93wr93i7+3z3po4hREwXAB+Eaxxun9e7H5t8/7/D8fXl9m8gsMp9jb0HtG3sWKzMhTHGGM/pMH1kjDGmjiwpGGOM8VhSMMYY47GkYIwxxmNJwRhjjMeSgjE1EJEH3GqoG9wKnMNE5C4Rad3UfTOmodglqcaEICIjgD8CF6hquYikAjHAUpzrwYuatIPGNBAbKRgTWgegSFXLAdwkMBHoCCwWkcUAInKxiCwTkTUi8ne3xpOvxv/v3Tr/K0Qkyz1/rYjkiLMnw+dNE5oxNbORgjEhuH/cl+CUyF4IzFXVz9x6QUNVtcgdPbwDjFfVUhH5N5y7gB9x2/23qj4mIjcCk1T1MhHZCIxT1XwRSVanhpIxzYaNFIwJQVUPA0NwNi/ZC8wVkZ8HNRuOs3HTl2457ZuALn7Pz/H7PMJ9/CXwkojcCkQ2TO+N+fGiTt7EmNOTqlYBnwKfuu/wbwpqIsACVZ1S07cIfqyqt4nIMOBSYLWIDFHVZlNx1BgbKRgTgoj0EpFsv1MDgZ1ACZDonvsKOM9vvSBeRHr6fc11fp+XuW16qOpyVZ2BMwLxL31sTJOzkYIxoSUAz7ilsitxKoROA6YA/xCRAlUd7U4pzRGRWPfrHsSpyAvQVkQ24Ozt7BtN/MFNNoJT+XJ9o0RjTB3ZQrMxDcB/Qbqp+2JMfdj0kTHGGI+NFIwxxnhspGCMMcZjScEYY4zHkoIxxhiPJQVjjDEeSwrGGGM8/w+k9dupUhOKHgAAAABJRU5ErkJggg==\n",
       "text/plain": [
        "<Figure size 432x288 with 1 Axes>"
       ]
@@ -560,7 +553,7 @@
     {
      "data": {
       "text/plain": [
-       "[0.3026159405708313, 0.8824800252914429]"
+       "[0.3063896596431732, 0.8806399703025818]"
       ]
      },
      "execution_count": 15,
@@ -609,1006 +602,1006 @@
     {
      "data": {
       "text/plain": [
-       "[array([0.9291455], dtype=float32),\n",
-       " array([0.96646625], dtype=float32),\n",
-       " array([0.9992888], dtype=float32),\n",
-       " array([0.6787984], dtype=float32),\n",
-       " array([0.8021081], dtype=float32),\n",
-       " array([0.21841279], dtype=float32),\n",
-       " array([0.00925558], dtype=float32),\n",
-       " array([0.03703438], dtype=float32),\n",
-       " array([0.9931043], dtype=float32),\n",
-       " array([0.77527326], dtype=float32),\n",
-       " array([0.57820714], dtype=float32),\n",
-       " array([0.9590181], dtype=float32),\n",
-       " array([0.23954743], dtype=float32),\n",
-       " array([0.99956185], dtype=float32),\n",
-       " array([0.99986804], dtype=float32),\n",
-       " array([0.89252], dtype=float32),\n",
-       " array([0.6362871], dtype=float32),\n",
-       " array([0.00453862], dtype=float32),\n",
-       " array([0.00022487], dtype=float32),\n",
-       " array([0.5878058], dtype=float32),\n",
-       " array([0.79077387], dtype=float32),\n",
-       " array([0.951337], dtype=float32),\n",
-       " array([0.02377308], dtype=float32),\n",
-       " array([0.01233012], dtype=float32),\n",
-       " array([2.1225938e-05], dtype=float32),\n",
-       " array([0.92153585], dtype=float32),\n",
-       " array([0.27290833], dtype=float32),\n",
-       " array([0.9678427], dtype=float32),\n",
-       " array([0.98325783], dtype=float32),\n",
-       " array([0.01623936], dtype=float32),\n",
-       " array([0.9855356], dtype=float32),\n",
-       " array([0.00476582], dtype=float32),\n",
-       " array([0.99998844], dtype=float32),\n",
-       " array([0.37596926], dtype=float32),\n",
-       " array([0.99066436], dtype=float32),\n",
-       " array([0.995574], dtype=float32),\n",
-       " array([0.92657614], dtype=float32),\n",
-       " array([0.00201875], dtype=float32),\n",
-       " array([0.04801914], dtype=float32),\n",
-       " array([0.9741373], dtype=float32),\n",
-       " array([0.11522112], dtype=float32),\n",
-       " array([0.26702783], dtype=float32),\n",
-       " array([0.9866962], dtype=float32),\n",
-       " array([0.9840716], dtype=float32),\n",
-       " array([0.7866682], dtype=float32),\n",
-       " array([0.8413062], dtype=float32),\n",
-       " array([0.5110062], dtype=float32),\n",
-       " array([0.9998983], dtype=float32),\n",
-       " array([0.80300266], dtype=float32),\n",
-       " array([0.914983], dtype=float32),\n",
-       " array([0.9802167], dtype=float32),\n",
-       " array([0.99499327], dtype=float32),\n",
-       " array([0.6802777], dtype=float32),\n",
-       " array([0.94399655], dtype=float32),\n",
-       " array([0.00192062], dtype=float32),\n",
-       " array([0.01228456], dtype=float32),\n",
-       " array([0.0108898], dtype=float32),\n",
-       " array([0.08844584], dtype=float32),\n",
-       " array([0.00843766], dtype=float32),\n",
-       " array([0.975319], dtype=float32),\n",
-       " array([0.9284045], dtype=float32),\n",
-       " array([0.28974563], dtype=float32),\n",
-       " array([0.00184639], dtype=float32),\n",
-       " array([0.7453014], dtype=float32),\n",
-       " array([0.08012109], dtype=float32),\n",
-       " array([0.00030172], dtype=float32),\n",
-       " array([0.9998647], dtype=float32),\n",
-       " array([0.9998778], dtype=float32),\n",
-       " array([0.99604577], dtype=float32),\n",
-       " array([0.9999883], dtype=float32),\n",
-       " array([0.03597537], dtype=float32),\n",
-       " array([0.9160098], dtype=float32),\n",
-       " array([0.1100359], dtype=float32),\n",
-       " array([0.21295448], dtype=float32),\n",
-       " array([0.45663244], dtype=float32),\n",
-       " array([0.9961888], dtype=float32),\n",
-       " array([0.3244737], dtype=float32),\n",
-       " array([0.99997187], dtype=float32),\n",
-       " array([0.12168489], dtype=float32),\n",
-       " array([0.04461992], dtype=float32),\n",
-       " array([0.05755902], dtype=float32),\n",
-       " array([0.16763222], dtype=float32),\n",
-       " array([0.87495023], dtype=float32),\n",
-       " array([0.21862818], dtype=float32),\n",
-       " array([0.01208456], dtype=float32),\n",
-       " array([0.10023356], dtype=float32),\n",
-       " array([0.29649988], dtype=float32),\n",
-       " array([0.99824023], dtype=float32),\n",
-       " array([0.39383507], dtype=float32),\n",
-       " array([0.3296326], dtype=float32),\n",
-       " array([0.68079424], dtype=float32),\n",
-       " array([0.7821922], dtype=float32),\n",
-       " array([0.99843687], dtype=float32),\n",
-       " array([0.998406], dtype=float32),\n",
-       " array([0.09952371], dtype=float32),\n",
-       " array([0.20677786], dtype=float32),\n",
-       " array([0.07357709], dtype=float32),\n",
-       " array([0.99855345], dtype=float32),\n",
-       " array([0.00219191], dtype=float32),\n",
-       " array([0.9438781], dtype=float32),\n",
-       " array([0.11632669], dtype=float32),\n",
-       " array([0.31134847], dtype=float32),\n",
-       " array([0.12542696], dtype=float32),\n",
-       " array([0.9763289], dtype=float32),\n",
-       " array([0.9640528], dtype=float32),\n",
-       " array([0.6855567], dtype=float32),\n",
-       " array([0.9791134], dtype=float32),\n",
-       " array([0.47091678], dtype=float32),\n",
-       " array([0.01279368], dtype=float32),\n",
-       " array([0.33637416], dtype=float32),\n",
-       " array([0.92876065], dtype=float32),\n",
-       " array([0.02191273], dtype=float32),\n",
-       " array([0.02534392], dtype=float32),\n",
-       " array([0.27622753], dtype=float32),\n",
-       " array([0.03425935], dtype=float32),\n",
-       " array([0.11640935], dtype=float32),\n",
-       " array([0.99640983], dtype=float32),\n",
-       " array([0.99434376], dtype=float32),\n",
-       " array([0.02413097], dtype=float32),\n",
-       " array([0.36645678], dtype=float32),\n",
-       " array([0.01748311], dtype=float32),\n",
-       " array([0.18354651], dtype=float32),\n",
-       " array([0.06130786], dtype=float32),\n",
-       " array([0.21773124], dtype=float32),\n",
-       " array([0.95380867], dtype=float32),\n",
-       " array([0.5504796], dtype=float32),\n",
-       " array([0.0219801], dtype=float32),\n",
-       " array([0.01981366], dtype=float32),\n",
-       " array([0.00031499], dtype=float32),\n",
-       " array([0.13779135], dtype=float32),\n",
-       " array([0.9984407], dtype=float32),\n",
-       " array([0.40540016], dtype=float32),\n",
-       " array([0.77313596], dtype=float32),\n",
-       " array([0.01747493], dtype=float32),\n",
-       " array([0.0557996], dtype=float32),\n",
-       " array([0.06081589], dtype=float32),\n",
-       " array([0.04389222], dtype=float32),\n",
-       " array([0.9974957], dtype=float32),\n",
-       " array([0.5977306], dtype=float32),\n",
-       " array([0.02096312], dtype=float32),\n",
-       " array([0.8821718], dtype=float32),\n",
-       " array([0.00831421], dtype=float32),\n",
-       " array([0.85648173], dtype=float32),\n",
-       " array([0.61687416], dtype=float32),\n",
-       " array([0.00555464], dtype=float32),\n",
-       " array([0.9578813], dtype=float32),\n",
-       " array([0.16184929], dtype=float32),\n",
-       " array([0.8980497], dtype=float32),\n",
-       " array([0.99223125], dtype=float32),\n",
-       " array([0.0132063], dtype=float32),\n",
-       " array([0.92253935], dtype=float32),\n",
-       " array([0.06267989], dtype=float32),\n",
-       " array([0.9891216], dtype=float32),\n",
-       " array([0.9419726], dtype=float32),\n",
-       " array([0.00210088], dtype=float32),\n",
-       " array([0.99873346], dtype=float32),\n",
-       " array([0.01406829], dtype=float32),\n",
-       " array([0.08238389], dtype=float32),\n",
-       " array([0.7304331], dtype=float32),\n",
-       " array([0.07515999], dtype=float32),\n",
-       " array([0.00137386], dtype=float32),\n",
-       " array([0.999446], dtype=float32),\n",
-       " array([0.06388262], dtype=float32),\n",
-       " array([0.1658486], dtype=float32),\n",
-       " array([0.99999976], dtype=float32),\n",
-       " array([0.9154651], dtype=float32),\n",
-       " array([0.9564062], dtype=float32),\n",
-       " array([0.9038421], dtype=float32),\n",
-       " array([0.9414884], dtype=float32),\n",
-       " array([0.023891], dtype=float32),\n",
-       " array([0.27174172], dtype=float32),\n",
-       " array([0.9541309], dtype=float32),\n",
-       " array([0.06518997], dtype=float32),\n",
-       " array([0.01072453], dtype=float32),\n",
-       " array([0.99960166], dtype=float32),\n",
-       " array([0.0525161], dtype=float32),\n",
-       " array([0.00074362], dtype=float32),\n",
-       " array([0.84470475], dtype=float32),\n",
-       " array([0.68433523], dtype=float32),\n",
-       " array([0.73134536], dtype=float32),\n",
-       " array([0.02615881], dtype=float32),\n",
-       " array([0.9435008], dtype=float32),\n",
-       " array([0.9924217], dtype=float32),\n",
-       " array([0.81417906], dtype=float32),\n",
-       " array([0.99532396], dtype=float32),\n",
-       " array([0.11175746], dtype=float32),\n",
-       " array([0.01164351], dtype=float32),\n",
-       " array([0.99615234], dtype=float32),\n",
-       " array([0.99891615], dtype=float32),\n",
-       " array([0.85309887], dtype=float32),\n",
-       " array([0.3838549], dtype=float32),\n",
-       " array([0.08728907], dtype=float32),\n",
-       " array([0.99386746], dtype=float32),\n",
-       " array([0.99560165], dtype=float32),\n",
-       " array([0.01668872], dtype=float32),\n",
-       " array([0.865859], dtype=float32),\n",
-       " array([0.00344433], dtype=float32),\n",
-       " array([0.10099232], dtype=float32),\n",
-       " array([0.18755046], dtype=float32),\n",
-       " array([0.17657793], dtype=float32),\n",
-       " array([0.99963737], dtype=float32),\n",
-       " array([0.1608229], dtype=float32),\n",
-       " array([0.99993265], dtype=float32),\n",
-       " array([0.9839023], dtype=float32),\n",
-       " array([0.8809537], dtype=float32),\n",
-       " array([0.16208851], dtype=float32),\n",
-       " array([0.9696871], dtype=float32),\n",
-       " array([0.9999682], dtype=float32),\n",
-       " array([0.06924216], dtype=float32),\n",
-       " array([0.03222934], dtype=float32),\n",
-       " array([0.01602055], dtype=float32),\n",
-       " array([0.16013445], dtype=float32),\n",
-       " array([0.13429435], dtype=float32),\n",
-       " array([0.9997141], dtype=float32),\n",
-       " array([0.5197483], dtype=float32),\n",
-       " array([0.02680757], dtype=float32),\n",
-       " array([0.9984849], dtype=float32),\n",
-       " array([0.00670389], dtype=float32),\n",
-       " array([0.04960306], dtype=float32),\n",
-       " array([0.05909725], dtype=float32),\n",
-       " array([0.07385926], dtype=float32),\n",
-       " array([0.01410465], dtype=float32),\n",
-       " array([0.4758584], dtype=float32),\n",
-       " array([0.9994578], dtype=float32),\n",
-       " array([0.00207514], dtype=float32),\n",
-       " array([0.98792577], dtype=float32),\n",
+       "[array([0.6597656], dtype=float32),\n",
+       " array([0.97529125], dtype=float32),\n",
+       " array([1.39369495e-05], dtype=float32),\n",
+       " array([0.9499197], dtype=float32),\n",
+       " array([0.69558215], dtype=float32),\n",
+       " array([0.98174447], dtype=float32),\n",
+       " array([0.01318819], dtype=float32),\n",
+       " array([0.9626703], dtype=float32),\n",
+       " array([0.98742026], dtype=float32),\n",
+       " array([0.00059057], dtype=float32),\n",
+       " array([0.6133139], dtype=float32),\n",
+       " array([0.978926], dtype=float32),\n",
+       " array([0.99840707], dtype=float32),\n",
+       " array([0.07168697], dtype=float32),\n",
+       " array([0.89191675], dtype=float32),\n",
+       " array([0.48994958], dtype=float32),\n",
+       " array([0.02672931], dtype=float32),\n",
+       " array([0.78033304], dtype=float32),\n",
+       " array([0.07513892], dtype=float32),\n",
+       " array([0.00686305], dtype=float32),\n",
+       " array([0.04119945], dtype=float32),\n",
+       " array([1.49263915e-05], dtype=float32),\n",
+       " array([0.99980336], dtype=float32),\n",
+       " array([0.8471216], dtype=float32),\n",
+       " array([0.00010777], dtype=float32),\n",
+       " array([0.9340031], dtype=float32),\n",
+       " array([0.8214722], dtype=float32),\n",
+       " array([0.9786547], dtype=float32),\n",
+       " array([0.00837058], dtype=float32),\n",
+       " array([0.9238503], dtype=float32),\n",
+       " array([0.00408007], dtype=float32),\n",
+       " array([0.18840362], dtype=float32),\n",
+       " array([0.999974], dtype=float32),\n",
+       " array([0.9948447], dtype=float32),\n",
+       " array([0.8062789], dtype=float32),\n",
+       " array([0.11027395], dtype=float32),\n",
+       " array([0.04690371], dtype=float32),\n",
+       " array([0.07576486], dtype=float32),\n",
+       " array([0.9307181], dtype=float32),\n",
+       " array([0.9578869], dtype=float32),\n",
+       " array([4.3841228e-05], dtype=float32),\n",
+       " array([0.97011423], dtype=float32),\n",
+       " array([0.372595], dtype=float32),\n",
+       " array([0.08670929], dtype=float32),\n",
+       " array([0.9922921], dtype=float32),\n",
+       " array([0.00444584], dtype=float32),\n",
+       " array([0.9995722], dtype=float32),\n",
+       " array([0.90575284], dtype=float32),\n",
+       " array([0.03082987], dtype=float32),\n",
+       " array([8.931183e-06], dtype=float32),\n",
+       " array([0.01119773], dtype=float32),\n",
+       " array([0.9681336], dtype=float32),\n",
+       " array([0.839909], dtype=float32),\n",
+       " array([0.00667274], dtype=float32),\n",
+       " array([0.99168044], dtype=float32),\n",
+       " array([0.99999154], dtype=float32),\n",
+       " array([1.9037469e-05], dtype=float32),\n",
+       " array([0.9974356], dtype=float32),\n",
+       " array([0.00046782], dtype=float32),\n",
+       " array([0.00524331], dtype=float32),\n",
+       " array([0.8870116], dtype=float32),\n",
+       " array([0.9076144], dtype=float32),\n",
+       " array([0.02826679], dtype=float32),\n",
+       " array([0.95415473], dtype=float32),\n",
+       " array([0.3839109], dtype=float32),\n",
+       " array([0.99069595], dtype=float32),\n",
+       " array([0.06462941], dtype=float32),\n",
+       " array([0.99408925], dtype=float32),\n",
+       " array([0.00728586], dtype=float32),\n",
+       " array([0.9963102], dtype=float32),\n",
+       " array([0.88912857], dtype=float32),\n",
+       " array([0.99318165], dtype=float32),\n",
+       " array([0.98711836], dtype=float32),\n",
+       " array([0.9997482], dtype=float32),\n",
+       " array([0.12893666], dtype=float32),\n",
+       " array([2.553328e-05], dtype=float32),\n",
+       " array([0.81136394], dtype=float32),\n",
+       " array([0.6672609], dtype=float32),\n",
+       " array([0.6661795], dtype=float32),\n",
+       " array([0.03229121], dtype=float32),\n",
+       " array([0.56833935], dtype=float32),\n",
+       " array([0.23906621], dtype=float32),\n",
+       " array([0.9886596], dtype=float32),\n",
+       " array([0.9827251], dtype=float32),\n",
+       " array([0.08567941], dtype=float32),\n",
+       " array([0.37140584], dtype=float32),\n",
+       " array([0.00025531], dtype=float32),\n",
+       " array([0.99791545], dtype=float32),\n",
+       " array([0.02411093], dtype=float32),\n",
+       " array([0.9877809], dtype=float32),\n",
+       " array([0.908092], dtype=float32),\n",
+       " array([0.8383248], dtype=float32),\n",
+       " array([0.00739653], dtype=float32),\n",
+       " array([0.00090695], dtype=float32),\n",
+       " array([0.9652902], dtype=float32),\n",
+       " array([0.01431155], dtype=float32),\n",
+       " array([0.93294597], dtype=float32),\n",
+       " array([0.99896336], dtype=float32),\n",
+       " array([0.9984067], dtype=float32),\n",
+       " array([0.93452567], dtype=float32),\n",
+       " array([0.99430794], dtype=float32),\n",
+       " array([0.36339617], dtype=float32),\n",
+       " array([0.8769031], dtype=float32),\n",
+       " array([0.9518878], dtype=float32),\n",
+       " array([0.83151025], dtype=float32),\n",
+       " array([0.9985399], dtype=float32),\n",
+       " array([0.0002125], dtype=float32),\n",
+       " array([0.714252], dtype=float32),\n",
+       " array([0.27901366], dtype=float32),\n",
+       " array([0.8523226], dtype=float32),\n",
+       " array([0.99559104], dtype=float32),\n",
+       " array([0.18001182], dtype=float32),\n",
+       " array([0.9432954], dtype=float32),\n",
+       " array([0.8350808], dtype=float32),\n",
+       " array([0.00853516], dtype=float32),\n",
+       " array([0.15583186], dtype=float32),\n",
+       " array([0.92990994], dtype=float32),\n",
+       " array([0.7541111], dtype=float32),\n",
+       " array([0.69654137], dtype=float32),\n",
+       " array([0.01848821], dtype=float32),\n",
+       " array([0.59170055], dtype=float32),\n",
+       " array([0.9971204], dtype=float32),\n",
+       " array([0.9903796], dtype=float32),\n",
+       " array([0.9991167], dtype=float32),\n",
+       " array([0.9316476], dtype=float32),\n",
+       " array([0.06031401], dtype=float32),\n",
+       " array([0.02550006], dtype=float32),\n",
+       " array([0.9999504], dtype=float32),\n",
+       " array([0.00857145], dtype=float32),\n",
+       " array([0.47920564], dtype=float32),\n",
+       " array([0.9485018], dtype=float32),\n",
+       " array([0.00464081], dtype=float32),\n",
+       " array([0.08251999], dtype=float32),\n",
+       " array([0.98797554], dtype=float32),\n",
+       " array([0.97623616], dtype=float32),\n",
+       " array([0.00270883], dtype=float32),\n",
+       " array([0.41065904], dtype=float32),\n",
+       " array([0.00041126], dtype=float32),\n",
+       " array([0.9735677], dtype=float32),\n",
+       " array([0.01444051], dtype=float32),\n",
+       " array([0.1193343], dtype=float32),\n",
+       " array([0.94883794], dtype=float32),\n",
+       " array([0.81132954], dtype=float32),\n",
+       " array([0.9701367], dtype=float32),\n",
+       " array([0.99988973], dtype=float32),\n",
+       " array([0.95782846], dtype=float32),\n",
+       " array([0.9999559], dtype=float32),\n",
+       " array([0.02463553], dtype=float32),\n",
+       " array([0.80905896], dtype=float32),\n",
+       " array([0.00272602], dtype=float32),\n",
+       " array([0.9443275], dtype=float32),\n",
+       " array([0.6925543], dtype=float32),\n",
+       " array([0.96254104], dtype=float32),\n",
+       " array([0.9993697], dtype=float32),\n",
+       " array([0.90027475], dtype=float32),\n",
+       " array([0.05616611], dtype=float32),\n",
+       " array([1.1050109e-05], dtype=float32),\n",
+       " array([0.8539005], dtype=float32),\n",
+       " array([0.7169908], dtype=float32),\n",
+       " array([0.06052893], dtype=float32),\n",
+       " array([0.03273512], dtype=float32),\n",
+       " array([0.98712534], dtype=float32),\n",
+       " array([0.00043659], dtype=float32),\n",
+       " array([0.9919195], dtype=float32),\n",
+       " array([0.5189989], dtype=float32),\n",
+       " array([0.01810263], dtype=float32),\n",
+       " array([0.00150598], dtype=float32),\n",
+       " array([0.06606124], dtype=float32),\n",
+       " array([0.00081787], dtype=float32),\n",
+       " array([0.01792734], dtype=float32),\n",
+       " array([0.9788325], dtype=float32),\n",
+       " array([0.95970446], dtype=float32),\n",
+       " array([0.09366837], dtype=float32),\n",
+       " array([0.01276378], dtype=float32),\n",
+       " array([0.9993555], dtype=float32),\n",
+       " array([0.027029], dtype=float32),\n",
+       " array([0.56499213], dtype=float32),\n",
+       " array([0.99708503], dtype=float32),\n",
+       " array([0.00154167], dtype=float32),\n",
+       " array([0.2801673], dtype=float32),\n",
+       " array([0.52925706], dtype=float32),\n",
+       " array([0.0010483], dtype=float32),\n",
+       " array([0.9990589], dtype=float32),\n",
+       " array([0.00761955], dtype=float32),\n",
+       " array([0.936439], dtype=float32),\n",
+       " array([0.9875731], dtype=float32),\n",
+       " array([0.05203724], dtype=float32),\n",
+       " array([0.9949458], dtype=float32),\n",
+       " array([0.12733188], dtype=float32),\n",
+       " array([0.01648956], dtype=float32),\n",
+       " array([0.7714576], dtype=float32),\n",
+       " array([0.7118609], dtype=float32),\n",
+       " array([0.09135327], dtype=float32),\n",
+       " array([0.94923663], dtype=float32),\n",
+       " array([0.00418737], dtype=float32),\n",
+       " array([0.39404547], dtype=float32),\n",
+       " array([0.98599905], dtype=float32),\n",
+       " array([0.7954801], dtype=float32),\n",
+       " array([0.42050537], dtype=float32),\n",
+       " array([0.02979656], dtype=float32),\n",
+       " array([0.9153005], dtype=float32),\n",
+       " array([0.7568136], dtype=float32),\n",
+       " array([0.5575319], dtype=float32),\n",
+       " array([0.9995894], dtype=float32),\n",
+       " array([0.9746347], dtype=float32),\n",
+       " array([6.51397e-05], dtype=float32),\n",
+       " array([0.14501932], dtype=float32),\n",
+       " array([0.97661], dtype=float32),\n",
+       " array([0.01651403], dtype=float32),\n",
+       " array([0.73719937], dtype=float32),\n",
+       " array([0.9063153], dtype=float32),\n",
+       " array([0.997982], dtype=float32),\n",
+       " array([0.91056806], dtype=float32),\n",
+       " array([0.00447078], dtype=float32),\n",
+       " array([0.09257668], dtype=float32),\n",
+       " array([0.9366054], dtype=float32),\n",
+       " array([0.9811677], dtype=float32),\n",
+       " array([0.0012391], dtype=float32),\n",
+       " array([0.00391587], dtype=float32),\n",
+       " array([0.00012618], dtype=float32),\n",
+       " array([0.0366583], dtype=float32),\n",
+       " array([0.00550616], dtype=float32),\n",
+       " array([0.890634], dtype=float32),\n",
+       " array([0.00715845], dtype=float32),\n",
+       " array([0.72381204], dtype=float32),\n",
+       " array([0.19576788], dtype=float32),\n",
+       " array([0.99990416], dtype=float32),\n",
+       " array([0.0158124], dtype=float32),\n",
+       " array([0.61522424], dtype=float32),\n",
+       " array([0.9689464], dtype=float32),\n",
+       " array([0.04064468], dtype=float32),\n",
+       " array([0.00022891], dtype=float32),\n",
+       " array([0.02944768], dtype=float32),\n",
+       " array([0.999653], dtype=float32),\n",
+       " array([0.40116826], dtype=float32),\n",
+       " array([0.9913776], dtype=float32),\n",
+       " array([0.0029448], dtype=float32),\n",
+       " array([0.32557806], dtype=float32),\n",
+       " array([0.6863088], dtype=float32),\n",
+       " array([0.00081112], dtype=float32),\n",
+       " array([0.97927356], dtype=float32),\n",
+       " array([0.19653757], dtype=float32),\n",
+       " array([0.9705768], dtype=float32),\n",
+       " array([0.04453946], dtype=float32),\n",
+       " array([0.00284266], dtype=float32),\n",
+       " array([0.03559921], dtype=float32),\n",
+       " array([0.9526187], dtype=float32),\n",
+       " array([0.7230885], dtype=float32),\n",
+       " array([0.8201464], dtype=float32),\n",
+       " array([0.00017875], dtype=float32),\n",
+       " array([0.97747767], dtype=float32),\n",
+       " array([0.5449069], dtype=float32),\n",
+       " array([0.09639208], dtype=float32),\n",
+       " array([0.90544367], dtype=float32),\n",
+       " array([0.167667], dtype=float32),\n",
+       " array([0.9997439], dtype=float32),\n",
+       " array([0.9310318], dtype=float32),\n",
+       " array([0.37656942], dtype=float32),\n",
+       " array([0.0002848], dtype=float32),\n",
+       " array([0.0001366], dtype=float32),\n",
+       " array([0.7440771], dtype=float32),\n",
+       " array([0.88802665], dtype=float32),\n",
+       " array([0.9152749], dtype=float32),\n",
+       " array([0.5734805], dtype=float32),\n",
+       " array([0.9993099], dtype=float32),\n",
+       " array([0.49408263], dtype=float32),\n",
+       " array([0.8506351], dtype=float32),\n",
+       " array([0.00250183], dtype=float32),\n",
+       " array([0.9945287], dtype=float32),\n",
+       " array([0.9684286], dtype=float32),\n",
+       " array([0.90822536], dtype=float32),\n",
+       " array([0.9937883], dtype=float32),\n",
+       " array([0.99190396], dtype=float32),\n",
+       " array([0.01760691], dtype=float32),\n",
+       " array([0.5422416], dtype=float32),\n",
+       " array([0.29439396], dtype=float32),\n",
+       " array([0.99019873], dtype=float32),\n",
+       " array([0.06950508], dtype=float32),\n",
+       " array([0.00818285], dtype=float32),\n",
+       " array([0.9632261], dtype=float32),\n",
+       " array([0.99473333], dtype=float32),\n",
+       " array([0.25060079], dtype=float32),\n",
+       " array([0.00048786], dtype=float32),\n",
+       " array([0.01472425], dtype=float32),\n",
+       " array([0.00318411], dtype=float32),\n",
+       " array([0.00093868], dtype=float32),\n",
+       " array([0.83109117], dtype=float32),\n",
+       " array([0.00123343], dtype=float32),\n",
+       " array([0.9713263], dtype=float32),\n",
+       " array([0.04610278], dtype=float32),\n",
+       " array([0.05665827], dtype=float32),\n",
+       " array([0.5868943], dtype=float32),\n",
+       " array([0.98522806], dtype=float32),\n",
+       " array([0.03351312], dtype=float32),\n",
+       " array([0.02006613], dtype=float32),\n",
+       " array([0.00033519], dtype=float32),\n",
+       " array([0.67317265], dtype=float32),\n",
+       " array([0.30107507], dtype=float32),\n",
+       " array([3.784242e-05], dtype=float32),\n",
+       " array([0.6087148], dtype=float32),\n",
+       " array([0.997804], dtype=float32),\n",
+       " array([0.32963577], dtype=float32),\n",
+       " array([0.03810342], dtype=float32),\n",
+       " array([0.99538136], dtype=float32),\n",
+       " array([0.5548133], dtype=float32),\n",
+       " array([0.9353912], dtype=float32),\n",
+       " array([0.9966528], dtype=float32),\n",
+       " array([0.00378726], dtype=float32),\n",
+       " array([0.43726218], dtype=float32),\n",
+       " array([0.95121735], dtype=float32),\n",
+       " array([0.9728295], dtype=float32),\n",
+       " array([3.875886e-06], dtype=float32),\n",
+       " array([0.98975426], dtype=float32),\n",
+       " array([0.9864806], dtype=float32),\n",
+       " array([0.00165366], dtype=float32),\n",
+       " array([0.1064606], dtype=float32),\n",
+       " array([0.89174306], dtype=float32),\n",
+       " array([0.00587977], dtype=float32),\n",
+       " array([0.98498905], dtype=float32),\n",
+       " array([0.06515972], dtype=float32),\n",
+       " array([0.06025562], dtype=float32),\n",
+       " array([0.0166713], dtype=float32),\n",
+       " array([0.93327284], dtype=float32),\n",
+       " array([0.36270353], dtype=float32),\n",
+       " array([0.99993503], dtype=float32),\n",
+       " array([0.75670844], dtype=float32),\n",
+       " array([0.8717547], dtype=float32),\n",
+       " array([0.3455405], dtype=float32),\n",
+       " array([0.79031855], dtype=float32),\n",
+       " array([0.28538352], dtype=float32),\n",
+       " array([0.9997949], dtype=float32),\n",
+       " array([0.26040974], dtype=float32),\n",
+       " array([0.9983621], dtype=float32),\n",
+       " array([0.04919887], dtype=float32),\n",
+       " array([0.00535334], dtype=float32),\n",
+       " array([0.33617225], dtype=float32),\n",
+       " array([0.07422278], dtype=float32),\n",
+       " array([0.15734425], dtype=float32),\n",
+       " array([0.8681399], dtype=float32),\n",
+       " array([3.36514e-05], dtype=float32),\n",
+       " array([0.220001], dtype=float32),\n",
+       " array([0.03030171], dtype=float32),\n",
+       " array([0.00071725], dtype=float32),\n",
+       " array([0.20411605], dtype=float32),\n",
+       " array([0.38738677], dtype=float32),\n",
+       " array([0.99825364], dtype=float32),\n",
+       " array([0.97874314], dtype=float32),\n",
+       " array([0.9536651], dtype=float32),\n",
+       " array([0.99999595], dtype=float32),\n",
+       " array([0.9274589], dtype=float32),\n",
+       " array([0.67642564], dtype=float32),\n",
+       " array([0.86876076], dtype=float32),\n",
+       " array([0.99380374], dtype=float32),\n",
+       " array([0.00764247], dtype=float32),\n",
+       " array([0.00141049], dtype=float32),\n",
+       " array([0.44760624], dtype=float32),\n",
+       " array([0.7392404], dtype=float32),\n",
+       " array([0.94820905], dtype=float32),\n",
+       " array([0.01543296], dtype=float32),\n",
+       " array([0.0030313], dtype=float32),\n",
+       " array([0.9983657], dtype=float32),\n",
+       " array([0.9877472], dtype=float32),\n",
+       " array([0.14449687], dtype=float32),\n",
+       " array([0.0175909], dtype=float32),\n",
+       " array([0.9933814], dtype=float32),\n",
+       " array([0.1099957], dtype=float32),\n",
+       " array([0.502743], dtype=float32),\n",
+       " array([0.0021092], dtype=float32),\n",
+       " array([0.4014902], dtype=float32),\n",
+       " array([8.531843e-05], dtype=float32),\n",
+       " array([0.0042778], dtype=float32),\n",
+       " array([0.91485137], dtype=float32),\n",
+       " array([0.02211919], dtype=float32),\n",
+       " array([0.00567074], dtype=float32),\n",
+       " array([0.06237838], dtype=float32),\n",
+       " array([0.9416742], dtype=float32),\n",
+       " array([0.0665731], dtype=float32),\n",
+       " array([0.8300122], dtype=float32),\n",
+       " array([0.93574494], dtype=float32),\n",
+       " array([0.99325573], dtype=float32),\n",
+       " array([0.24700274], dtype=float32),\n",
+       " array([0.99896765], dtype=float32),\n",
+       " array([0.93945384], dtype=float32),\n",
+       " array([0.18341716], dtype=float32),\n",
+       " array([0.00710799], dtype=float32),\n",
+       " array([0.00717159], dtype=float32),\n",
+       " array([0.9978796], dtype=float32),\n",
+       " array([0.39169902], dtype=float32),\n",
+       " array([0.9921503], dtype=float32),\n",
+       " array([0.33547845], dtype=float32),\n",
+       " array([0.97284275], dtype=float32),\n",
        " array([0.99999547], dtype=float32),\n",
-       " array([3.411692e-06], dtype=float32),\n",
-       " array([0.7747846], dtype=float32),\n",
-       " array([0.91780055], dtype=float32),\n",
-       " array([0.9927326], dtype=float32),\n",
-       " array([0.2352484], dtype=float32),\n",
-       " array([0.00142602], dtype=float32),\n",
-       " array([0.32317147], dtype=float32),\n",
-       " array([0.00565691], dtype=float32),\n",
-       " array([0.53445995], dtype=float32),\n",
-       " array([0.8927338], dtype=float32),\n",
-       " array([0.13075478], dtype=float32),\n",
-       " array([0.92551], dtype=float32),\n",
-       " array([0.06454863], dtype=float32),\n",
-       " array([0.945902], dtype=float32),\n",
-       " array([0.98765355], dtype=float32),\n",
-       " array([0.00029585], dtype=float32),\n",
-       " array([0.02011549], dtype=float32),\n",
-       " array([0.03295863], dtype=float32),\n",
-       " array([0.00324995], dtype=float32),\n",
-       " array([0.01008756], dtype=float32),\n",
-       " array([0.9823131], dtype=float32),\n",
-       " array([0.27388793], dtype=float32),\n",
-       " array([0.5470663], dtype=float32),\n",
-       " array([0.00781587], dtype=float32),\n",
-       " array([0.005428], dtype=float32),\n",
-       " array([0.9992046], dtype=float32),\n",
-       " array([0.11337327], dtype=float32),\n",
-       " array([0.0242104], dtype=float32),\n",
-       " array([0.06808829], dtype=float32),\n",
-       " array([0.9719501], dtype=float32),\n",
-       " array([0.960842], dtype=float32),\n",
-       " array([0.05452807], dtype=float32),\n",
-       " array([0.9993693], dtype=float32),\n",
-       " array([0.7771042], dtype=float32),\n",
-       " array([0.99957746], dtype=float32),\n",
-       " array([0.05997226], dtype=float32),\n",
-       " array([0.903902], dtype=float32),\n",
-       " array([0.86632144], dtype=float32),\n",
-       " array([0.99996936], dtype=float32),\n",
-       " array([0.69629955], dtype=float32),\n",
-       " array([0.8225713], dtype=float32),\n",
-       " array([0.00246663], dtype=float32),\n",
-       " array([0.0023386], dtype=float32),\n",
-       " array([0.9990294], dtype=float32),\n",
-       " array([0.9977755], dtype=float32),\n",
-       " array([0.9861735], dtype=float32),\n",
-       " array([0.03454849], dtype=float32),\n",
-       " array([0.08657297], dtype=float32),\n",
-       " array([0.9999199], dtype=float32),\n",
-       " array([0.9969818], dtype=float32),\n",
-       " array([0.05295636], dtype=float32),\n",
-       " array([0.99947375], dtype=float32),\n",
-       " array([0.01684208], dtype=float32),\n",
-       " array([0.00564773], dtype=float32),\n",
-       " array([0.00795649], dtype=float32),\n",
-       " array([0.9999298], dtype=float32),\n",
-       " array([0.06059966], dtype=float32),\n",
-       " array([0.9730349], dtype=float32),\n",
-       " array([0.3703475], dtype=float32),\n",
-       " array([0.00036947], dtype=float32),\n",
-       " array([0.3769037], dtype=float32),\n",
-       " array([0.00783887], dtype=float32),\n",
-       " array([0.0283057], dtype=float32),\n",
-       " array([0.04452141], dtype=float32),\n",
-       " array([0.47198933], dtype=float32),\n",
-       " array([0.99648005], dtype=float32),\n",
-       " array([0.9991768], dtype=float32),\n",
-       " array([0.99995506], dtype=float32),\n",
-       " array([0.1129663], dtype=float32),\n",
-       " array([0.01932632], dtype=float32),\n",
-       " array([0.01605185], dtype=float32),\n",
-       " array([0.9423293], dtype=float32),\n",
-       " array([0.06175272], dtype=float32),\n",
-       " array([0.99719644], dtype=float32),\n",
-       " array([0.10867236], dtype=float32),\n",
-       " array([0.02944934], dtype=float32),\n",
-       " array([0.9235288], dtype=float32),\n",
-       " array([0.47749949], dtype=float32),\n",
-       " array([0.88871026], dtype=float32),\n",
-       " array([0.11335868], dtype=float32),\n",
-       " array([0.9990363], dtype=float32),\n",
-       " array([0.03595558], dtype=float32),\n",
-       " array([0.19236687], dtype=float32),\n",
-       " array([0.99891937], dtype=float32),\n",
-       " array([0.28199562], dtype=float32),\n",
-       " array([0.01422782], dtype=float32),\n",
-       " array([0.0095924], dtype=float32),\n",
-       " array([0.05586888], dtype=float32),\n",
-       " array([0.90418166], dtype=float32),\n",
-       " array([0.06067238], dtype=float32),\n",
-       " array([0.12407589], dtype=float32),\n",
-       " array([0.00962292], dtype=float32),\n",
-       " array([0.01531602], dtype=float32),\n",
-       " array([0.00537257], dtype=float32),\n",
-       " array([0.3383139], dtype=float32),\n",
-       " array([0.9790156], dtype=float32),\n",
-       " array([0.99629223], dtype=float32),\n",
-       " array([0.999041], dtype=float32),\n",
-       " array([0.4543444], dtype=float32),\n",
-       " array([0.9975923], dtype=float32),\n",
-       " array([0.994676], dtype=float32),\n",
-       " array([0.9955705], dtype=float32),\n",
-       " array([0.00852212], dtype=float32),\n",
-       " array([0.00965995], dtype=float32),\n",
-       " array([0.04810032], dtype=float32),\n",
-       " array([0.03414589], dtype=float32),\n",
-       " array([0.19549885], dtype=float32),\n",
-       " array([0.04221633], dtype=float32),\n",
-       " array([0.999081], dtype=float32),\n",
-       " array([0.95819485], dtype=float32),\n",
-       " array([0.02422899], dtype=float32),\n",
-       " array([0.00078607], dtype=float32),\n",
-       " array([0.01256537], dtype=float32),\n",
-       " array([0.573112], dtype=float32),\n",
-       " array([0.97446126], dtype=float32),\n",
-       " array([0.01481443], dtype=float32),\n",
-       " array([0.91800165], dtype=float32),\n",
-       " array([0.04669483], dtype=float32),\n",
-       " array([0.12667258], dtype=float32),\n",
-       " array([0.9989146], dtype=float32),\n",
-       " array([0.11938414], dtype=float32),\n",
-       " array([0.8915277], dtype=float32),\n",
-       " array([0.01737923], dtype=float32),\n",
-       " array([0.9999982], dtype=float32),\n",
-       " array([0.96499765], dtype=float32),\n",
-       " array([0.02043628], dtype=float32),\n",
-       " array([0.6315207], dtype=float32),\n",
-       " array([0.9999362], dtype=float32),\n",
-       " array([0.34459296], dtype=float32),\n",
-       " array([0.98566204], dtype=float32),\n",
-       " array([0.97014564], dtype=float32),\n",
-       " array([0.99786866], dtype=float32),\n",
-       " array([0.01015446], dtype=float32),\n",
-       " array([0.8746796], dtype=float32),\n",
-       " array([0.9308818], dtype=float32),\n",
-       " array([0.00047523], dtype=float32),\n",
-       " array([0.99945456], dtype=float32),\n",
-       " array([0.00871587], dtype=float32),\n",
-       " array([0.87762976], dtype=float32),\n",
-       " array([0.00176486], dtype=float32),\n",
-       " array([0.9776403], dtype=float32),\n",
-       " array([0.00964555], dtype=float32),\n",
-       " array([0.38256386], dtype=float32),\n",
-       " array([0.9978903], dtype=float32),\n",
-       " array([0.48501348], dtype=float32),\n",
-       " array([0.9758839], dtype=float32),\n",
-       " array([0.76296157], dtype=float32),\n",
-       " array([0.00493866], dtype=float32),\n",
-       " array([0.05346973], dtype=float32),\n",
-       " array([0.9999949], dtype=float32),\n",
-       " array([0.00160436], dtype=float32),\n",
-       " array([0.00270788], dtype=float32),\n",
-       " array([0.2500689], dtype=float32),\n",
-       " array([0.01582536], dtype=float32),\n",
-       " array([0.8722655], dtype=float32),\n",
-       " array([0.95772576], dtype=float32),\n",
-       " array([0.9999635], dtype=float32),\n",
-       " array([0.86149096], dtype=float32),\n",
-       " array([0.12169719], dtype=float32),\n",
-       " array([0.28068733], dtype=float32),\n",
-       " array([0.00027394], dtype=float32),\n",
-       " array([0.00176786], dtype=float32),\n",
-       " array([0.00266076], dtype=float32),\n",
-       " array([0.00955428], dtype=float32),\n",
-       " array([0.06166862], dtype=float32),\n",
-       " array([0.96516556], dtype=float32),\n",
-       " array([0.99725515], dtype=float32),\n",
-       " array([0.86680585], dtype=float32),\n",
-       " array([0.7473102], dtype=float32),\n",
-       " array([0.09695191], dtype=float32),\n",
-       " array([0.00296136], dtype=float32),\n",
-       " array([0.00260568], dtype=float32),\n",
-       " array([0.9957995], dtype=float32),\n",
-       " array([0.99882144], dtype=float32),\n",
-       " array([0.00024917], dtype=float32),\n",
-       " array([0.9272133], dtype=float32),\n",
-       " array([0.00036355], dtype=float32),\n",
-       " array([0.9843275], dtype=float32),\n",
-       " array([0.02331734], dtype=float32),\n",
-       " array([0.10120519], dtype=float32),\n",
-       " array([0.47101545], dtype=float32),\n",
-       " array([0.02735368], dtype=float32),\n",
-       " array([0.9993734], dtype=float32),\n",
-       " array([0.9975063], dtype=float32),\n",
-       " array([0.9815989], dtype=float32),\n",
-       " array([0.9998317], dtype=float32),\n",
-       " array([0.08344828], dtype=float32),\n",
-       " array([0.9794001], dtype=float32),\n",
-       " array([0.9987134], dtype=float32),\n",
-       " array([0.00018663], dtype=float32),\n",
-       " array([0.8894404], dtype=float32),\n",
-       " array([0.04193362], dtype=float32),\n",
-       " array([0.99497104], dtype=float32),\n",
-       " array([0.04068065], dtype=float32),\n",
-       " array([0.00536961], dtype=float32),\n",
-       " array([0.06312026], dtype=float32),\n",
-       " array([0.05857188], dtype=float32),\n",
-       " array([0.68636453], dtype=float32),\n",
-       " array([0.9688917], dtype=float32),\n",
-       " array([0.19696496], dtype=float32),\n",
-       " array([0.06663571], dtype=float32),\n",
-       " array([0.03183109], dtype=float32),\n",
-       " array([0.96671313], dtype=float32),\n",
-       " array([0.05208089], dtype=float32),\n",
-       " array([0.09314187], dtype=float32),\n",
-       " array([0.9926622], dtype=float32),\n",
-       " array([0.92366344], dtype=float32),\n",
-       " array([0.9675213], dtype=float32),\n",
-       " array([0.01860829], dtype=float32),\n",
-       " array([0.9165048], dtype=float32),\n",
-       " array([0.94261616], dtype=float32),\n",
-       " array([0.0835857], dtype=float32),\n",
-       " array([0.00041597], dtype=float32),\n",
-       " array([0.00907566], dtype=float32),\n",
-       " array([0.94010156], dtype=float32),\n",
-       " array([0.9840152], dtype=float32),\n",
-       " array([0.00041544], dtype=float32),\n",
-       " array([0.8775654], dtype=float32),\n",
-       " array([0.32026634], dtype=float32),\n",
-       " array([0.01421119], dtype=float32),\n",
-       " array([0.0130173], dtype=float32),\n",
-       " array([0.9884545], dtype=float32),\n",
-       " array([0.04571154], dtype=float32),\n",
-       " array([1.6011174e-05], dtype=float32),\n",
-       " array([0.00115296], dtype=float32),\n",
-       " array([0.9790429], dtype=float32),\n",
-       " array([0.64808387], dtype=float32),\n",
-       " array([4.6414338e-05], dtype=float32),\n",
-       " array([0.5914479], dtype=float32),\n",
-       " array([0.00711486], dtype=float32),\n",
-       " array([0.82269937], dtype=float32),\n",
-       " array([0.66748506], dtype=float32),\n",
-       " array([0.9879972], dtype=float32),\n",
-       " array([6.153964e-05], dtype=float32),\n",
-       " array([0.99550027], dtype=float32),\n",
-       " array([0.00570012], dtype=float32),\n",
-       " array([0.46884078], dtype=float32),\n",
-       " array([0.00010328], dtype=float32),\n",
-       " array([0.03197822], dtype=float32),\n",
-       " array([0.9972145], dtype=float32),\n",
-       " array([0.05284659], dtype=float32),\n",
-       " array([0.9037368], dtype=float32),\n",
-       " array([0.04048614], dtype=float32),\n",
+       " array([0.04805868], dtype=float32),\n",
+       " array([0.6807831], dtype=float32),\n",
+       " array([0.38082442], dtype=float32),\n",
+       " array([0.7750744], dtype=float32),\n",
+       " array([0.99722785], dtype=float32),\n",
+       " array([0.77780694], dtype=float32),\n",
+       " array([0.9519044], dtype=float32),\n",
+       " array([0.00215464], dtype=float32),\n",
+       " array([0.29531085], dtype=float32),\n",
+       " array([0.9999316], dtype=float32),\n",
+       " array([0.7214245], dtype=float32),\n",
+       " array([0.8033163], dtype=float32),\n",
+       " array([0.6166736], dtype=float32),\n",
+       " array([0.26327613], dtype=float32),\n",
+       " array([0.21962917], dtype=float32),\n",
+       " array([0.10679483], dtype=float32),\n",
+       " array([0.04216451], dtype=float32),\n",
+       " array([0.00307667], dtype=float32),\n",
+       " array([0.99923015], dtype=float32),\n",
+       " array([0.00597921], dtype=float32),\n",
+       " array([0.99360764], dtype=float32),\n",
+       " array([0.973897], dtype=float32),\n",
+       " array([0.13671698], dtype=float32),\n",
+       " array([0.44968152], dtype=float32),\n",
+       " array([0.07701934], dtype=float32),\n",
+       " array([0.05103498], dtype=float32),\n",
+       " array([0.9994609], dtype=float32),\n",
+       " array([0.07936312], dtype=float32),\n",
+       " array([0.8839954], dtype=float32),\n",
+       " array([1.365624e-06], dtype=float32),\n",
+       " array([0.00480004], dtype=float32),\n",
+       " array([0.12765045], dtype=float32),\n",
+       " array([0.9904794], dtype=float32),\n",
+       " array([0.6438497], dtype=float32),\n",
+       " array([0.8862176], dtype=float32),\n",
+       " array([7.784928e-05], dtype=float32),\n",
+       " array([0.19045115], dtype=float32),\n",
+       " array([0.00067149], dtype=float32),\n",
+       " array([0.9358372], dtype=float32),\n",
+       " array([0.02452566], dtype=float32),\n",
+       " array([0.9958995], dtype=float32),\n",
+       " array([0.550974], dtype=float32),\n",
+       " array([0.30900526], dtype=float32),\n",
+       " array([0.99798125], dtype=float32),\n",
+       " array([0.01287526], dtype=float32),\n",
+       " array([0.01379994], dtype=float32),\n",
+       " array([0.12119947], dtype=float32),\n",
+       " array([0.665414], dtype=float32),\n",
+       " array([0.00102568], dtype=float32),\n",
+       " array([0.2067204], dtype=float32),\n",
+       " array([0.0050051], dtype=float32),\n",
+       " array([0.00433443], dtype=float32),\n",
+       " array([0.39867714], dtype=float32),\n",
+       " array([0.00024582], dtype=float32),\n",
+       " array([0.00571835], dtype=float32),\n",
+       " array([0.00590702], dtype=float32),\n",
+       " array([0.5449246], dtype=float32),\n",
+       " array([0.97699547], dtype=float32),\n",
+       " array([0.00366751], dtype=float32),\n",
+       " array([0.13479914], dtype=float32),\n",
+       " array([0.98704463], dtype=float32),\n",
+       " array([0.0312269], dtype=float32),\n",
+       " array([0.00039572], dtype=float32),\n",
+       " array([0.7193606], dtype=float32),\n",
+       " array([0.07044102], dtype=float32),\n",
+       " array([0.03585317], dtype=float32),\n",
+       " array([0.17524014], dtype=float32),\n",
+       " array([0.14926364], dtype=float32),\n",
+       " array([0.21622558], dtype=float32),\n",
+       " array([0.47393447], dtype=float32),\n",
+       " array([0.8796138], dtype=float32),\n",
+       " array([0.57277304], dtype=float32),\n",
+       " array([0.9692422], dtype=float32),\n",
+       " array([0.9952886], dtype=float32),\n",
+       " array([0.95525163], dtype=float32),\n",
+       " array([0.3414528], dtype=float32),\n",
+       " array([0.6035593], dtype=float32),\n",
+       " array([0.03257844], dtype=float32),\n",
+       " array([0.01301803], dtype=float32),\n",
+       " array([0.47819394], dtype=float32),\n",
+       " array([1.6677832e-08], dtype=float32),\n",
+       " array([0.22340754], dtype=float32),\n",
+       " array([0.9999951], dtype=float32),\n",
+       " array([0.96137166], dtype=float32),\n",
+       " array([0.9981943], dtype=float32),\n",
+       " array([0.05160893], dtype=float32),\n",
+       " array([0.99629396], dtype=float32),\n",
+       " array([0.9625849], dtype=float32),\n",
+       " array([0.0002911], dtype=float32),\n",
+       " array([0.980667], dtype=float32),\n",
+       " array([0.9892765], dtype=float32),\n",
+       " array([0.9987301], dtype=float32),\n",
+       " array([0.9874142], dtype=float32),\n",
+       " array([0.9936329], dtype=float32),\n",
+       " array([0.997771], dtype=float32),\n",
+       " array([0.5043148], dtype=float32),\n",
+       " array([0.8399789], dtype=float32),\n",
+       " array([0.9929483], dtype=float32),\n",
+       " array([0.31873196], dtype=float32),\n",
+       " array([0.0675632], dtype=float32),\n",
+       " array([0.00233161], dtype=float32),\n",
+       " array([0.98852634], dtype=float32),\n",
+       " array([0.9999845], dtype=float32),\n",
+       " array([0.08548676], dtype=float32),\n",
+       " array([0.00016344], dtype=float32),\n",
+       " array([0.06375157], dtype=float32),\n",
+       " array([0.98533106], dtype=float32),\n",
+       " array([0.9875267], dtype=float32),\n",
+       " array([0.02328171], dtype=float32),\n",
+       " array([0.7528208], dtype=float32),\n",
+       " array([0.6718994], dtype=float32),\n",
+       " array([0.7016442], dtype=float32),\n",
+       " array([0.29562166], dtype=float32),\n",
+       " array([0.21487534], dtype=float32),\n",
+       " array([0.05325569], dtype=float32),\n",
+       " array([0.98829865], dtype=float32),\n",
+       " array([0.0206712], dtype=float32),\n",
+       " array([0.39194584], dtype=float32),\n",
+       " array([0.05182257], dtype=float32),\n",
+       " array([0.12892328], dtype=float32),\n",
+       " array([0.98039585], dtype=float32),\n",
+       " array([0.07023581], dtype=float32),\n",
+       " array([0.998417], dtype=float32),\n",
+       " array([0.7812852], dtype=float32),\n",
+       " array([0.09137525], dtype=float32),\n",
+       " array([0.8678507], dtype=float32),\n",
+       " array([0.9933328], dtype=float32),\n",
+       " array([0.3079019], dtype=float32),\n",
+       " array([0.8708483], dtype=float32),\n",
+       " array([0.9929174], dtype=float32),\n",
+       " array([0.85494846], dtype=float32),\n",
+       " array([0.9882675], dtype=float32),\n",
+       " array([0.9930362], dtype=float32),\n",
+       " array([0.44101492], dtype=float32),\n",
+       " array([0.00028029], dtype=float32),\n",
+       " array([0.98733073], dtype=float32),\n",
+       " array([0.94348913], dtype=float32),\n",
+       " array([3.119138e-05], dtype=float32),\n",
+       " array([0.980949], dtype=float32),\n",
+       " array([0.9913406], dtype=float32),\n",
+       " array([0.99495846], dtype=float32),\n",
+       " array([0.9629638], dtype=float32),\n",
+       " array([0.0100573], dtype=float32),\n",
+       " array([0.02189975], dtype=float32),\n",
+       " array([0.99831617], dtype=float32),\n",
+       " array([0.98490876], dtype=float32),\n",
+       " array([0.54414076], dtype=float32),\n",
+       " array([0.06107181], dtype=float32),\n",
+       " array([0.9978096], dtype=float32),\n",
+       " array([0.9745584], dtype=float32),\n",
+       " array([0.00242021], dtype=float32),\n",
+       " array([0.03076136], dtype=float32),\n",
+       " array([0.35039175], dtype=float32),\n",
+       " array([0.83999205], dtype=float32),\n",
+       " array([0.99990547], dtype=float32),\n",
+       " array([0.05263938], dtype=float32),\n",
+       " array([0.8979464], dtype=float32),\n",
+       " array([0.03534276], dtype=float32),\n",
+       " array([0.00471485], dtype=float32),\n",
+       " array([0.99737906], dtype=float32),\n",
+       " array([0.929945], dtype=float32),\n",
+       " array([0.01993787], dtype=float32),\n",
+       " array([0.9856134], dtype=float32),\n",
+       " array([0.7457446], dtype=float32),\n",
+       " array([0.99158585], dtype=float32),\n",
+       " array([0.9860604], dtype=float32),\n",
+       " array([0.03886136], dtype=float32),\n",
+       " array([0.96496195], dtype=float32),\n",
+       " array([0.31795], dtype=float32),\n",
+       " array([0.99946743], dtype=float32),\n",
+       " array([0.996521], dtype=float32),\n",
+       " array([0.03773015], dtype=float32),\n",
+       " array([0.00583928], dtype=float32),\n",
+       " array([0.99041665], dtype=float32),\n",
+       " array([0.9955739], dtype=float32),\n",
+       " array([0.01058325], dtype=float32),\n",
+       " array([0.00011865], dtype=float32),\n",
+       " array([0.8401856], dtype=float32),\n",
+       " array([0.63474256], dtype=float32),\n",
+       " array([0.9829626], dtype=float32),\n",
+       " array([0.01037378], dtype=float32),\n",
+       " array([0.26479724], dtype=float32),\n",
+       " array([0.21121329], dtype=float32),\n",
+       " array([0.9914016], dtype=float32),\n",
+       " array([0.9588108], dtype=float32),\n",
+       " array([0.99756277], dtype=float32),\n",
+       " array([0.30543897], dtype=float32),\n",
+       " array([0.99640626], dtype=float32),\n",
+       " array([0.30586973], dtype=float32),\n",
+       " array([0.9993086], dtype=float32),\n",
+       " array([0.9949649], dtype=float32),\n",
+       " array([0.6421015], dtype=float32),\n",
+       " array([0.14092435], dtype=float32),\n",
+       " array([0.01815344], dtype=float32),\n",
+       " array([0.00090887], dtype=float32),\n",
+       " array([0.9869277], dtype=float32),\n",
+       " array([0.22545609], dtype=float32),\n",
+       " array([0.9994192], dtype=float32),\n",
+       " array([0.10223134], dtype=float32),\n",
+       " array([0.9989011], dtype=float32),\n",
+       " array([0.02059738], dtype=float32),\n",
+       " array([0.88542646], dtype=float32),\n",
+       " array([0.9960936], dtype=float32),\n",
+       " array([0.9262567], dtype=float32),\n",
+       " array([0.9434017], dtype=float32),\n",
+       " array([0.98046255], dtype=float32),\n",
+       " array([0.9889431], dtype=float32),\n",
+       " array([0.7408156], dtype=float32),\n",
+       " array([0.00285646], dtype=float32),\n",
+       " array([0.9890942], dtype=float32),\n",
+       " array([0.7398897], dtype=float32),\n",
+       " array([0.9671184], dtype=float32),\n",
+       " array([0.99998057], dtype=float32),\n",
+       " array([0.9491266], dtype=float32),\n",
+       " array([0.54299086], dtype=float32),\n",
+       " array([0.00412416], dtype=float32),\n",
+       " array([0.6694579], dtype=float32),\n",
+       " array([0.95415497], dtype=float32),\n",
+       " array([0.01549284], dtype=float32),\n",
+       " array([0.0003646], dtype=float32),\n",
+       " array([0.99999607], dtype=float32),\n",
+       " array([0.9999577], dtype=float32),\n",
+       " array([0.00113213], dtype=float32),\n",
+       " array([0.9941749], dtype=float32),\n",
+       " array([0.9958812], dtype=float32),\n",
+       " array([0.99189734], dtype=float32),\n",
+       " array([0.0017188], dtype=float32),\n",
+       " array([0.985795], dtype=float32),\n",
+       " array([0.9998721], dtype=float32),\n",
+       " array([0.99999976], dtype=float32),\n",
+       " array([0.98263794], dtype=float32),\n",
+       " array([0.58947575], dtype=float32),\n",
+       " array([0.00927054], dtype=float32),\n",
+       " array([0.9716789], dtype=float32),\n",
+       " array([0.84313625], dtype=float32),\n",
+       " array([0.96165526], dtype=float32),\n",
+       " array([0.9851811], dtype=float32),\n",
+       " array([0.9854842], dtype=float32),\n",
+       " array([0.00322469], dtype=float32),\n",
+       " array([0.9309462], dtype=float32),\n",
+       " array([0.20306274], dtype=float32),\n",
+       " array([0.04456307], dtype=float32),\n",
+       " array([0.9654337], dtype=float32),\n",
+       " array([0.01055153], dtype=float32),\n",
+       " array([0.99989104], dtype=float32),\n",
+       " array([0.03129936], dtype=float32),\n",
+       " array([0.2108155], dtype=float32),\n",
+       " array([0.98949], dtype=float32),\n",
+       " array([0.99999154], dtype=float32),\n",
+       " array([0.94526803], dtype=float32),\n",
+       " array([0.99107426], dtype=float32),\n",
+       " array([0.99824476], dtype=float32),\n",
+       " array([0.99930096], dtype=float32),\n",
+       " array([0.9494158], dtype=float32),\n",
+       " array([0.59529406], dtype=float32),\n",
+       " array([0.00836287], dtype=float32),\n",
+       " array([0.99950933], dtype=float32),\n",
+       " array([0.8118227], dtype=float32),\n",
+       " array([0.6227854], dtype=float32),\n",
+       " array([0.9727045], dtype=float32),\n",
+       " array([0.99001336], dtype=float32),\n",
+       " array([0.61210626], dtype=float32),\n",
+       " array([0.00018276], dtype=float32),\n",
+       " array([0.09038408], dtype=float32),\n",
+       " array([0.08299794], dtype=float32),\n",
+       " array([0.0105845], dtype=float32),\n",
+       " array([0.16678979], dtype=float32),\n",
+       " array([0.9531919], dtype=float32),\n",
+       " array([0.9998332], dtype=float32),\n",
+       " array([5.249855e-05], dtype=float32),\n",
+       " array([0.00057517], dtype=float32),\n",
+       " array([0.997013], dtype=float32),\n",
+       " array([0.12925929], dtype=float32),\n",
+       " array([0.07413327], dtype=float32),\n",
+       " array([0.98919934], dtype=float32),\n",
+       " array([0.5382614], dtype=float32),\n",
+       " array([0.9996692], dtype=float32),\n",
+       " array([0.8613375], dtype=float32),\n",
+       " array([0.71423596], dtype=float32),\n",
+       " array([0.09667405], dtype=float32),\n",
+       " array([0.9979893], dtype=float32),\n",
+       " array([0.00794561], dtype=float32),\n",
+       " array([0.00175152], dtype=float32),\n",
+       " array([0.21769904], dtype=float32),\n",
+       " array([0.94123036], dtype=float32),\n",
+       " array([0.96663105], dtype=float32),\n",
+       " array([0.01070287], dtype=float32),\n",
+       " array([0.07400733], dtype=float32),\n",
+       " array([0.012168], dtype=float32),\n",
+       " array([0.01236583], dtype=float32),\n",
        " array([0.998744], dtype=float32),\n",
-       " array([0.99049884], dtype=float32),\n",
-       " array([0.01359443], dtype=float32),\n",
-       " array([0.9997179], dtype=float32),\n",
-       " array([0.99999964], dtype=float32),\n",
-       " array([0.98721075], dtype=float32),\n",
-       " array([0.00063402], dtype=float32),\n",
-       " array([0.820883], dtype=float32),\n",
-       " array([0.4547376], dtype=float32),\n",
-       " array([0.891108], dtype=float32),\n",
-       " array([0.16223074], dtype=float32),\n",
-       " array([0.9726654], dtype=float32),\n",
-       " array([0.9003827], dtype=float32),\n",
-       " array([0.99944574], dtype=float32),\n",
-       " array([0.7704998], dtype=float32),\n",
-       " array([0.95534146], dtype=float32),\n",
-       " array([0.0062368], dtype=float32),\n",
-       " array([0.9787254], dtype=float32),\n",
-       " array([0.00126028], dtype=float32),\n",
-       " array([0.7004171], dtype=float32),\n",
-       " array([0.09580212], dtype=float32),\n",
-       " array([0.97376186], dtype=float32),\n",
-       " array([0.9920665], dtype=float32),\n",
-       " array([0.12573221], dtype=float32),\n",
-       " array([0.96389884], dtype=float32),\n",
-       " array([0.9980578], dtype=float32),\n",
-       " array([0.00578512], dtype=float32),\n",
-       " array([0.01206519], dtype=float32),\n",
-       " array([0.9992673], dtype=float32),\n",
-       " array([0.07898365], dtype=float32),\n",
-       " array([0.00214792], dtype=float32),\n",
-       " array([0.05026222], dtype=float32),\n",
-       " array([0.99995553], dtype=float32),\n",
-       " array([0.99840695], dtype=float32),\n",
-       " array([0.99762005], dtype=float32),\n",
-       " array([1.], dtype=float32),\n",
-       " array([0.7520437], dtype=float32),\n",
-       " array([0.03680907], dtype=float32),\n",
-       " array([0.5764651], dtype=float32),\n",
-       " array([0.99977905], dtype=float32),\n",
-       " array([0.20490777], dtype=float32),\n",
-       " array([0.00173326], dtype=float32),\n",
-       " array([0.9939528], dtype=float32),\n",
-       " array([1.9042971e-05], dtype=float32),\n",
-       " array([0.95325047], dtype=float32),\n",
-       " array([0.9958734], dtype=float32),\n",
-       " array([0.03266709], dtype=float32),\n",
-       " array([0.02785043], dtype=float32),\n",
-       " array([0.05891791], dtype=float32),\n",
-       " array([0.985227], dtype=float32),\n",
-       " array([0.00210726], dtype=float32),\n",
-       " array([0.9943926], dtype=float32),\n",
-       " array([0.02131884], dtype=float32),\n",
-       " array([0.99827754], dtype=float32),\n",
-       " array([0.8846837], dtype=float32),\n",
-       " array([0.99997437], dtype=float32),\n",
-       " array([0.9946067], dtype=float32),\n",
-       " array([0.99978703], dtype=float32),\n",
-       " array([0.00300169], dtype=float32),\n",
-       " array([0.00031111], dtype=float32),\n",
-       " array([0.9819504], dtype=float32),\n",
-       " array([0.00375891], dtype=float32),\n",
-       " array([0.63086605], dtype=float32),\n",
-       " array([0.83123654], dtype=float32),\n",
-       " array([0.63774806], dtype=float32),\n",
-       " array([0.6908987], dtype=float32),\n",
-       " array([0.15456767], dtype=float32),\n",
-       " array([0.4055819], dtype=float32),\n",
-       " array([0.0910763], dtype=float32),\n",
-       " array([0.24731727], dtype=float32),\n",
-       " array([0.994842], dtype=float32),\n",
-       " array([0.38033506], dtype=float32),\n",
-       " array([0.98958546], dtype=float32),\n",
-       " array([0.9998734], dtype=float32),\n",
-       " array([0.99897206], dtype=float32),\n",
-       " array([0.8665204], dtype=float32),\n",
-       " array([0.00639246], dtype=float32),\n",
-       " array([0.9556338], dtype=float32),\n",
-       " array([0.9666423], dtype=float32),\n",
-       " array([0.9984849], dtype=float32),\n",
-       " array([0.02892273], dtype=float32),\n",
-       " array([0.995031], dtype=float32),\n",
-       " array([0.72732645], dtype=float32),\n",
-       " array([0.998869], dtype=float32),\n",
-       " array([0.01101623], dtype=float32),\n",
-       " array([0.9236663], dtype=float32),\n",
-       " array([0.0053416], dtype=float32),\n",
-       " array([0.9376339], dtype=float32),\n",
-       " array([0.9097744], dtype=float32),\n",
-       " array([0.32959577], dtype=float32),\n",
-       " array([0.69777864], dtype=float32),\n",
-       " array([0.02821315], dtype=float32),\n",
-       " array([0.8764768], dtype=float32),\n",
-       " array([0.01698065], dtype=float32),\n",
-       " array([0.05337453], dtype=float32),\n",
-       " array([0.00699902], dtype=float32),\n",
-       " array([0.01098163], dtype=float32),\n",
-       " array([0.02664173], dtype=float32),\n",
-       " array([0.19115667], dtype=float32),\n",
-       " array([0.01039254], dtype=float32),\n",
-       " array([0.7853336], dtype=float32),\n",
-       " array([0.13310696], dtype=float32),\n",
-       " array([0.12221986], dtype=float32),\n",
-       " array([0.99144626], dtype=float32),\n",
-       " array([0.12488245], dtype=float32),\n",
-       " array([0.10422938], dtype=float32),\n",
-       " array([0.03960704], dtype=float32),\n",
-       " array([0.96108264], dtype=float32),\n",
-       " array([0.00321886], dtype=float32),\n",
-       " array([0.9612626], dtype=float32),\n",
-       " array([0.77753425], dtype=float32),\n",
-       " array([0.992634], dtype=float32),\n",
-       " array([0.08653396], dtype=float32),\n",
-       " array([0.573064], dtype=float32),\n",
-       " array([0.994193], dtype=float32),\n",
-       " array([0.9994568], dtype=float32),\n",
-       " array([0.00164113], dtype=float32),\n",
-       " array([0.02974954], dtype=float32),\n",
-       " array([0.00094306], dtype=float32),\n",
-       " array([0.01469964], dtype=float32),\n",
-       " array([0.01007712], dtype=float32),\n",
-       " array([0.5073839], dtype=float32),\n",
-       " array([0.00891581], dtype=float32),\n",
-       " array([0.01340619], dtype=float32),\n",
-       " array([0.19153547], dtype=float32),\n",
-       " array([0.00785635], dtype=float32),\n",
-       " array([0.00529725], dtype=float32),\n",
-       " array([0.9981645], dtype=float32),\n",
-       " array([0.89561176], dtype=float32),\n",
-       " array([0.44029003], dtype=float32),\n",
-       " array([0.9998586], dtype=float32),\n",
-       " array([0.09905386], dtype=float32),\n",
-       " array([0.9964311], dtype=float32),\n",
-       " array([0.9990119], dtype=float32),\n",
-       " array([0.85216373], dtype=float32),\n",
-       " array([0.9999999], dtype=float32),\n",
-       " array([0.9584791], dtype=float32),\n",
-       " array([0.00405316], dtype=float32),\n",
-       " array([0.9539604], dtype=float32),\n",
-       " array([0.9477786], dtype=float32),\n",
-       " array([0.99440527], dtype=float32),\n",
-       " array([0.9769205], dtype=float32),\n",
-       " array([0.89980936], dtype=float32),\n",
-       " array([0.00249994], dtype=float32),\n",
-       " array([0.8010222], dtype=float32),\n",
-       " array([0.20035101], dtype=float32),\n",
-       " array([0.00624598], dtype=float32),\n",
-       " array([0.80348605], dtype=float32),\n",
-       " array([0.19786465], dtype=float32),\n",
-       " array([0.0849141], dtype=float32),\n",
-       " array([0.9986082], dtype=float32),\n",
-       " array([0.13313013], dtype=float32),\n",
-       " array([0.16162278], dtype=float32),\n",
-       " array([0.993683], dtype=float32),\n",
-       " array([0.55489], dtype=float32),\n",
-       " array([0.07838932], dtype=float32),\n",
-       " array([0.008188], dtype=float32),\n",
-       " array([0.00667501], dtype=float32),\n",
-       " array([0.16437183], dtype=float32),\n",
-       " array([0.96623826], dtype=float32),\n",
-       " array([0.8830661], dtype=float32),\n",
-       " array([0.9948244], dtype=float32),\n",
-       " array([0.98182225], dtype=float32),\n",
-       " array([0.98158526], dtype=float32),\n",
-       " array([0.03434094], dtype=float32),\n",
-       " array([0.3056716], dtype=float32),\n",
-       " array([0.98550564], dtype=float32),\n",
-       " array([0.03481789], dtype=float32),\n",
-       " array([0.99663454], dtype=float32),\n",
-       " array([0.9985311], dtype=float32),\n",
-       " array([0.9939167], dtype=float32),\n",
-       " array([0.10510859], dtype=float32),\n",
-       " array([0.00508491], dtype=float32),\n",
-       " array([0.00165993], dtype=float32),\n",
-       " array([0.8245149], dtype=float32),\n",
-       " array([0.9556251], dtype=float32),\n",
-       " array([0.9887638], dtype=float32),\n",
-       " array([0.17581154], dtype=float32),\n",
-       " array([0.00011383], dtype=float32),\n",
-       " array([0.00106234], dtype=float32),\n",
-       " array([0.01681961], dtype=float32),\n",
-       " array([0.00525573], dtype=float32),\n",
-       " array([0.9957818], dtype=float32),\n",
-       " array([0.8603696], dtype=float32),\n",
-       " array([0.00199909], dtype=float32),\n",
-       " array([0.04100141], dtype=float32),\n",
-       " array([0.00017335], dtype=float32),\n",
-       " array([0.9992545], dtype=float32),\n",
-       " array([0.9659639], dtype=float32),\n",
-       " array([0.8995427], dtype=float32),\n",
-       " array([0.12103864], dtype=float32),\n",
-       " array([0.710389], dtype=float32),\n",
-       " array([0.69011146], dtype=float32),\n",
-       " array([0.05472863], dtype=float32),\n",
-       " array([0.01922707], dtype=float32),\n",
-       " array([0.8451995], dtype=float32),\n",
-       " array([0.99947244], dtype=float32),\n",
-       " array([0.09135215], dtype=float32),\n",
-       " array([0.00159073], dtype=float32),\n",
-       " array([0.05302845], dtype=float32),\n",
-       " array([0.98199064], dtype=float32),\n",
-       " array([0.50679266], dtype=float32),\n",
-       " array([0.17369047], dtype=float32),\n",
-       " array([0.9998796], dtype=float32),\n",
-       " array([0.6899049], dtype=float32),\n",
-       " array([0.00706529], dtype=float32),\n",
-       " array([0.00957006], dtype=float32),\n",
-       " array([0.78653455], dtype=float32),\n",
-       " array([0.00385365], dtype=float32),\n",
-       " array([0.3887317], dtype=float32),\n",
-       " array([0.03332283], dtype=float32),\n",
-       " array([0.04271935], dtype=float32),\n",
-       " array([0.00691515], dtype=float32),\n",
-       " array([0.51850504], dtype=float32),\n",
-       " array([0.00992748], dtype=float32),\n",
-       " array([0.29605916], dtype=float32),\n",
-       " array([0.00028048], dtype=float32),\n",
-       " array([0.9928925], dtype=float32),\n",
-       " array([0.00124131], dtype=float32),\n",
-       " array([0.9764488], dtype=float32),\n",
-       " array([0.7932969], dtype=float32),\n",
-       " array([0.97071785], dtype=float32),\n",
-       " array([0.98584795], dtype=float32),\n",
-       " array([0.07081781], dtype=float32),\n",
-       " array([0.04931527], dtype=float32),\n",
-       " array([0.15695621], dtype=float32),\n",
-       " array([0.9994659], dtype=float32),\n",
-       " array([0.5096887], dtype=float32),\n",
-       " array([0.9917842], dtype=float32),\n",
-       " array([0.9981895], dtype=float32),\n",
-       " array([0.12094765], dtype=float32),\n",
-       " array([0.11894753], dtype=float32),\n",
-       " array([0.90818393], dtype=float32),\n",
-       " array([0.98540974], dtype=float32),\n",
-       " array([0.94457304], dtype=float32),\n",
-       " array([0.9998313], dtype=float32),\n",
-       " array([0.9907136], dtype=float32),\n",
-       " array([0.72061336], dtype=float32),\n",
-       " array([0.99645764], dtype=float32),\n",
-       " array([0.8892745], dtype=float32),\n",
-       " array([0.999595], dtype=float32),\n",
-       " array([0.4836757], dtype=float32),\n",
-       " array([0.97150517], dtype=float32),\n",
-       " array([0.95611787], dtype=float32),\n",
-       " array([0.01308193], dtype=float32),\n",
-       " array([0.00030093], dtype=float32),\n",
-       " array([0.99999774], dtype=float32),\n",
-       " array([0.00015023], dtype=float32),\n",
-       " array([0.00129508], dtype=float32),\n",
-       " array([0.02929328], dtype=float32),\n",
-       " array([0.15967111], dtype=float32),\n",
-       " array([0.04437197], dtype=float32),\n",
-       " array([0.00077572], dtype=float32),\n",
-       " array([0.00117108], dtype=float32),\n",
-       " array([0.02031642], dtype=float32),\n",
-       " array([0.78561157], dtype=float32),\n",
-       " array([0.6640868], dtype=float32),\n",
-       " array([0.96112865], dtype=float32),\n",
-       " array([0.9977581], dtype=float32),\n",
-       " array([0.24247183], dtype=float32),\n",
-       " array([0.3876404], dtype=float32),\n",
-       " array([0.00950658], dtype=float32),\n",
-       " array([0.12216215], dtype=float32),\n",
-       " array([0.00086611], dtype=float32),\n",
-       " array([0.99396956], dtype=float32),\n",
-       " array([0.77168643], dtype=float32),\n",
-       " array([0.00386494], dtype=float32),\n",
-       " array([0.9824516], dtype=float32),\n",
-       " array([0.9933523], dtype=float32),\n",
-       " array([0.95625204], dtype=float32),\n",
-       " array([0.00056513], dtype=float32),\n",
-       " array([0.949406], dtype=float32),\n",
-       " array([0.91392314], dtype=float32),\n",
-       " array([0.998437], dtype=float32),\n",
-       " array([0.00321832], dtype=float32),\n",
-       " array([0.3081614], dtype=float32),\n",
-       " array([0.18779686], dtype=float32),\n",
-       " array([0.86671174], dtype=float32),\n",
-       " array([0.99736756], dtype=float32),\n",
-       " array([0.37101072], dtype=float32),\n",
-       " array([0.97278196], dtype=float32),\n",
-       " array([0.03783982], dtype=float32),\n",
-       " array([0.98899263], dtype=float32),\n",
-       " array([0.99997747], dtype=float32),\n",
-       " array([0.33761773], dtype=float32),\n",
-       " array([0.9922051], dtype=float32),\n",
-       " array([0.9986929], dtype=float32),\n",
-       " array([0.9734451], dtype=float32),\n",
-       " array([0.00104312], dtype=float32),\n",
-       " array([0.00905259], dtype=float32),\n",
-       " array([0.9999858], dtype=float32),\n",
-       " array([0.2685942], dtype=float32),\n",
-       " array([8.498155e-07], dtype=float32),\n",
-       " array([0.00194448], dtype=float32),\n",
-       " array([0.9610404], dtype=float32),\n",
-       " array([0.06272461], dtype=float32),\n",
-       " array([0.9734326], dtype=float32),\n",
-       " array([0.9998591], dtype=float32),\n",
-       " array([0.02384088], dtype=float32),\n",
-       " array([0.00458063], dtype=float32),\n",
-       " array([0.8619814], dtype=float32),\n",
-       " array([0.3280481], dtype=float32),\n",
-       " array([0.58994853], dtype=float32),\n",
-       " array([0.00738818], dtype=float32),\n",
-       " array([0.9968213], dtype=float32),\n",
-       " array([0.94588715], dtype=float32),\n",
-       " array([0.89741385], dtype=float32),\n",
-       " array([0.0001792], dtype=float32),\n",
-       " array([7.942238e-05], dtype=float32),\n",
-       " array([0.2487981], dtype=float32),\n",
-       " array([0.99818295], dtype=float32),\n",
-       " array([0.06495794], dtype=float32),\n",
-       " array([0.61300725], dtype=float32),\n",
-       " array([0.00142293], dtype=float32),\n",
-       " array([0.7782267], dtype=float32),\n",
-       " array([0.70798534], dtype=float32),\n",
-       " array([0.15175731], dtype=float32),\n",
-       " array([0.99284136], dtype=float32),\n",
-       " array([0.9841339], dtype=float32),\n",
-       " array([0.00554728], dtype=float32),\n",
-       " array([0.0500682], dtype=float32),\n",
-       " array([1.7751601e-06], dtype=float32),\n",
-       " array([0.12731266], dtype=float32),\n",
-       " array([0.01886535], dtype=float32),\n",
-       " array([0.9990376], dtype=float32),\n",
-       " array([0.27495182], dtype=float32),\n",
-       " array([0.90534323], dtype=float32),\n",
-       " array([0.8381721], dtype=float32),\n",
-       " array([0.12258686], dtype=float32),\n",
-       " array([0.23695664], dtype=float32),\n",
-       " array([0.04559099], dtype=float32),\n",
-       " array([0.798738], dtype=float32),\n",
-       " array([0.9249577], dtype=float32),\n",
-       " array([0.5790399], dtype=float32),\n",
-       " array([0.7356898], dtype=float32),\n",
-       " array([0.9420959], dtype=float32),\n",
-       " array([0.80315626], dtype=float32),\n",
-       " array([0.907965], dtype=float32),\n",
-       " array([0.18890426], dtype=float32),\n",
-       " array([0.04044292], dtype=float32),\n",
-       " array([0.00435959], dtype=float32),\n",
-       " array([0.01255109], dtype=float32),\n",
-       " array([0.973041], dtype=float32),\n",
-       " array([0.89595586], dtype=float32),\n",
-       " array([0.15041849], dtype=float32),\n",
-       " array([0.7386434], dtype=float32),\n",
-       " array([0.01395628], dtype=float32),\n",
-       " array([0.00037464], dtype=float32),\n",
-       " array([0.30354175], dtype=float32),\n",
-       " array([0.92193896], dtype=float32),\n",
-       " array([0.95892274], dtype=float32),\n",
-       " array([1.066259e-06], dtype=float32),\n",
-       " array([0.96353555], dtype=float32),\n",
-       " array([0.14788437], dtype=float32),\n",
-       " array([0.9997639], dtype=float32),\n",
-       " array([0.01777537], dtype=float32),\n",
-       " array([0.9861092], dtype=float32),\n",
-       " array([0.13082978], dtype=float32),\n",
-       " array([0.0002504], dtype=float32),\n",
-       " array([0.8804745], dtype=float32),\n",
-       " array([0.9967051], dtype=float32),\n",
-       " array([0.56104803], dtype=float32),\n",
-       " array([0.36787862], dtype=float32),\n",
-       " array([0.8360154], dtype=float32),\n",
-       " array([0.9998766], dtype=float32),\n",
-       " array([0.00568995], dtype=float32),\n",
-       " array([0.00194393], dtype=float32),\n",
-       " array([0.00631262], dtype=float32),\n",
-       " array([0.03533027], dtype=float32),\n",
-       " array([0.9103368], dtype=float32),\n",
-       " array([0.9982439], dtype=float32),\n",
-       " array([0.97841996], dtype=float32),\n",
-       " array([0.00286406], dtype=float32),\n",
-       " array([0.0708506], dtype=float32),\n",
-       " array([0.9432028], dtype=float32),\n",
-       " array([0.9654381], dtype=float32),\n",
-       " array([0.05079986], dtype=float32),\n",
-       " array([0.8743878], dtype=float32),\n",
-       " array([0.00240675], dtype=float32),\n",
-       " array([0.98993146], dtype=float32),\n",
-       " array([0.07532773], dtype=float32),\n",
-       " array([0.22899462], dtype=float32),\n",
-       " array([0.00091621], dtype=float32),\n",
-       " array([0.9989504], dtype=float32),\n",
-       " array([0.39317238], dtype=float32),\n",
-       " array([0.3326581], dtype=float32),\n",
-       " array([0.01213577], dtype=float32),\n",
-       " array([0.99774724], dtype=float32),\n",
-       " array([0.9886003], dtype=float32),\n",
-       " array([0.79621345], dtype=float32),\n",
-       " array([0.79079646], dtype=float32),\n",
-       " array([0.93861336], dtype=float32),\n",
-       " array([0.07021908], dtype=float32),\n",
-       " array([0.7411332], dtype=float32),\n",
-       " array([0.969042], dtype=float32),\n",
-       " array([0.9099184], dtype=float32),\n",
-       " array([0.02733893], dtype=float32),\n",
-       " array([0.9999924], dtype=float32),\n",
-       " array([0.9897418], dtype=float32),\n",
-       " array([0.03869773], dtype=float32),\n",
-       " array([0.97638786], dtype=float32),\n",
-       " array([0.08542448], dtype=float32),\n",
-       " array([0.05407662], dtype=float32),\n",
-       " array([0.9999993], dtype=float32),\n",
-       " array([0.14986295], dtype=float32),\n",
-       " array([0.999286], dtype=float32),\n",
-       " array([0.24805169], dtype=float32),\n",
-       " array([0.01673634], dtype=float32),\n",
-       " array([0.01463007], dtype=float32),\n",
-       " array([0.3670616], dtype=float32),\n",
-       " array([0.9926224], dtype=float32),\n",
-       " array([0.6253009], dtype=float32),\n",
-       " array([0.03401245], dtype=float32),\n",
-       " array([0.00030278], dtype=float32),\n",
-       " array([0.96080303], dtype=float32),\n",
-       " array([0.04573576], dtype=float32),\n",
-       " array([0.04926103], dtype=float32),\n",
-       " array([0.5770354], dtype=float32),\n",
-       " array([0.02184597], dtype=float32),\n",
-       " array([0.9933947], dtype=float32),\n",
-       " array([0.00422782], dtype=float32),\n",
-       " array([0.7942074], dtype=float32),\n",
-       " array([0.14047284], dtype=float32),\n",
-       " array([0.90892816], dtype=float32),\n",
-       " array([0.79335517], dtype=float32),\n",
-       " array([0.02081179], dtype=float32),\n",
-       " array([0.03224256], dtype=float32),\n",
-       " array([0.00269347], dtype=float32),\n",
-       " array([0.7325373], dtype=float32),\n",
-       " array([0.86657375], dtype=float32),\n",
-       " array([0.9994041], dtype=float32),\n",
-       " array([0.99819297], dtype=float32),\n",
-       " array([0.306308], dtype=float32),\n",
-       " array([0.9358532], dtype=float32),\n",
-       " array([0.00968082], dtype=float32),\n",
-       " array([0.22723815], dtype=float32),\n",
-       " array([0.88686043], dtype=float32),\n",
-       " array([0.00376564], dtype=float32),\n",
-       " array([0.9558993], dtype=float32),\n",
-       " array([0.03709094], dtype=float32),\n",
-       " array([0.9284992], dtype=float32),\n",
-       " array([0.01156035], dtype=float32),\n",
-       " array([0.6904194], dtype=float32),\n",
-       " array([0.7789368], dtype=float32),\n",
-       " array([0.06749155], dtype=float32),\n",
-       " array([0.83822256], dtype=float32),\n",
-       " array([0.00499537], dtype=float32),\n",
-       " array([0.96375054], dtype=float32),\n",
-       " array([0.99763095], dtype=float32),\n",
-       " array([0.00083689], dtype=float32),\n",
-       " array([0.1384925], dtype=float32),\n",
-       " array([0.99911016], dtype=float32),\n",
-       " array([0.18213369], dtype=float32),\n",
-       " array([0.01104294], dtype=float32),\n",
-       " array([0.9997731], dtype=float32),\n",
-       " array([0.00157826], dtype=float32),\n",
-       " array([0.45021382], dtype=float32),\n",
-       " array([0.70889956], dtype=float32),\n",
-       " array([0.99980146], dtype=float32),\n",
-       " array([0.14717786], dtype=float32),\n",
-       " array([0.9981312], dtype=float32),\n",
-       " array([0.99910754], dtype=float32),\n",
-       " array([0.00473733], dtype=float32),\n",
-       " array([0.00330126], dtype=float32),\n",
-       " array([0.17611578], dtype=float32),\n",
-       " array([0.69635725], dtype=float32),\n",
-       " array([0.39411786], dtype=float32),\n",
-       " array([0.26741236], dtype=float32),\n",
-       " array([0.56975543], dtype=float32),\n",
-       " array([0.06516983], dtype=float32),\n",
-       " array([0.70290774], dtype=float32),\n",
-       " array([0.1079508], dtype=float32),\n",
-       " array([0.9905323], dtype=float32),\n",
-       " array([0.07408904], dtype=float32),\n",
-       " array([0.99945086], dtype=float32),\n",
-       " array([0.08830733], dtype=float32),\n",
-       " array([0.47597456], dtype=float32),\n",
-       " array([0.08325432], dtype=float32),\n",
-       " array([0.9963791], dtype=float32),\n",
-       " array([0.99327046], dtype=float32),\n",
-       " array([0.987528], dtype=float32),\n",
-       " array([0.2695155], dtype=float32),\n",
-       " array([0.01687575], dtype=float32),\n",
-       " array([0.0887219], dtype=float32),\n",
-       " array([0.00404755], dtype=float32),\n",
-       " array([0.8474386], dtype=float32),\n",
-       " array([0.02510497], dtype=float32),\n",
-       " array([0.00147101], dtype=float32),\n",
-       " array([0.00696711], dtype=float32),\n",
-       " array([0.01805459], dtype=float32),\n",
-       " array([0.37892923], dtype=float32),\n",
-       " array([0.32513785], dtype=float32),\n",
-       " array([0.00713208], dtype=float32),\n",
-       " array([0.05214171], dtype=float32),\n",
-       " array([0.9894679], dtype=float32),\n",
-       " array([0.74764496], dtype=float32),\n",
-       " array([0.0094498], dtype=float32),\n",
-       " array([0.05753988], dtype=float32),\n",
-       " array([0.9815139], dtype=float32),\n",
-       " array([0.994449], dtype=float32),\n",
-       " array([0.0733721], dtype=float32),\n",
-       " array([0.03602724], dtype=float32),\n",
+       " array([0.00689602], dtype=float32),\n",
+       " array([0.9943845], dtype=float32),\n",
+       " array([0.81676173], dtype=float32),\n",
+       " array([0.9999511], dtype=float32),\n",
+       " array([0.00488768], dtype=float32),\n",
+       " array([0.33030394], dtype=float32),\n",
+       " array([1.3143154e-05], dtype=float32),\n",
+       " array([0.97804296], dtype=float32),\n",
+       " array([0.97198254], dtype=float32),\n",
+       " array([0.98943126], dtype=float32),\n",
+       " array([0.9713336], dtype=float32),\n",
+       " array([0.44176972], dtype=float32),\n",
+       " array([0.9996177], dtype=float32),\n",
+       " array([0.97341985], dtype=float32),\n",
+       " array([0.9889993], dtype=float32),\n",
+       " array([0.9999981], dtype=float32),\n",
+       " array([0.9906634], dtype=float32),\n",
+       " array([0.7313937], dtype=float32),\n",
+       " array([4.6735212e-07], dtype=float32),\n",
+       " array([0.9986381], dtype=float32),\n",
+       " array([0.5398831], dtype=float32),\n",
+       " array([0.5327877], dtype=float32),\n",
+       " array([0.99454075], dtype=float32),\n",
+       " array([0.7781688], dtype=float32),\n",
+       " array([0.00171901], dtype=float32),\n",
+       " array([0.9790917], dtype=float32),\n",
+       " array([1.7694707e-05], dtype=float32),\n",
+       " array([0.22174618], dtype=float32),\n",
+       " array([0.00032948], dtype=float32),\n",
+       " array([0.98750776], dtype=float32),\n",
+       " array([0.9930167], dtype=float32),\n",
+       " array([0.7805735], dtype=float32),\n",
+       " array([0.874757], dtype=float32),\n",
+       " array([0.10298155], dtype=float32),\n",
+       " array([0.00014828], dtype=float32),\n",
+       " array([0.01591556], dtype=float32),\n",
+       " array([0.96804875], dtype=float32),\n",
+       " array([0.91091835], dtype=float32),\n",
+       " array([0.00087433], dtype=float32),\n",
+       " array([0.02600787], dtype=float32),\n",
+       " array([0.00168016], dtype=float32),\n",
+       " array([0.93263006], dtype=float32),\n",
+       " array([0.19706792], dtype=float32),\n",
+       " array([0.9951959], dtype=float32),\n",
+       " array([0.024617], dtype=float32),\n",
+       " array([0.9766921], dtype=float32),\n",
+       " array([0.04694933], dtype=float32),\n",
+       " array([0.9548745], dtype=float32),\n",
+       " array([0.01036863], dtype=float32),\n",
+       " array([0.9931427], dtype=float32),\n",
+       " array([0.01541146], dtype=float32),\n",
+       " array([0.02152353], dtype=float32),\n",
+       " array([0.78170955], dtype=float32),\n",
+       " array([0.5403529], dtype=float32),\n",
+       " array([0.22647694], dtype=float32),\n",
+       " array([0.00169189], dtype=float32),\n",
+       " array([0.9999125], dtype=float32),\n",
+       " array([0.00411672], dtype=float32),\n",
+       " array([0.92826104], dtype=float32),\n",
+       " array([0.78801906], dtype=float32),\n",
+       " array([0.9407639], dtype=float32),\n",
+       " array([0.98959863], dtype=float32),\n",
+       " array([0.9553592], dtype=float32),\n",
+       " array([0.01293176], dtype=float32),\n",
+       " array([0.01023797], dtype=float32),\n",
+       " array([0.03475741], dtype=float32),\n",
+       " array([0.9997676], dtype=float32),\n",
+       " array([0.97791255], dtype=float32),\n",
+       " array([0.00023193], dtype=float32),\n",
+       " array([0.00889389], dtype=float32),\n",
+       " array([0.957821], dtype=float32),\n",
+       " array([0.8767215], dtype=float32),\n",
+       " array([0.12694164], dtype=float32),\n",
+       " array([0.00611601], dtype=float32),\n",
+       " array([0.3953812], dtype=float32),\n",
+       " array([0.0004641], dtype=float32),\n",
+       " array([0.9987463], dtype=float32),\n",
+       " array([0.01019047], dtype=float32),\n",
+       " array([0.5764202], dtype=float32),\n",
+       " array([0.01138657], dtype=float32),\n",
+       " array([0.5458222], dtype=float32),\n",
+       " array([0.9942966], dtype=float32),\n",
+       " array([0.00240233], dtype=float32),\n",
+       " array([0.7553514], dtype=float32),\n",
+       " array([0.0881868], dtype=float32),\n",
+       " array([0.34226933], dtype=float32),\n",
+       " array([0.4873583], dtype=float32),\n",
+       " array([0.33895075], dtype=float32),\n",
+       " array([0.03251609], dtype=float32),\n",
+       " array([0.00574167], dtype=float32),\n",
+       " array([0.988293], dtype=float32),\n",
+       " array([0.00064724], dtype=float32),\n",
+       " array([0.17580858], dtype=float32),\n",
+       " array([0.94925475], dtype=float32),\n",
+       " array([0.18242276], dtype=float32),\n",
+       " array([0.9117029], dtype=float32),\n",
+       " array([0.717524], dtype=float32),\n",
+       " array([0.9948232], dtype=float32),\n",
+       " array([0.41392937], dtype=float32),\n",
+       " array([0.39889827], dtype=float32),\n",
+       " array([0.21468543], dtype=float32),\n",
+       " array([0.00194653], dtype=float32),\n",
+       " array([0.8318713], dtype=float32),\n",
+       " array([0.9048981], dtype=float32),\n",
+       " array([0.00159927], dtype=float32),\n",
+       " array([0.01717789], dtype=float32),\n",
+       " array([0.99441886], dtype=float32),\n",
+       " array([0.9672044], dtype=float32),\n",
+       " array([0.9958038], dtype=float32),\n",
+       " array([0.8791019], dtype=float32),\n",
+       " array([0.9852657], dtype=float32),\n",
+       " array([0.09051052], dtype=float32),\n",
+       " array([0.00520805], dtype=float32),\n",
+       " array([0.4179241], dtype=float32),\n",
+       " array([0.02102872], dtype=float32),\n",
+       " array([0.999458], dtype=float32),\n",
+       " array([0.07276529], dtype=float32),\n",
+       " array([0.89086306], dtype=float32),\n",
+       " array([0.58678746], dtype=float32),\n",
+       " array([0.9981602], dtype=float32),\n",
+       " array([0.98019546], dtype=float32),\n",
+       " array([0.81768113], dtype=float32),\n",
+       " array([0.3091106], dtype=float32),\n",
+       " array([0.7304271], dtype=float32),\n",
+       " array([0.00713154], dtype=float32),\n",
+       " array([0.10799696], dtype=float32),\n",
+       " array([0.00034327], dtype=float32),\n",
+       " array([0.97954047], dtype=float32),\n",
+       " array([0.9953832], dtype=float32),\n",
+       " array([0.06257767], dtype=float32),\n",
+       " array([0.8372882], dtype=float32),\n",
+       " array([1.8113557e-05], dtype=float32),\n",
+       " array([0.04951284], dtype=float32),\n",
+       " array([0.04139359], dtype=float32),\n",
+       " array([0.5803639], dtype=float32),\n",
+       " array([0.01002938], dtype=float32),\n",
+       " array([0.44129696], dtype=float32),\n",
+       " array([0.88426584], dtype=float32),\n",
+       " array([0.01807107], dtype=float32),\n",
+       " array([0.87367356], dtype=float32),\n",
+       " array([0.09437197], dtype=float32),\n",
+       " array([0.98715776], dtype=float32),\n",
+       " array([0.06557368], dtype=float32),\n",
+       " array([0.9997048], dtype=float32),\n",
+       " array([0.5877887], dtype=float32),\n",
+       " array([0.10160982], dtype=float32),\n",
+       " array([0.2194032], dtype=float32),\n",
+       " array([0.996086], dtype=float32),\n",
+       " array([0.70603895], dtype=float32),\n",
+       " array([0.0575645], dtype=float32),\n",
+       " array([0.58087355], dtype=float32),\n",
+       " array([0.9330629], dtype=float32),\n",
+       " array([0.004917], dtype=float32),\n",
+       " array([0.19366205], dtype=float32),\n",
+       " array([0.99521846], dtype=float32),\n",
+       " array([0.9976768], dtype=float32),\n",
+       " array([0.01894422], dtype=float32),\n",
+       " array([0.5626045], dtype=float32),\n",
+       " array([0.99873656], dtype=float32),\n",
+       " array([0.98620087], dtype=float32),\n",
+       " array([0.20380375], dtype=float32),\n",
+       " array([0.00324226], dtype=float32),\n",
+       " array([0.03813465], dtype=float32),\n",
+       " array([0.07607552], dtype=float32),\n",
+       " array([0.02199142], dtype=float32),\n",
+       " array([0.7561464], dtype=float32),\n",
+       " array([0.9669124], dtype=float32),\n",
+       " array([0.86246103], dtype=float32),\n",
+       " array([0.189888], dtype=float32),\n",
+       " array([6.4221174e-05], dtype=float32),\n",
+       " array([0.61084515], dtype=float32),\n",
+       " array([0.9931891], dtype=float32),\n",
+       " array([0.95753783], dtype=float32),\n",
+       " array([0.96757764], dtype=float32),\n",
+       " array([0.99537355], dtype=float32),\n",
+       " array([0.05853846], dtype=float32),\n",
+       " array([0.9369336], dtype=float32),\n",
+       " array([0.99967706], dtype=float32),\n",
+       " array([0.48768336], dtype=float32),\n",
+       " array([0.38854727], dtype=float32),\n",
+       " array([0.16301523], dtype=float32),\n",
+       " array([0.44746688], dtype=float32),\n",
+       " array([0.9951616], dtype=float32),\n",
+       " array([0.9310025], dtype=float32),\n",
+       " array([0.9793833], dtype=float32),\n",
+       " array([0.9996581], dtype=float32),\n",
+       " array([0.06153212], dtype=float32),\n",
+       " array([0.99993515], dtype=float32),\n",
+       " array([8.6169755e-05], dtype=float32),\n",
+       " array([0.14121674], dtype=float32),\n",
+       " array([0.001046], dtype=float32),\n",
+       " array([0.96887445], dtype=float32),\n",
+       " array([0.9940006], dtype=float32),\n",
+       " array([0.20827933], dtype=float32),\n",
+       " array([1.3143304e-05], dtype=float32),\n",
+       " array([0.70770514], dtype=float32),\n",
+       " array([0.00062637], dtype=float32),\n",
+       " array([0.09923268], dtype=float32),\n",
+       " array([0.00062528], dtype=float32),\n",
+       " array([0.9974062], dtype=float32),\n",
+       " array([0.6399337], dtype=float32),\n",
+       " array([0.9582232], dtype=float32),\n",
+       " array([5.5980826e-07], dtype=float32),\n",
+       " array([0.98064935], dtype=float32),\n",
+       " array([0.9810916], dtype=float32),\n",
+       " array([0.02825824], dtype=float32),\n",
+       " array([0.00210933], dtype=float32),\n",
+       " array([0.03763315], dtype=float32),\n",
+       " array([0.9897635], dtype=float32),\n",
+       " array([0.38776097], dtype=float32),\n",
+       " array([0.01495247], dtype=float32),\n",
+       " array([0.00611806], dtype=float32),\n",
+       " array([0.998847], dtype=float32),\n",
+       " array([0.01276735], dtype=float32),\n",
+       " array([0.00442079], dtype=float32),\n",
+       " array([0.2124616], dtype=float32),\n",
+       " array([0.01237443], dtype=float32),\n",
+       " array([0.01144132], dtype=float32),\n",
+       " array([0.92837715], dtype=float32),\n",
+       " array([0.02206292], dtype=float32),\n",
+       " array([0.98381627], dtype=float32),\n",
+       " array([0.00593874], dtype=float32),\n",
+       " array([0.26435003], dtype=float32),\n",
+       " array([0.02000471], dtype=float32),\n",
+       " array([0.84790653], dtype=float32),\n",
+       " array([0.9852173], dtype=float32),\n",
+       " array([0.9987846], dtype=float32),\n",
+       " array([0.99995625], dtype=float32),\n",
+       " array([0.17164613], dtype=float32),\n",
+       " array([0.18840362], dtype=float32),\n",
+       " array([0.9717937], dtype=float32),\n",
+       " array([0.74185556], dtype=float32),\n",
+       " array([0.00340732], dtype=float32),\n",
+       " array([0.01526649], dtype=float32),\n",
+       " array([0.61485744], dtype=float32),\n",
+       " array([0.9119215], dtype=float32),\n",
+       " array([0.02722141], dtype=float32),\n",
+       " array([0.39047685], dtype=float32),\n",
+       " array([0.19983715], dtype=float32),\n",
+       " array([0.00018045], dtype=float32),\n",
+       " array([0.76507735], dtype=float32),\n",
+       " array([0.00108664], dtype=float32),\n",
+       " array([0.8838372], dtype=float32),\n",
+       " array([0.9674925], dtype=float32),\n",
+       " array([0.00014587], dtype=float32),\n",
+       " array([0.01428808], dtype=float32),\n",
+       " array([0.6684856], dtype=float32),\n",
+       " array([0.03062288], dtype=float32),\n",
+       " array([0.46116126], dtype=float32),\n",
+       " array([0.16899237], dtype=float32),\n",
+       " array([0.9975586], dtype=float32),\n",
+       " array([0.91609216], dtype=float32),\n",
+       " array([0.9852622], dtype=float32),\n",
+       " array([0.5730661], dtype=float32),\n",
+       " array([0.19011642], dtype=float32),\n",
+       " array([0.9962901], dtype=float32),\n",
+       " array([0.00494908], dtype=float32),\n",
+       " array([0.9681047], dtype=float32),\n",
+       " array([0.03208594], dtype=float32),\n",
+       " array([0.00147857], dtype=float32),\n",
+       " array([0.12340485], dtype=float32),\n",
+       " array([0.996431], dtype=float32),\n",
+       " array([0.9512111], dtype=float32),\n",
+       " array([0.9922307], dtype=float32),\n",
+       " array([0.02449521], dtype=float32),\n",
+       " array([0.9568155], dtype=float32),\n",
+       " array([0.99991953], dtype=float32),\n",
+       " array([0.9982376], dtype=float32),\n",
+       " array([0.1572257], dtype=float32),\n",
+       " array([0.34052122], dtype=float32),\n",
+       " array([0.6778389], dtype=float32),\n",
+       " array([0.9513396], dtype=float32),\n",
+       " array([0.99644357], dtype=float32),\n",
+       " array([0.3379453], dtype=float32),\n",
+       " array([0.9816772], dtype=float32),\n",
+       " array([0.01320378], dtype=float32),\n",
+       " array([0.00027732], dtype=float32),\n",
        " array([0.99997675], dtype=float32),\n",
-       " array([0.6763087], dtype=float32),\n",
-       " array([0.9927671], dtype=float32),\n",
-       " array([0.02451441], dtype=float32),\n",
-       " array([0.86146873], dtype=float32),\n",
-       " array([0.04389035], dtype=float32),\n",
-       " array([0.9999443], dtype=float32),\n",
-       " array([0.809564], dtype=float32),\n",
-       " array([0.99578035], dtype=float32),\n",
-       " array([0.4989446], dtype=float32),\n",
-       " array([0.02612785], dtype=float32),\n",
-       " array([0.87981015], dtype=float32),\n",
-       " array([0.6465501], dtype=float32),\n",
-       " array([0.576932], dtype=float32),\n",
-       " array([0.03007537], dtype=float32),\n",
-       " array([0.00870073], dtype=float32),\n",
-       " array([0.9998024], dtype=float32),\n",
-       " array([0.08114275], dtype=float32),\n",
-       " array([0.68397623], dtype=float32),\n",
-       " array([0.9999337], dtype=float32),\n",
-       " array([0.0099621], dtype=float32),\n",
-       " array([0.99060285], dtype=float32),\n",
-       " array([0.00027312], dtype=float32),\n",
-       " array([0.9289166], dtype=float32),\n",
-       " array([0.9932289], dtype=float32),\n",
-       " array([0.02628781], dtype=float32),\n",
-       " array([0.99826354], dtype=float32),\n",
-       " array([0.6789669], dtype=float32),\n",
+       " array([0.49815693], dtype=float32),\n",
+       " array([0.00038428], dtype=float32),\n",
+       " array([0.03885539], dtype=float32),\n",
+       " array([0.5476643], dtype=float32),\n",
+       " array([0.9998455], dtype=float32),\n",
+       " array([0.9970118], dtype=float32),\n",
+       " array([0.5124474], dtype=float32),\n",
+       " array([0.38307184], dtype=float32),\n",
+       " array([0.99099356], dtype=float32),\n",
+       " array([0.25695708], dtype=float32),\n",
+       " array([0.9953335], dtype=float32),\n",
+       " array([0.97055674], dtype=float32),\n",
+       " array([0.4068285], dtype=float32),\n",
+       " array([1.4898453e-06], dtype=float32),\n",
+       " array([0.66622144], dtype=float32),\n",
+       " array([0.99686724], dtype=float32),\n",
+       " array([0.00997034], dtype=float32),\n",
+       " array([0.2946419], dtype=float32),\n",
+       " array([0.70338255], dtype=float32),\n",
+       " array([0.02406825], dtype=float32),\n",
+       " array([0.99934345], dtype=float32),\n",
+       " array([0.03414964], dtype=float32),\n",
+       " array([0.00095879], dtype=float32),\n",
+       " array([0.99705076], dtype=float32),\n",
+       " array([0.21492238], dtype=float32),\n",
+       " array([0.87716794], dtype=float32),\n",
+       " array([0.47392538], dtype=float32),\n",
+       " array([0.24244678], dtype=float32),\n",
+       " array([0.03492213], dtype=float32),\n",
+       " array([0.9038005], dtype=float32),\n",
+       " array([0.51358217], dtype=float32),\n",
+       " array([0.3492779], dtype=float32),\n",
+       " array([0.37952748], dtype=float32),\n",
+       " array([0.9956209], dtype=float32),\n",
+       " array([0.05870749], dtype=float32),\n",
+       " array([0.93354183], dtype=float32),\n",
+       " array([0.45190257], dtype=float32),\n",
+       " array([0.99952877], dtype=float32),\n",
+       " array([0.35226253], dtype=float32),\n",
        " ...]"
       ]
      },

From 1d722fa75d9d1069bb7ec03c7b2a18b4ec71b138 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 18 Mar 2019 16:58:14 +0800
Subject: [PATCH 29/46] Add files via upload

---
 keras/3.6-classifying-newswires.ipynb | 554 ++++++++++++++++++++++++++
 1 file changed, 554 insertions(+)
 create mode 100644 keras/3.6-classifying-newswires.ipynb

diff --git a/keras/3.6-classifying-newswires.ipynb b/keras/3.6-classifying-newswires.ipynb
new file mode 100644
index 0000000..09ac085
--- /dev/null
+++ b/keras/3.6-classifying-newswires.ipynb
@@ -0,0 +1,554 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=8g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# # Classifying newswires: a multi-class classification example\n",
+    "\n",
+    "----\n",
+    "\n",
+    "In the previous section we saw how to classify vector inputs into two mutually exclusive classes using a densely-connected neural network. \n",
+    "But what happens when you have more than two classes? \n",
+    "\n",
+    "In this section, we will build a network to classify Reuters newswires into 46 different mutually-exclusive topics. Since we have many \n",
+    "classes, this problem is an instance of \"multi-class classification\", and since each data point should be classified into only one \n",
+    "category, the problem is more specifically an instance of \"single-label, multi-class classification\". If each data point could have \n",
+    "belonged to multiple categories (in our case, topics) then we would be facing a \"multi-label, multi-class classification\" problem."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The Reuters dataset\n",
+    "\n",
+    "\n",
+    "We will be working with the _Reuters dataset_, a set of short newswires and their topics, published by Reuters in 1986. It's a very simple, \n",
+    "widely used toy dataset for text classification. There are 46 different topics; some topics are more represented than others, but each \n",
+    "topic has at least 10 examples in the training set.\n",
+    "\n",
+    "Like IMDB and MNIST, the Reuters dataset comes packaged as part of Keras API of Analytics Zoo. Let's take a look right away:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import reuters\n",
+    "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(nb_words=10000)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Like with the IMDB dataset, the argument `nb_words=10000` restricts the data to the 10,000 most frequently occurring words found in the \n",
+    "data.\n",
+    "\n",
+    "We have 8,982 training examples and 2,246 test examples:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "word_index = reuters.get_word_index()\n",
+    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Preparing the data\n",
+    "\n",
+    "We can vectorize the data with the exact same code as in our previous example:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "def vectorize_sequences(sequences, dimension=10000):\n",
+    "    results = np.zeros((len(sequences), dimension))\n",
+    "    for i, sequence in enumerate(sequences):\n",
+    "        results[i, sequence] = 1.\n",
+    "    return results\n",
+    "\n",
+    "x_train = vectorize_sequences(train_data)\n",
+    "x_test = vectorize_sequences(test_data)\n",
+    "# this part pending to modify, one-hot or integer issue"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Building our network\n",
+    "\n",
+    "\n",
+    "This topic classification problem looks very similar to our previous movie review classification problem: in both cases, we are trying to \n",
+    "classify short snippets of text. There is however a new constraint here: the number of output classes has gone from 2 to 46, i.e. the \n",
+    "dimensionality of the output space is much larger. \n",
+    "\n",
+    "In a stack of `Dense` layers like what we were using, each layer can only access information present in the output of the previous layer. \n",
+    "If one layer drops some information relevant to the classification problem, this information can never be recovered by later layers: each \n",
+    "layer can potentially become an \"information bottleneck\". In our previous example, we were using 16-dimensional intermediate layers, but a \n",
+    "16-dimensional space may be too limited to learn to separate 46 different classes: such small layers may act as information bottlenecks, \n",
+    "permanently dropping relevant information.\n",
+    "\n",
+    "For this reason we will use larger layers. Let's go with 64 units:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<zoo.pipeline.api.keras.models.Sequential at 0x7f7c44a79e80>"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "model = models.Sequential()\n",
+    "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
+    "model.add(layers.Dense(64, activation='relu'))\n",
+    "model.add(layers.Dense(46, activation='softmax'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "There are two other things you should note about this architecture:\n",
+    "\n",
+    "* We are ending the network with a `Dense` layer of size 46. This means that for each input sample, our network will output a \n",
+    "46-dimensional vector. Each entry in this vector (each dimension) will encode a different output class.\n",
+    "* The last layer uses a `softmax` activation. You have already seen this pattern in the MNIST example. It means that the network will \n",
+    "output a _probability distribution_ over the 46 different output classes, i.e. for every input sample, the network will produce a \n",
+    "46-dimensional output vector where `output[i]` is the probability that the sample belongs to class `i`. The 46 scores will sum to 1.\n",
+    "\n",
+    "The best loss function to use in this case is `categorical_crossentropy`. It measures the distance between two probability distributions: \n",
+    "in our case, between the probability distribution output by our network, and the true distribution of the labels. By minimizing the \n",
+    "distance between these two distributions, we train our network to output something as close as possible to the true labels."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createRMSprop\n",
+      "creating: createZooKerasSparseCategoricalCrossEntropy\n",
+      "creating: createZooKerasSparseCategoricalAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='sparse_categorical_crossentropy',\n",
+    "              metrics=['accuracy'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Validating our approach\n",
+    "\n",
+    "Let's set apart 1,000 samples in our training data to use as a validation set:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "x_val = x_train[:1000]\n",
+    "partial_x_train = x_train[1000:]\n",
+    "\n",
+    "y_val = train_labels[:1000]\n",
+    "partial_y_train = train_labels[1000:]    # this line would return list\n",
+    "partial_y_train = np.array(partial_y_train)    # convert list to ndarray"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's train our network for 20 epochs:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "dir_name = '3-5 ' + str(time.ctime())\n",
+    "model.set_tensorboard('./', dir_name)\n",
+    "model.fit(partial_x_train,\n",
+    "          partial_y_train,\n",
+    "          nb_epoch=20,\n",
+    "          batch_size=512,\n",
+    "          validation_data=(x_val, y_val))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 512 records in 0.03322949 seconds. Throughput is 15408.001 records/second. Loss is 0.36856997.\n",
+    "Top1Accuracy is Accuracy(correct: 808, count: 1000, accuracy: 0.808)_"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd8leX5+PHPdZKTvRdZQAKEEPYIS0RAUcGB4rYubNVqtWprba2t1dr6q1Zr/bprVdwTxVVxM0QUDBiREQgjQBKy917374/n5JBAAgFycgi53q/XeeU5z7xOAs917vHctxhjUEoppQBs7g5AKaXUsUOTglJKKSdNCkoppZw0KSillHLSpKCUUspJk4JSSiknTQqqW4mIh4hUiciA7tzXnURkiIh0e99tEZktIllt3m8Rkeld2fcIrvWsiNx5pMcf5Lx/F5EXuvu8yn083R2Aci8RqWrz1g+oB5od739pjHn1cM5njGkGArp7377AGJPcHecRkWuAy40xM9uc+5ruOLc6/mlS6OOMMc6bsuOb6DXGmC86219EPI0xTT0Rm1Kq52n1kTooR/XAmyLyuohUApeLyFQR+U5EykRkr4g8KiJ2x/6eImJEJMHx/hXH9iUiUiki34pI4uHu69g+V0S2iki5iDwmIt+IyIJO4u5KjL8UkW0iUioij7Y51kNE/i0ixSKyA5hzkN/Pn0Tkjf3WPSEiDzuWrxGRzY7Ps93xLb6zc2WLyEzHsp+IvOyIbSMwYb99/ywiOxzn3Sgi8xzrRwGPA9MdVXNFbX6397Q5/nrHZy8WkfdEJKYrv5tDEZH5jnjKROQrEUlus+1OEckVkQoRyWjzWaeIyDrH+nwRebCr11MuYIzRl74wxgBkAbP3W/d3oAE4G+tLhC8wEZiMVdIcBGwFbnLs7wkYIMHx/hWgCEgF7MCbwCtHsG8UUAmc49j2W6ARWNDJZ+lKjO8DwUACUNL62YGbgI1APBAOrLD+q3R4nUFAFeDf5twFQKrj/dmOfQQ4GagFRju2zQay2pwrG5jpWH4IWAaEAgOBTfvtexEQ4/ib/MwRQz/HtmuAZfvF+Qpwj2P5NEeMYwEf4Engq678bjr4/H8HXnAspzjiONnxN7oT2OJYHgHsAqId+yYCgxzL3wOXOpYDgcnu/r/Ql19aUlBdsdIY86ExpsUYU2uM+d4Ys9oY02SM2QE8A8w4yPGLjDFpxphG4FWsm9Hh7nsWkG6Med+x7d9YCaRDXYzxH8aYcmNMFtYNuPVaFwH/NsZkG2OKgfsPcp0dwAasZAVwKlBqjElzbP/QGLPDWL4CvgQ6bEzez0XA340xpcaYXVjf/tte9y1jzF7H3+Q1rISe2oXzAlwGPGuMSTfG1AF3ADNEJL7NPp39bg7mEuADY8xXjr/R/ViJZTLQhJWARjiqIHc6fndgJfckEQk3xlQaY1Z38XMoF9CkoLpiT9s3IjJMRP4nInkiUgHcC0Qc5Pi8Nss1HLxxubN9Y9vGYYwxWN+sO9TFGLt0LaxvuAfzGnCpY/lnjvetcZwlIqtFpEREyrC+pR/sd9Uq5mAxiMgCEfnRUU1TBgzr4nnB+nzO8xljKoBSIK7NPofzN+vsvC1Yf6M4Y8wW4Dasv0OBozoy2rHr1cBwYIuIrBGRM7r4OZQLaFJQXbF/d8z/YH07HmKMCQL+glU94kp7sapzABARof1NbH9HE+NeoH+b94fqMvsWMFtE4rBKDK85YvQFFgH/wKraCQE+62IceZ3FICKDgKeAG4Bwx3kz2pz3UN1nc7GqpFrPF4hVTZXThbgO57w2rL9ZDoAx5hVjzDSsqiMPrN8LxpgtxphLsKoI/wW8IyI+RxmLOkKaFNSRCATKgWoRSQF+2QPX/AgYLyJni4gncAsQ6aIY3wJuFZE4EQkH/nCwnY0xecBK4AVgizEm07HJG/ACCoFmETkLOOUwYrhTRELEeo7jpjbbArBu/IVY+fFarJJCq3wgvrVhvQOvA78QkdEi4o11c/7aGNNpyeswYp4nIjMd174dqx1otYikiMgsx/VqHa8WrA9whYhEOEoW5Y7P1nKUsagjpElBHYnbgKuw/sP/B6tB2KWMMfnAxcDDQDEwGPgB67mK7o7xKay6/5+wGkEXdeGY17Aajp1VR8aYMuA3wGKsxtoLsJJbV9yNVWLJApYAL7U573rgMWCNY59koG09/OdAJpAvIm2rgVqP/wSrGmex4/gBWO0MR8UYsxHrd/4UVsKaA8xztC94A//EagfKwyqZ/Mlx6BnAZrF6tz0EXGyMaTjaeNSREatqVqneRUQ8sKorLjDGfO3ueJQ6XmhJQfUaIjLHUZ3iDdyF1WtljZvDUuq4oklB9SYnAjuwqiZOB+YbYzqrPlJKHQGtPlJKKeXk8pKCY8iAH0TkgAY2EfEWawiFbY6+3AmujkcppVTnemJAvFuAzUBQB9t+gfX05xARuQR4AKuHSaciIiJMQkJCtweplFLHs7Vr1xYZYw7WjRtwcVJwPDZ/JnAf1lg1+zsHuMexvAh4XETEHKROKyEhgbS0tO4OVSmljmsicqgn8wHXVx89Avyezh9EicPxKL+xhmMuxxqArB0RuU5E0kQkrbCw0FWxKqVUn+eypOB4erPAGLP2aM9ljHnGGJNqjEmNjDxk6UcppdQRcmVJYRrWI+9ZwBvAySLyyn775OAY38UxdEEw1tOqSiml3MBlbQrGmD8CfwRwTKbxO2PM5fvt9gHWY/HfYg0B8NXB2hOUUu7R2NhIdnY2dXV17g5FHYKPjw/x8fHY7Z0NfXVwPT4dp4jcC6QZYz4AngNeFpFtWGPDXNLT8SilDi07O5vAwEASEhKwBqhVxyJjDMXFxWRnZ5OYmHjoAzrQI0nBGLMMa6IOjDF/abO+DriwJ2JQSh25uro6TQi9gIgQHh7O0XTI0WEulFJdogmhdzjav1OfSQoZeRX885MMymsa3R2KUkods/pMUthdXMOTy7azq6Ta3aEopQ5TWVkZTz755BEde8YZZ1BWVtbl/e+55x4eeuihI7rW8aDPJIXYEF8Acstq3RyJUupwHSwpNDU1HfTYjz/+mJCQEFeEdVzqM0khPtRKCtmlmhSU6m3uuOMOtm/fztixY7n99ttZtmwZ06dPZ968eQwfPhyAc889lwkTJjBixAieeeYZ57EJCQkUFRWRlZVFSkoK1157LSNGjOC0006jtvbg94P09HSmTJnC6NGjmT9/PqWlpQA8+uijDB8+nNGjR3PJJVanyeXLlzN27FjGjh3LuHHjqKysdNFvw7V6vEuquwT72vHz8iC3TPtZK3U0/vrhRjblVnTrOYfHBnH32SM63X7//fezYcMG0tPTAVi2bBnr1q1jw4YNzq6Xzz//PGFhYdTW1jJx4kTOP/98wsPbj5qTmZnJ66+/zn//+18uuugi3nnnHS6/fP/Hp/a58soreeyxx5gxYwZ/+ctf+Otf/8ojjzzC/fffz86dO/H29nZWTT300EM88cQTTJs2jaqqKnx8fI721+IWfaakICLEhviSU1bj7lCUUt1g0qRJ7friP/roo4wZM4YpU6awZ88eMjMzDzgmMTGRsWPHAjBhwgSysrI6PX95eTllZWXMmDEDgKuuuooVK1YAMHr0aC677DJeeeUVPD2t79bTpk3jt7/9LY8++ihlZWXO9b1N74z6CMWF+GpJQamjdLBv9D3J39/fubxs2TK++OILvv32W/z8/Jg5c2aHT197e3s7lz08PA5ZfdSZ//3vf6xYsYIPP/yQ++67j59++ok77riDM888k48//php06bx6aefMmzYsCM6vzv1mZICWI3N2tCsVO8TGBh40Dr68vJyQkND8fPzIyMjg+++++6orxkcHExoaChff/01AC+//DIzZsygpaWFPXv2MGvWLB544AHKy8upqqpi+/btjBo1ij/84Q9MnDiRjIyMo47BHfpYScGH4uoGahua8fXycHc4SqkuCg8PZ9q0aYwcOZK5c+dy5plntts+Z84cnn76aVJSUkhOTmbKlCndct0XX3yR66+/npqaGgYNGsTChQtpbm7m8ssvp7y8HGMMN998MyEhIdx1110sXboUm83GiBEjmDt3brfE0NN63RzNqamp5kgn2Xl3XTa/fetHvrxtBoMjA7o5MqWOX5s3byYlJcXdYagu6ujvJSJrjTGphzq2T1UfRQdZvQHyK7RdQSmlOtKnkkK/YE0KSil1MH0rKThLCvVujkQppY5NfSopBHh7EuDtSV65lhSUUqojfSopAPQL8tbqI6WU6kQfTAo+5GlSUEqpDrksKYiIj4isEZEfRWSjiPy1g30WiEihiKQ7Xte4Kp5W0UE+FGibglLHvYAAq9t5bm4uF1xwQYf7zJw5k0N1cX/kkUeoqdk3PM7hDsXdmWN1iG5XlhTqgZONMWOAscAcEenoiZI3jTFjHa9nXRgPYPVAyq+oo6Wldz2foZQ6MrGxsSxatOiIj98/KRzvQ3G7LCkYS5Xjrd3xcvudOCrQm6YWQ0lNg7tDUUp10R133METTzzhfN/6LbuqqopTTjmF8ePHM2rUKN5///0Djs3KymLkyJEA1NbWcskll5CSksL8+fPbjX10ww03kJqayogRI7j77rsBa5C93NxcZs2axaxZs4B9Q3EDPPzww4wcOZKRI0fyyCOPOK/Xm4fodukwFyLiAawFhgBPGGNWd7Db+SJyErAV+I0xZk8H57kOuA5gwIABRxVTmL8XAGU1DUQEeB9ib6XU/m795FbS89K79Zxjo8fyyJxHOt1+8cUXc+utt3LjjTcC8NZbb/Hpp5/i4+PD4sWLCQoKoqioiClTpjBv3rxO5yl+6qmn8PPzY/Pmzaxfv57x48c7t913332EhYXR3NzMKaecwvr167n55pt5+OGHWbp0KREREe3OtXbtWhYuXMjq1asxxjB58mRmzJhBaGhorx6i26UNzcaYZmPMWCAemCQiI/fb5UMgwRgzGvgceLGT8zxjjEk1xqRGRkYeVUytSaGkWudqVqq3GDduHAUFBeTm5vLjjz8SGhpK//79McZw5513Mnr0aGbPnk1OTg75+fmdnmfFihXOm/Po0aMZPXq0c9tbb73F+PHjGTduHBs3bmTTpk0HjWnlypXMnz8ff39/AgICOO+885yD5/XmIbp7ZEA8Y0yZiCwF5gAb2qwvbrPbs8A/XR1LqJ+VFEq1+kipI3Kwb/SudOGFF7Jo0SLy8vK4+OKLAXj11VcpLCxk7dq12O12EhISOhwy+1B27tzJQw89xPfff09oaCgLFiw4ovO06s1DdLuy91GkiIQ4ln2BU4GM/faJafN2HrDZVfG0CnWUFEqrNSko1ZtcfPHFvPHGGyxatIgLL7wQsL5lR0VFYbfbWbp0Kbt27TroOU466SRee+01ADZs2MD69esBqKiowN/fn+DgYPLz81myZInzmM6G7Z4+fTrvvfceNTU1VFdXs3jxYqZPn37Yn+tYG6LblSWFGOBFR7uCDXjLGPORiNwLpBljPgBuFpF5QBNQAixwYTwAhDlKCtrQrFTvMmLECCorK4mLiyMmxvo+edlll3H22WczatQoUlNTD/mN+YYbbuDqq68mJSWFlJQUJkyYAMCYMWMYN24cw4YNo3///kybNs15zHXXXcecOXOIjY1l6dKlzvXjx49nwYIFTJo0CYBrrrmGcePGHbSqqDPH0hDdfWro7FbJf17CVSckcOcZOhSwUl2hQ2f3Ljp09mEK8/eiRKuPlFLqAH0yKYT4eVGm1UdKKXWAPpkUwvztWlJQ6jD1tqrmvupo/059MimE+nlRWqPPKSjVVT4+PhQXF2tiOMYZYyguLj6qB9p65DmFY42VFLSkoFRXxcfHk52dTWFhobtDUYfg4+NDfHz8ER/fN5OCvxfltY00txg8bB0/Dq+U2sdut5OYmOjuMFQP6JPVR2F+doyB8lqtQlJKqbb6ZFIIdY5/pFVISinVVt9MCjr+kVJKdahvJwUtKSilVDt9Myn42wEtKSil1P76ZFJonVNBn1VQSqn2+mRS8LV74OVp0+ojpZTaT59MCiJCmJ8OiqeUUvvrk0kBrG6pWn2klFLt9d2k4GfXhmallNpP300K/jr+kVJK7c+VczT7iMgaEflRRDaKyF872MdbRN4UkW0islpEElwVz/5C/eza0KyUUvtxZUmhHjjZGDMGGAvMEZEp++3zC6DUGDME+DfwgAvjaSciwJuy2kbqm5p76pJKKXXMc1lSMJYqx1u747X/YOznAC86lhcBp4hIjwxbOjDcD2NgT0lNT1xOKaV6BZe2KYiIh4ikAwXA58aY1fvtEgfsATDGNAHlQHgH57lORNJEJK27xnNPCPcHYGeRJgWllGrl0qRgjGk2xowF4oFJIjLyCM/zjDEm1RiTGhkZ2S2xJUZYSSGrqLpbzqeUUseDHul9ZIwpA5YCc/bblAP0BxARTyAYKO6JmEL8vAjxs7OzWJOCUkq1cmXvo0gRCXEs+wKnAhn77fYBcJVj+QLgK9ODk8AmhPtrSUEppdpw5XScMcCLIuKBlXzeMsZ8JCL3AmnGmA+A54CXRWQbUAJc4sJ4DpAY4c/qHT1SMFFKqV7BZUnBGLMeGNfB+r+0Wa4DLnRVDIcSE+xDQWU9xhh6qNOTUkod0/rsE81gTbbT1GKorG9ydyhKKXVM6NNJIcTPmmynrFoHxlNKKejjSWHfZDs63IVSSkEfTwohjrmaSzQpKKUU0MeTQmhr9ZEmBaWUAvp4UnBWH2mbglJKAX08KQT52LGJtikopVSrPp0UbDYh2FdnYFNKqVZ9OimA9ayCztWslFIWTQr+XtrQrJRSDpoU/OyUaEOzUkoBmhQI8/eiqKre3WEopdQxoc8nhcSIAAor6ynXdgWllNKkMCw6EICtBZVujkQppdyvzyeFoY6kkJGnSUEppfp8UogN9iHQ25OtmhSUUkqTgogwNDqQLZoUlFLKpXM09xeRpSKySUQ2isgtHewzU0TKRSTd8fpLR+dytYHhfuSU1brj0kopdUxx5RzNTcBtxph1IhIIrBWRz40xm/bb72tjzFkujOOQQv30ATallAIXlhSMMXuNMescy5XAZiDOVdc7GiG+dqobmmloanF3KEop5VY90qYgIgnAOGB1B5unisiPIrJEREZ0cvx1IpImImmFhYXdHl+IYwjtslotLSil+jaXJwURCQDeAW41xlTst3kdMNAYMwZ4DHivo3MYY54xxqQaY1IjIyO7PcYQ39bJdvQBNqVU3+bSpCAidqyE8Kox5t39txtjKowxVY7ljwG7iES4MqaOhDqm5dSkoJTq61zZ+0iA54DNxpiHO9kn2rEfIjLJEU+xq2LqTIhjWk6dV0Ep1de5svfRNOAK4CcRSXesuxMYAGCMeRq4ALhBRJqAWuASY4xxYUwdak0KOv6RUqqvc1lSMMasBOQQ+zwOPO6qGLoqxFF9pCUFpVRf1+efaAbw9/LA7iGU1WpJQSnVt2lSwBrqIthXH2BTSilNCg6hfnbtfaSU6vM0KTiE+NlZsiGP5Vu7/+E4pZTqLTQpOAyPCQLg359vdXMkSinlPq7sktqr/PWckeRX1JOpM7AppfowLSm0EervRUVdk7vDUEopt9Gk0EaQryfl2i1VKdWHaVJoI8jHTkNTC3WNze4ORSml3EKTQhtBjtFSK+q0tKCU6ps0KbQR3JoUtApJKdVHaVJoI8jH6oxVXquNzUqpvkmTQhtafaSU6us0KbSh1UdKqb5Ok0IbQT6aFJRSfVuXkoKIDBYRb8fyTBG5WURCXBtazwt0tCnoA2xKqb6qqyWFd4BmERkCPAP0B15zWVRu4mP3wNvTpiUFpVSf1dWk0GKMaQLmA48ZY24HYg52gIj0F5GlIrJJRDaKyC0d7CMi8qiIbBOR9SIy/vA/QvcK9rXrU81KqT6rq0mhUUQuBa4CPnKssx/imCbgNmPMcGAKcKOIDN9vn7lAkuN1HfBUF+NxmQBvT974fg9Lftrr7lCUUqrHdTUpXA1MBe4zxuwUkUTg5YMdYIzZa4xZ51iuBDYDcfvtdg7wkrF8B4SIyEFLIK52SkoUAM9/s9OdYSillFt0aehsY8wm4GYAEQkFAo0xD3T1IiKSAIwDVu+3KQ7Y0+Z9tmNdu6/pInIdVkmCAQMGdPWyR+RPZw6ntrGZ99NzMcYgIi69nlJKHUu62vtomYgEiUgYsA74r4g83MVjA7Aaqm81xlQcSZDGmGeMManGmNTIyMgjOcVhSY4OorKuidzyOpdfSymljiVdrT4KdtzQz8Oq7pkMzD7UQSJix0oIrxpj3u1glxysnkyt4h3r3ColOhCALXlHlMOUUqrX6mpS8HTU9V/EvobmgxKr3uU5YLMxprNSxQfAlY5eSFOAcmOM21t4hzqSwua9OgubUqpv6ep0nPcCnwLfGGO+F5FBQOYhjpkGXAH8JCLpjnV3AgMAjDFPAx8DZwDbgBqsBm23C/KxExvsw7aCKneHopRSPaqrDc1vA2+3eb8DOP8Qx6wEDtpKa4wxwI1diaGnxYX6sre81t1hKKVUj+pqQ3O8iCwWkQLH6x0RiXd1cO4UHezLXm1oVkr1MV1tU1iIVf8f63h96Fh33IoN9mFveR1WYUYppfqGriaFSGPMQmNMk+P1AuD6vqFuFBPsQ0NTCyXVDfyY9yNVDdq+oJQ6/nU1KRSLyOUi4uF4XQ4UuzIwd4sO9gXgm6xMJj07iTu/vNPNESmllOt1NSn8HKs7ah7W08YXAAtcFNMxITbEB4CbX9nFAK95PL7mcb7Z/Y2bo1JKKdfqUlIwxuwyxswzxkQaY6KMMedyiN5HvV2Mo6QAUF9yMSHeMVzz4TXUNWnjs1Lq+HU0M6/9ttuiOAaF+3s5l8f1jyHF53dkFGXw9xV/d2NUSinlWkeTFI7rkeJsNuGes4fzzg1TmZEUwd78ofxs5JU88M0DpOelH/oESinVCx1NUjju+2oumJbIhIFhnDQ0khYDZw/8A+G+4fzig1/Q1KJTdiqljj8HTQoiUikiFR28KrGeV+gTxvYPIdDHkx+ymnjijCdYt3cd/1r1L3eHpZRS3e6gScEYE2iMCergFWiM6eq4Sb2ep4eNE4dEsCKzkPNSzuO8lPO4e9ndbC3e6u7QlFKqWx1N9VGfctLQSPaW17GtoIrH5z6Or92Xaz64hhbT4u7QlFKq22hS6KITh0QAsCarhJjAGB4+7WG+3v01/0n7j5sjU0qp7qNJoYviQnzx8rSxu7gGgAVjF3DqoFP5/Re/Z3f5bjdHp5RS3UOTQhfZbEL/UF92OZKCiPDM2c9gjOH6j67XgfOUUscFTQqHYUCYH7tLapzvE0IS+H+n/D+WbFvCqz+96sbIlFKqe2hSOAwDwvzYU1LTrlRw48QbmRo/lVs+uYWC6gI3RqeUUkfPZUlBRJ53TMizoZPtM0WkXETSHa+/uCqW7tI/zI/K+iZyyvbNyOZh8+DZec9S1VDFzUtudmN0Sil19FxZUngBmHOIfb42xox1vO51YSzdYkCYHwAnPrCUb7YVOdcPjxzOXSfdxZsb3+T9jPfdFZ5SSh01lyUFY8wKoMRV53eHIVEBzuV31+W02/aHaX9gdL/R/OrjX1FWV9bToSmlVLdwd5vCVBH5UUSWiMiIznYSketEJE1E0goLC3syvnYGRQbw0a9P5KzRMXyVkU9T874H1+wedp6f9zx5VXnc/tntbotRKaWOhjuTwjpgoDFmDPAY8F5nOxpjnjHGpBpjUiMj3TsL6Mi4YM4aHUtpTSO3L1pPY5vEMCF2Ar+b+jue/eFZvtzxpRujVEqpI+O2pGCMqTDGVDmWPwbsIhLhrngOx2nD+/HLGYNY/EMOX27Ob7ftnpn3MCRsCNd9dB3VDdVuilAppY6M25KCiESLiDiWJzli6RXzPttswu9OSybIx5MvNrfvhupr9+W5ec+xo3QHf1l6zHeoUkqpdlzZJfV14FsgWUSyReQXInK9iFzv2OUCYIOI/Ag8ClxietFjwXYPG7OGRbFobTYPfbqF5pZ9oZ808CRuSL2BR1Y/wurs1W6MUimlDo/0ovswAKmpqSYtLc3dYQDwxaZ8rnnJiuXJy8ZTUt3A5VMGAlBRX8GIJ0cQ7B3M2uvW4u3p7c5QlVJ9nIisNcakHmo/d/c+6tVmD+/H+zdOA+BXr67jz+9toLKuEYAg7yCePvNpNhZu5NZPbqWmseZgp1JKqWOCJoWjNCouGD8vD+f74qoG5/KZQ8/k15N+zdNrn2boY0N5Mf1FnX9BKXVM06RwlGw2ISUmyPm+uLq+3fZH5z7K8gXLiQ2MZcH7C5jwzAS+2PFFT4eplFJdokmhGwxvkxSK2pQUWp008CS+u+Y7Xj//dUprSzn15VM549Uz2FDQ4bBQSinlNpoUusFpI/oRHeQDQFFVfYf72MTGJSMvIeOmDB489UFW7VnFmKfHcN2H17G3cm9PhquUUp3SpNANpidFsuL3s4D2bQod8fH04Xcn/I7tN2/n15N+zQvpL5D0WBL3Lr9XH3ZTSrmdJoVu4uVpI8jHkyeXbeOaFw/dZTbcL5xH5jzCphs3MTdpLncvu5ukx5J4bt1zNLc090DESqneoqS2hFV7VrG1eKvLr+Xp8iv0IREB3uwoquaLzfkUVNTxyJeZeNqEe88Z2ekxQ8KG8PaFb7Nqzypu++w2rvnwGv5v9f/x4KkPcvqQ03sweqWUOzW3NJNVlkVGUQZbireQUZThfBXWWAOB3n7C7fzz1H+6NA5NCt0oxM/uXP58cz6vrd4NcNCk0OqE/iew6uerWLRpEXd8eQdzXp3DaYNP44HZDzA2eqzLYlZKdV1tYy17KvbgIR7YPex42jyx2xw/PezOZQ+bR6fnqKyvdN70txRtIaPYuvFnFmdS37yvTTLSL5JhEcM4J/kchkUMY1jEsB65F2hS6EbF1fvaE/60+PB7FokIF464kHnJ83jy+yf524q/Me4/45gSP4UFYxZw8ciLCfEJ6c6QlVKdMMawo3QH32V/Z71yviM9L52mlqZDHivIAYnC7mGnxbS0m7bXQzwYHDaY5PBk5g6Z67z5J4cnE+4X7sqP13nsOsyN785uAAAgAElEQVRF9xly58c0tRhEoO2vdeNfT8ff+/Dzb2ltKc//8DwL0xeysXAjPp4+zB82n6vHXs3JiScf9NuIUseq6oZqPt3+Kav2rKKffz8SQhJIDE0kISSBcN9wHONk9rjyunK+z/3emQRW56ymqMaaYdHf7s+kuElMjpvM8MjhGAyNzY00tTTR2OL42cH7/bcZDIkhic6b/+CwwXh5ePXI5+vqMBeaFLrRl5vzeSttD6ePiGb51kJSB4Zy1/sbWX77TAaG+x/xeY0xpOWm8UL6C7y24TXK6sroH9SfK8dcyYKxCxgSNqQbP4VS3a+ktoQPt3zI4ozFfLb9M2qbarHb7DS2NLbbz9/uT0JIgpUoQhKdy62vMN+wbkkazS3NbCrc1K4UsLlwMwbrfjg8cjhT4qYwOX4yU+KnMCJyRK//EqZJ4RiwYmshVz6/hkXXTyU1IaxbzlnXVMcHWz5gYfpCPtv+GS2mhRMHnMjVY6/mwuEXEugd2C3XUepo5VTk8F7Ge7yb8S7Ls5bTbJqJD4rn3ORzmZ8yn5MGnkR1QzW7ynexs3QnWWVZ1qs8i52lO9lZtpOK+op25wz0CiQhJIGBIQPx9vCm2TTTYlpobmmm2TQ7f+6/rsW0tNueVZZFVUMVAOG+4UyJn8LkOCsBTIybeFxW02pSOAZsyq3gjEe/pl+QN1dOTeDGWUMwxnD1C99TVtPI3WcPZ9yA0CM+f05FDi+vf5mF6QvZWrwVP7sfFwy/gKvHXs1JA0/CJtrjWO1T11RHTkUO2RXZ7V5eHl4MDBnIwOCBzhvukd4UtxRtYXHGYhZnLGZNzhoAksOTOS/lPOYPm09qbOphfdMvqysjqyzrgKSxq2wXjS2NeIgHHjYP50+b2Lq0LiYghinxU5gSP4XBoYPdVmXVkzQpHAMKK+uZeJ81ztGI2CD+d/N0ymsaGXPvZwBcPS2Bu8/udGrqLjPG8F32dyxMX8ibG9+kor6CxJBErhh9BTMTZjI+ZjzBPsFHfR117KpuqCan8sAbfnZFNnsq9pBdke2sH28r2DuYhuYGaptq260P8g6yEkRroggeyMCQfcsRfhGICMYY1u1dx+KMxby7+V02F20GIDU2lfnD5jN/2HxSIlN65HegDq6rSUF7H7lQmP++BqSdRdUYY8gu2zeE9tb8ym65jogwtf9UpvafyiNzHmHx5sW88OML/G3F37h3xb0AJIUlkRqbSmpsKhNiJjA+ZrxWNfUy5XXlZJZkklmcaf1ss1xSW3LA/uG+4cQHxRMfFM/kuMnEB8XTP6i/c11cUBwBXgEYYyisKWRX2S6yyrLYVb7LWi63vpkvy1pGZUP7f6t+dj8GBg+kqqHK2UWzdXKpc4edS//g/j31a1HdTJOCC3nY9hVJaxqaKaisJ6fU+kaW3C+QLXlV7fYvrqrn4c+38sczUgg4gt5KYP1nvWz0ZVw2+jKKaopYm7uWtXvXkpabxsrdK3l9w+uA1WUuOSKZCTETnMlibPRYArwCjvDTqu5Q3VDNtpJt7W74W4u3klmS2a4rI0D/oP4MDR/KhcMvJCEkwXmzjw+KJy4wDl+7b5euKSJE+UcR5R/FxLiJB2w3xjircZwJw7EsIvx15l85O/lsIvx6xRTr6hBclhRE5HngLKDAGHPA01uO+Zn/DzgDqAEWGGPWuSqeY8GOwmpyy6ykMGtYFE8v305xVT3hAdasbG+vzebV1bsJ8bNz++nD2JJXyWcb87jp5CFHVOcZ4RfB6UNOb/dkdEF1AWtzrSSRtjeNZVnLePWnVwErUaREpjhLE6OiRjEsYhjRAdF9os61uxhjqGuqo7y+nPK6cirqK5zLna0rqiliW8k2cipz2p0rJiCGpPAkzh56NklhSQwNH0pSeBKDQwd3+aZ/tESEUN9QQn1DGRczrkeuqdzHlSWFF4DHgZc62T4XSHK8JgNPOX4eV+45ezg5ZbX89+ud7CyqJqesFm9PG1MHh/P08u1sza9iqiMpeHtaDcOrthcDsPiHHJ5evp0rT0gg2Nfe6TUOR5R/FHOT5jI3aa5z3d7Kvazdu9ZKFnvT+Gz7Z7z0474/W6BXYLuHaoZFDCM5IpmksKQ+Oc1oZX0l20u3k1mcybaSbc5v9jmVOdbNvq78gK6WHQnwCiDYO5hgn2BCfUKZPWg2SWFJJIUnkRSWxJCwIVrFp3qcy5KCMWaFiCQcZJdzgJeM1dL9nYiEiEiMMea4Gkd6wbREWloML327i51FVeSU1RIX4suwaOs/+9b8SqYOtp5cLK2xbiQ/7C7j1IeXM7SftU9+RV23JYWOxATGcFbgWZw19CzA+qa7t2ovmwo3tXsUf2nWUl5e/7LzOJvYSAxJJDkimWHhVqJoTR6RfpG9unRRUV9h3exbb/yl+5bzq/Pb7RsdEM2QsCFMjpvsvMl39jPIO4hgb+tnb+/3ro5P7mxTiAP2tHmf7Vh3QFIQkeuA6wAGDBjQI8F1J5tNSOoXwPrscuqaWogN8SUq0Bs/Lw+yivcNl13aZpiMzIIqKhzzPe8tr3MmiJ4gIsQGxhIbGMvsQbPbbatqqGJr8dYDxm35audX1DXVOffzs/sR5R9FpF+ks7663bJ/+/VdKXE0tzRT1VBFZUMllfWV7ZYrG6z3VQ1VzqdHm02z9bOl+aDv2y7nVuaSWZzpHICsVWxgLEPChnBm0pkkhVvf4oeEDWFw6GD9Nq+OK72iodkY8wzwDFhdUt0czhGZNiSC51fuxNfuwdyRMYgIA8P9WberlNP+vZx/nDeakuoGBkf68+CFYzjvyVXkV1iDY+WV1x7i7D0nwCuA8THjGR8zvt36FtPC7vLdzlEd95TvoaCmgMLqQnIrc0nPS6ewppCG5o7nmwjyDnImiCDvIKobq62bvuOGX1lfeUC3ya6wic0aoEw8nAOVdfbew+ZBdEA05ySfc8CN39/ryJ9IV6o3cWdSyAHa9luLd6w7Lp2UFMl/lu+gsbmJWcMiAUiM8OPjn/IAuOWNH+gf6keYvxcxwT7tjs0r73g2t2OJTWzOoQjmDJnT4T7GGCrqKyisKaSg2koYBdUF1nLruppCSmpLCPAKoH9QfwK9Awn0CiTAK4BAr8D27x3Lgd77tgd4BThHr/QQj15dhaWUO7gzKXwA3CQib2A1MJcfb+0JbaUmhOJr9yAhwp/ThkcDtBsPKbu0lpqGZlIHhhIZ4I1NoMVRJsqrOHZKCkdDRKz6dZ9gHa9JqWOUK7ukvg7MBCJEJBu4G7ADGGOeBj7G6o66DatL6tWuiuVY4O3pwX+umEBsiC82x/MLCeF+APjaPahtbKakuoHwAC88PWxEBnq3qT6qa3eub7cX4+khTOym8ZSUUqqVK3sfXXqI7Qa40VXXPxadNDSy3fsER0nhotR43kzbQ11jC6F+1lPQ0cG+5FfUI2I1NLcyxvDbt9IJ8fNiyS3T263/9+dbOXtMLEk92CitlDq+6IhpbpQcHUi4vxezh/cjPtQqNbQOjRETZLUrDI4MIKesltoGa97mjbkV7C2vY2t+pXMdQGFVPY9+tY330o/bZhmlVA/QpOBGIX5erL3rVKYnRRIb4utcBxDtaGy+cupAquqb+NPinwD4YrPVR765xbAxt9x5rnxHY3TrMBpKKXUkNCkcI+JCrCRQ22BN9Tcw3A9Pm3DO2Dh+Pi2R99JzKK9pZNX2YgaEWaWK9D1lzuPzKqwqptyyOpRS6kj1iucU+oLfnDqU4qoGzhodC8ClkwaQOjCMYF87p4+I5rmVO/l2RzFb8yuZOzKa5q2GH3YfmBRyyrSkoJQ6cpoUjhFRgT48c+W+oc597B6MirfmQBjbPwRfuwfvp+dQVtNIUlQgTc2GTzfm0djcgt3DRr6jMTqvoo6m5hbKahuJCOh74xIppY6OVh/1Al6eNiYPCmPJButBt6H9AjklJYqKuibSskqBfSWF5hbDym1FpP79C95Ys9ttMSuleidNCr3E2Y5qJYCh/QKYnhSJl4eNLx0Nz/kV+9oSvs60Zth6+POt3PTaOgorj/0nopVSxwZNCr3EmaNjnMuRgd74e3syZXA4X2ZYE6/sLa8jKcqaIGftLqv0UFBZz0fr9/Lcyp3OYwsr61m2pf1kLUop1UqTQi/hY/fggfNHcdupQ53j+cxOiWJnUTUXPLWKbQVVpCaE4mET1meXtTv2tdW7qK63ejUt/GYnP3/h+3bPOCilVCtNCr3IxRMH8OtTkpzvTx4WBUDarlLG9g/hjFExDIrwd46Z5O1p47enDqWiromvM62hoHeX1NBi4LsdxXyz7cCJ3JVSfZv2PurF4kP9uHLqQIZEBXDl1AQAhsUEkVlQxUlDI1m4YCItxvCf5dt5dfVuMvOryHY83Hb1C98D8P6N0xjTP8RdH0EpdYzRpNDL3XtO++mvU2IC+fBHiAvxxcMmeCBMHhTOVxkFfJ1ZxP4jSd/z4UYW/2paD0aslDqWafXRcSYlOgiA+NB9k7qf4JjuE8DsN0XRT9nl/N8XmTz8+dYeiU8pdWzTpHCcGR0fTJi/F+MG7KsSunJqAm/9ciq+9vZzAs9KjqSpxfDE0m28tnoXxpEx3lizm3W7S3s0bqXUsUGTwnEmPMCbdXedygmDI5zrvDxtTEoMIyXGGlI70NuqNZw31nr2oaG5haKqBgoq6zHGcMe7P3Hek6s6vUZdYzNvfb/HmUSUUscPTQp9yIhYa9iMKYPD8fK0cerwaGxt2hg25JRTWtPofF9S3X4+5Q055bS0GD5av5ffv7OeH/a07/qqlOr9NCn0IfPGxjJ3ZDR/O2ckr/xiMgHeniSE+2MTEIENORXkthlQ77ONec7lTzbkcdZjK3kvPYcdhVUA7Cis7vHPoJRyLZcmBRGZIyJbRGSbiNzRwfYFIlIoIumO1zWujKevm5gQxlOXTyA62IdJidZUnqkJoaQmhDE4MoD30nPaPbvw0GdbKHAMn7H4h2wAsoprnMlgR2EVxVX1fLLhuJ1aW6k+x2VJQUQ8gCeAucBw4FIRGd7Brm8aY8Y6Xs+6Kh7Vsfvmj+Kln0/iz2emUFxVzz+WZADw2rWTqaht4sll29lVXM2KrVayyCuvZWeRlRR2FlXzyBeZXP/KugPmkVZK9U6uLClMArYZY3YYYxqAN4BzXHg9dQTsHjZ87B7MTI7i3HFxAHjYhCmJ4cxIjuSj9bnMe/wbvDxtBPva2VFYzc5iKylkFlSxxFFK+MHRW+nttD08+GmGez6MUuqouTIpxAF72rzPdqzb3/kisl5EFolI/45OJCLXiUiaiKQVFha6IlaFVb0E1vDbNptw5qgYiqoaaGhqYfGvTuC04f1I21VKQ1MLwb52thVUUVRlNUav211KS4vhkS8yeWbFDufYSi98s1Orl5TqRdzd0PwhkGCMGQ18DrzY0U7GmGeMManGmNTIyMgeDbAvaU0KrU5OiSLc34vbT09mUGQACRH+zm1zRkQDkBjhz4jYINbtLuOHPaXklNXS2GxYt7uUpuYW7vlwE9e/sq5HP4dS6si5cpiLHKDtN/94xzonY0xxm7fPAv90YTzqEKKDrXmiTxpqJd4gHzvf/2k2Nke/1f6OuaG9PW3ce+4Izh0Xx8SEUB74JIMXv93FO+ty8PK00dTcwuodxQT72tud//usEgZHBhDm79WDn0opdThcWVL4HkgSkUQR8QIuAT5ou4OIxLR5Ow/Y7MJ4VBdsuvd0nm0zLaitzYMMUxLDmJgQygc3nYi3pwdTB4fj6WFjyqBwGppaWJSWzdRB4YyKC+azTfl8t2Nfzv/wx1wufPpbrn9lbbuH3gor6/n5C9+36wqrlHIflyUFY0wTcBPwKdbN/i1jzEYRuVdE5jl2u1lENorIj8DNwAJXxaO6xs/LEy/Pjv9ZRAX58Pb1J5AcHdhu/aTEMDxsQkNzC9OTIrhm+iAy8ir5+//25fjbF/1IRIAXa3aW8O66HO54Zz1b8ipZsmEvX2UU8LePNrn0cymlusalo6QaYz4GPt5v3V/aLP8R+KMrY1CuF+hjZ0x8MOt2lzE9KZLk6ED2lNbwztpsvD092LS3grrGFu4+O5mnlm3n3o82UV7byJ7SGpKirASzemcJjc0t2D0O73tKXWMzw+76hHvOHs6CaYmu+HhK9SnubmhWx4n54+MZPyCEof2sKUF/NXMIX942k39eMNq5z5j4EOaMjKa81hpK45ttxbyfbjUzlVQ38HZaNn98dz1vp+1hV3E1H/yYC8ADn2Qw6u5Pefm7XQdct3XgvqeWb3fp51Oqr9D5FFS3uGLKQK6YMvCA9TGOxmsvTxtJ/QKYMzKaZ1bsYP64OFZtLyK/op5LJ/XnnbU53PPhRhqaWnh9zR5C/eyU1jQyrn8IX2cWUlnfxD+XZHDJxP7tShPfbbfaLaICfXrmgyp1nNOSgnKpMH8vvD1tDI8Jwu5hY2x8CLefnsyts5O46WRratFJiWGMHxhCQ1MLs1OimJgQ6hyY74Mfc9lVVEO/IG8q65tYu6v9kN7fOhqz9x+8Tyl1ZLSkoFxKRJg9vB9j4635HWw24cZZQwC4NMQXX7sHc0fGkFtWx3c7Sjh7TCzThkSQvruM/6zYzrNf76CyvonrZw7mkS+28uKqLAaE+REb4ktVfRPpjpFac8trqWtsxme/OSMAtuRV8sb3u/nzmcPxsMkB25VS+2hJQbncEz8bz7UnDTpgvaeHjQsmxONj9+DC1HiuOTGR00dEExHgzezh/ThrdKyzxDAqLpjpSZEs2ZDH/Ce/obiqnpWZhTQ2Gy6dNABjYNhdnzi7wTY2t7CtoIpHv8zk8aXbWPhNljOBdIeCyjreWZutc0qo446WFNQxISrQhz+f1X68xNNG9OPuDzYCMCjSn6cvn0DarhIWLPyeBQu/p7G5hSAfT84fH8fra3YD8NH6XGoamrj2pbWANWRHq+VbC5kwMJSahiZqG5oJD/AG4MFPM/Dy8OCW2UkHjbGspoHnVu7kupMG8afFG/h8Uz6j44MJ8rWTW1bLuAGh3fb7UMpdtKSgjlkxwfvmmY4N9sXL08YJgyN46rLx7CyqJiOvkpnJUST12/fcRFZRDbe99SOJEf5cfUICqQOtG7WP3cbyrda4WX/7aDPzHv+GFkfCeGLpdv79xVaWbilgQ045JdUNVNU3cd1Laazavm8o8Q/X7+Wxr7ZxzYtpVNc3AbAmq4SHPt3Cpf/9jpqGpi5/NmMMG3PLj7qk0ZMlFWMMZTXadnO805KCOqa9+PNJZOZXtnuy+pSUfiy7fSZrdpYwYWAowb52vrnjZO589yfnjf/ZqyYyYWAomfmVPLdyJ1GB3jy2dBtb8ytZvbOYnLJa0rPLSG6TUF74JovlWwuJC/HlZ5MH8NmmfDbmVnDO2FiumT6ILXkVgPVMxQDHkB/f7Shha14ldY0tfJ1ZxOmOMaEO5Yml23jos608d1Uqp6T0O6LfTX1TM8l//oQ/nZHSYfVcd3t9zR7uXPwTX902g0GRAS6/nnIPLSmoY9qMoZFcM/3AG15EgDdnjIqhX5DVFTUuxJchUdaNKsDbk7H9rYbtpH6B3H/+aK6elkiAlyf3fLDROUnQ55vynXND2D2Ebx3dW3PKavnv1zsYEhVATlktTy7bzsJvdrIxtwI/L6she3dJDQDLMgrYWlAJwHs/5FDf1IwxhsU/ZPPl5nzAmqBo1kPLnM9kZJfW8NBnWwErwXSmvKaRnIMM/7G3zJrD4r6PD390mIc/38rVC9ewx/E5umLlNivhfp1ZdIg9VW+mSUEdNwY7vr2OHxh6QC+jUH8vrjtpEKscN/4gH0/eXZftnGf69BHRNDS3OPevrm/i8Z+NY/ntM5mUGMaitdlsyq3gjFH7husK9Paksr4JYyAh3I8lG/L42X9X86/PtvKbN3/kFy+m8a/PtvC7t9ezs6iaDx0P472fbv2MDvIhLavzpHDF86uZdv9XNDRZcX2yIa/d/vkV+yY2OljyaOut7/dw3/828eKqLJZuKeTWN9M73K+puYU31uxuNyaVp826XbQmT3V80qSgjhuDIq2hvScldNzg+/MTE4kIsEZoffKyCZTWNHLXexsAOK1NtU9SVAB/mDOMYdFBDAz357LJA9hbXkd9UwtTBoU7R3ldMC3BecxLP5/MbacOZe2uUh5fuo354+KYnhTBY19to7nFMDzGGl7cGMN7P+QwKSGMc8fFsW53GSc+8BU5ZbXUNDSxIafcec712dbyVxn55JbVcv0ra7ng6W+d2/PaJIVPN1jzaZdUN3DHO+tZ2cm3+ffSc3j5u13UNjQjAmt3lbK7eF9poam5BWMMD3++lTve/YnT/r2Cnxxx7Cm19vtmexFNbRLokahvaj6q45XraFJQx42x/UM4f3y8cwa5/fl7e/L3c0fyixMTOTEpgt+fnuzcNs5R3TQ40p/PfzujXZXVmaNiuOWUJMYPCGF6UgQDw632hP6hfvzjvFGcOCSC/mG+XHvSIEL97Hh52rhj7jB+NdN6HiMh3I8rpg6kpLqBzzblk1lQxdljYzlxSAQA2aW1PL9yJz/772rOemwlz369gwxH+wXA899k8dCnW5zvWxvICyrqAasq7etMq2rn45/28sb3e7j8udWs2VmCMca5P8Cu4hrqGltoaG7h147nRW598wdG3/Mp/+/jzUz5x5fc+mY6Ty7bzlmjY/D18uAvH2zAGMOekhpC/exU1jXx3Y6SdrEYY3jw0wyeXLatw999WU2DM5FszC0n+c+f8Md3f3LOAd5W40ESzp6SGh7+bAtb8io73acn5ZXXsb2w6pD7GWOorGvkkS+2UlnXeFjXKK9tpK6x55KoNjSr44aP3YN/XTTmoPvMGRnDnJFWFdBVJyTw9/9tZmRcEHEhvgR4ezIsJuiAYzw9bPzm1KH85tShACSG+/PD7jL6BfswY2gkl04a4Lz+/eePprahmX5BPkQFejNnRDTTkiKcvaBab+6npvQjOtiHr26bwV3vb+C5lTvxtAkpMUHtRpc9eVgUX2UUsAYI9/eiuLqBncXVDIrwJ7+iDl+7B2eMiubttGzqm5pZu6uUAG9PfOw2fvNmOmU1DQT4eLLsd7MQsR7yazV1cATZZbWszCyioq6JZ1bsAKzqrVA/O/84bxRLNuTx+0Xr+Wj9XoqqGrj55CE8t3Ini3/I4a20PXyyMY95Y2IZEObHE0u342ETzhoVy4BwPzbklLMhp5xnVuxgR1E1vnYPnrsqlU17rYT3+prdvLsum+cXTOT1NbvZkFOOv7cndY3NfPHbGSz+IYdnVuxgUmIYA8L8CPHz4m+OwRR3l9SQGBHA4Ch/PkjP5c4zUtpNAtWZxuYWPETadVxoq66xmYc/38oVUwaStquEuSNj8LF7dPpg5BXPrSazoIp3bjiBCQNDefyrTBIi/DlrdKxzn13F1Zz/1Lf0C/JmY24FYf5eXDk14ZCxgpVM5j/xDeMGhB7y33Z30aSg+iy7h43195yGYD1p/dTl44kL8T3kcQPDrZtP67hObbXtfSQiPH3FBMD6Rj0kKoDMgipGxAY5JzQaFBnAghMS+WF3GQ9fNJZZwyL5YlMBN75mzVb313kjmJgQRm1DEyen9OPcJ77hlH8tZ3R8MAL0C/LmpKRIXvp2F0szCvk+q4TpSRGM6R/C/UsyiAz0Jr+inpXbikiM8KNtD9aECD8evmgsAEt+2ssNr64jMtAbLw8bt5ySRKCPnfnj4vjnJxk8/LnVMJ7UL5DZw/vxzrpsAKYOCmfR2mxsAjOTI1m1vZi7P9hAZKA3b6VZ+wztF8Adc4fx4qosHv0qk+ggH6KDfFh49UQuevpbrnhuNXYPG5GB1k0TrKqzuz/YiJ+XBy99u28gRJtYJa+P1u+lqU0JqMVYN9/rZwxm/rg4SmoaiAjwJiOvgi83F3DhhHiignz4/aL1LN9ayAc3TSM+1A9jDD/772pOTIrgxllDnNPJ/m/9XnLKavlmWzG7iqv5Kaec166dQl1DMyc4SnjlNY1kFlilhDvf/YmPbj7R2YHg8035nDwsinPGxvHcyp0UVdVTVGWV7B78dAvrs8u5Z94IArw7vwW/9G0WRVUN7CiqJruslr+cNZxgP3un+3cXTQqqTwvy2fefbHpS16Z6PSUlivXZZc5qpK6w2YTbT0/mly+v5eRhUe22nTq8H+vvPg1Px0B/Z46O4c/vWQMCxof6csPMwQDtqhC2F1RR3dDM5MQwTkyKYHCkP9e/Yj2wd/W0RK4+IYFZyVEkRvgz4W+fc+1LaQS2uQF5e9ro12YQwVnDoogK9OayyQO5+ZQhiFjfpO0eNuaPi+O/X+8EYECYHzfNGkJEgDcnJkVwwuBwZj24jMKqev52zki+yijgng83YhPhhpnWDXpQhD+eHjaMsUa8BZidEkVKTBA/PzGR//sykwcvHMNpw/uxblcpP3t2NXe8+xOVdU0sXDCR1TtLsInwzrpsZg6NJD7Ul3s+tObfOGdsLJ9vyucLR0+v297+kceXbiOruJqfT0vkp+xy1mSV8MKqLL74jVX6ALj2pbX8bPIAbGKNn/XtjmLeWZvNDkdvtNaG+0VrsxEBY+Cip7+lqcXw8c3TKatp4A/vrgfg3LGxvJeey//W75uL/P30XJZsyMPH7sHbadmcPSaWE4eEs3lvJS+symLR2mwamlrYvLeCZ69KZUCYn/N3/vqa3eRX1PHksu3OTgYNTS3c8+FGrp0+iOGxB5Zmu5P0tsf0U1NTTVpamrvDUOqwGWP44MdcZgyNJMTv4FOSltU0UFRVz5Co9hMa3fjqOgaE+1FT38SL3+5iVnIkC6+eRF55HXe9v4EdhVU8v2CiszQDcNXza5zPbwBMGBhKXWMz/7t5ertz1zU24+VhO6BqJbu0hj+/t4HRccHcMnvoAT270rJKKKisd/bMWrurhABv+wGTMZVWNzD5/31ptWecPITbTkumpfPA5kEAAAp0SURBVMWwo6jK+TmNMST+0ZqCZUx8MO/dOM15s2xpMdhsQkZeBXMe+ZozRkXz5GUTePyrTB76bCtnjIpmYLg/r63ezeTEMD7bZCWKuSOj+WRjHrOSraq4WcmRLN1S2C62U4ZF4ekhjI4PoaahiSeWbmdWciQDw/35xYmJPL18O6+utp6av3RSf77OLCK7tBZfuwfLbp/J1H98SUSANwWV9QyK8P//7d17cFTlHcbx7xNCQonKTYuxqBDrpVQtBQo4WpV6A6qDtir4j6idUrXO6LS2UuxQdNqptdYZa2sdrdZLO0IVHemMFFFJGW9cyyVRkVi5iEqCgA0ggZBf/zhv1kOym2zE5Oxpfp+ZnT37nrPLk3c3vHnfc/Z9OXfowExvo6ykB8/d+E2OHVDGe9t384tnq3nxrdrMv11SXERxkbj69MH0611ywPAhRJNKnvuVLzJnxWauPauCn1xwEp+FpOVmNrLd47xRcC59Vmzcznfue5WvDerDszec0eaxNbU7WfDGlsxf6Yunn8PexqbMmttdafbSjdwyZw2PXjOKs07I3jO77q/LmVf1Yc4vyTU1Gbf9o5rvjhjEqYP6sm5LPRfe+zKPXTOK0RUDMt/ynv7MGuZXb+GlH5/FHfPeYtbSTQC88KMzua/yHTZ8tJvlG7ZzTP/eLPrp2Mzr19TuZPw9i3jwypGcfWLUq6vfs49XarbyfPUWng69jZkXDWXMcQM46cjDuOovS6gMDc27v56AJDZ+tJubn1zF1acPZvwp8ZWHYdqc1cxauokJpxzJvv1GcZGYF64gG/fVI6mt30NZaTF19Q0MPeow7r58GB/v3kdjU1NmepaOKohGQdI44B6gB/BnM7ujxf5S4DFgBPARMMnM1rf1mt4oOBf9RX3n/LVMOLmcUwb1yes5m7bt5v0dnzC6YkAnp2vb+zs+obxPr0wPoKX6PfvY2dB4wDQn7Wnc35QZfotrPkFcV9/A2LsqKRKsnHE+RUWioXE/Y39bybiTy5lx0YHzbu1qaKQsy3j/tl17ufeldextbOKXF5+c+Rne3bqLsXdVMvCwUhZPP7fdvCs2bue2udU8cvUo+pWVYGZMf6aKVZt2MPsHYygrKWa/WXRivEiUFrc+yd1RiTcKknoAbwPnAe8BS4ErzOyN2DHXA6ea2bWSJgOXmNmktl7XGwXn3Gcxv/pDduzey6RvHJMpq9+zj149e3R4Gdhslq3fRt/ePVsN+XWEmeVsLA9Wvo1CZ55oHgXUmNl/QqBZwEQgvkL7RGBm2H4K+IMkWdrGtJxzBS/bvFSH9vr8ruYZObj/Qb9GZzUIHdGZX177ErAp9vi9UJb1GDNrBD4GWvVtJU2VtEzSsrq6upa7nXPOfU5S8Y1mM3vAzEaa2cgjjsjvskHnnHMd15mNwmbg6NjjQaEs6zGSioE+RCecnXPOJaAzG4WlwPGShkgqASYDc1scMxeYErYvBV7y8wnOOZecTjvRbGaNkm4A5hNdkvqwmVVLuh1YZmZzgYeAxyXVANuIGg7nnHMJ6dRpLszsOeC5FmUzYtt7gMs6M4Nzzrn8peJEs3POua7hjYJzzrmM1M19JKkO2NDuga0dDqR5cVnPn6w0509zdvD8n5djzazda/pT1yh8VpKW5fMV70Ll+ZOV5vxpzg6ev6v58JFzzrkMbxScc85ldKdG4YGkAxwkz5+sNOdPc3bw/F2q25xTcM45177u1FNwzjnXDm8UnHPOZXSLRkHSOElrJdVImpZ0nnxIWi9pjaSVkpaFsv6SFkhaF+77JZ2zmaSHJdVKqoqVZc2ryO/D+7Fa0vDkkufMPlPS5lD/KyVNiO37Wci+VtIFyaT+lKSjJS2U9Iakakk3hvKCr/82sqei/iX1krRE0qqQ/7ZQPkTS4pBzdpgUFEml4XFN2D84yfxZmdn/9Y1oMr53gAqgBFgFDE06Vx651wOHtyi7E5gWtqcBv0k6ZyzbmcBwoKq9vMAEYB4gYAywuACzzwRuznLs0PAZKgWGhM9Wj4TzlwPDw/ahRMvgDk1D/beRPRX1H+rwkLDdE1gc6vTvwORQfj9wXdi+Hrg/bE8GZif52cl26w49hcyyoGa2F2heFjSNJgKPhu1HgYsTzHIAM1tENNNtXK68E4HHLPI60FdSedckbS1H9lwmArPMrMHM3gVqiD5jiTGzD8xsRdiuB94kWtWw4Ou/jey5FFT9hzrcGR72DDcDvkW0xDC0rvvm9+Qp4BwVwhqcMd2hUchnWdBCZMDzkpZLmhrKBprZB2H7Q2BgMtHylitvWt6TG8LwysOxobqCzh6GI75O9Bdrquq/RXZISf1L6iFpJVALLCDqveywaIlhODBjXksQJ6k7NAppdYaZDQfGAz+UdGZ8p0X9z9RcT5y2vMCfgOOAYcAHwO+SjdM+SYcAc4CbzOy/8X2FXv9Zsqem/s1sv5kNI1pdchRwUsKRDkp3aBTyWRa04JjZ5nBfCzxD9GHb0tzND/e1ySXMS668Bf+emNmW8MveBDzIp0MUBZldUk+i/1T/ZmZPh+JU1H+27GmrfwAz2wEsBE4jGpJrXq8mnrHglyDuDo1CPsuCFhRJZZIObd4GzgeqOHD50inAs8kkzFuuvHOBK8NVMGOAj2PDHAWhxRj7JUT1D1H2yeEqkiHA8cCSrs4XF8akHwLeNLO7Y7sKvv5zZU9L/Us6QlLfsP0F4Dyi8yILiZYYhtZ1X9hLECd9prsrbkRXW7xNNNZ3a9J58shbQXSFxSqgujkz0djji8A64AWgf9JZY5mfIOrm7yMaQ/1errxEV2z8Mbwfa4CRBZj98ZBtNdEvcnns+FtD9rXA+AKo+zOIhoZWAyvDbUIa6r+N7Kmof+BU4N8hZxUwI5RXEDVWNcCTQGko7xUe14T9FUl/flrefJoL55xzGd1h+Mg551yevFFwzjmX4Y2Cc865DG8UnHPOZXij4JxzLsMbBedykHRrmPlydZipc7SkmyT1Tjqbc53FL0l1LgtJpwF3A2ebWYOkw4lm2X2V6Lr+rYkGdK6TeE/BuezKga1m1gAQGoFLgaOAhZIWAkg6X9JrklZIejLM4dO8HsaditbEWCLpy6H8MklVYf79Rcn8aM7l5j0F57II/7m/DPQm+jbwbDP7l6T1hJ5C6D08TfSt2l2SbiH65urt4bgHzexXkq4ELjezCyWtAcaZ2WZJfS2aL8e5guE9BeeysGiO/BHAVKAOmC3pqhaHjSFa9OWVMHXyFODY2P4nYvenhe1XgEckfZ9oASjnCkpx+4c41z2Z2X6gEqgMf+FPaXGIgAVmdkWul2i5bWbXShoNfBtYLmmEmRXULJmue/OegnNZSDpR0vGxomHABqCeaNlIgNeB02PnC8oknRB7zqTY/WvhmOPMbLGZzSDqgcSngXYucd5TcC67Q4B7w7TIjUSzWk4FrgD+Kel9MxsbhpSekFQanvdzohl5AfpJWg00hOcB/DY0NiKawXRVl/w0zuXJTzQ71wniJ6STzuJcR/jwkXPOuQzvKTjnnMvwnoJzzrkMbxScc85leKPgnHMuwxsF55xzGd4oOOecy/gfTuCcbTgquGwAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "train_loss = np.array(model.get_train_summary('Loss'))\n",
+    "val_loss = np.array(model.get_validation_summary('Loss'))\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "plt.plot(train_loss[:,0],train_loss[:,1],label='train loss')\n",
+    "plt.plot(val_loss[:,0],val_loss[:,1],label='validation loss',color='green')\n",
+    "plt.title('Training and validation loss')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It seems that the network starts overfitting after 8 epochs. Let's train a new network from scratch for 8 epochs, then let's evaluate it on \n",
+    "the test set:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasSparseCategoricalCrossEntropy\n",
+      "creating: createZooKerasSparseCategoricalAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = models.Sequential()\n",
+    "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
+    "model.add(layers.Dense(64, activation='relu'))\n",
+    "model.add(layers.Dense(46, activation='softmax'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='sparse_categorical_crossentropy',\n",
+    "              metrics=['accuracy'])\n",
+    "model.fit(partial_x_train,\n",
+    "          partial_y_train,\n",
+    "          nb_epoch=8,\n",
+    "          batch_size=512,\n",
+    "          validation_data=(x_val, y_val))\n",
+    "y_test = np.array(test_labels).astype('float32')\n",
+    "results = model.evaluate(x_test, y_test)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[0.9659086465835571, 0.8032057285308838]"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "results"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our approach reaches an accuracy of ~80%. With a balanced binary classification problem, the accuracy reached by a purely random classifier \n",
+    "would be 50%, but in our case it is closer to 19%, so our results seem pretty good, at least when compared to a random baseline:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "0.19011576135351738"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import copy\n",
+    "\n",
+    "test_labels_copy = copy.copy(test_labels)\n",
+    "np.random.shuffle(test_labels_copy)\n",
+    "float(np.sum(np.array(test_labels) == np.array(test_labels_copy))) / len(test_labels)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generating predictions on new data\n",
+    "\n",
+    "We can verify that the `predict` method of our model instance returns a probability distribution over all 46 topics. Let's generate topic \n",
+    "predictions for all of the test data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "predictions = model.predict(x_test).collect()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Each entry in `predictions` is a vector of length 46:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(46,)"
+      ]
+     },
+     "execution_count": 18,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "predictions[0].shape"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The coefficients in this vector sum to 1:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "0.99999994"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "np.sum(predictions[0])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The largest entry is the predicted class, i.e. the class with the highest probability:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "4"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "np.argmax(predictions[0])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Further experiments\n",
+    "\n",
+    "* Try using larger or smaller layers: 32 units, 128 units...\n",
+    "* We were using two hidden layers. Now try to use a single hidden layer, or three hidden layers."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Wrapping up\n",
+    "\n",
+    "\n",
+    "Here's what you should take away from this example:\n",
+    "\n",
+    "* If you are trying to classify data points between N classes, your network should end with a `Dense` layer of size N.\n",
+    "* In a single-label, multi-class classification problem, your network should end with a `softmax` activation, so that it will output a \n",
+    "probability distribution over the N output classes.\n",
+    "* _Categorical crossentropy_ is almost always the loss function you should use for such problems. It minimizes the distance between the \n",
+    "probability distributions output by the network, and the true distribution of the targets.\n",
+    "* There are two ways to handle labels in multi-class classification:\n",
+    "    ** Encoding the labels via \"categorical encoding\" (also known as \"one-hot encoding\") and using `categorical_crossentropy` as your loss \n",
+    "function.\n",
+    "    ** Encoding the labels as integers and using the `sparse_categorical_crossentropy` loss function.\n",
+    "* If you need to classify data into a large number of categories, then you should avoid creating information bottlenecks in your network by having \n",
+    "intermediate layers that are too small."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From 64cba48eba12f6199927beb38c3039cfa8b74de2 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 21 Mar 2019 08:56:43 +0800
Subject: [PATCH 30/46] Add files via upload

---
 keras/3.7-regression.ipynb | 797 +++++++++++++++++++++++++++++++++++++
 1 file changed, 797 insertions(+)
 create mode 100644 keras/3.7-regression.ipynb

diff --git a/keras/3.7-regression.ipynb b/keras/3.7-regression.ipynb
new file mode 100644
index 0000000..0f37622
--- /dev/null
+++ b/keras/3.7-regression.ipynb
@@ -0,0 +1,797 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=8g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Predicting house prices: a regression example\n",
+    "\n",
+    "\n",
+    "----\n",
+    "\n",
+    "\n",
+    "In our two previous examples, we were considering classification problems, where the goal was to predict a single discrete label of an \n",
+    "input data point. Another common type of machine learning problem is \"regression\", which consists of predicting a continuous value instead \n",
+    "of a discrete label. For instance, predicting the temperature tomorrow, given meteorological data, or predicting the time that a \n",
+    "software project will take to complete, given its specifications.\n",
+    "\n",
+    "Do not mix up \"regression\" with the algorithm \"logistic regression\": confusingly, \"logistic regression\" is not a regression algorithm, \n",
+    "it is a classification algorithm."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The Boston Housing Price dataset\n",
+    "\n",
+    "\n",
+    "We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the \n",
+    "suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
+    "\n",
+    "The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in \n",
+    "total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has \n",
+    "a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, \n",
+    "others between 0 and 100...\n",
+    "\n",
+    "Let's take a look at the data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import boston_housing\n",
+    "(train_data, train_targets), (test_data, test_targets) =  boston_housing.load_data()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(404, 13)"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "train_data.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(102, 13)"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_data.shape"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, we have 404 training samples and 102 test samples. The data comprises 13 features. The 13 features in the input data are as \n",
+    "follow:\n",
+    "\n",
+    "1. Per capita crime rate.\n",
+    "2. Proportion of residential land zoned for lots over 25,000 square feet.\n",
+    "3. Proportion of non-retail business acres per town.\n",
+    "4. Charles River dummy variable (= 1 if tract bounds river; 0 otherwise).\n",
+    "5. Nitric oxides concentration (parts per 10 million).\n",
+    "6. Average number of rooms per dwelling.\n",
+    "7. Proportion of owner-occupied units built prior to 1940.\n",
+    "8. Weighted distances to five Boston employment centres.\n",
+    "9. Index of accessibility to radial highways.\n",
+    "10. Full-value property-tax rate per $10,000.\n",
+    "11. Pupil-teacher ratio by town.\n",
+    "12. 1000 * (Bk - 0.63) ** 2 where Bk is the proportion of Black people by town.\n",
+    "13. % lower status of the population.\n",
+    "\n",
+    "The targets are the median values of owner-occupied homes, in thousands of dollars:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([22.6, 50. , 23. ,  8.3, 21.2, 19.9, 20.6, 18.7, 16.1, 18.6,  8.8,\n",
+       "       17.2, 14.9, 10.5, 50. , 29. , 23. , 33.3, 29.4, 21. , 23.8, 19.1,\n",
+       "       20.4, 29.1, 19.3, 23.1, 19.6, 19.4, 38.7, 18.7, 14.6, 20. , 20.5,\n",
+       "       20.1, 23.6, 16.8,  5.6, 50. , 14.5, 13.3, 23.9, 20. , 19.8, 13.8,\n",
+       "       16.5, 21.6, 20.3, 17. , 11.8, 27.5, 15.6, 23.1, 24.3, 42.8, 15.6,\n",
+       "       21.7, 17.1, 17.2, 15. , 21.7, 18.6, 21. , 33.1, 31.5, 20.1, 29.8,\n",
+       "       15.2, 15. , 27.5, 22.6, 20. , 21.4, 23.5, 31.2, 23.7,  7.4, 48.3,\n",
+       "       24.4, 22.6, 18.3, 23.3, 17.1, 27.9, 44.8, 50. , 23. , 21.4, 10.2,\n",
+       "       23.3, 23.2, 18.9, 13.4, 21.9, 24.8, 11.9, 24.3, 13.8, 24.7, 14.1,\n",
+       "       18.7, 28.1, 19.8, 26.7, 21.7, 22. , 22.9, 10.4, 21.9, 20.6, 26.4,\n",
+       "       41.3, 17.2, 27.1, 20.4, 16.5, 24.4,  8.4, 23. ,  9.7, 50. , 30.5,\n",
+       "       12.3, 19.4, 21.2, 20.3, 18.8, 33.4, 18.5, 19.6, 33.2, 13.1,  7.5,\n",
+       "       13.6, 17.4,  8.4, 35.4, 24. , 13.4, 26.2,  7.2, 13.1, 24.5, 37.2,\n",
+       "       25. , 24.1, 16.6, 32.9, 36.2, 11. ,  7.2, 22.8, 28.7, 14.4, 24.4,\n",
+       "       18.1, 22.5, 20.5, 15.2, 17.4, 13.6,  8.7, 18.2, 35.4, 31.7, 33. ,\n",
+       "       22.2, 20.4, 23.9, 25. , 12.7, 29.1, 12. , 17.7, 27. , 20.6, 10.2,\n",
+       "       17.5, 19.7, 29.8, 20.5, 14.9, 10.9, 19.5, 22.7, 19.5, 24.6, 25. ,\n",
+       "       24.5, 50. , 14.3, 11.8, 31. , 28.7, 16.2, 43.5, 25. , 22. , 19.9,\n",
+       "       22.1, 46. , 22.9, 20.2, 43.1, 34.6, 13.8, 24.3, 21.5, 24.4, 21.2,\n",
+       "       23.8, 26.6, 25.1,  9.6, 19.4, 19.4,  9.5, 14. , 26.5, 13.8, 34.7,\n",
+       "       16.3, 21.7, 17.5, 15.6, 20.9, 21.7, 12.7, 18.5, 23.7, 19.3, 12.7,\n",
+       "       21.6, 23.2, 29.6, 21.2, 23.8, 17.1, 22. , 36.5, 18.8, 21.9, 23.1,\n",
+       "       20.2, 17.4, 37. , 24.1, 36.2, 15.7, 32.2, 13.5, 17.9, 13.3, 11.7,\n",
+       "       41.7, 18.4, 13.1, 25. , 21.2, 16. , 34.9, 25.2, 24.8, 21.5, 23.4,\n",
+       "       18.9, 10.8, 21. , 27.5, 17.5, 13.5, 28.7, 14.8, 19.1, 28.6, 13.1,\n",
+       "       19. , 11.3, 13.3, 22.4, 20.1, 18.2, 22.9, 20.6, 25. , 12.8, 34.9,\n",
+       "       23.7, 50. , 29. , 30.1, 22. , 15.6, 23.3, 30.1, 14.3, 22.8, 50. ,\n",
+       "       20.8,  6.3, 34.9, 32.4, 19.9, 20.3, 17.8, 23.1, 20.4, 23.2,  7. ,\n",
+       "       16.8, 46.7, 50. , 22.9, 23.9, 21.4, 21.7, 15.4, 15.3, 23.1, 23.9,\n",
+       "       19.4, 11.9, 17.8, 31.5, 33.8, 20.8, 19.8, 22.4,  5. , 24.5, 19.4,\n",
+       "       15.1, 18.2, 19.3, 27.1, 20.7, 37.6, 11.7, 33.4, 30.1, 21.4, 45.4,\n",
+       "       20.1, 20.8, 26.4, 10.4, 21.8, 32. , 21.7, 18.4, 37.9, 17.8, 28. ,\n",
+       "       28.2, 36. , 18.9, 15. , 22.5, 30.7, 20. , 19.1, 23.3, 26.6, 21.1,\n",
+       "       19.7, 20. , 12.1,  7.2, 14.2, 17.3, 27.5, 22.2, 10.9, 19.2, 32. ,\n",
+       "       14.5, 24.7, 12.6, 24. , 24.1, 50. , 16.1, 43.8, 26.6, 36.1, 21.8,\n",
+       "       29.9, 50. , 44. , 20.6, 19.6, 28.4, 19.1, 22.3, 20.9, 28.4, 14.4,\n",
+       "       32.7, 13.8,  8.5, 22.5, 35.1, 31.6, 17.8, 15.6])"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "train_targets"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The prices are typically between \\$10,000  and  \\$50,000. If that sounds cheap, remember this was the mid-1970s, and these prices are not \n",
+    "inflation-adjusted."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Preparing the data\n",
+    "\n",
+    "\n",
+    "It would be problematic to feed into a neural network values that all take wildly different ranges. The network might be able to \n",
+    "automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice to deal \n",
+    "with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we \n",
+    "will subtract the mean of the feature and divide by the standard deviation, so that the feature will be centered around 0 and will have a \n",
+    "unit standard deviation. This is easily done in Numpy:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mean = train_data.mean(axis=0)\n",
+    "train_data -= mean\n",
+    "std = train_data.std(axis=0)\n",
+    "train_data /= std\n",
+    "\n",
+    "test_data -= mean\n",
+    "test_data /= std"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that the quantities that we use for normalizing the test data have been computed using the training data. We should never use in our \n",
+    "workflow any quantity computed on the test data, even for something as simple as data normalization."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Building our network\n",
+    "\n",
+    "\n",
+    "Because so few samples are available, we will be using a very small network with two \n",
+    "hidden layers, each with 64 units. In general, the less training data you have, the worse overfitting will be, and using \n",
+    "a small network is one way to mitigate overfitting."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "def build_model():\n",
+    "    # Because we will need to instantiate\n",
+    "    # the same model multiple times,\n",
+    "    # we use a function to construct it.\n",
+    "    model = models.Sequential()\n",
+    "    model.add(layers.Dense(64, activation='relu',\n",
+    "                           input_shape=(train_data.shape[1],)))\n",
+    "    model.add(layers.Dense(64, activation='relu'))\n",
+    "    model.add(layers.Dense(1))\n",
+    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
+    "    return model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our network ends with a single unit, and no activation (i.e. it will be linear layer). \n",
+    "This is a typical setup for scalar regression (i.e. regression where we are trying to predict a single continuous value). \n",
+    "Applying an activation function would constrain the range that the output can take; for instance if \n",
+    "we applied a `sigmoid` activation function to our last layer, the network could only learn to predict values between 0 and 1. Here, because \n",
+    "the last layer is purely linear, the network is free to learn to predict values in any range.\n",
+    "\n",
+    "Note that we are compiling the network with the `mse` loss function -- Mean Squared Error, the square of the difference between the \n",
+    "predictions and the targets, a widely used loss function for regression problems.\n",
+    "\n",
+    "We are also monitoring a new metric during training: `mae`. This stands for Mean Absolute Error. It is simply the absolute value of the \n",
+    "difference between the predictions and the targets. For instance, a MAE of 0.5 on this problem would mean that our predictions are off by \n",
+    "\\$500 on average."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Validating our approach using K-fold validation\n",
+    "\n",
+    "\n",
+    "To evaluate our network while we keep adjusting its parameters (such as the number of epochs used for training), we could simply split the \n",
+    "data into a training set and a validation set, as we were doing in our previous examples. However, because we have so few data points, the \n",
+    "validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot \n",
+    "depending on _which_ data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a \n",
+    "high _variance_ with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
+    "\n",
+    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions \n",
+    "(typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining \n",
+    "partition. The validation score for the model used would then be the average of the K validation scores obtained."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then let's start our training:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "processing fold # 0\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 1\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 2\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 3\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n"
+     ]
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "k = 4\n",
+    "num_val_samples = len(train_data) // k\n",
+    "num_nb_epoch = 50\n",
+    "all_scores = []\n",
+    "for i in range(k):\n",
+    "    print('processing fold #', i)\n",
+    "    # Prepare the validation data: data from partition # k\n",
+    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "\n",
+    "    # Prepare the training data: data from all other partitions\n",
+    "    partial_train_data = np.concatenate(\n",
+    "        [train_data[:i * num_val_samples],\n",
+    "         train_data[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "    partial_train_targets = np.concatenate(\n",
+    "        [train_targets[:i * num_val_samples],\n",
+    "         train_targets[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "\n",
+    "    # Build the model (already compiled)\n",
+    "    model = build_model()\n",
+    "    # Train the model (in silent mode, verbose=0)\n",
+    "    #model.fit(partial_train_data, partial_train_targets,\n",
+    "    #          nb_epoch=num_nb_epoch, batch_size=1, verbose=0)\n",
+    "    model.fit(partial_train_data, partial_train_targets,\n",
+    "              nb_epoch=num_nb_epoch, batch_size=16)\n",
+    "\n",
+    "    # Evaluate the model on the validation data\n",
+    "    #val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
+    "    val_mse, val_mae = model.evaluate(val_data, val_targets)\n",
+    "    all_scores.append(val_mae)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 16 records in 0.011235845 seconds. Throughput is 1424.0139 records/second. Loss is 8.708786._\n",
+    "\n",
+    "_INFO - Trained 16 records in 0.009535034 seconds. Throughput is 1678.0223 records/second. Loss is 5.3613434._\n",
+    "\n",
+    "_INFO - Trained 16 records in 0.008636178 seconds. Throughput is 1852.6713 records/second. Loss is 18.106756._\n",
+    "\n",
+    "_INFO - Trained 16 records in 0.009207628 seconds. Throughput is 1737.6897 records/second. Loss is 7.0931993._"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[3.291872501373291, 2.496018171310425, 2.221175193786621, 2.6994853019714355]"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "all_scores"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "2.677137792110443"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "np.mean(all_scores)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can notice, the different runs do indeed show rather different validation scores, from 2.1 to 2.9. Their average (2.4) is a much more \n",
+    "reliable metric than any single of these scores -- that's the entire point of K-fold cross-validation. In this case, we are off by \\\\$2,400 on \n",
+    "average, which is still significant considering that the prices range from \\\\$10,000 to \\\\$50,000. \n",
+    "\n",
+    "Let's try training the network for a bit longer: 500 epochs. To keep a record of how well the model did at each epoch, we will modify our training loop \n",
+    "to save the per-epoch validation score log:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "processing fold # 0\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 1\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 2\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 3\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n"
+     ]
+    }
+   ],
+   "source": [
+    "num_epochs = 500\n",
+    "all_mae_histories = []\n",
+    "for i in range(k):\n",
+    "    print('processing fold #', i)\n",
+    "    # Prepare the validation data: data from partition # k\n",
+    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "\n",
+    "    # Prepare the training data: data from all other partitions\n",
+    "    partial_train_data = np.concatenate(\n",
+    "        [train_data[:i * num_val_samples],\n",
+    "         train_data[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "    partial_train_targets = np.concatenate(\n",
+    "        [train_targets[:i * num_val_samples],\n",
+    "         train_targets[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "\n",
+    "    # Build the model (already compiled)\n",
+    "    model = build_model()\n",
+    "    # Train the model (in silent mode, verbose=0)\n",
+    "    import time\n",
+    "    dir_name = '3-7 ' + str(time.ctime())\n",
+    "    model.set_tensorboard('./', dir_name)\n",
+    "    history = model.fit(partial_train_data, partial_train_targets,\n",
+    "                        validation_data=(val_data, val_targets),\n",
+    "                        nb_epoch=num_epochs, batch_size=16)\n",
+    "    \n",
+    "    #mae_history = history.history['val_mean_absolute_error']\n",
+    "    mae_history = model.get_validation_summary(\"Loss\")\n",
+    "    all_mae_histories.append(mae_history)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can then compute the average of the per-epoch MAE scores for all folds:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([[[1.90000000e+01, 4.05375427e+02, 1.55307042e+09],\n",
+       "        [3.80000000e+01, 2.64351837e+02, 1.55307042e+09],\n",
+       "        [5.70000000e+01, 1.50977859e+02, 1.55307042e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 2.07635689e+01, 1.55307053e+09],\n",
+       "        [9.48100000e+03, 2.02473850e+01, 1.55307053e+09],\n",
+       "        [9.50000000e+03, 2.02105141e+01, 1.55307053e+09]],\n",
+       "\n",
+       "       [[1.90000000e+01, 4.76980957e+02, 1.55307053e+09],\n",
+       "        [3.80000000e+01, 3.29584198e+02, 1.55307053e+09],\n",
+       "        [5.70000000e+01, 1.80655548e+02, 1.55307053e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 1.73588219e+01, 1.55307064e+09],\n",
+       "        [9.48100000e+03, 1.78555279e+01, 1.55307064e+09],\n",
+       "        [9.50000000e+03, 1.73744106e+01, 1.55307064e+09]],\n",
+       "\n",
+       "       [[1.90000000e+01, 4.62182434e+02, 1.55307064e+09],\n",
+       "        [3.80000000e+01, 3.34037567e+02, 1.55307064e+09],\n",
+       "        [5.70000000e+01, 2.06141006e+02, 1.55307064e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 1.72124062e+01, 1.55307075e+09],\n",
+       "        [9.48100000e+03, 1.75751667e+01, 1.55307075e+09],\n",
+       "        [9.50000000e+03, 1.74055386e+01, 1.55307075e+09]],\n",
+       "\n",
+       "       [[1.90000000e+01, 5.21177673e+02, 1.55307075e+09],\n",
+       "        [3.80000000e+01, 3.99685974e+02, 1.55307075e+09],\n",
+       "        [5.70000000e+01, 2.67611786e+02, 1.55307075e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 1.75390892e+01, 1.55307085e+09],\n",
+       "        [9.48100000e+03, 1.76337471e+01, 1.55307085e+09],\n",
+       "        [9.50000000e+03, 1.91227703e+01, 1.55307085e+09]]])"
+      ]
+     },
+     "execution_count": 47,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "all_mae_histories = np.array(all_mae_histories)\n",
+    "all_mae_histories"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the `all_mae_histories` is a 3-d array, the last dimension are 3-element tuples. This 3-d array is built up with four 2-d arrays and all the first element of every 2-d array are equal. The first element of tuple stands for the training step and the third element stands for time stamp. You do need to worry about them, let's just calculate the average value through the first axis of this 3-d array. Actually we just want the second elements of this array, which stand for the MAE results. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 48,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([[1.90000000e+01, 4.66429123e+02, 1.55307058e+09],\n",
+       "       [3.80000000e+01, 3.31914894e+02, 1.55307058e+09],\n",
+       "       [5.70000000e+01, 2.01346550e+02, 1.55307058e+09],\n",
+       "       ...,\n",
+       "       [9.46200000e+03, 1.82184715e+01, 1.55307069e+09],\n",
+       "       [9.48100000e+03, 1.83279567e+01, 1.55307069e+09],\n",
+       "       [9.50000000e+03, 1.85283084e+01, 1.55307069e+09]])"
+      ]
+     },
+     "execution_count": 48,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "average_mae_history = np.mean(all_mae_histories, axis=0)\n",
+    "average_mae_history"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, this operation does not mess up the first elements since they are all equal through the first axis. And we do not need to care about the third element because it is useless at this time.\n",
+    "\n",
+    "Let's plot this:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 49,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXmYHFW5/79vVXfP9MxklmSy7wsk7EtCWIPBQEBRQUWW64LKFS8uV/TqFfW64YaiICo/ERFUVFwAQUVEQCAsAUwgQIAkZN+3SSazT3dXnd8fVaf61KmleybT05Pp9/M880x3dS2nejnveXcSQoBhGIapXIxyD4BhGIYpLywIGIZhKhwWBAzDMBUOCwKGYZgKhwUBwzBMhcOCgGEYpsIpmSAgoslE9BgRvUZErxLRp9ztI4noYSJ6w/3fVKoxMAzDMIWhUuURENF4AOOFEC8Q0QgAywFcCOCDAPYJIa4jomsANAkhPl+SQTAMwzAFKZlGIITYIYR4wX3cDuB1ABMBXADgV+5uv4IjHBiGYZgyUTKNwHcRomkAlgA4GsBmIUSju50A7JfPtWOuBHAlANTW1s6dM2dOScb22o42NKaTmNCYLsn5GYZhysXy5cv3CiFGF9qv5IKAiOoAPAHgW0KIe4moVZ34iWi/ECLWTzBv3jyxbNmykoxv3jcfwTlHjsV33nVMSc7PMAxTLohouRBiXqH9Sho1RERJAPcA+K0Q4l538y7XfyD9CLtLOYZCJE2CZdvlHALDMExZKWXUEAH4BYDXhRA3KC/9BcDl7uPLAdxfqjEUg2kQchYX3mMYpnJJlPDcpwN4P4BXiGiFu+2LAK4D8EciugLAJgAXl3AMBUmaBrI2CwKGYSqXkgkCIcRTACji5UWlum5fSRhsGmIYprKp+Mxi0yBk2TTEMEwFU/GCIGkayFmsETAMU7lUvCBImIQc+wgYhqlgWBBw1BDDMBUOCwLDgMUaAcMwFQwLApOQ5aghhmEqGBYEbBpiGKbCYUFgGuwsZhimomFBYBCHjzIMU9GwIGCNgGGYCqfiBUHSIOTYWcwwTJnZ1tpdtgjGihcEXH2UYZhyc6A7i7OufxwPrtxRlutXvCBImAbXGmIYpqy092SRsWzsausty/UrXhBwYxqGYfrCqp1t6MlaA3pOaZUY6PMWS8ULAjYNMQxTLK1dGZz3wyfxubtfHtDzSj8lC4IykeSoIYZhiqSjNwcAWL5x34CeV5qnuzMsCMqCyVFDDMP0EacT78DhmYZyLAjKQtJtTPPAyzs4sYxhmFiE0J8LdGVyB33ejDv3dGfKMwdVvCBImM5b8PHfvYBbn1xf5tEwDHMoIBWCnz+5Hkd+5SHsbus5qPPJRShrBGXCNPIq3q4DB/dhMgwzvJEagRQEf3vZifvf0ce545+v7sS21m7vufRT9rCPoDwkzbwgGGi7H8MwwwvblQSE4uaKZ9e3BExHQgh8/Hcv4K7nNnvbsqwRlJeEUfFvAcP0mY7eHHrLNGmVE0sKAk0OhMUdbm/txqW3PhsINc1YNrKW8L1/OY4aKi8Jk7UAhukrR3/1Ibz7p8+UexiDjqwFVMys0d7jaAJrdrb7tvdkndW/WtEgn0fAzuKyoGoEbBlimOJZua2t3EMYdDxBoE0WYVOHQLj20OsmjakF5rKcWVxeEgbP/gzDFEdfNAKZnmRokkCu+tX8pXJnFifKctUhBJuGGIYplrxG4N8u1/ZCCPz+31tw2Jg6VCdNd1//zt3uZK+ahrzMYhYE5UHmETAMwxQip5mG1Cn+6399FUeMq8cX7n0FAHD/x08P7APkV/1qAmu+6Nww8xEQ0e1EtJuIVirbjiOipUT0ChH9lYjqS3X9YlFNQ8WGhDEMc+jSlcnh50vW96sJjK2lFgtl+x1Pb8T/3pOPEJKrez0w0RMEdtBZ3J21IPT05UGglMvhXwI4T9t2G4BrhBDHAPgzgM+V8PpFwT4ChhnarNjSis7egy/jIPnRo2vxrb+/jvtXbOvzsXLlrs8adohQ6XCjhgI+gpztOxfgNxP15gZfKyiZIBBCLAGgl+g7HMAS9/HDAN5dqusXS9LkqCGGGaq09WRx4c1P45N3vThg55Qr8n2dmT4fa2t5BHLKyITUKet0E8l0H0FeI8gfk1WOL4fDeLAN5K8CuMB9/B4Akwf5+gFM1ggYZsgiSy68vPXAgJ2zJuU4cfuTvJWPGvLPG5mQVbwsWS33PNCVxV9e2u5N9L48Ap8gGEYaQQQfBvAxIloOYASASJFMRFcS0TIiWrZnz56SDUiNGmKRwDClRwiB6x5chde2F85DkFPlQGrraTeapz8ROlFRQ2HmnLxpyHn+ibtewH/f9SLe2NXhOxfgFwrliBwaVEEghFglhFgshJgL4C4A62L2vVUIMU8IMW/06NElG1OSo4YYZlDpzlq45Yl1eM8thTOTpSmmv4r7HU9vwL0vbPVtS7saQdfBaASaJAgTBNKvIX0EW/c7ReakpqCag1QzUTlMQ4MaPkpEY4QQu4nIAPB/AG4ZzOuHoZqG2EfAMKVHBsVki2gRm3fO9u/H+fW/vgYAeNeJk7xtchLvz4Sbi0go6w05V0ev5V7PeS6FmhQavqih4aoRENFdAJYCmE1EW4noCgCXEdEaAKsAbAdwR6muXyxJX4kJlgQMU2pk4bZiOgNGmWIOBrkS78+EqzuLJaGmod6suy/5js14UUOqszgvCIaVRiCEuCzipZtKdc3+oGoE5YjfZZhKw3InvWLC+KWwKEYOSN/DxSdNxszRdZG/52xOdgMLn3DX7GrHhMY06qqC02OuDz6CTlcjkFOMlHuyLHVYHgFQGVFDQw61HwF3qmSY0mP1YcElV8rFaOvbD/TgZ0vW40N3/BsA0NYTnnsgNYLOkBaTQggsvnEJLr/9+dBj7YioobCS3PmoIcKn/7DCa0Qjt0flEVRC1NCQQy0xYXETe4YpOX3J6PV8BEWoBKa7k5yUW7vCgxIz7jnbQwSFHNvyTfvDxxOlEYRM3nLCNwzgzy9uC2zPqkXnlFVoOXoSsCBQTEO5fqScM0ylcbAm1D4JAmkaKiAIdrf3eAJACo+whLE1u9pxx9MbAITH/oclhqnYUc7imKghXXvoDNEIcrZAVcKZjsvRpYyLzimmIb2OCMMwQQ52vdQ3QVBc1ND8bz2Kppqk75jWrmxgv7f96Clvss+GTPphwiFsPLpkijMN6deRvgN/HoGNEdVJ9Hb0skZQDtTGNLkiwtkYptI52AVTXwSBnESLySPY70788vz7XdOQXGkD/hV/mAWgkCDwWlVq28OjhnK+8ejbs76oIRsjqhOR5yo1LAgM1VnMgoBhCnGwgqAvJthcEc5iPcrGcwa7E67sCxB1bhVVUNz82FrM/cbDvtctK9xUFeYj6PR8ASJ0u55HkE6aMKg8PgI2DZnsI2CYvtBfOZC1bCQM6pMgKaYj2IFuvwlIHiMjcaK0iUKmoesfWg3A8QsY7kmk7AhqBMHJW15fD0LJeePLb89YNpIJA+mkyeGj5SDpixpiQcAwheiPRtDWk8VhX3oQ/+/xdX0ywXqTZYwk0AWBnGgL/Z5DTUMhwqFdKYHtOYuLKDHhXSfiftXx9eZsVCcMVCdNdGctPLN2L9p6gj6OUlHxgsD0RQ1x+CjDFKI/66V9HY69/vf/3twnQSIna72mv4ouCCRZL+Io/NhincVtyvkDJSa8kNXouSPsOoBfQPTmbFQnTVQnTfz2uc34j9uew6+f2Rh5zoGm4gWB30dQxoEwzCHCwfgIhOibCVZOorGmoZDoICC6iYz+ukqoIFBW5pElJmLMOVE2fzWPoDdroSphoDqZn5IH02lc8T4CdbXACWUMUxhR5M9kza52WLbAEePrvQldiL6ZYIupNdQaoRHIJC1bCOw80INbnvAXOw6zAISZhtq686YhtQheW08WO9xs4bhJOyrDWb4XpkHozdmoSppeZVTAiS460JVFfTpR8jpoFa8RqLCzmGEKE6YRPL56d2Ci/cbfXsPX/vIqgPxEaQvRr8zi/piGVF/Bz59cj19qppZiw0dVjUAtj3HujUuwu70XQLwg6IhpsymFUW/WcnwEibwgWL5pP4679p949PXdkccPFCwIFNhZzDCFCRMEH7zj37juwVW+be09OW+ClJEwfRUE2SK09GIEwfiG6sDrckW+q60Hb+xqB1DYRyCtBrYQ2HGgx9uuRw2pJp4wZG7DfW7piZ6cjaqk4dMIZP+CkXWp2HMNBCwIFFgjYBhnUnt9R3T3sGJ/Jj1ZK1CD3+63aShaI2iLchZ7xeUsPL46vMth1rLxpusfwzk3Oq3UQ01DimlHvqwLQz2PYGJjOnK8ADCi2smC/vw9r7jHW6hOmKhSNAJZIqO5tir2XAMBCwIFmwUBw+Ar972Kt9z0JHa19YS+XmytIb8gcDUCW/jMK4XOpcbi5ywbLR29uHXJOt9xUcXlVGfwU2v3hu9jC6/ap2WL0AifMI1A9zPrpqGRtc4qPpUIn2LTKf/23hCNwDvXIGgEFe8sVmGNgGGA5Zudyptt3VmMrQ+aVIrXCGyk3TlMrpgd05C/zINaCl5HOnzX7OrA2378FKaOqsFDr+7C02tbsGZXO5Z+YVFB01AcatXPPe29hX0E7su6ANNNQw3ujY+sSWFniEBV6yDlLNstOmeiWhMcVQkDtSHCYaBhjUCBfQQMU5hiw0e7s5Y3YcoVs4A/TLvoIm8AVu1s91bvT6zZ49noD3RnQ1feuSLiwdU+ANsPdPvGM7I25Sv5sKutB7e7lUv1uULXCCY1OaahKI1ALYEtj60O0Qia66oGpXMiCwIF1ggYJk/Ur6FYQdCTtbwJ02caUjSCgoJAs8HUaBNl1rJxoDuLUbVB80kxv+ftbvinfCwn5dNnjcJjn12IaaNq8djq3ViyZg9+vmS9t69+avU+TIPwP4sPx6cWHYa3Hzc+9LqfPvtwAEBzXcpzpFclzEBdpJEh91UKWBAocB4Bw0QnYP308XX498Z9RdUasm2B3pztCQ25kncidfL77enojT2PHutfk/Jbs7uzFg50Z9FUE5wwozJ6337cBHzv3ccCAC64+Wlv+84DPZ6GcNsHTkJDOol0ysSutl584PbnYSomrDjfhmkQRlQn8elzDkfandhlZVHJMZPqcdHcSUiZhk8jMLXCSKMGwT8AsCAAAHz8rJkA2DTEMCr6XPfdf6zCe25ZGtAI1CJpMuBCNleRPylPIxDCN7lv3NsZO4ZsAY2gO+MIgrCVc9TvOWfZoSab3pztrezl67WK4EkaxdUlMxVTzrTmWud8pv961QkTqYSBjGV7gqAqYQbe8/EN8dFHAwU7iwF87tw52La/Gyu2tJZ7KAxTduQ8JiKMQ/ocqCZMWULAAPkicQDVWew3LW1q6Yodi27n1wVBS0cGWUsEBIEQIiBEJBMa04GVN+BoEFnLhmmQ97pqs1cdwnHmMbVszfnHjMfO83swoTGNj/32BW97VdL0tIG8aSgonGaOro28zkDCgsDFNAz2ETAM8t3Aola9+iTY4YuzF0iajskGADbv68ItT6zzVr1Zy/bZ/Tfti9cI9N9kQosw2tnmJl1pguD1He2hJSTu+NBJOHXGKDy+Opit+8NH3gAAz5wDALVV+cf7u9SaQ9FjVk1IRIT/XDADti1w+qxReHptCwDHDJRKGMgqGkF10gwI3+nNgyMI2DTkYhrcoYxhVKIEgW4fVzUCOXGr5qLrHlzlhWDmbIEed+Jrqklie2t4rkL+fFotf+03us09fvQIf9LVTY+uCf09nzV7DKqTpq8zoY4azqr6JNR8hTiNwAyJ8jEMwp0fPtl7nnY1gkzO9grWhWkEE5vYNDSoNNWksK8r42tCwTCViJzH1NW4mmypy4duZdK33MlXr7i5bX8+OkdOfLVVidCGLpID3Vn85tnNvm26uUeeV40aInKycsPMPxJds1BJKdm9ar8Sv0YQ7ywOwzAIBjnvX3XS8RHYAuhy36sqRRO5bP5kjBlRjcPHjIi8zkDCgsBl6qhaZHI2drT1FEwPZ5hKQJ38fUJBmwSzSuikXMHrXba2qoLA3b8mZcaGj/72uU2BbS9u2e97LsM/m+vyGsH05lq0dGYwMiSSSJI0ozUCtU6Q+h7s78pg4ezRSBhGbAmORJwAMhwHcXXS9MYgtaWqhIGGtFN64qgJDXjfKVMjzzPQsGnIZdqoGgDApgJRDAxTKfh66iomGt30rvbktTzTkH+nrfvzTmEpCNKpREAQCCG8yX1HiNnoxc2tMAi4+79OBaAIAsU0NLquynEixxjy4yZr1USjlsPY15nBxMY0RtYmYzWCOIuCtEhJHwEA7G5zQmgb0klcccZ0fPltR+LSkyZHnqMUsCBwmeo6ZTa0sCBgGMDvI4jSCIQQmkbgmoY0jaBTMRXJyT+dNAIZufe8sA2nXfcvLN+0Hy9vDY/iG1df7dnOpSBQTUNj6qtxoDuLHs08pcbyJ2I1gryJxvbVNHLyFQyi2PDRQhoBkA8fBYBt7j2MHlGFqoSJK86YHju+UsCCwGV8fTUSBvlUWIapRGRJA3Xytyx18s/vawt/4pYV4izWkX6BmhCN4CU3hPvlra3YcaAHl8ybjAWHNfv2SSYM1CSdSX37gR6MqE6gSjHnjHG1gz0dvb6GNn/75Bn5cyg+gi+99Qifs1nVCPRClI01SRgGhUYNzR7r2PPjNALTIKQSBgyDkHLHsL21GyOqEoGs4sGkZIKAiG4not1EtFLZdjwRPUtEK4hoGRHNL9X1+4phEJpqU9jfGV7JkGEqBTmNqZn2al8AdZVs2cJngonSCFQ8jSBlYuv+bm/yBxwHMgB09ubQnbWQTpmB6B/TIFQr1TubalK+SB05qe/rzHiT+pmHj8bUUflQTDVq6CNnzvBN/mopaL3Oj6MRhDuLj53U4J47TiMgr7Cc1Ai2H+gORD0NNqXUCH4J4Dxt2/cAfF0IcTyAr7jPhwyjalNoYUHAMAD8oZpWhGnI1kxDltJxKwopCGqSJjKW7SvzIM037b05dGcs1KRMn50ecDSSlJkvx9BUm/JN7DOU2HvpkNWDhPSKp+rkrTqLv/DWOb7XRtY6QidMEMhy0WZMaKppkLfyT5nO/+2tPT5ndzkomSAQQiwBsE/fDKDefdwAYHuprt8fRtamvGYQDFOpyEWwz0dgqZN/ft8Tv/GwzzRUlEZg5aOGomjtzCJnC6STZsAebwsBIvKOH1mThDr3HjmhHouPHOvci7tND+nUbfDqc1UjGDOiGjdecrz3vLEmCSLylZEGgFNnjPK0kjjzfkIRBFIY7evMoHnE4NQUimKwfQRXA7ieiLYA+D6AL0TtSERXuuajZXv2hHcXGmiaWBAwjIe6Elf9BWpCWVfG8gsCL48gOiy0N2uDyB83L8/ZlXGS03a4NfzTqaAgkM+lg7ipJuWb6A0izB7n2OvlkQFBoD1XTUt6m0m1LpF0FqssOKwZd115ineNWI3AJC9zWT3vuPryhqxHjpiI/ld5/B7ttW/383pXAfi0EGIygE8D+EXUjkKIW4UQ84QQ80aPHt3Py/WNUbUptBSohsgwlYLlCwtVfQT+/bIhJqSekEQxOcFmLBsmkc8u/7W/vIqcZXsCZM1Op4dwTSoR1AikIHDNKU21fkFgGoRGN4dAZj3rgkDO5c2uOUed2/UcgypNEKgvv/vESfjp++YCgCcgYnLVkDAM731QBcFbjxkXfdAgEKcRXKo81lfuuu2/WC4HcK/7+E8AhoyzGHBMQ209ucjytQxTCXiZxVa4RhBIKAszDWWCgkBW8szknMJuakXOXy3dhKXrWzyT0k5PIzBCTEPO/zrXsdxUk/St6ImcbUA+wklfxcuxXDZ/im+/sH1VU9GI6oTv9aMn1nvjkNvjyleYBnmakHr/c6c2RR4zGMQJAop4HPa8WLYDeJP7+M0A3ujneUqCVDX3d2WwfNN+7G6Pr4PCMIc67T1ZfP+h1b4wzrCic34fQbQg0BvRqMiIIE8QaLV1crZAdybnhX8CQDoZ1AikyUp1Fvs0AqJAfwJdI2iqTWHFV87xGsTEoYamGoY/h0D1LciHVcnoaVX1Eaj3PxhdyOKIKzEhIh6HPQ9ARHcBWAigmYi2AvgqgI8AuImIEgB6AFzZp9GWmDo3YqGjJ4d3//QZTB6ZxpP/++Yyj4phSsfNj63DLU+sw/jGarz3ZKekQVitIb+PwH+OjE9zcIRCmEYgnbu9OctZGWuCYH9nBl0ZC001Kexud0y06ZQZKDwnTUNybm9IJ30TqWMaSvqOOW3mqMB4GhVhoVb91OdkfZxdiiM8qfom3Mdx+QAnTm3C2BHV7vid/ae6VQ3KSZwgOI6I2uCs/tPuY7jPgx2tNYQQl0W8NLdvQxw8pBNnj/sl3LKPk8sYP5YtcMPDq/Hh06d7NupDGTnp7esIBklIv0BLR6/3mwCCGkEuNKEsaF6VGkGvpxH4J8zP/PElAMBxkxu9bTUpM+CTkBoBeTZ53f6f9xFMbEzj3o+d5tMywlBvKUoQyCbyasayrokA8YLg2+88xnssQ2UvPWlK7NgGg0hBIIQoX5pbmUi7dsMtnF3MRPDU2r24+bF1WLe7E7e8f8iuaQA4pQvGjKiKLbAmJyO1lLSc2nK2QCZnY+43H/EdE3QWFxc+KjWCTM5GIkQj8PZTJtJ0MqgRSGFz3KQGPPzaLoxt8K9LTYO84m0dvTmMrS+4btXwSwIpJMa451HvLaPcuxQK1RH3pTN1VC2euebNGN/Q1/ENPH0KHyWiWiJ6HxE9UKoBlROpEWzeF981ialcvISpmPLJQ4HujIVFP3gcf35xW+x+I9xVepvSXEYuiS1b4NdLNwaO0TUC1b/glaGOEQS9ORsGBX0EErUrWDpl4toLjvZf3xUEVy2chfs+fjpOnOJ3tJrkCILxDdW49oKjQq8Rh64RTB5Zg2MnNeD6i5w+x+q9qSYwqaGkY/IjdCY0psvuHwCKKENNRCkA5wP4DwDnArgHwC0lHldZkIJgqysI0mWs/cEMTYpp3D4U6M5a6MnaPpNOGPJ22nuygddytsCtS9YHj9HehC5lMpQaQVhmsRc1ZNlIGhSIzpH4BEHSxFmzx2Djdedj3Z4OLPrBE55GYhqE4xUzkoTIeW3pFxaFnr+vVCdN/OUT+TpF6v2qgkBqRofivBEpCIhoMYDLACwG8BiAXwM4SQjxoUEa26Ajv4Bb3JK53VkLD726E+ceVd4YX4bpK9JuH1fqAciv5ttVjcCloycXmmCpl6FWnaff/vvreHHz/lCNQIZNZnI2qqoToa0kAf9EqmYfS1OSXnJCJ64hTRTqGQsdrRbUU+/T6z18CAqCONPQPwDMAHCGEOJ9Qoi/AhjWAfZSEKimoY/euRyAswq6+bG12MJmI+YQwGsZGdP4BcjbuFWNQE6Ee93kSn1i1U1D6qp4874u/GzJ+lBBIF0VB7qzqKtKRNb0b6pJes2hVMerjOfXK4IGrnOQppZCh6v3e/G8fN8A6SDXM5MPBeJMQyfCSSp7hIjWA/g9gENP1PUBuRLZ1RZUp3e19eL6h1bjzy9uwyOfcVIhWjp6kTAMNGihaszwZyjYdeOwYkw0KlIjUM0dcprd60YSjW+o9pVn1+fhrkwOSZN8GcZhEXfqBF2fTob2FL7zivk4dlIjLj9tGp5Z1+IXBMniNIKD/Wg+smBG7OvyvVryubMwRQn9lBpBdeLQmyYjRZcQYoUQ4hohxEw4OQDHA0gS0YNENKTi/weKONueXBWpJSjmfvMRHHftP0s+LobpK9JeHRbGGbZfWH/ilk7nuz6hwV8HR/cRdGes0MkvpUUrqcKzIZ3EjNG1+iFYcNhoNKSTmNRU41ttA3nTUCE/TX+EtLynX314vq9cdRg/uuwEnH3EWExo9Ef7eIJgmJmGPIQQzwghPglgEoAbAZxS0lGViSiVbseBblzys6UACv+wGGYoEFfzZ+m6Fky75gHs7ej1NIKwXIAWqRFoE56uEXRmLFSHRMqMbfDH7qsmpvrqJOZOHYl7rjqt2FvyBMtbjh54n528pWQR/oW5U5tw2+XzAhVMpSBIp4aRaYiITox4aS+An5RmOOWFyKkMqNs3b3l8Hda7vYzjyusywx+5GtVXxUMNaabpDVm43LpkHQBgxeZWTxBkLYHenIXbntzgTWhS+x2vaQRhPoIwbXpGc53PRKTmM8g4/zlulVDAWWnHQUR4/kuLvGNLwcG0iOw+hE1DcT6CZQBWwpn4Ab8zXcCpFTTsSKeCgmCo24MZRidOI5AQ5ctDZCwbyzbux/UPrfZel32GdROILgi6MjmMGRFMipreXIsn1uzBxMY0zpozGhfPm4RbnnCEkJzM1aSydxw3oeB9hV1nIOlPxJHkc+fOwf6uLBYcPjjVkgeSOPH3GQBtALoB3AHg7UKIs9y/YSkEgLyf4PxjxgMI1hlhKhu5Jhjqi4Oc1yksqBHIaZwIPtNQWJ/hdNL01eQBgjb6rowValaVPoCGdBLfvPAYr8SEs815LFfgR02oDxxfDvTOZX1h1pg6/PGjp3rVSA8l4pzFPxRCnAHgkwAmA3iUiP5IRMdHHTMckCGkE5vS+PhZM5GzxUFHITDMYJOL0QjkRE4gL3zUMQ0FhUZjTTJg9tE1gt6c7XOQvu+UKZjUlEZ9td+E4/MRKOadB/77DPzuI2V2O7q3FFdCejhTUHQJIdYT0f0A0gDeD+BwACtKPbByITsX1VcnYNmOil3IHCzc1nkMM1SQoZlhwQ3e15ng9RvOWHZo2YzGmlSgpWRYGL+aDfyNC44GEeH+FU55C/nTULuCqXb+oyY0FLyfUuM5iw9CIziUietQNoOIvkhEzwH4OoCXABwhhPjjoI2uDIxzC0BVJ02vFkqmQKMajiSqHIayj7grk8NZ338cz61vie0LIB3dhPx3O2fZoWakxnQyUDsnLKFLdZB6VUHdiV8KAlUjGKrmE+MgfASHMnF60FoAF8PJMF4KYAqAq4joM0T0mcEYXDk4x2163ZuzPf9AVlOZN7gRRJLOTDA9nxne/GvVbizbuK/cw/DR0pHBhr2deHV7G7IhPgLLFrj9qQ1eOQmBfB7njMARAAAgAElEQVSBLfxJZZKm2qBpSJ77J/+Rj/KpShqYNaYO31CKvMkEMtnoRjW71A5RQVCpxH0a1yKvMdUNwliGBJfMm4xszsa7507C/Su2A0CgdeWFNz+Nl7662Hve1WtV0DvESC66ZSk2Xnd+uYfhIVf3Hb05rwqoqhEsXdeCa//2mvc8p/kF1FLUkoZ0KlDLf2+7k1+gJoylTMPLuJfo5lLV/N6XCp2DgaolVSJx/Qi+NojjGDIkTAMfPH06gHwruayWCn+g21+pMewHxAxPhrBlyPMLdPTmvKgh1Wy5YW+Hb/8bH17j++52hnyPG2uSgQY8Nz6yBoC/uFoyJLpOD8VUNQLd78CUl8p0kRdJVYSPYKaWGt/FpqGKQe+fO5TIegXkcvmoISUkdPWudt/+r+1o8xVYbA8RBE0xdbRUjSAsI9fwQm39zwGgJjm0TEPvcctZDIeuc/2BBUEM8ouuNt6YNqomoAF0hthWmeHJUM4o9pmGXEGQs4VXPuKNXR2RxwJO2Wkd2bVPNmVRURvLhHVB0x2vqqloqJmGPrZwJt741ltKmrU8lGFBEEPeNJQXBM11VdjflfVNCF1sGqoYClW+LCeeaagn6zNn7uvKYHd7T8CkqaMucKQ2LB3F75k3OdDtS022DDMNGZ6zOEhUd7JyQUSxLT2HO8V0KKsC8G4A09T9hRDXlm5YQwNZ/1wXBJmc7W9IMcTbFjIDx1CzDH30zmWoSpj40WUn+ExDltL05a03PYm9HRnMaA6vqpkyDWQs26cRLD5qHC6eNwmnzWz2tuk2f58gCDENmaTZhpghSzEi8H4AFwDIAehU/oY9+TyC/K9/VJ2Tbr+/K7+64jyCyqFQUxSdu5dvxeW3P1+i0QAPvboLf3nJiW5TTUOqRiD7CoSVkACAOreBveojaOvOYsFho32Tv37vVUruQLhpyPnPYmDoU4zHZpIQ4rySj2QIkgrJI5DOpH0d+RZ+UT8wZvgR1VUris/+6aWDvuZDr+7EqTNHBUo26EjTkKMRBMfZ0plBTcoM5AscPrYOz67f54saagvpYaz7wlJFmoYGi4+eOcOrEsz0jWI0gmeI6JiSj2QIIp3FftOQoxHIph0AawSVxGBHDa3f04GP3rkc19zzcsF9s4pGkAsZZ2/ODnWGHjep0TtOFo9rC/En6OGlaqJZmEZwMJU8+8MX3noEfv6BeYN6zeFCMYLgDADLiWg1Eb1MRK8QUeFv5TAgrMSETKVXJ3/WCCqHwfYVt7jN48Pap+r4BEFEWZQwQXD0RKfWT0dPvpx0WDP7zt4YjSCkRo8ePsoMXYoxDb2l5KMYooSVmAgTDiwIKoe+moYOFmnGCUvAUtumAvnER8sWkUmOjVpewCXzJnuZwxnLxqSmNM44rBmXnjQ5cKyuEaiZwnERNywHhj4FNQIhxCYAjQDe7v41utuGPVIQqGn4XpJZjgVBJdLf8NG+Opkl3W6yYlgHsLnffMR7fPNja7F0XYv3fH9XJrA/ENQIrj7nMF9XrnTSxLffeQyOdc1FKm85xt8iUvUBhAkCr9w1qwRDnoKCgIg+BeC3AMa4f78hok8WcdztRLSbiFYq2/5ARCvcv41ENKTLWcvVvzrRy21/e3m7t419BMObnqyFL/75FezrzPQ7fLRYAdLS0YsbHl7jCY4ojUAXLNc/tBr3vLDVe97aFZ4z0Jj2N5mpSpi+DOGqiL7dALBw9hhfbSV1ei+2fPOnFh2GGy85rqh9mcGjGNPQFQBOFkJ0AgARfRdONdIfFzjul3B6G/9abhBCXCIfE9EPABzo43gHFTnpqzkDcuXz+Oo93rZy5BHYtsC3/v463nvyFMwYzRXvSsm9L2zD757bDJMIs8b07722bIGQRX2Ar9z/Kh54ZQfmTW3CmYeP9iJ1arRqnVk7fvERKQg001AqYSCZyE/ixfTb/eWHTsJdz2/2OYNDNYKQYz99zuEFz88MPsUIAgKgznQWijD7CSGWENG00BM6uuLFGOJ9j+VKSV18hWVE9mQtvPe2Z2EQ4c4rTh6Usb2xuwO/eGoDlq5rwd8/tWBQrlmpyAqepkH99hEUG20ktU9pjpSmoRpNihQ6X5RpqF4zDaVMw1cMTm9LGcbC2WOwcPYY3zb2ERzaFCMI7gDwHBH92X1+IYBfHOR1FwDYJYR4I2oHIroSwJUAMGXKlIO8XP9ImAYMKkYQ2Hh6bUtgeylRJyemtEh/UNKkfoePhoVzhiE/T3kdGalTrfcEsOLPV6xGkDTJZxqKKzIXR5hpaAhX42A0imlVeQMRPQ4njBQAPiSEePEgr3sZgLsKXPdWALcCwLx588r2lapKmD7TUCpk5VMOZ7FcMVYNsZotwxEZlplKGP2e3Ip1FuuCIN9EJn/8y1tbfcEKYbR2ZZA0KSAwdB8BEflMQ021hTUC//HOhB/uLBbePszQJlIQEFG9EKKNiEYC2Oj+yddGCiH61Z6JiBIA3gVgbn+OH2xSCcMnCMIm3rIIAtdBHefcYwYGWWIkaRpFOX33dWawvyuDmYrvplhnsScIhOwt4Kzs1bSAd/zk6YLn6cxYaKxJBjSD+nTwJ69O4iP7KAhMIuSEqOiCbcOBOI3gdwDeBmA5/H4fcp/P6Oc1zwawSgixteCeQwDdFBRmGgqr4y7Zsq8LrV1Z7O3oxYtbWnHlmTMGpF+rNA1VFeHcYw6OvGnIKMpHcN4Pl2B3e68vwqZYk5IUBDklOcw53m0pWeA8soAcEL5o0U1MgNOX2HvcR9OQ4dpOQ01D7n9iL8GQJ65D2dvc/9P7c2IiugvAQgDNRLQVwFeFEL8AcCkKmIWGEropKCAYTAO7DvREHn/xz5Zix4EeJAxCzhY4fnID3jxn7EGPS5qGwkxVlcDKbQewq60Hi444+PeyEFllYi1G+9vdHswC7qsgkCHJMnxUagQy0ziKdMoEZZ3vh+oEllQnTDz7hUU45TuPetsSB6kRAOGmobH1Tpby/Okj+3ROZvAppgz1o0KIRYW26QghLovY/sE+jbDM6Ksq/QtfW2X6KpHqtLjF6aSzsJCTr1jkKrVSTUNv+/FTADAoPYOlIEiaRqBgm22LQAMWiSo0ihUECfdcsuud/JyXrm/Buj0doaUfVJKmgaRpoLej19c5b0JDNbYf6EEqYWBcQ3Xk8SOLiBpSkYIrTBBMb67FY59diCkja/p0TmbwiZxFiKja9Q80E1ETEY10/6YBmDhYAyw3ugZgap6v2gJmnrlTm3zP+5thqpM3DVWmIBhMVNOQPqHHxfOrk3bfNQLLd+3Xd7Rh0Q+ewPUPrYo9PmkSPrVoFgB/qXQ9D0Hno2c6lt5iwkdVpAyMSiib3lzLkW2HAHHfjo8CuBrABDh+AvlptsFJFKsI9IlWX/0Vsvfr2nmxYYSFkKYD9hGUHmlzFxCBVpU5SyDqK6CWcn5w5U5ctXBmwWtJuSI1D71fdqEwZVsIvP/UaUglDLR2ZfGdBx3BcfrMUVi7uwO1VeHfl8+fNwdXn314nzuHGTEaAXPoEOcjuAnATUT0SSFEoSziYYv+w0hogiCsGJiKHuY3UEXL5IqRNQI/3/3HKvz08XUDajKS5jzbFoHon2xElU/AX8r5u/9YhWMnNeD0Wc2R+wP5ib9b0wiKRfYkuOQkJ/dm9rgRGFVbhTnjR+DS+VMwqSncTGMY1K8+wnE+AubQoZg8gh8T0dEAjgRQrWz/dfRRw4eAaciINg2FacD6D3mg6tnLFWNYQ5BK5qePrxvwc2ZcM5wtgq0q43w+eo/gPSFO5MC1XEGgm4aKRdcg1AzgI8bXe4/vuepUxMiwopEaAecKHNoU4yz+KpzonyMB/B1OWeqnoNQQGs7oUTl616XaVP4tDFOre7Uf8sCZhrji6WChlnfWfTxhGoEM4WzTHLtx2oNETvxRpqFCFHImS+ZOHZhInvnTRuKBV3ZUbPTacKGYT+8iAIsA7BRCfAjAcQAaSjqqIUQhjaCmKr5LU8A0dBCCQG02Ik0Hg10fvxKRE7gtROD9zoVoBFURXb6KiRhTncN/fnGr7/tTaNWdMCi0j0Ap+f57jsNfP3FGnzOSmaFFMYKgWwhhA8gRUT2A3QAG99tWRlKaM1Y3/xRyFusaQX/r2d/7wlbM+tKD2NzSBQDodleMAxWFdKiiO28Lbe8P8jN0BIH/tbAVu0za0k1DfdEI1uzqwKf/8JLv/IV6Fv/5Y6fjuncfW/AaA0k6ZeKYSRWzLhy2FCMIlhFRI4Cfw4keegFOGeqKQHfG6k02VB9B2KSsTxT99RH89SWn/8Ebu9sBAF3ZvN1aRwgR2nx8OBL1dg5kb2E5gVt28Lxhk7v8hqzf01lwXwD42RPrsHZ3R+g+qlYR1mZSRa0ZxDB9oZgOZR8TQrQKIW4BcA6Ay10TUUVQKJxO1QjC7P8D5Sz20vXd33qXV3ogeL5fPbMRx37tn9iyr6tf1zqUiJpcB8oXo17DFsHw0U0twfdYCv/nNrSEblc50O2EeL7z/z2Nj965DBtburDgsGacOmOUe838vmF1glQ4cofpL3EJZSfqfwBGAki4jyuCQk4wNXw0bFIeMEGgtf3zqlIKge88+Dou+ukzsG2BVTvb8M/XdgEANrZ0hp5rOBE14Yf5Tixb4DfPbsKB7myfPgf5GYaFj67cFuytJPffur/bP1ZL4Orfv4g/Ltvibdvr9h1u78nhoVd3YW9HL2pSJt49d1LgvIVMQ+ywZfpL3BLjB+7/agDzALwER+s9FsAyAKeWdmhDg0Jx+qppSE4Sti3w3X+swn+cPGXATENyYpPKv7Q/2wL42RPrAQAzvvh33zEHaya/8eE12LK/CzdcfPzBnaiE5CI0grD3+e7lW/B/963E/923EpfNn4LvvOuYyPPe8PAazBxdiwuOn+j5CKwQH8ErIYKgN2dj8sg0tuzzC4KerIX7VmzHfSu24+J5jpstLKQ0aRqoC0n8KiQIWCNg+kvkN0cIcZYQ4iwAOwCcKISYJ4SYC+AEANsGa4DlRhUEx0wMOsXU8FEhHCGwrbUbP1uyHo++vjswIfXXWSwPk+Gr0gcQd76DjSi66dE3cO8LQ/ujjtIIwgRBW3c+tPKu5zfj+Gv/Gbqi39eZwY8efQOf+r3TUlt1zKt+oAkN1d6KXr2uZQs0hZRqaO0O+m3CCtSlEgZqUsE1WmHTEPsImP5RzBJithDiFflECLESwBGlG9LQQtZoOWFKI/76yTMCr+s+hJwtvBrwatEvSVyUz4HuLHa3+yuZrtnVjo17O32NSYD8pBYXHVMJ8URh4ZtAcZpXa1cWtz25XtuWwYnfeNi3zRMEwi9cq5NmwEchzUJhjt1tmqkICNcIqhJGaCmIws5i1giY/lHMN+dlIrqNiBa6fz8H8HKpBzZUqHV9AN2Z8AQufRVmC+H1i+3oDR4T58Q847p/Yf63HvVtW3zjEiz8/uNeDRrLFsjkbC+PIHbCc1/qyuS8uvbDDTkR//2VHZh2zQPe9jBNSRemAGBqxaD0kE8hhBehZQnhy8atSpqB3ABZDDCsnPP21rwg+PJ9K7F6Z3uoIEiZRmgxwxGuaUgNXHv+S/kiwOwjYPpLMd+cDwF4FcCn3L/X3G0VgfxB6vkAkjCNQAqCW54IljuI0wjiGtzISSxj2WhXQkPj5IBcvZ78rUdx9Fcfit7xEEYK1nuW+/scxRQF9aHPnXrCYMayPWFr2/6ooeqkEQgGkM/DTEPbFEFw57Ob8OQbewKmJcD5TtWGmYaqnW2L5uTLRqg9B9hHwPSXYmoN9QC40f2rOKQgiNIIpo2q9T23LIH9Mc1DDtZHkLOEr3RBnB9ACok4AXOos3pnG5ImoV4zm+SKlAS6RqAfpn7utvBHDVUn8qahtp4s6quT3oIhzIyj9zLozdmh36vqpBmqEdSnk/jH1QswdWQtjvnaQ8jZAglFI+Vyz0x/iQsf/aP7/xUieln/G7whlhe5MusOqe3z+GcXYly9v8mHJUSgUc133nUM7v3YaUgljKIKfYXZ/T1BYNs+80WchjGQ2bVDlf/6zQs447uPBSbe/moEen8BdfK2bL8GVp00kLVsPPLaLhz7tX9i+ab9niAoptNXJmeH1owaUZ0IrWqbMA3MGVePdMr0KoUmQ7qQMUxfidMIPuX+f9tgDGSoIp12YT9Y06BAf4KcbXumIUl9dRInTmmCSeT1ntXpVFbtXRkrsCKUK/+sJXzZpkW4CHw8u74Fu9p6cMHxw6u3kK4RFN0sXssU153PXZpGoAreatdH8MArOwAA6/Z04OgJTmRZMb1/s5YdusCor06Ghi2rJdBrUwm09+RgGoQ/XHkKnl67t+D1GCaKuH4EO9z/mwZvOEOPuhgfgVTFX/zyOfjby9vx5ftfhWUHNQJp3zcNitQIVKfh7vZe/OLB1/G5xXOUczjkLNsXjRQ34YVpBJfe+iwADDtBUK217Cw2X0N33utRQLppKCxqaFebE+m1akc7/vduR1kuFOEDxGkEyUApE8AvCKTGkDQJJ88YhZPdTGSG6Q+RgoCI2hG+qCQAQghRH/LasCOuxZ/8YTbVplDlFhqzbIFWTSOQgsE0CLYQ2HGgG6d+51/45YdO8urFy8kEAH7z7Cb85tnNSJl584Cc1LO28LqTmQbFmn8GsszCUMfSVvJhgoAQnFzlinzt7nYsXb8Px2q5Ij6h6+YISKRpSH52D726EwCw4LBmnFLExJyxbO+zVBlRHf6dU53B6ZQJ06BQgcEwfSUuoWyEEKI+5G9EpQgBAKgLid6QqGYhKRQsW2Cf5iye4DYLNw1CzraxfNN+AMCfluUjXf6glB2Q2bJqRImcf7I52wtRrEmasbbwgSy8NtTJ6ol7IfceFj4qV/wX3vwMvnzfykAmeFdW1wjyr1UlTNgCWOcWl9vW2o3JI9O484qTvQqkcfzh31vw2o62wHYpCE6Z4e8ZYGoaATuHmYGiaE8TEY0hoinyr5SDGkrURPR4BfyquvxRqgllAHDh8ROw6IixAJysYMtWm6Hnj3/yjb2Y2JgGkI/yUX0NnmnIzq8i0ykz1jRUTP37ctKbs3D7UxsiBdY/X92Jadc8gJ0HekJfV9F9L8VmVUuNQOZZ6D0EfKYhG1r4aPC7IT/DYogKSZb5AjddegLeM3eSZ2ZSI4TSqQSSLAiYAaKgICCidxDRGwA2AHgCwEYAD5Z4XEOGuNhsI0QQ2JpGcLRiajAN53U5AaiN57t6c5g6yuknK00NezsUQaA4i6VduSZlxpuGBqIXYQn56ePrcO3fXgvkAEh+/29HSwqr56OjO3mL1Yb08E09oUx14lta+GiYQ3dCHwRBFDJfYGx9Na5/z3HeddScgdqUiQTnDTADRME8AgDfAHAKgEeEECcQ0VkA3lfaYR0aJEJMQ50ZyxcJok4WCcNAzs0MBvLJaLYt0JmxMNYNRd3RKgWBahpyJqCclbdTp1OJ2Kgh3VyiIoSItC+/svUAHn59V/SJBwipOUXlOci6SsWs7nXtp1j/iO6sbdUc/fKzTCdNp9aQL3w0XiO47l3HIGsLfPm+lUWNRTJCKy4nL+nXCEzf949hDoZiBEFWCNFCRAYRGUKIx4johyUf2RDiGxcejdljRwS2mz6NwJnU/6VNoKpGYRjOpKYLAjnZjKmvAgDscE0h6upUrnhztpPpmjAIKZNiV75WjEaQsWyfRqLy9p885XseJzRKiXzriunCpieQSeEx4wsPYGJTGl97+1Ghx+nhm7pGIMNHa6sSIeGjwRX56BFV3uNL509BT9byBMGPLzsBG/d24gcPr/EdM3VUja+vgX5eKQfViX/h7DEFu+MxTLEUo1u2ElEdgCUAfktENwEY/oXuFd5/ylTMnx5s9q3GoMtJ60f/WuvbRy1B4eQRCM8hKV/rdCNTRrlJSHJyUssXyGMc05CNqoQBw41CiiJuVRxlnw6jVE7nQrJFCtpicgJ0jcArCyGALfu6ccWvloUeJyd6ORZdEDy73mkuU1tlwgopOgf4J2g9bNRQbvLtx00I7e178bzJ2Hjd+VhwWLM7Fv8bI81/qmnoHcdNwLUXHB16TwzTV4oRBBcA6AbwaQD/ALAOwNtLOahDhTCNQEfVCJw8AoFed6KXr3W6xeniYs+zniBwooaqkyYMihcEcc5ivUZOHMVMxG09WUy75oFIe38Y8rRRfo68aaiIMeoaQchB+qbalOmZhrzy3pogeHz1HlQnDSQMchrThGgEZowg0CN7wvwKctsvLj8JL39tceD1MNMQwwwkcSUmbiai04UQnUIISwiRE0L8SgjxIyFES9RxyvG3E9FuIlqpbf8kEa0ioleJ6HsDcRPlQl25qavCOePyZqQwQdClNZ6XDsk4QSAjhXJu7Hl10oRJVCB81P+imizVJ0FQxEy82TVt3PbUhqLPWwg5OXdncr48izB0Z3GYNqTb/yePrPHeVyNCIwCAmlTCywFRw0ulaS1OEOhm/KoQv4K8z1TCCG0+Y3saAQsCpjTEaQRrAHyfiDYS0feI6IQ+nvuXAM5TN7iO5gsAHCeEOArA9/t4ziGLagL4x9VnerVm1BWgQQRLCM8UJCdmKRhGVCcDE4dE9ijO2gI9OQtVCQNE8Y5UXSNQI2D6axq678VteDTEkSzH0Ze5qljT0OfveQUnf/vR2H0DeQQh78tuRZiMb6jG6bOavYldJpuFCYK0q33pOSKy7HOcINDNPNUhGkFU32VJ3kfAUUJMaYhLKLtJCHEqgDcBaAFwu7uS/yoRHV7oxEKIJQD2aZuvAnCdEKLX3Wd3/4c+tNDVdmnuUDWChOmYF2Sfgo0tnejszWH1rnYATjhoVEhgpysscpaN3qyNKndyivMD6A5UtSdBXzQC9TRX/2FFqL1dDsPoh1NZn7O37u/CtGsewPMb/F8fdcLUBY4eKhtmGtqpCILrLzoONSmnRITjDHe2yy5i5x41Nn8tA54Zbp8S0pv0wjqjBYHO7HEjMGtMna8WUWFB4GoEbBpiSkTBJYYQYpMQ4rtCiBMAXAbgQgCv9/N6hwNYQETPEdETRHRS1I5EdCURLSOiZXv27Onn5QYPuSqUq0Q5DenO4pwtvJX5Q6/uwtk3POFFldRWFU4SylnC9REYMA3ysoyj9lXpryAopqSzpxH0w3yhZ/NKAaDW7wf8oZ667T3KWayyQ0lMMwznsxLC2Vd3FqvVQ23buV5P1vaFusqEQHUsevE7namjavHIZ97kO3+mQOIf+wiYUlNMQlmCiN5ORL+Fk0i2GsC7+nm9BICRcPISPgfgjxQRlyiEuNXtkzxv9OjR/bzc4CEjiNJa+WA1e1hG+agF5tTJqSZlFmw36NQaypuG1Am9NmVi1pg677muLXT0qKahaAGiU4yzWPTRNPTSllbc8fRGZyxaCGeUVqGGeupfG90fYmlNZABg8758iKZJ5L3XW/d3e4JECgI1R8AWAgbl8zq+/c5jsPG680NNQ2EJiB88bRru+JB/zaNqLKcWqkvEpiGmxMQVnTsHjgbwVgDPA/g9gCuFEAcTOroVwL3C+YU+T0Q2gGYAQ3/JXwA5GaTdCUTOQSlfQhnhuQ37IlfjdVWJgj/2bM5xFo+qc1aUqq2fiHxmCt1c0t5PjaAYZ7HUPoo1DV10yzPeY91fEXWKnkx+v6RBUCs66ULPEiK4TXluGuRN2gu//7i3Xb4vqiCwbAHDIK/RvFzNy+MLfWZfe0cwh0EO5XcfORmnzowXBDabhpgSE5eR8gUAvwPwP0KI/QN0vfsAnAXgMdfPkAIwLAqpe4IgJQWB8+NV+8gaRLETcDplehpE0qTQ8M+c7YaPJkxkLBtZ7XzqZKE7UAfCWRyFNO/o9f2jUE9Z7FhUjUAXOLqd3bJFZGN7wNHOUjETa9qnETj3JR3FzXV+QWAYwK3vn4stIc3po5DvaViUkI68i/74XximGOL6Ebz5YE5MRHcBWAigmYi2AvgqgNvhOJ1XAsgAuFwMkzZaXtkHqRG4230+Aq02kTrBnn3EWFQlTG8ir0klQiNYZEJZddJAzrYD9nU1nyHnOkIlUYLgt89tQjpp4l0nToq9tzikgCtmrtrV1uM7p26mispiVktC69FS+qRvCxHr2zCJYutIqdm9jmkoP6YxI5xSIFJoJwwDi48aF3muMOT4dVNiGPOmjcSSNXs4fJQpGSXLURdCXBbx0iFfp+jwsXVYs6vDt82rSSN/2O68pOcRSBKaIPjcubOd/d2JvDZlhgoCp/qohaqEia6MhV6lnv2oupTP2ZzTEqBk4hrgd9B+6c+Os/pgBIFckRezatVDQXu1mvxRZ1A1At1voWs/OSteI1BNQ2GompxjGnIen33EWEwe6dQTkmfvTzlo+Z4WU676p+89EZtauoral2H6A3uf+sFfPnEGXvqqPwN0jFtjZvGRY33b4wSBiuw45WkEEXVksjmB9p4cRlQnYBCh152A508fid/+58m+a+Qsf7XM3YqTWnfQxmELx/H640ffUM7tn7ylhlHIn9kZUmCu17Jh2wJ/eWm7L4JHp8fXG8D/2ktbWgNjjgutNQ2KdMwnTfKF8ap9CI6f3OBpLHIyL9Ycpo9PXqsQtVUJHDmhYlqAMGWAq1b1g+qkGVidTR1Vi+e/uMgrOhYVPiq36XOUbiKoDTEZNNdVYWdbD7qzFppqU9h+oNszybz/lKmY1FTj8xHIAnWSW55Y5z2WGkExlrmcLbCxpctXLK07a2GEMllmi3QWy6Y8Kr1ZG39avgWfv+cV7O/MoLmuKuRIoFtxFhcat2X7NZlUwvD5Z8wYH0HCMHyC+rAxdZ6gUz93WXL6igXTY8cSxjlHjsNdz29GbUzjI4YZLFgjGEDG1Fd7q8V8oTB/+CjgZBvr5hY5Icj5TW9eDwAzmmu9EMjGmqRv0lXt1ZKcFb0qluaYsObpOpYtsGKLfwLX6/jLSbaQINivtfEEHB/BVtfRGva6d80YjSA4ZtVYGn4AABaBSURBVNvnQK7RBKtB5BPSAEIbwIxvqMZtl5/kaVA1ysTdkE5i43Xn4+J5k+MHE8K1FxyF5764KPRzZpjBhgVBibj1A/OwaM4YX/RJwhMEps+RSRQsPVwTslKc3lzrPW6qSfkmXekkTug+ggg7+Zpd7djd3uNV14zDsgVe2OQ3vXQFBIEs3uaYjRZ871/4+ys7AucKi4TqVZq4VyfNSCevz0dQQBLo/YX1lXeYj6DJzfZNmoYnQM+aMwYja1OeBpVODcxPJmkaXv8Jhik3vBwpEafPasbps5p926RGUJ30m4bSSTOvSbjbakNaZM4Y7RcEPp+D1AjU8FHLDk0GWzRnDB54ZQfufXFbUfkEli2wYa8/feSjdy7HQ58+U7lW3jTU2Wthy75ufP7ul/HWY8Zr5wperzeXb79ZnTAinbw9rvApxpy1rbXblx+gawRhUUNNtSlsbOlyKo0Kv/1falBpdtgywxDWCAYROanopYjVyUVOcgU1gtqkz6ma9DQC1W5vh66cT5vVjPaeXNFJZbYQ2Lq/y7dN1keSeMXbiLwm8T0h2cthGkFPxvJCSA0jutmO1AiKKUv9q6WbfM9157thBLOAm2qc/ICEQZ4wksI2rxHw2okZfrAgGEQSnkag2auVlb2nEYQ4i9XyEbppqCqkNn53xgr1EYzro0kikxOBuj8qu9t6cP1DqwE4piE52YcmxIUUWNu6vwvd7oq7N2tH+jXygqD4vAZJjfaeO87iCEFgGoHSz71Ky0qGGW6wIBhEVGexb7uyspdznG7KAPwaQWNN0he2KG3gqmmoozcXWoVTbacoiYuF39nWHdvkRgoBIGib11En+ROmNOLr7zgKnRkL63Y7eRk9WSvaR+CahvrTMS3UNJTw37P0ESTMfFVX030/ZdQQCwJmOMKCYBCRE7euEailiz3TkGLKkCWLiQi/+vB8XDJvMqoSpi9mf0S1KwiUCb0rQiNoqgmWNYiLZ9+4tyt0uxQy6hUeXbUbz2/Uq4/nUQVKyjRw9EQnPv61HW0AHHNSlI+gN2fh7uVbcfp1/4o8fxRhWphuGpJN42XvASD/mQ20s5hhhhJs8BxEpPlG1wjOVcoT5J3F+Y/mic+e5dXSf9Pho/Gmw51qrKppSO6vJkJ19OZCnbNhfXOTMZlgatVOAJjYmMa21m5kbRtVhhnIBP7vu170jeFAdxY5y8bUUbU+05BlC8xorvMd25MN92sAzqr8s396KXKcceiCzqSgaaha+XxkRM+kphoAeU2NfQTMcIS/1YPIaDdRSq2l859nTMfHFs7ynnt5BIopo6EmiYaQVbxfEDj7q6aL9p4clq4Lhoc2htTMjwv/39TS6Y2pM2N5yW8/e2I9PvnmWdEHAnjHj5/Cejfi6InPLfSVgjj7yLGo0aKjerIWshGmoULO7QWHNePJN8JrGOo5A6YZ1Aik1tBcV4V3nzgRjekkFh0xxrcPm4aY4QgLgkFkbIOzyjzQncVDV5+Jl7a04uKTwpORwqKGdKQVKGUaXv9caSKSfPn+VwPHJUwDDemkr5ZRXDmGTS1dGD2iCkIIdGYsz95+w8NrMG9qk6/hjc56Jex0b0cGOctGwiA8/6WzPRMVUV4A9mTtyNyHsPIUkvnTR+K6dx8baTbSJ30zJKFM7jOqNgUiwtlauRCABQEzPGGD5yAio3VauzKYPW5EqBCQoZdheQQ60vms7juiiLLGQNBPIAVBmHO5pTODSU1pyHJwqr29M2P5+vjGkTAcJ2zCJIx0J1siQnUif77eXLhfAwC2t/ob2P/f+Ufgv940E4AzsU9sTGPjdeeHHhvQCAwKmIukYJS9HsLQE/8YZjjA3+pBRNqdw6qKSvJRQ8VoBFIQ5PfVNYIo9PNL232UWWbaqFrPfKRG4Fi2HVsWQsU0CFnLDvgj1Mm1J2v7ooZGKPe2q90vCFIJA9Oba7xzx6ELAiMkoUzex6iIWkdAdIlshjmUYdPQICI1gv1dhQWBXOVfNn9K5L5y7qtTBUGRtWv0la0tgC37uny9dFUuOH4Cnlnn2N9V80jWEtjXGX0/OjlLBDptqZO0rhG8afZoHDWhAfev2IZVO/1JbETkCcNC87PuGDYNgurmvueq0zCpKY0Neztx2UnB93zm6Fqs23MwzfkYZujCgmAQqU8n0FSTxGcWz47cR4aPViVMrPz6ubE2aWka8gmCIk1DYbXtF3zvMbz0lcUhewNnHjYahGBfZtkf4YozpmPFltbQ6qKSrOWs9hNmcHUu6claPh9ByjRw1cKZeHZ9S0AQGJTXBPx9g4Pd3fTVv0H+1f3cqU0AgJ9/YF7o2P/88dNxIEaAM8yhDJuGBhEiwotfWYz3nzI1ch85fSUMQl1VItbk0RfT0G//82Tfcz2EVdKVDXfIGgZ5GohqGurKWMjkbKQSRmjWsErOFshawtc8B8hrQYA0Dfl7CwNB0w7g3L98XRUm/xGiRemCQBUCY+ujTUGS+uokJo+sKbgfwxyKsEYwxJCTYjFdr2SyU10RgkA3BUV1u9qjNK/RkZOnqqV09OSQsWykTCOwCjfIXxcoa9nIWbaXrRuGnlkstQfdtAM4958XBPntX37bkTj7yLF4/y+ez58n4v3840dPxbRRPMEzlQ1rBIcwclGrloyQpqGUaeATZ83CKNfmr/cJiBIE24powK4mVbW6ju8qt4eyiu6QtmyBrC1ik9d6cpYvoSyhaQS1KdOL9iHKC0P1/hKmgQWHjfadNyraZ/70kRjD5aCZCocFwRBDOocbQxLIdNp7HDPORLdTFpDXCDKWjc+eOxuzx40AALT1+E0+URPjG7s7AtvkvnL+VjWC/W7oaEqp4Z8/zi9snD7CdsBZLBlZm0J3xvJpFnJfqRE01qS8nAmDyNM44jSoD5w6FZeEOIAZhnFgQTDE+O9Fs7D2W28pKnx0xwFn9T5BEQRy8v3Q6dMAAOcf6/QDGKMVmqtKhGsEa7Ty0gBwnlsCQzqLVR/Bv1btds8X7COgF3pzTEPCVyobyOdONNel0N6Ti9UIGtJJz79hGPlKpHGd0f7nnNmhPgaGYRzYRzDEIKLIFbPOjgNOXP34Rr9pY8N33urZ89978lScfcRYjK2vxjcuPBoz3eY2URPjG7v8GsE/rl7g1QOSi241ami361OoSpiBGkG6IMhJ05AZ7ixurqvCml0dvjaYh411NBrp7K1PJ9Da5QoCyjeQMWI0gjifBMMwLAgOaeTKWDUNAcGkJ5nIpkYrRa2g1+xu9zl554yrD5w3LKQ1lTB8PYIBv8AA8s5iPXxUIpvWt3bnE9TefuwE7/wAUFeVRFUybxqS14yb66McxQzDOLAgOIT58WUnYun6vf3qfRsxF0MIJ7N2b0cwekhOp2GO5lBBoO1n2cIpMaGHj7r/PUHQlcXhY+tw91WnecJECoL66kTeNKT4CMIEm+l2OysmAothKhk2nB7CjGuoxjtPmNSvY+XEOao2hWveMsf3WnNErR0514aZrsKcxbogkM5iPabfu+4I57qtXVlUJUzUK8lxKfeaVUnTEwoG5WsjhZV+uP/jp+NjC2d6gmf5/50del2GqXRYI6hQDM+HMAWHj/X3BJgxujaQxQvkJ9uw8M+qZNBZrJuGnt3Qghc2t2LhbH9op0Q1DY1r8Gs5cvKvShieRkCKjyBMthw9sQFHT2zwno+qq8I/rl6ArfsKh8gyTCXBgqBCkeYSWyAQxXPMxEaMb0jj2EkNvu1GAY3gzivm46JblnrbdI3g3he2OcdHmGpkz+CerB1wKEurU1XS8ExTplFc+KjKnHH1Pr8HwzAlNA0R0e1EtJuIVirbvkZE24hohfv31lJdn4lHTpyWCNrQZ42pw5ffdiQuOH6ib/sNFx+Ps48YgzluboJKKmFg3rSRuOHi47xtYX2XAaCz1/I9v3LBDADAeEUL0E09vTnnmKqE6WkEQghMccs+HDmeJ3eG6S+l1Ah+CeAnAH6tbb9RCPH9El6XKQI5z9pKs/kTpzTiPfMmY9GcMaHHHD2xAbddfpI3KavIvAQ1LLXaFQSytaVEL8P9kTNn4CNnzsCutnyZ6ec3+Psey+bxjmnIOW/GsrH4sGb87ZNn4KgJLAgYpr+UTCMQQiwBEN3FnCkrsjSDLYRXGqI+ncRl86fExuQD4T4CKQDUmkDSNLT4KH+nr7ae8CqedTEltHuzeUEgryW3HT2xgfsEMMxBUA4fwSeI6AMAlgH4HyFEdN1ipmRMb651/9d5JR10X0EUYYJCmmtUjUCu3HWfQFRjnpqUiUvmTUZTbQpv1rSSvGko7yzOFKh2yjBMcQx2+OhPAcwEcDyAHQB+ELUjEV1JRMuIaNmePXsGa3wVw+KjxuGeq07FZfMne6Yh3UEbx+fOnY37P36691xOzmpJbLlIdxrI5I9t7wkvdU1E+O5Fx+Kat8zB/Okjfa9dccZ0TG+uxVuOGe85i3uzQRMVwzB9Z1A1AiHELvmYiH4O4G8x+94K4FYAmDdvXnRndabfzJ3qTLanzRyFiY1pfPysWUUfq+8rNQG1F7Kc+4UQSJgGMrn+r+BnjK7DY59dCCAvdHoO4nwMw+QZVEFAROOFEDvcp+8EsDJuf2ZwaKxJ4elr3nxQ55CCoLEmn4wmNQIhgKRBkIUjZEG8/nLCFKebmJ7/wDBM/yiZICCiuwAsBNBMRFsBfBXAQiI6Hk5VgY0APlqq6zODi1cmOq1qBPlcBRmi+pEF0/Gl8488qGudd/Q4LPncWZjCDWUYZkAomSAQQlwWsvkXpboeU15kIbmognJSEBRTXrsYWAgwzMDBmcXMQXHbB+bhry9vD33NMw1BeJFJtVXhSWYMw5QPFgTMQXH2kWNx9pFjQ1+Tsf1CwHMUD5RGwDDMwMHVR5mSoQajyph/1ggYZujByzNmwHnfKVPw2vY2JWooH/3bn94JDMOUFhYEzIDzzQuPAQD8a5WTNjJrTD7MU7a9ZBhm6MCmIaZkvHnOWNxz1al4n9Iic2x9VRlHxDBMGKwRMCVFZi9LuDgcwww9WBAwg8KNlxwHs8iidgzDDC4sCJhBob+9lRmGKT28RGMYhqlwWBAwDMNUOCwIGIZhKhwWBAzDMBUOCwKGYZgKhwUBwzBMhcOCgGEYpsJhQcAwDFPhsCBgGIapcFgQMAzDVDgsCBiGYSocFgQMwzAVDgsChmGYCocFAcMwTIXDgoBhGKbCYUHAMAxT4bAgYBiGqXBYEDAMw1Q4LAgYhmEqnJIJAiK6nYh2E9HKkNf+h4gEETWX6voMwzBMcZRSI/glgPP0jUQ0GcBiAJtLeG2GYRimSEomCIQQSwDsC3npRgD/C0CU6toMwzBM8SQG82JEdAGAbUKIl4io0L5XArjSfdpBRKv7edlmAHv7eexwgO+f75/vv3KZWsxOgyYIiKgGwBfhmIUKIoS4FcCtA3DdZUKIeQd7nkMVvn++f77/yr3/YhnMqKGZAKYDeImINgKYBOAFIho3iGNgGIZhNAZNIxBCvAJgjHzuCoN5QohKVtsYhmHKTinDR+8CsBTAbCLaSkRXlOpaBTho89IhDt9/ZcP3zxSEhODgHYZhmEqGM4sZhmEqHBYEDMMwFc6wFQREdB4RrSaitUR0TbnHM1AQ0WQieoyIXiOiV4noU+72kUT0MBG94f5vcrcTEf3IfR9eJqITlXNd7u7/BhFdXq576g9EZBLRi0T0N/f5dCJ6zr3PPxBRyt1e5T5f674+TTnHF9ztq4no3PLcSd8hokYiupuIVhHR60R0agV+/p92v/8rieguIqqupO/AgCOEGHZ/AEwA6wDMAJAC8BKAI8s9rgG6t/EATnQfjwCwBsCRAL4H4Bp3+zUAvus+fiuABwEQgFMAPOduHwlgvfu/yX3cVO7768P78BkAvwPwN/f5HwFc6j6+BcBV7uOPAbjFfXwpgD+4j490vxdVcMKa1wEwy31fRd77rwD8p/s4BaCxkj5/ABMBbACQVj77D1bSd2Cg/4arRjAfwFohxHohRAbA7wFcUOYxDQhCiB1CiBfcx+0AXofzw7gAzgQB9/+F7uMLAPxaODwLoJGIxgM4F8DDQoh9Qoj9AB5GSG2ooQgRTQJwPoDb3OcE4M0A7nZ30e9fvi93A1jk7n8BgN8LIXqFEBsArIXzvRnSEFEDgDMB/AIAhBAZIUQrKujzd0kASBNRAkANgB2okO9AKRiugmAigC3K863utmGFq+KeAOA5AGOFEDvcl3YCGOs+jnovDuX36Idw6lXZ7vNRAFqFEDn3uXov3n26rx9w9z9U7386gD0A7nBNY7cRUS0q6PMXQmwD8H04hSt3wPlMl6NyvgMDznAVBMMeIqoDcA+Aq4UQbeprwtF7h2VcMBG9DcBuIcTyco+lTCQAnAjgp0KIEwB0wjEFeQznzx8AXP/HBXCE4gQAtTi0tJkhx3AVBNsATFaeT3K3DQuIKAlHCPxWCHGvu3mXq/LD/b/b3R71Xhyq79HpAN7hZqb/Ho454CY4Jg+ZKa/ei3ef7usNAFpw6N7/VgBbhRDPuc/vhiMYKuXzB4CzAWwQQuwRQmQB3Avne1Ep34EBZ7gKgn8DOMyNIkjBcRD9pcxjGhBc2+YvALwuhLhBeekvAGTkx+UA7le2f8CNHjkFwAHXhPAQgMVE1OSusBa724Y0QogvCCEmCSGmwflc/yWEeC+AxwBc5O6m3798Xy5y9xfu9kvdiJLpAA4D8Pwg3Ua/EULsBLCFiGa7mxYBeA0V8vm7bAZwChHVuL8H+R5UxHegJJTbW12qPzjREmvgRAJ8qdzjGcD7OgOO2v8ygBXu31vh2DwfBfAGgEcAjHT3JwA3u+/DK3DqO8lzfRiOg2wtgA+V+9768V4sRD5qaAacH/FaAH8CUOVur3afr3Vfn6Ec/yX3fVkN4C3lvp8+3PfxAJa534H74ET9VNTnD+DrAFYBWAngTjiRPxXzHRjoPy4xwTAMU+EMV9MQwzAMUyQsCBiGYSocFgQMwzAVDgsChmGYCocFAcMwTIXDgoBhFIjoS25Vy5eJaAURnUxEVxNRTbnHxjClgsNHGcaFiE4FcAOAhUKIXiJqhlPd8xlwf21mGMMaAcPkGQ9grxCiFwDcif8iOPVsHiOixwCAiBYT0VIieoGI/uTWfQIRbSSi7xHRK0T0PBHNcre/x62b/xIRLSnPrTFMNKwRMIyLO6E/Baes8SNw6tY/4dY1mieE2OtqCffCyULtJKLPw8lgvdbd7+dCiG8R0QcAXCyEeBsRvQLgPCHENiJqFE7ZaIYZMrBGwDAuQogOAHMBXAmn1PMfiOiD2m6nwGlo8jQRrYBTw2aq8vpdyv9T3cdPA/glEX0ETtMkhhlSJArvwjCVgxDCAvA4gMfdlbzewpHgNHS5LOoU+mMhxH8R0clwmuksJ6K5QoiWgR05w/Qf1ggYxoWIZhPRYcqm4wFsAtAOpy0oADwL4HTF/v//27ljE4RjIIrD71UWWugMgns4hbiAU4gLWDqEK7iBWOoQ7iAIZ5EDUyqI/+J+X5MmB0n1uARubHvR1ay69Zx75hFxiYidWqfRjz4GBkdHALxNJB1sTyU91aZVbiStJZ1s3yNimc9FR9ujrNuqTbqVpJntm6RH1knSPgPGahNCr3+5DfAhPouBH+k/lYc+C/ANnoYAoDg6AgAojo4AAIojCACgOIIAAIojCACgOIIAAIp7ASk4f2RpzNUVAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "plt.plot(average_mae_history[:,0],average_mae_history[:,1])\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Validation MAE')\n",
+    "plt.ylim((14, 20))\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's plot this:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "According to this plot, it seems that validation MAE stops improving significantly after 150 epochs. Past that point, we start overfitting.\n",
+    "\n",
+    "Once we are done tuning other parameters of our model (besides the number of epochs, we could also adjust the size of the hidden layers), we \n",
+    "can train a final \"production\" model on all of the training data, with the best parameters, then look at its performance on the test data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Get a fresh, compiled model.\n",
+    "model = build_model()\n",
+    "# Train it on the entirety of the data.\n",
+    "model.fit(train_data, train_targets,\n",
+    "          nb_epoch=150, batch_size=16)\n",
+    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 51,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1.7991065979003906"
+      ]
+     },
+     "execution_count": 51,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_mae_score"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are still off by about \\$1,800."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Wrapping up\n",
+    "\n",
+    "\n",
+    "Here's what you should take away from this example:\n",
+    "\n",
+    "* Regression is done using different loss functions from classification; Mean Squared Error (MSE) is a commonly used loss function for \n",
+    "regression.\n",
+    "* Similarly, evaluation metrics to be used for regression differ from those used for classification; naturally the concept of \"accuracy\" \n",
+    "does not apply for regression. A common regression metric is Mean Absolute Error (MAE).\n",
+    "* When features in the input data have values in different ranges, each feature should be scaled independently as a preprocessing step.\n",
+    "* When there is little data available, using K-Fold validation is a great way to reliably evaluate a model.\n",
+    "* When little training data is available, it is preferable to use a small network with very few hidden layers (typically only one or two), \n",
+    "in order to avoid severe overfitting.\n",
+    "\n",
+    "This example concludes our series of three introductory practical examples. You are now able to handle common types of problems with vector data input:\n",
+    "\n",
+    "* Binary (2-class) classification.\n",
+    "* Multi-class, single-label classification.\n",
+    "* Scalar regression.\n",
+    "\n",
+    "In the next chapter, you will acquire a more formal understanding of some of the concepts you have encountered in these first examples, \n",
+    "such as data preprocessing, model evaluation, and overfitting."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From 024733581dd9aed443b9131b9c5993d99e7250bd Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 21 Mar 2019 08:57:35 +0800
Subject: [PATCH 31/46] Add files via upload

---
 keras/3.7-predicting-house-prices.ipynb | 797 ++++++++++++++++++++++++
 1 file changed, 797 insertions(+)
 create mode 100644 keras/3.7-predicting-house-prices.ipynb

diff --git a/keras/3.7-predicting-house-prices.ipynb b/keras/3.7-predicting-house-prices.ipynb
new file mode 100644
index 0000000..0f37622
--- /dev/null
+++ b/keras/3.7-predicting-house-prices.ipynb
@@ -0,0 +1,797 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=8g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Predicting house prices: a regression example\n",
+    "\n",
+    "\n",
+    "----\n",
+    "\n",
+    "\n",
+    "In our two previous examples, we were considering classification problems, where the goal was to predict a single discrete label of an \n",
+    "input data point. Another common type of machine learning problem is \"regression\", which consists of predicting a continuous value instead \n",
+    "of a discrete label. For instance, predicting the temperature tomorrow, given meteorological data, or predicting the time that a \n",
+    "software project will take to complete, given its specifications.\n",
+    "\n",
+    "Do not mix up \"regression\" with the algorithm \"logistic regression\": confusingly, \"logistic regression\" is not a regression algorithm, \n",
+    "it is a classification algorithm."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The Boston Housing Price dataset\n",
+    "\n",
+    "\n",
+    "We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the \n",
+    "suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
+    "\n",
+    "The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in \n",
+    "total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has \n",
+    "a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, \n",
+    "others between 0 and 100...\n",
+    "\n",
+    "Let's take a look at the data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import boston_housing\n",
+    "(train_data, train_targets), (test_data, test_targets) =  boston_housing.load_data()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(404, 13)"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "train_data.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(102, 13)"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_data.shape"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, we have 404 training samples and 102 test samples. The data comprises 13 features. The 13 features in the input data are as \n",
+    "follow:\n",
+    "\n",
+    "1. Per capita crime rate.\n",
+    "2. Proportion of residential land zoned for lots over 25,000 square feet.\n",
+    "3. Proportion of non-retail business acres per town.\n",
+    "4. Charles River dummy variable (= 1 if tract bounds river; 0 otherwise).\n",
+    "5. Nitric oxides concentration (parts per 10 million).\n",
+    "6. Average number of rooms per dwelling.\n",
+    "7. Proportion of owner-occupied units built prior to 1940.\n",
+    "8. Weighted distances to five Boston employment centres.\n",
+    "9. Index of accessibility to radial highways.\n",
+    "10. Full-value property-tax rate per $10,000.\n",
+    "11. Pupil-teacher ratio by town.\n",
+    "12. 1000 * (Bk - 0.63) ** 2 where Bk is the proportion of Black people by town.\n",
+    "13. % lower status of the population.\n",
+    "\n",
+    "The targets are the median values of owner-occupied homes, in thousands of dollars:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([22.6, 50. , 23. ,  8.3, 21.2, 19.9, 20.6, 18.7, 16.1, 18.6,  8.8,\n",
+       "       17.2, 14.9, 10.5, 50. , 29. , 23. , 33.3, 29.4, 21. , 23.8, 19.1,\n",
+       "       20.4, 29.1, 19.3, 23.1, 19.6, 19.4, 38.7, 18.7, 14.6, 20. , 20.5,\n",
+       "       20.1, 23.6, 16.8,  5.6, 50. , 14.5, 13.3, 23.9, 20. , 19.8, 13.8,\n",
+       "       16.5, 21.6, 20.3, 17. , 11.8, 27.5, 15.6, 23.1, 24.3, 42.8, 15.6,\n",
+       "       21.7, 17.1, 17.2, 15. , 21.7, 18.6, 21. , 33.1, 31.5, 20.1, 29.8,\n",
+       "       15.2, 15. , 27.5, 22.6, 20. , 21.4, 23.5, 31.2, 23.7,  7.4, 48.3,\n",
+       "       24.4, 22.6, 18.3, 23.3, 17.1, 27.9, 44.8, 50. , 23. , 21.4, 10.2,\n",
+       "       23.3, 23.2, 18.9, 13.4, 21.9, 24.8, 11.9, 24.3, 13.8, 24.7, 14.1,\n",
+       "       18.7, 28.1, 19.8, 26.7, 21.7, 22. , 22.9, 10.4, 21.9, 20.6, 26.4,\n",
+       "       41.3, 17.2, 27.1, 20.4, 16.5, 24.4,  8.4, 23. ,  9.7, 50. , 30.5,\n",
+       "       12.3, 19.4, 21.2, 20.3, 18.8, 33.4, 18.5, 19.6, 33.2, 13.1,  7.5,\n",
+       "       13.6, 17.4,  8.4, 35.4, 24. , 13.4, 26.2,  7.2, 13.1, 24.5, 37.2,\n",
+       "       25. , 24.1, 16.6, 32.9, 36.2, 11. ,  7.2, 22.8, 28.7, 14.4, 24.4,\n",
+       "       18.1, 22.5, 20.5, 15.2, 17.4, 13.6,  8.7, 18.2, 35.4, 31.7, 33. ,\n",
+       "       22.2, 20.4, 23.9, 25. , 12.7, 29.1, 12. , 17.7, 27. , 20.6, 10.2,\n",
+       "       17.5, 19.7, 29.8, 20.5, 14.9, 10.9, 19.5, 22.7, 19.5, 24.6, 25. ,\n",
+       "       24.5, 50. , 14.3, 11.8, 31. , 28.7, 16.2, 43.5, 25. , 22. , 19.9,\n",
+       "       22.1, 46. , 22.9, 20.2, 43.1, 34.6, 13.8, 24.3, 21.5, 24.4, 21.2,\n",
+       "       23.8, 26.6, 25.1,  9.6, 19.4, 19.4,  9.5, 14. , 26.5, 13.8, 34.7,\n",
+       "       16.3, 21.7, 17.5, 15.6, 20.9, 21.7, 12.7, 18.5, 23.7, 19.3, 12.7,\n",
+       "       21.6, 23.2, 29.6, 21.2, 23.8, 17.1, 22. , 36.5, 18.8, 21.9, 23.1,\n",
+       "       20.2, 17.4, 37. , 24.1, 36.2, 15.7, 32.2, 13.5, 17.9, 13.3, 11.7,\n",
+       "       41.7, 18.4, 13.1, 25. , 21.2, 16. , 34.9, 25.2, 24.8, 21.5, 23.4,\n",
+       "       18.9, 10.8, 21. , 27.5, 17.5, 13.5, 28.7, 14.8, 19.1, 28.6, 13.1,\n",
+       "       19. , 11.3, 13.3, 22.4, 20.1, 18.2, 22.9, 20.6, 25. , 12.8, 34.9,\n",
+       "       23.7, 50. , 29. , 30.1, 22. , 15.6, 23.3, 30.1, 14.3, 22.8, 50. ,\n",
+       "       20.8,  6.3, 34.9, 32.4, 19.9, 20.3, 17.8, 23.1, 20.4, 23.2,  7. ,\n",
+       "       16.8, 46.7, 50. , 22.9, 23.9, 21.4, 21.7, 15.4, 15.3, 23.1, 23.9,\n",
+       "       19.4, 11.9, 17.8, 31.5, 33.8, 20.8, 19.8, 22.4,  5. , 24.5, 19.4,\n",
+       "       15.1, 18.2, 19.3, 27.1, 20.7, 37.6, 11.7, 33.4, 30.1, 21.4, 45.4,\n",
+       "       20.1, 20.8, 26.4, 10.4, 21.8, 32. , 21.7, 18.4, 37.9, 17.8, 28. ,\n",
+       "       28.2, 36. , 18.9, 15. , 22.5, 30.7, 20. , 19.1, 23.3, 26.6, 21.1,\n",
+       "       19.7, 20. , 12.1,  7.2, 14.2, 17.3, 27.5, 22.2, 10.9, 19.2, 32. ,\n",
+       "       14.5, 24.7, 12.6, 24. , 24.1, 50. , 16.1, 43.8, 26.6, 36.1, 21.8,\n",
+       "       29.9, 50. , 44. , 20.6, 19.6, 28.4, 19.1, 22.3, 20.9, 28.4, 14.4,\n",
+       "       32.7, 13.8,  8.5, 22.5, 35.1, 31.6, 17.8, 15.6])"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "train_targets"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The prices are typically between \\$10,000  and  \\$50,000. If that sounds cheap, remember this was the mid-1970s, and these prices are not \n",
+    "inflation-adjusted."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Preparing the data\n",
+    "\n",
+    "\n",
+    "It would be problematic to feed into a neural network values that all take wildly different ranges. The network might be able to \n",
+    "automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice to deal \n",
+    "with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we \n",
+    "will subtract the mean of the feature and divide by the standard deviation, so that the feature will be centered around 0 and will have a \n",
+    "unit standard deviation. This is easily done in Numpy:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mean = train_data.mean(axis=0)\n",
+    "train_data -= mean\n",
+    "std = train_data.std(axis=0)\n",
+    "train_data /= std\n",
+    "\n",
+    "test_data -= mean\n",
+    "test_data /= std"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that the quantities that we use for normalizing the test data have been computed using the training data. We should never use in our \n",
+    "workflow any quantity computed on the test data, even for something as simple as data normalization."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Building our network\n",
+    "\n",
+    "\n",
+    "Because so few samples are available, we will be using a very small network with two \n",
+    "hidden layers, each with 64 units. In general, the less training data you have, the worse overfitting will be, and using \n",
+    "a small network is one way to mitigate overfitting."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "def build_model():\n",
+    "    # Because we will need to instantiate\n",
+    "    # the same model multiple times,\n",
+    "    # we use a function to construct it.\n",
+    "    model = models.Sequential()\n",
+    "    model.add(layers.Dense(64, activation='relu',\n",
+    "                           input_shape=(train_data.shape[1],)))\n",
+    "    model.add(layers.Dense(64, activation='relu'))\n",
+    "    model.add(layers.Dense(1))\n",
+    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
+    "    return model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Our network ends with a single unit, and no activation (i.e. it will be linear layer). \n",
+    "This is a typical setup for scalar regression (i.e. regression where we are trying to predict a single continuous value). \n",
+    "Applying an activation function would constrain the range that the output can take; for instance if \n",
+    "we applied a `sigmoid` activation function to our last layer, the network could only learn to predict values between 0 and 1. Here, because \n",
+    "the last layer is purely linear, the network is free to learn to predict values in any range.\n",
+    "\n",
+    "Note that we are compiling the network with the `mse` loss function -- Mean Squared Error, the square of the difference between the \n",
+    "predictions and the targets, a widely used loss function for regression problems.\n",
+    "\n",
+    "We are also monitoring a new metric during training: `mae`. This stands for Mean Absolute Error. It is simply the absolute value of the \n",
+    "difference between the predictions and the targets. For instance, a MAE of 0.5 on this problem would mean that our predictions are off by \n",
+    "\\$500 on average."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Validating our approach using K-fold validation\n",
+    "\n",
+    "\n",
+    "To evaluate our network while we keep adjusting its parameters (such as the number of epochs used for training), we could simply split the \n",
+    "data into a training set and a validation set, as we were doing in our previous examples. However, because we have so few data points, the \n",
+    "validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot \n",
+    "depending on _which_ data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a \n",
+    "high _variance_ with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
+    "\n",
+    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions \n",
+    "(typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining \n",
+    "partition. The validation score for the model used would then be the average of the K validation scores obtained."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then let's start our training:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "processing fold # 0\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 1\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 2\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 3\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n"
+     ]
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "k = 4\n",
+    "num_val_samples = len(train_data) // k\n",
+    "num_nb_epoch = 50\n",
+    "all_scores = []\n",
+    "for i in range(k):\n",
+    "    print('processing fold #', i)\n",
+    "    # Prepare the validation data: data from partition # k\n",
+    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "\n",
+    "    # Prepare the training data: data from all other partitions\n",
+    "    partial_train_data = np.concatenate(\n",
+    "        [train_data[:i * num_val_samples],\n",
+    "         train_data[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "    partial_train_targets = np.concatenate(\n",
+    "        [train_targets[:i * num_val_samples],\n",
+    "         train_targets[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "\n",
+    "    # Build the model (already compiled)\n",
+    "    model = build_model()\n",
+    "    # Train the model (in silent mode, verbose=0)\n",
+    "    #model.fit(partial_train_data, partial_train_targets,\n",
+    "    #          nb_epoch=num_nb_epoch, batch_size=1, verbose=0)\n",
+    "    model.fit(partial_train_data, partial_train_targets,\n",
+    "              nb_epoch=num_nb_epoch, batch_size=16)\n",
+    "\n",
+    "    # Evaluate the model on the validation data\n",
+    "    #val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
+    "    val_mse, val_mae = model.evaluate(val_data, val_targets)\n",
+    "    all_scores.append(val_mae)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 16 records in 0.011235845 seconds. Throughput is 1424.0139 records/second. Loss is 8.708786._\n",
+    "\n",
+    "_INFO - Trained 16 records in 0.009535034 seconds. Throughput is 1678.0223 records/second. Loss is 5.3613434._\n",
+    "\n",
+    "_INFO - Trained 16 records in 0.008636178 seconds. Throughput is 1852.6713 records/second. Loss is 18.106756._\n",
+    "\n",
+    "_INFO - Trained 16 records in 0.009207628 seconds. Throughput is 1737.6897 records/second. Loss is 7.0931993._"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[3.291872501373291, 2.496018171310425, 2.221175193786621, 2.6994853019714355]"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "all_scores"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "2.677137792110443"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "np.mean(all_scores)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can notice, the different runs do indeed show rather different validation scores, from 2.1 to 2.9. Their average (2.4) is a much more \n",
+    "reliable metric than any single of these scores -- that's the entire point of K-fold cross-validation. In this case, we are off by \\\\$2,400 on \n",
+    "average, which is still significant considering that the prices range from \\\\$10,000 to \\\\$50,000. \n",
+    "\n",
+    "Let's try training the network for a bit longer: 500 epochs. To keep a record of how well the model did at each epoch, we will modify our training loop \n",
+    "to save the per-epoch validation score log:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "processing fold # 0\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 1\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 2\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n",
+      "processing fold # 3\n",
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n"
+     ]
+    }
+   ],
+   "source": [
+    "num_epochs = 500\n",
+    "all_mae_histories = []\n",
+    "for i in range(k):\n",
+    "    print('processing fold #', i)\n",
+    "    # Prepare the validation data: data from partition # k\n",
+    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
+    "\n",
+    "    # Prepare the training data: data from all other partitions\n",
+    "    partial_train_data = np.concatenate(\n",
+    "        [train_data[:i * num_val_samples],\n",
+    "         train_data[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "    partial_train_targets = np.concatenate(\n",
+    "        [train_targets[:i * num_val_samples],\n",
+    "         train_targets[(i + 1) * num_val_samples:]],\n",
+    "        axis=0)\n",
+    "\n",
+    "    # Build the model (already compiled)\n",
+    "    model = build_model()\n",
+    "    # Train the model (in silent mode, verbose=0)\n",
+    "    import time\n",
+    "    dir_name = '3-7 ' + str(time.ctime())\n",
+    "    model.set_tensorboard('./', dir_name)\n",
+    "    history = model.fit(partial_train_data, partial_train_targets,\n",
+    "                        validation_data=(val_data, val_targets),\n",
+    "                        nb_epoch=num_epochs, batch_size=16)\n",
+    "    \n",
+    "    #mae_history = history.history['val_mean_absolute_error']\n",
+    "    mae_history = model.get_validation_summary(\"Loss\")\n",
+    "    all_mae_histories.append(mae_history)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can then compute the average of the per-epoch MAE scores for all folds:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([[[1.90000000e+01, 4.05375427e+02, 1.55307042e+09],\n",
+       "        [3.80000000e+01, 2.64351837e+02, 1.55307042e+09],\n",
+       "        [5.70000000e+01, 1.50977859e+02, 1.55307042e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 2.07635689e+01, 1.55307053e+09],\n",
+       "        [9.48100000e+03, 2.02473850e+01, 1.55307053e+09],\n",
+       "        [9.50000000e+03, 2.02105141e+01, 1.55307053e+09]],\n",
+       "\n",
+       "       [[1.90000000e+01, 4.76980957e+02, 1.55307053e+09],\n",
+       "        [3.80000000e+01, 3.29584198e+02, 1.55307053e+09],\n",
+       "        [5.70000000e+01, 1.80655548e+02, 1.55307053e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 1.73588219e+01, 1.55307064e+09],\n",
+       "        [9.48100000e+03, 1.78555279e+01, 1.55307064e+09],\n",
+       "        [9.50000000e+03, 1.73744106e+01, 1.55307064e+09]],\n",
+       "\n",
+       "       [[1.90000000e+01, 4.62182434e+02, 1.55307064e+09],\n",
+       "        [3.80000000e+01, 3.34037567e+02, 1.55307064e+09],\n",
+       "        [5.70000000e+01, 2.06141006e+02, 1.55307064e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 1.72124062e+01, 1.55307075e+09],\n",
+       "        [9.48100000e+03, 1.75751667e+01, 1.55307075e+09],\n",
+       "        [9.50000000e+03, 1.74055386e+01, 1.55307075e+09]],\n",
+       "\n",
+       "       [[1.90000000e+01, 5.21177673e+02, 1.55307075e+09],\n",
+       "        [3.80000000e+01, 3.99685974e+02, 1.55307075e+09],\n",
+       "        [5.70000000e+01, 2.67611786e+02, 1.55307075e+09],\n",
+       "        ...,\n",
+       "        [9.46200000e+03, 1.75390892e+01, 1.55307085e+09],\n",
+       "        [9.48100000e+03, 1.76337471e+01, 1.55307085e+09],\n",
+       "        [9.50000000e+03, 1.91227703e+01, 1.55307085e+09]]])"
+      ]
+     },
+     "execution_count": 47,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "all_mae_histories = np.array(all_mae_histories)\n",
+    "all_mae_histories"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the `all_mae_histories` is a 3-d array, the last dimension are 3-element tuples. This 3-d array is built up with four 2-d arrays and all the first element of every 2-d array are equal. The first element of tuple stands for the training step and the third element stands for time stamp. You do need to worry about them, let's just calculate the average value through the first axis of this 3-d array. Actually we just want the second elements of this array, which stand for the MAE results. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 48,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([[1.90000000e+01, 4.66429123e+02, 1.55307058e+09],\n",
+       "       [3.80000000e+01, 3.31914894e+02, 1.55307058e+09],\n",
+       "       [5.70000000e+01, 2.01346550e+02, 1.55307058e+09],\n",
+       "       ...,\n",
+       "       [9.46200000e+03, 1.82184715e+01, 1.55307069e+09],\n",
+       "       [9.48100000e+03, 1.83279567e+01, 1.55307069e+09],\n",
+       "       [9.50000000e+03, 1.85283084e+01, 1.55307069e+09]])"
+      ]
+     },
+     "execution_count": 48,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "average_mae_history = np.mean(all_mae_histories, axis=0)\n",
+    "average_mae_history"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, this operation does not mess up the first elements since they are all equal through the first axis. And we do not need to care about the third element because it is useless at this time.\n",
+    "\n",
+    "Let's plot this:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 49,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXmYHFW5/79vVXfP9MxklmSy7wsk7EtCWIPBQEBRQUWW64LKFS8uV/TqFfW64YaiICo/ERFUVFwAQUVEQCAsAUwgQIAkZN+3SSazT3dXnd8fVaf61KmleybT05Pp9/M880x3dS2nejnveXcSQoBhGIapXIxyD4BhGIYpLywIGIZhKhwWBAzDMBUOCwKGYZgKhwUBwzBMhcOCgGEYpsIpmSAgoslE9BgRvUZErxLRp9ztI4noYSJ6w/3fVKoxMAzDMIWhUuURENF4AOOFEC8Q0QgAywFcCOCDAPYJIa4jomsANAkhPl+SQTAMwzAFKZlGIITYIYR4wX3cDuB1ABMBXADgV+5uv4IjHBiGYZgyUTKNwHcRomkAlgA4GsBmIUSju50A7JfPtWOuBHAlANTW1s6dM2dOScb22o42NKaTmNCYLsn5GYZhysXy5cv3CiFGF9qv5IKAiOoAPAHgW0KIe4moVZ34iWi/ECLWTzBv3jyxbNmykoxv3jcfwTlHjsV33nVMSc7PMAxTLohouRBiXqH9Sho1RERJAPcA+K0Q4l538y7XfyD9CLtLOYZCJE2CZdvlHALDMExZKWXUEAH4BYDXhRA3KC/9BcDl7uPLAdxfqjEUg2kQchYX3mMYpnJJlPDcpwN4P4BXiGiFu+2LAK4D8EciugLAJgAXl3AMBUmaBrI2CwKGYSqXkgkCIcRTACji5UWlum5fSRhsGmIYprKp+Mxi0yBk2TTEMEwFU/GCIGkayFmsETAMU7lUvCBImIQc+wgYhqlgWBBw1BDDMBUOCwLDgMUaAcMwFQwLApOQ5aghhmEqGBYEbBpiGKbCYUFgGuwsZhimomFBYBCHjzIMU9GwIGCNgGGYCqfiBUHSIOTYWcwwTJnZ1tpdtgjGihcEXH2UYZhyc6A7i7OufxwPrtxRlutXvCBImAbXGmIYpqy092SRsWzsausty/UrXhBwYxqGYfrCqp1t6MlaA3pOaZUY6PMWS8ULAjYNMQxTLK1dGZz3wyfxubtfHtDzSj8lC4IykeSoIYZhiqSjNwcAWL5x34CeV5qnuzMsCMqCyVFDDMP0EacT78DhmYZyLAjKQtJtTPPAyzs4sYxhmFiE0J8LdGVyB33ejDv3dGfKMwdVvCBImM5b8PHfvYBbn1xf5tEwDHMoIBWCnz+5Hkd+5SHsbus5qPPJRShrBGXCNPIq3q4DB/dhMgwzvJEagRQEf3vZifvf0ce545+v7sS21m7vufRT9rCPoDwkzbwgGGi7H8MwwwvblQSE4uaKZ9e3BExHQgh8/Hcv4K7nNnvbsqwRlJeEUfFvAcP0mY7eHHrLNGmVE0sKAk0OhMUdbm/txqW3PhsINc1YNrKW8L1/OY4aKi8Jk7UAhukrR3/1Ibz7p8+UexiDjqwFVMys0d7jaAJrdrb7tvdkndW/WtEgn0fAzuKyoGoEbBlimOJZua2t3EMYdDxBoE0WYVOHQLj20OsmjakF5rKcWVxeEgbP/gzDFEdfNAKZnmRokkCu+tX8pXJnFifKctUhBJuGGIYplrxG4N8u1/ZCCPz+31tw2Jg6VCdNd1//zt3uZK+ahrzMYhYE5UHmETAMwxQip5mG1Cn+6399FUeMq8cX7n0FAHD/x08P7APkV/1qAmu+6Nww8xEQ0e1EtJuIVirbjiOipUT0ChH9lYjqS3X9YlFNQ8WGhDEMc+jSlcnh50vW96sJjK2lFgtl+x1Pb8T/3pOPEJKrez0w0RMEdtBZ3J21IPT05UGglMvhXwI4T9t2G4BrhBDHAPgzgM+V8PpFwT4ChhnarNjSis7egy/jIPnRo2vxrb+/jvtXbOvzsXLlrs8adohQ6XCjhgI+gpztOxfgNxP15gZfKyiZIBBCLAGgl+g7HMAS9/HDAN5dqusXS9LkqCGGGaq09WRx4c1P45N3vThg55Qr8n2dmT4fa2t5BHLKyITUKet0E8l0H0FeI8gfk1WOL4fDeLAN5K8CuMB9/B4Akwf5+gFM1ggYZsgiSy68vPXAgJ2zJuU4cfuTvJWPGvLPG5mQVbwsWS33PNCVxV9e2u5N9L48Ap8gGEYaQQQfBvAxIloOYASASJFMRFcS0TIiWrZnz56SDUiNGmKRwDClRwiB6x5chde2F85DkFPlQGrraTeapz8ROlFRQ2HmnLxpyHn+ibtewH/f9SLe2NXhOxfgFwrliBwaVEEghFglhFgshJgL4C4A62L2vVUIMU8IMW/06NElG1OSo4YYZlDpzlq45Yl1eM8thTOTpSmmv4r7HU9vwL0vbPVtS7saQdfBaASaJAgTBNKvIX0EW/c7ReakpqCag1QzUTlMQ4MaPkpEY4QQu4nIAPB/AG4ZzOuHoZqG2EfAMKVHBsVki2gRm3fO9u/H+fW/vgYAeNeJk7xtchLvz4Sbi0go6w05V0ev5V7PeS6FmhQavqih4aoRENFdAJYCmE1EW4noCgCXEdEaAKsAbAdwR6muXyxJX4kJlgQMU2pk4bZiOgNGmWIOBrkS78+EqzuLJaGmod6suy/5js14UUOqszgvCIaVRiCEuCzipZtKdc3+oGoE5YjfZZhKw3InvWLC+KWwKEYOSN/DxSdNxszRdZG/52xOdgMLn3DX7GrHhMY06qqC02OuDz6CTlcjkFOMlHuyLHVYHgFQGVFDQw61HwF3qmSY0mP1YcElV8rFaOvbD/TgZ0vW40N3/BsA0NYTnnsgNYLOkBaTQggsvnEJLr/9+dBj7YioobCS3PmoIcKn/7DCa0Qjt0flEVRC1NCQQy0xYXETe4YpOX3J6PV8BEWoBKa7k5yUW7vCgxIz7jnbQwSFHNvyTfvDxxOlEYRM3nLCNwzgzy9uC2zPqkXnlFVoOXoSsCBQTEO5fqScM0ylcbAm1D4JAmkaKiAIdrf3eAJACo+whLE1u9pxx9MbAITH/oclhqnYUc7imKghXXvoDNEIcrZAVcKZjsvRpYyLzimmIb2OCMMwQQ52vdQ3QVBc1ND8bz2Kppqk75jWrmxgv7f96Clvss+GTPphwiFsPLpkijMN6deRvgN/HoGNEdVJ9Hb0skZQDtTGNLkiwtkYptI52AVTXwSBnESLySPY70788vz7XdOQXGkD/hV/mAWgkCDwWlVq28OjhnK+8ejbs76oIRsjqhOR5yo1LAgM1VnMgoBhCnGwgqAvJthcEc5iPcrGcwa7E67sCxB1bhVVUNz82FrM/cbDvtctK9xUFeYj6PR8ASJ0u55HkE6aMKg8PgI2DZnsI2CYvtBfOZC1bCQM6pMgKaYj2IFuvwlIHiMjcaK0iUKmoesfWg3A8QsY7kmk7AhqBMHJW15fD0LJeePLb89YNpIJA+mkyeGj5SDpixpiQcAwheiPRtDWk8VhX3oQ/+/xdX0ywXqTZYwk0AWBnGgL/Z5DTUMhwqFdKYHtOYuLKDHhXSfiftXx9eZsVCcMVCdNdGctPLN2L9p6gj6OUlHxgsD0RQ1x+CjDFKI/66V9HY69/vf/3twnQSIna72mv4ouCCRZL+Io/NhincVtyvkDJSa8kNXouSPsOoBfQPTmbFQnTVQnTfz2uc34j9uew6+f2Rh5zoGm4gWB30dQxoEwzCHCwfgIhOibCVZOorGmoZDoICC6iYz+ukqoIFBW5pElJmLMOVE2fzWPoDdroSphoDqZn5IH02lc8T4CdbXACWUMUxhR5M9kza52WLbAEePrvQldiL6ZYIupNdQaoRHIJC1bCOw80INbnvAXOw6zAISZhtq686YhtQheW08WO9xs4bhJOyrDWb4XpkHozdmoSppeZVTAiS460JVFfTpR8jpoFa8RqLCzmGEKE6YRPL56d2Ci/cbfXsPX/vIqgPxEaQvRr8zi/piGVF/Bz59cj19qppZiw0dVjUAtj3HujUuwu70XQLwg6IhpsymFUW/WcnwEibwgWL5pP4679p949PXdkccPFCwIFNhZzDCFCRMEH7zj37juwVW+be09OW+ClJEwfRUE2SK09GIEwfiG6sDrckW+q60Hb+xqB1DYRyCtBrYQ2HGgx9uuRw2pJp4wZG7DfW7piZ6cjaqk4dMIZP+CkXWp2HMNBCwIFFgjYBhnUnt9R3T3sGJ/Jj1ZK1CD3+63aShaI2iLchZ7xeUsPL46vMth1rLxpusfwzk3Oq3UQ01DimlHvqwLQz2PYGJjOnK8ADCi2smC/vw9r7jHW6hOmKhSNAJZIqO5tir2XAMBCwIFmwUBw+Ar972Kt9z0JHa19YS+XmytIb8gcDUCW/jMK4XOpcbi5ywbLR29uHXJOt9xUcXlVGfwU2v3hu9jC6/ap2WL0AifMI1A9zPrpqGRtc4qPpUIn2LTKf/23hCNwDvXIGgEFe8sVmGNgGGA5Zudyptt3VmMrQ+aVIrXCGyk3TlMrpgd05C/zINaCl5HOnzX7OrA2378FKaOqsFDr+7C02tbsGZXO5Z+YVFB01AcatXPPe29hX0E7su6ANNNQw3ujY+sSWFniEBV6yDlLNstOmeiWhMcVQkDtSHCYaBhjUCBfQQMU5hiw0e7s5Y3YcoVs4A/TLvoIm8AVu1s91bvT6zZ49noD3RnQ1feuSLiwdU+ANsPdPvGM7I25Sv5sKutB7e7lUv1uULXCCY1OaahKI1ALYEtj60O0Qia66oGpXMiCwIF1ggYJk/Ur6FYQdCTtbwJ02caUjSCgoJAs8HUaBNl1rJxoDuLUbVB80kxv+ftbvinfCwn5dNnjcJjn12IaaNq8djq3ViyZg9+vmS9t69+avU+TIPwP4sPx6cWHYa3Hzc+9LqfPvtwAEBzXcpzpFclzEBdpJEh91UKWBAocB4Bw0QnYP308XX498Z9RdUasm2B3pztCQ25kncidfL77enojT2PHutfk/Jbs7uzFg50Z9FUE5wwozJ6337cBHzv3ccCAC64+Wlv+84DPZ6GcNsHTkJDOol0ysSutl584PbnYSomrDjfhmkQRlQn8elzDkfandhlZVHJMZPqcdHcSUiZhk8jMLXCSKMGwT8AsCAAAHz8rJkA2DTEMCr6XPfdf6zCe25ZGtAI1CJpMuBCNleRPylPIxDCN7lv3NsZO4ZsAY2gO+MIgrCVc9TvOWfZoSab3pztrezl67WK4EkaxdUlMxVTzrTmWud8pv961QkTqYSBjGV7gqAqYQbe8/EN8dFHAwU7iwF87tw52La/Gyu2tJZ7KAxTduQ8JiKMQ/ocqCZMWULAAPkicQDVWew3LW1q6Yodi27n1wVBS0cGWUsEBIEQIiBEJBMa04GVN+BoEFnLhmmQ97pqs1cdwnHmMbVszfnHjMfO83swoTGNj/32BW97VdL0tIG8aSgonGaOro28zkDCgsDFNAz2ETAM8t3Aola9+iTY4YuzF0iajskGADbv68ItT6zzVr1Zy/bZ/Tfti9cI9N9kQosw2tnmJl1pguD1He2hJSTu+NBJOHXGKDy+Opit+8NH3gAAz5wDALVV+cf7u9SaQ9FjVk1IRIT/XDADti1w+qxReHptCwDHDJRKGMgqGkF10gwI3+nNgyMI2DTkYhrcoYxhVKIEgW4fVzUCOXGr5qLrHlzlhWDmbIEed+Jrqklie2t4rkL+fFotf+03us09fvQIf9LVTY+uCf09nzV7DKqTpq8zoY4azqr6JNR8hTiNwAyJ8jEMwp0fPtl7nnY1gkzO9grWhWkEE5vYNDSoNNWksK8r42tCwTCViJzH1NW4mmypy4duZdK33MlXr7i5bX8+OkdOfLVVidCGLpID3Vn85tnNvm26uUeeV40aInKycsPMPxJds1BJKdm9ar8Sv0YQ7ywOwzAIBjnvX3XS8RHYAuhy36sqRRO5bP5kjBlRjcPHjIi8zkDCgsBl6qhaZHI2drT1FEwPZ5hKQJ38fUJBmwSzSuikXMHrXba2qoLA3b8mZcaGj/72uU2BbS9u2e97LsM/m+vyGsH05lq0dGYwMiSSSJI0ozUCtU6Q+h7s78pg4ezRSBhGbAmORJwAMhwHcXXS9MYgtaWqhIGGtFN64qgJDXjfKVMjzzPQsGnIZdqoGgDApgJRDAxTKfh66iomGt30rvbktTzTkH+nrfvzTmEpCNKpREAQCCG8yX1HiNnoxc2tMAi4+79OBaAIAsU0NLquynEixxjy4yZr1USjlsPY15nBxMY0RtYmYzWCOIuCtEhJHwEA7G5zQmgb0klcccZ0fPltR+LSkyZHnqMUsCBwmeo6ZTa0sCBgGMDvI4jSCIQQmkbgmoY0jaBTMRXJyT+dNAIZufe8sA2nXfcvLN+0Hy9vDY/iG1df7dnOpSBQTUNj6qtxoDuLHs08pcbyJ2I1gryJxvbVNHLyFQyi2PDRQhoBkA8fBYBt7j2MHlGFqoSJK86YHju+UsCCwGV8fTUSBvlUWIapRGRJA3Xytyx18s/vawt/4pYV4izWkX6BmhCN4CU3hPvlra3YcaAHl8ybjAWHNfv2SSYM1CSdSX37gR6MqE6gSjHnjHG1gz0dvb6GNn/75Bn5cyg+gi+99Qifs1nVCPRClI01SRgGhUYNzR7r2PPjNALTIKQSBgyDkHLHsL21GyOqEoGs4sGkZIKAiG4not1EtFLZdjwRPUtEK4hoGRHNL9X1+4phEJpqU9jfGV7JkGEqBTmNqZn2al8AdZVs2cJngonSCFQ8jSBlYuv+bm/yBxwHMgB09ubQnbWQTpmB6B/TIFQr1TubalK+SB05qe/rzHiT+pmHj8bUUflQTDVq6CNnzvBN/mopaL3Oj6MRhDuLj53U4J47TiMgr7Cc1Ai2H+gORD0NNqXUCH4J4Dxt2/cAfF0IcTyAr7jPhwyjalNoYUHAMAD8oZpWhGnI1kxDltJxKwopCGqSJjKW7SvzIM037b05dGcs1KRMn50ecDSSlJkvx9BUm/JN7DOU2HvpkNWDhPSKp+rkrTqLv/DWOb7XRtY6QidMEMhy0WZMaKppkLfyT5nO/+2tPT5ndzkomSAQQiwBsE/fDKDefdwAYHuprt8fRtamvGYQDFOpyEWwz0dgqZN/ft8Tv/GwzzRUlEZg5aOGomjtzCJnC6STZsAebwsBIvKOH1mThDr3HjmhHouPHOvci7tND+nUbfDqc1UjGDOiGjdecrz3vLEmCSLylZEGgFNnjPK0kjjzfkIRBFIY7evMoHnE4NQUimKwfQRXA7ieiLYA+D6AL0TtSERXuuajZXv2hHcXGmiaWBAwjIe6Elf9BWpCWVfG8gsCL48gOiy0N2uDyB83L8/ZlXGS03a4NfzTqaAgkM+lg7ipJuWb6A0izB7n2OvlkQFBoD1XTUt6m0m1LpF0FqssOKwZd115ineNWI3AJC9zWT3vuPryhqxHjpiI/ld5/B7ttW/383pXAfi0EGIygE8D+EXUjkKIW4UQ84QQ80aPHt3Py/WNUbUptBSohsgwlYLlCwtVfQT+/bIhJqSekEQxOcFmLBsmkc8u/7W/vIqcZXsCZM1Op4dwTSoR1AikIHDNKU21fkFgGoRGN4dAZj3rgkDO5c2uOUed2/UcgypNEKgvv/vESfjp++YCgCcgYnLVkDAM731QBcFbjxkXfdAgEKcRXKo81lfuuu2/WC4HcK/7+E8AhoyzGHBMQ209ucjytQxTCXiZxVa4RhBIKAszDWWCgkBW8szknMJuakXOXy3dhKXrWzyT0k5PIzBCTEPO/zrXsdxUk/St6ImcbUA+wklfxcuxXDZ/im+/sH1VU9GI6oTv9aMn1nvjkNvjyleYBnmakHr/c6c2RR4zGMQJAop4HPa8WLYDeJP7+M0A3ujneUqCVDX3d2WwfNN+7G6Pr4PCMIc67T1ZfP+h1b4wzrCic34fQbQg0BvRqMiIIE8QaLV1crZAdybnhX8CQDoZ1AikyUp1Fvs0AqJAfwJdI2iqTWHFV87xGsTEoYamGoY/h0D1LciHVcnoaVX1Eaj3PxhdyOKIKzEhIh6HPQ9ARHcBWAigmYi2AvgqgI8AuImIEgB6AFzZp9GWmDo3YqGjJ4d3//QZTB6ZxpP/++Yyj4phSsfNj63DLU+sw/jGarz3ZKekQVitIb+PwH+OjE9zcIRCmEYgnbu9OctZGWuCYH9nBl0ZC001Kexud0y06ZQZKDwnTUNybm9IJ30TqWMaSvqOOW3mqMB4GhVhoVb91OdkfZxdiiM8qfom3Mdx+QAnTm3C2BHV7vid/ae6VQ3KSZwgOI6I2uCs/tPuY7jPgx2tNYQQl0W8NLdvQxw8pBNnj/sl3LKPk8sYP5YtcMPDq/Hh06d7NupDGTnp7esIBklIv0BLR6/3mwCCGkEuNKEsaF6VGkGvpxH4J8zP/PElAMBxkxu9bTUpM+CTkBoBeTZ53f6f9xFMbEzj3o+d5tMywlBvKUoQyCbyasayrokA8YLg2+88xnssQ2UvPWlK7NgGg0hBIIQoX5pbmUi7dsMtnF3MRPDU2r24+bF1WLe7E7e8f8iuaQA4pQvGjKiKLbAmJyO1lLSc2nK2QCZnY+43H/EdE3QWFxc+KjWCTM5GIkQj8PZTJtJ0MqgRSGFz3KQGPPzaLoxt8K9LTYO84m0dvTmMrS+4btXwSwIpJMa451HvLaPcuxQK1RH3pTN1VC2euebNGN/Q1/ENPH0KHyWiWiJ6HxE9UKoBlROpEWzeF981ialcvISpmPLJQ4HujIVFP3gcf35xW+x+I9xVepvSXEYuiS1b4NdLNwaO0TUC1b/glaGOEQS9ORsGBX0EErUrWDpl4toLjvZf3xUEVy2chfs+fjpOnOJ3tJrkCILxDdW49oKjQq8Rh64RTB5Zg2MnNeD6i5w+x+q9qSYwqaGkY/IjdCY0psvuHwCKKENNRCkA5wP4DwDnArgHwC0lHldZkIJgqysI0mWs/cEMTYpp3D4U6M5a6MnaPpNOGPJ22nuygddytsCtS9YHj9HehC5lMpQaQVhmsRc1ZNlIGhSIzpH4BEHSxFmzx2Djdedj3Z4OLPrBE55GYhqE4xUzkoTIeW3pFxaFnr+vVCdN/OUT+TpF6v2qgkBqRofivBEpCIhoMYDLACwG8BiAXwM4SQjxoUEa26Ajv4Bb3JK53VkLD726E+ceVd4YX4bpK9JuH1fqAciv5ttVjcCloycXmmCpl6FWnaff/vvreHHz/lCNQIZNZnI2qqoToa0kAf9EqmYfS1OSXnJCJ64hTRTqGQsdrRbUU+/T6z18CAqCONPQPwDMAHCGEOJ9Qoi/AhjWAfZSEKimoY/euRyAswq6+bG12MJmI+YQwGsZGdP4BcjbuFWNQE6Ee93kSn1i1U1D6qp4874u/GzJ+lBBIF0VB7qzqKtKRNb0b6pJes2hVMerjOfXK4IGrnOQppZCh6v3e/G8fN8A6SDXM5MPBeJMQyfCSSp7hIjWA/g9gENP1PUBuRLZ1RZUp3e19eL6h1bjzy9uwyOfcVIhWjp6kTAMNGihaszwZyjYdeOwYkw0KlIjUM0dcprd60YSjW+o9pVn1+fhrkwOSZN8GcZhEXfqBF2fTob2FL7zivk4dlIjLj9tGp5Z1+IXBMniNIKD/Wg+smBG7OvyvVryubMwRQn9lBpBdeLQmyYjRZcQYoUQ4hohxEw4OQDHA0gS0YNENKTi/weKONueXBWpJSjmfvMRHHftP0s+LobpK9JeHRbGGbZfWH/ilk7nuz6hwV8HR/cRdGes0MkvpUUrqcKzIZ3EjNG1+iFYcNhoNKSTmNRU41ttA3nTUCE/TX+EtLynX314vq9cdRg/uuwEnH3EWExo9Ef7eIJgmJmGPIQQzwghPglgEoAbAZxS0lGViSiVbseBblzys6UACv+wGGYoEFfzZ+m6Fky75gHs7ej1NIKwXIAWqRFoE56uEXRmLFSHRMqMbfDH7qsmpvrqJOZOHYl7rjqt2FvyBMtbjh54n528pWQR/oW5U5tw2+XzAhVMpSBIp4aRaYiITox4aS+An5RmOOWFyKkMqNs3b3l8Hda7vYzjyusywx+5GtVXxUMNaabpDVm43LpkHQBgxeZWTxBkLYHenIXbntzgTWhS+x2vaQRhPoIwbXpGc53PRKTmM8g4/zlulVDAWWnHQUR4/kuLvGNLwcG0iOw+hE1DcT6CZQBWwpn4Ab8zXcCpFTTsSKeCgmCo24MZRidOI5AQ5ctDZCwbyzbux/UPrfZel32GdROILgi6MjmMGRFMipreXIsn1uzBxMY0zpozGhfPm4RbnnCEkJzM1aSydxw3oeB9hV1nIOlPxJHkc+fOwf6uLBYcPjjVkgeSOPH3GQBtALoB3AHg7UKIs9y/YSkEgLyf4PxjxgMI1hlhKhu5Jhjqi4Oc1yksqBHIaZwIPtNQWJ/hdNL01eQBgjb6rowValaVPoCGdBLfvPAYr8SEs815LFfgR02oDxxfDvTOZX1h1pg6/PGjp3rVSA8l4pzFPxRCnAHgkwAmA3iUiP5IRMdHHTMckCGkE5vS+PhZM5GzxUFHITDMYJOL0QjkRE4gL3zUMQ0FhUZjTTJg9tE1gt6c7XOQvu+UKZjUlEZ9td+E4/MRKOadB/77DPzuI2V2O7q3FFdCejhTUHQJIdYT0f0A0gDeD+BwACtKPbByITsX1VcnYNmOil3IHCzc1nkMM1SQoZlhwQ3e15ng9RvOWHZo2YzGmlSgpWRYGL+aDfyNC44GEeH+FU55C/nTULuCqXb+oyY0FLyfUuM5iw9CIziUietQNoOIvkhEzwH4OoCXABwhhPjjoI2uDIxzC0BVJ02vFkqmQKMajiSqHIayj7grk8NZ338cz61vie0LIB3dhPx3O2fZoWakxnQyUDsnLKFLdZB6VUHdiV8KAlUjGKrmE+MgfASHMnF60FoAF8PJMF4KYAqAq4joM0T0mcEYXDk4x2163ZuzPf9AVlOZN7gRRJLOTDA9nxne/GvVbizbuK/cw/DR0pHBhr2deHV7G7IhPgLLFrj9qQ1eOQmBfB7njMARAAAgAElEQVSBLfxJZZKm2qBpSJ77J/+Rj/KpShqYNaYO31CKvMkEMtnoRjW71A5RQVCpxH0a1yKvMdUNwliGBJfMm4xszsa7507C/Su2A0CgdeWFNz+Nl7662Hve1WtV0DvESC66ZSk2Xnd+uYfhIVf3Hb05rwqoqhEsXdeCa//2mvc8p/kF1FLUkoZ0KlDLf2+7k1+gJoylTMPLuJfo5lLV/N6XCp2DgaolVSJx/Qi+NojjGDIkTAMfPH06gHwruayWCn+g21+pMewHxAxPhrBlyPMLdPTmvKgh1Wy5YW+Hb/8bH17j++52hnyPG2uSgQY8Nz6yBoC/uFoyJLpOD8VUNQLd78CUl8p0kRdJVYSPYKaWGt/FpqGKQe+fO5TIegXkcvmoISUkdPWudt/+r+1o8xVYbA8RBE0xdbRUjSAsI9fwQm39zwGgJjm0TEPvcctZDIeuc/2BBUEM8ouuNt6YNqomoAF0hthWmeHJUM4o9pmGXEGQs4VXPuKNXR2RxwJO2Wkd2bVPNmVRURvLhHVB0x2vqqloqJmGPrZwJt741ltKmrU8lGFBEEPeNJQXBM11VdjflfVNCF1sGqoYClW+LCeeaagn6zNn7uvKYHd7T8CkqaMucKQ2LB3F75k3OdDtS022DDMNGZ6zOEhUd7JyQUSxLT2HO8V0KKsC8G4A09T9hRDXlm5YQwNZ/1wXBJmc7W9IMcTbFjIDx1CzDH30zmWoSpj40WUn+ExDltL05a03PYm9HRnMaA6vqpkyDWQs26cRLD5qHC6eNwmnzWz2tuk2f58gCDENmaTZhpghSzEi8H4AFwDIAehU/oY9+TyC/K9/VJ2Tbr+/K7+64jyCyqFQUxSdu5dvxeW3P1+i0QAPvboLf3nJiW5TTUOqRiD7CoSVkACAOreBveojaOvOYsFho32Tv37vVUruQLhpyPnPYmDoU4zHZpIQ4rySj2QIkgrJI5DOpH0d+RZ+UT8wZvgR1VUris/+6aWDvuZDr+7EqTNHBUo26EjTkKMRBMfZ0plBTcoM5AscPrYOz67f54saagvpYaz7wlJFmoYGi4+eOcOrEsz0jWI0gmeI6JiSj2QIIp3FftOQoxHIph0AawSVxGBHDa3f04GP3rkc19zzcsF9s4pGkAsZZ2/ODnWGHjep0TtOFo9rC/En6OGlaqJZmEZwMJU8+8MX3noEfv6BeYN6zeFCMYLgDADLiWg1Eb1MRK8QUeFv5TAgrMSETKVXJ3/WCCqHwfYVt7jN48Pap+r4BEFEWZQwQXD0RKfWT0dPvpx0WDP7zt4YjSCkRo8ePsoMXYoxDb2l5KMYooSVmAgTDiwIKoe+moYOFmnGCUvAUtumAvnER8sWkUmOjVpewCXzJnuZwxnLxqSmNM44rBmXnjQ5cKyuEaiZwnERNywHhj4FNQIhxCYAjQDe7v41utuGPVIQqGn4XpJZjgVBJdLf8NG+Opkl3W6yYlgHsLnffMR7fPNja7F0XYv3fH9XJrA/ENQIrj7nMF9XrnTSxLffeQyOdc1FKm85xt8iUvUBhAkCr9w1qwRDnoKCgIg+BeC3AMa4f78hok8WcdztRLSbiFYq2/5ARCvcv41ENKTLWcvVvzrRy21/e3m7t419BMObnqyFL/75FezrzPQ7fLRYAdLS0YsbHl7jCY4ojUAXLNc/tBr3vLDVe97aFZ4z0Jj2N5mpSpi+DOGqiL7dALBw9hhfbSV1ei+2fPOnFh2GGy85rqh9mcGjGNPQFQBOFkJ0AgARfRdONdIfFzjul3B6G/9abhBCXCIfE9EPABzo43gHFTnpqzkDcuXz+Oo93rZy5BHYtsC3/v463nvyFMwYzRXvSsm9L2zD757bDJMIs8b07722bIGQRX2Ar9z/Kh54ZQfmTW3CmYeP9iJ1arRqnVk7fvERKQg001AqYSCZyE/ixfTb/eWHTsJdz2/2OYNDNYKQYz99zuEFz88MPsUIAgKgznQWijD7CSGWENG00BM6uuLFGOJ9j+VKSV18hWVE9mQtvPe2Z2EQ4c4rTh6Usb2xuwO/eGoDlq5rwd8/tWBQrlmpyAqepkH99hEUG20ktU9pjpSmoRpNihQ6X5RpqF4zDaVMw1cMTm9LGcbC2WOwcPYY3zb2ERzaFCMI7gDwHBH92X1+IYBfHOR1FwDYJYR4I2oHIroSwJUAMGXKlIO8XP9ImAYMKkYQ2Hh6bUtgeylRJyemtEh/UNKkfoePhoVzhiE/T3kdGalTrfcEsOLPV6xGkDTJZxqKKzIXR5hpaAhX42A0imlVeQMRPQ4njBQAPiSEePEgr3sZgLsKXPdWALcCwLx588r2lapKmD7TUCpk5VMOZ7FcMVYNsZotwxEZlplKGP2e3Ip1FuuCIN9EJn/8y1tbfcEKYbR2ZZA0KSAwdB8BEflMQ021hTUC//HOhB/uLBbePszQJlIQEFG9EKKNiEYC2Oj+yddGCiH61Z6JiBIA3gVgbn+OH2xSCcMnCMIm3rIIAtdBHefcYwYGWWIkaRpFOX33dWawvyuDmYrvplhnsScIhOwt4Kzs1bSAd/zk6YLn6cxYaKxJBjSD+nTwJ69O4iP7KAhMIuSEqOiCbcOBOI3gdwDeBmA5/H4fcp/P6Oc1zwawSgixteCeQwDdFBRmGgqr4y7Zsq8LrV1Z7O3oxYtbWnHlmTMGpF+rNA1VFeHcYw6OvGnIKMpHcN4Pl2B3e68vwqZYk5IUBDklOcw53m0pWeA8soAcEL5o0U1MgNOX2HvcR9OQ4dpOQ01D7n9iL8GQJ65D2dvc/9P7c2IiugvAQgDNRLQVwFeFEL8AcCkKmIWGEropKCAYTAO7DvREHn/xz5Zix4EeJAxCzhY4fnID3jxn7EGPS5qGwkxVlcDKbQewq60Hi444+PeyEFllYi1G+9vdHswC7qsgkCHJMnxUagQy0ziKdMoEZZ3vh+oEllQnTDz7hUU45TuPetsSB6kRAOGmobH1Tpby/Okj+3ROZvAppgz1o0KIRYW26QghLovY/sE+jbDM6Ksq/QtfW2X6KpHqtLjF6aSzsJCTr1jkKrVSTUNv+/FTADAoPYOlIEiaRqBgm22LQAMWiSo0ihUECfdcsuud/JyXrm/Buj0doaUfVJKmgaRpoLej19c5b0JDNbYf6EEqYWBcQ3Xk8SOLiBpSkYIrTBBMb67FY59diCkja/p0TmbwiZxFiKja9Q80E1ETEY10/6YBmDhYAyw3ugZgap6v2gJmnrlTm3zP+5thqpM3DVWmIBhMVNOQPqHHxfOrk3bfNQLLd+3Xd7Rh0Q+ewPUPrYo9PmkSPrVoFgB/qXQ9D0Hno2c6lt5iwkdVpAyMSiib3lzLkW2HAHHfjo8CuBrABDh+AvlptsFJFKsI9IlWX/0Vsvfr2nmxYYSFkKYD9hGUHmlzFxCBVpU5SyDqK6CWcn5w5U5ctXBmwWtJuSI1D71fdqEwZVsIvP/UaUglDLR2ZfGdBx3BcfrMUVi7uwO1VeHfl8+fNwdXn314nzuHGTEaAXPoEOcjuAnATUT0SSFEoSziYYv+w0hogiCsGJiKHuY3UEXL5IqRNQI/3/3HKvz08XUDajKS5jzbFoHon2xElU/AX8r5u/9YhWMnNeD0Wc2R+wP5ib9b0wiKRfYkuOQkJ/dm9rgRGFVbhTnjR+DS+VMwqSncTGMY1K8+wnE+AubQoZg8gh8T0dEAjgRQrWz/dfRRw4eAaciINg2FacD6D3mg6tnLFWNYQ5BK5qePrxvwc2ZcM5wtgq0q43w+eo/gPSFO5MC1XEGgm4aKRdcg1AzgI8bXe4/vuepUxMiwopEaAecKHNoU4yz+KpzonyMB/B1OWeqnoNQQGs7oUTl616XaVP4tDFOre7Uf8sCZhrji6WChlnfWfTxhGoEM4WzTHLtx2oNETvxRpqFCFHImS+ZOHZhInvnTRuKBV3ZUbPTacKGYT+8iAIsA7BRCfAjAcQAaSjqqIUQhjaCmKr5LU8A0dBCCQG02Ik0Hg10fvxKRE7gtROD9zoVoBFURXb6KiRhTncN/fnGr7/tTaNWdMCi0j0Ap+f57jsNfP3FGnzOSmaFFMYKgWwhhA8gRUT2A3QAG99tWRlKaM1Y3/xRyFusaQX/r2d/7wlbM+tKD2NzSBQDodleMAxWFdKiiO28Lbe8P8jN0BIH/tbAVu0za0k1DfdEI1uzqwKf/8JLv/IV6Fv/5Y6fjuncfW/AaA0k6ZeKYSRWzLhy2FCMIlhFRI4Cfw4keegFOGeqKQHfG6k02VB9B2KSsTxT99RH89SWn/8Ebu9sBAF3ZvN1aRwgR2nx8OBL1dg5kb2E5gVt28Lxhk7v8hqzf01lwXwD42RPrsHZ3R+g+qlYR1mZSRa0ZxDB9oZgOZR8TQrQKIW4BcA6Ay10TUUVQKJxO1QjC7P8D5Sz20vXd33qXV3ogeL5fPbMRx37tn9iyr6tf1zqUiJpcB8oXo17DFsHw0U0twfdYCv/nNrSEblc50O2EeL7z/z2Nj965DBtburDgsGacOmOUe838vmF1glQ4cofpL3EJZSfqfwBGAki4jyuCQk4wNXw0bFIeMEGgtf3zqlIKge88+Dou+ukzsG2BVTvb8M/XdgEANrZ0hp5rOBE14Yf5Tixb4DfPbsKB7myfPgf5GYaFj67cFuytJPffur/bP1ZL4Orfv4g/Ltvibdvr9h1u78nhoVd3YW9HL2pSJt49d1LgvIVMQ+ywZfpL3BLjB+7/agDzALwER+s9FsAyAKeWdmhDg0Jx+qppSE4Sti3w3X+swn+cPGXATENyYpPKv7Q/2wL42RPrAQAzvvh33zEHaya/8eE12LK/CzdcfPzBnaiE5CI0grD3+e7lW/B/963E/923EpfNn4LvvOuYyPPe8PAazBxdiwuOn+j5CKwQH8ErIYKgN2dj8sg0tuzzC4KerIX7VmzHfSu24+J5jpstLKQ0aRqoC0n8KiQIWCNg+kvkN0cIcZYQ4iwAOwCcKISYJ4SYC+AEANsGa4DlRhUEx0wMOsXU8FEhHCGwrbUbP1uyHo++vjswIfXXWSwPk+Gr0gcQd76DjSi66dE3cO8LQ/ujjtIIwgRBW3c+tPKu5zfj+Gv/Gbqi39eZwY8efQOf+r3TUlt1zKt+oAkN1d6KXr2uZQs0hZRqaO0O+m3CCtSlEgZqUsE1WmHTEPsImP5RzBJithDiFflECLESwBGlG9LQQtZoOWFKI/76yTMCr+s+hJwtvBrwatEvSVyUz4HuLHa3+yuZrtnVjo17O32NSYD8pBYXHVMJ8URh4ZtAcZpXa1cWtz25XtuWwYnfeNi3zRMEwi9cq5NmwEchzUJhjt1tmqkICNcIqhJGaCmIws5i1giY/lHMN+dlIrqNiBa6fz8H8HKpBzZUqHV9AN2Z8AQufRVmC+H1i+3oDR4T58Q847p/Yf63HvVtW3zjEiz8/uNeDRrLFsjkbC+PIHbCc1/qyuS8uvbDDTkR//2VHZh2zQPe9jBNSRemAGBqxaD0kE8hhBehZQnhy8atSpqB3ABZDDCsnPP21rwg+PJ9K7F6Z3uoIEiZRmgxwxGuaUgNXHv+S/kiwOwjYPpLMd+cDwF4FcCn3L/X3G0VgfxB6vkAkjCNQAqCW54IljuI0wjiGtzISSxj2WhXQkPj5IBcvZ78rUdx9Fcfit7xEEYK1nuW+/scxRQF9aHPnXrCYMayPWFr2/6ooeqkEQgGkM/DTEPbFEFw57Ob8OQbewKmJcD5TtWGmYaqnW2L5uTLRqg9B9hHwPSXYmoN9QC40f2rOKQgiNIIpo2q9T23LIH9Mc1DDtZHkLOEr3RBnB9ACok4AXOos3pnG5ImoV4zm+SKlAS6RqAfpn7utvBHDVUn8qahtp4s6quT3oIhzIyj9zLozdmh36vqpBmqEdSnk/jH1QswdWQtjvnaQ8jZAglFI+Vyz0x/iQsf/aP7/xUieln/G7whlhe5MusOqe3z+GcXYly9v8mHJUSgUc133nUM7v3YaUgljKIKfYXZ/T1BYNs+80WchjGQ2bVDlf/6zQs447uPBSbe/moEen8BdfK2bL8GVp00kLVsPPLaLhz7tX9i+ab9niAoptNXJmeH1owaUZ0IrWqbMA3MGVePdMr0KoUmQ7qQMUxfidMIPuX+f9tgDGSoIp12YT9Y06BAf4KcbXumIUl9dRInTmmCSeT1ntXpVFbtXRkrsCKUK/+sJXzZpkW4CHw8u74Fu9p6cMHxw6u3kK4RFN0sXssU153PXZpGoAreatdH8MArOwAA6/Z04OgJTmRZMb1/s5YdusCor06Ghi2rJdBrUwm09+RgGoQ/XHkKnl67t+D1GCaKuH4EO9z/mwZvOEOPuhgfgVTFX/zyOfjby9vx5ftfhWUHNQJp3zcNitQIVKfh7vZe/OLB1/G5xXOUczjkLNsXjRQ34YVpBJfe+iwADDtBUK217Cw2X0N33utRQLppKCxqaFebE+m1akc7/vduR1kuFOEDxGkEyUApE8AvCKTGkDQJJ88YhZPdTGSG6Q+RgoCI2hG+qCQAQghRH/LasCOuxZ/8YTbVplDlFhqzbIFWTSOQgsE0CLYQ2HGgG6d+51/45YdO8urFy8kEAH7z7Cb85tnNSJl584Cc1LO28LqTmQbFmn8GsszCUMfSVvJhgoAQnFzlinzt7nYsXb8Px2q5Ij6h6+YISKRpSH52D726EwCw4LBmnFLExJyxbO+zVBlRHf6dU53B6ZQJ06BQgcEwfSUuoWyEEKI+5G9EpQgBAKgLid6QqGYhKRQsW2Cf5iye4DYLNw1CzraxfNN+AMCfluUjXf6glB2Q2bJqRImcf7I52wtRrEmasbbwgSy8NtTJ6ol7IfceFj4qV/wX3vwMvnzfykAmeFdW1wjyr1UlTNgCWOcWl9vW2o3JI9O484qTvQqkcfzh31vw2o62wHYpCE6Z4e8ZYGoaATuHmYGiaE8TEY0hoinyr5SDGkrURPR4BfyquvxRqgllAHDh8ROw6IixAJysYMtWm6Hnj3/yjb2Y2JgGkI/yUX0NnmnIzq8i0ykz1jRUTP37ctKbs3D7UxsiBdY/X92Jadc8gJ0HekJfV9F9L8VmVUuNQOZZ6D0EfKYhG1r4aPC7IT/DYogKSZb5AjddegLeM3eSZ2ZSI4TSqQSSLAiYAaKgICCidxDRGwA2AHgCwEYAD5Z4XEOGuNhsI0QQ2JpGcLRiajAN53U5AaiN57t6c5g6yuknK00NezsUQaA4i6VduSZlxpuGBqIXYQn56ePrcO3fXgvkAEh+/29HSwqr56OjO3mL1Yb08E09oUx14lta+GiYQ3dCHwRBFDJfYGx9Na5/z3HeddScgdqUiQTnDTADRME8AgDfAHAKgEeEECcQ0VkA3lfaYR0aJEJMQ50ZyxcJok4WCcNAzs0MBvLJaLYt0JmxMNYNRd3RKgWBahpyJqCclbdTp1OJ2Kgh3VyiIoSItC+/svUAHn59V/SJBwipOUXlOci6SsWs7nXtp1j/iO6sbdUc/fKzTCdNp9aQL3w0XiO47l3HIGsLfPm+lUWNRTJCKy4nL+nXCEzf949hDoZiBEFWCNFCRAYRGUKIx4johyUf2RDiGxcejdljRwS2mz6NwJnU/6VNoKpGYRjOpKYLAjnZjKmvAgDscE0h6upUrnhztpPpmjAIKZNiV75WjEaQsWyfRqLy9p885XseJzRKiXzriunCpieQSeEx4wsPYGJTGl97+1Ghx+nhm7pGIMNHa6sSIeGjwRX56BFV3uNL509BT9byBMGPLzsBG/d24gcPr/EdM3VUja+vgX5eKQfViX/h7DEFu+MxTLEUo1u2ElEdgCUAfktENwEY/oXuFd5/ylTMnx5s9q3GoMtJ60f/WuvbRy1B4eQRCM8hKV/rdCNTRrlJSHJyUssXyGMc05CNqoQBw41CiiJuVRxlnw6jVE7nQrJFCtpicgJ0jcArCyGALfu6ccWvloUeJyd6ORZdEDy73mkuU1tlwgopOgf4J2g9bNRQbvLtx00I7e178bzJ2Hjd+VhwWLM7Fv8bI81/qmnoHcdNwLUXHB16TwzTV4oRBBcA6AbwaQD/ALAOwNtLOahDhTCNQEfVCJw8AoFed6KXr3W6xeniYs+zniBwooaqkyYMihcEcc5ivUZOHMVMxG09WUy75oFIe38Y8rRRfo68aaiIMeoaQchB+qbalOmZhrzy3pogeHz1HlQnDSQMchrThGgEZowg0CN7wvwKctsvLj8JL39tceD1MNMQwwwkcSUmbiai04UQnUIISwiRE0L8SgjxIyFES9RxyvG3E9FuIlqpbf8kEa0ioleJ6HsDcRPlQl25qavCOePyZqQwQdClNZ6XDsk4QSAjhXJu7Hl10oRJVCB81P+imizVJ0FQxEy82TVt3PbUhqLPWwg5OXdncr48izB0Z3GYNqTb/yePrPHeVyNCIwCAmlTCywFRw0ulaS1OEOhm/KoQv4K8z1TCCG0+Y3saAQsCpjTEaQRrAHyfiDYS0feI6IQ+nvuXAM5TN7iO5gsAHCeEOArA9/t4ziGLagL4x9VnerVm1BWgQQRLCM8UJCdmKRhGVCcDE4dE9ijO2gI9OQtVCQNE8Y5UXSNQI2D6axq678VteDTEkSzH0Ze5qljT0OfveQUnf/vR2H0DeQQh78tuRZiMb6jG6bOavYldJpuFCYK0q33pOSKy7HOcINDNPNUhGkFU32VJ3kfAUUJMaYhLKLtJCHEqgDcBaAFwu7uS/yoRHV7oxEKIJQD2aZuvAnCdEKLX3Wd3/4c+tNDVdmnuUDWChOmYF2Sfgo0tnejszWH1rnYATjhoVEhgpysscpaN3qyNKndyivMD6A5UtSdBXzQC9TRX/2FFqL1dDsPoh1NZn7O37u/CtGsewPMb/F8fdcLUBY4eKhtmGtqpCILrLzoONSmnRITjDHe2yy5i5x41Nn8tA54Zbp8S0pv0wjqjBYHO7HEjMGtMna8WUWFB4GoEbBpiSkTBJYYQYpMQ4rtCiBMAXAbgQgCv9/N6hwNYQETPEdETRHRS1I5EdCURLSOiZXv27Onn5QYPuSqUq0Q5DenO4pwtvJX5Q6/uwtk3POFFldRWFU4SylnC9REYMA3ysoyj9lXpryAopqSzpxH0w3yhZ/NKAaDW7wf8oZ667T3KWayyQ0lMMwznsxLC2Vd3FqvVQ23buV5P1vaFusqEQHUsevE7namjavHIZ97kO3+mQOIf+wiYUlNMQlmCiN5ORL+Fk0i2GsC7+nm9BICRcPISPgfgjxQRlyiEuNXtkzxv9OjR/bzc4CEjiNJa+WA1e1hG+agF5tTJqSZlFmw36NQaypuG1Am9NmVi1pg677muLXT0qKahaAGiU4yzWPTRNPTSllbc8fRGZyxaCGeUVqGGeupfG90fYmlNZABg8758iKZJ5L3XW/d3e4JECgI1R8AWAgbl8zq+/c5jsPG680NNQ2EJiB88bRru+JB/zaNqLKcWqkvEpiGmxMQVnTsHjgbwVgDPA/g9gCuFEAcTOroVwL3C+YU+T0Q2gGYAQ3/JXwA5GaTdCUTOQSlfQhnhuQ37IlfjdVWJgj/2bM5xFo+qc1aUqq2fiHxmCt1c0t5PjaAYZ7HUPoo1DV10yzPeY91fEXWKnkx+v6RBUCs66ULPEiK4TXluGuRN2gu//7i3Xb4vqiCwbAHDIK/RvFzNy+MLfWZfe0cwh0EO5XcfORmnzowXBDabhpgSE5eR8gUAvwPwP0KI/QN0vfsAnAXgMdfPkAIwLAqpe4IgJQWB8+NV+8gaRLETcDplehpE0qTQ8M+c7YaPJkxkLBtZ7XzqZKE7UAfCWRyFNO/o9f2jUE9Z7FhUjUAXOLqd3bJFZGN7wNHOUjETa9qnETj3JR3FzXV+QWAYwK3vn4stIc3po5DvaViUkI68i/74XximGOL6Ebz5YE5MRHcBWAigmYi2AvgqgNvhOJ1XAsgAuFwMkzZaXtkHqRG4230+Aq02kTrBnn3EWFQlTG8ir0klQiNYZEJZddJAzrYD9nU1nyHnOkIlUYLgt89tQjpp4l0nToq9tzikgCtmrtrV1uM7p26mispiVktC69FS+qRvCxHr2zCJYutIqdm9jmkoP6YxI5xSIFJoJwwDi48aF3muMOT4dVNiGPOmjcSSNXs4fJQpGSXLURdCXBbx0iFfp+jwsXVYs6vDt82rSSN/2O68pOcRSBKaIPjcubOd/d2JvDZlhgoCp/qohaqEia6MhV6lnv2oupTP2ZzTEqBk4hrgd9B+6c+Os/pgBIFckRezatVDQXu1mvxRZ1A1At1voWs/OSteI1BNQ2GompxjGnIen33EWEwe6dQTkmfvTzlo+Z4WU676p+89EZtauoral2H6A3uf+sFfPnEGXvqqPwN0jFtjZvGRY33b4wSBiuw45WkEEXVksjmB9p4cRlQnYBCh152A508fid/+58m+a+Qsf7XM3YqTWnfQxmELx/H640ffUM7tn7ylhlHIn9kZUmCu17Jh2wJ/eWm7L4JHp8fXG8D/2ktbWgNjjgutNQ2KdMwnTfKF8ap9CI6f3OBpLHIyL9Ycpo9PXqsQtVUJHDmhYlqAMGWAq1b1g+qkGVidTR1Vi+e/uMgrOhYVPiq36XOUbiKoDTEZNNdVYWdbD7qzFppqU9h+oNszybz/lKmY1FTj8xHIAnWSW55Y5z2WGkExlrmcLbCxpctXLK07a2GEMllmi3QWy6Y8Kr1ZG39avgWfv+cV7O/MoLmuKuRIoFtxFhcat2X7NZlUwvD5Z8wYH0HCMHyC+rAxdZ6gUz93WXL6igXTY8cSxjlHjsNdz29GbUzjI4YZLFgjGEDG1Fd7q8V8oTB/+CjgZBvr5hY5Icj5TW9eDwAzmmu9EMjGmqRv0lXt1ZKcFb0qluaYsObpOpYtsGKLfwLX6/jLSbaQINivtfEEHB/BVtfRGva6d80YjSA4ZtVYGn4AABaBSURBVNvnQK7RBKtB5BPSAEIbwIxvqMZtl5/kaVA1ysTdkE5i43Xn4+J5k+MHE8K1FxyF5764KPRzZpjBhgVBibj1A/OwaM4YX/RJwhMEps+RSRQsPVwTslKc3lzrPW6qSfkmXekkTug+ggg7+Zpd7djd3uNV14zDsgVe2OQ3vXQFBIEs3uaYjRZ871/4+ys7AucKi4TqVZq4VyfNSCevz0dQQBLo/YX1lXeYj6DJzfZNmoYnQM+aMwYja1OeBpVODcxPJmkaXv8Jhik3vBwpEafPasbps5p926RGUJ30m4bSSTOvSbjbakNaZM4Y7RcEPp+D1AjU8FHLDk0GWzRnDB54ZQfufXFbUfkEli2wYa8/feSjdy7HQ58+U7lW3jTU2Wthy75ufP7ul/HWY8Zr5wperzeXb79ZnTAinbw9rvApxpy1rbXblx+gawRhUUNNtSlsbOlyKo0Kv/1falBpdtgywxDWCAYROanopYjVyUVOcgU1gtqkz6ma9DQC1W5vh66cT5vVjPaeXNFJZbYQ2Lq/y7dN1keSeMXbiLwm8T0h2cthGkFPxvJCSA0jutmO1AiKKUv9q6WbfM9157thBLOAm2qc/ICEQZ4wksI2rxHw2okZfrAgGEQSnkag2auVlb2nEYQ4i9XyEbppqCqkNn53xgr1EYzro0kikxOBuj8qu9t6cP1DqwE4piE52YcmxIUUWNu6vwvd7oq7N2tH+jXygqD4vAZJjfaeO87iCEFgGoHSz71Ky0qGGW6wIBhEVGexb7uyspdznG7KAPwaQWNN0he2KG3gqmmoozcXWoVTbacoiYuF39nWHdvkRgoBIGib11En+ROmNOLr7zgKnRkL63Y7eRk9WSvaR+CahvrTMS3UNJTw37P0ESTMfFVX030/ZdQQCwJmOMKCYBCRE7euEailiz3TkGLKkCWLiQi/+vB8XDJvMqoSpi9mf0S1KwiUCb0rQiNoqgmWNYiLZ9+4tyt0uxQy6hUeXbUbz2/Uq4/nUQVKyjRw9EQnPv61HW0AHHNSlI+gN2fh7uVbcfp1/4o8fxRhWphuGpJN42XvASD/mQ20s5hhhhJs8BxEpPlG1wjOVcoT5J3F+Y/mic+e5dXSf9Pho/Gmw51qrKppSO6vJkJ19OZCnbNhfXOTMZlgatVOAJjYmMa21m5kbRtVhhnIBP7vu170jeFAdxY5y8bUUbU+05BlC8xorvMd25MN92sAzqr8s396KXKcceiCzqSgaaha+XxkRM+kphoAeU2NfQTMcIS/1YPIaDdRSq2l859nTMfHFs7ynnt5BIopo6EmiYaQVbxfEDj7q6aL9p4clq4Lhoc2htTMjwv/39TS6Y2pM2N5yW8/e2I9PvnmWdEHAnjHj5/Cejfi6InPLfSVgjj7yLGo0aKjerIWshGmoULO7QWHNePJN8JrGOo5A6YZ1Aik1tBcV4V3nzgRjekkFh0xxrcPm4aY4QgLgkFkbIOzyjzQncVDV5+Jl7a04uKTwpORwqKGdKQVKGUaXv9caSKSfPn+VwPHJUwDDemkr5ZRXDmGTS1dGD2iCkIIdGYsz95+w8NrMG9qk6/hjc56Jex0b0cGOctGwiA8/6WzPRMVUV4A9mTtyNyHsPIUkvnTR+K6dx8baTbSJ30zJKFM7jOqNgUiwtlauRCABQEzPGGD5yAio3VauzKYPW5EqBCQoZdheQQ60vms7juiiLLGQNBPIAVBmHO5pTODSU1pyHJwqr29M2P5+vjGkTAcJ2zCJIx0J1siQnUif77eXLhfAwC2t/ob2P/f+Ufgv940E4AzsU9sTGPjdeeHHhvQCAwKmIukYJS9HsLQE/8YZjjA3+pBRNqdw6qKSvJRQ8VoBFIQ5PfVNYIo9PNL232UWWbaqFrPfKRG4Fi2HVsWQsU0CFnLDvgj1Mm1J2v7ooZGKPe2q90vCFIJA9Oba7xzx6ELAiMkoUzex6iIWkdAdIlshjmUYdPQICI1gv1dhQWBXOVfNn9K5L5y7qtTBUGRtWv0la0tgC37uny9dFUuOH4Cnlnn2N9V80jWEtjXGX0/OjlLBDptqZO0rhG8afZoHDWhAfev2IZVO/1JbETkCcNC87PuGDYNgurmvueq0zCpKY0Neztx2UnB93zm6Fqs23MwzfkYZujCgmAQqU8n0FSTxGcWz47cR4aPViVMrPz6ubE2aWka8gmCIk1DYbXtF3zvMbz0lcUhewNnHjYahGBfZtkf4YozpmPFltbQ6qKSrOWs9hNmcHUu6claPh9ByjRw1cKZeHZ9S0AQGJTXBPx9g4Pd3fTVv0H+1f3cqU0AgJ9/YF7o2P/88dNxIEaAM8yhDJuGBhEiwotfWYz3nzI1ch85fSUMQl1VItbk0RfT0G//82Tfcz2EVdKVDXfIGgZ5GohqGurKWMjkbKQSRmjWsErOFshawtc8B8hrQYA0Dfl7CwNB0w7g3L98XRUm/xGiRemCQBUCY+ujTUGS+uokJo+sKbgfwxyKsEYwxJCTYjFdr2SyU10RgkA3BUV1u9qjNK/RkZOnqqV09OSQsWykTCOwCjfIXxcoa9nIWbaXrRuGnlkstQfdtAM4958XBPntX37bkTj7yLF4/y+ez58n4v3840dPxbRRPMEzlQ1rBIcwclGrloyQpqGUaeATZ83CKNfmr/cJiBIE24powK4mVbW6ju8qt4eyiu6QtmyBrC1ik9d6cpYvoSyhaQS1KdOL9iHKC0P1/hKmgQWHjfadNyraZ/70kRjD5aCZCocFwRBDOocbQxLIdNp7HDPORLdTFpDXCDKWjc+eOxuzx40AALT1+E0+URPjG7s7AtvkvnL+VjWC/W7oaEqp4Z8/zi9snD7CdsBZLBlZm0J3xvJpFnJfqRE01qS8nAmDyNM44jSoD5w6FZeEOIAZhnFgQTDE+O9Fs7D2W28pKnx0xwFn9T5BEQRy8v3Q6dMAAOcf6/QDGKMVmqtKhGsEa7Ty0gBwnlsCQzqLVR/Bv1btds8X7COgF3pzTEPCVyobyOdONNel0N6Ti9UIGtJJz79hGPlKpHGd0f7nnNmhPgaGYRzYRzDEIKLIFbPOjgNOXP34Rr9pY8N33urZ89978lScfcRYjK2vxjcuPBoz3eY2URPjG7v8GsE/rl7g1QOSi241ami361OoSpiBGkG6IMhJ05AZ7ixurqvCml0dvjaYh411NBrp7K1PJ9Da5QoCyjeQMWI0gjifBMMwLAgOaeTKWDUNAcGkJ5nIpkYrRa2g1+xu9zl554yrD5w3LKQ1lTB8PYIBv8AA8s5iPXxUIpvWt3bnE9TefuwE7/wAUFeVRFUybxqS14yb66McxQzDOLAgOIT58WUnYun6vf3qfRsxF0MIJ7N2b0cwekhOp2GO5lBBoO1n2cIpMaGHj7r/PUHQlcXhY+tw91WnecJECoL66kTeNKT4CMIEm+l2OysmAothKhk2nB7CjGuoxjtPmNSvY+XEOao2hWveMsf3WnNErR0514aZrsKcxbogkM5iPabfu+4I57qtXVlUJUzUK8lxKfeaVUnTEwoG5WsjhZV+uP/jp+NjC2d6gmf5/50del2GqXRYI6hQDM+HMAWHj/X3BJgxujaQxQvkJ9uw8M+qZNBZrJuGnt3Qghc2t2LhbH9op0Q1DY1r8Gs5cvKvShieRkCKjyBMthw9sQFHT2zwno+qq8I/rl6ArfsKh8gyTCXBgqBCkeYSWyAQxXPMxEaMb0jj2EkNvu1GAY3gzivm46JblnrbdI3g3he2OcdHmGpkz+CerB1wKEurU1XS8ExTplFc+KjKnHH1Pr8HwzAlNA0R0e1EtJuIVirbvkZE24hohfv31lJdn4lHTpyWCNrQZ42pw5ffdiQuOH6ib/sNFx+Ps48YgzluboJKKmFg3rSRuOHi47xtYX2XAaCz1/I9v3LBDADAeEUL0E09vTnnmKqE6WkEQghMccs+HDmeJ3eG6S+l1Ah+CeAnAH6tbb9RCPH9El6XKQI5z9pKs/kTpzTiPfMmY9GcMaHHHD2xAbddfpI3KavIvAQ1LLXaFQSytaVEL8P9kTNn4CNnzsCutnyZ6ec3+Psey+bxjmnIOW/GsrH4sGb87ZNn4KgJLAgYpr+UTCMQQiwBEN3FnCkrsjSDLYRXGqI+ncRl86fExuQD4T4CKQDUmkDSNLT4KH+nr7ae8CqedTEltHuzeUEgryW3HT2xgfsEMMxBUA4fwSeI6AMAlgH4HyFEdN1ipmRMb651/9d5JR10X0EUYYJCmmtUjUCu3HWfQFRjnpqUiUvmTUZTbQpv1rSSvGko7yzOFKh2yjBMcQx2+OhPAcwEcDyAHQB+ELUjEV1JRMuIaNmePXsGa3wVw+KjxuGeq07FZfMne6Yh3UEbx+fOnY37P36691xOzmpJbLlIdxrI5I9t7wkvdU1E+O5Fx+Kat8zB/Okjfa9dccZ0TG+uxVuOGe85i3uzQRMVwzB9Z1A1AiHELvmYiH4O4G8x+94K4FYAmDdvXnRndabfzJ3qTLanzRyFiY1pfPysWUUfq+8rNQG1F7Kc+4UQSJgGMrn+r+BnjK7DY59dCCAvdHoO4nwMw+QZVEFAROOFEDvcp+8EsDJuf2ZwaKxJ4elr3nxQ55CCoLEmn4wmNQIhgKRBkIUjZEG8/nLCFKebmJ7/wDBM/yiZICCiuwAsBNBMRFsBfBXAQiI6Hk5VgY0APlqq6zODi1cmOq1qBPlcBRmi+pEF0/Gl8488qGudd/Q4LPncWZjCDWUYZkAomSAQQlwWsvkXpboeU15kIbmognJSEBRTXrsYWAgwzMDBmcXMQXHbB+bhry9vD33NMw1BeJFJtVXhSWYMw5QPFgTMQXH2kWNx9pFjQ1+Tsf1CwHMUD5RGwDDMwMHVR5mSoQajyph/1ggYZujByzNmwHnfKVPw2vY2JWooH/3bn94JDMOUFhYEzIDzzQuPAQD8a5WTNjJrTD7MU7a9ZBhm6MCmIaZkvHnOWNxz1al4n9Iic2x9VRlHxDBMGKwRMCVFZi9LuDgcwww9WBAwg8KNlxwHs8iidgzDDC4sCJhBob+9lRmGKT28RGMYhqlwWBAwDMNUOCwIGIZhKhwWBAzDMBUOCwKGYZgKhwUBwzBMhcOCgGEYpsJhQcAwDFPhsCBgGIapcFgQMAzDVDgsCBiGYSocFgQMwzAVDgsChmGYCocFAcMwTIXDgoBhGKbCYUHAMAxT4bAgYBiGqXBYEDAMw1Q4LAgYhmEqnJIJAiK6nYh2E9HKkNf+h4gEETWX6voMwzBMcZRSI/glgPP0jUQ0GcBiAJtLeG2GYRimSEomCIQQSwDsC3npRgD/C0CU6toMwzBM8SQG82JEdAGAbUKIl4io0L5XArjSfdpBRKv7edlmAHv7eexwgO+f75/vv3KZWsxOgyYIiKgGwBfhmIUKIoS4FcCtA3DdZUKIeQd7nkMVvn++f77/yr3/YhnMqKGZAKYDeImINgKYBOAFIho3iGNgGIZhNAZNIxBCvAJgjHzuCoN5QohKVtsYhmHKTinDR+8CsBTAbCLaSkRXlOpaBTho89IhDt9/ZcP3zxSEhODgHYZhmEqGM4sZhmEqHBYEDMMwFc6wFQREdB4RrSaitUR0TbnHM1AQ0WQieoyIXiOiV4noU+72kUT0MBG94f5vcrcTEf3IfR9eJqITlXNd7u7/BhFdXq576g9EZBLRi0T0N/f5dCJ6zr3PPxBRyt1e5T5f674+TTnHF9ztq4no3PLcSd8hokYiupuIVhHR60R0agV+/p92v/8rieguIqqupO/AgCOEGHZ/AEwA6wDMAJAC8BKAI8s9rgG6t/EATnQfjwCwBsCRAL4H4Bp3+zUAvus+fiuABwEQgFMAPOduHwlgvfu/yX3cVO7768P78BkAvwPwN/f5HwFc6j6+BcBV7uOPAbjFfXwpgD+4j490vxdVcMKa1wEwy31fRd77rwD8p/s4BaCxkj5/ABMBbACQVj77D1bSd2Cg/4arRjAfwFohxHohRAbA7wFcUOYxDQhCiB1CiBfcx+0AXofzw7gAzgQB9/+F7uMLAPxaODwLoJGIxgM4F8DDQoh9Qoj9AB5GSG2ooQgRTQJwPoDb3OcE4M0A7nZ30e9fvi93A1jk7n8BgN8LIXqFEBsArIXzvRnSEFEDgDMB/AIAhBAZIUQrKujzd0kASBNRAkANgB2okO9AKRiugmAigC3K863utmGFq+KeAOA5AGOFEDvcl3YCGOs+jnovDuX36Idw6lXZ7vNRAFqFEDn3uXov3n26rx9w9z9U7386gD0A7nBNY7cRUS0q6PMXQmwD8H04hSt3wPlMl6NyvgMDznAVBMMeIqoDcA+Aq4UQbeprwtF7h2VcMBG9DcBuIcTyco+lTCQAnAjgp0KIEwB0wjEFeQznzx8AXP/HBXCE4gQAtTi0tJkhx3AVBNsATFaeT3K3DQuIKAlHCPxWCHGvu3mXq/LD/b/b3R71Xhyq79HpAN7hZqb/Ho454CY4Jg+ZKa/ei3ef7usNAFpw6N7/VgBbhRDPuc/vhiMYKuXzB4CzAWwQQuwRQmQB3Avne1Ep34EBZ7gKgn8DOMyNIkjBcRD9pcxjGhBc2+YvALwuhLhBeekvAGTkx+UA7le2f8CNHjkFwAHXhPAQgMVE1OSusBa724Y0QogvCCEmCSGmwflc/yWEeC+AxwBc5O6m3798Xy5y9xfu9kvdiJLpAA4D8Pwg3Ua/EULsBLCFiGa7mxYBeA0V8vm7bAZwChHVuL8H+R5UxHegJJTbW12qPzjREmvgRAJ8qdzjGcD7OgOO2v8ygBXu31vh2DwfBfAGgEcAjHT3JwA3u+/DK3DqO8lzfRiOg2wtgA+V+9768V4sRD5qaAacH/FaAH8CUOVur3afr3Vfn6Ec/yX3fVkN4C3lvp8+3PfxAJa534H74ET9VNTnD+DrAFYBWAngTjiRPxXzHRjoPy4xwTAMU+EMV9MQwzAMUyQsCBiGYSocFgQMwzAVDgsChmGYCocFAcMwTIXDgoBhFIjoS25Vy5eJaAURnUxEVxNRTbnHxjClgsNHGcaFiE4FcAOAhUKIXiJqhlPd8xlwf21mGMMaAcPkGQ9grxCiFwDcif8iOPVsHiOixwCAiBYT0VIieoGI/uTWfQIRbSSi7xHRK0T0PBHNcre/x62b/xIRLSnPrTFMNKwRMIyLO6E/Baes8SNw6tY/4dY1mieE2OtqCffCyULtJKLPw8lgvdbd7+dCiG8R0QcAXCyEeBsRvQLgPCHENiJqFE7ZaIYZMrBGwDAuQogOAHMBXAmn1PMfiOiD2m6nwGlo8jQRrYBTw2aq8vpdyv9T3cdPA/glEX0ETtMkhhlSJArvwjCVgxDCAvA4gMfdlbzewpHgNHS5LOoU+mMhxH8R0clwmuksJ6K5QoiWgR05w/Qf1ggYxoWIZhPRYcqm4wFsAtAOpy0oADwL4HTF/v//27ljE4RjIIrD71UWWugMgns4hbiAU4gLWDqEK7iBWOoQ7iAIZ5EDUyqI/+J+X5MmB0n1uARubHvR1ay69Zx75hFxiYidWqfRjz4GBkdHALxNJB1sTyU91aZVbiStJZ1s3yNimc9FR9ujrNuqTbqVpJntm6RH1knSPgPGahNCr3+5DfAhPouBH+k/lYc+C/ANnoYAoDg6AgAojo4AAIojCACgOIIAAIojCACgOIIAAIp7ASk4f2RpzNUVAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "plt.plot(average_mae_history[:,0],average_mae_history[:,1])\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Validation MAE')\n",
+    "plt.ylim((14, 20))\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's plot this:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "According to this plot, it seems that validation MAE stops improving significantly after 150 epochs. Past that point, we start overfitting.\n",
+    "\n",
+    "Once we are done tuning other parameters of our model (besides the number of epochs, we could also adjust the size of the hidden layers), we \n",
+    "can train a final \"production\" model on all of the training data, with the best parameters, then look at its performance on the test data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasMeanSquaredError\n",
+      "creating: createZooKerasMAE\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Get a fresh, compiled model.\n",
+    "model = build_model()\n",
+    "# Train it on the entirety of the data.\n",
+    "model.fit(train_data, train_targets,\n",
+    "          nb_epoch=150, batch_size=16)\n",
+    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 51,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1.7991065979003906"
+      ]
+     },
+     "execution_count": 51,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_mae_score"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are still off by about \\$1,800."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Wrapping up\n",
+    "\n",
+    "\n",
+    "Here's what you should take away from this example:\n",
+    "\n",
+    "* Regression is done using different loss functions from classification; Mean Squared Error (MSE) is a commonly used loss function for \n",
+    "regression.\n",
+    "* Similarly, evaluation metrics to be used for regression differ from those used for classification; naturally the concept of \"accuracy\" \n",
+    "does not apply for regression. A common regression metric is Mean Absolute Error (MAE).\n",
+    "* When features in the input data have values in different ranges, each feature should be scaled independently as a preprocessing step.\n",
+    "* When there is little data available, using K-Fold validation is a great way to reliably evaluate a model.\n",
+    "* When little training data is available, it is preferable to use a small network with very few hidden layers (typically only one or two), \n",
+    "in order to avoid severe overfitting.\n",
+    "\n",
+    "This example concludes our series of three introductory practical examples. You are now able to handle common types of problems with vector data input:\n",
+    "\n",
+    "* Binary (2-class) classification.\n",
+    "* Multi-class, single-label classification.\n",
+    "* Scalar regression.\n",
+    "\n",
+    "In the next chapter, you will acquire a more formal understanding of some of the concepts you have encountered in these first examples, \n",
+    "such as data preprocessing, model evaluation, and overfitting."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From 6ec5eb4acc5fddc2d2336f0cf1dbda65c5edb0b0 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 21 Mar 2019 08:57:44 +0800
Subject: [PATCH 32/46] Delete 3.7-regression.ipynb

---
 keras/3.7-regression.ipynb | 797 -------------------------------------
 1 file changed, 797 deletions(-)
 delete mode 100644 keras/3.7-regression.ipynb

diff --git a/keras/3.7-regression.ipynb b/keras/3.7-regression.ipynb
deleted file mode 100644
index 0f37622..0000000
--- a/keras/3.7-regression.ipynb
+++ /dev/null
@@ -1,797 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "First of all, set environment variables and initialize spark context:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "env: SPARK_DRIVER_MEMORY=8g\n",
-      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
-      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
-     ]
-    }
-   ],
-   "source": [
-    "%env SPARK_DRIVER_MEMORY=8g\n",
-    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
-    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
-    "\n",
-    "from zoo.common.nncontext import *\n",
-    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Predicting house prices: a regression example\n",
-    "\n",
-    "\n",
-    "----\n",
-    "\n",
-    "\n",
-    "In our two previous examples, we were considering classification problems, where the goal was to predict a single discrete label of an \n",
-    "input data point. Another common type of machine learning problem is \"regression\", which consists of predicting a continuous value instead \n",
-    "of a discrete label. For instance, predicting the temperature tomorrow, given meteorological data, or predicting the time that a \n",
-    "software project will take to complete, given its specifications.\n",
-    "\n",
-    "Do not mix up \"regression\" with the algorithm \"logistic regression\": confusingly, \"logistic regression\" is not a regression algorithm, \n",
-    "it is a classification algorithm."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## The Boston Housing Price dataset\n",
-    "\n",
-    "\n",
-    "We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the \n",
-    "suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
-    "\n",
-    "The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in \n",
-    "total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has \n",
-    "a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, \n",
-    "others between 0 and 100...\n",
-    "\n",
-    "Let's take a look at the data:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras.datasets import boston_housing\n",
-    "(train_data, train_targets), (test_data, test_targets) =  boston_housing.load_data()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "(404, 13)"
-      ]
-     },
-     "execution_count": 3,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "train_data.shape"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "(102, 13)"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "test_data.shape"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "As you can see, we have 404 training samples and 102 test samples. The data comprises 13 features. The 13 features in the input data are as \n",
-    "follow:\n",
-    "\n",
-    "1. Per capita crime rate.\n",
-    "2. Proportion of residential land zoned for lots over 25,000 square feet.\n",
-    "3. Proportion of non-retail business acres per town.\n",
-    "4. Charles River dummy variable (= 1 if tract bounds river; 0 otherwise).\n",
-    "5. Nitric oxides concentration (parts per 10 million).\n",
-    "6. Average number of rooms per dwelling.\n",
-    "7. Proportion of owner-occupied units built prior to 1940.\n",
-    "8. Weighted distances to five Boston employment centres.\n",
-    "9. Index of accessibility to radial highways.\n",
-    "10. Full-value property-tax rate per $10,000.\n",
-    "11. Pupil-teacher ratio by town.\n",
-    "12. 1000 * (Bk - 0.63) ** 2 where Bk is the proportion of Black people by town.\n",
-    "13. % lower status of the population.\n",
-    "\n",
-    "The targets are the median values of owner-occupied homes, in thousands of dollars:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "array([22.6, 50. , 23. ,  8.3, 21.2, 19.9, 20.6, 18.7, 16.1, 18.6,  8.8,\n",
-       "       17.2, 14.9, 10.5, 50. , 29. , 23. , 33.3, 29.4, 21. , 23.8, 19.1,\n",
-       "       20.4, 29.1, 19.3, 23.1, 19.6, 19.4, 38.7, 18.7, 14.6, 20. , 20.5,\n",
-       "       20.1, 23.6, 16.8,  5.6, 50. , 14.5, 13.3, 23.9, 20. , 19.8, 13.8,\n",
-       "       16.5, 21.6, 20.3, 17. , 11.8, 27.5, 15.6, 23.1, 24.3, 42.8, 15.6,\n",
-       "       21.7, 17.1, 17.2, 15. , 21.7, 18.6, 21. , 33.1, 31.5, 20.1, 29.8,\n",
-       "       15.2, 15. , 27.5, 22.6, 20. , 21.4, 23.5, 31.2, 23.7,  7.4, 48.3,\n",
-       "       24.4, 22.6, 18.3, 23.3, 17.1, 27.9, 44.8, 50. , 23. , 21.4, 10.2,\n",
-       "       23.3, 23.2, 18.9, 13.4, 21.9, 24.8, 11.9, 24.3, 13.8, 24.7, 14.1,\n",
-       "       18.7, 28.1, 19.8, 26.7, 21.7, 22. , 22.9, 10.4, 21.9, 20.6, 26.4,\n",
-       "       41.3, 17.2, 27.1, 20.4, 16.5, 24.4,  8.4, 23. ,  9.7, 50. , 30.5,\n",
-       "       12.3, 19.4, 21.2, 20.3, 18.8, 33.4, 18.5, 19.6, 33.2, 13.1,  7.5,\n",
-       "       13.6, 17.4,  8.4, 35.4, 24. , 13.4, 26.2,  7.2, 13.1, 24.5, 37.2,\n",
-       "       25. , 24.1, 16.6, 32.9, 36.2, 11. ,  7.2, 22.8, 28.7, 14.4, 24.4,\n",
-       "       18.1, 22.5, 20.5, 15.2, 17.4, 13.6,  8.7, 18.2, 35.4, 31.7, 33. ,\n",
-       "       22.2, 20.4, 23.9, 25. , 12.7, 29.1, 12. , 17.7, 27. , 20.6, 10.2,\n",
-       "       17.5, 19.7, 29.8, 20.5, 14.9, 10.9, 19.5, 22.7, 19.5, 24.6, 25. ,\n",
-       "       24.5, 50. , 14.3, 11.8, 31. , 28.7, 16.2, 43.5, 25. , 22. , 19.9,\n",
-       "       22.1, 46. , 22.9, 20.2, 43.1, 34.6, 13.8, 24.3, 21.5, 24.4, 21.2,\n",
-       "       23.8, 26.6, 25.1,  9.6, 19.4, 19.4,  9.5, 14. , 26.5, 13.8, 34.7,\n",
-       "       16.3, 21.7, 17.5, 15.6, 20.9, 21.7, 12.7, 18.5, 23.7, 19.3, 12.7,\n",
-       "       21.6, 23.2, 29.6, 21.2, 23.8, 17.1, 22. , 36.5, 18.8, 21.9, 23.1,\n",
-       "       20.2, 17.4, 37. , 24.1, 36.2, 15.7, 32.2, 13.5, 17.9, 13.3, 11.7,\n",
-       "       41.7, 18.4, 13.1, 25. , 21.2, 16. , 34.9, 25.2, 24.8, 21.5, 23.4,\n",
-       "       18.9, 10.8, 21. , 27.5, 17.5, 13.5, 28.7, 14.8, 19.1, 28.6, 13.1,\n",
-       "       19. , 11.3, 13.3, 22.4, 20.1, 18.2, 22.9, 20.6, 25. , 12.8, 34.9,\n",
-       "       23.7, 50. , 29. , 30.1, 22. , 15.6, 23.3, 30.1, 14.3, 22.8, 50. ,\n",
-       "       20.8,  6.3, 34.9, 32.4, 19.9, 20.3, 17.8, 23.1, 20.4, 23.2,  7. ,\n",
-       "       16.8, 46.7, 50. , 22.9, 23.9, 21.4, 21.7, 15.4, 15.3, 23.1, 23.9,\n",
-       "       19.4, 11.9, 17.8, 31.5, 33.8, 20.8, 19.8, 22.4,  5. , 24.5, 19.4,\n",
-       "       15.1, 18.2, 19.3, 27.1, 20.7, 37.6, 11.7, 33.4, 30.1, 21.4, 45.4,\n",
-       "       20.1, 20.8, 26.4, 10.4, 21.8, 32. , 21.7, 18.4, 37.9, 17.8, 28. ,\n",
-       "       28.2, 36. , 18.9, 15. , 22.5, 30.7, 20. , 19.1, 23.3, 26.6, 21.1,\n",
-       "       19.7, 20. , 12.1,  7.2, 14.2, 17.3, 27.5, 22.2, 10.9, 19.2, 32. ,\n",
-       "       14.5, 24.7, 12.6, 24. , 24.1, 50. , 16.1, 43.8, 26.6, 36.1, 21.8,\n",
-       "       29.9, 50. , 44. , 20.6, 19.6, 28.4, 19.1, 22.3, 20.9, 28.4, 14.4,\n",
-       "       32.7, 13.8,  8.5, 22.5, 35.1, 31.6, 17.8, 15.6])"
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "train_targets"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The prices are typically between \\$10,000  and  \\$50,000. If that sounds cheap, remember this was the mid-1970s, and these prices are not \n",
-    "inflation-adjusted."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Preparing the data\n",
-    "\n",
-    "\n",
-    "It would be problematic to feed into a neural network values that all take wildly different ranges. The network might be able to \n",
-    "automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice to deal \n",
-    "with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we \n",
-    "will subtract the mean of the feature and divide by the standard deviation, so that the feature will be centered around 0 and will have a \n",
-    "unit standard deviation. This is easily done in Numpy:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "mean = train_data.mean(axis=0)\n",
-    "train_data -= mean\n",
-    "std = train_data.std(axis=0)\n",
-    "train_data /= std\n",
-    "\n",
-    "test_data -= mean\n",
-    "test_data /= std"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Note that the quantities that we use for normalizing the test data have been computed using the training data. We should never use in our \n",
-    "workflow any quantity computed on the test data, even for something as simple as data normalization."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Building our network\n",
-    "\n",
-    "\n",
-    "Because so few samples are available, we will be using a very small network with two \n",
-    "hidden layers, each with 64 units. In general, the less training data you have, the worse overfitting will be, and using \n",
-    "a small network is one way to mitigate overfitting."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from zoo.pipeline.api.keras import models\n",
-    "from zoo.pipeline.api.keras import layers\n",
-    "\n",
-    "def build_model():\n",
-    "    # Because we will need to instantiate\n",
-    "    # the same model multiple times,\n",
-    "    # we use a function to construct it.\n",
-    "    model = models.Sequential()\n",
-    "    model.add(layers.Dense(64, activation='relu',\n",
-    "                           input_shape=(train_data.shape[1],)))\n",
-    "    model.add(layers.Dense(64, activation='relu'))\n",
-    "    model.add(layers.Dense(1))\n",
-    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
-    "    return model"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Our network ends with a single unit, and no activation (i.e. it will be linear layer). \n",
-    "This is a typical setup for scalar regression (i.e. regression where we are trying to predict a single continuous value). \n",
-    "Applying an activation function would constrain the range that the output can take; for instance if \n",
-    "we applied a `sigmoid` activation function to our last layer, the network could only learn to predict values between 0 and 1. Here, because \n",
-    "the last layer is purely linear, the network is free to learn to predict values in any range.\n",
-    "\n",
-    "Note that we are compiling the network with the `mse` loss function -- Mean Squared Error, the square of the difference between the \n",
-    "predictions and the targets, a widely used loss function for regression problems.\n",
-    "\n",
-    "We are also monitoring a new metric during training: `mae`. This stands for Mean Absolute Error. It is simply the absolute value of the \n",
-    "difference between the predictions and the targets. For instance, a MAE of 0.5 on this problem would mean that our predictions are off by \n",
-    "\\$500 on average."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Validating our approach using K-fold validation\n",
-    "\n",
-    "\n",
-    "To evaluate our network while we keep adjusting its parameters (such as the number of epochs used for training), we could simply split the \n",
-    "data into a training set and a validation set, as we were doing in our previous examples. However, because we have so few data points, the \n",
-    "validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot \n",
-    "depending on _which_ data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a \n",
-    "high _variance_ with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
-    "\n",
-    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions \n",
-    "(typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining \n",
-    "partition. The validation score for the model used would then be the average of the K validation scores obtained."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Then let's start our training:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "processing fold # 0\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n",
-      "processing fold # 1\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n",
-      "processing fold # 2\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n",
-      "processing fold # 3\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n"
-     ]
-    }
-   ],
-   "source": [
-    "import numpy as np\n",
-    "\n",
-    "k = 4\n",
-    "num_val_samples = len(train_data) // k\n",
-    "num_nb_epoch = 50\n",
-    "all_scores = []\n",
-    "for i in range(k):\n",
-    "    print('processing fold #', i)\n",
-    "    # Prepare the validation data: data from partition # k\n",
-    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
-    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
-    "\n",
-    "    # Prepare the training data: data from all other partitions\n",
-    "    partial_train_data = np.concatenate(\n",
-    "        [train_data[:i * num_val_samples],\n",
-    "         train_data[(i + 1) * num_val_samples:]],\n",
-    "        axis=0)\n",
-    "    partial_train_targets = np.concatenate(\n",
-    "        [train_targets[:i * num_val_samples],\n",
-    "         train_targets[(i + 1) * num_val_samples:]],\n",
-    "        axis=0)\n",
-    "\n",
-    "    # Build the model (already compiled)\n",
-    "    model = build_model()\n",
-    "    # Train the model (in silent mode, verbose=0)\n",
-    "    #model.fit(partial_train_data, partial_train_targets,\n",
-    "    #          nb_epoch=num_nb_epoch, batch_size=1, verbose=0)\n",
-    "    model.fit(partial_train_data, partial_train_targets,\n",
-    "              nb_epoch=num_nb_epoch, batch_size=16)\n",
-    "\n",
-    "    # Evaluate the model on the validation data\n",
-    "    #val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
-    "    val_mse, val_mae = model.evaluate(val_data, val_targets)\n",
-    "    all_scores.append(val_mae)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "_INFO - Trained 16 records in 0.011235845 seconds. Throughput is 1424.0139 records/second. Loss is 8.708786._\n",
-    "\n",
-    "_INFO - Trained 16 records in 0.009535034 seconds. Throughput is 1678.0223 records/second. Loss is 5.3613434._\n",
-    "\n",
-    "_INFO - Trained 16 records in 0.008636178 seconds. Throughput is 1852.6713 records/second. Loss is 18.106756._\n",
-    "\n",
-    "_INFO - Trained 16 records in 0.009207628 seconds. Throughput is 1737.6897 records/second. Loss is 7.0931993._"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[3.291872501373291, 2.496018171310425, 2.221175193786621, 2.6994853019714355]"
-      ]
-     },
-     "execution_count": 9,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "all_scores"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "2.677137792110443"
-      ]
-     },
-     "execution_count": 10,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "np.mean(all_scores)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "As you can notice, the different runs do indeed show rather different validation scores, from 2.1 to 2.9. Their average (2.4) is a much more \n",
-    "reliable metric than any single of these scores -- that's the entire point of K-fold cross-validation. In this case, we are off by \\\\$2,400 on \n",
-    "average, which is still significant considering that the prices range from \\\\$10,000 to \\\\$50,000. \n",
-    "\n",
-    "Let's try training the network for a bit longer: 500 epochs. To keep a record of how well the model did at each epoch, we will modify our training loop \n",
-    "to save the per-epoch validation score log:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 13,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "processing fold # 0\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n",
-      "processing fold # 1\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n",
-      "processing fold # 2\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n",
-      "processing fold # 3\n",
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n"
-     ]
-    }
-   ],
-   "source": [
-    "num_epochs = 500\n",
-    "all_mae_histories = []\n",
-    "for i in range(k):\n",
-    "    print('processing fold #', i)\n",
-    "    # Prepare the validation data: data from partition # k\n",
-    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
-    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
-    "\n",
-    "    # Prepare the training data: data from all other partitions\n",
-    "    partial_train_data = np.concatenate(\n",
-    "        [train_data[:i * num_val_samples],\n",
-    "         train_data[(i + 1) * num_val_samples:]],\n",
-    "        axis=0)\n",
-    "    partial_train_targets = np.concatenate(\n",
-    "        [train_targets[:i * num_val_samples],\n",
-    "         train_targets[(i + 1) * num_val_samples:]],\n",
-    "        axis=0)\n",
-    "\n",
-    "    # Build the model (already compiled)\n",
-    "    model = build_model()\n",
-    "    # Train the model (in silent mode, verbose=0)\n",
-    "    import time\n",
-    "    dir_name = '3-7 ' + str(time.ctime())\n",
-    "    model.set_tensorboard('./', dir_name)\n",
-    "    history = model.fit(partial_train_data, partial_train_targets,\n",
-    "                        validation_data=(val_data, val_targets),\n",
-    "                        nb_epoch=num_epochs, batch_size=16)\n",
-    "    \n",
-    "    #mae_history = history.history['val_mean_absolute_error']\n",
-    "    mae_history = model.get_validation_summary(\"Loss\")\n",
-    "    all_mae_histories.append(mae_history)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We can then compute the average of the per-epoch MAE scores for all folds:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 47,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "array([[[1.90000000e+01, 4.05375427e+02, 1.55307042e+09],\n",
-       "        [3.80000000e+01, 2.64351837e+02, 1.55307042e+09],\n",
-       "        [5.70000000e+01, 1.50977859e+02, 1.55307042e+09],\n",
-       "        ...,\n",
-       "        [9.46200000e+03, 2.07635689e+01, 1.55307053e+09],\n",
-       "        [9.48100000e+03, 2.02473850e+01, 1.55307053e+09],\n",
-       "        [9.50000000e+03, 2.02105141e+01, 1.55307053e+09]],\n",
-       "\n",
-       "       [[1.90000000e+01, 4.76980957e+02, 1.55307053e+09],\n",
-       "        [3.80000000e+01, 3.29584198e+02, 1.55307053e+09],\n",
-       "        [5.70000000e+01, 1.80655548e+02, 1.55307053e+09],\n",
-       "        ...,\n",
-       "        [9.46200000e+03, 1.73588219e+01, 1.55307064e+09],\n",
-       "        [9.48100000e+03, 1.78555279e+01, 1.55307064e+09],\n",
-       "        [9.50000000e+03, 1.73744106e+01, 1.55307064e+09]],\n",
-       "\n",
-       "       [[1.90000000e+01, 4.62182434e+02, 1.55307064e+09],\n",
-       "        [3.80000000e+01, 3.34037567e+02, 1.55307064e+09],\n",
-       "        [5.70000000e+01, 2.06141006e+02, 1.55307064e+09],\n",
-       "        ...,\n",
-       "        [9.46200000e+03, 1.72124062e+01, 1.55307075e+09],\n",
-       "        [9.48100000e+03, 1.75751667e+01, 1.55307075e+09],\n",
-       "        [9.50000000e+03, 1.74055386e+01, 1.55307075e+09]],\n",
-       "\n",
-       "       [[1.90000000e+01, 5.21177673e+02, 1.55307075e+09],\n",
-       "        [3.80000000e+01, 3.99685974e+02, 1.55307075e+09],\n",
-       "        [5.70000000e+01, 2.67611786e+02, 1.55307075e+09],\n",
-       "        ...,\n",
-       "        [9.46200000e+03, 1.75390892e+01, 1.55307085e+09],\n",
-       "        [9.48100000e+03, 1.76337471e+01, 1.55307085e+09],\n",
-       "        [9.50000000e+03, 1.91227703e+01, 1.55307085e+09]]])"
-      ]
-     },
-     "execution_count": 47,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "all_mae_histories = np.array(all_mae_histories)\n",
-    "all_mae_histories"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "As you can see, the `all_mae_histories` is a 3-d array, the last dimension are 3-element tuples. This 3-d array is built up with four 2-d arrays and all the first element of every 2-d array are equal. The first element of tuple stands for the training step and the third element stands for time stamp. You do need to worry about them, let's just calculate the average value through the first axis of this 3-d array. Actually we just want the second elements of this array, which stand for the MAE results. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 48,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "array([[1.90000000e+01, 4.66429123e+02, 1.55307058e+09],\n",
-       "       [3.80000000e+01, 3.31914894e+02, 1.55307058e+09],\n",
-       "       [5.70000000e+01, 2.01346550e+02, 1.55307058e+09],\n",
-       "       ...,\n",
-       "       [9.46200000e+03, 1.82184715e+01, 1.55307069e+09],\n",
-       "       [9.48100000e+03, 1.83279567e+01, 1.55307069e+09],\n",
-       "       [9.50000000e+03, 1.85283084e+01, 1.55307069e+09]])"
-      ]
-     },
-     "execution_count": 48,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "average_mae_history = np.mean(all_mae_histories, axis=0)\n",
-    "average_mae_history"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "As you can see, this operation does not mess up the first elements since they are all equal through the first axis. And we do not need to care about the third element because it is useless at this time.\n",
-    "\n",
-    "Let's plot this:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 49,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXmYHFW5/79vVXfP9MxklmSy7wsk7EtCWIPBQEBRQUWW64LKFS8uV/TqFfW64YaiICo/ERFUVFwAQUVEQCAsAUwgQIAkZN+3SSazT3dXnd8fVaf61KmleybT05Pp9/M880x3dS2nejnveXcSQoBhGIapXIxyD4BhGIYpLywIGIZhKhwWBAzDMBUOCwKGYZgKhwUBwzBMhcOCgGEYpsIpmSAgoslE9BgRvUZErxLRp9ztI4noYSJ6w/3fVKoxMAzDMIWhUuURENF4AOOFEC8Q0QgAywFcCOCDAPYJIa4jomsANAkhPl+SQTAMwzAFKZlGIITYIYR4wX3cDuB1ABMBXADgV+5uv4IjHBiGYZgyUTKNwHcRomkAlgA4GsBmIUSju50A7JfPtWOuBHAlANTW1s6dM2dOScb22o42NKaTmNCYLsn5GYZhysXy5cv3CiFGF9qv5IKAiOoAPAHgW0KIe4moVZ34iWi/ECLWTzBv3jyxbNmykoxv3jcfwTlHjsV33nVMSc7PMAxTLohouRBiXqH9Sho1RERJAPcA+K0Q4l538y7XfyD9CLtLOYZCJE2CZdvlHALDMExZKWXUEAH4BYDXhRA3KC/9BcDl7uPLAdxfqjEUg2kQchYX3mMYpnJJlPDcpwN4P4BXiGiFu+2LAK4D8EciugLAJgAXl3AMBUmaBrI2CwKGYSqXkgkCIcRTACji5UWlum5fSRhsGmIYprKp+Mxi0yBk2TTEMEwFU/GCIGkayFmsETAMU7lUvCBImIQc+wgYhqlgWBBw1BDDMBUOCwLDgMUaAcMwFQwLApOQ5aghhmEqGBYEbBpiGKbCYUFgGuwsZhimomFBYBCHjzIMU9GwIGCNgGGYCqfiBUHSIOTYWcwwTJnZ1tpdtgjGihcEXH2UYZhyc6A7i7OufxwPrtxRlutXvCBImAbXGmIYpqy092SRsWzsausty/UrXhBwYxqGYfrCqp1t6MlaA3pOaZUY6PMWS8ULAjYNMQxTLK1dGZz3wyfxubtfHtDzSj8lC4IykeSoIYZhiqSjNwcAWL5x34CeV5qnuzMsCMqCyVFDDMP0EacT78DhmYZyLAjKQtJtTPPAyzs4sYxhmFiE0J8LdGVyB33ejDv3dGfKMwdVvCBImM5b8PHfvYBbn1xf5tEwDHMoIBWCnz+5Hkd+5SHsbus5qPPJRShrBGXCNPIq3q4DB/dhMgwzvJEagRQEf3vZifvf0ce545+v7sS21m7vufRT9rCPoDwkzbwgGGi7H8MwwwvblQSE4uaKZ9e3BExHQgh8/Hcv4K7nNnvbsqwRlJeEUfFvAcP0mY7eHHrLNGmVE0sKAk0OhMUdbm/txqW3PhsINc1YNrKW8L1/OY4aKi8Jk7UAhukrR3/1Ibz7p8+UexiDjqwFVMys0d7jaAJrdrb7tvdkndW/WtEgn0fAzuKyoGoEbBlimOJZua2t3EMYdDxBoE0WYVOHQLj20OsmjakF5rKcWVxeEgbP/gzDFEdfNAKZnmRokkCu+tX8pXJnFifKctUhBJuGGIYplrxG4N8u1/ZCCPz+31tw2Jg6VCdNd1//zt3uZK+ahrzMYhYE5UHmETAMwxQip5mG1Cn+6399FUeMq8cX7n0FAHD/x08P7APkV/1qAmu+6Nww8xEQ0e1EtJuIVirbjiOipUT0ChH9lYjqS3X9YlFNQ8WGhDEMc+jSlcnh50vW96sJjK2lFgtl+x1Pb8T/3pOPEJKrez0w0RMEdtBZ3J21IPT05UGglMvhXwI4T9t2G4BrhBDHAPgzgM+V8PpFwT4ChhnarNjSis7egy/jIPnRo2vxrb+/jvtXbOvzsXLlrs8adohQ6XCjhgI+gpztOxfgNxP15gZfKyiZIBBCLAGgl+g7HMAS9/HDAN5dqusXS9LkqCGGGaq09WRx4c1P45N3vThg55Qr8n2dmT4fa2t5BHLKyITUKet0E8l0H0FeI8gfk1WOL4fDeLAN5K8CuMB9/B4Akwf5+gFM1ggYZsgiSy68vPXAgJ2zJuU4cfuTvJWPGvLPG5mQVbwsWS33PNCVxV9e2u5N9L48Ap8gGEYaQQQfBvAxIloOYASASJFMRFcS0TIiWrZnz56SDUiNGmKRwDClRwiB6x5chde2F85DkFPlQGrraTeapz8ROlFRQ2HmnLxpyHn+ibtewH/f9SLe2NXhOxfgFwrliBwaVEEghFglhFgshJgL4C4A62L2vVUIMU8IMW/06NElG1OSo4YYZlDpzlq45Yl1eM8thTOTpSmmv4r7HU9vwL0vbPVtS7saQdfBaASaJAgTBNKvIX0EW/c7ReakpqCag1QzUTlMQ4MaPkpEY4QQu4nIAPB/AG4ZzOuHoZqG2EfAMKVHBsVki2gRm3fO9u/H+fW/vgYAeNeJk7xtchLvz4Sbi0go6w05V0ev5V7PeS6FmhQavqih4aoRENFdAJYCmE1EW4noCgCXEdEaAKsAbAdwR6muXyxJX4kJlgQMU2pk4bZiOgNGmWIOBrkS78+EqzuLJaGmod6suy/5js14UUOqszgvCIaVRiCEuCzipZtKdc3+oGoE5YjfZZhKw3InvWLC+KWwKEYOSN/DxSdNxszRdZG/52xOdgMLn3DX7GrHhMY06qqC02OuDz6CTlcjkFOMlHuyLHVYHgFQGVFDQw61HwF3qmSY0mP1YcElV8rFaOvbD/TgZ0vW40N3/BsA0NYTnnsgNYLOkBaTQggsvnEJLr/9+dBj7YioobCS3PmoIcKn/7DCa0Qjt0flEVRC1NCQQy0xYXETe4YpOX3J6PV8BEWoBKa7k5yUW7vCgxIz7jnbQwSFHNvyTfvDxxOlEYRM3nLCNwzgzy9uC2zPqkXnlFVoOXoSsCBQTEO5fqScM0ylcbAm1D4JAmkaKiAIdrf3eAJACo+whLE1u9pxx9MbAITH/oclhqnYUc7imKghXXvoDNEIcrZAVcKZjsvRpYyLzimmIb2OCMMwQQ52vdQ3QVBc1ND8bz2Kppqk75jWrmxgv7f96Clvss+GTPphwiFsPLpkijMN6deRvgN/HoGNEdVJ9Hb0skZQDtTGNLkiwtkYptI52AVTXwSBnESLySPY70788vz7XdOQXGkD/hV/mAWgkCDwWlVq28OjhnK+8ejbs76oIRsjqhOR5yo1LAgM1VnMgoBhCnGwgqAvJthcEc5iPcrGcwa7E67sCxB1bhVVUNz82FrM/cbDvtctK9xUFeYj6PR8ASJ0u55HkE6aMKg8PgI2DZnsI2CYvtBfOZC1bCQM6pMgKaYj2IFuvwlIHiMjcaK0iUKmoesfWg3A8QsY7kmk7AhqBMHJW15fD0LJeePLb89YNpIJA+mkyeGj5SDpixpiQcAwheiPRtDWk8VhX3oQ/+/xdX0ywXqTZYwk0AWBnGgL/Z5DTUMhwqFdKYHtOYuLKDHhXSfiftXx9eZsVCcMVCdNdGctPLN2L9p6gj6OUlHxgsD0RQ1x+CjDFKI/66V9HY69/vf/3twnQSIna72mv4ouCCRZL+Io/NhincVtyvkDJSa8kNXouSPsOoBfQPTmbFQnTVQnTfz2uc34j9uew6+f2Rh5zoGm4gWB30dQxoEwzCHCwfgIhOibCVZOorGmoZDoICC6iYz+ukqoIFBW5pElJmLMOVE2fzWPoDdroSphoDqZn5IH02lc8T4CdbXACWUMUxhR5M9kza52WLbAEePrvQldiL6ZYIupNdQaoRHIJC1bCOw80INbnvAXOw6zAISZhtq686YhtQheW08WO9xs4bhJOyrDWb4XpkHozdmoSppeZVTAiS460JVFfTpR8jpoFa8RqLCzmGEKE6YRPL56d2Ci/cbfXsPX/vIqgPxEaQvRr8zi/piGVF/Bz59cj19qppZiw0dVjUAtj3HujUuwu70XQLwg6IhpsymFUW/WcnwEibwgWL5pP4679p949PXdkccPFCwIFNhZzDCFCRMEH7zj37juwVW+be09OW+ClJEwfRUE2SK09GIEwfiG6sDrckW+q60Hb+xqB1DYRyCtBrYQ2HGgx9uuRw2pJp4wZG7DfW7piZ6cjaqk4dMIZP+CkXWp2HMNBCwIFFgjYBhnUnt9R3T3sGJ/Jj1ZK1CD3+63aShaI2iLchZ7xeUsPL46vMth1rLxpusfwzk3Oq3UQ01DimlHvqwLQz2PYGJjOnK8ADCi2smC/vw9r7jHW6hOmKhSNAJZIqO5tir2XAMBCwIFmwUBw+Ar972Kt9z0JHa19YS+XmytIb8gcDUCW/jMK4XOpcbi5ywbLR29uHXJOt9xUcXlVGfwU2v3hu9jC6/ap2WL0AifMI1A9zPrpqGRtc4qPpUIn2LTKf/23hCNwDvXIGgEFe8sVmGNgGGA5Zudyptt3VmMrQ+aVIrXCGyk3TlMrpgd05C/zINaCl5HOnzX7OrA2378FKaOqsFDr+7C02tbsGZXO5Z+YVFB01AcatXPPe29hX0E7su6ANNNQw3ujY+sSWFniEBV6yDlLNstOmeiWhMcVQkDtSHCYaBhjUCBfQQMU5hiw0e7s5Y3YcoVs4A/TLvoIm8AVu1s91bvT6zZ49noD3RnQ1feuSLiwdU+ANsPdPvGM7I25Sv5sKutB7e7lUv1uULXCCY1OaahKI1ALYEtj60O0Qia66oGpXMiCwIF1ggYJk/Ur6FYQdCTtbwJ02caUjSCgoJAs8HUaBNl1rJxoDuLUbVB80kxv+ftbvinfCwn5dNnjcJjn12IaaNq8djq3ViyZg9+vmS9t69+avU+TIPwP4sPx6cWHYa3Hzc+9LqfPvtwAEBzXcpzpFclzEBdpJEh91UKWBAocB4Bw0QnYP308XX498Z9RdUasm2B3pztCQ25kncidfL77enojT2PHutfk/Jbs7uzFg50Z9FUE5wwozJ6337cBHzv3ccCAC64+Wlv+84DPZ6GcNsHTkJDOol0ysSutl584PbnYSomrDjfhmkQRlQn8elzDkfandhlZVHJMZPqcdHcSUiZhk8jMLXCSKMGwT8AsCAAAHz8rJkA2DTEMCr6XPfdf6zCe25ZGtAI1CJpMuBCNleRPylPIxDCN7lv3NsZO4ZsAY2gO+MIgrCVc9TvOWfZoSab3pztrezl67WK4EkaxdUlMxVTzrTmWud8pv961QkTqYSBjGV7gqAqYQbe8/EN8dFHAwU7iwF87tw52La/Gyu2tJZ7KAxTduQ8JiKMQ/ocqCZMWULAAPkicQDVWew3LW1q6Yodi27n1wVBS0cGWUsEBIEQIiBEJBMa04GVN+BoEFnLhmmQ97pqs1cdwnHmMbVszfnHjMfO83swoTGNj/32BW97VdL0tIG8aSgonGaOro28zkDCgsDFNAz2ETAM8t3Aola9+iTY4YuzF0iajskGADbv68ItT6zzVr1Zy/bZ/Tfti9cI9N9kQosw2tnmJl1pguD1He2hJSTu+NBJOHXGKDy+Opit+8NH3gAAz5wDALVV+cf7u9SaQ9FjVk1IRIT/XDADti1w+qxReHptCwDHDJRKGMgqGkF10gwI3+nNgyMI2DTkYhrcoYxhVKIEgW4fVzUCOXGr5qLrHlzlhWDmbIEed+Jrqklie2t4rkL+fFotf+03us09fvQIf9LVTY+uCf09nzV7DKqTpq8zoY4azqr6JNR8hTiNwAyJ8jEMwp0fPtl7nnY1gkzO9grWhWkEE5vYNDSoNNWksK8r42tCwTCViJzH1NW4mmypy4duZdK33MlXr7i5bX8+OkdOfLVVidCGLpID3Vn85tnNvm26uUeeV40aInKycsPMPxJds1BJKdm9ar8Sv0YQ7ywOwzAIBjnvX3XS8RHYAuhy36sqRRO5bP5kjBlRjcPHjIi8zkDCgsBl6qhaZHI2drT1FEwPZ5hKQJ38fUJBmwSzSuikXMHrXba2qoLA3b8mZcaGj/72uU2BbS9u2e97LsM/m+vyGsH05lq0dGYwMiSSSJI0ozUCtU6Q+h7s78pg4ezRSBhGbAmORJwAMhwHcXXS9MYgtaWqhIGGtFN64qgJDXjfKVMjzzPQsGnIZdqoGgDApgJRDAxTKfh66iomGt30rvbktTzTkH+nrfvzTmEpCNKpREAQCCG8yX1HiNnoxc2tMAi4+79OBaAIAsU0NLquynEixxjy4yZr1USjlsPY15nBxMY0RtYmYzWCOIuCtEhJHwEA7G5zQmgb0klcccZ0fPltR+LSkyZHnqMUsCBwmeo6ZTa0sCBgGMDvI4jSCIQQmkbgmoY0jaBTMRXJyT+dNAIZufe8sA2nXfcvLN+0Hy9vDY/iG1df7dnOpSBQTUNj6qtxoDuLHs08pcbyJ2I1gryJxvbVNHLyFQyi2PDRQhoBkA8fBYBt7j2MHlGFqoSJK86YHju+UsCCwGV8fTUSBvlUWIapRGRJA3Xytyx18s/vawt/4pYV4izWkX6BmhCN4CU3hPvlra3YcaAHl8ybjAWHNfv2SSYM1CSdSX37gR6MqE6gSjHnjHG1gz0dvb6GNn/75Bn5cyg+gi+99Qifs1nVCPRClI01SRgGhUYNzR7r2PPjNALTIKQSBgyDkHLHsL21GyOqEoGs4sGkZIKAiG4not1EtFLZdjwRPUtEK4hoGRHNL9X1+4phEJpqU9jfGV7JkGEqBTmNqZn2al8AdZVs2cJngonSCFQ8jSBlYuv+bm/yBxwHMgB09ubQnbWQTpmB6B/TIFQr1TubalK+SB05qe/rzHiT+pmHj8bUUflQTDVq6CNnzvBN/mopaL3Oj6MRhDuLj53U4J47TiMgr7Cc1Ai2H+gORD0NNqXUCH4J4Dxt2/cAfF0IcTyAr7jPhwyjalNoYUHAMAD8oZpWhGnI1kxDltJxKwopCGqSJjKW7SvzIM037b05dGcs1KRMn50ecDSSlJkvx9BUm/JN7DOU2HvpkNWDhPSKp+rkrTqLv/DWOb7XRtY6QidMEMhy0WZMaKppkLfyT5nO/+2tPT5ndzkomSAQQiwBsE/fDKDefdwAYHuprt8fRtamvGYQDFOpyEWwz0dgqZN/ft8Tv/GwzzRUlEZg5aOGomjtzCJnC6STZsAebwsBIvKOH1mThDr3HjmhHouPHOvci7tND+nUbfDqc1UjGDOiGjdecrz3vLEmCSLylZEGgFNnjPK0kjjzfkIRBFIY7evMoHnE4NQUimKwfQRXA7ieiLYA+D6AL0TtSERXuuajZXv2hHcXGmiaWBAwjIe6Elf9BWpCWVfG8gsCL48gOiy0N2uDyB83L8/ZlXGS03a4NfzTqaAgkM+lg7ipJuWb6A0izB7n2OvlkQFBoD1XTUt6m0m1LpF0FqssOKwZd115ineNWI3AJC9zWT3vuPryhqxHjpiI/ld5/B7ttW/383pXAfi0EGIygE8D+EXUjkKIW4UQ84QQ80aPHt3Py/WNUbUptBSohsgwlYLlCwtVfQT+/bIhJqSekEQxOcFmLBsmkc8u/7W/vIqcZXsCZM1Op4dwTSoR1AikIHDNKU21fkFgGoRGN4dAZj3rgkDO5c2uOUed2/UcgypNEKgvv/vESfjp++YCgCcgYnLVkDAM731QBcFbjxkXfdAgEKcRXKo81lfuuu2/WC4HcK/7+E8AhoyzGHBMQ209ucjytQxTCXiZxVa4RhBIKAszDWWCgkBW8szknMJuakXOXy3dhKXrWzyT0k5PIzBCTEPO/zrXsdxUk/St6ImcbUA+wklfxcuxXDZ/im+/sH1VU9GI6oTv9aMn1nvjkNvjyleYBnmakHr/c6c2RR4zGMQJAop4HPa8WLYDeJP7+M0A3ujneUqCVDX3d2WwfNN+7G6Pr4PCMIc67T1ZfP+h1b4wzrCic34fQbQg0BvRqMiIIE8QaLV1crZAdybnhX8CQDoZ1AikyUp1Fvs0AqJAfwJdI2iqTWHFV87xGsTEoYamGoY/h0D1LciHVcnoaVX1Eaj3PxhdyOKIKzEhIh6HPQ9ARHcBWAigmYi2AvgqgI8AuImIEgB6AFzZp9GWmDo3YqGjJ4d3//QZTB6ZxpP/++Yyj4phSsfNj63DLU+sw/jGarz3ZKekQVitIb+PwH+OjE9zcIRCmEYgnbu9OctZGWuCYH9nBl0ZC001Kexud0y06ZQZKDwnTUNybm9IJ30TqWMaSvqOOW3mqMB4GhVhoVb91OdkfZxdiiM8qfom3Mdx+QAnTm3C2BHV7vid/ae6VQ3KSZwgOI6I2uCs/tPuY7jPgx2tNYQQl0W8NLdvQxw8pBNnj/sl3LKPk8sYP5YtcMPDq/Hh06d7NupDGTnp7esIBklIv0BLR6/3mwCCGkEuNKEsaF6VGkGvpxH4J8zP/PElAMBxkxu9bTUpM+CTkBoBeTZ53f6f9xFMbEzj3o+d5tMywlBvKUoQyCbyasayrokA8YLg2+88xnssQ2UvPWlK7NgGg0hBIIQoX5pbmUi7dsMtnF3MRPDU2r24+bF1WLe7E7e8f8iuaQA4pQvGjKiKLbAmJyO1lLSc2nK2QCZnY+43H/EdE3QWFxc+KjWCTM5GIkQj8PZTJtJ0MqgRSGFz3KQGPPzaLoxt8K9LTYO84m0dvTmMrS+4btXwSwIpJMa451HvLaPcuxQK1RH3pTN1VC2euebNGN/Q1/ENPH0KHyWiWiJ6HxE9UKoBlROpEWzeF981ialcvISpmPLJQ4HujIVFP3gcf35xW+x+I9xVepvSXEYuiS1b4NdLNwaO0TUC1b/glaGOEQS9ORsGBX0EErUrWDpl4toLjvZf3xUEVy2chfs+fjpOnOJ3tJrkCILxDdW49oKjQq8Rh64RTB5Zg2MnNeD6i5w+x+q9qSYwqaGkY/IjdCY0psvuHwCKKENNRCkA5wP4DwDnArgHwC0lHldZkIJgqysI0mWs/cEMTYpp3D4U6M5a6MnaPpNOGPJ22nuygddytsCtS9YHj9HehC5lMpQaQVhmsRc1ZNlIGhSIzpH4BEHSxFmzx2Djdedj3Z4OLPrBE55GYhqE4xUzkoTIeW3pFxaFnr+vVCdN/OUT+TpF6v2qgkBqRofivBEpCIhoMYDLACwG8BiAXwM4SQjxoUEa26Ajv4Bb3JK53VkLD726E+ceVd4YX4bpK9JuH1fqAciv5ttVjcCloycXmmCpl6FWnaff/vvreHHz/lCNQIZNZnI2qqoToa0kAf9EqmYfS1OSXnJCJ64hTRTqGQsdrRbUU+/T6z18CAqCONPQPwDMAHCGEOJ9Qoi/AhjWAfZSEKimoY/euRyAswq6+bG12MJmI+YQwGsZGdP4BcjbuFWNQE6Ee93kSn1i1U1D6qp4874u/GzJ+lBBIF0VB7qzqKtKRNb0b6pJes2hVMerjOfXK4IGrnOQppZCh6v3e/G8fN8A6SDXM5MPBeJMQyfCSSp7hIjWA/g9gENP1PUBuRLZ1RZUp3e19eL6h1bjzy9uwyOfcVIhWjp6kTAMNGihaszwZyjYdeOwYkw0KlIjUM0dcprd60YSjW+o9pVn1+fhrkwOSZN8GcZhEXfqBF2fTob2FL7zivk4dlIjLj9tGp5Z1+IXBMniNIKD/Wg+smBG7OvyvVryubMwRQn9lBpBdeLQmyYjRZcQYoUQ4hohxEw4OQDHA0gS0YNENKTi/weKONueXBWpJSjmfvMRHHftP0s+LobpK9JeHRbGGbZfWH/ilk7nuz6hwV8HR/cRdGes0MkvpUUrqcKzIZ3EjNG1+iFYcNhoNKSTmNRU41ttA3nTUCE/TX+EtLynX314vq9cdRg/uuwEnH3EWExo9Ef7eIJgmJmGPIQQzwghPglgEoAbAZxS0lGViSiVbseBblzys6UACv+wGGYoEFfzZ+m6Fky75gHs7ej1NIKwXIAWqRFoE56uEXRmLFSHRMqMbfDH7qsmpvrqJOZOHYl7rjqt2FvyBMtbjh54n528pWQR/oW5U5tw2+XzAhVMpSBIp4aRaYiITox4aS+An5RmOOWFyKkMqNs3b3l8Hda7vYzjyusywx+5GtVXxUMNaabpDVm43LpkHQBgxeZWTxBkLYHenIXbntzgTWhS+x2vaQRhPoIwbXpGc53PRKTmM8g4/zlulVDAWWnHQUR4/kuLvGNLwcG0iOw+hE1DcT6CZQBWwpn4Ab8zXcCpFTTsSKeCgmCo24MZRidOI5AQ5ctDZCwbyzbux/UPrfZel32GdROILgi6MjmMGRFMipreXIsn1uzBxMY0zpozGhfPm4RbnnCEkJzM1aSydxw3oeB9hV1nIOlPxJHkc+fOwf6uLBYcPjjVkgeSOPH3GQBtALoB3AHg7UKIs9y/YSkEgLyf4PxjxgMI1hlhKhu5Jhjqi4Oc1yksqBHIaZwIPtNQWJ/hdNL01eQBgjb6rowValaVPoCGdBLfvPAYr8SEs815LFfgR02oDxxfDvTOZX1h1pg6/PGjp3rVSA8l4pzFPxRCnAHgkwAmA3iUiP5IRMdHHTMckCGkE5vS+PhZM5GzxUFHITDMYJOL0QjkRE4gL3zUMQ0FhUZjTTJg9tE1gt6c7XOQvu+UKZjUlEZ9td+E4/MRKOadB/77DPzuI2V2O7q3FFdCejhTUHQJIdYT0f0A0gDeD+BwACtKPbByITsX1VcnYNmOil3IHCzc1nkMM1SQoZlhwQ3e15ng9RvOWHZo2YzGmlSgpWRYGL+aDfyNC44GEeH+FU55C/nTULuCqXb+oyY0FLyfUuM5iw9CIziUietQNoOIvkhEzwH4OoCXABwhhPjjoI2uDIxzC0BVJ02vFkqmQKMajiSqHIayj7grk8NZ338cz61vie0LIB3dhPx3O2fZoWakxnQyUDsnLKFLdZB6VUHdiV8KAlUjGKrmE+MgfASHMnF60FoAF8PJMF4KYAqAq4joM0T0mcEYXDk4x2163ZuzPf9AVlOZN7gRRJLOTDA9nxne/GvVbizbuK/cw/DR0pHBhr2deHV7G7IhPgLLFrj9qQ1eOQmBfB7njMARAAAgAElEQVSBLfxJZZKm2qBpSJ77J/+Rj/KpShqYNaYO31CKvMkEMtnoRjW71A5RQVCpxH0a1yKvMdUNwliGBJfMm4xszsa7507C/Su2A0CgdeWFNz+Nl7662Hve1WtV0DvESC66ZSk2Xnd+uYfhIVf3Hb05rwqoqhEsXdeCa//2mvc8p/kF1FLUkoZ0KlDLf2+7k1+gJoylTMPLuJfo5lLV/N6XCp2DgaolVSJx/Qi+NojjGDIkTAMfPH06gHwruayWCn+g21+pMewHxAxPhrBlyPMLdPTmvKgh1Wy5YW+Hb/8bH17j++52hnyPG2uSgQY8Nz6yBoC/uFoyJLpOD8VUNQLd78CUl8p0kRdJVYSPYKaWGt/FpqGKQe+fO5TIegXkcvmoISUkdPWudt/+r+1o8xVYbA8RBE0xdbRUjSAsI9fwQm39zwGgJjm0TEPvcctZDIeuc/2BBUEM8ouuNt6YNqomoAF0hthWmeHJUM4o9pmGXEGQs4VXPuKNXR2RxwJO2Wkd2bVPNmVRURvLhHVB0x2vqqloqJmGPrZwJt741ltKmrU8lGFBEEPeNJQXBM11VdjflfVNCF1sGqoYClW+LCeeaagn6zNn7uvKYHd7T8CkqaMucKQ2LB3F75k3OdDtS022DDMNGZ6zOEhUd7JyQUSxLT2HO8V0KKsC8G4A09T9hRDXlm5YQwNZ/1wXBJmc7W9IMcTbFjIDx1CzDH30zmWoSpj40WUn+ExDltL05a03PYm9HRnMaA6vqpkyDWQs26cRLD5qHC6eNwmnzWz2tuk2f58gCDENmaTZhpghSzEi8H4AFwDIAehU/oY9+TyC/K9/VJ2Tbr+/K7+64jyCyqFQUxSdu5dvxeW3P1+i0QAPvboLf3nJiW5TTUOqRiD7CoSVkACAOreBveojaOvOYsFho32Tv37vVUruQLhpyPnPYmDoU4zHZpIQ4rySj2QIkgrJI5DOpH0d+RZ+UT8wZvgR1VUris/+6aWDvuZDr+7EqTNHBUo26EjTkKMRBMfZ0plBTcoM5AscPrYOz67f54saagvpYaz7wlJFmoYGi4+eOcOrEsz0jWI0gmeI6JiSj2QIIp3FftOQoxHIph0AawSVxGBHDa3f04GP3rkc19zzcsF9s4pGkAsZZ2/ODnWGHjep0TtOFo9rC/En6OGlaqJZmEZwMJU8+8MX3noEfv6BeYN6zeFCMYLgDADLiWg1Eb1MRK8QUeFv5TAgrMSETKVXJ3/WCCqHwfYVt7jN48Pap+r4BEFEWZQwQXD0RKfWT0dPvpx0WDP7zt4YjSCkRo8ePsoMXYoxDb2l5KMYooSVmAgTDiwIKoe+moYOFmnGCUvAUtumAvnER8sWkUmOjVpewCXzJnuZwxnLxqSmNM44rBmXnjQ5cKyuEaiZwnERNywHhj4FNQIhxCYAjQDe7v41utuGPVIQqGn4XpJZjgVBJdLf8NG+Opkl3W6yYlgHsLnffMR7fPNja7F0XYv3fH9XJrA/ENQIrj7nMF9XrnTSxLffeQyOdc1FKm85xt8iUvUBhAkCr9w1qwRDnoKCgIg+BeC3AMa4f78hok8WcdztRLSbiFYq2/5ARCvcv41ENKTLWcvVvzrRy21/e3m7t419BMObnqyFL/75FezrzPQ7fLRYAdLS0YsbHl7jCY4ojUAXLNc/tBr3vLDVe97aFZ4z0Jj2N5mpSpi+DOGqiL7dALBw9hhfbSV1ei+2fPOnFh2GGy85rqh9mcGjGNPQFQBOFkJ0AgARfRdONdIfFzjul3B6G/9abhBCXCIfE9EPABzo43gHFTnpqzkDcuXz+Oo93rZy5BHYtsC3/v463nvyFMwYzRXvSsm9L2zD757bDJMIs8b07722bIGQRX2Ar9z/Kh54ZQfmTW3CmYeP9iJ1arRqnVk7fvERKQg001AqYSCZyE/ixfTb/eWHTsJdz2/2OYNDNYKQYz99zuEFz88MPsUIAgKgznQWijD7CSGWENG00BM6uuLFGOJ9j+VKSV18hWVE9mQtvPe2Z2EQ4c4rTh6Usb2xuwO/eGoDlq5rwd8/tWBQrlmpyAqepkH99hEUG20ktU9pjpSmoRpNihQ6X5RpqF4zDaVMw1cMTm9LGcbC2WOwcPYY3zb2ERzaFCMI7gDwHBH92X1+IYBfHOR1FwDYJYR4I2oHIroSwJUAMGXKlIO8XP9ImAYMKkYQ2Hh6bUtgeylRJyemtEh/UNKkfoePhoVzhiE/T3kdGalTrfcEsOLPV6xGkDTJZxqKKzIXR5hpaAhX42A0imlVeQMRPQ4njBQAPiSEePEgr3sZgLsKXPdWALcCwLx588r2lapKmD7TUCpk5VMOZ7FcMVYNsZotwxEZlplKGP2e3Ip1FuuCIN9EJn/8y1tbfcEKYbR2ZZA0KSAwdB8BEflMQ021hTUC//HOhB/uLBbePszQJlIQEFG9EKKNiEYC2Oj+yddGCiH61Z6JiBIA3gVgbn+OH2xSCcMnCMIm3rIIAtdBHefcYwYGWWIkaRpFOX33dWawvyuDmYrvplhnsScIhOwt4Kzs1bSAd/zk6YLn6cxYaKxJBjSD+nTwJ69O4iP7KAhMIuSEqOiCbcOBOI3gdwDeBmA5/H4fcp/P6Oc1zwawSgixteCeQwDdFBRmGgqr4y7Zsq8LrV1Z7O3oxYtbWnHlmTMGpF+rNA1VFeHcYw6OvGnIKMpHcN4Pl2B3e68vwqZYk5IUBDklOcw53m0pWeA8soAcEL5o0U1MgNOX2HvcR9OQ4dpOQ01D7n9iL8GQJ65D2dvc/9P7c2IiugvAQgDNRLQVwFeFEL8AcCkKmIWGEropKCAYTAO7DvREHn/xz5Zix4EeJAxCzhY4fnID3jxn7EGPS5qGwkxVlcDKbQewq60Hi444+PeyEFllYi1G+9vdHswC7qsgkCHJMnxUagQy0ziKdMoEZZ3vh+oEllQnTDz7hUU45TuPetsSB6kRAOGmobH1Tpby/Okj+3ROZvAppgz1o0KIRYW26QghLovY/sE+jbDM6Ksq/QtfW2X6KpHqtLjF6aSzsJCTr1jkKrVSTUNv+/FTADAoPYOlIEiaRqBgm22LQAMWiSo0ihUECfdcsuud/JyXrm/Buj0doaUfVJKmgaRpoLej19c5b0JDNbYf6EEqYWBcQ3Xk8SOLiBpSkYIrTBBMb67FY59diCkja/p0TmbwiZxFiKja9Q80E1ETEY10/6YBmDhYAyw3ugZgap6v2gJmnrlTm3zP+5thqpM3DVWmIBhMVNOQPqHHxfOrk3bfNQLLd+3Xd7Rh0Q+ewPUPrYo9PmkSPrVoFgB/qXQ9D0Hno2c6lt5iwkdVpAyMSiib3lzLkW2HAHHfjo8CuBrABDh+AvlptsFJFKsI9IlWX/0Vsvfr2nmxYYSFkKYD9hGUHmlzFxCBVpU5SyDqK6CWcn5w5U5ctXBmwWtJuSI1D71fdqEwZVsIvP/UaUglDLR2ZfGdBx3BcfrMUVi7uwO1VeHfl8+fNwdXn314nzuHGTEaAXPoEOcjuAnATUT0SSFEoSziYYv+w0hogiCsGJiKHuY3UEXL5IqRNQI/3/3HKvz08XUDajKS5jzbFoHon2xElU/AX8r5u/9YhWMnNeD0Wc2R+wP5ib9b0wiKRfYkuOQkJ/dm9rgRGFVbhTnjR+DS+VMwqSncTGMY1K8+wnE+AubQoZg8gh8T0dEAjgRQrWz/dfRRw4eAaciINg2FacD6D3mg6tnLFWNYQ5BK5qePrxvwc2ZcM5wtgq0q43w+eo/gPSFO5MC1XEGgm4aKRdcg1AzgI8bXe4/vuepUxMiwopEaAecKHNoU4yz+KpzonyMB/B1OWeqnoNQQGs7oUTl616XaVP4tDFOre7Uf8sCZhrji6WChlnfWfTxhGoEM4WzTHLtx2oNETvxRpqFCFHImS+ZOHZhInvnTRuKBV3ZUbPTacKGYT+8iAIsA7BRCfAjAcQAaSjqqIUQhjaCmKr5LU8A0dBCCQG02Ik0Hg10fvxKRE7gtROD9zoVoBFURXb6KiRhTncN/fnGr7/tTaNWdMCi0j0Ap+f57jsNfP3FGnzOSmaFFMYKgWwhhA8gRUT2A3QAG99tWRlKaM1Y3/xRyFusaQX/r2d/7wlbM+tKD2NzSBQDodleMAxWFdKiiO28Lbe8P8jN0BIH/tbAVu0za0k1DfdEI1uzqwKf/8JLv/IV6Fv/5Y6fjuncfW/AaA0k6ZeKYSRWzLhy2FCMIlhFRI4Cfw4keegFOGeqKQHfG6k02VB9B2KSsTxT99RH89SWn/8Ebu9sBAF3ZvN1aRwgR2nx8OBL1dg5kb2E5gVt28Lxhk7v8hqzf01lwXwD42RPrsHZ3R+g+qlYR1mZSRa0ZxDB9oZgOZR8TQrQKIW4BcA6Ay10TUUVQKJxO1QjC7P8D5Sz20vXd33qXV3ogeL5fPbMRx37tn9iyr6tf1zqUiJpcB8oXo17DFsHw0U0twfdYCv/nNrSEblc50O2EeL7z/z2Nj965DBtburDgsGacOmOUe838vmF1glQ4cofpL3EJZSfqfwBGAki4jyuCQk4wNXw0bFIeMEGgtf3zqlIKge88+Dou+ukzsG2BVTvb8M/XdgEANrZ0hp5rOBE14Yf5Tixb4DfPbsKB7myfPgf5GYaFj67cFuytJPffur/bP1ZL4Orfv4g/Ltvibdvr9h1u78nhoVd3YW9HL2pSJt49d1LgvIVMQ+ywZfpL3BLjB+7/agDzALwER+s9FsAyAKeWdmhDg0Jx+qppSE4Sti3w3X+swn+cPGXATENyYpPKv7Q/2wL42RPrAQAzvvh33zEHaya/8eE12LK/CzdcfPzBnaiE5CI0grD3+e7lW/B/963E/923EpfNn4LvvOuYyPPe8PAazBxdiwuOn+j5CKwQH8ErIYKgN2dj8sg0tuzzC4KerIX7VmzHfSu24+J5jpstLKQ0aRqoC0n8KiQIWCNg+kvkN0cIcZYQ4iwAOwCcKISYJ4SYC+AEANsGa4DlRhUEx0wMOsXU8FEhHCGwrbUbP1uyHo++vjswIfXXWSwPk+Gr0gcQd76DjSi66dE3cO8LQ/ujjtIIwgRBW3c+tPKu5zfj+Gv/Gbqi39eZwY8efQOf+r3TUlt1zKt+oAkN1d6KXr2uZQs0hZRqaO0O+m3CCtSlEgZqUsE1WmHTEPsImP5RzBJithDiFflECLESwBGlG9LQQtZoOWFKI/76yTMCr+s+hJwtvBrwatEvSVyUz4HuLHa3+yuZrtnVjo17O32NSYD8pBYXHVMJ8URh4ZtAcZpXa1cWtz25XtuWwYnfeNi3zRMEwi9cq5NmwEchzUJhjt1tmqkICNcIqhJGaCmIws5i1giY/lHMN+dlIrqNiBa6fz8H8HKpBzZUqHV9AN2Z8AQufRVmC+H1i+3oDR4T58Q847p/Yf63HvVtW3zjEiz8/uNeDRrLFsjkbC+PIHbCc1/qyuS8uvbDDTkR//2VHZh2zQPe9jBNSRemAGBqxaD0kE8hhBehZQnhy8atSpqB3ABZDDCsnPP21rwg+PJ9K7F6Z3uoIEiZRmgxwxGuaUgNXHv+S/kiwOwjYPpLMd+cDwF4FcCn3L/X3G0VgfxB6vkAkjCNQAqCW54IljuI0wjiGtzISSxj2WhXQkPj5IBcvZ78rUdx9Fcfit7xEEYK1nuW+/scxRQF9aHPnXrCYMayPWFr2/6ooeqkEQgGkM/DTEPbFEFw57Ob8OQbewKmJcD5TtWGmYaqnW2L5uTLRqg9B9hHwPSXYmoN9QC40f2rOKQgiNIIpo2q9T23LIH9Mc1DDtZHkLOEr3RBnB9ACok4AXOos3pnG5ImoV4zm+SKlAS6RqAfpn7utvBHDVUn8qahtp4s6quT3oIhzIyj9zLozdmh36vqpBmqEdSnk/jH1QswdWQtjvnaQ8jZAglFI+Vyz0x/iQsf/aP7/xUieln/G7whlhe5MusOqe3z+GcXYly9v8mHJUSgUc133nUM7v3YaUgljKIKfYXZ/T1BYNs+80WchjGQ2bVDlf/6zQs447uPBSbe/moEen8BdfK2bL8GVp00kLVsPPLaLhz7tX9i+ab9niAoptNXJmeH1owaUZ0IrWqbMA3MGVePdMr0KoUmQ7qQMUxfidMIPuX+f9tgDGSoIp12YT9Y06BAf4KcbXumIUl9dRInTmmCSeT1ntXpVFbtXRkrsCKUK/+sJXzZpkW4CHw8u74Fu9p6cMHxw6u3kK4RFN0sXssU153PXZpGoAreatdH8MArOwAA6/Z04OgJTmRZMb1/s5YdusCor06Ghi2rJdBrUwm09+RgGoQ/XHkKnl67t+D1GCaKuH4EO9z/mwZvOEOPuhgfgVTFX/zyOfjby9vx5ftfhWUHNQJp3zcNitQIVKfh7vZe/OLB1/G5xXOUczjkLNsXjRQ34YVpBJfe+iwADDtBUK217Cw2X0N33utRQLppKCxqaFebE+m1akc7/vduR1kuFOEDxGkEyUApE8AvCKTGkDQJJ88YhZPdTGSG6Q+RgoCI2hG+qCQAQghRH/LasCOuxZ/8YTbVplDlFhqzbIFWTSOQgsE0CLYQ2HGgG6d+51/45YdO8urFy8kEAH7z7Cb85tnNSJl584Cc1LO28LqTmQbFmn8GsszCUMfSVvJhgoAQnFzlinzt7nYsXb8Px2q5Ij6h6+YISKRpSH52D726EwCw4LBmnFLExJyxbO+zVBlRHf6dU53B6ZQJ06BQgcEwfSUuoWyEEKI+5G9EpQgBAKgLid6QqGYhKRQsW2Cf5iye4DYLNw1CzraxfNN+AMCfluUjXf6glB2Q2bJqRImcf7I52wtRrEmasbbwgSy8NtTJ6ol7IfceFj4qV/wX3vwMvnzfykAmeFdW1wjyr1UlTNgCWOcWl9vW2o3JI9O484qTvQqkcfzh31vw2o62wHYpCE6Z4e8ZYGoaATuHmYGiaE8TEY0hoinyr5SDGkrURPR4BfyquvxRqgllAHDh8ROw6IixAJysYMtWm6Hnj3/yjb2Y2JgGkI/yUX0NnmnIzq8i0ykz1jRUTP37ctKbs3D7UxsiBdY/X92Jadc8gJ0HekJfV9F9L8VmVUuNQOZZ6D0EfKYhG1r4aPC7IT/DYogKSZb5AjddegLeM3eSZ2ZSI4TSqQSSLAiYAaKgICCidxDRGwA2AHgCwEYAD5Z4XEOGuNhsI0QQ2JpGcLRiajAN53U5AaiN57t6c5g6yuknK00NezsUQaA4i6VduSZlxpuGBqIXYQn56ePrcO3fXgvkAEh+/29HSwqr56OjO3mL1Yb08E09oUx14lta+GiYQ3dCHwRBFDJfYGx9Na5/z3HeddScgdqUiQTnDTADRME8AgDfAHAKgEeEECcQ0VkA3lfaYR0aJEJMQ50ZyxcJok4WCcNAzs0MBvLJaLYt0JmxMNYNRd3RKgWBahpyJqCclbdTp1OJ2Kgh3VyiIoSItC+/svUAHn59V/SJBwipOUXlOci6SsWs7nXtp1j/iO6sbdUc/fKzTCdNp9aQL3w0XiO47l3HIGsLfPm+lUWNRTJCKy4nL+nXCEzf949hDoZiBEFWCNFCRAYRGUKIx4johyUf2RDiGxcejdljRwS2mz6NwJnU/6VNoKpGYRjOpKYLAjnZjKmvAgDscE0h6upUrnhztpPpmjAIKZNiV75WjEaQsWyfRqLy9p885XseJzRKiXzriunCpieQSeEx4wsPYGJTGl97+1Ghx+nhm7pGIMNHa6sSIeGjwRX56BFV3uNL509BT9byBMGPLzsBG/d24gcPr/EdM3VUja+vgX5eKQfViX/h7DEFu+MxTLEUo1u2ElEdgCUAfktENwEY/oXuFd5/ylTMnx5s9q3GoMtJ60f/WuvbRy1B4eQRCM8hKV/rdCNTRrlJSHJyUssXyGMc05CNqoQBw41CiiJuVRxlnw6jVE7nQrJFCtpicgJ0jcArCyGALfu6ccWvloUeJyd6ORZdEDy73mkuU1tlwgopOgf4J2g9bNRQbvLtx00I7e178bzJ2Hjd+VhwWLM7Fv8bI81/qmnoHcdNwLUXHB16TwzTV4oRBBcA6AbwaQD/ALAOwNtLOahDhTCNQEfVCJw8AoFed6KXr3W6xeniYs+zniBwooaqkyYMihcEcc5ivUZOHMVMxG09WUy75oFIe38Y8rRRfo68aaiIMeoaQchB+qbalOmZhrzy3pogeHz1HlQnDSQMchrThGgEZowg0CN7wvwKctsvLj8JL39tceD1MNMQwwwkcSUmbiai04UQnUIISwiRE0L8SgjxIyFES9RxyvG3E9FuIlqpbf8kEa0ioleJ6HsDcRPlQl25qavCOePyZqQwQdClNZ6XDsk4QSAjhXJu7Hl10oRJVCB81P+imizVJ0FQxEy82TVt3PbUhqLPWwg5OXdncr48izB0Z3GYNqTb/yePrPHeVyNCIwCAmlTCywFRw0ulaS1OEOhm/KoQv4K8z1TCCG0+Y3saAQsCpjTEaQRrAHyfiDYS0feI6IQ+nvuXAM5TN7iO5gsAHCeEOArA9/t4ziGLagL4x9VnerVm1BWgQQRLCM8UJCdmKRhGVCcDE4dE9ijO2gI9OQtVCQNE8Y5UXSNQI2D6axq678VteDTEkSzH0Ze5qljT0OfveQUnf/vR2H0DeQQh78tuRZiMb6jG6bOavYldJpuFCYK0q33pOSKy7HOcINDNPNUhGkFU32VJ3kfAUUJMaYhLKLtJCHEqgDcBaAFwu7uS/yoRHV7oxEKIJQD2aZuvAnCdEKLX3Wd3/4c+tNDVdmnuUDWChOmYF2Sfgo0tnejszWH1rnYATjhoVEhgpysscpaN3qyNKndyivMD6A5UtSdBXzQC9TRX/2FFqL1dDsPoh1NZn7O37u/CtGsewPMb/F8fdcLUBY4eKhtmGtqpCILrLzoONSmnRITjDHe2yy5i5x41Nn8tA54Zbp8S0pv0wjqjBYHO7HEjMGtMna8WUWFB4GoEbBpiSkTBJYYQYpMQ4rtCiBMAXAbgQgCv9/N6hwNYQETPEdETRHRS1I5EdCURLSOiZXv27Onn5QYPuSqUq0Q5DenO4pwtvJX5Q6/uwtk3POFFldRWFU4SylnC9REYMA3ysoyj9lXpryAopqSzpxH0w3yhZ/NKAaDW7wf8oZ667T3KWayyQ0lMMwznsxLC2Vd3FqvVQ23buV5P1vaFusqEQHUsevE7namjavHIZ97kO3+mQOIf+wiYUlNMQlmCiN5ORL+Fk0i2GsC7+nm9BICRcPISPgfgjxQRlyiEuNXtkzxv9OjR/bzc4CEjiNJa+WA1e1hG+agF5tTJqSZlFmw36NQaypuG1Am9NmVi1pg677muLXT0qKahaAGiU4yzWPTRNPTSllbc8fRGZyxaCGeUVqGGeupfG90fYmlNZABg8758iKZJ5L3XW/d3e4JECgI1R8AWAgbl8zq+/c5jsPG680NNQ2EJiB88bRru+JB/zaNqLKcWqkvEpiGmxMQVnTsHjgbwVgDPA/g9gCuFEAcTOroVwL3C+YU+T0Q2gGYAQ3/JXwA5GaTdCUTOQSlfQhnhuQ37IlfjdVWJgj/2bM5xFo+qc1aUqq2fiHxmCt1c0t5PjaAYZ7HUPoo1DV10yzPeY91fEXWKnkx+v6RBUCs66ULPEiK4TXluGuRN2gu//7i3Xb4vqiCwbAHDIK/RvFzNy+MLfWZfe0cwh0EO5XcfORmnzowXBDabhpgSE5eR8gUAvwPwP0KI/QN0vfsAnAXgMdfPkAIwLAqpe4IgJQWB8+NV+8gaRLETcDplehpE0qTQ8M+c7YaPJkxkLBtZ7XzqZKE7UAfCWRyFNO/o9f2jUE9Z7FhUjUAXOLqd3bJFZGN7wNHOUjETa9qnETj3JR3FzXV+QWAYwK3vn4stIc3po5DvaViUkI68i/74XximGOL6Ebz5YE5MRHcBWAigmYi2AvgqgNvhOJ1XAsgAuFwMkzZaXtkHqRG4230+Aq02kTrBnn3EWFQlTG8ir0klQiNYZEJZddJAzrYD9nU1nyHnOkIlUYLgt89tQjpp4l0nToq9tzikgCtmrtrV1uM7p26mispiVktC69FS+qRvCxHr2zCJYutIqdm9jmkoP6YxI5xSIFJoJwwDi48aF3muMOT4dVNiGPOmjcSSNXs4fJQpGSXLURdCXBbx0iFfp+jwsXVYs6vDt82rSSN/2O68pOcRSBKaIPjcubOd/d2JvDZlhgoCp/qohaqEia6MhV6lnv2oupTP2ZzTEqBk4hrgd9B+6c+Os/pgBIFckRezatVDQXu1mvxRZ1A1At1voWs/OSteI1BNQ2GompxjGnIen33EWEwe6dQTkmfvTzlo+Z4WU676p+89EZtauoral2H6A3uf+sFfPnEGXvqqPwN0jFtjZvGRY33b4wSBiuw45WkEEXVksjmB9p4cRlQnYBCh152A508fid/+58m+a+Qsf7XM3YqTWnfQxmELx/H640ffUM7tn7ylhlHIn9kZUmCu17Jh2wJ/eWm7L4JHp8fXG8D/2ktbWgNjjgutNQ2KdMwnTfKF8ap9CI6f3OBpLHIyL9Ycpo9PXqsQtVUJHDmhYlqAMGWAq1b1g+qkGVidTR1Vi+e/uMgrOhYVPiq36XOUbiKoDTEZNNdVYWdbD7qzFppqU9h+oNszybz/lKmY1FTj8xHIAnWSW55Y5z2WGkExlrmcLbCxpctXLK07a2GEMllmi3QWy6Y8Kr1ZG39avgWfv+cV7O/MoLmuKuRIoFtxFhcat2X7NZlUwvD5Z8wYH0HCMHyC+rAxdZ6gUz93WXL6igXTY8cSxjlHjsNdz29GbUzjI4YZLFgjGEDG1Fd7q8V8oTB/+CjgZBvr5hY5Icj5TW9eDwAzmmu9EMjGmqRv0lXt1ZKcFb0qluaYsObpOpYtsGKLfwLX6/jLSbaQINivtfEEHB/BVtfRGva6d80YjSA4ZtVYGn4AABaBSURBVNvnQK7RBKtB5BPSAEIbwIxvqMZtl5/kaVA1ysTdkE5i43Xn4+J5k+MHE8K1FxyF5764KPRzZpjBhgVBibj1A/OwaM4YX/RJwhMEps+RSRQsPVwTslKc3lzrPW6qSfkmXekkTug+ggg7+Zpd7djd3uNV14zDsgVe2OQ3vXQFBIEs3uaYjRZ871/4+ys7AucKi4TqVZq4VyfNSCevz0dQQBLo/YX1lXeYj6DJzfZNmoYnQM+aMwYja1OeBpVODcxPJmkaXv8Jhik3vBwpEafPasbps5p926RGUJ30m4bSSTOvSbjbakNaZM4Y7RcEPp+D1AjU8FHLDk0GWzRnDB54ZQfufXFbUfkEli2wYa8/feSjdy7HQ58+U7lW3jTU2Wthy75ufP7ul/HWY8Zr5wperzeXb79ZnTAinbw9rvApxpy1rbXblx+gawRhUUNNtSlsbOlyKo0Kv/1falBpdtgywxDWCAYROanopYjVyUVOcgU1gtqkz6ma9DQC1W5vh66cT5vVjPaeXNFJZbYQ2Lq/y7dN1keSeMXbiLwm8T0h2cthGkFPxvJCSA0jutmO1AiKKUv9q6WbfM9157thBLOAm2qc/ICEQZ4wksI2rxHw2okZfrAgGEQSnkag2auVlb2nEYQ4i9XyEbppqCqkNn53xgr1EYzro0kikxOBuj8qu9t6cP1DqwE4piE52YcmxIUUWNu6vwvd7oq7N2tH+jXygqD4vAZJjfaeO87iCEFgGoHSz71Ky0qGGW6wIBhEVGexb7uyspdznG7KAPwaQWNN0he2KG3gqmmoozcXWoVTbacoiYuF39nWHdvkRgoBIGib11En+ROmNOLr7zgKnRkL63Y7eRk9WSvaR+CahvrTMS3UNJTw37P0ESTMfFVX030/ZdQQCwJmOMKCYBCRE7euEailiz3TkGLKkCWLiQi/+vB8XDJvMqoSpi9mf0S1KwiUCb0rQiNoqgmWNYiLZ9+4tyt0uxQy6hUeXbUbz2/Uq4/nUQVKyjRw9EQnPv61HW0AHHNSlI+gN2fh7uVbcfp1/4o8fxRhWphuGpJN42XvASD/mQ20s5hhhhJs8BxEpPlG1wjOVcoT5J3F+Y/mic+e5dXSf9Pho/Gmw51qrKppSO6vJkJ19OZCnbNhfXOTMZlgatVOAJjYmMa21m5kbRtVhhnIBP7vu170jeFAdxY5y8bUUbU+05BlC8xorvMd25MN92sAzqr8s396KXKcceiCzqSgaaha+XxkRM+kphoAeU2NfQTMcIS/1YPIaDdRSq2l859nTMfHFs7ynnt5BIopo6EmiYaQVbxfEDj7q6aL9p4clq4Lhoc2htTMjwv/39TS6Y2pM2N5yW8/e2I9PvnmWdEHAnjHj5/Cejfi6InPLfSVgjj7yLGo0aKjerIWshGmoULO7QWHNePJN8JrGOo5A6YZ1Aik1tBcV4V3nzgRjekkFh0xxrcPm4aY4QgLgkFkbIOzyjzQncVDV5+Jl7a04uKTwpORwqKGdKQVKGUaXv9caSKSfPn+VwPHJUwDDemkr5ZRXDmGTS1dGD2iCkIIdGYsz95+w8NrMG9qk6/hjc56Jex0b0cGOctGwiA8/6WzPRMVUV4A9mTtyNyHsPIUkvnTR+K6dx8baTbSJ30zJKFM7jOqNgUiwtlauRCABQEzPGGD5yAio3VauzKYPW5EqBCQoZdheQQ60vms7juiiLLGQNBPIAVBmHO5pTODSU1pyHJwqr29M2P5+vjGkTAcJ2zCJIx0J1siQnUif77eXLhfAwC2t/ob2P/f+Ufgv940E4AzsU9sTGPjdeeHHhvQCAwKmIukYJS9HsLQE/8YZjjA3+pBRNqdw6qKSvJRQ8VoBFIQ5PfVNYIo9PNL232UWWbaqFrPfKRG4Fi2HVsWQsU0CFnLDvgj1Mm1J2v7ooZGKPe2q90vCFIJA9Oba7xzx6ELAiMkoUzex6iIWkdAdIlshjmUYdPQICI1gv1dhQWBXOVfNn9K5L5y7qtTBUGRtWv0la0tgC37uny9dFUuOH4Cnlnn2N9V80jWEtjXGX0/OjlLBDptqZO0rhG8afZoHDWhAfev2IZVO/1JbETkCcNC87PuGDYNgurmvueq0zCpKY0Neztx2UnB93zm6Fqs23MwzfkYZujCgmAQqU8n0FSTxGcWz47cR4aPViVMrPz6ubE2aWka8gmCIk1DYbXtF3zvMbz0lcUhewNnHjYahGBfZtkf4YozpmPFltbQ6qKSrOWs9hNmcHUu6claPh9ByjRw1cKZeHZ9S0AQGJTXBPx9g4Pd3fTVv0H+1f3cqU0AgJ9/YF7o2P/88dNxIEaAM8yhDJuGBhEiwotfWYz3nzI1ch85fSUMQl1VItbk0RfT0G//82Tfcz2EVdKVDXfIGgZ5GohqGurKWMjkbKQSRmjWsErOFshawtc8B8hrQYA0Dfl7CwNB0w7g3L98XRUm/xGiRemCQBUCY+ujTUGS+uokJo+sKbgfwxyKsEYwxJCTYjFdr2SyU10RgkA3BUV1u9qjNK/RkZOnqqV09OSQsWykTCOwCjfIXxcoa9nIWbaXrRuGnlkstQfdtAM4958XBPntX37bkTj7yLF4/y+ez58n4v3840dPxbRRPMEzlQ1rBIcwclGrloyQpqGUaeATZ83CKNfmr/cJiBIE24powK4mVbW6ju8qt4eyiu6QtmyBrC1ik9d6cpYvoSyhaQS1KdOL9iHKC0P1/hKmgQWHjfadNyraZ/70kRjD5aCZCocFwRBDOocbQxLIdNp7HDPORLdTFpDXCDKWjc+eOxuzx40AALT1+E0+URPjG7s7AtvkvnL+VjWC/W7oaEqp4Z8/zi9snD7CdsBZLBlZm0J3xvJpFnJfqRE01qS8nAmDyNM44jSoD5w6FZeEOIAZhnFgQTDE+O9Fs7D2W28pKnx0xwFn9T5BEQRy8v3Q6dMAAOcf6/QDGKMVmqtKhGsEa7Ty0gBwnlsCQzqLVR/Bv1btds8X7COgF3pzTEPCVyobyOdONNel0N6Ti9UIGtJJz79hGPlKpHGd0f7nnNmhPgaGYRzYRzDEIKLIFbPOjgNOXP34Rr9pY8N33urZ89978lScfcRYjK2vxjcuPBoz3eY2URPjG7v8GsE/rl7g1QOSi241ami361OoSpiBGkG6IMhJ05AZ7ixurqvCml0dvjaYh411NBrp7K1PJ9Da5QoCyjeQMWI0gjifBMMwLAgOaeTKWDUNAcGkJ5nIpkYrRa2g1+xu9zl554yrD5w3LKQ1lTB8PYIBv8AA8s5iPXxUIpvWt3bnE9TefuwE7/wAUFeVRFUybxqS14yb66McxQzDOLAgOIT58WUnYun6vf3qfRsxF0MIJ7N2b0cwekhOp2GO5lBBoO1n2cIpMaGHj7r/PUHQlcXhY+tw91WnecJECoL66kTeNKT4CMIEm+l2OysmAothKhk2nB7CjGuoxjtPmNSvY+XEOao2hWveMsf3WnNErR0514aZrsKcxbogkM5iPabfu+4I57qtXVlUJUzUK8lxKfeaVUnTEwoG5WsjhZV+uP/jp+NjC2d6gmf5/50del2GqXRYI6hQDM+HMAWHj/X3BJgxujaQxQvkJ9uw8M+qZNBZrJuGnt3Qghc2t2LhbH9op0Q1DY1r8Gs5cvKvShieRkCKjyBMthw9sQFHT2zwno+qq8I/rl6ArfsKh8gyTCXBgqBCkeYSWyAQxXPMxEaMb0jj2EkNvu1GAY3gzivm46JblnrbdI3g3he2OcdHmGpkz+CerB1wKEurU1XS8ExTplFc+KjKnHH1Pr8HwzAlNA0R0e1EtJuIVirbvkZE24hohfv31lJdn4lHTpyWCNrQZ42pw5ffdiQuOH6ib/sNFx+Ps48YgzluboJKKmFg3rSRuOHi47xtYX2XAaCz1/I9v3LBDADAeEUL0E09vTnnmKqE6WkEQghMccs+HDmeJ3eG6S+l1Ah+CeAnAH6tbb9RCPH9El6XKQI5z9pKs/kTpzTiPfMmY9GcMaHHHD2xAbddfpI3KavIvAQ1LLXaFQSytaVEL8P9kTNn4CNnzsCutnyZ6ec3+Psey+bxjmnIOW/GsrH4sGb87ZNn4KgJLAgYpr+UTCMQQiwBEN3FnCkrsjSDLYRXGqI+ncRl86fExuQD4T4CKQDUmkDSNLT4KH+nr7ae8CqedTEltHuzeUEgryW3HT2xgfsEMMxBUA4fwSeI6AMAlgH4HyFEdN1ipmRMb651/9d5JR10X0EUYYJCmmtUjUCu3HWfQFRjnpqUiUvmTUZTbQpv1rSSvGko7yzOFKh2yjBMcQx2+OhPAcwEcDyAHQB+ELUjEV1JRMuIaNmePXsGa3wVw+KjxuGeq07FZfMne6Yh3UEbx+fOnY37P36691xOzmpJbLlIdxrI5I9t7wkvdU1E+O5Fx+Kat8zB/Okjfa9dccZ0TG+uxVuOGe85i3uzQRMVwzB9Z1A1AiHELvmYiH4O4G8x+94K4FYAmDdvXnRndabfzJ3qTLanzRyFiY1pfPysWUUfq+8rNQG1F7Kc+4UQSJgGMrn+r+BnjK7DY59dCCAvdHoO4nwMw+QZVEFAROOFEDvcp+8EsDJuf2ZwaKxJ4elr3nxQ55CCoLEmn4wmNQIhgKRBkIUjZEG8/nLCFKebmJ7/wDBM/yiZICCiuwAsBNBMRFsBfBXAQiI6Hk5VgY0APlqq6zODi1cmOq1qBPlcBRmi+pEF0/Gl8488qGudd/Q4LPncWZjCDWUYZkAomSAQQlwWsvkXpboeU15kIbmognJSEBRTXrsYWAgwzMDBmcXMQXHbB+bhry9vD33NMw1BeJFJtVXhSWYMw5QPFgTMQXH2kWNx9pFjQ1+Tsf1CwHMUD5RGwDDMwMHVR5mSoQajyph/1ggYZujByzNmwHnfKVPw2vY2JWooH/3bn94JDMOUFhYEzIDzzQuPAQD8a5WTNjJrTD7MU7a9ZBhm6MCmIaZkvHnOWNxz1al4n9Iic2x9VRlHxDBMGKwRMCVFZi9LuDgcwww9WBAwg8KNlxwHs8iidgzDDC4sCJhBob+9lRmGKT28RGMYhqlwWBAwDMNUOCwIGIZhKhwWBAzDMBUOCwKGYZgKhwUBwzBMhcOCgGEYpsJhQcAwDFPhsCBgGIapcFgQMAzDVDgsCBiGYSocFgQMwzAVDgsChmGYCocFAcMwTIXDgoBhGKbCYUHAMAxT4bAgYBiGqXBYEDAMw1Q4LAgYhmEqnJIJAiK6nYh2E9HKkNf+h4gEETWX6voMwzBMcZRSI/glgPP0jUQ0GcBiAJtLeG2GYRimSEomCIQQSwDsC3npRgD/C0CU6toMwzBM8SQG82JEdAGAbUKIl4io0L5XArjSfdpBRKv7edlmAHv7eexwgO+f75/vv3KZWsxOgyYIiKgGwBfhmIUKIoS4FcCtA3DdZUKIeQd7nkMVvn++f77/yr3/YhnMqKGZAKYDeImINgKYBOAFIho3iGNgGIZhNAZNIxBCvAJgjHzuCoN5QohKVtsYhmHKTinDR+8CsBTAbCLaSkRXlOpaBTho89IhDt9/ZcP3zxSEhODgHYZhmEqGM4sZhmEqHBYEDMMwFc6wFQREdB4RrSaitUR0TbnHM1AQ0WQieoyIXiOiV4noU+72kUT0MBG94f5vcrcTEf3IfR9eJqITlXNd7u7/BhFdXq576g9EZBLRi0T0N/f5dCJ6zr3PPxBRyt1e5T5f674+TTnHF9ztq4no3PLcSd8hokYiupuIVhHR60R0agV+/p92v/8rieguIqqupO/AgCOEGHZ/AEwA6wDMAJAC8BKAI8s9rgG6t/EATnQfjwCwBsCRAL4H4Bp3+zUAvus+fiuABwEQgFMAPOduHwlgvfu/yX3cVO7768P78BkAvwPwN/f5HwFc6j6+BcBV7uOPAbjFfXwpgD+4j490vxdVcMKa1wEwy31fRd77rwD8p/s4BaCxkj5/ABMBbACQVj77D1bSd2Cg/4arRjAfwFohxHohRAbA7wFcUOYxDQhCiB1CiBfcx+0AXofzw7gAzgQB9/+F7uMLAPxaODwLoJGIxgM4F8DDQoh9Qoj9AB5GSG2ooQgRTQJwPoDb3OcE4M0A7nZ30e9fvi93A1jk7n8BgN8LIXqFEBsArIXzvRnSEFEDgDMB/AIAhBAZIUQrKujzd0kASBNRAkANgB2okO9AKRiugmAigC3K863utmGFq+KeAOA5AGOFEDvcl3YCGOs+jnovDuX36Idw6lXZ7vNRAFqFEDn3uXov3n26rx9w9z9U7386gD0A7nBNY7cRUS0q6PMXQmwD8H04hSt3wPlMl6NyvgMDznAVBMMeIqoDcA+Aq4UQbeprwtF7h2VcMBG9DcBuIcTyco+lTCQAnAjgp0KIEwB0wjEFeQznzx8AXP/HBXCE4gQAtTi0tJkhx3AVBNsATFaeT3K3DQuIKAlHCPxWCHGvu3mXq/LD/b/b3R71Xhyq79HpAN7hZqb/Ho454CY4Jg+ZKa/ei3ef7usNAFpw6N7/VgBbhRDPuc/vhiMYKuXzB4CzAWwQQuwRQmQB3Avne1Ep34EBZ7gKgn8DOMyNIkjBcRD9pcxjGhBc2+YvALwuhLhBeekvAGTkx+UA7le2f8CNHjkFwAHXhPAQgMVE1OSusBa724Y0QogvCCEmCSGmwflc/yWEeC+AxwBc5O6m3798Xy5y9xfu9kvdiJLpAA4D8Pwg3Ua/EULsBLCFiGa7mxYBeA0V8vm7bAZwChHVuL8H+R5UxHegJJTbW12qPzjREmvgRAJ8qdzjGcD7OgOO2v8ygBXu31vh2DwfBfAGgEcAjHT3JwA3u+/DK3DqO8lzfRiOg2wtgA+V+9768V4sRD5qaAacH/FaAH8CUOVur3afr3Vfn6Ec/yX3fVkN4C3lvp8+3PfxAJa534H74ET9VNTnD+DrAFYBWAngTjiRPxXzHRjoPy4xwTAMU+EMV9MQwzAMUyQsCBiGYSocFgQMwzAVDgsChmGYCocFAcMwTIXDgoBhFIjoS25Vy5eJaAURnUxEVxNRTbnHxjClgsNHGcaFiE4FcAOAhUKIXiJqhlPd8xlwf21mGMMaAcPkGQ9grxCiFwDcif8iOPVsHiOixwCAiBYT0VIieoGI/uTWfQIRbSSi7xHRK0T0PBHNcre/x62b/xIRLSnPrTFMNKwRMIyLO6E/Baes8SNw6tY/4dY1mieE2OtqCffCyULtJKLPw8lgvdbd7+dCiG8R0QcAXCyEeBsRvQLgPCHENiJqFE7ZaIYZMrBGwDAuQogOAHMBXAmn1PMfiOiD2m6nwGlo8jQRrYBTw2aq8vpdyv9T3cdPA/glEX0ETtMkhhlSJArvwjCVgxDCAvA4gMfdlbzewpHgNHS5LOoU+mMhxH8R0clwmuksJ6K5QoiWgR05w/Qf1ggYxoWIZhPRYcqm4wFsAtAOpy0oADwL4HTF/v//27ljE4RjIIrD71UWWugMgns4hbiAU4gLWDqEK7iBWOoQ7iAIZ5EDUyqI/+J+X5MmB0n1uARubHvR1ay69Zx75hFxiYidWqfRjz4GBkdHALxNJB1sTyU91aZVbiStJZ1s3yNimc9FR9ujrNuqTbqVpJntm6RH1knSPgPGahNCr3+5DfAhPouBH+k/lYc+C/ANnoYAoDg6AgAojo4AAIojCACgOIIAAIojCACgOIIAAIp7ASk4f2RpzNUVAAAAAElFTkSuQmCC\n",
-      "text/plain": [
-       "<Figure size 432x288 with 1 Axes>"
-      ]
-     },
-     "metadata": {
-      "needs_background": "light"
-     },
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "import matplotlib.pyplot as plt\n",
-    "plt.plot(average_mae_history[:,0],average_mae_history[:,1])\n",
-    "plt.xlabel('Steps')\n",
-    "plt.ylabel('Validation MAE')\n",
-    "plt.ylim((14, 20))\n",
-    "plt.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's plot this:"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "According to this plot, it seems that validation MAE stops improving significantly after 150 epochs. Past that point, we start overfitting.\n",
-    "\n",
-    "Once we are done tuning other parameters of our model (besides the number of epochs, we could also adjust the size of the hidden layers), we \n",
-    "can train a final \"production\" model on all of the training data, with the best parameters, then look at its performance on the test data:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 50,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "creating: createZooKerasSequential\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createZooKerasDense\n",
-      "creating: createRMSprop\n",
-      "creating: createZooKerasMeanSquaredError\n",
-      "creating: createZooKerasMAE\n"
-     ]
-    }
-   ],
-   "source": [
-    "# Get a fresh, compiled model.\n",
-    "model = build_model()\n",
-    "# Train it on the entirety of the data.\n",
-    "model.fit(train_data, train_targets,\n",
-    "          nb_epoch=150, batch_size=16)\n",
-    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 51,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "1.7991065979003906"
-      ]
-     },
-     "execution_count": 51,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "test_mae_score"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We are still off by about \\$1,800."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Wrapping up\n",
-    "\n",
-    "\n",
-    "Here's what you should take away from this example:\n",
-    "\n",
-    "* Regression is done using different loss functions from classification; Mean Squared Error (MSE) is a commonly used loss function for \n",
-    "regression.\n",
-    "* Similarly, evaluation metrics to be used for regression differ from those used for classification; naturally the concept of \"accuracy\" \n",
-    "does not apply for regression. A common regression metric is Mean Absolute Error (MAE).\n",
-    "* When features in the input data have values in different ranges, each feature should be scaled independently as a preprocessing step.\n",
-    "* When there is little data available, using K-Fold validation is a great way to reliably evaluate a model.\n",
-    "* When little training data is available, it is preferable to use a small network with very few hidden layers (typically only one or two), \n",
-    "in order to avoid severe overfitting.\n",
-    "\n",
-    "This example concludes our series of three introductory practical examples. You are now able to handle common types of problems with vector data input:\n",
-    "\n",
-    "* Binary (2-class) classification.\n",
-    "* Multi-class, single-label classification.\n",
-    "* Scalar regression.\n",
-    "\n",
-    "In the next chapter, you will acquire a more formal understanding of some of the concepts you have encountered in these first examples, \n",
-    "such as data preprocessing, model evaluation, and overfitting."
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.5.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

From fb2f9ecfc8fa7ce6f800b6d8ee3493707a3b1cd2 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 21 Mar 2019 16:09:14 +0800
Subject: [PATCH 33/46] Add files via upload

---
 keras/4.4-overfitting-and-underfitting.ipynb | 704 +++++++++++++++++++
 1 file changed, 704 insertions(+)
 create mode 100644 keras/4.4-overfitting-and-underfitting.ipynb

diff --git a/keras/4.4-overfitting-and-underfitting.ipynb b/keras/4.4-overfitting-and-underfitting.ipynb
new file mode 100644
index 0000000..d28bebd
--- /dev/null
+++ b/keras/4.4-overfitting-and-underfitting.ipynb
@@ -0,0 +1,704 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=32g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=32g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Overfitting and underfitting\n",
+    "\n",
+    "\n",
+    "----\n",
+    "\n",
+    "\n",
+    "In all the examples we saw in the previous chapter -- movie review sentiment prediction, topic classification, and house price regression -- \n",
+    "we could notice that the performance of our model on the held-out validation data would always peak after a few epochs and would then start \n",
+    "degrading, i.e. our model would quickly start to _overfit_ to the training data. Overfitting happens in every single machine learning \n",
+    "problem. Learning how to deal with overfitting is essential to mastering machine learning.\n",
+    "\n",
+    "The fundamental issue in machine learning is the tension between optimization and generalization. \"Optimization\" refers to the process of \n",
+    "adjusting a model to get the best performance possible on the training data (the \"learning\" in \"machine learning\"), while \"generalization\" \n",
+    "refers to how well the trained model would perform on data it has never seen before. The goal of the game is to get good generalization, of \n",
+    "course, but you do not control generalization; you can only adjust the model based on its training data.\n",
+    "\n",
+    "At the beginning of training, optimization and generalization are correlated: the lower your loss on training data, the lower your loss on \n",
+    "test data. While this is happening, your model is said to be _under-fit_: there is still progress to be made; the network hasn't yet \n",
+    "modeled all relevant patterns in the training data. But after a certain number of iterations on the training data, generalization stops \n",
+    "improving, validation metrics stall then start degrading: the model is then starting to over-fit, i.e. is it starting to learn patterns \n",
+    "that are specific to the training data but that are misleading or irrelevant when it comes to new data.\n",
+    "\n",
+    "To prevent a model from learning misleading or irrelevant patterns found in the training data, _the best solution is of course to get \n",
+    "more training data_. A model trained on more data will naturally generalize better. When that is no longer possible, the next best solution \n",
+    "is to modulate the quantity of information that your model is allowed to store, or to add constraints on what information it is allowed to \n",
+    "store. If a network can only afford to memorize a small number of patterns, the optimization process will force it to focus on the most \n",
+    "prominent patterns, which have a better chance of generalizing well.\n",
+    "\n",
+    "The processing of fighting overfitting in this way is called _regularization_. Let's review some of the most common regularization \n",
+    "techniques, and let's apply them in practice to improve our movie classification model from  the previous chapter."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note: in this notebook we will be using the IMDB test set as our validation set. It doesn't matter in this context.\n",
+    "\n",
+    "Let's prepare the data using the code from Chapter 3, Section 5:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import imdb\n",
+    "import numpy as np\n",
+    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(nb_words=10000)\n",
+    "\n",
+    "def vectorize_sequences(sequences, dimension=10000):\n",
+    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
+    "    results = np.zeros((len(sequences), dimension))\n",
+    "    for i, sequence in enumerate(sequences):\n",
+    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
+    "    return results\n",
+    "\n",
+    "x_train = vectorize_sequences(train_data)\n",
+    "x_test = vectorize_sequences(test_data)\n",
+    "\n",
+    "y_train = np.asarray(train_labels).astype('float32')\n",
+    "y_test = np.asarray(test_labels).astype('float32')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Fighting overfitting\n",
+    "\n",
+    "## Reducing the network's size\n",
+    "\n",
+    "\n",
+    "The simplest way to prevent overfitting is to reduce the size of the model, i.e. the number of learnable parameters in the model (which is \n",
+    "determined by the number of layers and the number of units per layer). In deep learning, the number of learnable parameters in a model is \n",
+    "often referred to as the model's \"capacity\". Intuitively, a model with more parameters will have more \"memorization capacity\" and therefore \n",
+    "will be able to easily learn a perfect dictionary-like mapping between training samples and their targets, a mapping without any \n",
+    "generalization power. For instance, a model with 500,000 binary parameters could easily be made to learn the class of every digits in the \n",
+    "MNIST training set: we would only need 10 binary parameters for each of the 50,000 digits. Such a model would be useless for classifying \n",
+    "new digit samples. Always keep this in mind: deep learning models tend to be good at fitting to the training data, but the real challenge \n",
+    "is generalization, not fitting.\n",
+    "\n",
+    "On the other hand, if the network has limited memorization resources, it will not be able to learn this mapping as easily, and thus, in \n",
+    "order to minimize its loss, it will have to resort to learning compressed representations that have predictive power regarding the targets \n",
+    "-- precisely the type of representations that we are interested in. At the same time, keep in mind that you should be using models that have \n",
+    "enough parameters that they won't be underfitting: your model shouldn't be starved for memorization resources. There is a compromise to be \n",
+    "found between \"too much capacity\" and \"not enough capacity\".\n",
+    "\n",
+    "Unfortunately, there is no magical formula to determine what the right number of layers is, or what the right size for each layer is. You \n",
+    "will have to evaluate an array of different architectures (on your validation set, not on your test set, of course) in order to find the \n",
+    "right model size for your data. The general workflow to find an appropriate model size is to start with relatively few layers and \n",
+    "parameters, and start increasing the size of the layers or adding new layers until you see diminishing returns with regard to the \n",
+    "validation loss.\n",
+    "\n",
+    "Let's try this on our movie review classification network. Our original network was as such:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras import models\n",
+    "from zoo.pipeline.api.keras import layers\n",
+    "\n",
+    "original_model = models.Sequential()\n",
+    "original_model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "original_model.add(layers.Dense(16, activation='relu'))\n",
+    "original_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "original_model.compile(optimizer='rmsprop',\n",
+    "                       loss='binary_crossentropy',\n",
+    "                       metrics=['acc'])\n",
+    "\n",
+    "import time\n",
+    "dir_name = '4-4 ' + str(time.ctime())\n",
+    "original_model.set_tensorboard('./', dir_name)\n",
+    "original_model.fit(x_train, y_train,\n",
+    "                   nb_epoch=20,\n",
+    "                   batch_size=512,\n",
+    "                   validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 512 records in 0.024455326 seconds. Throughput is 20936.135 records/second. Loss is 0.01585226.\n",
+    "Top1Accuracy is Accuracy(correct: 21341, count: 25000, accuracy: 0.85364)_"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's try to replace it with this smaller network:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "smaller_model = models.Sequential()\n",
+    "smaller_model.add(layers.Dense(4, activation='relu', input_shape=(10000,)))\n",
+    "smaller_model.add(layers.Dense(4, activation='relu'))\n",
+    "smaller_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "smaller_model.compile(optimizer='rmsprop',\n",
+    "                      loss='binary_crossentropy',\n",
+    "                      metrics=['acc'])\n",
+    "\n",
+    "dir_name = '4-4 ' + str(time.ctime())\n",
+    "smaller_model.set_tensorboard('./', dir_name)\n",
+    "smaller_model.fit(x_train, y_train,\n",
+    "                  nb_epoch=20,\n",
+    "                  batch_size=512,\n",
+    "                  validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "original_val_loss = np.array(original_model.get_validation_summary(\"Loss\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the smaller network starts overfitting later than the reference one (after 6 epochs rather than 4) and its performance \n",
+    "degrades much more slowly once it starts overfitting.\n",
+    "\n",
+    "Now, for kicks, let's add to this benchmark a network that has much more capacity, far more than the problem would warrant:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd0VNXax/HvTgdCb4EECKFDOqGJCEiVKk26dBQ7Kl4rei33Xi5eBGkKSK8qiiAoCoIgKqQACUkgCT2hJZSQXvf7R4a8ESkBMjmTzPNZaxaZM2dmnjkzzG/22efsrbTWCCGEEAA2RhcghBDCckgoCCGEyCehIIQQIp+EghBCiHwSCkIIIfJJKAghhMgnoSCEECKfhIIQQoh8EgpCCCHy2RldwL2qVq2adnd3N7oMIYQoUYKDgxO01tXvtl6JCwV3d3eCgoKMLkMIIUoUpdTpwqwnu4+EEELkk1AQQgiRT0JBCCFEvhLXp3ArWVlZxMbGkp6ebnQp4i6cnJxwc3PD3t7e6FKEELdQKkIhNjaW8uXL4+7ujlLK6HLEbWituXz5MrGxsdSvX9/ocoQQt1Aqdh+lp6dTtWpVCQQLp5SiatWq0qITwoKVilAAJBBKCHmfhLBspSYUhBCitMrOyeWjrRHEXUsz+3NJKBSzXr16ce3atTuuM336dHbs2HFfj79792769OlzX/ctrFOnTuHp6fnA6wgh7i43V/PGN2Es3nuS3ccumf35SkVHc0mgtUZrzbZt2+667vvvv18MFQkhLJ3Wmg+2RvBVcCwvdGnEyDb1zP6c0lIoIrNmzcLT0xNPT09mz54N5P1abtKkCU8++SSenp6cPXsWd3d3EhISAPjggw9o0qQJDz/8MMOHD+fjjz8GYOzYsXz99ddA3rAe7777Lv7+/nh5eXH06FEADhw4QLt27fDz8+Ohhx7i2LFjd6xv+fLlPP7443Tr1g13d3fmzZvHrFmz8PPzo23btly5cgWAQ4cO0bZtW7y9vRkwYABXr14FIDg4GB8fH3x8fJg/f37+4+bk5DBt2jRatWqFt7c3n3/+eRFuVSGs2yc7olm27xTj29dnatdGxfKcpa6l8M8t4UScu16kj9m8dgXe7dvitrcHBwezbNky9u/fj9aaNm3a0LFjRypXrkx0dDQrVqygbdu2f7lPYGAgGzdu5PDhw2RlZeHv70/Lli1v+fjVqlUjJCSEBQsW8PHHH7NkyRKaNm3K3r17sbOzY8eOHbz55pts3Ljxjq/jyJEjHDx4kPT0dBo2bMiMGTM4ePAgU6dOZeXKlbz00ks8+eSTzJ07l44dOzJ9+nT++c9/Mnv2bMaNG8e8efN45JFHmDZtWv5jfvHFF1SsWJHAwEAyMjJo37493bt3lw5lIR7Q4j0n+HRnNE8EuPFOn2bF9n9KWgpF4LfffmPAgAGUK1cOZ2dnBg4cyN69ewGoV6/e3wIBYN++ffTv3x8nJyfKly9P3759b/v4AwcOBKBly5acOnUKgMTERIYMGYKnpydTp04lPDz8rnV27tyZ8uXLU716dSpWrJj/nF5eXpw6dYrExESuXbtGx44dARgzZgx79uzh2rVrXLt2jUceeQSA0aNH5z/mTz/9xMqVK/H19aVNmzZcvnyZ6OjoQmw1IcTtrN1/ho+2RdLbqxb/HuhdrD+ySl1L4U6/6I1Qrly5B34MR0dHAGxtbcnOzgbgnXfeoXPnznz77becOnWKTp06FfpxAGxsbPKv29jY5D/uvdJaM3fuXHr06PGX5TfCSwhxb747FMdbm8Lo1KQ6nwz1xdameFvd0lIoAh06dGDTpk2kpqaSkpLCt99+S4cOHe54n/bt27NlyxbS09NJTk7m+++/v6fnTExMxNXVFcjrLygKFStWpHLlyvmtnFWrVtGxY0cqVapEpUqV+O233wBYs2ZN/n169OjBwoULycrKAiAqKoqUlJQiqUcIa7Mz8iKvfHmYVu5VWDiyJQ52xf8VXepaCkbw9/dn7NixtG7dGoCJEyfi5+d3x1/LrVq1ol+/fnh7e1OzZk28vLyoWLFioZ/ztddeY8yYMXz44Yf07t37QV9CvhUrVvD000+TmpqKh4cHy5YtA2DZsmWMHz8epRTdu3fPX3/ixImcOnUKf39/tNZUr16dTZs2FVk9QliL348nMGVNCM1rV+CLMQGUcbA1pA6ltTbkie9XQECAvnmSncjISJo1a2ZQRfcvOTkZZ2dnUlNTeeSRR1i0aBH+/v5Gl2V2JfX9EsJcDp65ysgl+3GrXIYNk9tRuZxDkT+HUipYax1wt/WkpWCgyZMnExERQXp6OmPGjLGKQBBC/FXk+euMXRZI9fKOrJ7QxiyBcC8kFAy0du1ao0sQQhjoZEIKo784QBl7W1ZPaEONCk5GlyQdzUIIYYS4a2mMWrKfXK1ZPbENdaqUNbokQEJBCCGKXXxSBqOX7Od6ehYrx7emYQ1no0vKJ6EghBDFKDE1i9Ff7Od8YjrLxrbC07XwRx0WBwkFIYQoJikZ2YxdfoAT8SkserIlAe5VjC7pbyQULJizc16T0tKGoe7UqRM3HxZ8P+sIYU3Ss3KYvCqI0NhEPh3uR4dG1Y0u6ZYkFEqh+x2yQghhHlk5uTy39iD7Yi4zc7A3PT1djC7ptiQUikBKSgq9e/fGx8cHT09PNmzYAOQNe/3GG2/g6+tLQEAAISEh9OjRgwYNGvDZZ58BeSewdenSJX9o7O++++6Oz3W7oap3795Nhw4d6NevH82bN//b/ZydnZk2bRotWrSga9euHDhwgE6dOuHh4cHmzZuBvLmux40bh5eXF35+fuzatQuAtLQ0hg0bRrNmzRgwYABpaf8/+9NPP/1Eu3bt8Pf3Z8iQISQnJz/4BhWiFMnMzuW5tSHsiLzI+/1bMNDfzeiS7qjUnafw0o8vcejCoSJ9TF8XX2b3nH3b23/88Udq167N1q1bgbxxiW6oW7cuhw4dYurUqYwdO5Z9+/aRnp6Op6cnTz/9NE5OTnz77bdUqFCBhIQE2rZtS79+/W47KuLthqoGCAkJ4ciRI9SvX/9v90tJSeHRRx9l5syZDBgwgLfffpuff/6ZiIgIxowZQ79+/Zg/fz5KKcLCwjh69Cjdu3cnKiqKhQsXUrZsWSIjIwkNDc0/yS4hIYEPP/yQHTt2UK5cOWbMmMGsWbOYPn36fW9rIUqTzOxcnl0bws8RF3mvb3OebOdudEl3VepCwQheXl688sor/OMf/6BPnz5/GQyvX79++eskJydTvnx5ypcvj6OjI9euXaNcuXK8+eab7NmzBxsbG+Li4rh48SIuLrduXv7000+EhobmT8KTmJhIdHQ0Dg4OtG7d+paBAODg4EDPnj3za3F0dMTe3j5/2GzIGwL8+eefB6Bp06bUq1ePqKgo9uzZwwsvvACAt7c33t7eAPz5559ERETQvn17ADIzM2nXrt2DbEohSo2M7ByeXRPCjshLvN+/RYkIBCiFoXCnX/Tm0rhxY0JCQti2bRtvv/02Xbp0yf+1XHB46puHrs7OzmbNmjXEx8cTHByMvb097u7upKen3/a5bjdU9e7du+84TLe9vX1+66Moh83u1q0b69atu6/7C1FaZWTn8MzqEHYevcQH/VswuoQEAkifQpE4d+4cZcuWZdSoUUybNo2QkJBC3zcxMZEaNWpgb2/Prl27OH369B3XN+dQ1R06dMgfFjsqKoozZ87QpEkTHnnkkfwhOY4cOUJoaCgAbdu2Zd++fcTExAB5u6iioqKKpBYhSqr0rByeXhXMzqOX+PBxzxIVCFAKWwpGCAsLY9q0adjY2GBvb8/ChQsLfd+RI0fSt29fvLy8CAgIoGnTpndc35xDVT/zzDNMmTIFLy8v7OzsWL58OY6OjkyZMoVx48bRrFkzmjVrlj9taPXq1Vm+fDnDhw8nIyMDgA8//JDGjRsXST1ClDTpWTk8vTqY3cfi+dcAL0a0qWt0SffMrENnK6V6AnMAW2CJ1vo/N93+CdDZdLUsUENrXelOj1mahs62VvJ+idIoPSuHp1YF82tUPP8e6MXw1pYVCIYPna2UsgXmA92AWCBQKbVZax1xYx2t9dQC6z8P+JmrHiGEMJf0rBwmrQzit5gEZgzyYmgrywqEe2HOPoXWQIzW+oTWOhNYD/S/w/rDAemxFEKUKH8JhIHeJToQwLyh4AqcLXA91rTsb5RS9YD6wC/3+2QlbQY5ayXvkyhN0jJzmLgiLxD+O8ibJ1rVMbqkB2YpRx8NA77WWufc6kal1GSlVJBSKig+Pv5vtzs5OXH58mX5wrFwWmsuX76Mk5PxE4kI8aDSMnOYsCKQfccT+HiwD0MCSn4ggHmPPooDCm4lN9OyWxkGPHu7B9JaLwIWQV5H8823u7m5ERsby60CQ1gWJycn3Nws+zR/Ie4mNTObCcuD2H/yMrOe8GGAX+n5TJszFAKBRkqp+uSFwTBgxM0rKaWaApWBP+73iezt7W97Jq8QQhSl1Mxsxi0LJPDUFWY94cvjfrfcK15imW33kdY6G3gO2A5EAl9qrcOVUu8rpfoVWHUYsF7Lvh8hhIVLychmrCkQPhla+gIBzHzymtZ6G7DtpmXTb7r+njlrEEKIopCSkddCCDp9hdnD/OjnU9voksxCzmgWQoi7SEzNYuLKQELOXGPOMD/6ltJAAAkFIYS4o+PxyUxcEUTs1VQ+HeZHb+9aRpdkVhIKQghxG3ui4nl2bQgOtjasm9TWIudULmoSCkIIcROtNSt+P8UHWyNpVMOZxU8GUKdKWaPLKhYSCkIIUUBWTi7Tvwtn3YEzdG1Wk9nDfHF2tJ6vSut5pUIIcRdXUzKZsiaYP09cYUqnBkzr3gQbm1tPjVtaSSgIIQQQfTGJCSuCuHA9nU+Glq6zlO+FhIIQwurtOnqJ59cdxMnelvWT2+Jft7LRJRlGQkEIYbW01izZe5J//RBJ81oVWPxkALUrlTG6LENJKAghrFJGdg5vf3uEr4JjeczThf894UNZB/lKlC0ghLA6CckZTFkdTOCpq7zwaENe6trY6jqUb0dCQQhhVY5euM6E5UEkJGcwd3jpHrLifkgoCCGsxs8RF3lp/UGcnez46ul2eLtVMrokiyOhIIQo9TKyc1iy9yQf/3QML9eKLBodgEtFmQHwViQUhBCl1unLKaw9cIavg2K5nJJJX5/azBzsjZO9rdGlWSwJBSFEqZKVk8vOyIus2X+GvdEJ2NooujarwYg29XikUTWUkg7lO5FQEEKUCrFXU9kQeJb1gWeJT8qgdkUnXu7WmCcC6siuonsgoSCEKLFycjW7jl5i7YEz7Dp2CYDOTWowsk1dOjWpga0cZnrPJBSEECXOhcR0NgSeZUPgGc4lplOjvCPPdW7I0FZ1cKtsHUNcm4uEghCiRMjN1eyNSWDNn6fZefQSObmaDo2qMb1vc7o0q4m9rY3RJZYKEgpCCIt3ITGdEUv+5ER8ClXLOTCpgwfDW9ehXtVyRpdW6kgoCCEs3odbI4i7msacYb709HTB0U4OKTUXCQUhhEXbF5PA96Hnmdq1Mf19XY0up9STnXBCCIuVkZ3DO98doV7VsjzV0cPocqyCtBSEEBbri99OciI+hWXjWslZyMVEWgpCCIsUdy2NuTtj6N68Jp2b1DC6HKshoSCEsEgfbIlAo5net7nRpVgVCQUhhMXZfewSP4Zf4PlHG8nJaMVMQkEIYVHSs3J4d3M4HtXKMbFDfaPLsTrS0SyEsCiL9pzg9OVUVk1oLecjGEBaCkIIi3H2Sirzd8XQ26sWHRpVN7ocqyShIISwGP/cEo6tjeLtPs2MLsVqmTUUlFI9lVLHlFIxSqnXb7POE0qpCKVUuFJqrTnrEUJYrh0RF9kReYmXujaiVsUyRpdjtczWp6CUsgXmA92AWCBQKbVZax1RYJ1GwBtAe631VaWUHIwshBVKy8zhvS3hNKrhzLj20rlsJHO2FFoDMVrrE1rrTGA90P+mdSYB87XWVwG01pfMWI8QwkIt3B1D7NU03u/vKUNgG8ycW98VOFvgeqxpWUGNgcZKqX1KqT+VUj3NWI8QwgKdTEjhs19P0N+3Nu0aVDW6HKtn9CGpdkAjoBPgBuxRSnlpra8VXEkpNRmYDFC3bt3irlEIYSZaa97dHI6jnQ1v9ZLOZUtgzpZCHFCnwHU307KCYoHNWussrfVJIIq8kPgLrfUirXWA1jqgenU5TE2I0mJ7+AX2RMUztVtjalRwMrocgXlDIRBopJSqr5RyAIYBm29aZxN5rQSUUtXI2510wow1CSEsRGpmNu9viaCpS3mebFfP6HKEidlCQWudDTwHbAcigS+11uFKqfeVUv1Mq20HLiulIoBdwDSt9WVz1SSEsBxzf4nhXGI6HzzuiZ10LlsMs/YpaK23AdtuWja9wN8aeNl0EUJYiZhLySzZe4JB/m60cq9idDmiAIlnIUSx0loz/bsjlLG35Y1eTY0uR9xEQkEIUay+Dz3P78cvM61HE6o5OxpdjriJhIIQotgkZ2Tz4dYIPF0rMKKNdC5bIqPPUxBCWJE5O6K4eD2Dz0a1xNZGGV2OuAVpKQghisWxC0ks3XeKYa3q4Fe3stHliNuQUBBCmF1uruad745Q3smO13pK57Ilk1AQQpjV6cspDFv0JwdOXuH1nk2pUs7B6JLEHUifghDCLHJzNav+PM1/fjiKnY3iv4O9GdLSzeiyxF1IKAghitzZK6lM+/owf564QsfG1fnPIC+ZOKeEkFAQQhSZ3FzNmgNn+Pe2SGyUYsYgL54IqINScqRRSSGhIIQoErFXU/nHxlD2xVzm4YbVmDHYG9dK0jooaSQUhBAPRGvN+sCzfLQ1Eq01/xrgxfDW0jooqSQUhBD37dy1NP6xMZS90Qk81KAqMwZ5U6dKWaPLEg9AQkEIcc+01nwVHMsHWyLIztV80L8FI9vUw0bOUi7xJBSEEPfkQmI6b3wTyq5j8bSpX4WZg32oW1VaB6WFhIIQolC01nwTEsd7W8LJysnlvb7NebKdu7QOShkJBSHEXSWmZfHKl4fYEXmJVu6VmTnYB/dq5YwuS5hBoUJBKdUAiNVaZyilOgHewEqt9TVzFieEMF5CcgZPfnGA6EtJvN27GePa15cRTkuxwo59tBHIUUo1BBYBdYC1ZqtKCGERziem8cTnf3AiIZklY1oxsYOHBEIpV9jdR7la62yl1ABgrtZ6rlLqoDkLE0IY61RCCiOX7Od6WhYrx7ehdX2ZS9kaFDYUspRSw4ExQF/TMnvzlCSEMNqxC0mM+mI/2Tm5rJvcFk/XikaXJIpJYXcfjQPaAR9prU8qpeoDq8xXlhDCKIfPXmPooj+wUfDlU+0kEKxMoVoKWusI4AUApVRloLzWeoY5CxNCFL8/T1xmwvJAqjg7sGZCWzn/wAoVqqWglNqtlKqglKoChACLlVKzzFuaEKI47Tp6iTFLD1CrUhm+euohCQQrVdjdRxW11teBgeQditoG6Gq+soQQxen70HNMWhlEo5rOfPlUO1wqOhldkjBIYUPBTilVC3gC+N6M9QghitmXgWd5Yd1B/OpWYu2ktjJdppUrbCi8D2wHjmutA5VSHkC0+coSQhSHL347yWsbQ3m4UXVWjm9DBSc5qNDaFbaj+SvgqwLXTwCDzFWUEMK8tNZ8ujOGT3ZE8ZinC7OH+eJoZ2t0WcICFLaj2U0p9a1S6pLpslEpJTNwC1ECaa3517ZIPtkRxSB/N+YO95NAEPkKu/toGbAZqG26bDEtE0KUIDm5mje/DWPx3pOMfcidmYO9sbMt7NeAsAaFPaO5uta6YAgsV0q9ZI6ChBDmkZWTy9QNh/g+9DzPdW7IK90by5SZFk5rzbmkcwSdCyL4fDD9m/SnZe2WZn3OwobCZaXUKGCd6fpw4LJ5ShJCFLX4pAz+sTGUX45e4vXHmvJ0xwZGlyRu4VzSOYLPBeeHQNC5IC6mXATAVtniVsHNYkJhPDAX+ATQwO/A2LvdSSnVE5gD2AJLtNb/uen2scBMIM60aJ7WekkhaxJC3MXZK6ks2nOCL4POkpWTy4ePezKqbT2jyxLAheQLfwuA88nnAbBRNjSv3pyeDXsSUDuAlrVa4uPiQ1l7859QWNijj04D/QouM+0+mn27+yilbIH5QDcgFghUSm02DZlR0Aat9XP3VLUQ4o6iLyaxcPdxvjt8DhsFA/3ceKqjBx7VnY0uzWrtPrWbvaf35gdAXFLeb2GFoln1ZnT16JofAL4uvpRzMGYSoweZee1l7hAKQGsgxnT4Kkqp9UB/4OZQEEIUkYNnrrJg93F+jrhIGXtbxj7kzsQO9alVsYzRpVmtnNwcpm6fytwDc1EomlRrQuf6nWlZqyUBtQPwdfHF2cFywvpBQuFuPVSuwNkC12OBNrdYb5BS6hEgCpiqtT57i3WEELehtWZfzGUW7I7h9+OXqVjGnhe6NGLsQ+5ydrLBkjOTGb5xON9Hfc/UtlP5Z6d/Ut6xvNFl3dGDhIIuguffAqwzTfP5FLACePTmlZRSk4HJAHXr1i2CpxWi5MvN1fwUcZGFu2M4HJtIjfKOvNWrGcPb1MXZUaZfN1rs9Vj6rutL2MUwFvRawJRWU4wuqVDu+MlRSiVx6y9/BdytPRpH3rSdN7jx/x3KAGitCx7BtAT4760eSGu9iLxpQAkICCiKMBKixMrKyWXzoXMs/PU4MZeSqVe1LP8a4MWglq5yEpqFCDkfQt91fUnKSOL7Ed/Ts2FPo0sqtDuGgtb6Qdo5gUAj04Q8ccAwYETBFZRStbTW501X+wGRD/B8QpRq6Vk5bAg8y6I9J4i7lkZTl/J8OtyPXp4ucgKaBdlybAvDNg6japmq7Bu/D6+aXkaXdE/M1sY0zen8HHkD6dkCS7XW4Uqp94EgrfVm4AWlVD8gG7hCIQ5zFcLa5ORqvgw6y/9+OkZCciYB9SrzweMt6Nykhpx8ZkG01szZP4eXt79My9ot2TxsM7XK1zK6rHumtC5Ze2MCAgJ0UFCQ0WUIUSwOnrnKu5vDCY1NpLV7FV7t0YTW9asYXZa4SXZuNi/+8CILghYwoOkAVg9cXSznFNwLpVSw1jrgbutJb5QQFighOYMZPxzlq+BYalZwZM4wX/r51JaWgQW6nnGdYV8P44eYH3i13avM6DYDG1Vyd+dJKAhhQbJzcln152lm/RxFWmYOT3X04PlHG8nRRBbqTOIZ+qztQ0R8BJ/3+ZzJLScbXdIDk0+aEBbizxOXeW9zOEcvJNGhUTXe7duChjUs56Qm8VdB54Lou64vqVmp/DDyB7o16GZ0SUVCQkEIg11ITOdf2yLZfPgcrpXK8NmolvRoUVN2FVmwTUc3MWLjCGqUq8GO0TtoUaOF0SUVGQkFIQySmZ3L0n0n+XRnNNm5mhe6NGJKxwaUcZBzDSyV1ppZf8xi2s/TaO3amu+GfUdN55pGl1WkJBSEMMCeqHje2xzOiYQUujaryfQ+zalb1bKOVhF/lZWTxfM/PM/nwZ8zuPlgVj6+kjL2pW9MKQkFIYrR2SupfLg1gu3hF3GvWpZlY1vRuWkNo8sSdxF3PY5x343j5xM/83r71/moy0cl+gijO5FQEKIYZGbnsnD3cRbsjsFGKab1aMLEDvVlWAoLl5WTxaf7P+W9X98jKyeLJX2XMMF/gtFlmZWEghBmdjk5gylrQjhw8gq9vWvxVq9m1K5U+nY7lDZ7Tu/hma3PEB4fTu9Gvfn0sU/xqOxhdFlmJ6EghBlFnr/OxBVBJCRnMGeYL/19XY0uSdzFheQLTPt5GqtDV1OvYj02Dd1Evyb9rOZoMAkFIczkxyMXePnLQ5R3suPLp9rhU6eS0SWJO8jOzWZh4ELe3vU26dnpvNXhLd7s8KbFDVdhbhIKQhQxrTVzf4lh1s9R+NSpxOLRLalRwcnossQd/HH2D57Z9gyHLhyim0c35vWaR+OqjY0uyxASCkIUodTMbKZ9FcrWsPMM9HPlXwO9cLKXzmRLFZ8Sz+s7XmfpoaW4lnflqyFfMajZIKvZVXQrEgpCFJG4a2lMWhFE5IXrvNmrKZM6eFj1l4sly8nNYUnIEt7Y+QZJmUlMe2ga0ztOt6i5ko0ioSBEEQg8dYWnVwXnnaU8Rs49sGRB54KYsnUKQeeC6OTeiXmPzStVw1Q8KAkFIR7QhsAzvL3pCG6Vy7L4yQAZxM5CXUm7wls73+Lz4M+p6VyTNQPXMNxzuLTmbiKhIMR9ys7J5aNtkSzbd4oOjaoxb7g/FcvaG12WuMmF5AssDFzI/MD5XEu/xottXuS9Tu9R0ami0aVZJAkFIe7DtdRMnlt7kN9iEhjfvj5v9moq8yRbmOBzwczZP4f1R9aTnZtN78a9+bDzh/i4+BhdmkWTUBDiHsVcSmLiiiDirqXx30HePNGqjtElCZPs3Gy+O/ods/fP5rczv+Hs4MyUgCk83+Z5GlZpaHR5JYKEghD34JejF3lh3SGc7G1YN6ktAe4yX7IluJZ+jS9CvmDugbmcTjyNeyV3ZnWfxXi/8bKb6B5ZTSgcTTjKtuhtvNzuZaNLEfchJSObLYfP0aOFC5XLORT78+fmahbtPcGMH4/SvFYFFj0ZgKuMX2S4qMtRfLr/U5YfWk5KVgod63Vkds/Z9G3cF1sbOT/kflhNKGyN2sqrP79KjwY95PCzEmjOzmgW7TnBh1sjGf9wfSZ2qE8FJ/N36ubman6KuMicndFEnr9Ob+9afDzYRybCMZDWmp0ndzL7z9lsjd6Kg60DI7xG8GKbF/F18TW6vBJPaa2NruGeBAQE6KCgoHu+X0JqAq6zXJkSMIXZPWeboTJhLhevp/PIf3fxcMNqONrbsC3sAhXL2PNURw/GPuROWYei/21zcxjUr1aO5x9tyAA/VzmE0SBpWWmsDl3NnP1zCI8Pp0a5GjwT8AxPBzxd6mY/MwelVLDWOuBu61lNS6Fa2WoMaDqAlYdX8p+u/8HJTsaiKSnm/hJNTq7m3b4tqFu1LEfiEpn1cxT//fEYS387yZRODRnZpm6RDCdxqzD4ZKgPfb1ry9FFBolPiWfegXnMD5zP5bTL+Lr4suLxFQxtMRRHO0ejyyt1rCYUACY9BmHXAAAdg0lEQVT5T2JD+AY2RmxkpPdIo8sRhXDmcirrD5xlaKs6+dNVerpWZOnYVgSfvsqsn4/xwfcRLN5zgucebcgTAXVwsLv3L++8MLjA7B3RHL2QJGFgAY5fOc7//vgfyw4tIz07nX5N+vFKu1foULeDtNbMyGp2HwHk6lwaz22MawVXfh37axFXJszh5S8PsTX0PL9O64xLxVu37n4/nsD/fooi+PRV6lQpw4tdGvO4b+G+zG8OA49q5Xi+S0MJAwMFxgUy8/eZbIzciJ2NHU96P8krD71C02pNjS6tRJPdR7dgo2yY6D+RN3a+wbGEYzSp1sToksQdRF9MYtPBOCY8XP+2gQDwUINqtHu6Kruj4vnfT8d49avDLNgdw9SujentVQsbm7//qrxVGEjLwDhaa36M+ZGZv89k16ldVHSsyGsPvcYLbV6gVvlaRpdnVayqpQB5p7zX+aQOL7Z5kY+7f1yElYmiNmV1MHui4tn7j0epUsjDULXWbA+/yKyfjxF1MZmmLuV5pXsTujargVLqlmHwQpdG9PWpje0twkOYV2ZOJuuPrOfj3z8m7FIYbhXcmNp2KpP8J1HesbzR5ZUq0lK4DRdnF/o16ceKwyv46NGPpKPKQoXFJvLDkQu80KVRoQMBQClFT08XujWvyfeh5/jk5ygmrQzCx60iA/xcWR94Nj8MZg/1lTAwSFJGEotDFvPJn58Qez0WzxqerHx8JUM9h+JgW/znoYj/Z3WhADDZfzLfRH7DpqObGOo51OhyxC18/NMxKpW1Z2KH+vd1f1sbRX9fV3p71eKbkDjm7IzmvS0REgYGO590nk/3f8rCoIUkZiTSyb0Ti/osomfDntJ5bCGsMhS6NehGvYr1WByyWELBAgWeusKvUfG8/ljTBz5Bzc7Whida1aG/X22iLiTTvHYFCQMDhF4M5dP9n7IqdBXZudkMajaIaQ9No5VrK6NLEzexylC40eH8zq53OH7lOA2qNDC6JGGitWbmj8eoXt6RMe3ci+xxHe1s8XKTMXCKU1ZOFt8e/ZZ5B+ax98xenOycmOA3gZfbvSyD01kwqz3MYpzvOGyUDUtClhhdiihgT3QCB05d4flHG8pQEiXUuaRzvLf7PerNrsfQr4cSlxTHx90+Ju7lOBb0XiCBYOHMGgpKqZ5KqWNKqRil1Ot3WG+QUkorpe7aM15UXCu40qdxH5YdWkZWTlZxPa24A601H28/hmulMgxrVdfocsQ90Fqz9/Rehn49lHqz6/H+r+/j6+LL1hFbiX4+mlceeoUqZWRE2ZLAbLuPlFK2wHygGxALBCqlNmutI25arzzwIrDfXLXcziT/SWw+tpktUVsY2GxgcT+9uMn28AuExSUyc7D3fZ2VLIpfSmYKa8LWMO/APMIuhVHJqRIvtnmRKQFTZLdsCWXOPoXWQIzW+gSAUmo90B+IuGm9D4AZwDQz1nJLPRv2xK2CG4tDFksoGCwnV/O/n6LwqF6OAX6uRpcj7iL6cjQLAhew7NAyEjMS8XXxZUnfJQz3Gk5Z+7JGlycegDlDwRU4W+B6LNCm4ApKKX+gjtZ6q1LqtqGglJoMTAaoW7fodivY2dgx3nc8H+z5gFPXTuFeyb3IHlvcm+8OxRF9KZn5I/zljGILlZObw7bobcwLnMdPx3/C3saewc0H82yrZ3mozkNySGkpYdj/PqWUDTALeOVu62qtF2mtA7TWAdWrVy/SOsb7jQdg6cGlRfq4ovAys3OZvSOa5rUq8Jini9HliJukZ6ezIHABDec2pN/6foRfCuf9Tu9zZuoZ1g5aS/u67SUQShFzthTigIKT17qZlt1QHvAEdps+UC7AZqVUP631/Y9jcY/qVapHz4Y9+eLgF0zvOB07G6s8StdQXwad5cyVVJaNbXXLcYqEMZIzk/ks6DP+98f/uJB8gXZu7ZjZbSb9m/TH3tb8ExwJY5izpRAINFJK1VdKOQDDgM03btRaJ2qtq2mt3bXW7sCfQLEGwg2T/CdxLukcP0T/UNxPbfXSs3KY+0s0LetVplOTom0FivtzLf0aH/z6AfVm12Paz9NoUb0Fu8bsYt/4fQxuPlgCoZQz289irXW2Uuo5YDtgCyzVWocrpd4HgrTWm+/8CMWnT+M+uDi7sChkEX2b9DW6HKuy6o/TXLyewZxhfrILwmDxKfF88ucnzDswj6TMJPo27subHd6krVtbo0sTxcis+0q01tuAbTctm36bdTuZs5Y7sbe1Z5zvOGbsm0Hs9VjcKrgZVYpVSc7IZuGvx+nQqBptPaoaXY7Vir0ey8e/f8yi4EWkZ6czpMUQ3nz4TXxcfIwuTRhADvMwmeg/kVydKx3OxWjpbye5kpLJq91lXgsjnLh6gqe2PIXHHA/mHZjHEy2eIOLZCDYM3iCBYMWkV9XEo7IHXT268sXBL3irw1vY2sgQC+Z0LTWTxXtO0L15TXzqVDK6HKsSGR/Jv3/7N2vD1mJrY8sEvwm81v416le+vxFpRekiLYUCJvtP5kziGX46/pPRpZR6n/16guTMbF6RVkKxOXj+IEO+GkKLBS3YGLmRF9u8yMkXT7Kwz0IJBJFPWgoF9G/an+plq7M4ZDGPNXrM6HJKrUvX01n++0n6+9SmiYvMrmVOqVmpfBX+FYtCFvH72d+p4FiBNzu8yUttX6Ja2WpGlycskIRCAQ62Doz1Hcsnf37C+aTzMjesmczfFUNWjualro2NLqXUOnThEIuDF7MmbA2JGYk0rtqYmd1mMtF/IpWcZHeduD0JhZtM9J/IzN9nsvzQct7o8IbR5ZQ6Z6+ksvbAGZ4IqIN7tXJGl1OqJGUksf7IehaHLCbwXCCOto4MaTGESf6T6FC3gxzyKwpFQuEmjas2ppN7J5YcXMI/Hv4HNkq6XYrSpzujUUrxQhcZU78oaK0JOhfE4pDFrDuyjuTMZFpUb8GcnnMY5T1KhqsW90xC4RYm+U9i5Dcj+eXkL3T16Gp0OaVGzKVkNobEMq59fWpVLGN0OSVaYnoia8LWsDhkMYcuHKKsfVmGthjKJP9JtHVrK60Ccd8kFG5hYLOBVClThcUhiyUUitAnO6JwsrdlSicZZ/9+aK35I/YPFocsZsORDaRlp+Hr4suCXgsY4TWCik4y3ah4cBIKt+Bk58ST3k8yP3A+8SnxVC8nY/I8CK01mw+fY2voeZ5/tCHVnB2NLqlEibsex9qwtaw4vILw+HCcHZwZ7T2aSS0n0bJWS2kViCIloXAbk1pOYvb+2aw4vIJXH3rV6HJKrBPxyby7OZy90Ql4uVZkYgcPo0sqEZIykvgm8htWha7il5O/oNG0dWvL4r6LGeY5DGcHZ6NLFKWU0lobXcM9CQgI0EFBxTOQ6sNLHyY+NZ6jzx6VX2P3KC0zhwW7Y/j81xM42tvwavcmjGpbD1sZGvu2snOz+fn4z6wKXcWmo5tIy07Do7IHo71HM8p7lEx4Lx6IUipYax1wt/WkpXAHk/wnMfa7sew5vYeO7h2NLqfE2BFxkfe2hBN7NY2Bfq680asZ1cvLLqNb0Vpz8MJBVh1exboj67iYcpHKTpUZ4zOG0T6jaefWTn6QiGIloXAHQ1oM4cUfX2RRyCIJhUI4eyWVf24JZ0fkJRrVcGb95LYy+ultnEk8w5rQNawOW01EfAQOtg70adyH0d6jeazhYzjaSYgKY0go3EFZ+7KM8h7FkpAlfNrzU6qWlS+4W8nIzmHxnhPM/SUGWxvFm72aMq59fexlruW/uJ5xna8jvmZV6Cp+PfUrGk37Ou35rPdnDGkxRM4pEBZBQuEuJvlPYn7gfFaFruKlti8ZXY7F2Rsdz7vfhXMiIYVeXi6806e5nINQQFZOFtuPb2d16Gq+O/Yd6dnpNKrSiH92+icjvUfiUVk63oVlkVC4Cx8XH1q7tmZxyGJebPOi7N81uZCYzgdbI9gaeh73qmVZMb41HRvLobuQ109wIO4Aq0NXsz58PQmpCVQtU5UJfhMY7T2a1q6t5XMkLJaEQiFM9p/MxC0T+SP2Dx6q85DR5RgqKyeX5ftOMXtHFNm5mpe7NWbyIx442cv8EzFXYvL7CWKuxOBk50T/Jv0Z5T2KHg16yNzGokSQUCiEoZ5DeWn7SywKXmSVoaC1JjtXE3L6KtO/C+fYxSQebVqD9/q2oG7VskaXZ6iE1AS+DP+S1aGr+SP2DxSKzvU78+bDbzKw2UA5y1iUOBIKheDs4MxIr5GsOLyCZ1s9SyvXVkaXdM/+PHGZL347SVpmDlk5uWTl5JKdq8nK0WSbrmflaLJz8/7Nyskl+8a/uf9/LotrpTIsGt2Sbs1rWu0ukLSsNL6P+p7VYavZFr2N7NxsPGt4MqPrDEZ4jZA5vkWJJievFVLs9Vg6Lu9IQmoCP4z8ocS0GLTWLNpzgv9uP0bVcg64VS6Dna0N9rYKe1sb7GwK/G2rsLexwd5O3bTcBnsbRaVyDgzyd6Wsg/X9lsjOzWbv6b2sCVvDVxFfcT3jOrXL12ak10hGeY/Cu6a30SUKcUdy8loRc6vgxq9jf+XRFY/SfVV3to3cxiP1HjG6rDu6np7FtK8Osz38Ir28XJgxyJvyTrJfu7DiU+LZfnw7W6O3sj1mO1fTr+Ls4Mzg5oMZ5TWKTu6dZC5vUepYTSj8GhXPd4fi6NqsJh0aVbuvL8cbwdBlZRd6ru7JluFb6OLRxQzVPrhjF5J4enUwZ66k8nbvZkx4uL7V7u4prFydy8HzB9kWvY1tMdvYH7sfjaZmuZr0b9qf3o1606tRL8raW3c/iijdrCYUzl9LY2fkJb4JicPeVtHWoypdmtagS7Oa1KlS+P/ktcrXYvfY3XRd2ZU+6/rw7dBv6dmwpxkrv3ffHYrj9Y1hODvZsW5SW1rXl5OibicxPZEdJ3awNXorP8T8wIXkCygUrV1b816n9+jVqBf+tfxlsiVhNayqTyE7J5fg01fZefQSOyIvciI+BYCmLuXp0iwvIHzdKmFTiEHbLqdeptuqboTHh/P1kK/p26TvfdVUlDKzc/loawQr/jhNa/cqzBvhR40KTkaXZVG01hxNOMrW6K1si97G3jN7yc7NppJTJXo06EHvRr3p0bAHNcrVMLpUIYpUYfsUrCoUbnYyIYWdkRfZEXmRwFNXycnVVHN2oHOTvIDo0Kga5Rxv35i6mnaVHqt7cPDCQTYM3sDAZgOLpK77cT4xjWfWhHDwzDUmdajPaz2byjATJjcmp1kbtpat0Vs5de0UAN41venVsBe9GvWiXZ122NlYTcNZWCEJhXuUmJrF7qhL7Ii8xO5jl0hKz8bBzoaHGlSlS7OadGlag9qV/j58Q2J6Ir3W9mJ/7H5WD1zNMM9hRV7b3eyLSeD5dQfJyMph5hAfennVKvYaLNGF5AusOryKpYeWcjThKGXty9LVoyu9G/XmsYaPUadiHaNLFKLYSCg8gKycXAJPXWFn5CV2Rl7k1OVUAHzrVOLdvs3xq1v5L+snZSTRZ10ffjvzG8v6L+NJnyfNWt8Nubmahb8e538/HaNBdWc+G92SBtWte/KVrJwstkVvY+mhpWyN2kqOzuHhug8z3nc8Q1oMkclphNWSUCgiWmuOx+ftZlr++ykuXk9nwsP1eblbE8o4/P/hiCmZKfRf359fTv7C4r6LmeA/wax1JaZl8cqXh9kReZG+PrX5z0CvO+7qKu0i4iNYdnAZK0NXcinlEi7OLozxGcM433E0qdbE6PKEMJyEghkkpWfxnx+Osmb/GdyrluU/g7z/Ml9AWlYaA78cyI8xP7Kg1wKmtJpiljoizl1nyppg4q6m8VbvZox9yN0qDze9nnGdDUc2sPTQUv6M/RM7Gzv6Nu7LeL/x9GzYU/oIhChAQsGM/jh+mX9sDOXMlVRGta3L6481w9n0Kz0jO4MhXw1hS9QWZveYzYttXyzS594YHMtbm8KoWMae+SP8CXC3rsNNtdbsOb2HpYeW8lX4V6Rlp9G8enMm+E1glPcoOWpIiNuQUDCz1Mxs/vdTFEv3naRWBSf+NdCLTk3yvpAyczIZsXEEGyM3MqPrDF5r/9oDP19yRjb/3hbJmv1naOtRhbnD/a1mikutNeHx4Ww6uonlh5Zz/OpxKjhWYLjncMb7jadV7VZW2VIS4l5YRCgopXoCcwBbYInW+j833f408CyQAyQDk7XWEXd6TEsJhRtCzlzlta9DibmUzCB/N97p04xKZR3Izs1m9LejWX9kPe93ep93Or5zz4+dk6v5/XgCG4Nj+TH8AulZuTzdsQGvdm+MXSk/3PRK2hV2nNjB9pjtbD++nbikOAA6u3dmvN94BjYbKGcWC3EPDA8FpZQtEAV0A2KBQGB4wS99pVQFrfV109/9gGe01nc8PdjSQgHypqOcuzOGhb8ep0o5Bz7o70lPTxdycnMYv3k8Kw+v5O0Ob/N+5/cL9Ys2+mISG0Pi2HQwjgvX06ngZEdfn9oMCaiDb51KxfCKil9Obg6B5wL5MeZHth/fzoG4A+TqXCo5VaKrR1d6NuhJj4Y9ZARSIe6TJQyI1xqI0VqfMBW0HugP5IfCjUAwKQeUrH1ZJo52trzaowmPebnw2tehPL06mN5etXivXwuW9V+Gg40DH+79kOTMZKZ3nE7lMpX/9hhXUjLZfCiOjSFxhMUlYmuj6NS4OtP7NufRpjVK5SQ2cdfj2H58Oz/G/MiOEzu4mn41f4iJtzu8TY+GPWjt2lo6jIUoRub83+YKnC1wPRZoc/NKSqlngZcBB+DRWz2QUmoyMBmgbt26RV5oUWlRuyKbnm3Poj0nmLMjmn3HE3ivbws+6/MZjnaOzN4/mwVBC+jVqBcjvUbSzeMx/ohJYmNILLuOXiI7V9OidgWm92lOP9/aVHMuXX0G6dnp7D29Nz8IwuPDAajlXIvHmz5OjwY96OrRlaplq97lkYQQ5mLO3UeDgZ5a64mm66OBNlrr526z/gigh9Z6zJ0e1xJ3H91KzKUkXvs6lJAz13i0aQ0+fLwF51IjWB26mjWh60lIu4gNZSmT/RCuDt0Y7d+LwS3r0tSlgtGlF4nkzGTCLoZx+OJhDl04xKELhzh88TDp2ek42DrQoW4HejToQc+GPfGs4SkdxUKYmSX0KbQD3tNa9zBdfwNAa/3v26xvA1zVWt9x/sKSEgqQ11G8/PdTzNx+FHsbGwa1dGNvdDwx8dfJsQ+jfJX9nEndRWp2MrXL12ZYi2GM9B6Jn4tfifmS1FpzLunc3778oy9Ho017Ays6VsTXxRc/Fz+6enSlk3snyjmUM7hyIayLJYSCHXkdzV2AOPI6mkdorcMLrNNIax1t+rsv8O7dii5JoXDD6cspvL4xjD9OXKa1exUGtXTlMa9aVHCyJy0rjS1RW1gbtpZt0dvIys2iabWmjPQayQivEXhU9jC6/HxZOVkcu3zsL1/+hy4cIiE1IX8dj8oe+NT0wdfFN//fuhXrlpiQE6K0MjwUTEX0AmaTd0jqUq31R0qp94EgrfVmpdQcoCuQBVwFnisYGrdSEkMB8n5Rp2Tm5J/kditX0q7wdcTXrAlbw57TewBo59aOkV4jeaLFE1QvV93sdSZlJHHy2klOXD3xt8vJayfJzMkEwNHWEc8anvi6+OYHgHdNb5moXggLZRGhYA4lNRTu1ZnEM6wLW8easDWEXQrDVtnSoV4HXJxdqOBQgQqOhbuUcyj3lwlisnOzib0em/clf9X05X/t/7/4C/7qh7xdPx6VPfCo7EGDyg3wcfHBp6YPTao1kaOChChBJBRKkbCLYawNW8vOkztJzEjkesZ1rmdcJzUr9a73Vaj8gLC1sSX2eizZudn5t9vZ2FGvYj08KntQv1L9/AC4cbnV4bNCiJLHEs5TEEXEq6YX/6759/757NxskjKS/hIUd7pk5mRSt2Ldv3zpu1Vwk1/8Qoh88m1QgtnZ2FG5TGX5NS+EKDKlewAdIYQQ90RCQQghRD4JBSGEEPkkFIQQQuSTUBBCCJFPQkEIIUQ+CQUhhBD5JBSEEELkK3HDXCil4oHTRtdhsGpAwl3XKt1kG8g2uEG2Q+G2QT2t9V1H1SxxoSBAKRVUmDFMSjPZBrINbpDtULTbQHYfCSGEyCehIIQQIp+EQsm0yOgCLIBsA9kGN8h2KMJtIH0KQggh8klLQQghRD4JBQujlKqjlNqllIpQSoUrpV40La+ilPpZKRVt+reyablSSn2qlIpRSoUqpfyNfQVFRyllq5Q6qJT63nS9vlJqv+m1blBKOZiWO5qux5hudzey7qKklKqklPpaKXVUKRWplGpnbZ8FpdRU0/+FI0qpdUopJ2v4LCilliqlLimljhRYds/vvVJqjGn9aKXUmLs9r4SC5ckGXtFaNwfaAs8qpZoDrwM7tdaNgJ2m6wCPAY1Ml8nAwuIv2WxeBCILXJ8BfKK1bghcBSaYlk8ArpqWf2Jar7SYA/yotW4K+JC3Pazms6CUcgVeAAK01p6ALTAM6/gsLAd63rTsnt57pVQV4F2gDdAaePdGkNyW1louFnwBvgO6AceAWqZltYBjpr8/B4YXWD9/vZJ8AdxMH/pHge8BRd7JOXam29sB201/bwfamf62M62njH4NRbANKgInb34t1vRZAFyBs0AV03v7PdDDWj4LgDtw5H7fe2A48HmB5X9Z71YXaSlYMFPT1w/YD9TUWp833XQBqGn6+8Z/mhtiTctKutnAa0Cu6XpV4JrWOtt0veDrzN8GptsTTeuXdPWBeGCZaTfaEqVUOazos6C1jgM+Bs4A58l7b4Oxvs/CDff63t/zZ0JCwUIppZyBjcBLWuvrBW/TeZFfag8bU0r1AS5prYONrsVgdoA/sFBr7Qek8P+7CwCr+CxUBvqTF5C1gXL8fZeKVTLXey+hYIGUUvbkBcIarfU3psUXlVK1TLfXAi6ZlscBdQrc3c20rCRrD/RTSp0C1pO3C2kOUEkpZWdap+DrzN8GptsrApeLs2AziQVitdb7Tde/Ji8krOmz0BU4qbWO11pnAd+Q9/mwts/CDff63t/zZ0JCwcIopRTwBRCptZ5V4KbNwI0jB8aQ19dwY/mTpqMP2gKJBZqXJZLW+g2ttZvW2p28TsVftNYjgV3AYNNqN2+DG9tmsGn9Ev/rWWt9ATirlGpiWtQFiMCKPgvk7TZqq5Qqa/q/cWMbWNVnoYB7fe+3A92VUpVNra7upmW3Z3RHilz+1rH0MHlNwlDgkOnSi7z9ojuBaGAHUMW0vgLmA8eBMPKO0jD8dRTh9ugEfG/62wM4AMQAXwGOpuVOpusxpts9jK67CF+/LxBk+jxsAipb22cB+CdwFDgCrAIcreGzAKwjrx8li7xW44T7ee+B8abtEQOMu9vzyhnNQggh8snuIyGEEPkkFIQQQuSTUBBCCJFPQkEIIUQ+CQUhhBD5JBSEuA2l1Fum0TlDlVKHlFJtlFIvKaXKGl2bEOYih6QKcQtKqXbALKCT1jpDKVUNcAB+J+8Y8ARDCxTCTKSlIMSt1QIStNYZAKYQGEze+Du7lFK7AJRS3ZVSfyilQpRSX5nGrEIpdUop9V+lVJhS6oBSqqFp+RDTvACHlVJ7jHlpQtyetBSEuAXTl/tvQFnyzhzdoLX+1TQeU4DWOsHUevgGeExrnaKU+gd5Z9a+b1pvsdb6I6XUk8ATWus+SqkwoKfWOk4pVUlrfc2QFyjEbUhLQYhb0FonAy3Jm7AkHtiglBp702ptgebAPqXUIfLGoqlX4PZ1Bf5tZ/p7H7BcKTWJvAljhLAodndfRQjrpLXOAXYDu02/8G+eylABP2uth9/uIW7+W2v9tFKqDdAbCFZKtdRal6ZRPEUJJy0FIW5BKdVEKdWowCJf4DSQBJQ3LfsTaF+gv6CcUqpxgfsMLfDvH6Z1Gmit92utp5PXAik4rLEQhpOWghC35gzMVUpVIm/e7BjydiUNB35USp3TWnc27VJap5RyNN3vbSDK9HdlpVQokGG6H8BMU9go8ka7PFwsr0aIQpKOZiHMoGCHtNG1CHEvZPeREEKIfNJSEEIIkU9aCkIIIfJJKAghhMgnoSCEECKfhIIQQoh8EgpCCCHySSgIIYTI938r33HzSrBr9gAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "smaller_val_loss = np.array(smaller_model.get_validation_summary(\"Loss\"))\n",
+    "\n",
+    "plt.plot(original_val_loss[:,0], original_val_loss[:,1], label='original model')\n",
+    "plt.plot(smaller_val_loss[:,0], smaller_val_loss[:,1],label='smaller model',color='green')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the smaller network starts overfitting later than the reference one (the original one starts overfitting at about 150 to 200 steps, which is about 3 or 4 epochs) and its performance \n",
+    "degrades much more slowly once it starts overfitting.\n",
+    "\n",
+    "Now, for kicks, let's add to this benchmark a network that has much more capacity, far more than the problem would warrant:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "bigger_model = models.Sequential()\n",
+    "bigger_model.add(layers.Dense(512, activation='relu', input_shape=(10000,)))\n",
+    "bigger_model.add(layers.Dense(512, activation='relu'))\n",
+    "bigger_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "bigger_model.compile(optimizer='rmsprop',\n",
+    "                     loss='binary_crossentropy',\n",
+    "                     metrics=['acc'])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dir_name = '4-4 ' + str(time.ctime())\n",
+    "bigger_model.set_tensorboard('./', dir_name)\n",
+    "bigger_model.fit(x_train, y_train,\n",
+    "                 nb_epoch=20,\n",
+    "                 batch_size=512,\n",
+    "                 validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here's how the bigger network fares compared to the reference one. The dots are the validation loss values of the bigger network, and the \n",
+    "crosses are the initial network."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd8k+X+//HX1b3TPeiggFDEli0UQcAFKsM9QFBQXEdUnOc4cZ7z8+gX9aioHEQUUFEBJyJyHICDXYZskZHSlra0TXeb9vr9kaS2pUBH0qTt5/l45EFy5859X0nD/c5139dQWmuEEEIIGzdnF0AIIYRrkWAQQghRhwSDEEKIOiQYhBBC1CHBIIQQog4JBiGEEHVIMAghhKhDgkEIIUQdEgxCCCHq8HB2AZoqPDxcJyYmOrsYQgjRpmzatClHax3RmHXbXDAkJiayceNGZxdDCCHaFKXUocauK6eShBBC1CHBIIQQog4JBiGEEHW0uWsMDamsrMRoNFJWVubsoojT8PHxIS4uDk9PT2cXRQhxEu0iGIxGI4GBgSQmJqKUcnZxxElorcnNzcVoNNKlSxdnF0cIcRLt4lRSWVkZYWFhEgouTilFWFiY1OyEcHHtIhgACYU2Qv5OQri+dhMMQgjXtnjHYnJKcpxdDNEIDgsGpVS8UuoHpdROpdTvSql7G1hnpFKqQCmVZr096ajyuIpLL72U/Pz8U67z5JNPsmrVqmZt/8cff2Ts2LHNem1jHTx4kOTk5BavIzqOzKJMrl9yPfO2zHN2UUQjOPLisxl4QGu9WSkVCGxSSn2ntd5Zb701WmvHHslcgNYarTXLly8/7brPPPNMK5RIiNZzpOAIAIfyG935VjiRw2oMWusMrfVm6/1CYBcQ66j9OdusWbNITk4mOTmZV155BbD8ak5KSuLGG28kOTmZI0eOkJiYSE6OpTr97LPPkpSUxLBhw5gwYQIvvfQSAFOmTOHTTz8FLEOAzJw5k/79+5OSksLu3bsBWL9+PUOGDKFfv36cc8457Nmz55Tlmz9/PpdffjkXXXQRiYmJvP7668yaNYt+/fqRmprK8ePHAUhLSyM1NZXevXtzxRVXkJeXB8CmTZvo06cPffr04Y033qjZblVVFQ899BBnn302vXv35u2337bjpyraC6PJCMBh02Enl0Q0Rqs0V1VKJQL9gHUNPD1EKbUVOAo8qLX+vYHX3wbcBpCQkHDKfT395e/sPGpqYYnr6tUpiJnjzjrp85s2beLdd99l3bp1aK0ZPHgwI0aMICQkhH379vHee++Rmppa5zUbNmxgyZIlbN26lcrKSvr378+AAQMa3H54eDibN29m9uzZvPTSS8ydO5eePXuyZs0aPDw8WLVqFY8++ihLliw55fvYsWMHW7ZsoaysjDPOOIMXXniBLVu2cN999/H+++8zY8YMbrzxRl577TVGjBjBk08+ydNPP80rr7zC1KlTef311xk+fDgPPfRQzTbfeecdDAYDGzZsoLy8nKFDhzJq1Ci5yCzqqAmGAgmGtsDhF5+VUgHAEmCG1rr+EXsz0Flr3Qd4DfisoW1oredorQdqrQdGRDRqcMBWtXbtWq644gr8/f0JCAjgyiuvZM2aNQB07tz5hFAA+Pnnn7nsssvw8fEhMDCQcePGnXT7V155JQADBgzg4MGDABQUFHDNNdeQnJzMfffdx++/n5CnJzjvvPMIDAwkIiICg8FQs8+UlBQOHjxIQUEB+fn5jBgxAoCbbrqJ1atXk5+fT35+PsOHDwdg8uTJNdtcuXIl77//Pn379mXw4MHk5uayb9++RnxqoiOxBYPtlJJwbQ6tMSilPLGEwiKt9dL6z9cOCq31cqXUbKVUuNa62U0XTvXL3hn8/f1bvA1vb28A3N3dMZvNADzxxBOcd955LFu2jIMHDzJy5MhGbwfAzc2t5rGbm1vNdptKa81rr73G6NGj6yy3BZgQAOmF6QDkleVRWF5IoHegk0skTsWRrZIU8A6wS2s96yTrRFvXQyk1yFqeXEeVyVHOPfdcPvvsM0pKSiguLmbZsmWce+65p3zN0KFD+fLLLykrK6OoqIivvvqqSfssKCggNtZyyWb+/PnNLXodBoOBkJCQmtrOggULGDFiBMHBwQQHB7N27VoAFi1aVPOa0aNH8+abb1JZWQnA3r17KS4utkt5RPthqzEAHDFJrcHVObLGMBSYDGxXSqVZlz0KJABord8CrgbuVEqZgVLgeq21dmCZHKJ///5MmTKFQYMGATBt2jT69et3yl/NZ599NuPHj6d3795ERUWRkpKCwWBo9D4ffvhhbrrpJp577jnGjBnT0rdQ47333uOOO+6gpKSErl278u677wLw7rvvcvPNN6OUYtSoUTXrT5s2jYMHD9K/f3+01kRERPDZZw2eERQdmNFkJMGQwOGCwxwpOEKviF7OLpI4BdXWjsMDBw7U9Sfq2bVrF2eeeaaTStR8RUVFBAQEUFJSwvDhw5kzZw79+/d3drEcrq3+vUTzaK3xfd6Xy3tezuLfFzNn7BxuHXCrs4vV4SilNmmtBzZmXen57ES33XYbffv2pX///lx11VUdIhREx5Nbmkt5VTmDYgfhptzkVFIb0C5GV22rPvjgA2cXQQiHs11fSAxOpFNgJ2my2gZIjUEI4VC2YIgLiqu5ziBcmwSDEMKhagdDfFC8nEpqAyQYhBAOZTQZcVfuRPlHkWBI4EjBEap1tbOLJU5BgkEI4VBGk5FOgZ1wd3MnwZBAeVU52cXZzi6WOAUJBjs41RDT06ZNY+fO+gPKth21B/RryTqi40ovTCcuKA6A+KB4QDq5uToJBgebO3cuvXo5rjOP1prqaqmWC9dlNBlrgiHBYBkEUy5AuzYJBjsxm83ccMMNnHnmmVx99dWUlJQAMHLkSGwd8t555x169OjBoEGDuPXWW5k+fToAf/zxB6mpqaSkpPD4448TEBBQs90XX3yxZkjrmTNnAg0P511bYmIijzzyCH379mXgwIFs3ryZ0aNH061bN9566y3AEigPPfQQycnJpKSksHjx4prl06dPJykpiQsvvJBjx47VbHfTpk2MGDGCAQMGMHr0aDIyMhz0aYr2QmvNkYIjxAZahm+xBYMMpufa2l0/hhkrZpCWmXb6FZugb3RfXrn4lVOus2fPHt555x2GDh3KzTffzOzZs3nwwQdrnj969CjPPvssmzdvJjAwkPPPP58+ffoAcO+993LvvfcyYcKEmgM3WEYu3bdvH+vXr0drzfjx41m9ejUJCQknHc7bJiEhgbS0NO677z6mTJnCzz//TFlZGcnJydxxxx0sXbqUtLQ0tm7dSk5ODmeffTbDhw/n119/Zc+ePezcuZOsrCx69erFzTffTGVlJXfffTeff/45ERERLF68mMcee4x582RGLnFypnITxZXFNTWGUN9QfD18pcbg4qTGYCfx8fEMHToUgEmTJtUMOGezfv16RowYQWhoKJ6enlxzzTU1z/366681jydOnFizfOXKlaxcuZJ+/frRv39/du/eXTOk9cmG87YZP348YBlSe/DgwTXDbXt7e5Ofn8/atWuZMGEC7u7uREVFMWLECDZs2MDq1atrlnfq1Inzzz8fsATfjh07uOiii+jbty/PPfccRqPxpPsXAuo2VQVQSln6MsiEPS6t3dUYTvfL3lHqT0xjj4lqtNY88sgj3H777XWWHzx48LTDedceUrv+cNvNGWJba81ZZ53Fr7/+2uTXio6rfjAAxBvi5VSSi5Mag50cPny45qD5wQcfMGzYsDrPn3322fz000/k5eVhNpvrzLaWmppa8/ijjz6qWT569GjmzZtHUVERAOnp6XXO+bfEueeey+LFi6mqqiI7O5vVq1czaNAghg8fXrM8IyODH374AYCkpCSys7Nr3mNlZWWjJgcSHVtDwZAQJL2fXV27qzE4S1JSEm+88QY333wzvXr14s4776zzfGxsLI8++iiDBg0iNDSUnj171gyz/corrzBp0iSef/55Lr744prlo0aNYteuXQwZMgSAgIAAFi5ciLu7e4vLe8UVV/Drr7/Sp08flFL8+9//Jjo6miuuuILvv/+eXr16kZCQULNvLy8vPv30U+655x4KCgowm83MmDGDs85yrYmRhGsxmowoFDGBMTXLEgwJZBZlUlFVgZe7lxNLJ05Ka92mbgMGDND17dy584RlrqiwsFBrrXVlZaUeO3asXrp0qdZa6+LiYl1dXa211vrDDz/U48ePd1oZW0Nb+XuJlpv2+TQd/VJ0nWXvbH5H8xT6wPEDTipVxwRs1I08zkqNoRU99dRTrFq1irKyMkaNGsXll18OWJqBTp8+Ha01wcHB0tJHtBvGQmOd00hQty9Dl5AuziiWOA0Jhlb00ksvNbj83HPPZevWra1cGiEcL92UTrfQbnWWSe9n19duLj7rNjYTXUclf6eOxWgy1nRus4k3WIJBLkC7rnYRDD4+PuTm5spBx8VprcnNzcXHx8fZRRGtoLiimLyyvBNOJfl5+hHuFy7B4MLaxamkuLg4jEYj2dkyYqOr8/HxIS4u7vQrijYvvTAd4IRgAGReBhfXLoLB09OTLl3kIpYQrqShPgw2CYYE/sj7o7WLJBqpXZxKEkK4ntMFg5xKcl0SDEIIh7AFQ/2Lz2A5lWQqN1FQVtDaxRKNIMEghHAIo8lImG8Yvp6+JzxXM/y2XGdwSRIMQgiHqD1BT322JqsymJ5rkmAQQjjEqYJBZnJzbRIMQgiHqD3Xc30xATG4K3cJBhclwSCEsLtycznHio81eOEZwN3NndigWLnG4KIkGIQQdne08CjQcFNVG2my6rokGIQQdneqPgw2EgyuS4JBCGF3jQmG+KB4jCYj1bq6tYolGkmCQQhhd42tMVRWV5JVlNVaxRKNJMEghLA7o8lIkHcQgd6BJ11H5mVwXRIMQgi7a2jmtvqkL4PrclgwKKXilVI/KKV2KqV+V0rd28A6Sin1H6XUfqXUNqVUf0eVRwjRek7Vuc1GgsF1ObLGYAYe0Fr3AlKBu5RSveqtcwnQ3Xq7DXjTgeURQrQSo8lIXOCpgyHYJxh/T38ZFsMFOSwYtNYZWuvN1vuFwC6gfm+Xy4D3tcVvQLBSKsZRZRJCOJ652kxmUSaxQQ13brNRSlmarJqkxuBqWuUag1IqEegHrKv3VCxQ++eCkRPDQwjRhmQWZVKtq097KgmkL4OrcngwKKUCgCXADK21qZnbuE0ptVEptVGm7xTCtTWmqapNfFC8nEpyQQ4NBqWUJ5ZQWKS1XtrAKulAfK3HcdZldWit52itB2qtB0ZERDimsEIIu2hKMCQYEsgqzqLcXO7oYokmcGSrJAW8A+zSWs86yWpfADdaWyelAgVa6wxHlUkI4XhNqjFY52WwvUa4Bg8HbnsoMBnYrpRKsy57FEgA0Fq/BSwHLgX2AyXAVAeWRwjRCowmI74evoT4hJx23dpNVruFdnN00UQjOSwYtNZrAXWadTRwl6PKIIRofbY+DJaTBqcmfRlck/R8FkLYVWM6t9nY1pNhMVyLBIMQwq6aEgw+Hj5E+kdKjcHFSDAIIeymWlefckrPhkhfBtcjwSCEsJvs4mzM1eaTTunZkPigeDmV5GIkGIQQdtOUpqo2thqDpS2KcAUSDEIIu2lOMMQHxVNUUURBeYGjiiWaSIJBCGE3za0xgDRZdSUSDEIIuzGajHi6eRLh3/ihayQYXI8EgxDCboyFRmKDYnFTjT+02IbFkMH0XIcEgxDCbprSh8EmOiAaTzdPqTG4EAkGIYTdNCcY3JQbcUFxMmGPC5FgEELYhda6UVN6NiTeIPMyuBIJBiGEXRwvPU6ZuazJNQaQ3s+uRoJBCGEX6YWWObZON9dzQ+KD4kkvTKequsrexRLNIMEghLCL5vRhsEkwJGCuNpNZlGnvYolmkGAQQthFS4MBpC+Dq5BgEELYhdFkxE25ER0Q3eTXxgdZ+zLIYHouQYJBCGEXRpORmIAYPNyaPjGk1BhciwSDEMIumtOHwcbgYyDIO0iarLoICQYhhF20JBjAcjpJOrm5BgkGIYRdtDQYpC+D65BgEEK0mKncRGFFYYtrDHIqyTVIMAghWszWVLUpU3rWl2BIILskm9LKUnsVSzSTBIMQosXSTZZezy09lQTSZNUVSDAIIVqsJZ3bbGReBtchwSCEaDFbMHQK7NTsbUhfBtchwSCEaDGjyUikfyTeHt7N3obt+oScSnI+CQYhRIsZC1vWVBXA28Ob6IBoqTG4AAkGIUSLtbQPg430ZXANEgxCiBZr7sxt9cUHxcupJBcgwSCEaJGSyhKOlx63a41Ba22HkonmkmAQQrSIPfow2CQYEmqCRjiPBIMQokVqej03Y0rP+mReBtcgwSCEaBHbXM/2qjGA9GVwNgkGIUSL2GOcJBvp/ewaHBYMSql5SqljSqkdJ3l+pFKqQCmVZr096aiyCCEcx2gyEuITgr+Xf4u3FekfiZe7l9QYnKzpc/A13nzgdeD9U6yzRms91oFlEEI4mL36MAC4KTeZsMcFOKzGoLVeDUjTAiHaOXsGA1hOJ8mpJOdqVDAopboppbyt90cqpe5RSgXbYf9DlFJblVLfKKXOssP2hBCtzN7BIL2fna+xNYYlQJVS6gxgDhAPfNDCfW8GOmut+wCvAZ+dbEWl1G1KqY1KqY3Z2dkt3K0Qwl4qqirIKs6ybzAEJZBemI652my3bYqmaWwwVGutzcAVwGta64eAmJbsWGtt0loXWe8vBzyVUuEnWXeO1nqg1npgRERES3YrhLCjo4VHAfs0VbWJN8RTravJKMyw2zZF0zQ2GCqVUhOAm4CvrMs8W7JjpVS0UkpZ7w+yliW3JdsUQrQuezZVtZG+DM7X2FZJU4E7gOe11n8qpboAC071AqXUh8BIIFwpZQRmYg0TrfVbwNXAnUopM1AKXK9lgBQh2hR7zNxWn/R+dr5GBYPWeidwD4BSKgQI1Fq/cJrXTDjN869jac4qhGij7DlOko2tk5vUGJynsa2SflRKBSmlQrFcNP6vUmqWY4smhHB1RpORAK8AgryD7LbNIO8ggn2CJRicqLHXGAxaaxNwJfC+1nowcKHjiiWEaAtsM7dZLxfajczL4FyNDQYPpVQMcC1/XXwWQnRw9u7DYCN9GZyrscHwDPAt8IfWeoNSqiuwz3HFEkK0BRIM7VNjLz5/AnxS6/EB4CpHFUoI4frM1WYyCjPsMqVnffFB8RwvPU5xRbFdBucTTdPYi89xSqll1tFSjymlliil7P9tEEK0GVlFWVTpKofVGECarDpLY08lvQt8AXSy3r60LhNCdFCO6MNgI/MyOFdjgyFCa/2u1tpsvc0HZGwKITowe07pWZ/0fnauxgZDrlJqklLK3XqbhAxfIUSH5sgaQ2xgLAolweAkjQ2Gm7E0Vc0EMrAMZzHFQWUSQrQB6YXpeLt7E+YbZvdte7p7EhMYI9cYnKRRwaC1PqS1Hq+1jtBaR2qtL0daJQnRodmaqtq7c5uNNFl1npbM4Ha/3UohhGhzHNWHwUaCwXlaEgyO+ZkghGgTHB0MtmExZNDl1teSYJC/lhAdVLWuJr0w3eE1hjJzGTklOQ7bh2jYKXs+K6UKaTgAFODrkBIJIVxeTkkOFVUVDq8xgKWTW4S/tI5vTaesMWitA7XWQQ3cArXWjZ3kRwjRzjiyqaqN9GVwnpacShJCdFCOmNKzPgkG55FgEEI0WWvUGML9wvHx8JFhMZxAgkEI0WRGkxEPNw8i/SMdtg+lFPFB8Rw2SY2htUkwCCGaLL0wnU6BnXB3c3fofqQvg3NIMAghmszRfRhs4g3xcirJCSQYhBBN1lrBkBCUwNHCo1RWVTp8X+IvEgxCiCbRWluCwQEzt9UXb4hHozlaeNTh+xJ/kWAQQjRJflk+JZUlrVNjkCarTiHBIIRoktZoqmojweAcEgxCiCZpzWCoPSyGaD0SDEKIJnHklJ71+Xv5E+obKjWGVibBIIRoEqPJiEIRExDTKvuTvgytT4JBCNEkRpOR6IBoPN09W2V/tnkZROuRYBBCNImj52GoT2oMrU+CQQjRJK3Vuc0mPiie/LJ8CssLW22fHZ0EgxCiSVo7GGxNVuV0UuuRYBBCNFpheSEF5QVOCQY5ndR6JBiEEI2WXpgOtE4fBpt4g7Uvgwym12okGIQQjdaandtsOgV2wk25degaw6H8Q9zx1R18vvvzVtmfw+ZtVkrNA8YCx7TWyQ08r4BXgUuBEmCK1nqzo8ojhGi51pjSsz4PNw9iA2M75IQ9B/MP8s81/2R+2nwAuoV0a5X9OiwYgPnA68D7J3n+EqC79TYYeNP6rxDCRbVmr+fanD0vQ1V1FR///jGR/pGMTBzp8AmK/sz70xIIW+fjpty4tf+t/GPYP2pOqzmaw4JBa71aKZV4ilUuA97XWmvgN6VUsFIqRmud4agyCSFaxmgy1szF3JoSDAlsSN/Qqvu0OZR/iEnLJrH28FrAUluamDKRyb0nkxKVYtd9Hcg7wPOrn+f9be/jrty5Y8Ad/H3Y31v11B04tsZwOrFA7Z8ARusyCQYhXFRrN1W1iQ+KZ9muZVTratxU610a/XD7h9z59Z1U62rmXzYfX09fFmxbwMu/vcyLv7xIn6g+TO49mQkpE+gU2KnZ+/nj+B88v+Z53t/6Ph5uHtw58E7+PvTvrV4zs3FmMDSaUuo24DaAhIQEJ5dGiI6rtXs92yQYEiivKie7OJuogCiH789UbuKu5XexcNtChsQNYdGVi+gS0gWAa8+6luzibBb/vpgF2xbw4HcP8vCqh7mgywVM7j2ZK868ggCvgEbtZ//x/Ty3+jkWbluIp7sn0wdN5+GhD7coZOzBmcGQDtQ+YRZnXXYCrfUcYA7AwIEDteOLJoRoiNFkJDU2tdX3W7svg6OD4ZcjvzBp6SQOFRziqRFP8djwx/Bwq3uojPCPYPqg6UwfNJ09OXtYuG0hC7cv5MbPbsTvaz+uPPNKJqVM4oKuF5zwWoC9uXt5bvVzLNq+CG93b+4ZfA8PnfMQMYGtMzDh6TizueoXwI3KIhUokOsLQriuMnMZOSU5TjuVBI7t/WyuNvP0j08z/N3haDRrpq5h5siZDR7Ya0sKT+LZ85/lj3v+YM3UNUxKmcRXe7/i4kUXE/9yPA98+wBpmWlordmTs4fJyyZz5htn8unOT5kxeAYH7j3ArNGzXCYUwLHNVT8ERgLhSikjMBPwBNBavwUsx9JUdT+W5qpTHVUWIUTLpZtav3ObTefgzgA89v1jHCk4wsSUiUT4R9ht+3/m/cmkZZP45cgvTO49mdcvfZ0g76AmbcNNuTEsYRjDEobx6iWvsnzfchZsW8Br619j1m+z6BbSjT/z/8THw4f7U+/nwXMebJXTYs2hLI2C2o6BAwfqjRs3OrsYQnQ4Px38iZHvjWTV5FVc0PWCVt//vC3zmL1hNpsyNuHh5sGY7mOY0ncKl3a/FC93r2Zvd+G2hfzt67+hlOKtMW8xIWWCHUsNuSW5fLLzEz7b/Rm9o3rz4DkPEukfadd9NIZSapPWemCj1pVgEKL9KzOX8cpvr5ASmcLoM0af9vRIQxZtW8SkZZPYfdduksKTHFDKxtlxbAfvpb3Hgm0LyCrOItwvnInJE5nSdwp9o/ti6Tt7egVlBfxt+d/4YPsHDEsYxoIrFpAYnOjYwjtRU4KhTbRKEkI0X1FFEZd9dBnf//k9ANEB0UzuPZkpfafQK6JXo7fjrM5t9SVHJvPiqBf514X/4tv93/Le1vd4a9Nb/Gf9f0iJTGFK3ynckHLDKU/TrD28lklLJ2E0GXn2vGf5x7B/NCss2ysZK0mIdux46XEufP9Cfjr4E/PGz2PZdcsYFDuIWb/O4qzZZzF47mDe3PAmeaV5p92W0WTE4G1odFNMR/Nw82BMjzF8fM3HZDyQwexLZ+Pr6csDKx8gdlYs4z4cx5KdSyg3l9e8prKqkid/eJIR80fg7ubOzzf/zOPDH5dQqEdOJQnRTmUWZTJqwSj25O5h8dWLubzn5TXPZRVlsWj7It5Ne5cdx3bg7e7N5T0vZ2rfqVzY9cIGh3y4YvEV7Mvdx46/7WjNt9Fku7J38d7W93h/6/tkFGUQ6hvKhOQJjOk+hqd/epp16eu4qc9NvHbJawR6Bzq7uK1GrjEI0cEdzD/Ihe9fSGZRJp9d/xkXdr2wwfW01mzO2Mz8tPl8sOMDjpceJzYwlhv73MiUvlPoEdajZt2z/3s2Yb5hrJi0orXeRouYq82sOrCK97a+x7JdyyivKsfgbeDtsW9zXfJ1zi5eq5NgEKID252zm4sWXERRRRHLJy5nSPyQRr2u3FzOl3u/5N20d1mxfwXVuppz4s9hSp8pXHvWtZz5xplc2v1S5o6f6+B3YH/5ZfmsOrCK1LhUpzS3dQUSDOIEpZWlbMncwpC4IY1utSHans0Zmxm9cDTuyp2Vk1fSO6p3s7aTUZjBgm0LmJ82n105u/D18KXUXMrMETN5auRT9i20aBVNCQa5+NzOZRVlMfOHmXR+pTND5w3lv5v/6+wiCQdZc2gN5713Hn6efqyZuqbZoQAQExjDw0Mf5ve//c66aZZz8vFB8QxLGGbHEgtXJTWGdmrHsR28/OvLLNy+kIqqCsb2GEtWURYH8g6w7+59hPiGOLuIwo5W7F/BlYuvJMGQwHeTv2u1cftF2yE1hg5Ka82K/SsYvXA0KW+m8OGOD7ml3y3svms3X074kv+O+y95ZXk89eNTzi6qsKNPfv+E8R+OJyk8idVTV0soiBaTYGgHSitLmbt5LslvJnPJokvYnrWdf57/T47cd4TZY2bX9FLtE92HOwbcwRsb3mDHMdducmhvx4qPcesXtzJvyzyKKoqcXRy7mbdlHtcvuZ5BsYP44aYfnDLUgmh/5FRSG5ZVlMXsDbOZvXE2OSU59I3uy/2p93Nd8nUnHTsmtySXHq/3oG90X1ZNXtUhLkQXVRRx3nvnsfGo5XsT6BXIhOQJTOs/jYGdBrbZz+DlX1/m/pX3M7rbaJZetxQ/Tz9nF0m4MBkSwwXllOTwxPdP4OfpR1RAFNEB0UQznvlqAAAbUElEQVT5RxEVEEWUfxQR/hGN7n1Z//rBuB7juH/I/YzoPOK0B7kwvzCePe9Z7lp+F0t3LeWqXlfZ4+25rMqqSq7++Go2Z2zm8+s/J8w3jLlb5rJg2wLmbJ5D76jeTOs3jRt630Cob6izi9soWmue+vEpnln9DFedeRWLrlyEt4e3s4sl2hGpMbSS51c/z+M/PF7T7K8+hSLML6xOWNS5HxBFRVUFszfM5rsD3+Hr4cvUvlO5N/XeOp2QGsNcbWbAnAEUlBWw665d+Hr62uttuhStNTd9dhMLti3gv+P+y7T+02qeKygr4KMdHzF3y1w2Ht2It7s3V/W6imn9pjEicUSrTh/ZFNW6mvu/vZ9X173K1L5TmTNujgznIBpF+jG4GK01vWb3IsIvgp+m/ERRRRFZxVlkFWU1/G+t+/XPh8cExHD3oLu5bcBthPmFNbtMtiGUnx75NE+OeLKlb9El/WPVP3jh5xd4ZuQzPDHiiZOul5aZxjub32Hh9oXkl+XTLaQbt/S7hSl9p7jU5CnmajO3fnkr89PmM2PwDP5v9P+5bIAJ+9JacyCnGB9Pd2KDm/dDToLBxWzO2MyAOQN4a8xb3D7w9ia9tqSypCYkSipLGJYwrEVjz9d2/afX8/mez9l91+6aiVDai1d/e5UZ387gjgF3MHvM7EZdRyitLGXprqXM3TKXHw/+iLtyZ0yPMUzrN41Lul/ilF/m+WX57MnZw97cvSz+fTFf7/uap0c+zRPDn2iz10bE6ZVWVLHVmM+mQ3lsPpTH5sN55JVUcvuIrjxyyZnN2qYEg4u5/9v7eX3962Q+mOlS57GPFBwh6fUkxvYYy8fXfOzs4tjN4h2LmbBkApf1vIxPr/m0wQHhTmdf7j7e2fIO89Pmk1WcRUxADDf1uYnkyGQi/COI8Iuo+bel5/crqio4kHeAvbl72ZOzhz25ltve3L0cKz5Ws56Xuxf/vvDf3Jt6b4v2J1yL1pqjBWV1QmDnURPmasuxuVuEP/0TQhjQOYQh3cLoHObfrP1IMLiQquoq4l6OIzUulWXXLXN2cU7w3OrneOKHJ/j+xu85r8t5zi5Oi33/5/dcsugSBsUOYuWklS2+flJZVcnX+75m7ua5fLP/G6p19QnrBHkH1QkK2/1I/8g6y4N9gjGajJYDf85fB/8DeQeo0lU124v0j6RHWA+SwpJICkuy3A9PomtIV7vVFoXzVJir2ZlhqgmCTYfyyDSVAeDr6U6feAMDOluCoF98CCH+9vmbSzC4kO/++I5RC0fxyTWfcHWvq51dnBOUVpbSa3YvArwC2HL7ljZ9ITMtM43h7w4nwZDAmqlr7N6721RuIqMwg+ySbLKLs+v+W5LNseJjdZZXVleedFs+Hj50D+1OUnhSnQDoEdZDeqW3I1prMgrK2Hokn7Qj+Ww+nMc2YwHlZssPjNhg35oQGNA5hJ7RgXi4O+a6kTRXdSGLti8iyDuIsT3GOrsoDfL19GXWqFlc+fGVvL3xbe4adJezi9QsB/MPcsmiSzD4GPjmhm8ccnAN8g4iyDuIJE4/raXWGlO5qU5g5JXl0SmwE0lhScQb4uXCcTtUUFrJdmMBaUfySDtSwFZjPtmFlomCPN0VybEGJqd2ZkDnEPp3DiEqyMfJJW6YBIMDlVSWsGTXEq7tdS0+Hq75BQC4vOflXNj1Qp744QmuS76OcL9wZxepSXJKchi9cDRl5jLWTl3rEkNCKKUw+Bgw+Bg4I/QMZxdHOEC5uYpdGYVsPZJvqREY8zmQXVzzfNcIf849I5w+8cH0jQ+mZ0wg3h5Nv97lDBIMDvTlni8pqihiUu9Jzi7KKSmlePXiV+n9Zm+e+P4J3hz7prOL1GjFFcWM/WAshwsO893k7zgr8ixnF0m0Q+aqag7mFrPNWFBzWmhnhonKKsup+IhAb/rGB3NV/zj6xAWTEmfA4Ovp5FI3nwSDAy3cvpDYwFhGJI5wdlFOq1dEL+4edDevrnuV2wbcRr+Yfs4u0mmZq81c9+l1bDi6gSXXLpEhoUWLaa3JLipnd0YhezIL2Z1ZyO5ME/uOFVFhvS7g7+VOSpyBm4d1oW9cMH3ig4kx+LSr5sMSDA6SU5LDiv0ruC/1vjZzLnnmyJks2r6Ie1bcw+opq136i6615vYvb+frfV/z5pg368xnLERjlFSY2ZtVxJ5MkyUAMgrZk1XI8eKKmnUiA71Jig7kpiGdSYoOonecgW4RAbi7ue7/DXuQYHCQj3//GHO1mRtSbnB2URot2CeYf17wT2798lY+2vERE1Im2G3b1bqazRmbSTAk2GUE0Cd/eJJ5afN4cviT3DHwDjuUULQ31dWa/NJKcovKySmqILuonD+OFbE708SezEIOHS/B1ijT19OdHtGBjOoVRVJ0IEnRgfSMDiLUTk1F2xppruogQ+cNxVRuYtsd21z6l3d9VdVVDJ47mMyiTPZM34O/V/M609S2zriOe1bcw/r09QAkBicyOHYwg2IHMTh2MP1i+jVpZNDZG2Zz1/K7mNZvGnPGzWlTn69omZIKM7lFFeQUlf/1b/Ffj3OLbcsryCupoKq67vHNTUFiuD89owNJigqiZ0wgPaMDiQ/xw62d1wKkuaqTHcg7wC9HfuFfF/yrzR203N3c+c8l/2HovKH8a+2/eO7855q9rYzCDB753yO8t/U9ogOieePSNyipLGF9+np+Nf7K4t8XW/ap3Okd1bsmKAbFDqJneM8Geywv3bWU6cunM67HON4c+2ab+3xF01RXazYeyuPztHS+2ZFZ5zRPbQHeHoQFeBHm70V8qB/9EoIJ8/e2LAvwJtzf8m/nMD98PNtGyyBnkmBwgEXbFgEwMWWik0vSPOfEn8Pk3pN58ZcXmdp3Kt1CuzXp9eXmcl5d9yrPrn6WiqoK/j707zx27mMEegfWWS+zKJP16etZn76edenr+GjHR7y96W3AMmfCwE4D/6pZxA1m//H9TFwykdS4VD66+qM23RlPnNquDBOfpx3ly61HSc8vxdfTnYt6RdGrUxBh/l6EB/x10A/z95KDvZ3JqSQ701pz5htnEhUQxU9TfnJ2cZrtaOFRkl5P4oIuF/DZ9Z816jVaa77e9zX3fXsf+4/vZ1yPccwaPavR7firdTV7c/dagsK4jnXp69iatRVztRkAN+VGj7AerJ26tkUjywrXdOR4CV9sPcoXaUfZk1WIh5tieI8ILuvbiQvPjMLfW34ItIScSnKiTRmb2JO7hweGPODsorRIp8BOPH7u4/zjf/9g5R8rGdVt1CnX35OzhxnfzmDF/hX0DO/JihtWMPqM0U3ap5tyo2d4T3qG9+TGPjcCUGYuY0vGFtanr+dwwWHuTb1XQqEdyS0qZ/n2DD5LO8qmQ3kAnJ0YwrOXJ3NpcjRhATIBkTNIjcHO7ltxH7M3zibzgcw2P+ZNubmc5DeT8XDzYNsd2/B0P7HDTkFZAc/89Az/Wf8f/Dz9eGrEU0wfNL3BdYUAKC43893OLD5PS2f1vhyqqjVJUYFc1q8T43p3Ij5Upih1BKkxOIm52syHOz5kTPcxbT4UALw9vHll9CuM/XAsr69/nfuG3FfzXLWu5t0t7/Lo94+SXZzNLf1u4fkLnpfJ6MUJKszVHD5ewr6sQr7Zkcl3O7MorawiNtiX24Z35bK+negZHeTsYopaJBjs6Ps/vyerOMvlh8BoijE9xnBp90t56qenmJgykaiAKH458gv3fHMPmzI2cU78OSyfuJwBnQY4u6jCiWwH/0O5xfyZU8zB3GIO5ZbwZ04xR/NLsbUaDfHz5KoBsVzWN5YBCSHtvoloWyXBYEcLty3E4G3g0u6XOrsodvXy6JdJnp3M3d/cjbeHNwu3LaRTYCcWXrGQiSkTpcloB9HYgz9AkI8HXcL9GdA5hCv7x9El3I/OYf6kxBrwdNCw0sJ+JBjspLiimGW7l3H9Wde79EiqzdEjrAczUmfw4i8v4u3uzaPDHuWRcx8hwCvA2UUTDlJurmJ3RiHb0wvYbixge3oBe7MKa2YVg78O/v0T6h78u4T5E+znKT8Y2jCHBoNS6mLgVcAdmKu1/n/1np8CvAikWxe9rrWe68gyOcoXe76gqKKIG3q3nSEwmmLmiJmE+4Vzda+r6RrS1dnFEXZUbq5ib2YR29Lz2ZFewDajJQRsI4cG+3mSEmvg1qSudI8MIDHcn8Qwf0Lk4N9uOSwYlFLuwBvARYAR2KCU+kJrvbPeqou11tMdVY7Wsmj7IuKC4hjeebizi+IQ/l7+PDz0YWcXQ7RQhbmavVmFbLPWAnakF7A786/how2+nvSOMzDt3K6kxBpIiTUQF+IrAdDBOLLGMAjYr7U+AKCU+gi4DKgfDG1ednE2K/av4IEhD7SZkVRF+1NdrcktriDLVMaxwjKOmcrJMpWTVVjGMVMZR/PL2H+siIoqy/DRQT4epMQZuGWYJQR6x0kICAtHBkMscKTWYyMwuIH1rlJKDQf2AvdprY80sI5L+/j3j6nSVe2qNZJwLYVllRjzSi0HfVM5WaYysgrLyDKVc6ywnGOmMrILy+tcA7AJ8/ciMsiHqCBvzu0eTkqcpSaQEOonISAa5OyLz18CH2qty5VStwPvAefXX0kpdRtwG0BCQkLrlrARFm5fSEpkCilRKc4uimijSirMGPNKOXK8BGNeKca8Eo4cL8WYb/m3oLTyhNcE+3kSFehDZJA33SPDiQryJirIh0jrsqggHyICvPHykFqsaBpHBkM6UHvy3Tj+usgMgNY6t9bDucC/G9qQ1noOMAcsPZ/tW8yW+eP4H/xm/I0XLnzB2UURLsxcZWnqefh4CUesB35jXilGaxDk1hs11NvDjbgQX+JC/OgbH0xciB+xwb50CrYc+CMCvWXgOOEwjgyGDUB3pVQXLIFwPVBnuFGlVIzWOsP6cDywy4HlcYhF2xehUExItt+kNqLt0lqTZSqvmQzGNj3k/uy/poYE8HJ3IzbEl7gQX0Z1spzbjw/1s4aBLxEB3nKaRziNw4JBa21WSk0HvsXSXHWe1vp3pdQzwEat9RfAPUqp8YAZOA5McVR5HEFrzcJtCxmROIJ4Q/zpXyDaFVNZZc3Bv+aWVVjntE90kA9J0YEM6x5Oj6hAEsP8iA/1IyLAW3r9Cpfl0GsMWuvlwPJ6y56sdf8R4BFHlsGRNh7dyL7j+/j70L87uyjtXm5ROT/sySbA251ogy8xBh/CA7wdPvduaUUVmaYyMgvKOJpfyr5jljmC92QWcrSgrGa9QG8PkqIDGds7xjI1ZJRleshgv445NaRo25x98blNW7htIV7uXlzV6ypnF6Vd0toye9fC3w7xzfbMmmaWNh5uiqggH6INlluM9X6MwZeYYB9iDJaLrx4NDMGgteZ4cQWZpjKyTGVkFpSTWVBqCQFTOVkFZWSayk646OvprugWEcCgLqEkRQeRFB1AUnQQnQw+cupHtBsSDM1krjbz0e8fMa7HOIJ9gp1dnHalsKySZVvSWfTbYfZkFRLo7cHEwQlcPSAOpbD8ei8oI7OglIwCy6/5XUdN/G9XFmWVdcPDTUFkoCUwwgO8KSi1hkFB+QlBoxREBHgTbfChc5gfg7uGWoInyBIyUQYfEkL9ZKwf0e5JMDTTqgOrOFZ8jBtS2ucQGM6wI72AResO8XnaUUoqqkiJNfDCVSmM69MJP6+/vqpndTI0+HqtNQWllTVhkWENj6PWx8a8Egy+nvRPCLHUMqwH/SjDqWsXQnQ0EgzNtGj7IoJ9gtvdSKqtrayyii+3HmXRusOkHcnHx9ON8X06ccPgzvSJb1pNTClFsJ8XwX5enBkj4/sL0VwSDM1QXFHMsl3LmJgyEW8PmXqwOf7ILuKDdYf5dJORgtJKukX48+TYXlzVPw6Dn8z+JoQzSTA0w+d7Pqe4sliGwGiiyqpqvtuZxcLfDvHLH7l4uitGnxXNDYM7k9o1VC7eCuEiJBiaYeG2hcQHxTMsYZizi+Jyqqs12UXlHDlewhHrsA62+3uzijheXEFssC8PjU7i2oHxRARKjUsIVyPB0ETHio+x8o+VPHTOQx12JNWC0krLwb72wT+vpGacn3Jz3dY+kYHexIf6MTIpgrG9YxjRI9Lh/Q+EEM3XYYKhrLKKkooqgnw8WtTyZPGOxVTpqjYxIU9uUTn7jhVZblmFHMotwVxdjdZYbmjLdIzW+5ZlltY9ln/r3q+squZofimmMnOd/QT5eBAf6kf3yEDO7xlJfKgf8SF+xIdaxvqRMX2EaFs6TDD8sPsYdy7aDIC/lztBvp4YfD0J8vEkyNeTIF+POo8t963LrI+D/TxZtH0RfaL6kByZ7OR3ZKG1Jqeogn3HCtl/rIi9WYXsy7KEwfFaA7MFeHuQGO6Ht4c7CkubfaWU5b4bKNysy0Ch6j6vQAHubm4M6hJa56AfH+qHwVcuFgvRnnSYYOgZE8RT43pRUGrGVFaJqbSSgtJKTGWVpOeXsivDsqyw3HzSbVSqdI76rCPZ/2/cvziNhDA/EsP86RxmmevWkVMdmquqyS2uYL/11//eY0Xszypi37FC8kr+6p0b6O1B96gALjoziu5RAXSPCqR7ZAAx0jNXCNFIHSYYuoT70yW8y2nXq6rWFJWZa0KjoPSvEFmwcyVHDym6+o/mtwO5LEtLR9caBDzQx4PEMH9rYFjConOoH4nh/kQG/jVaptYaU5mZvOIKcosryCuu4HhJvX+LK8krqeB4seVWf2iGIB8PekQFcnFyNN0jAy0hEBlIVJCMyimEaJkOEwwZhRmkZabh7eGNt7s3Ph4+NffrLwv09TqhLb3WmifWLef8Lufx+Y3jAct1C2NeCQdzSjiYW8zh4yUczC1hR3oBK3ZkUlVrNi0fTzeig3woKq8iv6SiwZm2wDIcc4i/J6H+3oT6e3JWpyBC/b0I8fMiPMCLrhEBdI8MICJQAkAI4RgdJhjWHF7DdZ9e1+j1Pd088fawhoW7N57unhzMP8gjw/4aDNbH050zIgM5IzLwhNfbLtQezC3hUG4xh3JLyDSVEeTjQYifF6H+lluIvxehfn/d9/dylwO+EMKplNYuNSHaaQ0cOFBv3Lixya87Xnqcvbl7KTeXU15VTpm5rOZ+udn62Hr/ZMt8PXx55eJX8Pfyd8A7E0IIx1FKbdJaD2zMuh2mxhDqG0pqXKqziyGEEC6vY/bQEkIIcVISDEIIIeqQYBBCCFGHBIMQQog6JBiEEELUIcEghBCiDgkGIYQQdUgwCCGEqKPN9XxWSmUDh5xdDicLB3KcXQgnk8/AQj4H+QygcZ9BZ611RGM21uaCQYBSamNju7a3V/IZWMjnIJ8B2P8zkFNJQggh6pBgEEIIUYcEQ9s0x9kFcAHyGVjI5yCfAdj5M5BrDEIIIeqQGoMQQog6JBhcjFIqXin1g1Jqp1Lqd6XUvdbloUqp75RS+6z/hliXK6XUf5RS+5VS25RS/Z37DuxHKeWulNqilPrK+riLUmqd9b0uVkp5WZd7Wx/vtz6f6Mxy25NSKlgp9alSardSapdSakhH+y4ope6z/l/YoZT6UCnl0xG+C0qpeUqpY0qpHbWWNflvr5S6ybr+PqXUTY3ZtwSD6zEDD2itewGpwF1KqV7AP4D/aa27A/+zPga4BOhuvd0GvNn6RXaYe4FdtR6/ALystT4DyANusS6/BcizLn/Zul578SqwQmvdE+iD5fPoMN8FpVQscA8wUGudDLgD19MxvgvzgYvrLWvS314pFQrMBAYDg4CZtjA5Ja213Fz4BnwOXATsAWKsy2KAPdb7bwMTaq1fs15bvgFx1i/++cBXgMLSgcfD+vwQ4Fvr/W+BIdb7Htb1lLPfgx0+AwPwZ/330pG+C0AscAQItf5tvwJGd5TvApAI7Gju3x6YALxda3md9U52kxqDC7NWg/sB64AorXWG9alMIMp63/Yfx8ZoXdbWvQI8DFRbH4cB+Vprs/Vx7fdZ8xlYny+wrt/WdQGygXetp9TmKqX86UDfBa11OvAScBjIwPK33UTH+y7YNPVv36zvhASDi1JKBQBLgBlaa1Pt57Ql+tttczKl1FjgmNZ6k7PL4mQeQH/gTa11P6CYv04dAB3iuxACXIYlJDsB/px4eqVDcuTfXoLBBSmlPLGEwiKt9VLr4iylVIz1+RjgmHV5OhBf6+Vx1mVt2VBgvFLqIPARltNJrwLBSikP6zq132fNZ2B93gDktmaBHcQIGLXW66yPP8USFB3pu3Ah8KfWOltrXQksxfL96GjfBZum/u2b9Z2QYHAxSikFvAPs0lrPqvXUF4CtRcFNWK492JbfaG2VkAoU1Kpqtkla60e01nFa60QsFxq/11rfAPwAXG1drf5nYPtsrrau3+Z/RWutM4EjSqkk66ILgJ10oO8CllNIqUopP+v/Ddtn0KG+C7U09W//LTBKKRVirX2Nsi47NWdfXJHbCRebhmGpHm4D0qy3S7GcJ/0fsA9YBYRa11fAG8AfwHYsrTec/j7s+HmMBL6y3u8KrAf2A58A3tblPtbH+63Pd3V2ue34/vsCG63fh8+AkI72XQCeBnYDO4AFgHdH+C4AH2K5rlKJpfZ4S3P+9sDN1s9jPzC1MfuWns9CCCHqkFNJQggh6pBgEEIIUYcEgxBCiDokGIQQQtQhwSCEEKIOCQYhTkIp9Zh1VM9tSqk0pdRgpdQMpZSfs8smhCNJc1UhGqCUGgLMAkZqrcuVUuGAF/ALljbiOU4toBAOJDUGIRoWA+RorcsBrEFwNZbxen5QSv0AoJQapZT6VSm1WSn1iXWMK5RSB5VS/1ZKbVdKrVdKnWFdfo11XoGtSqnVznlrQpya1BiEaID1AL8W8MPSw3Sx1von6/hNA7XWOdZaxFLgEq11sVLq71h64D5jXe+/WuvnlVI3AtdqrccqpbYDF2ut05VSwVrrfKe8QSFOQWoMQjRAa10EDMAy6Uk2sFgpNaXeaqlAL+BnpVQalrFrOtd6/sNa/w6x3v8ZmK+UuhXLpDNCuByP068iRMekta4CfgR+tP7Srz8togK+01pPONkm6t/XWt+hlBoMjAE2KaUGaK3b0+ifoh2QGoMQDVBKJSmlutda1Bc4BBQCgdZlvwFDa10/8FdK9aj1mutq/furdZ1uWut1WusnsdREag+JLIRLkBqDEA0LAF5TSgVjmYd7P5bTShOAFUqpo1rr86ynlz5USnlbX/c4sNd6P0QptQ0ot74O4EVr4Cgso2RubZV3I0QTyMVnIRyg9kVqZ5dFiKaSU0lCCCHqkBqDEEKIOqTGIIQQog4JBiGEEHVIMAghhKhDgkEIIUQdEgxCCCHqkGAQQghRx/8HtAZXgpiZG8QAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "bigger_val_loss = np.array(bigger_model.get_validation_summary(\"Loss\"))\n",
+    "\n",
+    "plt.plot(original_val_loss[:,0], original_val_loss[:,1], label='original model')\n",
+    "plt.plot(bigger_val_loss[:,0], bigger_val_loss[:,1],label='bigger model',color='green')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The bigger network starts overfitting almost right away, after just one epoch, and overfits much more severely. Its validation loss is also \n",
+    "more noisy.\n",
+    "\n",
+    "Meanwhile, here are the training losses for our two networks:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the bigger network gets its training loss near zero very quickly. The more capacity the network has, the quicker it will be \n",
+    "able to model the training data (resulting in a low training loss), but the more susceptible it is to overfitting (resulting in a large \n",
+    "difference between the training and validation loss)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Adding weight regularization\n",
+    "\n",
+    "\n",
+    "You may be familiar with _Occam's Razor_ principle: given two explanations for something, the explanation most likely to be correct is the \n",
+    "\"simplest\" one, the one that makes the least amount of assumptions. This also applies to the models learned by neural networks: given some \n",
+    "training data and a network architecture, there are multiple sets of weights values (multiple _models_) that could explain the data, and \n",
+    "simpler models are less likely to overfit than complex ones.\n",
+    "\n",
+    "A \"simple model\" in this context is a model where the distribution of parameter values has less entropy (or a model with fewer \n",
+    "parameters altogether, as we saw in the section above). Thus a common way to mitigate overfitting is to put constraints on the complexity \n",
+    "of a network by forcing its weights to only take small values, which makes the distribution of weight values more \"regular\". This is called \n",
+    "\"weight regularization\", and it is done by adding to the loss function of the network a _cost_ associated with having large weights. This \n",
+    "cost comes in two flavors:\n",
+    "\n",
+    "* L1 regularization, where the cost added is proportional to the _absolute value of the weights coefficients_ (i.e. to what is called the \n",
+    "\"L1 norm\" of the weights).\n",
+    "* L2 regularization, where the cost added is proportional to the _square of the value of the weights coefficients_ (i.e. to what is called \n",
+    "the \"L2 norm\" of the weights). L2 regularization is also called _weight decay_ in the context of neural networks. Don't let the different \n",
+    "name confuse you: weight decay is mathematically the exact same as L2 regularization.\n",
+    "\n",
+    "In Analytics-zoo Keras API, weight regularization is added by passing _weight regularizer instances_ to layers as keyword arguments. Let's add L2 weight \n",
+    "regularization to our movie review classification network:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createL2Regularizer\n",
+      "creating: createZooKerasDense\n",
+      "creating: createL2Regularizer\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras import regularizers\n",
+    "\n",
+    "l2_model = models.Sequential()\n",
+    "l2_model.add(layers.Dense(16, W_regularizer=regularizers.l2(0.001),\n",
+    "                          activation='relu', input_shape=(10000,)))\n",
+    "l2_model.add(layers.Dense(16, W_regularizer=regularizers.l2(0.001),\n",
+    "                          activation='relu'))\n",
+    "l2_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "l2_model.compile(optimizer='rmsprop',\n",
+    "                 loss='binary_crossentropy',\n",
+    "                 metrics=['acc'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`l2(0.001)` means that every coefficient in the weight matrix of the layer will add `0.001 * weight_coefficient_value` to the total loss of \n",
+    "the network. Note that because this penalty is _only added at training time_, the loss for this network will be much higher at training \n",
+    "than at test time.\n",
+    "\n",
+    "Here's the impact of our L2 regularization penalty:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dir_name = '4-4 ' + str(time.ctime())\n",
+    "l2_model.set_tensorboard('./', dir_name)\n",
+    "l2_model.fit(x_train, y_train,\n",
+    "             nb_epoch=20,\n",
+    "             batch_size=512,\n",
+    "             validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 512 records in 0.024366594 seconds. Throughput is 21012.373 records/second. Loss is 0.13651785.\n",
+    "Top1Accuracy is Accuracy(correct: 21684, count: 25000, accuracy: 0.86736)_"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XdYFOf6//H3QxdFJIoIIoINQUQpii0ajT1qYonfmKLGGDXR9GN+OSmmnZyTk5NjTEzTGI2mauztJJYk9qgoYBfsgNhQmkjbfX5/7LpBRUVl2QXu13Xtxe7M7Oy9s8N+duaZeUZprRFCCCEAHGxdgBBCCPshoSCEEMJCQkEIIYSFhIIQQggLCQUhhBAWEgpCCCEsJBSEEEJYSCgIIYSwkFAQQghh4WTrAm5VnTp1dGBgoK3LEEKICmXHjh3ntNbeN5uuwoVCYGAgsbGxti5DCCEqFKXU8dJMJ7uPhBBCWEgoCCGEsJBQEEIIYVHh2hRKUlhYSEpKCnl5ebYuRVQQbm5u+Pv74+zsbOtShLArlSIUUlJS8PDwIDAwEKWUrcsRdk5rTXp6OikpKQQFBdm6HCHsSqXYfZSXl0ft2rUlEESpKKWoXbu2bFkKUYJKEQqABIK4JbK+CFGyShMKQghRWRmMmvdW7CM145LVX0tCoZz17duXjIyMG04zadIk1qxZc1vz/+OPP+jXr99tPbe0jh07RlhY2B1PI4S4Oa01f1+4i682HOWPg2es/nqVoqG5ItBao7Vm5cqVN532nXfeKYeKhBD2TmvNP1bsZ15sCs92a8IjMQ2t/pqypVBGJk+eTFhYGGFhYUyZMgUw/VoODg5m+PDhhIWFkZycTGBgIOfOnQPg3XffJTg4mE6dOjFs2DA+/PBDAEaOHMn8+fMBU7ceb775JpGRkbRs2ZIDBw4AsG3bNtq3b09ERAQdOnTg4MGDN6zvm2++4YEHHqBHjx4EBgby6aefMnnyZCIiImjXrh3nz58HID4+nnbt2hEeHs7AgQO5cOECADt27KBVq1a0atWKzz77zDJfg8HAxIkTadOmDeHh4UybNq0Ml6oQVdvU3w7x9cajjOwQyAs9mpXLa1a6LYW3l+1l38msMp1nqF9N3uzf4rrjd+zYwaxZs9i6dStaa2JiYujSpQteXl4kJSUxe/Zs2rVrd8Vztm/fzoIFC0hISKCwsJDIyEiioqJKnH+dOnXYuXMnn3/+OR9++CEzZsygefPmbNiwAScnJ9asWcOrr77KggULbvg+9uzZQ1xcHHl5eTRp0oR///vfxMXF8cILLzBnzhyef/55hg8fztSpU+nSpQuTJk3i7bffZsqUKTz++ON8+umndO7cmYkTJ1rm+fXXX+Pp6cn27dvJz8+nY8eO9OzZUxpyhbhDszYdZfLqRAZF1mdSv9By+5+SLYUysHHjRgYOHEj16tWpUaMGgwYNYsOGDQA0bNjwmkAA2LRpE/fffz9ubm54eHjQv3//685/0KBBAERFRXHs2DEAMjMzefDBBwkLC+OFF15g7969N62za9eueHh44O3tjaenp+U1W7ZsybFjx8jMzCQjI4MuXboAMGLECNavX09GRgYZGRl07twZgMcee8wyz1WrVjFnzhxat25NTEwM6enpJCUllWKpCSGuZ8GOFN5eto+eoT58MDgcB4fy+5FV6bYUbvSL3haqV69+x/NwdXUFwNHRkaKiIgDeeOMNunbtyqJFizh27Bj33HNPqecD4ODgYHns4OBgme+t0lozdepUevXqdcXwy+ElhLg1v+w5xcT5CXRqUoepD0fg5Fi+v91lS6EM3H333SxevJjc3FwuXrzIokWLuPvuu2/4nI4dO7Js2TLy8vLIyclh+fLlt/SamZmZ1K9fHzC1F5QFT09PvLy8LFs53377LV26dKFWrVrUqlWLjRs3AvD9999bntOrVy+++OILCgsLAUhMTOTixYtlUo8QVc3GpHM8+2McrRrUYtpjUbg6OZZ7DZVuS8EWIiMjGTlyJG3btgVg9OjRRERE3PDXcps2bRgwYADh4eH4+PjQsmVLPD09S/2aL7/8MiNGjOAf//gH9913352+BYvZs2czbtw4cnNzadSoEbNmzQJg1qxZjBo1CqUUPXv2tEw/evRojh07RmRkJFprvL29Wbx4cZnVI0RVseP4BcZ8G0sj7+rMGtmG6q62+XpWWmubvPDtio6O1ldfZGf//v2EhITYqKLbl5OTQ40aNcjNzaVz585Mnz6dyMhIW5dVZVTU9UZUPvvTsvi/aVvwqu7Cz+PaU9fDrcxfQym1Q2sdfbPpZEvBhsaMGcO+ffvIy8tjxIgREghCVEHHzl3ksa+34e7ixHdPxFglEG6FhIIN/fDDD7YuQQhhQ2mZl3hkxlaMWvPd6Bga3OVu65KkoVkIIWwhPSefR2dsJetSIXNGtaVJXQ9blwTIloIQQpS7rLxCRszaRsqFS8wZ1Zaw+qU/yMTaZEtBCCHK0aUCA6O/ieVAWjZfPhpFTKPati7pCrKlIIQQ5aSgyMhT3+9g+/HzfPJQBF2b17V1SdeQLYUyUqNGjWuGTZ48mdDQUMLDw7n33ns5fvx4udf11ltvWTraK62lS5fy/vvv3/Fr33PPPVx9+HBZK9554J1MI4S1GYyaF+fF88fBs/xzYEv6t/KzdUklklCwooiICGJjY9m1axdDhgzh5Zdfvulzbre7ibJSVFTEgAEDeOWVV2xahxCVidaa1xfvZvmuNF7t25xhbQNsXdJ1SShYUdeuXXF3Nx1i1q5dO1JSUkqcbuTIkYwbN46YmBhefvllLl68yKhRo2jbti0REREsWbIEgNzcXIYOHUpoaCgDBw4kJibG8ku8+JbK/PnzGTly5DWv89VXX9GmTRtatWrF4MGDyc3NLfH1v/nmGyZMmABA69atLbdq1aqxbt2669Z36dIlHnroIUJCQhg4cCCXLpV8lajAwED+/ve/07p1a6Kjo9m5cye9evWicePGfPnll4Dpn2jixImEhYXRsmVL5s6daxk+YcIEgoOD6d69O2fO/HXRkR07dtClSxeioqLo1asXaWlppfughLAirTWTluzlx23JjO/amDGdG9u6pBuqdG0Kz//yPPGn4st0nq3rtWZK7yl3NI+vv/6aPn36XHd8SkoKmzdvxtHRkVdffZVu3boxc+ZMMjIyaNu2Ld27d+eLL77Ay8uLffv2sWfPHlq3bn1LNQwaNIgnn3wSgNdff52vv/6aZ5555prXL96XUny8aVkuW7aMDz74gA4dOvDmm2+WWN+0adNwd3dn//797Nq164Yn4wUEBBAfH88LL7zAyJEj2bRpE3l5eYSFhTFu3DgWLlxIfHw8CQkJnDt3jjZt2tC5c2e2bNnCwYMH2bdvH6dPnyY0NJRRo0ZRWFjIM888w5IlS/D29mbu3Lm89tprzJw585aWkRBl6XIgfPvnccZ2bsTfegbbuqSbqnShYI++++47YmNjWbdu3XWnefDBB3F0NHV+tWrVKpYuXWppC8jLy+PEiRNs3LiR5557DoCwsDDCw8NvqY49e/bw+uuvk5GRQU5OzhU9mxZ//aslJSUxceJEfv/9d5ydna9b3/r163n22WcBCA8Pv2F9AwYMAEzddufk5ODh4YGHhweurq5kZGSwceNGhg0bhqOjIz4+PnTp0oXt27ezfv16y3A/Pz+6desGwMGDB9mzZw89evQATBf/8fX1vaXlI0RZujoQXunTvEJcZ6TShcKd/qIva2vWrOG9995j3bp1lq6qX3vtNVasWAH89Uu8eBfbWmsWLFhAcHDpf1UUX9ny8vJKnGbkyJEsXryYVq1a8c033/DHH39Yxl2vi++cnByGDh3KV199ZfmSvZ36rla82+6ru/S+nXYVrTUtWrRgy5Ytt12TEGWlogYCSJuCVcXFxTF27FiWLl1K3bp/HXr23nvvER8fbwmEq/Xq1YupU6dyubPCuLg4wNTd9rx58wDYt28fu3fvtjzHx8eH/fv3YzQaWbRoUYnzzc7OxtfXl8LCwiu6v76RUaNG8fjjj1/RFfj16uvcubOl6449e/awa9euUr1GSe6++27mzp2LwWDg7NmzrF+/nrZt29K5c2fL8LS0NH7//XcAgoODOXv2rCUUCgsLS3XhISHKWkUOBKiEWwq2kpubi7+/v+Xxiy++yMqVK8nJyeHBBx8ETPvRly5detN5vfHGGzz//POEh4djNBoJCgpi+fLlPP3004wYMYLQ0FCaN29OixYtLN1tv//++/Tr1w9vb2+io6PJycm5Zr7vvvsuMTExeHt7ExMTQ3Z29g3rOH78OPPnzycxMdGyb37GjBnXre+pp57i8ccfJyQkhJCQkOteXrQ0Bg4cyJYtW2jVqhVKKT744APq1avHwIED+e233wgNDSUgIID27dsD4OLiwvz583n22WfJzMykqKiI559/nhYt7OuiS6Jyq+iBAFbuOlsp1Rv4GHAEZmit379q/EdAV/NDd6Cu1rrWjeZZmbrOvlUGg4HCwkLc3Nw4fPgw3bt35+DBg7i4uNi6tAqpqqw3onzYeyDYvOtspZQj8BnQA0gBtiullmqt912eRmv9QrHpnwEirFVPZZCbm0vXrl0pLCxEa83nn38ugSCEHbD3QLgV1tx91BY4pLU+AqCU+gm4H9h3nemHAW9asZ4Kz8PDw+pnCAshbk1lCgSwbkNzfSC52OMU87BrKKUaAkHAb7f7YhXtCnLCtmR9EWWhsgUC2M/RRw8B87XWhpJGKqXGKKVilVKxZ8+evWa8m5sb6enp8o8uSkVrTXp6Om5utr3ClajYKmMggHV3H6UCDYo99jcPK8lDwPjrzUhrPR2YDqaG5qvH+/v7k5KSQkmBIURJ3NzcrjhaTIhbUVkDAawbCtuBpkqpIExh8BDw8NUTKaWaA17AbZ915OzsTFBQ0O0+XQghSq0yBwJYcfeR1roImAD8CuwH5mmt9yql3lFKDSg26UPAT1r2/Qgh7FxlDwSw8slrWuuVwMqrhk266vFb1qxBCCHKQlUIBLCfhmYhhLBbRmPVCASQbi6EEOKGLuYX8eK8eH7de7rSBwJIKAghxHWdzLjE6NmxHDiVxZv9QxnZIbBSBwJIKAghRIniTlzgyTk7yC80MHNkG+4JrnvzJ1UCEgpCCHGVJfGpTJy/i3o13fjxyRia+njYuqRyI6EghBBmRqNmyppEPvntEG2D7uLLR6O4q3rV6nRSQkEIIYDcgiJempfA//acYmi0P/94oCUuTlXvAE0JBSFElXcqM4/Rc7az92QWr98XwhOdgip9g/L1SCgIIaq0hOQMnpwTy8X8ImYMj+beEB9bl2RTEgpCiCpr+a6TvDQvAW8PV759oiPB9apOg/L1SCgIIaocrTVT1iTx8dokoht68eVjUdSp4WrrsuyChIIQokrJKzTw0s8JrNiVxuBIf/45KAxXJ0dbl2U3JBSEEFXG6aw8xsyJZVdqJq/0ac7Yzo2qbIPy9UgoCCGqhD2pmYyeHUtWXiHTHo2iZ4t6ti7JLkkoCCEqLa018ckZfL/1BEsTTlKnugvzx3Ug1K+mrUuzWxIKQohK52J+EUviT/L91uPsPZmFu4sjgyP9ebFHM7w9pEH5RiQUhBCVxv60LL7fepzFcSfJyS+ieT0P3n0gjAda++Hh5mzr8ioECQUhRIWWV2hg5e40vvvzODtPZODi5EC/cF8eiWlIZEAtaUi+RRIKQogK6cjZHH7YeoL5O1PIyC0kqE51Xr8vhMGR/nhVsU7sypKEghCiwig0GFm97zTf/XmczYfTcXJQ9GpRj0diAmjfuLZsFZQBCQUhhN0rMhj59PdDfL/1BGez86lfqxp/69mMoW0aUNfDzdblVSoSCkIIuzd7y3GmrEmia7A3j7VvSJdmdXF0kK0Ca5BQEELYtTNZeXy0OpEuzbyZObKN7CKysqp3BQkhRIXyr/8doKDIyFsDWkgglAMJBSGE3dp6JJ1FcamM6dyIoDrVbV1OlSChIISwS4UGI5OW7KV+rWqM79rE1uVUGRIKQgi7NGfLcQ6ezmZS/1CquUjX1uVFQkEIYXfOZOUxxdy43DO0al8es7xJKAgh7M6//neAfGlctgkJBSGEXZHGZduSUBBC2I0ig5E3l0rjsi1JKAgh7MacLcc5cCqbN/pJ47KtSCgIIexC8TOXe7WQxmVbsWooKKV6K6UOKqUOKaVeuc40Q5VS+5RSe5VSP1izHiGE/ZLGZftgtb6PlFKOwGdADyAF2K6UWqq13ldsmqbA34GOWusLSqm61qpHCGG/LjcuT+jaRBqXbcyaWwptgUNa6yNa6wLgJ+D+q6Z5EvhMa30BQGt9xor1CCHskDQu2xdrhkJ9ILnY4xTzsOKaAc2UUpuUUn8qpXqXNCOl1BilVKxSKvbs2bNWKlcIYQvSuGxfbN3Q7AQ0Be4BhgFfKaVqXT2R1nq61jpaax3t7e1dziUKIazlTLapcbmzNC7bDWuGQirQoNhjf/Ow4lKApVrrQq31USARU0gIIaqA91eaGpfflsZlu2HNUNgONFVKBSmlXICHgKVXTbMY01YCSqk6mHYnHbFiTUIIO7Ht6HkWxqXyZOcgaVy2I1YLBa11ETAB+BXYD8zTWu9VSr2jlBpgnuxXIF0ptQ/4HZiotU63Vk1CCPtQZDAyackeaVy2Q1a9HKfWeiWw8qphk4rd18CL5psQooq43Lj85aNRuLvIVYHtia0bmoUQVYw0Lts3CQUhRLmSxmX7JqEghCg30rhs/yQUhBDlQhqXKwYJBSFEufjrzOUQaVy2YxIKQgiriz12vljjcj1blyNuQOJaCGE1eYUGJq9O5KsNR/DzrMa790vjsr2TUBBCWEV8cgYvzYvn8NmLDGsbwGv3hVDDVb5y7J18QkKIMlVQZOSTtUl8se4w3jVcmT2qLV2aSUeWFYWEghCizOw9mclL8xI4cCqbIVH+vNEvFM9qzrYuS9wCCQUhxB0rNBj54o/DfLI2Ca/qLswYHk33UDlbuSKSUBBC3JHE09m8NC+B3amZDGjlx9sDWuBV3cXWZYnbJKEghLgtBqPmqw1HmLwqkRpuTnzxSCR9WvrauixxhyQUhBC37MjZHF76OYG4Exn0blGPfwwMo04NV1uXJcqAhIIQotSMRs2szcf44JcDuDk78vFDrRnQyk/OPahEJBSEEKVyIj2Xv81PYNvR83RrXpd/DWqJT003W5clylipQkEp1RhI0VrnK6XuAcKBOVrrDGsWJ4SwD+sSz/LUdztwVIoPhoTzYJS/bB1UUqXt+2gBYFBKNQGmAw2AH6xWlRDCbvyy5xSjZ28nsHZ1fn2hM0OjG0ggVGKlDQWj+ZrLA4GpWuuJgBxmIEQltyguhfE/7KRlfU9+HNMOv1rVbF2SsLLStikUKqWGASOA/uZhcpqiEJXY91uP8/riPbRvVJuvhkdTXfotqhJKu6XwONAeeE9rfVQpFQR8a72yhBC2NH39YV5btIduwXWZObKNBEIVUqpPWmu9D3gWQCnlBXhorf9tzcKEEOVPa82UNUl8vDaJfuG+fPR/rXF2lMuuVCWl+rSVUn8opWoqpe4CdgJfKaUmW7c0IUR50lrz3or9fLw2iaHR/nz8UIQEQhVU2k/cU2udBQzCdChqDNDdemUJIcqTwah5ddEeZmw8ysgOgbw/KBxHBznCqCoqbSg4KaV8gaHAcivWI4QoZ4UGIy/Oi+fHbSeY0LUJb/YPxUECocoqbevRO8CvwCat9XalVCMgyXplCSHKQ36RgQk/xLF632le7h3M0/c0sXVJwsZK29D8M/BzscdHgMHWKkoIYX25BUWM/XYHG5LO8c79LRjePtDWJQk7UNqGZn+l1CKl1BnzbYFSyt/axQkhrCMrr5ARM7ex6dA5/jMkXAJBWJS2TWEWsBTwM9+WmYcJISqY8xcLeOSrrcSdyGDqsEgejG5g65KEHSltKHhrrWdprYvMt28AuRK3EBXMmaw8Hpq+hcTT2Xw1PJr7wqW3GnGl0oZCulLqUaWUo/n2KJBuzcKEEGUr5UIuD07bQsqFS8x6vA1dm9e1dUnCDpU2FEZhOhz1FJAGDAFGWqkmIUQZMhg1K3al8eCXW7hwsYDvRsfQoXEdW5cl7FSpQkFrfVxrPUBr7a21rqu1foBSHH2klOqtlDqolDqklHqlhPEjlVJnlVLx5tvo23gPQogSFBQZmRebTI/J6xj/w06quTjy05j2RAZ42bo0YcfupJerF4Ep1xuplHIEPgN6ACnAdqXUUnM/SsXN1VpPuIM6hBDFXCowMHf7CaavP8LJzDxa+NXk80ci6dWinpylLG7qTkLhZmtXW+CQ+ZwGlFI/AfcDV4eCEKIMZOUV8u2W48zceJT0iwW0CfTin4Na0qWZt1wUR5TanYSCvsn4+kBysccpQEwJ0w1WSnUGEoEXtNbJJUwjhLiO9Jx8Zm46ypzNx8nOL6JLM2/Gd21C26C7bF2aqIBuGApKqWxK/vJXQFlcgmkZ8KP52s9jgdlAtxLqGAOMAQgICCiDlxWi4juZcYmvNhzhx20nyC8y0iesHk/f04Sw+p62Lk1UYDcMBa21xx3MOxXTtZwv8zcPKz7/4oe1zgA+uE4d0zFdG5ro6OibbaEIUakdPXeRL/84zMK4FLSGByLqM65LY5rUrWHr0kQlYM3LKW0Hmpqv0pYKPAQ8XHwCpZSv1jrN/HAAsN+K9QhRoe07mcXnfxxi5e40nBwdGNY2gDGdG+Hv5W7r0kQlYrVQ0FoXKaUmYOpd1RGYqbXeq5R6B4jVWi8FnlVKDQCKgPPIuQ9CXONsdj7v/+8AC3amUMPViTGdGzOqUyB1PdxsXZqohJTWFWtvTHR0tI6NjbV1GUJYXZHByLd/Hmfy6kTyCg2M6hTE012a4OnubOvSRAWklNqhtY6+2XRyNW4h7NC2o+eZtGQPB05lc3fTOrw1oAWNvaXNQFifhIIQduRMdh7vrzzAwrhU/Dzd+PJR00lncp6BKC8SCkLYgUKDkTlbjjNldSL5RUbGd23M+K5NcHeRf1FRvmSNE8LG/jySzptL9nLwdDZdmnnz1oAWBNWpbuuyRBUloSCEjZzOyuOfK/ezJP4k9WtVY/pjUfQI9ZFdRcKmJBSEKGeFBiPfbDrGlDWJFBo1z3ZrwlP3NKGai6OtSxNCQkGI8rT58DneXLKXpDM5dGtelzf7h9KwtuwqEvZDQkGIcnDhYgFvLNnD8l1pNLirGjOGR9M91MfWZQlxDQkFIaws6XQ2T8yO5VRmHs93b8q4Lo1xc5ZdRcI+SSgIYUW/HTjNsz/G4+bsyE9j28lVz4Tdk1AQwgq01kxff4T3fzlAqG9NvhoejV+tsuhtXgjrklAQoozlFRp4ddFuFu5M5b6WvvznwXA5CU1UGLKmClGGzmTnMe7bHew8kcEL3Zvx7L1N5LwDUaFIKAhRRvakZjJmTiwXcgv54pFI+rT0tXVJQtwyCQUhysDK3Wm8NC8BL3dn5j/VnhZ+cklMUTFJKAhxB7TWfLL2EB+tSSQyoBbTHovG28PV1mUJcdskFIS4TZcKDPzt5wRW7E5jcKQ//xwUhquTnH8gKjYJBSFuQ1rmJZ6cE8vek1m81jeE0XcHSYOyqBQkFIS4RTtPXGDstzvIKzAwc0Qbujava+uShJ3LKcjhg00fcP7SeZwdnHF2dLb8dXJwKtUwZwdnWvq0JMAzwKq1SigIcQsW7kzhlYW78fV044fRMTT18bB1ScLOncw+Sb8f+pFwOoFabrUoNBRSZCyi0Gj6eyu+uO8LxkWPs1KlJhIKokLIyC3gh20nGNDKD38v93J//UKDkf+uSuTLdYdp36g2nz8SiVd1l3KvQ1Qse87soe/3fTl/6TzLhi2jb9O+V4zXWlsCotBQaAmKy/ev/tvQs6HVa5ZQEBXC5NWJzNlynI9WJzKsbQATujahbk03q79uocHIop2pTP09ieTzl3isXUMm9Q/F2dHB6q8tKra1R9YyaN4gqjtXZ/3j64n0jbxmGqWUadeQozM426DIEkgoCLuXfD6XH7edoH8rP2q4OvH91hPMi01mRIdAxnVubJVf7EUGI4viUpn62yFOnM8l3N+TdwaESfuBKJXZ8bMZvWw0wbWDWfnISqu3A5QlCQVh96b+loRSilf7NsfXsxpjOzdiyppEpq8/wg9/nuCJu4N4olMQHm53/lOryGBkcfxJpv6WxPH0XFrW9+TrEdF0a15Xji4SN6W15p117/DWure4N+heFgxdgKdbxTqRUUJB2LUjZ3NYsDOVEe0D8fU09TIaWKc6Ux6K4Kl7mjB59UGmrEli9uZjjOvSmOHtA2/rspZFBiNLzGFwLD2XFn41mTE8mntDJAxE6RQYChi7fCzfxH/DiFYjmN5/Oi6OFa/dSUJB2LUpa5JwcXTg6a6NrxkXXM+DaY9Fsyslgw9XJfKv/x3g641HmdCtCQ+1CcDF6eb7/Q1GzdKEVD5Ze4ij5y4S6luT6Y9F0SPUR8JAlFpmXiaD5w1m7dG1vNXlLSZ1mVRh1x8JBWG3DpzKYtmukzzVpTF1aly/64hw/1rMGdWWrUfS+XDVQSYt2cv09Ud47t6mDIyoj1MJjcIGo2ZZwkk+WZvEkXMXCfGtybTHougpYSBu0YnME/T9vi8H0w/yzf3fMKL1CFuXdEeU1trWNdyS6OhoHRsba+syRDkYMyeWLUfS2fhyNzzdS9deoLVmXeJZ/rsqkd2pmTTyrs6LPZrRN8wXBweFwahZvssUBofPXqR5PQ+e796MnqE+ODhIGIhbszNtJ/1+6MfFwossHLqQexvda+uSrksptUNrHX2z6WRLQdilhOQMVu07zYs9mpU6EMB0iN89wXXp0sybX/ee4r+rEpnwQxyhvocZGFGfubHJHDqTQ7CPB188EkmvFvUkDMRtWZm0kqE/D6W2e202PbaJsLphti6pTEgoCLv04aqDeLk7M6pT0G09XylF7zBfeoTWY2lCKh+tTuK9lftp5lODzx+JpLeEgbgDX8Z+yfiV42ldrzXLhy3H16PyXDtDQkHYna1H0tmQdI5X+zanhuudraKODoqBEf70C/ezbCFIGIjbZdRG/r7IHITuAAAalUlEQVTm73yw+QP6Nu3L3CFzqeFSw9ZllSkJBWFXtNb8d1UidT1ceaxdYJnN19nRgRDfmmU2P1H15BXlMXLxSObuncu4qHFM7TsVJ4fK9xVa+d6RqNA2JJ1j27HzvHN/i9s630CIkmTlZ7EzbSfZ+dkUGAquuBUaC68ZVtIt4XQC8afi+Xf3fzOxw8RKe5SaVUNBKdUb+BhwBGZord+/znSDgflAG621HFpURWmt+XDVQerXqsZDbSpOtwDC/pzKOcWG4xvYcGIDG09sJOF0AkZtLPXzXRxdrrlVd67O3CFzGdpiqBUrtz2rhYJSyhH4DOgBpADblVJLtdb7rprOA3gO2GqtWkTFsHrfaXalZPLBkPBSnXgmBJh+TCSdT2LjiY1sOLGBDcc3cPjCYQDcnd1p59+ONzq/QXv/9tRxr3PFF72zo/M1X/6OyrHSbgWUhjW3FNoCh7TWRwCUUj8B9wP7rpruXeDfwEQr1iLsnNGombw6kUZ1qjMoor6tyxF2rMhYRMKpBFMAmLcEzlw8A0Ad9zp0CujEU9FPcXfDu4moF2HqgVSUmjVDoT6QXOxxChBTfAKlVCTQQGu9Qil13VBQSo0BxgAEBMhuhcpo+e40DpzK5pNhESWegSxsLyMvg5quNXFQ5fv55BTksDVlK5uTN7PhxAa2pGwhpyAHgMBagfRq3Iu7A+6mU0AnmtdpXqV/5ZcFmzU0K6UcgMnAyJtNq7WeDkwH0xnN1q1MlLcig5EpqxNpXs+Dfi0rz/HelUV6bjoTV09kVvwsPFw8iPKLoo1fG9r4tSHaL5rAWoFl9kWsteZE5gk2J29mU/ImNidvtrQHKBQtfVoyPHw4dzc0hYB/Tf8yeV3xF2uGQirQoNhjf/OwyzyAMOAP8wpVD1iqlBogjc1Vy8K4VI6cu8j0x6LkHAI7orXm+93f88KvL3Dh0gUmtJmAURuJTYvl460fU2AoAEy7bKL9oon2jaZNfVNYlPZkrkJDIXGn4ticvNkSBCezTwJQ3bk67fzb8drdr9GxQUdi/GOo5VbLau9XmFgzFLYDTZVSQZjC4CHg4csjtdaZQJ3Lj5VSfwB/k0CoWvKLDHy8JolW/p70CPWxdTnC7ND5Qzy14inWHFlDTP0Ypg+fTrhPuGV8gaGA3ad3s/3kdmJPxrL95Hb+dfhfGLQBAD8PP8uWxOW/td1rk56bzpaULWw6sYnNKZvZnrqdS0WXAGjo2ZAuDbvQsUFHOjToQEuflpXyPAB7Z7UlrrUuUkpNAH7FdEjqTK31XqXUO0Cs1nqptV5bVBzztieTmnGJfw1qKfuC7UCBoYAPN3/Iu+vfxcXRhc/6fsbYqLE4Olx5zoiLowtRflFE+UVZhuUW5hKXFmcJie0nt7Pk4BLL+LrV61oahJ0cnIj0jWRs1Fg6NOhAhwYdqF9TDjCwB9JLqrCZSwUGuvzndwJrV2fu2HYSCja26cQmxi4fy96zexkSOoSPe3+Mn4ffHc0zMy+THWk72J66nf3n9hNcO5iOAR2J9ovG3dm9jCoXpSG9pAq7992fxzmTnc/UYRESCDaUkZfBK2teYdqOaTSo2YBlw5bRr1m/Mpm3p5sn3YK60S2oW5nMT1ifhIKwiZz8Ir5Yd5i7m9YhplFtW5dTJWmtmbd3Hs/98hxnc8/yQrsXeKfrO5WugzdxayQUhE3M3HiU8xcL+FvPYFuXUiUdyzjG+JXjWZm0kijfKFY+spJI30hblyXsgISCKHcZuQV8tf4IPUN9aNVADjEsT0XGIqb8OYU3/3gTheKjXh8xoe0EOcpHWMiaIMrd9PVHyCko4sWezWxdSpWyLXUbY5aNIeF0AgOCB/Bpn09p4Nng5k8UVYqEgihX53LymbXpGP3D/Wher+pc3yAzL5NtqdvwqeFDUK0gPFw9rPZaBYYC9p/dz67Tu0y3M6a/p3JO4efhx4KhCxjYfKA07osSSSiIcvX574cpMBh5vntTW5didem56Sw5uIQF+xew+vBqCo2FlnF13OvQyKuR6Var0V/3vRrhX9P/mvMCSqK1Ji0n7a8vf/Nt/7n9FBmLAHB1dKVF3Rb0btKbiHoRjGg1Ak83T6u9Z1HxSSiIcpOWeYnvth5ncGR9GnlXziNcTuWcYvGBxczfN58/jv2BQRto6NmQZ9o+Q68mvcjMy+TIhSMcuXCEoxlH2Z66nfn75lu+xMF0YldgrUCCagVdERY+1X04dP4QCacTLAGQfind8rwGNRsQ7hNO/2b9CfcJJ9wnnKa1m0p7gbglsraIcjP1t0NorXmmW+XaSkjOTGbh/oUs2L+AjSc2otE0q92Mlzu+zOCQwUT6Rt5wV02RsYiUrBRLWBS/zd83/4ovfjBdI6Bl3ZYMChlk+fJvWbclXtW8rP1WRRUgoSDKxY7jF5i3PZmHYwJocFf5n8m67OAyXv3tVTxcPPCv6U+Dmg1Mfz0bWB7Xq1GvVLttAI5cOMKCfQtYsH8BW1NN14cKqxvGpC6TGBI6hBbeLUq9z/7ylkFgrcAST/LKzMvkaMZR0rLTaHJXExp5NSp1nULcKunmQlhVZm4h/1l1gO+3nsDHw42lEzpSt6ZbudYwJ2EOo5aMolntZvh6+JKcmUxKVoqlI7bLHJUjvh6+fwXGVcHh6ujKiqQVLNi/gPhT8QBE+UYxOGQwg0MH06y2HE0l7Jd0c3GV7PxsVh9ZzaCQQbYupUrQWrMoLpV/rtzP+YsFjOwQyIs9muHhVr5Xwfpoy0e8uOpF7g26l0X/t8hy1I/WmvOXzpOSlUJKVgrJWclX3E84ncDyxOXXBAdAe//2fNjjQwaFDCLIK6hc348Q1lZlQuGDTR/w3ob32P7k9it6dhRlL+l0Nq8v3sPWo+eJCKjF7FFtaeFXvke8aK154/c3eG/DewwKGcQPg37A1cnVMl4pRW332tR2r02req2uO48LeRdMQZGZTEZeBvcE3iO9eYpKrcrsPsrMyyT402ACawWy+YnN5X5Jwaogt6CIT9YeYsaGI1R3deKVPs35v+gG5X7hHIPRwPiV45m2YxqjI0bzZb8vZR+8qPJKu/uoynwzerp58p8e/2Fr6lZmxc2ydTmVitaaX/eeosfk9Xy57jCDIuvz20tdGNY2oNwDocBQwMMLH2bajmm80vEVpvefLoEgxC2oMlsKYPry6vxNZw6cO8DBCQe5q9pdZVxd1ZN8Ppe3lu5l7YEzBPt48I+BYbQJtM1yzSnIYfC8waw6vIr/9PgPf+vwN5vUIYQ9ki2FEiil+KzvZ1y4dIE3fnvD1uVUaPlFBj77/RA9PlrHliPpvNY3hOXPdrJZIKTnptN9TnfWHFnDzAEzJRCEuE1VpqH5snCfcMa3Gc/UbVN5IvIJ6S74Nmw6dI43luzhyNmL9G1Zjzf6heLrWc1m9aRmpdLzu54cPn+YBUMX8EDzB2xWixAVXZXaUrjs7a5v413dm/Erx2PURluXU2Gcycrj2R/jeGTGVooMmlmPt+HzR6JsGgiJ6Yl0nNmR5Mxk/vfI/yQQhLhDVW5LAaCWWy0+6P4BI5eMZHb8bB6PeNzWJVldWuYl5semkFtooMhgpNCgKTQYKTL/LTRqCouMFBmvGmc0Wu4nn8+l0KB59t6mPH1PY9ycbduAG5cWR6/veqHR/D7idznUWIgyUKUamoszaiOdZ3XmYPpBEickVup+YzYfOseEH+M4f7EAZ0eFk4MDzo4KZ0cHnMx/Tbfrj3NyUHi5uzDunsYE1alu67fEumPr6P9jf7yqebHq0VUE15EruAlxI3JG8004KAc+6/sZkdMjeeP3N/i076e2LqnMaa2Ztv4IH/xygEbeNZg3tj1N6lb83kmXHlzK0J+H0sirEb8++qtcKEaIMlRl2hT2p2Xx3Z/HOZWZZxnWql4rno5+mi9iv7D0ZVNZZOcVMu67Hbz/vwP0CfNl8fiOlSIQZsfPZtBcU++g6x9fL4EgRBmrMruPPlmbxOTViQCE1a9J9xAfuof4UP8uI8GfBtO0dlM2PL6hUpzpnHg6m3Hf7uD4+Vz+3qc5T3QKqvBX2dJa89GfH/HSqpfo3qg7C4cutOrVy4SobEq7+6jKhILWmkNncli9/zRr9p0mLjkDrcHX0406PltYnvwG0/t9zZNRo6xQdflZlnCS/7dgF+4ujnz6cCTtGtW2dUm3pdBQSNypODad2MTG5I1sOrGJ0xdPMyR0CN8N/O6KfoyEEDcnoXAT53Ly+e3AGdbuP826xDMcUy9R5JDGY4GL6RvWmG7N61K7RsX54ik0GHn/fwf4euNRohp68dnDkdTzLN8uqu9EZl4mf6b8ycYTG9mUvImtqVvJLcwFIKhWEJ0COtE1sCvDWw2XbiuEuA0SCrcgr9DAnNg/GLeqJ3UdBuB2cTRKQVSAF/eG+NAjtC6NvWvY7S6YM9l5TPg+jm3HzjOyQyCv9g3Bxcm+d4MlZyZbAmDjiY3sOr0LjcZBORBRL4KODTrSKaATHQM64ufhZ+tyhajwJBRuw/gV4/lyx5f8dP8fpJzxYc3+0+w9mQVAw9ru9G3py1P3NKZmOV8T4Ea2HzvP+O93kpVXyPuDwnkgwv66dc4tzGX36d3Enoy1hEByVjIANVxq0M6/HZ0adKJTQCdi/GOo4VLxG8SFsDcSCrfhwqULNPu0GcG1g9nw+AaUUpzMuMTaA2dYs+80G5LO4lPTjX8OaknX4LpWqaG0tNZ8s/kY763Yj79XNb54NIoQ35o2rQng7MWzxJ+KJ+5UnOVvYnqi5cxxPw8/OgV0olMD01ZAuE+4XFheiHIgoXCbZsbN5ImlTzD7gdkMbzX8inHxyRlM/DmBpDM5DI70Z1K/UDzdy3+rIbegiFcW7GZpwkm6h/jw36Gt8KxWvnVorTly4Qjxp+KvCIHU7FTLNAGeAbSu15rWPq2J8I0gol4EAZ4BdrsbTojKTELhNhm1kQ5fd+BoxlEOTjhILbdaV4zPLzIwde0hvlh3mLuqu/DeA2H0bFHPavVc7ei5i4z7dgeJZ7L5W89gnurSuFyuWXAi8wS/Hf2NuLQ44k7FkXA6gax80641R+VI8zrNifCNsARAK59W1HavmEc+CVEZSSjcgZ1pO4meHs0zbZ/h4z4flzjNntRMJs7fxf60LPq38uPtAS24q7qLVetatfcUL81LwMlR8fFDEXRu5m211yoyFrEleQsrklawImkFe87sAcDd2Z1WPq1oXa81EfUiaF2vNWF1w6jmbLtO8YQQN2cXoaCU6g18DDgCM7TW7181fhwwHjAAOcAYrfW+G82zPEIB4OkVTzNtxzTixsYR7hNe4jQFRUa+XHeYqb8lUdPNmbfvb8F9LX3LdPdIXqGBX/eeYsHOVNYnniXc35PPH4nE38u9zF7jsvTcdH459Asrklbwy6FfuJB3AScHJzoFdOK+pvfRu0lvQuqEyCGhQlRANg8FpZQjkAj0AFKA7cCw4l/6SqmaWuss8/0BwNNa6943mm95hcL5S+dpNrUZId4hrB+5/oZf9AdOZfHy/F3sSsmkd4t6vPNAC+p63P45AlprYo9fYMGOFFbsSiM7v4j6tarxYLQ/47qUXe+kWmt2n9nNisQVLE9azp8pf2LURrzdvenTtA/9mvajZ+OeeLp5lsnrCSFsxx46xGsLHNJaHzEX9BNwP2AJhcuBYFYdsJt9WXdVu4v3u7/Pk8ue5Ltd3/FYq8euO23zejVZ+FQHvtpwlI/WJPLnR+m82T+UB1rXv6WthuTzuSzcmcrCuBSOp+fi7uJInzBfBkfVJybwLi7kn+dSURYaN1ydXG+rS47cwlx+O/obyxOXszJppeXQ0EjfSF67+zXua3ofbeq3qRTdfQghbp01txSGAL211qPNjx8DYrTWE66abjzwIuACdNNaJ91ovuW1pQCmRuf2X7fneMZxDk44WKpfzIfO5PDy/AR2nsjg3uZ1eW9gyxueWZyTX8TK3Wks2JHC1qPnAejQuDaDI/3p1cKHg+d3MX/ffObvn8+h84eueK6TgxNuTm64Orri6uSKq6Or6fF17mfmZ7L++HryivKo4VKDHo16cF/T++jTtI+cICZEJWcPu49KFQrFpn8Y6KW1HlHCuDHAGICAgICo48ePW6XmksSejKXtV215LuY5Pur9UameYzCaziH4z68HcHZw4PV+IQyNbmDZajAaNVuOpDN/Rwq/7DnFpUIDgbXdGRzpzwMRfpzM3WMJgmMZx3BUjtzb6F56NuqJo4MjeUV55Bflk2/Iv/a+If+645wdnOkW1I37mt5H54adpf8gIaoQewiF9sBbWute5sd/B9Ba/+s60zsAF7TWN/w5Xp5bCpeNWz6OGTtnEDc2jpY+LUv9vGPnLvLygl1sO3qeu5vWYULXJqxPOsuinamczMzDw82JfuF+DIr0JV8dYMH+BSzYv4DkrGScHZzp0bgHQ0KGMCB4gBzeKYS4I/YQCk6YGprvBVIxNTQ/rLXeW2yappd3Fyml+gNv3qxoW4RCem46zT5tRmCtQEa1HkX9mvXxr+lPfY/61K1e94ZH4xiNmu+3Hudf/ztAboEBBwWdm3nzQIQv1WsksTRxEQsPLORk9klcHF3o3aQ3Q0KG0D+4/zXnSAghxO2yeSiYi+gLTMF0SOpMrfV7Sql3gFit9VKl1MdAd6AQuABMKB4aJbFFKAD8tOcnRi4eSb4h/4rhjsoRXw9f6nv8FRT1a9a/5m96Nmw6dBon9wOsPbaUhQcWcubiGdyc3OjTpA9DQofQr1k/arravqsKIUTlYxehYA22CgUwNTyfuXiGlKwUUrNSSc1OtfxNyUqxPM4uyL7muV5uXmg0GXkZuDu7c1/T+xgSOoS+TftKB3BCCKuzh0NSKx0H5UC9GvWoV6Me0X7XX7ZZ+VnXhEZqVir5hnz6Nu1L7ya9cXcu+5PPhBDiTkkoWEFN15rU9K5JiHeIrUsRQohbImcoCSGEsJBQEEIIYSGhIIQQwkJCQQghhIWEghBCCAsJBSGEEBYSCkIIISwkFIQQQlhUuG4ulFJngfLrO9s+1QHO2boIG5NlIMvgMlkOpVsGDbXWN72we4ULBQFKqdjS9GFSmckykGVwmSyHsl0GsvtICCGEhYSCEEIICwmFimm6rQuwA7IMZBlcJsuhDJeBtCkIIYSwkC0FIYQQFhIKdkYp1UAp9btSap9Saq9S6jnz8LuUUquVUknmv17m4Uop9YlS6pBSapdSKtK276DsKKUclVJxSqnl5sdBSqmt5vc6VynlYh7uan58yDw+0JZ1lyWlVC2l1Hyl1AGl1H6lVPuqti4opV4w/y/sUUr9qJRyqwrrglJqplLqjFJqT7Fht/zZK6VGmKdPUkqNuNnrSijYnyLgJa11KNAOGK+UCgVeAdZqrZsCa82PAfoATc23McAX5V+y1TwH7C/2+N/AR1rrJpiu6f2EefgTwAXz8I/M01UWHwO/aK2bA60wLY8qsy4opeoDzwLRWuswTNd7f4iqsS58A/S+atgtffZKqbuAN4EYoC3w5uUguS6ttdzs+AYsAXoABwFf8zBf4KD5/jRgWLHpLdNV5Bvgb17puwHLAYXp5Bwn8/j2wK/m+78C7c33nczTKVu/hzJYBp7A0avfS1VaF4D6QDJwl/mzXQ70qirrAhAI7Lndzx4YBkwrNvyK6Uq6yZaCHTNv+kYAWwEfrXWaedQpwMd8//I/zWUp5mEV3RTgZcBoflwbyNBaF5kfF3+flmVgHp9pnr6iCwLOArPMu9FmKKWqU4XWBa11KvAhcAJIw/TZ7qDqrQuX3epnf8vrhISCnVJK1QAWAM9rrbOKj9OmyK+0h40ppfoBZ7TWO2xdi405AZHAF1rrCOAif+0uAKrEuuAF3I8pIP2A6ly7S6VKstZnL6Fgh5RSzpgC4Xut9ULz4NNKKV/zeF/gjHl4KtCg2NP9zcMqso7AAKXUMeAnTLuQPgZqKaWczNMUf5+WZWAe7wmkl2fBVpICpGitt5ofz8cUElVpXegOHNVan9VaFwILMa0fVW1duOxWP/tbXickFOyMUkoBXwP7tdaTi41aClw+cmAEpraGy8OHm48+aAdkFtu8rJC01n/XWvtrrQMxNSr+prV+BPgdGGKe7OplcHnZDDFPX+F/PWutTwHJSqlg86B7gX1UoXUB026jdkopd/P/xuVlUKXWhWJu9bP/FeiplPIyb3X1NA+7Pls3pMjtmoalTpg2CXcB8eZbX0z7RdcCScAa4C7z9Ar4DDgM7MZ0lIbN30cZLo97gOXm+42AbcAh4GfA1Tzczfz4kHl8I1vXXYbvvzUQa14fFgNeVW1dAN4GDgB7gG8B16qwLgA/YmpHKcS01fjE7Xz2wCjz8jgEPH6z15UzmoUQQljI7iMhhBAWEgpCCCEsJBSEEEJYSCgIIYSwkFAQQghhIaEgxHUopV4z9865SykVr5SKUUo9r5Ryt3VtQliLHJIqRAmUUu2BycA9Wut8pVQdwAXYjOkY8HM2LVAIK5EtBSFK5guc01rnA5hDYAim/nd+V0r9DqCU6qmU2qKU2qmU+tncZxVKqWNKqQ+UUruVUtuUUk3Mwx80XxcgQSm13jZvTYjrky0FIUpg/nLfCLhjOnN0rtZ6nbk/pmit9Tnz1sNCoI/W+qJS6v9hOrP2HfN0X2mt31NKDQeGaq37KaV2A7211qlKqVpa6wybvEEhrkO2FIQogdY6B4jCdMGSs8BcpdTIqyZrB4QCm5RS8Zj6omlYbPyPxf62N9/fBHyjlHoS0wVjhLArTjefRIiqSWttAP4A/jD/wr/6UoYKWK21Hna9WVx9X2s9TikVA9wH7FBKRWmtK1MvnqKCky0FIUqglApWSjUtNqg1cBzIBjzMw/4EOhZrL6iulGpW7Dn/V+zvFvM0jbXWW7XWkzBtgRTv1lgIm5MtBSFKVgOYqpSqhem62Ycw7UoaBvyilDqpte5q3qX0o1LK1fy814FE830vpdQuIN/8PID/mMNGYertMqFc3o0QpSQNzUJYQfEGaVvXIsStkN1HQgghLGRLQQghhIVsKQghhLCQUBBCCGEhoSCEEMJCQkEIIYSFhIIQQggLCQUhhBAW/x96cXiBYECZ4wAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "l2_val_loss = np.array(l2_model.get_validation_summary(\"Loss\"))\n",
+    "\n",
+    "plt.plot(original_val_loss[:,0], original_val_loss[:,1], label='original model')\n",
+    "plt.plot(l2_val_loss[:,0], l2_val_loss[:,1],label='L2-regularized model',color='green')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, the model with L2 regularization (dots) has become much more resistant to overfitting than the reference model (crosses), \n",
+    "even though both models have the same number of parameters.\n",
+    "\n",
+    "As alternatives to L2 regularization, you could use one of the following Analytics-zoo Keras API weight regularizers: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras import regularizers\n",
+    "\n",
+    "# L1 regularization\n",
+    "regularizers.l1(0.001)\n",
+    "\n",
+    "# L1 and L2 regularization at the same time\n",
+    "regularizers.l1_l2(l1=0.001, l2=0.001)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Adding dropout\n",
+    "\n",
+    "\n",
+    "Dropout is one of the most effective and most commonly used regularization techniques for neural networks, developed by Hinton and his \n",
+    "students at the University of Toronto. Dropout, applied to a layer, consists of randomly \"dropping out\" (i.e. setting to zero) a number of \n",
+    "output features of the layer during training. Let's say a given layer would normally have returned a vector `[0.2, 0.5, 1.3, 0.8, 1.1]` for a \n",
+    "given input sample during training; after applying dropout, this vector will have a few zero entries distributed at random, e.g. `[0, 0.5, \n",
+    "1.3, 0, 1.1]`. The \"dropout rate\" is the fraction of the features that are being zeroed-out; it is usually set between 0.2 and 0.5. At test \n",
+    "time, no units are dropped out, and instead the layer's output values are scaled down by a factor equal to the dropout rate, so as to \n",
+    "balance for the fact that more units are active than at training time.\n",
+    "\n",
+    "Consider a Numpy matrix containing the output of a layer, `layer_output`, of shape `(batch_size, features)`. At training time, we would be \n",
+    "zero-ing out at random a fraction of the values in the matrix:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This technique may seem strange and arbitrary. Why would this help reduce overfitting? Geoff Hinton has said that he was inspired, among \n",
+    "other things, by a fraud prevention mechanism used by banks -- in his own words: _\"I went to my bank. The tellers kept changing and I asked \n",
+    "one of them why. He said he didn’t know but they got moved around a lot. I figured it must be because it would require cooperation \n",
+    "between employees to successfully defraud the bank. This made me realize that randomly removing a different subset of neurons on each \n",
+    "example would prevent conspiracies and thus reduce overfitting\"_.\n",
+    "\n",
+    "The core idea is that introducing noise in the output values of a layer can break up happenstance patterns that are not significant (what \n",
+    "Hinton refers to as \"conspiracies\"), which the network would start memorizing if no noise was present. \n",
+    "\n",
+    "In Analytics-zoo Keras API you can introduce dropout in a network via the `Dropout` layer, which gets applied to the output of layer right before it, e.g.:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.add(layers.Dropout(0.5))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's add two `Dropout` layers in our IMDB network to see how well they do at reducing overfitting:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDropout\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDropout\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "dpt_model = models.Sequential()\n",
+    "dpt_model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
+    "dpt_model.add(layers.Dropout(0.5))\n",
+    "dpt_model.add(layers.Dense(16, activation='relu'))\n",
+    "dpt_model.add(layers.Dropout(0.5))\n",
+    "dpt_model.add(layers.Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "dpt_model.compile(optimizer='rmsprop',\n",
+    "                  loss='binary_crossentropy',\n",
+    "                  metrics=['acc'])\n",
+    "\n",
+    "dir_name = '4-4 ' + str(time.ctime())\n",
+    "dpt_model.set_tensorboard('./', dir_name)\n",
+    "dpt_model.fit(x_train, y_train,\n",
+    "              nb_epoch=20,\n",
+    "              batch_size=512,\n",
+    "              validation_data=(x_test, y_test))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 512 records in 0.017992654 seconds. Throughput is 28456.057 records/second. Loss is 0.112769656. \n",
+    "Top1Accuracy is Accuracy(correct: 21871, count: 25000, accuracy: 0.87484)_"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XdYVMf6wPHvUARBpNgjKtgLigUVey+JLSYaxdhiNzH1xpR7U031JvEaYxI1Ro3dWGMS04wx9gKKDQuIKFhRBAWk7vz+YN0fMRhRWc7Cvp/n2Yc9Z2d33z2c3ffMmTkzSmuNEEIIAeBgdABCCCFshyQFIYQQFpIUhBBCWEhSEEIIYSFJQQghhIUkBSGEEBaSFIQQQlhIUhBCCGEhSUEIIYSFk9EB3K2yZctqPz8/o8MQQogiJSws7LLWutydyhW5pODn50doaKjRYQghRJGilDqdn3Jy+kgIIYSFJAUhhBAWkhSEEEJYFLk2hbxkZmYSFxdHWlqa0aEIcVuurq74+vri7OxsdChC3FaxSApxcXF4eHjg5+eHUsrocIT4G601V65cIS4uDn9/f6PDEeK2isXpo7S0NMqUKSMJQdgspRRlypSR2qywecUiKQCSEITNk31UFAXFJikIIURxlW3SvPdjBGcTb1j9vSQpFLKHHnqIxMTEfyzzxhtvsHHjxnt6/c2bN9O7d+97em5+xcTEEBAQcN9lhBB3prXmtXWH+GrrKf48Hm/19ysWDc1FgdYarTUbNmy4Y9kpU6YUQkRCiKLgo1+Os2xPLJM61WRIy6pWfz+pKRSQadOmERAQQEBAANOnTwdyjpbr1KnD8OHDCQgIIDY2Fj8/Py5fvgzAO++8Q506dWjbti0hISF8/PHHAIwcOZJVq1YBOcN6vPnmmzRt2pSGDRty7NgxAPbs2UOrVq1o0qQJrVu35vjx4/8Y34IFC3j44Yfp1q0bfn5+zJw5k2nTptGkSROCg4NJSEgAIDw8nODgYBo1akT//v25evUqAGFhYQQGBhIYGMjnn39ued3s7GwmT55M8+bNadSoEbNnzy7ArSqEfftqSzRfbD7JkJZV+Vf32oXynsWupvD290eIOHetQF+z/gOlebNPg9s+HhYWxvz589m9ezdaa1q2bEmHDh3w9vYmMjKSb775huDg4L88Z+/evaxevZoDBw6QmZlJ06ZNadasWZ6vX7ZsWfbt28cXX3zBxx9/zNy5c6lbty5bt27FycmJjRs38u9//5vVq1f/4+c4fPgw+/fvJy0tjZo1azJ16lT279/P888/z8KFC3nuuecYPnw4n332GR06dOCNN97g7bffZvr06TzxxBPMnDmT9u3bM3nyZMtrfv3113h6erJ3717S09Np06YN3bt3l0ZVIe7TqrA43ttwlF4NK/FOv4BC+05JTaEAbNu2jf79++Pu7k6pUqV45JFH2Lp1KwDVqlX7W0IA2L59O/369cPV1RUPDw/69Olz29d/5JFHAGjWrBkxMTEAJCUlMXDgQAICAnj++ec5cuTIHePs1KkTHh4elCtXDk9PT8t7NmzYkJiYGJKSkkhMTKRDhw4AjBgxgi1btpCYmEhiYiLt27cHYNiwYZbX/PXXX1m4cCGNGzemZcuWXLlyhcjIyHxsNSHE7fwWcZGXVx+kXa2yTBsUiKND4R1kFbuawj8d0RvB3d39vl/DxcUFAEdHR7KysgB4/fXX6dSpE2vXriUmJoaOHTvm+3UAHBwcLMsODg6W171bWms+++wzevTo8Zf1N5OXEOLu7Iq+wlNL9xFQ2ZNZQ5vh4uRYqO8vNYUC0K5dO9atW0dqaiopKSmsXbuWdu3a/eNz2rRpw/fff09aWhrJycn88MMPd/WeSUlJVK5cGchpLygInp6eeHt7W2o5ixYtokOHDnh5eeHl5cW2bdsAWLJkieU5PXr04MsvvyQzMxOAEydOkJKSUiDxCGFvDp9NYuw3oVT1cWP+yOa4uxT+cXuxqykYoWnTpowcOZIWLVoAMGbMGJo0afKPR8vNmzenb9++NGrUiAoVKtCwYUM8PT3z/Z4vvfQSI0aM4N1336VXr173+xEsvvnmGyZMmEBqairVq1dn/vz5AMyfP59Ro0ahlKJ79+6W8mPGjCEmJoamTZuitaZcuXKsW7euwOIRwl6cupzCyPl7KF3SmUWjW+DjXsKQOJTW2pA3vldBQUH61kl2jh49Sr169QyK6N4lJydTqlQpUlNTad++PXPmzKFp06ZGhyWsqKjuq8K6Ll5L49Evd5Cakc3KCa2oUa5Ugb+HUipMax10p3JSUzDQuHHjiIiIIC0tjREjRkhCEMIOJaZmMPzrPVxNyWD5OOskhLshScFAS5cuNToEIYSBUjOyGLVgL6cup7BgVHMa+ub/FLK1SEOzEEIYICPLxMTF+wiPTWRGSBNa1yhrdEiA1BSEEKLQmUyaf608wJ8n4pn6aEN6BlQ0OiQLq9YUlFI9lVLHlVJRSqlX8ni8qlLqD6XUfqXUQaXUQ9aMRwghjKa15q3vj/D9gXO88mBdBjW3/nhGd8NqSUEp5Qh8DjwI1AdClFL1byn2GvCt1roJMBj4wlrxCCGELZi+MZKFO08zvn11JnSoYXQ4f2PNmkILIEprHa21zgCWA/1uKaOB0ub7nsA5K8ZjVY6OjjRu3JgGDRoQGBjIJ598gslkMiyedevWERERYch7lyp1970n8jOk+J3IsOHC1n2zI4ZPf4/ksSBfXnmwrtHh5MmabQqVgdhcy3FAy1vKvAX8qpR6GnAHuloxHqsqWbIk4eHhAFy6dIkhQ4Zw7do13n777b+Uy8rKwsnJ+k0569ato3fv3tSvf2vl7K9uDunt4GBMn4O7GVJciKLsu/CzvLn+CN3rV+D9/g1tdtBIo3sfhQALtNa+wEPAIqXU32JSSo1TSoUqpULj460/ycT9Kl++PHPmzGHmzJlorVmwYAF9+/alc+fOdOnSBa01kydPJiAggIYNG7JixQog50i3ffv29OrVizp16jBhwgRLbWPZsmU0bNiQgIAAXn75Zct75T4qX7VqFSNHjmTHjh2sX7+eyZMn07hxY06ePPmX+PIa0vvXX3+lVatWNG3alIEDB5KcnAzAhg0bqFu3Ls2aNeOZZ56xHIm/9dZblqG+AQICAv52BXdycjJdunSxDPv93Xff3fb9bw4pPmvWLBo3bkzjxo3x9/enU6dOALeN7+eff6Zu3bo0bdqUNWvW5Pn/kGHDhVFiLqfw5eaT9Ju5jWeXhxNc3YcZIU1wcjT6p/f2rHnIehaokmvZ17wut9FATwCt9U6llCtQFriUu5DWeg4wB3KuaP6nN33u5+cIvxB+f5HfonHFxkzvOf2unlO9enWys7O5dCnno+zbt4+DBw/i4+PD6tWrCQ8P58CBA1y+fJnmzZtbRiDds2cPERERVKtWjZ49e7JmzRpat27Nyy+/TFhYGN7e3nTv3p1169bx8MMP5/nerVu3pm/fvvTu3ZsBAwbkWSb3kN6XL1/m3XffZePGjbi7uzN16lSmTZvGSy+9xPjx49myZQv+/v6EhITc1TZwdXVl7dq1lC5dmsuXLxMcHEzfvn3/9v65TZgwgQkTJpCZmUnnzp154YUX/jG+sWPHsmnTJmrWrMmgQYNuG4sMGy4KS+TF62w4dIGfDp/n2IXrAAT6evLKg3UZFlwNV+fCHeDublkzKewFaiml/MlJBoOBIbeUOQN0ARYopeoBroDtVwXuQbdu3fDx8QFyhtoOCQnB0dGRChUq0KFDB/bu3Uvp0qVp0aIF1atXByAkJIRt27bh7OxMx44dKVeuHACPP/44W7ZsuW1SyI/cQ3rv2rWLiIgI2rRpA0BGRgatWrXi2LFjVK9eHX9/f0s8c+bMyfd7aK3597//zZYtW3BwcODs2bNcvHjxb++fl2effZbOnTvTp08ffvjhh9vG5+/vT61atQAYOnTobeO7OWy4h4fH34YNP3jwYJ7Dhg8cODDPYcN/+uknIKf2cvDgQcuESElJSURGRlK7duFMhiJsg9aaI+eu8fPhnERwMj4FpSComjev965Pz4CKVPYqaXSY+Wa1pKC1zlJKTQJ+ARyBeVrrI0qpKUCo1no98C/gK6XU8+Q0Oo/U9zkY090e0VtLdHQ0jo6OlC9fHsj/ENq3HmXe6agz9+NpaWl5lomNjbX8CE6YMIGePXv+JR6tNd26dWPZsmV/ed7NNpK8ODk5/aUhPa/3XrJkCfHx8YSFheHs7Iyfn5+l3D9tjwULFnD69Glmzpx5z/HdSoYNFwVJa014bKI5EVzgTEIqDgqCq5dhZGs/ejSoSPnSrkaHeU+semJLa71Ba11ba11Da/2eed0b5oSA1jpCa91Gax2otW6stf7VmvEUlvj4eCZMmMCkSZPy/FFv164dK1asIDs7m/j4eLZs2WIZYXXPnj2cOnUKk8nEihUraNu2LS1atODPP//k8uXLZGdns2zZMssRbYUKFTh69Cgmk4m1a9da3sPDw4Pr13OqrlWqVCE8PJzw8HAmTJjwt3iCg4PZvn07UVFRAKSkpHDixAnq1KlDdHS05UfuZtsH5EwTum/fPiDn1NipU6f+9rpJSUmUL18eZ2dn/vjjD06fPn3HbRcWFsbHH3/M4sWLLY3ft4uvbt26xMTEWNpMbk0ad0OGDRd3km3S7I6+wlvrj9D6w030/2IH87afwr+sOx8+0pC9/+nK0rHBDGvlV2QTAsgVzQXmxo0bNG7cmMzMTJycnBg2bBgvvPBCnmX79+/Pzp07CQwMRCnFf//7XypWrMixY8do3rw5kyZNIioqik6dOtG/f38cHBz48MMP6dSpE1prevXqRb9+Ob17P/zwQ3r37k25cuUICgqyNMAOHjyYsWPHMmPGDFatWkWNGrfvD12uXDkWLFhASEgI6enpALz77rvUrl2bL774wlKzaN68ueU5jz76KAsXLqRBgwa0bNkyz1Mmjz/+OH369KFhw4YEBQVRt+6du+DNnDmThIQESwNzUFAQc+fOvW18c+bMoVevXri5udGuXTtLIrwXMmy4uJ346+k8+uUOziSkUsLJgQ61yzG5Rx261KuAZ0lno8MrUDJ0tg3ZvHkzH3/88V1PuGNNN4f31lrz1FNPUatWLZ5//nmjwyqyisu+am8+2HCUr7ZG88ljgXSrX5FSBkx+c7/yO3S27faLEjbhq6++slyUl5SUxPjx440OSYhClZCSwaJdp+kb+AD9m/gWyYRwN4r3pytiOnbsmK+5lgvT888/LzUDYdfmbTvFjcxsnupU0+hQCkWxqSkUtdNgwv7IPlr0JN3I5JsdMTwYUJFaFTyMDqdQFIuk4OrqypUrV+RLJ2yW1porV67g6lp0e6XYowXbY7iensWkTrWMDqXQFIvTR76+vsTFxVEUhsAQ9svV1RVfX1+jwxD5dD0tk3nbT9G1XgXqP1D6zk8oJopFUnB2drZcdSuEEAVh8a4zJN3I5Jku9tGWcFOxOH0khBAFKTUji7lbo+lQuxyNfL2MDqdQSVIQQohbLN19hispGXZXSwBJCkII8RdpmdnM2RJNq+plaFbNx+hwCp0kBSGEyOXb0FguXU/naTusJYAkBSGEsMjIMjFr80mCqnnTqnoZo8MxhCQFIYQwW70vjnNJaTzdpZbdTpYkSUEIIYCsbBNfbI4i0NeT9rXKGh2OYSQpCCEE8F34OWITbjCps/3WEkCSghBCkG3SfP5HFPUqlaZrvfJGh2MoSQpCCLv346HzRF9O4enONe26lgCSFIQQds5k0szcFEmt8qXo2aCi0eEYTpKCEMKu/RpxgRMXk5nUuSYODvZdSwBJCkIIO6a15rNNUfiXdad3oweMDscmSFIQQtitP45f4si5a0zsWANHqSUAkhSEEHZKa82M36Pw9S5J/yaVjQ7HZkhSEELYpW1RlwmPTWRixxo4O8pP4U2yJYQQdumz36Oo5OnKgGYyG15ukhSEEHZnV/QV9sQkML59dVycHI0Ox6ZYNSkopXoqpY4rpaKUUq/k8fj/lFLh5tsJpVSiNeMRQgiAzzZFUraUC4NbVDU6FJtjtTmalVKOwOdANyAO2KuUWq+1jrhZRmv9fK7yTwNNrBWPEEIAhJ2+yvaoK/z7obq4Okst4VbWrCm0AKK01tFa6wxgOdDvH8qHAMusGI8QQjBzUyTebs483rKa0aHYJGsmhcpAbK7lOPO6v1FKVQP8gU1WjEcIYecOxSXxx/F4xrSrjruL1U6UFGm20tA8GFiltc7O60Gl1DilVKhSKjQ+Pr6QQxNCFBefbYqktKsTw1tJLeF2rJkUzgJVci37mtflZTD/cOpIaz1Hax2ktQ4qV65cAYYohLAXR89f49eIizzRxh8PV2ejw7FZ1kwKe4FaSil/pVQJcn74199aSClVF/AGdloxFiGEnZv5RxTuJRx5oo2f0aHYNKudVNNaZymlJgG/AI7APK31EaXUFCBUa30zQQwGlmuttbViEULYr8Nnk5i7NZoNh84zoUMNvNxKGB2STbNqS4vWegOw4ZZ1b9yy/JY1YxBC2B+tNVsiLzNny0m2R13BrYQjT7T25+nONY0OzeZJ87sQotjIyDKx/sA5vtoSzfGL1ynv4cLLPesypEVVPN2kHSE/JCkIIYq8pBuZLN19hgU7TnHxWjp1Knjw8cBA+gY+QAknW+lkWTRIUhBCFFlxV1OZty2GFXvPkJKRTduaZfnvgEDa1ypr93Mt3ytJCkKIIudQXBJzzI3HCugT+ABj2vnT4AFPo0Mr8iQpCCGKBJNJs/nEJeZsiWZXdAKlXJwY3dafka39eMCrpNHhFRuSFIQQNi82IZUx34Ry/OJ1Knm68p+H6jGoRRVKy0VoBU6SghDCpl1JTmf4vD0kpGTwv0GB9G70gMyUZkWSFIQQNislPYtRC/ZyLvEGS8e2pFk1H6NDKvYkKQghbFJGlomJS/Zx+Nw1Zg9tJgmhkEgdTAhhc0wmzcurD7LlRDzv9w+ga/0KRodkNyQpCCFszoc/H2Pt/rNM7lGHQc1lyszCJElBCGFTvtoSzZwt0YxoVY0nO9YwOhy7I0lBCGEz1u6P470NR+nVsBJv9GkgVyUbQJKCEMImbD5+ickrD9K6RhmmDQrE0UESghEkKQghDHcgNpEnl+yjdgUPZg9rhouTo9Eh2S1JCkIIQ0XHJ/PEgr2UKVWCBaOay1SZBpOkIIQwzKVraQyftwcFLBrVkvIerkaHZPfk4jUhhCGupWVahq9YPi4Yv7LuRockkJqCEMIAaZnZjP0mlJPxycwe1oxGvl5GhyTMpKYghChU2SbN8yvC2X0qgU8HN6ZdrXJGhyRykZqCEKLQaK15a/0Rfjp8gdd716df48pGhyRuIUlBCFFoPtsUxaJdpxnfoTqj2/obHY7IgyQFIUShWLbnDNN+O8EjTSvzSs+6RocjbkPaFIQQVmUyaWZvieajX47RsU45pj7aSIavsGGSFIQQVpOYmsG/vj3A78cu0athJT4a2EhmTbNxkhSEEFZxc+iKS9fTeLtvA4a3qiY1hCLAqilbKdVTKXVcKRWllHrlNmUeU0pFKKWOKKWWWjMeIYT1aa1ZuDOGgbN2ArByQmtGtPaThFBEWK2moJRyBD4HugFxwF6l1HqtdUSuMrWAV4E2WuurSqny1opHCGF9yelZvLrmEN8fOEfnuuWZ9lggXm4ljA5L3AVrnj5qAURpraMBlFLLgX5ARK4yY4HPtdZXAbTWl6wYjxDCio5duMaTi/cRcyWFl3rWYUL7GjjI8NdFjjWTQmUgNtdyHNDyljK1AZRS2wFH4C2t9c9WjEkIYQWrwuJ4bd0hPFydWTo2mODqZYwOSdwjoxuanYBaQEfAF9iilGqotU7MXUgpNQ4YB1C1qszXKoStSMvM5s3vjrAiNJZW1cvwaUhjGem0iLNmUjgLVMm17Gtel1scsFtrnQmcUkqdICdJ7M1dSGs9B5gDEBQUpK0WsRAi305dTuHJJfs4ev4akzrV5PlutWW2tGLAmr2P9gK1lFL+SqkSwGBg/S1l1pFTS0ApVZac00nRVoxJCFEAfjp0nj6fbeN80g3mP9GcF3vUkYRQTOSrpqCUqgHEaa3TlVIdgUbAwltP8+Smtc5SSk0CfiGnvWCe1vqIUmoKEKq1Xm9+rLtSKgLIBiZrra/c30cSQlhLRpaJD346yvztMTSu4sXnjzelsldJo8MSBUhpfeezMUqpcCAI8AM2AN8BDbTWD1k1ujwEBQXp0NDQwn5bIeze2cQbPLVkH+GxiYxq488rD9alhJNcnVxUKKXCtNZBdyqX3zYFk/nIvz/wmdb6M6XU/vsLUQhRFGit+eHgeV7/7jDZ2ZovH2/Kgw0rGR2WsJL8JoVMpVQIMALoY14ns2sLUcxdup7G6+sO88uRiwT6ejJ9cBP8ZdrMYi2/SeEJYALwntb6lFLKH1hkvbCEEEbSWvNd+Dne+v4IqRnZvPJgXca09cdJBrMr9vKVFMxDUzwDoJTyBjy01lOtGZgQwhgXr6Xxn7WH2Hj0Ek2revHfAYHULF/K6LBEIclv76PNQF9z+TDgklJqu9b6BSvGJoQoRFprVu87y5Tvj5CeZeK1XvV4oo2/dDW1M/k9feSptb6mlBpDTlfUN5VSB60ZmBCi8JxPusGraw6x+Xg8zf28+e+AQGk7sFP5TQpOSqlKwGPAf6wYjxCiEGmtWbE3lvd+PEqWSfNmn/qMaOUnA9nZsfwmhSnkXGi2XWu9VylVHYi0XlhCCGuLu5rKq2sOsTXyMsHVfZj6aCOqlZHagb3Lb0PzSmBlruVo4FFrBSWEsB6TSbN0zxk+2HAUDbzTrwGPt6wmtQMB5L+h2Rf4DGhjXrUVeFZrHWetwIQQBS82IZWXVh1kZ/QV2tQsw4ePNKKKj5vRYQkbkt/TR/OBpcBA8/JQ87pu1ghKCFGwTCbNol2nmfrzMRyU4v3+DQlpUUWmyCwEaVlpXL1xlaT0JNyc3fBy9aJUiVI4KNu85iO/SaGc1np+ruUFSqnnrBGQEKJgJaVm8q+V4Ww8eon2tcvxwSMNZRC7e5CUlsS56+e4mnaVqzeucjXtKgk3Eiz3b7c+LSvtb6/loBzwdPHEy9ULT9ecv5abi9dfl3PdqnlVw8vVy6qfM79J4YpSaiiwzLwcAshopkLYuMNnk5i4JIzziWm80bs+T7Txk9rBPZi/fz7jfxhPpikzz8c9SnjgXdIbb1dvfEr6ULdsXbxdc5a9S+asK+1SmrSsNBLTEvO8nUw4abl/PeN6nu/z+UOf82TzJ635UfOdFEaR06bwP0ADO4CRVopJCFEAVuw9w+vfHcHHrQQrxgfTrJqP0SEVOSZt4vVNr/P+tvfpWr0roxqP+suPv3dJb7xcvXByKNj5yrJMWVxLv/a3xBFYIbBA3ycv+e19dJqcK5otzKePplsjKCHEvUvLzOb1dYdZGRZHm5pl+HRwE8qWcjE6rCInLSuNketGsuLICsY0GcMXvb7A2bFwxgF1cnDCp6QPPiULP5HfT3p7AUkKQtiUmMspTDRPkfl055o811WmyLwX8SnxPLziYXbE7mBq16lMbj3Zbk673U9SsI8tJEQR8cuRC7y48gAOSjFvZBCd61YwOqQi6fjl4/Ra2ouz18+ycuBKBtQfYHRIhep+ksKdp2wTQlhdVraJj349zuw/o2lY2ZMvHm8q1x7coz9j/qT/iv44OTjxx4g/CPYNNjqkQvePSUEpdZ28f/wVIH3ahDDYpetpPL10P7tPJTCkZVXe6F0fV2dHo8MqkhYdWMTo9aOp4VODDUM24O/tb3RIhvjHpKC19iisQIQQd2d39BUmLdvP9bRMpj0WyCNNfY0OqUjSWvPW5reYsmUKnf07s2rgKrxLehsdlmEKth+VEMLqtNZ8tTWaqT8fp6qPG4tGt6BuxdJGh1UkpWelM3r9aJYcWsITjZ9gVu9ZlHAsYXRYhpKkIEQRci0tkxe/PcCvERfp2aAiHw1shIerTJd+L66kXqH/iv5sPbOV9zq/x6ttX7WbHkb/RJKCEEVExLlrPLkkjNirN3itVz1Gt/WXH7F7FHklkl5Le3Em6QzLHl3G4IDBRodkMyQpCFEErN0fxyurD+FZ0pnl44Jp7idXJ9+rbWe20W95PxSK34f/Tpuqbe78JDsiSUEIG5aZbeKDDceYt/0ULfx9+HxIU8p5yNXJ92rZoWWM/G4kfl5+/DjkR2r61DQ6JJsjSUEIG3U5OZ1JS/exKzqBka39+E+vejg72uZwy7ZIa8219GtcTLnIxeSL/Bz1M+9ve5/21dqzdtBaQ4aQKAqsmhSUUj2BTwFHYK7W+sNbHh8JfAScNa+aqbWea82YhCgKDsUlMX5RKFdSMvhkYCCPNpPuppDzQ59wI4GLKRe5lHKJi8kXLT/6F1NuuZ98kfTs9L88f1ijYXzV5ytcnKS2dTtWSwpKKUfgc3Im4okD9iql1mutI24pukJrPclacQhR1KwOi+PVtYcoV8qFVRNa09DX0+iQDLf19FbGfD+G6KvRZJmy/va4o3KkvHt5KpSqQAX3CtQrV4/ybv+/XKFUBXxL+1KvbD1pnL8Da9YUWgBR5vmcUUotB/oBtyYFIQQ57Qfv/XiUBTtiCK6e035QRkY3ZfHBxYxeP5pqntV4sdWLf/mhv/nXp6SPzc5kVtRYMylUBmJzLccBLfMo96hSqj1wAnheax2bRxkhirXLyek8uWQfe04lMLqtP68+WBcnO28/0Foz5c8pvPXnW3T068jqx1ZLO0AhMLqh+XtgmdY6XSk1HvgG6HxrIaXUOGAcQNWqVQs3QiGs7EBsIhMWh5GQksH0QY15uEllo0MyXHpWOmO+H8Pig4sZETiCOX3m2P2VxoXFmociZ4EquZZ9+f8GZQC01le01jdbguYCzfJ6Ia31HK11kNY6qFy5clYJVggjrAyNZeDsnTgoxeqJrSUhkHOlcbdF3Vh8cDHvdHqH+f3mS0IoRNasKewFaiml/MlJBoOBIbkLKKUqaa3Pmxf7AketGI8QNiMz28Q7P0SwcOdpWtcow8whTfFxlx++3FcaL31kKSENQ4wOye5YLSlorbOUUpOpYsoPAAAgAElEQVSAX8jpkjpPa31EKTUFCNVarweeUUr1BbKABGTeZ2EH4q+n89SSfeyJSWBsO39e7intB5DTw+jhFQ/LlcYGU1oXrblygoKCdGhoqNFhCHFP9p+5ysTF+0i8kcHURxvRr7GcLgJYcnAJo9aPkiuNrUgpFaa1DrpTOTk8EaIQaK35dm8sg2bvwskxp/1AEkLOdnl789sMXTuUVr6t2Dl6pyQEgxnd+0iIYm939BU++e0Ee04l0LZmWT4LaYK3tB/8pYfR8MDhfNXnK2lQtgGSFISwkrDTV/nfbyfYFnWZch4uvNWnPkODq0n7ATk9jB759hG2nN7CO53e4T/t/iNXGtsISQpCFLADsYlM++0Ef56Ip4x7CV7rVY+hwdVk7mSzqIQoHlryEKeTTksPIxskSUGIAnL4bBLTN55g49FLeLk583LPuoxoXQ23EvI1u2nbmW08vPxhADYN3yQ9jGyQ7K1C3KdjF64x/bdIfj5ygdKuTrzYvTYjWvvJNJm5aK1ZcmgJo9ePlh5GNk6SghD3KOrSdaZvjOTHQ+cpVcKJZ7vUYlRbfzxLSjKAnEQQfiGclRErWRWxisiESDpU68CaQWtkDCMbJklBiLt06nIKM36P5Lvws7g6O/JkxxqMbVcdLzfpOaO1Jux8GKsiVrEqYhUnr57EUTnS2b8zk1tPZnjgcJnLwMZJUhAin2ITUpnxeyRr9p/F2VExtl11xrWvbvfDW2ut2XtuLyuPrGTV0VXEJMbg5OBEF/8uvNr2VfrV7UdZt7JGhynySZKCEPmwMjSWV9ccwsFBMaKVHxM6Vqe8h6vRYRnGpE3sjtudUyM4uoozSWdwdnCmW41uvNH+DfrV7SeniIooSQpC3MG8baeY8kMEbWuW5eOBgVT0tM9kYNImdsbuZGXESlYfXU3ctThKOJage43uvNPpHfrU7oN3SW+jwxT3SZKCELehtebT3yOZvjGSng0q8mlIY1yc7Otag2xTNtvObGNVxCrWHFvDuevncHF0oUfNHnzQ5QP61O6Dp6tMF1qcSFIQIg8mk+bdH48yb/spBjTz5cNHGtrNlchZpiw2x2xmVcQq1h5by6WUS7g6udKzZk8G1h9I79q9Ke1S2ugwhZXYTVLIMmWx//x+mldubnQowsZlZZt4dc0hVobFMbK1H2/0ro+DQ/EegiEjO4NNpzaxKmIV646t48qNK7g5u9G7dm8G1BvAg7UepFSJUkaHKQqB3SSFtze/zdTtUzn93GkqeVQyOhxho9KzsnlueTg/Hb7As11q8VzXWsV2TJ60rDR+O/kbq4+u5rvj35GYlohHCQ/61OnDgHoD6FGzB27ObkaHKQqZ3SSFEY1H8N7W95gdNpu3Or5ldDjCBqVmZDF+URhbIy/zWq96jGlX3eiQCtyNzBv8HPUzq46u4vvj33M94zperl70q9OPAfUH0LV6V1yd7LMhXeSwm6RQ06cmD9V6iFmhs3i17atyAY34i6QbmYxasJf9Z67y30cb8VjzKnd+UhESdi6Mj3Z8xA8nfiAlM4UyJcswqMEgHq3/KJ39O8uQ1cLCbpICwDMtn6HH4h6sjFjJ0EZDjQ5H2IjLyekM/3oPkZeu8/mQpjzYsPicXjx86TBvbn6TNUfX4O3qzbBGwxhQfwAd/Drg5GBXX3+RT3a1V3Sr3o26Zevy6e5Pebzh48X2XLHIv7OJNxg2dzfnkm4wd0RzOtQuZ3RIBSLySiRv/fkWyw4tw8PFg7c7vs1zwc9JryFxR3aVFJRSPN3iaZ7a8BS7z+4m2DfY6JCEgaLjkxk6dzfX07NYPLolQX5F/wrcM0lnmPLnFBaEL8DFyYWX27zMi61fpIxbGaNDE0WEfXS8zmV44HBKu5Rmxu4ZRociDHTkXBKPzd5JepaJ5eOCi3xCOH/9PE9veJpan9Vi0cFFTGoxiZPPnOSDrh9IQhB3xe6SQqkSpRjdZDQrI1Zy7vo5o8MRBgiNSWDwnF2UcHTg2wmtaPBA0b0i90rqFV767SVqzKjBrLBZjAwcSdTTUUzvOZ2KpSoaHZ4oguwuKQA81fwpsk3ZzAqdZXQoopBtORHPsK/3ULaUCysntqZGuaJ5QVZSWhJv/vEm/p/68/GOjxlQfwDHnjrG7D6zqeJZvHpOicJll0mhhk8NetfuzazQWaRnpRsdjigkPx06z+hv9uJX1p1vx7eisldJo0O6aykZKXy47UP8P/VnypYp9KjZg8NPHmZh/4XU8KlhdHiiGLCrhubcnmn5DN+f+J4VR1YwPHC40eGI+5CakUX89XTir6dzyfzXcku+uT6NS9fTaVLFi/kjW+DpVrRmR0vJSGHuvrm8v+19LqVcoletXrzT6R2aVGpidGiimLHbpNDFvwv1ytZjxu4ZDGs0TLqn2rjE1AxW7I3lfFKa5Uf+5g9/Skb238o7KChbyoVyHi6U93ChXiUPqvq4MaqtP24lis5uf+rqKT7f+zlf7/+axLREOvt35p1O79C6SmujQxPFlFW/HUqpnsCngCMwV2v94W3KPQqsApprrUOtGVOu9+SZls8w8ceJ7IzbKV8yG3b8wnXGLgzlTEIqHq5Olh/6hr5elDP/8N9cd/O+t1sJHIvoIHZaa/6I+YMZu2ew/vh6HJQDA+oP4OkWT9OmahujwxPFnNWSglLKEfgc6AbEAXuVUuu11hG3lPMAngV2WyuW2xnWaBivbHyFGbtnSFKwUT8fvsAL34bj7uLE6omtaVat+E7ikpqZyuKDi5mxewZH4o9Q1q0s/273byYETcC3tK/R4Qk7Yc2aQgsgSmsdDaCUWg70AyJuKfcOMBWYbMVY8uRewp0xTccwfdd04q7FyRfPhphMmhmbcia4CazixeyhzYrtjGcxiTF8sfcL5u6by9W0qzSu2Jj5/eYzOGCwDE4nCp01ex9VBmJzLceZ11kopZoCVbTWP1oxjn/0VPOnMGmTdE+1IcnpWUxYHMb0jZE82tSXFeOCi11C0FqzOWYzj6x4hBozajBt5zS6Vu/K1ie2sm/cPkY2HikJQRjCsBY3pZQDMA0YmY+y44BxAFWrVi3QOPy9/elbpy+zw2bzWvvX5ItosNNXUhi7MJSoS8m83rs+o9r4FatOAKmZqSw9tJQZu2dw6NIhypQsw8ttXmZi0ES5vkDYBGsmhbNA7r3c17zuJg8gANhs/tJXBNYrpfre2tistZ4DzAEICgrSBR3oMy2f4bvj37H88HJGNh5Z0C8v8mlb5GWeWroPgIWjWtK2VlmDIyo4kVcimbtvLnP3zyXhRgKNKjTi675fExIQQknnone9hCi+rJkU9gK1lFL+5CSDwcCQmw9qrZMAy7deKbUZeLGweh/l1smvEw3KNWDG7hmMCBxRrI5MiwKtNV9vO8X7G45Ss3wpvhoeRLUy7kaHdd+upV/j2yPfsiB8Adtjt+OgHHi47sM82/JZ2lVtJ/uZsElWSwpa6yyl1CTgF3K6pM7TWh9RSk0BQrXW66313nfrZvfU8T+MZ3vsdtpWbWt0SHYjLTObf689xJp9Z+nRoAKfPNaYUi5F5zqCW5m0iU2nNrEgfAFrjq7hRtYN6paty4ddPmRoo6FULl35zi8ihIGU1gV+NsaqgoKCdGhowVcmUjJSqPK/KnSt3pVvB35b4K8v/u5CUhrjF4dxIDaR57vW5unONXEootcWRCVE8U34N3xz4Btir8Xi6eJJSEAIIxuPpEXlFlIrEIZTSoVprYPuVK7oHpIVsJvdU6ftnEZsUqw0+llZ2OmrTFgcRmp6FrOHNaNHg6I3oue19GusPLKSBQcWsO3MNhyUA91rdOejbh/Rt05faSsQRZJdDoh3O082fxKN5svQL40OpVj7NjSWkDm7KOnsyJon2xSphGDSJn6P/p1ha4dR8eOKjPl+DPEp8XzQ5QPOPHeGnx7/iUEBgyQhiCJLagq5+Hn50a9OP+aEzeH19q/LF7uAZWabeO/HoyzYEUPbmmWZOaQJXm5FY8L4mMQY5u2fxzcHvuFM0hk8XTwZHjickY1H0rJySzk9JIoNSQq3eKblM6w9tpZlh5cxqskoo8MpNqIuJfP6usPsjL7C6Lb+vPpgXZwcbbuimm3KZkPkBmaFzeKnyJ8A6F6jO1O7TqVfnX5y0CCKJWlovoXWmsBZgTgoB/aP3y9HgPchM9vEr0cusmhXDLuiE3BxcuD9/g15tJltDydy9tpZvt7/NXP3zSX2WiyVSlViTNMxjGk6hqqeBXvxpBCFRRqa79HN7qljvx/L1jNbaV+tvdEhFTnnEm+wbM8Zlu+NJf56OpW9SjK5Rx0eC6pCOQ8Xo8PLk0mb2Bi9kVmhs1h/fD3ZOpvuNbozved0+tTug7Nj0Zp/QYh7JUkhD0MaDuHljS8zY/cMSQr5ZDJptkTGs3jXGTYdu4gGOtUpz9DgqnSoXd5mh7GOT4lnfvh8ZofNJvpqNGXdyvKvVv9ibLOx1PSpaXR4QhQ6SQp5cHN2Y2zTsXy04yPOJJ2RUwb/ICElg29DY1m6+wxnElIpW6oEEzrUIKRFVar4uBkdXp601mw9s5VZobNYfXQ1GdkZtK/Wnnc7vcsj9R7Bxck2azNCFAZJCrfxZPMn+WjHR3yx9ws+7Jrn3EB2S2tN2OmrLN51mg2HLpCRbaKFvw8v9qhDzwYVKeFkmw3IV29cZeGBhcwOm83Ry0fxcvViYtBExjUbR/1y9Y0OTwibIEnhNqp6VqV/3f58te8r3ujwBm7OtnnUW5iS07NYu/8sS3ad5tiF65RycWJwiyo83rIadSp6GB1entKy0vgp8ieWH1nO+uPrSctKo2XllszvN5/HGjwm/1chbiFJ4R880/IZVh9dzdJDSxnTdIzR4RhGa83Cnaf56JfjJKdnUb9Sad7v35B+jR/A3QbHKcrMzmRj9EaWH1nO2qNruZ5xnfLu5RndZDRjmo6hccXGRocohM2yvW+0DWlXtR2BFQKZsXsGo5uMtsvuqYmpGUxedZDfIi7SvnY5nutaiyZVvGxuW2SbstlyegvLDy9n1dFVJNxIwMvVi8caPMbggMF09OuIk4Ps7kLciXxL/sHN7qmj14/mz9N/0tGvo9EhFard0Vd4bkU4l5PTea1XPUa18bepAeu01uyK28Xyw8v5NuJbLiRfwN3ZnYfrPszggMF0r9GdEo5F44ppIWyFJIU7CAkI4aXfXmLG7hl2kxSyTZrPNkUy4/dIqvq4sWZiGxr6ehodFpCTCMIvhLP88HJWHFnB6aTTuDi60Lt2bwYHDOahWg9JO4EQ90GSwh2UdC7JuGbjmLp9KjGJMfh5+RkdklWdT7rBs8vD2XMqgf5NKvPOwwE2Mb/BldQrfLbnM5YfXs7xK8dxcnCie43uvNv5XfrW6Utpl9JGhyhEsWCbfQet4GziDWZuiuRehvWYGDQRheKLvV9YITLb8VvERR78dCuHzybxycBA/jfI+AlvtNYsPbSUep/XY8qfU6hcujJzes/hwr8u8OOQHxnaaKgkBCEKkPGHgIVk3f6zfPzrCZLTs3m5Z527aiit4lmFR+o9wlf7vmJSi0nF7mK2tMxsPvzpGAt2xNDggdJ8FtKE6uVKGR0WMYkxTPxxIj9H/UyLyi3YOHwjjSo0MjosIYo1u6kpPNmxBkODqzLrz5N8tinqrp//Zoc30VrTZWEXzl8/b4UIjXEyPpn+X+xgwY4YRrXxZ82TrQ1PCFmmLD7Z8QkNvmjAtjPbmNFzBjtG7ZCEIEQhsJuaglKKKX0DuJFhYtpvJyjp7MjY9tXz/fwG5Ruw4fENdF/UnW6LuvHnyD8p41bGihH/1Y2MbEJPJ+BXxh1f75L33SVUa83KsDje/O4Irs4OfD0iiC71KhRQtPdu//n9jP1+LGHnw+hduzdfPPSFzIInRCGym6QA4OCgmPpoQ9Iys3lvw1FcSzgyLLhavp/fukpr1oes56ElD9FjcQ9+H/47nq7W75UTdzWVMd+EcuzCdQC83JwJeMCTgMqeBFQuTcPKnlT1cct3orielslr6w7zXfg5gqv7MH1QEyp6ulrzI9xRamYqb21+i2k7p1HWrSzfDviWAfUH2Nz1EEIUd3aVFACcHB3436DGpGVm8/q6w5R0dmTAXYzv39m/M6sfW03/Ff3ptbQXvwz9BfcS7laLd29MAhMWhZGRbeKTgYHcyMzmyLkkDp1N4utt0WRm5zSce7g6mRNFaXOy8MS/jPvfris4EJvIM8v3E5uQyr+61ebJTjUNH8H0t5O/Mf6H8ZxKPMWYJmP4b7f/4l3S29CYhLBXdjvJTlpmNmO+CWXHycvMCGlC70YP3NXzV0WsYtCqQXTy68QPQ37A1angj7RX7D3Da+sO4+vtxtwRQdS45Vx/RpaJExevc/hsTpI4fO4aR89fIyPLBIB7CUca5KpRXLiWxrRfT1Dew4VPQ5rQ3M+nwGO+G5dTL/PCLy+w6OAiapepzZzec+jg18HQmIQorvI7yY7dJgWA1IwsRszbw/4zicwa2oyu9e/unPrCAwsZsW4EvWv3Zs1jawpsIpasbBPvbTjK/O0xtKtVlpkhTfF0y99rZ2abiLyYzOFzSRw+m3OLOH+NtMycRNGjQQWmPtrI0LmRtdYsPriY5395nqT0JF5p8wr/af8fqyRWIUQOSQr5dD0tk8fn7ubY+evMG9mctrXK3tXzv9z7JU9ueJLHGjzG0keW4ujgeF/xJKVmMmnZPrZGXmZUG3/+/dD9z2WclW0i+nIKSTcyCarmbeh5+uir0Uz4YQK/Rf9GK99WzOkzh4DyAYbFI4S9kOk488nD1ZmFo1oweM4uxi4MZeHoFnd1WmVi84mkZKYw+bfJuDm78XXfr3FQ9/YjfjI+mbHfhBJ7NZWpjzZkUPOCuR7CydGB2hWMHdo6NTOVz/d8zpub38TJwYmZD85kYvOJ97ythBDWYfdJAcDLrQSLRrdk0JydPDF/L0vGtCSwile+n/9i6xdJzkjm7T/fppRzKWY8OOOuj8b/PBHPpKX7KOHowNKxwYaf7y8ImdmZ/HryV5YdXsZ3x78jOSOZfnX6MfOhmfiWzn/jvhCi8Fg1KSilegKfAo7AXK31h7c8PgF4CsgGkoFxWusIa8Z0O+U8XFg6JpiBs3cwfN4elo8Lpl6l/A+f8GaHN0nOSOaTnZ/gXsKdD7p8kK/EoLXm622neH/DUWpX8GDuiCB8vYvugG4mbWLL6S0sO7TMMoS1t6s3gxsMZmijobSv1l66mQphw6zWpqCUcgROAN2AOGAvEJL7R18pVVprfc18vy/wpNa65z+9bkG3KdwqNiGVx2bvJCPLxIrxrahZPv9X92qtefLHJ5kVNot3O73Lf9r/5x/Lp2dl89raw6wMi6NHgwpMe6yxTU5acydaa0LPhbLs8DJWHFnBuevncHN2o1+dfoQEhNCjZg8ZwloIg9lCm0ILIEprHW0OaDnQD7AkhZsJwcwdMLzVu4qPG4vHtGTQ7J08PncXK8e3pmqZ/B25K6X4vNfnpGSm8Nofr+Fewp3ngp/Ls2z89XQmLg4j9PRVnulck+e61rapuQryIyI+gmWHlrH8yHKiEqIo4ViCB2s+SEhACL1r97bq9RtCCOuwZlKoDMTmWo4DWt5aSCn1FPACUALobMV48q1GuVIsHtOSwXN2MWTuLr4d34oHvErm67kOyoF5/eaRkpnC8788j5uzG+OajftLmSPnkhj7TSgJqRnMHHL310gYKSYxhuWHl7Ps8DIOXjyIg3Kgs39nXm37Kv3r9peLzoQo4qx5+mgA0FNrPca8PAxoqbWedJvyQ4AeWusReTw2DhgHULVq1WanT5+2Ssy3OhiXyONf7aachwsrxreinIdLvp+bkZ3Bw8sf5ueon1nUfxGPN3ocgJ8OneeFbw/g5ebMnGFBNjN5ze1kZmey5+we/oj5gw2RG9gZtxOAVr6tCAkIYWCDgVQsVdHgKIUQd2L4dQpKqVbAW1rrHublVwG01h/cprwDcFVr/Y+/ktZuU7hVaEwCw77eQ1UfN5aPC8bbPX/nxrNNmmtpKfRb0ZsdsduY02sx1642ZfrGSJpU9WL2sGaU97C9i7WyTdmEXwhn06lNbIrZxNbTW0nJTEGhaFKpCQPrD2RwwOBiP9mQEMWNLSQFJ3IamrsAZ8lpaB6itT6Sq0wtrXWk+X4f4M07BV3YSQFge9Rlnliwl/IeLlTydCUjy0RGtiYz20RGlonM7JxbuuW+JtuUs11NpHLR5XUy1EnKZ7zG44H9eP+Rhrg6399FbgVFa82R+CM5SeDUJv48/SeJaYkA1Ctbj87+nens35kO1ToU6qiwQoiCZXhSMAfxEDCdnC6p87TW7ymlpgChWuv1SqlPga5AJnAVmJQ7aeTFiKQAOdcRzNp8EqXA2dEBZ0cHXJwccHZUlHByuGVdzq2E+fEsfZ0PQ4dw+noEPiV9qOlTM+fmXfP/7/vUpKxbWat319RaE5UQZakJ/HHqD+JT4wGo7l2dzn45SaCjX0cqeVSyaixCiMJjE0nBGoxKCvcr4UYCC8IXEHklkqirUUQlRHEm6QwmbbKUKe1SmhreNf6SKG7eKpWq9JeEobUmIzuD5IxkkjOSSclMsdy3rMv467pzyefYHLOZuGtxAFT2qGypCXTy60Q1r/wPIy6EKFokKRQB6VnpxCTGEJUQ9f+3q1GcTDjJqcRTZJmyLGVLOpXkAY8HSMtKsySB3I/fiauTKz4lfWhbtS2d/TrTyb8TtXxqyYVkQtgJW7hOQdyBi5MLdcrWoU7ZOn97LMuUxZmkM39JGOeTz+Pu7I67szulSpT6y829RB7rzOXcS7jj5CD/aiHEnckvhY1ycnCiund1qntXp3uN7kaHI4SwEzJEpRBCCAtJCkIIISwkKQghhLCQpCCEEMJCkoIQQggLSQpCCCEsJCkIIYSwkKQghBDCosgNc6GUigcKZ0IF21UWuGx0EAaTbSDb4CbZDvnbBtW01uXu9EJFLikIUEqF5mcMk+JMtoFsg5tkOxTsNpDTR0IIISwkKQghhLCQpFA0zTE6ABsg20C2wU2yHQpwG0ibghBCCAupKQghhLCQpGBjlFJVlFJ/KKUilFJHlFLPmtf7KKV+U0pFmv96m9crpdQMpVSUUuqgUqqpsZ+g4CilHJVS+5VSP5iX/ZVSu82fdYVSqoR5vYt5Ocr8uJ+RcRckpZSXUmqVUuqYUuqoUqqVve0LSqnnzd+Fw0qpZUopV3vYF5RS85RSl5RSh3Otu+v/vVJqhLl8pFJqxJ3eV5KC7ckC/qW1rg8EA08ppeoDrwC/a61rAb+blwEeBGqZb+OALws/ZKt5Fjiaa3kq8D+tdU3gKjDavH40cNW8/n/mcsXFp8DPWuu6QCA528Nu9gWlVGXgGSBIax0AOAKDsY99YQHQ85Z1d/W/V0r5AG8CLYEWwJs3E8ltaa3lZsM34DugG3AcqGReVwk4br4/GwjJVd5SrijfAF/zTt8Z+AFQ5Fyc42R+vBXwi/n+L0Ar830nczll9GcogG3gCZy69bPY074AVAZiAR/z//YHoIe97AuAH3D4Xv/3QAgwO9f6v5TL6yY1BRtmrvo2AXYDFbTW580PXQAqmO/f/NLcFGdeV9RNB14CTOblMkCi1jrLvJz7c1q2gfnxJHP5os4fiAfmm0+jzVVKuWNH+4LW+izwMXAGOE/O/zYM+9sXbrrb//1d7xOSFGyUUqoUsBp4Tmt9LfdjOiflF9tuY0qp3sAlrXWY0bEYzAloCnyptW4CpPD/pwsAu9gXvIF+5CTIBwB3/n5KxS5Z638vScEGKaWcyUkIS7TWa8yrLyqlKpkfrwRcMq8/C1TJ9XRf87qirA3QVykVAywn5xTSp4CXUsrJXCb357RsA/PjnsCVwgzYSuKAOK31bvPyKnKShD3tC12BU1rreK11JrCGnP3D3vaFm+72f3/X+4QkBRujlFLA18BRrfW0XA+tB272HBhBTlvDzfXDzb0PgoGkXNXLIklr/arW2ldr7UdOo+ImrfXjwB/AAHOxW7fBzW0zwFy+yB89a60vALFKqTrmVV2ACOxoXyDntFGwUsrN/N24uQ3sal/I5W7/978A3ZVS3uZaV3fzutszuiFFbn9rWGpLTpXwIBBuvj1EznnR34FIYCPgYy6vgM+Bk8AhcnppGP45CnB7dAR+MN+vDuwBooCVgIt5vat5Ocr8eHWj4y7Az98YCDXvD+sAb3vbF4C3gWPAYWAR4GIP+wKwjJx2lExyao2j7+V/D4wyb48o4Ik7va9c0SyEEMJCTh8JIYSwkKQghBDCQpKCEEIIC0kKQgghLCQpCCGEsJCkIMRtKKX+Yx6d86BSKlwp1VIp9ZxSys3o2ISwFumSKkQelFKtgGlAR611ulKqLFAC2EFOH/DLhgYohJVITUGIvFUCLmut0wHMSWAAOePv/KGU+gNAKdVdKbVTKbVPKbXSPGYVSqkYpdR/lVKHlFJ7lFI1zesHmucFOKCU2mLMRxPi9qSmIEQezD/u2wA3cq4cXaG1/tM8HlOQ1vqyufawBnhQa52ilHqZnCtrp5jLfaW1fk8pNRx4TGvdWyl1COiptT6rlPLSWica8gGFuA2pKQiRB611MtCMnAlL4oEVSqmRtxQLBuoD25VS4eSMRVMt1+PLcv1tZb6/HViglBpLzoQxQtgUpzsXEcI+aa2zgc3AZvMR/q1TGSrgN611yO1e4tb7WusJSqmWQC8gTCnVTGtdnEbxFEWc1BSEyINSqo5SqlauVY2B08B1wMO8bhfQJld7gbtSqnau5wzK9XenuUwNrfVurfUb5NRAcg9rLIThpKYgRN5KAZ8ppbzImTc7ipxTSSHAz0qpc1rrTuZTSsuUUi7m570GnDDf91ZKHQTSzc8D+MicbBQ5o10eKJRPI0Q+SUOzEFaQu0Ha6FiEuBty+kgIIYSF1BSEEEJYSE1BCGN6zFMAAAAoSURBVCGEhSQFIYQQFpIUhBBCWEhSEEIIYSFJQQghhIUkBSGEEBb/BwcIwcxRBXkCAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "dpt_val_loss = np.array(dpt_model.get_validation_summary(\"Loss\"))\n",
+    "\n",
+    "plt.plot(original_val_loss[:,0], original_val_loss[:,1], label='original model')\n",
+    "plt.plot(dpt_val_loss[:,0], dpt_val_loss[:,1],label='Dropout-regularized model',color='green')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Again, a clear improvement over the reference network.\n",
+    "\n",
+    "To recap: here the most common ways to prevent overfitting in neural networks:\n",
+    "\n",
+    "* Getting more training data.\n",
+    "* Reducing the capacity of the network.\n",
+    "* Adding weight regularization.\n",
+    "* Adding dropout."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From d92cd27cf4afd1239156aefe675ede80de2939f9 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 21 Mar 2019 16:36:09 +0800
Subject: [PATCH 34/46] Add files via upload

some writing style fix
---
 keras/4.4-overfitting-and-underfitting.ipynb | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/keras/4.4-overfitting-and-underfitting.ipynb b/keras/4.4-overfitting-and-underfitting.ipynb
index d28bebd..c2a801b 100644
--- a/keras/4.4-overfitting-and-underfitting.ipynb
+++ b/keras/4.4-overfitting-and-underfitting.ipynb
@@ -409,7 +409,7 @@
     "the \"L2 norm\" of the weights). L2 regularization is also called _weight decay_ in the context of neural networks. Don't let the different \n",
     "name confuse you: weight decay is mathematically the exact same as L2 regularization.\n",
     "\n",
-    "In Analytics-zoo Keras API, weight regularization is added by passing _weight regularizer instances_ to layers as keyword arguments. Let's add L2 weight \n",
+    "In Keras API of Analytics Zoo, weight regularization is added by passing _weight regularizer instances_ to layers as keyword arguments. Let's add L2 weight \n",
     "regularization to our movie review classification network:"
    ]
   },
@@ -518,7 +518,7 @@
     "As you can see, the model with L2 regularization (dots) has become much more resistant to overfitting than the reference model (crosses), \n",
     "even though both models have the same number of parameters.\n",
     "\n",
-    "As alternatives to L2 regularization, you could use one of the following Analytics-zoo Keras API weight regularizers: "
+    "As alternatives to L2 regularization, you could use one of the following Keras API of Analytics Zoo weight regularizers: "
    ]
   },
   {
@@ -568,7 +568,7 @@
     "The core idea is that introducing noise in the output values of a layer can break up happenstance patterns that are not significant (what \n",
     "Hinton refers to as \"conspiracies\"), which the network would start memorizing if no noise was present. \n",
     "\n",
-    "In Analytics-zoo Keras API you can introduce dropout in a network via the `Dropout` layer, which gets applied to the output of layer right before it, e.g.:"
+    "In Keras API of Analytics Zoo you can introduce dropout in a network via the `Dropout` layer, which gets applied to the output of layer right before it, e.g.:"
    ]
   },
   {

From 2b88f7880fee9c6f496947389dcc45f090b07dd7 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 21 Mar 2019 16:37:59 +0800
Subject: [PATCH 35/46] Add files via upload

---
 keras/5.1-introduction-to-convnets.ipynb | 300 +++++++++++++++++++++++
 1 file changed, 300 insertions(+)
 create mode 100644 keras/5.1-introduction-to-convnets.ipynb

diff --git a/keras/5.1-introduction-to-convnets.ipynb b/keras/5.1-introduction-to-convnets.ipynb
new file mode 100644
index 0000000..0fbedc1
--- /dev/null
+++ b/keras/5.1-introduction-to-convnets.ipynb
@@ -0,0 +1,300 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=8g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=8g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 5.1 - Introduction to convnets\n",
+    "\n",
+    "\n",
+    "----\n",
+    "\n",
+    "First, let's take a practical look at a very simple convnet example. We will use our convnet to classify MNIST digits, a task that you've already been \n",
+    "through in Chapter 2, using a densely-connected network (our test accuracy then was 97.8%). Even though our convnet will be very basic, its \n",
+    "accuracy will still blow out of the water that of the densely-connected model from Chapter 2.\n",
+    "\n",
+    "The 6 lines of code below show you what a basic convnet looks like. It's a stack of `Conv2D` and `MaxPooling2D` layers. We'll see in a \n",
+    "minute what they do concretely.\n",
+    "Importantly, a convnet takes as input tensors of shape `(image_height, image_width, image_channels)` (not including the batch dimension). \n",
+    "In our case, we will configure our convnet to process inputs of size `(28, 28, 1)`, which is the format of MNIST images. We do this via \n",
+    "passing the argument `input_shape=(28, 28, 1)` to our first layer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasConvolution2D\n",
+      "creating: createZooKerasMaxPooling2D\n",
+      "creating: createZooKerasConvolution2D\n",
+      "creating: createZooKerasMaxPooling2D\n",
+      "creating: createZooKerasConvolution2D\n"
+     ]
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras import layers\n",
+    "from zoo.pipeline.api.keras import models\n",
+    "\n",
+    "model = models.Sequential()\n",
+    "model.add(layers.Conv2D(32, nb_col=3, nb_row=3, activation='relu', input_shape=(1,28,28)))\n",
+    "model.add(layers.MaxPooling2D((2, 2)))\n",
+    "model.add(layers.Conv2D(64, nb_col=3, nb_row=3, activation='relu'))\n",
+    "model.add(layers.MaxPooling2D((2, 2)))\n",
+    "model.add(layers.Conv2D(64, nb_col=3, nb_row=3, activation='relu'))\n",
+    "\n",
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_In Keras one could see model summary directly in output, in Keras API of Analytics Zoo, summary is printed in console, the same as INFO._"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "From summary you can see that the output of every `Conv2D` and `MaxPooling2D` layer is a 3D tensor of shape `(height, width, channels)`. The width \n",
+    "and height dimensions tend to shrink as we go deeper in the network. The number of channels is controlled by the first argument passed to \n",
+    "the `Conv2D` layers (e.g. 32 or 64).\n",
+    "\n",
+    "The next step would be to feed our last output tensor (of shape `(3, 3, 64)`) into a densely-connected classifier network like those you are \n",
+    "already familiar with: a stack of `Dense` layers. These classifiers process vectors, which are 1D, whereas our current output is a 3D tensor. \n",
+    "So first, we will have to flatten our 3D outputs to 1D, and then add a few `Dense` layers on top:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasFlatten\n",
+      "creating: createZooKerasDense\n",
+      "creating: createZooKerasDense\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<zoo.pipeline.api.keras.models.Sequential at 0x7f81700d54a8>"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "model.add(layers.Flatten())\n",
+    "model.add(layers.Dense(64, activation='relu'))\n",
+    "model.add(layers.Dense(10, activation='softmax'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are going to do 10-way classification, so we use a final layer with 10 outputs and a softmax activation. Now here's what our network \n",
+    "looks like:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, our `(3, 3, 64)` outputs were flattened into vectors of shape `(576,)`, before going through two `Dense` layers.\n",
+    "\n",
+    "Now, let's train our convnet on the MNIST digits. We will reuse a lot of the code we have already covered in the MNIST example from Chapter \n",
+    "2."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### CNN input shape\n",
+    "_Once we get the dataset, we need to reshape the images. In Keras the shape of the dataset is `(sample_size, height, width, channel)`, like the Keras code below:\n",
+    "    \n",
+    "    train_images = train_images.reshape((60000, 28, 28, 1))\n",
+    "In Keras API of Analytics Zoo, the default order is theano-style NCHW `(sample_size, channel, height, width)`, so you can process data like following:\n",
+    "\n",
+    "Alternatively, you can also use tensorflow-style NHWC as Keras default just by setting `Convolution2D(dim_ordering=\"tf\")`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As you can see, our `(3, 3, 64)` outputs were flattened into vectors of shape `(576,)`, before going through two `Dense` layers.\n",
+    "\n",
+    "Now, let's train our convnet on the MNIST digits. We will reuse a lot of the code we have already covered in the MNIST example from Chapter \n",
+    "2."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Using TensorFlow backend.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from keras.datasets import mnist\n",
+    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n",
+    "\n",
+    "train_images = train_images.reshape((60000, 1, 28, 28))\n",
+    "train_images = train_images.astype('float32') / 255\n",
+    "\n",
+    "test_images = test_images.reshape((10000, 1, 28, 28))\n",
+    "test_images = test_images.astype('float32') / 255"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createRMSprop\n",
+      "creating: createZooKerasSparseCategoricalCrossEntropy\n",
+      "creating: createZooKerasSparseCategoricalAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='sparse_categorical_crossentropy',\n",
+    "              metrics=['acc'])\n",
+    "\n",
+    "model.fit(train_images, train_labels, nb_epoch=5, batch_size=64)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Trained 64 records in 0.03212866 seconds. Throughput is 1991.9911 records/second. Loss is 0.0023578003."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_loss, test_acc = model.evaluate(test_images, test_labels)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "0.9912999868392944"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "test_acc"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "While our densely-connected network from Chapter 2 had a test accuracy of 97.8%, our basic convnet has a test accuracy of 99.1%: we \n",
+    "decreased our error rate by over 50% (relative). Not bad! "
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From 5dc2650d7b841a08bae2d6619331141f48f28110 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Mon, 25 Mar 2019 14:31:05 +0800
Subject: [PATCH 36/46] Add files via upload

---
 ...erstanding-recurrent-neural-networks.ipynb | 441 ++++++++++++++++++
 1 file changed, 441 insertions(+)
 create mode 100644 keras/6.2-understanding-recurrent-neural-networks.ipynb

diff --git a/keras/6.2-understanding-recurrent-neural-networks.ipynb b/keras/6.2-understanding-recurrent-neural-networks.ipynb
new file mode 100644
index 0000000..5b19bf2
--- /dev/null
+++ b/keras/6.2-understanding-recurrent-neural-networks.ipynb
@@ -0,0 +1,441 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First of all, set environment variables and initialize spark context:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: SPARK_DRIVER_MEMORY=16g\n",
+      "env: PYSPARK_PYTHON=/usr/bin/python3.5\n",
+      "env: PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+      "Prepending /home/litchy/.local/lib/python3.5/site-packages/bigdl/share/conf/spark-bigdl.conf to sys.path\n"
+     ]
+    }
+   ],
+   "source": [
+    "%env SPARK_DRIVER_MEMORY=16g\n",
+    "%env PYSPARK_PYTHON=/usr/bin/python3.5\n",
+    "%env PYSPARK_DRIVER_PYTHON=/usr/bin/python3.5\n",
+    "\n",
+    "from zoo.common.nncontext import *\n",
+    "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Understanding recurrent neural networks\n",
+    "\n",
+    "----\n",
+    "\n",
+    "In this section we will build recurrent neural networks to finish the same task as we did in chapter 3."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## A first recurrent layer in Keras API of Analytics Zoo\n",
+    "\n",
+    "The process we just naively implemented in Numpy corresponds to an actual layer: the `SimpleRNN` layer:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.layers import SimpleRNN"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "There is just one minor difference: `SimpleRNN` processes batches of sequences, like all other Keras API of Analytics Zoo layers, not just a single sequence like \n",
+    "in our Numpy example. This means that it takes inputs of shape `(batch_size, timesteps, input_features)`, rather than `(timesteps, \n",
+    "input_features)`.\n",
+    "\n",
+    "Like all recurrent layers in Keras API of Analytics Zoo, `SimpleRNN` can be run in two different modes: it can return either the full sequences of successive \n",
+    "outputs for each timestep (a 3D tensor of shape `(batch_size, timesteps, output_features)`), or it can return only the last output for each \n",
+    "input sequence (a 2D tensor of shape `(batch_size, output_features)`). These two modes are controlled by the `return_sequences` constructor \n",
+    "argument. Let's take a look at an example:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from zoo.pipeline.api.keras.models import Sequential\n",
+    "from zoo.pipeline.api.keras.layers import Embedding, SimpleRNN"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Following is the preprocessing method. You do not need to care about the detail of its implementation. Basically this `pad_sequences` method fix all the sequences to a same length."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def pad_sequences(sequences, maxlen=None, dtype='int32',\n",
+    "                  padding='pre', truncating='pre', value=0.):    \n",
+    "    lengths = [len(s) for s in sequences]\n",
+    "\n",
+    "    nb_samples = len(sequences)\n",
+    "    if maxlen is None:\n",
+    "        maxlen = np.max(lengths)\n",
+    "\n",
+    "    # take the sample shape from the first non empty sequence\n",
+    "    # checking for consistency in the main loop below.\n",
+    "    sample_shape = tuple()\n",
+    "    for s in sequences:\n",
+    "        if len(s) > 0:\n",
+    "            sample_shape = np.asarray(s).shape[1:]\n",
+    "            break\n",
+    "\n",
+    "    x = (np.ones((nb_samples, maxlen) + sample_shape) * value).astype(dtype)\n",
+    "    for idx, s in enumerate(sequences):\n",
+    "        if not len(s):\n",
+    "            continue  # empty list/array was found\n",
+    "        if truncating == 'pre':\n",
+    "            trunc = s[-maxlen:]\n",
+    "        elif truncating == 'post':\n",
+    "            trunc = s[:maxlen]\n",
+    "        else:\n",
+    "            raise ValueError('Truncating type \"%s\" not understood' % truncating)\n",
+    "\n",
+    "        # check `trunc` has expected shape\n",
+    "        trunc = np.asarray(trunc, dtype=dtype)\n",
+    "        if trunc.shape[1:] != sample_shape:\n",
+    "            raise ValueError('Shape of sample %s of sequence at position %s is different from expected shape %s' %\n",
+    "                             (trunc.shape[1:], idx, sample_shape))\n",
+    "\n",
+    "        if padding == 'post':\n",
+    "            x[idx, :len(trunc)] = trunc\n",
+    "        elif padding == 'pre':\n",
+    "            x[idx, -len(trunc):] = trunc\n",
+    "        else:\n",
+    "            raise ValueError('Padding type \"%s\" not understood' % padding)\n",
+    "    return x"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's try to use such a model on the IMDB movie review classification problem. First, let's preprocess the data. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "input_train shape: (25000, 500)\n",
+      "input_test shape: (25000, 500)\n"
+     ]
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras.datasets import imdb\n",
+    "\n",
+    "max_features = 10000  # number of words to consider as features\n",
+    "maxlen = 500  # cut texts after this number of words (among top max_features most common words)\n",
+    "batch_size = 32\n",
+    "\n",
+    "(input_train, y_train), (input_test, y_test) = imdb.load_data(nb_words=max_features)\n",
+    "input_train = pad_sequences(input_train, maxlen=maxlen)\n",
+    "input_test = pad_sequences(input_test, maxlen=maxlen)\n",
+    "print('input_train shape:', input_train.shape)\n",
+    "print('input_test shape:', input_test.shape)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It is sometimes useful to stack several recurrent layers one after the other in order to increase the representational power of a network. \n",
+    "In such a setup, you have to get all intermediate layers to return full sequences:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Specify input shape\n",
+    "_One could add an embedding layer as our first layer in Keras as following:_\n",
+    "    \n",
+    "    model = Sequential()\n",
+    "    model.add(Embedding(10000, 32))\n",
+    "_In Keras API of Analytics Zoo, you need to specify the input shape of first layer, in this example, the sequence length is 500, as is shown above, so we could build our model as following:_"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasEmbedding\n",
+      "creating: createZooKerasSimpleRNN\n",
+      "creating: createZooKerasSimpleRNN\n",
+      "creating: createZooKerasSimpleRNN\n",
+      "creating: createZooKerasSimpleRNN\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = Sequential()\n",
+    "model.add(Embedding(10000, 32, input_shape=(500,)))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32, return_sequences=True))\n",
+    "model.add(SimpleRNN(32))  # This last layer only returns the last outputs.\n",
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's train a simple recurrent network using an `Embedding` layer and a `SimpleRNN` layer:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasEmbedding\n",
+      "creating: createZooKerasSimpleRNN\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras.layers import Dense\n",
+    "\n",
+    "model = Sequential()\n",
+    "model.add(Embedding(max_features, 32, input_shape=(500,)))\n",
+    "model.add(SimpleRNN(32))\n",
+    "model.add(Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
+    "\n",
+    "import time\n",
+    "dir_name = '6-2 ' + str(time.ctime())\n",
+    "model.set_tensorboard('./', dir_name)\n",
+    "model.fit(input_train, y_train,\n",
+    "          nb_epoch=10,\n",
+    "          batch_size=128,\n",
+    "          validation_split=0.2)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 128 records in 0.046239497 seconds. Throughput is 2768.1963 records/second. Loss is 0.16970885.\n",
+    "\n",
+    "Top1Accuracy is Accuracy(correct: 4167, count: 5000, accuracy: 0.8334)_"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's display the training and validation loss:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnXeclNXV+L9ntrJL22WXusCCoPQmIooIWBDsvURjiS2Jxp+via8Ya0x8rTHGxFhjiTEaY48NG4gFCyBdkKosdZe+bN+5vz+emdlnZp+pO7MzO5zv57Mwc5925pln7rn3nHPPEWMMiqIoigLgSrYAiqIoSuqgSkFRFEXxoUpBURRF8aFKQVEURfGhSkFRFEXxoUpBURRF8aFKQVEURfGhSkFRFEXxoUpBURRF8ZGZbAGipaioyJSWliZbDEVRlDbF/PnzK4wxxeH2a3NKobS0lHnz5iVbDEVRlDaFiPwQyX5qPlIURVF8qFJQFEVRfKhSUBRFUXy0OZ+CoiitT319PWVlZdTU1CRbFCUMubm5lJSUkJWVFdPxqhQURQlLWVkZHTp0oLS0FBFJtjhKEIwxbN++nbKyMvr16xfTOdR8pChKWGpqaujSpYsqhBRHROjSpUuLZnSqFBRFiQhVCG2Dln5PqhSiYHtlLe8t3ZxsMRRFURKGKoUouPTZefz8nwvYua8u2aIoyn7Frl27+Nvf/hbTsccffzy7du2KeP/bb7+d+++/P6ZrpQOqFKJgw44qABrcJsmSKMr+RSil0NDQEPLYd955h86dOydCrLRElUIMqGlVUVqXGTNmsGbNGkaNGsX111/P7NmzmThxIieffDJDhgwB4NRTT+Xggw9m6NChPP74475jS0tLqaioYP369QwePJjLL7+coUOHMnXqVKqrq0Ned+HChYwfP54RI0Zw2mmnsXPnTgAeeughhgwZwogRIzj33HMB+OSTTxg1ahSjRo1i9OjR7N27N0F3I7FoSGoU6PxAUeB3/13G8k174nrOIT07cttJQ4Nuv/vuu1m6dCkLFy4EYPbs2SxYsIClS5f6Qi+feuopCgsLqa6u5pBDDuGMM86gS5cufudZtWoVL7zwAk888QRnn302r7zyChdccEHQ61544YX85S9/YdKkSdx666387ne/48EHH+Tuu+9m3bp15OTk+ExT999/Pw8//DATJkygsrKS3Nzclt6WpKAzhSgwxlILOlFQlOQzbtw4v1j8hx56iJEjRzJ+/Hg2bNjAqlWrmh3Tr18/Ro0aBcDBBx/M+vXrg55/9+7d7Nq1i0mTJgFw0UUXMWfOHABGjBjB+eefzz//+U8yM62x9YQJE7juuut46KGH2LVrl6+9rdE2pU4yGpqn7M+EGtG3Jvn5+b7Xs2fP5sMPP2Tu3Lnk5eUxefJkx1j9nJwc3+uMjIyw5qNgvP3228yZM4f//ve/3HnnnSxZsoQZM2Zwwgkn8M477zBhwgRmzpzJoEGDYjp/MtGZgqIoKU+HDh1C2uh3795NQUEBeXl5rFixgi+//LLF1+zUqRMFBQV8+umnADz33HNMmjQJt9vNhg0bmDJlCvfccw+7d++msrKSNWvWMHz4cG644QYOOeQQVqxY0WIZkoHOFBRFSXm6dOnChAkTGDZsGNOnT+eEE07w2z5t2jQeffRRBg8ezEEHHcT48ePjct1nn32Wn//851RVVdG/f3+efvppGhsbueCCC9i9ezfGGK655ho6d+7MLbfcwqxZs3C5XAwdOpTp06fHRYbWRrx28rbC2LFjTbKK7Iy64312VdXz7S3HUpCfnRQZFCUZfPfddwwePDjZYigR4vR9ich8Y8zYcMeq+SgK2pj+VBRFiRpVCjGgukFRlHRFlUIUuD1ThbZmclMURYkUVQpRsLfGWk6vKkFRlHQlYUpBRJ4SkW0isjTI9vNFZLGILBGRL0RkZKJkURRFUSIjkTOFZ4BpIbavAyYZY4YDvwceD7FvSqHWI0VR0pWEKQVjzBxgR4jtXxhjdnrefgmUJEqWeGPUgKQoKU/79u0B2LRpE2eeeabjPpMnTyZciPuDDz5IVVWV7320qbiDkaopulPFp3Ap8G6yhYgY1QmK0mbo2bMnL7/8cszHByqFdE/FnXSlICJTsJTCDSH2uUJE5onIvPLy8tYTLgiqExSldZkxYwYPP/yw7713lF1ZWcnRRx/NmDFjGD58OG+88UazY9evX8+wYcMAqK6u5txzz2Xw4MGcdtppfrmPfvGLXzB27FiGDh3KbbfdBlhJ9jZt2sSUKVOYMmUK0JSKG+CBBx5g2LBhDBs2jAcffNB3vbacojupaS5EZATwJDDdGLM92H7GmMfx+BzGjh2rfbKiJJFr37uWhVsWxvWco7qP4sFpDwbdfs4553Dttddy1VVXAfDSSy8xc+ZMcnNzee211+jYsSMVFRWMHz+ek08+OWjSykceeYS8vDy+++47Fi9ezJgxY3zb7rzzTgoLC2lsbOToo49m8eLFXHPNNTzwwAPMmjWLoqIiv3PNnz+fp59+mq+++gpjDIceeiiTJk2ioKCgTafoTtpMQUT6AK8CPzXGfJ8sOWJBHc2K0rqMHj2abdu2sWnTJhYtWkRBQQG9e/fGGMNvf/tbRowYwTHHHMPGjRvZunVr0PPMmTPH1zmPGDGCESNG+La99NJLjBkzhtGjR7Ns2TKWL18eUqbPPvuM0047jfz8fNq3b8/pp5/uS57XllN0J2ymICIvAJOBIhEpA24DsgCMMY8CtwJdgL95tHpDJHk5UgF1NCv7M6FG9InkrLPO4uWXX2bLli2cc845ADz//POUl5czf/58srKyKC0tdUyZHY5169Zx//33880331BQUMDFF18c03m8tOUU3YmMPjrPGNPDGJNljCkxxvzdGPOoRyFgjLnMGFNgjBnl+WsTCgF0pqAoyeCcc87hxRdf5OWXX+ass84CrFF2165dycrKYtasWfzwww8hz3HkkUfyr3/9C4ClS5eyePFiAPbs2UN+fj6dOnVi69atvPtuU9xLsLTdEydO5PXXX6eqqop9+/bx2muvMXHixKg/V6ql6NbU2TGgOkFRWp+hQ4eyd+9eevXqRY8ePQA4//zzOemkkxg+fDhjx44NO2L+xS9+wSWXXMLgwYMZPHgwBx98MAAjR45k9OjRDBo0iN69ezNhwgTfMVdccQXTpk2jZ8+ezJo1y9c+ZswYLr74YsaNGwfAZZddxujRo0OaioKRSim6NXV2FJTOeBuAz2ccRa/O7ZIig6IkA02d3bbQ1NmtTFtTpIqiKJGiSiEGVCcoipKuqFJQFCUidIbcNmjp96RKIQb0t6Hsb+Tm5rJ9+3ZVDCmOMYbt27e3aEGbRh8pihKWkpISysrKSIU0M0pocnNzKSmJPb+oKoUY0MVryv5GVlYW/fr1S7YYSiug5qMY0Bm0oijpiiqFGFCdoChKuqJKIQbU2aYoSrqiSkFRFEXxoUohBnSeoChKuqJKIQbUeqQoSrqiSiEmVCsoipKeqFJQFEVRfKhSiAE1HymKkq6oUogB1QmKoqQrqhRiQGcKiqKkK6oUYkBzHymKkq6oUlAURVF8qFKIATUfKYqSriRMKYjIUyKyTUSWBtkuIvKQiKwWkcUiMiZRssQbVQqKoqQriZwpPANMC7F9OjDQ83cF8EgCZYkr6lNQFCVdSZhSMMbMAXaE2OUU4B/G4kugs4j0SJQ88URnCoqipCvJ9Cn0AjbY3pd52hRFUZQk0SYczSJyhYjME5F5WiNWURQlcSRTKWwEetvel3jammGMedwYM9YYM7a4uLhVhAuFmo8URUlXkqkU3gQu9EQhjQd2G2M2J1GeiFFHs6Io6Upmok4sIi8Ak4EiESkDbgOyAIwxjwLvAMcDq4Eq4JJEyRJvdKagKEq6kjClYIw5L8x2A1yVqOsriqIo0dMmHM2phk4UFEVJV1QpxIBR+5GiKGmKKoUYUJWgKEq6okohBnSioChKuqJKQVEURfGhSiEmdKqgKEp6okohBtR8pChKuqJKIQZUJyiKkq6oUogBnSkoipKuqFKIgUa3agVFUdITVQox0OB2J1sERVGUhKBKIQbqG1UptBbvLNlM6Yy32bCjKtmiKMp+gSqFGKhrUPNRa/HqAqvExneb9yRZEkXZP1ClEANqPlIUJV1RpRADaj5qfXRupiitgyqFGKhvTO8u6ut1O3hjoWNlVEVR0pyEFdlJZ9J9pnD2Y3MBOGVUryRLoihKa6MzhRiob0hvpaAoyv6LKoUocIn1f2VtQ3IF2Q+RZAugKPsJqhSiIDvTul07q+qTLMn+R3p7cRQldVClEAXenEe7q1UpKIqSnqhSiAG35j5SFCVNSahSEJFpIrJSRFaLyAyH7X1EZJaIfCsii0Xk+ETK01K8qsCtaVIVRUlTEqYURCQDeBiYDgwBzhORIQG73Qy8ZIwZDZwL/C1R8sQTnSgoipKuJHKmMA5YbYxZa4ypA14ETgnYxwAdPa87AZsSKE/L8SiDRp0ptBqiYUeK0qokcvFaL2CD7X0ZcGjAPrcD74vIr4B84JgEyhM3jCoFRVHSlGQ7ms8DnjHGlADHA8+JSDOZROQKEZknIvPKy8tbXUgvxjNV0Hx4iqKkK4lUChuB3rb3JZ42O5cCLwEYY+YCuUBR4ImMMY8bY8YaY8YWFxcnSNzIUUezoijpSiKVwjfAQBHpJyLZWI7kNwP2+RE4GkBEBmMpheRNBcLg1QXqaFYUJV1JmFIwxjQAVwMzge+wooyWicgdInKyZ7dfA5eLyCLgBeBi0wYM9jpTUBQlXUlollRjzDvAOwFtt9peLwcmJFKGeKLrFBRFSXeS7Whuk6j5SFGUdEWVQhR4LVttwMKVdugtV5TWQZVCDKj5SFGUdEWVQhR4VUGj2o8URUlTVCnEgOoERVHSFVUKUeC1GqlPQVEUL5+uKmfL7ppkixE3IlIKInKAiOR4Xk8WkWtEpHNiRUtddKbQemg+PCXV+enfv+aEhz5NthhxI9KZwitAo4gMAB7HSl/xr4RJleIEczSv3lZJfWPqJ0a6b+YKFm7YlWwxFCVt2L6vLtkixI1IlYLbs0L5NOAvxpjrgR6JEyu1caq8tm1PDcc88Am3v7ms1eWpbWikdMbbvPj1jxHt//CsNZz68OcJlkpRlLZIpEqhXkTOAy4C3vK0ZSVGpMRSvrc2puPsfgQn85G3bvOXa7fHdP6WsLvKuvYfP/i+1a+daNRSpyitS6RK4RLgMOBOY8w6EekHPJc4seLPrqo6jv/zpxxy54f8/bN1HHjTu/zhreUx1Vt2Mh95i8FoJ6YoSlsmotxHnhxF1wCISAHQwRhzTyIFizezV5azfPMeAH7/1nIAnvxsHU9+tg6Av50/hkNKC9m2t4blm/bw2rcbufO04fQrym92Lic9Im1EK7S1yCl1NCtK6xKRUhCR2cDJnv3nA9tE5HNjzHUJlC2unDq6F9f+e2HQ7b98fkGztin3z2bt/x2PyyV+aRacZhfezktXOyuK0paJNEtqJ2PMHhG5DPiHMeY2EVmcSMESwQf/cyTLNu3hgOL2vL5wIxku4au121lUtjvoMXWNbnJdGX5tTh2/yzNTSHWVoDpLUZRQRKoUMkWkB3A2cFMC5UkoA7t1YGC3DgAML+nka69rcHP43R9RUdk8rKy2wU1uVoZfZx/Kp5DqM4XUlk5RlGQTqaP5DqxiOWuMMd+ISH9gVeLEal2yM118/dtj+NVRA+ic5x9U9dtXl7B1j/9qRad+3zdTSEKvG80l25pPQVGU1iVSR/N/gP/Y3q8FzkiUUMnA5RJ+PfUgGt2Gv81e42t/e8lm3l6ymSuO7O9rCzUbSPU+N8XFC0HblVxR2hKRprkoEZHXRGSb5+8VESlJtHDJ4JqjB9KjU26z9sfnrPW9bgwVkhpnrdAQwQrpaCJ0Ul1pKYqSXCI1Hz0NvAn09Pz919OWduRmZTD3xqP9ZgaBuEP00/Hsc/+7aBMDbnqXNeWVcTxrW0WDUxWlNYhUKRQbY542xjR4/p4BihMoV9K5MoRScJoNeJvi6Wh+b9kWAJZt2hO3c5oEm2H+9+VFXPbsNwm9hqIoiSPS6KPtInIB8ILn/XlA6+dzaEU652UH3RZqEXQ8zTPesXE4k1R0juaYxYmIl+aVJfYCiqIklEhnCj/DCkfdAmwGzgQuTpBMKUGGK7i5wm0MO/bVMeTW95j/w07AVmshjjL4VklHun+U5//XV5El0EsN1BmiKK1BRErBGPODMeZkY0yxMaarMeZUIog+EpFpIrJSRFaLyIwg+5wtIstFZJmItIl03G5j+HrdDqrqGjnjkS8onfE2u6qtNQ7xdDQ3zRTidkq/c/32tSXxO7GiKGlBSyqvhUxxISIZwMPAdGAIcJ6IDAnYZyBwIzDBGDMUuLYF8sSdP50z0rHdbZqijbxs9lRecjIt7amp5w9vLae2oTGq6zelU4pMK+hYWlGUltISpRDOWjEOWG2MWWuMqQNeBE4J2Ody4GFjzE4AY8y2FsgTd6YPcy4Z4Tam2YcPVarzTx98z5OfreOV+Rujun5CZgptTHVEaUFrE2zdU8NBN7/L0o3B06sobYN0XAzaEqUQ7m70AjbY3pd52uwcCBwoIp+LyJciMq0F8sSdYH4Ft9s42PuN719jDK9/u9E3M2hotLY1hIpldUAiXCUdzXOZzGd44r0f86c0rPkQLbNXbqO2wc2zX6xPtiiK0oyQSkFE9orIHoe/vVjrFVpKJjAQmIwV0fSEU+1nEblCROaJyLzy8vI4XDYyMoIMU41pPk1qminAJ9+Xc+2/F/LH960O0Ktboq3d4JsphNkvmtF/Msc1G3ZU8+eP0iY7SsyIrrlIG9JwohBaKRhjOhhjOjr8dTDGhAtn3YhVy9lLiafNThnwpjGm3hizDvgeS0kEyvG4MWasMWZscXHrLY9wBZkpNBrTzKzh7e/dxviqsG3x+Bm8I/5AnVBV18CBN73LTM96hGZEuErauzmSriYdp7uKosSPlpiPwvENMFBE+olINnAu1qpoO69jzRIQkSIsc9JaUhynBWq+0XqoNQwB7zfsqKau0c39M1cC8M36HeyIsgD4j9urmLsmrZeMKErKko5DrEgXr0WNMaZBRK7Gyq6aATxljFkmIncA84wxb3q2TRWR5UAjcL0xJqV7uEyXOEYfuUPohKYMqiag3Xus1X7Wo3MZ2LU9H1w3CWgyMzids7qukSuem8enqyp8bZE8oOn4ELdV9Lto+6TjzDthSgHAGPMO8E5A26221wYrtLXNVHDLzBAa3aaZXXjZJiuSxG2aO6GbkuUR0N7ckbxqW6Vtu+eFw3P3+eoKP4UQKWn4DCuKEkcSaT5KCx44eyQFthoLWRkua1YQMFN47BPL6mXvdL0vA2cEOLQ7jTiadEJcY1KVFEHdzW2fdPw5qVIIw+ljSujbJd/3PjvD5ZkpOGPv+L0dvTiU6lxfsY8Gj83JbZxH8MFmGMGIyNGclo9xE5W1Dfz141U0RhnppSiKhSqFCJgxfZDvdWZG6LUDTs2BnXtNfSOT75/NdS8tBCxF4ljiM4RPQXHm7ne/4/73v+fdpZuTLYqyH5CO5lhVChEwvn8XLhjfB4Cte2oB+OMHKx33NQGrnXdV1VFTZy1i83b89Z7COUs3Wimxy3ZWs3VvbbNzRTtTiIS2+hBHKndVrXWva+ujWyiYDNroV6GkOaoUIiQv298n7+3QAwnsvEbd8QHPzv0h7PmP+eMnzdqizX0UCW2hI6qpb6SuIcZOPYj/JqVQZ0Kr8uaiTfS/8W1q6qPLPRYJ6WiOVaUQIZHm4GlwG371wrdA8w441IrmascHNrI0F14i2e3jFSmVXsqRQbe8x7QH58R0rJrclEDufW8FbgPb9jSfjSvNUaUQIa44ZGYzAf+Ho2mmED9+859FcTxb4lhbsS+m40KF8SpKvEnlCWmsqFKIkGB5kEIS8MB4TRpRRxM5OaEdxEllq8T2ytYZpSUkjDdBpGOHksqkY8bdRKBKIUKC5UGKhlDptZ0INVNoax3KwX/4MKbjok0elwjnfLzRvik6Nu2qZvbK1Dd7pguqFCIkDjrBpwwiDaGXKH0K6Ui0I371KaQfJ/7lMy5++ptki+FIOv42VSlESDx9CpFGxjSNeiPbf9veWtYH2OIXbthFZW1DpCK2eVyeJzodf6z7K9EmiQyGPhORoUohQoIV3AlF4Cg3Zp+Ch1Vb91I6421Wb6sMah+dfP9s3+uqugZOffhzfvn8ggglTj2irz3gnSmkfg/QFmRMBxLpS0jH71CVQoQUd8gB4JeTD4j4mFVbK/3eR+9T8K/D8OaiTQC8sySy1bo1ngVcS8p2RbR/OpHKo8LmVfsUJXVQpRAhZ44p4S/njebXUw+K+Bh7xlOwF+KJ7Hjv7MSbxyfajs5b/jOWWU4oNu+u5uKnv2ZvTX3M53juyx94N0LlBvDonLWc/djcsPu5EhDGqyjBSOXBR6yoUogQl0s4aWTPFnWwj36yBoh8yunNs1QfUNs5Ugm8D2yweg6x8uAHq5i9spy3FseeX+iW15fyiyjMWos27OLrdTvC7hetHyZeVNc1+mpyK6lJWzX17K6qZ/At77VaMS1VCkkg3Exhp8exlp1hfT31DbE9zN4ZhitIOdBY8YbnpmIqiWTVPx5863tM/VNsq7CV1uG3ry2J+znj/Quoa6xj095NLNqyiI/WfsSLS1/kto//yGbzDy567Upe++61OF+xOQktsrO/k5Uh1DcGOJvdJmS6C4Axf/iAdXedQKYnlMZrBop2pONVCt7ZTbxGz97ReCpnp06Gvvphe1V0B6Tw/UtHPl/dukUdjTHsrt1N+b5yKqoqKK+y/q+oqrDaqiuabdtT65xTjUyhprojy8tHc9rg0xIqtyqFBBKoEAAajQnbYXm3+8xHDueJhIYApRCuE1+0YRfZmeEnj64kmWgiIRVl21fbQLusDN8Ma3d17L4YJbnsrtnN+l3rfR152e6t7Mr8ikbZw9n/ebZZ59/gdg4Hz83MpTivmOL8YoryihhQOICivCKK8ooozrPavNvWbBGufn4VEwd046YjD034Z1Sl0Mo0ug27qiOLu/Z25g2NsWUM9R73444qtuyuoSA/K+T+pzz8edBtlZ6OLcMlTeYom5a5+90VPp9JMgmM2AqHMYbvt1ZyUPcOCZGnuq6RobfN5PKJ/bjphCF8v3Uvv39reUKupcSHvbV7Wb1jNat2rGLV9lXW/57X5VXlzQ/IFFx0YPHWHhTnFzOgcACHlRzm2MF73+dn5zc/TxC27ypHWBvHTxgaVQoJxtDAlpwbyHb3J8c9iI9WdeeX/9gcke3bO9htCOjh3IaIOpZHbJ30KwvKuPSIfg7XaF5T2olht83kjDEl/PHskY4+ikCF0Og2cY96ioaPV2zjZw6fN5B/f7OBGa8u4blLxzFxYHHc5dhba80KXvt2IzedMKRZmLISOZE+q4E4/db21e0L2vFv3bfVb9+eHXoysHAgpxx0CgMKB9C/oD/d2nejKK+I3IzOTLl3HkIGK64+IebPFoybXlvC3hprttFajnJVCgnGzV5cJo99GZ9QmfkuJ/7nT7hyO5DtPpAc9yDP34G4aD5y8D4EdQEzhXk/7GB9GPt1o9vw6oKN/udzeKaMCb+4xzsjeGVBWYBSCP6Q1jW4aZedEfrECeSz1RV8tXY7h/bvEnK/ZZssG+66in0JUQreW9RUktVWrjXuV0tvInlW7VTXV7N6x2oqGuewO3M9DbKJSc/cy+odq9m0d5Pfvt3bd2dA4QCOH3g8AwsHMrDLQAYWDmRA4YCQo/rd1fUIiXvOn//qx4SdOxiqFBJMBgV0q/s9Bjf1soFa10rqXCuoda1gd+YCEANGyDK9yXEPItt9EDnuQbiN29ehBDqmI4n6CbSpiziPNCLpmBoDztVktw9+TE19Y1KUgj0dSTTpERLlgvB+V5HcMzt7a+rJynCRm5U8xZoITvrLZ5w4ogdXTop8EagXp1tX01DDmh1rWLVjlTXyt436y/aUNe2YBS7TiQb3UI7tf2yzjr9DTmLMh7Fw/8yVHD24K6P7FCTl+qoUWgnBRbbpS3ZjX2icCoCbfdS6vqfWtYI610qqMuZSmfk+AJ3umkGeDKI28wCenn8Q10/vZVsRHf56gbsI4mhndxtDRhBTVm1DI9kZLl8UkxevwzRQWfgf2/JymDOXbYn6GPtIMpR8TvsnAu+ty4hgdmVn+O3v06cwjzn/OyVRoiWFJRt3s2Tjbq6cdAAN7gYq6yrZW7uXvXV7/V7b23ZlLsAt1Vz25ivsq6+09qvby4+7f2TD7g1+g50u7bowsMtAppRO8XX897y1i+27CnGRz+c/i7OJJwGDib/OWs1fZ61m/d3xN0dFQkKVgohMA/4MZABPGmPuDrLfGcDLwCHGmHmJlCmVcJFPO/do2rlHA9ZIvkE2UetaQW3DCna6VlKfOR/ETdF9v6Mot5TqrP6sqjyYOulNlukbdOoa2PmIOEfkeJuq65ovvDro5ve4+YTB/OTQPv5y2zq4v3+2jsMPaG6iicdCriufmx/1MfY+vqbezfJNexjSs2PQ/RMdpOSd5cViC/9xh2UirGuso3xfOdv2bWPrvq3W/5XW/9uqrNe1jbVkujL9/rJcWWHbsjKC77OmvJr+RR3pkJMTdD8Riahj977fmLMVt1STd2cd1Q3Vkd2ITEFoxwdrC2if3Z4OOR3okN2BI/ocYXX8tlF/Qbvmo+u/vTOLnUQZLrwfkzClICIZwMPAsUAZ8I2IvGmMWR6wXwfg/wFfJUqWRDH3xqM47K6P43Y+Qcgyvchq7EX7xqMBcFNNnWsVow6oYEPlQlZUL2Deno8hF8Tkku0eSI7H5JTjPogMrB9FYGcnBKnLgGHDjiom3jvLUabXvt3IOYf09r1fW15JdV2D7xrBHN4VlbX07eJsiw00h328Yit9u+TTq3M7x/395HVwNu6urifTJeTnZPqN/G97Yyn76hr57IYplBTkhT13KKrqGvhq7Q76dsnjjYWbuPaYgRF19F7l3LRWxLrnhmp21P7A5z/ua97ZV21jS/YKGmUXhffsY2fMvmaVAAAgAElEQVTNTsdz52bm0i2/G13zu9Iuqx1V9VU0uBtocDdQ31jve+1rczu0NdbTaOK7ElsQX8dt78T7du7L6o0FCLn8/PBhvvYOOZ79PK8D24bcOgtBWHXLdLIyUmu9bVtdJR2KRM4UxgGrjTFrAUTkReAUILAX+T1wD3B9AmVJCLmZibf3umhHrnsEQzr04qhel/Pw7NUM613L15u+pNa1klrXCvZkvgZi/bAz3d2Y8PihnDhoMrUCWaYPQrY1U3Cw6BgDa8pDR8TYzUdH/fETx/ZA1mzbx8F9Cx23Bc5ifvZM0+SwfU7oR7LRbXzrN7yM/N375GdnsOyOaX4d9T7P7GdXVT0lQcyzkQzg73lvBY/M9o+uOn98H7p2yAVg6cbdGAxudrN46+KmUfy+baysKKMiazF7a/dyyBONrN+5mYrcbSD1/GMd/GOd/7UK2xXSLb8bkEG2u5Tzh4+ia35XurW3Ov+u+V19iqB9dvu4JNczxtBoGv0UxdqK3Zzy8Kf06pzNi1eOa6ZQvErHbdy+jt/biedl5QWVq3TG2wDcNzVy08j+VFckFdbXJFIp9AI22N6XAX4rL0RkDNDbGPO2iARVCiJyBXAFQJ8+fYLt1urYnZr9i/Jjrisc6bW+XLsdQVi2IZd8JpPfOBkAN7XUudZ4fBMr+GrTZ3yx+U3IbTr+qtnZ/ObzXPbluhCThZCDkMWRzxRT15DJ1uwaS3mYbOt/shCyWVndnrs+e5/dmWXNti2u+JFqV4XjcTtqtmNMiWPnEGoNQbjaD8GO9SoAp66opaYsu0IwuGmUcj5c+x6b961h8dZlvLdyHuW5azGyj5GP+h+b5crC7eqEUEBR3gG0d5WyYI+bDDpzSO9Srjt6nK/DL84rJivDWkvi7Tz/cnxknednqypYW1HJhYeVRv35RIRMsUxCXrq0yyaTQnIz8uhXED60186sldvoU5jHAcXto5YlFKk4Ko93H54COiF5jmYRcQEPABeH29cY8zjwOMDYsWNT4LZ5sPVA/YvbJ1Qp1DW4WfCjcwpsFznkuoeQ6x7ia2uQcmpdK2iQLRjqOOyAjvQtyuL5r1djqMNQj5E68rM6UFtfiVsqrXZp2maoo7KmnnvnNoDDure/rwBynOX9xcdw3Zx2lHYupV9BP3ZkCZmmG5nu7izY3A03lbiIvtMI66h10ArXvbSIT66PzmHb4G5gzY41LC9fzu7M16mXH6l3baBeyjBSy/mvW/vlZXShsa4n+WYyWe5ePHH+VLrmd+WDpdXMXFzDsxdPZtqfP2VAp/a8e/4kXp5fxm9WLQJgcKeeHDdgtN91N+2qZvPumqhkBbjg75b1NRal4ERLfmSXeKqkxdtRmgodZiDxFikV8oklUilsBHrb3pd42rx0AIYBsz2jye7AmyJycltxNtvXZmVnJjaMJdqHJdMUk9nYFHd/5sAhnDqqJ+997l8red0KK/FejyCrpof16shjF4zhsHve8yiTOozUY6jj7LHdeHH+2qZ227ZjhnTCZFRQXr2BTXs3UpmxCiOW0jzsqf+DduAy+Yx57EDKs9uR6e5mKQ3TjUzTnQzTFZeDxgl3H5wWKoXKSdTgrqVO1vHl5rWsnVXBdxXfsbx8Od9v/556tycdRRZkuIvJMr1p7x5Glrs3T11wGhNLR3HFMyuY90OTzf+MIVZH+NNH3gYyfBFQ3mclnPxH3jur2WLFZOL0VBtj+PNHqzjz4JIW+2pag7ZUviIVvvlEKoVvgIEi0g9LGZwL/MS70RizGyjyvheR2cBv2opCAH/zkdcB9vtTh3HL60vjfq1Q9vtICOZohuaL4/yPExAXLnLx2aM8J+qedwC57lzH447seRD3zVwJWCPG0hlv00glDbKFyybn89dP59IgW2mXUUudrKA6cx5G/NcVZJgCMt3dyPAoikzTjY/WZTGs60B6d+rtZ+7wyRukA6isq2RFxQqWly/nu/LvWF6xnOXly1mzYy0m182jS8ElLvoX9GdI8RBOPPBEhhQPYXDRYM74y3pc+Hd+h/Q4jMJ24R3j3qznvmfF9iU4fR/xVghfrK6gfW4mI0o6x+2ca8orefDDVXz03Tb++6sjYjrHuop9TLl/Nv/5+WEcUursewokUJ++uqCM5778gdd+OSEmGeJBvH0AoQYNrTWJSJhSMMY0iMjVwEyskNSnjDHLROQOYJ4x5s1EXbu1sHdAmQkuDvzu0uhj9u08/9UPMS0m27qnho9XbHPc1ugOrkzqHRRNBu3JMAMYVTSSTg09ANi42nI+Gdw0sosG11YaZAsNstX3V+taQZV8CuLm5Bcfss4lGfTu1Jt+nftRkZVJpunG84t38cNeFzWuMpu5x/rrcFdTzhohk575/Tm872h6Zh3D8g0duWbiJK4/5ihyM5sruXaZFc3WXUQ6c/NmuHU5rGiOhI++28rRg7tFdYydnzxpmZWiNeV4O7tQPqHq+th9NZ+tsr6PNxZujFwpBNy7615aFPP1U5UUsB4l1qdgjHkHeCeg7dYg+05OpCyJwH+m4F8lLdVYU76PG1+NPp/8tr21QWc+T3y6zrEdYHtl8NXETh2q4CKTQjLdhcDgZtsNDTRIBRMOcnPcCBfrdq2z/nauoyZjBY2ykwte+6e1s8fqJCaHLFNCrnsoh5aM4OojJjOkeAhT71+BVGfy0m9O4LY3lvLD+h/o3WGwo0IAyHQJtYHy+NJXBP2YgK2mhcv72e2fOTyrtlW2SClEwrQH5zD5oK7MmD7I1+YV00nGeFhjvPchmvoXqdBhBhJvkVLhM+qK5hZg7xC85qNIdMKVR/Zn1bbKoCPwdOC5L3/wvX541mq/bbE8+EImWaY7/Tv05tIxI/y2lc54Gze1NMo2GsSaUWWZPmSYYsRTR2rLhgxOGzzNcy5/eSD0j9vlkNgv1Gf4p+2zB3Z+0X72UF2m222NnaNNPGiM4fWFGzl+eA9yMjNYsWUvK7bs9VcKkayab0EP1jQTieKYmK/WdkiFCKvUWgnSxhCEl648jD+cOoxBPazcKb0LwzveSovyOW10r0SLlzJ4fQteWvLg2xcvfbG6whe66SKHLNObdu5DaOc+hEzTzacQAKo8pg57R/bqgjK27rHmAO8s2cwvn3deQe3U6X6xxrr2pl3NI4Vuts2s3AGdn32WFMldCNVpHnnfLEbf8X4EZ/Hng+Vb+Z9/L+LBD1dFIID134/bqyjbWRVWJjt3vr08aNr3UDORYARTQsmM7Y/3pVPB0KAzhRbgEhjXr5Bx/QoxxjC4R0fG2JJY9SvKZ51DmKrbmJQIPUsWLXnwvXUP3l68mav+FXmNZ19yQdu17Tbpb9Y7rxoGfzOhlxkeU9zGXaFTNQR+z9F2YE7X9lK2M8I0EQHsrLJMexV7A41idvzlPPI+a8W73TcR7pM88ek6Dj+giCmDulr72xWiz/wWhfkoSLvbQEY8bFoBHPXH2YwrLeTuM0aE3zlOpMLiNZ0ptAD7D1ZE/BQCBP+C3cZad5Cu9C8KXUCkJc99Q6Ob+T/siEoh2IlFGYfqmMMScLnlm/fGfq444R28hzI7hb5Nkd8PezSV/Zy+mUI05qMgMiVqgLW2fB8vfrMh5D7xNvckXyWoUmgR4R7odtlNE7FuHXM43WMycol/GOgdpwxNiHzJYmxp6JS/LfkRry6v5IxH5sZ8fLhAgCc/XcvKLf4dd0tqBdkvt2rrXl74Orr8+PaRdNnOKhb8GHxGY6fRbfh+q7MC8q2dCKUUvNcPdZEIvkb7/fYzncXyDLSyUogX2/bUNHumguGUiqa1UaXQAkJNfQvzs3niwoN9GUZdIr5RU352JrX1Td/+0YO7MbpP/OLIk024pGVLN+6O+dz//DL2oiNfrt3OoFveC7nPH97+jhMe+hSwwkFfXVDWogpy9g5ryx5//4MxsKuqjuv/s4iqOuf0HvZLH3HPLE7/2xehr+c27NhXx8VPf83UP80Jug80pfMOJXd5SBNTeL5YU8HCDbs852y+ParooyBaIZpF7jW2MNp/f/MjPwYsbKyqa+DdJZsjlikSxXj43R9z3IPO30Xz0yVfwalSSAD/uvxQ3rlmIiUFefxyslVMxCVClSc/T7vsDN9MISfTRc9OzqGQbZXszNCPVbgpeaKIdJTuVd6XPjuP615a1CLzkXekvGlXtWOn+OePVvGf+WW88LXzPYnkyjttxYQajeHsx+by6aqKoPvbM7cGNXF6xix7ahoc06oDrK3Yxx8CsuQGKrd/zP2BUz21v/2qznleRqNvvcfUN7rZYksFEs1MwTsYc7sNN7yyhNMf8a9LfvPrS/nF8wsiHrhEcuVoFiSmgqNZlUIMXDUldNWoww8ooruno7fHs99y4mCmDunGkQOLfT6Fyyb2QySa8VLqk5nE2syhyI4x7bKrBb+Sx+dYBdcrKuvYFyLZ3+/fWs7u6vpm7ZE4YrfubeogG92G1dsiy3orEtycZu9oN+z0H03bRXryM/+1KkNunRn0uva+OzAqKxK8h//21SWMv+sj27kiP4f41otYB1UErKfZ6HHe76lp/l20BqHMaq1lJVOlEAPXHzco4hWiTeUYhb5d8nn8wrG0y87g1FG96JCbyVkHW+mh4pECOVUwJvYOOJHkZMUm04YdsUX5AMxdu933eq9DR2Ofhby3tLnZ4h9z1zP2Dx/4tdXUN/qNyL2F3SGyxZO+mYLNpBmIvQP6at2OkOd75nNLMYTrSJ0dzdbnP+exuTw+x5aN1hge+mgVP2zf59cG8P7yrX7njWam4L3bwY7wmgpDLNb3Ix1DUlPvl5tmdMy10otOOci/KHyfLnksuf04SsNE6gA8d+m4hMiWKNwm+gVVrUFOFPUvghUPagk3vOK/otzgbx5y6hDWlO+jorLObwR59B8/8RuRP/vFet/rYJ283QRkjz5aXNZkJvlg+Vafzd1eytS+on3Wim0cbaupAXD7f5fz1drtjLg99JoJf0ez9b/383+1bgf/984K3/aNu6p54IPvmXTf7KZjfMcGhPkG6cA37qrm9jeX+d3XpnM4H+NTCklyXqtPYT+gID+bL2YcxS0nDgm5X6gudOLA4rB2+gfPGRWDdInBbUyLInYShb3zDMffA8wikfLkp2uj2t+vpnSIYWJ9Y9O2wLURu6qaRuhut3E0323a3XSMt8N7bM5azn6sKZLr8n/M8ynDYJ3iM0Hu4bcbnNO6e5m3fodfd9dkPvKX1XsPGhqbXz/akNQbXl7MM1+s95U1hSYFEuwYrzyR1PeG8J34p6vKQ25vdr7k6wRVCq1Bz87tyGypOSXMw3LKqJ6+1wV5DsUPWplQ4Y7JojVSUv/h7e+i2t/eKc4KkfYkmLMX4LPVTU7lRtO8Mh34d7KhlI+3Aw1m2/7ke+dO7v6AVeuBnPnoXL+O2O7X8DvP+9Z5nDrlr9ftYMit77Gnxt834zaGH7bvo3TG23y3eY+vvcHBBtRoDMs37fFru/e9Fb5Fpt5bF1gyNlIe+mgVby3eBFgr7n/696992x54f6Uv2umxT9awpKy5M1uVguIjnEsh3IjE3rkcY0ugduWk/i2SKxaMMSlpPkpF7HfpoxBK4ZJnvg66zU6j21BT37wztFe0C6UUPl1VgTEmatt2JArXbubx7r+2vJKBNzXlzHxk9hq27alx7JSv+tcCXwSfHbeBmcusnFevzC/ztTtFjT36yRqOf+hTv/Uef5u9hqueX+B3TKSfP7ATf+CD77n6X98CzUOQH/p4NZf9wypAdNe7Kzjpr585fJbkawVVCm2EaJ6VUCGUD5w9kttPCm7KKu6QQ48Whsi6TQtXAe8nuI2JeHFwsKp7gZzzmPPCPruPJFzH8+XaHTGPlENhH9h4cyLNXLbVzzQGsGzTnojNN2B9nhWeleL2x87pGfSOzu+y+S+gKcuxd4Ybj2zHTooyMAV7IMlXCaoUUoZwQalRRVjYThX44y4pyGNk7+AL5YIlMIsGg1GlEAG19e6oFm9FwvogVeYW2mz+4Z6lmvrGhETB2M8ZqtNdvnlPVJ2y2xhe/dYq6ljfaHzO8lCP4JKAdQh7axt4b+lm34K+MlsY7ikOI3ovdik/+s4/KsrpM4T7GWvuI6WJMH1DND9S/3hw//DQ2oZGRvcJnoaiJXb3y47o57tmRWXLVsI60a1jkILQbZSa+sZWLxX5/da9zUbmgTS6TUI6J/s5H5sT3CF/38yVEYeEgv9v45kv1vtWrUczMFlbvo+f/3OBb02K3Te0yMH278X+mS59tqloZH2j23GA9eOOKjaFSKKYAjpBs6SmOu2yoq+WZidwVFjrYG+24xT1ESne8NpEPdg/m9CPu95dEX7HNsJnqyuCprdIBMc88EnYhW1gDQwSPVMIR1TmoyAndnJrhdMT8VovVFPfGHSANcmTcdaJVFAKOlNIYR69YAwf/2YSEO3KT8MJI6xyl4EPWU1D6BKKThEbkdI0MkvMkz0pYK1HOhCpryAeRKIQAH7+z/ksKou/XNHMPkKVem1+3uZtby3exKyVzSOlwpnOop0hBdtdRIKawAJnaiu27GGOJ6orZI3mVvI4qFJIEbzd6UPnjfa1Hdy3kB6drOLwXTtEbjoxBsZ56t4GPpjDenYC4NELDnY81holBn/4Th3VM+jspUOuNfFsn5OYCWj3jumVIyqVefGb2BMPBuOwuz+OeN9oXFtHOoy8vRFAzc8b3nTmRGWIFCXBiMQPWN/oZtqDn3LhU19T3+hWR7PShDe2vLh9Dp3aWesM7GGd/7z0UM4b15uHfzLGL3ro7WuO4MPrJjU7n/dQe4TLktun+kw804Z1Z+6NR3HfmSP4YsZRvuOMCT3Nf/Dc0cy7+RjHbdOHdefXxx7Izzy+hXjTOS87IedVmhMqg2qsROM8TsSK8khkCLZ52G0zfTOtrXtqWL1tL+sr9vGyLQTW/zwmosSP9gWVdQ3ulAhJVZ9CinD/WSN57JO1jOtX6DMV2X+WA7t14K7TmypA3f5f60cz1DPyt2NoCq2zP+SBjrcendpx1tjezY4f378L/120yfc+O8PlV/8hP8hMIDPDxa+OHui4raX86qgBCTmv4kyyFx8GRgfFi7BKIcT2dRX7KO6Qw6H/91HQfbwYt+W8DofdoV3f6FafgtJEj07tuP3koXFZ9GUMjPKEnU622eEjicaYMKAL953pX37w6hZ2yJku4fQxzWtSX3fsgUGP6V/clBNq4sAifj31oBbJoERHImYKqUA4B3ao7TmZLq7798KIrhPLiH/UHR/46lck8/YnVCmIyDQRWSkiq0VkhsP260RkuYgsFpGPRKRvIuVpKxx1kFXTNtasng1uN0N7dmLF76dx3NDuvhlHJA/akxceQq7NZ/D5jKO4esoAfnfyUI4b2i3EkcHpXZjHDdMGNWs/uG/w0Njjhnb3vS4pyIvpuooSSLjoulAziW9/3BVy1bmdWM1Ac9c0pSxJ1pqFhCkFEckAHgamA0OA80QkcCntt8BYY8wI4GXg3kTJ05a4+4wRfHbDFPKyo7PuPXrBGKApT05uDOGs7bL9j+nVuR0ul3DR4aU89tOxUZ8P4JIJpXTrmMvjP/V3bnt9J07Yf1S32Xwoc66fEpMMSnSsijBSqa0Rbh1OqM78Tx9+H/F1ogmptdPeE6xhTPLCUxM5UxgHrDbGrDXG1AEvAqfYdzDGzDLGeJcOfgmUJFCeNkN2pium0bE3NXTgUvoXrhjPeeN6kxMm02qiuPCwUgCG9fL3f4SSx/6DsCu3Pl2C35cxtpKm3vKmqVrwR0kO4XwKn6/eHnJ7pMTaodsj+ybe6x9VlQ5FdnoBdvd7mactGJcC7zptEJErRGSeiMwrL48uFe3+hNfcZK9DCzCmTwF3nT4i5MKcT66fzOzfTI67TPbV1D07t2P93Sf4zEZOo7ZD+xVy1sElvjKmTvzr8kOdr2VTMt5Pav+RHda/C8VRhPZGw3Cbwgs1A4qFESXNgwmU2MjPadli0Ej5xGGNRCTUhUiR3loTh5RwNIvIBcBY4D6n7caYx40xY40xY4uL028BU7zwrmkY0rNj1Mf27ZIfUcGfaMlySOP815+M5pqjBzKoe4dm2/JzMrnvrJEhw0+DOUHtTnonp/oLV4xnSI/o700k2Gc9d5wylOuPi59jvKWhuGnqM46J4b1aR8H+7yuLYzquPsQCjVDb4kkilcJGwB7vWOJp80NEjgFuAk42xsQ/Yc5+RL+ifN761RHcOH1wwq9lX2TnxPRhlqPYybbao1M7rjv2QMeZSyT9V7AZjz25nLcjzfJ01k9cGNwf0ruwHW/96ogIrhwcuy/mlFG9/KK+YuGFy8e36Hgv824+JqqFj+lOPLKfJpL6EFlUv22l1e+JVArfAANFpJ+IZAPnAm/adxCR0cBjWAohMre+AsCCW45lvsMismG9OoWt0hYPTh7ZM+T2+84aCYTPd/PhdZN8q68hslFt4D7elc5TbdFR9545ghunD2K0JzTX60AMJk6gvwPgqYsjd6wHlvqMpvSnE3bfSTdPp/7iFeOZe+NR/PUnoRUywB/PGsnMa4+kqH0OU4d0D7v//kJrFFpqCXWtNBsIRcJ6D2NMA3A1MBP4DnjJGLNMRO4QkZM9u90HtAf+IyILReTNIKdTAijMz6ZL+9QYAR55YDEXjO/j15bpK4Ae+kc4oGv7ADt/eK0Q6Du+/riDWH7HcT6HNlj358pJBzDWo3AO6tbcVBWIvXodwBRPaHAknDSyh9/77DCV9l668rBmbf++oml2YP+Mt588lHvPHMGh/ay0J4O6hzeBnXFwCQd5zHO3nTSEr397dMj937hqQthzJprbTxrCuH6F4XdsAYmoExFPUkEpJHRFszHmHeCdgLZbba+d8yUobYp//GxcszavUogoNM/WAdpnATcdP5hBPZp35l7z0ajenfnX5YeGDN29fGI/Thvdi+6ewkGhYr+PGFDEGwubVnIHM1PdftIQ34pyLyeN6Mn/e7FpYVNWZmjldpCDP+XQ/l2aru25KYX52eTnZHK2beV5tAFVmRkuuobJG9U3RFRXa5HhkoR7U7/fmtqhtu8u2RJye6M78VUNU8LRrKQmnSOo9RxsZOd9cEeUBC/o40WCvL78yP5MHNjcNj+kR0eG9erILScOCbuWIzPD5VMI8eLiCf1YcMuxfm2BaSGyAmYKNx3v7+dpn5PJsF7BR/xefeTUAQRTVktunxr0fABvXj2BSyaUOm5LdloLsD5XonP/LN+8J/xOScRbIzsYreFsVqWgOPLZDVPChqh+fdPRjrMEsH7gr181gX9c4rzdjj1SJyeCBXe5WRm89auJIVdEB+PuM0Zw9tgSR+frYQd0cTjC4rTR/tHUhfmhI4IClcJ5h/qb1zJcwlu/muh7//xl/mG23s7RKdIqWPfdITeLF68Yz2M/dc6AO6KkM0cNitwkFgqX4DP7/c8xwdOVRDOqzXQlXik4ccaYtrM8qjXMS6oUFEdKCvLChkJ27ZAbctX0qN6d6RTBbKNvl3zW/N/x/HzSAfzu5KFRyxoNvTq3494zR/p12t5+qKQgj0W3WqPtwL74T+eMiuo6gT6F3DDO/wkDigC46LC+XHP0QF+UTLQD+PH9u/ilCAkksM/1pjvPckXXFXRql8XtJ1nf1amjewY1P0WjFDJcwp/OGRVywWFg2vb3rp0YZM/IOHFED+46fXiLztGaLAlRBS5eqFJQUoIMlzBj+qCwI/Bw9C/Kjyj6yt4Hjrfb8j3tLa0xHbg+IzOM49nL704ZxnXHHuhTChkO6zycRtN/PjcypRV45EPnjubjX0+iXXYGh3pMgccPDx+t5BLhhBE9WH/3CfTtku87NpBhDmtmenVu57hvr87t6NslnztOGRb0uoH3dVD3juR7woG9616G9erI+rtPCCn/jdOtXFzdO+aSneniwXNG0bvQWa54cebBLZ+RBEvVHU80dbaSVnx43aSIfJXeTv+pi8f6Run2dm/Xc9roXvQucO4sTh3Vk9dtjum/XzSWbh6HbqQj5JtPGExpl+aLBn1KwUE5OQXQRLpSO9DR3i47g/7F7QF49mfjqKxtoKh9DqUz3g55nkCxfn/qMHbsq+fDgOL1T1w4lveWbWHRhl3MW7+Tk0f1pENuVrN6CW/96ghfWHCg3fyGaYO45z2rDKt3hnfx4aUcO8QKQf7faYO47c1lvm3ew286fjC7qut4eNaaZvJnBKSWP3V0L95fvoUNO4LXT24p0UQ+9eiUy+bdNQmTJRSqFJS0IlKHqbcDKe2S77emwHu4VzmEMhs9cPYoLj2ivy+9yNGDm9ZJiAgPnD2S615a5Gubc/0Uyitr6FfU3td22cT+jufu2yWf00b34rKJzQsWOUVQSUTL/prPFOzKKzcrI2wSxXGlhXy9fgeBno2czAwGdmvPh99t5ZIJpTz9+XoAurTP4fxD+3L+oU0JkP/tUNXNvk4kNyA78C8mH+BTCt5iVJdMKKWvR5l6FZR3FuHtfC8/sj8fLPdXUl68zvrWKnEJ0a2RCOZbaY3MqWo+UvZLnrhwLJcd0Y9+Aak9vJ1rJNYjl0sYXtKJA4OsgTh9TAlDbeaTPl3yOLhvYUQmMq993amIUktC7QM7lWjNZN6ItFCHFYVZPzOmjxUgcOUkZ4V4xpgSfnt881TrAJkOvg8J2GYPgw62gtmrC1vTrx1N/fNg33FriKtKQdkv6VeUz80nDgka3hmvWPBXfnE4i8OEikaL0yiyR4Rht4GHBvucXkf5FzOO8tnqB3Rtz7WeSCOno/oUWs7mkoJ2XD1lAFcc6dzpD+zWgVV3Tg+ajiUzw8UVRzonRHSUV/wVud1MU11v1VaeMMA/ssx7Fvu9zAjjbL/2mJZVFWxoNDz7s3ERBVMEMzW1hhJT85Gi2MjNcvHzSQc0W6EM8Mj5Y6LOgBqJSSZaSgJ8HJ/dMCXiVOvNlEIQpXjEwCI+XrtDnIEAAAxPSURBVLGNju2yfDOb208aysBu7enbJa/ZuguAcw/pTe+CPCYM6BIyIy80D9mNlLvPGM49762kp81ZHWjys88Uqjy1RXoX5AFNabF95iPb/RjWs6NfGdpArj3mQMb378LqbZXc/PrSqGVvcBsmHVjMpAOLuejwUoCgvpvA0FOv2U5nCorSyohYUVBOZpvpw3twuM0pnSw65Gb5omvOGFMSVe2NwE6lS3tnU9ZffzKaN6+eQPucTN/ovNEYsjJcfHL9FKY6hL2KCEcMLAqrEOxEW+/i8AOKeOOqCX5KJc8TfeQNr7WbjI4d0o2enXKb+WZ85iPbHekWZtU3WJFqF4zvy+Lbp7Li99P8/B9De3Z0LDvrJRqfwt6aBr/3Fxxm+WRaw6egMwVFaaOEC7t0wht2ecO0QUwd2s1vxG0nLzvTtxrdpxSisIlHyle/Pdo3mg/FSSN7Bh3FnzyyF5t21XDuIb15f/lWrprSVFO8a4dcvrjRyvs0vFcnlmy04vy9isveT08b1p3Tv+/F5EFd6V+UT2F+Noff/bHjNTvmWjPGb246hhmvLKG4Qw63e8xCry5olgwaaF7nJBSTDypmtq0mQ2uuN1eloCj7EYO6d+TT/51CSUG7iEf0J47oyeyV5QwoDp9UMFq6tM8h+DryJh46dxQPBokEy3CJTxGEUpR/OHUYpzz8OdDkf7CPvHOzMnjAdo3Nu5vCUx/+yRjHc3bIzeLh8523BbKnur5Z26zfTGbK/bObtV955AH+SsE3s0k8qhQUZT+jd2F0ye/OPLiEk0b2aHE68FjwOrxFBId1fFHhnfEM7tGRvoVW1FmwyDGwHMNgLao7YURzH1MoBvfoyHe2PEtXHtmf0x3SaQRGvwXDF3KsjmZFUVKBZCiE2b+Z7CtkHw+8kwLBcqS/efWEkJXYvIvonKoHhuLbW46lXXYGg255z9d2o4NjPhTtc5o+9/ybj/ElynOqVhhvVCkoipKSxLs8rNep7I08DZfB15td97qp0ZVWLYgyVctdpw+nb2EeP3nyK1/bcFtd7kyXi9F9CnjjqgmOxaDijUYfKYqyX+B1Kke6YC8vO5P1d58QtspgMNbffQIXH15KQZikkOeN68PhA4r4w6n+OZ+8MxRvPq6RvTsnvJYCqFJQFGU/wbvGZEiP8JXr4sXtJw/l21sjW7x4wfi+fu+9YbetXelCzUeKouwX9CvK56UrD2NESeJNMPFg0oHFvLt0S6vMDuyoUlAUZb8h0TWg48mfzhnFr6dWha0uGG/UfKQoipKC5GZlMKBr4qONAtGZgqIoSopw+0lDGFua3NmMKgVFUZQU4eIJzetntDYJNR+JyDQRWSkiq0VkhsP2HBH5t2f7VyJSmkh5FEVRlNAkTCmISAbwMDAdGAKcJyJDAna7FNhpjBkA/Am4J1HyKIqiKOFJ5ExhHLDaGLPWGFMHvAicErDPKcCzntcvA0dLNHl3FUVRlLiSSKXQC9hge1/maXPcxxjTAOyGiJImKoqiKAmgTYSkisgVIjJPROaVl5eHP0BRFEWJiUQqhY1Ab9v7Ek+b4z4ikgl0wl4zz4Mx5nFjzFhjzNji4uIEiasoiqIkUil8AwwUkX4ikg2cC7wZsM+bwEWe12cCH5vWqDenKIqiOJKwdQrGmAYRuRqYCWQATxljlonIHcA8Y8ybwN+B50RkNbADS3EoiqIoSULa2sBcRMqBH2I8vAioiKM48URli41UlS1V5QKVLVbaumx9jTFh7e9tTim0BBGZZ4wZm2w5nFDZYiNVZUtVuUBli5X9RbY2EX2kKIqitA6qFBRFURQf+5tSeDzZAoRAZYuNVJUtVeUClS1W9gvZ9iufgqIoihKa/W2moCiKooRgv1EK4dJ4J/javUVklogsF5FlIvL/PO2FIvKBiKzy/F/gaRcRecgj62IRGdMKMmaIyLci8pbnfT9POvPVnvTm2Z72Vk13LiKdReRlEVkhIt+JyGGpct9E5H883+dSEXlBRHKTdd9E5CkR2SYiS21tUd8nEbnIs/8qEbnI6Vpxku0+z3e6WEReE5HOtm03emRbKSLH2drj+ht2ksu27dciYkSkyPM+6ffM0/4rz31bJiL32trjd8+MMWn/h7V4bg3QH8gGFgFDWvH6PYAxntcdgO+x0onfC8zwtM8A7vG8Ph54FxBgPPBVK8h4HfAv4C3P+5eAcz2vHwV+4Xn9S+BRz+tzgX8nWK5ngcs8r7OBzqlw37CSOa4D2tnu18XJum/AkcAYYKmtLar7BBQCaz3/F3heFyRItqlApuf1PTbZhnh+nzlAP8/vNiMRv2EnuTztvbEW3f4AFKXQPZsCfAjkeN53TcQ9S9iPOZX+gMOAmbb3NwI3JlGeN4BjgZVAD09bD2Cl5/VjwHm2/X37JUieEuAj4CjgLc+DX2H70frun+fHcpjndaZnP0mQXJ2wOl4JaE/6faMpw2+h5z68BRyXzPsGlAZ0IlHdJ+A84DFbu99+8ZQtYNtpwPOe136/Te99S9Rv2EkurDT+I4H1NCmFpN8zrAHHMQ77xfWe7S/mo0jSeLcKHrPBaOAroJsxZrNn0xagm+d1a8v7IPC/gNvzvguwy1jpzAOv35rpzvsB5cDTHtPWkyKSTwrcN2PMRuB+4EdgM9Z9mE9q3Dcv0d6nZP1OfoY1Ck+6bCJyCrDRGLMoYFMq3LMDgYke8+MnInJIImTbX5RCSiAi7YFXgGuNMXvs24ylyls9FExETgS2GWPmt/a1IyATawr9iDFmNLAPywziI4n3rQCrSFQ/oCeQD0xrbTkiJVn3KRwichPQADyfArLkAb8Fbk22LEHIxJqZjgeuB14SiX9Rsv1FKUSSxjuhiEgWlkJ43hjzqqd5q4j08GzvAWzztLemvBOAk0VkPVZ1vKOAPwOdxUpnHnj9iNKdx4kyoMwY85Xn/ctYSiIV7tsxwDpjTLkxph54FetepsJ98xLtfWrV34mIXAycCJzvUVrJlu0ALCW/yPN7KAEWiEj3JMvlpQx41Vh8jTWzL4q3bPuLUogkjXfC8GjzvwPfGWMesG2ypw6/CMvX4G2/0BPxMB7YbTMDxBVjzI3GmBJjTCnWffnYGHM+MAsrnbmTbK2S7twYswXYICIHeZqOBpaTAvcNy2w0XkTyPN+vV7ak3zcb0d6nmcBUESnwzISmetrijohMwzJZnmyMqQqQ+VyxorX6AQOBr2mF37AxZokxpqsxptTzeyjDChDZQgrcM+B1LGczInIglvO4gnjfs3g4RNrCH1b0wPdY3vibWvnaR2BN3RcDCz1/x2PZlD8CVmFFFRR69hfgYY+sS4CxrSTnZJqij/p7HqzVwH9oinjI9bxf7dneP8EyjQLmee7d61gRHilx34DfASuApcBzWNEfSblvwAtYvo16rM7s0ljuE5Z9f7Xn75IEyrYay97t/T08atv/Jo9sK4Hptva4/oad5ArYvp4mR3Mq3LNs4J+e520BcFQi7pmuaFYURVF87C/mI0VRFCUCVCkoiqIoPlQpKIqiKD5UKSiKoig+VCkoiqIoPlQpKEoQROQmTzbKxSKyUEQOFZFrPStfFSUt0ZBURXFARA4DHgAmG2NqPSmUs4EvsGLUK5IqoKIkCJ0pKIozPYAKY0wtgEcJnImV52iWiMwCEJGpIjJXRBaIyH88+a0QkfUicq+ILBGRr0VkgKf9LLHqLywSkTnJ+WiKEhydKSiKA57O/TMgD2s18L+NMZ94cuKMNcZUeGYPr2KtIN0nIjdgrWC+w7PfE8aYO0XkQuBsY8yJIrIEmGaM2SginY0xu5LyARUlCDpTUBQHjDGVwMHAFVjpu//tSeBmZzxWgZPPRWQhVn6hvrbtL9j+P8zz+nPgGRG5HKsIiqKkFJnhd1GU/RNjTCMwG5jtGeFfFLCLAB8YY84LdorA18aYn4vIocAJwHwROdgYk+hsqYoSMTpTUBQHROQgERloaxqFVZ5xL1ZJVYAvgQk2f0G+J3ull3Ns/8/17HOAMeYrY8ytWDMQe2pjRUk6OlNQFGfaA38Rq6B8A1YGzCuwyi++JyKbjDFTPCalF0Qkx3PczVhZKQEKRGQxUOs5DuA+j7IRrAymgRW+FCWpqKNZURKA3SGdbFkUJRrUfKQoiqL40JmCoiiK4kNnCoqiKIoPVQqKoiiKD1UKiqIoig9VCoqiKIoPVQqKoiiKD1UKiqIoio//D1f3zA5TpOKOAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "train_loss = np.array(model.get_train_summary('Loss'))\n",
+    "val_loss = np.array(model.get_validation_summary('Loss'))\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "plt.plot(train_loss[:,0],train_loss[:,1],label='train loss')\n",
+    "plt.plot(val_loss[:,0],val_loss[:,1],label='validation loss',color='green')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As a reminder, in chapter 3, our very first naive approach to this very dataset got us to 88% test accuracy. Unfortunately, our small \n",
+    "recurrent network doesn't perform very well at all compared to this baseline (only up to 85% validation accuracy). Part of the problem is \n",
+    "that our inputs only consider the first 500 words rather the full sequences -- \n",
+    "hence our RNN has access to less information than our earlier baseline model. The remainder of the problem is simply that `SimpleRNN` isn't very good at processing long sequences, like text. Other types of recurrent layers perform much better. Let's take a look at some \n",
+    "more advanced layers."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## A concrete LSTM example in Keras API of Analytics Zoo\n",
+    "\n",
+    "Now let's switch to more practical concerns: we will set up a model using a LSTM layer and train it on the IMDB data. Here's the network, \n",
+    "similar to the one with `SimpleRNN` that we just presented. We only specify the output dimensionality of the LSTM layer, and leave every \n",
+    "other argument (there are lots) to the Keras API of Analytics Zoo defaults, which has good defaults, and things will almost always \"just work\" without you \n",
+    "having to spend time tuning parameters by hand."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "creating: createZooKerasSequential\n",
+      "creating: createZooKerasEmbedding\n",
+      "creating: createZooKerasLSTM\n",
+      "creating: createZooKerasDense\n",
+      "creating: createRMSprop\n",
+      "creating: createZooKerasBinaryCrossEntropy\n",
+      "creating: createZooKerasBinaryAccuracy\n"
+     ]
+    }
+   ],
+   "source": [
+    "from zoo.pipeline.api.keras.layers import LSTM\n",
+    "\n",
+    "model = Sequential()\n",
+    "model.add(Embedding(max_features, 32, input_shape=(500,)))\n",
+    "model.add(LSTM(32))\n",
+    "model.add(Dense(1, activation='sigmoid'))\n",
+    "\n",
+    "model.compile(optimizer='rmsprop',\n",
+    "              loss='binary_crossentropy',\n",
+    "              metrics=['acc'])\n",
+    "\n",
+    "dir_name = '6-2 ' + str(time.ctime())\n",
+    "model.set_tensorboard('./', dir_name)\n",
+    "model.fit(input_train, y_train,\n",
+    "          nb_epoch=10,\n",
+    "          batch_size=128,\n",
+    "          validation_split=0.2)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "_INFO - Trained 128 records in 0.335889472 seconds. Throughput is 381.07776 records/second. Loss is 0.14791179.\n",
+    "\n",
+    "Top1Accuracy is Accuracy(correct: 4358, count: 5000, accuracy: 0.8716)_"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXd8FMX7xz9zl0uO9EoJLQkgJRAIHZEmRcAGooCCYlfsX38W9Gv7qvgFK18UC/auiA0BBQtIkd57DxBKgJBC+pX5/XG3d3t7u7d7NZfkeb9eeeVudmZ2dm93npnneeYZxjkHQRAEQQCArrYbQBAEQYQPJBQIgiAIByQUCIIgCAckFAiCIAgHJBQIgiAIByQUCIIgCAckFAiCIAgHJBQIgiAIByQUCIIgCAcRtd0Ab0lNTeUZGRm13QyCIIg6xaZNm85xztPU8tU5oZCRkYGNGzfWdjMIgiDqFIyxo1rykfqIIAiCcEBCgSAIgnBAQoEgCIJwUOdsCgRBhB6TyYT8/HxUVVXVdlMIFYxGI1q0aAGDweBTeRIKBEGokp+fj7i4OGRkZIAxVtvNIRTgnKOwsBD5+fnIzMz0qQ5SHxEEoUpVVRVSUlJIIIQ5jDGkpKT4NaMjoUAQhCZIINQN/P2dSCjUQ/aeLsXGvPO13QyCIOogJBTqISNnrcS1766p7WYQRMAoLi7G22+/7VPZ0aNHo7i4WHP+5557Dq+++qpP56oPkFAgCCLs8SQUzGazx7KLFy9GYmJiMJpVLyGhQBBE2DNt2jQcOnQI3bp1w6OPPorly5djwIABuOqqq9CpUycAwJgxY9CjRw9kZ2dj7ty5jrIZGRk4d+4c8vLy0LFjR9xxxx3Izs7GiBEjUFlZ6fG8W7duRd++fZGTk4OxY8eiqKgIADB79mx06tQJOTk5mDhxIgDg77//Rrdu3dCtWzfk5ubiwoULQbobwYVcUgmC8Ir//LILu0+WBrTOTunxePbKbMXjM2bMwM6dO7F161YAwPLly7F582bs3LnT4Xr50UcfITk5GZWVlejVqxfGjRuHlJQUl3oOHDiAr7/+Gu+//z7Gjx+P77//HpMnT1Y870033YQ333wTgwYNwjPPPIP//Oc/mDVrFmbMmIEjR44gKirKoZp69dVXMWfOHPTv3x9lZWUwGo3+3pZagWYKBEHUSXr37u3iiz979mx07doVffv2xfHjx3HgwAG3MpmZmejWrRsAoEePHsjLy1Osv6SkBMXFxRg0aBAAYMqUKVixYgUAICcnB5MmTcIXX3yBiAjb2Lp///54+OGHMXv2bBQXFzvS6xp1s9UEQdQankb0oSQmJsbxefny5fjjjz+wZs0aREdHY/DgwbK++lFRUY7Per1eVX2kxKJFi7BixQr88ssvmD59Onbs2IFp06bh8ssvx+LFi9G/f38sWbIEHTp08Kn+2oRmCgRBhD1xcXEedfQlJSVISkpCdHQ09u7di7Vr1/p9zoSEBCQlJWHlypUAgM8//xyDBg2C1WrF8ePHMWTIEMycORMlJSUoKyvDoUOH0KVLFzz++OPo1asX9u7d63cbagOaKRAEEfakpKSgf//+6Ny5M0aNGoXLL7/c5fjIkSPx7rvvomPHjmjfvj369u0bkPN++umnuPvuu1FRUYGsrCx8/PHHsFgsmDx5MkpKSsA5xwMPPIDExEQ8/fTTWLZsGXQ6HbKzszFq1KiAtCHUMM55bbfBK3r27Mlpkx3PZExbBADIm3G5Sk6C0MaePXvQsWPH2m4GoRG534sxtolz3lOtLKmPCIIgCAckFAiCIAgHJBQIgiAIByQUCIIgCAckFAiCIAgHJBQIgiAIByQUCIKol8TGxgIATp48iWuvvVY2z+DBg6Hm4j5r1ixUVFQ4vnsbiluJcA3RTUKBIIh6TXp6OubPn+9zealQqO+huEkoEAQR9kybNg1z5sxxfBdG2WVlZRg6dCi6d++OLl264Oeff3Yrm5eXh86dOwMAKisrMXHiRHTs2BFjx451iX00depU9OzZE9nZ2Xj22WcB2ILsnTx5EkOGDMGQIUMAOENxA8Drr7+Ozp07o3Pnzpg1a5bjfHU5RDeFuSAIwise+u0hbD29NaB1dmvaDbNGzlI8PmHCBDz00EO49957AQDz5s3DkiVLYDQa8eOPPyI+Ph7nzp1D3759cdVVVynuU/zOO+8gOjoae/bswfbt29G9e3fHsenTpyM5ORkWiwVDhw7F9u3b8cADD+D111/HsmXLkJqa6lLXpk2b8PHHH2PdunXgnKNPnz4YNGgQkpKS6nSIbpopEAQR9uTm5uLMmTM4efIktm3bhqSkJLRs2RKcczz55JPIycnBsGHDcOLECRQUFCjWs2LFCkfnnJOTg5ycHMexefPmoXv37sjNzcWuXbuwe/duj21atWoVxo4di5iYGMTGxuKaa65xBM+ryyG6aabgBzvyS1BpsqB3ZnJtN4UgQoanEX0wue666zB//nycPn0aEyZMAAB8+eWXOHv2LDZt2gSDwYCMjAzZkNlqHDlyBK+++io2bNiApKQk3HzzzT7VI1CXQ3TTTMEPrnxrFca/t6a2m0EQDYIJEybgm2++wfz583HdddcBsI2yGzduDIPBgGXLluHo0aMe6xg4cCC++uorAMDOnTuxfft2AEBpaSliYmKQkJCAgoIC/Prrr44ySmG7BwwYgJ9++gkVFRUoLy/Hjz/+iAEDBnh9XeEWoptmCiqcLqnC5mNFGN2lWW03hSAaNNnZ2bhw4QKaN2+OZs1s7+OkSZNw5ZVXokuXLujZs6fqiHnq1Km45ZZb0LFjR3Ts2BE9evQAAHTt2hW5ubno0KEDWrZsif79+zvK3HnnnRg5ciTS09OxbNkyR3r37t1x8803o3fv3gCA22+/Hbm5uR5VRUqEU4huCp2twqBXluFoYQUOTB8Fg951YhWuIarDtV1E3YVCZ9ctKHR2EDlR5JsukCAIoi5CQkEjdWxCRRAE4RMkFDTCQVKBaNjUNVVzQ8Xf34mEgkbofSAaMkajEYWFhSQYwhzOOQoLC/1a0BZU7yPG2EgA/wOgB/AB53yG5HgrAJ8CSLTnmcY5XxzMNhEE4T0tWrRAfn4+zp49W9tNIVQwGo1o0aKFz+WDJhQYY3oAcwAMB5APYANjbAHnXLxM8CkA8zjn7zDGOgFYDCAjWG3yBxogEQ0Zg8GAzMzM2m4GEQKCqT7qDeAg5/ww57wGwDcArpbk4QDi7Z8TAJwMYnv8gmwKBEE0BIIpFJoDOC76nm9PE/McgMmMsXzYZgn3y1XEGLuTMbaRMbaxtqavNFMgCKIhUNuG5usBfMI5bwFgNIDPGWNubeKcz+Wc9+Sc90xLSwt5IwHQPIEgiAZBMIXCCQAtRd9b2NPE3AZgHgBwztcAMAJIRRhCXheEEsv2ncHvu5UjcxJEXSKYQmEDgHaMsUzGWCSAiQAWSPIcAzAUABhjHWETCmHl3sAl/wlCyi0fb8Adn4Uu9ApBBJOgCQXOuRnAfQCWANgDm5fRLsbY84yxq+zZ/g/AHYyxbQC+BnAzD9MheXi2imjoVJstNIslAkpQ1ynY1xwslqQ9I/q8G0B/abmwhN47IsworqhBt+d/x6OXtce9Q9rWdnOIekJtG5rrDOSSSoQbZy5UAwB+2iI11RGE75BQUEHY6dVKMoEIU+jRJAIJCQWNkN6WIIiGAAkFjZBIIAiiIUBCQSM0USAIoiFAQkEjZGgmCKIhQEJBBe72gSDCA6aehSC8hoSCRkgmEATRECChoBGyKRAE0RAgoaARsikQ4Qq5SxOBhISCRui9IwiiIUBCQSMkEwiCaAiQUNAITdGJcIOR+xERBEgoaIRkAkEQDQESCioIgzESCgRBNARIKKjg3HmNpAJBEPWfoG6yE07UmK0Y984/OF9eg/svbYsJvVqCeaGUpZkCQRANgQYjFGb+thc7TpQAAKb9sAPTftgBAJjUpxVuvSQTkXodmiYY8dSPO3H34DbITI1xKU8ygQhX6NkkAkmDEQp3D2qDD1cdcUv/ct0xfLnuGABg0EVp+Hv/WXy78Tj2vTgSURF6Rz7yPiIIoiHQYIRCWlwU8mZc7pJ2prQKs/86gC/W2oTC3/vPOo7d88VmfHhzL8d3EgkEQTQEGrShuXG8ES+O6YK3bsh1O/bn3jMu32miQIQftFCBCDwNWigIXJGTjm3PjMDNF2e4pE/6YK3oG0kFgiDqPyQU7CREG/DcVdl44epsR9rqg4WwWG3CgGYKvrFk12kcP19R280gCEIjJBQkTOjVSjadZIJv3PX5Jlw2a0VtN4MgCI2QUJAQGaHDpqeGuaXTTMF3Kmostd2E+g09m0QAIaEgQ0pslJtgsJJUIMIMCohHBAMSCgokx0S6fH/m5504X15TS60hCIIIDSQUFGCMITs93vF9Q14Rur/wO46cK6/FVhEEQQQXEgoeuO2STLe0Yz540pgs1gbpgUOrwEMD3WUikJBQ8ECE3v326H1Q5E5ftAcDXl6GMxeqAtGsOgPJBIKoe5BQ8MBFTWLd0nQ+3LGVB2zhM0oqTP42qU5BxnmC8B3OOX7eegImizWk5yWh4IEOTeMRoXOdGdzw/jrUmL37kbwJ0V2fsJJMIAifWbLrNB78Zive/PNASM9LQkGFj0RB8QTe/Cu0P1JdhWYKBOE7RXbNwpkL1SE9LwkFFaQzBQA46+OPRF0kEUga5vyTCDZBFQqMsZGMsX2MsYOMsWkKecYzxnYzxnYxxr4KZnt8QU71Y/ZSL9JQX16aKRCE/4T6NQrafgqMMT2AOQCGA8gHsIExtoBzvluUpx2AJwD055wXMcYaB6s9viIzUfDapiDQ0PpIsikQRN0jmDOF3gAOcs4Pc85rAHwD4GpJnjsAzOGcFwEA5/wMwgydXSr0aJ3kSFuw7SSKvFjdLEw2eANTINFMITTQehAikARTKDQHcFz0Pd+eJuYiABcxxlYzxtYyxkYGsT0+obP36BbJsPflJfs018EaqAKJh9aTjiCIAFDbhuYIAO0ADAZwPYD3GWOJ0kyMsTsZYxsZYxvPnj0rPRxU9PaZgnQ09vX6YyFtR10kXGYK+05fQMa0RS7brRIEIU8whcIJAC1F31vY08TkA1jAOTdxzo8A2A+bkHCBcz6Xc96Tc94zLS0taA2Wo3N6PMbmNsdr47v6XVc49JG/7y7AharQLKILF6GwIe88AOC3nadruSWBpaGufyGCSzCFwgYA7RhjmYyxSAATASyQ5PkJtlkCGGOpsKmTDgexTV4TodfhjQnd0LZxnGIeq4pFNVze3ePnK3DHZxvxr2+3huR8gTQ0nyqpVL3PSggqQNK9E4Q6QRMKnHMzgPsALAGwB8A8zvkuxtjzjLGr7NmWAChkjO0GsAzAo5zzwmC1yV9+fXCAbPon/+RpKl/bfVKlybbZzdHC0ATnEzphOQ8ub8gvqkC///6FWX/s96m8cP7avv8E4Q21NZYMqk2Bc76Yc34R57wN53y6Pe0ZzvkC+2fOOX+Yc96Jc96Fc/5NMNvjLx2bxcumP7/Q5mVbbbagrNoctPNXmSxuBm9fCFXfKDTVXzVHQaltseDKg+d8Ki/MFMJFnRUoaOZDBIPaNjTXK8bO+Qedn10StPo7PP0bHvxmS9DqDzRCJ1zb2jNBJtG6CYJQh4SCl7w9qbvisd2nSj2WDcQ6hYXbT/ldR6g6aatDfRSYM/o6MGb13KZQP6+KqC1IKHhJ91ZJ6pkAbDpahFMllagxWxusl4ijD67ly9c5Zgr1q/usX1dDhAtBC3NRX4mKkJej8zYcd/k+7p1/HJ/jjOF1m0NnUwiMoVnAV9nqtCkEph0EUZ+hmYKXxETJd/CPfb9dscyFKpvx2dNA1WLlWO2jIVUroR6wOwzNATqz7+oje/mAtIIg6jckFLwkUmGm4C8frDyMSR+sw197C4JSPxD6TjFc1DX11/uotltA1EdIKIQJeYXlAIBTJfL7ONdlI2ltm1QCuXjtga+3IGPaIr/rIYhwhYRCELjyzVWy6UKfVFFjxo0frsPhs2WOY04PGfk6AyETQt03B0qO+StUHC6pAQjQt2DbSf8rCRh1d6BAaCfU0ZVJKASBHSdKPB5fdeAcVh44h5cW73WkCf2e0mi2bqo+wqPN9dX7iCCCQXi5xdRzODgypi2S9cZRM4aS54zvsHrufUSyjggkmmYKjLE2jLEo++fBjLEH5EJcE54ROiVn+AfnMZ2a+ihMRt3eEC6dlXPxXJg0KECEy/0lgkNt2eK0qo++B2BhjLUFMBe2kNhht59yqPjxnosRHan3upxUNST+zdU8ZOpiBxAuTRbuc32dKRBEINEqFKz2qKdjAbzJOX8UQLPgNSu8yW2VhDG50k3k1PHUJ6nF5yF9uO/o7E853UMi2FTWWPDPoeCuNwo2WoWCiTF2PYApABba0wzBaVLdYFTnpl6XcZspiKYKwgIvJUNzIPuzULm3Bvo0vlZXX20K9exy6gVP/LAdN7y/DkftLuZ1Ea1C4RYA/QBM55wfYYxlAvg8eM0Kfwa0834HOGknKV7pq6Y/DMQoN9Q6ynCxg9AmO0So2F9gczMXohjURTR5H3HOdwN4AAAYY0kA4jjnM4PZsPqIdC8EV0Oz7b9S5x+IUW5D7RPr6yY79e16iPBAq/fRcsZYPGMsGcBmAO8zxl4PbtPqHyaL8lusquKogx1AuHRawoyMbApEXSTUj61W9VEC57wUwDUAPuOc9wEwLHjNqp+YLK5Lal1tCjaUHgDq0Hxfkc3q6UxBIFzUdET9QKtQiGCMNQMwHk5Dc4Pn0g6N8djI9prz10iFgotNwa73VnjB66RNIVwMzQFtRfhAwoAIBlqFwvMAlgA4xDnfwBjLAnAgeM2qG3x0cy/cMSBLc37pTEGM2mhWnFxlsmD3Sc+7vIUD1GkRRN1Dk1DgnH/HOc/hnE+1fz/MOR8X3KbVDSK82EHG7MGm4DSGOvNcqDJh3eFCAK4zhUfnb8fo2StxvrzGy9YS9RF/ZmQFpVV4668D5JlFONBqaG7BGPuRMXbG/vc9Y6xFsBtXF/Bmq02p9xFk1imIs9zz5WZMmLsWJRUmlxd/89EiALZoq+FMwKKkBqaasJq5lFWb8ek/eX51xoG4v/d/vQWvLt2vur840XDQqj76GMACAOn2v1/saYQXuLmkij7LuU3usb+o1RZLvTWShoJwvHX/WbALzy7YhZUHanf1qzCwsFptGz2dKqms1fYQtY9WoZDGOf+Yc262/30CwPvVW/WUnq2TNOWrMluUD8rGPnKKDTlDs6+CwptiF/37V9z52UbfTlSP8VfdUlRhAgBUmpSfiZ+3nsCLC3d7aINfTXDheFEFXly0B3d4+VvvPlmK13/fH7iGEA4CtY2tt2gVCoWMscmMMb39bzKAwmA2rC4xf+rF2PrMcNV85y5Uu3wXVE9/7C7AF2uP2tNkCvLac0mtsVixdLdvW4SGy+wmGO3wt04tWscHv9mKD1Yd8e9EGhFmsaWV3qkkx8xZjdl/HoDZgxMFUbfQKhRuhc0d9TSAUwCuBXBzkNpUJzHo1W/l7L8OunwX+oXbP9voMBoHc+e1UHfS4aTDDzT1Zd2IMBr1dR9rs307O29sa0R4o9X76Cjn/CrOeRrnvDHnfAwA8j4SYTR4H0rbm/coIEJBOK//VdUpgiGcAlWjP79rIK/L1wV+9UM0EmL82Y7z4YC1oh6g1zG8OKaz1+V+3npC8ZhYaARyZBqqFzngg2k/KwxspFn/yoebYFbbDlYNcmmtP/gjFMLtua51vNWrMtj0xmpwBKYjb6jvbTCuO5zUR/40RRh4qG0Hq9oG35tAhBn+CAV6DiSYVUKZtkmLcfmupocVjp4rq0Z5tdlexl3ttGzfGSzfd0a1faHW8dMDooXaXacg1MF8tCmEkWz0ivyiChw5V3f3PAgmHkNnM8YuQP6pZQAaBaVFdRhPUVABmcVrMsjluHz2Kug9rJy+5eMNAIC8GZer1u+JOz/biIEXpWFy39Z+1SMQLiqFYLQiUN5HYXKLRIZm38qHy3Vo5ZKZywCovzObjhbh2PlyjM3Vtla3PtjbPc4UOOdxnPN4mb84zrmmvRgaEs0SjB6PS2cS3jw/gkDR+/HUCS9uQUkVMqYtwqLtp1yOL91dgKd+2ulz/UEnjN44T7Oun7eewPoj5z2Wry0fdCXUovSqUV89zca98w/+9e222m5GSPFHfURIuLpbulsahxnFEZ/DghJYPYS5cBZw5pHrA81Wjvwi31adClWX19gWTH29/phP9Wg+X8ArDB8jqKcR9YPfbMX499YE/JzBQGpLCJfZHeEk1L8ICYUAwhhDZITrLa3W7UVJxHycNN6NU+ZF4HAao92EBGxrGa54c6Wm80nf36W7TsvW6cgvebyqPa2wDgCh7F+2Hi/GzhMlHvMEsjn1rfMUrsdnQ3M9uh1frD2KYa//7VPZ+nAfgioUGGMjGWP7GGMHGWPTPOQbxxjjjLGewWxPKJCqd4zWzmhWPRsGa2vkWd9AQeRjqGGHAQAmhQ585wlbzCNvVQx3fr4JLy3eozl/jbn+rEIdM2c1rnhzleyxoNgUglCntwTUxdZRZzhcWe3y1E87cfBMWW03o9YImlBgjOkBzAEwCkAnANczxjrJ5IsD8CCAdcFqSyiRswdH8tZoUvNftMQjMOlO4VTUQzhveB811sB7P3wlUQmdL6/BBysPg3Pu1olUB10oBKaDCcfVsgEzNPvfFL+Q2hI8qcU+WHkYG/I820rqO9XmauQV58FslQ8HEoaPqtcE01jcG8BBzvlhAGCMfQPgagDSCF8vAJgJ4NEgtiVk6BSeCgYGVjkY6eiBYsOnuKBfgK/y/kG07jZEW/v7ZHgsLK92U1dJR/+PfrcNf+49g54ZyW57P0h3gvNEaZUJ8UYDisprUFheg7aNY1XLhM2gMyixj/yrNNy8j4Sb5Mkl9cVFtlmonMdO+FxHcCitLsV7G9/DG2vfwKmyU4jUR6Jtclu0T2lv+0u1/a+x1v0Q5MEUCs0BHBd9zwfQR5yBMdYdQEvO+SLGWL0QCkkxkbhQrRxUTI84pJjuQ6x5OExR7+Fc1AwYLd2RbLobBu5uqPbE2Lf/cUuTejiVVtmicdaYrYiIdA3F4Y366M7PNuLh4e0dBlR/3V/rOuHQCQbF4ycMriucsKAIpREL0OqNSSipLsGwrGF4euDTOFpyFHvP7cWec3uwcP9CmKwmRxmdMR43/dIJ3dOzXQRGVlIWDHpDLV6NNmrNrZQxpgPwOjQE1mOM3QngTgBo1apVcBvmJ1/e3gcDXl6mmi+Kt8fo9E/x4/6PUWz4HCej7kWCeTwSzOPAEAkgMFNR8aIkaUfmTce2Ia/Ia4+agEe58Llc+MY+Chec6iMfPbzq2R0xsVOYunAq8o0fAjDjujbX4vH+j6NHeg+3vGarGUeKjmBf4T48+tOvyL9wCIyV4pf9v+DD8g8d+SJ0EchKynKbXXRI7YDU6NSwUZMGUyicANBS9L2FPU0gDkBnAMvtN6MpgAWMsas45y5B3TnncwHMBYCePXuG9dPXMjkaKx8bgleW7MOCbSc95t167ALiLVch2tIfRYYPUGL4EuX6ZUg2TcXukwMCMhoVHjMr524vrrgDWLbX84poLQvvwh4vLqHKZEG1yYqEaPmRXX0zyHLJf6/L15PbseXUFpw1zESFfjU+2hqBWMtQxJvHYt51dyqWidBFoF1KO7RLaYd3jAmoPF+KT6+4BJ2bJ6C4qhj7zu3DvsJ9zv+F+7Dk0BLUWJzb6SYZkxxCQhAYx8viwWEK+b0NplDYAKAdYywTNmEwEcANwkHOeQmAVOE7Y2w5gEekAqEu0jI5GrOvz1UVCoX2cNkRSEGa6XFUWobjvOEdnIl6Gr3e+R1JptsQgRS/2iLYODh3f3HFQuGWTza4HNt8rAiv/LbPr3MH+mH2dRzlSzvGv7cG2/NLFNVkgZKR/oywwzmm06oD5/DJP0fw/k09w2YErATnHMvzlmPG6hlYemgpmL4R4s1jsefh2bj4pS1+1Z1oTESfFn3Qp4WL5hwWqwVHS466CYzfD/+OT7d96sxo1OHDg+nY82UXdEjtgAnZE9zqCjRBEwqcczNj7D4ASwDoAXzEOd/FGHsewEbO+YJgnbuu0sjaHenVc1ASMR8lEd+hUr8BiaYbEWe5HAzeh+YGAJ3dDl1lsripozx1bI/P344DGtzy7vxsIwa1T8OkPu6hMQI9mg7lgGl7vtqaBz8NzXAK63BAi/eRx/KS77d+ugE1ZiuqzVavwsqfKqnEjvwSjMhu6pJ+rLACVWYLLmoS51sDZeCw4oc9P2Dm6plYf2I9msQ0wX+H/hdvL8yEDrFoEtsMgKtQqLQv/GwU6fma1H5XvU6PrKQsZCVlYVS7US7HSqtLsb9wPz7d8A8+37AGTeLP4+SFk1ietxw5TXLqrlAAAM75YgCLJWnPKOQdHMy21BUYIpFovgExlsE4b3gXRZFzUW79E8k19yCKt/e6PmGmcNun7hOwQHTaS3cXYOnuAlmh4A/zN+Xjke+2YduzIxTzLNt3Bs0SjOjQNN5jXUHpeP2tMwCD54AuxrPXVmO2ImPaIix/ZDAyUmNUSonKS26yr+rGsXP+wenSKrcZ2sBXtMUqUqKs2oyV+89iVJdmqDZXo0y/FCURP2DcvHy0SWqDdy9/F1O6TYExwoh3Fy4CIP9+dHluCcxWHlRHi/ioePRM74nDzZvglzVtcEWLFnhtfFdYuRUWa3AXnAK0ojlsMfB0NK75D1Krp8HCinA66hEUGt6CBd4tqvE0dff03mqZJajhbbew93Qp+r70JwrLqvHBStsCv5PFyiE9bvl4A0bO0rb6O9DUBxMLAIc3g7T/25Zf7Fe1vgqF06VVmvNuyDuPe7/c7HEVv8Bj87fhri9X4YmlLyFrdhYKI2dDhyh8e+232HffPtzV8y4YI1xjl8nVqhYJOZjomC4k3ksU1C6MYWCIsV6CRlXdUWz4Ehf0v6BCvwZJplsRY7lU09oGTzmU9MeBUvt4W817fx/G6dIqLN93VhTS2f/2iEsfK6xAnDECSTGRftYZPlIhEL+XtAbfUiCQAAAgAElEQVSvQ2grpQfxNt3y8QaUVZvx0tguig4BAHCm/AyWHv8fThh/wIw15bg081KYz90NozUX47OvUCwXTntmhBKaKdQBdIhGsukONKuehQjeFIWRb6Ag8gnUMM8B7X7fXYC/959VPK40wgrVu2C2WHHkXDnGvr0aT/64w7m6Fr53usVVxag2V8se4+AY+MoyDH51uU91u9QVMEOzP20I3A8lrctqten3tZ5DKZvPLq4aypXZ1wPpFHqxI0VHcO+ie9F6Vmscqv4CRmtXfH31X/jzpj/RyNpddVAViM2L6iIkFOoQkTwLTatfQXLNfTDp8nAq6n4URXwCK+Sn3Hd85tmRi3NbpNSlu067pFf5EChvy7EiNyGj1rG/snQfhry6HFuOFeOrdcccqi5xSA4GJluLxWqBieWjXL8S//7z3zgT+R/kR92MpJlJSH89HQ8veRh7zu5x1CempNK20OjJH3cgY9oir6/Vdm3+ES59hlI7Nh8rQr///oX5m/L9qj8ULq7Ssc32gu2Y9MMktHuzHd7f/D4mdZmEi2M+RVrNk+jW1H2dQSDaEMiytQ2pj0JEh6Zx2Hv6gt/1MOgQZxmJaEs/FBk+RqlhPsr1K5BsugvRVu+8Eqyc44kfdrilP7dgl2pZzrmLvWLs2/9g6uA2eHxkB1Emz3WsOVTo8l0c+oGL0i5Ul6JKtxNHKk/jjgWfYvuZ7dhRsAOVRpu9YeZqPXSsOaKs2Xh6+EhsOb0Fb61/C2+sfQMDWw9E3ybjweG+Wvyrdb6HDteix9ZCba93UDq78KxuOlqE63q2VMilXpHPez57k5dzcM6x8thKzFg1A78e/BWxkbH4V99/4aG+D6F5fHNc+tpyAOVe7UcSTirCUEJCIUS8Nr4rPlqVh+83+zfyEtAjAammhxBrGYbzhrdxNuoFNLL0QbLpLkTwxgCAuKgIjyE3LAov7O5T6vFbOHefIv++uwAmsxVNE4y4fUCWpjpcE6wwsZNYe+o4DlWtwZnI/Rj65SnkXzgKRAEFFcDJvcno2qQr7upxFz5fCURaM3HkhbvQ/qk/AADTLrF5hZwpP4NPtn6CuZvm4uWj90FnjEPUhdEwscEwcA2dXJAJhO9+MLsswaDqbzt9nylwaJlPcVix6MACvLXxNazNX4u06DRMv3Q6pvaciqRGSY58Vh+uRyz3pYOgUFBbs0kSCkFk5rguePx720g8Oz1BNoKqvwihuUsjfkZJxFc4GTUVCebrEW++Gox5/nmVBrta4uRZOIdO8tharRwfrDoCALh9QJbHDqGspgxFpp24oN+BGt0RmNgRzN53DCZjBd7YAgA6RLB0dG3SG1e3uxHfrdEjp0kOlj4w1vFy/rjcpvqJiohyq79xTGM81v8xPHLxI5i57Hu8uGw2DlbMBzd+iyhLZ3y5vRgc0Y6QIt5Sl9UDcrgtbLQ/HFqfWaVRNfcxEK/a7a2x1KBM/wdKI77H5J+OIzMxE3NGz8Et3W5BI4P7TsHOZ137Dyee5cgNguSoy7YEARIKQWRCr1YOoQAoR1D1F4YIJJjHIcYyAOcNc1Fs+ATl+r+gx32wRS1XwI8pv6wBUebyODhqUICf9/6MbQXbbH+nt+FQ0SFbhkiA8RhEWjPRJfFq5J1Oxb8GDcfv2ww4WmjBrOEDUVJpwsLVa3DgFHDkXDmy0tQjtAromA65jQcgzRSNLk05Vp36HmX6JZj842TojHGIsVyKvefaoENqB/XKJNdVUWPG8fO+7YIXCAIdBkWMxSEUtD2zSm3xpILhnKPcVI7iqmLHX4VuPaysDG+tP4SymhJH+tnIvbCiDD3mPoOiyiKcrTiLssgyGKwZeG/0p7i1xw2I0Cl3Z8L1SAdCZosVEXp506o4q1VmECR/TapZwh4SCiEk2KOICN4YjWueQoVuPc4b3sUR9hgMUVlgiADAwLgOtm7A9p+BAXpnms0bg6G8PBLlkRYwez44yjFH2h2/zEekXo9CQ76jvLk6EqUGCwCGqQsXYs2xrThu3AHOyjHmWwBgaJfcFt2adsOUrlPw84YInC5sCj1PAwPD8PSW+ObEcWQldIGOHQZQDqskPMfQ1//Gkf96u3DIVoFRn4wE87WIN1+Dj+6MxlUfP4cL+oXoOOdnDGw9EHf1uAvXdLzGzV9dDisH7vtqC/5SiRkFAEXlNYiJinAJcx7uA0rHnuAqUwUrqmBFOfae2wN+vhxFlUUorirGBf0qWFk5nl72J6osF1BcbevchePCn4VLnBrsk76Hl9r+xxhikGhMhInpoeOxSI9rhU5pnZAYlYh5q1JhtPbAmA7DPAoEwCn0pMKv7b9/xawJ3TAmt7lbGfEsp96sS9EACYUQEiqdZLS1N4zVOahp9BNK+V77aM0K2P9zh9OnxfafCcdsx6usDGZmdqQ5j1vt5TiW5x0EhxWV+gpHWkWNFdDb6p+3OwrpMW0QYxmEWF0bNDZehKKSpvhz0mi0TI4GAPyzdSXOcaf9wrnxjPMNlC6CCoRHCIMOQ7OGIs1UBYupCPddcQJzN8/FpB8mIaVRCqZ0nQITawcDb6moS+acY/0RbRvO5L7wO4Z1bIwPpvTyvfEeCER/Jb2vZqutRzRZK7DrzC4cKT6CUv1CmFkBxs37CEeKjiCvOA9FjYoAABd/IqnQrpX7aGs0EhslItGYiChdHBrHNkb71PZIjLKlCX9JjZKQaEzEje/vhA4xWDftKjSJTXYs1hK8xH653jkg+GWFsPJY/foEYSC3qG7xjlPyQkF0ZxuS0ZmEQggJhk1B8Vwwwlg5EepjXncyYqORV1jhMc/2qSMQbzQounTmPXc5Vh44ixs/XI/oSD1KSiy2OQYDKmrMGDNnNfYXuK6aZi7B+5wjO289UZ5bsAsTe7dCx2bxjvqEuqTokYTHL5mMR/s/ir+O/IW5m+Zi9vrZMBvNDtvDtdnjZFe7eiPj/9gjP6PQ0qEpGzm976g2HytC+yZxiImyvfpWVMPE8rH5TBEu6NfAzM7AzApwvuIsyo2n8NK2Ery0zV44EmA8CnvOZiEzKRP9WvTDV2vKoeNxePmavmiZmObo5Ie+sgE6xGDTU6OQGhsFk8WKdv/+FdntUvH5ZKeX3Kaj52HQ65DTIhEAEMVta0xSGqXCoNcWM0nbPVTOq1Tc1dCsqSn1AhIKIaSuGKGUvJLEvLP8kKv7qQxK1RwoKHMTCIB4a0inIJCL7uqJgtJqfLrmKH7deRrr/z3M5ZinEAU6psOwrGEYljUMBWUF6DBjGsr0v+HGnybjoSUPYkrXKbijxx2ia+OyqhWtLphysyIl1Iycnk5psphwvPQ4jhQdwa4zB/H0wmVoklSKtKQS5BXn4eSFk4AReGk9bKN7HoEInoZYpCPa0hf9WnfErX37oHVCa4yfcwg6JGLRhCHItMdFWrTSNigY034YUmKdBn89Dri0TRihrzvsOrsa9478pk3eDAW8WSDnTV7xb6m1XF15xz1BQiGESI12jeOicOaC/Orb2sSqwWNEk1CQSTNbuGOUKkW8b7GSDhiwLT4rV3C1lXsphRrMFm0vdpPYJg7bw7u3ReGjre9j9vrZeH3t64iK7Iw4y0hUmXrLGmG19jme+o57v9ws2373c3FYUYFK63msOLrCodI5Uuz8n1+aD6tYOR6hQ3V5GpqndsSINiOwdr8eZ4sTcP+gfnj/r3LokQQGPZo3aoQT5ZUYkp6FiZ07wmrl0MPWoQ95dblMJ67QRvsRbzfx8W7xmjdCQXu9roZmjWXqwYyChEIImdCrJT5bc9TxvXdmMhZuP4XPb+uNGz9cX4stcyWYG+qYrVbFF0wunPRVb63GWIm+d9jrf+OsijCVFUhapJ1Le3QYkjEUo9qNQEFZAT7Z+gme+n0WzkW+igFffIRIPgQRbDgMvIWjjC9hHTjnqDRXorCiEIWVhZi/6zdYcQFW/QVY2QU8+vtfKKo6j8KKQpyvPI/CSvv/ivOwNDLjuAUY9ImzvvS4dGQmZmJg64HISMhAZlImMhIzwE1puPWjw2jfJBFLpwwCAIx9ezUqC4vRMSkHEdjudq8cKj3Va7D9X3ngLC5uk+qWbvEg5GXr05TL9RyeEOS3N8+21YeZgqNNddgGQUIhyKx/cih0djVDdnoC/vy/QRj62t8AgJnjcnBTvwykxbn62fdonYRNR4tC3lYBLeojLcipUkweRuuCNkYc5gIAftxywiWfJ4Egq3mXqDC8QSjbJLYJHr/kcby9sBOqdNuRm7URSw//BG78HlGWzoizjEJJ5RDoGIMZ52FlpVhxdAUKKwpxQb8cVlaKaX+sdHT8648dw5moc7h9aRWuX1SMaovomiTLLt7d2Agp0SlIbpSMlEYp6Ny4M5KNyagxReOnzaVIiEzEezeMQGZSJloltFL0ntqeXwyGYzDIuGBKOzGn95FwH9Tv3fJ9Z3Dzxxvw6GXOEO+cA5+tycMn/+QB8GbE7Y2axz2tqLwG20+UYNBFaQDEAw73zIqnEtsUNI4nxKvytVBSaUJ0pF72N3E2I7QChoRCkGkc7/qCijutRgY9emcm42hhuUses5bVY0FEbRQu8PC3Wz0eV1IfKeldnftJ+zEN9/BS+lKldITIoEMjaze8Mex+TP54KQ5XLESZfgnORb6CpJdft3l02ddOOUbvdk+c19cYkBKdgpRGKeCIRARvhtzGGejeoiVSGtk7/egU3PP5Puh4PHQ8FjrEoU9GUzw8vD36tbHtwic8H1uOF2PZ+jVIjorEZW2Hq15LjdlWLipCRii4eR+5rlNQnSmA40yp7bnJO1fukv7Mz+phU9zr8yYvx89bT8DKOcbm2mZtd36+ERvyirDjuRGIMxocz5w34wIXQ3OQOuau/1mKoR0a48Obg+OZ5gskFEKMeKQsPKhuumnGcGmHxpp84GuTHyQjeCXEHU6NxQqDPhTWOBl3Qh/e67krDuN/fx7AvhdHIirC6Q3DOWDUJSPBfB3izeNQpduGtNQDuK57W8z+/TR0iMM3tw1HYlQSrn5rG3Q8Dkf/O84h+B76Zgt+2noS93Ttimu6t3A558NWV4+uDXlFuP/rzdj4lK3jb/fUr2iZFI3Xxne1t0XbhdXYhYncqFTaWVrsz6nYI8wjXPajz8Ld24B4D35jG6AIQkHwnquosSDOaBDl9WIGIroSbyeZ3mT/M8zec4qSGmKqRRFIFdctcI6Pwmjk4DOOvtj5ipgtyjYFcTHv1AfuPVKgZgpvLz8IACitdDVsc3BHyGbb7CEXuQn34NGLn0CcZTRiLAMwNGsoujXLRQRvDB0aufzemjtbO+Lzcw4cO+90GZa7n5xzbDsu2SiHC+d2JimuaObCTMF5vWrIqU583pPAT0Oz0WD7capMtvfNcZ1e9O5aruO2TzZg9p8HNNdZFyChEGLk9NrSB67umqjkEV+y2coVdfsOVYXXaxPkzyU9LhYeWkNmO1VarhUXV5jcZnhWzt3Of/y8/HoPb+dKNTIqRU8ePd9sOI6r56zGH7sLnPk91C89JqiP9ArCa8uxIpRWmVzKi0Ofi9sY6UFfroR3rqPuaUb7rK7a7Gowl3/05M+lxdD8594zeP33/TJtqrtvMQmFENPVvkhHjPT5eWSE93sxhyPifX8FaixWxRfMJXS2F++URcPLa2uPQrqHMsJIWbrGYdIH69zCMFus3K2uQGzoo4pM8/OLbMJojyjirXBvZGdRkkSTXQgJcYGkZca+/Q9u+2SDS5p4kySXdB+0hd4NCuRmCjahIMwUBHwVNt728XLZK2pss73dJ9WjENcmJBRCjE5mwZP0Qc1t5S446iJyL5LZojxTcHYq3OfFS46OT9wOmXxq7ZRiMlsxRNLBS39Lq7V2YuTIXVe8XY/+2u/7MemDtQA8X6fbBkn2rxE6wdDsXnjb8RKX/LJrRBTS1VAS1HLqH7l7LsSZEg9IbHlDIxSk7DxRgk7PLMG9X27G6Nkr8euOU4p5tx4vRoEXe1UHGhIKYYD0efPknlbXWXOoUFGvK3Sy3s4UXNRHViHNlmixckfHoFTnL9tP4oGvt3jUN5vsW4e6tFfS2Vm4+0xBEYc3jO+9jTMUiPux6EinUXz1wUKXc4mzq61DiNBrs324xAnyw7/fWZ88cvXJCSzprIV5uN9KTXQ1NHt3HdLsP9mdMhbZhcGBM+4r+gXGzFmNS19dHvL9GwTqb+8TxvTJTHb5niLZRD4ilEGSgkTGtEV48kf3Xd0+Wn1EceMfp9HTuzDj4hf2+vfto2L793u/3IxHvrMF71F6rx+etw0Ltp100ZFLqTa76/SlbbSpj7S1Wa9gq/CFKrPFZeTJOcdamWB9jlPJnFJJHjpnCurIua9yONcIqCHeT1zptsi1U25NonSxmkMoeOHtLT6X97+Ta35hnxFH+1RKl9d4vyVuoCChUAt8emtvrHtyqON7YrSrUPAUrrhxnPuGMuFKQan8egeTwjqMg/bRU43ZqhqyWYxYHXWi2HV/g98k+097Ki+3sE7oROQMvVKhwDnX3HkI1ydUu7/gAtYdLvRQQhnOgalfbsbqg+cAAPM2Hsei7e7qCbW9DeRwzt48X5dYTSTdsUwrUz5yrupXaqvc/ZVLEwSR9JivsY98ld1l1WYMfW25av2eG+LbuX2FhEItYDTo0USyqG3WhG6Oz56mjTOvzQlau0JFYVmNbLrgr11aZXLr3D2h5HEk7RjVXsKKGjOun7sWO084deUmu1SQi5skFVwWGe8jJYTOVjCSj3hjBSbMXautsJ0/9hS4fD9fbruvRxUi3DpUa170Mg7PLbV8LmXEahfNp1KuUIRap26xcmQ9sQjr82wzJeGalYSEHOIYXFrPK0XIvuVYEQ6dLfeYR4mv1h31nCFI0OK1MGFMbnOkJzZye9GleLPxeLjy7ALPK1w/lEy11ZDr7EsqTbj3K22B5QS255dgzeFCF7WXULWccdzNpmDVPvpzqI/8sEy/v9L1Pql1XJ68jxSN8MJ/mcmdNJaUUnjvQHofqbl0V5stsmofb1Y0M7ivlfH1Z/KkOlP7vTYfK/Z4PFiQUAgjemcmo7fE3iAlWFt61mW0vrBqL6Gw0rqixgK9jrl0QHKCyt37yPMYvLCs2hFe2qk+CpxuQK2u40XKsy/FovZ71vX5pR7LuKhaJMV9eWKVfitP61DkkMbx8t37yMuZgspxxiQzEStHldmC6Mja75JJfRTGNE9034Dc5GWkz4aAUhhtKervta37qqyxuC24kpvBuRmaOcehs06vEmlH8seeAvyy7SQAp1BYvOMUpi/araX5qngSCqsPnsMLC5XPo+yBow3ORTMncWeqsbzm9mi0KUjzi50Y1JBb6GblwF97Pc/iXc8r1KWeBwDe+GM/Oj2zxKOzQ6iofbFEKLJ62qVuK2+ratErIVwZ8PIyTfnU+gPBmFxpssCgZ6hUeT+lqryDZ8pww/vrHN+LKlwrePx7m1rq4jYpDqGw8WgRNooi4j79006VViojVZWImfHrXsdn8X1QCnPhqNOLmYyc3v61pft88qTxTn0k/9mWX5JXprziJEniZnvrJxsVcsqz7Xixw/gvhTHmcp/e/MsWTuVClbYBTjAhoVDHqDKTUPAVtZlCtX31q8lsRZRBfRKtpsl7/hd528mcZYfw0Wp5u8nna5WNiztPlGDOsoOKxz0F191xQrzQTKZjVLEpaEEu9tGSXfKj60tm/oWcFgmKdSkuXpNNdiaKnQRs+W0RVIUAeWrqo5IKk0PwcA/CRsraw4Xom5XibBHnuHrOasX8e09fkI1GHA7u6KQ+qmNUmbSrj96Z1F3xWMM0TXh+s4WZgtnKcU7BQ0qMmtus3NoGAIoCQUojg+sexVe8uQq/7lR2sbU4VCXe/7jK6hrtdXhz1vyiSizeoXwtwnnv+nwj2jy52JEu75Lq/PzgN1tcjpkt3BFBVZpXoKLGjMfmb0NxRQ32n7kgey41YTLR7jnm2LfBY27gl20nMfAV9xluOLyXNFOoI/xvYjd0aBqPkxpdNR+9rD1GdWmmeDwqQueVgKkPqHX0JrMgFLTdFzWjf5lGW4cS3qzVALSreuRyKRX1xqbgLBM447l0piErFKziztv12GlJuAi58msPn8faw+cRG2XAyM5NHemewlxYrO57dJdXm31y93VN1Fw8aNBMoY5wdbfmaN80DkM6NFbMM75nC7e0dyZ1R4emcW7p4eDlEG4IQe+kwe+UqFZR5Skt0tOKt5oE6epdJeQ6I6XOTKvXzZbjRV6HA/eENyuapV48YqROCJ4Ep/QeuKqPXI9Vmtx/e7Ezgq/3QG7Xw1DLCRIKdZgB7VIxqU8rx3fxyHVCr5YAgFFdmskKEnFsnIyU6CC2su4gp0v2xIY8z1um+ts5lnppdPQvjpJ36VIe/Gar5rUA0iB1sudVWtFsr3x/gVPNI446Ku1UpWGtPbXNFnNL7FrLXY6JESKeihGr+3ydLcktkgx1GG4SCnWYz2/r43iI2jeJc4zUpo/tjNRYZzgMuYFjbJRzphChEIBPbc1EfUPrDKG26lNDEGqCJ4sa5dVmnCqpcikrxZvOTevEprhS3V6jdOsGvLwMM37dixFvrHCkPS9ytVVToVlVgha6zDo8zRRkPKoCMftW864KBUEVCoyxkYyxfYyxg4yxaTLHH2aM7WaMbWeM/ckYax3M9tRF5t7YAy+O6ax4XFBR3DkwS3GkJqdOaBQp3lpS/qkLA0eIkLJEQ5wkbwi5UNA4ohRyTZy71hFORKmsb4NUz4XkOlTpM+ip4/7Yg6FedVc/7jloodKCNalQkFMNRhl0TiO/jz+93DNTb9RHjDE9gDkARgHoBOB6xlgnSbYtAHpyznMAzAfwcrDaU1cZkd0Uk/sqy8oax2YoIp8Tt1WcrmUm9GzpsjhL6SVpaKunt+eXqGfygrIQL0R64/f9KFFbXAE4fnCxm6rSCJtDu/pCmKn+scfznsOfrXF3uz0m2aFOyZUV8BxaXktsJM8bMYnVR+J6XfOZZTZU4tx/I7vWgH/BJJgzhd4ADnLOD3POawB8A+BqcQbO+TLOufA0rAXgbiklPCKE4W7bONbRiUsfYJNEhzvz2hzH7CEp2qD40N3oQRgR6igFQgsWJgvXvDpaGpH1vRWHZfPZ9Ozazm/R6LUlFzJk0CvLXb7P/G2vWx4BT15d6vGflGcTFTVmyV7YzowvS9pjtrgHPxSf29duXG4GUp9sCs0BHBd9z7enKXEbgF/lDjDG7mSMbWSMbTx79qxclgbL5L6tseaJS5GdnuBQ90hfDLmwz5P62Dr8JQ8NdHmAn786GwCQlRrj4p5H1A20rB7mgOaIrFbONaul7v5is3qmIKMml6xceVe/eRvzcfcXmxzfxZe9TrI/hUlmW1mrlTvXKfjYj18+e5V7m0PsOR4WhmbG2GQAPQG8Ineccz6Xc96Tc94zLS0ttI0LQ67v3dLhMcQYQ7OERo7PgPsDKTf6uLJrOvJmXI7G8UaXh1tQKzFmq69vVsMyNtd5NHRG3nZYgQzaF2y0RIrVev2eLltODTV98R7Ncbi8oT6pj04AaCn63sKe5gJjbBiAfwO4inMuvysL4cJ/r8nB8keHuKU71UeSmYJZ5UURyQzBAC2U+HBKL98bSoScFQcCO5N+Zck+LNh6MqB1BhO1DvSlxXs1CwVPapunftrp5j6662QpDtu3bA3kAr765H20AUA7xlgmYywSwEQAC8QZGGO5AN6DTSB4tk4RqsjFngG8W0Ql+FoLdcRERaBVMq1jqCtoCajmbYf12PfbfW2OXxwUhZzQipYOVOvI21OuvacvYPMx5XUqgR3c15OZAufcDOA+AEsA7AEwj3O+izH2PGPsKnu2VwDEAviOMbaVMbZAoTpCAzqHN5z86ssru6bj+6kXu5UTvyTCTEGsMgiUE1JWWkxgKiL8YueJUvVMYcCw11eoZ/IBtY2sBLQYrUNBqGcKQY11wDlfDGCxJO0Z0edhwTx/Q0PJ+0jQc07o2RI9Wie5lRM//MJKZ3GanGvq7Otz8cDXW9zSPVEfdo0j6j7iAHkeUemMxXtKe1nUK+qTTYEINQreR4JHSnSUXloCgHOq+99ruiDeaAAAlz2k5bryq7qme928hrbugajbhIt9vT7ZFIgQo1PwProyxxYttbWCbUB46IZ2aIystFjc1K81Hr2svTODvS///LbefrXPH5nw0tgufp07kDw4tF1tN4EIAV9vOOZz2UCuLQj1OgUKlVmPcOyGKHmIbrskEzf2a42oCKWZgi0/Ywx6HcPzV7uG1RDqlYYKbhpvdAtN7Al/ZgoxCrOc2kDLBjxE3WfR9lM+lyX1EREWKNkUGGOKAgEA0uJswfOUdn0S1j9E6Fwfl3dv7OFd+7x42rLT413LhpHqSbp/M0EEkxDLBBIK9QmlFc1qfHJLb7xybQ6SYiJljwvdcYTetWNuEh/lnhmQ9XACvDM0i6O8AuElFKIi6LUhVAhgRx7qxYP0dNcnfNzkpGmCEdf1bKl4XOiPpTOJZgmNsP7JoS5pvz44wLHWoUPTOGx/boSoHu0duzRrOEVslQvINrxTk1poCRGuBHLxWqghoVCPcKxTCPB884ocm6dRSqz7zKBxvBHf3d3P8d1i5ejQNA63XZKJ927s4fBmArwzNEuz6sJIKsjZFBrHyc+aiPChf9uUkJ0rkPGKTBYrDp8tC1yFKpBQqEco2RT85f5L22LbMyPQPLGR7PFeGcno3Dzefm4OnY7h6Ss6oXWK62I1b7p1qboonNRHkXp3+0y4uC8Syhg92NUCjdYgglrYfKwYl772Ny6EKBQ7CYV6hNKKZn9hjCEh2jnif+DStjLnVhdIUvXRmicuVczbtkmspH5gzg3dHd+v69ECv9x3CT691d1Ndt2TQ/HX/w1ySRNvlegvBr27gAq122BWauBWh792XVfZ9NTYKNwxIDNg56ltQuk1Fgw7gNXKKUMAABXjSURBVNzmRMGAhEI9ggVppiAmb8bleHhEe7f0FLuRWs6D6dkrbXsrSY8kKxi2AeARyTl0jLl4JBkNenRpkYBBF6Vh0EWukXObxBuRleYUKt/d3Q9TB7dRPJe3RMoYmkPtIfLsVdkBqWdir5a4TCFEutGgC6sZmlY6NI2TTTcGcGCgRjCEwlfrfV834Q0kFOoRgleMoRb076+P74b/XJXt5krqCaagUPru7n5uxlzGXNdJeDMb6pWR7OY5pYUh7Z3CRuyGKicUpB5fX9zWx+vzeYMvv/GdA7Pc0gZelKboTRVnNHjlHKDGm9fnBqwuTySKZrViQikUgrG2YPafBwJepxwkFOoRk/u2xl2DsnB3AEfFWkmKicSUizNkOxEhRXpIqb/pleG+h4Nex1yMzeJ3Tkv/2K6x/OjRE0KcqHuHtMHcm5xrMuTWKUgHhpe0S/X6fAItkuRtN/4y6KI0TO7byiWNQXl9SlxUREC9vkLloSVdZCkQSpuCyRJ4oRAquxUJhXqE0aDHE6M6IjoyvBaqZzdPAAD0znTt7MWv7odTenqsQ8eYSweVFO1UPWkZzQ7v1MRhDNeK3r7ajoG5qFHk7q945tK1ZaJX53E/r4ae2IfOWscYXhzTBW0bO1VrvTOTFe+fXsdQbZZ3o9Ex5RG5gHSDJk3XFQCUVF7GANkUtISSn7fhuGoeXwjFmgUSCkTQ6ZWRjPX/HoorJUH0xJ3R0I6eR5GMuS5+u3+o09jdM8M98qscndMTVPPcJVKxTLm4NSb1aYW7BmWhawtnR2/QMyz910CXcuKZy0/3yC/e00ogOk+5UblQr7hzlHMzFtDpgKKKGtljVg70bO15V77XxndzPX8Q7RODRao+xZlCgNRHN/VT37vc014L/uDN3ii+QkKBCAmN44xuaToGxBsj8IJ9X+js9HjFYHM6xlyEiDhsx90D2+Bfwy5yK5OZGoOERs7RrLSzmNSnFVZPuxTvicN1MKfNIDoyAtPHdkGc0YCEaINDnRSh17m557ruP+F6nggdw1s35DoM7mrIqXNGd/Fuv+z3b3KfeXVvZRNsWsN06BjzuDJ39vXdlA/CdTLz1e19grrWROxsoCR8ArUSXW7xohSznyP6HaJFn2JCEQcpvPQMRL1GalhmjGH7c5c5vi96YIBiWZ09WJ/sMR3DPUPa4I0/9rukS91ShZc5KzUGheU1mG6PvBpndL4GhWU1+PPhQY5tFcXMHJeD33cXyEabFfzSr7BHpBVz8KXRjs//+WW343Nuq0RsOVbsll8vEyTK7d550B/d0j/DLe2y7CaIsF+/mkdR1xYJ2JZf4jEPIK9GEyN2AxZUiL5y35C2eGvZQcXj4itSUocFaqYgdVpY9MAluHz2qoDULaD0rPsrbLRAMwUiZHirPXhhjDNaq455Vj8Y9Dp8P/ViLBYJFiaZXQgj8Ot7t8K2Z50jsXijAU9d3hEAUFppQsvkaDc3VwBo2zgWUwe3gU5i9AYAq5Vj27MjMGuCc/Q8a0I3TB/bWVqNg49vdu5/LcR66tYyEXID0UU7tEfslNN5uxrmVYSC3SaiY0x1ZLrxqWGKIdXFsbS0TBKu6d5cNn1Au1QM7djYY9k4o3hG6CwnJlA2BelMIRixsJR+IysJBaI+4a324Ma+rdGpmc04bDTowVSe1h6tk9DJg0tsul3lI2cgbWM3vlYpGFbVMFs5EhoZHKNxABiT2xyT+ijrn8WqrY1PDcMHN/XEF7f3kZ0pSPHUr1/f29XDqGm8Efdf6lTLyZVtmWy7NzPHdXHo53UMMCl0QsJ+G6mxUZoWBqrZSUZmN0WzBHcVo1bG5DoFinCuXInBX2mmIH0eUmMjcc/gNnh4uLtKEnD93QD36MG+cvslzoWCSkIhv6gyIOfyBAkFIoR4r1Ousu8v3ShS77ehcsrFGXjrhlyM697C7ZjgriicTwsjOjVBL7uR29sR3Ge39nbMYi63q5yGdWqC2KgIRRdRTzxwaVsYDTqkJxgdnd8t/W3Xu/bJoejSwqm+EW7ju5OdthRhQtAvK9URt0fHGCwyrpV5My7HvUOchn7h0nt5MPirzU5G5zRTVI0wxhRNG89c0QkHpo9yETrNEmwCrnG80WV2IBUKz1+djZ/v7e/2XMUZDXhsZAfFqMEjJEb8QHlVPSQSQkp1Lt2tbX9pfyCbAhEy4ht5/7hV2Jf2NzLo/V5dq9cxR3A/KUIIhGovhMLcm3pi2d4zuOWTDZpj3RyYPsrFPrLl6eGINbreFy3CT5oju3kCdorsMwDw7JXyq56F+yjeuEhoPmNOYyZjgFlDZDfByG7bt0Mn68aq9ttF6JiiYGWQXzH+y32XuAg7gRv6tEKP1kkYmd0Uc5YdxKkS20ZQ0kWHN/XLAODeAXtqaYemcW42i0AJhdioCDx3ZSf899e9irPqUHj10kyBCBmN44z47aEBSI2Nkl0VLEerFJt+PNYY4dUmPd6SGmPT6bdJi1XJ6YpgW9DqP27Q61w6kaSYSNnV22pIOyarlSNCr3NRXykh5BGPzAVBoNM5R+WMMZwtc3VJlerpAWfcJz1jWPn4ENlzqnVmeh2D4G0pXd+gdD+UOmODTofRXZpBp2N4ThQORGkGJq3HkzFXdnFmADvqm/tnYt+Lo1zOs+yRwRjf0za7DaZbrwDNFIiQ0qFpPNY9OVRzALn3JvfAjhMliDcaYA6ij3arlGh8e2df5LTwbuGZrxsbeULLey/Vg3ujvRJCZJgtMkKBOTt5HQOeGNUBE+euBQDMu6uf2wJEAI5gie2bxqFxnBH/m9jNsZufgNpoOipCh/REm02hVXI01h4+7zimNMtQCnWiF3kHXZbtdOVV9F6T1F/pYbaYmepuxPfkCSbHkPZpWLbvrGq+L2/vg63Hi5GZGuNYTxKKEPIkFIiQY3s5tT3cSTGRGHiRYPgM7gvRJ8v7ePt6mQ7WX4ROJjM1BkckrrEvj8tBy+RoXNTENWyHN6GaBZdKsZAVhIqOMfTKSEZ0pB53DWqD7q2cdgI5gQAA2ekJ+OqOPo7FbFd3c/ciUlt1nhwTiVv6Z6J1SgwMeoZ5G/Ml53B3IFC6ZKk30G2XZOLDVUcUDcJSYaFU76wJ3RxeUFueHo7cF34HYLN3aWFsbnOkxUXhX8MuwvGiCox4Y4XH/P3bpqJ/W9vMTBDaoVgVTuojos4QThvtCKTZR3Adm3kXQsMTQt/1wtWdkZ0ej4m9nLviXdO9Ofq1cRdeFi92dRHUR2LPIodNAbZVzrufH+kiENS4uE2qZpWglG4tE3FRkzjodQzDOzVx65Q55y5GYuE8SrMzaTuevqIT8mZcrtg+aUcreLB1lgiiMbnNHa6vYiO01BtJiSu7NsOTozuiUaTeTairIVxqKNRHJBQIwg/aNYnD91P74cnRHQNWpzBT4OBY9MAAzBiX4xA6SiNFbzRrTvWRsxDnTmOxHN38jOfkiZ/u7e/S6Us7e6mOX1j1Lt3ESUBp3YDYE0kcpkR8ye0ax+LNibZorrmtkvDHw64LIMVkpcU4Qrf8eM/FeO/GHi5xpV67ritu7e90M5WqmVY9PsThCqyG05ivKbtfkPqIqFP0ykjC5L7qsWdCSQ+VGEDecnHbFKw6eM6xrgKwhYk4fK5MsdP2xiV2ZOdm+GnrSWSLYkHNGJeDl3/biySZNRz7XhwZkhGqgFQds/LAOZfv13Rv7uISK0UpjIeQnpHiqn4b3bmZY7X0XYPauGwoJdwPub0//vq/wY7PufZZ1WXZTTH4lWXIK6xAj9ZJGNejBQpKq7Boxym3mUqLpGi0TIrG8fPqaw8EoRAK9REJBaJO8d3d/gWbqwvcPbANruqajhZJTqNmUkwkesS4Cp+pg9tgya7T0DHmVVjqkZ2b4sD0US5eT8M7NVGsI8qPkNNTB7fBO8sPOb4ve2Qwdp4owf1fb1Es0y8rBS+Py8Fbyw7i2PkKr9ujJDiFTrlG4jL78PCLcOslmdh9stRtH+ek6EiM6twUt4kWlqnh9N6y/X9xTGd0aBqHfjI2q47N4vHPoULFVeECfTKT8ck/eciRccENNCQUCCLM0OmYi0BQ4vGRHfD4yA4+nUNLULdAIG1jZmoMzl6odny/S2bjH8YYxvdqiXkbj+PY+QrERrl2U76Gq4ixx2pqIlk5rdMxJMdEyu6BodMxvCNa5KeFrNQYHC2scKjEkmIicb9CoMdpozrgipxmjpmGEqO6NMPmp4d73K0wUJBQIAgipAgakB6tk/CEB1uM4FH12njXPaSVNsv59s6+2HLcPcCgQFJMJN6e1F12E6dA8r/rc7HpaBGaxKuH7TDodaoCQSAUAgEgoUAQRIhx7iXu2Q4iBAnMSrUZlId3aoLfdxcoeqH1yUpRdSse3cU9im2giTcaMKS95wB+4QwJBYIgQoqwslhtX4dXr+2KZfvOoJ3dKDznhu64UGUKevsaOiQUCIIIKTktEvDA0Ha4QRLNVUpCtMEl+mlkhM7jTnFEYCChQBBESGGMKYalJmofWrxGEARBOAiqUGCMjWSM7WOMHWSMTZM5HsUY+9Z+fB1jLCOY7SEIgiA8EzShwBjTA5gDYBSATgCuZ4xJdy6/DUAR57wtgDcAzAxWewiCIAh1gjlT6A3gIOf8MOe8BsA3AK6W5LkawKf2z/MBDGVq4RQJgiCIoBFModAcwHHR93x7mmwezrkZQAkA7+MXEwRBEAGhThiaGWN3MsY2MsY2nj2rvjkFQRAE4RvBFAonALQUfW9hT5PNwxiLAJAAoFBaEed8Lue8J+e8Z1paWpCaSxAEQQRTKGwA0I4xlskYiwQwEcACSZ4FAKbYP18L4C+udZ9GgiAIIuCwYPbBjLHRAGYB0AP4iHM+nTH2PICNnPMFjDEjgM8B5AI4D2Ai5/ywSp1nARz1sUmpAM6p5qodqG3eE67tAqhtvkJt8x6t7WrNOVdVtQRVKIQbjLGNnPOetd0OOaht3hOu7QKobb5CbfOeQLerThiaCYIgiNBAQoEgCIJw0NCEwtzaboAHqG3eE67tAqhtvkJt856AtqtB2RQIgiAIzzS0mQJBEAThgQYjFNQitgb53C0ZY8sYY7sZY7sYYw/a05MZY78zxg7Y/yfZ0xljbLa9rdsZY91D0EY9Y2wLY2yh/XumPXLtQXsk20h7ekgj2zLGEhlj8xljexljexhj/cLlvjHG/mX/PXcyxr5mjBlr674xxj5ijJ1hjO0UpXl9nxhjU+z5DzDGpsidKwDtesX+e25njP3IGEsUHXvC3q59jLHLROkBf3/l2iY69n+MMc4YS7V/D9k989Q2xtj99nu3izH2sig9cPeNc17v/2BbJ3EIQBaASADbAHQK4fmbAehu/xwHYD9skWNfBjDNnj4NwEz759EAfgXAAPQFsC4EbXwYwFcAFtq/z4Nt3QgAvAtgqv3zPQDetX+eCODbILfrUwC32z9HAkgMh/sGW9yuIwAaie7XzbV13wAMBNAdwE5Rmlf3CUAygMP2/0n2z0lBaNcIABH2zzNF7epkfzejAGTa31l9sN5fubbZ01sCWALbeqjUUN8zD/dtCIA/AETZvzcOxn0L2sscTn8A+gFYIvr+BIAnarE9PwMYDmAfgGb2tGYA9tk/vwfgelF+R74gtacFgD8BXApgof3BPyd6cR33z/6y9LN/jrDnY0FqVwJsHS+TpNf6fYMzmGOy/T4sBHBZbd43ABmSTsSr+wTgegDvidJd8gWqXZJjYwF8af/s8l4K9yyY769c22CL2NwVQB6cQiGk90zh95wHYJhMvoDet4aiPtISsTUk2NUGuQDWAWjCOT9lP3QaQBP751C3dxaAxwBY7d9TABRzW+Ra6flDGdk2E8BZAB/bVVv/3969hVhVR3Ec//7ANNQHjYgMIS+pr4aGIxaYhViJvXRF0C4kPfoSURNCQS8VQvRQUEEPiZg1+FhEqZGZkqJOdJ1ISstsHroZiMbqYf3nuDmcsZk45+xT8/vAYWb23jPnP2tmz5r/5az/K5Km0QNxi4iTwHPAd8CPZBwO0RtxGzHeONVxnzxA/gfeE+2SdDtwMiKONp2qvW3AQuCGMvy4V9J1nWjbREkKPUHSdOAtYHNE/FY9F5nKu74UTNJa4HREHOr2c4/BJLIL/WJEXAucIYdBGmqM20xyP5C5wFXANGBNt9sxVnXF6WIk9QPngW11twVA0lTgcWBL3W0ZxSSyZ9oHPAK8IbV//5mJkhTGUrG1oyRdQiaEbRExUA7/JGlWOT8LOF2Od7O9K4B1ko6TGyGtAp4HZigr1zY//5gq27bJCeBERBwoH79JJoleiNvNwLcR8XNEnAMGyFj2QtxGjDdOXYufpPuAtcD6krB6oV3zySR/tNwPs4HDkq7sgbZB3g8DkQ6SPfvL2922iZIUxlKxtWNKNn8V+DwitlZOVavEbiTnGkaObygrHvqAXyvDAG0VEY9FxOyImEPG5f2IWA/sJivXtmpbVyrbRsQp4HtJi8qhm4DP6IG4kcNGfZKmlp/vSNtqj1vFeOP0DrBa0szSE1pdjrWVpDXkcOW6iPizqb33KFdqzQUWAAfp0v0bEYMRcUVEzCn3wwlygcgpao5ZsYucbEbSQnLyeJh2x60dEyL/hQe5euArcja+v8vPfT3ZdT8GHCmPW8kx5feAr8lVBZeV60Xub/0NMAgs7VI7V3Jh9dG88os1BOzkwoqHS8vHQ+X8vA63aTHwSYndLnKFR0/EDXgS+AL4lKz2O6WuuAHbybmNc+Qfswf/TZzIMf6h8ri/Q+0aIse6R+6FlyrX95d2fQncUjne9vu3Vduazh/nwkRz12J2kbhNBl4vv2+HgVWdiJtf0WxmZg0TZfjIzMzGwEnBzMwanBTMzKzBScHMzBqcFMzMrMFJwWwUkvpLNcpjko5IWiZpc3nlq9n/kpekmrUgaTmwFVgZEWdLCeXJwEfkGvXhWhto1iHuKZi1NgsYjoizACUJ3EHWOdotaTeApNWS9ks6LGlnqW+FpOOSnpE0KOmgpGvK8TuV+y8clfRBPd+a2ejcUzBrofxx/xCYSr4aeEdE7C01cZZGxHDpPQyQryA9I+lR8hXMT5XrXo6IpyVtAO6KiLWSBoE1EXFS0oyI+KWWb9BsFO4pmLUQEX8AS4BNZPnuHaWIW1UfucHJPklHyPpCV1fOb6+8XV7e3we8JukhchMUs54y6Z8vMZuYIuIvYA+wp/yHv7HpEgHvRsS9o32J5vcj4mFJy4DbgEOSlkREp6ulmo2ZewpmLUhaJGlB5dBicnvG38ktVQE+BlZU5gumleqVI+6uvN1frpkfEQciYgvZA6mWNjarnXsKZq1NB15Qbip/nqyAuYncfvFtST9ExI1lSGm7pCnl854gq1ICzJR0DDhbPg/g2ZJsRFYwbd7hy6xWnmg264DqhHTdbTEbDw8fmZlZg3sKZmbW4J6CmZk1OCmYmVmDk4KZmTU4KZiZWYOTgpmZNTgpmJlZw981pVCHuC64kgAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "train_loss = np.array(model.get_train_summary('Loss'))\n",
+    "val_loss = np.array(model.get_validation_summary('Loss'))\n",
+    "\n",
+    "plt.plot(train_loss[:,0],train_loss[:,1],label='train loss')\n",
+    "plt.plot(val_loss[:,0],val_loss[:,1],label='validation loss',color='green')\n",
+    "plt.xlabel('Steps')\n",
+    "plt.ylabel('Loss')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

From c875f97f6bdd37d4e540fedbcf01ef61582fcb58 Mon Sep 17 00:00:00 2001
From: Kai Huang <huangkaivision@gmail.com>
Date: Mon, 25 Mar 2019 19:53:24 +0800
Subject: [PATCH 37/46] Add bert run_classifier original code (#13)

---
 tensorflow/bert/run_classifier.py | 981 ++++++++++++++++++++++++++++++
 1 file changed, 981 insertions(+)
 create mode 100644 tensorflow/bert/run_classifier.py

diff --git a/tensorflow/bert/run_classifier.py b/tensorflow/bert/run_classifier.py
new file mode 100644
index 0000000..817b147
--- /dev/null
+++ b/tensorflow/bert/run_classifier.py
@@ -0,0 +1,981 @@
+# coding=utf-8
+# Copyright 2018 The Google AI Language Team Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""BERT finetuning runner."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import collections
+import csv
+import os
+import modeling
+import optimization
+import tokenization
+import tensorflow as tf
+
+flags = tf.flags
+
+FLAGS = flags.FLAGS
+
+## Required parameters
+flags.DEFINE_string(
+    "data_dir", None,
+    "The input data dir. Should contain the .tsv files (or other data files) "
+    "for the task.")
+
+flags.DEFINE_string(
+    "bert_config_file", None,
+    "The config json file corresponding to the pre-trained BERT model. "
+    "This specifies the model architecture.")
+
+flags.DEFINE_string("task_name", None, "The name of the task to train.")
+
+flags.DEFINE_string("vocab_file", None,
+                    "The vocabulary file that the BERT model was trained on.")
+
+flags.DEFINE_string(
+    "output_dir", None,
+    "The output directory where the model checkpoints will be written.")
+
+## Other parameters
+
+flags.DEFINE_string(
+    "init_checkpoint", None,
+    "Initial checkpoint (usually from a pre-trained BERT model).")
+
+flags.DEFINE_bool(
+    "do_lower_case", True,
+    "Whether to lower case the input text. Should be True for uncased "
+    "models and False for cased models.")
+
+flags.DEFINE_integer(
+    "max_seq_length", 128,
+    "The maximum total input sequence length after WordPiece tokenization. "
+    "Sequences longer than this will be truncated, and sequences shorter "
+    "than this will be padded.")
+
+flags.DEFINE_bool("do_train", False, "Whether to run training.")
+
+flags.DEFINE_bool("do_eval", False, "Whether to run eval on the dev set.")
+
+flags.DEFINE_bool(
+    "do_predict", False,
+    "Whether to run the model in inference mode on the test set.")
+
+flags.DEFINE_integer("train_batch_size", 32, "Total batch size for training.")
+
+flags.DEFINE_integer("eval_batch_size", 8, "Total batch size for eval.")
+
+flags.DEFINE_integer("predict_batch_size", 8, "Total batch size for predict.")
+
+flags.DEFINE_float("learning_rate", 5e-5, "The initial learning rate for Adam.")
+
+flags.DEFINE_float("num_train_epochs", 3.0,
+                   "Total number of training epochs to perform.")
+
+flags.DEFINE_float(
+    "warmup_proportion", 0.1,
+    "Proportion of training to perform linear learning rate warmup for. "
+    "E.g., 0.1 = 10% of training.")
+
+flags.DEFINE_integer("save_checkpoints_steps", 1000,
+                     "How often to save the model checkpoint.")
+
+flags.DEFINE_integer("iterations_per_loop", 1000,
+                     "How many steps to make in each estimator call.")
+
+flags.DEFINE_bool("use_tpu", False, "Whether to use TPU or GPU/CPU.")
+
+tf.flags.DEFINE_string(
+    "tpu_name", None,
+    "The Cloud TPU to use for training. This should be either the name "
+    "used when creating the Cloud TPU, or a grpc://ip.address.of.tpu:8470 "
+    "url.")
+
+tf.flags.DEFINE_string(
+    "tpu_zone", None,
+    "[Optional] GCE zone where the Cloud TPU is located in. If not "
+    "specified, we will attempt to automatically detect the GCE project from "
+    "metadata.")
+
+tf.flags.DEFINE_string(
+    "gcp_project", None,
+    "[Optional] Project name for the Cloud TPU-enabled project. If not "
+    "specified, we will attempt to automatically detect the GCE project from "
+    "metadata.")
+
+tf.flags.DEFINE_string("master", None, "[Optional] TensorFlow master URL.")
+
+flags.DEFINE_integer(
+    "num_tpu_cores", 8,
+    "Only used if `use_tpu` is True. Total number of TPU cores to use.")
+
+
+class InputExample(object):
+  """A single training/test example for simple sequence classification."""
+
+  def __init__(self, guid, text_a, text_b=None, label=None):
+    """Constructs a InputExample.
+
+    Args:
+      guid: Unique id for the example.
+      text_a: string. The untokenized text of the first sequence. For single
+        sequence tasks, only this sequence must be specified.
+      text_b: (Optional) string. The untokenized text of the second sequence.
+        Only must be specified for sequence pair tasks.
+      label: (Optional) string. The label of the example. This should be
+        specified for train and dev examples, but not for test examples.
+    """
+    self.guid = guid
+    self.text_a = text_a
+    self.text_b = text_b
+    self.label = label
+
+
+class PaddingInputExample(object):
+  """Fake example so the num input examples is a multiple of the batch size.
+
+  When running eval/predict on the TPU, we need to pad the number of examples
+  to be a multiple of the batch size, because the TPU requires a fixed batch
+  size. The alternative is to drop the last batch, which is bad because it means
+  the entire output data won't be generated.
+
+  We use this class instead of `None` because treating `None` as padding
+  battches could cause silent errors.
+  """
+
+
+class InputFeatures(object):
+  """A single set of features of data."""
+
+  def __init__(self,
+               input_ids,
+               input_mask,
+               segment_ids,
+               label_id,
+               is_real_example=True):
+    self.input_ids = input_ids
+    self.input_mask = input_mask
+    self.segment_ids = segment_ids
+    self.label_id = label_id
+    self.is_real_example = is_real_example
+
+
+class DataProcessor(object):
+  """Base class for data converters for sequence classification data sets."""
+
+  def get_train_examples(self, data_dir):
+    """Gets a collection of `InputExample`s for the train set."""
+    raise NotImplementedError()
+
+  def get_dev_examples(self, data_dir):
+    """Gets a collection of `InputExample`s for the dev set."""
+    raise NotImplementedError()
+
+  def get_test_examples(self, data_dir):
+    """Gets a collection of `InputExample`s for prediction."""
+    raise NotImplementedError()
+
+  def get_labels(self):
+    """Gets the list of labels for this data set."""
+    raise NotImplementedError()
+
+  @classmethod
+  def _read_tsv(cls, input_file, quotechar=None):
+    """Reads a tab separated value file."""
+    with tf.gfile.Open(input_file, "r") as f:
+      reader = csv.reader(f, delimiter="\t", quotechar=quotechar)
+      lines = []
+      for line in reader:
+        lines.append(line)
+      return lines
+
+
+class XnliProcessor(DataProcessor):
+  """Processor for the XNLI data set."""
+
+  def __init__(self):
+    self.language = "zh"
+
+  def get_train_examples(self, data_dir):
+    """See base class."""
+    lines = self._read_tsv(
+        os.path.join(data_dir, "multinli",
+                     "multinli.train.%s.tsv" % self.language))
+    examples = []
+    for (i, line) in enumerate(lines):
+      if i == 0:
+        continue
+      guid = "train-%d" % (i)
+      text_a = tokenization.convert_to_unicode(line[0])
+      text_b = tokenization.convert_to_unicode(line[1])
+      label = tokenization.convert_to_unicode(line[2])
+      if label == tokenization.convert_to_unicode("contradictory"):
+        label = tokenization.convert_to_unicode("contradiction")
+      examples.append(
+          InputExample(guid=guid, text_a=text_a, text_b=text_b, label=label))
+    return examples
+
+  def get_dev_examples(self, data_dir):
+    """See base class."""
+    lines = self._read_tsv(os.path.join(data_dir, "xnli.dev.tsv"))
+    examples = []
+    for (i, line) in enumerate(lines):
+      if i == 0:
+        continue
+      guid = "dev-%d" % (i)
+      language = tokenization.convert_to_unicode(line[0])
+      if language != tokenization.convert_to_unicode(self.language):
+        continue
+      text_a = tokenization.convert_to_unicode(line[6])
+      text_b = tokenization.convert_to_unicode(line[7])
+      label = tokenization.convert_to_unicode(line[1])
+      examples.append(
+          InputExample(guid=guid, text_a=text_a, text_b=text_b, label=label))
+    return examples
+
+  def get_labels(self):
+    """See base class."""
+    return ["contradiction", "entailment", "neutral"]
+
+
+class MnliProcessor(DataProcessor):
+  """Processor for the MultiNLI data set (GLUE version)."""
+
+  def get_train_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")
+
+  def get_dev_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "dev_matched.tsv")),
+        "dev_matched")
+
+  def get_test_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "test_matched.tsv")), "test")
+
+  def get_labels(self):
+    """See base class."""
+    return ["contradiction", "entailment", "neutral"]
+
+  def _create_examples(self, lines, set_type):
+    """Creates examples for the training and dev sets."""
+    examples = []
+    for (i, line) in enumerate(lines):
+      if i == 0:
+        continue
+      guid = "%s-%s" % (set_type, tokenization.convert_to_unicode(line[0]))
+      text_a = tokenization.convert_to_unicode(line[8])
+      text_b = tokenization.convert_to_unicode(line[9])
+      if set_type == "test":
+        label = "contradiction"
+      else:
+        label = tokenization.convert_to_unicode(line[-1])
+      examples.append(
+          InputExample(guid=guid, text_a=text_a, text_b=text_b, label=label))
+    return examples
+
+
+class MrpcProcessor(DataProcessor):
+  """Processor for the MRPC data set (GLUE version)."""
+
+  def get_train_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")
+
+  def get_dev_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "dev.tsv")), "dev")
+
+  def get_test_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "test.tsv")), "test")
+
+  def get_labels(self):
+    """See base class."""
+    return ["0", "1"]
+
+  def _create_examples(self, lines, set_type):
+    """Creates examples for the training and dev sets."""
+    examples = []
+    for (i, line) in enumerate(lines):
+      if i == 0:
+        continue
+      guid = "%s-%s" % (set_type, i)
+      text_a = tokenization.convert_to_unicode(line[3])
+      text_b = tokenization.convert_to_unicode(line[4])
+      if set_type == "test":
+        label = "0"
+      else:
+        label = tokenization.convert_to_unicode(line[0])
+      examples.append(
+          InputExample(guid=guid, text_a=text_a, text_b=text_b, label=label))
+    return examples
+
+
+class ColaProcessor(DataProcessor):
+  """Processor for the CoLA data set (GLUE version)."""
+
+  def get_train_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")
+
+  def get_dev_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "dev.tsv")), "dev")
+
+  def get_test_examples(self, data_dir):
+    """See base class."""
+    return self._create_examples(
+        self._read_tsv(os.path.join(data_dir, "test.tsv")), "test")
+
+  def get_labels(self):
+    """See base class."""
+    return ["0", "1"]
+
+  def _create_examples(self, lines, set_type):
+    """Creates examples for the training and dev sets."""
+    examples = []
+    for (i, line) in enumerate(lines):
+      # Only the test set has a header
+      if set_type == "test" and i == 0:
+        continue
+      guid = "%s-%s" % (set_type, i)
+      if set_type == "test":
+        text_a = tokenization.convert_to_unicode(line[1])
+        label = "0"
+      else:
+        text_a = tokenization.convert_to_unicode(line[3])
+        label = tokenization.convert_to_unicode(line[1])
+      examples.append(
+          InputExample(guid=guid, text_a=text_a, text_b=None, label=label))
+    return examples
+
+
+def convert_single_example(ex_index, example, label_list, max_seq_length,
+                           tokenizer):
+  """Converts a single `InputExample` into a single `InputFeatures`."""
+
+  if isinstance(example, PaddingInputExample):
+    return InputFeatures(
+        input_ids=[0] * max_seq_length,
+        input_mask=[0] * max_seq_length,
+        segment_ids=[0] * max_seq_length,
+        label_id=0,
+        is_real_example=False)
+
+  label_map = {}
+  for (i, label) in enumerate(label_list):
+    label_map[label] = i
+
+  tokens_a = tokenizer.tokenize(example.text_a)
+  tokens_b = None
+  if example.text_b:
+    tokens_b = tokenizer.tokenize(example.text_b)
+
+  if tokens_b:
+    # Modifies `tokens_a` and `tokens_b` in place so that the total
+    # length is less than the specified length.
+    # Account for [CLS], [SEP], [SEP] with "- 3"
+    _truncate_seq_pair(tokens_a, tokens_b, max_seq_length - 3)
+  else:
+    # Account for [CLS] and [SEP] with "- 2"
+    if len(tokens_a) > max_seq_length - 2:
+      tokens_a = tokens_a[0:(max_seq_length - 2)]
+
+  # The convention in BERT is:
+  # (a) For sequence pairs:
+  #  tokens:   [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP]
+  #  type_ids: 0     0  0    0    0     0       0 0     1  1  1  1   1 1
+  # (b) For single sequences:
+  #  tokens:   [CLS] the dog is hairy . [SEP]
+  #  type_ids: 0     0   0   0  0     0 0
+  #
+  # Where "type_ids" are used to indicate whether this is the first
+  # sequence or the second sequence. The embedding vectors for `type=0` and
+  # `type=1` were learned during pre-training and are added to the wordpiece
+  # embedding vector (and position vector). This is not *strictly* necessary
+  # since the [SEP] token unambiguously separates the sequences, but it makes
+  # it easier for the model to learn the concept of sequences.
+  #
+  # For classification tasks, the first vector (corresponding to [CLS]) is
+  # used as the "sentence vector". Note that this only makes sense because
+  # the entire model is fine-tuned.
+  tokens = []
+  segment_ids = []
+  tokens.append("[CLS]")
+  segment_ids.append(0)
+  for token in tokens_a:
+    tokens.append(token)
+    segment_ids.append(0)
+  tokens.append("[SEP]")
+  segment_ids.append(0)
+
+  if tokens_b:
+    for token in tokens_b:
+      tokens.append(token)
+      segment_ids.append(1)
+    tokens.append("[SEP]")
+    segment_ids.append(1)
+
+  input_ids = tokenizer.convert_tokens_to_ids(tokens)
+
+  # The mask has 1 for real tokens and 0 for padding tokens. Only real
+  # tokens are attended to.
+  input_mask = [1] * len(input_ids)
+
+  # Zero-pad up to the sequence length.
+  while len(input_ids) < max_seq_length:
+    input_ids.append(0)
+    input_mask.append(0)
+    segment_ids.append(0)
+
+  assert len(input_ids) == max_seq_length
+  assert len(input_mask) == max_seq_length
+  assert len(segment_ids) == max_seq_length
+
+  label_id = label_map[example.label]
+  if ex_index < 5:
+    tf.logging.info("*** Example ***")
+    tf.logging.info("guid: %s" % (example.guid))
+    tf.logging.info("tokens: %s" % " ".join(
+        [tokenization.printable_text(x) for x in tokens]))
+    tf.logging.info("input_ids: %s" % " ".join([str(x) for x in input_ids]))
+    tf.logging.info("input_mask: %s" % " ".join([str(x) for x in input_mask]))
+    tf.logging.info("segment_ids: %s" % " ".join([str(x) for x in segment_ids]))
+    tf.logging.info("label: %s (id = %d)" % (example.label, label_id))
+
+  feature = InputFeatures(
+      input_ids=input_ids,
+      input_mask=input_mask,
+      segment_ids=segment_ids,
+      label_id=label_id,
+      is_real_example=True)
+  return feature
+
+
+def file_based_convert_examples_to_features(
+    examples, label_list, max_seq_length, tokenizer, output_file):
+  """Convert a set of `InputExample`s to a TFRecord file."""
+
+  writer = tf.python_io.TFRecordWriter(output_file)
+
+  for (ex_index, example) in enumerate(examples):
+    if ex_index % 10000 == 0:
+      tf.logging.info("Writing example %d of %d" % (ex_index, len(examples)))
+
+    feature = convert_single_example(ex_index, example, label_list,
+                                     max_seq_length, tokenizer)
+
+    def create_int_feature(values):
+      f = tf.train.Feature(int64_list=tf.train.Int64List(value=list(values)))
+      return f
+
+    features = collections.OrderedDict()
+    features["input_ids"] = create_int_feature(feature.input_ids)
+    features["input_mask"] = create_int_feature(feature.input_mask)
+    features["segment_ids"] = create_int_feature(feature.segment_ids)
+    features["label_ids"] = create_int_feature([feature.label_id])
+    features["is_real_example"] = create_int_feature(
+        [int(feature.is_real_example)])
+
+    tf_example = tf.train.Example(features=tf.train.Features(feature=features))
+    writer.write(tf_example.SerializeToString())
+  writer.close()
+
+
+def file_based_input_fn_builder(input_file, seq_length, is_training,
+                                drop_remainder):
+  """Creates an `input_fn` closure to be passed to TPUEstimator."""
+
+  name_to_features = {
+      "input_ids": tf.FixedLenFeature([seq_length], tf.int64),
+      "input_mask": tf.FixedLenFeature([seq_length], tf.int64),
+      "segment_ids": tf.FixedLenFeature([seq_length], tf.int64),
+      "label_ids": tf.FixedLenFeature([], tf.int64),
+      "is_real_example": tf.FixedLenFeature([], tf.int64),
+  }
+
+  def _decode_record(record, name_to_features):
+    """Decodes a record to a TensorFlow example."""
+    example = tf.parse_single_example(record, name_to_features)
+
+    # tf.Example only supports tf.int64, but the TPU only supports tf.int32.
+    # So cast all int64 to int32.
+    for name in list(example.keys()):
+      t = example[name]
+      if t.dtype == tf.int64:
+        t = tf.to_int32(t)
+      example[name] = t
+
+    return example
+
+  def input_fn(params):
+    """The actual input function."""
+    batch_size = params["batch_size"]
+
+    # For training, we want a lot of parallel reading and shuffling.
+    # For eval, we want no shuffling and parallel reading doesn't matter.
+    d = tf.data.TFRecordDataset(input_file)
+    if is_training:
+      d = d.repeat()
+      d = d.shuffle(buffer_size=100)
+
+    d = d.apply(
+        tf.contrib.data.map_and_batch(
+            lambda record: _decode_record(record, name_to_features),
+            batch_size=batch_size,
+            drop_remainder=drop_remainder))
+
+    return d
+
+  return input_fn
+
+
+def _truncate_seq_pair(tokens_a, tokens_b, max_length):
+  """Truncates a sequence pair in place to the maximum length."""
+
+  # This is a simple heuristic which will always truncate the longer sequence
+  # one token at a time. This makes more sense than truncating an equal percent
+  # of tokens from each, since if one sequence is very short then each token
+  # that's truncated likely contains more information than a longer sequence.
+  while True:
+    total_length = len(tokens_a) + len(tokens_b)
+    if total_length <= max_length:
+      break
+    if len(tokens_a) > len(tokens_b):
+      tokens_a.pop()
+    else:
+      tokens_b.pop()
+
+
+def create_model(bert_config, is_training, input_ids, input_mask, segment_ids,
+                 labels, num_labels, use_one_hot_embeddings):
+  """Creates a classification model."""
+  model = modeling.BertModel(
+      config=bert_config,
+      is_training=is_training,
+      input_ids=input_ids,
+      input_mask=input_mask,
+      token_type_ids=segment_ids,
+      use_one_hot_embeddings=use_one_hot_embeddings)
+
+  # In the demo, we are doing a simple classification task on the entire
+  # segment.
+  #
+  # If you want to use the token-level output, use model.get_sequence_output()
+  # instead.
+  output_layer = model.get_pooled_output()
+
+  hidden_size = output_layer.shape[-1].value
+
+  output_weights = tf.get_variable(
+      "output_weights", [num_labels, hidden_size],
+      initializer=tf.truncated_normal_initializer(stddev=0.02))
+
+  output_bias = tf.get_variable(
+      "output_bias", [num_labels], initializer=tf.zeros_initializer())
+
+  with tf.variable_scope("loss"):
+    if is_training:
+      # I.e., 0.1 dropout
+      output_layer = tf.nn.dropout(output_layer, keep_prob=0.9)
+
+    logits = tf.matmul(output_layer, output_weights, transpose_b=True)
+    logits = tf.nn.bias_add(logits, output_bias)
+    probabilities = tf.nn.softmax(logits, axis=-1)
+    log_probs = tf.nn.log_softmax(logits, axis=-1)
+
+    one_hot_labels = tf.one_hot(labels, depth=num_labels, dtype=tf.float32)
+
+    per_example_loss = -tf.reduce_sum(one_hot_labels * log_probs, axis=-1)
+    loss = tf.reduce_mean(per_example_loss)
+
+    return (loss, per_example_loss, logits, probabilities)
+
+
+def model_fn_builder(bert_config, num_labels, init_checkpoint, learning_rate,
+                     num_train_steps, num_warmup_steps, use_tpu,
+                     use_one_hot_embeddings):
+  """Returns `model_fn` closure for TPUEstimator."""
+
+  def model_fn(features, labels, mode, params):  # pylint: disable=unused-argument
+    """The `model_fn` for TPUEstimator."""
+
+    tf.logging.info("*** Features ***")
+    for name in sorted(features.keys()):
+      tf.logging.info("  name = %s, shape = %s" % (name, features[name].shape))
+
+    input_ids = features["input_ids"]
+    input_mask = features["input_mask"]
+    segment_ids = features["segment_ids"]
+    label_ids = features["label_ids"]
+    is_real_example = None
+    if "is_real_example" in features:
+      is_real_example = tf.cast(features["is_real_example"], dtype=tf.float32)
+    else:
+      is_real_example = tf.ones(tf.shape(label_ids), dtype=tf.float32)
+
+    is_training = (mode == tf.estimator.ModeKeys.TRAIN)
+
+    (total_loss, per_example_loss, logits, probabilities) = create_model(
+        bert_config, is_training, input_ids, input_mask, segment_ids, label_ids,
+        num_labels, use_one_hot_embeddings)
+
+    tvars = tf.trainable_variables()
+    initialized_variable_names = {}
+    scaffold_fn = None
+    if init_checkpoint:
+      (assignment_map, initialized_variable_names
+      ) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
+      if use_tpu:
+
+        def tpu_scaffold():
+          tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
+          return tf.train.Scaffold()
+
+        scaffold_fn = tpu_scaffold
+      else:
+        tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
+
+    tf.logging.info("**** Trainable Variables ****")
+    for var in tvars:
+      init_string = ""
+      if var.name in initialized_variable_names:
+        init_string = ", *INIT_FROM_CKPT*"
+      tf.logging.info("  name = %s, shape = %s%s", var.name, var.shape,
+                      init_string)
+
+    output_spec = None
+    if mode == tf.estimator.ModeKeys.TRAIN:
+
+      train_op = optimization.create_optimizer(
+          total_loss, learning_rate, num_train_steps, num_warmup_steps, use_tpu)
+
+      output_spec = tf.contrib.tpu.TPUEstimatorSpec(
+          mode=mode,
+          loss=total_loss,
+          train_op=train_op,
+          scaffold_fn=scaffold_fn)
+    elif mode == tf.estimator.ModeKeys.EVAL:
+
+      def metric_fn(per_example_loss, label_ids, logits, is_real_example):
+        predictions = tf.argmax(logits, axis=-1, output_type=tf.int32)
+        accuracy = tf.metrics.accuracy(
+            labels=label_ids, predictions=predictions, weights=is_real_example)
+        loss = tf.metrics.mean(values=per_example_loss, weights=is_real_example)
+        return {
+            "eval_accuracy": accuracy,
+            "eval_loss": loss,
+        }
+
+      eval_metrics = (metric_fn,
+                      [per_example_loss, label_ids, logits, is_real_example])
+      output_spec = tf.contrib.tpu.TPUEstimatorSpec(
+          mode=mode,
+          loss=total_loss,
+          eval_metrics=eval_metrics,
+          scaffold_fn=scaffold_fn)
+    else:
+      output_spec = tf.contrib.tpu.TPUEstimatorSpec(
+          mode=mode,
+          predictions={"probabilities": probabilities},
+          scaffold_fn=scaffold_fn)
+    return output_spec
+
+  return model_fn
+
+
+# This function is not used by this file but is still used by the Colab and
+# people who depend on it.
+def input_fn_builder(features, seq_length, is_training, drop_remainder):
+  """Creates an `input_fn` closure to be passed to TPUEstimator."""
+
+  all_input_ids = []
+  all_input_mask = []
+  all_segment_ids = []
+  all_label_ids = []
+
+  for feature in features:
+    all_input_ids.append(feature.input_ids)
+    all_input_mask.append(feature.input_mask)
+    all_segment_ids.append(feature.segment_ids)
+    all_label_ids.append(feature.label_id)
+
+  def input_fn(params):
+    """The actual input function."""
+    batch_size = params["batch_size"]
+
+    num_examples = len(features)
+
+    # This is for demo purposes and does NOT scale to large data sets. We do
+    # not use Dataset.from_generator() because that uses tf.py_func which is
+    # not TPU compatible. The right way to load data is with TFRecordReader.
+    d = tf.data.Dataset.from_tensor_slices({
+        "input_ids":
+            tf.constant(
+                all_input_ids, shape=[num_examples, seq_length],
+                dtype=tf.int32),
+        "input_mask":
+            tf.constant(
+                all_input_mask,
+                shape=[num_examples, seq_length],
+                dtype=tf.int32),
+        "segment_ids":
+            tf.constant(
+                all_segment_ids,
+                shape=[num_examples, seq_length],
+                dtype=tf.int32),
+        "label_ids":
+            tf.constant(all_label_ids, shape=[num_examples], dtype=tf.int32),
+    })
+
+    if is_training:
+      d = d.repeat()
+      d = d.shuffle(buffer_size=100)
+
+    d = d.batch(batch_size=batch_size, drop_remainder=drop_remainder)
+    return d
+
+  return input_fn
+
+
+# This function is not used by this file but is still used by the Colab and
+# people who depend on it.
+def convert_examples_to_features(examples, label_list, max_seq_length,
+                                 tokenizer):
+  """Convert a set of `InputExample`s to a list of `InputFeatures`."""
+
+  features = []
+  for (ex_index, example) in enumerate(examples):
+    if ex_index % 10000 == 0:
+      tf.logging.info("Writing example %d of %d" % (ex_index, len(examples)))
+
+    feature = convert_single_example(ex_index, example, label_list,
+                                     max_seq_length, tokenizer)
+
+    features.append(feature)
+  return features
+
+
+def main(_):
+  tf.logging.set_verbosity(tf.logging.INFO)
+
+  processors = {
+      "cola": ColaProcessor,
+      "mnli": MnliProcessor,
+      "mrpc": MrpcProcessor,
+      "xnli": XnliProcessor,
+  }
+
+  tokenization.validate_case_matches_checkpoint(FLAGS.do_lower_case,
+                                                FLAGS.init_checkpoint)
+
+  if not FLAGS.do_train and not FLAGS.do_eval and not FLAGS.do_predict:
+    raise ValueError(
+        "At least one of `do_train`, `do_eval` or `do_predict' must be True.")
+
+  bert_config = modeling.BertConfig.from_json_file(FLAGS.bert_config_file)
+
+  if FLAGS.max_seq_length > bert_config.max_position_embeddings:
+    raise ValueError(
+        "Cannot use sequence length %d because the BERT model "
+        "was only trained up to sequence length %d" %
+        (FLAGS.max_seq_length, bert_config.max_position_embeddings))
+
+  tf.gfile.MakeDirs(FLAGS.output_dir)
+
+  task_name = FLAGS.task_name.lower()
+
+  if task_name not in processors:
+    raise ValueError("Task not found: %s" % (task_name))
+
+  processor = processors[task_name]()
+
+  label_list = processor.get_labels()
+
+  tokenizer = tokenization.FullTokenizer(
+      vocab_file=FLAGS.vocab_file, do_lower_case=FLAGS.do_lower_case)
+
+  tpu_cluster_resolver = None
+  if FLAGS.use_tpu and FLAGS.tpu_name:
+    tpu_cluster_resolver = tf.contrib.cluster_resolver.TPUClusterResolver(
+        FLAGS.tpu_name, zone=FLAGS.tpu_zone, project=FLAGS.gcp_project)
+
+  is_per_host = tf.contrib.tpu.InputPipelineConfig.PER_HOST_V2
+  run_config = tf.contrib.tpu.RunConfig(
+      cluster=tpu_cluster_resolver,
+      master=FLAGS.master,
+      model_dir=FLAGS.output_dir,
+      save_checkpoints_steps=FLAGS.save_checkpoints_steps,
+      tpu_config=tf.contrib.tpu.TPUConfig(
+          iterations_per_loop=FLAGS.iterations_per_loop,
+          num_shards=FLAGS.num_tpu_cores,
+          per_host_input_for_training=is_per_host))
+
+  train_examples = None
+  num_train_steps = None
+  num_warmup_steps = None
+  if FLAGS.do_train:
+    train_examples = processor.get_train_examples(FLAGS.data_dir)
+    num_train_steps = int(
+        len(train_examples) / FLAGS.train_batch_size * FLAGS.num_train_epochs)
+    num_warmup_steps = int(num_train_steps * FLAGS.warmup_proportion)
+
+  model_fn = model_fn_builder(
+      bert_config=bert_config,
+      num_labels=len(label_list),
+      init_checkpoint=FLAGS.init_checkpoint,
+      learning_rate=FLAGS.learning_rate,
+      num_train_steps=num_train_steps,
+      num_warmup_steps=num_warmup_steps,
+      use_tpu=FLAGS.use_tpu,
+      use_one_hot_embeddings=FLAGS.use_tpu)
+
+  # If TPU is not available, this will fall back to normal Estimator on CPU
+  # or GPU.
+  estimator = tf.contrib.tpu.TPUEstimator(
+      use_tpu=FLAGS.use_tpu,
+      model_fn=model_fn,
+      config=run_config,
+      train_batch_size=FLAGS.train_batch_size,
+      eval_batch_size=FLAGS.eval_batch_size,
+      predict_batch_size=FLAGS.predict_batch_size)
+
+  if FLAGS.do_train:
+    train_file = os.path.join(FLAGS.output_dir, "train.tf_record")
+    file_based_convert_examples_to_features(
+        train_examples, label_list, FLAGS.max_seq_length, tokenizer, train_file)
+    tf.logging.info("***** Running training *****")
+    tf.logging.info("  Num examples = %d", len(train_examples))
+    tf.logging.info("  Batch size = %d", FLAGS.train_batch_size)
+    tf.logging.info("  Num steps = %d", num_train_steps)
+    train_input_fn = file_based_input_fn_builder(
+        input_file=train_file,
+        seq_length=FLAGS.max_seq_length,
+        is_training=True,
+        drop_remainder=True)
+    estimator.train(input_fn=train_input_fn, max_steps=num_train_steps)
+
+  if FLAGS.do_eval:
+    eval_examples = processor.get_dev_examples(FLAGS.data_dir)
+    num_actual_eval_examples = len(eval_examples)
+    if FLAGS.use_tpu:
+      # TPU requires a fixed batch size for all batches, therefore the number
+      # of examples must be a multiple of the batch size, or else examples
+      # will get dropped. So we pad with fake examples which are ignored
+      # later on. These do NOT count towards the metric (all tf.metrics
+      # support a per-instance weight, and these get a weight of 0.0).
+      while len(eval_examples) % FLAGS.eval_batch_size != 0:
+        eval_examples.append(PaddingInputExample())
+
+    eval_file = os.path.join(FLAGS.output_dir, "eval.tf_record")
+    file_based_convert_examples_to_features(
+        eval_examples, label_list, FLAGS.max_seq_length, tokenizer, eval_file)
+
+    tf.logging.info("***** Running evaluation *****")
+    tf.logging.info("  Num examples = %d (%d actual, %d padding)",
+                    len(eval_examples), num_actual_eval_examples,
+                    len(eval_examples) - num_actual_eval_examples)
+    tf.logging.info("  Batch size = %d", FLAGS.eval_batch_size)
+
+    # This tells the estimator to run through the entire set.
+    eval_steps = None
+    # However, if running eval on the TPU, you will need to specify the
+    # number of steps.
+    if FLAGS.use_tpu:
+      assert len(eval_examples) % FLAGS.eval_batch_size == 0
+      eval_steps = int(len(eval_examples) // FLAGS.eval_batch_size)
+
+    eval_drop_remainder = True if FLAGS.use_tpu else False
+    eval_input_fn = file_based_input_fn_builder(
+        input_file=eval_file,
+        seq_length=FLAGS.max_seq_length,
+        is_training=False,
+        drop_remainder=eval_drop_remainder)
+
+    result = estimator.evaluate(input_fn=eval_input_fn, steps=eval_steps)
+
+    output_eval_file = os.path.join(FLAGS.output_dir, "eval_results.txt")
+    with tf.gfile.GFile(output_eval_file, "w") as writer:
+      tf.logging.info("***** Eval results *****")
+      for key in sorted(result.keys()):
+        tf.logging.info("  %s = %s", key, str(result[key]))
+        writer.write("%s = %s\n" % (key, str(result[key])))
+
+  if FLAGS.do_predict:
+    predict_examples = processor.get_test_examples(FLAGS.data_dir)
+    num_actual_predict_examples = len(predict_examples)
+    if FLAGS.use_tpu:
+      # TPU requires a fixed batch size for all batches, therefore the number
+      # of examples must be a multiple of the batch size, or else examples
+      # will get dropped. So we pad with fake examples which are ignored
+      # later on.
+      while len(predict_examples) % FLAGS.predict_batch_size != 0:
+        predict_examples.append(PaddingInputExample())
+
+    predict_file = os.path.join(FLAGS.output_dir, "predict.tf_record")
+    file_based_convert_examples_to_features(predict_examples, label_list,
+                                            FLAGS.max_seq_length, tokenizer,
+                                            predict_file)
+
+    tf.logging.info("***** Running prediction*****")
+    tf.logging.info("  Num examples = %d (%d actual, %d padding)",
+                    len(predict_examples), num_actual_predict_examples,
+                    len(predict_examples) - num_actual_predict_examples)
+    tf.logging.info("  Batch size = %d", FLAGS.predict_batch_size)
+
+    predict_drop_remainder = True if FLAGS.use_tpu else False
+    predict_input_fn = file_based_input_fn_builder(
+        input_file=predict_file,
+        seq_length=FLAGS.max_seq_length,
+        is_training=False,
+        drop_remainder=predict_drop_remainder)
+
+    result = estimator.predict(input_fn=predict_input_fn)
+
+    output_predict_file = os.path.join(FLAGS.output_dir, "test_results.tsv")
+    with tf.gfile.GFile(output_predict_file, "w") as writer:
+      num_written_lines = 0
+      tf.logging.info("***** Predict results *****")
+      for (i, prediction) in enumerate(result):
+        probabilities = prediction["probabilities"]
+        if i >= num_actual_predict_examples:
+          break
+        output_line = "\t".join(
+            str(class_probability)
+            for class_probability in probabilities) + "\n"
+        writer.write(output_line)
+        num_written_lines += 1
+    assert num_written_lines == num_actual_predict_examples
+
+
+if __name__ == "__main__":
+  flags.mark_flag_as_required("data_dir")
+  flags.mark_flag_as_required("task_name")
+  flags.mark_flag_as_required("vocab_file")
+  flags.mark_flag_as_required("bert_config_file")
+  flags.mark_flag_as_required("output_dir")
+  tf.app.run()

From 4361683c7b79ddf0e7115be73c78fca909269a70 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Tue, 26 Mar 2019 14:47:51 +0800
Subject: [PATCH 38/46] Add files via upload

---
 keras/3.5-classifying-movie-reviews.ipynb | 61 +++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 9c38c15..2e87dc0 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -31,6 +31,13 @@
     "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [this link would be added after merge]()."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -1653,6 +1660,60 @@
     "* As they get better on their training data, neural networks eventually start _overfitting_ and end up obtaining increasingly worse results on data \n",
     "never-seen-before. Make sure to always monitor performance on data that is outside of the training set.\n"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## \\* Memory saving\n",
+    "To run this notebook based on codes above, you need 32g `SPARK_DRIVER_MEMORY`, which is a bit expensive. Following is a viable memory saving approach which could save your `SPARK_DRIVER_MEMORY` to 12g.\n",
+    "\n",
+    "Taking a review of the time you have compiled the model, and prepared the `ndarray` type of datasets. And in old code above, the next step you would do is fit:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.fit(partial_x_train,\n",
+    "          partial_y_train,\n",
+    "          nb_epoch=20,\n",
+    "          batch_size=512,\n",
+    "          validation_data=(x_val, y_val))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Just hold on here! Before you call this `fit` method, use following code to do the training to save the memory:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from bigdl.util.common import to_sample_rdd\n",
+    "\n",
+    "train = to_sample_rdd(partial_x_train, partial_y_train)\n",
+    "val = to_sample_rdd(x_val, y_val)\n",
+    "\n",
+    "model.fit(train, None,\n",
+    "          nb_epoch=20,\n",
+    "          batch_size=512,\n",
+    "          validation_data=val)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This code zip the training data and label into RDD. The reason why it works is that every time when `fit` method takes `ndarray` as input, it transforms the `ndarray` to RDD and some memory is taken for cache in this process. And in this notebook, we use the same dataset as input repeatedly. If we call this operation only once and reuse the RDD afterwards, all the subsequential memory use would be saved."
+   ]
   }
  ],
  "metadata": {

From c257c264a101cf3e4f4d2303974d839dc2ef2494 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 13:26:49 +0800
Subject: [PATCH 39/46] Update 3.5-classifying-movie-reviews.ipynb

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 2e87dc0..1bbf7b5 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [this link would be added after merge]()."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](https://render.githubusercontent.com/view/ipynb?commit=453119be8480aa5e7b6c11071cf72510ccc8d7fe&enc_url=68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f696e74656c2d616e616c79746963732f7a6f6f2d7475746f7269616c732f343533313139626538343830616135653762366331313037316366373235313063636338643766652f6b657261732f332e352d636c617373696679696e672d6d6f7669652d726576696577732e6970796e623f746f6b656e3d414e675f3838585a706332444f457544334778614b38394e3631494c7a766a716b7335636d77635a7741253344253344&nwo=intel-analytics%2Fzoo-tutorials&path=keras%2F3.5-classifying-movie-reviews.ipynb&repository_id=172436179&repository_type=Repository#*-Memory-saving)."
    ]
   },
   {

From ab198a5895b87cb4bf92648cddcefc2a22b56a76 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 13:44:52 +0800
Subject: [PATCH 40/46] Update 3.5-classifying-movie-reviews.ipynb

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 1bbf7b5..85dd72e 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](https://render.githubusercontent.com/view/ipynb?commit=453119be8480aa5e7b6c11071cf72510ccc8d7fe&enc_url=68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f696e74656c2d616e616c79746963732f7a6f6f2d7475746f7269616c732f343533313139626538343830616135653762366331313037316366373235313063636338643766652f6b657261732f332e352d636c617373696679696e672d6d6f7669652d726576696577732e6970796e623f746f6b656e3d414e675f3838585a706332444f457544334778614b38394e3631494c7a766a716b7335636d77635a7741253344253344&nwo=intel-analytics%2Fzoo-tutorials&path=keras%2F3.5-classifying-movie-reviews.ipynb&repository_id=172436179&repository_type=Repository#*-Memory-saving)."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](*-memory-saving)."
    ]
   },
   {

From a6f5a5818f444f89872b67c25bcc0a66a2ac5e34 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 14:04:05 +0800
Subject: [PATCH 41/46] Update 3.5-classifying-movie-reviews.ipynb

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 85dd72e..0206619 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](*-memory-saving)."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](#*-memory-saving)."
    ]
   },
   {

From e007c136075745e8efb48cd4c78037e5ae40c497 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 14:05:35 +0800
Subject: [PATCH 42/46] Update 3.5-classifying-movie-reviews.ipynb

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 0206619..c44415d 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](#*-memory-saving)."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](##\*-memory-saving)."
    ]
   },
   {

From cb6371fdd9908a17be2f47611eb651bdc13217aa Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 14:09:30 +0800
Subject: [PATCH 43/46] Update 3.5-classifying-movie-reviews.ipynb

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index c44415d..f5c2251 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](##\*-memory-saving)."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](#\*-memory-saving)."
    ]
   },
   {

From 4f6c88490f97aca85be7db14f516e58f5b6279a4 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 15:28:00 +0800
Subject: [PATCH 44/46] Update 3.5-classifying-movie-reviews.ipynb

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index f5c2251..0c73aba 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here](#\*-memory-saving)."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [here]()."
    ]
   },
   {

From 24683102be42959443137f1ca06f7807d4e86abb Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Wed, 27 Mar 2019 15:37:08 +0800
Subject: [PATCH 45/46] Add files via upload

---
 keras/3.5-classifying-movie-reviews.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keras/3.5-classifying-movie-reviews.ipynb b/keras/3.5-classifying-movie-reviews.ipynb
index 2e87dc0..2a2d22c 100644
--- a/keras/3.5-classifying-movie-reviews.ipynb
+++ b/keras/3.5-classifying-movie-reviews.ipynb
@@ -35,7 +35,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach [this link would be added after merge]()."
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach at the end of this notebook."
    ]
   },
   {

From 07f0401a1b82fa6e5be3fa9d4ee182545babab83 Mon Sep 17 00:00:00 2001
From: Jiaming Song <litchilitchy@qq.com>
Date: Thu, 4 Apr 2019 14:55:45 +0800
Subject: [PATCH 46/46] Add files via upload

---
 keras/4.4-overfitting-and-underfitting.ipynb | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/keras/4.4-overfitting-and-underfitting.ipynb b/keras/4.4-overfitting-and-underfitting.ipynb
index c2a801b..3e9c188 100644
--- a/keras/4.4-overfitting-and-underfitting.ipynb
+++ b/keras/4.4-overfitting-and-underfitting.ipynb
@@ -31,6 +31,13 @@
     "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that you have to allocate 32g memory to `SPARK_DRIVER_MEMORY` if you are about to finish the contents in this notebook. Perhaps there is no such memory left on your machine, see memory saving approach at [Chapter 3.5](https://github.com/intel-analytics/zoo-tutorials/blob/master/keras/3.7-predicting-house-prices.ipynb)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},