Predict_Formation_Energy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction of Formation Energy of Solids Using Neural Network\n",
    "\n",
    "\n",
    "This work was reported by Jha *et al.*[1] and in this exercise, we will implement, train, and validate a model to predict the formation energy of given solid using its formula as the input.\n",
    "\n",
    "\n",
    "1. Jha, D., Ward, L., Paul, A., Liao, W.-K., Choudhary, A., Wolverton, C., & Agrawal, A. **(2018)**. ElemNet : Deep Learning the Chemistry of Materials From Only Elemental Composition. Scientific Reports, 8(1), 17593. http://doi.org/10.1038/s41598-018-35934-y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Runtime Install necessary packages\n",
    "\n",
    "Set runtime to GPU.\n",
    "Clone the repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: numpy in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (1.17.3)\n",
      "Requirement already satisfied: matplotlib in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (3.1.1)\n",
      "Requirement already satisfied: pandas in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (0.25.2)\n",
      "Requirement already satisfied: scikit-learn in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (0.21.3)\n",
      "Requirement already satisfied: tensorflow in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (2.0.0)\n",
      "Requirement already satisfied: cycler>=0.10 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from matplotlib) (0.10.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from matplotlib) (1.1.0)\n",
      "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from matplotlib) (2.4.4)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from matplotlib) (2.8.1)\n",
      "Requirement already satisfied: pytz>=2017.2 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from pandas) (2019.3)\n",
      "Requirement already satisfied: joblib>=0.11 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from scikit-learn) (0.14.0)\n",
      "Requirement already satisfied: scipy>=0.17.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from scikit-learn) (1.3.1)\n",
      "Requirement already satisfied: absl-py>=0.7.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (0.8.1)\n",
      "Requirement already satisfied: gast==0.2.2 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (0.2.2)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (1.1.0)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (3.9.2)\n",
      "Requirement already satisfied: keras-applications>=1.0.8 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (1.0.8)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (1.1.0)\n",
      "Requirement already satisfied: grpcio>=1.8.6 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (1.16.1)\n",
      "Requirement already satisfied: wheel>=0.26 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (0.33.6)\n",
      "Requirement already satisfied: six>=1.10.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (1.13.0)\n",
      "Requirement already satisfied: tensorflow-estimator<2.1.0,>=2.0.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (2.0.0)\n",
      "Requirement already satisfied: wrapt>=1.11.1 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (1.11.2)\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (3.1.0)\n",
      "Requirement already satisfied: google-pasta>=0.1.6 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (0.1.7)\n",
      "Requirement already satisfied: astor>=0.6.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (0.8.0)\n",
      "Requirement already satisfied: tensorboard<2.1.0,>=2.0.0 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorflow) (2.0.0)\n",
      "Requirement already satisfied: setuptools in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib) (41.6.0.post20191030)\n",
      "Requirement already satisfied: h5py in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from keras-applications>=1.0.8->tensorflow) (2.9.0)\n",
      "Requirement already satisfied: markdown>=2.6.8 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorboard<2.1.0,>=2.0.0->tensorflow) (3.1.1)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /home/bala/anaconda3/envs/tf20/lib/python3.6/site-packages (from tensorboard<2.1.0,>=2.0.0->tensorflow) (0.16.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install numpy matplotlib pandas scikit-learn tensorflow-gpu==2.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "parent_dir = 'ML_Workshop/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error, mean_absolute_error\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import re\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Descriptor\n",
    "\n",
    "The descriptor is the composition of the given system. The discriptor will be an array of length 86 (dataset contains only 86 elements)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>formula</th>\n",
       "      <th>energy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ge4Mg2O12Sn2</td>\n",
       "      <td>-2.167457</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cr6O16</td>\n",
       "      <td>-1.338311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AlBa2Fe3O8</td>\n",
       "      <td>-2.265546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mg2Mo3O8</td>\n",
       "      <td>-2.300022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BiF6Mg</td>\n",
       "      <td>-2.932162</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        formula    energy\n",
       "0  Ge4Mg2O12Sn2 -2.167457\n",
       "1        Cr6O16 -1.338311\n",
       "2    AlBa2Fe3O8 -2.265546\n",
       "3      Mg2Mo3O8 -2.300022\n",
       "4        BiF6Mg -2.932162"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(parent_dir + 'data/inorg/mp.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a3c683b70>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAARj0lEQVR4nO3df4xlZ13H8feHll9BoIXiWHc3bhMaTXHlRyZtDSaOVNulJSx/IEEr7GLN/lMSiGug4B9VkKTGQJWA6IY2LgiWBiTdAFrW0htjQqEtP1ragl1hsbspVNhSXYjo6Nc/5hm8U2Y6Mzt37r3b5/1KNnPOc8+993uenfnc557znHtTVUiS+vCESRcgSRofQ1+SOmLoS1JHDH1J6oihL0kdOX3SBTyWs846q7Zv3z7pMgD4/ve/z9Oe9rRJlzE17I+l7I+l7I+lxt0fd95553eq6jnL3TbVob99+3buuOOOSZcBwGAwYG5ubtJlTA37Yyn7Yyn7Y6lx90eSb650m4d3JKkjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI1N9Ra5GZ/tVn1y2/cg1l425EkmT5Ehfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI2sK/SRHktyd5EtJ7mhtz0pyKMn97eeZrT1J3p3kcJK7krxo6HF2t+3vT7J7c3ZJkrSS9Yz0f6WqXlBVs239KuCWqjoXuKWtA7wUOLf92wu8DxZeJICrgQuA84GrF18oJEnjsZHDO7uAA235APCKofYP1ILbgDOSnA1cAhyqquNV9TBwCNi5geeXJK3TWr9EpYBPJyngL6tqPzBTVQ+2278FzLTlLcADQ/c92tpWal8iyV4W3iEwMzPDYDBYY4mb68SJE1NTy8nYt2N+2faT3adTvT9Gzf5Yyv5Yapr6Y62h/0tVdSzJTwKHknx1+MaqqvaCsGHtBWU/wOzsbM3NzY3iYTdsMBgwLbWcjD0rfXPW5XMn9Xinen+Mmv2xlP2x1DT1x5oO71TVsfbzIeDjLByT/3Y7bEP7+VDb/BiwbejuW1vbSu2SpDFZNfSTPC3J0xeXgYuBrwAHgcUZOLuBm9ryQeC1bRbPhcAj7TDQzcDFSc5sJ3Avbm2SpDFZy+GdGeDjSRa3/3BV/X2S24Ebk1wBfBN4Vdv+U8ClwGHgB8DrAKrqeJK3A7e37d5WVcdHtieSpFWtGvpV9XXg+cu0fxe4aJn2Aq5c4bGuB65ff5mSpFHwilxJ6oihL0kdWeuUTelxb/tK01qvuWzMlUibx9DvnEEn9cXDO5LUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOeHGWurLSxWhSLxzpS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6subQT3Jaki8m+URbPyfJ55IcTvKRJE9q7U9u64fb7duHHuMtrf1rSS4Z9c5Ikh7bekb6bwDuG1r/Y+Daqnou8DBwRWu/Ani4tV/btiPJecCrgecBO4E/T3LaxsqXJK3Hmr4uMclW4DLgHcDvJgnwEuA32yYHgD8A3gfsassAHwXe07bfBdxQVT8EvpHkMHA+8NmR7IlGaqWvFTxyzWVjrkTSKK31O3L/FHgT8PS2/mzge1U139aPAlva8hbgAYCqmk/ySNt+C3Db0GMO3+dHkuwF9gLMzMwwGAzWui+b6sSJE1NTy8nYt2N+9Y3WYLEPTtX+OJl+WMt+nqr9sVnsj6WmqT9WDf0kLwMeqqo7k8xtdkFVtR/YDzA7O1tzc5v+lGsyGAyYllpOxp4RfSH4kcvngFO3P06mHxb3+bGcqv2xWeyPpaapP9Yy0n8x8PIklwJPAZ4B/BlwRpLT22h/K3CsbX8M2AYcTXI68Ezgu0Pti4bvI0kag1VP5FbVW6pqa1VtZ+FE7Geq6nLgVuCVbbPdwE1t+WBbp93+maqq1v7qNrvnHOBc4PMj2xNJ0qrWekx/OW8GbkjyR8AXgeta+3XAB9uJ2uMsvFBQVfckuRG4F5gHrqyq/9nA80uS1mldoV9VA2DQlr/OwuybR2/zn8Cvr3D/d7AwA0iSNAFekStJHTH0JakjGzmmrymz0gVVkrTIkb4kdcTQl6SOeHhHj0se6pKW50hfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BEvztK6LF70tG/H/JKvHvQL06VTgyN9SeqIoS9JHTH0JakjHtOXVrHSh7d5HkOnIkf6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xHn6GgnnskunBkf6ktQRQ1+SOrJq6Cd5SpLPJ/lyknuS/GFrPyfJ55IcTvKRJE9q7U9u64fb7duHHustrf1rSS7ZrJ2SJC1vLSP9HwIvqarnAy8Adia5EPhj4Nqqei7wMHBF2/4K4OHWfm3bjiTnAa8GngfsBP48yWmj3BlJ0mNbNfRrwYm2+sT2r4CXAB9t7QeAV7TlXW2ddvtFSdLab6iqH1bVN4DDwPkj2QtJ0pqsafZOG5HfCTwXeC/wL8D3qmq+bXIU2NKWtwAPAFTVfJJHgGe39tuGHnb4PsPPtRfYCzAzM8NgMFjfHm2SEydOTE0tK9m3Y371jUZk5qlre75J9dk4+mJ4306F349xsj+Wmqb+WFPoV9X/AC9IcgbwceDnNqugqtoP7AeYnZ2tubm5zXqqdRkMBkxLLSvZs8K0yc2wb8c877x79V+fI5fPbX4xyxhHXwzv26nw+zFO9sdS09Qf65q9U1XfA24FfhE4I8niX/1W4FhbPgZsA2i3PxP47nD7MveRJI3BqkO1JM8B/ruqvpfkqcCvsXBy9lbglcANwG7gpnaXg239s+32z1RVJTkIfDjJu4CfBs4FPj/i/dGU8aItabqs5fDO2cCBdlz/CcCNVfWJJPcCNyT5I+CLwHVt++uADyY5DBxnYcYOVXVPkhuBe4F54Mp22EjrtFKQStJqVg39qroLeOEy7V9nmdk3VfWfwK+v8FjvAN6x/jKl5fkCKK2PV+RKUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1ZE3fnCWNmp+zL02GI31J6oihL0kdMfQlqSMe059ifivU/7MvpNFwpC9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkecp6+p4nx8aXM50pekjhj6ktQRQ1+SOrJq6CfZluTWJPcmuSfJG1r7s5IcSnJ/+3lma0+Sdyc5nOSuJC8aeqzdbfv7k+zevN2SJC1nLSdy54F9VfWFJE8H7kxyCNgD3FJV1yS5CrgKeDPwUuDc9u8C4H3ABUmeBVwNzALVHudgVT086p2SxmH4pPO+HfPsaet+EYym2aoj/ap6sKq+0Jb/A7gP2ALsAg60zQ4Ar2jLu4AP1ILbgDOSnA1cAhyqquMt6A8BO0e6N5Kkx7SuKZtJtgMvBD4HzFTVg+2mbwEzbXkL8MDQ3Y62tpXaH/0ce4G9ADMzMwwGg/WUuGlOnDgx9lr27Zgf6/Otx8xTp7u+cRvuj2n5nZ2kSfy9TLNp6o81h36SnwA+Bryxqv49yY9uq6pKUqMoqKr2A/sBZmdna25ubhQPu2GDwYBx17Jniues79sxzzvv9jKPRcP9ceTyuckWMwUm8fcyzaapP9Y0eyfJE1kI/A9V1d+25m+3wza0nw+19mPAtqG7b21tK7VLksZkLbN3AlwH3FdV7xq66SCwOANnN3DTUPtr2yyeC4FH2mGgm4GLk5zZZvpc3NokSWOylvfnLwZeA9yd5Eut7a3ANcCNSa4Avgm8qt32KeBS4DDwA+B1AFV1PMnbgdvbdm+rquMj2QtJ0pqsGvpV9U9AVrj5omW2L+DKFR7reuD69RQoSRodr8iVpI44/WIK+MmSksbFkb4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdeT0SRfQk+1XfXLSJUjqnCN9SerIqqGf5PokDyX5ylDbs5IcSnJ/+3lma0+Sdyc5nOSuJC8aus/utv39SXZvzu5Ikh7LWkb6fwXsfFTbVcAtVXUucEtbB3gpcG77txd4Hyy8SABXAxcA5wNXL75QSJLGZ9XQr6p/BI4/qnkXcKAtHwBeMdT+gVpwG3BGkrOBS4BDVXW8qh4GDvHjLySSpE12sidyZ6rqwbb8LWCmLW8BHhja7mhrW6n9xyTZy8K7BGZmZhgMBidZ4midOHFiw7Xs2zE/mmKmwMxTH1/7s1HD/TEtv7OTNIq/l8eTaeqPDc/eqapKUqMopj3efmA/wOzsbM3NzY3qoTdkMBiw0Vr2PI5m7+zbMc8773by16Lh/jhy+dxki5kCo/h7eTyZpv442dk7326HbWg/H2rtx4BtQ9ttbW0rtUuSxuhkQ/8gsDgDZzdw01D7a9ssnguBR9phoJuBi5Oc2U7gXtzaJEljtOr78yR/A8wBZyU5ysIsnGuAG5NcAXwTeFXb/FPApcBh4AfA6wCq6niStwO3t+3eVlWPPjksSdpkq4Z+Vf3GCjddtMy2BVy5wuNcD1y/ruokSSPlFbmS1BFDX5I6YuhLUkcMfUnqiKEvSR3xkspN4Ofm922l//8j11w25kqkH+dIX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0JekjjhPfwOcjy/pVONIX5I6YuhLUkcMfUnqiKEvSR3xRO4abL/qk+zbMc8eT9xKOsU50pekjjjSl8bEj1zWNHCkL0kdMfQlqSMe3pEmzMM+GidH+pLUEUf60pTyHYA2gyN9SeqII/0hfmqmTgW+A9BGdBn6hrukXo099JPsBP4MOA14f1VdM+4apMcj3wFoLcYa+klOA94L/BpwFLg9ycGqunecdUg9OZl3tr5QPH6Ne6R/PnC4qr4OkOQGYBewKaHvYRzp5Gz0b2e1Dyj0RWVyUlXje7LklcDOqvqdtv4a4IKqev3QNnuBvW31Z4Gvja3Ax3YW8J1JFzFF7I+l7I+l7I+lxt0fP1NVz1nuhqk7kVtV+4H9k67j0ZLcUVWzk65jWtgfS9kfS9kfS01Tf4x7nv4xYNvQ+tbWJkkag3GH/u3AuUnOSfIk4NXAwTHXIEndGuvhnaqaT/J64GYWpmxeX1X3jLOGDZi6Q04TZn8sZX8sZX8sNTX9MdYTuZKkyfKzdySpI4a+JHXE0D8JSfYlqSRnTbqWSUryJ0m+muSuJB9Pcsakaxq3JDuTfC3J4SRXTbqeSUqyLcmtSe5Nck+SN0y6pmmQ5LQkX0zyiUnXAob+uiXZBlwM/Ouka5kCh4Cfr6pfAP4ZeMuE6xmroY8VeSlwHvAbSc6bbFUTNQ/sq6rzgAuBKzvvj0VvAO6bdBGLDP31uxZ4E9D9GfCq+nRVzbfV21i47qInP/pYkar6L2DxY0W6VFUPVtUX2vJ/sBB0WyZb1WQl2QpcBrx/0rUsMvTXIcku4FhVfXnStUyh3wb+btJFjNkW4IGh9aN0HnKLkmwHXgh8brKVTNyfsjBI/N9JF7Jo6j6GYdKS/APwU8vc9PvAW1k4tNONx+qPqrqpbfP7LLy1/9A4a9N0SvITwMeAN1bVv0+6nklJ8jLgoaq6M8ncpOtZZOg/SlX96nLtSXYA5wBfTgILhzK+kOT8qvrWGEscq5X6Y1GSPcDLgIuqv4s+/FiRR0nyRBYC/0NV9beTrmfCXgy8PMmlwFOAZyT566r6rUkW5cVZJynJEWC2qrr9JMH2hTjvAn65qv5t0vWMW5LTWTiBfRELYX878Jun0FXmI5WF0dAB4HhVvXHS9UyTNtL/vap62aRr8Zi+NuI9wNOBQ0m+lOQvJl3QOLWT2IsfK3IfcGOvgd+8GHgN8JL2+/ClNsrVFHGkL0kdcaQvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JH/g8wgyveX4YgVAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visulaize the energy histogram\n",
    "df['energy'].hist(bins=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Descriptor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Ge': 4, 'Mg': 2, 'O': 12, 'Sn': 2}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def parse_formula(formula):\n",
    "    \n",
    "    split = re.findall('[A-Z][^A-Z]*', formula)\n",
    "    result = {}\n",
    "    for el in split:\n",
    "        head = el.rstrip('0123456789')\n",
    "        tail = el[len(head):]\n",
    "        result[head] = int(tail) if tail != '' else 1\n",
    "    \n",
    "    return result\n",
    "    \n",
    "parse_formula('Ge4Mg2O12Sn2')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# build template for descriptor\n",
    "elements = ['H', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Na', 'Mg', 'Al', 'Si', 'P', 'S', 'Cl', 'K', 'Ca', 'Sc', 'Ti', 'V', \n",
    "            'Cr', 'Mn', 'Fe', 'Co', 'Ni', 'Cu', 'Zn', 'Ga', 'Ge', 'As', 'Se', 'Br', 'Kr', 'Rb', 'Sr', 'Y', 'Zr', 'Nb', \n",
    "            'Mo', 'Tc', 'Ru', 'Rh', 'Pd', 'Ag', 'Cd', 'In', 'Sn', 'Sb', 'Te', 'I', 'Xe', 'Cs', 'Ba', 'La', 'Ce', 'Pr', \n",
    "            'Nd', 'Pm', 'Sm', 'Eu', 'Gd', 'Tb', 'Dy', 'Ho', 'Er', 'Tm', 'Yb', 'Lu', 'Hf', 'Ta', 'W', 'Re', 'Os', 'Ir', \n",
    "            'Pt', 'Au', 'Hg', 'Tl', 'Pb', 'Bi', 'Ac', 'Th', 'Pa', 'U', 'Np', 'Pu']\n",
    "desc_positions = dict(zip(elements, range(len(elements))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'H': 0, 'Li': 1, 'Be': 2, 'B': 3, 'C': 4, 'N': 5, 'O': 6, 'F': 7, 'Na': 8, 'Mg': 9, 'Al': 10, 'Si': 11, 'P': 12, 'S': 13, 'Cl': 14, 'K': 15, 'Ca': 16, 'Sc': 17, 'Ti': 18, 'V': 19, 'Cr': 20, 'Mn': 21, 'Fe': 22, 'Co': 23, 'Ni': 24, 'Cu': 25, 'Zn': 26, 'Ga': 27, 'Ge': 28, 'As': 29, 'Se': 30, 'Br': 31, 'Kr': 32, 'Rb': 33, 'Sr': 34, 'Y': 35, 'Zr': 36, 'Nb': 37, 'Mo': 38, 'Tc': 39, 'Ru': 40, 'Rh': 41, 'Pd': 42, 'Ag': 43, 'Cd': 44, 'In': 45, 'Sn': 46, 'Sb': 47, 'Te': 48, 'I': 49, 'Xe': 50, 'Cs': 51, 'Ba': 52, 'La': 53, 'Ce': 54, 'Pr': 55, 'Nd': 56, 'Pm': 57, 'Sm': 58, 'Eu': 59, 'Gd': 60, 'Tb': 61, 'Dy': 62, 'Ho': 63, 'Er': 64, 'Tm': 65, 'Yb': 66, 'Lu': 67, 'Hf': 68, 'Ta': 69, 'W': 70, 'Re': 71, 'Os': 72, 'Ir': 73, 'Pt': 74, 'Au': 75, 'Hg': 76, 'Tl': 77, 'Pb': 78, 'Bi': 79, 'Ac': 80, 'Th': 81, 'Pa': 82, 'U': 83, 'Np': 84, 'Pu': 85}\n"
     ]
    }
   ],
   "source": [
    "print(desc_positions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_descriptor(formula):\n",
    "    \n",
    "    elem_counts = parse_formula(formula)\n",
    "    desc = np.zeros(shape=86)\n",
    "    for k, v in elem_counts.items():\n",
    "        desc[desc_positions.get(k)] = v\n",
    "        \n",
    "    return desc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.  0.  0.  0.  0.  0. 12.  0.  0.  2.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  4.  0.  0.  0.  0.  0.  0.  0.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  2.  0.  0.  0.  0.  0.  0.  0.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]\n"
     ]
    }
   ],
   "source": [
    "print(get_descriptor('Ge4Mg2O12Sn2'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(67847, 86)\n",
      "(67847,)\n"
     ]
    }
   ],
   "source": [
    "# get datasets\n",
    "x = []\n",
    "y = []\n",
    "\n",
    "for row in df.itertuples():\n",
    "    x.append(get_descriptor(row.formula))\n",
    "    y.append(float(row.energy))\n",
    "\n",
    "x = np.stack(x)\n",
    "y = np.array(y)\n",
    "\n",
    "print(x.shape)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(54277, 86)\n",
      "(13570, 86)\n"
     ]
    }
   ],
   "source": [
    "# train test split\n",
    "X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=35)\n",
    "\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model\n",
    "\n",
    "We will start with a model with six hidden layers with [1024, 512, 256, 128, 64, 32] neurons. We will use relu activations, ADAM optimizer and mean squared error for loss function.\n",
    "\n",
    "Architecture used in the paper is 1024x4-512x3-256x3-128x3-64x2-32x1-1. Dropout [0.2, 0.1, 0.3, 0.2]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential([\n",
    "    Dense(1024, input_shape=(86,), activation='relu'),  # x 4\n",
    "    Dropout(0.2),\n",
    "    Dense(512, activation='relu'),  # x 3\n",
    "    Dropout(0.1),\n",
    "    Dense(256, activation='relu'),  # x 3\n",
    "    Dropout(0.3),\n",
    "    Dense(128, activation='relu'),  # x 3\n",
    "    Dropout(0.2),\n",
    "    Dense(64, activation='relu'),\n",
    "    Dense(32, activation='relu'),\n",
    "    Dense(1, activation='linear'),\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 1024)              89088     \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 512)               524800    \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 256)               131328    \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 64)                8256      \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 32)                2080      \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 788,481\n",
      "Trainable params: 788,481\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss='mse',\n",
    "              metrics=['mae'])\n",
    "\n",
    "# print summary \n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 3686 samples, validate on 410 samples\n",
      "Epoch 1/100\n",
      "3686/3686 - 4s - loss: 1.5726 - mae: 0.9398 - val_loss: 1.0557 - val_mae: 0.8066\n",
      "Epoch 2/100\n",
      "3686/3686 - 2s - loss: 0.7300 - mae: 0.6370 - val_loss: 0.4359 - val_mae: 0.4901\n",
      "Epoch 3/100\n",
      "3686/3686 - 2s - loss: 0.3181 - mae: 0.4176 - val_loss: 0.2184 - val_mae: 0.3181\n",
      "Epoch 4/100\n",
      "3686/3686 - 2s - loss: 0.2119 - mae: 0.3369 - val_loss: 0.1928 - val_mae: 0.3107\n",
      "Epoch 5/100\n",
      "3686/3686 - 2s - loss: 0.1800 - mae: 0.3102 - val_loss: 0.2620 - val_mae: 0.3853\n",
      "Epoch 6/100\n",
      "3686/3686 - 2s - loss: 0.1683 - mae: 0.3015 - val_loss: 0.1663 - val_mae: 0.3041\n",
      "Epoch 7/100\n",
      "3686/3686 - 2s - loss: 0.1440 - mae: 0.2776 - val_loss: 0.2040 - val_mae: 0.3591\n",
      "Epoch 8/100\n",
      "3686/3686 - 2s - loss: 0.1216 - mae: 0.2499 - val_loss: 0.1408 - val_mae: 0.2806\n",
      "Epoch 9/100\n",
      "3686/3686 - 2s - loss: 0.1095 - mae: 0.2398 - val_loss: 0.1337 - val_mae: 0.2560\n",
      "Epoch 10/100\n",
      "3686/3686 - 2s - loss: 0.1028 - mae: 0.2294 - val_loss: 0.1182 - val_mae: 0.2499\n",
      "Epoch 11/100\n",
      "3686/3686 - 2s - loss: 0.1032 - mae: 0.2306 - val_loss: 0.1403 - val_mae: 0.2800\n",
      "Epoch 12/100\n",
      "3686/3686 - 2s - loss: 0.0912 - mae: 0.2163 - val_loss: 0.1287 - val_mae: 0.2660\n",
      "Epoch 13/100\n",
      "3686/3686 - 2s - loss: 0.0823 - mae: 0.2084 - val_loss: 0.1010 - val_mae: 0.2247\n",
      "Epoch 14/100\n",
      "3686/3686 - 2s - loss: 0.0778 - mae: 0.2032 - val_loss: 0.0943 - val_mae: 0.2020\n",
      "Epoch 15/100\n",
      "3686/3686 - 2s - loss: 0.0770 - mae: 0.2018 - val_loss: 0.1103 - val_mae: 0.2376\n",
      "Epoch 16/100\n",
      "3686/3686 - 2s - loss: 0.0696 - mae: 0.1913 - val_loss: 0.0887 - val_mae: 0.1884\n",
      "Epoch 17/100\n",
      "3686/3686 - 2s - loss: 0.0665 - mae: 0.1828 - val_loss: 0.0871 - val_mae: 0.2032\n",
      "Epoch 18/100\n",
      "3686/3686 - 2s - loss: 0.0631 - mae: 0.1818 - val_loss: 0.0722 - val_mae: 0.1768\n",
      "Epoch 19/100\n",
      "3686/3686 - 2s - loss: 0.0579 - mae: 0.1753 - val_loss: 0.0763 - val_mae: 0.1855\n",
      "Epoch 20/100\n",
      "3686/3686 - 2s - loss: 0.0640 - mae: 0.1843 - val_loss: 0.0906 - val_mae: 0.2135\n",
      "Epoch 21/100\n",
      "3686/3686 - 2s - loss: 0.0528 - mae: 0.1655 - val_loss: 0.0759 - val_mae: 0.1838\n",
      "Epoch 22/100\n",
      "3686/3686 - 2s - loss: 0.0531 - mae: 0.1669 - val_loss: 0.0931 - val_mae: 0.2018\n",
      "Epoch 23/100\n",
      "3686/3686 - 2s - loss: 0.0466 - mae: 0.1583 - val_loss: 0.0768 - val_mae: 0.1833\n",
      "Epoch 24/100\n",
      "3686/3686 - 2s - loss: 0.0480 - mae: 0.1574 - val_loss: 0.0872 - val_mae: 0.2110\n",
      "Epoch 25/100\n",
      "3686/3686 - 2s - loss: 0.0452 - mae: 0.1554 - val_loss: 0.0901 - val_mae: 0.1984\n",
      "Epoch 26/100\n",
      "3686/3686 - 2s - loss: 0.0428 - mae: 0.1515 - val_loss: 0.0919 - val_mae: 0.2048\n",
      "Epoch 27/100\n",
      "3686/3686 - 2s - loss: 0.0419 - mae: 0.1509 - val_loss: 0.0882 - val_mae: 0.2037\n",
      "Epoch 28/100\n",
      "3686/3686 - 2s - loss: 0.0422 - mae: 0.1505 - val_loss: 0.0794 - val_mae: 0.1861\n",
      "Epoch 29/100\n",
      "3686/3686 - 2s - loss: 0.0378 - mae: 0.1429 - val_loss: 0.0841 - val_mae: 0.1904\n",
      "Epoch 30/100\n",
      "3686/3686 - 2s - loss: 0.0441 - mae: 0.1507 - val_loss: 0.0883 - val_mae: 0.1972\n",
      "Epoch 31/100\n",
      "3686/3686 - 2s - loss: 0.0440 - mae: 0.1514 - val_loss: 0.0899 - val_mae: 0.1908\n",
      "Epoch 32/100\n",
      "3686/3686 - 2s - loss: 0.0435 - mae: 0.1536 - val_loss: 0.0807 - val_mae: 0.1729\n",
      "Epoch 33/100\n",
      "3686/3686 - 2s - loss: 0.0362 - mae: 0.1401 - val_loss: 0.0766 - val_mae: 0.1833\n",
      "Epoch 34/100\n",
      "3686/3686 - 2s - loss: 0.0397 - mae: 0.1446 - val_loss: 0.0791 - val_mae: 0.1819\n",
      "Epoch 35/100\n",
      "3686/3686 - 2s - loss: 0.0366 - mae: 0.1402 - val_loss: 0.0800 - val_mae: 0.1780\n",
      "Epoch 36/100\n",
      "3686/3686 - 2s - loss: 0.0343 - mae: 0.1361 - val_loss: 0.1152 - val_mae: 0.2305\n",
      "Epoch 37/100\n",
      "3686/3686 - 2s - loss: 0.0393 - mae: 0.1447 - val_loss: 0.0789 - val_mae: 0.1722\n",
      "Epoch 38/100\n",
      "3686/3686 - 2s - loss: 0.0334 - mae: 0.1345 - val_loss: 0.0798 - val_mae: 0.1790\n",
      "Epoch 39/100\n",
      "3686/3686 - 2s - loss: 0.0307 - mae: 0.1304 - val_loss: 0.0710 - val_mae: 0.1614\n",
      "Epoch 40/100\n",
      "3686/3686 - 2s - loss: 0.0307 - mae: 0.1300 - val_loss: 0.0750 - val_mae: 0.1709\n",
      "Epoch 41/100\n",
      "3686/3686 - 2s - loss: 0.0293 - mae: 0.1266 - val_loss: 0.0808 - val_mae: 0.1814\n",
      "Epoch 42/100\n",
      "3686/3686 - 2s - loss: 0.0287 - mae: 0.1224 - val_loss: 0.0805 - val_mae: 0.1845\n",
      "Epoch 43/100\n",
      "3686/3686 - 2s - loss: 0.0306 - mae: 0.1291 - val_loss: 0.0741 - val_mae: 0.1704\n",
      "Epoch 44/100\n",
      "3686/3686 - 2s - loss: 0.0293 - mae: 0.1265 - val_loss: 0.0695 - val_mae: 0.1623\n",
      "Epoch 45/100\n",
      "3686/3686 - 2s - loss: 0.0313 - mae: 0.1313 - val_loss: 0.0819 - val_mae: 0.1859\n",
      "Epoch 46/100\n",
      "3686/3686 - 2s - loss: 0.0299 - mae: 0.1272 - val_loss: 0.0770 - val_mae: 0.1773\n",
      "Epoch 47/100\n",
      "3686/3686 - 2s - loss: 0.0325 - mae: 0.1305 - val_loss: 0.0789 - val_mae: 0.1765\n",
      "Epoch 48/100\n",
      "3686/3686 - 2s - loss: 0.0303 - mae: 0.1275 - val_loss: 0.0735 - val_mae: 0.1683\n",
      "Epoch 49/100\n",
      "3686/3686 - 2s - loss: 0.0264 - mae: 0.1195 - val_loss: 0.0815 - val_mae: 0.1807\n",
      "Epoch 50/100\n",
      "3686/3686 - 2s - loss: 0.0284 - mae: 0.1243 - val_loss: 0.0767 - val_mae: 0.1749\n",
      "Epoch 51/100\n",
      "3686/3686 - 2s - loss: 0.0258 - mae: 0.1195 - val_loss: 0.0774 - val_mae: 0.1668\n",
      "Epoch 52/100\n",
      "3686/3686 - 2s - loss: 0.0287 - mae: 0.1242 - val_loss: 0.0761 - val_mae: 0.1781\n",
      "Epoch 53/100\n",
      "3686/3686 - 2s - loss: 0.0279 - mae: 0.1231 - val_loss: 0.0776 - val_mae: 0.1713\n",
      "Epoch 54/100\n",
      "3686/3686 - 2s - loss: 0.0320 - mae: 0.1292 - val_loss: 0.0877 - val_mae: 0.1886\n",
      "Epoch 55/100\n",
      "3686/3686 - 3s - loss: 0.0295 - mae: 0.1239 - val_loss: 0.0752 - val_mae: 0.1713\n",
      "Epoch 56/100\n",
      "3686/3686 - 2s - loss: 0.0262 - mae: 0.1180 - val_loss: 0.0765 - val_mae: 0.1610\n",
      "Epoch 57/100\n",
      "3686/3686 - 2s - loss: 0.0274 - mae: 0.1198 - val_loss: 0.0758 - val_mae: 0.1660\n",
      "Epoch 58/100\n",
      "3686/3686 - 2s - loss: 0.0270 - mae: 0.1199 - val_loss: 0.0896 - val_mae: 0.2039\n",
      "Epoch 59/100\n",
      "3686/3686 - 2s - loss: 0.0271 - mae: 0.1202 - val_loss: 0.0844 - val_mae: 0.1956\n",
      "Epoch 60/100\n",
      "3686/3686 - 3s - loss: 0.0238 - mae: 0.1146 - val_loss: 0.0819 - val_mae: 0.1731\n",
      "Epoch 61/100\n",
      "3686/3686 - 3s - loss: 0.0243 - mae: 0.1127 - val_loss: 0.0787 - val_mae: 0.1762\n",
      "Epoch 62/100\n",
      "3686/3686 - 2s - loss: 0.0245 - mae: 0.1141 - val_loss: 0.0727 - val_mae: 0.1674\n",
      "Epoch 63/100\n",
      "3686/3686 - 3s - loss: 0.0241 - mae: 0.1139 - val_loss: 0.0754 - val_mae: 0.1788\n",
      "Epoch 64/100\n",
      "3686/3686 - 2s - loss: 0.0236 - mae: 0.1115 - val_loss: 0.0755 - val_mae: 0.1643\n",
      "Epoch 65/100\n",
      "3686/3686 - 2s - loss: 0.0235 - mae: 0.1112 - val_loss: 0.0724 - val_mae: 0.1639\n",
      "Epoch 66/100\n",
      "3686/3686 - 2s - loss: 0.0214 - mae: 0.1078 - val_loss: 0.0734 - val_mae: 0.1606\n",
      "Epoch 67/100\n",
      "3686/3686 - 2s - loss: 0.0231 - mae: 0.1089 - val_loss: 0.0773 - val_mae: 0.1816\n",
      "Epoch 68/100\n",
      "3686/3686 - 2s - loss: 0.0240 - mae: 0.1142 - val_loss: 0.0756 - val_mae: 0.1718\n",
      "Epoch 69/100\n",
      "3686/3686 - 3s - loss: 0.0242 - mae: 0.1121 - val_loss: 0.0782 - val_mae: 0.1810\n",
      "Epoch 70/100\n",
      "3686/3686 - 3s - loss: 0.0272 - mae: 0.1199 - val_loss: 0.0704 - val_mae: 0.1630\n",
      "Epoch 71/100\n",
      "3686/3686 - 2s - loss: 0.0229 - mae: 0.1106 - val_loss: 0.0730 - val_mae: 0.1660\n",
      "Epoch 72/100\n",
      "3686/3686 - 2s - loss: 0.0229 - mae: 0.1078 - val_loss: 0.0640 - val_mae: 0.1549\n",
      "Epoch 73/100\n",
      "3686/3686 - 2s - loss: 0.0200 - mae: 0.1044 - val_loss: 0.0680 - val_mae: 0.1611\n",
      "Epoch 74/100\n",
      "3686/3686 - 2s - loss: 0.0221 - mae: 0.1075 - val_loss: 0.0750 - val_mae: 0.1665\n",
      "Epoch 75/100\n",
      "3686/3686 - 2s - loss: 0.0203 - mae: 0.1051 - val_loss: 0.0676 - val_mae: 0.1566\n",
      "Epoch 76/100\n",
      "3686/3686 - 2s - loss: 0.0186 - mae: 0.1021 - val_loss: 0.0742 - val_mae: 0.1624\n",
      "Epoch 77/100\n",
      "3686/3686 - 2s - loss: 0.0188 - mae: 0.0997 - val_loss: 0.0698 - val_mae: 0.1567\n",
      "Epoch 78/100\n",
      "3686/3686 - 3s - loss: 0.0193 - mae: 0.1026 - val_loss: 0.0735 - val_mae: 0.1590\n",
      "Epoch 79/100\n",
      "3686/3686 - 2s - loss: 0.0180 - mae: 0.0990 - val_loss: 0.0762 - val_mae: 0.1654\n",
      "Epoch 80/100\n",
      "3686/3686 - 3s - loss: 0.0188 - mae: 0.0996 - val_loss: 0.0662 - val_mae: 0.1535\n",
      "Epoch 81/100\n",
      "3686/3686 - 3s - loss: 0.0178 - mae: 0.0981 - val_loss: 0.0693 - val_mae: 0.1686\n",
      "Epoch 82/100\n",
      "3686/3686 - 2s - loss: 0.0189 - mae: 0.1002 - val_loss: 0.0731 - val_mae: 0.1651\n",
      "Epoch 83/100\n",
      "3686/3686 - 2s - loss: 0.0178 - mae: 0.0977 - val_loss: 0.0683 - val_mae: 0.1575\n",
      "Epoch 84/100\n",
      "3686/3686 - 2s - loss: 0.0180 - mae: 0.0984 - val_loss: 0.0669 - val_mae: 0.1568\n",
      "Epoch 85/100\n",
      "3686/3686 - 2s - loss: 0.0164 - mae: 0.0938 - val_loss: 0.0679 - val_mae: 0.1536\n",
      "Epoch 86/100\n",
      "3686/3686 - 2s - loss: 0.0175 - mae: 0.0972 - val_loss: 0.0783 - val_mae: 0.1760\n",
      "Epoch 87/100\n",
      "3686/3686 - 2s - loss: 0.0203 - mae: 0.1019 - val_loss: 0.0713 - val_mae: 0.1611\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 88/100\n",
      "3686/3686 - 2s - loss: 0.0186 - mae: 0.0980 - val_loss: 0.0780 - val_mae: 0.1692\n",
      "Epoch 89/100\n",
      "3686/3686 - 2s - loss: 0.0215 - mae: 0.1019 - val_loss: 0.0775 - val_mae: 0.1716\n",
      "Epoch 90/100\n",
      "3686/3686 - 2s - loss: 0.0204 - mae: 0.1033 - val_loss: 0.0746 - val_mae: 0.1665\n",
      "Epoch 91/100\n",
      "3686/3686 - 2s - loss: 0.0179 - mae: 0.0966 - val_loss: 0.0746 - val_mae: 0.1659\n",
      "Epoch 92/100\n",
      "3686/3686 - 2s - loss: 0.0230 - mae: 0.1053 - val_loss: 0.0746 - val_mae: 0.1659\n",
      "Epoch 93/100\n",
      "3686/3686 - 2s - loss: 0.0212 - mae: 0.1044 - val_loss: 0.0796 - val_mae: 0.1679\n",
      "Epoch 94/100\n",
      "3686/3686 - 2s - loss: 0.0186 - mae: 0.0978 - val_loss: 0.0755 - val_mae: 0.1645\n",
      "Epoch 95/100\n",
      "3686/3686 - 3s - loss: 0.0178 - mae: 0.0954 - val_loss: 0.0731 - val_mae: 0.1642\n",
      "Epoch 96/100\n",
      "3686/3686 - 2s - loss: 0.0163 - mae: 0.0904 - val_loss: 0.0697 - val_mae: 0.1553\n",
      "Epoch 97/100\n",
      "3686/3686 - 2s - loss: 0.0157 - mae: 0.0907 - val_loss: 0.0748 - val_mae: 0.1663\n",
      "Epoch 98/100\n",
      "3686/3686 - 3s - loss: 0.0154 - mae: 0.0908 - val_loss: 0.0672 - val_mae: 0.1533\n",
      "Epoch 99/100\n",
      "3686/3686 - 2s - loss: 0.0139 - mae: 0.0868 - val_loss: 0.0691 - val_mae: 0.1558\n",
      "Epoch 100/100\n",
      "3686/3686 - 3s - loss: 0.0145 - mae: 0.0885 - val_loss: 0.0737 - val_mae: 0.1633\n"
     ]
    }
   ],
   "source": [
    "epochs = 100\n",
    "history = model.fit(x=X_train[:4096], y=y_train[:4096], batch_size=32, validation_split=0.1, verbose=2, epochs=epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visulaizing the training curves"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeUAAAHiCAYAAADWNdTaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdeZxU1Zn/8c9zq3oButkXZROiooAsYosaN4iJonGNRiEu0TE6k20yifEXkknUcczEZDLGLEZjEnVMFDTuE1GjhsS4i6goooKArEIDsjXQ6/n9cW51Vzdd1QVdUvcW3/fr1a/uvnW76jR6+qnnnOecY845REREpPCCQjdAREREPAVlERGRiFBQFhERiQgFZRERkYhQUBYREYkIBWUREZGIUFCOOTNLmNlWMxuaz3tFZM9SXxZQUN7jwo6U+mgys+1p35+/q8/nnGt0zlU455bl895dZWbXmZkzs6+2uX5FeP37ba4fEF7/ZZvryfB6TZt/q2/lu80inVHkffmOfD+v5EZBeQ8LO1KFc64CWAaclnbtrrb3m1lyz7dyt70HXNTm2kXh9ba+CGwApppZSTuPj07/t3LO3ZDntop0SpH3ZSkQBeWICd+l3mNmM8xsC3CBmR1lZi+a2UYzW21mv0gFsrTMclj4/R/Dxx8zsy1m9oKZDd/Ve8PHTzaz98xsk5n90syeM7OLszT/BaC3mR0U/vx4/P9jr7X5HQ24EPguYMBnO/8vJxItMe/LmX6n0Wb297D9b5rZZ9MeO9XMFoSvv8LMvhle729ms8Kf2WBmz+zuv+neQEE5ms4C7gZ6APcADcA3gL7A0cAU4J+z/PwXgB8AvfHv4P9zV+81s/7AvcCV4esuASbm0PY/0JItXwTc2c49k4ABwEzgT/isWaQYxbkvt2JmpcCfgUeBfsA3gXvM7IDwltuBS51zlcBY4O/h9SuBxeHP7AO0msqS1hSUo+lZ59z/OeeanHPbnXOvOOdecs41OOcWA7cCx2f5+fucc3Occ/XAXcD43bj3VOB159zD4WM/A9bl0PY/AOeH7/7PC5+zrS8CjzrnNuP/YJ1iZn3a3DMvfGed+jghh9cWiZo49+W2jgZKgf92ztU7554CHgOmho/XA6PMrNI5t8E5Nzft+kBgqHOuzjmnTDkLBeVoWp7+jZkdbGaPmtmHZrYZuBb/jjeTD9O+3gZU7Ma9A9Pb4fzJJSs6arhzbgn+Xfp/AW8551a1+V26AWfTEqyfDdswrc1TjXXO9Uz7eLqj1xaJoNj25XYMBJa51qcYfQAMCr8+CzgdWGZmfzOzI8Lr14f3PW1m75vZlbvx2nsNBeVoant012+At4ADnHPdgavwc7Efp9XA4NQ34TzwoMy3t3IncAXtD12fjf9jcauZfRi+zgA0hC3FKe59Od0qYEj48ylDgZUA4QjA6UB//DD3zPD6ZufcN51zw4Azge+YWbbRgb2agnI8VAKbgBozG0n2Oah8+TMwwcxOC6tGv4GfE8rF3cCJwP3tPPZF4LfAGPzw2njgOOCw8HcTKWZx6csJMytP+ygDnsfPiV9hZiVm9ingFPy8chcz+4KZdQ+HyLcATQDh6+4fBvNNQGPqMdmZgnI8XIEPZlvw77Tv+bhf0Dm3Bj8nfAOwHtgfX0Vdm8PPbnPOPeWc25F+3fxGB5OAG51zH6Z9vAw8Retseb61Xgf6P/n5zUQKKi59+QJge9rHu865WuA04Az8nPQvgC845xaGP/NF4INwWP7S8DkADgL+CmwFngN+7pz7R95+wSJjracHRNpnZgn88NU56lAi8aW+HG3KlCUjM5tiZj3Doasf4KsoXy5ws0RkF6kvx4eCsmRzDH59YTVwEnBWOIQlIvGivhwTGr4WERGJCGXKIiIiEaGgLCIiEhEFO7Wkb9++btiwYYV6eZHYePXVV9c553JdI14Q6s8iHculLxcsKA8bNow5c+YU6uVFYsPMPih0Gzqi/izSsVz6soavRUREIkJBWUREJCIUlEVERCKiYHPK8vGpr69nxYoV7Nixo+ObJTLKy8sZPHgwJSUlhW6KRIT6cjx1pi8rKBehFStWUFlZybBhw2h9yppElXOO9evXs2LFCoYPH17o5khEqC/HT2f7soavi9COHTvo06ePOnGMmBl9+vRRRiStqC/HT2f7soJykVInjh/9N5P26P+L+OnMfzMFZcm79evXM378eMaPH88+++zDoEGDmr+vq6vL6TkuueQS3n333az33HTTTdx11135aDLHHHPMTkNNp556Kj179mx17ac//Sldu3Zly5YtzdeeeuopevTo0fw7jh8/ntmzZ+elXflmZreZ2VozeyvLPZPM7HUzm29mf9+T7ZNoiWtffv311/PyXIWgOWXJuz59+jR3imuuuYaKigq+/e1vt7rHOYdzjiBo/33h7bff3uHrfPWrX+18Y9NUVlby4osvcuSRR7JhwwbWrFmz0z0zZszgsMMO46GHHuLCCy9svj558mQeeuihvLbnY3IH8CvgzvYeNLOewK+BKc65ZWbWfw+2TSImrn05zpQpyx6zaNEiRo0axfnnn8/o0aNZvXo1l19+OVVVVYwePZprr722+d7Uu92GhgZ69uzJ9OnTGTduHEcddRRr164F4Pvf/z433nhj8/3Tp09n4sSJHHTQQTz//PMA1NTUcPbZZzNq1CjOOeccqqqqMr6Lnjp1KjNnzgTgvvvu45xzzmn1+HvvvUdDQwPXXHMNM2bMyPu/z57gnHsG2JDlli8ADzjnloX3r90jDZNYiXpfbmv79u188YtfZMyYMUyYMIFnnnkGgDfffJPDDz+c8ePHM3bsWBYvXsyWLVs4+eSTGTduHIcccgj33XdfPv/pOqRMucj9x//N5+1Vm/P6nKMGdufq00bv1s++88473HnnnVRVVQFw/fXX07t3bxoaGpg8eTLnnHMOo0aNavUzmzZt4vjjj+f666/nW9/6FrfddhvTp0/f6bmdc7z88ss88sgjXHvttTz++OP88pe/ZJ999uH+++/njTfeYMKECRnb9pnPfIZLL72UpqYm7rnnHn7/+9/zox/9qPnxGTNmMHXqVCZNmsQll1zCunXr6Nu3LwCzZ89m/Pjxzfc+9NBDxHQv6BFAiZn9DagEfu6cazerlj1LfTn3vtzWL37xC8rKynjzzTeZP38+p5xyCgsXLuTXv/413/72tznvvPOora3FOcfDDz/MsGHDeOyxx5rbvCcpU5Y9av/992/uxOAD3YQJE5gwYQILFizg7bff3ulnunTpwsknnwzAYYcdxtKlS9t97s997nM73fPss88ydepUAMaNG8fo0Zn/AJWUlHDkkUcyc+ZMGhsbGTx4cKvHZ86cydSpU0kkEpx55pmt3kFPnjyZ119/vfkjpgEZ/Bv1w4DPAicBPzCzEe3daGaXm9kcM5tTXV29J9soERDlvtzWs88+ywUXXADA6NGjGThwIIsWLeKTn/wk1113HT/5yU9Yvnw55eXljB07lscff5zp06fz3HPP0aNHj5xfJx+UKRe53X0X/HHp1q1b89cLFy7k5z//OS+//DI9e/bkggsuaHcZQWlpafPXiUSChoaGdp+7rKysw3s6MnXqVD7/+c9z3XXXtbr+2muvsXjxYiZPngxAbW0tI0aM4F/+5V9263UibAWw3jlXA9SY2TPAOOC9tjc6524FbgWoqqpye7SVeyH15fy78MILOeqoo3j00UeZMmUKt912G8cddxxz5sxh1qxZTJ8+nZNPPpnvfe97H1sb2lKmLAWzefNmKisr6d69O6tXr+aJJ57I+2scffTR3HvvvYCfP2rv3Xu6SZMmMX36dM4777xW12fMmMF1113H0qVLWbp0KatWrWLJkiWsWLEi720usIeBY8wsaWZdgSOABQVuk0RcFPtyumOPPba5unvBggWsXr2aAw44gMWLF3PAAQfwjW98g1NPPZV58+axcuVKKioquPDCC7niiiuYO3du3n+XbJQpS8FMmDCBUaNGcfDBB7Pffvtx9NFH5/01vv71r3PRRRcxatSo5o9sw1FBEHDllVcCNL9Dd85xzz338PTTTzffZ2aceeaZ3HPPPYwbN26nOeWrr76as846K++/T2eZ2QxgEtDXzFYAVwMlAM65W5xzC8zscWAe0AT8zjmXcfmUCESvL5900knNW1wee+yx3HbbbfzzP/8zY8aMoaSkhDvvvJPS0lLuvvtuZsyYQUlJCQMHDuSaa67h+eefZ/r06QRBQGlpKbfcckvef5dszLnCjDpVVVU5nb/68ViwYAEjR44sdDMioaGhgYaGBsrLy1m4cCEnnngiCxcuJJmM5vvR9v7bmdmrzrmqDD8SCerPHw/15RZ7S1+O5m8D1DY0sqOuie5dktrRRnbb1q1bOeGEE2hoaMA5x29+85vIduJitml7PaWJgC6liUI3RWJqb+nLkf2N7nhuKT967B3m/8dJdCuLbDMl4nr27Mmrr75a6Gbs9Sb+8CkuPnoY3z1ZWZ/snr2lL0e20CsR+Oy4sUDD6yKSP4EZ6soiHYt+UG5UTxaJu8CgqUl9WaQj0Q/KenstEnuBmfqySA4iH5T17lok/oJAw9ciuYhuUA4rrhsUlGNn8uTJO20ecOONN/LlL385689VVFQAsGrVqp0Og0iZNGkSHS29ufHGG9m2bVvz96eccgobN27MpelZXXPNNZgZixYtavVaZtaqTa+//jpmxuOPP97q5xOJRKvjHa+//vpOtykuAoMmReXYKea+/NOf/rTTz/NxiGxQDlLD1wrKsTNt2rTm05ZSZs6cybRp03L6+YEDB3bqZJa2HXnWrFk7nYu8u8aMGdPqd/vTn/600x68M2bM4JhjjtnpJKkuXbq02h+7vY34i1VgpqAcQ8Xcl6MqskE5qaAcW+eccw6PPvpo8yHoqW0pjz322Oa1hhMmTGDMmDE8/PDDO/380qVLOeSQQwB/5NrUqVMZOXIkZ511Ftu3b2++78tf/nLzUXFXX3014E+DWbVqFZMnT27ep3rYsGGsW7cOgBtuuIFDDjmEQw45pPmouKVLlzJy5Eguu+wyRo8ezYknntjqddKdeeaZzW1+//336dGjR/NJUeB3//rTn/7EHXfcwZNPPtnu/r97IzOjsanQrZBdVcx9uT3tPWdNTQ2f/exnm49yvOeeewCYPn06o0aNYuzYsTudMd0ZkV0ArEKvPHlsOnz4Zn6fc58xcHLmodfevXszceJEHnvsMc444wxmzpzJueeei5lRXl7Ogw8+SPfu3Vm3bh1HHnkkp59+esYNYm6++Wa6du3KggULmDdvXqvj2n74wx/Su3dvGhsbOeGEE5g3bx7/+q//yg033MDs2bNbBUuAV199ldtvv52XXnoJ5xxHHHEExx9/PL169WLhwoXMmDGD3/72t5x77rncf//9zafKpOvevTtDhgzhrbfe4uGHH+a8885rdYj7888/z/Dhw9l///2ZNGkSjz76KGeffTbg/yilb8X53e9+d6c9totVYP4Ni3SC+nKzfPTltjI95+LFixk4cCCPPvoo4I9yXL9+PQ8++CDvvPMOZpaXIfWUyGbKKvSKt/Rhr/ThLucc3/ve9xg7diyf/vSnWblyJWvWrMn4PM8880xzhxo7dixjx45tfuzee+9lwoQJHHroocyfP7/DDeqfffZZzjrrLLp160ZFRQWf+9zn+Mc//gHA8OHDmwNmtiPlwJ8kNXPmTB566KGd9rdOnbmcui99CLvt8PXeEpDB92cNX8dTMfflXJ5zzJgxPPnkk3znO9/hH//4Bz169KBHjx6Ul5dz6aWX8sADD9C1a9ecXiMX0c2UVeiVH1neBX+czjjjDL75zW8yd+5ctm3bxmGHHQbAXXfdRXV1Na+++iolJSUMGzZst4Z4lyxZwk9/+lNeeeUVevXqxcUXX9ypoeLUUXHgC7KyDXmdeuqpXHnllVRVVdG9e/fm642Njdx///08/PDD/PCHP8Q5x/r169myZQuVlZW73bZi4OeUC92KmFNfzsmu9OVcjBgxgrlz5zJr1iy+//3vc8IJJ3DVVVfx8ssv8/TTT3Pffffxq1/9ir/+9a+dep2UyGbKKvSKt4qKCiZPnsw//dM/tSoK2bRpE/3796ekpITZs2fzwQcfZH2e4447jrvvvhuAt956i3nz5gH+qLhu3brRo0cP1qxZw2OPPdb8M5WVlWzZsmWn5zr22GN56KGH2LZtGzU1NTz44IMce+yxu/y7de3alR//+Mf8+7//e6vrTz/9NGPHjmX58uUsXbqUDz74gLPPPpsHH3xwl1+j2Jiqr2OrmPtyLs+5atUqunbtygUXXMCVV17J3Llz2bp1K5s2beKUU07hZz/7GW+88UanXjtdZDPlVKGXOnJ8TZs2jbPOOqtV9eb555/PaaedxpgxY6iqquLggw/O+hxf/vKXueSSSxg5ciQjR45sfpc+btw4Dj30UA4++GCGDBnS6qi4yy+/nClTpjBw4EBmz57dfH3ChAlcfPHFTJw4EYAvfelLHHrooTkPb6VLDVGnmzFjxk7D2WeffTY333wzF1100U5zylOmTNlrlkUFZpqKirFi7MvXXXddczEXwIoVK9p9zieeeIIrr7ySIAgoKSnh5ptvZsuWLZxxxhns2LED5xw33HBDzq/bkcge3Tj73bVccvsrPPCVTzJhaK892LL403Fv8VWsRzdO/unfGDOoB7+YdugebFX8qS/H1+725cgOXydV6CVSNDR8LZKbyAZlFXqJFA+dEiWSm8gG5UCZskjRCExFmyK5iGxQTmrzkE7RRg3xU8z/zbTN5u4r5v8vilVn/ptFNiinMmUNX++68vJy1q9fr84cI6k1zeXl5YVuysdC65R3j/py/HS2L0d/SZR68i4bPHgwK1asoLq6utBNkV1QXl7O4MGDC92Mj0UQKOPbHerL8dSZvhzZoByo0Gu3lZSUMHz48EI3Q6RZYKapqN2gvrz3iezwtfa+FikeGr4WyU1kg7IKvUSKh06JEslNZIOy9r4WKR6qvhbJTWSDclJBWaRo+L2vC90KkeiLbFBWoZdI8TDTVJRILiIblFXoJVI8EoFpTlkkB5ENyir0Eikeqr4WyU1kg7IKvUSKh06JEslNZIOyCr1Eiocv9FJfFulIZIOyMmWR4pEINHwtkovIBuXUecoKyiLxF2j4WiQn0Q3KKvQSKRqmQi+RnEQ/KDeqJ4vEXWBa3iiSi+gGZVOmLFIs/Jyy+rJIRyIblIPA/C5AenctEnumva9FchLZoAw+W1ZQFom/wAzFZJGORTsoBzoYXaQYBNr7WiQn0Q/KKvQSiT0d3SiSm2gHZVOmLFIMdHSjSG6iHZQTmlMWKQaBoVOiRHIQ7aCsQi+RvDKz28xsrZm91cF9h5tZg5mdk4/X1SlRIrmJdlDW2kaRfLsDmJLtBjNLAD8G/pKvFw0CFXqJ5CLyQblBhV4ieeOcewbY0MFtXwfuB9bm63X9kij1ZZGORDooByr0EtmjzGwQcBZwcw73Xm5mc8xsTnV1ddZ7NXwtkptIB+WkCr1E9rQbge845zqslXbO3eqcq3LOVfXr1y/rvTolSiQ3yUI3IBsVeonscVXATPN7z/cFTjGzBufcQ515UlNfFslJtIOyCr1E9ijn3PDU12Z2B/DnzgZk8H1ZXVmkY5EPyir0EskfM5sBTAL6mtkK4GqgBMA5d8vH9boavhbJTaSDsrbmE8kv59y0Xbj34ny9rvqySG4iX+jVoHkokdgzbbMpkpNIB+VAxSEiRSERaPhaJBeRDspJFXqJFAUNX4vkJtJBOVChl0hRMG0eIpKTSAflhN5dixSFwPznJkVmkawiHZS1o5dIcQj8ZiR6ky3SgUgHZRV6iRSHRJAKygVuiEjERTooJwMdSCFSDCw1fK3+LJJVpIOyCr1EikNq+FoxWSS7SAdlFXqJFIdUoZdGvkSyi3ZQVqGXSFFQoZdIbqIblNct4rCtf4fGhkK3REQ6qXn4WlttimSVU1A2sylm9q6ZLTKz6e08PtTMZpvZa2Y2z8xO6XTL3p3FP626hoSr7fRTiUhhBSr0EslJh0HZzBLATcDJwChgmpmNanPb94F7nXOHAlOBX3e+ZQn/ubGx008lIoUVhFFZc8oi2eWSKU8EFjnnFjvn6oCZwBlt7nFA9/DrHsCqTrfMfFB2TkFZJO40pyySm1yC8iBgedr3K8Jr6a4BLggPTZ8FfL29JzKzy81sjpnNqa6u7qBlYaas895EYk9LokRyk69Cr2nAHc65wcApwB/MbKfnds7d6pyrcs5V9evXr4OW+aBsTfV5aqKIFIrmlEVyk0tQXgkMSft+cHgt3aXAvQDOuReAcqBvp1qm4WuRopHKlLXEUSS7XILyK8CBZjbczErxhVyPtLlnGXACgJmNxAflDsanO2pZavhaQVkk7lKFXkqURbLrMCg75xqArwFPAAvwVdbzzexaMzs9vO0K4DIzewOYAVzsXCe7X5AMG6CgLBJ3Gr4WyU0yl5ucc7PwBVzp165K+/pt4Oi8tiwcvg5U6CUSey3V1wVuiEjERXdHr8A3zSfqIhJnOiVKJDfRDcphpmyuic6OhItIYTVnykqVRbKKblAO55QTNKliUyTmEoGGr0VyEeGg7DPlBE3amk8k5lToJZKb6AZlawnKqvUSiTfTNpsiOYluUA4LvQKaaFBUFom1ljnlAjdEJOIiHJRb5pTVkUXiLRH+pVGmLJJddINyOHydtEZlyiIxp+FrkdxENyiHhV6BCr1EYk+bh4jkJrpBWYVeIkVD1dciuYluUE7LlDV8LRJvCW0eIpKTyAflpDJlkdgzDV+L5CS6QTlt+FqZski8pYavtWWuSHbRDcppw9eahxKJt9R5yiraFMkuukE5LVNuVKIsEmsthV6FbYdI1EU3KDdvHqJ1yiJxF2idskhOIhyUfdO0JEok/lJBWXPKItlFNyinhq9NhV4icae9r0VyE92g3Fzo5TTkJRJzYUxWoZdIByIclP2ccpJGFXqJxFwi0PC1SC6iG5RNO3qJ5JuZ3WZma83srQyPn29m88zsTTN73szG5eN1tfe1SG6iG5QD7X0t8jG4A5iS5fElwPHOuTHAfwK35uNFtfe1SG6ShW5ARtZSfa15KJH8cM49Y2bDsjz+fNq3LwKD8/G6qW02G5Uqi2QV4Uy5ZZ1yo1JlkUK4FHgsH0/UMqecj2cTKV7RzZSbh6+dCr1E9jAzm4wPysdkuedy4HKAoUOHZn0+DV+L5Ca6mXJaoZcyZZE9x8zGAr8DznDOrc90n3PuVudclXOuql+/flmfU4VeIrmJblAOtPe1yJ5mZkOBB4ALnXPv5e95/WedpyySXXSHr1OFXtaoQi+RPDGzGcAkoK+ZrQCuBkoAnHO3AFcBfYBfh8VZDc65qs6+bmpOWcPXItlFOCgbzhJhpqxUWSQfnHPTOnj8S8CX8v26Gr4WyU10h68BmoNyoRsiIp1hKvQSyUmkg7ILEir0EikCOiVKJDeRDsoECZLKlEViL9DmISI5iXZQtjBT1rtrkVhLaE5ZJCfRDspBOKesVFkk1sLFFJpTFulAtINyqtBL/Vgk1lrmlAvcEJGIi3ZQDhLa+1qkCKS22dRUlEh2MQjKKvQSibuWdcoKyiLZRDoomyUIrEkdWSTmNHwtkptIB+VUptygSWWRWAu097VITiIelJN+nbLeXovEWvM6ZfVlkawiHZQtSJAw7eglEndBoHXKIrmIdFDGEiRxKvQSKQKBaZtNkY5EOygHAQkVeokUhcBMfVmkAxEPyn5OWYVeIvEXmGnUS6QD0Q7KllCmLFIkTMPXIh2KdlAOT4lqUKGXSOwlAg1fi3Qk2kHZUtXXhW6IiHSWn1MudCtEoi3aQTnMlLXhgEj8mWmbTZGORD4oJ2iiQUFZJPYCM73BFulAtIOyCr1EioafUy50K0SiLdpBWZmySNEINHwt0qGIB+UkCc0pixQFU6GXSIeiHZQtCM9TVk8WibvAdEqUSEeiHZQ1fC1SNBLaZlOkQ9EOypYgQIVeIsVAw9ciHYt2UA6SJGhUpixSBIJA22yKdCTiQTmhQi+RIhGY0aigLJJVtINyOHytQi+R+Eto+FqkQ9EOykGgoCxSJLTNpkjHIh6U/ZyyhrxE4i8w05yySAeiHZQtoXXKIsXgJ/tzce1d6BRWkeyiHZSDBIFTUBaJvfptlFGrUS+RDkQ7KKvQS6Q4WIIkTsPXIh1IFroBWQUKyiJFIfBb5qori2QX7Uw5SBA4FXqJxJ4lSNCo6muRDkQ7KKe22dTba5F4C5IkcBr1EulAtINyOHzd0KiSTZFYC3fnU6Iskl3Eg7Kf8nZNjQVuiIh0iiVImA6XEelItIOyhc1TUBaJt3B3PgVlkeyiHZSDBADOKSiLxJqlDpcpdENEoi3aQdl8UFamLBJzqRPflCmLZBXtoBzOKTc1NRS4ISLFwcxuM7O1ZvZWhsfNzH5hZovMbJ6ZTcjPCysoi+Qi4kE5lSkrKIvkyR3AlCyPnwwcGH5cDtycl1cNV1JoRZRIdtEOymGhl2vU8LVIPjjnngE2ZLnlDOBO570I9DSzfTv9wqk9B5Qpi2QV7aAcZspNmlMW2VMGAcvTvl8RXuuc5m02FZRFsol4UPZzyjopSiR6zOxyM5tjZnOqq6s7uDm1O9+eaZtIXEU7KIfV1wlrpEG9WWRPWAkMSft+cHhtJ865W51zVc65qn79+mV/VlVfi+Qk2kE5HL5O0ERDozqzyB7wCHBRWIV9JLDJObe6088aZsqKySLZRfvoRlNQFsknM5sBTAL6mtkK4GqgBMA5dwswCzgFWARsAy7JywunjmFVVBbJKtpBOcyUA5o0fC2SB865aR087oCv5v2FTdtsiuQiFsPXSZpoUKGXSHwFCRJOw9ciHYl2UE4bvq7X8Y0i8aV1yiI5iXZQThu+1pIokRhLzSmrH4tklVNQNrMpZvZuuB/u9Az3nGtmb5vZfDO7Oz+t81PePlNWZxaJLVVfi+Skw0IvM0sANwGfwe/u84qZPeKcezvtngOB7wJHO+c+MrP+eWlduM1mAq1TFom1IEHgGmlCUVkkm1wy5YnAIufcYmJytcEAACAASURBVOdcHTATvz9uusuAm5xzHwE459bmp3VaEiVSFFR9LZKTXIJyLnvhjgBGmNlzZvaimWU7hSZ3zTt6qfpaJNaCBEYTqtcUyS5f65ST+KPeJuG35XvGzMY45zam32Rml+OPg2Po0KEdP2tq72uaaFBvFokvSxC4JpwyZZGscsmUc9kLdwXwiHOu3jm3BHgPH6Rb2aW9cqHVOmUVeonEWKAlUSK5yCUovwIcaGbDzawUmIrfHzfdQ/gsGTPrix/OXtzp1pmWRIkUBUsQ0Ii6sUh2HQZl51wD8DXgCWABcK9zbr6ZXWtmp4e3PQGsN7O3gdnAlc659Z1vXar6uol6VV+LxFcQEDhlyiIdyWlO2Tk3C79Rffq1q9K+dsC3wo/8SVunrOprkRgzX+jVpFRZJKto7+jVvM1mI43KlEXiK0iEmXKhGyISbdEOykH63tfqzSKx1TynrH4skk20g7Lp6EaRohAkMJ0SJdKhaAdl7eglUhyCJIFrpFFRWSSreARl7eglEm/aZlMkJ9EOypaeKWv4WiS2ggTmGnEO7eolkkW0g3L68LUyZZH4Mj+nDGheWSSLiAfl9L2v1ZNFYivcZhOc5pVFsoh2UA7PU07SqB29ROKseSWF07yySBbRDspB2pIoZcoi8ZW2Za5iskhm0Q7KpjllkaKQtueAMmWRzKIdlMM55RJzqr4WibO0ok2d+CaSWcSDsu/IJYEyZZFYSxv1UlcWySzaQTnsyD5TVk8Wia20+hCtUxbJLNpBOSwOSZr2vhaJNWXKIjmJdlAGCJKUmNMpUSJxllZ9rUIvkcyShW5AhyxBiTXpPGWROEuvvlaqLJJRDDLlBEnNKYvEW6Dha5FcxCJTTtJEvXqySHylMmXT8LVINrHJlDV8LRJjrTJlBWWRTOIRlGlUoZdInKVXX+v9tUhG0Q/KltCOXiJxF1Zfa5tNkeyiH5SDBAnTjl4isWYavhbJRQyCclLV1yJxp+prkZxEPyhbQBLt6CUSazolSiQn0Q/K4fC1Cr1EYkzV1yI5iX5QDtcp67g3kRhT9bVITqIflINkmCmrJ4vElqqvRXISg6CcCOeU1ZFFYqs5U3YKyiJZRD8oW0Cg4WuReAvSt9kscFtEIiz6QTnMlDV8LdJ5ZjbFzN41s0VmNr2dx4ea2Wwze83M5pnZKfl5YRV6ieQiBkE5SWBNWqcs0klmlgBuAk4GRgHTzGxUm9u+D9zrnDsUmAr8Oi8vnlZ97RSURTKKflA2zSmL5MlEYJFzbrFzrg6YCZzR5h4HdA+/7gGsyssrt1qnnJdnFClK0T+6MUgQUKvNQ0Q6bxCwPO37FcARbe65BviLmX0d6AZ8Oi+vHFZfJ2lUfYhIFjHIlAMSaPhaZA+ZBtzhnBsMnAL8wcza/TthZpeb2Rwzm1NdXZ39WQP//l9LokSyi35QDpIEKvQSyYeVwJC07weH19JdCtwL4Jx7ASgH+rb3ZM65W51zVc65qn79+mV/ZUufU96dpovsHWIQlBMknIa8RPLgFeBAMxtuZqX4Qq5H2tyzDDgBwMxG4oNyB2lwDrTNpkhOoh+ULUEQFnqpalNk9znnGoCvAU8AC/BV1vPN7FozOz287QrgMjN7A5gBXOzy0fHSCr30Blsks5gUevmh64YmR0nCCtwgkfhyzs0CZrW5dlXa128DR+f9hcNCLw1fi2QX/Uw5SBDQCKB32CJxlZpTNg1fi2QT/aBsCQLnM2UVe4nEVGqbTZzWKYtkEf2gnJYpa1mUSEylVV9rxEsks+gH5bRMWbt6icRU0FLopYJNkcyiH5SDJNZc6KXha5FYanUgRYHbIhJhMQjKAYHT8LVIrKVVX6vQSySz6AdlS7QEZb3FFomnVgdSqB+LZBL9oJwoIXANADSo+loknrSjl0hOYhCUS7EmH5TrNXwtEk/pc8p6by2SUQyCcgmBqwe0eYhIbAUavhbJRQyCcilBkw/K9XqLLRJPOiVKJCexCMrmmvyhFBq+FomnsPo6sCYaFZVFMopBUC4BoIQGFXqJxJizhHb0EulADIJyKQClNGhJlEicBQrKIh2JTVAuoUE7eonEWdrZ6CLSvhgE5Zbhay2JEomxICBBk6ahRLKIQVAOM2Vr0LCXSJyFc8rKlEUyi35QDnymXEqDzlMWibMgoVUUIh2IflBuVX2tziwSV2YJEtak2hCRLGIQlFsKvTR8LRJjQYIScxq+FskiNkG5lAbt6CUSZ5YgiVOhl0gWMQjKqeHrRg1fi8RZkCAZqNBLJJsYBOWW6mt1ZpEYs9SSKPVjkUziE5S1zaZIvAUJkuZU6CWSRQyCcsuSKGXKIjFmCUpMmbJINjEIyi2ZstYpi8RYkAyXRCkoi2QSg6AcZsra0Usk3oKw+lr9WCSjGARlnymXW6P2vhaJMwt8pqwRL5GMYhOUy4JGdWaROAsSJLX3tUhWMQjKfvi6LGhUZxaJM0uQMG0eIpJNDIJymClbo5ZSiMRZoFOiRDoSr6CsOWWR+EodSKF+LJJRDIJyOHytHb1E4q05U9aIl0gm0Q/KZhCUUGoq9BKJtdQ2m3pzLZJRstANyEmilDIaqVdnFomvVKas4WuRjKKfKQMkSvzmIerMIvGVqr7Wm2uRjGISlEvDva81fC0SW0GCAE1DiWQTn6CsHb1E4s0SJHDaLlcki5gE5ZLwPGW9wxaJrSAgoIl69WORjGISlMPha2XKIvFlCQKn/QZEsolNUNbRjSIxpx29RDoUk6BcQpk1sr1eQVkktixBgE6JEskmJkG5lFJroKa2odAtEYk1M5tiZu+a2SIzm57hnnPN7G0zm29md+ftxYMwKCtTFskoJpuHlFBq26nZoaAssrvMLAHcBHwGWAG8YmaPOOfeTrvnQOC7wNHOuY/MrH/+GpAgcNo8RCSb+GTKNFBTp6As0gkTgUXOucXOuTpgJnBGm3suA25yzn0E4Jxbm7dXDwKMJi2JEskip6Ccy5BXeN/ZZubMrCp/TaS50GtHveajRDphELA87fsV4bV0I4ARZvacmb1oZlMyPZmZXW5mc8xsTnV1dcevHs4pa0mUSGYdBuW0Ia+TgVHANDMb1c59lcA3gJfy3UgSJSTxWXJNXWPen15EmiWBA4FJwDTgt2bWs70bnXO3OueqnHNV/fr16/iZA78kyjmULYtkkEumnMuQF8B/Aj8GduSxfV6ilKQLg7KKvUR210pgSNr3g8Nr6VYAjzjn6p1zS4D38EG68yyB4bNkbQQk0r5cgnKHQ15mNgEY4px7NI9ta5EoJUk9ANs0ryyyu14BDjSz4WZWCkwFHmlzz0P4LBkz64sfzl6cl1cPfKEXoGIvkQw6XehlZgFwA3BFDvfu2hxUSqKERJMPxltrNXwtsjuccw3A14AngAXAvc65+WZ2rZmdHt72BLDezN4GZgNXOufW56UBFmDO918tixJpXy5Lojoa8qoEDgH+ZmYA+wCPmNnpzrk56U/knLsVuBWgqqoq916ZKCVwPlPW8LXI7nPOzQJmtbl2VdrXDvhW+JFf4TplQAWbIhnkkilnHfJyzm1yzvV1zg1zzg0DXgR2CsidkiglaPJBeauCskg8WQILh69V6CXSvg6Dco5DXh+vRAnWpDllkVgLEs3D1/UKyiLtymlHr46GvNpcn9T5ZrWRKMUa6wCnOWWRuLIEhsO0/7VIRrHZ0ctwJGjSnLJIXAUJAJ0UJZJFTIJyCUDLoRTOwc/Hw2t3FbhhIpIz839uEmj/a5FMYhKUSwHoWRoWetXVwEdLoHpBgRsmIjkL/GyZPylKw9ci7YlJUPaZco9Sx7baRqjb6q/X1RSwUSKyS9KHr5Upi7QrJkHZZ8rdSx1b6xqgdou/Xru1gI0SkV1iPijrTGWRzGIVlHuUOj+nXLvZX69TUBaJjVaZsoavRdoTk6Dsh68rk6mgHGbKCsoi8dFc6OW0eYhIBjEJyi3D1zW1jRq+FomjoGX4WpuHiLQvVkG5ItlETV1DSzBWpiwSH6bha5GOxCQo++HripK2w9eqvhaJjdScsqnQSySTmARlnyl3SzT5dcqpQi8NX4vER3r1tZZEibQrXkE52cSO+iaadqQy5S1+dy8Rib5W22xq+FqkPTEJyn74ulsyPIt1+yZ/3TVBw45CtUpEdkVYfa1MWSSzmARlnyl3TaSC8paWx2q3wsblcOsk2Lq2AI0TkZykZcpaEiXSvlgF5S6BP7axacfmlsfqtsCqubDqNVj7diFaJyK5SKu+rtfwtUi7YhKU/fB1Kig3F3qBr8DetiG8rsIvkcgKVOgl0pGYBGWfKZeHw9fUboFEWfj1VtgeBmWtWxaJLtN5yiIdiVVQTmXKVrcVKvfxj9VtTcuUt7T30yISBdr7WqRDMQnKfvi61Br8t/VboXJf/1jdVti+0X+toCwSXenV18qURdoVk6AcDl+bz5ST9TXQPQzKGr4WiQedpyzSoVgF5VJrIEkDyaYdUDnQP9Zq+FpBWSSyrGWbzUZVX4u0Kx5BOUgARgkNVFi4WUjlAP+5bits/6jlaxGJpjBTLg2cTokSySAeQdkMEqVYYz39S+v8ta59fAadPnytOWWR6Aoz5RJzKvQSySAeQRl8sVdjPX1KwqBcVgmlFT4QK1MWib60TFmFXiLti1lQrmPf8nr/fSoob/kQmnxVtuaURSIsrL4uCZwKvUQyiFFQLoXGOj5RGQ57lVZCWQVsWtZyjzJlkegKM+WkKVMWySRmQbmeoRV+WVR9STefKW9c7h8v76lMWSTKLG34WnPKIu2KUVD2w9eDuvih6pXbk1DaDXaEG4f0HOoPpxCRaAoz5ZLA6ZQokQxiFJT98PWAch+UF29O+OHrlJ5Dfabs1NlFIimt+lpLokTaF6Og7Kuv+yZrAVi0scnPK6f03A+a6qGhtkANFJGsUpmyae9rkUxiFJR9plzetI0aylm8bnvrTLnHYP9ZxV4i0ZSqvjbtfS2SSeyCMrWb2RF0Y/G6Gj+nDFDWHbr09F9rAxGRaEqbU1amLNK+eAXl2s1Qu4WGZDcWV9f46muALr1avlamLBJNQRLQkiiRbOITlD8xCVa/ActegLLurNtay/agq3+sa++WoWwtixKJpuaDZRq1eYhIBvEJykf8iz8ZausaEl26A1Bd699506V3S9GXMmWRaEqWAVBudVoSJZJBfIJyaVeY/F0Ayrv5oLx6h5+jokuvtExZc8oikZTsAkA5ddTr6EaRdsUnKAOMPx+GHEn50AmUJgLeXhe+2+7a2++FDcqURaIqkQRLUEa9MmWRDJKFbsAuCRJw6RMkgTPWvcFf5r3LJQnC4WvNKYtEXkkXyqijXnPKIu2KV6ac5vLjPsFHDX6Oiq69VX0tkgMzm2Jm75rZIjObnuW+s83MmVlVXhuQLKecOi2JEskgtkH5wAGVHHTgCLbQhdreB/uhsWQXzSmLZGBmCeAm4GRgFDDNzEa1c18l8A3gpbw3IllOqYavRTKKbVAG+MKk8YzZ8XserznAXyirUFAWyWwisMg5t9g5VwfMBM5o577/BH4M7Mh7C0rKKVWhl0hGsQ7KE/br5Qu+Vm/2F0orNHwtktkgYHna9yvCa83MbAIwxDn36MfSgmQXSl0tjZpTFmlXrINySSLgE/26sXBNGIjLKlToJbKbzCwAbgCuyPH+y81sjpnNqa6uzu1FkmWUujqdEiWSQayDMvi55Xc/DIesSyuVKYtkthIYkvb94PBaSiVwCPA3M1sKHAk8kqnYyzl3q3OuyjlX1a9fv9xaUNKFUqfNQ0QyiX1QHtG/gpUbt1NT26A5ZZHsXgEONLPhZlYKTAUeST3onNvknOvrnBvmnBsGvAic7pybk7cWJMspcbXUq/papF2xD8oHDvCbhixcu1VzyiJZOOcagK8BTwALgHudc/PN7FozO32PNCJZRomr097XIhnEa/OQdhy0jw/K763ZwviySs0pi2ThnJsFzGpz7aoM907KewNKulDSpOFrkUxinykP7d2VsmTAwjVb/FabypRFoitZTtJpSZRIJrEPyonA2L9fBe+tSRu+VocXiaZkOcmmWpyDJmXLIjuJfVAGGDGgIsyUw60262ta37BlDWz/aM83TERaKykn2eT3JFG2LLKzogjKBw6oZNWmHWwv6ekvbF3b+oYZ58GDX97zDROR1pLlJJvqAKd5ZZF2FEVQHhFWYH9g4eZE1e+2POgcrF0A7/8V6rYVoHUi0ixZjuEopUEnRYm0oyiC8if6dQPgvcaB/kL1Oy0Pbl0DDTugsRaWPluA1olIs5IuADopSiSDogjKQ3p1JTBYtMmg+yBY917Lgx8tbfl64V/2eNtEJE3SH7dappOiRNpVFEG5NBkwuFdXFq+rgX4Htc6UU0G5zwGw6Ek/nC0ihZH0mXKZaf9rkfYURVAGGNa3G0vX10C/g6H6vZZlUR8tBQyqLvVfr3+/gK0U2cuVlAN++FonRYnsrGiC8vA+XVm6bhuu7wi/JGrzCv/AR0v9kPbBp/jvFz1ZsDaK7PWSPiiXUa8lUSLtKJqgPKxvN7bWNrCx2yf8hVQF9kdLodcw/9G1b+uhbRHZs5JpmbKGr0V2UlRBGWCJDfYXUsF3wxIfkAEq9/EbiYhIYaSqr61OJ0WJtKNogvLwPj4oL9paCt36+6Bctw22ftgSlCsG+CVSIlIYqr4WyapogvLgXl1IBsbS5grsd2HjMv9geqasoCxSOMmWdcraPERkZ0UTlJOJgKG9u7JkXQ30Hwlr5sPyl/yDvYf7z6lMWQUmIoVR0lLopc1DRHZWNEEZ/LzyknU1cPiXoKkRnviefyA9U25qgO0bCtZGkb1aqtDL6mjQ8LXIToorKPfpxgfrw2VRk7/nj3EsrYCuffwNFQP85y0fFq6RInuztOrrmtqGAjdGJHqKKigfOKCC7fWNvLtmCxz1NRh8OAwYDWb+hsp9/OetCsoiBZG2Tnnj9voCN0YkeooqKJ80eh9KEwH3vLIcEkm46BE4/08tN1T095+1LEqkMNKGrzdtU1AWaauognLvbqWcdMg+PDB3JTvqG6G0K5T3aLmhQpmySEEFAS5RRherZ5MyZZGdFFVQBph2+BA2ba/n8bfaCbylXaGsuzJlkQKyZDmVyUY2bq8rdFNEIqfogvKRn+jDsD5dufulZe3fUDFAmbJIIZWU0z3RwEYNX4vspOiCchAY5x0+lJeXbmBx9dadb2i71ea2DfDCTTrSUWRPSZZTkdDwtUh7ii4oA5x92CASgXHvnBU7P9g2U57/gF/PvO69PddAkb1ZspxuiQYFZZF2FGVQ7l9ZzuSD+nP/3BU7b3pfuQ9sXduSGW8KA3dN9Z5tpMjeqqScrlav4WuRdhRlUAY47/AhVG+p5cm31/DS4vWs3rTdP1AxAOq3Qe0W//2mlf5zzbrCNLQjq9/Q0LoUl2QXVV+LZFC0QXnyQf3oV1nGV+6ay3m3vsiVf5rnH0jt6pU6mGJzGJS3RTAor3wVfnMcLH+50C0RyZ9kGWVWz+YdOilKpK1koRvwcUkmAq46dRTPv7+e6i21PPNeNTW1DXSrTNtqs++BLUG5Zn3hGptJamg91UaRYlDShTLW4Bxs2VFPz66lhW6RSGQUbaYMcNq4gfzoc2O45Ohh1DU28eLi9WkbiISnRW1e5b+PYqa8LTw4Y/tHhW2HSD4lyyl1fo2y5pVFWivqoJxSNawXXUoS/P29augx2F/8aIkPxI3hBgZRnFPeFmbvCspSTJLllIRBedP2eljzNjz/ywI3SiQa9oqgXJZM8Mn9+/igXFYBPffzfwg2pS2ZimKmnArGOzYWth0i+VRSTrJpB4A/lOK1P8Bfvg8NtQVumEjh7RVBGeD4g/rxwfptLF1X40+OWjO/Za62+6BozikrU5ZilOxC0Jgavq5rmULS/+cie1FQHtEPgL++sxb6j4L1i2DDYv/gvuOimSk3B2VlylJEkmUEjT5T3ry9Hras9te3RfCNscgettcE5f36dGPMoB787wtLaeg3ClwjvP9XSJRB3xH+D0K29cBL/gE7Nu+x9gJphV4KylJESrpgjXUENPlCr+ZiSwVlkb0mKAN844QD+WD9Nv6yro+/8MHz0H0gdOsHTQ2Z525r1sP/ngZzbmv/8dotLX9Y8knD11KMkmUA9CxtYuO2WmXKImn2qqB8wsj+jBvcg+tfqsclynzldY/B0K2vvyHTvPL6RYCD9Qvbf/zpa/0mH415Xt6xPcyUVeglxSTZBYD+5Y7GrdX+DTG0jAyJ7MX2qqBsZnzzMyNYtqmOdV2G+YvdB0HXMChneqe+4X3/+aMP2n/8w7f83tkfPJe/xjbWw45N/mtlylJMSsoB6NfFEaSyZFBQFiHHoGxmU8zsXTNbZGbT23n8W2b2tpnNM7OnzWy//Dc1P44f0Y/PjtmXZzb19xd6DIJu4XB2pmKv9WFQ3rCk/cdTQXvB//nPK+fC0uegoROHuKcCceW+fq9uLReRYpH0QblPWROl29JObNPwtUjHQdnMEsBNwMnAKGCamY1qc9trQJVzbixwH/CTfDc0X8yM688ew9quBwDw2qZubE309A/WrIOlz8IbM1v/UCrobl65c3Cs3RLuo22w4M8+gN9+CtxxCvzkE/D+7N1raCpr6L2//xzVYq8NS+BnY/zBGSK5CINy79ImuuxYG17r0jJdI7IXyyVTnggscs4tds7VATOBM9JvcM7Nds5tC799ERic32bmV2V5CaeeeBIAP5tTywm3vOUf2LYOnvoPeLzNYEBq6RTOD2GvnAs3jvH7Z6ceO/iz/pzmu86BIAln3eqH6V77w+41MpU19EkF5YgOYa9+HTYtg79dX+iWSFyU+Dnl3mWNVNatBUv4feiVKYvkFJQHAcvTvl8RXsvkUuCxzjRqTxgyYQpNU+/hsosvZc32gLqgiw+wK+f4AJgq+nIO1i+Gfcf77z9aCu8+BhuXwZJnWoa2j/wKBCX+OT59NYw7D4YdC8te2r0Gtg3KUS322hzOCb47Cz58s7BtkXhIq77u0bAOVzEAKvorKIuQ50IvM7sAqAL+O8Pjl5vZHDObU11dnc+X3nVmBAdP4dgRAzjyE71Z7ypxC/4Mrsk/nqq0rqmGui1wwKf99x8tgeVhoF0xp2Voe+B4GHmqD8RVl/prQ46AzStab+f5/my488yOi1q2tx2+jmimvGUVJEqhtBL+8T+Fbo3EQeW+AAxqXEE/twFXuQ907aNCLxFyC8orgSFp3w8Or7ViZp8G/h043TnXblWSc+5W51yVc66qX79+u9Pej8W5VUNY01iB7dgIFv6TrAuDcioTHnIElHTzy6NWvuqvrXjFZ9GV+0JpNzjndrjoEQjC5xh6hP+cCuL12+GRf4XFs+HJq7I3Ki7D15tX+7XeE78E8x/yQ/oSWZEo2uw7AroP5oBNL7KPbaC2yz7QpbeCsgi5BeVXgAPNbLiZlQJTgUfSbzCzQ4Hf4APy2vw38+N18iH7stl6+G8O+IzP/FKZcmrOuM/+0GsYvPc41G2FHkPhw3mwdn5LNmvWEpABBhwCJV1h+cv+++d+4edf9/+Un2temmUJ1bYN/mcrw6Mmo1rotWU1VA6EA08EnIawIywyRZtmcOCnGbj+Rfa1DVRbb58p123p3IoFkSLQYVB2zjUAXwOeABYA9zrn5pvZtWZ2enjbfwMVwJ/M7HUzeyTD00VSl9IE3XoPAGDb0MnQ+xMtmfKG930hSs+h0Hu4n0sGOOJyv+nB6jegzyfaf+JECQw6DJa9COsWwbM/g9FnwXl/9M8368rMjdq2wWcPZT0Ai3CmvNJnyv3Dv+1r3ipseySb6BRtHngiiYYaKm077+/oDl17++vbN/h+8dfrPpaXFYm6nOaUnXOznHMjnHP7O+d+GF67yjn3SPj1p51zA5xz48OP07M/Y/QMH+pH6e7ZOAL6HNB6+LrXfj7A9hrmr3XtC2M+3/LDfQ7I/MRDjvDZ4x/OhNKucOJ1fqj7yK/4LHvTTjMB3rb1/g9VEECXnoUt9Hru5/CXH+x83blw+Hpf38YeQ/zpWxJV0SnaHH6cL4wE5m3u5jNl8MsS590D7z3xsbysSNTtVTt6ZdP7yPN5pOdF3PhqA3W99vcFXY31PlNODU+ngvKQiX5YuUc41Z56vD1Dj/SHX+zYDBc+6Lf1BB+sAVa83P7Pbd/Q8oeqvOfuZ8rr3/dBNdthGx155ff+o7GhTRs/gsZaP3wNPlte8/buv45ERkdFm+E9u1+4WVYJ+30SgDkbyqkrDfcKWP2638kuvThSZC+ioJyy7ziGfO5aNu1o4NkNvfzQ9LIXYe071PcbyXm/eYE3t4dDbIMPDz9X+c99sgTl/Y6GQy+EC+7zR0Sm7DPGb5iQmm9e8ao/tSoV+FKZMkCXXh0H5ff/Cr841C/XSvfq7b6o7P2nO/43aM/m1bDxA6ivgTVt5otTh3B099W0DBgN697VvGB05a1oE/JQuDnC7xWwtLE3C7eW+mupzXa2b4C6bRl+UKR4KSinOXRoL44+oA+/mue//+j+fwPXxJ9LpvDSkg08sGYfv+RpVDgNd+BJ/oSpXsMzP2lpVzjjVz67TpcogUETfGV2Qy3cdTb84Sy44WB4fYafU05lyl16ZS/0ev5X8IfP+aK01/7Y+rG1C/znZ29s/2c3Lts5A063/MWWr5e92Pqx1L7FqUx5wGj/Zmbde5mfrz3rFrbO5jcug43Ls/+M7I5oFW0e/iU2nXMvy90AXq0O/xQtTtsBb3OGqR2RIqag3Mavv3AY5518AgC9ti5i0/6n8ou5/vSnF1Y1wsV/bsmMx02FK95r3mB/lw0+HFbPg7fu95nw8dP9/PRD/+LnkLukMuVw+LpmnT/XOd3m1fDkD+Cgk2HMubD4760z1bXv+Crupf/w66rTbdsAv5oID381cxuXvegz+sqBOwflnTLlQ/zn9uaVP3wzBI9g9wAAIABJREFU8ylcr/zOZ/PLXoSmJr+O+/5LM7dJdkvkijaTZfQ45CSG9+3Gc6vCN2Tb1rcsS9ykN2ay91FQbqNH1xLOO24sTV18lvrVpcewZF0NIwZU8N6aLdTUpmWVbZdA7aohR0BTPTx1jV9idfx34KKH4aDP+sdTR0qmhq/v+ye48ww/P53yxt1+w5MTr4NRp/tlJal10Ts2+c1LjvwKlPfwVa2v3gFbwwTo3cegYTvMm+nXGKfMfwhuGO2D7rIX/DD9sKN90Eyfm05lyhXhsq0++/vlZGve8sE1tU/4js3w+xP9m4f2rA6HJl75HSx8ws/jr5zr13VLXkWxaHPC0F68vGwLrrTCX9jvaP9Z88qyF1JQziDY7yhWDDiBZ7cOYt8e5Xz7xINocvDWyk35e5HU3PTWNXDo+T7AJ8vg3P+Fz/6PXz4FYVDeAEv+7ovGVrzirzvnh6v3O9oHxOHH+323Fz3lH69+t+V1PnOtH1b+v2/4AzOaGv2pVj2GwMBD4c//1rLxxyu/88H80W/7YymHHuUL1rZ+6LcZTdm8yg/fJ8P5wEQJ9DvID0HecjT89gT/Ou886k+6WvTUzgVnTU0+iw6S8PbD8Lcf+UypqR5WvZ6/f+tc1W71b0gW/HnPv/Ze6nMTBvHRtno2WXd/YcQUwBSUZa+koJzJeX9k4GX3Mm3iEL57ykgO268XAK8vz+PSpIp+fk00BuPPb7meKIHDv9SSKZeHlan9DvYBK5UJf/C8n0c+9MLwvu4+gKaC8tqwErr/wXDYxTB9OZz1G78xytw7fXHYyNP84Rm1W+GZn/o/hEufhe6D4d1H/ZuAoUfAkCP9cy1P28t78yq/RjndgEN8kN2wxBeGvTsL3vyTf2zrmp2Htj9a4rP7I7/sA/HqN+CocDg9U2X6x2nNW/4NSarN8rE7+oC+nDF+IMu2h9NAg6v86oZMywVFipiCciZmBMkkP/rcWE4fN5A+FWUM6d2FN1bkeb3whC/CxMug55DM9/Qc4oPxqTf6YqrU3O5rf/B7To9K2//hgBPCwLLKF3mVdPND4+Az8THnQr+R8Nh3/HKmkadDvxEw/gsw93/h+V8CDs6/1/+cBTB4IvQf6TcyWfpsy2uldvNKd+gFMG4afO0Vv4Rs9n/B4r/BuC/4x9tWgX8YDl0fcrbP9Eu6wrFX+DcrywsUlMGPSjQ1+q83LvdTB786vGUeXfLqB6eOYnPQnSbMv7HrMVhzyrJXUlDeBeOH9OL1ZXkOysf8G5yScSmod/Bp8M35sN9RPmNdMcfPC7/1AIz9vK/wThlxsv/8+l0+KPc/uPW8dxDAcd/2Ablb/5aq8GO/5eemX7oFBlX54P/52+Hkn/gMPEjAgZ+BBY+0zPVuXtVS5JUy7Bg46xb/RuKor/ls3TXC0f/q1zEvahOUV8/zQ9f9R8GZN8Mls/xw/eCJPih3Zn11W1vWwN9+3HpOvq1UJr/9I5+1f/AC/KrKD8FvWgH3XLjzmdrO+SVtdTX5a+tepm9FGV0GjuK1pgOorisJg7KGr2Xvo6C8C8YN7sGqTTu45pH5XPj7l1i/NeMSzvwKgpZh4iFH+DXDqUz38Mta39v/YL9U64Wb/DByv5E7P9/os/w88oQLfbAFn9WOm+q/Hnuu/zy4ymfxKRMu9MVjC/4M9Tv8PHfbTDnd+PP9sq4Bh/hMe/9P+cKx9OD14Tw/LJ8sgx6DfLsAhhwONWv9GulFT+0cSFPLqJqasv7TUVfTEthf/g387b/8nHrqyMm21sz3ByaAH95/6hpfBf+1OX7of+UcuPtcv4/5a3f5+ffffgp+9yn43WfaX8q19h14O1Y7zxZE+Wf/i/PqfsDsd9ZC90F+SVQ+35SJxICC8i6oGuaXKP3xxQ94dtE6bn9u6Z5vROrkqfkP+DXTA9qeJwBM+o7P9LZv8MGwrSABl/8NTmhzUtWk78H4C2Dsee2/9rDjoOd+8NqdMOf3/lq/EZnbWtoVLrgfzg7v3f9T0FgHj14BL//Wz2Ovngf7jN35Z1M7nv3xbP/x9LUtj9VugbvP88uoFv818+uvfgN+PNwvOQM/jN5jqJ/Hvn3KztXdTU1+R7Lhx/vNXV66xa/TPvZbPvMfdTp85j/9m50nfwAPf8X/LtvWw6Tv+uHW334K7r8Mnvnvloz68e/AfZfsvNZ81evZ14jvZUYN7En/HhU8tWCNL0Bs2KEzlmWvo6C8C8YP6ck9lx/Jy//+aU4atQ93vrCUrbV7+I9qjyEt2enEy9u/Z9Bh/rQr8Jlzzs89CM68ya+Lbk8Q+DnjJc/4vbBHnuaH1rMZeGhLG/Y72rftrQdg1rfh1kk+G963naDcf5SfL/9oqV+7/dZ9fv21c/B//+YDa1l3eOU2f//cP/hNVFKaGv19jbXw+t0+IK56zY8GTL3LP+/Lt7Z+zU3LfNHZgNH+DURNtT+WM1VIB34Y/v8thv+3BL4xD/7tLfj6XJg0HS590v+uy170Byq89kdfrLT4735TlYVPtjzP1rU+Y//L97P/++1FzIwTRg7gHwvX/f/2zjs8yirtw/eZmUw6CekhCaQQuoTeBQFRQJorroVVVPhU7Kufrn6urqvuuuruKjasqCiKiAWkiAqC0gm9hBZKCoH0TNokU873xxlI6AGSzEDOfV25Mm+Z933eM3Pmd8pznofqANd3XM8ra5oYWpTPk96JoYT4m7n3qiQsVjtfrs1oXAOEgKTBari57cgznzfs75A0VM3N1iddJijnr7BkNQd8Puu0vXzgf5bCX4/ChG+UIMPpe8oGI9zypTr/2pdUz3/fz7B1thLowf8HPSfBnkVK5H94CH56Ws39gorVfXgjRF6heshp89SceeJV6q/11Sprl7XWErdj88mRndRxgAF/Pn1wGL8QlagkOA6MJrUvoh1M/AEe2aoaI6vfVmvAkaoBsXtBzft/e1X1BHtOrnv5NQGGto+g0uZgi8W1ZlnPK2uaGCZ3G3Cp0iUumL6Jobz16z4OFpTTLymM4Z2iMBpEw9985L9VD9B4lo8vsiPc9m393zsoBu5cpLyjvQMv7BqufLpMXgq75tcMVZ9MwpXqf4RNZeZaM015R8f1hgGPql7UiteVZ3RwnFo6NvcB1ZNf/5Hq7Q79G7w/SA1/e/nVrA0f+iy8NxAWPal6/zHdakQ5or3K5HXnopqlYOf7fP0eVHb99m9lb3g7NYxur1JzpanToftECDtLhrEmSN+kUPzNRn7KNtET9LIoTZND95QvgufGdKRTTDPmbTnM/V9sZNhry1mSdrThb2z2Ux7K7qJlHwiIuPjrhLVW3udna1yAWrd9xXgVKrS6Asa8pXrSzeNrerTXv6fmrm2VsPY9SLkJ/vChSgISkqiGolv1rwl0Ep2ilqNt+QI+GQnv9FHpApsngHeAEtZW/S48Ylv7sWr+2lahhszbXQfVZaonv/BxFfls0F8u7NqXMd4mI4PahvP97iqkyRcOrTzR2asgHWZNgLLzzEql0VwiaFG+CNpGBTJzch+2PHsN7/6pGyaDYPKMVL5Ords8mNXmYPqKA3y7UQ/RnZMutwJCDVvXdi4b945aRtWqn9o/aTHctwbGvg3+oUpcj0VGS7zqxGuOngqPbIM/zlDe5NmpaoShPjCa1NIz3xB1/4RBas34nDvVsrChf1MBMjSnMKJTNHll1eS0v1NNOyx4tMbLPnW6Gl358Un3GqnRNBB6+LoeMBgEwztFM6hNBHd/lsrjc7ayYl8+7aKa0SLYh7gQP7rGBSOEwOZwsmj7EXYfsfD9psNkF1cS6G1iVOcWmE26jXRGolPg4S0Q3PLE/QERJ/baa6fHPEaXCcrJqv1JTmlCqOsFt1Rz74ufUsFV6ovuE6Hb7eo+x7azUmHEv5TDm+a0DG4XgbfJwPumCTzX3wwrX1fOfn3uU6FYTb7KryDlZrV2XqO5jBDSTesAe/ToIVNTU8994iWG1ebg2bnbWbY7j9zSmnXMfxvdgTv7J/DPhWm8/9t+jAZB59gg+iSGMm1ZOl9M7k2/1mFutFzjqQghNkgpe7jbjrNR3/X57hmpbM0qYdVfBmP4fBzk7YLxH6uphtFTYfU7aqrioU3nnv7QaDyEutRl/W2uZ3y8jLwyXvXWSq02jlqsPPP9Dt5cuo+BbcL5dNVBxnZpwavjUzCbDJRX2floxQGW7MrVoqzRuLiuczQ/7TzKpqxiul/5GMwYA9/dCwYv6DAOzAEqvefhTSrQjEZzmaDHSxuQQB8vWkcE8uSIdhSWV3Pz+2uwOyWPDmtzfKja39tE38RQlu5Sy4N+3J5DRkHFCdcpLK9mxuqDOJw6upGmaTCkXQRmo4HZ67MgYaAK/VqSoTzqfYMhcbA6cf8yt9qp0dQ3WpQbgZS4YK7tGEleaRV/7BFLq1D/E44PbR/Bgfxynpu3g3s/38iN760iq6hGmP/z026enbuD5XtyG9t0jcYtBPp4MaFPS2ZvyGRbtkU5zYFKXALKiS+q86Ulyrm71J9Gcxa0KDcST45oz8A24Tw0NPmUY4PbKkelT1YdpH/rUCqrHdw+fR0FZVXklVbx9Qblnf3lOh3dSNN0+POwNoT6e/PXudtxJg9X69qvuLHmhMRBKr1n7VjqVsupCUM8he/vhe/ucbcVGg9Hi3IjkRDmz4y7ehEd5HvKsbgQP3rFh9ArIYSPJvbkozt6kl1UyZ8+WsfUJXuwOZwM7xjF0l255FqsbrBeo2l8mvl48fR17diSWcx/f96DjOl24rrxxKtULPWM1WrbXg3vDlDxyD0Nm1XFTD+6Q73WaM6AFmUP4fPJvfnq7j74eBnpGR/CB7f3ID2vjM/XZDCiUxR/GdEOh1Me7zVrNE2BcV1iGN89lrd+3cdT325jwdYclu/JQ0oJLfuqICz7l6uTt32tsort+O7UZCPu5sg2Ff/caYPcHe62RuPBaFH2EMwmA0LUhOgc2Cac927rTnJEAA8OSSYhzJ/eCSF8sTaDkkqbGy3VaBoPIQSvju/MvYOSmLU+k/u/2MjE6etU49Tsr9aX716ohq1XvQHeQSpyWu3kH57A4Y21Xm9ynx0aj0eLsgczuG0EPz86iPbRzQB4+OpkckutTJy+Dou1bsJcWe1g52HLuU/UaDwUIQRPjmjHL48OYvEjA+nRqjkvLUyjqLwa+kyBwgPwTl+1lnnEv1Sc9B0NEPf9YsjeAAGRKr+4FmXNWdCifAnRLymMt2/txvbsEga/uowbpq3ig9/2UzsAjJSS7dkl2BxObA4nk2esZ+QbvzNz7SGKyquZ9Ml6bvtoLTkldR/eO2qxcqig/NwnajQNSOuIANpGBfLCuE5YrHZeWbwL2o+Cm79QeZeD4pQjWIcxKo559Rm+s/ZqqCis2f75WZXf+2TKCyDtB1jzLpTnX5zx2RtVFLfoLiqPtkZzBrQoX2Jc0zGKT+7sxaA24dgdTv6xMI1HvtqM1eYAYN6Ww4x6cwVj3lrJo7O3sHJfAe2iAnn6u+1c8/pv/L43nw2Hihg59XfW7D93AvnyKjs3TFvFqDdXkFlYcc7zNZqGpn10M+7qH8+X6zJZsDUH2g6He1fA7XNV8pJON6hEID88osJyHoubfYyFj8Gb3VWO7fy9sHKqyiJmdY0o7f4RPh8PrybBV3+CH/8Cr3eGX/95YnKMqlKVgnP6CPhnrBLv02EtgYK90KKbSumZm+a+OW+HXT2fvfr0xxc8Bl/eCpacxrVLcxwtypcgA5LD+O9NXfj+/v48fm1b5m4+zORPUykoq+LFBWkkhfuTX1bFD1sOc8/AROY+0J+r20dgFIKv7unD/AcHEOTrxZPfbMXuUD9YNofztPd6dfFusosrcTol93+xkSq7ozEfVaM5Lf97bVu6tQzmsa83sz27RGUcC01SB1v2hY5/UMksZt8Oq9+qeWP+Ptj0OVQWqmxi6z4AYYQqC2ycATu+hy9vgtydcOVjcNdiuHelirG9/GX4/d9KmDd/oYR96YsqL3ZYsoqdfszprDbHhqtjXKIsHXBke8MX0ulY9k/1fMv/deqxzPWw/kOV93taX5UqtbQRst5pTkDHvr4M+Do1k8fnbCXE30xRRTXf39ef+FB/VqXnc01HleNZSolTcjzf8+IdR7jnsw3858YUIpv5cPdnqfzvNW25a0AC7y5P5/Vf9tAlLpi1BwqZ2Deevkmh3PPZBu7qn8Czozu4+YmbFk0x9nVdyCutYuxbK6iwOXh6ZHvGd489wVkShw1m3qg8nx/ZplKezpmkHMNiusORraoX3XYEWA5DYbrqwYYmKTE2etVcS0q1xnjrV0pYD29SubmH/wtie6he84dXQ1muyjqWcKXqiWetV+/Z9jU8cUBd/7UO0P1OaNEF2l4HAeE199m/XGXCMphUXu/+tVKbVpXBp6MgvD1c+w/l6GbJVulGhVCCuuN7uOYFZePJ7F8OM8aCdzOwV6psascaMlLCJ6MgfzdMmAOLnoDMtYCANsPV3H3CwJrkKueDlBf2vvrC6Ty/FKwlWZC3W0WPq2e761KXtShfJny2+iDPzN3BLb3ieOkPnc95vpSS695YQUmljVKrjYpqB04pmdgvno9XHqRby2BKrXaMBsE3U/rh723ib3O38+nqQ8eTZzidkmrX3HVZlR1vk5EQf3PDP2wTQ4vymdmfV8bjc7ay4VARkc28aRHsy629WnJjjzh1QsZamH4NXPtPlUHsk1Eqh3fH6+G9geqcyUtUvu0vbwYvPzUUfkysamOzwqejlUPZsL9DtztO/LEvSIcZ41Q40MhO6ofd6XLIjO0Jk39RAvVaJ7C4ljZGdFTpRr38YfWb8MtzylHN7A9FByD5GpWIwztAHVvxmhJsc4AKkmKvVLHAW/WHRY+r2ODSqQKrWEvUdcLaqDn39KXK2eyWWerZW3RVQlt2FBCw/gMY8Sr0vlvZlrsLts2GDZ+o93eZAKNeU9fNWKPWiAdEQKsBNeWQsVZNB3j5qkbFgd+UuI+bBp3+cOYPsvSIyhFv8lbbJwt5dYVqTBm9oMPYOn8/WPi48i/40zeqzGbfrj4XnyDVcOp2OzSLUaMXoCLELX5aefDH9lSpYuMHnpj0pLwA/EJOFWybFbx8zmqOFuUmxq4jFhLDAuqcAvJYb7m5nxez7u7Ln7/azM4cCz3jm/PZpN74eBlPOL+y2sF1b/yO1ebgmo5RzE7NpKK6ZjjbbDTw2aRe9E4MvehnsTmcXP/OSq5MDucvw9td9PUuZbQonx2nU/LtpmxWpeeTllNKWo6FF8Z25La+8eqET8dA5jolYIEtYMpK9aM6Z5ISpDvmq97UDw+pYeqz/ejbq5QYeQee/ritEta+qxzEWvWD5GuVwAdE1QhX0UElcpbDMHsixPdX89k5m5XAjn1biXDqdDXHG5oMVz4Kcx9Qjmx9pqi57GYtVCNi1RtqDXTiVfCHD9Tcd9Z68A9XPfj8PSpeeIuuMOQZNdS+6i346Wllj08wWIvVfaasAtNJDWubFX7/D/z2inKmsxyuETGAkEQVm7wwXXmZ+4Up8SzNUb14s79qyIx6DYozlU3XvFgjdHsWw1e3QVQnuO07Nee96HHofDP0nKQaBRtnKKFEqHOSXLHPM9fDho9Vo6TLrScK5c55MPs21Yjxba4aChWFKuVnRSHs/cl1zZOIvxLaj1HPXHZEPU/nm5QtGz+FlW+oe415S32mVWWw7CXYvUg16Mx+Z/z6aFHWnBUpJW8u3cegNuGkxAWTU1LJJ6sOcs/ApDP2eDdnFnPDtFUYBIzu3IKkiAC8jIJAHy/eW55Otd3JoocHEuTnddr315VZ6zJ48ttt+JmNrH5qKEG+F3e9SxktynWn2u7kvpkb+CUtl5t7xnFn/wTa2napH+cut6rhYB+1xPC405Y7h1bXvqeGioPiYMhf1Y9/bXvSl8K8h6AkUw07P7jhxPzhoIQpbS4MelKJeV2QUgllUJx6j71a3dd4lnq2cy6sflvN2bcfoxomOVsg9SM15BvcEloPhd73qsZCRaFq/FRZ1AhDzhYQBtWT7z0Fhr8EW2bBvAeUsBceUA2N4kMQkqRGCqRTzflfcSOk3AQ/PqWuO/JV5Ruw72cVQMZRDUlDVfmFJKiGw/xHoHk8jH0HZo5XDaoJX6u5fVCNgz2LVUNKGNTz+4ao0QmDQfXO9/6kltftWqAaPgBxfSBzjWo0BMXA1tnq8+l+Bwx7oeb7dRq0KGsahO3ZJYQGmE8JGbrFJdiD2oTz8vjOhPiZST1URF5pFcF+XqTEBRPgfeZsoTkllWQUVJASF8yQfy/DaBRkFlby1+vaM/nKxIZ+LI9Fi/L5UWV38OL8NGanZlJld3JlchhTBiUdT42aWVhBVlElfZMufkTnopFSzU9HdDjz0GdVGax5B6KuUPPflyIVhUoAk4bAytfV84S3Uw2Dln3h1q/g4Ao1ctBhjBruztulRLHTDUq0AY7uhA8GK+c6/3DVAOh1txL3Jc9DdWnNPX2CVLz0sNZqyN3pUI2EC6E4U90jpqsS/yXPw4r/qgZDXC+4+jlo2eecl9GirGl0pq84wAsLdmI2GgjxN5NTUhPnNyzAzBPXtmNI+whC/MwYDDU9AovVxti3VnIgv5wWQT4cLrEyc3Jvpv6ylxxLJcv+d/BxJ7WmhhblC6OovJov1mXw8cqD5JdVMa5LC4Z3iuaJOVsoq7IzZ0o/urVs7m4zmx4OO8y6BXK2qjnbLhNqhrKtFtUDP9voxYHflENd+9E1c9CgHPsK0lVPOzBKCfmZphkuFimhcL+ao6/r6ARalDVuIj2vjA9/309eaTWjU6JpGxXIkRIrbyzZy8aMYgB8vAw8PbI9t/WNR0rJlM838nPaUe67KonvNmXTJjKQ6Xf0ZNG2HKbM3HjiHGETQ4vyxWG1OXh3eTpvLt2HwylpGxlIqdWGn7eJ+Q8OOMV3QtMIHFs7fj5e0ZcBWpQ1HoWUkt/25nMwv5xf0o7y+958Rl4RxZESKxszinl6ZHv+Z2Di8XOFENgdTm75YA3rDxYxJqUFL17fiWY+XlRWO9iXW0anmGYnLoO5DNGiXD9szChiaVou916VxIZDRUycvo4+iSFEBPowpF0E47rGuNtEzWWOFmWNx+JwSl5amMbHqw7SLiqQ0SktuGdg4mkF1u5wMm1ZOlOX7CU+zJ9nRnXgHwt2sudoGUPaRfDc6I60DD2zx2NDklVUgd0haRXq12CNAy3KDcN/ftrNnA1ZOJyS3NIq7h+cxNguMezPKyclLui0aVY1motBi7LG47E7nJiMdRvCWpWez5TPN1JSaaO5nxd/7BHH52sO4ZTw/NiOJwSPONbTrm/S88qQUtI6IpCyKjsDXl5KcYWNsAAzr4zvzJB2kfV+Ty3KDYvd4eSZudv5cl3m8X1CwIDWYYzu3IJhHSJpfobVCOsOFHKooByzyYCf2UR4oDcpsUGX/eiN5sLQoqy57DiQX87MNYe4a0ACLYJ9OVxcyaOzN7NmfyHXdY7mmes6kJZj4f++28ZVbSN4fmxHvOoo+rXJtViZumQvdw1IICk8gFX78nlhQRppORb8zUaWPHYV87ce5sUFaTxydTILtuZQXGljyWODaOZTv8u3tCg3PFJKFm0/QpXdQcsQP5bvyef7TdlkuOK9B/t50dzPTGW1Ov70de35dXcur/+y95RrTejdkhfGdqLC5iC7qJI2kQEniHRJpQ1fL2Od4wloLh+0KGuaBA6n5J1f9/Hmr/sQQJXdSXSQDzklVga2CadrXDC5pVYGtYmgTWQAby7dx6GCch67pi39W4dRbXfiZRTHfzgrqu3c9N4atmWXEBZg5t5BSbzy425im/tyQ/dYpi7Zy7D2kWzKKCIuxI+v7unL1qxixr69kjv6xfO30R3r9fm0KLsHlXHNwop9+WQXV1BSacfHZODX3Xnkl1UBcEO3WB65Ohmbw0l5lYO5m7P5cMUBBrQOY1t2CSWVNtpFBXJzzziu7RTFr7vy+MeCncQ29+P927vTKtTfzU+paUy0KGuaFBkFFbz6025aBPvw56vb8O3GbJ6Zux0pJf5mE6VVavG/j5eBUH9vsosraRXqR1ZRJb0TQph+R08MQnDfzA0s3ZXLs6M6MG15OkctVaTEBvHpXb0I9jPz35/38MYS1UP6+I6eDG6ngjn89fttfLE2g//8MYVxXWLqbQhTi7JnYbHamLYsnVB/M5MGJJzwOUspeWXxbqYtS2dYh0j6J4UyZ2MW27Nrcpr3ig9h99FSpJQ8NDSZcV1jCAvwPt2tTqGovBpfsxEfLyMZBRWsPVDAFbFBtI0MrNP3ze5wsuFQEe2iml10gB/N+aNFWdPkKam04W0yYDIIlu/JY9eRUsZ3jyXI14tpy9JJy7EQHujNzLUZjLwiiqJyG6v3FxxfgpVRUME3G7OYfGUCga5haavNwbDXluNvNrHo4SuP/xiWVNi4ffpatmSV0CcxhHf/1J1gv4uPBa5F+dLDYrWdMI2xL7eUn3fmEhHozfVdY8h2TbusP1gEgMkgMJsMJIUHkBwRQHgzbyqqHKzeX0B5lZ3WEQEctVjZc7QMo0EQHeRDVlFN+sfY5r48MLg1o1JasCmjCLtD0q91KFabkx+351Bld2I2Gvjg9/2k55XjZRQMTA7njv7xDGgdhsVqx1JpI8DbhNlkwO6UNPMx1VvDUkqJwynr7D9yuaJFWaOpI9OWpfPyj7vwMgpeGd+Z67vGnvX8XIsVBEQEnhiFyeGUzFqfwd/n7aRXQgif3Nnzon+ItChfvuw9WsriHUeoqHZQaVPL/NJzy8gvq8ZkFPRKCKG5n5l9uWUE+3nRJzGUymoH6XllpMQFc2VyGNuzS5i1PpNNrhgAx2jmY6LK7qTKXpOWNTHcnymDktiXW8a3m7LJK60i0MdEqdV+im2JYf7cOSCB2Oa+WCptVNmc2J0Sf28jgT4mAn288DMb8TIaiAzOdrdKAAAJo0lEQVT0Od7zttocZBdXklNspdLmIKekklnrMtlztJQ7+sXz4NBkAr1NCIHHOcQ1lIPoMbQoazR1RErJzLUZtI8OpHurCwzFV4vZqZk8MWcr13eNoU1kIBarDZvdSY/4EIZ3ijqva2lRbnpIKZGSE6Lenev8n3ceZWtWCT3imyMlLNyWg6/ZyI3d44gM8qawvJqk8IDjjo9VdgdzNx9m46Ei4sP8CfE3U2q143A6cUpYsDWHbdkldbq/QUBKXDBOp2THYQt254m60i4qkOTIQOZvPXw85HiLIB/Gd4/Fx2xk1b4Cgvy86JsYSo/45iRHBB5POVtpc+CUnDVE77nKZmeOBUulnc6xQezNLWPu5myMQpAQ7s/A5HCC/bz4x4I05m05TPvoZvRLCmV0SgvaRNZvRDAtyhqNG3lh/k4+WnEAUMOTBoOg2u7k+bEdGdslhm83ZpESF3zOUI9alDXuQEolsFV2J0G+Jny8jJgMBsqq7JRabZRa7VRU27E7JXuOlvH73jy8jAa6t2pOm8gAooN8CfA24Wc2khDmjxCC7dkl/LorF4eUbMoo5re9eUipRLu4wsYRiwrL6+NlwCAEVXYnDpfARwf5EOTrxVGLFaNBENvcj4pqO5mFlQT7edHKFatASujYIoj4MD8yCir4fW8+u4+qmNhCqOPepmMNEzWKEOhtorzazqjOLcgsqmBrVgkOpyQ6yIeYYF8Sw/3pFBNEXIgfYf7e+HsbCfBWS+DOp2etRVmjcTO5FisBPib8zCaq7U7u/2IjP+88irfJQJXdyX1XJfHEOVJTalHWXK7kukQ4opkPUkoOFVSwMaOInYeVY5y3l4FmPl44pGTv0TJKrXaigrxxOCUZhRX4eploGeJHcUU1GYUVGITA7nQeb0x4mwx0igni+q4xxAT7simzmKhmPoxOicbfbCKjsIIfdxxhW1YJdw2IPz5KlldaxcJtOWzJLCa7uJK9uWUUllefYn+Iv5nW4QFYrDYslTZWPjnkrCKtRVmj8TCq7U6en78Dp4Rbe7WkU0zQOd9Tn6IshBgOTAWMwIdSyn+ddNwbmAF0BwqAm6SUB891XV2fNZ5Etd1JYXk1EYHedZ4COBtSSo5YrBwutlJQVkWlzUFxhY0dh0s4mF9BkJ8XYQFm/j6m01nXn9elLl/YIL1Go7kgzCYDL467wi33FkIYgbeBYUAWsF4IMU9KubPWaZOAIillayHEzcDLwE2Nb61Gc+GYTQaigs6QCvMCEEIQHeTbKKFXm7Z/ukbTtOgF7JNS7pdSVgOzgLEnnTMW+NT1eg4wVHiai6xGcxmjRVmjaTrEAJm1trNc+057jpTSDpQAoY1inUaj0aKs0WguDCHE3UKIVCFEal5enrvN0WguC7QoazRNh2wgrtZ2rGvfac8RQpiAIJTD1ylIKd+XUvaQUvYIDw9vAHM1mqaHFmWNpumwHkgWQiQIIczAzcC8k86ZB0x0vR4PLJXuWqKh0TRBtPe1RtNEkFLahRAPAItRS6KmSyl3CCGeB1KllPOAj4DPhBD7gEKUcGs0mkZCi7JG04SQUi4EFp6079lar63AjY1tl0ajUejha41Go9FoPAQtyhqNRqPReAhalDUajUaj8RC0KGs0Go1G4yFoUdZoNBqNxkPQoqzRaDQajYegRVmj0Wg0Gg9Bi7JGo9FoNB6CFmWNRqPRaDwELcoajUaj0XgIWpQ1Go1Go/EQhLsSwAgh8oBD5zgtDMhvBHPOF0+0S9tUNzzRJji7Xa2klB6dG/ESrs/aprrjiXZdajadsy67TZTrghAiVUrZw912nIwn2qVtqhueaBN4rl31iSc+o7ap7niiXZejTXr4WqPRaDQaD0GLskaj0Wg0HoKni/L77jbgDHiiXdqmuuGJNoHn2lWfeOIzapvqjifaddnZ5NFzyhqNRqPRNCU8vaes0Wg0Gk2TwWNFWQgxXAixWwixTwjxpJtsiBNC/CqE2CmE2CGEeNi1P0QI8bMQYq/rf3M32GYUQmwSQsx3bScIIda6yusrIYTZDTYFCyHmCCF2CSHShBB93V1WQog/uz677UKIL4UQPo1dVkKI6UKIXCHE9lr7TlsuQvGGy7atQohuDWlbY6Drcp3s86j6rOvyWe1o0PrskaIshDACbwMjgA7ALUKIDm4wxQ48JqXsAPQB7nfZ8SSwREqZDCxxbTc2DwNptbZfBl6TUrYGioBJbrBpKvCjlLIdkOKyz21lJYSIAR4CekgpOwFG4GYav6w+AYaftO9M5TICSHb93Q1Ma2DbGhRdl+uMp9VnXZfPzCc0ZH2WUnrcH9AXWFxr+yngKQ+way4wDNgNRLv2RQO7G9mOWNcHPwSYDwjUYnXT6cqvkWwKAg7g8lOotd9tZQXEAJlACGByldW17igrIB7Yfq5yAd4DbjndeZfin67LdbLFo+qzrst1sqfB6rNH9pSp+QCOkeXa5zaEEPFAV2AtECmlzHEdOgJENrI5rwNPAE7XdihQLKW0u7bdUV4JQB7wsWsY7kMhhD9uLCspZTbwbyADyAFKgA24v6zgzOXicd/9i8TjnsfD6jJ4Xn3Wdfn8qbf67Kmi7FEIIQKAb4BHpJSW2sekav40mgu7EGIUkCul3NBY96wjJqAbME1K2RUo56ThLTeUVXNgLOpHpgXgz6nDTm6nsculKeNJddlljyfWZ12XL4KLLRtPFeVsIK7WdqxrX6MjhPBCVeKZUspvXbuPCiGiXcejgdxGNKk/MEYIcRCYhRrymgoECyFMrnPcUV5ZQJaUcq1rew6qYruzrK4GDkgp86SUNuBbVPm5u6zgzOXiMd/9esJjnscD6zJ4Zn3Wdfn8qbf67KmivB5IdnnWmVET+vMa2wghhAA+AtKklP+tdWgeMNH1eiJqfqpRkFI+JaWMlVLGo8plqZRyAvArMN4dNrnsOgJkCiHaunYNBXbixrJCDXX1EUL4uT7LYza5taxcnKlc5gG3u7w2+wAltYbFLkV0XT4LnlifdV2+IOqvPjfWRP0FTKSPBPYA6cDTbrJhAGoYYiuw2fU3EjXnswTYC/wChLjJvquA+a7XicA6YB/wNeDtBnu6AKmu8voeaO7usgL+DuwCtgOfAd6NXVbAl6h5MBuqFzLpTOWCcvJ52/W934byNm3071Y9P7+uy3Wz0WPqs67LZ7WjQeuzjuil0Wg0Go2H4KnD1xqNRqPRNDm0KGs0Go1G4yFoUdZoNBqNxkPQoqzRaDQajYegRVmj0Wg0Gg9Bi7JGo9FoNB6CFmWNRqPRaDwELcoajUaj0XgI/w/7lC2+7L8VlQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['mae']\n",
    "val_acc = history.history['val_mae']\n",
    "\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs_range = range(epochs)\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(epochs_range, acc, label='Training MAE')\n",
    "plt.plot(epochs_range, val_acc, label='Validation MAE')\n",
    "plt.legend(loc='upper right')\n",
    "plt.title('Training MAE')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(epochs_range, loss, label='Training Loss')\n",
    "plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
    "plt.legend(loc='upper right')\n",
    "plt.title('Training Loss')\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE for the prediction is 0.0816 eV/atom and loss is 0.0816\n"
     ]
    }
   ],
   "source": [
    "y_pred = model.predict(X_test[:256])\n",
    "\n",
    "mse = mean_squared_error(y_test[:256], y_pred)\n",
    "mae = mean_squared_error(y_test[:256], y_pred)\n",
    "\n",
    "print('MAE for the prediction is {0} eV/atom and loss is {1}'.format(round(mae, 4), round(mse, 4)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}