Part III - Training and Development of Different LSTM Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lva4WL1eUAum"
   },
   "source": [
    "<h1 align=\"center\">PART III</h1>\n",
    "<h1 align=\"center\">Training and Development of Different LSTM Models</h1> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mkuAEyhzUAu0"
   },
   "source": [
    "To get started, let's download the necessary libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "PJYCN9LEUAu2"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import pickle\n",
    "import gensim, logging\n",
    "import gensim.models.keyedvectors as word2vec\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "RcrRvb5oXupu"
   },
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import LSTM\n",
    "from keras.layers.core import Dense\n",
    "from keras.models import load_model\n",
    "from keras.layers.core import Dropout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q2nLR8J5UAu7"
   },
   "source": [
    "Also let's write a style for alignment in the middle of all graphs, images, etc:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 17
    },
    "id": "AGpiAGeHUAu-",
    "outputId": "795c2fce-59fc-46fe-9253-f51303b82fe8"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       ".output_png {\n",
       "    display: table-cell;\n",
       "    text-align: center;\n",
       "    vertical-align: middle;\n",
       "}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML\n",
    "HTML(\"\"\"\n",
    "<style>\n",
    ".output_png {\n",
    "    display: table-cell;\n",
    "    text-align: center;\n",
    "    vertical-align: middle;\n",
    "}\n",
    "</style>\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QA8Xr_kuUAvC"
   },
   "source": [
    "<h2 align=\"center\">Preparation of input data</h2> \n",
    "\n",
    "Firstly, we will load the sample data we processed in the first part:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "OaiT9x7lUAvD"
   },
   "outputs": [],
   "source": [
    "with open('documents.pql', 'rb') as f:\n",
    "     docs = pickle.load(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "2VCXoOsbgCVy"
   },
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "pos = [doc for doc in docs if doc[1] == \"pos\"][:15000]\n",
    "neg = [doc for doc in docs if doc[1] == \"neg\"][:15000]\n",
    "\n",
    "docs = pos + neg\n",
    "random.shuffle(docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LvNMiyJfUAvG",
    "outputId": "fa641a85-b032-41a2-c03f-c9b9daba3b3c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of documents: 30000\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of documents:\", len(docs))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sc4ggoC9UAvL"
   },
   "source": [
    "Now we will load our word2vec model. \n",
    "\n",
    "This may take some time, as the model contains 325 250  words, so we will get a 325 250 x 300 embedding matrix that contains all the values of the word vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "NxCptR0qUAvO"
   },
   "outputs": [],
   "source": [
    "model = word2vec.KeyedVectors.load_word2vec_format('ubercorpus.lowercased.lemmatized.word2vec.300d.txt', binary=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u4SMf_dkUAvQ"
   },
   "source": [
    "Now let's get a list of all the words from our dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "6gYT_cCLUAvS"
   },
   "outputs": [],
   "source": [
    "words = list(model.index_to_key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_E6iFE2nUAvT"
   },
   "source": [
    "Just to make sure everything is loaded correctly, we can look at the dimensions of the dictionary list and the embedding matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ed2ShP8MUAvV",
    "outputId": "40630d64-188d-4ada-c057-a7b4bc698dd4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total words: 325250 \n",
      "\n",
      "Word-Vectors shape: (325250, 300)\n"
     ]
    }
   ],
   "source": [
    "print(\"Total words:\", len(words), \"\\n\\nWord-Vectors shape:\", model.vectors.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kz0v97JmUAvX"
   },
   "source": [
    "The shape of our sample for training will be `(H, 300, N)`, where:\n",
    "* `H` - number of samples;\n",
    "* `300` - dimension of each word; \n",
    "* `N` - number of words in each sentence.\n",
    "\n",
    "Let's analyze how many words are usually found in the reviews:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wTyGdPrxUAvY",
    "outputId": "27f6ddfa-2543-4cac-b7bd-beb00c4ee555"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average number of words in the review: 40\n",
      "\n",
      "Maximum review length: 363 \n",
      "Minimum review length: 1\n",
      "\n",
      "Review with more than 50 words: 7493 \n",
      "Review with more than 100 words: 2211 \n",
      "Review with more than 200 words: 323 \n",
      "Review with more than 300 words: 18 \n",
      "Review with more than 400 words: 0\n"
     ]
    }
   ],
   "source": [
    "max_word, min_word = 0, 100\n",
    "word50, word100, word200, word300, word400  = 0, 0, 0, 0, 0\n",
    "reviews_len = []\n",
    "\n",
    "for review, state in docs:\n",
    "    reviews_len.append(len(review))\n",
    "\n",
    "    if len(review) > max_word: max_word = len(review)\n",
    "    if len(review) < min_word: min_word = len(review)\n",
    "\n",
    "    if len(review) > 50: word50 += 1\n",
    "    if len(review) > 100: word100 += 1\n",
    "    if len(review) > 200: word200 += 1\n",
    "    if len(review) > 300: word300 += 1\n",
    "    if len(review) > 400: word400 += 1\n",
    "    \n",
    "print(\"Average number of words in the review:\", int(sum(reviews_len)/len(reviews_len)))\n",
    "print(\"\\nMaximum review length:\", max_word, \"\\nMinimum review length:\", min_word)\n",
    "print(\"\\nReview with more than 50 words:\", word50, \n",
    "      \"\\nReview with more than 100 words:\", word100,\n",
    "      \"\\nReview with more than 200 words:\", word200, \n",
    "      \"\\nReview with more than 300 words:\", word300,\n",
    "      \"\\nReview with more than 400 words:\", word400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LZdo8KFFUAva"
   },
   "source": [
    "Visualize the frequency of words in the review:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "yHTCVs34UAvb",
    "outputId": "25ae1107-05bd-4be4-db7b-a506c089f0a1"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAe+klEQVR4nO3de7xVVb338c9XvF8STCICFEzLzIwQLx0zTQsVTKw8ZY8d0ccnu9g5Wp1XYnXKSp8XnfOUZVc1SbSLkl0ktQzz0jnlDRRRMYMUEyRBwVsahv6eP8ZYMN3utffaMNZea22+79drvdacY95+ay7YvzXGHHNMRQRmZmYbapNWB2BmZgODE4qZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYtZCks6U9IMWHv8sSY9K+msLYxgtKSRt2qoYrAwnFCtC0g2SVknaotWxWGMk7QR8EtgjIl7Z6nis8zmh2AaTNBo4EAjgqCbs379cG7Ae52kn4LGIWN6MeLrj73Jgc0KxEo4HbgYuAqYASNpC0uOS9qytJGmopGclvSLPHylpXl7vD5L2qqy7WNLpkuYDf5O0qaSpkv4s6SlJCyS9q7L+IElfyc03D0j6WLUZRdL2ki6UtEzS0tzUM6i7D5OboWZKujgf6x5J4yvLQ9KulfmLJJ2Vpw+WtETSpyQtz8c7WtJESX+StFLSp7sccktJl+Vj3S7pjZV9v0rSTyWtyJ/r37rEebmkH0h6Ejihm8+yff4cKyQ9KOmzkjaR9HZgNvAqSU9LuqibbW+U9J48fUD+3JPy/KGS5uXpTfJ+H8yf+WJJ2+dlteaskyT9Bbguf1f/L39X9wOTuhz3BEn35/PxgKTjuvuerP04oVgJxwM/zK/DJA2LiNXAz4D3V9Z7L3BjRCyX9CZgOvAh4OXAecCsLk1m7yf9sRkcEWuAP5NqQtsDXwB+IGl4XveDwBHAWGAccHSXGC8C1gC7Am8CJgD/p4fPdBRwKTAYmAV8s6EzkbwS2BIYAXwOuAD4ALB3jv8/JI2prD8Z+AmwA/Aj4BeSNpO0CfBL4M68r0OB0yQd1mXby3OcP+wmlm+QztcuwEGk7+rEiLiWdL4ejohtI+KEbra9ETg4Tx8E3A+8tTJ/Y54+Ib/elo+zLS89XwcBrwMOI31XR5K+h/HAMbWVJG0DnAscERHbAf8EzOsmNmtHEeGXX+v9At4C/APYMc//Efh4nn478OfKur8Hjs/T3wG+1GVf9wEH5enFwP/u5djzgMl5+jrgQ5Vlbyc1wW0KDANWA1tVlr8fuL7Ofs8Erq3M7wE8W5kPYNfK/EXAWXn6YOBZYFCe3y6vv19l/bnA0ZVj3VxZtgmwjJR49gP+0iW2M4DvV7b9XQ/nZxDwHOkaSa3sQ8ANlViX9LD9ocD8PP1rUgK+Oc/fCLw7T/8W+Ghlu9fmfxObAqPz59+lsvw64MOV+QmV72ob4HHgPdXvy6/OeLmGYhtqCvCbiHg0z/8olwFcD2wtab98nWUs8PO8bGfgk7m563FJjwOjgFdV9v1Q9UCSjq80kT0O7AnsmBe/qsv61emdgc2AZZVtzwNe0cPnqvZ6eobULNVo+/9jEfF8nn42vz9SWf4s6Vf8S2KNiBeAJaTPszOpSap6jj5NSpAv2bYbO5I+94OVsgdJtZ1G3AS8RtIw0nd3MTBK0o7AvsDv8nqv6uYYtUTeXZxdv6u120bE34D3AR8mfV9XSdq9wXitxXyBzNabpK1IzViDtK7b6RbAYElvjIg7Jc0k1QYeAa6MiKfyeg8BZ0fE2T0cYu1Q2JJ2JjUdHQrcFBHP5zZ85VWWASMr246qTD9EqqHsGKnpbEM9A2xdmX8lKQmsr7Wx5maukcDDpCa6ByJitx627Wm48EdJNYWdgQW5bCdgaSNBRcQzkuYCpwJ3R8Rzkv4AfIJU86z9iHg4H6Nmpxz7I6z7TqpxLuPF389OXY57DXBN/vd1Ful7P7CRmK21XEOxDXE08DypSWhsfr0O+G9SWz2kGsv7gOPydM0FwIdz7UWStpE0SdJ2dY61DemP0goASSeSaig1M4FTJY2QNBg4vbYgIpYBvwG+Iull+SLyqyUdtJ6fex7wv/LF5cNJ1wc2xN6S3p1rQKeRkt/NwK3AU0qdE7bKx9tT0j6N7DTXkmYCZ0vaLiflTwB9ue/lRuBjrLteckOXeYAfAx+XNEbStsD/BS7rIXnPBP5N0khJQ4CptQWShkmanK+lrAaeBl7oQ7zWQk4otiGmkNrz/xIRf629SBdkj5O0aUTcAvyN1Mzxq9qGETGHdHH2m8AqYBHd9FKqrL8A+AqpGeYR4A2kazI1F5CSxnzgDuBq0q/kWtPT8cDmpF/qq0gXsoezfk4F3klq6z8O+MV67qfmClLSXQX8C+naxD9yQjiSlKgfINU4vke6yN6ofyWd//uB/yEl9el92P5G0nWg39WZJ+/vklz2APD3fNx6LgCuIXU2uJ3UeaNmE1LSexhYSUrWH+lDvNZCivADtmzgkXQE8N2I2LnXlc2sCNdQbEDITUITle5XGQF8nnUdAMysHzQ1oSjdnHZX7pkzJ5ftIGm2pIX5fUgul6RzJS2SNF/SuMp+puT1F0qaUu94tlET6d6UVaQmr3tJ94CYWT9papOXpMXA+EpvECT9J7AyIqZJmgoMiYjTJU0ktbtOJPW//3pE7CdpB2AO6QaoIPXh3zsiVjUtcDMz67NWNHlNBmbk6Rmsu6N5MnBxJDeTup4OJ91ZOzsiVuYkMhs4vL+DNjOznjX7PpQAfiMpgPMi4nxgWO7GCenmsdrNTyN48c1OS3JZvfIXkXQycDLANttss/fuu/teKDOzvpg7d+6jETF0fbdvdkJ5S0QsVRoMcLakP1YXRkTkZLPBcrI6H2D8+PExZ86cErs1M9toSHqw97Xqa2qTV0Qsze/LST1u9gUeqQ3ol99rQ2cv5cV3z47MZfXKzcysjTQtoeQ7n7erTZMGgLubNHJrrafWFNJNXeTy43Nvr/2BJ3LT2DXABElDco+wCbnMzMzaSDObvIYBP5dUO86PIuLXkm4DZko6iTQo3Hvz+leTengtIo2VdCJARKyU9CXgtrzeFyNiZRPjNjOz9TAg75T3NRQzs76TNDcixve+Zvd8p7yZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFbHRPQJ49NSrui1fPG1SP0diZjawuIZiZmZFOKGYmVkRTihmZlaEE4qZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYmZmRTihmJlZEU4oZmZWxEY3llc9HuPLzGzDuIZiZmZFOKGYmVkRTihmZlaEE4qZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYmZmRTihmJlZEU4oZmZWhBOKmZkV4YRiZmZFOKGYmVkRTihmZlaEE4qZmRXhhGJmZkU0PaFIGiTpDklX5vkxkm6RtEjSZZI2z+Vb5PlFefnoyj7OyOX3STqs2TGbmVnf9UcN5VTg3sr8l4FzImJXYBVwUi4/CViVy8/J6yFpD+BY4PXA4cC3JQ3qh7jNzKwPmppQJI0EJgHfy/MCDgEuz6vMAI7O05PzPHn5oXn9ycClEbE6Ih4AFgH7NjNuMzPru2bXUL4GfAp4Ic+/HHg8Itbk+SXAiDw9AngIIC9/Iq+/trybbdaSdLKkOZLmrFixovTnMDOzXjQtoUg6ElgeEXObdYyqiDg/IsZHxPihQ4f2xyHNzKxi0ybu+wDgKEkTgS2BlwFfBwZL2jTXQkYCS/P6S4FRwBJJmwLbA49Vymuq25iZWZtoWg0lIs6IiJERMZp0Uf26iDgOuB44Jq82BbgiT8/K8+Tl10VE5PJjcy+wMcBuwK3NitvMzNZPM2so9ZwOXCrpLOAO4MJcfiFwiaRFwEpSEiIi7pE0E1gArAFOiYjn+z9sMzPrSb8klIi4AbghT99PN720IuLvwD/X2f5s4OzmRWhmZhvKd8qbmVkRrWjy6iijp17VbfniaZP6ORIzs/bmGoqZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYmZmRTihmJlZEU4oZmZWhBOKmZkV4YRiZmZFOKGYmVkRTihmZlaEE4qZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYmZmRTihmJlZEU4oZmZWxKatDqBTjZ56Vbfli6dN6udIzMzag2soZmZWhBOKmZkV4YRiZmZFOKGYmVkRTihmZlaEE4qZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYmZmRTQtoUjaUtKtku6UdI+kL+TyMZJukbRI0mWSNs/lW+T5RXn56Mq+zsjl90k6rFkxm5nZ+mtmDWU1cEhEvBEYCxwuaX/gy8A5EbErsAo4Ka9/ErAql5+T10PSHsCxwOuBw4FvSxrUxLjNzGw9NJRQJL2hrzuO5Ok8u1l+BXAIcHkunwEcnacn53ny8kMlKZdfGhGrI+IBYBGwb1/jMTOz5mq0hvLt3Hz1UUnbN7pzSYMkzQOWA7OBPwOPR8SavMoSYESeHgE8BJCXPwG8vFrezTbVY50saY6kOStWrGg0RDMzK6ShhBIRBwLHAaOAuZJ+JOkdDWz3fESMBUaSahW7b0iwvRzr/IgYHxHjhw4d2qzDmJlZHQ1fQ4mIhcBngdOBg4BzJf1R0rsb2PZx4HrgzcBgSbUHe40ElubppaSERV6+PfBYtbybbczMrE00eg1lL0nnAPeSroG8MyJel6fPqbPNUEmD8/RWwDvy9tcDx+TVpgBX5OlZeZ68/LqIiFx+bO4FNgbYDbi1T5/SzMyartFHAH8D+B7w6Yh4tlYYEQ9L+mydbYYDM3KPrE2AmRFxpaQFwKWSzgLuAC7M618IXCJpEbCS1LOLiLhH0kxgAbAGOCUinu/TpzQzs6ZrNKFMAp6t/SGXtAmwZUQ8ExGXdLdBRMwH3tRN+f1000srIv4O/HOdfZ0NnN1grGZm1gKNXkO5FtiqMr91LjMzMwMaTyhbVu4pIU9v3ZyQzMysEzWaUP4maVxtRtLewLM9rG9mZhuZRq+hnAb8RNLDgIBXAu9rWlRmZtZxGkooEXGbpN2B1+ai+yLiH80Ly8zMOk2jNRSAfYDReZtxkoiIi5sSlZmZdZyGEoqkS4BXA/OA2j0gATihmJkZ0HgNZTywR75z3czM7CUa7eV1N+lCvJmZWbcaraHsCCyQdCvpwVkARMRRTYnKzMw6TqMJ5cxmBmFmZp2v0W7DN0raGdgtIq6VtDXgx/CamdlajQ5f/0HSY3nPy0UjgF80KygzM+s8jV6UPwU4AHgS1j5s6xXNCsrMzDpPowlldUQ8V5vJT1R0F2IzM1ur0YRyo6RPA1vlZ8n/BPhl88IyM7NO02hCmQqsAO4CPgRcTXq+vJmZGdB4L68XgAvyy8zM7CUaHcvrAbq5ZhIRuxSPyMzMOlJfxvKq2ZL07PcdyodTzuipV7U6BDOzjUpD11Ai4rHKa2lEfA2Y1OTYzMysgzTa5DWuMrsJqcbSl2epmJnZANdoUvhKZXoNsBh4b/FozMysYzXay+ttzQ7EzMw6W6NNXp/oaXlEfLVMOGZm1qn60strH2BWnn8ncCuwsBlBmZlZ52k0oYwExkXEUwCSzgSuiogPNCswMzPrLI0OvTIMeK4y/1wuMzMzAxqvoVwM3Crp53n+aGBGc0LqbD3dULl4mm/dMbOBq9FeXmdL+hVwYC46MSLuaF5YZmbWaRpt8gLYGngyIr4OLJE0pkkxmZlZB2r0EcCfB04HzshFmwE/aFZQZmbWeRqtobwLOAr4G0BEPAxs16ygzMys8zSaUJ6LiCAPYS9pm+aFZGZmnajRhDJT0nnAYEkfBK7FD9syM7OKXhOKJAGXAZcDPwVeC3wuIr7Ry3ajJF0vaYGkeySdmst3kDRb0sL8PqR2HEnnSlokaX51hGNJU/L6CyVN2YDPa2ZmTdJrt+GICElXR8QbgNl92Pca4JMRcbuk7YC5kmYDJwC/jYhpkqaSnld/OnAEsFt+7Qd8B9hP0g7A50nDv0Tez6yIWNWHWMzMrMkabfK6XdI+fdlxRCyLiNvz9FPAvcAIYDLrboqcQbpJklx+cSQ3k5rXhgOHAbMjYmVOIrOBw/sSi5mZNV+jd8rvB3xA0mJSTy+RKi97NbKxpNHAm4BbgGERsSwv+ivrhnAZATxU2WxJLqtX3vUYJwMnA+y0006okcDMzKyYHhOKpJ0i4i+kWsJ6kbQt6drLaRHxZLokk+TmtFjffVdFxPnA+QDjx4+PR0vs1MzMGtZbk9cvACLiQeCrEfFg9dXbziVtRkomP4yIn+XiR3JTFvl9eS5fCoyqbD4yl9UrNzOzNtJbQqm2HO3Slx3n3mEXAvd2eQDXLKDWU2sKcEWl/Pjc22t/4IncNHYNMEHSkNwjbEIuMzOzNtLbNZSoM92IA4B/Ae6SNC+XfRqYRrqv5STgQdY9m/5qYCKwCHgGOBEgIlZK+hJwW17vixGxso+xmJlZk/WWUN4o6UlSTWWrPA3rLsq/rN6GEfE/UPfa+KHdrB/AKXX2NR2Y3kusZmbWQj0mlIgY1F+BmJlZZ+vL8PVmZmZ1OaGYmVkRTihmZlaEE4qZmRXhhGJmZkU4oZiZWRFOKGZmVoQTipmZFdHo8PVWwOipV3VbvnjapH6OxMysPNdQzMysCCcUMzMrwgnFzMyKcEIxM7MinFDMzKwIJxQzMyvCCcXMzIpwQjEzsyKcUMzMrAgnFDMzK8IJxczMinBCMTOzIpxQzMysCCcUMzMrwgnFzMyK8PNQ2oCfk2JmA4FrKGZmVoQTipmZFeGEYmZmRTihmJlZEb4o38Z8sd7MOolrKGZmVoQTipmZFeGEYmZmRTQtoUiaLmm5pLsrZTtImi1pYX4fkssl6VxJiyTNlzSuss2UvP5CSVOaFa+ZmW2YZtZQLgIO71I2FfhtROwG/DbPAxwB7JZfJwPfgZSAgM8D+wH7Ap+vJSEzM2svTUsoEfE7YGWX4snAjDw9Azi6Un5xJDcDgyUNBw4DZkfEyohYBczmpUnKzMzaQH9fQxkWEcvy9F+BYXl6BPBQZb0luaxe+UtIOlnSHElzVqxYUTZqMzPrVcsuykdEAFFwf+dHxPiIGD906NBSuzUzswb1d0J5JDdlkd+X5/KlwKjKeiNzWb1yMzNrM/2dUGYBtZ5aU4ArKuXH595e+wNP5Kaxa4AJkobki/ETcpmZmbWZpg29IunHwMHAjpKWkHprTQNmSjoJeBB4b179amAisAh4BjgRICJWSvoScFte74sR0fVCv5mZtYGmJZSIeH+dRYd2s24Ap9TZz3RgesHQzMysCXynvJmZFeGEYmZmRTihmJlZEX4eSgfyc1LMrB25hmJmZkU4oZiZWRFOKGZmVoQTipmZFeGEYmZmRTihmJlZEU4oZmZWhO9DGUB8f4qZtZJrKGZmVoQTipmZFeGEYmZmRTihmJlZEb4ovxHwxXoz6w+uoZiZWRGuoWzEXHMxs5JcQzEzsyKcUMzMrAgnFDMzK8IJxczMivBFeXsJX6w3s/XhGoqZmRXhhGJmZkW4ycsa5qYwM+uJE4ptMCcaMwMnFGsiJxqzjYuvoZiZWRGuoVi/c83FbGByQrG24URj1tnc5GVmZkW4hmJtr17Npa9c0zFrLicU22j0lJjqJRs3w5k1rmMSiqTDga8Dg4DvRcS0FodkA0hfa0GuNZm9VEckFEmDgG8B7wCWALdJmhURC1obmdmGKZWY6nHCsv7UEQkF2BdYFBH3A0i6FJgMOKGY9aDZCas/OCl2jk5JKCOAhyrzS4D9qitIOhk4Oc+uZu6Rd/dTbBtiR+DRVgfRAMdZluPsA325x8VtEWMDOiXO127Ixp2SUHoVEecD5wNImhMR41scUq8cZ1mOs6xOiLMTYoTOinNDtu+U+1CWAqMq8yNzmZmZtYlOSSi3AbtJGiNpc+BYYFaLYzIzs4qOaPKKiDWSPgZcQ+o2PD0i7ulhk/P7J7IN5jjLcpxldUKcnRAjbCRxKiJKBWJmZhuxTmnyMjOzNueEYmZmRQy4hCLpcEn3SVokaWqr46mStFjSXZLm1brnSdpB0mxJC/P7kBbENV3Sckl3V8q6jUvJufn8zpc0roUxnilpaT6f8yRNrCw7I8d4n6TD+iPGfNxRkq6XtEDSPZJOzeXtdj7rxdlW51TSlpJulXRnjvMLuXyMpFtyPJflzjpI2iLPL8rLR7c4zoskPVA5n2NzeUu+93zsQZLukHRlni93LiNiwLxIF+z/DOwCbA7cCezR6rgq8S0GduxS9p/A1Dw9FfhyC+J6KzAOuLu3uICJwK8AAfsDt7QwxjOBf+9m3T3yd78FMCb/mxjUT3EOB8bl6e2AP+V42u181ouzrc5pPi/b5unNgFvyeZoJHJvLvwt8JE9/FPhunj4WuKyfzme9OC8Cjulm/ZZ87/nYnwB+BFyZ54udy4FWQ1k7REtEPAfUhmhpZ5OBGXl6BnB0fwcQEb8DVnYprhfXZODiSG4GBksa3qIY65kMXBoRqyPiAWAR6d9G00XEsoi4PU8/BdxLGumh3c5nvTjrack5zefl6Ty7WX4FcAhweS7vej5r5/ly4FBJamGc9bTke5c0EpgEfC/Pi4LncqAllO6GaOnpP0l/C+A3kuYqDRUDMCwiluXpvwLDWhPaS9SLq93O8cdyk8H0SnNhW8SYmwjeRPq12rbns0uc0GbnNDfRzAOWA7NJtaPHI2JNN7GsjTMvfwJ4eSvijIja+Tw7n89zJG3RNc6sv87n14BPAS/k+ZdT8FwOtITS7t4SEeOAI4BTJL21ujBS3bLt+nG3a1zAd4BXA2OBZcBXWhvOOpK2BX4KnBYRT1aXtdP57CbOtjunEfF8RIwljZCxL7B7i0PqVtc4Je0JnEGKdx9gB+D0VsUn6UhgeUTMbdYxBlpCaeshWiJiaX5fDvyc9J/jkVpVN78vb12EL1IvrrY5xxHxSP5P/AJwAeuaYFoao6TNSH+kfxgRP8vFbXc+u4uzXc9pju1x4HrgzaQmotqN2dVY1saZl28PPNaiOA/PTYsREauB79Pa83kAcJSkxaTLAYeQnjFV7FwOtITStkO0SNpG0na1aWACcDcpvil5tSnAFa2J8CXqxTULOD73UtkfeKLSlNOvurQ5v4t0PiHFeGzupTIG2A24tZ9iEnAhcG9EfLWyqK3OZ7042+2cShoqaXCe3or0TKR7SX+wj8mrdT2ftfN8DHBdrhG2Is4/Vn5EiHRtono++/V7j4gzImJkRIwm/W28LiKOo+S5bHaPgv5+kXpP/InUzvqZVsdTiWsXUi+ZO4F7arGR2iR/CywErgV2aEFsPyY1b/yD1IZ6Ur24SL1SvpXP713A+BbGeEmOYX7+xz+8sv5ncoz3AUf047l8C6k5az4wL78mtuH5rBdnW51TYC/gjhzP3cDncvkupIS2CPgJsEUu3zLPL8rLd2lxnNfl83k38APW9QRryfdeifdg1vXyKnYuPfSKmZkVMdCavMzMrEWcUMzMrAgnFDMzK8IJxczMinBCMTOzIpxQbECQ9Jk8yuv8PKrrfq2OaUPkUWqP6X3N9d7/wZL+qb+OZxuHjngEsFlPJL0ZOJI0eu5qSTuSRpu2+g4Gngb+0OI4bABxDcUGguHAo5GGtyAiHo2IhwEk7S3pxjwg5zWVO5f3Vnp2xZ2S/kv5OSuSTpD0zdqOJV0p6eA8PUHSTZJul/STPA5W7Tk3X8jld0naPZdvK+n7uWy+pPf0tJ/e5MEH/0vSbXl/H8rlB0u6QdLlkv4o6Yf5zmwkTcxlc5Wev3Gl0mCQHwY+nmtzB+ZDvFXSHyTd79qKrQ8nFBsIfgOMkvQnSd+WdBCsHavqG6TnUewNTAfOztt8H/jXiHhjIwfItZ7PAm+PNMDnHNJzJWoezeXfAf49l/0HaUiNN0TEXsB1DeynJyfl/e1DGmzwg3kYFEijBZ9Gem7JLsABkrYEziPd1b43MBQgIhaTnntxTkSMjYj/zvsYTrqD/khgWoMxma3lJi/reBHxtKS9gQOBtwGXKT2tcw6wJzA7/2AfBCzLYy4NjvSMFUjDjRzRy2H2J/2x/n3e1+bATZXltUEg5wLvztNvJ42ZVItzldKIrz3tpycTgL0qtYftSWNqPQfcGhFLAJSGUB9NatK6P9LzSyANX3My9f0i0qCQCyS1y2MUrIM4odiAEBHPAzcAN0i6izSo3Vzgnoh4c3Xd2iB+dazhxTX3LWubkZ5x8f46263O78/T8/+r3vbTE5FqVde8qDA1ya2uFPUWQz3VfTT9oVQ28LjJyzqepNdK2q1SNBZ4kDSI4dB80R5Jm0l6faThxR+X9Ja8/nGVbRcDYyVtImkU64Ybv5nUjLRr3tc2kl7TS2izgVMqcQ5Zz/3UXAN8JDflIek1SiNX13MfsIvWPQv8fZVlT5Ee/WtWjBOKDQTbAjMkLZA0n/xs9EiPgT4G+LKkO0kj6ta6yp4IfCs3D1V/jf8eeABYAJwL1B6TuwI4AfhxPsZN9P6gp7OAIZLuzsd/Wx/3c56kJfl1E+mxrQuA23MngvPooSYSEc+Sngv+a0lzSUnkibz4l8C7ulyUN9sgHm3YNnr5F/yVEbFni0MpTtK2+RpTbbj0hRFxTqvjsoHJNRSzge2DuRZ2D+ki/nktjscGMNdQzMysCNdQzMysCCcUMzMrwgnFzMyKcEIxM7MinFDMzKyI/w/WOmmw8o3T/QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(reviews_len, bins=50)\n",
    "plt.axis([0, 400, 0, 5000])\n",
    "plt.ylabel('Frequency')\n",
    "plt.xlabel('Sequence Length');\n",
    "plt.title(\"Average number of words\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UDmHmAUGUAvc"
   },
   "source": [
    "Our next function `fix_review_len()` will do a useful job, this function is designed to fix the size of reviews to a fixed size to feed them into a neural network with reviews of a certain length. Reviews whose length is less than fixed will be extended by zeros. This process does not affect the algorithm and reviews longer than the specified length will be truncated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kW4wcvQaUAvc",
    "outputId": "c971f32c-ed09-4cfb-8853-3bb0cee9d685"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3, 1, 2, 4, 5, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]\n"
     ]
    }
   ],
   "source": [
    "def fix_review_len(review, length):\n",
    "    if len(review) > length:\n",
    "        review = review[:length]\n",
    "    elif len(review) < length:\n",
    "        for i in range(length - len(review)):\n",
    "            zeros = [0] * 300\n",
    "            review.append(zeros)\n",
    "    return review\n",
    "\n",
    "example = [3, 1, 2, 4, 5]\n",
    "example = fix_review_len(example, 7)\n",
    "print(example) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IFC0ZjeFUAve"
   },
   "source": [
    "Now let's slightly improve our `sent_embed()` function from previous part and then update our dataset with word vectors.\n",
    "\n",
    "Based on the histogram data, as well as the average number of words in the files, we can say with confidence that most reviews will have less than 100 words, which is the maximum value of the length of the sequence that we set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "byCFbH6xUAve"
   },
   "outputs": [],
   "source": [
    "def sent_embed(words, docs):\n",
    "    x_sent_embed, y_sent_embed = [], [] \n",
    "    \n",
    "    max_seq_len = 50\n",
    "    \n",
    "    # recover the embedding of each sentence with the average of the vector that composes it\n",
    "    # sent - sentence, state - state of the sentence (pos/neg)\n",
    "    for sent, state in docs:\n",
    "        # average embedding of all words in a sentence\n",
    "        sent_embed = []\n",
    "        for word in sent:\n",
    "            try:\n",
    "                # if word is present in the dictionary - add its vector representation\n",
    "                sent_embed.append(model[word])\n",
    "            except KeyError:\n",
    "                # if word is not in the dictionary - add a zero vector\n",
    "                sent_embed.append([0] * 300)\n",
    "        \n",
    "        # add a sentence vector to the list\n",
    "        sent_embed = fix_review_len(sent_embed, max_seq_len)\n",
    "        x_sent_embed.append(sent_embed)\n",
    "        \n",
    "        # add a label to y_sent_embed\n",
    "        if state == 'pos': y_sent_embed.append(1)\n",
    "        elif state == 'neg': y_sent_embed.append(0)\n",
    "            \n",
    "    return x_sent_embed, y_sent_embed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "k-NTBtZKUAvf"
   },
   "outputs": [],
   "source": [
    "x, y = sent_embed(words, docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ys6ex88UUAvg",
    "outputId": "aa3f63c7-0da8-4eb0-9a6d-dcb5a786c918"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of X: (30000, 50, 300)\n"
     ]
    }
   ],
   "source": [
    "x = np.array(x)\n",
    "print(\"Shape of X:\", x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EQf3lg0BUAvh"
   },
   "source": [
    "<h3 align=\"center\">Split Corpus</h3>\n",
    "\n",
    "Now, for further work, we will divide our corpus for training, testing and development sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "sWwXlssdUAvi"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# train test\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=42)\n",
    "\n",
    "# train dev\n",
    "x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.25, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hn93dkv9UAvj",
    "outputId": "cfca084b-c387-4aae-9bb9-bac32deaa769"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of x_train: 16875 | Length of y_train: 16875\n",
      "Length of x_test:   7500 | Length of y_test:  7500\n",
      "Length of x_val:    5625 | Length of y_val:   5625\n"
     ]
    }
   ],
   "source": [
    "print('Length of x_train:', len(x_train), '| Length of y_train:', len(y_train))\n",
    "print('Length of x_test:  ', len(x_test), '| Length of y_test: ', len(y_test))\n",
    "print('Length of x_val:   ', len(x_val), '| Length of y_val:  ', len(y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sLZwdAMRUAvk",
    "outputId": "18607251-2566-43e7-867a-5861946aa3db"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of x_train set: (16875, 50, 300)\n"
     ]
    }
   ],
   "source": [
    "print(\"Shape of x_train set:\", x_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gk4hpYPmUAvl"
   },
   "source": [
    "<h2 align=\"center\">LSTM Model (Stochastic Gradient Descent)</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9I3W0MIMUAvl"
   },
   "source": [
    "Keras's benefit is that it is built on top of symbolic mathematical libraries such as TensorFlow and Theano for fast and efficient computation. This is needed with large neural networks.\n",
    "\n",
    "A downside of using these efficient libraries is that you must define the scope of your data upfront and for all time. Specifically, the batch size.\n",
    "\n",
    "Neural networks are trained using the stochastic gradient descent optimization algorithm.\n",
    "\n",
    "This involves using the current state of the model to make a prediction, comparing the prediction to the expected values, and using the difference as an estimate of the error gradient. This error gradient is then used to update the model weights and the process is repeated.\n",
    "\n",
    "The error gradient is a statistical estimate. The more training examples used in the estimate, the more accurate this estimate will be and the more likely that the weights of the network will be adjusted in a way that will improve the performance of the model. The improved estimate of the error gradient comes at the cost of having to use the model to make many more predictions before the estimate can be calculated, and in turn, the weights updated.\n",
    "\n",
    "Alternately, using fewer examples results in a less accurate estimate of the error gradient that is highly dependent on the specific training examples used.\n",
    "\n",
    "This results in a noisy estimate that, in turn, results in noisy updates to the model weights, e.g. many updates with perhaps quite different estimates of the error gradient. Nevertheless, these noisy updates can result in faster learning and sometimes a more robust model.\n",
    "\n",
    "The number of training examples used in the estimate of the error gradient is a hyperparameter for the learning algorithm called the “batch size,” or simply the “batch.”\n",
    "\n",
    "A batch size of 32 means that 32 samples from the training dataset will be used to estimate the error gradient before the model weights are updated. One training epoch means that the learning algorithm has made one pass through the training dataset, where examples were separated into randomly selected “batch size” groups.\n",
    "\n",
    "Historically, a training algorithm where the batch size is set to the total number of training examples is called “batch gradient descent” and a training algorithm where the batch size is set to 1 training example is called “stochastic gradient descent” or “online gradient descent.”\n",
    "\n",
    "A configuration of the batch size anywhere in between (e.g. more than 1 example and less than the number of examples in the training dataset) is called “minibatch gradient descent.”\n",
    "\n",
    "  - **Batch Gradient Descent**. Batch size is set to the total number of examples in the training dataset.\n",
    "  - **Stochastic Gradient Descent**. Batch size is set to one.\n",
    "  - **Minibatch Gradient Descent**. Batch size is set to more than one and less than the total number of examples in the training dataset. For shorthand, the algorithm is often referred to as stochastic gradient descent regardless of the batch size. Given that very large datasets are often used to train deep learning neural networks, the batch size is rarely set to the size of the training dataset.\n",
    "\n",
    "Smaller batch sizes are used for two main reasons:\n",
    "\n",
    "   - Smaller batch sizes are noisy, offering a regularizing effect and lower generalization error.\n",
    "   - Smaller batch sizes make it easier to fit one batch worth of training data in memory (i.e. when using a GPU). \n",
    "   - A third reason is that the batch size is often set at something small, such as 32 examples, and is not tuned by the practitioner. Small batch sizes such as 32 do work well generally.\n",
    "   \n",
    "Nevertheless, the batch size impacts how quickly a model learns and the stability of the learning process. It is an important hyperparameter that should be well understood and tuned by the deep learning practitioner."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zKrM1guPUAvm"
   },
   "source": [
    "<h3 align=\"center\">Setting Up and Creating the Network</h3>\n",
    "\n",
    "The network has one input, a hidden layer with 128 units, and an output layer with 1 unit.\n",
    "\n",
    "A mean squared error optimization function is used for this problem with the efficient ADAM optimization algorithm.\n",
    "\n",
    "So now let's configure and create the network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zkEEUIlhUAvn",
    "outputId": "f21ebb35-65a0-4a5e-d49b-34fde5681d72"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " lstm (LSTM)                 (None, 128)               219648    \n",
      "                                                                 \n",
      " dense (Dense)               (None, 1)                 129       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 219,777\n",
      "Trainable params: 219,777\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import LSTM\n",
    "from keras.layers.core import Dense\n",
    "\n",
    "# configure network\n",
    "n_neurons = 128\n",
    "n_epoch = 20\n",
    "n_batch = 1\n",
    "\n",
    "# design network\n",
    "model_batch = Sequential()\n",
    "model_batch.add(LSTM(n_neurons, input_shape=(x.shape[1], x.shape[2])))\n",
    "model_batch.add(Dense(1, activation='sigmoid'))\n",
    "model_batch.compile(loss='mean_squared_error', optimizer='adam', metrics=['binary_accuracy'])\n",
    "\n",
    "print(model_batch.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PSDwAFP2UAvo",
    "outputId": "661a6ba0-a9a3-44c5-a481-f4e90d51ecc4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "16875/16875 [==============================] - 560s 33ms/step - loss: 0.1972 - binary_accuracy: 0.6882\n",
      "Epoch 2/20\n",
      "16875/16875 [==============================] - 551s 33ms/step - loss: 0.1515 - binary_accuracy: 0.7828\n",
      "Epoch 3/20\n",
      "16875/16875 [==============================] - 544s 32ms/step - loss: 0.1356 - binary_accuracy: 0.8094\n",
      "Epoch 4/20\n",
      "16875/16875 [==============================] - 590s 35ms/step - loss: 0.1226 - binary_accuracy: 0.8309\n",
      "Epoch 5/20\n",
      "16875/16875 [==============================] - 600s 36ms/step - loss: 0.1130 - binary_accuracy: 0.8491\n",
      "Epoch 6/20\n",
      "16875/16875 [==============================] - 581s 34ms/step - loss: 0.1021 - binary_accuracy: 0.8636\n",
      "Epoch 7/20\n",
      "16875/16875 [==============================] - 565s 34ms/step - loss: 0.0946 - binary_accuracy: 0.8741\n",
      "Epoch 8/20\n",
      "16875/16875 [==============================] - 584s 35ms/step - loss: 0.0875 - binary_accuracy: 0.8826\n",
      "Epoch 9/20\n",
      "16875/16875 [==============================] - 577s 34ms/step - loss: 0.0830 - binary_accuracy: 0.8891\n",
      "Epoch 10/20\n",
      "16875/16875 [==============================] - 577s 34ms/step - loss: 0.0768 - binary_accuracy: 0.9010\n",
      "Epoch 11/20\n",
      "16875/16875 [==============================] - 573s 34ms/step - loss: 0.0777 - binary_accuracy: 0.8961\n",
      "Epoch 12/20\n",
      "16875/16875 [==============================] - 589s 35ms/step - loss: 0.0731 - binary_accuracy: 0.9045\n",
      "Epoch 13/20\n",
      "16875/16875 [==============================] - 633s 38ms/step - loss: 0.0717 - binary_accuracy: 0.9081\n",
      "Epoch 14/20\n",
      "16875/16875 [==============================] - 654s 39ms/step - loss: 0.0707 - binary_accuracy: 0.9079\n",
      "Epoch 15/20\n",
      "16875/16875 [==============================] - 711s 42ms/step - loss: 0.0689 - binary_accuracy: 0.9121\n",
      "Epoch 16/20\n",
      "16875/16875 [==============================] - 674s 40ms/step - loss: 0.0650 - binary_accuracy: 0.9156\n",
      "Epoch 17/20\n",
      "16875/16875 [==============================] - 701s 42ms/step - loss: 0.0658 - binary_accuracy: 0.9154\n",
      "Epoch 18/20\n",
      "16875/16875 [==============================] - 619s 37ms/step - loss: 0.0610 - binary_accuracy: 0.9227\n",
      "Epoch 19/20\n",
      "16875/16875 [==============================] - 609s 36ms/step - loss: 0.0608 - binary_accuracy: 0.9225\n",
      "Epoch 20/20\n",
      "16875/16875 [==============================] - 585s 35ms/step - loss: 0.0605 - binary_accuracy: 0.9241\n"
     ]
    }
   ],
   "source": [
    "model_batch_hist = model_batch.fit(x_train, np.asarray(y_train), batch_size=n_batch, epochs=n_epoch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jaq6sAhBUAvp",
    "outputId": "46b48807-da65-4fb9-948c-604ffd277a80"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5625/5625 [==============================] - 36s 6ms/step - loss: 0.1859 - binary_accuracy: 0.7673\n",
      "\n",
      "Model Accuracy: 76.73%\n"
     ]
    }
   ],
   "source": [
    "score_batch = model_batch.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\",str(round(score_batch[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 299
    },
    "id": "-ZBoMTiMUAvp",
    "outputId": "538b2cbb-952c-48e6-f9bd-e2514be2ebc4"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEaCAYAAADg2nttAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVzU1f7H8deZGQERRGdGQNxDtERxwyIyl+Bni1amVrc9tWtGVtTVTH/tXc1yy7R+er1ktv3SXK/VbUFTf0neTENzS3DLBSVAZVGU4Xt+f1CThMg6i/B5Ph48ZOZ7zsx7RuAz33O+3/NVWmuNEEIIcREmTwcQQgjh/aRYCCGEqJAUCyGEEBWSYiGEEKJCUiyEEEJUSIqFEEKICkmxEKIG3n33XSwWS5X6vPjii7Rv395FiYRwDSkWok568MEHUUoxZMiQMttWrlyJUqrKf+TdqV+/fjz00EOejiGEkxQLUWe1bt2aTz/9lOPHj5e6f968ebRp08ZDqYS4NEmxEHVWREQEMTExvPvuu877fvnlF77++muGDx9epv3nn39Oz5498fX1JTg4mISEBAoKCpzbDcPgueeeIzg4mICAAO68805OnDhR5nG+/vprrrnmGho2bEiLFi0YPnw42dnZtfraNm7cSJ8+fWjYsCFNmzbl7rvvJjMz07n98OHDDB06FLvdjp+fH5dddhlTp051bl+5ciXdu3fH39+fJk2acOWVV/Ljjz/WakZRt0ixEHXaqFGj+Oc//8nvq9r885//JC4ursyexbZt27jlllvo06cPW7duZeHChXz66aeMHj3a2Wb27NnMmDGDqVOnsmXLFnr27MlLL71U6nHWrFnDrbfeyl/+8he2bdvGihUrOHDgAEOGDKG2VtY5duwYAwYMoGXLlnz//fesWrWK7du3M2zYMGebhIQETp06RXJyMrt37yYpKYmWLVs6+99+++3cdddd7Nixg++++47ExESvHpYTXkALUQc98MADOi4uTp85c0ZbrVa9Zs0a7XA4dIsWLfTSpUv1ggULtNlsdra/9957da9evUo9xooVK7RSSh84cEBrrXWLFi30xIkTS7UZOnRoqcfp27evHj9+fKk2Bw8e1ID+8ccftdZav/DCCzo8PPyi+fv27atHjhx5wW3PPvusbtGihT579qzzvtTUVA3odevWaa21joqK0i+88MIF+2/ZskUDev/+/RfNIMT5ZM9C1Gl+fn7cd999zJ8/n88++wyHw8HNN99cpt2OHTvo06dPqfv69u2L1pqdO3eSm5vLkSNHiI2NLdWmd+/epW5v2rSJN954g4CAAOdXp06dAEhLS6uV17Rjxw5iYmLw8fFx3te1a1eCgoLYsWMHAImJiUyePJmrrrqK8ePHs379emfbqKgorr/+ejp37sxtt93GrFmzOHToUK1kE3WX7HeKOm/UqFH06NGDQ4cOMXz4cBo0aOCy5zIMg/Hjx3PfffeV2RYaGuqy5/2z4cOHc8MNN/DFF1/wzTffcOONN3LbbbfxwQcfYDab+fe//82mTZtITk5m6dKlPPPMM3zyyScMGjTIbRnFpUX2LESd16lTJ3r16sWGDRvKPRw1MjKy1KdvgHXr1qGUIjIyksaNG9OiRQtSUlJKtdmwYUOp29HR0ezYsYP27duX+QoICKiV1xMZGcnGjRs5d+6c876tW7dy6tQpOnfu7LyvefPmDB8+nPfee4+kpCQ+/PBDcnNzAVBKceWVVzJx4kTWr19P3759WbBgQa3kE3WT7FmIeuHLL7+ksLAQq9V6we3jxo2jR48ePPnkkzz88MMcOHCAxx57jHvuuYfWrVsD8Le//Y3nnnuOyy+/nJiYGP71r3+RnJxc6nFefvllBgwYwFNPPcX9999PYGAgaWlpfPLJJ8yZM4eGDRtWOnNOTg6pqaml7mvcuDFjxoxh1qxZPPjgg0ycOJGTJ0+SkJDAtddey7XXXgvAmDFjuOmmm+jYsSOFhYUsW7aMVq1aERgYSEpKCqtXr2bAgAE0b96ctLQ0tm3bxsiRI6vylor6xtOTJkK4wu8T3OX58wS31lp/9tlnukePHtrHx0fb7XY9evRonZ+f79xeXFysJ0yYoG02m/b399dDhw7VM2bMKPM469ev13FxcTogIED7+/vryy+/XD/xxBO6qKhIa135CW6gzNf111+vtdb6u+++09dee6328/PTQUFB+q677tLHjx939k9ISNARERHaz89PW61WfdNNN+nt27drrbXevn27vvHGG3VISIj28fHRrVu31mPHji01YS7Enymt5Up5QgghLk7mLIQQQlRIioUQQogKSbEQQghRISkWQgghKiTFQgghRIXq7HkWR48erXZfu91OVlZWLaapXZKvZiRfzUi+mvHmfGFhYeVukz0LIYQQFXLbnkVqaioLFizAMAzi4uIYPHhwqe2ffvopq1evxmw207hxYx555BGaNWsGwNq1a1m2bBkAQ4YMoV+/fu6KLYQQAjftWRiGQVJSEhMnTmTmzJls2LCBw4cPl2rTtm1bpkyZwrRp04iJieGDDz4AID8/nyVLljB58mQmT57MkiVLyM/Pd0dsIYQQv3FLsUhPTyc0NJSQkBAsFguxsbFs2rSpVJvOnTvj6+sLlFzhLCcnByjZI4mKinIu9xwVFVVmvRwhhBCu5ZZhqJycHGw2m/O2zWa76Nr+a9asoVu3bhfsa7VanYXkfMnJyc5F3aZMmYLdbq92XovFUqP+rib5akby1Yzkqxlvz1cerzsaav369ezbt48XX3yxSv3i4+OJj4933q7J0QbefLQCSL6aknw1I/lqxpvzefxoKKvVWuqC9dnZ2RdcKnrbtm0sX76cp59+2nmBmj/3zcnJKXeZaSGEEK7hlmIRHh5ORkYGmZmZOBwOUlJSiI6OLtVm//79zJ8/n6effpqgoCDn/d26dWPr1q3k5+eTn5/P1q1bnUNUtU0X5GF8+jFFe392yeMLIcSlyi3DUGazmREjRjBp0iQMw6B///60atWKRYsWER4eTnR0NB988AGFhYXMmDEDKNlVGz9+PAEBAQwdOpQJEyYAMGzYsFq74lgZyoT+18ec9fOD+MEVtxdCiHqizl7PorpncBe/8iQNGgdhPPFi7QaqRd485gmSr6YkX81Ivurz+JzFpUR1iKRoz3a0o8jTUYQQwmtIsfgTFREJ587BgXRPRxFCCK8hxeLPIjoBoNN2eDiIEEJ4DykWf6ICgzC3aofeI8VCCCF+J8XiAnw6dYP0nWij2NNRhBDCK0ixuIAGkV2h8AwcOuDpKEII4RWkWFyAT6fuAOg92z2cRAghvIMUiwsw25pBs1CZtxBCiN9IsSiH6hAJ6TvQhuHpKEII4XFSLMoT0Rny8yDjcMVthRCijpNiUQ7VIRIAnSbzFkIIIcWiPPYQaGIDmbcQQggpFuVRSqE6RKLTdlBH11oUQohKk2JxMRGRcDIHfj3m6SRCCOFRUiwu4o95CxmKEkLUb1IsLqZ5KwhoDD/LJLcQon6TYnERSin4bd5CCCHqMykWFVARkZB1HJ3zq6ejCCGEx0ixqMAf8xY7PZxECCE8R4pFRVq2hYb+cr6FEKJes7jriVJTU1mwYAGGYRAXF8fgwYNLbd+5cycLFy7k4MGDJCYmEhMT49z2wQcfsGXLFrTWdOnSheHDh5fMJ7iBMpmhfSeZtxBC1Gtu2bMwDIOkpCQmTpzIzJkz2bBhA4cPl15zyW63k5CQQO/evUvd//PPP/Pzzz8zbdo0pk+fzt69e9m5071DQioiEjIOoXNPuvV5hRDCW7ilWKSnpxMaGkpISAgWi4XY2Fg2bdpUqk1wcDBt2rQps8eglOLcuXM4HA6KioooLi4mKCjIHbH/yPDbvAXpMm8hhKif3DIMlZOTg81mc9622WykpaVVqm+HDh2IjIxk1KhRaK254YYbaNmyZZl2ycnJJCcnAzBlyhTsdnu181osllL9ddBVZPr44vvLXhoPuKXaj1tb/pzP20i+mpF8NSP5XMNtcxbVdezYMY4cOcLcuXMBeOWVV9i1axdXXHFFqXbx8fHEx8c7b2dlZVX7Oe12e9n+l3XkzLYfOFeDx60tF8znRSRfzUi+mpF81RcWFlbuNrcMQ1mtVrKzs523s7OzsVqtler7/fffExERgZ+fH35+fnTv3p09e/a4Kmq5VIfOcPgA+nS+259bCCE8zS3FIjw8nIyMDDIzM3E4HKSkpBAdHV2pvna7nV27dlFcXIzD4WDnzp20aNHCxYnLUh0iQWtI3+X25xZCCE9zyzCU2WxmxIgRTJo0CcMw6N+/P61atWLRokWEh4cTHR1Neno606ZNo6CggM2bN7N48WJmzJhBTEwM27dvZ+zYsQB069at0oWmVrXrAGYLes8OVFQv9z+/EEJ4kNvmLHr06EGPHj1K3XfnnXc6v2/fvr1zXuJ8JpOJUaNGuTxfRZSPL7SLkPMthBD1kpzBXQUqIhIOpqPPFno6ihBCuJUUiypQHSKhuBj2/ezpKEII4VZSLKoi/ApQJrSsEyWEqGekWFSBaugPrS9D75GLIQkh6hcpFlWkIiJh38/ooiJPRxFCCLeRYlFFqmMkOIrgQOWWKxFCiLpAikVVte8EIENRQoh6RYpFFamAxtCijZxvIYSoV6RYVIOKiIT03ejiYk9HEUIIt5BiUR0dIuHsGTi0z9NJhBDCLaRYVIOK+H3eQoaihBD1gxSLalBNbBDcXOYthBD1hhSLalIRkZC2E20Yno4ihBAuJ8WiujpEQkEeHP3F00mEEMLlpFhUk+rQGUCGooQQ9YIUi+qyBYPVDjLJLYSoB6RYVJNSChURiU7bgdba03GEEMKlpFjURIdIOHUCMjM8nUQIIVxKikUNqIjf5i1knSghRB0nxaImQltAYBDIJLcQoo6zuOuJUlNTWbBgAYZhEBcXx+DBg0tt37lzJwsXLuTgwYMkJiYSExPj3JaVlcXcuXPJzs4GYMKECQQHB7srermUUhARKWdyCyHqPLcUC8MwSEpK4tlnn8VmszFhwgSio6Np2bKls43dbichIYFVq1aV6T9nzhyGDBlCVFQUhYWFJX+kvYTqEInekoLOzkTZPF/AhBDCFdwyDJWenk5oaCghISFYLBZiY2PZtGlTqTbBwcG0adOmTCE4fPgwxcXFREVFAeDn54evr687YleKiogE5HwLIUTd5pY9i5ycHGw2m/O2zWYjLa1yV5o7evQojRo1Ytq0aWRmZtKlSxfuueceTKbSdS45OZnk5GQApkyZgt1ur3Zei8VS6f66aVN+bRSI3y97aTzo9mo/Z1VUJZ8nSL6akXw1I/lcw21zFtVlGAa7du3i9ddfx263M3PmTNauXct1111Xql18fDzx8fHO21lZWdV+TrvdXqX+Ovxyzvy0mXM1eM6qqGo+d5N8NSP5akbyVV9YWFi529wyDGW1Wp2T0wDZ2dlYrdZK923bti0hISGYzWauvPJK9u3zrutIqA6RcOwIOveEp6MIIYRLuKVYhIeHk5GRQWZmJg6Hg5SUFKKjoyvVt3379pw+fZrc3FwAtm/fXmpi3Bv8Pm9B2k7PBhFCCBdxyzCU2WxmxIgRTJo0CcMw6N+/P61atWLRokWEh4cTHR1Neno606ZNo6CggM2bN7N48WJmzJiByWTivvvu4+WXX0ZrzWWXXVZquMkrtA4HH1/0nh2ontd4Oo0QQtQ6pevowkZHjx6tdt/qjCkWz3gO8nIxvzCr2s9bWd485gmSr6YkX81Ivurz+JxFfaA6RMKRA+iCfE9HEUKIWifFopaoiM6gNaTLvIUQou6RYlFb2kWAxSJLfwgh6iQpFrVE+fhCuw5yJrcQok6SYlGLVERnOJiOLjzj6ShCCFGrpFjUItUhEgwD9u72dBQhhKhVUixqU/jl0NAf4/PFaKPY02mEEKLWSLGoRcqvIerOv8KeHeiv/+XpOEIIUWukWNQyFXsddI9Br3gffXi/p+MIIUStkGJRy5RSmO57FPwDMP45A11U5OlIQghRY1IsXEAFBmF64DE4chC98kNPxxFCiBqTYuEiKqoXqs8N6K+Wo/ds93QcIYSoESkWLqRuHw72EIx33kCfOe3pOEIIUW1SLFxI+TXENPIpyMlCfzzf03GEEKLapFi4mAq/HHXTMHTKavSW7zwdRwghqkWKhRuoQX+BNu0x3n8LfUouvSqEuPRIsXADZbFgGvkknC3EWDibOnq9KSFEHSbFwk1U81aooQ/ATz+g/+9LT8cRQogqkWLhRqr/QLiiK3pREjqz+pd9FUIId5Ni4UbKZML04BNgsWAkzUQXy2KDQohLg9uKRWpqKk888QSPPfYYK1asKLN9586djB8/nr/85S9s3LixzPbTp08zevRokpKS3BHXZZTVjrrnEdj3M/qLpZ6OI4QQleKWYmEYBklJSUycOJGZM2eyYcMGDh8+XKqN3W4nISGB3r17X/AxFi1axBVXXOGOuC5nurIP6so+6FX/iz6Y7uk4QghRIbcUi/T0dEJDQwkJCcFisRAbG8umTZtKtQkODqZNmzYopcr037dvH6dOnaJr167uiOsW6u7RENikZLHBc2c9HUcIIS7K4o4nycnJwWazOW/bbDbS0tIq1dcwDN577z0ee+wxfvrpp3LbJScnk5ycDMCUKVOw2+3VzmuxWGrUv1Lsds4+8RwnX0rE9/PFNH7oyUp3dUu+GpB8NSP5akbyuYZbikVNfPXVV3Tv3r1UsbmQ+Ph44uPjnbezsrKq/Zx2u71G/Sut5WWouJs589knnO3QGdWpe6W6uS1fNUm+mpF8NSP5qi8sLKzcbW4pFlarlezsbOft7OxsrFZrpfru2bOHXbt28dVXX1FYWIjD4cDPz4977rnHVXHdSg25H73jR4wFb2J6cTaqUYCnIwkhRBluKRbh4eFkZGSQmZmJ1WolJSWFxx9/vFJ9z2+3du1a9u7dW2cKBYDy8cX00FMYr45DfzQX9dexno4khBBluKVYmM1mRowYwaRJkzAMg/79+9OqVSsWLVpEeHg40dHRpKenM23aNAoKCti8eTOLFy9mxowZ7ojncapNe9Sgv6BXfogR1QvTVX09HUkIIUpRuo4uVHT0aPXPkPbEmKIuLsZ4/Rk4dhjTC2+irM3KbevNY54g+WpK8tWM5Ku+i81ZyBncXkKZzSWLDRYXY7z9KvpsoacjCSGEU6WLxfbt28nMzATgxIkTzJkzh7fffpuTJ0+6LFx9o4LDMP11LPyyDyNpBtowPB1JCCGAKhSLpKQkTKaS5u+99x7FxcUopZg3b57LwtVHquuVqDtGwI8b0UsXejqOEEIAVZjgzsnJwW63U1xczNatW3n77bexWCw8/PDDrsxXL6m4m+H4UfRXyzFCmmPqc4OnIwkh6rlKF4uGDRty8uRJDh06RMuWLfHz88PhcOBwOFyZr15SSsFf/orOOo7+cC7aHlLpE/aEEMIVKj0MdcMNNzBhwgTefPNNrr/+egB2795NixYtXBauPlNmM6ZR46B5K4y5r6GP/OLpSEKIeqzSexaDBw/myiuvxGQyERoaCpScmT169GiXhavvVEN/TI89j/HqWIzZL2OaOBXVuKmnYwkh6qEqHTobFhbmLBTbt2/n5MmTtG7d2iXBRAlla4ZpzLOQdxJjziRZoVYI4RGVLhYvvPACu3fvBmDFihXMmjWLWbNmsWzZMpeFEyVU2whMI/8GB9LQ77whh9QKIdyu0sXi0KFDdOjQAYDVq1fzwgsvMGnSJL7++muXhRN/UD2uRg19EL15AwUf/cPTcYQQ9Uyl5yx+XxXk2LFjALRs2RKAgoICF8QSF6IGDIbjRyhY+h4qsAmma+Ir7iSEELWg0sWiY8eOvPPOO5w4cYJevXoBJYUjMDDQZeFEaUopuHs0DXJPcO79t9C2YNTlUZ6OJYSoByo9DPXoo4/i7+9PmzZtuOOOO4CSxfpuuukml4UTZSmLhaBxkyA4DON/XkUfO1xxJyGEqKFK71kEBgZy9913l7qvR48etR5IVMzUKADT489jTB6L8ebLmCZMQwU29nQsIUQdVuli4XA4WLZsGevXr+fEiRM0bdqUPn36MGTIECwWr786a52j7CGYHv1vjOnPYrw9CdNTf0c1aODpWEKIOqrSf+U/+OAD9u7dy1//+leaNWvGr7/+ytKlSzl9+jQPPvigCyOK8qjwyzGNSMSY9zr63TfhoadK5jWEEKKWVbpYbNy4kalTpzontMPCwmjXrh3jxo2TYuFBKro3KjMDvfx9CGmOuuXuijsJIUQVVfnQWeF91I3DSlapXfUxRnAYpph+no4khKhjKl0srr76al577TWGDRvmvCzg0qVLufrqq12ZT1SCUgruS0BnZ6IXvolu3ATVqZunYwkh6pBKHzp777330qVLF5KSknjmmWd45513iIyMlMltL6EsDTA9MgFCWmC8+TJGyhpPRxJC1CGV/ktvsVi48847ufPOO533nTt3jvvuu4977723wv6pqaksWLAAwzCIi4tj8ODBpbbv3LmThQsXcvDgQRITE4mJiQHgwIEDzJ8/nzNnzmAymRgyZAixsbGVjV2vqEYBmJ6egjF3CnrBGxhZx1A33yWT3kKIGqvRbkFl/wgZhkFSUhLPPvssNpuNCRMmEB0d7VwyBMBut5OQkMCqVatK9fXx8WHMmDE0b96cnJwcnnnmGbp27UqjRo1qEr3OUv6NMD3+AvqDt9CrPoZfj8MDY1AWOaxWCFF9bhlDSk9PJzQ0lJCQEABiY2PZtGlTqWIRHBwMlC1AYWFhzu+tVitBQUHk5uZKsbgIZbHAA4+DPRS98kP0iSxMj0xANQrwdDQhxCWqwmKxffv2crdV9pKqOTk52Gw2522bzUZaWlql+p4vPT0dh8PhLDrnS05OJjk5GYApU6Zgt9ur/Pi/s1gsNervapXO9+CjnGnXntw5k1HTJtL0uemYg5t7Tz4PkXw1I/lqxtvzlafCYvE///M/F93urhd94sQJZs+ezaOPPorJVHZePj4+nvj4P1ZhzcrKqvZz/X60l7eqUr7InpgSX6L47clkjR2B6bHnUO06eE8+D5B8NSP5asab850/kvNnFRaLt956q8YBrFYr2dnZztvZ2dlYrdZK9z99+jRTpkzhrrvucl5TQ1Se6tgZ0zOvY7z5Esa0iZgeGovqHuPpWEKIS0iVLqtaXeHh4WRkZJCZmYnD4SAlJYXo6OhK9XU4HEybNo0+ffo4j5ASVaeat8Q0YSq0aIvxP69iJP/L05GEEJcQt0xwm81mRowYwaRJkzAMg/79+9OqVSsWLVpEeHg40dHRpKenM23aNAoKCti8eTOLFy9mxowZpKSksGvXLvLy8li7di1Qslx627Zt3RG9TlGNm2D62ySMpOnoRf/EyDqOumMEymT2dDQhhJdTuo6u43H06NFq9/XmMUWoeT5tFKOXvIv+eiV0vRLTX8eifP28Jp+rSb6akXw14835LjZn4ZZhKOFdlMmM6Y6RqLsfhm0/YEydiD51wtOxhBBeTIpFPWbqPxDTo/8NGYcwXh2HPvKLpyMJIbyUFIt6TnXthenpV8FRhPHaePSurZ6OJITwQlIsBKpNe0wTpkFTG8asFzH+s87TkYQQXkaKhQBA2ZphGv8ahF+BTpqJ8d03no4khPAiUiyE0++LENKxc8mqtRtWezqSEMJLSLEQpShfX0xjnoMruqIXvonx7deejiSE8AJSLEQZyte35CipTt3QC2djrP/C05GEEB4mxUJckPL5rWB0iUa//zbG2s89HUkI4UFSLES5VAOfkku1RvVCfzgX45vPPB1JCOEhUizERakGDTA98gx0uwr90TyM1asq7iSEqHOkWIgKKUsDTA8/Dd1j0B/Px/h6pacjCSHcTIqFqBRlaYBp1NPQMxa9OAnjy2WejiSEcCMpFqLSlMVScuGkXteil7yL8e8lno4khHATt1zPQtQdymKBkU+BMqGXvYdhGJgG3uHpWEIIF5NiIapMmc0wMhFMCr3iA4ziYky33OXpWEIIF5JiIapFmcww/ImSPYxV/4uhNeqWu1BKeTqaEMIFpFiIalMmMzz4OJhM6E8/BsOAwfd4OpYQwgWkWIgaUSYT3D+mpGB8vhiMYvSopzwdSwhRy6RYiBpTJhPcm1BSML5YSm5hAfq2+1H+AZ6OJoSoJW4rFqmpqSxYsADDMIiLi2Pw4MGltu/cuZOFCxdy8OBBEhMTiYmJcW5bu3Yty5aVHNc/ZMgQ+vXr567YopKUyQT3PAKNm1D42SeQugnTA2NQnXt6OpoQoha45TwLwzBISkpi4sSJzJw5kw0bNnD48OFSbex2OwkJCfTu3bvU/fn5+SxZsoTJkyczefJklixZQn5+vjtiiypSSmG65W6sU+ZBQ3+MWS9hvDcHfea0p6MJIWrILcUiPT2d0NBQQkJCsFgsxMbGsmnTplJtgoODadOmTZmjaVJTU4mKiiIgIICAgACioqJITU11R2xRTQ0iOmF6bibqhqHob5MxXnxMru0txCXOLcNQOTk52Gw2522bzUZaWlq1+lqtVnJycsq0S05OJjk5GYApU6Zgt9urnddisdSov6tdCvmaNQ+Dh//Gub4DyJ09ieIZz9HwhiEE3J+AqaG/x/N5+/sn+apP8rlGnZngjo+PJz4+3nk7Kyur2o9lt9tr1N/VLql89ubo/56OWv4BZ75czpnNKZgefBzVobN35PNCkq9mJF/1hYWFlbvNLcNQVquV7Oxs5+3s7GysVmu1+ubk5FS6r/AOyscX050jMY2dDIAx7b8xPp6PPnvWw8mEEJXllmIRHh5ORkYGmZmZOBwOUlJSiI6OrlTfbt26sXXrVvLz88nPz2fr1q1069bNxYmFK6gOkZheeBPV70b06lUYLz+BTt/l6VhCiEpwyzCU2WxmxIgRTJo0CcMw6N+/P61atWLRokWEh4cTHR1Neno606ZNo6CggM2bN7N48WJmzJhBQEAAQ4cOZcKECQAMGzaMgAA5fv9SpXz9UHePRne/GmPhbIzXJ6AG3Iq69R5UAx9PxxNClENprbWnQ7jC0aNHq93Xm8cUoe7k02dOo5csQK//Epq3wtw4iXwAABqRSURBVDQ8EdUuwmvyeYrkqxnJV30en7MQ4kJUQ39M9z2K6YkX4cxpjCnjMJZ/gC4q8nQ0IcSfSLEQHqc698D00mxUTH/054sx/v4keu9uT8cSQpxHioXwCso/ANPwJzA9/jwUnsZ4bTzGR/PQhXL2txDeQIqF8CqqSzSml+agrhuEXvs5xvNj0Fs3VdxRCOFSUiyE11F+/pj+8ldM418rWWNqzisY815H557wdDQh6i0pFsJrqfDLS9aYuvUedOpGjOcexdiQTB09gE8IrybFQng1ZWmAadCdmJ5/E1q0Rr/7JsaM59CZ1T80WghRdVIsxCVBNW+Jaexk1L0JcDAd48XHMb5Yii4u9nQ0IeoFKRbikqFMJkx9b8D00lvQuQd66UKMSU+hD6Z7OpoQdZ4UC3HJUU1tmBMmYnrkGcg9hTFpLMYnC2RhQiFcqM4sUS7qH9UjFtPlUeilC9FfLUdvScF0XwKqU3dPRxOizpE9C3FJU/4BJUuGjJsMZgvGzBconvUiOvU/aEPmM4SoLbJnIeoE1aEzphdmob9agf7mc4y3JoHVjrp2AKr3AFQTuQaKEDUhxULUGaqBD2rgHejrh8C27zHWfYFe+RH600XQ9SpM/W6Ejl08HVOIS5IUC1HnKIsFesRi7hGLzjyKXvclOiUZY0sKBIdRcNNQdNerUAGNPR1ViEuGFAtRp6ngMNTtw9GD70Fv3oBe9wX5784Gy1xUdG9Uvxvhso4opTwdVQivJsVC1AuqgQ8qpj/E9Cco/yQnVn6M3vgNeuM30LItqu8NqJh+KD9/T0cVwivJ0VCi3mnQtj2me0Zjmvou6r4EUAr94VyMscMx3n8bfeyIpyMK4XVkz0LUW8qvIarPDehrr4cDaei1/0Z/twa94WtUnxtQN9+FCpR5DSFAioUQJfMV7Tqg2nVAD30A/a+PSgrHxrWogbejrrsZ1aCBp2MK4VFuKxapqaksWLAAwzCIi4tj8ODBpbYXFRUxZ84c9u3bR2BgIImJiQQHB+NwOJg7dy779+/HMAz69OnDbbfd5q7Yop5RjZug7k1AXzcIY8m76CXvor/5HDX0QVT0NTIRLuott8xZGIZBUlISEydOZObMmWzYsIHDhw+XarNmzRoaNWrE7NmzGThwIB9++CEAGzduxOFwMH36dKZMmUJycjKZmZnuiC3qMRXWGvPjz2N68iXwa4j+x+sYr42Xa4OLesstxSI9PZ3Q0FBCQkKwWCzExsayaVPpS2X+8MMP9OvXD4CYmBi2b9/uvMhNYWEhxcXFnDt3DovFgr+/HLEi3EN16o7p+TdQ94+BrOMYU54uuWrfr8c8HU0It3LLMFROTg42m81522azkZaWVm4bs9mMv78/eXl5xMTE8MMPPzBq1CjOnTvHAw88QEBAQJnnSE5OJjk5GYApU6Zgt9urnddisdSov6tJvpqpVr7b7sa4YTCnV3xEwYoP0an/wX/g7TS6/QFMjQI9n8+NJF/NeHu+8nj9BHd6ejomk4l58+ZRUFDA888/T5cuXQgJCSnVLj4+nvj4eOftrKysaj+n3W6vUX9Xk3w1U6N8/zUYU/S16BUfcPpf/8vp1atQg+4qOU/DUju/TnX6/XMDyVd9YWFh5W5zS7GwWq1kZ2c7b2dnZ2O1Wi/YxmazUVxczOnTpwkMDOTbb7+lW7duWCwWgoKC6NixI3v37i1TLIRwF9XUhhr+BDpuUMl1ND7+B/qbzzANewC6XlXuJLg2DDhTAHm5kH8K8nPRebmQV/I9ebnogjzy2ndE9+qLahbq5lcmRPncUizCw8PJyMggMzMTq9VKSkoKjz/+eKk2PXv2ZO3atXTo0IGNGzcSGRmJUgq73c727dvp06cPhYWFpKWlMXDgQHfEFuKiVOtwTE+9Ats2YSxZgPHWZOjYBRURWVIM8nLR+ecVg/xcMIwLP5iPLwQGQUN/Tq/8EVZ8BJ17Yup/E0R2R5nM7n1xQvyJ0r/PIrvYli1bWLhwIYZh0L9/f4YMGcKiRYsIDw8nOjqac+fOMWfOHPbv309AQACJiYmEhIRQWFjI22+/zeHDh9Fa079/f2655ZYKn+/o0aPVzurNu4kg+WrKFfm0w4H+vy/R//pfKMiDRgEQEAQBjSGwccmihYFBENgYAhqjAn7/vqSN8vV1PlZTDLJX/i96/ZeQexLsISXDXNf8l1ecJFgf/39rkzfnu9gwlNuKhbtJsfCc+pzv9wsu1WRP4Pd82lGE/vE/6LWfwZ4dYGmA6tUb1e+mkpMIPXTOR33+/60N3pzP43MWQtQXtTlcpH4rDvTqjT5y8LflSL5Bf/cNtA5H9bsRdWXfUnslQriKLCQoxCVAtWhTsvjhtAWoe0aDowj93hyMpx/EWJQkix8Kl5M9CyEuIcrPH9XvJnTfGyFtJ3rt5+hvPkUnr4RO3TD1uwk695S1rEStk2IhxCVIKQUdIlEdItGnTqD/7yv0+i8x3p4MSkGQtWRi3B4Cv305v29qlaOrRJVJsRDiEqeCmqIG3Ym+cRhs34w+uBeyjqOzj6P3/AT/WQta4zySxWwBq/2PAmIL/uP7ZiEQ2EQWTBRlSLEQoo5QZjN0vRLV9cpS92tHEeT8WlJAso5D1nHIykRnHUen/qfkPBAoXUwaN3F+qaCmv31f8u+51m3QWpXcbugvhaWekGIhRB2nLA0gOAyCw7jQn3VdeAayM/8oJidzIPckOvcEnMpB/7IP8k46Tyg8cX5nSwNwFpMmqMZNSvZSmreC5q2gWWhJEROXPCkWQtRzyq8htGgDLdpcsJjAb0uVFORB7kkaY3Dq0EHIPVFy0uCpk+jck5Cdid73M+Sd+mMvxWKBkBao0JYQVlJAVPNWJffJJPwlRYqFEKJCymT67Qz0IHztdkwt2pXbVheehowj6IxfIOMwOuMQ+tA+2PIdaKOkkCgTNAuF5i1RYa0gtNVv/7YsKV7C69SbYqG1prCwEMMwKhxjPX78OGfPnnVTsqq7WD6tNSaTCT8/PxlLFh6h/PyhXQSqXUSp+3XROTh+BH300G9F5Ldisn0LFDv+2Btp1Q51ZR9Urz4oWzO35xcXVm+KRWFhIQ0aNMBSiWWkLRYLZi8eZ60on8PhoLCwkIYN5ROa8B6qgQ+0bIdqWXqvRDsckHUMjh5CH/0FvW0TeulC9NKF0L5TSeGIvgYVGOSh5ALqUbEwDKNShaIusFgsXr1nJMT5lMUCoS1LhqB6XA2D7kRnZqA3/R/6+/Xoj+aiP/4HXNG1ZHmT7jGohnK1THerH389od4NydS31yvqFhXcHDXwDhh4B/rwgZKi8f169II30B/4QJdoTFf1gS7RJXsswuXqTbEQQlyaVMu2qJZt0bfdB/t+Likam/4PY0tKyXke3WJQV/Yp2fOoxPCxLjoH+XnOa4zo/Dwo+O16I/l50Kw5KvY62Xv5EykWbpKTk8Odd94JwK+//orZbHZeLfCzzz7Dx6f8T0dbt25lyZIlvPLKK27JKoQ3UkpB+OWo8MvRd4yEn7eVFI4tG9HfrYHAIFT0NZxufznGsYySP/4FeSUXoPq9OBTkwdnC8p/E1w/OFqJXvI+6Jh513SBUcHP3vUgvVm+uZ3H69Gn8/Sv3ScFiseBwOFwRC4Dp06fTqFEjRo8e7bzP4XBUek6lMvmq8nprmzev1w+Sr6a8LZ8uOgc/bS4pHNs2QdG5kg3+jaBRYMkFqAIaowICoVFjCAj87fYf35e0C0RZGqAPpKFXr0Jv+haMYojqhSnuZrg8qlaGd73t/TufXM/iT4yP56MP7S9/u1JUtYaqVu0w/eWvVeqTmJiIr68vO3bsIDo6mltvvZXnn3+es2fP4ufnx4wZM2jfvj0pKSnMnTuX9957j+nTp3P06FEOHjzIkSNHeOihhxg5cmSVnleIukQ18IEeV6N6XI0+W4i1kT85Z4uqfea4ahuBGvkUeuiD6HX/Rq/7AmPr9yUnLcbdjLqqL8qn/l1DpF4WC2+SkZHBypUrMZvN5OXlsXz5ciwWC+vXr+e1115j/vz5Zfqkp6ezePFiCgoKuPbaa7n//vtpIGfDCoHy9cPcxIqqhU/uqokVdes96JtuL9lrSV6Ffm8OetlCVJ8bUf1vRDWx1ULqS0O9LBYV7QG4ehjqfIMGDXKeM5Gbm0tiYiL79+9HKUVRUdEF+8THx+Pr64uvry92u51ff/31oruPQojqUw18UNfEo2PjYM92jORV6H9/gv5yKapnb1T8zah2HTwd0+XcVixSU1NZsGABhmEQFxfH4MGDS20vKipizpw57Nu3j8DAQBITEwkODgbg4MGD/OMf/+DMmTMopXj11VcvOiF8KTl/XmHq1KnExsaSlJTEoUOHGDZs2AX7+J53GU2z2UxxcbHLcwpR3ymloGMXzB27oH89hl7zGXrD1+jv15VMvMfdjOp+dcl5I3WQW16VYRgkJSXx7LPPYrPZmDBhAtHR0bRs2dLZZs2aNTRq1IjZs2ezYcMGPvzwQ5588kmKi4uZPXs2Y8aMoW3btuTl5dXZk+vy8vIIDQ0FYPHixR5OI4Qoj2oWirpzJPrWu9Ab1qDXrEL/Yyq6qR11RdeSBRTNZjCZS5Z8N5tK/jWZKWjcGKOw8I/tJnPJ9+aStspiAYsPNGgAPj7QwOeP2w1K/+vOi1i55a9ueno6oaGhhISEABAbG8umTZtKFYsffviB22+/HYCYmBjeeecdtNZs3bqV1q1b07ZtWwACAwPdEdkjHnnkERITE5k1axZxcXGejiOEqIDy80fFDUL3vwl+2oyx5lP0rq1Q7Cg5kqr4/K+Soe38Ch6zSofWmM2/FY7fiojFB9UmHNOocdV9SeVyS7HIycnBZvtjIshms5GWllZuG7PZjL+/P3l5eWRkZKCUYtKkSeTm5hIbG8utt97qjtgu87e//e2C90dHR/Ptt986b48fPx4oKa6xsbHOvufPqaxZs8bFaYUQFVEmE3Tthblrr4u200Yx9qZNyTp+vGwxMX4rKA4HFBWVHALsOOf8Xv9+n/Or6I9/HefdZwt2yWv0+vGc4uJidu/ezauvvoqvry8vv/wyl112GV26dCnVLjk5meTkZACmTJmC3W4vtf348eNVGr7y9qGuivL9PvntCRaLxWPPXRmSr2YkX81YLBaatWzl6RhV5pa/iFarlezsbOft7Oxs59nLf25js9koLi7m9OnTBAYGYrPZuOKKK2jcuDEA3bt3Z//+/WWKRXx8PPHx8c7bfz7p5ezZs5VeSdadR0NVR2XynT171mMn/njzSUcg+WpK8tWMN+e72FGVJncECA8PJyMjg8zMTBwOBykpKURHR5dq07NnT9auXQvAxo0biYyMRClF165dOXToEGfPnqW4uJhdu3aVmuuorDp6onq56tvrFUK4llv2LMxmMyNGjGDSpEkYhkH//v1p1aoVixYtIjw8nOjoaK677jrmzJnDY489RkBAAImJiQAEBAQwcOBAJkyYgFKK7t2706NHjypnMJlMVVpS41LmcDgwmdzyOUAIUU/Um7WhqnKlPF9fX6++HsTF8nnDlfK8eTcbJF9NSb6a8eZ8sjYUJSfUVPbKcd78nwnen08IUffIWIUQQogKSbEQQghRISkWQgghKlRnJ7iFEELUHtmzuIBnnnnG0xEuSvLVjOSrGclXM96erzxSLIQQQlRIioUQQogKmV988cUXPR3CG1122WWejnBRkq9mJF/NSL6a8fZ8FyIT3EIIISokw1BCCCEqJMVCCCFEherN2lB/lpqayoIFCzAMg7i4OAYPHlxqe1FREXPmzGHfvn0EBgaSmJhIcLBrrkD1Z1lZWbz11lucPHkSpRTx8fHcdNNNpdrs2LGD119/3ZnpqquuYtiwYW7J97tHH30UPz8/TCYTZrOZKVOmlNqutWbBggX8+OOP+Pr6kpCQ4Lax2qNHjzJz5kzn7czMTO644w4GDhzovM/d7+Hbb7/Nli1bCAoKYvr06QDk5+czc+ZMfv31V5o1a8aTTz5JQEBAmb5r165l2bJlAAwZMoR+/fq5Jd/777/P5s2bsVgshISEkJCQQKNGjcr0rehnwVX5Fi9ezOrVq53Xu7nrrrsuuCp1Rb/vrso3c+ZM56Kmp0+fxt/fn6lTp5bp6473r8Z0PVRcXKzHjBmjjx07pouKivTYsWP1oUOHSrX54osv9Lx587TWWn/77bd6xowZbsuXk5Oj9+7dq7XW+vTp0/rxxx8vk2/79u361VdfdVumC0lISNCnTp0qd/vmzZv1pEmTtGEY+ueff9YTJkxwY7o/FBcX64ceekhnZmaWut/d7+GOHTv03r179VNPPeW87/3339fLly/XWmu9fPly/f7775fpl5eXpx999FGdl5dX6nt35EtNTdUOh8OZ9UL5tK74Z8FV+RYtWqRXrlx50X6V+X13Vb7zLVy4UH/yyScX3OaO96+m6uUwVHp6OqGhoYSEhGCxWIiNjWXTpk2l2vzwww/OT28xMTFs377dbRcUatq0qfMTeMOGDWnRogU5OTluee7a9MMPP9CnTx+UUnTo0IGCggJOnDjh9hw//fQToaGhNGvWzO3Pfb5OnTqV2WvYtGkTffv2BaBv375lfg6h5FNxVFQUAQEBBAQEEBUVRWpqqlvyde3a1XmFyQ4dOnj05/BC+SqjMr/vrs6ntea7777jmmuuqfXndZd6OQyVk5ODzWZz3rbZbKSlpZXbxmw24+/vT15ennN3110yMzPZv38/7du3L7Ntz549jBs3jqZNm3LffffRqpX7r+s7adIkAP7rv/6r1GVtoeQ9PP9ayDabjZycHJo2berWjBs2bCj3l9TT7+GpU6ec70eTJk04depUmTZ//nm1Wq0e+aO9Zs0aYmNjy91+sZ8FV/ryyy9Zv349l112Gffff3+ZP9iV+X13tV27dhEUFETz5s3LbeOp96+y6mWxuFQUFhYyffp0HnzwQfz9/Utta9euHW+//TZ+fn5s2bKFqVOn8uabb7o13yuvvILVauXUqVP8/e9/JywsjE6dOrk1Q0UcDgebN2/m7rvvLrPNG97D8ymlPHbBqoosW7YMs9nMtddee8HtnvpZGDBggHOeadGiRbz33nskJCS4/Hmr6mIfWODS+F2ql8NQVquV7Oxs5+3s7GysVmu5bYqLizl9+jSBgYFuy+hwOJg+fTrXXnstV111VZnt/v7++Pn5AdCjRw+Ki4vJzc11Wz7A+Z4FBQXRq1cv0tPTy2w//yJNF3qfXe3HH3+kXbt2NGnSpMw2b3gPg4KCnENzJ06cuOCe659/XnNyctz6Pq5du5bNmzfz+OOPl1vMKvpZcJUmTZpgMpkwmUzExcWxd+/eC2ar6PfdlYqLi/n+++8vulfmqfevKuplsQgPDycjI4PMzEwcDgcpKSlER0eXatOzZ0/Wrl0LwMaNG4mMjHTbpz6tNXPnzqVFixYMGjTogm1OnjzpnENJT0/HMAy3FrPCwkLOnDnj/H7btm20bt26VJvo6GjWr1+P1po9e/bg7+/vVUNQnn4PoeQ9WrduHQDr1q2jV69eZdp069aNrVu3kp+fT35+Plu3bqVbt25uyZeamsrKlSsZP348vr6+F2xTmZ8FVzl/Duz777+/4DBiZX7fXemnn34iLCys1FDY+Tz5/lVFvT2De8uWLSxcuBDDMOjfvz9Dhgxh0aJFhIeHEx0dzblz55gzZw779+8nICCAxMREQkJC3JJt9+7dPP/887Ru3dpZoO666y7np/QBAwbwxRdf8NVXX2E2m/Hx8eH++++nY8eObskHcPz4caZNmwaUfHLq3bs3Q4YM4auvvnJm1FqTlJTE1q1b8fHxISEhgfDwcLdlLCwsJCEhgTlz5jiH8c7P5+738I033mDnzp3k5eURFBTEHXfcQa9evZg5cyZZWVmlDp3du3cvX3/9NaNHjwZK5guWL18OlBw6279/f7fkW758OQ6HwzkPEBERwahRo8jJyWHevHlMmDCh3J8Fd+TbsWMHBw4cQClFs2bNGDVqFE2bNi2VDy78++6OfNdddx1vvfUWERERDBgwwNnWE+9fTdXbYiGEEKLy6uUwlBBCiKqRYiGEEKJCUiyEEEJUSIqFEEKICkmxEEIIUSEpFkJ4sTvuuINjx455OoYQstyHEFXx6KOPcvLkSUymPz5n9evXj5EjR3owlRCuJ8VCiCoaP348UVFRno4hhFtJsRCiFqxdu5bVq1fTtm1b1q9fT9OmTRk5ciRdunQBSs7YnT9/Prt37yYgIIBbb73VubKoYRisWLGCb775hlOnTtG8eXPGjRvnXLF327ZtTJ48mdzcXHr37s3IkSO9dsFBUXdJsRCilqSlpXHVVVeRlJTE999/z7Rp03jrrbcICAhg1qxZtGrVinnz5nH06FFeeeUVQkND6dy5M59++ikbNmxgwoQJNG/enIMHD5Zah2nLli28+uqrnDlzhvHjxxMdHe22taGE+J0UCyGqaOrUqc4LAgHce++9WCwWgoKCGDhwIEopYmNjWbVqFVu2bKFTp07s3r2bZ555Bh8fH9q2bUtcXBzr1q2jc+fOrF69mnvvvZewsDAA2rZtW+r5Bg8eTKNGjWjUqBGRkZEcOHBAioVwOykWQlTRuHHjysxZrF27FqvVWmp4qFmzZuTk5HDixAkCAgJo2LChc5vdbncup52dnX3RRSrPX17d19eXwsLC2nopQlSaHDorRC3JyckpdendrKwsrFYrTZs2JT8/37kM9fnboOTKbcePH3d7XiGqQoqFELXk1KlT/Pvf/8bhcPDdd99x5MgRunfvjt1up2PHjnz00UecO3eOgwcP8s033zivOhcXF8eiRYvIyMhAa83BgwfJy8vz8KsRojQZhhKiil577bVS51lERUXRq1cvIiIiyMjIYOTIkTRp0oSnnnrKeTGlJ554gvnz5/Pwww8TEBDA7bff7hzKGjRoEEVFRfz9738nLy+PFi1aMHbsWI+8NiHKI9ezEKIW/H7o7CuvvOLpKEK4hAxDCSGEqJAUCyGEEBWSYSghhBAVkj0LIYQQFZJiIYQQokJSLIQQQlRIioUQQogKSbEQQghRof8HPmIugkJUOvgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.style.use('ggplot')\n",
    "plt.plot(model_batch_hist.history['loss'])\n",
    "\n",
    "plt.title('Model Loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['Train'], loc='lower left')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WL3WSOAGUAvq"
   },
   "source": [
    "The results are not entirely satisfactory, but let's save this model and try a different approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DjRshqbRUAvq"
   },
   "outputs": [],
   "source": [
    "from keras.models import load_model\n",
    "\n",
    "model_batch.save('models/LSTM-Batch-Model.h5')  # save model\n",
    "del model_batch  # delete existing model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "OmOr8Pjfnp7L"
   },
   "outputs": [],
   "source": [
    "# returns a compiled model, identical to the previous one\n",
    "model_batch = load_model('models/LSTM-Batch-Model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xNbEOGJBUAvr"
   },
   "source": [
    "Let's check if everything is correct:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CYFNvYmlUAvr",
    "outputId": "e7eb2d3d-737e-4ace-ff23-a890ab4559d0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44/44 [==============================] - 5s 92ms/step - loss: 0.1080 - binary_accuracy: 0.8642\n",
      "\n",
      "Model Accuracy: 86.42%\n"
     ]
    }
   ],
   "source": [
    "score_batch = model_batch.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\", str(round(score_batch[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Qo2aS8F7UAvs"
   },
   "source": [
    "<h2 align=\"center\">LSTM Model (Mini-Batch Gradient Descent)</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3VbAeeOOUAvs"
   },
   "source": [
    "Mini-batch gradient descent is a variation of the gradient descent algorithm that splits the training dataset into small batches that are used to calculate model error and update model coefficients.\n",
    "\n",
    "Mini batch algorithm is the most favorable and widely used algorithm that makes precise and faster results using a batch of `m` training examples. In mini batch algorithm rather than using  the complete data set, in every iteration we use a set of `m` training examples called batch to compute the gradient of the cost function. Common mini-batch sizes range between 32 and 256."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xu8ESwoCUAvt"
   },
   "source": [
    "<h3 align=\"center\">Setting Up and Creating the Network</h3>\n",
    "\n",
    "We will use an LSTM network fit for 20 epochs and with batch size = 128.\n",
    "\n",
    "A mean squared error optimization function is used for this problem with the efficient ADAM optimization algorithm.\n",
    "\n",
    "So now let's configure and create the network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DeoBOnWMUAvt",
    "outputId": "e1dadf0a-b701-4f43-8e3d-2d1542cbbb37"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " lstm (LSTM)                 (None, 128)               219648    \n",
      "                                                                 \n",
      " dense (Dense)               (None, 1)                 129       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 219,777\n",
      "Trainable params: 219,777\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# configure network\n",
    "n_neurons = 128\n",
    "n_epoch = 20\n",
    "n_batch = 128\n",
    "\n",
    "# design network\n",
    "model_mini = Sequential()\n",
    "model_mini.add(LSTM(n_neurons, input_shape=(x.shape[1], x.shape[2])))\n",
    "model_mini.add(Dense(1, activation='sigmoid'))\n",
    "model_mini.compile(loss='mean_squared_error', optimizer='adam', metrics=['binary_accuracy'])\n",
    "\n",
    "print(model_mini.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "n6-fVhTrUAvu",
    "outputId": "4ac25462-5d24-4348-f1b5-ef1a1bb0d170"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "132/132 [==============================] - 30s 211ms/step - loss: 0.1943 - binary_accuracy: 0.6979\n",
      "Epoch 2/20\n",
      "132/132 [==============================] - 29s 216ms/step - loss: 0.1313 - binary_accuracy: 0.8207\n",
      "Epoch 3/20\n",
      "132/132 [==============================] - 42s 323ms/step - loss: 0.1052 - binary_accuracy: 0.8640\n",
      "Epoch 4/20\n",
      "132/132 [==============================] - 37s 282ms/step - loss: 0.0825 - binary_accuracy: 0.8991\n",
      "Epoch 5/20\n",
      "132/132 [==============================] - 35s 268ms/step - loss: 0.0666 - binary_accuracy: 0.9215\n",
      "Epoch 6/20\n",
      "132/132 [==============================] - 31s 235ms/step - loss: 0.0512 - binary_accuracy: 0.9419\n",
      "Epoch 7/20\n",
      "132/132 [==============================] - 49s 372ms/step - loss: 0.0411 - binary_accuracy: 0.9538\n",
      "Epoch 8/20\n",
      "132/132 [==============================] - 32s 238ms/step - loss: 0.0364 - binary_accuracy: 0.9591\n",
      "Epoch 9/20\n",
      "132/132 [==============================] - 28s 209ms/step - loss: 0.0346 - binary_accuracy: 0.9630\n",
      "Epoch 10/20\n",
      "132/132 [==============================] - 28s 214ms/step - loss: 0.0302 - binary_accuracy: 0.9668\n",
      "Epoch 11/20\n",
      "132/132 [==============================] - 37s 277ms/step - loss: 0.0317 - binary_accuracy: 0.9655\n",
      "Epoch 12/20\n",
      "132/132 [==============================] - 31s 237ms/step - loss: 0.0289 - binary_accuracy: 0.9685\n",
      "Epoch 13/20\n",
      "132/132 [==============================] - 28s 210ms/step - loss: 0.0263 - binary_accuracy: 0.9711\n",
      "Epoch 14/20\n",
      "132/132 [==============================] - 33s 251ms/step - loss: 0.0254 - binary_accuracy: 0.9732\n",
      "Epoch 15/20\n",
      "132/132 [==============================] - 44s 334ms/step - loss: 0.0293 - binary_accuracy: 0.9682\n",
      "Epoch 16/20\n",
      "132/132 [==============================] - 35s 264ms/step - loss: 0.0284 - binary_accuracy: 0.9687\n",
      "Epoch 17/20\n",
      "132/132 [==============================] - 34s 260ms/step - loss: 0.0219 - binary_accuracy: 0.9763\n",
      "Epoch 18/20\n",
      "132/132 [==============================] - 38s 288ms/step - loss: 0.0206 - binary_accuracy: 0.9779\n",
      "Epoch 19/20\n",
      "132/132 [==============================] - 28s 211ms/step - loss: 0.0214 - binary_accuracy: 0.9765\n",
      "Epoch 20/20\n",
      "132/132 [==============================] - 28s 210ms/step - loss: 0.0252 - binary_accuracy: 0.9726\n"
     ]
    }
   ],
   "source": [
    "model_mini_hist = model_mini.fit(x_train, np.asarray(y_train), batch_size=n_batch, epochs=n_epoch, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZMMLXNMgUAvu",
    "outputId": "aa447d3b-388b-43cb-a1f8-f9414d9c3d67"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44/44 [==============================] - 4s 86ms/step - loss: 0.1847 - binary_accuracy: 0.7820\n",
      "\n",
      "Model Accuracy: 78.2%\n"
     ]
    }
   ],
   "source": [
    "model_mini_score = model_mini.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\",str(round(model_mini_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "J-wkvW0IUAvv",
    "outputId": "21ea2bcd-27ae-4211-a616-b7bffacc2b77"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXyV5Z3//9cnOyErSVhD2EU2hRipilqtFVGs2KlWHFtAnTq2Y6uPtrZOW6vj2PnWbnZs/bXVKohTq9aOI61a1LrUpVbCIhAWCcgeIARIwpKQhM/vj3MHjyGBhOTkJDnv5+NxHrnPdd/3yecckry5r+u+r9vcHRERkdaKi3YBIiLSvSg4RESkTRQcIiLSJgoOERFpEwWHiIi0iYJDRETaRMEh0sHMbKiZuZkltGLbOWb2VmfUJdJRFBwS08xso5kdNrPcJu1Lgz/+Q6NTWdsCSKQzKThE4EPg2sYnZjYBSI1eOSJdm4JDBB4HZoU9nw3MD9/AzDLNbL6ZlZvZJjP7npnFBevizewnZrbbzDYA05vZ9xEzKzOzbWZ2r5nFt6dgMxtoZgvMbI+ZlZrZl8LWTTazYjOrMrOdZvazoD3FzP7HzCrMbJ+ZLTKzfu2pQ2KTgkME3gUyzGxM8Ad9JvA/Tbb5BZAJDAc+SShorg/WfQm4HJgEFAFXNdl3HlAPjAy2mQr8SztrfhLYCgwMvt9/mdmngnX/Dfy3u2cAI4Cng/bZwXsYDOQANwOH2lmHxCAFh0hI41HHxcBqYFvjirAw+Xd3r3b3jcBPgS8Gm3we+Lm7b3H3PcD/C9u3H3AZcJu7H3D3XcD9weudFDMbDEwBvu3uNe6+DPgtHx011QEjzSzX3fe7+7th7TnASHdvcPfF7l51snVI7FJwiIQ8DvwzMIcm3VRALpAIbApr2wQMCpYHAluarGs0JNi3LOge2gf8BujbjloHAnvcvbqFem4ETgHWBN1RlwftjwMLgSfNbLuZ/cjMEttRh8QoBYcI4O6bCA2SXwb8b5PVuwn9b31IWFsBHx2VlBHq/glf12gLUAvkuntW8Mhw93HtKHc70MfM0purx93Xufu1hMLpPuAZM+vt7nXu/h/uPhY4h1D32ixE2kjBIfKRG4FPufuB8EZ3byA0TvADM0s3syHA1/loHORp4Gtmlm9m2cAdYfuWAS8BPzWzDDOLM7MRZvbJNtSVHAxsp5hZCqGAeAf4f0HbaUHt/wNgZl8wszx3PwLsC17jiJldaGYTgq63KkJheKQNdYgACg6Ro9x9vbsXt7D6q8ABYAPwFvAE8Giw7mFCXUDvA0s49ohlFpAErAL2As8AA9pQ2n5Cg9iNj08ROn14KKGjj2eBu9z9lWD7aUCJme0nNFA+090PAf2D711FaBznDULdVyJtYrqRk4iItIWOOEREpE0UHCIi0iYKDhERaRMFh4iItElMzLqZm5vrQ4cOjXYZIiLdyuLFi3e7e17T9ogGh5lNI3Q6YDzwW3f/YZP1Xyc0Z089UA7cEFyIhZnNBr4XbHqvuz8WtJ9BaO6fXsALwK1+glPDhg4dSnFxS2dZiohIc8xsU3PtEeuqCi4yehC4FBgLXGtmY5tsthQocvfTCJ1f/qNg3z7AXcAngMnAXcGFVQC/IjSp3KjgMS1S70FERI4VyTGOyUCpu29w98OEZvOcEb6Bu7/m7geDp+8C+cHyJcDL7r7H3fcCLwPTzGwAkOHu7wZHGfOBKyP4HkREpIlIBscgPj7x21Y+moStOTcCL55g30HB8glf08xuCu5JUFxeXt7G0kVEpCVd4qwqM/sCofsY/LijXtPdH3L3Incvyss7ZmxHREROUiSDYxsfnzE0n7B7HDQys08D3wWucPfaE+y7jY+6s1p8TRERiZxIBsciYJSZDTOzJEI3rlkQvoGZTSJ0b4IrghvcNFoITDWz7GBQfCqwMJhptMrMzjIzIzR53HMRfA8iItJExE7Hdfd6M7uFUAjEA4+6e4mZ3QMUu/sCQl1TacAfQjnAZne/wt33mNl/EgofgHuCO6sBfIWPTsd9kY/GRUREpBPExOy4RUVFfjLXcTy3bBvVNfV84awhJ95YRKSHMbPF7l7UtL1LDI53VX9ZuYNfv7E+2mWIiHQpCo7jKCzIZuveQ+yqrol2KSIiXYaC4zgKh4QuVl+yad8JthQRiR0KjuMYPyiDpPg4lm7eG+1SRES6DAXHcSQnxDNuUAaLNyk4REQaKThOoLAgm+XbKjlcfyTapYiIdAkKjhM4Y0g2h+uPsKqsKtqliIh0CQqOEygsaBwgV3eViAgoOE6of2YKAzNTWKwBchERQMHRKpOGZLNURxwiIoCCo1UKC7LZXlnDjkpdCCgiouBohTMaLwRUd5WIiIKjNcYOyCA5IU4D5CIiKDhaJSkhjgmDMjVALiKCgqPVCodkU7Ktitr6hmiXIiISVQqOViosyOJwwxFWbtOFgCIS2xQcrdR4IaAmPBSRWKfgaKW+GSnkZ/fSmVUiEvMUHG1QWJDN4k17iYXb7YqItCSiwWFm08xsrZmVmtkdzaw/38yWmFm9mV0V1n6hmS0Le9SY2ZXBunlm9mHYuomRfA/hCguy2FlVy3ZdCCgiMSwhUi9sZvHAg8DFwFZgkZktcPdVYZttBuYA3wzf191fAyYGr9MHKAVeCtvkdnd/JlK1t+SMIX2A0ISHg7J6dfa3FxHpEiJ5xDEZKHX3De5+GHgSmBG+gbtvdPflwPFudnEV8KK7H4xcqa1z6oB0UhLjNM4hIjEtksExCNgS9nxr0NZWM4HfN2n7gZktN7P7zSy5uZ3M7CYzKzaz4vLy8pP4tsdKjI/jtPwslmzWPchFJHZ16cFxMxsATAAWhjX/O3AqcCbQB/h2c/u6+0PuXuTuRXl5eR1WU2FBNiXbKqmp04WAIhKbIhkc24DBYc/zg7a2+DzwrLvXNTa4e5mH1AJzCXWJdZrCgizqjzgrtlV25rcVEekyIhkci4BRZjbMzJIIdTktaONrXEuTbqrgKAQzM+BKYGUH1NpqhUN0R0ARiW0RCw53rwduIdTNtBp42t1LzOweM7sCwMzONLOtwNXAb8yspHF/MxtK6IjljSYv/TszWwGsAHKBeyP1HpqTm5bMkJxUDZCLSMyK2Om4AO7+AvBCk7bvhy0vItSF1dy+G2lmMN3dP9WxVbZdYUE2b5Xuxt0JHfiIiMSOLj043lUVFmRRXl3L1r2Hol2KiEinU3CchEkFuiOgiMQuBcdJOLV/OqlJ8RogF5GYpOA4CQnxcZyuCwFFJEYpOE5S4ZAsVpVVcfBwfbRLERHpVAqOk1RYkE3DEWf5Vl0IKCKxRcFxkjRALiKxSsFxkvr0TmJ4bm+WbNI4h4jEFgVHO0wqyGbpZt0RUERii4KjHQqHZFFx4DCbKqJ+qxARkU6j4GiHQo1ziEgMUnC0wyn90klLTlBwiEhMUXC0Q3ycMXFwlgbIRSSmKDjaqbAgizU7qjhQqwsBRSQ2KDjaadKQbI44vL9FRx0iEhsUHO1UOFgD5CISWxQc7ZSZmsjIvmma8FBEYoaCowMUFmTpQkARiRkKjg5QWJDN3oN1fLj7QLRLERGJOAVHBygcEhrnWKwbO4lIDIhocJjZNDNba2alZnZHM+vPN7MlZlZvZlc1WddgZsuCx4Kw9mFm9o/gNZ8ys6RIvofWGJmXRnpKgsY5RCQmRCw4zCweeBC4FBgLXGtmY5tsthmYAzzRzEsccveJweOKsPb7gPvdfSSwF7ixw4tvo7g4OzrhoYhITxfJI47JQKm7b3D3w8CTwIzwDdx9o7svB4605gXNzIBPAc8ETY8BV3ZcySevsCCLtTurqa6pi3YpIiIRFcngGARsCXu+NWhrrRQzKzazd82sMRxygH3u3niZdouvaWY3BfsXl5eXt7X2NissyMYdlulCQBHp4bry4PgQdy8C/hn4uZmNaMvO7v6Quxe5e1FeXl5kKgwzsSALMzRvlYj0eJEMjm3A4LDn+UFbq7j7tuDrBuB1YBJQAWSZWcLJvGYkZaQkckrfdF1BLiI9XiSDYxEwKjgLKgmYCSw4wT4AmFm2mSUHy7nAFGCVh66wew1oPANrNvBch1d+kgqHhC4EPHJEFwKKSM8VseAIxiFuARYCq4Gn3b3EzO4xsysAzOxMM9sKXA38xsxKgt3HAMVm9j6hoPihu68K1n0b+LqZlRIa83gkUu+hrSYVZFNVU8+G3fujXYqISMQknHiTk+fuLwAvNGn7ftjyIkLdTU33eweY0MJrbiB0xlaX03hHwMWb9jKyb3qUqxERiYyuPDje7QzP7U1mr0QNkItIj6bg6EBxcUZhQZYGyEWkR1NwdLDCgmzW7dpP5SFdCCgiPZOCo4M1TnioCwFFpKdScHSw0wdnEWeaKVdEei4FRwdLS07glH7pmvBQRHosBUcEnDEkm2Wb9+lCQBHpkRQcEVBYkE11bT3rdulCQBHpeRQcEdA4QK7TckWkJ1JwRMDQnFT69E7SALmI9EgKjggw04WAItJzKTgiZFJBNhvKD7Dv4OFolyIi0qEUHBHSOOHh0s26EFBEehYFR4ScPjiT+DjTOIeI9DgKjghJTUrg1P7pvPfhnmiXIiLSoRQcEXTZhAG8t3EPizcpPESk51BwRND1U4aSl57MD19cQ+iutyIi3Z+CI4JSkxK49aJRLNq4l7+u3hXtckREOoSCI8KuOXMww3J786OFa2jQ3FUi0gNENDjMbJqZrTWzUjO7o5n155vZEjOrN7OrwtonmtnfzazEzJab2TVh6+aZ2Ydmtix4TIzke2ivxPg4br9kNB/s3M//Ltka7XJERNotYsFhZvHAg8ClwFjgWjMb22SzzcAc4Ikm7QeBWe4+DpgG/NzMssLW3+7uE4PHsoi8gQ506fj+nD44i5+9/AE1dQ3RLkdEpF0iecQxGSh19w3ufhh4EpgRvoG7b3T35cCRJu0fuPu6YHk7sAvIi2CtEWVmfHvaaMoqa5j/943RLkdEpF0iGRyDgC1hz7cGbW1iZpOBJGB9WPMPgi6s+80suYX9bjKzYjMrLi8vb+u37XDnjMjlk6fk8eBr66k8qPuRi0j31aUHx81sAPA4cL27Nx6V/DtwKnAm0Af4dnP7uvtD7l7k7kV5eV3jYOXb006lqqaOX72x/sQbi4h0UZEMjm3A4LDn+UFbq5hZBvA88F13f7ex3d3LPKQWmEuoS6xbGDswgysnDmLu2x+yo7Im2uWIiJyUSAbHImCUmQ0zsyRgJrCgNTsG2z8LzHf3Z5qsGxB8NeBKYGWHVh1hX7/4FNzh5698EO1SREROSsSCw93rgVuAhcBq4Gl3LzGze8zsCgAzO9PMtgJXA78xs5Jg988D5wNzmjnt9ndmtgJYAeQC90bqPUTC4D6pXHdWAU8Xb6F0V3W0yxERaTOLhakwioqKvLi4ONplHFWxv5ZP/vh1zhmRw0OziqJdjohIs8xssbsf80eqSw+O91Q5acn86/nDeWnVTk27LiLdjoIjSm48bxi5acncpwkQRaSbUXBESWpSArd9ehTvbdzDq2s0AaKIdB8KjihqnADxvr9oAkQR6T4UHFGUGB/HN6dqAkQR6V4UHFF22YT+nJ6fyf2aAFFEuolWBYeZ9TazuGD5FDO7wswSI1tabDAzvn3pqWyvrOHxv2+KdjkiIifU2iOOvwEpZjYIeAn4IjAvUkXFmsYJEH/5WimVhzQBooh0ba0NDnP3g8A/Af+fu18NjItcWbHnW9NGU3mojl9rAkQR6eJaHRxmdjZwHaGJBwHiI1NSbBo3MJMrJw7k0bc0AaKIdG2tDY7bCE1n/mww39Rw4LXIlRWbvjF1NEfc+e+/agJEEem6WhUc7v6Gu1/h7vcFg+S73f1rEa4t5gzuk8oXzhrCU4u2ULprf7TLERFpVmvPqnrCzDLMrDehacxXmdntkS0tNt1y4UhSkxL48cI10S5FRKRZre2qGuvuVYTuf/EiMIzQmVXSwXLSkrnp/OEsLNEEiCLSNbU2OBKD6zauBBa4ex2gOTIi5MZzNQGiiHRdrQ2O3wAbgd7A38xsCFAVqaJiXe/kBG4NJkB8ba0mQBSRrqW1g+MPuPsgd78suN/3JuDCCNcW02aeOZihOan81wtrOFx/JNrliIgc1drB8Uwz+5mZFQePnxI6+pAISYyP487Lx1K6az8Pv7kh2uWIiBzV2q6qR4FqQvcC/zyhbqq5kSpKQi4a049p4/rzwF/XsbniYLTLEREBWh8cI9z9LnffEDz+Axh+op3MbJqZrTWzUjO7o5n155vZEjOrN7OrmqybbWbrgsfssPYzzGxF8JoPmJm18j10S3ddMZaEOOPO51ZqoFxEuoTWBschMzu38YmZTQEOHW8HM4sHHgQuBcYC15rZ2CabbQbmAE802bcPcBfwCWAycJeZZQerfwV8CRgVPKa18j10SwMye/GNqaN544Nynl9RFu1yRERaHRw3Aw+a2UYz2wj8EvjXE+wzGSgNjlAOA08CM8I3cPeN7r4caDr6ewnwsrvvcfe9wMvANDMbAGS4+7se+u/3fEKnCPdos88ZyvhBGfzHn1ZRVaPZc0Ukulp7VtX77n46cBpwmrtPAj51gt0GAVvCnm8N2lqjpX0HBcsn85rdVnyc8V+fnUDF/lp+snBttMsRkRjXpjsAuntVcAU5wNcjUE+HMbObGs8CKy8vj3Y57XZafhazzh7K4+9uYtmWfdEuR0RiWHtuHXuiQeltwOCw5/lBW2u0tO+2YPmEr+nuD7l7kbsX5eXltfLbdm3fmHoKfdOT+c7/rqC+Qdd2iEh0tCc4TnSKzyJglJkNM7MkYCawoJWvvRCYambZwaD4VGChu5cBVWZ2VnA21SzguZOsv9tJT0nk7s+MY1VZFfPe2RjtckQkRh03OMys2syqmnlUAwOPt6+71wO3EAqB1cDTwb087jGzK4LXP9PMtgJXA78xs5Jg3z3AfxIKn0XAPUEbwFeA3wKlwHpCky7GjGnj+/OpU/vys5c/YPu+457YJiISERYL1wYUFRV5cXFxtMvoMFv2HOTi+9/gvFF5PDyrKNrliEgPZWaL3f2YPzLt6aqSKBncJ5XbPn0KL6/ayUslO6JdjojEGAVHN3XjucM4tX86dy8o4UBtfbTLEZEYouDophLj4/jBZ8ezvbKG+1/WPcpFpPMoOLqxM4b04drJBcx9ZyMl2yujXY6IxAgFRzd3x7RTyU5N5DvPrqThSM8/0UFEok/B0c1lpibyveljeX/LPp74x6ZolyMiMUDB0QPMmDiQc0fm8qO/rGVXVU20yxGRHk7B0QOYGfdeOZ7ahiPc8+dV0S5HRHo4BUcPMTS3N7dcOJI/Ly/j9bW7ol2OiPRgCo4e5F8/OZzheb2587mVHDrcEO1yRKSHUnD0IMkJ8fzgygls2XOIX7y6LtrliEgPpeDoYc4ekcPnCvN56G8b+GBndbTLEZEeSMHRA313+hjSUhL47rMrOKJrO0Skgyk4eqA+vZP4zqVjWLRxL39YvOXEO4iItIGCo4e6uiifycP6cM+fVrFym6YjEZGOo+DoocyMX1w7iazUJK6ft4gtew5GuyQR6SEUHD1Yv4wU5l1/JrV1DcyZ+x77Dh6Odkki0gMoOHq4Uf3SeXhWEVv2HOJL84upqdP1HSLSPgqOGPCJ4Tncf81EFm3cy9efXqYzrUSkXRQcMWL6aQP43vQxvLBiB/c+vzra5YhINxbR4DCzaWa21sxKzeyOZtYnm9lTwfp/mNnQoP06M1sW9jhiZhODda8Hr9m4rm8k30NP8i/nDeeGKcN49O0P+e2bG6Jdjoh0UwmRemEziwceBC4GtgKLzGyBu4dP33ojsNfdR5rZTOA+4Bp3/x3wu+B1JgD/5+7Lwva7zt2LI1V7T/a96WPYWVXDvc+vpl9GCp85fWC0SxKRbiaSRxyTgVJ33+Duh4EngRlNtpkBPBYsPwNcZGbWZJtrg32lA8TFGT/9/OlMHtqHbzz9Pu9uqIh2SSLSzUQyOAYB4Zctbw3amt3G3euBSiCnyTbXAL9v0jY36Ka6s5mgAcDMbjKzYjMrLi8vP9n30COlJMbz0KwzKMhJ5ab5xZrTSkTapEsPjpvZJ4CD7r4yrPk6d58AnBc8vtjcvu7+kLsXuXtRXl5eJ1TbvWSlJjHv+jNJSYxnzqPvsVN3DhSRVopkcGwDBoc9zw/amt3GzBKATCC872QmTY423H1b8LUaeIJQl5ichPzsVOZefyaVh+qY/eh7VNfURbskEekGIhkci4BRZjbMzJIIhcCCJtssAGYHy1cBr7q7A5hZHPB5wsY3zCzBzHKD5UTgcmAlctLGDczk1188g9Jd+/ny/yzhcP2RaJckIl1cxIIjGLO4BVgIrAaedvcSM7vHzK4INnsEyDGzUuDrQPgpu+cDW9w9/LzRZGChmS0HlhE6Ynk4Uu8hVpw3Ko/7Pncab5Xu5o4/LifIbhGRZkXsdFwAd38BeKFJ2/fDlmuAq1vY93XgrCZtB4AzOrxQ4XNn5FNWeYifvPQBA7JSuP2SU6Ndkoh0URENDule/u3CkWyvrOHB19YzILMXXzhrSLRLEpEuSMEhR5kZ91wxjl1VNXz/uZX0y0jh4rH9ol2WiHQxXfp0XOl8CfFxPHDtJCbkZ/HV3y9h6ea90S5JRLoYBYccIzUpgUdmF9EvI4Ub5i2idNf+aJckIl2IgkOalZuWzPwbJhMfF8fsR9+jrPJQtEsSkS5CwSEtGpLTm3nXn0nVoTpmPaI7CIpIiIJDjmv8oEwemlXEpj0HuWHeIg4ero92SSISZQoOOaGzR+TwwMxJLNuyj6/8bgl1Dbq6XCSWKTikVaaN789/fXYCr68t51vPLNftZ0VimK7jkFabObmAigOH+fHCtWSnJnHn5WNoYVZ7EenBFBzSJl+5YAS799fy6NsfkpuexFcuGBntkkSkkyk4pE3MjDunj2XPgcP86C9r6ZOaxMzJBdEuS0Q6kYJD2iwuzvjxVaez72Ad33l2Bdm9k7hkXP9olyUinUSD43JSkhLi+NUXCjktP4uv/n6p7l0uEkMUHHLSUpMSmDvnTAr6pPKlx4op2V4Z7ZJEpBMoOKRdsnsnMf+GyaSnJDD70UVsqjgQ7ZJEJMIUHNJuA7N6Mf/GT9Bw5AhffOQ9dlXXRLskEYkgBYd0iJF905h7/WR2769l9qOLqKqpi3ZJIhIhCg7pMBMHZ/HrL5xB6a5q/uWxYmrqGqJdkohEQESDw8ymmdlaMys1szuaWZ9sZk8F6/9hZkOD9qFmdsjMlgWPX4ftc4aZrQj2ecB06XKXcv4pefzk6tNZtHEPX/v9Uuo1r5VIjxOx4DCzeOBB4FJgLHCtmY1tstmNwF53HwncD9wXtm69u08MHjeHtf8K+BIwKnhMi9R7kJMzY+Ig7rp8LC+t2sntzyzXkYdIDxPJI47JQKm7b3D3w8CTwIwm28wAHguWnwEuOt4RhJkNADLc/V13d2A+cGXHly7tNWfKML5x8Sk8u3QbM375Nmt3VEe7JBHpIJEMjkHAlrDnW4O2Zrdx93qgEsgJ1g0zs6Vm9oaZnRe2/dYTvCYAZnaTmRWbWXF5eXn73omclK9eNIp5159JxYHDfOaXbzHv7Q8J5b2IdGdddXC8DChw90nA14EnzCyjLS/g7g+5e5G7F+Xl5UWkSDmxC0b35S+3nce5I3O5+0+ruGHeInbvr412WSLSDpEMjm3A4LDn+UFbs9uYWQKQCVS4e627VwC4+2JgPXBKsH3+CV5TupjctGQemV3EPTPG8c76Cqb9/G+8tnZXtMsSkZMUyeBYBIwys2FmlgTMBBY02WYBMDtYvgp41d3dzPKCwXXMbDihQfAN7l4GVJnZWcFYyCzguQi+B+kgZsass4fyp6+eS25aMtfPXcTdC0o0cC7SDUUsOIIxi1uAhcBq4Gl3LzGze8zsimCzR4AcMysl1CXVeMru+cByM1tGaND8ZnffE6z7CvBboJTQkciLkXoP0vFO6ZfO//3bFOacM5R572zkygc1cC7S3VgsDFYWFRV5cXFxtMuQJl5bu4vb//A+1TX1fOeyMcw6e4juKCjShZjZYncvatreVQfHJQZcOLovL956PmePyOGuBSX8y2PFGjgX6QYUHBJVeenJzJ1zJnd/Zixvlu5m2s/f5I0PdPq0SFem4JCoMzPmTBnGglum0Kd3IrMffY97/rRKA+ciXZSCQ7qMU/tnsOCWc5l99hAefftDrnzwbVaXVUW7LBFpQoPj0iW9umYnt/9hORUHDnNafiaXTRjA9AkDGNwnNdqlicSMlgbHFRzSZVXsr+WPS7by/PIy3t8aui3tafmZTJ8wgMsUIiIRp+BQcHRrW/Yc5IUVZbyw4qMQOT04ElGIiESGgkPB0WM0hsjzK8pYHhYi008LhUh+tkJEpCMoOBQcPdLmioO8sLKM55eXsWJbECKDs5g+ob9CRKSdFBwKjh5vc8VBng+6sxpD5JR+aQzN6c3gPqkMzu7F4D6pFPRJJT87lV5J8VGuWKRrU3AoOGJKY4gUb9zDlr0H2bznIDV1H7+NbW5aMgV9egWhEgRKn14Mzk5lQGYKCfE6W11im4JDwRHT3J3d+w+zZe9BtuxpfBxi856DbNl7kLLKGhqOfPS7kBBnDMzqxSn90vj0mH58emw/ctOSo/gORDpfS8GREI1iRDqbmZGXnkxeejKFBdnHrK9rOELZvpqjwRIKlEMs3byXV1bvIu7ZFRQN6cPUcf24ZFx/ncUlMU1HHCLH4e6sKqtiYclOXirZwZpgCvixAzKOhsip/dM1q6/0SOqqUnBIB9hUcYCXSnaysGQHizfvxR0K+qRySRAikwqyiY87+RCpqWtgR2UNO6pq2FFZw+GGI4wdkMGofmkkJ2gwXzqXgqNJcNTV1bF161ZqamqiVFXnSUlJIT8/n8TExGiX0qPsqq7hlVW7WFiyg3fW76auwclNS+bisX2ZOq4/54zIOfrH3t2pqqkPC4VD7KisZUfVIXZU1lAWtO87WNfs90qMN07pl874gZmMH5TBuEGZjOmfoTPDJKIUHE2C48MPPyQ9PZ2cnJwe3c3g7lRUVLSbFIsAAA+oSURBVFBdXc2wYcOiXU6PVVVTx2trdvFSyU5eW7uLg4cbSE9OYMyADHbvr6WssoZDzcz2m5uWRP/MFPpnpIR97XX0eZzB6rJqVm6vZOW20GNvEC5xBiP7pjF+YCbjBmUyfmAGYwdmkJ6i/yBIx1BwNAmO1atXc+qpp/bo0Gjk7qxZs4YxY8ZEu5SYUFPXwNulu1lYsoMPdx+gb3rKx8JhQGYK/TJS6JuR3ObuJ3enrLImFCLbqyjZVsnK7ZXsrProBljDcnszbmAG4wdlMm5gBuMGZtKnd1JHv82IaTjiLN+6jzfX7Wbtzmp6J8WTnpJIekrC0a8ZKQlkpCSGtYfWJSXoFOqOpLOqmhELoQGx8z67ipTEeC4a04+LxvTr8Nc2C50mPDCrF1PH9T/avqu6hpLGINlWxbIt+/jz8rKj6/tnpDB2YAZjB2QwLjgyGZydSlw7xmM60pY9B/nbunLeWrebt0t3U1VTjxkMzk6ltr6B6pp6Dh4+8f1ZkhPiSE9JJCMIkwGZvbjurALOHZmr34MOFNPBIdJT9E1Poe/oFC4c3fdo276DhynZXsWq7VWsKgt9feOD8qPXq6QlJzBmQDrjBmYydkAoTDprEL6qpo6/r6/gzSAsNlYcBGBAZgrTxvfnvFF5TBmZ+7EjpfqGI+yvrae6pp6qmjqqa+qDR93HvlaFLS/evJe/lOxg/KAMbv7kCC4dP6BdJy9ISESDw8ymAf8NxAO/dfcfNlmfDMwHzgAqgGvcfaOZXQz8EEgCDgO3u/urwT6vAwOAQ8HLTHX3XZF8H5FQUVHBRRddBMCOHTuIj48nLy8PgPfee4+kpJa7FoqLi5k/fz4PPPBAp9Qq3VNWahJTRuYyZWTu0baaugY+2Fn9sTD5Q/EWDgT/m0+IM0b2TTt6dDKybxpZqUlHu4MyUhJJSWx7sNQ3HOH9oPvpzXW7WbZlHw1HnNSkeM4ensPsc4Zy3qg8RuT1bvHIICE+jqzUJLJSW9/tVlvfwP8t3cZv3tjALU8sZUjOWm46fzifK8w/qfchIREb4zCzeOAD4GJgK7AIuNbdV4Vt8xXgNHe/2cxmAp9192vMbBKw0923m9l4YKG7Dwr2eR34pru3+vzalsY4ukqf/913301aWhrf/OY3j7bV19eTkNBxud6V3q90LUeOOJv2HAzCpPLoUcqu6tpmt0+KjwuFSK/EjwVK4zhDRti4w6FgvOed9RVUB91Ppw3K5LxReZw3KpdJBdmdMi7RcMR5edUOfvXGBt7fso/ctGSunzKUL5w1hMxePfdkgvLqWvLST37Gg2iMcUwGSt19Q1DAk8AMYFXYNjOAu4PlZ4Bfmpm5+9KwbUqAXmaW7O7N/yS303/8qYRV2zv2FqVjB2Zw12fGtWmfOXPmkJKSwtKlS5kyZQozZ87k1ltvpaamhl69ejF37lxGjx7N66+/zk9+8hP+/Oc/c/fdd7N582Y2bNjA5s2bue222/ja177Woe9Fera4OGNYbm+G5fZm+mkDjraXV9eyseIA1TV1VB0Kdf9UNdNNVHWojl1VtUe7kJqORQzK6sXlpw3g3JF5nDMih+woDNTHxxnTxg/gknH9+fuGCn79xgZ+vHAtv3p9Pdd9ooAbzh1Gv4yUTq+ro+2vree9Dyt4Mxgr+mDnft781oUdPtNBJINjELAl7PlW4BMtbePu9WZWCeQAu8O2+RywpElozDWzBuCPwL3ezGGTmd0E3ARQUFDQzrfSebZu3co777xDfHw8VVVVvPnmmyQkJPDKK6/wne98hz/+8Y/H7LNmzRpee+01qqurGT16NF/+8pd1zYa0W+MULW0VPhYBkJ/dq8sMTJsZ54zI5ZwRuazcVslv/raBh9/cwNy3N/JPhYO46fzhDM9Li3aZrRbqAqw8elLBks17qT/iJCfEMXlYHz5XmB+Ra3269OC4mY0D7gOmhjVf5+7bzCydUHB8kdA4yce4+0PAQxDqqjre92nrkUEkXX311cTHh/6hKysrmT17NuvWrcPMqKtr/uKw6dOnk5ycTHJyMn379mXnzp3k5+d3ZtkiR53MWEQ0jB+UyS+uncTtU0fz8JsbeLp4C08Vb2HauP7c/MkRnD44K9olHsPd2bD7AG+t281bpbt5d30F1bWhLsDxAzP50vnDOXdkLmcMyY7oGE4kg2MbMDjseX7Q1tw2W80sAcgkNEiOmeUDzwKz3H194w7uvi34Wm1mTxDqEjsmOLqr3r17H12+8847ufDCC3n22WfZuHEjF1xwQbP7JCd/9L/C+Ph46uvrI12mSI9RkJPKf145nls/PYp5b29k/t838uLKHZw9PIc5U4YyLLc3uWnJZPVKjMrpy7v31/J26e6jRxXbK0OzXQzu04vLTx/IuSNzO70LMJLBsQgYZWbDCAXETOCfm2yzAJgN/B24CnjV3d3MsoDngTvc/e3GjYNwyXL33WaWCFwOvBLB9xBVlZWVDBo0CIB58+ZFtxiRHi43LZlvXjKamy8Ywe//sZnfvrWBf3188dH18XFGTu8kctKSyU1LIi8tmdz00HJuWjK5acnkBO19eicdcz+XuoYjVNfUsz9snCjUpRd2OnFt/cfGj8r21bB2Z2hizcxeiUwZmcO/jczlvJF5FOREb4bmiAVHMGZxC7CQ0Om4j7p7iZndAxS7+wLgEeBxMysF9hAKF4BbgJHA983s+0HbVOAAsDAIjXhCofFwpN5DtH3rW99i9uzZ3HvvvUyfPj3a5YjEhLTkBL50/nBmnTOEpZv3UV5dy+79waP6cOjrgcNsKD/A7v211NYfOeY1zCA7NYmMlAQOHG6guqbumBuJNScpIY705I+uhO+fmcIVEwdy3qhcxg3M7DLXoMT0lCOxdHpqrL1fkc7g7uyvrWf3/iBQgpApD55X19STlhxMmZKcQFrYtCmhgAgtpwWnL3e1GZA15YiISAczs+CPfyLDcnufeIceQjOCiYhIm8R0cMRCNx3EzvsUkc4Rs8GRkpJCRUVFj/+j2ng/jpSU7n9VrIh0DTE7xpGfn8/WrVspLy+PdikR13gHQBGRjhCzwZGYmKg74omInISY7aoSEZGTo+AQEZE2UXCIiEibxMSV42ZWDmw6yd1z+fg0712N6msf1dc+qq99unp9Q9w9r2ljTARHe5hZcXOX3HcVqq99VF/7qL726er1tURdVSIi0iYKDhERaRMFx4k9FO0CTkD1tY/qax/V1z5dvb5maYxDRETaREccIiLSJgoOERFpEwVHwMymmdlaMys1szuaWZ9sZk8F6/9hZkM7sbbBZvaama0ysxIzu7WZbS4ws0ozWxY8vt/ca0Wwxo1mtiL43sXNrDczeyD4/JabWWEn1jY67HNZZmZVZnZbk2069fMzs0fNbJeZrQxr62NmL5vZuuBrdgv7zg62WWdmszuxvh+b2Zrg3+9ZM8tqYd/j/ixEsL67zWxb2L/hZS3se9zf9QjW91RYbRvNbFkL+0b882s3d4/5B6H7l68HhgNJwPvA2CbbfAX4dbA8E3iqE+sbABQGy+nAB83UdwHw5yh+hhuB3OOsvwx4ETDgLOAfUfy33kHowqaofX7A+UAhsDKs7UfAHcHyHcB9zezXB9gQfM0OlrM7qb6pQEKwfF9z9bXmZyGC9d0NfLMV//7H/V2PVH1N1v8U+H60Pr/2PnTEETIZKHX3De5+GHgSmNFkmxnAY8HyM8BFZtYpd4539zJ3XxIsVwOrgUGd8b070Axgvoe8C2SZ2YAo1HERsN7dT3YmgQ7h7n8D9jRpDv8Zewy4spldLwFedvc97r4XeBmY1hn1uftL7l4fPH0XiNpc/S18fq3Rmt/1djtefcHfjc8Dv+/o79tZFBwhg4AtYc+3cuwf5qPbBL88lUBOp1QXJugimwT8o5nVZ5vZ+2b2opmN69TCwIGXzGyxmd3UzPrWfMadYSYt/8JG8/MD6OfuZcHyDqBfM9t0lc/xBkJHkM050c9CJN0SdKU92kJXX1f4/M4Ddrr7uhbWR/PzaxUFRzdiZmnAH4Hb3L2qyeolhLpfTgd+AfxfJ5d3rrsXApcC/2Zm53fy9z8hM0sCrgD+0MzqaH9+H+OhPosuea68mX0XqAd+18Im0fpZ+BUwApgIlBHqDuqKruX4Rxtd/ndJwRGyDRgc9jw/aGt2GzNLADKBik6pLvQ9EwmFxu/c/X+brnf3KnffHyy/ACSaWW5n1efu24Kvu4BnCXUJhGvNZxxplwJL3H1n0xXR/vwCOxu774Kvu5rZJqqfo5nNAS4HrgvC7Rit+FmICHff6e4N7n4EeLiF7xvtzy8B+CfgqZa2idbn1xYKjpBFwCgzGxb8r3QmsKDJNguAxjNYrgJebekXp6MFfaKPAKvd/WctbNO/cczFzCYT+rftlGAzs95mlt64TGgQdWWTzRYAs4Kzq84CKsO6ZTpLi//Ti+bnFyb8Z2w28Fwz2ywEpppZdtAVMzVoizgzmwZ8C7jC3Q+2sE1rfhYiVV/4mNlnW/i+rfldj6RPA2vcfWtzK6P5+bVJtEfnu8qD0Fk/HxA64+K7Qds9hH5JAFIIdXGUAu8BwzuxtnMJdVssB5YFj8uAm4Gbg21uAUoInSXyLnBOJ9Y3PPi+7wc1NH5+4fUZ8GDw+a4Aijr537c3oSDIDGuL2udHKMDKgDpC/ew3Ehoz+yuwDngF6BNsWwT8NmzfG4Kfw1Lg+k6sr5TQ+EDjz2DjWYYDgReO97PQSfU9HvxsLScUBgOa1hc8P+Z3vTPqC9rnNf7MhW3b6Z9fex+ackRERNpEXVUiItImCg4REWkTBYeIiLSJgkNERNpEwSEiIm2i4BDpAGbW0GQG3g6bddXMhobPsioSbQnRLkCkhzjk7hOjXYRIZ9ARh0gEBfdW+FFwf4X3zGxk0D7UzF4NJuT7q5kVBO39gntdvB88zgleKt7MHrbQ/VheMrNeUXtTEvMUHCIdo1eTrqprwtZVuvsE4JfAz4O2XwCPuftphCYLfCBofwB4w0OTLRYSunoYYBTwoLuPA/YBn4vw+xFpka4cF+kAZrbf3dOaad8IfMrdNwQTVe5w9xwz201oSoy6oL3M3XPNrBzId/fasNcYSugeHKOC598GEt393si/M5Fj6YhDJPK8heW2qA1bbkDjkxJFCg6RyLsm7Ovfg+V3CM3MCnAd8Gaw/FfgywBmFm9mmZ1VpEhr6X8tIh2jl5ktC3v+F3dvPCU328yWEzpquDZo+yow18xuB8qB64P2W4GHzOxGQkcWXyY0y6pIl6ExDpEICsY4itx9d7RrEeko6qoSEZE20RGHiIi0iY44RESkTRQcIiLSJgoOERFpEwWHiIi0iYJDRETa5P8HuIBv28/FJGUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model_mini_hist.history['loss'])\n",
    "\n",
    "plt.title('Model Loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['Train'], loc='lower left')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GEqsC9KlUAvw"
   },
   "source": [
    "This is a good value for the model, we can use it. Let's save our model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dkgO64lLUAvw"
   },
   "outputs": [],
   "source": [
    "model_mini.save('models/LSTM-Mini-Batch-Model.h5')  # save model\n",
    "del model_mini  # delete existing model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "B1-OiFG1YPYr"
   },
   "outputs": [],
   "source": [
    "# returns a compiled model, identical to the previous one\n",
    "model_mini = load_model('models/LSTM-Mini-Batch-Model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TzLZXiV_UAvx"
   },
   "source": [
    "Let's check if everything is correct:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IonSQZhuUAvx",
    "outputId": "d7561b64-0bf6-4e22-f4b0-dd2772deb2cc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44/44 [==============================] - 4s 87ms/step - loss: 0.1011 - binary_accuracy: 0.8848\n",
      "\n",
      "Model Accuracy: 88.48%\n"
     ]
    }
   ],
   "source": [
    "model_mini_score = model_mini.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\", str(round(model_mini_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aOj5rNBfUAvz"
   },
   "source": [
    "<h2 align=\"center\">Stacked LSTM Layers</h2>\n",
    "\n",
    "Now we will try the model with a 3 stacked LSTM layers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wnJQBxZTUAvz",
    "outputId": "b26f075c-d786-48e5-962f-170274b19005"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " lstm (LSTM)                 (None, 50, 100)           160400    \n",
      "                                                                 \n",
      " lstm_1 (LSTM)               (None, 50, 32)            17024     \n",
      "                                                                 \n",
      " lstm_2 (LSTM)               (None, 32)                8320      \n",
      "                                                                 \n",
      " dense (Dense)               (None, 1)                 33        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 185,777\n",
      "Trainable params: 185,777\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# configure network\n",
    "n_neurons = 100\n",
    "n_epoch = 20\n",
    "n_batch = 128\n",
    "\n",
    "# design network\n",
    "model3L = Sequential()\n",
    "\n",
    "model3L.add(LSTM(n_neurons, return_sequences=True, input_shape=(x.shape[1], x.shape[2])))\n",
    "model3L.add(LSTM(32, return_sequences=True))  # returns a sequence of vectors of dimension 32\n",
    "model3L.add(LSTM(32))  # return a single vector of dimension 32\n",
    "\n",
    "model3L.add(Dense(1, activation='sigmoid'))\n",
    "model3L.compile(loss='mean_squared_error', optimizer='adam', metrics=['binary_accuracy'])\n",
    "\n",
    "print(model3L.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "BDNLjOJYUAv0",
    "outputId": "296c34e2-4aad-4928-92b5-37fc1e8682e8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "132/132 [==============================] - 35s 261ms/step - loss: 0.1186 - binary_accuracy: 0.8389\n",
      "Epoch 2/20\n",
      "132/132 [==============================] - 33s 250ms/step - loss: 0.0980 - binary_accuracy: 0.8724\n",
      "Epoch 3/20\n",
      "132/132 [==============================] - 32s 245ms/step - loss: 0.0821 - binary_accuracy: 0.8949\n",
      "Epoch 4/20\n",
      "132/132 [==============================] - 33s 252ms/step - loss: 0.0657 - binary_accuracy: 0.9194\n",
      "Epoch 5/20\n",
      "132/132 [==============================] - 33s 248ms/step - loss: 0.0516 - binary_accuracy: 0.9393\n",
      "Epoch 6/20\n",
      "132/132 [==============================] - 32s 245ms/step - loss: 0.0472 - binary_accuracy: 0.9443\n",
      "Epoch 7/20\n",
      "132/132 [==============================] - 33s 250ms/step - loss: 0.0416 - binary_accuracy: 0.9512\n",
      "Epoch 8/20\n",
      "132/132 [==============================] - 32s 245ms/step - loss: 0.0369 - binary_accuracy: 0.9585\n",
      "Epoch 9/20\n",
      "132/132 [==============================] - 32s 244ms/step - loss: 0.0358 - binary_accuracy: 0.9583\n",
      "Epoch 10/20\n",
      "132/132 [==============================] - 32s 245ms/step - loss: 0.0304 - binary_accuracy: 0.9656\n",
      "Epoch 11/20\n",
      "132/132 [==============================] - 32s 245ms/step - loss: 0.0282 - binary_accuracy: 0.9688\n",
      "Epoch 12/20\n",
      "132/132 [==============================] - 32s 245ms/step - loss: 0.0262 - binary_accuracy: 0.9708\n",
      "Epoch 13/20\n",
      "132/132 [==============================] - 33s 248ms/step - loss: 0.0243 - binary_accuracy: 0.9733\n",
      "Epoch 14/20\n",
      "132/132 [==============================] - 33s 247ms/step - loss: 0.0225 - binary_accuracy: 0.9751\n",
      "Epoch 15/20\n",
      "132/132 [==============================] - 32s 246ms/step - loss: 0.0200 - binary_accuracy: 0.9787\n",
      "Epoch 16/20\n",
      "132/132 [==============================] - 33s 247ms/step - loss: 0.0210 - binary_accuracy: 0.9769\n",
      "Epoch 17/20\n",
      "132/132 [==============================] - 33s 246ms/step - loss: 0.0185 - binary_accuracy: 0.9801\n",
      "Epoch 18/20\n",
      "132/132 [==============================] - 32s 246ms/step - loss: 0.0211 - binary_accuracy: 0.9769\n",
      "Epoch 19/20\n",
      "132/132 [==============================] - 33s 248ms/step - loss: 0.0229 - binary_accuracy: 0.9739\n",
      "Epoch 20/20\n",
      "132/132 [==============================] - 33s 247ms/step - loss: 0.0223 - binary_accuracy: 0.9751\n"
     ]
    }
   ],
   "source": [
    "model3L_hist = model3L.fit(x_train, np.asarray(y_train), batch_size=n_batch, epochs=n_epoch, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xlqt5FW1UAv1",
    "outputId": "d8376769-34c8-4ddf-8ed0-733680e1c7b2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44/44 [==============================] - 6s 97ms/step - loss: 0.1776 - binary_accuracy: 0.7925\n",
      "\n",
      "Model Accuracy: 79.25%\n"
     ]
    }
   ],
   "source": [
    "model3L_score = model3L.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\", str(round(model3L_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "DFnp_aHKUAv2",
    "outputId": "7ebcc66f-a10e-4c32-aecd-3980378d0cb5"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV5bn+8e+TOQQImQAhgQQClOAIAZwnhOJQ7aAVO4it1Wqr1ePp6bHtr6312NNjT1utVVttrWPrWD1FxaoUxRkIKCqDEpAhjCEJMyHT8/tjLzTGDQaSvVfIvj/Xta+svda7sp/sDHfe9a71LnN3RERE2koKuwAREemaFBAiIhKVAkJERKJSQIiISFQKCBERiUoBISIiUSkgRA6QmRWbmZtZSjvaXmRmr8SjLpHOooCQhGBmK8yswczy26x/M/gjXxxOZfsXNCLxpICQRPIBcMGeJ2Z2GNAjvHJEujYFhCSS+4ELWz2fCtzXuoGZZZvZfWZWbWYrzez/mVlSsC3ZzH5tZpvMbDlwZpR97zKzdWa2xsxuMLPkjhRsZgPMbJqZ1ZpZpZld0mrbODOrMLOtZrbBzH4brM8wswfMrMbMNpvZXDPr15E6JDEpICSRvAH0NrORwR/uKcADbdr8HsgGhgAnEQmUbwTbLgHOAo4CyoFz2+x7D9AElAZtJgHf6mDNDwFVwIDg9f7bzE4Ntv0O+J279waGAo8E66cGX0MRkAdcBuzqYB2SgBQQkmj29CImAouBNXs2tAqNH7r7NndfAfwG+HrQ5MvAze6+2t1rgV+22rcfcAZwtbvvcPeNwE3B5zsgZlYEHAf8p7vXu/tbwJ/5qBfUCJSaWb67b3f3N1qtzwNK3b3Z3ee5+9YDrUMSlwJCEs39wFeAi2hzeAnIB1KBla3WrQQGBssDgNVttu0xONh3XXBYZzNwB9C3A7UOAGrdfdte6rkYGA4sCQ4jnRWsvx94FnjIzNaa2a/MLLUDdUiCUkBIQnH3lUQGq88AHm+zeROR/74Ht1o3iI96GeuIHLZpvW2P1cBuIN/d+wSP3u4+qgPlrgVyzaxXtHrcfam7X0AkhG4EHjOzLHdvdPefu3sZcCyRw2IXIrKfFBCSiC4GTnX3Ha1XunszkeP4vzCzXmY2GLiGj8YpHgG+Z2aFZpYDXNtq33XAc8BvzKy3mSWZ2VAzO2k/6koPBpgzzCyDSBC8BvwyWHd4UPsDAGb2NTMrcPcWYHPwOVrM7BQzOyw4ZLaVSOi17EcdIoACQhKQuy9z94q9bL4S2AEsB14B/gb8Jdj2JyKHbhYA8/lkD+RCIA1YBNQBjwGH7Edp24kMJu95nErktNxiIr2JJ4CfufuMoP1kYKGZbScyYD3F3XcB/YPX3kpknGUWkcNOIvvFdMMgERGJRj0IERGJSgEhIiJRKSBERCQqBYSIiETVbWaPzM/P9+Li4rDLEBE5qMybN2+TuxdE29ZtAqK4uJiKir2duSgiItGY2cq9bdMhJhERiUoBISIiUcU0IMxsspm9F8xjf22U7Sea2XwzazKzc1utP9LMXjezhWb2tpmdH8s6RUTkk2IWEME8MLcBpwNlwAVmVtam2Sois2r+rc36ncCFwURnk4GbzaxPrGoVEZFPiuUg9Tig0t2XA5jZQ8A5ROapASCYbx8z+9hEYu7+fqvltWa2ESjgownJREQkxmJ5iGkgH587v4qP5rFvNzMbR2QCtGVRtl0a3HKxorq6+oALFRGRT+rSg9RmdgiRWSi/EUxp/DHufqe7l7t7eUFB1NN4RUTkAMUyINbw8ZurFNLq9o6fxsx6A08DP251K8VOt3lnAzc9/z7vrd/26Y1FRBJILANiLjDMzErMLI3IvXmntWfHoP0TwH3u/lgMawTgD7OW8bfZe71WREQkIcUsINy9CbiCyA1WFgOPuPtCM7vezM4GMLOxZlYFnAfcYWYLg92/DJwIXGRmbwWPI2NRZ58eaZx+aH+eeHMN9Y3NsXgJEZGDUkyn2nD36cD0Nut+2mp5LpFDT233e4CPbvMYc1PGDuIfb61l+jvr+OLoT5QjIpKQuvQgdbwcPSSXkvwsHpqz+tMbi4gkCAUEYGacP7aIOStqqdyowWoREVBAfOhLowtJSTL1IkREAgqIQEGvdCaW9ePv86vY3aTBahERBUQrU8YNom5nI88t3BB2KSIioVNAtHJCaT4D+2Ty0NxVYZciIhI6BUQrSUmRwepXK2tYWbMj7HJEREKlgGjjvPJCkgwenqvBahFJbAqINg7JzuSUEX15dF4Vjc2fmB9QRCRhKCCimDJuENXbdjNzycawSxERCY0CIopTRhTQr3c6D83RYLWIJC4FRBQpyUmcN6aIWe9Xs3bzrrDLEREJhQJiL84fW0SLwyMVGqwWkcSkgNiLotwenDAsn0fmrqa5xcMuR0Qk7hQQ+zBl7CDWbqnnpaW637WIJB4FxD5MLOtHXlaaBqtFJCEpIPYhLSWJL40p5F+LN7JxW33Y5YiIxJUC4lOcP7aIphbnsXlVYZciIhJXCohPMbSgJ+NLcnl47mpaNFgtIglEAdEOF4wbxMqanbyxvCbsUkRE4kYB0Q6TD+1PdmYqf9NgtYgkEAVEO2SkJvOFowby3MIN1O5oCLscEZG4UEC00wXjBtHQ3MLj8zVYLSKJQQHRTiP69+KoQX14cM4q3DVYLSLdnwJiP1wwdhDLqndQsbIu7FJERGJOAbEfzjriEHqmp/CgBqtFJAEoIPZDj7QUzj5yANPfWceWXY1hlyMiElMKiP10wdhB1De28I+31oRdiohITCkg9tNhhdmMGtCbB+es1mC1iHRrCogDMGXcIBav28rbVVvCLkVEJGYUEAfgnCMHkJmazENzNVgtIt1XTAPCzCab2XtmVmlm10bZfqKZzTezJjM7t822qWa2NHhMjWWd+6t3RipnHn4I095ay47dTWGXIyISEzELCDNLBm4DTgfKgAvMrKxNs1XARcDf2uybC/wMGA+MA35mZjmxqvVAXDCuiB0NzTy5YG3YpYiIxEQsexDjgEp3X+7uDcBDwDmtG7j7Cnd/G2hps+9ngefdvdbd64DngckxrHW/jR6Uw7C+PXlw7uqwSxERiYlYBsRAoPVfz6pgXafta2aXmlmFmVVUV8f3vtFmxpRxg1iwejOL122N62uLiMTDQT1I7e53unu5u5cXFBTE/fW/eNRA0pKTdM9qEemWYhkQa4CiVs8Lg3Wx3jducrLSmHxof554cw31jc1hlyMi0qliGRBzgWFmVmJmacAUYFo7930WmGRmOcHg9KRgXZczZVwRW+ubmKbBahHpZmIWEO7eBFxB5A/7YuARd19oZteb2dkAZjbWzKqA84A7zGxhsG8t8F9EQmYucH2wrss5ZkgeZYf05rYXKmlqbjvWLiJy8LLuMl1EeXm5V1RUhPLazy/awCX3VfCrLx3Ol8cWffoOIiJdhJnNc/fyaNsO6kHqruK0kX05vDCbW2YupaFJvQgR6R4UEJ3AzPi3icOpqtvFo/N0XYSIdA8KiE5y8vACRg/qw60zK9ndpDOaROTgp4DoJGbGNRNHsG5LPQ/NUS9CRA5+CohOdFxpHuNKcrnthUpdFyEiBz0FRCeK9CKGs3Hbbh54Y2XY5YiIdIgCopMdPSSP40rz+OOsZexs0FTgInLwUkDEwDUTh7NpewP3va5ehIgcvBQQMTBmcC4nDS/gjlnL2K4bConIQUoBESP/NnE4dTsbuefVD8IuRUTkgCggYuTIoj6cNrIvd760nK31jWGXIyKy3xQQMXT1acPZWt/EXS+rFyEiBx8FRAwdOjCbyaP685dXPmDzzoawyxER2S8KiBi7euIwtjc08aeXl4ddiojIflFAxNhn+vfmzMMO4e5XV1C7Q70IETl4KCDi4OrThlHf2Mwds5aFXYqISLspIOKgtG8vzjlyIPe+voLqbbvDLkdEpF0UEHHyvQnDaGx2/vCiehEicnBQQMRJSX4WXzxqIA/MXsmGrfVhlyMi8qkUEHH0vQnDaGlxbnuhMuxSREQ+lQIijopye3BeeREPzVnNms27wi5HRGSfFBBxdsWppQDcOlO9CBHp2hQQcTawTyZTxhXxaMVqVtfuDLscEZG9UkCE4Dsnl5KUZNzyr6VhlyIislcKiBD0z87ga+MH8/iba/hg046wyxERiUoBEZLLTh5CarJ6ESLSdSkgQtK3VwZTjynmH2+toXLjtrDLERH5BAVEiL590lAyU5O5eYZ6ESLS9SggQpSblcZFxxXz9DvrWLJ+a9jliIh8jAIiZJecMISeaSkaixCRLiemAWFmk83sPTOrNLNro2xPN7OHg+2zzaw4WJ9qZvea2TtmttjMfhjLOsPUp0ekFzH9nfW8t15jESLSdcQsIMwsGbgNOB0oAy4ws7I2zS4G6ty9FLgJuDFYfx6Q7u6HAWOAb+8Jj+7om8eVkJWWzC0z1YsQka4jlj2IcUCluy939wbgIeCcNm3OAe4Nlh8DJpiZAQ5kmVkKkAk0AN32IH1OVhpTjy1m+jvrWLpBvQgR6RpiGRADgdWtnlcF66K2cfcmYAuQRyQsdgDrgFXAr929tu0LmNmlZlZhZhXV1dWd/xXE0bdOGEJmajK3aI4mEekiuuog9TigGRgAlAD/bmZD2jZy9zvdvdzdywsKCuJdY6fKzUrjwmOKeerttVRu3B52OSIiMQ2INUBRq+eFwbqobYLDSdlADfAV4J/u3ujuG4FXgfIY1tolXHJCCRkpydyqsQgR6QJiGRBzgWFmVmJmacAUYFqbNtOAqcHyucBMd3cih5VOBTCzLOBoYEkMa+0S8nqmc+Exg5m2YC3Lq9WLEJFwxSwggjGFK4BngcXAI+6+0MyuN7Ozg2Z3AXlmVglcA+w5FfY2oKeZLSQSNHe7+9uxqrUrueTEIaSnJOt+ESISupRYfnJ3nw5Mb7Pup62W64mc0tp2v+3R1ieC/J7pfO3oQdz1ygdcOWEYJflZYZckIgmqqw5SJ7RLTxxKWkqSehEiEioFRBdU0Cudr44fzP+9tYaVNbpfhIiEQwHRRX37pCGkJJl6ESISGgVEF9W3VwZfGT+Ix99cw6oa3btaROJPAdGFXXbSUJKTjNteUC9CROJPAdGF9eudwQVji/j7/CpW16oXISLxpYDo4i47eShJZtz+onoRIhJfCogu7pDsTM4fW8SjFVVU1akXISLxo4A4CFx+8lDM4PYXl4VdiogkkHYFhJllmVlSsDzczM42s9TYliZ7DOiTyZfLi3i0YjVrNu8KuxwRSRDt7UG8BGSY2UDgOeDrwD2xKko+6TunlALwR/UiRCRO2hsQ5u47gS8Ct7v7ecCo2JUlbQ3sk8m5Y4p4eO5q1m1RL0JEYq/dAWFmxwBfBZ4O1iXHpiTZm++cPJQWd/UiRCQu2hsQVwM/BJ4IpuweArwQu7IkmqLcHpw7ppAH565mw9b6sMsRkW6uXQHh7rPc/Wx3vzEYrN7k7t+LcW0SxXdPKaWlxfmDehEiEmPtPYvpb2bWO7i727vAIjP7j9iWJtEU5fbgi6MH8uCcVWxUL0JEYqi9h5jK3H0r8HngGaCEyJlMEoLvnlJKU4vzx1nLwy5FRLqx9gZEanDdw+eBae7eCHjsypJ9GZyXxeePHMhfZ69k4zb1IkQkNtobEHcAK4As4CUzGwxsjVVR8umuOLWUxuYW7lQvQkRipL2D1Le4+0B3P8MjVgKnxLg22YeS/Egv4oHZK6netjvsckSkG2rvIHW2mf3WzCqCx2+I9CYkRFecWkpDUwt/elm9CBHpfO09xPQXYBvw5eCxFbg7VkVJ+wwp6MnZRwzg/tdXsmm7ehEi0rnaGxBD3f1n7r48ePwcGBLLwqR9rjh1GLubmrn9BV0XISKdq70BscvMjt/zxMyOAzQhUBdQ2rcn548t4r7XV1C5cXvY5YhIN9LegLgMuM3MVpjZCuBW4Nsxq0r2y79PGkFmajK/eHpR2KWISDfS3rOYFrj7EcDhwOHufhRwakwrk3bL75nOlRNKeeG9al54b2PY5YhIN7Ffd5Rz963BFdUA18SgHjlAFx1bQkl+Fjc8tYjG5pawyxGRbqAjtxy1TqtCOiwtJYkfnzGSZdU7uP/1lWGXIyLdQEcCQlNtdDETRvblhGH53DzjfWp3NIRdjogc5PYZEGa2zcy2RnlsAwbEqUZpJzPjJ2eVsaOhmZuefz/sckTkILfPgHD3Xu7eO8qjl7unxKtIab/h/XrxtfGD+OvslSxZr+myROTAdeQQ06cys8lm9p6ZVZrZtVG2p5vZw8H22WZW3Grb4Wb2upktNLN3zCwjlrV2J1efNpxeGan811OLcNeRQBE5MDELCDNLBm4DTgfKgAvMrKxNs4uBOncvBW4Cbgz2TQEeAC5z91HAyUBjrGrtbnKy0vi304bxamUNzy/aEHY5InKQimUPYhxQGUzN0QA8BJzTps05wL3B8mPABDMzYBLwtrsvAHD3GndvjmGt3c5Xjx5Mad+e/GL6YnY36a0Tkf0Xy4AYCKxu9bwqWBe1jbs3AVuAPGA44Gb2rJnNN7MfRHsBM7t0zwyz1dXVnf4FHMxSk5P4yVllrKzZyT2vrgi7HBE5CMV0DKIDUoDjga8GH79gZhPaNnL3O9293N3LCwoK4l1jl3fS8AImfKYvv59ZqXtGiMh+i2VArAGKWj0vDNZFbROMO2QDNUR6Gy+5+yZ33wlMB0bHsNZu68dnjmR3UzO/fva9sEsRkYNMLANiLjDMzErMLA2YAkxr02YaMDVYPheY6ZHTbp4FDjOzHkFwnARoJroDMKSgJ1OPKeaReat5d82WsMsRkYNIzAIiGFO4gsgf+8XAI+6+0MyuN7Ozg2Z3AXlmVklkbqdrg33rgN8SCZm3gPnu/nSsau3urpwwjJweaVz/pE57FZH2s+7yB6O8vNwrKirCLqPL+tvsVfzoiXe49StHcdbhugheRCLMbJ67l0fb1lUHqaWTnT+2iJGH9OaX05dQ36jTXkXk0ykgEkRykvHTs8pYs3kXf3ppedjliMhBQAGRQI4ZmsfkUf25/cVlrN9SH3Y5ItLFKSASzI/OGEmzOzf+c0nYpYhIF6eASDCD8nrwreNLeOLNNcxfVRd2OSLShSkgEtB3TimloFc61z+5iJaW7nEWm4h0PgVEAuqZnsIPPjuCt1Zv5h8L2l7cLiISoYBIUF8aXcjhhdn8zzNL2LG7KexyRKQLUkAkqKQk42efK2PD1t38cdaysMsRkS5IAZHAxgzO5ewjBnDnS8tZWbMj7HJEpItRQCS4a0//DClJxhm/e5mbZ7yvw00i8iEFRIIb0CeTJ688nhOHF3DzjKWc9L8vcv8bK2lsbgm7NBEJmQJCGFLQkz98bQyPf+dYhuRn8ZP/e5fP3vQSz7yzTrO/iiQwBYR8aPSgHB7+9tH8+cJykpOMy/86ny/c/hqzl9eEXZqIhEABIR9jZpxW1o9/Xn0iv/rS4azfUs/5d77BxffM5b3128IuT0TiSPeDkH3a1dDMPa+t4PYXK9mxu4kvjS7kmknDOSQ7M+zSRKQT7Ot+EAoIaZe6HQ3c9kIl972+EjP4xnElXH7yULIzU8MuTUQ6QAEhnWZ17U5uev59nnhrDdmZqVxxSilfO3owGanJYZcmIgdAd5STTlOU24Pfnn8kT195AocX9uGGpxcz4TezeG7h+rBLE5FOpoCQA1I2oDf3fXMcf/3WeHpnpnLp/fP4wWML2K4L7US6DQWEdMhxpfn847vHccUppTw2r4rTf/cSFStqwy5LRDqBAkI6LC0lie9/dgSPfPsYDOPLd7zO/z67hIYmXY0tcjBTQEinKS/OZfpVJ3DemCJue2EZX7j9VZZu0LUTIgcrBYR0qp7pKdx47uHc+fUxrN9Sz1m/f4W7X/1Ad64TOQgpICQmJo3qzz+vPpHjS/P5+ZOLmHr3HNZvqQ+7LBHZDwoIiZmCXun8eWo5//2Fw6hYUcdnb36JJxesDbssEWknBYTElJnxlfGDeOaqExhSkMWVD77J1Q+9yZZdjWGXJiKfQgEhcVGcn8Wj3z6GayYO58m31zH55pd4rXJT2GWJyD4oICRuUpKT+N6EYTx++bFkpiXzlT/P5oanFlHf2Bx2aSIShQJC4u6Ioj48feUJXHjMYP78ygecc+urvF21OeyyRKSNmAaEmU02s/fMrNLMro2yPd3MHg62zzaz4jbbB5nZdjP7fizrlPjLTEvm+nMO5Z5vjKVuZwNn3/oq3390ARu36kwnka4iZgFhZsnAbcDpQBlwgZmVtWl2MVDn7qXATcCNbbb/FngmVjVK+E4e0Zd//ftJfPukIUx7ay2n/PpFbnuhUoedRLqAWPYgxgGV7r7c3RuAh4Bz2rQ5B7g3WH4MmGBmBmBmnwc+ABbGsEbpAnplpPLD00fy/DUnclxpPv/77HtMvGmW7oktErJYBsRAYHWr51XBuqht3L0J2ALkmVlP4D+Bn+/rBczsUjOrMLOK6urqTitcwjE4L4s7Lyznb98aT1ZaCpf/dT5T7nyDhWu3hF2aSELqqoPU1wE3ufv2fTVy9zvdvdzdywsKCuJTmcTcsaX5PHXl8dzw+UNZunE7Z/3+FX74+Nts2r477NJEEkpKDD/3GqCo1fPCYF20NlVmlgJkAzXAeOBcM/sV0AdoMbN6d781hvVKF5KSnMTXjh7M544YwC3/Wsq9r63gqQXruHJCKRcdW0JaSlf930ak+4jlb9lcYJiZlZhZGjAFmNamzTRgarB8LjDTI05w92J3LwZuBv5b4ZCYsjNT+clZZTz7bycytiSX/56+hEk3zeL5RRs0PiESYzELiGBM4QrgWWAx8Ii7LzSz683s7KDZXUTGHCqBa4BPnAorAjC0oCd/uWgs935zHCnJSVxyXwVfv2sO763XdOIisWLd5b+w8vJyr6ioCLsMiYPG5hb++sZKbpqxlO27m/jKuEGcV15IcX4WvTNSwy5P5KBiZvPcvTzqNgWEHKzqdjRw84z3eWD2KpqD+03kZqUxOK8HJXlZDM7Loji/B4PzsijJyyK7h8JDpC0FhHRrVXU7eXfNVlbW7GBFzc7Ix007WNvm/hN9eqRGQiMvCI0gPIrzssjNSgupepFw7SsgYnkWk0hcFOb0oDCnxyfW1zc2s7p254eh8cGmHays2cm8lXVMW7CW1v8bnTS8gF9+8TAG9MmMY+UiXZsCQrqtjNRkhvXrxbB+vT6xbXdTM1V1u1ixaQfvrNnCHbOW89mbXuInnyvjvDGFBBf0iyQ0HWISAVbV7OQ/HlvA7A9qOWVEAb/84uH0z84IuyyRmNvXISZdbSQCDMrrwYOXHM3Pzx7FG8trmXjTLB6bV6VrLSShKSBEAklJxtRji/nn1Scwsn9vvv/oAi6+t4INmoJcEpQCQqSNwXlZPHTp0fz0rDJeW7aJib+dxePz1ZuQxKOAEIkiKcn45vElPHPViQzv14trHlnAJffNY+M29SYkcSggRPahJD+Lh799DP/vzJG8vLSaSTe9xD/eWqPehCQEBYTIp0hOMr51whCmX3UCJflZXPXQW1z2wDyqt2n6ceneFBAi7TS0oCePXXYsPzrjM7zwXjWTbprFkwvWqjch3ZYCQmQ/JCcZl544lOnfO55BeVlc+eCbfOev81levV1BId2OLpQTOUBNzS386eUPuOn592lobmFgn0xOGJbPcaX5HDs0j7ye6WGXKPKpNFmfSAyt2byLmUs28urSTby2bBNb65sAGDWgN8eXRgJjXEkuGanJIVcq8kkKCJE4aW5x3lmzhVeWVvNK5SbmrayjsdlJS0mifHAOx5Xmc8KwfEYNyCY5SfM9SfgUECIh2dnQxJwPanm1chMvL93EkuAOeNmZqRw7NI/jh+VzfGk+g/OyQq5UEpWm+xYJSY+0FE4e0ZeTR/QFoHrbbl5btolXlm7ilcpNPPPuegAG9slkXEku40tyGT8kj+K8HppRVkKnHoRISNyd5Zt28GrlJt5YXsOcD2rZtL0BgL690iOBMSSPo0tyKe3bU4EhMaFDTCIHAXdnWfUOZn9Qw+zltcz+oIYNWyMX4+VmpTGuOJfxQ3IZX5LHZ/r3IkljGNIJdIhJ5CBgZpT27Ulp3558dfxg3J1VtTuDsIgExj8XRg5J9c5ICQ5J5TGuJJdRA3qTkqzLmqRzKSBEuigzY3BeFoPzsvjy2CIgckrtnA97GLXMWLwRgKy0ZEYPzmFscS7lxTkcVZRDZppOq5WO0SEmkYPYxq31zP6glrkrapnzQS3vbdiGO6QkGYcOzGZcSW4kNAbnkJOVFna50gVpDEIkQWzZ1cj8lXXMWVFLxYpaFqzeQkNzCwDD+vZkbEkuY4sjPY3CnB4hVytdQcIGRGNjI1VVVdTXd/85/DMyMigsLCQ1NTXsUqQLqW9s5u2qLcxdEellzFtRx7bdkSu9B2RnUF6cy9iSSA9jeL9eungvASXsIHVVVRW9evWiuLi4W58i6O7U1NRQVVVFSUlJ2OVIF5KRmsy4klzGleQCkSu931u/LXJIakUtbyyvYdqCtQD0TE/hiKJsRg/KYfTgHEYX5ZDdQ/9wJLJuHRD19fXdPhwgMpiZl5dHdXV12KVIF5ecZJQN6E3ZgN5MPbYYd2d17S7mrapl/srNzF9Vx+0vLqO5JXJkYWhB1keBMSiHYX176vTaBNKtAwLo9uGwR6J8ndK5zIxBeT0YlNeDLxxVCMCO3U0sqNrMm6s2M39lHTMWb+DReVUA9MpI4ciiPh+GxpFFfcjOVC/jQKzfUs+8lXW8uaqOrfWN5Galk5uV+vGPPdLI7ZlGVlpyKL/j3T4gRGT/ZKWncOzQfI4dmg9EDmGuqNnJ/JV1zF9Vx7yVdfx+5lJaHMygtKAnxw7N47SyfowvySMtRddjtNXQ1MKidVs/fA/nr6xj7ZbI2Gh6ShJ9eqRSu6OBxuboY8JpyUnkZqWRk5VGXuuPQYAU5WR+OJ1LZ1JAxFBNTQ0TJkwAYP369SQnJ1NQUADAnDlzSEvb+2mHFRUV3Hfffdxyyy1xqVVkb8yMkvwsSvKz+NKYSC9j++4mFqyO9DDmrarj4YrV3Pv6SgwieNsAAA1SSURBVHqmp3DS8AJOK+vLKSP60qdHYp5aW71t94dBMH9VHW9XbWF3U+RssoF9Mhk9OIdvBb2wskN6k5aShLuzfXcTdTsaqdmxm7qdDdRsb4h83NFAbavlqrqd1OxoYFswtfzoQX1iEhDd+iymxYsXM3LkyJAq+rjrrruOnj178v3vf//DdU1NTaSkdF5Gd6WvVxJLfWMzr1ZuYsbiDcxYvJHqbbtJTjLKB+cwsawfE0b2oyS/e85Y29TcwpL12z4MhHmr6lhduwuI/Oc/amBvxrQax+mfndFpr93Q1MLmnQ00NLcc8GnLoZ3FZGaTgd8BycCf3f1/2mxPB+4DxgA1wPnuvsLMJgL/A6QBDcB/uPvMjtTy8ycXsmjt1o58ik8oG9Cbn31u1H7tc9FFF5GRkcGbb77Jcccdx5QpU7jqqquor68nMzOTu+++mxEjRvDiiy/y61//mqeeeorrrruOVatWsXz5clatWsXVV1/N9773vU79WkQ6IiM1mQkjI0Hwixbn7TVbmLFoAzMWb+CGpxdzw9OLGVqQxWll/Zg4sh9HDcrZ71NqdzY0saZuF1Wbd1FVt4uqup2R53W7yO+ZzqSyfkwY2Tcud/Lb3dTMa5U1TH9nHc8v3sDmnY1AZJLFMYNzuPDoYkYPzmHUgN4xvVFUWkoSfXt3XuC0FbOAMLNk4DZgIlAFzDWzae6+qFWzi4E6dy81synAjcD5wCbgc+6+1swOBZ4FBsaq1nirqqritddeIzk5ma1bt/Lyyy+TkpLCjBkz+NGPfsTf//73T+yzZMkSXnjhBbZt28aIESO4/PLLdc2DdElJScaRRX04sqgP3//sCFbX7uRfQc/irpc/4I5Zy8nNSuPUz/TltJH9OGFYPlnpKWzfHQRA3U7WRAmBmh0NH3udtOQkBvTJYGBOJovXbWXG4g0kGYwZnMOksv5MGtWvU++zUd/YzEvvV/PMu+uZsWgD23Y30Ss9hdPK+nHyiALGDM5hYJ/MbnXCSCx7EOOASndfDmBmDwHnAK0D4hzgumD5MeBWMzN3f7NVm4VAppmlu/vuAy1mf//Tj6XzzjuP5OTIfxVbtmxh6tSpLF26FDOjsbEx6j5nnnkm6enppKen07dvXzZs2EBhYWE8yxY5IEW5PbjouBIuOq6ErfWNzHqvmhmLN/DcwvU8Nq+KtJQkstKSqdv58Z/9tJQkCnMyGdgnk0kDsinMyWz16EFBz/QPT7l1dxat28rzizbw3MIN/GL6Yn4xfTEj+vVi0qh+TCrrz6EDe+/3H++dDU28sKSaZ95dx8wlG9nZ0EyfHqlMPrQ/Zxx2CMeW5pGe0n3nvIplQAwEVrd6XgWM31sbd28ysy1AHpEexB5fAuZHCwczuxS4FGDQoEGdV3mMZWV99F/NT37yE0455RSeeOIJVqxYwcknnxx1n/T0j7rNycnJNDU1xbpMkU7XOyOVzx0xgM8dMYDG5hbmrqhl5uKN7GpspjCnx4cBMDAnk/ys9HZfc2FmjBqQzagB2Vx92nBW1+6MhMWi9dz2QiW/n1nJIdkZTCyLhMX4Ibmk7mX22231jcxcspHp76xj1vvV1De2kJeVxuePGsjph/bn6CF5e923u+nSZzGZ2Sgih50mRdvu7ncCd0JkkDqOpXWaLVu2MHBg5OjZPffcE24xInGUmpz0sdNpO1NRbg++eXwJ3zy+hNodDcxcspHnFq7nkYrV3Pf6SnpnpHDqZ/oysaw/J40ooKm5hecXbeCf767n5aWbaGhuoW+vdM4vL+L0ww5hbHFuQk5DEsuAWAMUtXpeGKyL1qbKzFKAbCKD1ZhZIfAEcKG7L4thnaH6wQ9+wNSpU7nhhhs488wzwy5HpNvJzUrj3DGFnDumkF0NzbxSuYnnFq5nxuIN/N9ba0lLTqLFnaYWZ2CfTL5+zGDOOKw/RxXlJPxV4zE7zTX4g/8+MIFIEMwFvuLuC1u1+S5wmLtfFgxSf9Hdv2xmfYBZwM/d/fH2vF5XP801HhLt6xXpiKbmFuYFV4qnJCcxeVR/Di/M7laDzO0RymmuwZjCFUTOQEoG/uLuC83seqDC3acBdwH3m1klUAtMCXa/AigFfmpmPw3WTXL3jbGqV0QSS0pyEuOH5DF+SF7YpXRZMR2DcPfpwPQ2637aarkeOC/KfjcAN8SyNhER2bduPxTfXa4U/zSJ8nWKSPx064DIyMigpqam2//x3HM/iIyM2F1RKSKJp0uf5tpRhYWFVFVVJcR9EvbcUU5EpLN064BITU3VHdZERA5Qtz7EJCIiB04BISIiUSkgREQkqm5zwyAzqwZWduBT5PPxSQK7GtXXMaqvY1Rfx3Tl+ga7e0G0Dd0mIDrKzCr2drl5V6D6Okb1dYzq65iuXt/e6BCTiIhEpYAQEZGoFBAfuTPsAj6F6usY1dcxqq9junp9UWkMQkREolIPQkREolJAiIhIVAkVEGY22czeM7NKM7s2yvZ0M3s42D7bzIrjWFuRmb1gZovMbKGZXRWlzclmtsXM3goeP432uWJc5wozeyd4/Yoo283Mbgnew7fNbHQcaxvR6r15y8y2mtnVbdrE9T00s7+Y2UYze7fVulwze97MlgYfc/ay79SgzVIzmxrH+v7XzJYE378ngjs8Rtt3nz8LMazvOjNb0+p7eMZe9t3n73sM63u4VW0rzOytvewb8/evw9w9IR5E7mq3DBgCpAELgLI2bb4D/DFYngI8HMf6DgFGB8u9iNyutW19JwNPhfw+rgDy97H9DOAZwICjgdkhfr/XE7kIKLT3EDgRGA2822rdr4Brg+VrgRuj7JcLLA8+5gTLOXGqbxKQEizfGK2+9vwsxLC+64Dvt+P7v8/f91jV12b7b4CfhvX+dfSRSD2IcUCluy939wbgIeCcNm3OAe4Nlh8DJlicblDr7uvcfX6wvA1YDAyMx2t3snOA+zziDaCPmR0SQh0TgGXu3pGr6zvM3V8icjvd1lr/nN0LfD7Krp8Fnnf3WnevA54HJsejPnd/zt2bgqdvAKHNI7+X96892vP73mH7qi/42/Fl4MHOft14SaSAGAisbvW8ik/+Af6wTfALsgWI+w1rg0NbRwGzo2w+xswWmNkzZjYqroVFOPCcmc0zs0ujbG/P+xwPU9j7L2bY72E/d18XLK8H+kVp01Xex28S6RFG82k/C7F0RXAI7C97OUTXFd6/E4AN7r50L9vDfP/aJZEC4qBgZj2BvwNXu/vWNpvnEzlkcgTwe+D/4l0fcLy7jwZOB75rZieGUMM+mVkacDbwaJTNXeE9/JBHjjV0yXPNzezHQBPw1700Cetn4Q/AUOBIYB2Rwzhd0QXsu/fQ5X+XEikg1gBFrZ4XBuuitjGzFCAbqIlLdZHXTCUSDn9198fbbnf3re6+PVieDqSaWX686gted03wcSPwBJGufGvteZ9j7XRgvrtvaLuhK7yHwIY9h92CjxujtAn1fTSzi4CzgK8GIfYJ7fhZiAl33+Duze7eAvxpL68b9vuXAnwReHhvbcJ6//ZHIgXEXGCYmZUE/2FOAaa1aTMN2HO2yLnAzL39cnS24HjlXcBid//tXtr03zMmYmbjiHz/4hlgWWbWa88ykcHMd9s0mwZcGJzNdDSwpdXhlHjZ639uYb+HgdY/Z1OBf0Rp8ywwycxygkMok4J1MWdmk4EfAGe7+869tGnPz0Ks6ms9pvWFvbxue37fY+k0YIm7V0XbGOb7t1/CHiWP54PIGTbvEzm74cfBuuuJ/CIAZBA5LFEJzAGGxLG244kcangbeCt4nAFcBlwWtLkCWEjkjIw3gGPj/P4NCV57QVDHnvewdY0G3Ba8x+8A5XGuMYvIH/zsVutCew+JBNU6oJHIcfCLiYxr/QtYCswAcoO25cCfW+37zeBnsRL4RhzrqyRy/H7Pz+GeM/sGANP39bMQp/ruD3623ibyR/+QtvUFzz/x+x6P+oL19+z5mWvVNu7vX0cfmmpDRESiSqRDTCIish8UECIiEpUCQkREolJAiIhIVAoIERGJSgEhsh/MrLnNjLGdNkuomRW3nhVUJGwpYRcgcpDZ5e5Hhl2ESDyoByHSCYK5/X8VzO8/x8xKg/XFZjYzmFjuX2Y2KFjfL7jXwoLgcWzwqZLN7E8WuSfIc2aWGdoXJQlPASGyfzLbHGI6v9W2Le5+GHArcHOw7vfAve5+OJFJ724J1t8CzPLIpIGjiVxNCzAMuM3dRwGbgS/F+OsR2StdSS2yH8xsu7v3jLJ+BXCquy8PJl1c7+55ZraJyFQQjcH6de6eb2bVQKG77271OYqJ3ANiWPD8P4FUd78h9l+ZyCepByHSeXwvy/tjd6vlZjROKCFSQIh0nvNbfXw9WH6NyEyiAF8FXg6W/wVcDmBmyWaWHa8iRdpL/52I7J/MNjeh/6e77znVNcfM3ibSC7ggWHclcLeZ/QdQDXwjWH8VcKeZXUykp3A5kVlBRboMjUGIdIJgDKLc3TeFXYtIZ9EhJhERiUo9CBERiUo9CBERiUoBISIiUSkgREQkKgWEiIhEpYAQEZGo/j+8G+C+ihuVQwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model3L_hist.history['loss'])\n",
    "\n",
    "plt.title('Model Loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['Train'], loc='lower left')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NXOzFtS9UAv2"
   },
   "source": [
    "Let's save our model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TQBHSUWcUAv3"
   },
   "outputs": [],
   "source": [
    "model3L.save('models/LSTM-3L-Model.h5')  # save model\n",
    "del model3L  # delete existing model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "UW1iX4lWY3QK"
   },
   "outputs": [],
   "source": [
    "# returns a compiled model, identical to the previous one\n",
    "model3L = load_model('models/LSTM-3L-Model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kmrOB9YMUAv4"
   },
   "source": [
    "Let's check if everything is correct:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VDa2MsFuUAv4",
    "outputId": "ade31e28-25b8-4ee2-904a-54be20e9e832"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44/44 [==============================] - 6s 96ms/step - loss: 0.0916 - binary_accuracy: 0.8939\n",
      "\n",
      "Model Accuracy: 89.39%\n"
     ]
    }
   ],
   "source": [
    "model3L_score = model3L.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\", str(round(model3L_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7P_6TlCeY8pr"
   },
   "source": [
    "<h2 align=\"center\">Bidirectional LSTM</h2>\n",
    "\n",
    "Bidirectional LSTMs are an extension of traditional LSTMs that can improve model performance on sequence classification problems.\n",
    "\n",
    "In problems where all timesteps of the input sequence are available, Bidirectional LSTMs train two instead of one LSTMs on the input sequence. The first on the input sequence as-is and the second on a reversed copy of the input sequence. This can provide additional context to the network and result in faster and even fuller learning on the problem.\n",
    "\n",
    "The idea of Bidirectional Recurrent Neural Networks (RNNs) is straightforward.\n",
    "\n",
    "It involves duplicating the first recurrent layer in the network so that there are now two layers side-by-side, then providing the input sequence as-is as input to the first layer and providing a reversed copy of the input sequence to the second."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zwElja8XZENL",
    "outputId": "3a713c59-59eb-4575-e094-51bb3741e964"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " bidirectional (Bidirectiona  (None, 50, 128)          186880    \n",
      " l)                                                              \n",
      "                                                                 \n",
      " bidirectional_1 (Bidirectio  (None, 128)              98816     \n",
      " nal)                                                            \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 128)               0         \n",
      "                                                                 \n",
      " dense (Dense)               (None, 1)                 129       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 285,825\n",
      "Trainable params: 285,825\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "from keras.layers.core import Dropout\n",
    "from keras.layers import Bidirectional\n",
    "\n",
    "# configure network\n",
    "n_epoch = 20\n",
    "n_batch = 64\n",
    "\n",
    "# design network\n",
    "model_Bi_LSTM = Sequential()\n",
    "model_Bi_LSTM.add(Bidirectional(LSTM(64, return_sequences=True), input_shape=(x.shape[1], x.shape[2])))\n",
    "model_Bi_LSTM.add(Bidirectional(LSTM(64)))\n",
    "model_Bi_LSTM.add(Dropout(0.5))\n",
    "model_Bi_LSTM.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "# try using different optimizers and different optimizer configs\n",
    "model_Bi_LSTM.compile('adam', 'binary_crossentropy', metrics=['binary_accuracy'])\n",
    "print(model_Bi_LSTM.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "J1xZy08-ZFtv",
    "outputId": "8fa01d83-7375-411f-9557-38b2680de5e3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "264/264 [==============================] - 72s 242ms/step - loss: 0.5121 - binary_accuracy: 0.7432 - val_loss: 0.4397 - val_binary_accuracy: 0.7984\n",
      "Epoch 2/20\n",
      "264/264 [==============================] - 76s 290ms/step - loss: 0.3840 - binary_accuracy: 0.8297 - val_loss: 0.4256 - val_binary_accuracy: 0.8034\n",
      "Epoch 3/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.3016 - binary_accuracy: 0.8766 - val_loss: 0.4322 - val_binary_accuracy: 0.8107\n",
      "Epoch 4/20\n",
      "264/264 [==============================] - 63s 240ms/step - loss: 0.2179 - binary_accuracy: 0.9155 - val_loss: 0.4585 - val_binary_accuracy: 0.8187\n",
      "Epoch 5/20\n",
      "264/264 [==============================] - 64s 242ms/step - loss: 0.1425 - binary_accuracy: 0.9473 - val_loss: 0.5824 - val_binary_accuracy: 0.8069\n",
      "Epoch 6/20\n",
      "264/264 [==============================] - 64s 244ms/step - loss: 0.0920 - binary_accuracy: 0.9675 - val_loss: 0.6255 - val_binary_accuracy: 0.8089\n",
      "Epoch 7/20\n",
      "264/264 [==============================] - 64s 243ms/step - loss: 0.0770 - binary_accuracy: 0.9719 - val_loss: 0.7419 - val_binary_accuracy: 0.8060\n",
      "Epoch 8/20\n",
      "264/264 [==============================] - 64s 242ms/step - loss: 0.0578 - binary_accuracy: 0.9795 - val_loss: 0.7371 - val_binary_accuracy: 0.8076\n",
      "Epoch 9/20\n",
      "264/264 [==============================] - 63s 241ms/step - loss: 0.0371 - binary_accuracy: 0.9887 - val_loss: 0.8497 - val_binary_accuracy: 0.8126\n",
      "Epoch 10/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.0362 - binary_accuracy: 0.9881 - val_loss: 0.8730 - val_binary_accuracy: 0.8020\n",
      "Epoch 11/20\n",
      "264/264 [==============================] - 63s 240ms/step - loss: 0.0267 - binary_accuracy: 0.9912 - val_loss: 0.8611 - val_binary_accuracy: 0.7947\n",
      "Epoch 12/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.0312 - binary_accuracy: 0.9895 - val_loss: 0.9707 - val_binary_accuracy: 0.8089\n",
      "Epoch 13/20\n",
      "264/264 [==============================] - 63s 240ms/step - loss: 0.0275 - binary_accuracy: 0.9903 - val_loss: 1.0035 - val_binary_accuracy: 0.8066\n",
      "Epoch 14/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.0165 - binary_accuracy: 0.9948 - val_loss: 1.0076 - val_binary_accuracy: 0.8012\n",
      "Epoch 15/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.0220 - binary_accuracy: 0.9928 - val_loss: 0.9767 - val_binary_accuracy: 0.8020\n",
      "Epoch 16/20\n",
      "264/264 [==============================] - 63s 240ms/step - loss: 0.0327 - binary_accuracy: 0.9882 - val_loss: 1.0082 - val_binary_accuracy: 0.8021\n",
      "Epoch 17/20\n",
      "264/264 [==============================] - 64s 241ms/step - loss: 0.0217 - binary_accuracy: 0.9929 - val_loss: 0.9770 - val_binary_accuracy: 0.8027\n",
      "Epoch 18/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.0116 - binary_accuracy: 0.9963 - val_loss: 1.1661 - val_binary_accuracy: 0.8117\n",
      "Epoch 19/20\n",
      "264/264 [==============================] - 63s 239ms/step - loss: 0.0131 - binary_accuracy: 0.9950 - val_loss: 1.0535 - val_binary_accuracy: 0.8096\n",
      "Epoch 20/20\n",
      "264/264 [==============================] - 63s 240ms/step - loss: 0.0235 - binary_accuracy: 0.9917 - val_loss: 0.9686 - val_binary_accuracy: 0.7993\n"
     ]
    }
   ],
   "source": [
    "model_Bi_LSTM_hist = model_Bi_LSTM.fit(x_train, np.asarray(y_train), epochs=n_epoch, batch_size=n_batch, verbose=1, validation_data=(x_val, np.asarray(y_val)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EFt5Tg70ZLB4",
    "outputId": "6b9a5b49-fb63-4096-9dd6-59a068b91e21"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "88/88 [==============================] - 6s 72ms/step - loss: 0.9686 - binary_accuracy: 0.7993\n",
      "\n",
      "Model Accuracy: 79.93%\n"
     ]
    }
   ],
   "source": [
    "model_Bi_LSTM_score = model_Bi_LSTM.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\",str(round(model_Bi_LSTM_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "f2uJjnDeZL68",
    "outputId": "31134fdc-9183-4883-9631-1cc56db6f52b"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwU5f3A8c8354acXOEKl4BcgoIRBayKbRWtFVRE8ESpVFu19ra2tmr766G2Vet94NmKikfRilKtNx4EEOS+j3CFMwmQO9/fH88ElpBAjt2dJPt9v17z2tmZ2Znvbjbz3XmeZ55HVBVjjDHRK8bvAIwxxvjLEoExxkQ5SwTGGBPlLBEYY0yUs0RgjDFRzhKBMcZEOUsExhyFiPQQERWRuDpsO0lEPolEXMaEiiUC06KIyDoRKRWRdtWWz/dO5j38iax+CcWYSLJEYFqitcDEqiciMgho5V84xjRtlghMS/QccGXQ86uAZ4M3EJF0EXlWRLaLyHoR+Y2IxHjrYkXkHhHZISJrgO/U8NonRWSLiGwSkT+ISGxjAhaRziIyQ0R2icgqEbk2aN0wEckRkQIR2SYif/OWB0TkeRHZKSJ7RGSOiHRoTBwmOlkiMC3R50CaiPT3TtATgOerbfMPIB04Bjgdlziu9tZdC5wHDAGygXHVXvs0UA709rY5C/heI2OeBuQCnb3j/VFEzvTW3Qfcp6ppQC/gJW/5Vd576Aq0Ba4DihoZh4lClghMS1V1VfBtYCmwqWpFUHL4laoWquo64K/AFd4m44F7VXWjqu4C/hT02g7AucDNqrpPVfOAv3v7axAR6QqMBH6pqsWq+hXwBAevasqA3iLSTlX3qurnQcvbAr1VtUJV56pqQUPjMNHLEoFpqZ4DLgUmUa1YCGgHxAPrg5atB7p4852BjdXWVenuvXaLVxyzB3gUyGxErJ2BXapaWEs8k4FjgWVe8c953vLngHeAaSKyWUTuEpH4RsRhopQlAtMiqep6XKXxucCr1VbvwP2a7h60rBsHrxq24IpbgtdV2QiUAO1UNcOb0lR1YCPC3Qy0EZHUmuJR1ZWqOhGXbP4CTBeRZFUtU9U7VHUAMAJXnHUlxtSTJQLTkk0GzlTVfcELVbUCV87+fyKSKiLdgZ9wsB7hJeAmEckSkdbALUGv3QLMAv4qImkiEiMivUTk9HrElehV9AZEJIA74c8G/uQtG+zF/jyAiFwuIu1VtRLY4+2jUkRGicggr6irAJfcKusRhzGAJQLTgqnqalXNqWX1jcA+YA3wCfAvYKq37nFckcsCYB6HX1FcCSQAS4DdwHSgUz1C24ur1K2azsQ1d+2Buzp4Dfidqr7rbT8aWCwie3EVxxNUtQjo6B27AFcP8iGuuMiYehEbmMYYY6KbXREYY0yUs0RgjDFRzhKBMcZEOUsExhgT5ZpdL4jt2rXTHj16+B2GMcY0K3Pnzt2hqu1rWhe2RCAiU3E3uOSp6nE1rL8M+CUgQCFwvaouONp+e/ToQU5ObS0CjTHG1ERE1te2LpxFQ0/j2j/XZi1wuqoOAn4PPBbGWIwxxtQibFcEqvrRkQYBUdXZQU8/B7LCFYsxxpjaNZXK4snAzNpWisgUrz/2nO3bt0cwLGOMafl8TwQiMgqXCH5Z2zaq+piqZqtqdvv2NdZ1GGOMaSBfWw15nWs9AZyjqjv9jMUYY6KVb1cEItIN15nXFaq6wq84jDEm2oWz+egLwBlAOxHJBX6HG9ADVX0E+C1udKWHRASgXFWzwxWPMcaYmoWz1dDEo6z/Ho0f59UYYyJjyb+h8xDI6Hb0bZsZ3yuLjTGmydvwBbx0JTxzPuzf5Xc0IWeJwBhjjubDv0AgAwo2uYRQUeZ3RCFlicAYY44kNwdWvwen/hjOfwDWfQxv/Rxa0KBeza7TOWOMiagP/gxJbeCk70FiCmxfCp/8HTL7w8nf9zu6kLArAmOMqc2mubDqvzDiBpcEAM78LfQ9F96+BVa95298IWKJwBhjavPhXZDUGoZNObgsJgYufAza94eXr4YdK/2LL0QsERhjTE02z4cVb8PwH0Ji6qHrElPh0mkQGw//uqTZtySyRGCMMTX58G4IpB96NRAsoxtM+Cfkb4SXJzXrlkSWCIwxprotC2H5f+CUH7pkUJtup8B374O1H7o6g2bKWg0ZY0x1H/4FEtPr1irohEshbynMvh/a94Nh14Y/vhCzKwJjjAm2dREsexNOuQ6SMur2mm/dDseOhpm/hNXvhzO6sLBEYIwxwT66CxLT4JTr6/6amFi48HFodyy8fBXsXB2++MLAEoExxlTZtsR1Lnfy912z0foIpLmWRDFxriVR0Z7wxBgGlgiMMabKR3dBQgqc8oOGvb51D7jkedi9DqZfDRXloYwubCwRGGMMQN4yWPy6ay7aqk3D99N9BJz3d1j9P3jn1tDFF0bWasgYYwA+uhviW8HwGxq/r6FXwPZl8NkDkNkPsq9p/D7DyK4IjDFm+wpY9AoM+x4ktw3NPr99J/Q5y/VUuvaj0OwzTCwRGGPMx/dAfBKMuCl0+4yJhYuehLa94cUrmnRLIksExpjotmMVfP0ynDQZktuFdt+BNJj4AojACxOgOD+0+w8RSwTGmOj28T0Qmxjaq4FgbY5xLYl2rYFnx8K+HeE5TiNYIjDGRK+dq2HhS64yNyUzfMfpcSqMfw7ylsDUs2HPhvAdqwEsERhjotfHf3NdSY8M09VAsH7nwhWvw77t8ORZ7ua1JsISgTEmOu1aCwtegBMnQWrHyByz+3C4eqabf2o0rP8sMsc9CksExpjo9PFfXXcQI2+O7HE7DITJsyA5E54bC8veiuzxa2CJwBgTfXavd1cDQ6+EtE6RP35GN7jmHZcUXrwM5j0b+RiCWCIwxkSfT/4GEgOn/ti/GJLbwpUz4JhRMONGd4Wi6ksoYetiQkSmAucBeap6XA3rBbgPOBfYD0xS1XnhiscYcxQV5VC2D0r3Q+m+g/NVj+UlkHWiaw4Zafm5sOpdd3NWt+HuZq2G2rMR5v/TXQ2kdwldjA2RmAITp8G/fwDv3Ql78+DsP0FMZH+jh7OvoaeBB4DarnnOAfp408nAw96jMSaUyktg5X9h2X9ci5WyqhO991g1X1Fat/11HAQDxsLAC6Btr/DFXbDZdQm9+DXY+MXB5SkdoP933fEbkhQ++bt79PNqIFhcAlzwGCS3h88fcvcZjH3YLY9UCOHasap+JCI9jrDJGOBZVVXgcxHJEJFOqrolXDEZEzUqK2Ddx+6O2SVvQEk+JLVxZdMJye6kk5DspvhWkNDKdb9cNR/vrTsw38oVpax+H5a8Dv/7vZs6HOclhbHQrk/j4y7cCktmwOJXYYPXoqbDIDjzNuh7ruvIbfFr7hf9nCdcheuA810M3UccPSnkb4L5z8GQyyCja+PjDZWYGDj7j+5ehndvh/074ZLnIDE1IocXDWOZlJcI3qylaOhN4M+q+on3/D3gl6qaU8O2U4ApAN26dTtx/fr1YYvZmGZLFTbNg0XTXQdqe7e5k3v/78Jx4+CY012b+VDIz3Un7CWvH/y1njngYFJo37fu+9qb5/3yfx3Wfwqo29fAC9z+2h97+GtK9sLKWe74K2ZBeZFLClVXCrUlhf/8DOY+BTfOg9bdG/TWw27+8zDjJug0GC6bHrJuL0Rkrqpm17iuOSSCYNnZ2ZqTc8RNjIku25fD19Pdr//dayE2wfV6OWicG0c3Pim8x8/fBEvfcCflDZ8D6gZxr0oKmf0Pf82+HbB0hvt1v+4T0Eo3zOPAC92JPLNf3Y9fug9WvFNLUhgL3Ue6pFCwGe47HgZfAmMeCNnbD4vlM+HlSZDWBa54LSRJq6kmgkeBD1T1Be/5cuCMoxUNWSIwBveLfNErLgFsXeiKbXp8AwZd7E6AdR10PdQKthxMCutnAwrt+sKAMS4pbVvkin3Wfgxa4Sp/D5z8+7vO2RqjdJ+7Ulj8uksO5UWuGKz/+a64ZekbcONcaNMzJG83rDZ8Dv8aD3FJcPkr0PGw02i9NNVE8B3gBlyroZOB+1V12NH2aYnARK39u9wJ9uvpXhEK0OVEd/IfeEHk7o6tq8KtXlL4t4tXK93yNse4eAde6NrRN/bkX5vgpLBylqsQP+FyGPtgeI4XDnlL4bkL3XuZ+AL0GNngXfmSCETkBeAMoB2wDfgdEA+gqo94zUcfAEbjmo9efbRiIbBEYKLUhi/cXahl+10RyqDxcNyF4W21E0p782DNB67uoOPg8J38a1O6z3Xn0HWY6xq6OdmzEZ67wHVUN24q9D+vQbvx7YogHCwRmKizay088U0IpMPFT/tzIjX+2rfTFRMNuRyyr27QLo6UCGzMYmOasqI97gRQWQGXvgztevsdkfFDclvXJUVseE7ZlgiMaaoqyuClK90VwZWvWxKIdmFKAmCJwJimSRXe/DGs/dDdZdrjVL8jMi2YdTpnTFP06X3uDthv/AxOuNTvaEwLZ4nAmKZmyQx493eueeWoX/sdjYkClgiMaUo2zYVXp0DWSTD2oYj3Qmmik33LjGkq9myEf02AlPYw4YXwdw1hjMcqi41pCooLXDPR8hK46g2XDIyJEEsExvitohymX+06j7v8lfp1uGZMCFgiMMZPqjDzF270re/eB71G+R2RiUJWR2CMnz5/GHKehBE3wYmT/I7GRClLBMb4ZflMeOdW6HcefOsOv6MxUcwSgTF+2LIApk+GzifAhY9bM1HjK/v2GRNpBZvhX5dAUmuYOM2NB2yMj6yy2Ji62JsHm+dDIMMNG5ic2bBf8SV7XRIoKXS9STa1wWRMVLJEYExN9ua5sXSrph3LD10fF4D0ri4pZHSDjO7evDe1anP4mAGVFfDK99xwjRNfbPTQg8aEiiUCYwAKt8H64BP/Crc8IQW6DXcdv3Ud5n7R71kPu9e5EaP2rIfcHCjec+j+ElKCkoOXKLYthhUz4dx74NizIv4WjamNJQITnQq3Hjzpr/806MSfCt2Hu5GgepwKHY+vWz/wxfkuMexe7yWK9d7zdbDmQyjb57Y7+ToYdm3Y3pYxDWGJwEQHVVj2Jqx6z538d650yxPT3C/+IVd4J/7BDRsAJJAOHQe5qaZj79/pRhtrLmMMm6hiicBEh4/vgf/9wZ34u4+AoVc27sRfHyKQ3M5NxjRBlghMy7dkhksCg8bDBY9ATKzfERnTpNh9BKZl27IQXvs+dMmG8/9hScCYGlgiMC1X4TZ4YaK7cWvCvyA+4HdExjRJVjRkWqayYnjxMijaBde8Dakd/I7ImCbLEoFpeVThjR9B7hwY/yx0Ot7viIxp0sJaNCQio0VkuYisEpFbaljfTUTeF5H5IrJQRM4NZzwmSnx6Lyyc5gZ+HzDG72iMafLClghEJBZ4EDgHGABMFJEB1Tb7DfCSqg4BJgAPhSseEyWWvQXv3gHHXQSn/dzvaIxpFsJ5RTAMWKWqa1S1FJgGVP95pkCaN58ObA5jPKal27rI9eXT+QQY8+Dhff0YY2oUzkTQBdgY9DzXWxbsduByEckF3gJurGlHIjJFRHJEJGf79u3hiNU0d3u3uxZCgTSY8ALEJ/kdkTHNht/NRycCT6tqFnAu8JyIHBaTqj6mqtmqmt2+ffuIB2mauPISeOkK2JfnmommdfI7ImOalXAmgk1A16DnWd6yYJOBlwBU9TMgANh9+KbuVOHNn8CGz2DsQ9BlqN8RGdPshDMRzAH6iEhPEUnAVQbPqLbNBuCbACLSH5cIrOzH1N1nD8BXz8Ppv3QVxMaYegtbIlDVcuAG4B1gKa510GIRuVNEzvc2+ylwrYgsAF4AJqmqhism08KsmAWzbnNNRE8/rHWyMaaOwnpDmaq+hasEDl7226D5JcDIcMZgWqi8pTD9Gtft89iHbfB3YxrB/ntM87Nvpxv3N6GVN/h7st8RGdOsWRcTpnkpL4WXrnQjjF39FqRXb5FsjKkvSwSm+VCFt37mxha+8AnIyvY7ImNaBEsEJjK2LoLXr3O/6BNT3ODuialuSkhxyxJT3ZjBNa5PhcWvwbxn4Bs/hcEX+/2OjGkxLBGY8Nu/C6ZdCmVFbmD4kr1QUgj7trv50kL3vLL86Pvqdx6M+k34YzYmilgiMOFVUQ7Tr4bCLXD1zNqLc1TdHcKlXpIoKfTmqxLFXtd30HHjrIWQMSFmicCE17u/gzUfuE7gjlSmL+JGEIsP2CDvxkSY/bQy4bPwZXfn70nXwpDL/Y7GGFMLSwQmPDZ/BTNugO4jYfSf/I7GGHMElghM6O3bAS9eDq3awsXPQGy83xEZY47A6ghMaFWUwcuTXIuga96GFOs23JimzhKBCa1Zt8G6j+GCR6HzEL+jMcbUgRUNmdD56l/wxcNwyg/g+Al+R2OMqSNLBCY0Ns2FN26GnqfBt3/vdzTGmHqwRGAab28eTLscUjrAuKch1kocjWlO7D/WNE5Vb6BFu2HyLEhu63dExph6skRgGuedX7nxgi96EjoN9jsaY0wDWNGQabi5z8CcJ2DETTBonN/RGGMayBKBaZiNc9zYAL3OhG/d7nc0xphGsERg6q9wq7tzOK2zKxKKifU7ImNMI1gdgamf8hJ48QooKYArXoVWbfyOyBjTSJYITN1VDRWZ+6XrQ6jDQL8jMsaEQJ2KhkQkWURivPljReR8EbGexKJNzlSY96wbKnLgWL+jMcaESF3rCD4CAiLSBZgFXAE8Ha6gTBOzZyO8/SuY+Qvo/W0Y9Wu/IzLGhFBdi4ZEVfeLyGTgIVW9S0S+CmdgpgnYughm3w9fTz84TOQ5f7HKYWNamDonAhEZDlwGTPaWHfVsICKjgfu8bZ9Q1T/XsM144HZAgQWqemkdYzLhoOp6D/30Plj1LsQnw8nXwSnXQ0ZXv6MzxoRBXRPBzcCvgNdUdbGIHAO8f6QXiEgs8CDwbSAXmCMiM1R1SdA2fbz9jlTV3SKS2ZA3YUKgsgKWznAJYPN8SG4PZ94GJ02GpNZ+R2eMCaM6JQJV/RD4EMCrNN6hqjcd5WXDgFWqusZ73TRgDLAkaJtrgQdVdbd3nLz6hW8arXQ/fPVPN7bw7nXQphecdy8cP9ENJG+MafHqlAhE5F/AdUAFMAdIE5H7VPXuI7ysC7Ax6HkucHK1bY719v8prvjodlV9u4bjTwGmAHTr1q0uIZuj2b8LvnwcvnwU9u+ELie67qP7fcfqAIyJMnUtGhqgqgUichkwE7gFmAscKRHU9fh9gDOALOAjERmkqnuCN1LVx4DHALKzs7WRx4xuu9fDZw/C/OegbD/0ORtG/gi6j3AVwsaYqFPXRBDv3TcwFnhAVctE5Ggn5E1AcO1ilrcsWC7whaqWAWtFZAUuMcypY1ymror2wFs/h0WvuBP+oPEw4kboMMDvyIwxPqtrIngUWAcswP1q7w4UHOU1c4A+ItITlwAmANVbBL0OTASeEpF2uKKiNXWMydSVKrz+A1j5jmv9c8r1kJ7ld1TGmCairpXF9wP3By1aLyKjjvKachG5AXgHV/4/1WtxdCeQo6ozvHVnicgSXP3Dz1V1Z0PeiDmCzx+C5f+Bs/8Iw3/odzTGmCZGVI9e5C4i6cDvgNO8RR8Cd6pqfhhjq1F2drbm5ORE+rDN18Y58NRoOHY0XPK81QMYE6VEZK6qZte0rq5dTEwFCoHx3lQAPBWa8EzY7N8FL09y3UWPecCSgDGmRnWtI+ilqhcFPb/Dupho4ior4bXrYO82mPyO3RRmjKlVXa8IikTk1KonIjISKApPSCYkZt/vKofP/qO7R8AYY2pR1yuC64BnvboCgN3AVeEJyTTa+s/gvTthwFgYdq3f0Rhjmri6thpaABwvImne8wIRuRlYGM7gTAPs2wHTr4GMbnD+/VYvYIw5qnqNWayqBapadf/AT8IQj2mMykp4dYrrMmL8MxBIP/prjDFRrzFDVdpPzabmk7/C6vfgvL9Dp+P9jsYY00zU64qgGuvzpylZ+zG8/0c3eMyJV/sdjTGmGTliIhCRQhEpqGEqBDpHKMaQ2LSniF+/9jUl5RV+hxJ6e/PglcmuC+nv3mv1AsaYejli0ZCqpkYqkHBbsrmAf36xgUB8LLed14I6WquscEmguACueA0SW8yfzBgTIY0pGmpWvj2gA1cN786Tn6zl/WUtaPybD++CtR/BuXdDh4F+R2OMaYaiJhEA/Orc/vTvlMZPX17AtoJiv8NpvNXvw4d/caOJDbnc72iMMc1UVCWCQHws/5g4hKLSCn784ldUVDbj+u6CLfDqtdC+L3znr1YvYIxpsKhKBAC9M1O4/fwBzF69k0c+XO13OA1TUe7qBUr3wcXPQEKy3xEZY5qxqEsEAOOzu3Le4E787b8rmLt+t9/h1N8Hf4T1n7r7BTL7+R2NMaaZi8pEICL88cJBdM4IcNML88kvKvM7pLpb+S58/FcYeiUcP8HvaIwxLUBUJgKAtEA8908YwraCYm599WvqMkCP7/JzXb1Ah+PgnLv8jsYY00JEbSIAGNKtNT89qy//+XoL0+Zs9DucIyvYDNMnQ0WpqxeIT/I7ImNMC9GYvoZahO+fdgyfrtrBHW8sJrt7a/p0aCI3ZJWXwIbPYNW7sOo9yFvill/0JLTr7W9sxpgWpU5jFjcl4RizOK+gmHPu+5h2KYn8+4aRBOJjQ7r/Otu52p30V7/nbhIr2w+xCdBtOPT+Fhx7tmsuaowx9XSkMYuj54qgpNCN4ZueBTGHnugz0wL8dfzxTHpqDn/4zxL+MHZQhGLaC+s+8X71vwu717rlbY6BEy5zJ/8ep0JiSmTiMcZEpehJBKv/By9dCbGJ0LYXtO3tpnZ9oG1vzujam2u/0ZPHP17Lqb3bMfq4TqGPQdUV8VSd+Dd87sr841tBz9Ng+A+h15kuPmOMiZDoSQSdh8B374Odq2DHKndCXv4WVJYf2OTWVm25MCWTZdM7kL9lJOlZ/aFtH2jTE+ISD99nZQUU57srjaLd1aZqy/bvgoJNbjB5gMyBcPL33a/+bsNr3r8xxkRA9CSCjG5w4qRDl1WUwe71sHMl7FyF7FhJz60raLfpK9I//eDgdhLjXp/e1d3NW3WSL84/wgHFjRCW1Prg1O5YV9TT60xI7xKGN2mMMfUXPYmgJrHxrgVOUCucAPD2/E3c9uJsbhkWz2W9S72riJWuCWertq5IKfgE36rNoc+TWrskEONTpbMxxtRDWBOBiIwG7gNigSdU9c+1bHcRMB04SVVD2ySoAcYO6cJHK/tw25xN9DrhG5wyeLzfIRljTNiE7YYyEYkFHgTOAQYAE0XksBFhRCQV+BHwRbhiaYjfjzmO7m2TuXnaV+zeV+p3OMYYEzbhvLN4GLBKVdeoaikwDRhTw3a/B/4CNKkBApIT4/jHxCHs3FfCz6cvaB5dUBhjTAOEMxF0AYL7bcj1lh0gIkOBrqr6nyPtSESmiEiOiORs37499JHW4rgu6dxyTn/eXZrHM7PXRey4xhgTSb71NSQiMcDfgJ8ebVtVfUxVs1U1u3379uEPLsg1I3twZr9M/vjWMhZvPlIrIWOMaZ7CmQg2AV2Dnmd5y6qkAscBH4jIOuAUYIaI1HgLtF9EhLvHDSajVTw3vjCfPfutvsAY07KEMxHMAfqISE8RSQAmADOqVqpqvqq2U9UeqtoD+Bw4vym0GqqubUoi9044gdxdRVz08Gw27trvd0jGGBMyYUsEqloO3AC8AywFXlLVxSJyp4icH67jhsuIXu14dvIwtheWcOHDs1m0yYqJjDEtg/U+Wk8rthUyaeqX7Ckq46HLhnJG30zfYjHGmLo6Uu+jUT0wTUMc2yGV1344ku5tk5n8TA4vNfUBbYwx5igsETRAh7QAL33/FEb0assvXlnI3/+7wu4zMMY0W5YIGig1EM/USScx7sQs7ntvJb+YvpCyikq/wzLGmHqL7k7nGik+Noa7xw2mc0YS97+3km2FJTx02VBSEu1jNcY0H3ZF0Egiwk++fSx/uWgQn67awfhHPiOvoEn1lmGMMUdkiSBELjmpG09clc26nfu44KHZrNxW6HdIxhhTJ5YIQmhU30xenDKckvJKLnp4Nl+s2el3SMYYc1SWCEJsUFY6r/1gBO1TE7niyS95Y8Fmv0MyxpgjskQQBl3btOKV60dwfNd0bnxhPo9/tMaalxpjmixLBGGS0SqB5yafzHcGdeL/3lrKHW8soaLSkoExpumxdo5hFIiP5R8Th9ApPcATn6xlS34R900YQiDexjI2xjQddkUQZjExwm/OG8BvzxvArCXbuPyJL6wra2NMk2KJIEKuObUnD146lIW5+Yx75DM27SnyOyRjjAEsEUTUuYM68ezkYWwrKObChz5l6ZYCv0MyxhhLBJF2yjFtefm64QjC+Ec+47PVdq+BMcZflgh80K9jGq/+YAQd0wNcNfVL3lxo9xoYY/xjicAnnTOSePm64QfuNZj6yVq/QzLGRClLBD6qutfgrAEduPPNJfzpraVU2r0GxpgIs0Tgs0B8LA9ddiJXnNKdRz9aw09e+orSchvXwBgTOXZDWRMQGyPcOWYgHdMD3P3OcnbsLeWRK060cQ2MMRFhVwRNhIjww1G9uXvcYD5bs5NLHv2MvEIb18AYE36WCJqYi7O78sRV2azZvo+LHp7Nmu17/Q7JGNPCWSJogkb1zWTalFPYX1LBuEc+Y/6G3X6HZIxpwSwRNFHHd83gletHkJIYx6WPf8H/lm3zOyRjTAtliaAJ69EumVeuH0GvzGSufXYuL87Z4HdIxpgWyBJBE9c+NZFpU4YzoldbfvnK1/xp5lIb18AYE1JhTQQiMlpElovIKhG5pYb1PxGRJSKyUETeE5Hu4YynuUpJjOPJq07ispO78eiHa7j66Tnk7y/zOyxjTAsRtkQgIrHAg8A5wABgoogMqLbZfCBbVQcD04G7whVPc5cQF8P/XTCIP14wiM9W7+D8Bz9hxbZCv8MyxrQA4bwiGAasUtU1qloKTAPGBG+gqu+r6n7v6edAVhjjaREuPbmba1FUWsHYBz/l7UVb/A7JGNPMhTMRdAE2Bj3P9ZbVZjIws6YVIjJFRHJEJGf79u0hDLF5OrF7G9644Z4DtIYAABP1SURBVFT6dEjluufn8bdZy62PImNMgzWJymIRuRzIBu6uab2qPqaq2aqa3b59+8gG10R1TA/w4pRTuPjELO7/3yqmPJdDYbHVGxhj6i+ciWAT0DXoeZa37BAi8i3g18D5qloSxnhanEB8LHeNG8ydYwbywfLtjH3wU1bbncjGmHoKZyKYA/QRkZ4ikgBMAGYEbyAiQ4BHcUkgL4yxtFgiwpXDe/D8905m9/4yxj7wqd18ZoypF1ENX9myiJwL3AvEAlNV9f9E5E4gR1VniMi7wCCgqsZzg6qef6R9Zmdna05OziHLysrKyM3Npbi45XfSFggEyMrKIj4+/rB1m/YUMeXZHJZsKeCn3z6WH47qjYj4EKUxpqkRkbmqml3junAmgnCoKRGsXbuW1NRU2rZt26JPfKrKzp07KSwspGfPnjVuU1RawS2vLuTfX23mnOM6cs/Fx5Ns3VkbE/WOlAiaRGVxYxUXF7f4JACuGKht27ZHvPJJSojl3ktO4Nfn9uedxVu58KHZrN+5L4JRGmOamxaRCIAWnwSq1OV9igjXnnYMz1wzjK0FxZz/wKd8vNKa3RpjatZiEoE53Df6tOeNG06lU3qAq6Z+yaMfrrb7DYwxh7FEEAI7d+7khBNO4IQTTqBjx4506dLlwPPS0tIjvjYnJ4ebbropbLF1a9uKV64fwejjOvKnmcs4/8FP+GTljrAdzxjT/LSIyuKlS5fSv39/nyI61O23305KSgo/+9nPDiwrLy8nLi50FbYNeb+qymvzN/HXWSvYtKeIb/Rpxy9H9+O4Lukhi8sY03QdqbK4xTUnueONxSzZXBDSfQ7onMbvvjuwXq+ZNGkSgUCA+fPnM3LkSCZMmMCPfvQjiouLSUpK4qmnnqJv37588MEH3HPPPbz55pvcfvvtbNiwgTVr1rBhwwZuvvnmkF0tiAgXDs3i3EGdeP7z9Tzw/irO+8cnjD2hMz89qy9d27QKyXGMMc1Pi0sETUlubi6zZ88mNjaWgoICPv74Y+Li4nj33Xe59dZbeeWVVw57zbJly3j//fcpLCykb9++XH/99TXeM9BQgfhYvveNY7g4uyuPfriaqZ+u5T9fb+HyU7pz45l9aJOcELJjGWOahxaXCOr7yz2cLr74YmJjYwHIz8/nqquuYuXKlYgIZWU19wv0ne98h8TERBITE8nMzGTbtm1kZYW+U9b0pHh+MbofVw7vwb3vruCZ2et4OSeX604/hmtO7UmrhBb31TDG1MIqi8MoOTn5wPxtt93GqFGjWLRoEW+88Uat9wIkJiYemI+NjaW8vDysMXZMD/DniwYz68enMaJXW+6ZtYLT7/6Af36xnvKKyrAe2xjTNFgiiJD8/Hy6dHG9cD/99NP+BlOD3pmpPHZlNq9cP5zubVrx69cWcdbfP+LtRVtobg0KjDH1Y4kgQn7xi1/wq1/9iiFDhoT9V35jnNi9DS9fN5zHr8wmJka47vl5XPjwbL5cu8vv0IwxYWLNR5uhSL3f8opKXp23ib/9dwVbC4o5o297Tu3djl6ZKfRun0KXjCRiYqLjjm5jmruoaj5qQicuNobxJ3Xlu8d35unZ65j66Vo+WH6wq4qk+FiOaZ9Mr/Yp9M48OPVom0xCnF1sGtNcWCIwR5WUEMv1Z/Ti+jN6sWtfKavy9rIqby+rt7vHuet3M2PB5gPbx8YI3dq0OiRB9GqfTO/MFFIDoWsKa4wJDUsEpl7aJCcwrGcbhvVsc8jy/aXlrNm+70ByqJo+XJFHWcXB4scebVsxOCuDwVnpDM7K4LguadZU1Rif2X+gCYlWCXEc1yX9sC4ryioq2bhrP6vy9rIyby8Lc/eQs27XgSuIGIHemSmHJIf+nVJJjIv1420YE5UsEZiwio+N4Zj2KRzTPoWzgu71yyss5uvcfBbm5rMwdw/vL8tj+txc7zVC346pLjl0ccnh2A4pxMVavYMx4WCJwPgiMzXAN/sH+Gb/DoDrFG9zfjFf5+5hgZcc3liwmX99sQGAQHwMAzqlMaRba4Z2a82Qbhl0zkjy8y0Y02JYIgiBUaNGccstt3D22WcfWHbvvfeyfPlyHn744cO2P+OMM7jnnnvIzq6xJVdUEhG6ZCTRJSOJ0cd1AqCyUlm/az8Lc/ewMDefBRv38Pzn63nyk7UAdEwLMKRbBkO7tWZo9wwGdk4nEG9FSsbUlyWCEJg4cSLTpk07JBFMmzaNu+66y8eomr+YGKFnu2R6tktmzAnuruzS8kqWbS1g3vrdzN+4h3kbdjNz0VbAFSkN6JzOkK4ZDO3emiFdM8hqnRQ1o9cZ01AtLxHMvAW2fh3afXYcBOf8udbV48aN4ze/+Q2lpaUkJCSwbt06Nm/ezAsvvMBPfvITioqKGDduHHfccUdo44pCCXExXsVyBpO8ZXmFxXy1YQ/zNrjEMG3OBp6evQ6A9qmJBxLD4Kx0UhPjiYlxTVxjpGoKeh4jxHrLYrxlsSJIDMTHxBCIj7HEYlqclpcIfNCmTRuGDRvGzJkzGTNmDNOmTWP8+PHceuuttGnThoqKCr75zW+ycOFCBg8e7He4LU5maoCzBnbkrIEdAddSafnWQuZv2H0gOcxasi0kx0qKj6VDWiKZaQE6pAXokJpIh7QAmWnusUNagMzURJITG/+vpaoUl1Wyt6Sc/aXl7C+tID42hqSEWJLi3ZQYF9Ni7+5WVXbvL2PzniK25BezraCY1EAcXTKS6JyRRGZqojUgCJGWlwiO8Ms9nKqKh6oSwZNPPslLL73EY489Rnl5OVu2bGHJkiWWCCIgPjbmQFPWK4a7ZTv3lrB4cwEl5ZVUVCqqSoUqlerqIioqlUqtmji4TaVSoe6kVFah7NxbwrbCErYVuIrt/xYUU1x2eC+tqYlxtE9LpENqgA5ekmiXkkh5pbKvpJx9peXsL6lgX2m597zCnexLKrwTv1tXlx5gAvExBxJDIChJJCXEEog/9HlmWuKBupjOGUl0SAsQ61Mi2VtSzpY9RWzOLz7w6E76RWzZU8zm/KIaP9sqsTFCx7QAnTMCdPbeT+eMJLoEPU+zGxjrpOUlAp+MGTOGH//4x8ybN4/9+/fTpk0b7rnnHubMmUPr1q2ZNGlSrV1Pm/Brm5LIace2D/l+VZXCknLyCorZVlBCXqF73FZQTJ73OHfDbrYVlFBa7k5qMQLJCXG0SowlOTHOzSfEkpkaoFXbWFIS42iVEEdyYuyBx+SEOJISYimrqKS4rIKi0gqKyiopKqsIeu6mYm9+b0k52wtL3PqyCvaXVFBYcmiHh1UnU5cYAnRpffCEmuU9Hu3qprzCXbUUFpdTUFxGYXG5N5Ud8lhQXMbW/GK2eCf8guJDYxGBzNREOmck0b9TGmf2y6STd2LvlO6S1t6SMnJ3F7F5j9vH5j1FbNpTxLwNu/nPwi2UVx6aOVMT47z345JDp/QAHdOT6JgWoGO6m1JCcPUWCqpKQVE5WwoOJsKt+cVs3lPMVm/Z+JO6ct3pvUJ+7KbxCbQAKSkpjBo1imuuuYaJEydSUFBAcnIy6enpbNu2jZkzZ3LGGWf4HaYJMREhLRBPWiCe3pmptW5XlTASYmNIjPOvnmFfSTlb8osOOZlu8qac9bt5s4aTaXpSvDuZpgcoq9TDTvD7SyuOetyEuBjSAnFkpgbIap3EsJ5t6JR+6Am6Q1qA+KMW9QRq/ZwrKpUde0vY5CUINxUfeD5/4x727D98QKjUxDg6pAcOxFDTY5vkhHr9zVSVcu9Ks7xSqahQyior2bm39MAJvuoqaGv+wZN+9c8yRqCDl7T6d0ojq3V4mkyHNRGIyGjgPiAWeEJV/1xtfSLwLHAisBO4RFXXhTOmcJo4cSIXXHAB06ZNo1+/fgwZMoR+/frRtWtXRo4c6Xd4xkdVCcNvyYlx9M5MPeLJNK+wKkF4j7uLDpTTx3sn9E7pAVIT40kNxJEaqHqsfT4Sd4rHxsiBepqh3VrXuE1xWQVb84vZWlB86GN+MVsKilm5bQd5hcVUy4UkxMaQmZZIQmxM0Am+8pATffDy6q+vSYy4+q2O6QH6dUxlVN9MOqW7q5+O6a7Iq31KZOpBwtYNtYjEAiuAbwO5wBxgoqouCdrmB8BgVb1ORCYAF6jqJUfar3VDHX3v15hIKq+oZMfeUi9JFB1IEtvyiymvVOJjY4iNEeJiJOgxhrhYqXl51fNYoU1ywoGTfaQru/3qhnoYsEpV13hBTAPGAEuCthkD3O7NTwceEBHR5jZIgjGmxYiLjTlQf0DXDL/DiYhwpqMuwMag57neshq3UdVyIB9oW31HIjJFRHJEJGf79u3VVxtjjGmEZtEIV1UfU9VsVc1u377mlh/RchERLe/TGBM54UwEm4CuQc+zvGU1biMicUA6rtK4XgKBADt37mzxJ0lVZefOnQQCAb9DMca0IOGsI5gD9BGRnrgT/gTg0mrbzACuAj4DxgH/a0j9QFZWFrm5uURDsVEgECArK8vvMIwxLUjYEoGqlovIDcA7uOajU1V1sYjcCeSo6gzgSeA5EVkF7MIli3qLj4+nZ8+eoQrdGGOiSljvI1DVt4C3qi37bdB8MXBxOGMwxhhzZM2istgYY0z4WCIwxpgoF7Y7i8NFRLYD6xv48nbAjhCGE2pNPT5o+jFafI1j8TVOU46vu6rW2P6+2SWCxhCRnNpusW4Kmnp80PRjtPgax+JrnKYeX22saMgYY6KcJQJjjIly0ZYIHvM7gKNo6vFB04/R4msci69xmnp8NYqqOgJjjDGHi7YrAmOMMdVYIjDGmCjXIhOBiIwWkeUiskpEbqlhfaKIvOit/0JEekQwtq4i8r6ILBGRxSLyoxq2OUNE8kXkK2/6bU37CmOM60Tka+/YOTWsFxG53/v8ForI0AjG1jfoc/lKRApE5OZq20T88xORqSKSJyKLgpa1EZH/ishK77HG8RNF5Cpvm5UiclUE47tbRJZ5f8PXRKTGUViO9n0IY3y3i8imoL/jubW89oj/72GM78Wg2NaJyFe1vDbsn1+jqWqLmnAd3K0GjgESgAXAgGrb/AB4xJufALwYwfg6AUO9+VTccJ7V4zsDeNPHz3Ad0O4I688FZgICnAJ84ePfeivuRhlfPz/gNGAosCho2V3ALd78LcBfanhdG2CN99jam28dofjOAuK8+b/UFF9dvg9hjO924Gd1+A4c8f89XPFVW/9X4Ld+fX6NnVriFcGBITJVtRSoGiIz2BjgGW9+OvBNEZFIBKeqW1R1njdfCCzl8JHbmroxwLPqfA5kiEgnH+L4JrBaVRt6p3nIqOpHuB50gwV/z54Bxtbw0rOB/6rqLlXdDfwXGB2J+FR1lrqRAQE+x40Z4otaPr+6qMv/e6MdKT7v3DEeeCHUx42UlpgIQjZEZrh5RVJDgC9qWD1cRBaIyEwRGRjRwECBWSIyV0Sm1LC+Lp9xJEyg9n8+Pz+/Kh1UdYs3vxXoUMM2TeWzvAZ3lVeTo30fwukGr+hqai1Fa03h8/sGsE1VV9ay3s/Pr05aYiJoFkQkBXgFuFlVC6qtnocr7jge+AfweoTDO1VVhwLnAD8UkdMifPyjEpEE4Hzg5RpW+/35HUZdGUGTbKstIr8GyoF/1rKJX9+Hh4FewAnAFlzxS1M0kSNfDTT5/6eWmAgiNkRmQ4lIPC4J/FNVX62+XlULVHWvN/8WEC8i7SIVn6pu8h7zgNdwl9/B6vIZh9s5wDxV3VZ9hd+fX5BtVUVm3mNeDdv4+lmKyCTgPOAyL1kdpg7fh7BQ1W2qWqGqlcDjtRzX788vDrgQeLG2bfz6/OqjJSaCA0Nker8aJ+CGxAxWNUQmNGKIzIbwyhOfBJaq6t9q2aZjVZ2FiAzD/Z0ikqhEJFlEUqvmcRWKi6ptNgO40ms9dAqQH1QEEim1/grz8/OrJvh7dhXw7xq2eQc4S0Rae0UfZ3nLwk5ERgO/AM5X1f21bFOX70O44guud7qgluPW5f89nL4FLFPV3JpW+vn51YvftdXhmHCtWlbgWhP82lt2J+4LDxDAFSmsAr4EjolgbKfiiggWAl9507nAdcB13jY3AItxLSA+B0ZEML5jvOMu8GKo+vyC4xPgQe/z/RrIjvDfNxl3Yk8PWubr54dLSluAMlw59WRcvdN7wErgXaCNt2028ETQa6/xvourgKsjGN8qXPl61fewqiVdZ+CtI30fIhTfc973ayHu5N6penze88P+3yMRn7f86arvXdC2Ef/8GjtZFxPGGBPlWmLRkDHGmHqwRGCMMVHOEoExxkQ5SwTGGBPlLBEYY0yUs0RgTDUiUlGth9OQ9WgpIj2Ce7A0pimI8zsAY5qgIlU9we8gjIkUuyIwpo68fuXv8vqW/1JEenvLe4jI/7zO0d4TkW7e8g5eP/8LvGmEt6tYEXlc3HgUs0Qkybc3ZQyWCIypSVK1oqFLgtblq+og4AHgXm/ZP4BnVHUwruO2+73l9wMfquv8bijuzlKAPsCDqjoQ2ANcFOb3Y8wR2Z3FxlQjIntVNaWG5euAM1V1jddx4FZVbSsiO3DdH5R5y7eoajsR2Q5kqWpJ0D564MYf6OM9/yUQr6p/CP87M6ZmdkVgTP1oLfP1URI0X4HV1RmfWSIwpn4uCXr8zJufjev1EuAy4GNv/j3gegARiRWR9EgFaUx92C8RYw6XVG0g8rdVtaoJaWsRWYj7VT/RW3Yj8JSI/BzYDlztLf8R8JiITMb98r8e14OlMU2K1REYU0deHUG2qu7wOxZjQsmKhowxJsrZFYExxkQ5uyIwxpgoZ4nAGGOinCUCY4yJcpYIjDEmylkiMMaYKPf/5//zxE9UBLgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model_Bi_LSTM_hist.history['loss'])\n",
    "plt.plot(model_Bi_LSTM_hist.history['val_loss'])\n",
    "\n",
    "plt.title('Model Loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['Train', 'Val'], loc='lower left')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "GZFmgZ8zZPV_",
    "outputId": "525a6a62-ed47-4518-d7f6-47d5ba2d8076"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5bnA8d+TfQ8JCQESIAFZREHACLiLVqVaRau1oq1abW1trd2s1a7WW7td23rbaxcX1NpWpNoFvVq1KmpVlLDve4AESALZ98zMc/84JzCECYYkk5nMPN/PZz5ztpnz5GTmPPO+7znvK6qKMcYY01VMqAMwxhgTnixBGGOMCcgShDHGmIAsQRhjjAnIEoQxxpiALEEYY4wJyBKEMb0kIoUioiIS14NtbxKR/wxEXMb0F0sQJiqISKmItItITpflK92TfGFoIju+RGPMQLIEYaLJTmB+54yITAFSQheOMeHNEoSJJk8BN/jN3wj80X8DEckUkT+KSJWI7BKR74pIjLsuVkQeEJEDIrIDuDTAax8TkX0iUi4iPxKR2L4ELCIjRWSxiFSLyDYR+ZzfupkiUiIi9SJSISK/dJcnicifROSgiNSKyDIRyetLHCY6WYIw0WQpkCEiJ7on7muBP3XZ5jdAJjAWOBcnoXzGXfc54GPAdKAYuLrLa58APMAJ7jYXAZ/tY8wLgTJgpLu/H4vI+e66/wH+R1UzgHHAInf5je7fMAoYCnwBaOljHCYKWYIw0aazFHEhsBEo71zhlzTuUdUGVS0FfgF82t3kGuBBVd2jqtXAT/xemwdcAnxVVZtUtRL4lft+vSIio4AzgW+paquqrgIe5XApqAM4QURyVLVRVZf6LR8KnKCqXlVdrqr1vY3DRC9LECbaPAVcB9xEl+olIAeIB3b5LdsF5LvTI4E9XdZ1GuO+dp9brVML/AEY1odYRwLVqtrQTTy3ABOATW410sfc5U8BLwMLRWSviPxcROL7EIeJUpYgTFRR1V04jdWXAH/rsvoAzq/vMX7LRnO4lLEPp9rGf12nPUAbkKOqQ9xHhqqe1Idw9wLZIpIeKB5V3aqq83GS0M+AZ0UkVVU7VPWHqjoZOAOnWuwGjDlOliBMNLoFOF9Vm/wXqqoXpx7/fhFJF5ExwNc53E6xCLhDRApEJAu42++1+4BXgF+ISIaIxIjIOBE59zjiSnQbmJNEJAknEbwL/MRdNtWN/U8AIvIpEclVVR9Q676HT0TmiMgUt8qsHifp+Y4jDmMASxAmCqnqdlUt6Wb1l4EmYAfwH+AvwAJ33SM4VTergRUcXQK5AUgANgA1wLPAiOMIrRGnMbnzcT7OZbmFOKWJvwM/UNV/u9vPBdaLSCNOg/W1qtoCDHf3XY/TzvImTrWTMcdFbMAgY4wxgVgJwhhjTECWIIwxxgRkCcIYY0xAliCMMcYEFDG9R+bk5GhhYWGowzDGmEFl+fLlB1Q1N9C6iEkQhYWFlJR0d+WiMcaYQERkV3frrIrJGGNMQJYgjDHGBBS0BCEiC0SkUkTWdbNeROTXbh/3a0Rkht+6G0Vkq/u4MVgxGmOM6V4wSxBP4HQF0J2PAuPdx63A7wBEJBv4ATALmAn8wO33xhhjzAAKWoJQ1beA6mNsMg/4ozqWAkNEZARwMfCqqlarag3wKsdONMYYY4IglG0Q+RzZt36Zu6y75UcRkVvdIRdLqqqqghaoMcZEo0HdSK2qD6tqsaoW5+YGvIzXGGNML4XyPohyjhx8pcBdVg6c12X5kgGLyhhjPoSqUtnQxpaKBrZUNNLa4SUjKY6M5HjnkRRPZnIcGUnOfFJ8bKhD7pVQJojFwO0ishCnQbpOVfeJyMs4A7N3NkxfBNwTqiCNMd2ra+lg9Z5aPD4fw9KTGJaRyNDURGJjJNSh9QtVpaqhjS0VjWypaGBrpZMQtlY0UN/q6fH7JMTFuMkizk0enYnESSp56YmMz0vnhGFpDEtPRCQ8jl/QEoSIPI1TEsgRkTKcK5PiAVT198CLOMM+bgOagc+466pF5L+AZe5b3ecOEG+MCSGfT9lW1ciKXTWs2F3Dit21bKtsPGq72BghNy2RvIxEhmUkOc/pSYfn3emslARiwiSRqCpVjW1sPZQInCSwpaKRupaOQ9tlpcQzPi+dy04ZyYS8dMbnpTEhL530pDjqWzzUt3ZQ39JBfauHupbO6Q53+vD62uZ2dlc3U9/irPP4Do/Lk54UxwnD0hg/LM19dhJH/pDkAT9eETNgUHFxsVpXG8b0n7qWDlbtqT2UEFbtqaXB/dU8JCWe6aOGMGN0FjPGZJGSEEtlQxuV9a1U1LdRUd9KxaH5VmqaO456//hYJ5EMy0hiXG4aM8Y47zchLz3oJZC6lg5W7q5hxa4alu+uYf3eemr9YsxMjmdCXhrj89KZMCzNTQbp5KQl9Puv+85SytbKRrZVNrK1soFt7vSBxvZD2yXFxzAu93DiOMFNHGOGphAf2/vmZBFZrqrFAddZgjBm4HV4fZTVtFB6oImdB5yhsYemJZCdmkBWSsKh6cS4gam79vmU7VWNTslgVy0rdtewraoRVRCBiXnpTB+dxYzRQ5gxJouxOanHdaJs83ipamijov5w0qhocBJJZX0bG/fVc7DJORmmJcZxyqhMJ/mMzmL66CEMSUno9d+mquyubqak1EkGy0tr2FLZgCrECJw4IoOpBZmMH5bOhLx0JuSlkRsm1Tw1Te1sq2o8lDC2VjayvbKR8tqWQ9vExwrnjM/lsZtO69U+jpUgIqazPmPCjc+n7K9vpfRAEzvcRLDzQBOlB5rYXd18RLVCd1ITYslOSyA7NZGhXZJHdmoC2SkJZKclECNCa4eXlg4vbe5zS7vv6GUdXlo7fM5zu5dWj5fmdi/bKxsP1al3lg4uP2UkM8ZkccqoIaQl9u1UkRgXS0FWCgVZKQHXd57E/RPUb5dsx+seo7G5qYcSxowxQxg/rPtSRpvHy7ryOpbvqqGk1Cn9dP4ST0+MY/qYLC6dOoJTx2QxbdQQUvv4twVTVmoCp6Vmc1ph9hHLm9o8bK9qZGtFI9uqGvv8/+mOlSBMVKtsaGXRsj0caGwnMS6GxLgYEuJiSIyLJeHQdJdlsTEkxjvPSfExxMfGcKCxjR1VbgI42MSOKue5tcN3aF9J8TEUDk1lbG4qRTmph6YLh6YSI0J1czvVTe0cbGynxm+6uqmN6uYO57mxnYNN7bR5fMf4q44mAsnxsSTFx5IcH0tifAzJ7nRSfCyjslN6XToIluZ2D6v31LFid41THbS7lupuShntXh8rdtVQsquGtWV1tHud4zNmaAqnjsni1DFZFI/JZvywtLBp9wgXVsVkTBdbKhp49O0d/GPlXjp8PtIS42j3+Gj3+ujLVyIuRhidnUJRjpMEinJTKRrqPOelJ/Xbyam53eMmD+ehKEl+CaDzuTMZJMbFhMVJvy9UlV0H3VKGW9LYtL+ezoJYQmwMJ+dnUFyYzYzRTlLITU8MbdCDgFUxGYNzgnl3+0EeeXsHSzZXkRQfwydPG8XNZxVRlJN6aJsOr9Lu9dHW4aXd66Pd46PN0/ns9Zt2nts9PrJTEyjKSaUgK5m4PjQY9lRKQhwp2XGMyg5cZROJRITCnFQKc1L5+IwCwKlqWVteR1yMcHJ+5qC93yBcWYIwEa/D6+OFNXt55K2dbNhXT05aAt+4cALXzx5DduqRjZ8iQkKckBAXE7R6XdN/UhPjmD12aKjDiFj2DTARq761g6ff380T75ayr66VE4al8bOrpjBvWr790jSmByxBmIhTVtPM4++UsvCD3TS1ezlj3FB+fOUUzp2Qaw2UxhwHSxAmYqwpq+Xht3bw0rr9CPCxqSP47NljOTk/M9ShGTMoWYIwg1qH18frmyp57D87+WBnNemJcdxyVhE3nVHIyCHJoQ7PmEHNEoQZlDbsrefZ5WX8c1U5B5vayR+SzHcvPZFPnjaK9KT4UIdnTESwBGEGjQONbfxjZTnPrShn4756EmJjuODEYVw1o4BzJ+b2qT8aY8zRLEGYsNbu8fH6pgqeXV7Gks1VeHzKKQWZ3DfvJC6bOpKs1N730WOMOTZLECbsqCpry+t4bnkZ/1y9l9rmDoalJ3LL2UVcPaOA8XnpoQ7RmKhgCcKEjcr6Vv6+spznVpSxpaKRhLgYLpqcx9WnFnDWCTkDcoeyMeYwSxAmZPz71lm8ei9vbanCpzBj9BDuv/JkPjZ1JJnJ1uBsTKhYgjAD5kBjG6v31LJ6Ty2ryupYU1Z7aJCWEZlJ3HbeOD4+o4BxuWkhjtQYA5YgTJA0tXlYV17H6rJaVu+pY9We2kODnMQITMhLZ+5Jwzll1BCmFmQyaXhGxIxjbEyksARh+szj9bG5ooHVe+qcEkJZLVsqGg51w1yQlcy00UO46YxCThk1hJPzM0hJsI+eMeHOvqWmT7ZWNHDDgg/YV9cKOKORnVIwhItOGs60UZlMLRhCTpr1yW/MYGQJwvTa5v0NXP/oUkSEBz85jemjhzA6O2XQD0xjjHFYgjC9sml/Pdc98j5xMcLTt862hmVjIpAlCHPcNuyt5/pHl5IYF8vTt84+NBqbMSay2J1H5risK6/jukeXkhQfy0JLDsZENCtBmB5bW1bHpx57n7TEOJ7+3GxGD42e8ZCNiUZWgjA9snpPLdc/upS0xDgW3mrJwZhoYAnCfKiVu2v41KPvk5kSzzOfn82obEsOxkQDq2Iyx7R8Vw03LviA7NQEnr51Nvk2SpsxUcNKEKZbJaXV3PDY++SkJfDM5y05GBNtLEGYgD7YWc0NCz4gLyOJhbeezohMSw7GRJugJggRmSsim0Vkm4jcHWD9GBF5TUTWiMgSESnwW+cVkVXuY3Ew4zRHWrrjIDc9/gHDM5NYeOtshmcmhTokY0wIBK0NQkRigYeAC4EyYJmILFbVDX6bPQD8UVWfFJHzgZ8An3bXtajqtGDFZwJ7d/sBbnmihPysZP7yuVkMS7fkYEy0CmYJYiawTVV3qGo7sBCY12WbycDr7vQbAdabAfTOtgPc/MQyRmUn8/TnZltyMCbKBTNB5AN7/ObL3GX+VgMfd6evBNJFZKg7nyQiJSKyVESuCLQDEbnV3aakqqqqP2OPOm9tqeLmJ5ZRODSVv3xuNrnp1gOrMdEu1I3UdwLnishK4FygHPC668aoajFwHfCgiIzr+mJVfVhVi1W1ODc3d8CCjjRLNlfy2T+WUJTjJAfrntsYA8G9D6IcGOU3X+AuO0RV9+KWIEQkDbhKVWvddeXu8w4RWQJMB7YHMd6otPNAE59/ajkn5Kbx58/OIis1IdQhGWPCRDBLEMuA8SJSJCIJwLXAEVcjiUiOiHTGcA+wwF2eJSKJndsAZwL+jdumH/h8yt3PrSEhLobHP3OaJQdjzBGCliBU1QPcDrwMbAQWqep6EblPRC53NzsP2CwiW4A84H53+YlAiYisxmm8/mmXq59MP3h62W7e31nNdy45kbwMa5A2xhxJVDXUMfSL4uJiLSkpCXUYg8b+ulYu/OWbnJyfyV8+N8tGgTMmSonIcre99yihbqQ2IaCqfPcfa+nw+fjJx6dYcjDGBGQJIgq9sGYf/95YydcvnEChDfhjjOmGJYgoU9PUzr2L1zMlP5ObzywKdTjGmDBm3X1Hmf/6vw3UtXTw1C2ziIu13wfGmO7ZGSKKLNlcyd9WlPOFc8cxeWRGqMMxxoQ5SxBRoqnNw3f+vo6xuancfv4JoQ7HGDMIWBVTlPjvlzezt66Fv37+dJLiY0MdjjFmELASRBRYvquGJ98r5dOzx1BcmB3qcIwxg4QliAjX5vHyrefWMCIjibvmTgp1OMaYQcSqmCLcQ29sZ1tlI4/fdBppifbvNsb0nJUgItim/fX89o1tXDFtJHMmDQt1OMaYQcYSRITy+pRvPbeWjOR4vn/ZSaEOxxgzCFmCiFCPv7OT1Xtq+cFlk8m2bryNMb1gCSIC7T7YzAOvbOb8ScO4/JSRoQ7HGDNIWYKIMKrKt/++lriYGH50xcnWU6sxptcsQUSYvy4v4z/bDvCtuRMZOSQ51OEYYwYxSxARpLK+lR+9sIHTCrO4ftaYUIdjjBnkLEFEkB8sXk+rx8dPr5pKTIxVLRlj+sYSRIT417p9vLRuP1+5YDzjctNCHY4xJgJYgogAdc0dfO+f6zlxRAa3njM21OEYYyKE9b0QAX784kYONrax4MbTiLdBgIwx/cTOJoPcuvI6ninZw2fPHsuUgsxQh2OMiSCWIAa5n7+8mSEp8TYIkDGm31mCGMSW7jjIW1uq+OJ548hIig91OMaYCGMJYpBSVX7+r03kZSRyw+mFoQ7HGBOBLEEMUq9trGTF7lruuGC8DSFqjAkKSxCDkM+nPPDKZgqHpnBN8ahQh2OMiVCWIAah59fsZdP+Br524QS7rNUYEzR2dhlkOrw+fvnqFk4ckcFlU60rb2NM8FiCGGSeWbaHXQeb+ebFE6y/JWNMUAU1QYjIXBHZLCLbROTuAOvHiMhrIrJGRJaISIHfuhtFZKv7uDGYcQ4WLe1efv3aVorHZDFnoo0xbYwJrqAlCBGJBR4CPgpMBuaLyOQumz0A/FFVpwL3AT9xX5sN/ACYBcwEfiAiWcGKdbB48r1SKhvauGvuJBsIyBgTdMEsQcwEtqnqDlVtBxYC87psMxl43Z1+w2/9xcCrqlqtqjXAq8DcIMYa9upaOvjdku2cNzGXmUXZoQ7HGBMFgpkg8oE9fvNl7jJ/q4GPu9NXAukiMrSHr0VEbhWREhEpqaqq6rfAw9Gjb++grqWDOy+aGOpQjDFRItSN1HcC54rISuBcoBzw9vTFqvqwqharanFubm6wYgy5qoY2HvvPTi6dOoKT861DPmPMwAhmd9/lgP9dXAXuskNUdS9uCUJE0oCrVLVWRMqB87q8dkkQYw1rD72xjTaPj29cOCHUoRhjokgwSxDLgPEiUiQiCcC1wGL/DUQkR0Q6Y7gHWOBOvwxcJCJZbuP0Re6yqFNW08xf3t/NJ04tYKyNFGeMGUBBSxCq6gFuxzmxbwQWqep6EblPRC53NzsP2CwiW4A84H73tdXAf+EkmWXAfe6yqPPgv7eCwFc+Mj7UoRhjokxQR5RT1ReBF7ss+77f9LPAs928dgGHSxRRaWtFA39bUcbNZxYxIjM51OEYY6JMqBupzTH84pUtpCTE8cU5NhiQMWbgWYIIU6v31PKv9fv57NlFZKcmhDocY0wUsgQRph54ZTPZqQl89uyxoQ7FGBOlLEGEoXe3H+DtrQf44nnjSEsMajORMcZ0yxJEmHGGEt3MiMwkPjV7TKjDMcZEMUsQYebVDRWs2lPLV2woUWNMiFmCCCNedyjRsTmpXH1qwYe/wBhjgsgSRBhZvLqcLRWNfP2iCcTZUKLGmBCzs1CYaPc4Q4meNDKDS04eEepwjDHGEkS4eGbZbvZUt/DNiyfaUKLGmLDQowQhIqmdneqJyAQRuVxE4oMbWvRobvfw69e3MbMom3MnRG635caYwaWnJYi3gCQRyQdeAT4NPBGsoKLNE++WUtXQxl0XT7ShRI0xYaOnCUJUtRln7IbfquongJOCF1b0aGzz8Psl2zl/0jCKC20oUWNM+OhxghCR04Hrgf9zl9lF+v3g2ZI91Ld6uOMC687bGBNeepogvoozoM/f3TEdxgJvBC+s6ODzKY+/W8qpY7KYNmpIqMMxxpgj9KijH1V9E3gTwG2sPqCqdwQzsGjw+qZKdh1s5q6LJ4U6FGOMOUpPr2L6i4hkiEgqsA7YICLfDG5okW/BOzsZmZnExSflhToUY4w5Sk+rmCaraj1wBfASUIRzJZPppY376nl3+0FuPKPQ7po2xoSlnp6Z4t37Hq4AFqtqB6DBCyvyLfjPTpLjY7n2tNGhDsUYYwLqaYL4A1AKpAJvicgYoD5YQUW6A41t/HPVXq4+tYDMFLvf0BgTnnraSP1r4Nd+i3aJyJzghBT5/rx0N+1eHzedWRjqUIwxpls9baTOFJFfikiJ+/gFTmnCHKc2j5enlu5izsRcxuWmhTocY4zpVk+rmBYADcA17qMeeDxYQUWyF1bv40BjGzefVRTqUIwx5ph6OuDxOFW9ym/+hyKyKhgBRTJVZcE7Oxk/LI2zTsgJdTjGGHNMPS1BtIjIWZ0zInIm0BKckCLXBzurWb+3npvPKrJO+YwxYa+nJYgvAH8UkUx3vga4MTghRa4F7+wkKyWeK6fnhzoUY4z5UD0qQajqalU9BZgKTFXV6cD5QY0swuw+2MwrGyq4btZokuKtn0NjTPg7rlt4VbXevaMa4OtBiCdiPfleKbEifHp2YahDMcaYHulLHw9Wid5DDa0dPLNsD5dOHcHwzKRQh2OMMT3SlwRhXW300LPLy2hs8/CZM+3SVmPM4HHMBCEiDSJSH+DRAIz8sDcXkbkisllEtonI3QHWjxaRN0RkpYisEZFL3OWFItIiIqvcx+97/ReGmNenPGFjPhhjBqFjXsWkqum9fWMRiQUeAi4EyoBlIrJYVTf4bfZdYJGq/k5EJgMvAoXuuu2qOq23+w8XNuaDMWawCmY/0zOBbaq6Q1XbgYXAvC7bKJDhTmcCe4MYT0gs+I+N+WCMGZyCmSDygT1+82XuMn/3Ap8SkTKc0sOX/dYVuVVPb4rI2UGMM2g27K3nvR025oMxZnAK9VlrPvCEqhYAlwBPuUOa7gNGu/dbfB34i4hkdH2xiNza2YFgVVXVgAbeE4+/Y2M+GGMGr2AmiHJglN98gbvM3y3AIgBVfQ9IAnJUtU1VD7rLlwPbgQldd6CqD6tqsaoW5+bmBuFP6D0b88EYM9gFM0EsA8aLSJGIJADXAou7bLMbuABARE7ESRBVIpLrNnIjImOB8cCOIMba72zMB2PMYNfTvpiOm6p6ROR24GUgFligqutF5D6gRFUXA98AHhGRr+E0WN+kqioi5wD3iUgH4AO+oKrVwYq1v9mYD8aYSBC0BAGgqi/iND77L/u+3/QG4MwAr3sOeC6YsQWTjflgjIkEoW6kjjg25oMxJlJYguhnNuaDMSZSWILoZzbmgzEmUliC6Ec25oMxJpJYguhHNuaDMSaSWILoJzbmgzEm0liC6Cc25oMxJtJYgugHNuaDMSYSWYLoB51jPtxspQdjTASxBNEPbMwHY0wksgTRR3trW3hvx0Gunz3GxnwwxkQUO6P10bJSpw/BcyeEV3fjxhjTV5Yg+mhZaTVpiXFMGt7r4buNMSYsWYLoo2U7a5g+eohVLxljIo6d1fqgtrmdzRUNzCzMDnUoxhjT7yxB9EFJaQ0ApxVZgjDGRB5LEH2wbFc18bFiN8cZYyKSJYg+WLazmin5mdZzqzEmIlmC6KXWDi9ry+tCW73kaYP1f4dnb4b3HwZvR+hiMcZEnKCOSR3JVu6upcOroWmgrtwEK5+C1U9D80FIzIR1z8GyR2Huj+GEjwx8TMaYiGMJopdK3BvkiscMUIJoa3RKCyufgj3vQ0w8TLoEpt8A4+bAln/By9+BP10F4y+Ci+6H3AkDE5sxJiJZguilD0qrmZiXTmZKfPB2ogrlK2DFk04Job0RcibART+CqddCmt/d25MudUoOHzwMb/4cfnc6nPY5OPcuSLGrrIwxx88SRC94vD5W7KrhyhlBGne6uRrWPAMrnoLK9RCfAiddCTNugFGzQCTw6+IS4YwvO8njjfvhgz/AmoUw5ztw6mcg1v7dAal2f0yNiWJ2xuiFTfsbaGr3clp/tj/4fFD6Fqz4I2x8HrztMHIGfOxBOPkqSMro+Xul5cJlD8Jpn4V/3Q0v3um0T1z8Yzjhgv6LebDqaIHy5VD6Dux6B8pKYPjJcP53oeicUEdnBpv966B+L6QPh4yRkJwNMZFx/Y8liF74YKfT/tAvCaKmFNb81WlbqN0FSZnOr/0Zn4bhU/r23sNPhhufh80vuu0TH4fxF8PF90PO+L7HPli0NTjtNrvedR7ly50EjEDeyTDlatj6Kjx5GRSdCxd8HwqKQx21CWc+r9Pu995vYdd/jlwXEw/pIyBjhJM00kceTh7pIw6vS0gNTezHwRJELywrrSZ/SDIjhyT37g2aDsL6v8HavzonLoDCs+H878GJH4P4Xr5vICKH2yfe/wO89d/w29kw81anfSI5q2/v7/O6V1JlQHyYjMXdXA2733MTwjuwbzWoDyQWRk6HWV+AMWfC6FmH//6OVihZAG//Ah69ACZ8FM7/Tt+T9EDoaAVfByRah5FB19YIq/4MS38HNTshc5TTJjhqFjTsg4b9TmmiYZ/zqNgA215z2g+7Ssw4nCxGzXI+l2HWXiiqGuoY+kVxcbGWlJQEfT+qymn3/5uzx+fyq09O6/kL25tg80uwZhFsfw18Hhg2GaZ8wvkFO2R08IL211gFb/wIlj/pnBznfDtw+0RbIzRW+D0qnQ9/Y6U77043VTknX8T5hZQ9FrIKnefsIne+6PiqyHpK1TmuTVWwd+XhEkLlemd9bCIUnAZjznAeBadBYtqx37OtEd7/HbzzG2irg5M+7hyjcC1xbfgnPP9VaKl2LnfOGAmZ+ZCRD5kF7rM7n5EPCSmhjnhwqitzfmAtf9L5XBTMhNO/CJMu61nbXlsD1O+Dhr1HJ5G6cti7AhLSYfYX4PQv9f2H23EQkeWqGrDIbAniOO080MScB5bw4yuncN2sDzmpez2wYwmsXQQbX4COJudLOuVqmHKNUwUUKvvXwr/ugdK3IXeS8/BPBoF+8cTEQeowSM+DtDxIGwZpwyE11zlBVe+E6h3OL6umqiNfm5JzZMLwTyApQ52STkeL8+u/+aDfo+t8l2XetsP7iE91SgVjznBKCCNn9L5U01ID7/4Glv4ePC1wynVw3rcGLpF/mNY6ePEu5yKEkdNh8hXOSae+3DmZ1Zcf/T8A58STUeCXNEY6iSTvZOcHS4TUnfebsuXw3v86iRhg8uUw+0sw6rT+3U/FenjzZ85+EjNg9m0w+4uQHPxufCxB9KNFy/Zw13NrePVr5zA+L0CRXtWp416zyKlGaqpy2hUmXwFTr4HRZ4TPl1AVNv0fLPkpeFqdk/6hk3/e0fPH02NKMlwAABR3SURBVPjW1nBkwqje4c7vdE5e+H3uEtKdUkhHU/fvl5zl7D9lqN/Dbz5vMgw/pf+v1Gqsgv/8EpY95sRY/Bk4+xtOnXKo7Hwb/nGbkxDO+SaccyfEBrjc2tPmHOv6vc6v1Poy97n88HxLzeHtU3Kg6GynHaboHCd5R+PVXV4PbHoBlv7WqQJOzIRTb4CZn4cho4K77/3r4M2fOheqJGa6ieK2oCYKSxD96M6/rua1jRWs+N6FiP+X58BWp01hzSLnhBibCBPnOiWF8Rc6l6AaR0er0yB/KIGUOie4lK4JwH0kDQn9Jbp1ZU77zco/OY2Qs26FM786sHXGHa3w+n/Bew/B0HFw5cNQcGrf3rO92fnbyktgx5uw802n2gOc+vWicw4njIwRff8bwllrnXNp+ft/gLrdTlXprNtg+vUD376zf63zw23TC06iOP2LTqJIyuz3XYUsQYjIXOB/gFjgUVX9aZf1o4EngSHuNner6ovuunuAWwAvcIeqvnysfQ1Ugjj3v99gQl46j1w9FnYvdRpBd77p/EMR54s09Ro48bKg/DNNiFXvcL64axY5J43Tv+RUBQSjjcXfvtXwt89D1UbnBsgL7wtOe4IqHNzmfKZ3vOlUQXaWMnImHE4YhWeFXYNqQD6vU5LytjuPo6Y7nCrEzS85yaG9wamenP1FmPhRiAlxR5z71jhVT5tecM4np9/uNGb34+ctJAlCRGKBLcCFQBmwDJivqhv8tnkYWKmqvxORycCLqlroTj8NzARGAv8GJqiqt7v9BT1BNFRQt2kJ//jnX/lYZilDm7Y6y2MTIf9U50qhk6+K/F9ZxlG50bkZcePzTtVX8c3OD4Pcif27H58X3nkQ3viJU5q64qGB7WvL54OKtW7p4i3nIoCOJkBgxFQ3YZzntGl0tDhVlV2fPa1O6cfT4j4HWOfrcKrw1Of8zepzkpX6QL2H1x1ar37LvM6J3tvhtEl528HT7kyrr2d/Z0yc8/2dfZvTphNu9q12fphsftEpUZ9+O8z6fL8kilAliNOBe1X1Ynf+HgBV/YnfNn8Adqjqz9ztf6GqZ3TdVkRedt/rve721+8Jonb34cskd73r/KoCmjQRb/5MMiad2/eGUDP47V3pfHG3vuKcjIZPdRLFyVc5DcB9Ub0T/v4F2LPUacP62K9C/6vd0+60se18yyll7PnAObn3lMRAXLJT5RqfDHFJznNsvLPu0CPWfRbnOSY2wHo5PB8bD7EJziMu0Z1PdOcT3Ol4d53/du507sTQtiv11N6VsORnsOUlp12uM1H0oQosVAniamCuqn7Wnf80MEtVb/fbZgTwCpAFpAIfUdXlIvK/wFJV/ZO73WPAS6r6bHf761OC6CxWdyaDXe9C3R5nXVKm07A85gwe2T2CX29IZsW9lxBvY1Abfw37Yd3fnCvW9q7EqW4822mDmnz58VU3qjp31P/rHueX7aUPOJdDh2ODcXuzk8Baao884ccludNJTkLofI6ND8+/Y7ApX+FUPW35l5MozrgDzvpar47tsRJEqG+Umw88oaq/cEsQT4lIj6/9FJFbgVsBRo/u5eWHtXvgkfOhqdKZT811LpM848vO87DJh+oh/7H8baaMjrfkYI6WPtxpSDz9i3Bgm5Mo1iyCxbfD/30DJlzslCzGX3TsCxYaK2HxHc4vxKJz4IrfOZehhquEFBh3fqijiD75M+C6Z5zS3JKfOt3FBCHxBjNBlAP+14QVuMv83QLMBVDV90QkCcjp4WtR1YeBh8EpQfQqyoyRMOEiyC92Gt6GnhDwQDe0drBxXz23nx+mN0yZ8JFzgnNz3Xn3HHnJ88bF7iXP85wSwZizjrxseOML8Pwdzs1/c3/qXFYZLpdEm/CUfypc/1enwT0IgpkglgHjRaQI5+R+LXBdl212AxcAT4jIiUASUAUsBv4iIr/EaaQeD3wQlChjYmHeQx+62fJdNfiU0AwQZAYnEadPp4Jip6PEnUucfrfWPudUIaWPhClXwYnzYPkTsOpPMOIU5/LVYZNCHb0ZTIJ0GX3QEoSqekTkduBlnEtYF6jqehG5DyhR1cXAN4BHRORrOHdO3aROo8h6EVkEbAA8wJeOdQXTQFhWWk1sjDB9dPDvbDQRKDbOufrohI9A+6+cq1HW/tXp0+fd3zgNrWffCed+y2lUNSYM2I1yPXTNH96jrcPLP28/K2j7MFGo6aDT0DjsRKde2ZgBdqxGaqvg7IE2j5dVe2optuol099Shzp36lpyMGHIEkQPrC2ro93j698BgowxJsxZguiBD0o7BwgauC54jTEm1CxB9EBJaQ3jclMZmmYd7hljoocliA/h8yklpdVWvWSMiTqWID7E5ooG6ls9liCMMVHHEsSHKHHbH2YWWYIwxkQXSxAf4oPSGvIyEinISg51KMYYM6AsQRyDqrJsp9P+INYDpTEmyliCOIaymhb217da9ZIxJipZgjiGZYfuf7AEYYyJPqEeDyKoOjo6KCsro7W1tVevH67tPDZvBFpTzsbao3obDytJSUkUFBQQHx8f6lCMMREiohNEWVkZ6enpFBYW9qoNYfP+BhLiYijKSQ1CdP1HVTl48CBlZWUUFRWFOhxjTISI6Cqm1tZWhg4d2qvk4PH6aPN4SU2MDUJk/UtEGDp0aK9LSsYYE0hEJwig11cfNbU7w0+kJgyOQpZdZWWM6W8RnyB6q6nNg4iQnBD+JQhjjAkGSxDdaG73kBIfS0wffpkfPHiQadOmMW3aNIYPH05+fv6h+fb29mO+tqSkhDvuuKPX+zbGmL4aHPUnA8zrU1rafeSm923ox6FDh7Jq1SoA7r33XtLS0rjzzjsPrfd4PMTFBf4XFBcXU1wccJAnY4wZEFGTIH74/Ho27K3v0bZen9La4SUpPpbYmO5LEJNHZvCDy046rjhuuukmkpKSWLlyJWeeeSbXXnstX/nKV2htbSU5OZnHH3+ciRMnsmTJEh544AFeeOEF7r33Xnbv3s2OHTvYvXs3X/3qV610YYwJuqhJEMfD547Tfazk0BdlZWW8++67xMbGUl9fz9tvv01cXBz//ve/+fa3v81zzz131Gs2bdrEG2+8QUNDAxMnTuS2226zex6MMUEVNQnieH7p76hqxOtTxuelByWWT3ziE8TGOo3fdXV13HjjjWzduhURoaOjI+BrLr30UhITE0lMTGTYsGFUVFRQUFAQlPiMMQaskfooPlWa272kJgYvd6amHr7x7nvf+x5z5sxh3bp1PP/8893ey5CYeHg0u9jYWDweT9DiM8YYsARxlNYOLz5VUgbo8ta6ujry8/MBeOKJJwZkn8YY0xOWILpoanNvkAtiCcLfXXfdxT333MP06dOtVGCMCSuiboPsYFdcXKwlJSVHLNu4cSMnnnjicb1P6YEmWj1eJg3P6M/wBkRv/l5jTHQTkeWqGvCaeitB+FFVmts9g6Z7DWOMCSZLEH7aPD48Ph0UHfQZY0ywWYLw09TmtAFYCcIYYyxBHKG53UtcTAwJcXZYjDHGzoR+mto8pCbGWtfZxhhDkBOEiMwVkc0isk1E7g6w/lcissp9bBGRWr91Xr91i4MZJ0C7x0e710eKVS8ZYwwQxAQhIrHAQ8BHgcnAfBGZ7L+Nqn5NVaep6jTgN8Df/Fa3dK5T1cuDFWen5na3/aGfG6jnzJnDyy+/fMSyBx98kNtuuy3g9ueddx5dL9c1xphQCGYJYiawTVV3qGo7sBCYd4zt5wNPBzGeY2pq8xIjQnJ8/yaI+fPns3DhwiOWLVy4kPnz5/frfowxpr8Fsz4lH9jjN18GzAq0oYiMAYqA1/0WJ4lICeABfqqq/+hTNC/dDfvXdrs6q8NDNoIcT4IYPgU++tNjbnL11Vfz3e9+l/b2dhISEigtLWXv3r08/fTTfP3rX6elpYWrr76aH/7whz3frzHGDIBwaaS+FnhWVb1+y8a4d/ddBzwoIuO6vkhEbhWREhEpqaqq6vXOFcXnC0733tnZ2cycOZOXXnoJcEoP11xzDffffz8lJSWsWbOGN998kzVr1vT7vo0xpi+CWYIoB0b5zRe4ywK5FviS/wJVLXefd4jIEmA6sL3LNg8DD4PT1cYxoznGL/2Glg5KDzYxNieNhKT+PySd1Uzz5s1j4cKFPPbYYyxatIiHH34Yj8fDvn372LBhA1OnTu33fRtjTG8FswSxDBgvIkUikoCTBI66GklEJgFZwHt+y7JEJNGdzgHOBDYEK9Cmdg8iErQeXOfNm8drr73GihUraG5uJjs7mwceeIDXXnuNNWvWcOmll3bbzbcxxoRK0BKEqnqA24GXgY3AIlVdLyL3iYj/VUnXAgv1yF4DTwRKRGQ18AZOG0TwEkSbl+T4WGKCNIJcWloac+bM4eabb2b+/PnU19eTmppKZmYmFRUVh6qfjDEmnAT1on9VfRF4scuy73eZvzfA694FpgQztk4+n9LS4SUnLSGo+5k/fz5XXnklCxcuZNKkSUyfPp1JkyYxatQozjzzzKDu2xhjeiPq7wrzqpKZFE96kMd/uOKKK/AvJHU3ONCSJUuCGocxxvRU1CeI+NgYRg9NCXUYxhgTdsLlMldjjDFhJuITRKSMmPdhouXvNMYMnIhOEElJSRw8eDDiT56qysGDB0lKSgp1KMaYCBLRbRAFBQWUlZXRl7usB4ukpCQKCgpCHYYxJoJEdIKIj4+nqKgo1GEYY8ygFNFVTMYYY3rPEoQxxpiALEEYY4wJSCLlCh8RqQJ29eEtcoAD/RROMFh8fWPx9Y3F1zfhHN8YVc0NtCJiEkRfiUiJO/5EWLL4+sbi6xuLr2/CPb7uWBWTMcaYgCxBGGOMCcgSxGEPhzqAD2Hx9Y3F1zcWX9+Ee3wBWRuEMcaYgKwEYYwxJiBLEMYYYwKKqgQhInNFZLOIbBORuwOsTxSRZ9z174tI4QDGNkpE3hCRDSKyXkS+EmCb80SkTkRWuY/vB3qvIMdZKiJr3f2XBFgvIvJr9xiuEZEZAxjbRL9js0pE6kXkq122GdBjKCILRKRSRNb5LcsWkVdFZKv7nNXNa290t9kqIjcOYHz/LSKb3P/f30VkSDevPeZnIYjx3Ssi5X7/w0u6ee0xv+9BjO8Zv9hKRWRVN68N+vHrM1WNigcQC2wHxgIJwGpgcpdtvgj83p2+FnhmAOMbAcxwp9OBLQHiOw94IcTHsRTIOcb6S4CXAAFmA++H8P+9H+cmoJAdQ+AcYAawzm/Zz4G73em7gZ8FeF02sMN9znKnswYovouAOHf6Z4Hi68lnIYjx3Qvc2YP//zG/78GKr8v6XwDfD9Xx6+sjmkoQM4FtqrpDVduBhcC8LtvMA550p58FLhARGYjgVHWfqq5wpxuAjUD+QOy7n80D/qiOpcAQERkRgjguALaral/uru8zVX0LqO6y2P9z9iRwRYCXXgy8qqrVqloDvArMHYj4VPUVVfW4s0uBkPUj383x64mefN/77FjxueeOa4Cn+3u/AyWaEkQ+sMdvvoyjT8CHtnG/IHXA0AGJzo9btTUdeD/A6tNFZLWIvCQiJw1oYA4FXhGR5SJya4D1PTnOA+Fauv9ihvoY5qnqPnd6P5AXYJtwOY4345QIA/mwz0Iw3e5WgS3opoouHI7f2UCFqm7tZn0oj1+PRFOCGBREJA14DviqqtZ3Wb0Cp8rkFOA3wD8GOj7gLFWdAXwU+JKInBOCGI5JRBKAy4G/BlgdDsfwEHXqGsLyWnMR+Q7gAf7czSah+iz8DhgHTAP24VTjhKP5HLv0EPbfpWhKEOXAKL/5AndZwG1EJA7IBA4OSHTOPuNxksOfVfVvXderar2qNrrTLwLxIpIzUPG5+y13nyuBv+MU5f315DgH20eBFapa0XVFOBxDoKKz2s19rgywTUiPo4jcBHwMuN5NYkfpwWchKFS1QlW9quoDHulmv6E+fnHAx4FnutsmVMfveERTglgGjBeRIvcX5rXA4i7bLAY6rxa5Gni9uy9Hf3PrKx8DNqrqL7vZZnhnm4iIzMT5/w1kAksVkfTOaZzGzHVdNlsM3OBezTQbqPOrThko3f5yC/UxdPl/zm4E/hlgm5eBi0Qky61CuchdFnQiMhe4C7hcVZu72aYnn4VgxeffpnVlN/vtyfc9mD4CbFLVskArQ3n8jkuoW8kH8oFzhc0WnKsbvuMuuw/niwCQhFMtsQ34ABg7gLGdhVPVsAZY5T4uAb4AfMHd5nZgPc4VGUuBMwb4+I11973ajaPzGPrHKMBD7jFeCxQPcIypOCf8TL9lITuGOIlqH9CBUw9+C0671mvAVuDfQLa7bTHwqN9rb3Y/i9uAzwxgfNtw6u87P4edV/aNBF481mdhgOJ7yv1srcE56Y/oGp87f9T3fSDic5c/0fmZ89t2wI9fXx/W1YYxxpiAoqmKyRhjzHGwBGGMMSYgSxDGGGMCsgRhjDEmIEsQxhhjArIEYcxxEBFvlx5j+62XUBEp9O8V1JhQiwt1AMYMMi2qOi3UQRgzEKwEYUw/cPv2/7nbv/8HInKCu7xQRF53O5Z7TURGu8vz3LEWVruPM9y3ihWRR8QZE+QVEUkO2R9lop4lCGOOT3KXKqZP+q2rU9UpwP8CD7rLfgM8qapTcTq9+7W7/NfAm+p0GjgD525agPHAQ6p6ElALXBXkv8eYbtmd1MYcBxFpVNW0AMtLgfNVdYfb6eJ+VR0qIgdwuoLocJfvU9UcEakCClS1ze89CnHGgBjvzn8LiFfVHwX/LzPmaFaCMKb/aDfTx6PNb9qLtROaELIEYUz/+aTf83vu9Ls4PYkCXA+87U6/BtwGICKxIpI5UEEa01P268SY45PcZRD6f6lq56WuWSKyBqcUMN9d9mXgcRH5JlAFfMZd/hXgYRG5BaekcBtOr6DGhA1rgzCmH7htEMWqeiDUsRjTX6yKyRhjTEBWgjDGGBOQlSCMMcYEZAnCGGNMQJYgjDHGBGQJwhhjTECWIIwxxgT0/3CQEM5aSOuJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model_Bi_LSTM_hist.history['binary_accuracy'])\n",
    "plt.plot(model_Bi_LSTM_hist.history['val_binary_accuracy'])\n",
    "\n",
    "plt.title('Model Loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['Train', 'Val'], loc='lower left')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "b70FyN5kZQoY"
   },
   "outputs": [],
   "source": [
    "model_Bi_LSTM.save('models/LSTM-Bi-Model.h5')  # save model\n",
    "del model_Bi_LSTM  # delete existing model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "lH6yewEOZR8c"
   },
   "outputs": [],
   "source": [
    "# returns a compiled model, identical to the previous one\n",
    "model_Bi_LSTM = load_model('models/LSTM-Bi-Model.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DPXLkLhYrFko",
    "outputId": "7a835d49-428b-4c5f-ac2f-3967114993d0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "88/88 [==============================] - 9s 75ms/step - loss: 0.4190 - binary_accuracy: 0.9093\n",
      "\n",
      "Model Accuracy: 90.93%\n"
     ]
    }
   ],
   "source": [
    "model_Bi_LSTM_score = model_Bi_LSTM.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\", str(round(model_Bi_LSTM_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "av6jpxzJUAv5"
   },
   "source": [
    "<h2 align=\"center\">CNN + LSTM</h2>\n",
    "\n",
    "This approach is based entirely on [this work](https://www.aclweb.org/anthology/P16-2037.pdf). As indicated in the work:\n",
    "\n",
    "<cite>\n",
    "    \n",
    "    Dimensional sentiment analysis aims to recognize continuous numerical values in multiple dimensions such as the valencearousal (VA) space. \n",
    "    \n",
    "    Compared to the categorical approach that focuses on sentiment classification such as binary classification (i.e., positive and negative), the dimensional approach can provide more fine-grained sentiment analysis. This study proposes a regional CNN-LSTM model consisting of two parts: regional CNN and LSTM to predict the VA ratings of texts. \n",
    "   \n",
    "    Unlike a conventional CNN which considers a whole text as input, the proposed regional CNN uses an individual sentence as a region, dividing an input text into several regions such that the useful affective information in each region can be extracted and weighted according to their contribution to the VA prediction. Such regional information is sequentially integrated across regions using LSTM for VA prediction. \n",
    "\n",
    "</cite>\n",
    "\n",
    "In short, word vectors of vocabulary words are trained from a large corpus using the word2vec toolkit. For each given text, the  regional CNN model uses the sentence as a region to divide the text into R-domains, ie r1, ..., ri rj, rk, ..., rR. In each region, useful affective functions can be removed when word vectors pass sequentially through the convolutional layer and the  maxpooling  layer. Then such local (regional) features are sequentially integrated between regions using LSTM to construct a text vector for VA prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RLcp2YbBUAv5",
    "outputId": "dc6fa980-bd39-4a38-f20f-35320f927074"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " conv1d_2 (Conv1D)           (None, 50, 128)           153728    \n",
      "                                                                 \n",
      " max_pooling1d_2 (MaxPooling  (None, 16, 128)          0         \n",
      " 1D)                                                             \n",
      "                                                                 \n",
      " bidirectional (Bidirectiona  (None, 16, 128)          98816     \n",
      " l)                                                              \n",
      "                                                                 \n",
      " lstm_3 (LSTM)               (None, 32)                20608     \n",
      "                                                                 \n",
      " dropout_1 (Dropout)         (None, 32)                0         \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 1)                 33        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 273,185\n",
      "Trainable params: 273,185\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import Conv1D, MaxPooling1D, Flatten\n",
    "from keras.layers import Bidirectional\n",
    "\n",
    "# configure network\n",
    "n_epoch = 20\n",
    "n_batch = 64\n",
    "\n",
    "# design network\n",
    "model_CNN_LSTM = Sequential()\n",
    "\n",
    "model_CNN_LSTM.add(Conv1D(filters=128, kernel_size=4, input_shape=(x.shape[1], x.shape[2]), padding='same', activation='relu'))\n",
    "model_CNN_LSTM.add(MaxPooling1D(pool_size=3))\n",
    "model_CNN_LSTM.add(Bidirectional(LSTM(64, return_sequences = True)))\n",
    "model_CNN_LSTM.add(LSTM(32, recurrent_dropout = 0.4))\n",
    "model_CNN_LSTM.add(Dropout(0.2))\n",
    "model_CNN_LSTM.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model_CNN_LSTM.compile(loss='binary_crossentropy', optimizer='adam', metrics=['binary_accuracy'])\n",
    "\n",
    "print(model_CNN_LSTM.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "a3uQzmQsUAv6",
    "outputId": "b3835d7b-1978-435f-8f81-89664e75fe5a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "264/264 [==============================] - 34s 102ms/step - loss: 0.5486 - binary_accuracy: 0.7165\n",
      "Epoch 2/20\n",
      "264/264 [==============================] - 28s 105ms/step - loss: 0.4245 - binary_accuracy: 0.8054\n",
      "Epoch 3/20\n",
      "264/264 [==============================] - 28s 105ms/step - loss: 0.3418 - binary_accuracy: 0.8536\n",
      "Epoch 4/20\n",
      "264/264 [==============================] - 27s 102ms/step - loss: 0.2504 - binary_accuracy: 0.9016\n",
      "Epoch 5/20\n",
      "264/264 [==============================] - 27s 102ms/step - loss: 0.1758 - binary_accuracy: 0.9352\n",
      "Epoch 6/20\n",
      "264/264 [==============================] - 27s 102ms/step - loss: 0.1236 - binary_accuracy: 0.9575\n",
      "Epoch 7/20\n",
      "264/264 [==============================] - 28s 105ms/step - loss: 0.0946 - binary_accuracy: 0.9681\n",
      "Epoch 8/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0805 - binary_accuracy: 0.9723\n",
      "Epoch 9/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0592 - binary_accuracy: 0.9799\n",
      "Epoch 10/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0571 - binary_accuracy: 0.9819\n",
      "Epoch 11/20\n",
      "264/264 [==============================] - 28s 105ms/step - loss: 0.0529 - binary_accuracy: 0.9828\n",
      "Epoch 12/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0394 - binary_accuracy: 0.9877\n",
      "Epoch 13/20\n",
      "264/264 [==============================] - 30s 112ms/step - loss: 0.0404 - binary_accuracy: 0.9867\n",
      "Epoch 14/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0497 - binary_accuracy: 0.9825\n",
      "Epoch 15/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0330 - binary_accuracy: 0.9902\n",
      "Epoch 16/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0357 - binary_accuracy: 0.9877\n",
      "Epoch 17/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0382 - binary_accuracy: 0.9874\n",
      "Epoch 18/20\n",
      "264/264 [==============================] - 27s 104ms/step - loss: 0.0340 - binary_accuracy: 0.9888\n",
      "Epoch 19/20\n",
      "264/264 [==============================] - 27s 103ms/step - loss: 0.0340 - binary_accuracy: 0.9895\n",
      "Epoch 20/20\n",
      "264/264 [==============================] - 27s 104ms/step - loss: 0.0283 - binary_accuracy: 0.9910\n"
     ]
    }
   ],
   "source": [
    "model_CNN_LSTM_hist = model_CNN_LSTM.fit(x_train, np.asarray(y_train), epochs=n_epoch, batch_size=n_batch, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6Xn1ZX1FUAv7",
    "outputId": "fe68e529-9651-41e4-dde1-3493b5ba2fc1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "88/88 [==============================] - 4s 36ms/step - loss: 1.0092 - binary_accuracy: 0.7858\n",
      "\n",
      "Model Accuracy: 78.58%\n"
     ]
    }
   ],
   "source": [
    "model_CNN_LSTM_score = model_CNN_LSTM.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\",str(round(model_CNN_LSTM_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "iyGfw-x_UAv7",
    "outputId": "806a0b35-8c32-43c4-cbfc-e52812e21fe1"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3/8dcnkz0kgZCFhATCLltUNrW4ixWEYmtri9209dbaXuvS20Xb29b22vbXzba23lbtdWuvVdvq1SqKuKNWMQKC7BAggARCgCQQsn9/f8yJjjGBQDJzMjPv5+Mxj8ycc2bmnclk3nOW+Y455xARkfiV4HcAERHxl4pARCTOqQhEROKcikBEJM6pCERE4pyKQEQkzqkIRI7AzErNzJlZYg+WvdzMXo5ELpG+pCKQmGFmW82s2cxyO01f7r2Yl/qT7NgKRSTSVAQSa7YAl3ZcMLPJQLp/cUT6PxWBxJo/A58PuXwZcF/oAmaWbWb3mVm1mW0zs/80swRvXsDMfmlme82sApjbxXX/x8x2mdlOM7vZzAK9CWxmRWb2mJntM7NNZvalkHkzzKzczOrMbLeZ3eJNTzWzv5hZjZkdMLM3zKygNzkkfqkIJNa8BmSZ2XjvBXoB8JdOy/wOyAZGAmcRLI4vePO+BMwDTgamAZ/odN17gFZgtLfMh4F/62XmB4AdQJF3fz8xs3O9eb8FfuucywJGAQ950y/zfocSYDBwFXC4lzkkTqkIJBZ1rBWcD6wFdnbMCCmHG51z9c65rcCvgM95i3wS+I1zbrtzbh/w05DrFgAXAtc55w455/YAv/Zu77iYWQkwE/i2c67RObcC+BPvrdW0AKPNLNc5d9A591rI9MHAaOdcm3PuTedc3fHmkPimIpBY9Gfg08DldNosBOQCScC2kGnbgKHe+SJge6d5HYZ7193lbY45ANwO5PciaxGwzzlX302eK4CxwDpv8888b/qfgUXAA2b2jpn93MySepFD4piKQGKOc24bwZ3GFwIPd5q9l+C76eEh04bx3lrDLoKbW0LnddgONAG5zrmB3inLOTexF3HfAXLMLLOrPM65jc65SwmWzc+Av5tZhnOuxTn3Q+fcBOBDBDdnfR6R46AikFh1BXCuc+5Q6ETnXBvB7ew/NrNMMxsOfJ339iM8BFxjZsVmNgi4IeS6u4CngV+ZWZaZJZjZKDM76xhypXg7elPNLJXgC/6rwE+9aWVe9r8AmNlnzSzPOdcOHPBuo93MzjGzyd6mrjqC5dZ+DDlE3qUikJjknNvsnCvvZvbXgENABfAycD9wlzfvToKbXN4ClvHBNYrPA8nAGmA/8Heg8BiiHSS4U7fjdC7Bw11LCa4dPAL8wDn3jLf8bGC1mR0kuON4gXPuMDDEu+86gvtBXiS4uUjkmJm+mEZEJL5pjUBEJM6pCERE4pyKQEQkzqkIRETiXNSNhJibm+tKS0v9jiEiElXefPPNvc65vK7mRV0RlJaWUl7e3VGBIiLSFTPb1t08bRoSEYlzKgIRkTinIhARiXMqAhGROKciEBGJcyoCEZE4pyIQEYlzcVMEyyr387On1vkdQ0Sk34mbInh7Zy1/eGEz66vqj76wiEgciZsimDOpkASDx1e+43cUEZF+JW6KIC8zhVNHDubxlbvQl/GIiLwnbooAYF5ZEVv2HmL1O3V+RxER6TfiqghmTxpCIMF4fOUuv6OIiPQbcVUEORnJzBydy+Mr39HmIRERT1wVAcC8skJ27D/MWztq/Y4iItIvxF0RXDBhCEkB4/G3dPSQiAjEYRFkpydx5pg8nli1i/Z2bR4SEYm7IgCYd2Ihu2obWVa53+8oIiK+i8simDW+gOTEBB09JCJCnBZBZmoS54wLbh5q0+YhEYlzcVkEEPxwWXV9E0u37PM7ioiIr+K2CM4bn09aUkBjD4lI3IvbIkhPTuTc8fk89XYVrW3tfscREfFN3BYBwEfKCqk51My/Kmr8jiIi4puwFoGZzTaz9Wa2ycxu6GL+5WZWbWYrvNO/hTNPZ2ePyycjOcDjb+noIRGJX2ErAjMLALcBc4AJwKVmNqGLRR90zp3knf4UrjxdSU0KcP6EAp5aXUVzqzYPiUh8CucawQxgk3OuwjnXDDwAXBTG+zsu88qKqD3cwiub9vodRUTEF+EsgqHA9pDLO7xpnX3czFaa2d/NrKSrGzKzK82s3MzKq6ur+zTkGWNzyUxN5J86ekhE4pTfO4v/CZQ658qAxcC9XS3knLvDOTfNOTctLy+vTwOkJAa4YOIQFq/eTWNLW5/etohINAhnEewEQt/hF3vT3uWcq3HONXkX/wRMDWOebs0rK6S+qZWXNvTt2oaISDQIZxG8AYwxsxFmlgwsAB4LXcDMCkMuzgfWhjFPt2aOzmVQepLGHhKRuJQYrht2zrWa2dXAIiAA3OWcW21mPwLKnXOPAdeY2XygFdgHXB6uPEeSFEhg9qQhPLriHQ43t5GWHPAjhoiIL8JWBADOuYXAwk7Tvh9y/kbgxnBm6Kl5ZUX8del2nl+/hwsnFx79CiIiMcLvncX9xikjcsgdkKyxh0Qk7qgIPImBBOZMKuS5dXs41NTqdxwRkYhREYSYV1ZIY0s7z6zd7XcUEZGIURGEmF6aQ0FWio4eEpG4oiIIkZBgXDi5kBfXV1PX2OJ3HBGRiFARdDKvrIjmtnYWr9bmIRGJDyqCTqYMG8jQgWk6ekhE4oaKoBMzY25ZIUs27uVAQ7PfcUREwk5F0IV5ZYW0tjsWra7yO4qISNipCLoweWg2w3LSdfSQiMQFFUEXzIx5ZYW8urmGmoNNR7+CiEgUUxF0Y15ZEW3tjqe0eUhEYpyKoBvjCzMZmZehL7YXkZinIuhGcPNQEa9vqWFPfaPfcUREwkZFcAQfKSuk3cGTq7R5SERil4rgCMYUZDKuIFMfLhORmKYiOIp5ZYW8sXU/u2oP+x1FRCQsVARHMe/EIgCe0GcKRCRGqQiOYkRuBhOLsvThMhGJWSqCHphXVsSK7QfYvq/B7ygiIn1ORdAD88qCX2b/xCqtFYhI7FER9EBJTjonlgzU0UMiEpNUBD100YlFvL2zjuWV+/2OIiLSp1QEPfSp6SXkZCTz62c2+h1FRKRPqQh6KCMlkavOGslLG6op37rP7zgiIn1GRXAMPndqKbkDUrhl8Qa/o4iI9BkVwTFISw7w1bNH8ermGv61ucbvOCIifUJFcIw+fcowCrJS+PXiDTjn/I4jItJrKoJjlJoU4OpzRrN06z5e3rTX7zgiIr0W1iIws9lmtt7MNpnZDUdY7uNm5sxsWjjz9JVPTi+hKDuVW7RWICIxIGxFYGYB4DZgDjABuNTMJnSxXCZwLfB6uLL0tZTEAF87bwzLKw/wwvpqv+OIiPRKONcIZgCbnHMVzrlm4AHgoi6W+y/gZ0BUfQ3YJ6YWU5KTprUCEYl64SyCocD2kMs7vGnvMrMpQIlz7okj3ZCZXWlm5WZWXl3dP96BJwUSuObcMazaWcviNbv9jiMictx821lsZgnALcB/HG1Z59wdzrlpzrlpeXl54Q/XQx87eSilg9P59TMbaW/XWoGIRKdwFsFOoCTkcrE3rUMmMAl4wcy2AqcCj0XLDmOAxEAC184aw9pddTy1Wt9rLCLRKZxF8AYwxsxGmFkysAB4rGOmc67WOZfrnCt1zpUCrwHznXPlYczU5+afOJRReRn8evEG2rRWICJRKGxF4JxrBa4GFgFrgYecc6vN7EdmNj9c9xtpgQTjullj2bjnoIapFpGoZNF2xMu0adNceXn/Wmlob3fM+e0SWtraefr6M0kM6HN6ItK/mNmbzrkuN73rFasPJCQY158/hoq9h3h0hdYKRCS6qAj6yAUThzCxKIvfPruRlrZ2v+OIiPSYiqCPmBnXzxpL5b4GHl62w+84IiI9piLoQ+eNz+fE4mxufXYTza1aKxCR6KAi6ENmxvXnj2XngcM8VL796FcQEekHVAR97KyxeUwdPojfP7eJxpY2v+OIiByViqCPmRlfP38sVXWNPLC00u84IiJHpSIIgw+NGswpI3K47YXNHG7WWoGI9G8qgjDoWCuorm/if1/f5nccEZEjUhGEySkjB3P66Fz+8MJmDjW1+h1HRKRbKoIwuv78sdQcaua+f2mtQET6LxVBGE0dPoizx+Vx+0ubqW9s8TuOiEiXVARhdv2ssRxoaOGeV7b6HUVEpEsqgjA7sWQgs8YXcOeSCmoPa61ARPofFUEEXH/+GOoaW/mfl7f4HUVE5ANUBBEwsSibOZOGcNfLW6ht0FqBiPQvKoII+dq5YzjY1Mpf9LkCEelnVAQRMqEoizPH5nH3K1s1BpGI9Csqggi66syR7D3YxMPLdvodRUTkXSqCCDpt1GAmD83mziUVtLVH13dFi0jsUhFEkJnx5bNGsmXvIRavqfI7jogIoCKIuNkThzAsJ50/vFiBc1orEBH/qQgiLDGQwJfOGMFb2w+wdMs+v+OIiKgI/HDJtBIGZyRz+0sVfkcREVER+CE1KcBlHyrluXV7WF9V73ccEYlzKgKffO7U4aQlBbhDawUi4jMVgU8GZSTzqeklPLpiJ7tqD/sdR0TimIrAR1ecPgIH3KXB6ETERz0qAjPLMLME7/xYM5tvZknhjRb7SnLSmVdWyP2vV2qIahHxTU/XCF4CUs1sKPA08DngnqNdycxmm9l6M9tkZjd0Mf8qM1tlZivM7GUzm3As4WPBlWeO5FBzm77kXkR809MiMOdcA3Ax8N/OuUuAiUe8glkAuA2YA0wALu3ihf5+59xk59xJwM+BW44pfQyYWJTNGWNyuetlDUYnIv7ocRGY2WnAZ4AnvGmBo1xnBrDJOVfhnGsGHgAuCl3AOVcXcjEDiMuP2l511ij2HmzikeUajE5EIq+nRXAdcCPwiHNutZmNBJ4/ynWGAttDLu/wpr2Pmf27mW0muEZwTVc3ZGZXmlm5mZVXV1f3MHL0+NCowUwamsWdL2kwOhGJvB4VgXPuRefcfOfcz7ydxnudc12+aB8r59xtzrlRwLeB/+xmmTucc9Occ9Py8vL64m77FTPjqrNGUbH3EIvX7PY7jojEmZ4eNXS/mWWZWQbwNrDGzL55lKvtBEpCLhd707rzAPDRnuSJRR2D0f3xxc0ajE5EIqqnm4YmeNvzPwo8CYwgeOTQkbwBjDGzEWaWDCwAHgtdwMzGhFycC2zsYZ6Y0zEY3YrtB3hj636/44hIHOlpESR5nxv4KPCYc66Fo+zYdc61AlcDi4C1wEPe/oUfmdl8b7GrzWy1ma0Avg5cdly/RYz4xNQScjKSuf3FzX5HEZE4ktjD5W4HtgJvAS+Z2XCg7ojXAJxzC4GFnaZ9P+T8tT1OGgfSkgNcdlopv35mAxt21zO2INPvSCISB3q6s/hW59xQ59yFLmgbcE6Ys8Wlz5+mwehEJLJ6urM428xu6TiE08x+RfC4f+ljGoxORCKtp/sI7gLqgU96pzrg7nCFindXnD6Cdgd3v7LV7ygiEgd6WgSjnHM/8D4lXOGc+yEwMpzB4llJTjpzJ2swOhGJjJ4WwWEzO73jgpnNBLTdIoyuPHMkB5tauf/1Sr+jiEiM62kRXAXcZmZbzWwr8Hvgy2FLJUwa6g1G98oWDUYnImHV06OG3nLOnQiUAWXOuZOBc8OaTLjqrFFU1zfxfxqMTkTC6Ji+ocw5VxcyYujXw5BHQnQMRnfHSxW0azA6EQmT3nxVpfVZCumSmfHlM73B6NZqMDoRCY/eFIHeokbAnElDKMlJ02B0IhI2RywCM6s3s7ouTvVAUYQyxrXgYHQjWV55gPJtGoxORPreEYvAOZfpnMvq4pTpnOvpOEXSS5dMLWFQepIGoxORsOjNpiGJkLTkAJd9qJRn1u5h1Y5av+OISIxREUSJL54+gsEZyfzXE2u0r0BE+pSKIEpkpSZx/fljWbplH4tWV/kdR0RiiIogiiyYXsLYggH8ZOE6mlr1aWMR6RsqgiiSGEjgP+dOoHJfA/e9us3vOCISI1QEUebMsXmcMy6PW5/bSM3BJr/jiEgMUBFEoe/OHU9Dcxu/eWaj31FEJAaoCKLQ6PxMPnPKMO5fWsnG3fV+xxGRKKciiFLXzRpLenKAHy9c63cUEYlyKoIolZORzLXnjeGF9dW8uKHa7zgiEsVUBFHsc6cNZ/jgdG5+fA2tbe1+xxGRKKUiiGIpiQFunDOejXsO8tc3tvsdR0SilIogyl0wsYBTRuTw68UbqGvUF92LyLFTEUQ5M+N78yawv6GZ257b5HccEYlCKoIYMGloNh+fUszdr2ylsqbB7zgiEmVUBDHimxeMIzFg/PRJHU4qIsdGRRAjCrJSueqsUTz5dhVLt+zzO46IRJGwFoGZzTaz9Wa2ycxu6GL+181sjZmtNLNnzWx4OPPEui+dMZLC7FT+6/E1tLfrOwtEpGfCVgRmFgBuA+YAE4BLzWxCp8WWA9Occ2XA34GfhytPPEhLDvDt2Sewamctjyzf6XccEYkS4VwjmAFscs5VOOeagQeAi0IXcM4975zr2Lv5GlAcxjxxYf6JRZxYnM3PF62jobnV7zgiEgXCWQRDgdBPOe3wpnXnCuDJrmaY2ZVmVm5m5dXVGk7hSBISgoeT7q5r4o6XKvyOIyJRoF/sLDazzwLTgF90Nd85d4dzbppzblpeXl5kw0WhaaU5zC0r5PYXK6iqbfQ7joj0c+Esgp1AScjlYm/a+5jZLOC7wHznnL5ppY/cMPsE2pzj54vW+R1FRPq5cBbBG8AYMxthZsnAAuCx0AXM7GTgdoIlsCeMWeJOSU46X5w5goeX7WTljgN+xxGRfixsReCcawWuBhYBa4GHnHOrzexHZjbfW+wXwADgb2a2wswe6+bm5Dj8+zmjyB2QzM2Pr8U5HU4qIl1LDOeNO+cWAgs7Tft+yPlZ4bz/eJeZmsTXzx/Hdx5ZxVNvVzFncqHfkUSkH+oXO4slfD41vYQThmTy0yfX0dTa5nccEemHVAQxLpBgfHfueCr3NXDvq1v9jiMi/ZCKIA6cMSaPc0/I53fPbmLHfo1OKiLvpyKIE9+fFxzd48t/fpPDzdpEJCLvURHEidLcDH576Ums2VXHDQ+v1FFEIvIuFUEcOfeEAr7x4XE8uuId7lyi4SdEJEhFEGe+evYo5k4u5P89uY4XN2jcJhFREcQdM+MXl5QxtiCTr92/jK17D/kdSUR8piKIQ+nJidz5+WkkJBhX/rmcg00arloknqkI4lRJTjq3fXoKm/Yc5D8eWqFvNBOJYyqCODZzdC7fuXA8i1bv5vfPb/I7joj4REUQ5644fQQXnzyUWxZvYPGa3X7HEREfqAjinJnxk4snU1aczfUPrmDTnnq/I4lIhKkIhNSkAH/87FRSkxL40n1vUnu4xe9IIhJBKgIBoGhgGn/47FS272vg2geW06adxyJxQ0Ug75pemsNN8yfywvpqfvX0er/jiEiEhPWLaST6fPbU4ax+p47/fmEzE4qymFdW5HckEQkzrRHIB/xw/kSmDh/EN/+2kjXv1PkdR0TCTEUgH5CcmMAfPjuF7LQkrvxzOfsONfsdSUTCSEUgXcrPTOX2z01lT30TV9+/jNa2dr8jiUiYqAikWyeWDOQnH5vMq5tr+MnCdX7HEZEw0c5iOaJPTC1m9Tu13PXKFiYWZfHxqcV+RxKRPqYikKP6zoXjWbernhsfXkV9YwufP62UhATzO5aI9BFtGpKjSgoEdx7PHD2Ym/65hs/d9TrvHDjsdywR6SMqAumRgenJ3HX5dH568WSWVx7ggt+8xMPLdui7j0VigIpAeszMuHTGMJ689gzGFWTy9Yfe4it/WUbNwSa/o4lIL6gI5JgNH5zBg18+jRvmnMBz6/ZwwW+W8IyGsBaJWioCOS6BBOOqs0bx6NUzyR2QzL/dV863/76S+kaNXCoSbVQE0ivjC7N49OqZfPXsUfztze3M+e0SXq+o8TuWiByDsBaBmc02s/VmtsnMbuhi/plmtszMWs3sE+HMIuGTkhjgW7NP4KEvn0YgwVhw52v8+Ik1NLa0+R1NRHogbEVgZgHgNmAOMAG41MwmdFqsErgcuD9cOSRyppXmsPCaM/j0jGHcuWQL83//Mm/vrPU7logcRTjXCGYAm5xzFc65ZuAB4KLQBZxzW51zKwENZBMjMlIS+fHHJnP3F6ZzoKGFj972Cr9/bqPGKhLpx8JZBEOB7SGXd3jTjpmZXWlm5WZWXl1d3SfhJLzOGZfP09efyexJQ/jl0xv4xB//RUX1Qb9jiUgXomJnsXPuDufcNOfctLy8PL/jSA8NTE/m95+ewq2XnsyWvYeY/ZslXHlfOY+u2MnBpla/44mIJ5xjDe0ESkIuF3vTJM7MP7GIGaU53P7SZp5cVcXTa3aTnJjA2WPzmFtWyHnjCxiQomGvRPwSzv++N4AxZjaCYAEsAD4dxvuTfmxIdio/+MhEvjd3Assq9/PEql0sXLWLp9fsJiUxgbPH5XHhZJWCiB8snGPFmNmFwG+AAHCXc+7HZvYjoNw595iZTQceAQYBjUCVc27ikW5z2rRprry8PGyZJXLa2x1vVu7niZW7ePLtXeyua3q3FOaWFXHeCflkqBRE+oSZvemcm9blvGgbNExFEJtCS2Hhql3sqQ+Wwjnj8rmwrFClINJLKgKJKu3tjvJt+1m46v2lcN74fK48cxQnlQz0O6JI1FERSNQKLYVHV+xkf0ML54zL4/rzx1JWrEIQ6SkVgcSEg02t3PvqVu5cUsGBhhZmjc/nulljmTQ02+9oIv2eikBiSn1jC/e8EiyEusZWzp9QwHWzxjCxSIUg0h0VgcSkusYW7n55K396uYL6xlYumFjAdbPGMr4wy+9oIv2OikBiWu3hFu56eQt3vbyF+qZWLpw8hGvPG8u4IZl+RxPpN1QEEhdqG1r408sV3P3KVg41t3Lh5EKuO28MYwpUCCIqAokrBxqauXNJBfe8spWGljbmlRVx7XmjGZ2vQpD4FfNF0NLSwo4dO2hsbPQpVeSkpqZSXFxMUlKS31H6vX2HgoVw76tbOdzSxuyJQyjNzWBASiIZyQEyUhKD573TgJREBqQmMiA5kYyUAImBqBiTUaRHYr4ItmzZQmZmJoMHD8bMfEoWfs45ampqqK+vZ8SIEX7HiRo1B5u4Y0kF/3hzBwcaWmht79lzPiUx4X1FkZmaSN6AFPIyU8jPSqEgM5X8rBTyM1PJz0xhYHpSTD//JLodqQhi4jP7jY2NlJaWxvw/oZkxePBg9J0Mx2bwgBRunDOeG+eMxzlHU2s7h5paOdTUxsGmVg41twZ/eqeDTW0h59+bVt/YwtqqOl7c0NTlMNrJiQnkDUjxyiGFgqxgQeRnppKXlUJRdhpj8geQkBDbz1OJPjFRBEDMl0CHePk9w8XMSE0KkJoUYPCA47+dhuZW9tQ1sae+id11jeypb2JPfSPVdU3srm+kovoQr1Xso/Zwy/uuNyQrlQsmFnDBpCHMKM3R5ifpF2KmCEQiKT05kdLcREpzM464XGNLG9VeSVRUH+KZtbt5sHw79/5rG4PSk5g1voDZk4Ywc3QuqUmBCKXvmZqDTSyvPMDy7ftZXnmAzdUHOX10HpfOKGHq8EF6UxJDVAR9oKamhvPOOw+AqqoqAoEAHd+ktnTpUpKTk7u9bnl5Offddx+33nprRLJKZKUmBSjJSackJ52pw3O4ZFoJDc2tvLShmqferuKpt6v425s7yEgOcM4J+cyeNISzx+VH/DsZWtraWbernuXb97Ns236Wbz/AtpoGABITjPGFWUwrzWHR6ir+sWwHY/IHsGDGMC4+eSiDMrp/fkt0iImdxWvXrmX8+PE+JXq/m266iQEDBvCNb3zj3Wmtra0kJvbdP3Z/+n2ld5pb23l1814Wra7i6dW7qTnUTHJiAmeOyeWCiUOYNb4gLC+0u+saWV4ZfKe/vPIAK3ceoLGlHYC8zBSmDBvIlGGDOHnYICYPzSYtObi2cqiplSdW7uL+pZWs2H6A5MQE5kwawoLpwzh1ZI7WEvqxmN9ZHOqH/1zNmnfq+vQ2JxRl8YOPHPH7cj7g8ssvJzU1leXLlzNz5kwWLFjAtddeS2NjI2lpadx9992MGzeOF154gV/+8pc8/vjj3HTTTVRWVlJRUUFlZSXXXXcd11xzTZ/+LtK/JCcmcPa4fM4el8/NH3WUb93HU6urWPR2Fc+s3UMgwThlRA6zJw3hnHH5pCcHaGlztLS1e6fg+ea2dlpa33+5NXReWzsHG1tZubOWFZUH2HngMABJAWNiUTafnjGck4cNZMrwQRRlp3b7gp6Rksgnp5fwyeklrKuq44Gl23l42Q4eXfEOI3Mz+NT0Ej4+tZjcASmRfBill2KuCPqTHTt28OqrrxIIBKirq2PJkiUkJibyzDPP8J3vfId//OMfH7jOunXreP7556mvr2fcuHF85Stf0WcG4kQgwThl5GBOGTmY78+bwKqdtcHNR6ur+P6jq4HVvb6PouxUTh4+iC/MLGXK8EFMKMw67n0TJwzJ4qb5E7lhzgksXLWLvy6t5KdPruOXT6/nwxOGsGBGCTNH5Yb9KCnnHIeag0d11Te2Ut/YQl1jK/WNrdQdfm9aQ3Mb2WlJ7x7NVZAVPPx3cEZy3O+0j7kiONZ37uF0ySWXEAgE/8lqa2u57LLL2LhxI2ZGS0tLl9eZO3cuKSkppKSkkJ+fz+7duykuLo5kbOkHzIyy4oGUFQ/kW7NPYNOeel7dXINzkBRIIClgJCcmeOcTSAwYyYGEd+clBRLenZ+YEFw2NTFAdnrfv6lITQpw8ZRiLp5SzKY99fx16Xb+sWwHT6zaRUlOGgumD+OSqcXkZ6V2ef2OF/Lawy3UNrRQ19hC7eEW6g57P70X9LrDwXkdL/IdL/wHm1ppO8pnQxITjLTkAAebWum8NTzBgocYF3ifCSnISiEvM/V9hwAXZKWSOyB2CyPmiqA/ych474iS733ve5xzzjk88sgjbN26lbPPPrvL66SkvLdKHQgEaG394PHqEn9G52dGxRAZo/Mz+d68CXzzgnEsWl3FA0u384tF67ll8UL/TkkAAAirSURBVAbOHJNLenKi9+L+3ot9XeORX8jNIDMlkay0JDJTk8hKTWTowDSyUjPJTE0kMzXp3Z9Zae9dzgqZl5YUCL4Ba2tn78Em9tSFHPbr/dxd18juukZW7qil5lDTBwoDgqVxvMyMgswUSnLSGT44nWE56QwbnMGwnHSG56T7+oFEFUGE1NbWMnToUADuuecef8OIhFlqUoCLThrKRScNZcveQzzwRiWL3q4ikGBkpyWRk5HMiNwMslKTyE4LvoBnpyWFXH7v54CURAJ9tHkpKZBAYXYahdlpR1yuta2dvQeb2VPfyO467zMi9U1HXfM4krZ2R1VtI9v2NfD8+mqq65veNz8zJbFTSaR7JZFB0cDUsK6NqAgi5Fvf+haXXXYZN998M3PnzvU7jkjEjMjNePeT3dEiMZDAkOxUhmR3vTmrLzQ0t7J932Eq9zWwreYQ2/c1ULmvgfW763l27R6a29rfXTaQYAwdmMZ/fHgsF500tM+z6PDRKBRvv69IvGlvd1TVNVLplUNlTfDnp6aXMHN07nHdZlwdPioiEu0SEoyigWkUDUzj1JGDw39/Yb8HERHp12KmCKJtE9fxipffU0QiJyaKIDU1lZqamph/kez4PoLU1PDtwBKR+BMT+wiKi4vZsWNHXIzT3/ENZSIifSUmiiApKUnf2CUicpxiYtOQiIgcPxWBiEicUxGIiMS5qPtksZlVA9uO8+q5wN4+jNPXlK93lK/3+ntG5Tt+w51zeV3NiLoi6A0zK+/uI9b9gfL1jvL1Xn/PqHzhoU1DIiJxTkUgIhLn4q0I7vA7wFEoX+8oX+/194zKFwZxtY9AREQ+KN7WCEREpBMVgYhInIvJIjCz2Wa23sw2mdkNXcxPMbMHvfmvm1lpBLOVmNnzZrbGzFab2bVdLHO2mdWa2Qrv9P1I5fPuf6uZrfLuu7yL+WZmt3qP30ozmxLBbONCHpcVZlZnZtd1Wibij5+Z3WVme8zs7ZBpOWa22Mw2ej8HdXPdy7xlNprZZRHK9gszW+f9/R4xs4HdXPeIz4UwZ7zJzHaG/B0v7Oa6R/x/D2O+B0OybTWzFd1cNyKPYa8452LqBASAzcBIIBl4C5jQaZmvAn/0zi8AHoxgvkJginc+E9jQRb6zgcd9fAy3ArlHmH8h8CRgwKnA6z7+rasIflDG18cPOBOYArwdMu3nwA3e+RuAn3VxvRygwvs5yDs/KALZPgwkeud/1lW2njwXwpzxJuAbPXgOHPH/PVz5Os3/FfB9Px/D3pxicY1gBrDJOVfhnGsGHgAu6rTMRcC93vm/A+eZmUUinHNul3NumXe+HlgL9P23UYfXRcB9Lug1YKCZFfqQ4zxgs3PueD9p3meccy8B+zpNDn2e3Qt8tIurXgAsds7tc87tBxYDs8OdzTn3tHOu1bv4GuDr2ObdPH490ZP/9147Uj7vteOTwF/7+n4jJRaLYCiwPeTyDj74QvvuMt4/Qy0Q/i8G7cTbJHUy8HoXs08zs7fM7EkzmxjRYOCAp83sTTO7sov5PXmMI2EB3f/z+fn4dShwzu3yzlcBBV0s0x8eyy8SXMPrytGeC+F2tbf56q5uNq31h8fvDGC3c25jN/P9fgyPKhaLICqY2QDgH8B1zrm6TrOXEdzccSLwO+D/IhzvdOfcFGAO8O9mdmaE7/+ozCwZmA/8rYvZfj9+H+CC2wj63bHaZvZdoBX4324W8fO58AdgFHASsIvg5pf+6FKOvDbQ7/+fYrEIdgIlIZeLvWldLmNmiUA2UBORdMH7TCJYAv/rnHu483znXJ1z7qB3fiGQZGa5kcrnnNvp/dwDPEJw9TtUTx7jcJsDLHPO7e48w+/HL8Tujk1m3s89XSzj22NpZpcD84DPeEX1AT14LoSNc263c67NOdcO3NnNffv6XPRePy4GHuxuGT8fw56KxSJ4AxhjZiO8d40LgMc6LfMY0HF0xieA57r7R+hr3vbE/wHWOudu6WaZIR37LMxsBsG/U0SKyswyzCyz4zzBnYpvd1rsMeDz3tFDpwK1IZtAIqXbd2F+Pn6dhD7PLgMe7WKZRcCHzWyQt+njw960sDKz2cC3gPnOuYZulunJcyGcGUP3O32sm/vuyf97OM0C1jnndnQ10+/HsMf83lsdjhPBo1o2EDya4LvetB8RfNIDpBLcpLAJWAqMjGC20wluIlgJrPBOFwJXAVd5y1wNrCZ4BMRrwIcimG+kd79veRk6Hr/QfAbc5j2+q4BpEf77ZhB8Yc8Omebr40ewlHYBLQS3U19BcL/Ts8BG4Bkgx1t2GvCnkOt+0XsubgK+EKFsmwhuW+94DnYcRVcELDzScyGCj9+fvefXSoIv7oWdM3qXP/D/Hol83vR7Op53Icv68hj25qQhJkRE4lwsbhoSEZFjoCIQEYlzKgIRkTinIhARiXMqAhGROKciEOnEzNo6jXDaZyNamllp6AiWIv1Bot8BRPqhw865k/wOIRIpWiMQ6SFvXPmfe2PLLzWz0d70UjN7zhsc7VkzG+ZNL/DG+n/LO33Iu6mAmd1pwe+jeNrM0nz7pURQEYh0Ja3TpqFPhcyrdc5NBn4P/Mab9jvgXudcGcHB2271pt8KvOiCg99NIfjJUoAxwG3OuYnAAeDjYf59RI5InywW6cTMDjrnBnQxfStwrnOuwhs4sMo5N9jM9hIc/qDFm77LOZdrZtVAsXOuKeQ2Sgl+/8AY7/K3gSTn3M3h/81EuqY1ApFj47o5fyyaQs63oX114jMVgcix+VTIz395518lOOolwGeAJd75Z4GvAJhZwMyyIxVS5FjonYjIB6V1+iLyp5xzHYeQDjKzlQTf1V/qTfsacLeZfROoBr7gTb8WuMPMriD4zv8rBEewFOlXtI9ApIe8fQTTnHN7/c4i0pe0aUhEJM5pjUBEJM5pjUBEJM6pCERE4pyKQEQkzqkIRETinIpARCTO/X/o7BAVEQzjxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model_CNN_LSTM_hist.history['loss'])\n",
    "\n",
    "plt.title('Model Loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['Train'], loc='lower left')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VwaDpGlgUAv8"
   },
   "source": [
    "Let's save our model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8F65fjrOUAv8"
   },
   "outputs": [],
   "source": [
    "model_CNN_LSTM.save('models/LSTM-CNN-Model.h5')  # save model\n",
    "del model_CNN_LSTM  # delete existing model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "bY795DW7aSSW"
   },
   "outputs": [],
   "source": [
    "# returns a compiled model, identical to the previous one\n",
    "model_CNN_LSTM = load_model('models/LSTM-CNN-Model.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QbpzmiLmrJhn",
    "outputId": "8efe9a56-8684-4063-c701-c288100e456a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44/44 [==============================] - 5s 83ms/step - loss: 0.4531 - binary_accuracy: 0.9015\n",
      "\n",
      "Model Accuracy: 90.15%\n"
     ]
    }
   ],
   "source": [
    "model_CNN_LSTM_score = model_CNN_LSTM.evaluate(x_val, np.asarray(y_val), batch_size=n_batch)\n",
    "print(\"\\nModel Accuracy:\", str(round(model_CNN_LSTM_score[1] * 100, 2)) + '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q-J5xACQUAv9"
   },
   "source": [
    "<h3 align=\"center\">Testing</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wtQJDhyRUAv-"
   },
   "source": [
    "Let's try to check the test data with all but one of the test examples to see if the model can predict it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RqRsML2oUAv-",
    "outputId": "833f4a23-9721-4817-807e-8cb60a462b0a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of test data (except 1): 7499\n"
     ]
    }
   ],
   "source": [
    "print(\"Length of test data (except 1):\", len(x_test[:-1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DUkLTUycUAv-",
    "outputId": "c22593b6-148d-4896-fcab-d57fe9da14f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "235/235 [==============================] - 6s 21ms/step - loss: 0.4305 - binary_accuracy: 0.9059\n",
      "[0.4304787218570709, 0.9058541059494019]\n"
     ]
    }
   ],
   "source": [
    "score_test = model_CNN_LSTM.evaluate(x_test[:-1], np.asarray(y_test[:-1]))\n",
    "print(score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "s244cTtsUAv_",
    "outputId": "6f98d594-32d9-4080-9313-1f80e29e1c4b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1/1 [==============================] - 0s 43ms/step - loss: 0.0015 - binary_accuracy: 1.0000\n",
      "[0.00153376127127558, 1.0]\n"
     ]
    }
   ],
   "source": [
    "score_test_one = model_CNN_LSTM.evaluate(x_test[-1:], np.asarray(y_test[-1:]))\n",
    "print(score_test_one)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nONPOUPSUAv_",
    "outputId": "5a9d4e0d-ccbb-4e38-de94-1e8cbfd760b2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0]\n",
      " [1]\n",
      " [1]\n",
      " ...\n",
      " [0]\n",
      " [0]\n",
      " [0]]\n"
     ]
    }
   ],
   "source": [
    "predict_test = (model_CNN_LSTM.predict(x_test[:-1]) > 0.5).astype(\"int32\")\n",
    "print(predict_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-731qOryUAwA"
   },
   "outputs": [],
   "source": [
    "predict_test_one = (model_CNN_LSTM.predict(x_test[-1:]) > 0.5).astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hE3ZtOhjUAwA",
    "outputId": "e1d3114a-c4a0-4da4-f9f8-4358cede44ba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted value: 1 \n",
      "Actual value: 1\n"
     ]
    }
   ],
   "source": [
    "print(\"Predicted value:\", predict_test_one[0][0], \"\\nActual value:\", y_test[-1:][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aeY1BnFPUAwB"
   },
   "source": [
    "Prediction works great, now let's visualize the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xgYEmZemUAwB"
   },
   "source": [
    "<h2 align=\"center\">Visualization of Classification Report</h2>\n",
    "\n",
    "We will need our method from previous part to visualize our data, so we will use them now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "6busKszmUAwB"
   },
   "outputs": [],
   "source": [
    "def plot_classification_report(classification_report, title='Classification Report', cmap='RdBu'):\n",
    "    \n",
    "    lines = classification_report.split('\\n')\n",
    "\n",
    "    classes, plotMat, support, class_names = [], [], [], []\n",
    "    \n",
    "    for line in lines[2 : (len(lines) - 5)]:\n",
    "        t = line.strip().split()\n",
    "        if len(t) < 2: continue\n",
    "        classes.append(t[0])\n",
    "        v = [float(x) for x in t[1: len(t) - 1]]\n",
    "        support.append(int(t[-1]))\n",
    "        class_names.append(t[0])\n",
    "        plotMat.append(v)\n",
    "\n",
    "    xlabel = 'Metrics'\n",
    "    ylabel = 'Classes'\n",
    "    \n",
    "    xticklabels = ['Precision', 'Recall', 'F1-score']\n",
    "    yticklabels = ['{0} ({1})'.format(class_names[idx], sup) for idx, sup  in enumerate(support)]\n",
    "    \n",
    "    figure_width = 25\n",
    "    figure_height = len(class_names) + 7\n",
    "    correct_orientation = False\n",
    "    \n",
    "    heatmap(np.array(plotMat), title, xlabel, ylabel, xticklabels, yticklabels, figure_width, figure_height, correct_orientation, cmap=cmap)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "-tHtC2VQUAwC"
   },
   "outputs": [],
   "source": [
    "def heatmap(AUC, title, xlabel, ylabel, xticklabels, yticklabels, figure_width=40, figure_height=20, correct_orientation=False, cmap='RdBu'):\n",
    "    fig, ax = plt.subplots()\n",
    "    c = ax.pcolor(AUC, edgecolors='k', linestyle='dashed', linewidths=0.2, cmap=cmap)\n",
    "\n",
    "    # put the major ticks at the middle of each cell\n",
    "    ax.set_yticks(np.arange(AUC.shape[0]) + 0.5, minor=False)\n",
    "    ax.set_xticks(np.arange(AUC.shape[1]) + 0.5, minor=False)\n",
    "\n",
    "    # set tick labels\n",
    "    ax.set_xticklabels(xticklabels, minor=False)\n",
    "    ax.set_yticklabels(yticklabels, minor=False)\n",
    "\n",
    "    # set title and x/y labels\n",
    "    plt.title(title)\n",
    "    plt.xlabel(xlabel)\n",
    "    plt.ylabel(ylabel)      \n",
    "\n",
    "    # remove last blank column\n",
    "    plt.xlim( (0, AUC.shape[1]) )\n",
    "\n",
    "    # turn off all the ticks\n",
    "    ax = plt.gca()    \n",
    "    for t in ax.xaxis.get_major_ticks():\n",
    "        t.tick1line.set_visible = False\n",
    "        t.tick2line.set_visible = False\n",
    "    for t in ax.yaxis.get_major_ticks():\n",
    "        t.tick1line.set_visible = False\n",
    "        t.tick2line.set_visible = False\n",
    "\n",
    "    # add color bar\n",
    "    plt.colorbar(c)\n",
    "\n",
    "    # add text in each cell \n",
    "    show_val(c)\n",
    "\n",
    "    # proper orientation (origin at the top left instead of bottom left)\n",
    "    if correct_orientation:\n",
    "        ax.invert_yaxis()\n",
    "        ax.xaxis.tick_top()       \n",
    "\n",
    "    # resize \n",
    "    fig = plt.gcf()\n",
    "    fig.set_size_inches(cm_to_inch(figure_width, figure_height))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "vZnxizJCUAwC"
   },
   "outputs": [],
   "source": [
    "def show_val(pc, fmt=\"%.2f\", **kw):\n",
    "    pc.update_scalarmappable()\n",
    "    ax = pc.axes\n",
    "    for p, color, value in zip(pc.get_paths(), pc.get_facecolors(), pc.get_array()):\n",
    "        x, y = p.vertices[:-2, :].mean(0)\n",
    "        if np.all(color[:3] > 0.5):\n",
    "            color = (0.0, 0.0, 0.0)\n",
    "        else:\n",
    "            color = (1.0, 1.0, 1.0)\n",
    "        ax.text(x, y, fmt % value, ha=\"center\", va=\"center\", color=color, **kw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "RX_DtRg-UAwD"
   },
   "outputs": [],
   "source": [
    "def cm_to_inch(*dim):\n",
    "    inch = 2.54\n",
    "    return tuple(i/inch for i in dim[0]) if type(dim[0]) == tuple else tuple(i/inch for i in dim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LWLefyJHUAwD"
   },
   "source": [
    "<h2 align=\"center\">Comparison of Models</h2> \n",
    "\n",
    "A useful tool when predicting the probability of a binary outcome is the Receiver Operating Characteristic curve, or ROC curve.\n",
    "\n",
    "It is a plot of the false positive rate (x-axis) versus the true positive rate (y-axis) for a number of different candidate threshold values between 0.0 and 1.0. Put another way, it plots the false alarm rate versus the hit rate.\n",
    "\n",
    "The true positive rate is calculated as the number of true positives divided by the sum of the number of true positives and the number of false negatives. It describes how good the model is at predicting the positive class when the actual outcome is positive.\n",
    "\n",
    "The false positive rate is calculated as the number of false positives divided by the sum of the number of false positives and the number of true negatives.\n",
    "\n",
    "It is also called the false alarm rate as it summarizes how often a positive class is predicted when the actual outcome is negative.\n",
    "\n",
    "To make this clear:\n",
    "* Smaller values on the x-axis of the plot indicate lower false positives and higher true negatives.\n",
    "* Larger values on the y-axis of the plot indicate higher true positives and lower false negatives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 445
    },
    "id": "EZq-3kksUAwE",
    "outputId": "55034629-13b2-4524-b306-63b9c75e4f46"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report LSTM Stochastic Gradient Descent Model:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.84      0.90      0.87      3745\n",
      "           1       0.89      0.83      0.86      3755\n",
      "\n",
      "    accuracy                           0.87      7500\n",
      "   macro avg       0.87      0.87      0.87      7500\n",
      "weighted avg       0.87      0.87      0.87      7500\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkcAAAD+CAYAAAAwGaFjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZyd4/3/8dd7ZrIJWQgRWSS2EltqrVK1i53SonZapaXfWlq0SqpVWkq1VH+0xNLaq4IQaq2tEltIbBFLNrJIEBJJZj6/P+5r4p7JzJkTmTMzJ/N+Ph7nMee+7uu+r+s+555zPue6rvu6FRGYmZmZWaaitStgZmZm1pY4ODIzMzPLcXBkZmZmluPgyMzMzCzHwZGZmZlZjoMjMzMzsxwHR7bck7SDpMktVNZcSWu1RFlLq/7rIGmcpB1asUrLNUlHS3qihPu/T9JRueXfSJop6X1JA9K5WFmCctvsOW7WXBwctTOS3pG0SyPrfi7p7fThN1nSLSl9XEqbK6la0vzc8s/Tl0BIurTe/vZL6cML1KfBMtO6RyV9r5kOvdk1VL+IWDEiJn6Jfa0k6ZL0/nwq6T1Jt0vauvlqXFdEbBgRjy7rfiQNk3RjE3nekTRP0ieS5kh6StIJktrkZ1CxgY2k3SU9no5rhqTHJO3bEnWMiD0i4rpUjwHAacDgiFg9It5L52L1spTRnOd4EWXVniNzU4A3XNKKzV3OUtSlwc9Jax/a5AeTtbz0C/QIYJeIWBHYAngIFn+JrpjS/wucVLscEb9Nu3gL+I6kqtxujwLe+DJltieSOgEPAxsDewPdgA2Am4E9GtmmqqH0Nm6fiFgJWBO4EDgD+HvrVunLk3QQcBtwPdAP6A2cA+zTCtUZAMyKiOmtUHZz2id9FgwBvgqc1ZKFl+n/lZVCRPjRjh7AO2TBSP30y4E/FrH9o8D36qUdDTwB3A/sldJWBt4HLgKGN7KvRssEzgeqgfnAXODylP51YDTwUfr79dw2KwPXAlOB2cC/U/oOwGSyX9bTgWnAMbnt9gJeAD4GJgHDcus6AzcCs4A5qczeBeoXwDrpeRfgD8C7qb5PAF0aONbvpTp1beK1D+BHwJvA2yntslTnj4HngG/k8ncBhqfXYjzwU2ByQ+cC2Q+lM8mC3FnArcDKad3AVPZRwHvATOAXad1QYAGwML0OLxV73gFbATXARmm5E3BxKuMD4K+1rxfQC7gnvQcfkgXpFWldf+BfwIxU98tzZRwLvJpeg1HAmvVezxPS6zkHuAIQWWA6P72/c4E5DRyPUj1/WuD9Ohp4Irdc6L3aChiT1n0AXFLo/Mv/LwK7APPSazk3vee171lVE/8bPdPrOiOl3wP0a+J/MH+OdycLDmeQnedn596Xo8nO+YvTvt8G9ij2swn4PXBvbvlrwFPpdXgJ2KHe59IFwLPpNbyLdP6m9fsC49K2jwIb1Cv3DGAs8DlwU3ot56Xj/llzff76UT6PVq+AHy38hjceHB1O9qXzU7IWnMpGtn+UxoOj7wK3pLQfAv8P+A2NB0cFy6xfVvqAn03W2lQFHJqWV0nr7wVuSR/4HYBvpvQdgEXAeSl9T+AzoGdu/cZkAcImZF9O+6d1PwDuBlYAKoHNgW4FXov8F8cVKU/ftO3XgU4NvA43N/YaNbDvB9Pr0CX3Gq6SXo/TyALSzmndhWRBxMpkAcQrNB4c/R/wDFkLSKf03t2U1g1MZV9NFnBtSvYlskFaPwy48Uued+8BJ6bnlwIjUn1XSq/7BWndBWTBUof0+AZZgFJJ9kV5KdCVLJjYLm2zHzCBLNipIvvifqre63kP0IOs5WUGMDR/Thc4nvXT9oMK5Kmzjybeq6eBI9LzFYGvLc35R/oBkCur9j2rDY4a+99YBTgw7X8lspawfzfx/54/x68nC0RWSmW+ARyXO/6FwPdT3U8kC87U1DlCdh6+DFyWlvuSBYh7kv2f7pqWV83VcwqwUToP7iCdk8B6wKdpmw7Az9J50TFX7otk/yNd6tfFj/b5aPUK+NHCb3iBf3rgMOA/6YNkFnBGA3ka+rA8mtQqQhZYdCf7ot2WAsFRU2XWL4ssKHq23vZPp/L7kP3a69lAGTuQ/QqsyqVNJ30BNZD/j8Cl6fmxZL9WNynytQhgnfQBPg/YtIj35D/AhbnlIWS/cD8GXq+3752a2Nfs2jKBiaQv+7R8PI0HR68CO+fW9SH7Yqviiy/afrn1zwKHpOfD+PLB0TPAL8gCnU+BtXPrtuGLFrLzyL6E16m3/TZkQU1VA/u+j/RFnZYryILiNXOv53a59bcCZ+bP6QLHs23avnOBPE3tI/9ePQ78CuhVL09R5x8FgiMK/G80sM8hwOwiz/FKslbDwbl1PwAezR3/hNy6FdK2qxc4R+YCn6R8DwE90rozgBvq5R8FHJWrZ/5/aHCqWyXwS+DWeufBFFLLUyr32GLOVz/az8NjjmyxiPhHROxC9kv6BODXknZfiu3nkf1CPZusNefJZi5zDbKm+7x3yX5V9gc+jIjZjWw7KyIW5ZY/I/uFjqStJT2SBtR+lOrRK+W7gexD+GZJUyX9XlKHpo4rbd+ZrJuqKbPIvsAAiIgXI6IH8C2yVpy8SfkFSadLelXSR5LmkAWmtXVfo17++q9d3prAnWmw9ByyYKmarAux1vu554tfv2XUl6z1cFWyL8/ncnW4P6VD1j07AXhA0kRJZ6b0/sC79d7b/DFdltvfh2RBWN9mOKZZ6W+fgrlymnivjiNr4XhN0mhJe6f0L3v+5TX6vyFpBUn/T9K7kj4mC9J6FHmVWy+ylpj8eVX7/1hr8esbEZ+lp4Ve4/0jG5e2A1nrXO3rsybw7dr3Mr1+21H39a9/rndI29f53IiImpS3byPbmjk4siVFxMKIuI2sD36jpdz8erIug4JXLxVZZtTLNpXsQzJvANmvwEnAypJ6LGV9Af5J1p3TPyK6k3XfKFevX0XEYLJusb2BIxupX95MsrEaaxdR/kPAbpK6FpF3cZmSvkHWRfAdslaBHmRjm5SyTCP7Yqw1oMB+J5GNB+mRe3SOiClLU6elIWlLsi+oJ8her3nAhrnyu0c2OJeI+CQiTouItcjGj5wqaedU7wGNDKSdBPyg3jF1iYinmuGYXk/7P7DIYy34XkXEmxFxKLAa8Dvgdkldmzj/ilXof+M04CvA1hHRDdi+tsrpb1Pn+ELq/k/W/j8uk4h4jGzs1MUpaRJZy1H+vewaERfmNqt/ri9MdazzuSFJKW++nvWP80ud07b8cHDUPnWQ1Dn3qEqXLu+VLimvkLQHsCHwv6Xc92Nkfft/bipjEWV+AOTnUxkJrCfpu6nOB5M1n98TEdPIulH+IqmnpA6Stqc4K5H9sp4vaSuysVO1ddxR0sbpl/THZB+4NY3Ub7H06/Qa4BJJa0iqlLRNujKtvuvJApk7JW2U8nYmG4fVVL0XkbqVJJ1DdqVbrVuBs9Lr0Q84ucC+/gqcL2nNdNyrStqvifJrfQAMLPayfEndUsvIzWTdcS+n1+tq4FJJq6V8fWtbESXtLWmd9MX2EVmrVg1Z99404EJJXdP5vG3umM6StGHaR3dJ316KY+onqWNDKyMigFOBX0o6Jh1ThaTtJF3VwCYF3ytJh0taNb0Oc1JyTRPnX1Ga+N9YiSwonSNpZeDcBl6Hxs7xarJz7Pz0P7xmek2W6odRAX8EdpW0adrnPsqmTqhM7/MO6byudbikwZJWIOuGvT1Xx70k7Zxa3U4jGzNXKEhu9LitfXBw1D6NJPtArH0MI/vg/TnZANk5ZFeKnBgRSzWJXWQeiogPi8jeVJmXAQdJmi3pTxExi+yX82lk3Ro/A/aOiJkp/xFkXx6vkY0p+kmR1f4hcJ6kT8guxb41t2514PZU11fJgr8bGqpfA/s9nWxQ6WiyLp3f0cD/XETMB3Yku6Ls3lTW68CWZC0NjRlF1vX0Blm3wXzqdg/8KqW/DTyQq3dDLiNrPXsgvQ7PAMXOsXRb+jtL0vMF8t2d9j2JbJzRJcAxufVnkHWdPZO6eP5D1qoBsG5anks2zuwvEfFI+vLbh2wMzHtkVyUeDBARd5K95jen/b1CI1MjNOBhsqub3pc0s6EMEXF7KutYstaJD8jG2N3VQPam3quhwDhJc8nei0NSN3Wh829pNPa/8UeysYIzyd7z++tt19Q5fjLZWLGJZC2A/yT7UbDMImIG2Q+HcyJiEtkA+5+TBZiTyC7kyP8/3UDW2vQ+WZf2j9N+XicbDP9nsuPch2zKgAUFir8AODt14Z3eHMdj5UXZDyAzM7PyJOlRslbIv7V2XWz54JYjMzMzsxwHR2ZmZtbmSRoq6XVJE/TFFav59WtKekjSWGW3vumXW3eUpDfT46gmy3K3mpmZmbVl6aKEN8gu+JlMNpbz0IgYn8tzG9kFOtdJ2onsTghHpIsNxpBd5BJkM9RvXmDqF7ccmZmZWZu3FdmkohPTYPqbyQbp5w0mu5gC4JHc+t2BByOidr6vB8kugmiUgyMzMzNr6/pS9wrPydSdyBOyWwl9Kz0/AFhJ0ipFbluH70BcQMcVVoouPVZtOqNZAR0//pCK6urWroaVucoVOrHyaqtkCw0Nh5CWTGtKc+ynLdWlufbTlurSXPvJ7WPc62/OjIgW+3Kr6NYvWDS/YJ6YN2sc2RQXta6KiIbmDCvkdOBySUeTzfY+hWxOtKXm4KiAqu6rM+joy1q7Glbmvnb3JfDKhNauhpW5/vt8kzMu+VlrV8OWAx36b1ToVkLNr/pzOmxwQMEsC57/2/yIKDTx7RTqzoLej3qzsUfEVFLLkaQVgQMjYo6kKWS3pMlv+2ih+rhbzczMzEpKFZUFH0UYDawraVCauf4QsolrvyhD6pWbqf8svpiQdBTZLZp6SuoJ7JbSGuXgyKzEvkQjutkS5DPJypaWOThKN5c+iSyoeRW4NSLGSTpP0r4p2w7A65LeILtp9vlp2w+BX5MFWKOB85q6i4O71czMzKxkJFHRocHbFC6ViBhJdvurfNo5uee3k91up6Ftr2Epbm3j4MjMzMxKR6KiuK6zNsPBkZmZmZVUkeOK2gwHR2ZmZlYykqio6tDa1VgqDo7MzMyshOSWIzMzM7PF5ODIzMzM7AvNdLVaS3JwZFZiQQNT/5uZtRPCA7LNrJ7AE0Hasqtp6P5aZuXA3WpmZmZmORKVVe5WMzMzMwOyW9+45cjM6nJviDUDj12zcubgyMzMzKyWKqhwt5qZ5XkwtjWHCvlMsjIlUKVbjswsz19qZtaOecyRmZmZWZ7kbjUzq0t4TLYtO7mD1spYRUV5nb8OjszMzKxkJCEHR2ZmZmZfUJmNvXRwZGZmZqUjqKyqaO1aLJXyqq2ZmZmVlezGsyr4KGo/0lBJr0uaIOnMBtYPkPSIpBckjZW0Z0rvKOlaSS9LeknSDk2V5ZYjMzMzKyEt8zxdkiqBK4BdgcnAaEkjImJ8LtvZwK0RcaWkwcBIYCDwfYCI2FjSasB9kraMiJrGynPLkZmZmZWOoKKqouCjCFsBEyJiYkQsAG4G9quXJ4Bu6Xl3YGp6Phh4GCAipgNzgC0KFebgyMzMzEpGyi7lL/QoQl9gUm55ckrLGwYcLmkyWavRySn9JWBfSVWSBgGbA/0LFebgyMzMzEpKFYUfQC9JY3KP479EMYcCwyOiH7AncIOkCuAasmBqDPBH4CmgutCOPObIrMR8N3VrDjU+j6xMSaKyssm2mJkRUairawp1W3v6pbS844ChABHxtKTOQK/UlXZKrj5PAW8UqoxbjsxKzF9p1ix8IlkZa4ar1UYD60oaJKkjcAgwol6e94CdASRtAHQGZkhaQVLXlL4rsKjeQO4luOXIzMzMSmpZZ8iOiEWSTgJGAZXANRExTtJ5wJiIGAGcBlwt6RSynxNHR0SkK9RGSaoha206oqnyHByZmZlZyUgU063WpIgYSTbQOp92Tu75eGDbBrZ7B/jK0pTl4Mis1NwdYs3AY9esfPneamZmVgJldmsqsy+kS/nLiYMjsxIrtxsuWtskfB5ZeRJQUVle56+DIzMzMysdtxyZmZmZ1eUxR2ZWh/CYbFt27lazclXkJJBtioMjMzMzKym3HJmZmZklElQ6ODIzMzPLCNGxyt1qZmZmZhm3HNny5Ovr9uJnew+mokLcOXoS1z4+sc761bt35tff3pSVOldRIfGnUa/zxBszqKoUv9x/Ywb37U5NBBfdM54xb3/YSkdhrW3w7t/kO5edQ0VlJU/+7RZG/e7KOut79l+Do6/7A116dKOisoJ/n/k7XrnvUQZuuSmHXXUBkA3ovGfYH3nx36Na4xCsjRj1yBOcOuxCqqurOfbQA/nZj75XZ/17U6Zx7Ck/Z87Hn1BdXc1vzzqFPXbaHoCxr77OD888j0/mzkWq4Jl7bqZz506tcRjtjnBwZMuJCsFZ+27ICdc8ywcfz+cfP9yWx16bzsTpcxfn+f6O6/DAy9O47X/vsdZqK3L5UVuw50WPcuCWAwD49p/+S8+uHbni6C057C9PEr5kq91RRQWHXnEel+16OLMnv89Zo0cwdsSDTHt1wuI8e559Es/dei+P//VG+mywDieNHM4vBm3HlFde54It9qGmuppuq6/K2S/dx9i7/0NNdXUrHpG1lurqan589m+4759X06/P6nxt74PZe9cdGbze2ovz/PZP/4+D9t6dE448hPFvvMW+R53IhKcfYNGiRRz14zMZftkFbDp4fWbNnkOHDv76aykVgk5l1q1WstpKukbSdEmvNJHvJ5KOTM9/LWmspBclPSBpjZT+05T2oqRXJFVLWjmte0fSy2ndmNx+h0makttuz5S+saThpTru5cVG/XowadZnTJk9j0XVwaix09hhg9518gTQtVP2AbNipypmfPw5AGuttiLPvjUTgNmfLuCT+QvZsG/3Fq2/tQ0DtxrC9AnvMvPtSVQvXMjom+9mk/12q5MnAjp3WxGAzt27MWfqBwAsnDd/cSDUoXMnHF23b8+++DJrDxzAWmv2p2PHDhy87x7c/cDDdfJI4pO5nwLw0Sef0Kf3qgA8+PhTbLzBemw6eH0AVunZg8rKypY9gPZMorKi8KOtKWXoPBy4HLi+sQySqoBjgc1S0kUR8cu07sfAOcAJEXERcFFK3wc4JSLy/TQ7RsTMBoq4NCIuzidExMuS+kkaEBHvfblDW/6t1r0z7380f/HyBx/NY+P+Perk+etDb3LlMVtx6DZr0qVjFT/4+/8AeGPax+ywQW/uHzuN3t07M3iN7vTu3oVXJn/Uosdgra9n397MnjR18fKcydMYtPWQOnnuGXYp//fADex48lF07LoCl+1y2OJ1A7cawpHX/J6V1+zL8CNOdatROzb1/en0W2P1xct9+/Tm2RderpPnnFN+yB6HHc8V1/6TT+fN4/5/Xg3AGxPfRRJ7HnY8Mz6czcH77sHpJx7bovVvz7JuNbccARARjwNNDTTZCXg+IhalbT7OretKw3PnHQrctIzVuxs4ZBn30e4N3WQNRjw/md1/9wgnDR/Nb76zKRL8+7nJfPDRfP75w2356V6Deem92dTUtN9f/TWeArKgLQ/dl6eH385Z/bfh8j2P4ZgbLl18P7p3nn2R8zbajQu33JehZ51IVaf2O0akxi1nTbr5rpEc9e39eGf0Q4y47i8c85OzqKmpoXrRIp4a/QLX//l3PPav6/n3/Q/x8BPPtHZ12w0JOlZVFHy0Na1do22B5/IJks6XNAk4jKzlKL9uBWAocEcuOYAHJD0n6fh6+z8pddNdI6lnLn0M8I2GKiTpeEljJI1Z9Fn7bemY/tF8Vu/eefFy7+5dmJ66zWodsEU/Hnh5GgBjJ82hU1UlPVboSHVNcPHIVzn48ic45cbnWKlzB96d9WmL1t/ahtlTPqBn/zUWL/fo14fZUz6ok2fb4w7muVvvBeDtZ56nqnMnVuy1cp0877/2FvPnfsYaG61X+kpbm7TG6qsxeer7i5enTPuAvquvVifP8Fv+xUH77A7ANpsPYf7nC5j54Wz69unNdltvTq+Ve7JCly7sseM3eOGV8S1a//ZMlF+3WmsHR32AGfmEiPhFRPQH/gGcVC//PsCT9brUtouIzYA9gB9J2j6lXwmsDQwBpgF/yG0zHViDBkTEVRGxRURsUbVC+x0nM27KRwzo1ZU1enahqlLsvkkfHnu17pfatDnz2HrtVQAYtGpXOlZVMPvTBXTuUEHnDll//tfW6cWimqgzkNvaj3dHv8Rq6w5klYH9qOzQgS0P2YexIx6sk+fD96ay/s7bArD6+mvToXMnPpkxi1UG9qMijQtZeUBfVl9/bWa9M7nFj8Hahi033YgJ77zH2+9NZsGChdwy4j723nXHOnn6r9GHh5/IuvdfffMt5s//nFVXWZndvrktr7z2Jp/Nm8eiRYt4/H9j2GDdtRsqxkqkUir4aGtae7j+PKBzI+v+AYwEzs2lHUK9LrWImJL+Tpd0J7AV8HhELP4ml3Q1cE9us86pbGtEdU1w4YhxXHnMVlQI7npuMm9Nn8uJu6zL+Mkf8dhr07nkvtc454CNOGzbQRBw7u1jAVi5ayf+csyW1ARM/3g+Z9/2YisfTStrx70hNdXV3HLSOfx41PVUVFby1DW3Mm38m+zzq1N4d8zLjL37P9xx2m84/OoL2fmU44gIrjv6dADW2W5Ldj/zRKoXLiJqarjph7/k01mzW/mIWk+05xMJqKqq4rJf/5y9Dv8B1dXVHH3wAWz4lXUYdvHlbL7Jhuyz2478/pc/5YQzzuWyv12PJP5+yW+QRM8e3fnJ949km70PQYihO32DPXf+ZmsfUrtR261WThQl7MeWNBC4JyI2amT9CUC/iDg7La8bEW+m5ycD34yIg9Jyd+BtoH9EfJrSugIVEfFJev4gcF5E3C+pT0RMS/lOAbaOiEPS8oHArhFxQqH6r9Bn3VjvmMuW7UWwdm/rEX9A495q7WpYmet70PacdekZrV0NWw506L/RcxGxRUuVt/q6G8aRl9xaMM9F+zZdJ0lDgcuASuBvEXFhvfUDgOuAHinPmRExUlIH4G9kF39VAddHxAWFyipZy5Gkm4AdgF6SJgPnRsTf62W7D7ght3yhpK8ANcC7QD54OQB4oDYwSnoDd6bBm1XAPyPi/rTu95KGkP1ufwf4QW67HYF7v/zRmZm1LLXBrgezYtSOOVqmfUiVwBXArsBkYLSkERGRHzx2NnBrRFwpaTBZ79NA4NtAp4jYOI1dHi/ppoh4p7HyShYcRcShReR5V9Ks2hajiDiwQN7hZNMD5NMmAps2kv+IhtIldQK2AH7SVP3MmoO/1Kw5+CyyctVM3WpbARPS9z6Sbgb2A/LBUQDd0vPuwNRcetc0fVAXYAGQvzp+CW2hE/BMsoHZLWUAWVPbohYs08zMrF2qvX3IMl6t1heYlFuenNLyhgGHp96qkcDJKf124FOyi7PeAy6ud2HXElo9OIqI19OcSC1V3psR8WhLlWdmZtauqajgqFftNDrpUX9qnmIcCgyPiH7AnsANkirIWp2qya5SHwScJmmtQjtq7avVzJZ7ol1fsGbNRO5YszIlVEy32swmBmRPAfrnlvultLzjyOZCJCKeltQZ6AV8F7g/IhYC0yU9STa8ZiKNaPWWIzMzM1t+qbiWo6aMBtaVNEhSR7KpfUbUy/MesHNWpjYgm7ZnRkrfKaV3Bb4GvFaoMAdHZmZmVjLNMeYojRM+CRgFvEp2Vdo4SedJ2jdlOw34vqSXyOZEPDqy+YquAFaUNI4syLo2IsYWKs/damZmZlYykuhYuextMRExkmygdT7tnNzz8WS3Jau/3Vyyy/mL5uDIzMzMSqqizKY0cXBkZmZmJSOgsrxiIwdHZmZmVjoSdGiGbrWW5ODIzMzMSsrdamZWR3u/m7qZtW/uVjOzJQS+L5Ytu5pwkG3lSRJV7lYzMzMz+4JbjszMzMwS4TFHZlafe0OsGXjsmpUrX61mZmZmVo+71cysjjL7TLA2qty6JcxqCZXd+evgyKzUyuxDwcysOWXdauX1OejgyMzMzEqqssx+JDo4Misx4THZtuzkDlorU75azczMzCzH3WpmZmZmdcjdamZmZma13K1mZmZmluNuNTMzM7N6yqzhiKWez1tST0mblKIyZmZmtvypQAUfxZA0VNLrkiZIOrOB9QMkPSLpBUljJe2Z0g+T9GLuUSNpSOH6FlehRyV1k7Qy8DxwtaRLijoaMzMza7cEVFYUfjS5D6kSuALYAxgMHCppcL1sZwO3RsRXgUOAvwBExD8iYkhEDAGOAN6OiBcLlVdsy1H3iPgY+BZwfURsDexS5LZmZmbWXinrViv0KMJWwISImBgRC4Cbgf3q5QmgW3reHZjawH4OTdsWVOyYoypJfYDvAL8ochszw3dTt+ZR4/PIypSWouusgL7ApNzyZGDrenmGAQ9IOhnoSsONOAezZFC1hGJbjs4DRgFvRcRoSWsBbxa5rVm75q80axY+kayMFdGt1kvSmNzj+C9RzKHA8IjoB+wJ3CBpcZwjaWvgs4h4pakdFdVyFBG3AbfllicCBy5trc3MzKz9KaLdaGZEbFFg/RSgf265X0rLOw4YChART0vqDPQCpqf1hwA3FVPfYgdkryfpIUmvpOVNJJ1dzLZmZmbWftVOAlnoUYTRwLqSBknqSBbojKiX5z1gZwBJGwCdgRlpuYJsaFCT442g+G61q4GzgIUAETE2VczMmuLuEGsGHrtm5axShR9NiYhFwElkQ3xeJbsqbZyk8yTtm7KdBnxf0ktkLURHR0TtP872wKTU89WkYgdkrxARz6pudLeoyG3NzGwZldskema1sivSlv0EjoiRwMh6aefkno8Htm1k20eBrxVbVrHB0UxJa5N+A0s6CJhWbCFm7VlzfCiYadmv9jFrNRVldvoWGxz9CLgKWF/SFOBt4PCS1crMzMyWC6K4rrO2pNir1SYCu0jqClRExCelrZaZmZktL8qtBb3Yq9X+T1I34DPgUknPS9qttFUzWz6U10eCtVXuVrOypaxbrdCjrSn2arVj0+1DdgNWIbs3yYUlq5WZmZktF2q71ZblarWWVj6NeiMAABvHSURBVOyYo9qq70l2b7VxKrc2MjMzM2sV5RYyFBscPSfpAWAQcJaklYCa0lXLzMzMlgfZJJCtXYulU2xwdBwwBJgYEZ9JWhk4pnTVMjMzs+WDqFxOW462AV6MiE8lHQ5sBlxWumq1DV2rP2PLj54nd986RNPNg9mEnF/MZluTJuisux81OalbRE3uee0MuapTftPTrgdfTBCa1S2WqIuPCUp3TFNX6MRKW2/a0GHUmfU4soQl6l3ccSy5XGc7NT2gt+5+gtrF/H6abBr3MRVZl6U/pvcWVfPk+Hey+kQgiYqKijrbFnw5Ampq6p6rRFBRUVG3Dk2MRI2aev9/NdnzysrK4usC1FTnOh98THW2a7Fjaikqv0lMiw2OrgQ2lbQp2fTcfwOuB75Zqoq1Bat8+hFHjBnZdEazAk7tsSFjp1e3djWszB2wxpr0/er2rV0Ns6WmCFTvx0FbV+zVaovS/Un2Ay6PiCuAlUpXLTMzM1teqGZRwUdbU2zL0SeSziKbFXv7dHfbDqWrlpmZmS0fAqK8ruEqtuXoYOBz4LiIeB/oB1xUslqZmZnZ8iOi8KONKfb2Ie8Dl+SW3yMbc2RmZmbWuFhOW44kfU3SaElzJS2QVC3po1JXzszMkrb349qsaMvrmKPLgUOA24AtgCOB9UpVKbPlShtsMrbyU38qALPysZy2HAFExASgMiKqI+JaYGjpqmVmZmbLhSALjgo92phiW44+k9QReFHS74FpLEVgZWZmZu1VoOq213VWSLEBzhFAJXAS8CnQHziwVJUyW564N8SaQ3jQkZWzZmg5kjRU0uuSJkg6s4H1AyQ9IukFSWMl7Zlbt4mkpyWNk/SypM6Fyir2arV309N5wK+KOgozMzOzZrhcX1IlcAWwKzAZGC1pRESMz2U7G7g1Iq6UNBgYCQyUVAXcCBwRES9JWgVYWKi8gsGRpJcpcI1ERGxSzEGZtWvldlMha5OavAecWRvWDFekbQVMiIiJAJJuJrtrRz44CqBbet4dmJqe7waMjYiXACJiVlOFNdVy9C2gNzCpXnp/4P2mdm5mZmbtXbNcrdaXurHIZGDrenmGAQ9IOhnoCuyS0tcDQtIoYFXg5oj4faHCmhpzdCnwUUS8m38AH6V1ZmZmZoU1Peaol6QxucfxX6KUQ4HhEdEP2BO4Id3urArYDjgs/T1A0s6FdtRUy1HviHh5iWOMeFnSwC9RcbN2x70h1hzcrWblShHFdKvNjIgtCqyfQtZrVatfSss7jjTNUEQ8nQZd9yJrZXo8ImYCSBoJbAY81FhhTbUc9SiwrksT25qZmZlBTU3hR9NGA+tKGpSmFjoEGFEvz3vAzgCSNgA6AzOAUcDGklZIg7O/Sd2xSktoKjgaI+n79RMlfQ94roiDMTMzs3atiZvOFnElW0QsIptOaBTwKtlVaeMknSdp35TtNOD7kl4CbgKOjsxssvvDjgZeBJ6PiHsLlddUt9pPgDslHcYXwdAWQEfggCaPxszMzNq3aJar1YiIkWSX5+fTzsk9Hw9s28i2N5Jdzl+UgsFRRHwAfF3SjsBGKfneiHi42ALMzMysPSu/e6sVOwnkI8AjJa6LmZmZLY+Wx+DIzMzM7EuJIBYVnJC6zXFwZGZmZiUUUFPd2pVYKg6OzMzMrHQCB0dmZtb8Yhlv3GnWWiKCWOhuNTPL85eaNQefRla23K1mZmZm9oUIwsGRmdXhX/zWDMInkpUtX61mZvX4K83M2jUPyDazJfhu6tYMfBZZ+Ypiby7bZjg4MjMrBw6yrVxFEAsXtHYtloqDIzMzMyud8NVqZmZmZnWEu9XMLM+9IdYc5BPJylUELHK3mpmZmRmQTUPhliMzMzOzWr6U38zMzCzHV6uZmZmZ5ZXfPEcVrV0BMzMzW87VVBd+FEHSUEmvS5og6cwG1g+Q9IikFySNlbRnSh8oaZ6kF9Pjr02V5ZYjMzMzK50Iapbx3mqSKoErgF2BycBoSSMiYnwu29nArRFxpaTBwEhgYFr3VkQMKbY8B0dmZmZWOhFE9TJ3q20FTIiIiQCSbgb2A/LBUQDd0vPuwNQvW5i71czMzKxkIiCqawo+itAXmJRbnpzS8oYBh0uaTNZqdHJu3aDU3faYpG80VZhbjsxKLaK1a2DLgajxeWRlKoKahYuaytVL0pjc8lURcdVSlnQoMDwi/iBpG+AGSRsB04ABETFL0ubAvyVtGBEfN7YjB0dmZmZWUkW0Ds2MiC0KrJ8C9M8t90tpeccBQwEi4mlJnYFeETEd+DylPyfpLWA9YAyNcLeaNarn17Zmi1tvYsvbb6X/kUcssb5T795s8pc/s9n1w9nsxuvp+fVtlli/7SP/od9hh7ZUla0N2u3rG/HKnb9l/F0X8NNj9lxi/YA+q3D/X0/nuVt+xYNX/4y+q/VcvO6Ifb7OuLsuYNxdF3DEPl9vyWpbG/T4ww+y2zabsfNWm/L//nTJEuunTp7E4Qfsxb47bcfe39yGR/8zCoC7br+FfXbcdvFjvd7dGf/y2JaufrsVEdRUVxd8FGE0sK6kQZI6AocAI+rleQ/YGUDSBkBnYIakVdOAbiStBawLTCxUWMmCo6Yuucvl+6Ok7dPzv0t6KV2Cd7ukFVP6pblL8N6QNKfePrpJmizp8lzao6n82u1WS+knSTq2NEe9HKmoYJ2fns4rPzmNMYd8l1V324UVBg2sk2XAsUcz4z8P8/yRR/PaL89h3Z+eXmf9Wj/5MR8+/UzL1bmNas+9ahUV4rIzD2efky5l0wPP5uChW7PBWmvUyfO7U77DP+59is0PPpfzrxrBb04+EICe3bryi+P3Y7sjfsO2h/+aXxy/Hz1WWqE1DqNNCNrxiQRUV1cz7IzT+NtNd3DfE6O551+38+brr9XJ85dLL2KPfQ9gxMNPcOlV1zLsjNMA2O+gg7n7kSe5+5EnufiKq+g3YE0Gb7xJaxxG+5S61Qo9mt5FLAJOAkYBr5JdlTZO0nmS9k3ZTgO+L+kl4Cbg6IgIYHtgrKQXgduBEyLiw0LllSQ4yl1ytwcwGDg0XVZXP98qwNci4vGUdEpEbBoRm5BFgCcBRMQpETEkXYb3Z+Bf9Xb1a+BxlnRY7XapWQ3gGuoO0rIGrDR4MPMmT2b+1KnEokXMePA/rLJ9vTFsEVR27QpAZdcV+XzmzMWrVtl+e+ZPncpnE99uyWpbG7PlRmvx1qTpvD1lBgsXVXPrqP+xzw51r6bdYK01eOTZVwF4dPRr7LPDV4GsxemhZ8Yx++NPmfPJZzz0zDh233bjFj8GaxvGPj+GNQetxYCBg+jYsSN7HXAgD91/b71cYu4nnwAw9+OPWK336kvs5547b2fvAw5qgRpbXjMMyCYiRkbEehGxdkScn9LOiYgR6fn4iNg2xRFDIuKBlH5HRGyY0jaLiLubKqtULUeLL7mLiAVA7SV39R0I3F+7UDs4Stntp7tAgz+VDiWLCEl5Nwd6Aw8UU7GI+Ax4R9JWxR1K+9RptVX5/IMPFi9/Pn0GHVddtU6ed6/+O72H7s7Wd/+bjS69mLf+kDVzV3TpQv8jD+fdv13TonVus9rx3dT7rtaDyR988QNtygezWWPVnnXyjH1jEvvvtDkA+++0Gd1W7MLK3buyxqr1tp0+mzVW7dEyFW+D1I7PI4D3359Gn779Fi+v3mcNPphW90rtH//sLEbccQvbbbo+3/vutznngouW2M+9/77DwVELi8huPFvo0daUKjgq5pI7gG2B5/IJkq4F3gfWJ2slyq9bExgEPJyWK4A/AHX7c75wbepS+6XqfrKMAZq8lM8KW3W3XXn/3pH8b5/9eeWU0/nKsHNAYs3vH8fkm26mZt681q6ilYEzLr2V7Tf/Cs/edC7f2PwrTP7gQ6qXfU4Ua4fu+dftfOvgw3jipdf42z9v4/QfHU9N7ov3xedG02WFFVhvgyU6MqyUAqoXLCr4aGta+2q1PsCMfEJEHJO65f4MHAxcm1t9CHB7RNSO3vohMDIiJjfwq+qwiJgiaSXgDuAI4Pq0bjpZ8LUESccDxwP07tjxyx5X2ft8+gw69e69eLnTaquyYEadt4rV992bV/7vVAA+eeUVKjp2pEOPHnTbcDCr7rgja530I6pWWpGoCWo+X8DU2+9o0WOw1jdl+hz69V558XLf3j2ZOmN2nTzTZszhO6dfAUDXLp04YOfN+WjuPKbOmMP2m3/li21X68njz73eMhW3Nmf11fswbcrkxcvvT5tK7z51x6/d9s/ruebmbNTFV7fcms/nf87sWbNYJbV6u9WolTTPJJAtqlQtR8Vccgcwj2w0eR0p+LmZrNst7xByXWrANsBJkt4BLgaOlHRh2seU9PcT4J9kXX21OqeylxARV0XEFhGxRY+qDo0d33Lvk1dfpUv/fnTu0wdVVbHqrrsw6/En6uT5/P0P6LFlduVll4FrUtGxIwtnz+alH/yQZw84kGcPOJApN9/KpOuua9eBUXvuDBkz7m3WGdCbgWv0okNVJd/ZfWvuefTFOnlW6bHi4i6jM47di+vuys6zB556hV222ZAeK61Aj5VWYJdtNuSBp15p8WNoM9rziQRs/NXNeWfiRCa9+w4LFizg3jvvYOfd6179uEbffjz138cAmPDG6yz4fD4r9+oFQE1NDffddSd77V//a8VKLaDsutVK1XK0+JI7sqDoEOC7DeR7FVgHeDR1e60dERPS832BxZciSFof6Ak8XZsWEYfl1h8NbBERZ0qqAnpExExJHYC9gf/kyl0PeLJZjnR5VV3NhIsvYaM/XYoqKnn/7nv47O23WfP47/HJq6/x4X+fYOKf/sy6Z51J30MPhgje+PX5rV3rtqkdf6lVV9fwk9/dyL1/OZWKigquu+sJxk+cyrkn7s9z49/hnsde5JtbfIVfn3wQRPDf59/gxxfcCMDsjz/lt1ffzVM3/hKA86+6m9kff9qah9Oq1J5PJKCqqopzL7yIYw8+gOrqag767hGsu/4G/PHC37DxkM3YeeienPmr33L2qScz/K9XgMSFf7pyceA9+uknWb1vXwYMHNTKR9IORVDTBrvOClGU6DrjdDfcPwKVwDW1I8vr5fkG8IOIODyNH/ov2X1RBLwEnJgbpD0M6BwRDU4LkAuOTpLUlezqtQ6p/P8Ap9Z2x0l6Htg1ImYVOob1u64YV23kyz1t2ZzaY0PGTi/urtNmjTlg2zU5/9xTW7sathxYd7VuzzUx4WKz2mT1XjHiqL0K5hn0++tbtE5NKdmYo4gYSXZvk0J5/ivpAkk9ImIO2QDtxvIOa2Jfw4Hh6fmnwOYN5ZP0VWBcU4GRmZmZLbug/MYctfaAbMgmbRoAzGkqYzPpBfyyhcoyMzNr3yKoWbiwtWuxVFo9OIqI/7VweQ+2ZHlmZmbtWhR1b7U2pdWDIzMzM1ueuVvNzMzMbLGIoLqI+6e1JQ6OzMzMrHTcrWZmZiVRmllXzEovIKrL6wR2cGRWaiWaS8zal1LNSWdWalm3WnnN9ebgyMzMzEqqxi1HZmZmZpnwmCMzq8+9IdYcwoOOrFxFUL3AwZGZmZlZpgxbjipauwJmyz2177upW/OQzyMrUwHU1ETBRzEkDZX0uqQJkpa4Cb2kAZIekfSCpLGS9mxg/VxJpzdVlluOzMzMrHQiqF6wbFerSaoErgB2BSYDoyWNiIjxuWxnA7dGxJWSBgMjgYG59ZcA9xVTnoMjMzMzK5lonnmOtgImRMREAEk3A/sB+eAogG7peXdgau0KSfsDbwOfFlOYu9XMSsy9IdYc3K1mZSsFR4UeRegLTMotT05pecOAwyVNJms1OhlA0orAGcCviq2ygyMzMzMrnTQJZKEH0EvSmNzj+C9R0qHA8IjoB+wJ3CCpgixoujQi5ha7I3ermZmZWckERU0COTMitiiwfgrQP7fcL6XlHQcMBYiIpyV1BnoBWwMHSfo90AOokTQ/Ii5vrDAHR2ZmZlY6Ec1xKf9oYF1Jg8iCokOA79bL8x6wMzBc0gZAZ2BGRHyjNoOkYcDcQoERODgyMzOzEopgmSeBjIhFkk4CRgGVwDURMU7SecCYiBgBnAZcLekUsgaro+NL3pTQwZGZmZmVVNQs+ySQETGSbKB1Pu2c3PPxwLZN7GNYMWU5ODIzM7PSifCNZ83MzMxqRUDNMk4C2dIcHJmZmVnpRFFXq7UpDo7MzMrAlxxXatYGFD3RY5vh4Mis1PylZs3Bp5GVqQhY2AwDsluSgyMzMzMrmQDKrOHIwZFZyZXZh4K1TeETycpYdZm1oDs4Miux8vpIMDNrXjUBC2rK65PQwZFZqflu6tYMfBZZOXO3mpmZNT8H2VamgnC3mpmZmVktD8g2MzMzywmPOTKz+twbYs1BPpGsjLnlyMzMzCzJutXKKzpycGRmZmYlE7hbzczMzGyxCHermZmZmdXhbjUzMzOzJAgWOjgyMzMzy5TjPEcVrV0BMzMzW37VXq1W6FEMSUMlvS5pgqQzG1g/QNIjkl6QNFbSnil9K0kvpsdLkg5oqiy3HJmZmVnJNMckkJIqgSuAXYHJwGhJIyJifC7b2cCtEXGlpMHASGAg8AqwRUQsktQHeEnS3RGxqLHy3HJkVmpl1tdubVOU2aXQZnnVUfhRhK2ACRExMSIWADcD+9XLE0C39Lw7MBUgIj7LBUKdU76C3HJUwKIOHZjVZ3Uq0FLdEjsiG4BWX8VSznBb08CXqtDSzbgcUNNQXXxMLXZMPWpgg34rNVQDoqFtJZbuHuyN7WfpfvtE1JSwLj6mZT2mDixkyjtv1SlfFUvxekRQU7Pk61FRUbFU07hHTU2Dx1FRWVl8XYCa6uol0nxMtLljag7NNAlkX2BSbnkysHW9PMOABySdDHQFdqldIWlr4BpgTeCIQq1GAGroxbOMpBnAu61djzauFzCztSthywWfS9YcfB41bc2IWLWlCpN0P9n7UkhnYH5u+aqIuCq3j4OAoRHxvbR8BLB1RJyUy3MqWVzzB0nbAH8HNorcryRJGwDXAdtHRL68OtxyVEBLnjzlStKYiNiiteth5c/nkjUHn0dtT0QMbYbdTAH655b7pbS844ChqcynJXUmC8qm5+ryqqS5wEbAmMYK85gjMzMza+tGA+tKGiSpI3AIMKJenveAnWFxC1FnYEbapiqlrwmsD7xTqDC3HJmZmVmblq40OwkYBVQC10TEOEnnAWMiYgRwGnC1pFPIhjodHREhaTvgTEkLgRrghxFRsOvVY45smUg6Pt8vbPZl+Vyy5uDzyJqDgyMzMzOzHI85MjMzM8txcNSOSKpO06e/Iuk2SSs0wz7Pk7RLgfUnSDpyWcux8lHvPLtbUo9m3v87knql53Obc9/W8nLnS+1joKRV0m0g5kq6vLXraO2Pu9XaEUlzI2LF9PwfwHMRcUlufVVTE2OZNaXeeXYd8EZEnN+M+3+H7FYAM/NlWXlq6D2U1BX4Ktnl1hvl57IpcV38GWiAW47as/8C60jaQdJ/JY0AxkuqlHSRpNHpxn0/qN1A0hmSXk437rswpQ1Pk3Mh6UJJ49N2F6e0YZJOT8+HSHomrb9TUs+U/qik30l6VtIbkr7R0i+GlczTZDPbImltSfdLei6dc+un9N7pfHgpPb6e0v+d8o6TdHwrHoO1sIj4NCKeoO6kgEuQtGH63Hgxfa6sm9KPTMsvSbohpQ2U9HBKf0jSgJQ+XNJfJf0P+H1j56m1L76Uvx1K8z3sAdyfkjYj+3X2dvoS+igitpTUCXhS0gNk80LsRzYj6WeSVq63z1WAA4D106WTDXWlXA+cHBGPpcsvzwV+ktZVRcRWyu6ifC65ad+tPCm7UeTOZLPUAlwFnBARbyqbyv8vwE7An4DHIuKAtE1tK8KxEfGhpC5kN5m8IyJmtfBhWOl1kfRiev52RDR5x/ScE4DLIuIfyua+qZS0IdkNSL+eWhdrP6v+DFwXEddJOpbsvNs/reuX8ldLeoiGz1NrRxwctS/5D6H/kn1pfR14NiLeTum7AZvUtgaR3bxvXbJg5dqI+AwgIj6st++PyH7l/V3SPcA9+ZWSugM9IuKxlHQdcFsuy7/S3+fI7qJs5av2POsLvAo8KGlFsnPtNn1xf6hO6e9OwJEAEVFNdi4B/FhS7Rdlf7Lz0MHR8mdeRAz5kts+DfxCUj/gXymg2Qm4rXYem9xn1TbAt9LzG4Df5/ZzWwqMCp2n1o44OGpflvgQSh8An+aTyFp3RtXLt3uhHacJurYiayk4CDiJpfu19Xn6W43Py3I3LyKGKBvwPwr4ETAcmFPsl6CkHcgC8m1SS+WjZLPdWjuWguVz0+L3IuKfqTtsL2BkfhjAUqr9DKxgKc5TW355zJHVNwo4UVIHAEnrpcGRDwLHpC88GuhWWxHoHhEjgVOATfPrI+IjYHZuPNERwGPYciu1Mv6YbNbaz4C3JX0bQJnac+Qh4MSUXplaGbsDs1NgtD7wtRY/AGtzIuLOiBiSHmMkrQVMjIg/AXcBmwAPA99OXf35z6qnyG45AXAYWet5/f1/TOPnqbUj/oVu9f2NrFvreWXNSjOA/SPifklDgDGSFgAjgZ/ntlsJuEvZjf4EnNrAvo8C/poCrInAMaU7DGsLIuIFSWOBQ8m+kK6UdDbQAbgZeAn4P+AqSceRtRyeSDYe7gRJrwKvA8+0Rv2t9Si7KrEb0FHS/sBuETG+XrbvAEcouy3E+8Bv0zi184HHJFUDLwBHAycD10r6KdnnWmOfP42dp9aO+FJ+MzMzsxx3q5mZmZnlODgyMzMzy3FwZGZmZpbj4MjMzMwsx8GRmZmZWY6DIzNbJpJC0o255SpJM9JM6YW2G5JuF9PY+i0k/ak562pmVgwHR2a2rD4FNkr3QAPYFZhSxHZDgAaDI2V3Rx8TET9upjqamRXNwZGZNYeRZLdwgGzCx5tqV0jqKumadPf0FyTtl24Seh5wcLqj+sGShkm6QdKTwA2SdqhtfZK0oqRrJb2c7qp+YJpNe7ikV1L6KS190Ga2fHJwZGbN4WbgkDRD+ibA/3LrfgE8HBFbATsCF5HNPHwOcEu6FcQtKe9gYJeIOLTe/n8JfBQRG0dE7S0ihgB9I2KjiNgYuLZUB2dm7YuDIzNbZhExluy2M4eStSLl7QacKelF4FGyG8gOaGRXIyJiXgPpuwBX5MqbTXYLmrUk/VnSUODjZTkGM7NaDo7MrLmMAC4m16WWCDgwd8PQARHxaiP7+LSR9CWkAGlTsoDrBLL7ApqZLTMHR2bWXK4BfhURL9dLHwWcnG5kjKSvpvRPyG5YXIwHgR/VLkjqKakXUBERdwBnA5stS+XNzGo5ODKzZhERkyOioUvvf002xmispHFpGeARYHDtgOwmdv8boGcafP0S2dilvsCjqbvuRuCsZjkQM2v3FBGtXQczMzOzNsMtR2ZmZmY5Do7MzMzMchwcmZmZmeU4ODIzMzPLcXBkZmZmluPgyMzMzCzHwZGZmZlZjoMjMzMzs5z/D2h9lXZaipNRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 708.661x255.118 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "\n",
    "batch_prediction = (model_batch.predict(x_test) > 0.5).astype(\"int32\")\n",
    "print('Classification Report LSTM Stochastic Gradient Descent Model:\\n', classification_report(y_test, batch_prediction))\n",
    "\n",
    "plot_classification_report(classification_report(y_test, batch_prediction), title='LSTM Stochastic Gradient Descent Classification Report', cmap='RdBu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 445
    },
    "id": "CQ-miZUdUAwF",
    "outputId": "faad88ed-66bc-47c3-8fb2-c6803653ba1b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report LSTM Mini-Batch Model:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.89      0.90      0.89      3745\n",
      "           1       0.90      0.88      0.89      3755\n",
      "\n",
      "    accuracy                           0.89      7500\n",
      "   macro avg       0.89      0.89      0.89      7500\n",
      "weighted avg       0.89      0.89      0.89      7500\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlQAAAD+CAYAAAD8gJuIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeZgU1dn+8e89MyCKCihiZDEuIUZcI4ioWYgrLrjEFeOWmJjlxURjjBiNGhLjviTRmJ9GFk2iAd/4ikgUxS1xC+AObqiogAqoiKIGmXl+f9QZ6WlnunuY6Rlm5v5cV11TferUqVNN0f30OadOKSIwMzMzs1VX0doVMDMzM2vrHFCZmZmZNZEDKjMzM7MmckBlZmZm1kQOqMzMzMyayAGVmZmZWRM5oDJrRZI+kLRZc+ctJ0njJP2mGcu7T9J3m6u8vLI3Tu9bZXq9oaQHJL0v6VJJv5D05zIc91uSpjZ3uWa2+nJAZe2CpLmS9mhg2y8kvZK+WOdJ+ntKn5XSPpBULenjnNe/kHS8pJB0eV55B6b0cQ0cb2jafkte+nYp/b7atIhYOyJeLuUcC+VNQc7yVPf3Jc2U9PVSyk37N/j+NZWkzpLOlfSipGXpWGMkbVKO4+WKiNfS+1adkk4EFgPrRsSpEfHbiGhSMCdpk/TvWpVz3L9GxF5NKbeBYw2VVJPz7/y8pG8393EaUZd5rXFss9WRAypr1yQdBxwD7BERawODgGkAEbFV+rJdG/gXMLL2dUT8NhXxEnB47pclcBzwQpFDLwJ2lrR+I/driovSuawLXA38o7ZlppXdDBwAHAV0A7YDZgK7t0JdPg/MjrY9o/GCnH/nU4BrJW3RkhXI+/9gZjigsvZvR+DOiHgJICLejIhrGrH/m8DTwN4AktYDdgEmFdlvOfB/wJFpv0rgCOCvuZlSy8YX0vo4SVdJuj21PjwqafP68haSgoW/AesBG6Z9N5d0j6S3JS2W9FdJ3dO2G4CNgdtSy8fPU/pXJD0kaYmk1yUdn3OYHg3VM+/89gD2BA6MiOkRsSIi3ouIqyLiunryN1jPtP10SfNzWmd2T+mDJc2QtFTSW5IuS+mfth6lFsXjgJ+n89wjtZz9Jaf8es9Z0n6SHk/lvy7p3JxqP5D+Lknl7pxaN/+dU+4ukqZLei/93SVn232Sfi3pwXReUyX1bPhfOBOZKcA7wLaprApJoyS9lN7DCemazX0vTpS0QNIbkn6WU481JF2Rti1I62ukbUOVte6eLulN4Ebgn0BvrWzV7V2szmbtmQMqa+8eAY6VdJqkQavYYnM9cGxaPxK4FfhvI/fbG3gGWFBknyOBXwE9gDnAeY2tbDrHY4FXgLdqk4Hzgd7AlkA/4FyAiDgGeA0YnlrnLpL0ebIvzD8AGwDbA0+sQj33AP4TEa+XWv2G6plaYUYCO0bEOmTv6dy03++A30XEusDmwIT8giPieLKA9qJ0nnfXOXDhc15G9p52B/YDfijpoLTta+lv91Tuw3nlrgfcDvweWB+4DLg9r/XyKODbQC+gM/AzikjB0wFAT7J/A4CTgIOAr5O9h+8CV+Xt+g2gP7AXcLpWdvWeCQxJ570dMBg4K2e/z5EF6Z9P78U+pNaytBS7ts3aNQdU1q5FxF/IvmT2Bu4HFko6vZHF3AIMldSN7Ivk+hKP/RCwXgoESt3vloj4T0SsIPvy374R9fyZpCXAB8AVwC9rxw5FxJyIuCsi/hsRi8i+1AuNsToKuDsiboyITyLi7YjIDahKref6wBulnkCRelYDawADJHWKiLm1LY/AJ8AXJPWMiA8i4pFSj5mjwXOOiPsi4umIqImIp8haaEodo7Yf8GJE3JBa6G4EngOG5+QZGxEvRMRHZMFgoX/33unf+SOya/OnEfF42vYD4MyImBcR/yULRg/N66L7VUQsi4ingbHAiJT+LWB0RCxM7/2vyLrLa9UA56R/m49KPHezDsMBlbV7aYDwHmStCz8Afi1p70bs/xFZC8NZwPoR8WAjDn8DWavKN8i+/Ip5M2f9Q2Dt/AxaeefaB5I+yNl0SUR0B9YiGyt2saR90j4bSropdZctBf5C1rLRkH5k48dWuZ7J28BGBcqpo1A9I2IOcDJZkLAw5avtZjoB+CLwXOpS27/UY+Zo8Jwl7STpXkmLJL1Hdh0V7ZZLegOv5qW9CvTJeV3q+wlZq1B3sjFUvwd2y9n2eeCW1GW5BHiWLBDdMCdPbmvhq6l+9dUzdxvAooj4uEC9zDo0B1TWYaRWh4nAU8DWjdz9euBUsi/4xrgB+BEwJSI+bOS+9cq5c612QH3+9oiIZ4AHyVpHAH4LBLBN6hY7mqx77dPd8op5nazrrKnuBgZL6lti/oL1jIi/RcRXyAKHAC5M6S9GxAiyLrMLgZsldW1kXQud89/Ixs31i4huwJ9y6lVsgPuCVN9cGwPzG1m/OlIL1OnANjndj68D+0RE95ylS0TkHqtfXj1qu+ry65m7DT57nm15YL9Zs3NAZe1JJ0ldcpaqNDh4P0nrpDEn+wBbAY82suz7yQZX/6ExO0XEK2RdQ2c28nhNIulLwFeAWSlpHbKuwPck9QFOy9vlLSB3jqu/AntIOjy9j+tLakz3IwBpnNJdZK0mA1NZ60j6gaTv1LNLg/WUtIWk3dJA6Y/Jurxq0rajJW0QETXAkrRLTSOrW+ic1wHeiYiPJQ0m6x6stSgdq6E5wqYAX5R0VCr3CGAAMLmR9fuMiFgOXAqcnZL+BJyXxoMhaQNJB+bt9ktJa0naimzc1t9T+o3AWWmfnqnMQj8g3gLWT13hZh2eAyprT6aQfcnWLucCS4FfkA26XgJcBPwwIv7dQBn1Sq0+0yLincZWKiL+3UIDdmvvXlsGTCUbH/P/0rZfATsA75F1X/4jb9/zyb5Ml0j6WUS8BuxL1ir3Dtng7O1WsV6Hkv3b/D0d/xmyLsm768lbqJ5rABeQzSP1Jllr1Blp2zBgVuoC/R1wZGPH+RQ55x8BoyW9TxZoTMjZ70OyQfkPpvdvSF65bwP7p3LfBn4O7B8RixtTvwLGABtLGk527pOAqamujwA75eW/n2wQ+zSybuLaCUh/A8wga8F9GngspdUrIp4jC8JeTuftu/ysQ1O06elYzMysFMomUn0F6JRuJjCzZuQWKjMzM7MmckBlZmZmqz1Jw5RN6DtH0qh6tn9e0jRJTymbMLdvzrbjlD3+6kVlT9CoTR8o6elU5u8lKaWvJ+mulP8uST2K1c8BlZlZB5Dm7ZK7+6wtShMWX0U2oewAYISkAXnZLgGuj4htgdFkY0NrJ9c9h2w84WDgnJwA6Wrge2ST3fYnG48JMAqYFhH9ycYbfiaAy+eAyszMzFZ3g4E5EfFyurv1JiD/DtYBwD1p/d6c7XsDd0XEOxHxLtmdx8MkbUT2oPRHIhtQfj3ZkwZI+45P6+Nz0hvkgMrMzMxWd32oOyntPOpOjgvwJPDNtH4wsE56xFND+/ZJ6/WVuWFE1D7l4U3qTo5bLz8xvICqzmtGpzXXae1qWBvXdcV/qaxp7JRIZnVVdl2T9T7XK5tNs767s6U6M7UW09Ad3mkISYuW43Mqb13yy5k9e/biiNigUQU0QcW6fYMVhSfZj4/enkU2v1yta6JxD7KH7BmYVyp7qPkDZJPnVjeyjM/WLSIkFZ0SwQFVATVVXanebFWeYGG20sEfvUTl8/lPHjFrnL7fHMap113R2tWwdmCtNdds2Q+k6v/SacuDC2ZZ/tifP46IQQWyzKfuLP99yXvaQJrv75sAktYGDomIJZLmA0Pz9r0v7d83L722zLckbRQRb6SuwYUFTwB3+ZmVXeN+R5o1wBeStWGqqCy4lGA60F/SppI6A0eSTWK78hhST0m1cc0ZZJPeAtwJ7CWpRxqMvhdwZ+rSWyppSLq771jg1rTPJKD2bsDjctIb5IDKrNz8RWhmHZqaHFClu1NHkgVHzwITImKWpNGSDkjZhgLPS3qBbMzTeWnfd4BfkwVl04HROU+9+BHwZ7KnB7wE/DOlXwDsKelFYI/0uiB3+ZmZmVnZSKKiU+cmlxMRU8geY5WbdnbO+s3AzQ3sO4aVLVa56TOAretJfxvYvTH1c0BlZmZm5SNRUVq3XpvmgMrMzMzKqsRxUm2aAyozMzMrG0lUVHVq7WqUnQMqMzMzKyO5hcrMzMysSeSAyszMzKxpmukuv9WdAyozMzMrG+FB6WbWDIo+AMqsBDV+HqS1Ve7yMzMzM2siicoqd/mZmZmZrTL5Lj8zaw4R7vSzZuDLyNowB1RmZmZmTaEKKtzlZ2ZNpdaugLULFRUVrV0Fs1UjUKVbqMysyRxSmVnH5TFUZmZmZk0ldYguP7chm5WZ3EBlzcHXkbVhFRUquJRC0jBJz0uaI2lUPds3lnSvpMclPSVp35TeWdJYSU9LelLS0JS+jqQncpbFkq5I246XtChn23eL1c8tVGZmZlY2klCJQVOBMiqBq4A9gXnAdEmTImJ2TrazgAkRcbWkAcAUYBPgewARsY2kXsA/Je0YEe8D2+ccYybwj5zy/h4RI0uto1uozMzMrKwkFVxKMBiYExEvR8Ry4CbgwLw8Aayb1rsBC9L6AOAegIhYCCwBBuXV74tAL+Bfq3B6gAMqMzMzKydBZVVFwaUEfYDXc17PS2m5zgWOljSPrHXqpJT+JHCApCpJmwIDgX55+x5J1iKVO+PbIanr8GZJ+fk/wwGVmZmZlU32cGQVXICekmbkLCeuwqFGAOMioi+wL3CDpApgDFkANgO4AngIqM7b90jgxpzXtwGbRMS2wF3A+GIH9xgqMzMzKyNRUbxbb3FEDCqwfT51W5X6prRcJwDDACLiYUldgJ6pm++UT2sjPQS8kPN6O6AqImbWpkXE2znl/hm4qNgJuIXKzMzMykdQUVVRcCnBdKC/pE0ldSZrUZqUl+c1YHcASVsCXYBFktaS1DWl7wmsyBvMPoK6rVNI2ijn5QHAs8Uq6BYqMzMzKxuJkqdGaEhErJA0ErgTqATGRMQsSaOBGRExCTgVuFbSKWQD1I+PiEh39t0pqYasVeuYvOIPJ+sizPVjSQcAK4B3gOOL1dEBlZmZmZWVmqE/LCKmkA02z007O2d9NrBrPfvNBbYoUO5m9aSdAZzRmPo5oDIrsyiexayoujcfmbUdkqisbP8jjBxQmZm1AQ6orC1r6sSebYEDKjMzMysrB1Rm1gzcsmBmHZeEu/zMrOncU2PNwteRtVlNf5ZfW+CAysysDSjxeWdmq59mmDahLXBAZVZmov1/kFj5OaCytkpARWX7v34dUJmZmVn5uIXKzMzMrOk8hsrMmsw9NdYsfB1ZG+WJPc3MzMyagVuozMzMzJpAgkoHVGZmZmarTojOVe7yMzMzM1t1HaSFqv2HjLbK9tpla5655bfMvvV8Tvv2vp/ZvvFG63PHn37GzL//iruu/Tl9evX4dNsxw3dh1q3nM+vW8zlm+C4tWW1bzQzY++uc+9w0Rr94H3uf/sPPbO/Rrzen3HMjv3jsds568p9svc9QACqqqjhu3KX88qk7OGf23ew96kctXHNb3UydOpXttt2Wrbfaiksuvvgz219/7TWG7b03Q4YMYfCOO3LHHXcAsHz5ck488UR2HDSInQYP5oEHHmjpqndoIguoCi3tgQMqq1dFhfjdqKMZPvJytjvkLI4YthNbbta7Tp4LTzmcv97+EAOPOIfzrpnEb046BIAe63blzBMP5CvH/IZdj/41Z554IN3XWas1TsNamSoqGHHVaK7c53h+NWBPdhxxABtt+YU6efY9ayQzJ9zOb3fYj+uOPIkRf/wNAAMP25eqNTrz622H8duB+/O17x/F+p/v2xqnYauB6upqTjn5ZP7v1lt57PHHmThxIs8++2ydPBdceCHfPOQQHnnkEcZffz0n/+QnAIwZMwaA6TNmcNvkyYwaNYqampoWP4eOqkKwRlVFwaUUkoZJel7SHEmj6tm+saR7JT0u6SlJ+6b0zpLGSnpa0pOShubsc18q84m09Erpa0j6ezrWo5I2KXqepb0djSdpjKSFkp4pku9kScem9V+nN+EJSVMl9U7pp+Wc7DOSqiWtl7bNTW/SE5Jm5JR7rqT5OfvVvrHbSBpXrvNuL3bcejNeen0hr8xfxCcrqplw56MMH7p9nTxbbtabe/+TfaDdN/05hg/9MpC1bE17ZBbvLl3Gkvc/ZNojs9h7121a/Bys9W0yeHsWznmVxa+8TvUnnzD9ptvY9sC96uSJgC7rrg1Al27rsmTBW5+mr9F1TSoqK+m8ZhdWLF/OR0vfb/FzsNXDjOnT2Xzzzdl0003p3Lkzhx52GJMnT66TRxLvL10KwNL33mOjjTYC4LnnnmPo0KEA9OrVi+7dujFz5swWrX+HpsKtU6W0UEmqBK4C9gEGACMkDcjLdhYwISK+DBwJ/DGlfw8gIrYB9gQulZQb/3wrIrZPy8KUdgLwbkR8AbgcuLBYHcvZQjUOGFYog6Qq4DvA31LSxRGxbURsD0wGzgaIiItrTxY4A7g/It7JKeobafugvENcnvMmTUllPQ30lbRxU0+wPevTqzvz3lr5Fs9/6116b9CjTp6nXnidg3YbCMBBu+3AumuvyXrdutJ7g7x9F75L7w26t0zFbbXSo8+GvPv6gk9fL5n3Bj36bFgnz+RzL2enow/i/NcfZuSUsfz9pHMAeOzmKfx32Udc+MZ/+O1rD3HXJdfy4bvvtWj9bfWxYMEC+vRd2ULZp08fFsyfXyfPmWeeyU033cQXNt+cgw8+mEsvuwyAbbbZhtsnT2bFihXMnTuXxx9/nPnz5rVo/TuyrMuvouBSgsHAnIh4OSKWAzcBB+blCWDdtN4NqP3wGQDcA5ACpiVAfryQ70BgfFq/GdhdRZ7/VLaAKiIeAN4pkm034LGIWJH2WZqzrSv1P199BHBjE6t3G1n0ak1w+uUT+NrALfjPjefw1YFbMO+td6iudjN6vvouYltpxxEH8PC4mzmj385cue+3+fYNlyOJTQdvR1RXc3rvnThr06+yx6nfpeem/Vq7uq3GXVTFTZwwgaOPPpo5L73ELbfcwndPOIGamhqOO+44+vTpw6677sppp53GTkOGUFFZ2drV7TAk6FxVUXApQR/g9ZzX81JarnOBoyXNA6YAJ6X0J4EDJFVJ2hQYCOR+mIxNPVm/zAmaPj1eilHeA9YvVMHWHkO1K1Cn3VXSeZJeB75FaqHK2bYWWavX/+YkBzBV0kxJJ+aVPzJ1IY6RlNu8MgP4an0VknSipBmSZsSKj1ftrNqB+QuX0HfD9T593WfDHixY9G6dPG8sWsLhP7uKwSN+xdlX/gOA9z74iAWL8vbt1YMFi5a0TMVttfLu/Lfo0W/l2LvufTfi3flv1cmz6wlHMHPC7QC88shjVHVZg7V7rseORx3IrDvup2bFCt5f9DYvPTiTzw/atkXrb6uP3r1712lVmj9/Pr371P0+HT9+PIccko3l3GnIED7++GMWL15MVVUVF118MY8++igTJ07kvSVL6N+/f4vWvyMTJXX59az97k1L/vd5KUYA4yKiL7AvcEPq2htDFoDNAK4AHgKq0z7fSl2BX03LMat6nq0dUG0ELMpNiIgzI6If8FdgZF7+4cCDed19X4mIHcj6Vf9H0tdS+tXA5sD2wBvApTn7LATqjrBeefxrImJQRAxSVZdVPK22b8asV/jCxhuySe+edKqq5PC9d2LyfU/UybN+97WpDeZP/85+jL/13wBMfegZ9th5K7qvsxbd11mLPXbeiqkPFRxKZ+3Uq9OfpFf/TVh/k75UdurEjkcO56lJd9XJ885rC/jS7rsC8LkvbU6nLmvw/qK3eee1BWyxW3aHaOe11mSzIV/mzedeavFzsNXDwEGDmDNnDnPnzmX58uXcPHEi++23X508ffv149777gOycVMff/wxG2ywAR9++CHLli0DYNq0aVRVVbHlllu29Cl0aJVSwQVYXPvdm5Zr8oqYT91Wpb4pLdcJwASAiHgY6AL0jIgVEXFKGv5zINAdeCHlm5/+vk82/Ghw/vHS8KRuwNuFzrG156H6iOyE6/NXsia7c3LSjiSvuy/nzVgo6RayN+OBiPj0Z7Cka8nGZNXqko5tDaiuruHkC//C7X/8KRUVFYy/9d/MfnkB5/zwIGbOnsvk+5/g64O24NcnHQoR/OuxF/jx+X8B4N2ly/jttbfx0F9+CcB519zGu0uXtebptK7ouJ1+NdXV/H3k2fz4zuupqKzkoTETeGP2iwz/1Sm8OuNpnrrtbv731N9w9LUXsPspJxARjD/+ZwDcf9X1HDv2Ys5+ZiqSeGjsROY//Vwrn5G1lqqqKi67/HIOGD6c6upqjj3uOAYMGMDo0aPZYYcd2H///bngggv4nx/9iCv/8AeQuObaa5HEokWLOGD4cCoqKujduzfXXXdda59Oh1Lb5ddE04H+qctuPlk8cFRenteA3YFxkrYk+65flHq3FBHLJO0JrIiI2SlQ6h4RiyV1AvYH7k5lTQKOAx4GDgXuiSj8Ya4i25sk3WY4OSK2bmD7D4C+EXFWet0/Il5M6ycBX4+IQ9PrbsArQL+IWJbSugIVEfF+Wr8LGB0Rd0jaKCLeSPlOAXaKiCPT60OAPSPiB4XqX7FWz6ja4oCmvQnW4R370Ryqnn+ttathbVzvI/fitLG/b+1qWDuw1pprzqznJq6y+Vz/reLYyyYUzHPxAVsXrVO6W/8KoBIYExHnSRoNzIiISemuv2uBtcmGA/08IqamWOROoIYsGDshIl5NccMDQKdU5t3ATyOiWlIX4Abgy2TjwY+MiJcL1a9sLVSSbgSGkvWLzgPOiYj8nwX/JKtwrQskbUF20q8CuQHPwcDU2mAq2RC4JXU7VQF/i4g70raLJG1P9qbOBb6fs983gNtX/ezMStc+pqyz1lbkBiOz1VbtGKqmSnfrT8lLOztnfTbZ2Oz8/eYCW9STvoxsgHp9x/oYOKwx9StbQBURI0rI86qkt2tbpiLikAJ5x5FNxZCb9jKwXQP56x1YJmkNstslTy5WP7Pm4S9CazoHVNZWNVOX32qvtcdQAYwiG5z+Ygsdb2NgVO1UDWZmZlY+tY+eae9aPaCKiOeB51vweC/ScsGbmdunrHn4QrK2qoM8HLnVAyqzdq/9f46YmTVIyF1+ZmZmZk0ht1CZmZmZNY3HUJmZmZk1kSQ6V7rLz8zMzKxJKjrAtB8OqMzMzKxsBFS2/3jKAZWZmZmVjwSd3OVnZmZm1jTu8jMzMzNrAnf5mVmziNaugLULNTU1rV0Fs1UiiSp3+ZmZmZk1jVuozMzMzJpAdIwxVO2/Dc6slUW408+agS8ja6Nq7/IrtJRWjoZJel7SHEmj6tm+saR7JT0u6SlJ+6b0zpLGSnpa0pOShqb0tSTdLuk5SbMkXZBT1vGSFkl6Ii3fLVY/t1CZmZlZWTW1y09SJXAVsCcwD5guaVJEzM7JdhYwISKuljQAmAJsAnwPICK2kdQL+KekHdM+l0TEvZI6A9Mk7RMR/0zb/h4RI0uto1uozMqs/Td0W0uoqPDHtbVNQlSo8FKCwcCciHg5IpYDNwEH5uUJYN203g1YkNYHAPcARMRCYAkwKCI+jIh7U/py4DGg76qep/+HmpWdQyoz67iyLj8VXErQB3g95/W8lJbrXOBoSfPIWqdOSulPAgdIqpK0KTAQ6Fe3juoODAem5SQfkroOb5ZUJ399HFCZmZlZWVVKBRegp6QZOcuJq3CYEcC4iOgL7AvcIKkCGEMWgM0ArgAeAqprd5JUBdwI/D4iXk7JtwGbRMS2wF3A+GIH9xgqszLrADe3WEvwdWRtVIl3+S2OiEEFts+nbqtS35SW6wRgGEBEPCypC9AzdfOd8ml9pIeAF3L2uwZ4MSKuqE2IiLdztv8ZuKjYCbiFyszMzMqmmbr8pgP9JW2aBpAfCUzKy/MasHt2TG0JdAEWpbv5uqb0PYEVtYPZJf2GbLzVyXXrrI1yXh4APFusgm6hMjMzszL6tFtvlUXECkkjgTuBSmBMRMySNBqYERGTgFOBayWdQjZA/fiIiHRn352SashatY4BkNQXOBN4DnhMWR2vjIg/Az+WdACwAngHOL5YHR1QmZmZWdk018SeETGFbLB5btrZOeuzgV3r2W8usEU96fNooDM9Is4AzmhM/RxQmZmZWdnUdvm1dw6ozMzMrKw6ws05jR6ULqmHpG3LURkzMzNrfypQwaU9KKmFStJ9ZKPcq4CZwEJJD0bET8tYNzMzM2vjBJT4uL42rdRT7BYRS4FvAtdHxE7AHuWrlpmZmbULyrr8Ci3tQakBVVWak+FwYHIZ62PW7kRrV8DahQhfSdY2qUh3X4fq8gNGk8398GBETJe0GfBi+aplZma5HFBZW9YRuvxKCqgiYiIwMef1y8Ah5aqUmZmZtR/tow2qsJJiRklflDRN0jPp9baSzipv1czaC7csmFnHVTuxZ6GlPSi1Ee5ashlDPwGIiKfInqNjZkW4p8aaha8ja8MqVXhpD0odQ7VWRPxHdaPIFWWoj5mZ1UPt5Fe8dTzZnXzt//otNaBaLGlz0m8kSYcCb5StVmbtiDrE6AErt47whWTtV0UHuHxLDaj+B7gG+JKk+cArwNFlq5WZmZm1C6L9dOsVUupdfi8De0jqClRExPvlrZaZmZm1Fx2hhbXUu/x+Imld4EPgckmPSdqrvFUzax86wOeItQRfR9ZWKevyK7S0B6Xe5fed9OiZvYD1gWOAC8pWKzMzM2sXarv82vtdfqUGVLWnuy/Zs/xm4d9LZmZmVgJJBZcSyxgm6XlJcySNqmf7xpLulfS4pKck7ZvSO0saK+lpSU9KGpqzz8CUPkfS75UqI2k9SXdJejH97VGsfqUGVDMlTSULqO6UtA5QU+K+ZmZm1kFlE3s2rctPUiVwFbAPMAAYIWlAXrazgAkR8WWyuTL/mNK/BxAR2wB7ApdKqo1/rk7b+6dlWEofBUyLiP7AtPS6oFIDqhNSYTtGxIdAJ+DbJe5rZmZmHZaoVOGlBIOBORHxckQsB24CDszLE8C6ab0bsCCtDwDuAYiIhcASYNuajacAAB3NSURBVJCkjYB1I+KRyB6WeT1wUNrnQGB8Wh+fk96gUqdN2Bl4IiKWSToa2AH4XYn7tlk91lmDfb++GVJFnYHFxeYVyv5dVk5rXJOmyq6oyI1fi89OFLGyETBSufnNoyWUUuehqhFBgM+JljunNx98k3V32q6+0yByjh9Zwmeav4s1h+c/NLf2teq+GSW+Hzm1ic8ev2jTvM+pxLo0/pzmffxfZs6YkdUkAqQ616pU+N2ICGpqauq8BlBFRZ06FHsMSE1NzWf+/wFUVFbmnkbR96O6unplGVlBPida9pxajEq6OaenpBk5r6+JiGtyXvcBXs95PQ/YKa+Mc4Gpkk4CugJ7pPQngQMk3Qj0AwamvzWpnNwy+6T1DSOidr7NN4ENi51AqQHV1cB2krYDTgX+TBbJfb3E/dukz/f9HNdc8LPWroa1cd84ahF3vPJea1fD2rgjem/CwEGDWrsaZo2mCFT8GVyLI6KpF/gIYFxEXCppZ+AGSVsDY4AtgRnAq8BDQHXDxdQVESGp6AmUGlCtSAUeCFwZEddJOqHUypiZmVnHpZomP61uPlmrUq2+KS3XCaQxUBHxsKQuQM/UzXfKp3WRHgJeAN5N5dRX5luSNoqIN1LX4MJiFSx1DNX7ks4gmx399jSYq1OJ+5qZmVmHFRA1hZfipgP9JW0qqTPZoPNJeXleA3YHkLQl0AVYJGmtNDE5kvYkaySanbr0lkoaku7uOxa4NZU1CTgurR+Xk96gUluojgCOAk6IiDclbQxcXOK+ZmZm1pEV7/IrsnuskDQSuBOoBMZExCxJo4EZETGJbEjStZJOIRvGdnzqXetFNkNBDVkL1DE5Rf8IGAesCfwzLZDNtTkh9ca9ChxerI6lPnrmTeCynNevkY2hMjMzM2tYRKmtUEWKiSnAlLy0s3PWZwO71rPfXGCLBsqcAWxdT/rbpNauUpX66JkhkqZL+kDScknVkjzK1qwEuXeImZl1RKpZUXBpD0rt8ruSrL9yIjCIrJ/xi+WqlFm70sSmbjOoOz2HWdvSPC1Uq7tSB6UTEXOAyoiojoixrJxN1MzMzKx+QXMMSl/tldpC9WEaVf+EpIuAN2hEMGZmZmYdVaDq9tGtV0ipQdExZKPqRwLLyOaCOKRclTJrTzyGysw6PLdQZSLi1bT6EfCr8lXHzMzM2pWIDjGWtGBAJelpaPjndURs2+w1MmtnVj7U3GzVFX3moNlqrL3cyVdIsRaqb5I9EPD1vPR+ZA8LNDMzMyvAd/kBXA68FxGv5i7Ae2mbmZmZWWEeQ8WGEfF0fmJEPC1pk7LUyKydEe6qsebg68jaJkW4yw/oXmDbms1ZETMzM2unatpHK1Qhxbr8Zkj6Xn6ipO8CM8tTJTMzM2s/YuWdfg0t7UCxFqqTgVskfYuVAdQgoDNwcDkrZmZmZu1A+C4/IuItYBdJ32Dl05hvj4h7yl4zMzMzawc6xl1+pU7seS9wb5nrYmZmZu1RBwioPOOgmZmZlU8EseKTgkspJA2T9LykOZJG1bN9Y0n3Snpc0lOS9k3pnSSNl/S0pGclnZHSt5D0RM6yVNLJadu5kubnbNu3WP1KfTiymZmZ2SoIqKluUgmSKoGrgD2BecB0SZMiYnZOtrOACRFxtaQBwBRgE+AwYI2I2EbSWsBsSTdGxPPA9jnlzwduySnv8oi4pNQ6uoXKzMzMyifIAqpCS3GDgTkR8XJELAduAg6s50jrpvVuwIKc9K6SqsimfFoOLM3bd3fgpZxnFzeaAyqzMouGH4dpVrJoJ7eWW8cTEcQnnxRcStCHuo/Bm5fScp0LHC1pHlnr1Ekp/WZgGfAG8BpwSUS8k7fvkcCNeWkjU9fhGEk9ilXQAZVZufmL0JqFryNrq6KUFqqekmbkLCeuwoFGAOMioi+wL3CDsqfTDwaqgd7ApsCpkjar3UlSZ+AAYGJOWVcDm5N1Cb4BXFrs4B5DZWZmZuUTQRTv1lscEYMKbJ8P9Mt53Tel5ToBGJYdMh6W1AXoCRwF3BERnwALJT1INqfmy2m/fYDH0lRRqcor1yVdC0wudgJuoTIrM7crmFnH1ix3+U0H+kvaNLUoHQlMysvzGtlYKCRtCXQBFqX03VJ6V2AI8FzOfiPI6+6TtFHOy4OBZ4pV0C1UZmXnkMrMOrDaQelNKSJihaSRwJ1AJTAmImZJGg3MiIhJwKnAtZJOSUc9PiJC0lXAWEmzyJ4yPjYinoJPA6w9ge/nHfIiSduncubWs/0zHFCZlZ1auwJmZq0omuXhyBExhWyweW7a2Tnrs4Fd69nvA7KpE+orcxmwfj3pxzS2fg6ozMpMckBlTefryNqsCOKT5a1di7JzQGVmZmblE02f2LMtcEBlZtYmuIXK2q5ohi6/1Z0DKrMyk78Izawji4AV7vIzMzMzW2VBuIXKzMzMrEmaYdqEtsABlZmZmZWP7/IzMzMza6rmmYdqdeeAyszMzMrLXX5mZmZmTRBBTWnP62vTHFCZmZlZ+UQQ1e7yMzMzM1tlETigMrOmi2j/HyRWfr6OrM2KoOaTFa1di7JzQGVmZmZl1RFaqCpauwK2+po6dSrbbbstW2+1FZdcfPFntr/+2msM23tvhgwZwuAdd+SOO+4AYPny5Zx44onsOGgQOw0ezAMPPNDSVbfVyF67bM0zt/yW2beez2nf3vcz2zfeaH3u+NPPmPn3X3HXtT+nT68en247ZvguzLr1fGbdej7HDN+lJattqyF/JrVNEUFNdXXBpT0oW0AlaZik5yXNkTSqQL4rJH0trV8n6UlJT0m6WdLaKf1ySU+k5QVJS/LKWFfSPElX5qTdl45fu1+vlD5S0nfKc9btR3V1NaecfDL/d+utPPb440ycOJFnn322Tp4LLryQbx5yCI888gjjr7+ek3/yEwDGjBkDwPQZM7ht8mRGjRpFTQeYg6QhQbR2FVpNRYX43aijGT7ycrY75CyOGLYTW27Wu06eC085nL/e/hADjziH866ZxG9OOgSAHut25cwTD+Qrx/yGXY/+NWeeeCDd11mrNU7DVgP+TGrDUpdfoaUUxeIKSRtLulfS4ymO2Deld5I0XtLTkp6VdEbOPnNT+hOSZuSkryfpLkkvpr898o+XrywBlaRK4CpgH2AAMELSgHryrQ8MiYjanwunRMR2EbEt8BowEiAiTomI7SNie+APwD/yivo1UN9Pjm/V7hcRC1PaGOCkJp5iuzdj+nQ233xzNt10Uzp37syhhx3G5MmT6+SRxPtLlwKw9L332GijjQB47rnnGDp0KAC9evWie7duzJw5s0Xrb6uHHbfejJdeX8gr8xfxyYpqJtz5KMOHbl8nz5ab9ebe/2RfjPdNf47hQ78MZC1b0x6ZxbtLl7Hk/Q+Z9sgs9t51mxY/B1s9+DOpbYvqmoJLMSXGFWcBEyLiy8CRwB9T+mHAGhGxDTAQ+L6kTXL2+0aKEwblpI0CpkVEf2Bael1QuVqoBgNzIuLliFgO3AQcWE++Q4A7al9ExFIASQLWhHp/2o8Abqx9IWkgsCEwtZSKRcSHwFxJg0s7lY5pwYIF9Onb99PXffr0YcH8+XXynHnmmdx00018YfPNOfjgg7n0sssA2Gabbbh98mRWrFjB3Llzefzxx5k/b16L1n91ItTaVWg1fXp1Z95b73z6ev5b79J7g7o/9J564XUO2m0gAAfttgPrrr0m63XrSu8N8vZd+C69N+jeMhVfDWUfix2XP5Parojs4ciFlhKUElcEsG5a7wYsyEnvKqmKLLZYDiwtcrwDgfFpfTxwULEKliug6gO8nvN6XkrLtytQ52eCpLHAm8CXyFqjcrd9HtgUuCe9rgAuBX7WQD3Gpma8X6rup9EM4Ksln43Va+KECRx99NHMeeklbrnlFr57wgnU1NRw3HHH0adPH3bddVdOO+00dhoyhIrKytaubuvp4F+ExZx++QS+NnAL/nPjOXx14BbMe+sdqjvAAFZrfv5MWk0FVC9fUXApQSlxxbnA0ZLmAVNY2Rt1M7AMeIOs9+uSiKj9tRbAVEkzJZ2YU9aGEfFGWn+TrOGmoNa+y28jYFFuQkR8OzXt/QE4Ahibs/lI4OaIqB3B9iNgSkTMq+fX27ciYr6kdYD/BY4Brk/bFpIFbJ+R3tATAfr167eq59Xm9e7du84vuPnz59O7T91rd/z48dx6660A7DRkCB9//DGLFy+mV69eXJQzYPQbQ4fSv3//lqm4rVbmL1xC3w3X+/R1nw17sGDRu3XyvLFoCYf/7CoAuq65BgfvPpD3PviIBYuW8LWBW6zct1cPHpj5fMtU3FY7/kxqw0qb2LNn7hgm4JqIuKaRRxoBjIuISyXtDNwgaWuy1q1qoDfQA/iXpLsj4mXgKylW6AXcJem5nGFIqfoRkooOhi1XC9V8IDca6ZvS8n0EdMlPTAHTTWRdgrmOJKe7D9gZGClpLnAJcKykC1IZ89Pf94G/kb2htbqkY39GRFwTEYMiYlDPDTZo6PzavYGDBjFnzhzmzp3L8uXLuXniRPbbb786efr268e9990HZGMUPv74YzbYYAM+/PBDli1bBsC0adOoqqpiyy23bOlTWI103BaqGbNe4Qsbb8gmvXvSqaqSw/feicn3PVEnz/rd1/60O+v07+zH+Fv/DcDUh55hj523ovs6a9F9nbXYY+etmPrQMy1+DquPjnsdgT+T2rKAUrr8Ftd+96YlP5gqJa44AZgAEBEPk33X9wSOAu6IiE/SeOoHgUEpX22ssBC4hZWxwluSNgJIfxdSRLlaqKYD/SVtSnbCR5KdUL5ngS8A96Uuuc0jYk5aPwB4rjajpC+RRZYP16ZFxLdyth8PDIqIUamftHtELJbUCdgfuDvnuF8ke0OtAVVVVVx2+eUcMHw41dXVHHvccQwYMIDRo0ezww47sP/++3PBBRfwPz/6EVf+4Q8gcc211yKJRYsWccDw4VRUVNC7d2+uu+661j6dVtWRvwarq2s4+cK/cPsff0pFRQXjb/03s19ewDk/PIiZs+cy+f4n+PqgLfj1SYdCBP967AV+fP5fAHh36TJ+e+1tPPSXXwJw3jW38e7SZa15OtaK/JnUhkVQU1q3XiGlxBWvAbsD4yRtSRZQLUrpu5G1WHUFhgBXpPWKiHg/re8FjE5lTQKOAy5If28tVkFFlOeW7nS74hVAJTAmIs6rJ89Xge9HxNFpPNS/yAaUCXgS+GHOQPVzgS4RUe9I+5yAamR6Yx4AOqXj3w38tLarUNJjwJ4R8Xahc9hh4MB48EHHXdY03zjqDB575b3Wroa1cUd8dROuuejnrV0NawfWWnPNmXl3tJXVtp/rGZOO269gnk0vur5oneqLKySNBmZExKR019+1wNpkDWM/j4ipaQqmsWR3BwoYGxEXS9qMrFUKsgamv9XGKmkWggnAxsCrwOE5467qVbYxVBExhWxQWKE8/5J0vqTuEbGEbJB6Q3nPLVLWOGBcWl9GdmvkZ0j6MjCrWDBlZmZmTRc0z8OR64srIuLsnPXZ1BNHRMQHZFMn5Ke/DGzXwLHeJmvtKllrD0oHOJUsAlxSLGMz6Qn8soWOZWZm1rFFUPPJJ61di7Jr9YAqIh5t4ePd1ZLHMzMz69CiYzzLr9UDKjMzM2vPmqfLb3XngMrMzMzKJiKoLvF5fW2ZAyozMzMrH3f5mVlziHofSWlm1kEERHX7/xx0QGVWbmWa6806loj2/wvf2qesy6+6eMY2zgGVmZmZlVWNW6jMzMzMVl14DJWZNQePoTKzDi2C6uUOqMzMzMxWnVuozKw5ZM/9NmsaSa1dBbNVEkBNTftvqXdAZWZmZuUTQfVy3+VnZmZmtsrC81CZWXMQ7qqx5uDryNqoDhJQeXCHmZmZlU+a2LPQUgpJwyQ9L2mOpFH1bN9Y0r2SHpf0lKR9U3onSeMlPS3pWUlnpPR+Kf9sSbMk/SSnrHMlzZf0RFr2LVY/t1CZmZlZ2QRNn9hTUiVwFbAnMA+YLmlSRMzOyXYWMCEirpY0AJgCbAIcBqwREdtIWguYLelG4L/AqRHxmKR1gJmS7sop8/KIuKTUOrqFyszMzMongqiuKbiUYDAwJyJejojlwE3AgflHAtZN692ABTnpXSVVAWsCy4GlEfFGRDyWVTHeB54F+qzqaTqgMjMzs7KJgOrlNQWXEvQBXs95PY/PBj/nAkdLmkfWOnVSSr8ZWAa8AbwGXBIR7+TuKGkT4MvAoznJI1PX4RhJPYpV0AGVmZmZlVXU1BRcgJ6SZuQsJ67CYUYA4yKiL7AvcIOyiQAHA9VAb2BT4FRJm9XuJGlt4H+BkyNiaUq+Gtgc2J4sELu02ME9hsrMzMzKJ6KUMVSLI2JQge3zgX45r/umtFwnAMOyQ8bDkroAPYGjgDsi4hNgoaQHgUHAy5I6kQVTf42If6yscrxVuy7pWmBysRNwC5WZmZmVTQTULK8uuJRgOtBf0qaSOgNHApPy8rwG7A4gaUugC7Aope+W0rsCQ4DnlD1+4Drg2Yi4LLcgSRvlvDwYeKZYBd1CZWZmZuUTTb/LLyJWSBoJ3AlUAmMiYpak0cCMiJgEnApcK+mU7KgcHxEh6SpgrKRZZBO6jY2IpyR9BTgGeFrSE+lQv4iIKcBFkrZP5cwFvl+sjg6ozMosaP8T2ln5Rfg6srYqmmVizxToTMlLOztnfTawaz37fUA2dUJ++r9pYMbciDimsfVzQGVWbv4itGbh68japgj4pKakO/naNAdUZmZmVjYBdIAnzzigMiu3DvA5YmZWUHUHaKl3QGVWdu3/g8TMrCE1Actr2v/noAMqs7Krd8yjmVmH4S4/M2uybKoTs6bxdWRtVRDu8jMzMzNrCg9KNzOz1YhbqKxtCo+hMrPmIH8RmlkH5xYqMzMzsybIuvzaf0TlgMrMzMzKJnCXn5mZmVmTRLjLz8zMzKzJ3OVnZmZm1gRB8IkDKjMzM7NV11Hmoapo7QqYmZlZ+1V7l1+hpRSShkl6XtIcSaPq2b6xpHslPS7pKUn7pvROksZLelrSs5LOKFampE0lPZrS/y6pc7H6OaAyMzOzsqmd2LPQUoykSuAqYB9gADBC0oC8bGcBEyLiy8CRwB9T+mHAGhGxDTAQ+L6kTYqUeSFweUR8AXgXOKFYHR1QmZVZRE1rV8HaAV9H1pZVR+GlBIOBORHxckQsB24CDszLE8C6ab0bsCAnvaukKmBNYDmwtKEylT04czfg5rT/eOCgYhX0GKoC/vvxx8yZM4eKiopGzXVdE0HU04RZWdG4+LW65rMfoJKoaMRDUgOoqaccn1PLnVP3NSvYsu869exZ//Gz/8uNOZOGymncedT3hd18dfE5NfWcqljBnDlz6hy/Pf+/KYXPqWHNdU7NoZkm9uwDvJ7zeh6wU16ec4Gpkk4CugJ7pPSbyYKvN4C1gFMi4h1JDZW5PrAkIlbkpPcpVkEHVAXMnj178bbbbPNqa9djNdcTWNzalbB2wddSAU8B4688r7Wr0Rb4Oiru8y15sEUsv/OP8WrPItm6SJqR8/qaiLimkYcaAYyLiEsl7QzcIGlrspaoaqA30AP4l6S7G1l2UQ6oCoiIDVq7Dqs7STMiYlBr18PaPl9L1hx8Ha1+ImJYMxQzH+iX87pvSst1AjAsHfNhSV3IAuyjgDsi4hNgoaQHgUFkrVP1lfk20F1SVWqlqu9Yn+ExVGZmZra6mw70T3ffdSYbdD4pL89rwO4AkrYEugCLUvpuKb0rMAR4rqEyI+srvRc4NJV7HHBrsQo6oDIzM7PVWmopGgncCTxLdjffLEmjJR2Qsp0KfE/Sk8CNwPEpOLoKWFvSLLIgamxEPNVQmams04GfSppDNqbqumJ1VH2D1sxKJenEVejnNvsMX0vWHHwdWWtxQGVmZmbWRO7yMzMzM2siB1QdiKRqSU9IekbSRElrNUOZoyXtUWD7DyQd29TjWNuRd53dJql7M5c/V1LPtP5Bc5ZtLS/neqldNpG0fnqEyAeSrmztOpqVwl1+HYikDyJi7bT+V2BmRFyWs70qZyIzs1WSd52NB16IiGabQEnSXGBQRCzOPZa1TfX9G6Y7sb4MbA1sHREjW6gu/gy0VeYWqo7rX8AXJA2V9C9Jk4DZkiolXSxpenq45Pdrd5B0enq45JOSLkhp4yQdmtYvkDQ77XdJSjtX0s/S+vaSHknbb5HUI6XfJ+lCSf+R9IKkr7b0m2Fl8zBphmFJm0u6Q9LMdM19KaVvmK6HJ9OyS0r/v5R3lqQTW/EcrIVFxLKI+DfwcaF8krZKnxtPpM+V/in92PT6SUk3pLRNJN2T0qdJ2jilj5P0J0mPAhc1dJ2aFeOJPTsgZc8z2ge4IyXtQPYr8JX0xfVeROwoaQ3gQUlTgS+RTd2/U0R8KGm9vDLXBw4GvhQR0UA3z/XASRFxv6TRwDnAyWlbVUQMVvZ08HNY+cgAa6OUPXh0d1bebnwN8IOIeFHSTmQPLt0N+D1wf0QcnPapba34Tno8xJrAdEn/GxFvt/BpWPmtKemJtP5KRBzciH1/APwuIv6a5hGqlLQV2UNyd0mtmLWfVX8AxkfEeEnfIbvuap/P1jflr5Y0jfqvU7OCHFB1LLkfXP8i+6LbBfhPRLyS0vcCtq1tdSJ7wGR/sgBnbER8CBAR7+SV/R7Zr8nrJE0GJudulNQN6B4R96ek8cDEnCz/SH9nApus8hna6qD2OutDNrfLXZLWJrvWJmrlc83WSH93A44FiIhqsmsJ4MeSar9c+5Fdhw6o2p+PImL7Vdz3YeBMSX2Bf6QgaDdgYkQshjqfVTsD30zrNwAX5ZQzMQVTha5Ts4IcUHUsn/ngSh8ay3KTyFqR7szLt3ehgiNihaTBZC0Sh5JNltaYX3X/TX+r8XXZ1n0UEdsru+nhTuB/gHFkDxst6YtT0lCyIH7n1CJ6H9msx9aBpQD7nPTyuxHxt9RVtx8wJXeIQiPVfgZW0Ijr1CyXx1BZvjuBH0rqBCDpi2mA6F3At9OXJPV0+a0NdIuIKcApwHa52yPiPeDdnPFRxwD3Y+1Was38MdnsxR8Cr0g6DECZ2mtkGvDDlF6ZWjO7Ae+mYOpLZI+KsA4uIm6JiO3TMkPSZsDLEfF7skeDbAvcAxyWhiHkflY9RPZoEYBvkbXS55e/lIavU7OC3BJg+f5M1uX2mLLmq0XAQRFxh6TtgRmSlgNTgF/k7LcOcKuyh1EK+Gk9ZR8H/CkFZS8D3y7fadjqICIel/QU2VPgvwVcLeksoBNwE/Ak8BPgGkknkLVQ/pBsfN8PJD0LPA880hr1t9aj7G7OdYHOkg4C9oqI2XnZDgeOkfQJ8Cbw2zTu7jzgfknVwOPA8cBJwFhJp5F9rjX0+dPQdWpWkKdNMDMzM2sid/mZmZmZNZEDKjMzM7MmckBlZmZm1kQOqMzMzMyayAGVmZmZWRM5oDKzJpEUkv6S87pK0qI0Y36h/bZPjxpqaPsgSb9vzrqamZWLAyoza6plwNbpmXsAewLzS9hve6DegEpSVUTMiIgfN1MdzczKygGVmTWHKWSP/4BsEs8bazdI6ippjKT/SHpc0oHpQbajgSMkPSHpCEnnSrpB0oPADZKG1rZySVpb0lhJT0t6StIhaVb1cZKeSemntPRJm5nVckBlZs3hJuDINFP+tsD/b++OVbMIwigMv1/uIGAXSGEZDGoKeyGIvVXuQBB7RQ2IdtpJWtOkSWGTTgRNa6cR8QbsRUUsj8XMD0tISHD+EALv0+3sMrvTHXaHPR8n5x4B75PcAG4CL2h/oN4EdnuNyG6/dgVYT7JxaP4nwM8kq0lm9SLXgKUkV5KsAttntThJOomBStKwJAe0yqIN2tuqqVvAg6r6BOzTSo6Xj5lqL8nfI8bXga3J/X7Q6osuV9WrqroN/BpZgySNMFBJmpc94CWTz31dAXcmpbbLSb4dM8ef096sh6qrtJB2l9ZDKUnnwkAlaV5eA0+TfDk0/ha438u2qarrffw3rVT7NN4B92YHVbVYVZeAhSRvgMfA2sjDS9IIA5WkuUjyPclRvzl4RtszdVBVX/sxwAdgZbYp/YTpnwOLfQP6Z9perCVgv39K3AEezmUhkvQfKsl5P4MkSdKF5hsqSZKkQQYqSZKkQQYqSZKkQQYqSZKkQQYqSZKkQQYqSZKkQQYqSZKkQQYqSZKkQf8ARor9db+RThgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 708.661x255.118 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "mini_prediction = (model_mini.predict(x_test) > 0.5).astype(\"int32\")\n",
    "print('Classification Report LSTM Mini-Batch Model:\\n', classification_report(y_test, mini_prediction))\n",
    "\n",
    "plot_classification_report(classification_report(y_test, mini_prediction), title='LSTM Mini-Batch Classification Report', cmap='RdBu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 445
    },
    "id": "EqQEJ4pKUAwF",
    "outputId": "9e8ae9ec-1509-4f04-8aa1-01fce62522e7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report LSTM Stacked Layers Model:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.87      0.89      3745\n",
      "           1       0.88      0.93      0.90      3755\n",
      "\n",
      "    accuracy                           0.90      7500\n",
      "   macro avg       0.90      0.90      0.90      7500\n",
      "weighted avg       0.90      0.90      0.90      7500\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkcAAAD+CAYAAAAwGaFjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXgV1f3H8fcnYRVkFyuLihZFVERF3Ct139e6UPfaWv0V21ptq9WqxVq1bnWrrbbuu7bWjYpWRa0ruKCColRRFhUQUFEUSL6/P2YCk5Dc3JDcJDf5vJ5nntw5c2bmTDK593vPOXOOIgIzMzMzS5Q0dQHMzMzMmhMHR2ZmZmYZDo7MzMzMMhwcmZmZmWU4ODIzMzPLcHBkZmZmluHgyKwZkBSSvt0Axxkn6YcNUabmTtIISTMKePy/SPptZv1ESZ9IWiipZ/pznQKcd5KkEQ19XDPLn4Mja1YkTZO0cw3bfiPp/fRDaYaku9L0SWnaQkllkr7OrP9G0jFp8HFZlePtl6bfmKM81Z4z3VZUgYikcyTd2tTlqAtJwyWNkbRA0jxJL0k6tjHOHREnRMS5aTnaApcCu0ZE54j4NP35Xn3OIelGSb+vct4NI2JcfY5bw7nGZf435kr6p6Q1Gvo8dShL0fzvWOvj4MiKgqSjgSOBnSOiMzAMeByWfZh0TtOfAUZVrEfEH9JD/A84RFKbzGGPBt5ZmXNa3VX53eeTf2vgCeAp4NtAT+BEYI+GL12tVgc6AJOa4NwNaVR6L38b6Axc3JgnV8KfO9bs+Sa1YrEFMDYi/gcQER9HxLV12P9j4A1gNwBJPYBtgAdW5pySzgO2B65Kv4lflaZfLmm6pM8lvSxp+4qDSSpNa6L+J+mLdHv/qieVtF16jBHp+g8kvSVpvqSxktbK5N1F0tuSPkvLoDr8TrLnPC1TrsmSDkjT26U1Nhtn8vaW9JWk1dL1vSW9ltbuPCdpSCbvNEm/lvQ68KWkNun6zPRcUyTtVEOxLgJuiogLI2JuJF6OiEPqcg3ptm9Leir9Pc3N1DpK0mWSZqd/szckbZRuu1HS7yWtB0xJD7VA0hPp9mVNoZI6SrpE0gfpOf4rqWO67R5JH6fpT0vaME0/Hjgc+FV6Dz2Y+Z3tnL5uL+lPkmaly58ktU+3jVBSm3lKWv6P8q1Vi4gFwL+AoZnf0SBJj6V/7ymSDslsu1FJM+Nj6e/3qSr34TaSxqfXOF7SNplt4ySdJ+lZ4CvgFqr53zFrThwcWbF4AThK0i8lDZNUuhLHuBk4Kn19GHA/8M3KnDMizqByLdWodNN4kg+cHsDtwD2SOqTbfgGMBPYEugA/IPmwWEbS7sAdwEERMU7SfsBvgAOB1dJz3pHm7QX8EzgT6EVSO7ZtnX4jy/2P5AOrK/A74FZJa0TEYuBO4IhM3pHA4xExR9KmwPXAj0lqdv4KPFDxAZ7JvxfQDVgXGAVsERGrkgSr06oWRtIqwNbAvfW9hnTbucCjQHegH3Blmr4r8B1gvXS/Q4BPsweNiHeADdPVbhGxYzXnvhjYnCTg7gH8CihPt/0bGAj0Bl4BbkuPe236+o/pPbRPNcc9A9iK5J7aBBhO8veu8K203H2B44CrJXWv9reTIaknyT01NV3vBDxGcs/2Jvn/+LOkwZndDif5PfYCXqu4DiVfNB4GriC5By4FHk7PUeFI4HhgVeAYqv/fMWs2HBxZUYiIW4GTSD5MnwJmS/p1HQ9zHzBCUleSIOnmhj5nRNya9kdZGhGXAO2B9dPNPwTOjIgpaS3IxIjIfhAfTBJc7BERL6VpJwDnR8RbEbEU+AMwNP3WvicwKSLujYglwJ9IasjqLCLuiYhZEVEeEXcB75J8EAPcBIyUVFErdSTJt39IPvD+GhEvRkRZRNxEEnBulTn8FRExPSIWAWXp72SwpLYRMa2iZq6K7iTvTx810DUsAdYC+kTE1xHx30z6qsAgQOnvOe9zAihpJvoB8LOImJn+Hp6LiG/Scl0fEV+k6+cAm6T3YD4OB0ZHxOyImEMS9B2Z2b4k3b4kIsYAC1l+v1XnCkmfAXNJgpyT0vS9gWkRcUN6774K/IPknqzwcEQ8nV7HGcDWSmo+9wLejYhb0n3vAN4GssHejRExKd2+JM9rN2syDo6saETEbRGxM0kNxAnAuZJ2q8P+i0i+4Z4J9IyIZxv6nJJOVdIE9pmkBSTf6nulm/uT1G7U5OfA3RHxZiZtLeDytMlqATCPpOmsL9AHmJ4pa2TX60LSUVreNLYA2Kii3BHxIkkN1whJg0j6q1Q0R64FnFKxX7pv/7RsFbJlnJpe5zkkweadkrJ5K8wnqXnJu8NwrmsgqckR8JKSDvw/SMvzBHAVcHVanmsldcn3nKleJP2RVvjbKmlKvUBJc9/nLK8l61U1bw36AB9k1j+g8u/20zRorvAVSV+imvw0IroCQ1heiwbJ33HLKn/Hw0lqpipk/44LSe7FPtWUsaKcfavb16wYODiyopN+S74HeJ3kA7AubgZOAer01FYN54xsHiX9i35F0jTTPSK6AZ+xvB/QdJJmpZocDOwv6WeZtOnAjyOiW2bpGBHPkdSqLOuzlNbsrNCHqTZpLdR1JM1dPdNyv5kpNyS1R0eQ1FrcGxFfZ8p3XpXyrZLWHlSo9HuKiNsjYjuSD+QALqxapoj4CngeOKghriHtL/ajiOhD0gT454r+QhFxRURsDgwmaV77ZT7nzJgLfE31f9vvA/sBO5MEymtXFDn9GdXskzWL5PdUYc00rV4i4g3g9yTNcCL5Oz5V5e/YOSJOzOyWvdc6kzQfzqqmjBXlnJk9ZdUi1PcazArJwZE1R20ldcgsbZQ8jr+XpFUllUjag6QfyIt1PPZTwC4s73NSozzO+QmQHedmVWApMAdoI+kskr5FFf5GUvM0UIkhVfplzAJ2An4mqeJD6S/A6VreiberpIqmjoeBDSUdqORJsJ9S+Zt+dUqq/G7bA51IPqzmpOc4lhWDzluBA0gCpGxz5HXACZK2TK+pU8XvrLqTS1pf0o7peb8GFrG8b05VvwKOSft89Uz330TSndXkzXkNkg6WVFFLMj/NWy5pi7TsbYEv0zLVVJ5qRUQ5Sb+rSyX1SWuLtk6vcVWSZsZPgVVImkWzqt5DVd0BnClptbSP2VnUMbDP4SaSp/D2BR4C1pN0pKS26bKFpA0y+fdU8rBAO5K+Ry9ExHRgTLrv99P/1UNJAs2Hcpy7tus2a1IOjqw5GkPyoVmxnAN8TtIx+UNgAfBH4MRM35G8pH19Ho+IeXlkr+2clwPfU/IU2RXAWOARkuEBPiD5oM02J1wK3E3SMfhz4O9Axyrl+5AkQDpN0g8j4j6SmpU702aZN0kfZY+IuSS1TReQfPgOBGprKhxJ5d/t/yJiMnAJSU3NJ8DGVY+Tfgi+QhJUPJNJnwD8iKRpaj5JB99jcpy/fVreuST9o3oDp1eXMa0d2zFd3pM0D7iW5P6omre2a9gCeFHSQpImwZ9FMkZRF5IAbz7J3+xTkqfk6upUkqchx5M0N11I8v56c3rcmcBkkk7+WX8n6X+1QNK/qjnu74EJJDWWb5D8DX5fTb46i6Sz/eXAbyPiC5LO6YeRBOkfp9eQ7Vh/O3B2en2bk3bST/vN7U1SI/spSVC7d3p/1qTq/45Zs6Kkm4KZWW6SrgdmRcSZtWa2FkXJQKkz/Le31qJOg7KZWeskaW2SR783bdqSmJkVnpvVzCwnSeeSNOddFBHvN3V5zKx1krS7kgFKp0o6rZrta0l6XNLrSgYf7ZdJfyV9mnWSpBNqPZeb1czMzKw5UzII7zskD9TMIOnfNzLtb1iR5x7goYi4SdKOwLERcWT6EIEi4pv0Scs3gW0iosYnP11zZGZmZs3dcGBqRLwXy0fu369KnsEk8zECPFmxPSIWVwzKSvKQQa2xj4MjMzMza+76Uvnp3xlUHmgUYCJJ30hIhh5ZNTMMSH8lczxOBy7MVWsE7pCdU9f27aN3505NXQwrcp+UlbAk/D3E6qdThzZ8a7UeyeiJ1XWHkOo063BNXSqkuhylYY7jaypsWaoeZ/LkyXMjYrU6HaAeSrr0C5Z+nTNPLPp0EsnwJxWujbpNLg7JkBpXSToGeJpkCI0yWDYcyRAlI/L/S9K9EfFJTQdycJRDnw7tuWmLobVnNMvh/2JdXp9d1tTFsCK3x/Zrce0f6zqdoNmKVunYsep0L4VV9g1tNzggZ5bFr/zt64gYliPLTCrPANCPyqOwk9YGHQjLRnE/KCIWVM0j6U2SSaprnNjaX2fNCq1uX/DMauAbyYqXSkpzLnkYDwyUNCDtYH0Yy+d4TM4h9VIyETQkg8ten6b3k9Qxfd0d2A6YkutkDo7MCqxulehmZi2N6h0cpRMsjyKZieAtkkm6J0kaLWnfNNsIYIqkd0imxjkvTd+AZIT8iSRTSF2czi9YIzermZmZWcFIoqRtu3ofJyLGUGX6oIg4K/P6XqppKouIx4AhdTmXgyMzMzMrHImS/JrOmg0HR2ZmZlZQefYrajYcHJmZmVnBSKKkTdumLkadODgyMzOzApJrjszMzMyWkYMjMzMzs+Ua6Gm1xuTgyMzMzApGuEO2mVVR09xIZnURUd7URTBbOW5WMzMzM8uQKG3jZjUzMzMzIJlCyTVHZlaZm9XMrJVzcGRmZmZWQSWUuFnNzCqRmroE1gLI95EVK4FKXXNkZmZmBrjPkZmZmVllkpvVzKwyN4dYw/B9ZMWrpKS47l8HR2ZmZlYwkpCDIzMzM7Pliq0G3cGRmZmZFY6gtE1JU5eiThwcmZmZWcEkE8+65sjMzMwsJUrcrGZmZmaWEpS4Wc3MzMwsIRXfo/zFFcqZmZlZ0VFJ7iWvY0i7S5oiaaqk06rZvpakxyW9LmmcpH5p+lBJz0ualG47tLZzOTgyK7SIpi6BtQDh+8iKlCRKS0tyLnkcoxS4GtgDGAyMlDS4SraLgZsjYggwGjg/Tf8KOCoiNgR2B/4kqVuu8zk4Miswf6RZw/CdZMVLJcq55GE4MDUi3ouIxcCdwH5V8gwGnkhfP1mxPSLeiYh309ezgNnAarlO5uDIzMzMCqoBgqO+wPTM+ow0LWsicGD6+gBgVUk9K5VDGg60A/6X62QOjszMzKxgJPJpVuslaUJmOX4lTnUqsIOkV4EdgJlA2fJyaA3gFuDYiCjPdSA/rWZWaO4rYmatWl61Q3MjYliO7TOB/pn1fmnaMmmT2YEAkjoDB0XEgnS9C/AwcEZEvFBbYVxzZGZmZoWTPsqfa8nDeGCgpAGS2gGHAQ9UOo3US1r27NvpwPVpejvgPpLO2vfmczIHR2aFVmQjw1rzVGwTd5pVEFBSqpxLbSJiKTAKGAu8BdwdEZMkjZa0b5ptBDBF0jvA6sB5afohwHeAYyS9li5Dc53PzWpmZmZWOA00CGREjAHGVEk7K/P6XmCFmqGIuBW4tS7ncnBkZmZmBeWJZ82sEjeHWMPwfWTFqWIQyGLi4MjMzMwKyjVHZmZmZikJSh0cmZmZmSWEaNfGzWpmZmZmiSKsOSquUM4aVdfNt2Djv93EkOtvYY1DRq6wvd1qvRl04SVseNVf2eia6+i6xZYAqLSUdU75NRtd8zc2vvYG1jh0xX2t9dh1m414874/MPn+8/nlsXuusH3NNXryyF9O5eW7fsdj1/2Kvr27L0t/8fazGX/nObx277n86HsjGrnk1tw8+uijbDJkCBttuCEXX3TRCts//OAD9txjD4ZvsQW77borM2bMWLbt1ltvZeONNmLjjTbi1lvr9FS31ZNIgqNcS3PjmiOrXkkJa/3kZ0z5zS9ZPHcOG15xDfNfeI6vP/xgWZY+I49g3tNPMfvhB+iw5lqsf+75TDz6+/TYfgfUti1vnvhDStq3Z+Nrb+DTcU+w+JNPmvCCrCmUlIjLTzuCPU+8hBmfzOP5287ioade4633Zi3Lc+HJh3Dbw89xy4PPMWKLQfz+pIM49rd/46M5C9j+6PNYvGQpnTq259V7z+Whp17jozkLmvCKrKmUlZVx8s9/zkMPP0zfvn3Zfrvt2Gvvvdlggw2W5Tn99NP5/uGHc8QRRzBu3DjOPuss/n799cybN48/nHce/332WSSx7TbbsNdee9G9e/cmvKLWo0TQvsia1QpWWknXS5ot6c1a8v1c0lHp63MlvZ6OXvmopD5p+i8zo1q+KalMUo902zRJb6TbJmSOe46kmZn99kzTN5Z0Y6Guu6XovP4gvvloJt98/BGxdCmfPvUE3bfepkquoHSVVQBo06kTiz/9NE2Fkg4doaSEknbtiSVLKPvyq8a9AGsWtthoHf43fTbvz5zDkqVl3D32RfYZUXlg2g3W6cOTL70FwLjxb7PPiE0BWLK0jMVLlgLQvl0bSjwkQqs2Yfx41l13XQYMGEC7du343sEH89BDD1XK8/bbbzNihx0A2GGHHZZt/89jj7HjTjvRo0cPunfvzo477cRjjz7a6NfQail3rVFzrDkqZCh3I7B7rgyS2gA/AG5Pky6KiCERMRR4CDgLICIuioihafrpwFMRMS9zqO+m26tOWndZxX7pyJpExBtAP0lr1vcCW7K2PXvxzZzZy9YXz51Lu56rVcoz89ab6Lnjzgy95S7WG30+H/z5CgDmP/MU5V8vYtPb72WTW+7go3/cTdnCLxq1/NY89O3djRmfLP9XnfnJfPqsVvnb+uvvTGf/HTcHYP8dN6NL54706NoJgH6rd+flu37He/++mItv/LdrjVqxWbNm0bdfv2Xrffv2ZdbMSvOOsvHGG3P//fcDcP/99/PFF1/w6aefMmvWLPpV3XfWLKxxJM1qJTmX5qZgJYqIp4F5tWTbEXglnTOFiPg8s60TSSVEVSOBO+pZvAdJJq2zeug5YkfmPjaW1448lHfOOp11f3k6SHRafxBRXs5rhx/MxKMP51sHHUL7b63R1MVtMhHV3cZW4deX3c13Nl+fl+44m+03X58Zn8yjrKwcgBmfzGfzQ89mg/1O58h9tqF3jy5NXNqmE1He1EVo9v5w/vk888wzbLXVVvz3mWfo06cPpaWlTV2sVk+Cdm1Kci7NTVOXaFvg5WyCpPMkTQcOJ605ymxbhaQ26h+Z5AAelfSypOOrHH9U2kx3vaTs19UJwPbVFUjS8ZImSJqwYPHilbuqFmDJp3Npv1rvZevtevVi8adzKuXptduezHt6HAAL35qM2rWjTZeu9PzuTnz28niirIylny1g4aQ36TRwvcYsvjUTM2cvoN/qPZat9129O7PmzK+U56M5Czjk1KsZPvJ3nHXVPwH4bOGiFfJMmjqT7TYbWPhCW7PUp08fZmY6WM+cOZM+ffuukOfOu+7ihRde4Jzf/Q6Abt260adPn0qds2fOnEmfPn0ap+CGcLNaXa0BVPrEjYgzIqI/cBvJDLxZ+wDPVmlS2y4iNgP2AH4i6Ttp+jXAusBQ4CPgksw+s4Fq/zMi4tqIGBYRw7q1a7eSl1X8Fk55m/Z9+tJu9W+hNm3oucOOLHjh+Up5Fs/+hC6bbgZAh/5rUtKuHUs/W8Di2bPpsknSb6SkfQc6D9qARTOmN/o1WNObMOl9vr3m6qzdpxdt25RyyG5b8tC41yrl6dmt87IpVn79g7246f7/AtC3d3c6tG8LQLdVV2HbTQfyzrSPG/cCrNnYfNgwpk6dyrRp01i8eDH33nMPe+21V6U8c+fOpbw8qWG76KKLOOroowHYeZddePw//2H+/PnMnz+fx//zH3beZZdGv4bWrFTKuTQ3Tf202iKgQw3bbiOZfffsTNphVGlSi4iZ6c/Zku4DhgNPR8SyR6MkXUfSh6lCh/TcVpPycj7485UMOu9CKCllzqP/ZtEH0+h75DF8+e47LHjhOT687i8M+NkpfOuA7xERvHfJHwH45MF/JY/y//V6BMx5bCyL3n+vaa+nSbXeZrWysnJ+fuGtPPznX1BSUsJN9/+Xye/N4uwT9+flydN46KnX2GHY+px70vcggmdeeYefnp88Zj1owBr88ReHEiR9Fi69eSxvTp2Z83zWcrVp04ZLL7uMfffZh7KyMo46+mgGDx7M6NGj2Wyzzdh777155umnOeuss5In0rbbjj/96U8A9OjRg9NOP53tt9sOgNN/8xt69OiR63TWgCqa1YqJCtkfQtLawEMRsVEN208A+kXEmen6wIh4N319ErBDRHwvXe8KvA/0j4gv07ROQElEfJG+fgwYHRGPSFojIj5K850MbBkRh6XrBwG7RMQJucq/QdcucdPWVft4m9XN/5Wvw+tz3F/E6ueQ7dbkuotOa+piWAuwSseOL1fzAFPBfGvghnHUpXfnzHPRvhs1aplqU7CaI0l3ACOAXpJmAGdHxN+rZPs3cEtm/QJJ6wPlwAdANng5AHi0IjBKrQ7cl1bJtwFuj4hH0m1/lDSU5Gv7NODHmf2+Czy88ldnVgfNsMrYio98H1mRquhzVEwKFhxFRK3DIkfEB5I+ragxioiDcuS9kWR4gGzae8AmNeQ/srp0Se2BYcDPayufmZmZ1U8xNqs1dZ8jgNNIOma/20jnWxM4rWL4ADMzMyuciulDikmTB0cRMQWY0ojne5fGC8TMkncGs3rzjWRFqggnnm3y4MispZM/1MysFRNys5qZmZlZBbnmyMzMzGw59zkyMzMzy5BEu9LialYrrtKamZlZ0SmRci75kLS7pCmSpkpaYURUSWtJejydU3WcpH6ZbY9IWiDpoar7VVvevK/MzMzMrI4ElCr3UusxpFLgapJ5VAcDIyUNrpLtYuDmiBgCjAbOz2y7CKh2/MPqODgyMzOzgpGgbWlJziUPw4GpEfFeRCwG7gT2q5JnMPBE+vrJ7PaIeBz4It8yOzgyMzOzgmqAZrW+wPTM+ow0LWsicGD6+gBgVUk9V6q8K7OTmZmZWT7ybFbrJWlCZjl+JU51KrCDpFeBHYCZQNnKlNlPq5kVWEQ0dRGsBYgob+oimK0USbSpvelsbkQMy7F9JtA/s94vTVsmImaR1hxJ6gwcFBEL6l5i1xyZmZlZgdW3QzYwHhgoaYCkdsBhwAPZDJJ6SaqIa04Hrl/Z8jo4MjMzs4IR9e9zlE4WPwoYC7wF3B0RkySNlrRvmm0EMEXSO8DqwHnLyiA9A9wD7CRphqTdcp3PzWpmheZmNTNrxSqeVquviBgDjKmSdlbm9b3AvTXsu31dzuXgyMzMzAoqz6azZsPBkVmh5Tn6q1ku8n1kRUrkPwp2c+HgyMzMzAomaVZzcGRmZma2TKlrjswsy80h1jB8H1lxqnharZg4ODIzM7OCcbOamZmZWSVys5qZmZlZBTermZmZmWW4Wc3MzMysiiKrOKr73GqSuksaUojCmJmZWctTgnIuzU1eNUeSxgH7pvlfBmZLejYiflHAspmZmVmRE9AAU6s1qnyL2zUiPgcOBG6OiC2BnQtXLDMzM2sRlDSr5Vqam3yDozaS1gAOAR4qYHnMWp6Ipi6BtQDh+8iKlGppUivaZjVgNDAWeDYixktaB3i3cMUyazn8kWYNw3eSFa9ia1bLKziKiHuAezLr7wEHFapQZmZm1nI0v7qh3PKK5SStJ+lxSW+m60MknVnYopmZmVmxqxgEMtfS3ORb0XUdcDqwBCAiXgcOK1ShzFoU9xUxs1auVLmX5ibfPkerRMRLVWYXX1qA8piZmVkLkjyR1gwjoBzyDY7mSlqXtEegpO8BHxWsVGYtSZG9KVjzVGwfLmZZJUV2++YbHP0EuBYYJGkm8D5wRMFKZWZmZi2CaJ5NZ7nk+7Tae8DOkjoBJRHxRWGLZWZmZi1FsdV85vu02s8kdQG+Ai6T9IqkXQtbNLOWodjeFKy58n1kRUpJs1qupbnJ92m1H6TTh+wK9ASOBC4oWKnMzMysRahoViump9XyDY4qir4nydxqk/DXGDMzM8uDpJxLnsfYXdIUSVMlnVbN9rXSMRlflzROUr/MtqMlvZsuR9d2rnyDo5clPUoSHI2VtCpQnue+ZmZm1kolg0DWr1lNUilwNbAHMBgYKWlwlWwXk1TgDCGZ9uz8dN8ewNnAlsBw4GxJ3XOdL9/g6DjgNGCLiPgKaAscm+e+ZmZm1mqJUuVe8jAcmBoR70XEYuBOYL8qeQYDT6Svn8xs3w14LCLmRcR84DFg91wny/dR/q2B1yLiS0lHAJsBl+e5b9Fa2K4Dz/QbnFT7LUtVrcPWlFcZEbliNu1KQ6RXOmbtx4nkQMlgWtnS1FKYoPJs3snL8DXReNfUbcpMduixajXXEZVGz46KiUUlqlxJzvMn5V7xOFLl7z61H6dyZXBynMpV3rUew9eUoyz1u6ay2R/y8lOPJUWJSDq5lizfVyrJec9HBOXl5ZXWCVBJ5d9H9pjVKS8vr/z/V57+35Rmy1J7U0lZWVmmLL6mprimRqO8hnvrJWlCZv3aiLg2s94XmJ5Zn0FSE5Q1ETiQJD45AFhVUs8a9u2bqzD5BkfXAJtI2gQ4BfgbcDOwQ577F6X57bpxz9r7N3UxrMht+uIFxBuvNHUxrMj133tzhkU69m7FB019OzeI9NtGJq2ux2zIsjTUcXxNhSvLSlAEqn0apbkRMayepzoVuErSMcDTwEygLOceNcg3hFwaSQi6H3BVRFwNrPhV2MzMzKwKlS/NueRhJtA/s94vTVsmImZFxIERsSlwRpq2IJ99q8o3OPpC0ukko2I/rKQeuG2e+5qZmVmrFRDluZfajQcGShogqR1wGPBANoOkXlreTn06cH36eiywq6TuaUfsXdO0GuUbHB0KfAMcFxEfk0RdF+W5r5mZmbVmEbmXWnePpcAokqDmLeDuiJgkabSkfdNsI4Apkt4BVgfOS/edB5xLEmCNB0anaTXKd/qQj4FLM+sfkvQ5MjMzM6tZRL61Q7UcJsYAY6qknZV5fS9wbw37Xs/ymqRa5Tt9yFaSxktaKGmxpDJJn+V7ErPWLKj9W5FZbXwXWTFrgD5HjSrfp9WuImnfuwcYBhwFrFeoQpm1JMnD42b1E3k0PZg1Tw1Tc9SY8h7wICKmAqURURYRN1DLAEpmZmZmyTAE9e6Q3ajyrTn6Ku0d/pqkPwIfUVc25+8AABj6SURBVIfAyqxV8xd+M2vVApU1v6azXPINcI4ESkl6in9JMl7AQYUqlJmZVeZmNStqLbHmKCI+SF8uAn5XuOKYmZlZi5Ln4/rNSc7gSNIb5GgUSGe+NbMcSpBb1qzeSvKYnMqsuWqOT6TlUlvN0YEkAylNr5LeH/i4ICUyMzOzFqTlPa12GfBZRHyQXYDP0m1mZmZmubWwPkerR8QbVRMj4g1JaxekRGZmtgK5Wc2KlCJaXLNatxzbOjZkQcxaLH+mmVlrV978aodyqa1ZbYKkH1VNlPRD4OXCFMnMzMxajlomnW2GT7LVVnP0c+A+SYezPBgaBrQDDihkwczMzKwFiBb2tFpEfAJsI+m7wEZp8sMR8UTBS2ZmZmYtQPE9rZbvIJBPAk8WuCxmZmbWErXE4MjMzMxspUQQS5c0dSnqxMGRmZmZFVBAeVlTF6JOHByZmZlZ4QQOjsyssvDMatYAohk+7myWj4gglrhZzcwyAo8DafXn0MiKl5vVzMzMzJaLIBwcmVkl/spvZq2an1YzM7MCcJ8jK1pF2CG7trnVzKye3N/IGoLvIytekUw8m2vJg6TdJU2RNFXSadVsX1PSk5JelfS6pD3T9HaSbpD0hqSJkkbUdi7XHJkVmJBb1qzeJIdHVqQiiCWL63UISaXA1cAuwAxgvKQHImJyJtuZwN0RcY2kwcAYYG3gR0kxYmNJvYF/S9oiouZhu11zZGZmZoUT6dNquZbaDQemRsR7EbEYuBPYr+qZgC7p667ArPT1YOCJpCgxG1gADMt1MgdHZmbFwBVHVsSivDznkoe+wPTM+ow0Lesc4AhJM0hqjU5K0ycC+0pqI2kAsDnQP9fJ3KxmVmj+ULMGIN9IVqwiYGmtzWq9JE3IrF8bEdfW8UwjgRsj4hJJWwO3SNoIuB7YAJgAfAA8B+SsrnJwZGZmZgUTRD61Q3MjIldT10wq1/b0S9OyjgN2B4iI5yV1AHqlTWknV2SS9BzwTq7CuFnNzMzMCqfiUf769TkaDwyUNEBSO+Aw4IEqeT4EdgKQtAHQAZgjaRVJndL0XYClVTpyr8A1R2ZmZlY4DfC0WkQslTQKGAuUAtdHxCRJo4EJEfEAcApwnaSTSUKyYyIi0ifUxkoqJ6ltOrK28zk4MjMzswKKvMcyynmUiDEkHa2zaWdlXk8Gtq1mv2nA+nU5l4MjMzMzK6wiGyHbwZGZmZkVTgTlnlvNzMzMLBVBlNW/Wa0xOTgyMzOzgonAwZGZVVZOePg+q7fy8Ax9VqQiKF+ytKlLUScOjszMzKygXHNkLcZW6/Tk5N3Wo0TigddmcstzH1Ta/q2uHThj78F0X6Utn3+9lLP/9SZzvviGgat35ld7DKJT+zaUlwc3PjuN/0z+pImuwpra4N124JDLz6KktJRn/3YXYy+8ptL27v37cMxNl9CxWxdKSkv412kX8ua/xzH8+/uxyy9/vCxf3yGD+MNmezNjYs6x26wFe3TCm5z6l7spKy/nmN2345eH7F5p+4ez5/GjS27gs4WLKCsv59xjD2D34RuzeMlSRl15K6+8+wElKuHiEw7hO0Pq9GS31UNEUF7mp9UAkLQ7cDnJYE1/i4gLasj3J+CfEfG0pL+TzJQrkqG9j4mIhZIuA76b7rIK0DsiumWO0QWYDPwrIkalaeOANYBFabZdI2J2OojUVxFxfcNecctSIjh1j/X56W2vMvvzr7nhuOE8885cps39clmek3YayL/f+Igxr3/E5mt35/92/Da/u38SXy8pZ/T9k5g+fxG9OrfjxuO25IX/fcrCb4qrWrXBtOLWEJWUMPLq0Vy+yxHMn/Exp49/gNcfeIyP3pq6LM+eZ47i5bsf5um/3MoaG3ybUWNu5IwB2/HS7ffz0u33A9Bno/U58V/XturAKFrzjQSUlZXz86vv4OE//Jy+vbqz3c/OZ+8th7DBWn2W5bnwjoc5aPthHL/3Drz1wSz2P+sqpgzfmOsfeQaACdeczewFn7P/b6/kv5efTkmJJ4loFEXYrFaQO0NSKXA1sAcwGBgpaXA1+XoCW0XE02nSyRGxSUQMIRkGfBRARJwcEUMjYihwJfDPKoc6F3iaFR1esV86twokE9CdVE1eyxjcpysz5i1i1oJFLC0PHpv0Cd9Zb7VKeQas1okJ0+YB8PK0+cu2T5/3FdPnJzHp3IWLmf/VYrqv0rZxL8CahbWHD2X21A+Y+/50ypYsYfydDzJkv10r5YmADl06A9ChaxcWzFqxlnGLkfsy4c4HG6XMzVbrjo0Y/877rNunNwPWWI12bdtw8A7DeOiFiZXySOLzr5L3ns++WsQaPbsC8PaHHzFik0EA9O7Wha6dOvLyu5Vrwq2woqw859LcFCpsHg5MjYj3ImIxcCewXzX5DgIeqViJiM8BJAnoSPVvByOBOypWJG0OrA48mk/BIuIrYJqk4fldSuu02qrtmf3518vWZ3/xNaut2r5Snnc/WciI9XsDMGL91ejUvg1dOlYOggb36ULb0hJmzF9Ea9WaO2N377s686fPWra+YMZHdO+7eqU8D51zGVsesT/nT3+eUWNu4K6Tzl7hOMMO3Zvxd1SdRql1Sd4WW69ZcxfQb7Xuy9b79urOzE8XVMpzxhH7cOeTL7LuEb/mgLOu4tITDwNg4wH9eOiFiSwtK2Pax3N5deqHzJgzv1HL35pFJBPP5lqam0IFR32B6Zn1GWlaVdsCL2cTJN0AfAwMIqklym5bCxgAPJGulwCXAKfWUI4bJL0m6beq/M4yAdg+76uxal35n3fYbK3u3PTDLdl0re7M/vxrysuXx7M9O7fj7P025NwHJ7XqL71q1eFR7bYYuS/P33gvp/ffmqv2PJZjb7msUiCw9vChLP5qEbMm5ZxEu8XzXVS7u8e9xBE7b8P/br2Q+0aP4riLbqC8vJyjd9uWvr26s+1P/8Av/3o3W22wLqVuUms8AWWLl+Zcmpum7pC9BjAnmxARx6bNclcChwI3ZDYfBtwbERU9u/4PGBMRM6r5VnV4RMyUtCrwD5KJ5m5Ot80mCb5WIOl44HiAdl17r+x1Fb05X3xD7y4dlq33XrUDc774plKeuQsXc9q9rwPQsW0p3x3Ue1m/olXalXLpoUP5y5P/Y9LMzxuv4NaszJ/5Cd37L+8T0q3fGsyfWbnZbNvjDuXK3Y8G4P0XXqFNh/Z07tWDL+Z8CsAWh+3T6muNDPr06laptmfm3Pn07dmtUp6bxj7L/b//KQBbbbAuXy9ZwtzPF9K7Wxcu+vEhy/KN+MWFDOzbet/fG10RDgJZqNB5JtA/s94vTatqEdChamIa/NxJ0uyWdRiZJjVga2CUpGnAxcBRki5IjzEz/fkFcDtJU1+FDizvqF313NdGxLCIGNa2U9earq/Fe2vW5/Tv0ZE1unWgTYnYZcPVeeadSnEsXTu2XfZt9uht1+bBiUnzSZsSceHBmzDmjY948u3ZWOv1wfiJ9B64Nj3X7kdp27Zscdg+vP7AY5XyzPtwFoN2SuaK/NagdWnbof2ywEgSmx+yl/sb4Wa1YeutzdRZs5n28VwWL1nKPU9NYK+tNqmUp3/vHox77W0g6Wf09eIlrNZ1Vb76ejFffp18uXv8lcm0KS2p1JHbCiug6JrVClVzNB4YKGkASVB0GPD9avK9BXwbGJc2e60bEVPT1/sCb1dklDQI6A48X5EWEYdnth8DDIuI0yS1AbpFxFxJbYG9gf9kzrse8GyDXGkLVRbBxY9M4fKRm1JSIh56bRbvz/2SH+2wDm/P+pxn3p3LZmslT6hFBK99uICLHkn+XDsPXp1N1+xG145t2WtI8gZ07oOTePeThU15SU2nFX+mlZeVcdeos/jp2JspKS3luevv5qPJ77LP707mgwlv8PqD/+Efp/yeI667gJ1OPo6I4KZjlreSD/zOlsyb/hFz35+e4yzWGrQpLeWyEw9jnzMvp6ysnKN33ZbBa/Vh9M0PsNl6a7H3VptwwQ+/x/9dcStX3vc4Elz3i2OQxJzPPmefM66gpET06dmNv5/6g6a+nNYlgvJm2HSWi6JAo65K2hP4E8mj/NdHxHnV5Nke+HFEHJH2H3oG6ELycTIRODHTSfscoENEnFbD+Y4hCY5GSepE8vRa2/T8/wF+UdEcJ+kVYJeI+DTXNXTuu15sdOKf63ztZlmb3n0B8cbU2jOa5dB/78055Se7NXUxrAXouMePX46IYY11viHf6hUPHL1XzjwD/nhzo5apNgXrcxQRY4AxteR5RtL5krpFxAKSDto15T2nlmPdCNyYvv4S2Ly6fJI2BSbVFhiZmZlZ/QXF1+eoqTtkA5wCrAksqC1jA+kF/LaRzmVmZta6RVC+ZElTl6JOmjw4iogXG/l8j9Wey8zMzBpEeG41MzMzsww3q5mZmZktExGUFdncag6OzMzMrHDcrGZmVbX22dStYfgusqIVEGXFdQc7ODIrsKBVjwNpDaRQY9KZFVrSrFZWe8ZmxDPvmZmZWUGVl0XOJR+Sdpc0RdJUSSsMCC1pTUlPSnpV0uvpYNRIaivpJklvSHpL0um1ncs1R2aF5i/8ZtaKRQP0OUonpL8a2AWYAYyX9EBETM5kOxO4OyKukTSYZCDqtYGDgfYRsbGkVYDJku6IiGk1nc/BkZlZEXCzmhWtCMoW17tD9nBgakS8ByDpTmA/IBscBckUZABdgVmZ9E7pvKsdgcXA57lO5uDIzMzMCqdhnlbrC2RnoJ4BbFklzznAo5JOAjoBO6fp95IEUh8BqwAnR8S8XCdznyOzAitxd2xrACXyfWTFKYDy8si5AL0kTcgsx6/EqUYCN0ZEP2BP4JZ0UvvhQBnQBxgAnCJpnVwHcs2RmZmZFU4EZYtrfVptbkQMy7F9JtA/s94vTcs6Dtg9OWU8L6kDyXyq3wceiYglwGxJzwLDgPdqOplrjszMzKxgIh3nKNeSh/HAQEkDJLUDDgMeqJLnQ2AnAEkbAB2AOWn6jml6J2Ar4O1cJ3NwZGZWBORmNStWDRAcRcRSYBQwFniL5Km0SZJGS9o3zXYK8CNJE4E7gGMieZLhaqCzpEkkQdYNEfF6rvO5Wc2s0PyZZmatWQMNAhkRY0gez8+mnZV5PRnYtpr9FpI8zp83B0dmZmZWMAF5D/TYXDg4MjMzs8KJ8MSzZmZmZhUiaIhBIBuVgyMzMzMrqCh3cGRmZmaWiPwnl20uHByZmZlZwURAee2DQDYrDo7MzMyscMJPq5lZFUFxvSlY85SMZWdWjPIeBbvZcHBkVmCBx4G0+iuujxaz5SJgiTtkm5mZmSUCKLKKIwdHZgVXZG8KZmYNrazImoUdHJmZFQH3ObJiVR6wuLy47l8HR2YF5v5G1hB8H1kxc7OamVUi5JY1qzfJ4ZEVpyDcrGZmZmZWwR2yzcysMFxxZEUq3OfIzFbgDzVrAPKNZEXMNUdmZmZmqaRZrbiiIwdHZmZmVjCBm9XMzMzMlolws5qZmZlZJW5WMzMzM0sFwRIHR2ZmZmaJYhznqKSpC2BmZmYtV8XTarmWfEjaXdIUSVMlnVbN9jUlPSnpVUmvS9ozTT9c0muZpVzS0Fzncs2RmZmZFUxDDAIpqRS4GtgFmAGMl/RAREzOZDsTuDsirpE0GBgDrB0RtwG3pcfZGPhXRLyW63yuOTIrsHLPrGYNoLzI+myYZZVF7iUPw4GpEfFeRCwG7gT2q5IngC7p667ArGqOMzLdNyfXHOXQljLWYD6UqE6j00aUU93noUrqFotGeXk1BwEp/+MEAdVF7L6mRrsmdetMpw0HVLNjei0r7Ko6jaqdfGZWc5w6TlQa1X74ijodxteUoyzpwaoeJ8/CLOnQjqkLFmf2q9v/DQTl1fzflJSIulxIRDnV/VpL6vh/U17N/42vCZrbNTWEBhoEsi8wPbM+A9iySp5zgEclnQR0Anau5jiHsmJQtQJV/+ZhAJLmAB80dTmauV7A3KYuhLUIvpesIfg+qt1aEbFaY51M0iMkf5dcOgBfZ9avjYhrM8f4HrB7RPwwXT8S2DIiRmXy/IIkrrlE0tbA34GNIqI83b4l8LeI2Li2MrvmKIfGvHmKlaQJETGsqcthxc/3kjUE30fNT0Ts3gCHmQn0z6z3S9OyjgN2T8/5vKQOJEHZ7HT7YcAd+ZzMfY7MzMysuRsPDJQ0QFI7kkDngSp5PgR2ApC0AUlt1Jx0vQQ4hDz6G4GDIzMzM2vmImIpMAoYC7xF8lTaJEmjJe2bZjsF+JGkiSQ1RMfE8r5D3wGmR8R7+ZzPfY6sXiQdn20XNltZvpesIfg+sobg4MjMzMwsw81qZmZmZhkOjloRSWXp0OlvSrpH0ioNcMzRkqobS6Ji+wmSjqrveax4VLnPHpTUrYGPP01Sr/T1woY8tjW+zP1SsawtqWc6DcRCSVc1dRmt9XGzWisiaWFEdE5f3wa8HBGXZra3STu9ma20KvfZTcA7EXFeAx5/GjAsIuZmz2XFqbq/oaROwKbARiTj1IyqdueGL4vfAw1wzVFr9gzwbUkjJD0j6QFgsqRSSRdJGp9O3Pfjih0k/VrSG5ImSrogTbsxHZwLSRdImpzud3Gado6kU9PXQyW9kG6/T1L3NH2cpAslvSTpHUnbN/YvwwrmeZKRbZG0rqRHJL2c3nOD0vTV0/thYrpsk6b/K807SdLxTXgN1sgi4suI+C+VBwVcgaQN0/eN19L3lYFp+lHp+kRJt6Rpa0t6Ik1/XNKaafqNkv4i6UXgjzXdp9a6eBDIVkhSG2AP4JE0aTOSb2fvpx9Cn0XEFpLaA89KehQYRDLk+pYR8ZWkHlWO2RM4ABgUEVFDU8rNwEkR8ZSk0cDZwM/TbW0iYriSWZTPpvph362IKJkocieSUWoBrgVOiIh3lYxU+2dgR+AK4KmIOCDdp6IW4QcRMU9SR5JJJv8REZ828mVY4XWUVDEJ6PsRcUAd9j0BuDwibkvHvimVtCHJBKTbpLWLFe9VVwI3RcRNkn5Act/tn27rl+Yvk/Q41d+n1oo4OGpdsm9Cz5B8aG0DvBQR76fpuwJDKmqDSCbvG0gSrNwQEV8BRMS8Ksf+jORb3t8lPQQ8lN0oqSvQLSKeSpNuAu7JZPln+vNlYO2VvkJrDirus74k45E8Jqkzyb12j5bPJdY+/bkjcBRARJSR3EsAP5VU8UHZn+Q+dHDU8iyKiKErue/zwBmS+gH/TAOaHYF7ImIuVHqv2ho4MH19C/DHzHHuSQOjXPeptSIOjlqXFd6E0jeAL7NJJLU7Y6vk2y3XgSNiqaThJDUF3yMZrKsu37a+SX+W4fuy2C2KiKFKOvyPBX4C3AgsyPdDUNIIkoB867SmchzJaLfWiqXB8tnp6g8j4va0OWwvYEy2G0AdVbwHllCH+9RaLvc5sqrGAidKagsgab20c+RjwLHpBx7VNKt1BrpGxBjgZGCT7PaI+AyYn+lPdCTwFNZipbWMPyUZtfYr4H1JBwMoUXGPPA6cmKaXprWMXYH5aWA0CNiq0S/Amp2IuC8ihqbLBEnrAO9FxBXA/cAQ4Ang4LSpP/te9RzJlBMAh5PUnlc9/ufUfJ9aK+Jv6FbV30iatV5RUq00B9g/Ih6RNBSYIGkxMAb4TWa/VYH7lUz0J+AX1Rz7aOAvaYD1HnBs4S7DmoOIeFXS68BIkg+kaySdCbQlmeNoIvAz4FpJx5HUHJ5I0h/uBElvAVOAF5qi/NZ0lDyV2AVoJ2l/YNeImFwl2yHAkZKWAB8Df0j7qZ0HPCWpDHgVOAY4CbhB0i9J3tdqev+p6T61VsSP8puZmZlluFnNzMzMLMPBkZmZmVmGgyMzMzOzDAdHZmZmZhkOjszMzMwyHByZWb1ICkm3ZtbbSJqTjpSea7+h6XQxNW0fJumKhiyrmVk+HByZWX19CWyUzoEGsAswM4/9hgLVBkdKZkefEBE/baAympnlzcGRmTWEMSRTOEAy4OMdFRskdZJ0fTp7+quS9ksnCR0NHJrOqH6opHMk3SLpWeAWSSMqap8kdZZ0g6Q30lnVD0pH075R0ptp+smNfdFm1jI5ODKzhnAncFg6QvoQ4MXMtjOAJyJiOPBd4CKSkYfPAu5Kp4K4K807GNg5IkZWOf5vgc8iYuOIqJgiYijQNyI2ioiNgRsKdXFm1ro4ODKzeouI10mmnRlJUouUtStwmqTXgHEkE8iuWcOhHoiIRdWk7wxcnTnffJIpaNaRdKWk3YHP63MNZmYVHByZWUN5ALiYTJNaSsBBmQlD14yIt2o4xpc1pK8gDZA2IQm4TiCZF9DMrN4cHJlZQ7ke+F1EvFElfSxwUjqRMZI2TdO/IJmwOB+PAT+pWJHUXVIvoCQi/gGcCWxWn8KbmVVwcGRmDSIiZkREdY/en0vSx+h1SZPSdYAngcEVHbJrOfzvge5p5+uJJH2X+gLj0ua6W4HTG+RCzKzVU0Q0dRnMzMzMmg3XHJmZmZllODgyMzMzy3BwZGZmZpbh4MjMzMwsw8GRmZmZWYaDIzMzM7MMB0dmZmZmGQ6OzMzMzDL+HyHn3VThA0uxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 708.661x255.118 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "stack_layer_prediction = (model3L.predict(x_test) > 0.5).astype(\"int32\")\n",
    "print('Classification Report LSTM Stacked Layers Model:\\n', classification_report(y_test, stack_layer_prediction))\n",
    "\n",
    "plot_classification_report(classification_report(y_test, stack_layer_prediction), title='LSTM Stacked Layers Classification Report', cmap='RdBu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 445
    },
    "id": "tC135dMAaj1E",
    "outputId": "efe098bd-25a9-4198-f289-f472d5e5c121"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report Bidirectional LSTM Model:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.89      0.91      3745\n",
      "           1       0.90      0.92      0.91      3755\n",
      "\n",
      "    accuracy                           0.91      7500\n",
      "   macro avg       0.91      0.91      0.91      7500\n",
      "weighted avg       0.91      0.91      0.91      7500\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAD+CAYAAADMDIqpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgcVdn+8e89k4RA9gUCWdhkDQjBBBAUiawRZOcFIjsqggRfQfwBsgoiCCjKor4oYVVZVRAiYQ0iiySBJGwSYliyQXYIAUwy8/z+qDOhZzLp7kmmZ9I99+e66krXqVNVp3oq3U+fc+ocRQRmZmZmVlhVaxfAzMzMrFw4cDIzMzMrkgMnMzMzsyI5cDIzMzMrkgMnMzMzsyI5cDIzMzMrkgMnqyiSfivpgjzbQ9JmxeRt5nK9Kmloic+xcbq+dqU8T0uR9LakvUp07N0kvZGzvqWkCZIWSfpeqe4NST+S9PvmPq6ZtRwHTlZW0pfpJ5I+krRA0kOSBtRtj4hTIuLSYo7VlLxNLOMtkn7S4FzbRMSY5j5XU+QLRNIX+lvpfZ0u6a6U/mpK+0hSjaRPc9Z/JOmEFKxd0+B4B6X0W/KUp6ukX0p6Nx3vP2m9d7NeeCMi4umI2DIn6f8BT0ZEl4i4tjnuDUlDJU1vcN6fRsS3Vue4KznXCenv85GkDyVNlPT15j5PE8ryz9Y4t1lLcOBk5eiAiOgMbAC8D1zX3CeolFqbYkg6HjgW2Cu9r0OAx2F5wNc5pT8NjKhbj4ifpkP8BziiwXt2PDA5zzk7pHNsAwwDugK7APOAnZr1AouzEfBqK5y3OT2X/k7dgV8Dd0rq3pIFaEv/b6ztcuBkZSsiPgXuBQbWpTWs7ZH0Q0mzJM2UdFLu/rl562oHJJ0t6T3gZklVks5JNSHzJN0tqWfO/l+W9KykhZKmpV/aJwNHA/8v/fr/W8q7vLZH0lqpZmVmWn4paa0G5fiBpNmp7CfmnHN/SS+lWoVpki5uhrdyR2B0RPwnva/vRcSNTdj/PeBlYN9Uxp7ArsADefY5DtgQOCQiXouI2oiYHRGXRsSohpkl7STpufRez5J0fQq+UOaa9H59KOllSdumbftJei01wc2QdFZKX14bJOkJ4KvA9elvtkUj99FBypryPkz3w7CUfqKk19Pxp0r6TkrvBPwd6JtTQ9dX0sWS7sg57oGpVm+hpDGSts7Z9raksyRNkvSBpLskdSz0x4iIWuB2oBOweTrWWpKuTrV77ytrilw7971QVoM4N5336JxydJN0m6Q5kt6RdL6kqrTtBEnPpPd/HnAX8Ftgl3TNCwuV16zcOHCysiVpHeBI4PmVbB8GnAXsTfYFUqi/zPpAT7Lah5OB04GDgd2BvsAC4IZ07I3IvhivA9YFBgETUsDxB+DKVCtzQCPnOQ/4Ytpne7IalvMblKMb0A/4JnCDpB5p22KyoKM7sD9wqqSDC1xXIc8DxykLModIql6FY9yWygVwFHA/8N88+fcCHo6Ij4o8fg1wBtCbrGZqT+C7ads+wFeALcjetyPIaq4AbgK+ExFdgG2BJxoeOCL2oH5tWr2aMkk7pev7Idn7/hXg7bR5NvB1shqzE4FrJH0hIhYDXwNm5tTQzWxw3C2APwHfJ7uHRgF/qwsIkyPIauQ2AbYDTij0RqW/34nAUuCdlHxFen8GAZuR3VsX5uy2Ptl724+stvBGSXVNmdeRva+bkv1fOC4dv87OwFSgD3AMcAqp9isiWrTGy6wlOHCycvTX9Ev2A7Kg6KqV5DsCuDkiXklfZBcXOG4tcFFE/DciPiH7AjgvIqZHxH/T/ocra474BvBYRPwpIpZGxLyImFBk+Y8GLkk1LHOAH5M1ldVZmrYvTbUvHwFbAkTEmIh4OdXQTCL74t29yPM2KiLuIAsS9wWeAmZLOruJh/kLMFRSN7Iv1tsK5O8FzGpCGcdHxPMRsSwi3gb+j8+ueynQBdgKUES8HhGzcrYNlNQ1IhZExIvFX9Jy3wRGRsSj6X2fERH/TuV6KCL+E5mngEeA3Yo87pHAQ+m4S4GrgbXJauvqXBsRMyNiPvA3ssBnZb6Y/l98mo51TETMliSyHwJnRMT8iFgE/JQswM11Qbr3nwIeImt+rU75zo2IRem9/zn179eZEXFd+tt8UuS1m5UtB05Wjg5Ov2Q7AiOApySt30i+vsC0nPV3GsmTa05q/quzEfCX1IyyEHidrOajDzCArG/PqujboCzvpLQ68yJiWc76x0BnAEk7S3oyNZt8QBbcrXZn6oj4Q0TsRVajcgpwqaR9m7D/J2RftucDvSLimQK7zCPro1aU1Hz2oKT3JH1I9sXfO537CeB6strA2ZJulNQ17XoYsB/wjqSnJO1S7DlzrPRvLelrkp6XND/dI/tR/N+j3n2QmtimkdX61Hkv5/Xy+2Alnk//L3qQNZPWBXDrAusA43Pu5YdTep0F6cdFnbp7sjfQnhXv19wy5v4fM6t4DpysbEVETUT8mSyY+XIjWWaRfenV2bDQIRusTwO+FhHdc5aOETEjbftckcdpaCZZUJZbrpkrydvQH8m+FAdERDey/iQqct+CUi3XPcAksqatprgN+AFwR6GMwGPAvqkvUDF+A/wb2DwiugI/Iue605Nwg8n6u21B1qxGRIyNiIOA9YC/AncXeb5cjf6tlfVLu4+sdqdPClpG5ZSrSfdBqhkaAMxYhTIul5o/TwWOlbQDMBf4BNgm5z7uljqS1+nR4G9Rd0/OJau1a3i/5pax4XUWum6zsubAycpW6hR8ENkv7NcbyXI3cIKkgak/1EVNPMVvgctSfyYkrZvOB1k/pr0kHSGpnaRekuqaUd4n6w+yMn8Czk/H603W16SYYAOyJqn5EfFp6nvzjSZeU3tJHXOWdqmD7/6SuijrEP81sqfd/tXEYz9F1nRazFOOt5MFJPdJ2iqdt1fqoLxfI/m7AB8CH0naiiwwAEDSjqkmrj1ZH7BPgVpJHSQdLalbagr7kKw5tqluAk6UtGcqZ79Uhg7AWsAcYFl63/bJ2e99oFdqvmzM3cD+6bjtyYLO/wLPrkIZ60lNe78HLkw1Wb8j63+1HkC6hoY1ij9O79luZP227omImlTOy9L9sRFwJvnv1/eB/g36aplVDAdOVo7+Jukjsi/Cy4DjI2KFR8kj4u/AL8k6BE+hkY7BBfyKrHbnEUmLyDpR75yO/S5Zs8wPgPnABLKO3pB90Q5MzSJ/beS4PwHGkdXqvAy8mNKK8V3gklSeC2l6DcoostqHuuVisvfxR8C7wELgSuDUiGjSWDypn8/j6Uu7UN7/knUQ/zfwaCrDC2RNQ40FbGeRBYmLyIKAu3K2dU1pC8iakebxWb+3Y4G3U/PeKWT9y5okIl4gdfwm61f3FLBR6iv0PbK/wYJUvgdy9vs3WZA8Nd0LfRsc9w2yztTXkdXsHEA21MaSppZxJX4J7CdpO+Bssv8Dz6f34jFSv7nkvXQNM8l+FJxS14+LrP/bYrIO4P8kq/Ucmee8T5AN7fCepLnNdC1mawxFuFbVzKytUjai/R0R0b+1y2JWDlzjZGZmZlYkB05mZma2xpM0TNIbkqZIOqeR7RtJelzZoLFjJPVP6YOUDaD7atp2ZM4+m0j6VzrmXcX0zXNTnZmZma3R0phik8keQJkOjAWGR8RrOXnuAR6MiFsl7QGcGBHHKhtsNiLizdTXcDywdUQslHQ38OeIuFPSb4GJEfGbfGVxjZOZmZmt6XYCpkTE1PQAxZ3AQQ3yDOSzh4CerNseEZMj4s30eibZiP/rpiFA9iCbugvgVrLZIvJy4GRmZmZrun7UH2x1OvUHYgWYCByaXh8CdJHUKzdDGsalA9mgtr2AhTkDDjd2zBV4Jus8enTpHH3X7blCupo43GBjraFNPUZzHWdlLbO+ptJd07SFn7C0ptnGqLQ2qmOHarr18NRvtvreefPfcyNi3cI5m0dV1/7Bsk/z5olP5r1KNgZbnRujaZONQzZsyfWSTgD+QTZQa03dRkkbkI0hd3xE1GpVPuBx4JTXphv04plrz2vtYliZ2/3XE3lpmqfwstWz6479OPa0U1q7GFYBhg/qX2j6qeZV81/ab31I3ixLXvz9pxExJE+WGdSfCaI/DUbZT81whwJI6gwcFhEL03pXsmmhzouIuonh5wHdJbVLtU4rHLMxbqozMysLrrW08qWq6rxLEcYCm6en4DqQTT79QG4GSb0l1cU155IGak35/wLcFhF1/ZmI7Om4J4HDU9LxwP2FCuLAyazE/HVnzcH3kZUvrXbglGqERgCjyabYujsiXpV0iaQDU7ahwBuSJpNNxn5ZSj8C+ArZFFwT0lI3RdbZwJmSppD1ebqpUFncVGdmZmYlI4mq9qs/dWFEjCKbNio37cKc1/fy2RNyuXnuYCXzK0bEVLIn9ormwMnMzMxKR6KquOa4suDAyczMzEqqyH5MZcGBk5mZmZWMJKratW/tYjQbB05mZmZWQnKNk5mZmVlR5MDJzMzMrDjN9FTdmsKBk1mJrWRGGDOzNkG4c7iZNcXKJtMza4KorW3tIpitGjfVmZmZmRVJorqdm+rMzMzMCpKfqjOzpgj3crJm4PvIypkDJzMzM7NiqIoqN9WZWbEkz2tvq09VVa1dBLNVI1C1a5zMrGgOnMys7XIfJzMzM7NiSW6qM7Piub7JmofvJCtfVVWVc/86cDIzKwOV87VjbY0k5MDJzMzMrDiV9JCMAyczMzMrHUF1u8p5KtSBk5mZmZVMNsmva5zMzMzMiiCq3FRnZmZmVgRBlZvqzMzMzAqTKms4gsoJAc3MzGyNpKr8S1HHkIZJekPSFEnnNLJ9I0mPS5okaYyk/jnbHpa0UNKDDfa5RdJbkiakZVChcjhwMiux8KT21gzCN5KVKUlUV1flXYo4RjVwA/A1YCAwXNLABtmuBm6LiO2AS4DLc7ZdBRy7ksP/MCIGpWVCobI4cDIrOX/hWTNw4GRlTFXKuxRhJ2BKREyNiCXAncBBDfIMBJ5Ir5/M3R4RjwOLVv9KHDiZmZlZiTVD4NQPmJazPj2l5ZoIHJpeHwJ0kdSriGNflpr3rpG0VqHMDpzMzMqA65usXEkU01TXW9K4nOXkVTjVWcDukl4CdgdmADUF9jkX2ArYEegJnF3oJH6qzqzEwl951ix8H1m5KqpWaW5EDMmzfQYwIGe9f0pbLiJmkmqcJHUGDouIhflOGhGz0sv/SrqZLPjKyzVOZmZloJLm+rI2Jg1HkG8pwlhgc0mbSOoAHAU8UO80Um9p+TN65wIjCxZN2iD9K+Bg4JVC+7jGyazEVOyztmb5OHCyMiWgqnr17t+IWCZpBDAaqAZGRsSrki4BxkXEA8BQ4HJJAfwDOG15GaSnyZrkOkuaDnwzIkYDf5C0birmBOCUQmVx4GRmZmal00wDYEbEKGBUg7QLc17fC9y7kn13W0n6Hk0thwMnMzMzKylP8mtmRaucjwtrTfKdZGWqbgDMSuHAyczMzErKNU5mZmZmRZCg2oGTmZmZWWFCdGjnpjozMzOzwlzjZG3FI+Ne4azf3k1NbS0nDPsyPzxiWL3t77w/j1OuuZW5H3xEjy6dGPnDk+i/bg8A7nj0Oa64M3tq9Jyj9uOYvXdp8fLbmmGfXbflFz/8BlVV4ua/Ps1VN9d7mpgNN+jFjRedyLo9ujD/w8WccN7vmDF7AdtvMYDrzjuWrp3WpqamlituepB7HhnbSldhrW3CM09y25UXUVtbw1cPGc5BJ42ot33OzOn838U/4MMF8+jctTun/fRaevXpC8Dl3z2aKZNeYssdduT/XXdraxS/TROVFThVTt2ZNauamlq+f8OfuP/S03np/y7mnjFjef2dmfXynPv7ezl6z10Y+5sL+dE39ufCW/4CwPxFi7nsjw/yj1+ew9O/PIfL/vggCxYtbo3LsFZWVSV+dc4xHDDiGrY/7HyOHLYzW2/at16en51xBH946FkGH3kRl934AD85/TAAPv50CSdd8HsGHX4BXx9xDVefNZxunddujcuwVlZbU8PNl5/P2TfcztV/fpJnH76f6f+ZXC/PH35xKbt9/XCuvOcxDv3OGdx57RXLtx1w/Kl897JftXSxLakSrNWuKu9STkpWWkkjJc2WlHf4cknfl3Rcen1pmqF4gqRHJPVN6T9MaRMkvSKpRlLPtO1tSS+nbeNyjnuxpBk5++2X0j8v6ZZSXXelGDv5LT7Xdz022WBdOrRvx//sPoQHn59YL8+/353F7oO2BGD37bfkweey7Y+Of5U9d9ianl060aNLJ/bcYWseGf9qi1+Dtb4dt92U/0ybzVsz5rB0WQ13j/4XBwwdVC/P1pv25ckXXgdgzNh/c8DQHQB48933mfLubABmzVnInAWLWLdnl5a9AFsjTHllAusP2Jg+/TeiXfsO7LLvQYwb80i9PNOnvsm2O30JgG123JXxOdu33fnLrL1OpxYts+WQqK7Kv5STUoZ5twDD8mWQ1A44CfhjSroqIraLiEHAg8CFABFxVUQMSunnAk9FxPycQ301bW84QeA1dfulEUeJiJeB/pI2XN0LrGQz5y5c3uwG0K93D2bMqz9X4uc37c/9z7wEwP3PvsSiTz5l3ocfNbrvzLl551m0CtVvve5Mf/+z/6oz3l9A35x7A2DS5GkcvMdgAA7e4wt07bw2PbvV/5Ibss0mdGhXzX+mzSl9oW2Ns2D2LHqtv8Hy9V591mfB7Fn18my0xda88HjWDDz2ib/zyeKPWLRwQYuW0xqXNdVV5V3KSclKGxH/AOYXyLYH8GJELEv7fJizrRONTwc+HPjTahbvb2QTBNpquPxbh/P0y5P54mk/4emX36Rvr+5l9x+gJUR4Vvt8zr7mbr4yeEte+NNF7DZ4S6a/P5+amtrl29fv3Y1bfvJtvnXxyDb9XkZtbeFMbdjRZ17A6+Of55wj9+X1cc/Tc731qfLn0RpBgg7tqvIu5aS1O4d/CRifmyDpMuA44APgqw22rUNWi5XbKzCAR9Kkfv8XETfmbBuRmgHHAT+IiLqfH+OAc4ArGxZI0snAyQAD1uu56ldW5vr27s70OZ/9WpsxdwH9enWvn6dXd+664FQAPvrkU/76zxfp3nkd+vbuztOTJtfbd7fttmiZgtsaZcbshfTv89n/o359ejBzTv1agFlzFnLEWTcA0GnttThkz8F88NEnAHTp1JH7r/0+F95wHy+8PLXlCm5rlB7rbcC89z6rYZr3/nv0WG+Denl6rrc+Z/7i9wB8+vFiXnh8FJ26dmvRclrjRPk1x+XT2mHeBkC9uveIOC8iBgB/oH6ABHAA8EyDZrovR8QXgK8Bp0n6Skr/DfA5YBAwC/h5zj6zgfo9VD87/40RMSQihqzbrfMqXlb5G7LFxkyZOZu335vLkqXLuOepcez/xe3r5Zn7wUfUpl/BV931MMfvk/Uv2HvwNjz24mssWLSYBYsW89iLr7H34G1a/Bqs9Y179S0227APG/ftTft21Ryx7848OGZCvTy9undGyj5Uzz5pf269/58AtG9XzT0/H8EdDz7Lnx8bv8Kxre343Dbb8967bzF7xrssW7qE50bfz+Dd966X58MF85d/Ht1/0/UMPfjI1iiqrUS1lHcpJ61d4/QJ0HEl2/5ANgvyRTlpR9GgmS4iZqR/Z0v6C7AT8I+IeL8uj6TfkfWZqtMxndtWol11NdecehQHnP8rampqOX6fLzFwo75cctsDfGGLjfj6F7fnH5Pe4MJb/ooEX952c3753eEA9OzSiXOH78+X//dyAH70jf3p2aXtdsyMRluc24aamlq+/7M7eOjXZ1JVVcWt9/+T16bO5KJTD2b8a2/z4FMT2H3Illx6+uEQwdMvTuZ7l98BwP/ssyO7fWELenXvzHEHZkH5ty68iYmTp7XmJbWatnsXQXW7dpxwzqVcfurR1NbWMvSgIxmw2Zbc8+ur2GTg9gwZug+vj3s2e5JOYuvBO3PiuZct3//iEw9l5ttT+PTjxZy2zxBOvvhqtt91aOtdUBtT11RXKVTKPgOSNgYejIhtV7L9FKB/RJyf1jePiDfT69OB3SPi8LTeDXgLGBARi1NaJ6AqIhal148Cl0TEw5I2iIhZKd8ZwM4RcVRaPwzYOyJOyVf+wVtsFM9ce97qvQnW5n3lhpeYMP2/rV0MK3N7Dd6A407/bmsXwyrA8EH9xzfyMFXJrL/5NnHcL+7Om+eqA7dt0TKtjpLVOEn6EzAU6C1pOnBRRNzUINvfgdtz1q+QtCVQC7wD5AY2hwCP1AVNSR/gL6mavx3wx4h4OG27UtIgsh9qbwPfydnvq8BDq351Zk1RXtXQtmZSmTVnmNWptD5OJQucImJ4EXnekTSvrqYpIg7Lk/cWsiEOctOmAtuvJP+xjaVLWgsYAny/UPnMmoO/8KxZ+D6yMlVpTXWt3ccJsqfbNgDebKHzbQicUzcEgpmZmZVOpU250uqBU0S8AbzRgud7k5YL0szMmknlfPFYG+NJfs2sKSrn48Jak+8jK1dCbqozMzMzK4Zc42RmZmZWHPdxMjMzMyuSJDpUV05TXeVciZmZma2RqqS8SzEkDZP0hqQpks5pZPtGkh6XNEnSGEn9c7Y9LGmhpAcb7LOJpH+lY94lqUPBaymqtGZmZmarQEC18i8FjyFVAzeQzUs7EBguaWCDbFcDt0XEdsAlwOU5264CGhvf8WfANRGxGbAA+GahsjhwMjMzs5KRoH11Vd6lCDsBUyJiakQsAe4EDmqQZyDwRHr9ZO72iHgcWFS/XBKwB3BvSroVOLhQQRw4mZmZWUk1Q1NdPyB3hu/pKS3XRODQ9PoQoIukXnmO2QtYmDMgdmPHXPFaiimtma26tjyrvZlZkU11vSWNy1lOXoVTnQXsLuklYHdgBlDTbBeS+Kk6s1ILh062+qK2trWLYLZKJNGucHPc3IgYkmf7DGBAznr/lLZcRMwk1ThJ6gwcFhEL8xxzHtBdUrtU67TCMRvjGiczMzMrqdXtHA6MBTZPT8F1AI4CHsjNIKm3pLq45lxgZL4DRkSQ9YU6PCUdD9xfqCAOnMzMzKxkxOr3cUo1QiOA0cDrwN0R8aqkSyQdmLINBd6QNBnoA1y2vAzS08A9wJ6SpkvaN206GzhT0hSyPk83FSqLm+rMSizcy8mage8jK1d1T9WtrogYBYxqkHZhzut7+ewJuYb77raS9KlkT+wVzYGTmZmZlVSRzXFlwYGTWYmpyFFxzfJRlXtWWHkSxY8OXg4cOJmVXOV8YJiZNVXWVFc5n4MOnMzMzKykql3jZGbFqpyPC2tdvpOsPNU9VVcpHDiZmZWByvnasbbGTXVmZmZmRZOb6szMzMyK4aY6MzMzsyK5qc7MzMysCSqowqnpc9VJ6iFpu1IUxszMzCpPFcq7lJOiapwkjQEOTPnHA7MlPRMRZ5awbGZmZlbmBDTDVHVrjGIvpVtEfAgcCtwWETsDe5WuWGZmZlYRlDXV5VvKSbGBUztJGwBHAA+WsDxmFSc8qb01g/CNZGVKBZrpKrKpDrgEGA08ExFjJW0KvFm6YplVEn/hWTNw4GRlrJKa6ooKnCLiHuCenPWpwGGlKpSZmZlVjvKqU8qvqBhQ0haSHpf0SlrfTtL5pS2amZnVcX2Tlau6ATDzLeWk2Mqz3wHnAksBImIScFSpCmVWScJfedYsfB9Z+apW/qWcFNvHaZ2IeEH1o8JlJSiPmZk1QmX2q9ysTvbkXOXcv8UGTnMlfY70k0fS4cCskpXKrIJIFdQr0lpPBX3xWNtTVUG3b7GB02nAjcBWkmYAbwHHlKxUZmZmVhFE+TXH5VPsU3VTgb0kdQKqImJRaYtlZmZmlaKSmuqKfarufyV1BT4GrpH0oqR9Sls0s8pQOR8X1prkO8nKlbKmunxLOSm288VJacqVfYBewLHAFSUrlZmZmVWEuqa6SnmqrtjAqe6y9iObq+5V/EPazMzMiiAp71LkMYZJekPSFEnnNLJ9ozTm5CRJYyT1z9l2vKQ303J8TvqYdMwJaVmvUDmK7Rw+XtIjwCbAuZK6ALVF7mtmZmZtVDYA5moeQ6oGbgD2BqYDYyU9EBGv5WS7mqxy51ZJewCXA8dK6glcBAwhGx1gfNp3Qdrv6IgYV2xZiq1x+iZwDrBjRHwMtAdOLPYkZmZm1laJauVfirATMCUipkbEEuBO4KAGeQYCT6TXT+Zs3xd4NCLmp2DpUWDYql5NsTVOuwATImKxpGOALwC/WtWTlou5i5cyctzMNBz8Z3/YQtWKEVFvJvO6kaOrcsbzUXagvMeprc2t1MuOIupXaxau4gxqaxsri6+ppa6p28dz2L3n2o1cRdSbuHX5CONSvY7AhToFZ9ew4nEajh9V+Dj1K5GXX4maUBZfU56yrN41af57zJvySnbfpnyqyrlXCwwyGBFEzr26/DiqqrdfVYGqgdra2nrvR216XVVVXXRZAGpralYsi6+pRa+pxaioYch6S8qt9bkxIm7MWe8HTMtZnw7s3OAYE4FDyeKTQ4AuknqtZN9+Oes3S6oB7gN+EpF/Ru1iA6ffANtL2h74AfB74DZg9yL3L0vTFizh+/e929rFsDJ3wqfvU/Xvd1q7GFbmOh2yJ70227a1i2HWZIpA+WMRgLkRMWQ1T3UWcL2kE4B/ADOAmrx7ZM10M1IXpPvIHn67Ld8OxTbVLUsR2EHA9RFxA9ClyH3NzMysDVPtsrxLEWYAA3LW+6e05SJiZkQcGhE7AOeltIX59o2Iun8XAX8kaxLMq9jAaZGkc8lGC39IWd1y+yL3NTMzszYrIGrzL4WNBTaXtImkDsBRwAO5GST11mdt3+cCI9Pr0cA+knpI6kE2tNJoSe0k9U77tge+DrxSqCDFBk5HAv8FvhkR75FFa1cVua+ZmZm1ZRH5l4K7xzJgBFkQ9Dpwd0S8KukSSQembEOBNyRNBvoAl6V95wOXkgVfY4FLUtpaZAHUJGACWS3U7wqVpdgpV94DfpGz/i4F2gDNzMzMsuBo9UcwiohRwKgGaRfmvL4XuHcl+47ksxqourTFwOCmlqPYKVe+KGmspI8kLZFUI+mDpp7MzMxWTW8yZbkAABn9SURBVOHf5GZrrmbo47TGKPapuuvJ2hPvIRtA6jhgi1IVyqyS+AvPmkOBJ6TN1mDNU+O0pii2jxMRMQWojoiaiLiZ1Rg8yszMzNqIoDk6h68xiq1x+jj1Yp8g6UpgFk0IuszaNtcUmFlbFqimvJrj8ik2+DkWqCbr0b6YbDyEw0pVKLNK4hYWaw6+jaystbUap4ioG/b4E+DHpSuOmZmZVZQihxwoF3kDJ0kvk+eHTkRs1+wlMqswheYwMytGVXEToZqtkcrtybl8CtU4HUo2iNS0BukDgPdKUiIzMzOrIG3rqbprgA8i4p3cBfggbTMzMzPLrw31ceoTES83TIyIlyVtXJISmVUYt7BYc/BtZOVKEW2qqa57nm1rN2dBzMzMrELVlletUj6FmurGSfp2w0RJ3wLGl6ZIZmZmVjkKTPBbZk/cFapx+j7wF0lH81mgNAToABxSyoKZmZlZBYg29FRdRLwP7Crpq8C2KfmhiHii5CUzMzOzClBZT9UVOwDmk8CTJS6LmZmZVaK2FjiZmZmZrZIIYtnS1i5Fs3HgZGZmZiUUUFvT2oVoNg6czMzMrHQCB05mVrzyetDW1lRRZo9sm9WJCGKpm+rMzKwFOWyy8uWmOjMzM7PiRBAOnMysaG5iMbM2zU/VmVkTOGyy5uD7yMpWhXUOLzRXnZmtJs9qb83B95GVr8gm+c23FEHSMElvSJoi6ZxGtm8k6XFJkySNkdQ/Z9vxkt5My/E56YMlvZyOea2kgv/VHDiZlZy/8mz1FfF5brZmiiCWLsm7FCKpGrgB+BowEBguaWCDbFcDt0XEdsAlwOVp357ARcDOwE7ARZJ6pH1+A3wb2DwtwwqVxYGTmZmZlU6kp+ryLYXtBEyJiKkRsQS4EzioQZ6BQN1cuk/mbN8XeDQi5kfEAuBRYJikDYCuEfF8ZON93AYcXKggDpzMzMyspKK2Nu9ShH7AtJz16Skt10Tg0PT6EKCLpF559u2XXuc75gocOJmVmFtYrDn4NrKyFQHLluRfoLekcTnLyatwprOA3SW9BOwOzACavVe6n6ozMzOzkgmimFqluRExJM/2GcCAnPX+Ke2z80TMJNU4SeoMHBYRCyXNAIY22HdM2r9/g/R6x2yMa5zMzMysdOqGI1i9Pk5jgc0lbSKpA3AU8EBuBkm9JdXFNecCI9Pr0cA+knqkTuH7AKMjYhbwoaQvpqfpjgPuL1QQ1ziZmZlZ6aSn6lbvELFM0giyIKgaGBkRr0q6BBgXEQ+Q1SpdLimAfwCnpX3nS7qULPgCuCQi5qfX3wVuAdYG/p6WvBw4mZmZWQlF0WM15T1KxChgVIO0C3Ne3wvcu5J9R/JZDVRu+jhg26aUw4GTmZmZlVYFjRzuwMnMzMxKJ4Jaz1VnZmZmVoQIomb1m+rWFA6czMzMrGQicOBkZsXzrPbWHGrDd5KVqQhqly5r7VI0GwdOZmZmVlKucbI2YZ9dt+UXP/wGVVXi5r8+zVU313sKlA036MWNF53Iuj26MP/DxZxw3u+YMXsB228xgOvOO5aundampqaWK256kHseGbuSs1ilG7jv7hzxqwupqq7mmd/fxeif/abe9h4D+nLCrT9n7e5dqaqu4q/n/IxX/j6G6vbtOfr/fspGQz5P1AZ3/++PmfzU8610FdbaJjzzJLddeRG1tTV89ZDhHHTSiHrb58yczv9d/AM+XDCPzl27c9pPr6VXn74AXP7do5ky6SW23GFH/t91t7ZG8du0iKC2pnKeqivZyOGShkl6Q9IUSefkyfdLSV9Jr2+SNFHSJEn3piHTkXSNpAlpmSxpYYNjdJU0XdL1OWlj0vnr9lsvpY+QdFJprrpyVFWJX51zDAeMuIbtDzufI4ftzNab9q2X52dnHMEfHnqWwUdexGU3PsBPTj8MgI8/XcJJF/yeQYdfwNdHXMPVZw2nW+e1W+My1gjRhptYVFXF8Bsu4fqvncCPB+7NjsMPZIOtN6uXZ7/zRzD+7of46Rf256ajTmf4r38CwJe/fRQAl243jF/tfQyH/fw81IYn/mu7dxHU1tRw8+Xnc/YNt3P1n5/k2YfvZ/p/JtfL84dfXMpuXz+cK+95jEO/cwZ3XnvF8m0HHH8q373sVy1dbKuTmuryLeWkJIGTpGrgBuBrwEBguKSBjeTrBXwxIv6Rks6IiO0jYjvgXWAEQEScERGDImIQcB3w5waHupRslNCGjq7bLyJmp7SRwOmreYkVb8dtN+U/02bz1ow5LF1Ww92j/8UBQwfVy7P1pn158oXXARgz9t8cMHQHAN58932mvJu93bPmLGTOgkWs27NLy16ArRE23mkQs6e8w9y3plGzdClj7/wb2x20T708EdCxa2cAOnbrysKZ7wOwwcDNeeOJZwFYNGcenyz8kI2GbNeyF2BrhCmvTGD9ARvTp/9GtGvfgV32PYhxYx6pl2f61DfZdqcvAbDNjrsyPmf7tjt/mbXX6dSiZbb6oqY271JOSlXjtBMwJSKmRsQS4E7goEbyHQY8XLcSER8CpDlj1qbxH1nDgT/VrUgaDPQBHmkk7woi4mPgbUk7FXcpbVO/9boz/f35y9dnvL+Avuv2qJdn0uRpHLzHYAAO3uMLdO28Nj271f9wGrLNJnRoV81/ps0pfaHXUG23jgR69OvDgmkzl68vnD6LHv361Mvz4MXXsPMxB3P5tOcYMepm7jr9IgCmT3yd7Q7ci6rqanpt3J8NB3+eHgM2aNHyr0nacm3bgtmz6LX+Z3/7Xn3WZ8HsWfXybLTF1rzweNadYOwTf+eTxR+xaOGCFi2nNS4im+Q331JOShU49QOm5axPT2kNfQkYn5sg6WbgPWArstql3G0bAZsAT6T1KuDnwFkrKcfNqZnuAtX/1BkH7Fb01Vijzr7mbr4yeEte+NNF7DZ4S6a/P5+anF8O6/fuxi0/+Tbfunhkm26uatuhU2E7Dj+Q5265l3MH7ML1+53IibdfgySeHXk3C6e/x7nj/sYRv7yIqc+Op7bMfpk2J99F+R195gW8Pv55zjlyX14f9zw911ufqirPY79GCKhZsizvUk5au3P4BkC9qoiIODE19V0HHAncnLP5KODeiKjrZfZdYFRETG/k19jRETFDUhfgPuBY4La0bTZZYLYCSScDJwPQvu1W7c6YvZD+fXouX+/Xpwcz59T/9TZrzkKOOOsGADqtvRaH7DmYDz76BIAunTpy/7Xf58Ib7uOFl6e2XMFtjbJgxvv0GPBZ37ju/TdgwYz36+X50jeP5LphxwPw1vMv0q7jWnTu3ZNFc+Zxz5mXLs/3w2fuY/Zk30ttUY/1NmDee5/VMM17/z16rFe/9rHneutz5i9+D8CnHy/mhcdH0alrtxYtp61EhQ2AWapwfAYwIGe9f0pr6BOgY8PEFBjdSdaUl+socprpgF2AEZLeBq4GjpN0RTrGjPTvIuCPZM2HdTqmc68gIm6MiCERMUTtVihamzHu1bfYbMM+bNy3N+3bVXPEvjvz4JgJ9fL06t55efPB2Sftz633/xOA9u2quefnI7jjwWf582PjVzh2W9OWawreGTuR9TbfmF4b96e6fXt2POoAJj3waL0889+dyVZ7Zn1T1t/qc7TvuBaL5syj/dod6bBO9lDB1nt9mdply5j1+pQWv4Y1RVu+jz63zfa89+5bzJ7xLsuWLuG50fczePe96+X5cMF8alOTz/03Xc/Qg49sjaJaIwIqqqmuVDVOY4HNJW1CFjAdBXyjkXyvA5sBY1JT2uciYkp6fSDw77qMkrYCegDP1aVFxNE5208AhkTEOZLaAd0jYq6k9sDXgcdyzrsF8EyzXGmFqqmp5fs/u4OHfn0mVVVV3Hr/P3lt6kwuOvVgxr/2Ng8+NYHdh2zJpacfDhE8/eJkvnf5HQD8zz47stsXtqBX984cd2D2hfitC29i4uRp+U5ZudrwN15tTQ13jbiQ742+jarqap4deTezXnuTA358Bu+Me5lJf3uM+37wE4753RXsecY3iQhuPSFree+6Xm9OH30rURssnPEeNx97ZitfjbWW6nbtOOGcS7n81KOpra1l6EFHMmCzLbnn11exycDtGTJ0H14f92z2JJ3E1oN35sRzL1u+/8UnHsrMt6fw6ceLOW2fIZx88dVsv+vQ1rugtiaC2jJrjstHpep7Imk/4JdANTAyIi5rJM9uwHci4pjUX+lpoCvZV81E4NScDuMXAx0jotGhDXICpxGSOpE9Zdc+nf8x4My6Jj5JLwJ7R8S8fNdQtU7vaLflgU2+drNcJ3z6H6r+/U5rF8PKXKdD9mTHiy5p7WJYBRg+qP/4iBjSUufbbv3e8cDx++fNs8mVt7VomVZHyfo4RcQoYFSBPE9LulxS94hYSNZZfGV5Ly5wrFuAW9LrxcDgxvJJ2gF4tVDQZGZmZqsvqKw+Tq3dORzgB8CGwMJCGZtJb+CCFjqXmZlZ2xZB7dKlrV2KZtPqgVNE/KuFz/do4VxmZmbWLMJz1ZmZmZkVyU11ZmZmZkWJCGrKbD66fBw4mZmZWem4qc7MzFpaW560yMpcQNRUzh3swMmsxCrn48JaU9ue79HKWdZUV1M4Y5nwDIhmZmZWUrU1kXcphqRhkt6QNEXSCoNhS9pQ0pOSXpI0KQ3EjaQOkm6W9LKkiZKG5uwzJh1zQlrWK1QO1ziZlZxrCsys7Ypm6OMkqRq4AdgbmA6MlfRARLyWk+184O6I+I2kgWSDcG8MfDsrR3w+BUZ/l7RjRNQV6uiIGFdsWVzjZFZibmGx5uDbyMpWBDVLavMuRdgJmBIRUyNiCXAncFDDM5FN2wbQDZiZXg8EnsiKErPJBtxe5eldHDiZmZlZ6aQap3xLEfoBuTPFT09puS4GjpE0nay26fSUPhE4UFI7SZuQTck2IGe/m1Mz3QWSCk7L7sDJrMREwf+HZgVVFf48N1sjBVBbG3kXoLekcTnLyatwquHALRHRH9gPuF1SFTCSLNAaB/wSeBao661+dER8HtgtLccWOon7OJmZmVnpRFCzpOBTdXMjIl/z2Qzq1xL1T2m5vgkMy04Zz0nqCPROzXNn1GWS9CwwOeWbkf5dJOmPZE2Ct+UrqGuczMzMrGQijeOUbynCWGBzSZtI6gAcBTzQIM+7wJ4AkrYGOgJzJK0jqVNK3xtYFhGvpaa73im9PfB14JVCBXGNk1mJuYXFmoNvIytbzTAAZkQskzQCGA1UAyMj4lVJlwDjIuIB4AfA7ySdkZ2VEyIi0pN0oyXVktVS1TXHrZXS26djPgb8rlBZHDiZmZlZ6TTTAJgRMYqs03du2oU5r18DvtTIfm8DWzaSvpiso3iTOHAyMzOzkgkoepDLcuDAyczMzEonwpP8mpmZmRUjgmIHuSwLDpzMzMyspKLWgZOZmZlZYVH8RL7lwIGTmZmZlUwE1BYeALNsOHAyMzOz0gk/VWdmTVA5HxfWmiJ8J1m5Knp08LLgwMnMrAxUzteOtTURsNSdw83MzMwKC6CCKpwcOJmVnJtYzKyNq6mgz0EHTmYlVjkfF9aafB9ZuaoNWFJbOXewAyezEvOs9tYcfB9ZOXNTnZk1gb/ybPVJvo+sPAXhpjozMzOzYrhzuJmZmVmRwn2czKwp3MJizcG3kZUz1ziZmZmZFSFrqqucyMmBk5mZmZVM4KY6MzMzs6JEuKnOzMzMrGhuqjMzMzMrQhAsdeBkZmZmVliljeNU1doFMDMzs8pV91RdvqUYkoZJekPSFEnnNLJ9Q0lPSnpJ0iRJ+6X0DpJulvSypImShubsMzilT5F0rYoYot+Bk5mZmZVM3QCY+ZZCJFUDNwBfAwYCwyUNbJDtfODuiNgBOAr4dUr/dlaO+DywN/BzSXXxz2/S9s3TMqxQWRw4mZVYBdVQWyuqraA+Itb21ET+pQg7AVMiYmpELAHuBA5qkCeArul1N2Bmej0QeAIgImYDC4EhkjYAukbE8xERwG3AwYUK4j5OeXTsUMVm/TqlyTWbMm5vEI18yH0W4BZ5lKht5BjNVRZfU0tdU8ztRKdtNmns9EQjYZVQky4ju4SVvR9NOU5jn15q2sjnvqY8ZUkHa3icIguzbK0qFr/3br39VNWEezWC2toV79WqqqomDW8ftbWNvq9V1dXFlwWoralZIc3XxBp3Tc2hmQbA7AdMy1mfDuzcIM/FwCOSTgc6AXul9InAgZL+BAwABqd/a9Nxco/Zr1BBHDjl8ckHc+a+/ND177R2OdZwvYG5rV2INdmk1i5A+fC9lM+rY+Cum1q7FOXA91FhG7XkyeawZPSv453eBbJ1lDQuZ/3GiLixiacaDtwSET+XtAtwu6RtgZHA1sA44B3gWWDFKLdIDpzyiIh1W7sMazpJ4yJiSGuXw8qf7yVrDr6P1jwRUbDfUBFmkNUS1emf0nJ9k9RHKSKek9QR6J2a586oyyTpWWAysCAdJ98xV+A+TmZmZramGwtsLmkTSR3IOn8/0CDPu8CeAJK2BjoCcyStI6lTSt8bWBYRr0XELOBDSV9MT9MdB9xfqCCucTIzM7M1WkQskzQCGA1UAyMj4lVJlwDjIuIB4AfA7ySdQda16oSICEnrAaMl1ZLVKB2bc+jvArcAawN/T0tearzzpFlxJJ28Cu3QZivwvWTNwfeRlZoDJzMzM7MiuY+TmZmZWZEcOLUhkmokTZD0iqR7JK3TDMe8RNJeebafIum41T2PlY8G99nfJHVv5uO/Lal3ev1Rcx7bWl7O/VK3bCypV5o64yNJ17d2Gc1yuamuDZH0UUR0Tq//AIyPiF/kbG8XEctarYBWERrcZ7cCkyPismY8/tvAkIiYm3suK0+N/Q3TE1A7ANsC20bEiBYqiz8DrSDXOLVdTwObSRoq6WlJDwCvSaqWdJWksWmSxO/U7SDp7JxJEq9IabdIOjy9vkLSa2m/q1PaxZLOSq8HSXo+bf+LpB4pfYykn0l6QdJkSbu19JthJfMcaSReSZ+T9LCk8eme2yql90n3w8S07JrS/5ryvirp5Fa8BmthEbE4Iv4JfJovn6Rt0ufGhPS5snlKPy6tT5R0e0rbWNITKf1xSRum9Fsk/VbSv4ArV3afmtXxcARtkKR2ZBMlPpySvkD2q+6t9AX1QUTsKGkt4BlJjwBbkc0LtHNEfCypZ4Nj9gIOAbZKj3821jxzG3B6RDyVHiG9CPh+2tYuInZSNpv1RXw2VL6VKWWTcu4J1A13fSNwSkS8KWlnsgk49wCuBZ6KiEPSPnW1DydFxHxJawNjJd0XEfNa+DKs9NaWNCG9fisiDmnCvqcAv4qIPygb26da0jZkk73ummol6z6rrgNujYhbJZ1Edt/VzUvWP+WvkfQ4jd+nZoADp7Ym9wPqabIvtF2BFyLirZS+D7BdXS0S2USJm5MFMjdHxMcAETG/wbE/IPt1eJOkB4EHczdK6gZ0j4inUtKtwD05Wf6c/h0PbLzKV2hrgrr7rB/wOvCopM5k99o9+my+rbXSv3uQDTxHRNSQ3UsA35NU9yU6gOw+dOBUeT6JiEGruO9zwHmS+gN/TsHOHsA9ETEX6n1W7QIcml7fDlyZc5x7UtCU7z41Axw4tTUrfEClD4fFuUlktUKjG+TbN9+B0+BkO5HVMBwOjKBpv9L+m/6twfdlufskIgYpe/hgNHAa2QBzC4v9gpQ0lCxY3yXVcI4hGwXY2rAUSF+UVr8VEX9MTWz7A6NyuxY0Ud1nYBVNuE+tbXIfJ2toNHCqpPYAkrZIHTUfBU5MX4Y00lTXGegWEaPI5gTaPnd7RHwALMjpv3Qs8BRWsVLt5PfIRvP9GHhL0v8AKFN3jzwOnJrSq1PtZDdgQQqatgK+2OIXYGuciPhLRAxKyzhJmwJTI+JasqkytgOeAP4ndR/I/ax6lmyaDoCjyWrdGx7/Q1Z+n5oB/mVvK/o9WVPZi8qqo+YAB0fEw5IGAeMkLQFGAT/K2a8LcL+ySRUFnNnIsY8HfpuCr6nAiaW7DFsTRMRLkiaRzVp+NPAbSecD7YE7gYnA/wI3SvomWY3jqWT9706R9DrwBvB8a5TfWo+ypye7Ah0kHQzsExGvNch2BHCspKXAe8BPU7+4y4CnJNUALwEnAKcDN0v6Idnn2so+f1Z2n5oBHo7AzMzMrGhuqjMzMzMrkgMnMzMzsyI5cDIzMzMrkgMnMzMzsyI5cDIzMzMrkgMnM1stkkLSHTnr7STNSSPI59tvUJpiZ2Xbh0i6tjnLama2uhw4mdnqWgxsm+aUA9gbmFHEfoOARgMnZbPUj4uI7zVTGc3MmoUDJzNrDqPIpr2AbLDLP9VtkNRJ0sg0i/1Lkg5KE7JeAhyZZrY/UtLFkm6X9Axwu6ShdbVWkjpLulnSy2l2+8PSKOO3SHolpZ/R0hdtZm2PAyczaw53AkelkeO3A/6Vs+084ImI2An4KnAV2YjMFwJ3pekz7kp5BwJ7RcTwBse/APggIj4fEXXTagwC+kXEthHxeeDmUl2cmVkdB05mttoiYhLZVD3DyWqfcu0DnCNpAjCGbLLeDVdyqAci4pNG0vcCbsg53wKyaXs2lXSdpGHAh6tzDWZmxXDgZGbN5QHganKa6RIBh+VMzrphRLy+kmMsXkn6ClLwtD1ZMHYK2TyLZmYl5cDJzJrLSODHEfFyg/TRwOlp0mgk7ZDSF5FNDl2MR4HT6lYk9ZDUG6iKiPuA84EvrE7hzcyK4cDJzJpFREyPiMaGD7iUrE/TJEmvpnWAJ4GBdZ3DCxz+J0CP1BF8IllfqX7AmNQEeAdwbrNciJlZHoqI1i6DmZmZWVlwjZOZmZlZkRw4mZmZmRXJgZOZmZlZkRw4mZmZmRXJgZOZmZlZkRw4mZmZmRXJgZOZmZlZkRw4mZmZmRXp/wMydBRz2WZc4gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 708.661x255.118 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "lstm_bi_prediction = (model_Bi_LSTM.predict(x_test) > 0.5).astype(\"int32\")\n",
    "print('Classification Report Bidirectional LSTM Model:\\n', classification_report(y_test, lstm_bi_prediction))\n",
    "\n",
    "plot_classification_report(classification_report(y_test, lstm_bi_prediction), title='Bidirectional LSTM Classification Report', cmap='RdBu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 445
    },
    "id": "EeiUobHdUAwG",
    "outputId": "3d2a1377-a802-4e58-91a4-7901baa56ee8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report CNN + LSTM Model:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.91      0.90      0.91      3745\n",
      "           1       0.90      0.91      0.91      3755\n",
      "\n",
      "    accuracy                           0.91      7500\n",
      "   macro avg       0.91      0.91      0.91      7500\n",
      "weighted avg       0.91      0.91      0.91      7500\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAD+CAYAAAAnOzGqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xVVd3H8c93BlQEkZsSgor3IkNMUqvHIEvEzFv0eHksL11M0/KSpZaJoj6YWaaPdrFCJc1rWagkkYLmpQQVMUGFUJORROWmggkzv+ePvQbODDPnnIFzZubMfN+v135xztprr732zOac36y19lqKCMzMzMwsv6q2roCZmZlZJXDQZGZmZlYEB01mZmZmRXDQZGZmZlYEB01mZmZmRXDQZGZmZlYEB01mVnKSQtLOZSr7WEl/znn/cUnzJL0t6XBJf5J0fBnO+3NJ3y91uWZWORw0Wacl6X8kzUxftovSl+1/pX0Xpi/+I3Pyd0lpg9P7G9L7vXPy7Cyp5JOfNReESNpE0o8kLUzX8ZKkn6R9b+dsdZJW5bw/NucaT29U5ukp/cI89Rkg6dfp5/aWpOckXSSpe6mvvbGIuDkiRuUkjQOuiYgeEfGHiDgoIm7cmHNIOkHSw43Oe3JEXLwx5TZzrgslrU6/l2WSHpX00VKfpwV1uaktzm1WCRw0Wack6SzgJ8D/Av2B7YCfAoflZFsCXCSpOk9RS4BLNrAOF+YLTIp0HjAc2BvYAhgJPAmQgogeEdED+BdwSE7azen4F4DjGpV5fEpvrt59gMeAbsBHI2IL4ACgF7DTRl7PhtgeeLYNzltKt6XfUz9gGnBHa1dAUpfWPqdZpXHQZJ2OpC3JWidOjYjfR8Q7EbE6Iu6OiG/nZL0PeA/4Qp7ibgSGShpRxirn8xHgroh4NTIvRcTEFhw/A9hc0gcB0r+bpfTmnAW8BXwhIl4CiIhXIuL0iJjdOLOkgyU9JWmFpFdyA0VJm0m6SdKbqZVlhqT+ad8JkhaklqwXJR2bk/5wev1PYEfg7tRSs6mk6ZK+knOOr0qam8qZI+nDKf1cSf/MST8ipX8A+Dnw0frWn5R+g6RLGpU7X9ISSZMkbZOzLySdnLoNl0m6VpIK/TIiYg1wMzBQ0laprC1zWvVqJF1SH8inn8Ujkq6RtDy1+H0qpx7bpLotSXX9as6+CyXdmX7+K4CTge8CR6XrfrpQfc06GwdN1hl9lCwwuKtAvgC+D4yV1LWZPCvJWqsuLV31WuRvwFmSvi7pQ8V8MTfhN6xrbTo+vc/n08DvI6KuyPLfSeX3Ag4GTpF0eM75tgS2BfqSfXGvSt18VwMHpZasjwGzGhccETvRsBXtP7n7Jf03cGE6f0/gUODNtPufwH7p/BcBN0kaEBFzUz0eS2X2anxeSfsD44EjgQHAy8CtjbJ9liyoHZryHVjoByVpk1TXN4GlKfkGYA2wM7AnMAr4Ss5h+6Rr6QeMBX6fWgNJdVoIbAN8HvjfVPd6hwF3kv1ufk12L9+WrnuPQvU162wcNFln1Bd4I/1Vn1dETAJep+GXVGO/ALaTdFCJ6tcS44EfAMcCM4EatXwQ9E3AMSkwPDq9z6cvsKjYwiNiekQ8ExF1qSXqFqC+ZW51Km/niKiNiCciYkXaVwfsLqlbRCyKiA3pgvsKcHlEzEgtcfMj4uVUrztSC11dRNwGzCPr5izGscCEiHgyBWrnkbVMDc7Jc1lELIuIf5F1uQ3LU96RqUVrFfBV4PMRsSa1un0GOCO1iC4GriT7PdVbDPwktZbeBjwPHCxpW+DjwDkR8W5EzAJ+RcPu2MfSOLC6iFhV5LWbdVoOmqwzehPo14IxHOcD3yNrnVpP+tK8OG15SbonddcsA84Fzq1/L+meIuuTe+7aiLg2Ij5O1lpwKTAhdTEVW8a/gPlkrQzzIuKVAoe8Sda6UhRJ+0iaJul1ScvJWnH6pd2/AaYAt0p6VdLlkrpGxDvAUSnvIkn3Snp/sefMsS1ZK0xT9TpO0qyc38fuOfUqZBuy1iUAIuJtsp/LwJw8/855vRLokae821OLVn/gH8BeKX17oCvZz6C+nr8Ats45tiYarrz+cqrfNsCSiHir0b7cOhb6XZtZDgdN1hk9BvwHOLxQRoCImEoWVHw9T7bryYKWzxUo67MR0St9QV5G1hrRK22fLar2zZe9KiKuJevWGdLCwycC30r/FvIX4AhJxX5+/BaYBGwbEVuSjRdSqvPqiLgoIoaQdcF9ltQSEhFTIuIAsgDtOeCXLbieeq/QxOB0Sdun8k4D+qbfxz/q60XWNZvPq2QBTX153clazGo2oI5rRcQbwEnAhZIGpPr/B+iXc5/0jIgP5hw2sFG37Hapfq8CfSRt0Whfbh0bX2fJn/w060gcNFmnExHLgQuAa5XN67O5pK6SDpJ0eTOHfQ/4Tp4y15CNJzmn9DVea5M0cLp+q5Z0hqSRkropmxLheLKn6J5qYdm3kY2Vub2IvD8mGx90Ywo+kDRQ0o8lDW0i/xZkLR7vKpue4X/qd0j6ZBqLVQ2sIOuuq5PUX9JhKRj5D/A2WXddS/0KOFvSXsrsnOrcnSxAeD3V40SylqZ6rwGD0hijptwCnChpmKRNyVrp/l4/MH5jRMTzZK1v34mIRcCfgR9J6impStJOavjgwdbAN9M9/N/AB4DJqcXwUWB8ul+GAl8mf/fra8DgFgTEZp2K/2NYpxQRPyJ7Cux8si/OV8haHf7QTP5HgMcLFHsLLRjrswGeJRvzUr+dSNbt8yOyrqA3gFOBMRGxoCUFp1aqvxQzriUilpC1Cq0G/i7pLeB+YDlZi1xjXwfGpXwX0DAwex/ZQOQVwFzgQbIuuyqy38+rZNM6jABOack1pbreQdZl+VuyJ/7+APSJiDlkP7fHyAKFDwGP5Bz6ANnP+9+S3mii3L+QPSTwO7Lf+U40HGe0sX4InCRpa7KWt02AOWStiHfSsHv078AuZL//S8nGQ9UPdj8GGEz2c7wLGJvq3pz6qQ7elPRkaS7FrONQw65wMzOrFJJOAL4SEf/V1nUx6wzc0mRmZmZWBAdNZmZm1u5JGi3p+TRR67lN7N9e0v2SZiub5HZQzr77mnpKWdIOkv6eyrwtzzhGwEGTmVnFiogb3DVnnUF6WORa4CCyp4OPkdT4KeErgIkRMZRs1YfxOft+CHyxiaJ/AFwZETuTjRn8cr56OGgyMzOz9m5vYH5ELIiI98hmuz+sUZ4hZA9xQDah7Nr9EXE/2cMga6WpOvYne7gCsmWx8k5F46DJzMzM2ruBNJyMdSENJ2oFeJp1c+UdAWwhqW+eMvsCy3JWh2iqzAa8qnUe3bp0jZ5dm5wE2qxo72yyKbX++8Q2UhV11Pk+shJ4d8Xrb0TEVq11vqqeg4I17+bNE6vefBbIzXRdRFzXwlOdDVyTnip9iGwi19oWlpGXg6Y8etRWcURtviDVrLDHPv5p5r65IfMymq2zbe86XlrqoMlKYNb1LxfOVEK1/6HrB47Im+W9J3/1bkQMz5OlhmxZpHqDaDQDf0S8SmppktSDbM66ZXnKfBPoJalLam1ar8zG/D/QzMzMykpV1Xm3IswAdklPu21CNpnspAbnkPrlzGZ/HjAhX4FpzcZpwOdT0vHAH/Md46DJzKwCCBXOZNYuaaODptQSdBrZEkNzyRa5flbSOEmHpmwjgeclvUC2+PWla2sg/ZVsxvtPSVoo6cC06xzgLEnzycY4/TpfPdw9Z2ZmZmUjiaqueac/KkpETAYmN0q7IOf1nax7Eq7xsfs1k76A7Mm8ojhoMjMzs/KRqCquC67dc9BkZmZmZVXkuKV2z0GTmZmZlY0kqrp0betqlISDJjMzMysjuaXJzMzMrCA5aDIzMzMrrERPz7UHDprMyi2irWtgZtZmhAeCm1mRAgdNVgJRB3SMLx7rZNw9Z2ZmZlYEieou7p4zMzMzy0t+es7MzFqTO3mtkjloMjMzMytEVVS5e87MipGtTu92AttIUlvXwGzDCFTtliYzK4YcNJlZ5+UxTWZmZmbFkNw9Z2ZmZlaMqqqO0b3soMnMrAJkY+PMKo8k5KDJzMzMrDB1kAcZHDSZmZlZ+Qiqu1S1dS1KwkGTmZmZlU22YK9bmszMzMwKEFXunjMzMzMrQFDl7jkzMzOz/CRPOWBmZmZWFHWMhiYHTWZlF15CxUogAjxXk1UgSVRXd4yoyUGTWZmF152zkvB9ZJXLT8+ZmZmZFcFBk5mZtRq3M1mlknD3nJmZmVlhXnvOzMzMrLAONOVAx2gvM2vH1FGetbW21UFmVLbOR0BVtfJuRZUjjZb0vKT5ks5tYv/2ku6XNFvSdEmDcvYdL2le2o7PST9G0jPpmPsk9ctXB3+am5mZWfmklqZ8W8EipGrgWuAgYAhwjKQhjbJdAUyMiKHAOGB8OrYPMBbYB9gbGCupt6QuwFXAJ9Mxs4HT8tXDQZOZmZmVlaqUdyvC3sD8iFgQEe8BtwKHNcozBHggvZ6Ws/9AYGpELImIpcBUYDRpLWGguyQBPYFX81XCQZOZWQWQJ7a0ClU/uWW+rQgDgVdy3i9MabmeBj6XXh8BbCGpb3PHRsRq4BTgGbJgaQjw63yVcNBkZmZmZVVES1M/STNztpM24DRnAyMkPQWMAGqA2mbrJHUlC5r2BLYh6547L98J/PScmZmZlY0E1YW74N6IiOF59tcA2+a8H5TS1oqIV0ktTZJ6AGMiYpmkGmBko2OnA8PScf9Mx9wOrDfAPJdbmszMzKxshNikS1XerQgzgF0k7SBpE+BoYFKD80j9tO5x5fOACen1FGBUGvzdGxiV0mqAIZK2SvkOAObmq4RbmszMzKx8imtpyisi1kg6jSzYqQYmRMSzksYBMyNiEllr0nhJATwEnJqOXSLpYrLAC2BcRCwBkHQR8JCk1cDLwAn56uGgyZo15MARHHnVBVRVV/PIr25jyg9+1mB/n+0GctyEy+mxVR9WLlnOhC+cwbKafwOw73Fj+Mz52ZObky+5hr9N/F2r19/ah1Ef250ff/t/qKoS1//hr/zw+skN9m83oC/XjT2RrXpvwZIV73DC935JzeKlANx9zZnsM3QnHnlqHkecflVbVN/aCd9HlUtsfNAEEBGTgcmN0i7IeX0ncGczx05gXctTbvrPgZ8XWwd3z1mTVFXFMdeO45qDTuCiIQfwkWMOZcAHdm6QZ8wV3+VvE3/PJXscxL3jruLw8d8BYPPeW3Lw2NO5bJ/DuWzvwzh47Ols3qtnW1yGtbGqKnHVuV/gkNOuZI8x53PU6H34wI7bNMjzgzOP5OZ7H2Wvo8Zy6XWTuOQbY9bu+/HE+zjx/F+2drWtnfF9VNmqBJt2qcq7VYqy1VTSBEmLJf2jQL4zJB2XXl+cZuWcJenPkrZJ6d9OabMk/UNSbZqsCkkvpdk8Z0mamVPuhZJqco77TEr/kKQbynXdHcXgvYexeP7LvPHiK9SuXs2MW+9m6GGjGuQZMGQXnn/gUQCen/YYexx2AJC1UM2d+jArly5n5bIVzJ36MENGj2ztS7B24CO778g/X1nMizWvs3pNLbdP+TuHjBzWIM8HdtyGaY9nwwimz3iOQ0buuXbftMfn8tY777Zqna398X1U4SSqq/JvlaKc4d0NZJNHNSvNxvkl4Lcp6YcRMTQihgH3ABcARMQPI2JYSj8PeLC+PzL5ZNrfeOT9lfXHpWY9IuIZYJCk7Tb2Ajuy3gP7s/SVdXN8LVu4iN4D+zfIs/Dpuez5uexXPOyIA+nWcwu69+lV1LHWOQzcuhcLX1v3X7XmtaVss1XvBnlmv/AKh++/FwCH7/9hevboRp8tu7dqPa19831U2bLuuaq8W6UoW00j4iFgSYFs+wNPRsSadMyKnH3dgWjimGOAWzayeneTjby3jfC7sy9llxH78N0n72XXEfuydOEi6mrr2rpa7U6Efyb5nHPl7Xxir914/Jax7LfXbix8bQm1vo/W5/soL99H7ZdEKZ6eaxfaeiD4x4EnchMkXQocBywHPtlo3+ZkrVe5a8ME8Oc0Wv4XEXFdzr7TUtffTOBbafp00vtzgcsbVyhNqHUSQA+qN/zKKtzSmtfove26MQO9Bg1gac1rDfIsX7SYX4w5GYBNu2/OnmNGs2r5CpbWvMauI/dtcOwL0//WOhW3dqVm8TIG9e+z9v3A/r159fWlDfIsen0ZR559LQDdu23KEZ/ai+Vvr2rVelr75vuosonK6oLLp63DuwHA67kJEfG9iNgWuJn1F847BHikUdfcf0XEh8kW8TtV0idS+s+Ancgmr1oE/CjnmMVks3+uJyKui4jhETF8s04cNL0842m23mUwfQcPorprVz5y9CHMnjS1QZ7ufXujtPL66PO+zqMTbgdgzpQHGTJqPzbv1ZPNe/VkyKj9mDPlwVa/Bmt7M599kZ2368/gbfrRtUs1Rx64D/dMn9UgT99ePdbeR+d86WBu/OPDbVFVa8d8H1W+ainvVinauqVpFbBZM/tuJnu0cGxO2tE06pqLiJr072JJd5Et6vdQRKxtFpH0S7IxUvU2S+e2ZtTV1nLbaRfwzSkTqaqu5tEJt7NozjwOuehMXp75DLPv/gu7jdyXw8d/h4hg3kOPc+up2ZOfK5cuZ/LFV3PujGzesXvHXc3Kpcvb8nKsjdTW1nHGD27i3p+eRVVVFTf+8WHmLHiVsacczhNzXuKeB2cxYvhuXPyNz0MEf33yBb45/qa1xz/w63PZbYcB9Oi2KQvuu4KvXXQ9Ux97tg2vqO00NVahs/B9VNnqu+c6AkWU77+ipMHAPRGxezP7TwYGRcT56f0uETEvvf4GMCIiPp/ebwm8CGwbEe+ktO5AVUS8lV5PJZu06j5JAyJiUcp3JrBPRByd3o8BDoiIk/PVfyttGmMYsHE/BOv0Ht1/f54rNLrPrIDBvWp5aVnnbf220lk96/onCixZUlLv2+WDcdyPb8+b54eH7t6qddpQZWtpknQL2eyc/SQtBMZGROPVg/8E/Cbn/WWSdgPqyGbmzA1qjgD+XB8wJf2Bu1KTbBfgtxFxX9p3uaRhZH+gvQR8Lee4TwL3bvjVmRUvW52+M7cTmFln1pHGNJUtaIqIY4rI87KkN+tbmCJiTJ68N5BNY5CbtgDYo5n8X2wqXdKmwHDgjEL1MysJOWiyEqigcR9muTpS91xbj2mC7Cm2AcC8VjrfdsC59dMcmJmZWfmUahmV9qDNg6aIeB54vhXPN4/WC9DMzMw6txIs2NtetHnQZGZmhWVj48wqj5C758zMzMwKkVuazMzMzArzmCYzMzOzIkhik2p3z5mZmZkVVNVBpsxw0GRmZmZlI6C6Y8RMDprMzMysfCTo6u45MzMzs8LcPWdmxSnjothmZu2du+fMrGjhdeesFKIOqG7rWpi1mCS6uHvOzMzMrDC3NJmZmZkVIDymyczMWpE7ea1S+ek5MzMzsyK5e87MipKtTu92AttIHaR7wzofIXfPmVmR5KDJzDqvrHvOQZOZmZlZQdUdpKWpY4zMMjMzs3ap/um5fFtR5UijJT0vab6kc5vYv72k+yXNljRd0qCcfcdLmpe243PSN5F0naQXJD0naUy+OrilycysAmRj48wqTym65yRVA9cCBwALgRmSJkXEnJxsVwATI+JGSfsD44EvSuoDjAWGk42VeCIduxT4HrA4InaVVAX0yVcPtzSZmZlZGYlq5d+KsDcwPyIWRMR7wK3AYY3yDAEeSK+n5ew/EJgaEUtSoDQVGJ32fYksuCIi6iLijXyVcNBkZmZmZVOi7rmBwCs57xemtFxPA59Lr48AtpDUt7ljJfVK7y+W9KSkOyT1z1cJB01mZmZWNvXdc/k2oJ+kmTnbSRtwqrOBEZKeAkYANUBtnvxdgEHAoxHxYeAxsi6+vAeYmZmZlU0RjUlvRMTwPPtrgG1z3g9KaWtFxKukliZJPYAxEbFMUg0wstGx04E3gZXA71P6HcCX81WyxS1NknpLGtrS48zMzKxzqkJ5tyLMAHaRtIOkTYCjgUm5GST1S4O5Ac4DJqTXU4BRKX7pDYwCpkREAHezLqD6FJA7sHw9RbU0SZoOHJryPwEslvRIRJxVzPFmZmbWOQnY2KXnImKNpNPIAqBqYEJEPCtpHDAzIiaRBT/jJQXwEHBqOnaJpIvJAi+AcRGxJL0+B/iNpJ8ArwMn5qtHsd1zW0bECklfIXucb6yk2UVfrZmZmXVOKs0qQBExGZjcKO2CnNd3Anc2c+wE1rU85aa/DHyi2DoUG/t1kTQAOBK4p9jCzQwIL6FiJeD7yCqUCnTNFdk91y4U29I0jqxJ7JGImCFpR2Be+apl1nGE152zkvB9ZJVrY7vn2ouigqaIuINsVHn9+wVA3qnGzczMzIAKakvKr6jYT9KuaT2Xf6T3QyWdX96qmZlZPbczWaUq1dpz7UGxDWa/JHt8bzVARMwme9zPzMzMLK9q5d8qRbFjmjaPiMfVMBpcU4b6mJmZWQcigSqoNSmfYoOmNyTtRGohlvR5YFHZamXWgWRzrdW1dTWs0nWQLx3rnKo6yO1bbNB0KnAd8P40HfmLwBfKViszMzPrEERldcHlU+zTcwuAT0vqDlRFxFvlrZaZmZl1FB2le67Yp+dOl9STbGG7KyU9KWlUeatmZmb11GEe2rZOR1n3XL6tUhT79NyXImIF2SJ3fYEvApeVrVZmZmbWIdR3z3Wmp+fqL+kzZGvPPauO0tZmZmZmZdVRQoZig6YnJP0Z2AE4T9IW+HEgMzMzKyCb3LKta1EaxQZNXwaGAQsiYqWkPsCJ5auWmZmZdQyiupO1NH0UmBUR70j6AvBh4KryVat9qNu8Gys++KGGzYoqPCAzGq1GXv8+t5zsZUvKibWLnDcsp8CNGA0XjI0swde0Xl3Kd01bvrWYEX16N3EZ0WDl+rXXJDWoe3HXsX452fxQ6xQup2HjcVaOGv48CpXha8pTl427piWrVvDBbd639ujG11HwPm10HevqUppyGl9H4f83jTsrfE356lLKa3pqVt5DS08dZ5qxYoOmnwF7SNoD+BbwK2AiMKJcFWsPlkYX7ly9VVtXwyrcCe/+k6rHZ7d1NazCdfvQrty36j9tXQ2zFlMEavRHaqUq9um5NZH9mXQYcE1EXAtsUb5qmZmZWUehujV5t0pRbEvTW5LOI5sF/BPK2iy7lq9aZmZm1jEErNd1WZmKbWk6CvgP8OWI+DcwCPhh2WplZmZmHUdE/q1CFLuMyr+BH+e8/xfZmCYzMzOz5kUna2mStK+kGZLelvSepFpJy8tdOTMzy1TO3+Jm6+tsY5quAY4G7gCGA8cBu5arUmYdib/srCQqqAvDrKFO1tIEEBHzgeqIqI2I64HR5auWmZmZdQhBFjTl2ypEsS1NKyVtAsySdDmwiBYEXGadm1sIzKwzC1RbOV1w+RQb+HwRqAZOA94BtgXGlKtSZh2Je1WsFMLBt1WyztTSFBEvp5ergIvKVx0zMzPrUCpsWoF88gZNkp4hT99CRAwteY3MOphCa5KZFaPgmoxm7VglPSGXT6GWps8B/YFXGqVvC/y7LDUyMzOzDqTzPD13JbA8Il7O3YDlaZ+ZmZlZfp1kTFP/iHimcWJEPCNpcFlqZNbBuFfFSsHdvFapFNFhuucKtTT1yrOvWykrYmZmZh1UXV3+rQiSRkt6XtJ8Sec2sX97SfdLmi1puqRBOfuOlzQvbcc3cewkSf8oVIdCQdNMSV9tovCvAE8UKtzMzMw6uwKL9RbxZJ2kauBa4CBgCHCMpCGNsl0BTEwPqY0Dxqdj+wBjgX2AvYGxknrnlP054O1irqRQ99wZwF2SjmVdkDQc2AQ4opgTmJmZWScWJXl6bm9gfkQsAJB0K3AYMCcnzxDgrPR6GvCH9PpAYGpELEnHTiVb1eQWST3SMScBtxeqRN6gKSJeAz4m6ZPA7in53oh4oODlmZmZmZXm6bmBNHySfyFZy1Gup8me+r+KrGFnC0l9mzl2YHp9MfAjYGUxlSh2cstpZFGbmZmZWcsUDpr6SZqZ8/66iLiuhWc5G7hG0gnAQ0ANUNtcZknDgJ0i4sxiH24rdu05MzMzs5aLINasLpTrjYgYnmd/DdkckfUGpbSc08SrZC1NpG63MRGxTFINMLLRsdOBjwLDJb1EFg9tLWl6ROTmbcCL7pqZmVkZBdTV5t8KmwHsImkHSZsARwOTcjNI6iepPq45D5iQXk8BRknqnQaAjwKmRMTPImKbiBgM/BfwQr6ACRw0mZmZWTkFGx00RcQa4DSyAGgucHtEPCtpnKRDU7aRwPOSXiBbzeTSdOwSsrFLM9I2rn5QeEu5e86szDrGMpXW1sJ3klWoiCBWF+yeK6acycDkRmkX5Ly+E7izmWMnsK7lqan9L7HugbdmOWgyM6sEjpmsYkWxXXDtnoMmMzMzK58IwkGTmRWliNluzcw6rqKenqsIDprMyswhk5WCxzRZxaofCN4BOGgyKzOvTW8lId9JVqmi6EV52zsHTWZl5y8723i+i6xiRRCr32vrWpSEgyYzMzMrn/DTc2ZmZmZFCXfPmVkxPBTFSkHuoLNKFQFr3D1nZmZmllcQbmkyMzMzK8hTDpiZmZkVwU/PmZmZmRXD8zSZmZmZFcfdc2ZmZmYFRFDntefMzMzMCoggat09Z2ZmZpZXBA6azKw4XpveSiHCd5JVqAjqVq9p61qUhIMmMzMzK6uO0tJU1dYVsPZr1Md25x93/S9z/jieb5/4mfX2bzegL/f9/GyeuO0ipv7yOwzcuvfafXdfcyaLH7qGu646vTWrbO3QkANHcOFz9zNu3nQOPOeU9fb32W4gZ/zlZs5/+k+cNe1Weg1839p9+x43hnEvTGPcC9PY97gxrVlta2f8eVS5IoK62tq8W6UoW9AkabSk5yXNl3Runnw/kfSJ9PrXkp6WNFvSnZJ6pPQrJc1K2wuSljUqo6ekhZKuyUmbns5ff9zWKf00SV8qz1V3HFVV4qpzv8Ahp13JHmPO56jR+/CBHbdpkOcHZx7Jzfc+yl5HjeXS6yZxyTfWfan9eOJ9nHj+L1u72u1SZ+5WUVUVx1w7jmsOOoGLhmqNF/YAABIFSURBVBzAR445lAEf2LlBnjFXfJe/Tfw9l+xxEPeOu4rDx38HgM17b8nBY0/nsn0O57K9D+Pgsaezea+ebXEZ7UJ04o5efx5VuNQ9l2+rFGUJmiRVA9cCBwFDgGMkDWkiX19g34h4KCWdGRF7RMRQ4F/AaQARcWZEDIuIYcD/Ab9vVNTFwEOs79j64yJicUqbAHxjIy+xw/vI7jvyz1cW82LN66xeU8vtU/7OISOHNcjzgR23YdrjcwGYPuM5Dhm559p90x6fy1vvvNuqdbb2Z/Dew1g8/2XeePEValevZsatdzP0sFEN8gwYsgvPP/AoAM9Pe4w9DjsAyFqo5k59mJVLl7Ny2QrmTn2YIaNHtvYlWDvgz6PKF7V1ebdKUa6Wpr2B+RGxICLeA24FDmsi3xjgvvo3EbECQJKAbjQ9hvYY4Jb6N5L2AvoDfy6mYhGxEnhJ0t7FXUrnNHDrXix8bcna9zWvLWWbrXo3yDP7hVc4fP+9ADh8/w/Ts0c3+mzZvVXrWQk689r0vQf2Z+krr659v2zhInoP7N8gz8Kn57Ln50YDMOyIA+nWcwu69+lV1LGdSfax2Dn586iyRWQL9ubbKkW5gqaBwCs57xemtMY+DjyRmyDpeuDfwPvJWpVy920P7AA8kN5XAT8Czm6mHtenrrnvq+Enzkxgv6Kvxpp0zpW384m9duPxW8ay3167sfC1JdRW0F8MrafzftkV43dnX8ouI/bhu0/ey64j9mXpwkXU+T6yFvLnUTsWUPvemrxbpWjrp+cGAK/nJkTEial77/+Ao4Drc3YfDdwZEfWjxr4OTI6IhU38FXZsRNRI2gL4HfBFYGLat5gsKFuPpJOAkwDo2nn/SqlZvIxB/fusfT+wf29efX1pgzyLXl/GkWdfC0D3bptyxKf2Yvnbq1q1nta+La15jd7brht70mvQAJbWvNYgz/JFi/nFmJMB2LT75uw5ZjSrlq9gac1r7Dpy3wbHvjD9b61TcWtX/HlU4TrQ5JblammqAbbNeT8opTW2CtiscWIKim4l677LdTQ5XXPAR4HTJL0EXAEcJ+myVEZN+vct4LdkXYb1NkvnXk9EXBcRwyNiuLqsV7VOY+azL7Lzdv0ZvE0/unap5sgD9+Ge6bMa5Onbq8faLoNzvnQwN/7x4baoarvXmduZXp7xNFvvMpi+gwdR3bUrHzn6EGZPmtogT/e+vdfeR6PP+zqPTrgdgDlTHmTIqP3YvFdPNu/VkyGj9mPOlAdb/RraC3XiO8mfR5UtoMN0z5WrpWkGsIukHciCpaOB/2ki31xgZ2B66j7bKSLmp9eHAs/VZ5T0fqA38Fh9WkQcm7P/BGB4RJwrqQvQKyLekNQV+Czwl5zz7go8UpIr7aBqa+s44wc3ce9Pz6Kqqoob//gwcxa8ythTDueJOS9xz4OzGDF8Ny7+xuchgr8++QLfHH/T2uMf+PW57LbDAHp025QF913B1y66nqmPPduGV9SGOu93HXW1tdx22gV8c8pEqqqreXTC7SyaM49DLjqTl2c+w+y7/8JuI/fl8PHfISKY99Dj3HrqBQCsXLqcyRdfzbkzJgFw77irWbl0eVtejrURfx5VuAjqKqgLLh+V63FoSZ8BfgJUAxMi4tIm8uwHfC0ivpDGJ/0V6En2NfM0cErO4PALgc0iosnpC3KCptMkdSd7mq5rOv9fgLPqu/UkPQkcEBFv5ruGqs37RZfdDm3xtZvlOuHdf1L13MttXQ2rcK9+aFfuqx7U1tWwDmD1rOufiIjhrXW+oe/rF5OOPzhvnh0un9iqddpQZRvTFBGTgckF8vxV0nhJvSJiGdnA8ObyXligrBuAG9Lrd4C9msonaU/g2UIBk5mZmW28wGOaSulbwHateL5+wPdb8XxmZmadVwR1q1fn3YpRaNJsSdtLuj9NkD1d0qCcfcdLmpe241Pa5pLulfScpGfrx0Tn09ZPzxERf2/l800tnMvMzMxKIjZ+7bmcSbMPIJvGaIakSRExJyfbFcDEiLhR0v7AeOCLkvoAY4HhWW14QtIk4D/AFRExTdImwP2SDoqIPzVXj/bQ0mRmZmYdVpRiRvBiJs0eQprHEZiWs/9AYGpELImIpcBUYHRErIyIaQCpzCfJnvZvloMmMzMzK5uIoHb1mrxbEYqZNPtp4HPp9RHAFmm5toLHSuoFHALcn68SDprMzMysfKKotef6SZqZs520AWc6Gxgh6SlgBNmUR7X5D4E0TdEtwNURsSBf3jYf02RmZoWVZ3IYs1YQELUF7+A3Ckw5UHDS7Ih4ldTSJKkHMCYilkmqAUY2OnZ6zvvrgHkR8ZNClXRLk1mZ+cvOSqJMc+qZlVvWPVebdyvC2kmz06Dto4FJuRkk9UtzPgKcB0xIr6cAoyT1ltQbGJXSkHQJsCVwRjGVcNBkZmZmZVVXG3m3QiJiDXAaWbAzF7g9Ip6VNE5S/SzUI4HnJb0A9AcuTccuAS4mC7xmAOMiYkmakuB7ZAPIn5Q0S9JX8tXD3XNmZecWAjPrvKIEUw5k5aw/aXZEXJDz+k7gzmaOncC6lqf6tIW0cKErB01mZeZeFSuFcPBtlSqC2vc6xozgDprMzMysfErU0tQeOGgyKzO1rPXXrEmS7yOrTAHU1XWMllIHTWZmZlY+EdS+V9QTcu2egyYzMzMrmyhunqaK4KDJrMzcq2Kl4G5eq1gOmszMzMyKkCa37AgcNJmZmVnZBBQ1gWUlcNBkZmZm5RPhKQfMzMzMConAk1uamZmZFSPqHDSZmZmZ5RfFLcpbCRw0mZmZWdlEQJ0ntzQzMzMrIPz0nJkVqWN8VFhbC99JVrHCk1uamVkr6hjfOdYJRcBqDwQ3MzMzyy+ADtLQ5KDJrOyig3xamJltoNoO8jnooMmszDrGR4W1NY9pskpVF/BeXce4fx00mZWZ16a3kpDvJKtc7p4zsyL5y842nu8iq1RBuHvOzMzMrBAPBDczMzMrQnhMk5kVy0NRrBTkDjqrYG5pMjMzMysg657rGFGTgyYzMzMrm8Ddc2ZmZmYFRbh7zszMzKwo7p4zMzMzKyAIVjtoMjMzM8uvI83TVNXWFTAzM7OOq/7puXxbMSSNlvS8pPmSzm1i//aS7pc0W9J0SYNy9h0vaV7ajs9J30vSM6nMq6X8k8Q4aDIzM7OyqZ/cMt9WiKRq4FrgIGAIcIykIY2yXQFMjIihwDhgfDq2DzAW2AfYGxgrqXc65mfAV4Fd0jY6Xz0cNJmVWQdplbY2Fh1kTIh1TrWRfyvC3sD8iFgQEe8BtwKHNcozBHggvZ6Ws/9AYGpELImIpcBUYLSkAUDPiPhbZP/BJgKH56uExzTlsdkmVew8sDtZa11LZuONJj/gpJbFqBF1TZRRqrr4mlrrmuKN7nT/4A5NnZ5oIqQSatFlZJfQ3M+jJeU09cmlls1o7mvKU5dUWONyiqzMJj278f7e3Rsd13H/32xcXXxN+a5p9qwWFFMCJZrcciDwSs77hWQtR7meBj4HXAUcAWwhqW8zxw5M28Im0pvloCmPVctff+OZe695ua3r0c71A95o60q0Z7PbugKVw/eSlYLvo8K2b82Tvc57U34aL/crkG0zSTNz3l8XEde18FRnA9dIOgF4CKgBaltYRl4OmvKIiK3aug7tnaSZETG8rethlc/3kpWC76P2JyLyjhMqUg2wbc77QSkt9zyvkrU0IakHMCYilkmqAUY2OnZ6On5Qo/QGZTbmMU1mZmbW3s0AdpG0g6RNgKOBSbkZJPXTuj7V84AJ6fUUYJSk3mkA+ChgSkQsAlZI2jc9NXcc8Md8lXDQZGZmZu1aRKwBTiMLgOYCt0fEs5LGSTo0ZRsJPC/pBaA/cGk6dglwMVngNQMYl9IAvg78CpgP/BP4U756yE9k2MaQdNIG9Dubrcf3kpWC7yMrJwdNZmZmZkVw95yZmZlZERw0dSKSaiXNkvQPSXdI2rwEZY6T9Ok8+0+WdNzGnscqR6P77G5JvUpc/kuS+qXXb5eybGt9OfdL/TZYUl9J0yS9Lematq6jWT13z3Uikt6OiB7p9c3AExHx45z9XdJgO7MN1ug+uxF4ISIuLWH5LwHDI+KN3HNZZWrqdyipO7AnsDuwe0Sc1kp18Weg5eWWps7rr8DOkkZK+qukScAcSdWSfihpRlr08Gv1B0g6Jy1s+LSky1LaDZI+n15fJmlOOu6KlHahpLPT62GS/pb231W/9k9aWPEHkh6X9IKk/Vr7h2Fl8xhphl1JO0m6T9IT6Z57f0rvn+6Hp9P2sZT+h5T3WUknteE1WCuLiHci4mHg3Xz5JH0wfW7MSp8ru6T049L7pyX9JqUNlvRASr9f0nYp/QZJP5f0d+Dy5u5TM/Dklp2SpC5kix7el5I+TPbX3Ivpy2l5RHxE0qbAI5L+DLyfbB2ffSJipbIFEHPL7Es2bf37IyKa6ZKZCHwjIh6UNI5sAcUz0r4uEbG3pM+k9Ga7/KwyKFtg81PAr1PSdcDJETFP0j7AT4H9gauBByPiiHRMfavDlyJiiaRuwAxJv4uIN1v5Mqz8ukmqX9jjxYg4ogXHngxcFRE3K5u7p1rSB4HzgY+l1sj6z6r/A26MiBslfYnsvqtfZ2xQyl8r6X6avk/NHDR1MrkfTn8l+zL7GPB4RLyY0kcBQ+tbj4AtyVZ+/jRwfUSshLXzXuRaTvZX4a8l3QPck7tT0pZAr4h4MCXdCNyRk+X36d8ngMEbfIXWHtTfZwPJ5lOZqmx23o8Bd2jdWmubpn/3J5tUjoioJbuXAL4pqf4LdFuy+9BBU8ezKiKGbeCxjwHfkzQI+H0KdPYH7oiIN6DBZ9VHSbNFA78BLs8p544UMOW7T80cNHUy6304pQ+Gd3KTyFqDpjTKd2C+giNijaS9yVoWPk82CVlL/jr7T/q3Ft+XlW5VRAxT9qDBFOBU4AZgWbFfjpJGkgXqH00tm9OBzcpTXasUKYgem95+JSJ+m7rVDgYm5w4naKH6z8AqWnCfWufjMU3W2BTgFEldASTtmgZlTgVOTF+ENNE91wPYMiImA2cCe+Tuj4jlwNKc8UpfBB7EOqzUKvlN4FvASuBFSf8NoEz9PXI/cEpKr06tklsCS1PA9H5g31a/AGt3IuKuiBiWtpmSdgQWRMTVZMtfDAUeAP47DRnI/ax6lGzpDYBjyVrbG5e/gubvUzP/RW/r+RVZ99iTypqhXgcOj4j7JA0DZkp6D5gMfDfnuC2AP0rajKy16qwmyj4e+HkKvBYAJ5bvMqw9iIinJM0GjiH7ovqZpPOBrsCtwNPA6cB1kr5M1tJ4Ctl4u5MlzQWeB/7WFvW3tqPsKcmewCaSDgdGRcScRtmOBL4oaTXwb+B/0zi4S4EHJdUCTwEnAN8Arpf0bbLPteY+f5q7T8085YCZmZlZMdw9Z2ZmZlYEB01mZmZmRXDQZGZmZlYEB01mZmZmRXDQZGZmZlYEB01mtlEkhaSbct53kfR6mhk+33HD0rI5ze0fLunqUtbVzGxjOGgys431DrB7WiMO4ACgpojjhgFNBk3KVpufGRHfLFEdzcw2moMmMyuFyWRLWUA2keUt9TskdZc0Ia1G/5Skw9LiquOAo9IK9UdJulDSbyQ9AvxG0sj61ipJPSRdL+mZtEr9mDR7+A2S/pHSz2ztizazzsVBk5mVwq3A0WlG+KHA33P2fQ94ICL2Bj4J/JBspuULgNvSkhi3pbxDgE9HxDGNyv8+sDwiPhQR9UtlDAMGRsTuEfEh4PpyXZyZGThoMrMSiIjZZMvvHEPW6pRrFHCupFnAdLKFd7drpqhJEbGqifRPA9fmnG8p2VI8O0r6P0mjgRUbcw1mZoU4aDKzUpkEXEFO11wiYEzOQqvbRcTcZsp4p5n09aTAaQ+yQOxksnUTzczKxkGTmZXKBOCiiHimUfoU4BtpAWgk7ZnS3yJb6LkYU4FT699I6i2pH1AVEb8Dzgc+vDGVNzMrxEGTmZVERCyMiKamCLiYbAzTbEnPpvcA04Ah9QPBCxR/CdA7Dfp+mmxs1EBgeur2uwk4ryQXYmbWDEVEW9fBzMzMrN1zS5OZmZlZERw0mZmZmRXBQZOZmZlZERw0mZmZmRXBQZOZmZlZERw0mZmZmRXBQZOZmZlZERw0mZmZmRXh/wFk8vlTg7cVagAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 708.661x255.118 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "lstm_cnn_prediction = (model_CNN_LSTM.predict(x_test) > 0.5).astype(\"int32\")\n",
    "print('Classification Report CNN + LSTM Model:\\n', classification_report(y_test, lstm_cnn_prediction))\n",
    "\n",
    "plot_classification_report(classification_report(y_test, lstm_cnn_prediction), title='CNN + LSTM Classification Report', cmap='RdBu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "fHMruU4tUAwG"
   },
   "outputs": [],
   "source": [
    "from sklearn import metrics\n",
    "from sklearn.metrics import roc_curve, auc\n",
    "\n",
    "fprB, tprB, thresholdsB = metrics.roc_curve(y_test, model_batch.predict(x_test))\n",
    "fprMB, tprMB, thresholdsMB = metrics.roc_curve(y_test, model_mini.predict(x_test))\n",
    "fpr3L, tpr3L, thresholds3L = metrics.roc_curve(y_test, model3L.predict(x_test))\n",
    "fprBi, tprBi, thresholdsBi = metrics.roc_curve(y_test, model_Bi_LSTM.predict(x_test))\n",
    "fprCNN, tprCNN, thresholdsCNN = metrics.roc_curve(y_test, model_CNN_LSTM.predict(x_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 350
    },
    "id": "G9lJ49m0UAwH",
    "outputId": "2b4a4f3f-c911-46c7-91cd-03e058f75bf7"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAFNCAYAAAAD7RaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3wc1bnw8d+Zmd3VqndblizLFTdsMMZ0MKY3AwHT0isBErhJbm4gvd2bBLhvQri5aSQhEHABAzb1mmqaAWMMBmMbd0m2epd2V7szc94/ZrWSLNmWiyxZer6fj9HMmbMzz66Fnz1lziitNUIIIYQYeoyBDkAIIYQQ/UOSvBBCCDFESZIXQgghhihJ8kIIIcQQJUleCCGEGKIkyQshhBBDlCR5IcSgpJT6qVLqXwMdhxBHM0nyQhwhSqkdSqmwUqpVKVWplLpfKZW6R51TlVIvKaValFJNSqknlVJT96iTrpT6nVKqNH6urfH93L1cVymlblVKfaSUalNKlSulHlFKHduf77cv4p9BNP4+6pVSzyulJh/EeXYopc7tjxiFOJpJkhfiyLpMa50KHAccD9zRcUApdQqwAlgGjALGAh8AbyilxsXr+IEXgWnAhUA6cApQB8zZyzXvAW4DbgWygUnAE8AlBxq8Uso60Nf0wZ3xz6QIqAbu74drCDEsSZIXYgBorSuB/8NL9h3uBB7QWt+jtW7RWtdrrX8IvAX8NF7nc0AxcKXW+mOttau1rtZa/0Jr/cye11FKTQRuAa7XWr+ktW7XWoe01g9prX8dr/OKUuorXV7zBaXU6132tVLqFqXUZmCzUuqPSqm797jOMqXUt+Pbo5RSS5VSNUqp7UqpW/v4mYSAh4HpvR1XSs1XSq1XSjXGY54SL38w/pk8Ge8R+A+lVJJS6l9Kqbp4/dVKqRF9iUOIoUSSvBADQClVBFwEbInvJwOnAo/0Un0JcF58+1zgOa11ax8vdQ5QrrV+59Ai5grgJGAqsBC4VimlAJRSWcD5wCKllAE8idcDURi//r8ppS7Y3wXiQxefBtb2cmxS/Lr/BuQBz+Aldb/W+rNAKfFeEq31ncDngQxgNJADfB0IH/zbF+LoJEleiCPrCaVUC1CG1zX9k3h5Nt7/jxW9vKYC6Bhvz9lLnb050Pp786t4z0IYeA3QwBnxY1cDq7TWu4ETgTyt9c+11lGt9Tbgr8B1+zj3vyulGvG+8KQCX+ilzrXA01rr57XWMeBuIIj3xag3Mbz3PkFr7Wit12itmw/kDQsxFEiSF+LIukJrnQbMBSbTmbwbABco6OU1BUBtfLtuL3X25kDr701Zx4b2nmq1CLg+XnQD8FB8ewwwKt5F3hhP3t8H9tVVfrfWOlNrPVJrPV9rvbWXOqOAnV1icOMxFe7lnA/iDYcsUkrtVkrdqZTy7f9tCjG0SJIXYgBorVfiTTC7O77fBqwCFvRS/Rq8yXYALwAXKKVS+nipF4EipdTsfdRpA5K77I/sLeQ99hcCVyulxuB14y+Nl5cB2+NJu+NPmtb64j7Guze78b5AAN4dA3hd8bt6i09rHdNa/0xrPRWvtX8p3nwGIYYVSfJCDJzfAecppWbG928HPh+/3S1NKZWllPol3uz5n8XrPIiXSJcqpSYrpQylVI5S6vtKqR6JVGu9GfhfYKFSaq5Syh+flHadUur2eLX3gU8ppZKVUhOAL+8vcK31WrzehfuA/9NaN8YPvQO0KKW+p5QKKqVMpdR0pdSJB/MBdbEEuEQpdU68Rf4doB14M368ChjXUVkpdbZS6lillAk043Xfu4cYgxBHHUnyQgwQrXUN8ADw4/j+68AFwKfwxtF34t1md3o8WaO1bsebfLcReB4vgb2D1+3/9l4udSvwP8AfgEZgK3Al3gQ5gN8CUbxE+U86u9735+F4LA93eU8OXqv5OGA7nV8EMvp4zl5prTcBnwHujZ/zMryJdtF4lV8BP4wPEfw7Xm/Eo3ifzwZgJd4XJCGGFeUNrwkhhBBiqJGWvBBCCDFESZIXQgghhihJ8kIIIcQQJUleCCGEGKIkyQshhBBDVH88Uapf5ebm6pKSkoEOQwghhDgi1qxZU6u1zjuY1x51Sb6kpIR33313oMMQQgghjgil1M791+qddNcLIYQQQ5QkeSGEEGKIkiQvhBBCDFGS5IUQQoghSpK8EEIIMURJkhdCCCGGKEnyQgghxBDVb0leKfV3pVS1UuqjvRxXSqnfK6W2KKXWKaVm9VcsQgghxHDUny35+4EL93H8ImBi/M/XgD/2YyxCCCHEsNNvK95prV9VSpXso8rlwANaaw28pZTKVEoVaK0r+ismIYQQYlDQGnDR2kW7Gtd1aA7ZoDXadXG1dywSbjmkywzksraFQFmX/fJ4mSR5IYToSrtg14MbBdxEgkC7gO78o12vvL0CYrUQ2Q5GIH5Md5ysS33vp9YQsxWuC67jFbsutEctolEDrTXajaHDO9C+kWjXe41GeT+1oqE5Gcu00VqBBler+LkVbuJyHfUh3O4nGrXYWZFPWnKo+9vVXXZsL8lpFX8fHf/tUqdjX2uN1l3reBeO2Q46/j69Ay5uR328eDtO52oXUCggxWwn7PoSMTW1jsBxLXbVTiE7vTx+TcM7B977Y49tLw6F6thG0RLO7ttfu2rB9b3Wp7p7c1SsXa+U+hpelz7FxcUDHI0Q4qikXdAxcCMQq6MzKTrxY463Ha0EFGgb7AZoWgVWWjxhOJ2JVMe3W1bTbo7Hti3ceN5144lMa0VbKMju2jwMw/FeCuiOrBLehrZy0BjxROO9UGuNjjWBdtDKisem0FrRHM4mZgfwWe24romrDVzXJOYECEdT0PF9Vxu42mJb5TSy0yoAheOatEayqG0uJD25Nv46P+2xlAH5Kzma1TcXHbZzKVyUckGBgQsqim2+j2OuBWUf0rkHMsnvAkZ32S+Kl/Wgtf4L8BeA2bNn697qCCGOYm7MS8CJ5Ol42247xOq9Y5GdXssUDS3vgi8PtINtg22DY2uaqzfRYk/GdR3clo/RvgJc26amPoWALxxPfAZam7hasbtuPClJzTiuxY7qqaQF63Fci6idxMdlpxDw5WAaY+MtUJVotXXddtyvD/CHt38NbSN6lDWHcnuUGWYE4glHKQcAO5ZBUuoOQGMoDUpjJ5rE3r73DcXEiWbhT9vSvdz76uIlsa6vMSIYZhiMGG4sEzOl4xksnf/Ed2z5DZcW2/C+PAFea7mzZ6JrUtB4reuOaq424u9Fo520+Eu91jtGDLSfmBsgaDh0vMhV7Vg6FQU4GLTEAqSbUaIOpPmScK0mgj4Dyx/CUApXK7L8MTSKVEtjKEApDKXQ0RBJGfkErQAKH6ggyjDJSAqSFFAoJ4o/q4BgwKIgzeSjLaU89dpaGhqbATjhhBP4xz/+sb+/4r0ayCS/HPiGUmoRcBLQJOPxQuyH1p3dtnYjuCG6ddN2dN927KPBCUOsGpQfryXqQLQKnBYwU6DrP9hdunC773f8Aw2uowlHfDh2DLdtE65VSGubhdO8DtdX7HX5akU4EiAUCWIqG0cr3HA5FU0zSA604LreP4yuC62hIA2t+aQFG3FdE0ebbN59PPkZZfGEbCR+RmITqWosIT15Fo5r0RbJ7NeP+4BbuFYzdCQzFf87UBra8yFYCsGyeG9uR2Ly/ht1O9Oh7nLURdPuxru7OyiNz86k3VeNbYTQ8eto5eIYIVxlg3LR8TJwUdqi3VcbL/OSWcxsQisn/sfFVZGOnuYeXDcAgG0XxHsdDCyrimh0PN6LlFeO8nJ7/ESmWY9jj/bK42UKiGqTTL/2Sg0T5WtC6UyUUtSFoDADTOXgN3MwlMJQJoZp4WgF2iDoNynKVmQmjyBgBUj2pxF1gkwqzMc0LAKWwcSCZPLS/QR9Boaxlzc2iLS0tPDHP/6RjRs3AjB69GhuuOEGjjnmGL7xjW8c9Hn7LckrpRYCc4FcpVQ58BPAB6C1/hPwDHAxsAUIAV/sr1iE6Hdae928ut3r5o1WxlukMW98VLdD64dgpkLzGxAY7dXTjvez8WXw5aIjZWgrG9eBUHsqNY0jcFwzkTi1q3G1QX3rCAzlsrN6Cqnx5KhR3X7uqJ5KalITzeEstE6Kt0B98THSdK8lqhWVjWMJ+NowDcdrocZbqmiF29GNHG+1drSCj4Sm0N4fn91bK9Tyh1HKJdaegj97C5FYGK0UjoqPBUczCQfL0LjE3Ag6noD9diatSdvQysF0g4T85WhlE7OasI02bDP+RQoXrfB+drQdlZtIy/HslqC1gVIuWiscNxO0ARiYZgOOk43jpnktyq5JUitQGtNoIGYXx4+DhYOBJqZ9aBTptCXyccerXbOVUKyEUbqOVpVClqogPZaGEY8wtT0FA4XSGhNQZGAF02kKORRnZ5Nm2phKYSoFzTX48sfjNw3ykhxMAwylUNE2fPklpKenkZJbgJGcRmDkBMbkBTEUKDX4k+lglZKSQigUIjU1lU996lOceeaZGMah3wCntNb7rzWIzJ49W8vz5MUREWuAyFbaI+00Vpbh1L5GXWseFTV5mIZNaUUeKUlh3NC2xLjolsqZ5GWUoxNjol4LtLqpiJgTICXgdQ1XNxWjcDHNWCLZeon06FmfyudvQRkOro7hugFcOxlf2g5iWoNycLSNiubiJu3CNdvQysHRGqI5hIPb0Lh47U0H7QaIGmFiZnM8+ToobdLuq8dr13oJWeMNattmM64RQ+PgGtFE6/RgtbWdHv/sDRQ2GgPHyaIjhXoJ2vsCpJSNbefhOploTK+b2k3F1C5ZtGLiUqXyAVjgPk2mbsbEwcVgGpvxaRsDN37mzp/xgYB4utfkUUcKISxcUAZoFzNjBIExM1CmDwwDu7aMpAkngmmhDAunuZqkcScAYOWXYGWNQpk+lGGCYWKmZqH8XnexGFi2bfPyyy9zwgknkJ3tTcSrqKggPT2dlJTuPUhKqTVa69kHc52jYuKdEPukNYQ3QbQGcNCuQzTq4tiacEsN4ZCXRB1H47qwdUcKjg0ffhQgPd3BdVx2VyaBjhKwIriuxnEtKhrG4bM0MTsdmBb/05s53fZ210/Ya6hdW6AaA9sJ9KijlPfHjU/S8gcUJeOSUApsHYvnHU1dTYxR4zThNk3mCId2O0zUiWIYUNVSTUOoHlyDLe1vkpmaiouDq11cbeNol6Zwg9fGUxpXxXCMCF276bXq2oFMvPs53pUc3+4Lr/fAh1JRlLKJxQr36PIdB3QkUbxtDWADqXSdraxwUUYYu31cvL7XJjVUGzF7FGCCNrxrOrn4lIVyLUzldRGHXYvxuhQMA0uDqS1C2k8qDjP0Ror1boJEGK9L44lYY+ImkjBoMmjFh5eoTVxMHEwc76/Fn4wyvYSKMnFDjSh/Ev5Rk3CaqvGPmgSGSWDMTNxwC/6iySjTh45G8OWPRfn8oAyszJFeIjctzJRMlOXv8+ctBr+PPvqIhQsXUlFRwbZt27jxxhsBKCgoOOzXkiQvBo1Y00Yqy1uItru4rkt9PZiRj70JV9qG9l1sr5xEUygHcDHcZlztw9VQ1TgGxzV7SbCH9j9NzE7qPFPOLkzDprl9FCPz2ykpjtDc4qN4dBTDVBhp0zEMk1jUJSffT9SO0NzexO6mXSgFO2rLSM8w0MrBxQYzRm2kgrZYG6mBZFxcb8IYLrZj897O9yjJKaHVifHa7o97BpcJ1Me3S/fxJgLQEOu9HIgn4BiOPRbXGQmqlXB7MW6X7uSO8dZE97KycZxswOu+19qP1kle3Xi3tLft59jR6aQGDBpbwuQFNdoKMyGpkWSfwm2uQscaMX0BlHZQ2sGp3UFbciGjjVqCqh1bG4xU9ZjaRrdUo5wYprkd5cTIohkDN5FwjXhCTiKKSd+/iACY6Xkoy49dv4vg1LOwMkeAYaGUwmltIFA8HUwLHWvHyhyJL28MRkoWRjANMz0PMzn9gK4nhp+qqioWL17M+++/D0B+fj4nnXRSv15Tkrw4vLSG0AYIb4bQBtxIDWVVBTTV1lLbmEVtYw6WGWVnqY/yuom0hHPwWZEuyTRpjxPOYc+Wcl8FA2FM06U1lELRyBpMQ2PE/+yuzuKsOZsImI2MLMrEtEDrAFkjR2Am5WMk5WNYSfj8BvWRKlrcOj7aVUN9az3pahvhWITHyz+ioGgkyyu34miXxlAjzeFmkv3JhKKh/QfYB++HPuhRZjtZoE0MI4SrAzh2LsQ7e02jCdseieumoIwIsVgRjpOF4+TGE3E8Wcdb0q5OZn8LX2YlmzSEHI4bZVHV4nDcKJNzxxsk1W4k3a0nzbKxt79LXqqJhY3ptmPvWIsvJR1lWtjbyw/8jTfv53hiIrRBx31pVm4xyvQRnHI6yvThtNQSKDnOq2b6sLILva7rQDL+EeMwgmleN7fpA8OU8WTRb8LhME899RQrVqzAcRwCgQDz58/n3HPPxefz9eu1JcmLfdPau1dY27RHbMJtMUq3VNFcU0rlJ6tJToaKhhIcx8DQzWwoP5GWcA4ZyWm0Rs7Hcff/C9y1tQxgmTHGFJRiGprKuhFMLPwIAsVgeWOgMSfIyAKTkaMsfIFUDJ/XlR21HTKzDbQ/REpmDFsnUdVUjaMbKK2rwHEdNldtITM5gwLXYaN2cVyXDeVvk5Xszex1q10c1+GD8nWEo+H9xv7RrvU9ynpL8I6TQTRWgmG0eDOS461djYGhIjhuOtoNxhfVMBITrjQWrpuG1iaZyekYZOMzTRSwq7adE8enYxmKUNShKDuJyYUpWIbyJkoZikjUZVphkBGpYEaaSDIcgjqEXbUNQ2nat7+O21KNFUzHba6mfdPrmGk56JYafNj4sDsTbkdvQQPQ820nuN47wGns+TlYeSXoaBgrIx8ruxDfyPG4oSZ8I8bjL5gElje2rEwTIzXHGztWBiiFUoaXjE0LMzUbTB/qMExMEqK/1dTU8Nxzz6G15rTTTuOqq64iM7N/7wzpIBPvhjHtRGiprcSN1kGsBt34JlX1uaxe7QO7nh1Vk2gJZxOJJROJph7Stfw+hzFFDfiTAvj9BuPHxcAIMuaYInLyk0hLD2L6LLThUlpXRlVzFaV1ZfgtH7brELOjrN7+Ltkp2VQ1V1PdXE1FUwV+K0Bda91h+kT2LRotwTCbiESOxXW8/0G9CVrZgMZxstHah+umx2egm3RtJc+blkV5fTunHZNJZWM7M4rT8FsKy1SYhqI5bDM2L0hywMQyFAUZFsXpDmZrDZaOoh0HN9xM6OOVmMkZtK5ehpGcGR/At9Gug11fgd1YgZWRj9Nci45FDvl9GymZaDuGbm/DP3oayrC8ZGuYRLauJvuqH2D4ktB2lEDJcfExaa+FbGbko0wLI5iOEUg+5FiEOFrs2rWLwsLCxP6KFSuYMGEC48aNO+BzHcrEO0nyQ5jWmrrKZj56801Ktzbx+nvTGJVbih3TVDeNOejzpgWbaAlnMHvaJpTpJ7doHK1tBiXjTKxgAMf0E8yM4PpCNIbrqYtU4rg2H5Z/xPbaHeysK6UgowDbiVFa761sbBkmtntoM6Qt08IyLCLxxDYubxy2A5VNNeSnTaGmuRrbLiE5YKHIob7Voa1do7XXmradfG/SFh1jyppYrDg+1tzJb6l4a1nREnHITfNx4vh0dje0c8qkTLTWzD8hD59pYJqKiSOTSfJ5yV47Nu2lHxLd/QnKMIhWbgWgfft7mGl5hDe+jt2w+5A+h94ofxAdDaMCKfjyxuCGm7GyCvCPOgYn1ERSyXFYOaNRhoGVXYQvvwTlD0piFuIANTQ08Oijj7Jq1Spuu+02Zs6cecjnlNn1gtrqKEsfriQ1OcbGD3ZTXZ8RXzsZvMUEvSUYd9f2XBbYb0VI8reC8tMaTiErI4ptw7nnugQyRpA/Op2kFEV6FoTsNrbVNBBzqvjLK4+i0WzessU70QEsZbStZlu3/Y4EbxomTnz7uOKZmMpkbN5YTMMENJZhMTZ3AqGYTXs0DcdJIcmXw2NvhzlmVCoxW7PwzUovnMrO8+9k/0wDUpMsfKZi9rh0DEOREjD53JkF5KX5SEmyGJGx91nOWmvsunLsulLa1jyEXVtK67vLqcofi7aj2PW9Lui4f4aJMiz8hZPBtIhVb8eXM5rkmefhhltInnpm521R8VujrIwRKF8A5Qt4XdtCiH4Vi8VYsWIFTz31FO3t7ViWRU1NzUCHJUn+aNRQF6NsRyu7t5Xy2NLexiR7jvVkpdUzZXw5k6akMX7GVMxgHoHkAFGjjfq2Bqqaory8cSU7y9axqmozJbljWLG+L6mxu4AVwDIt2trbmJA/ntrWWibkTyAcDXNCySwc12HSyEkU54wmOyUby7QI+pJIT0rHNMxuiz/EbJfNlSHuenInb21uwnE1jaE913F2gGoA3t7S82lNfksRtTUnT8zg3OnZaKA4J4minAA+08AyFUXZAdKD1n5XxXLDrUQrywitf5nwxtcxghm4rfW0l60HpbBre5/iHqve3qPMyitBWX6Sxs/27m0ePxsdjeAfPQ1cl0DxdKycQoyArCkuxGCmtWbt2rUsXrw4kdRnzZrFNddcQ35+/gBHJ0l+0Nu2OcTyR6r4eF0rwaBDONx1EYueCX7siA85a9pjmIFsxs48ifQcl3faxvLOjnVsaq3n7aZGNj3yP9S31bOvoZodtd0TfFpSKqZhYhoWda115KfnM7tkFu/ueI8fXfZ9irKLGJc39qDeY2vEZl1piHv/r4xtVSFqW2LUtvR2z1dP6UGTkydm4LcM6ltjXHPyCCzTYESGn7OmZu31dVprdKQVu6mKyHtv0xxuJlq5FbuxAqUM2ss3YNfswAimox0bfYCz5a3cYnAdMs75Mr78sd5YteXDCKRgJB3a/AYhxODx2muvcf/99wNQWFjI9ddfz9SpUwc2qC4kyQ9Crqt57KEyVjzd1K28e4KHgqxtZKVWMXpkLWdfVER5UjF/fu0Vbtq6Hcssw966pk/XG5kxgsqmKmYUHcvkgmMoyR3D1FFTyUvLJSM5g7SktEN6P6W1Ef76YjlbqsJEbZe61hgVDe1YpqKmef/JfGSmH79l8IMrxnLmlCzSgyY+q/dZ1dqOEtn2Hm64mYbnNhPd/Qmx6m1Etr2HL7eYWOUWMH3g9O1LhBvu5V4upfDljyVpwhyCE0/CjbVjZY/CTMshqeR4lNW/t8QIIQaW1jpxy+WcOXN4/vnnmTt3LnPnzsU0B9dqgjLxbhCJRV1+9cNPKC/tmYAuOP6fzJ7wPKOKUqk1i9hY18BLrSfyfvnHVDRWENtL0uoY405LSqUoq4hzps6jILOArORMxuWNpSCzb4vFtIRtVm9rZkdNGIXi/R0tlNaGyU33E3M0WytDbKoIkZFsYTsa19XYribm9P33Kz1oMntcOp8/axQTRyYzNn//62HbTdW0rX2Omge+0+frJHTcY21aKCtA2klXogIpmClZ+EdNRPmD+AundK44ZvrkXmohhjHHcVi5ciWvvPIK3//+90lK8ibluq57WNaZ3xuZeHeUs5s38PPbd1BZ3/35xIZh8+3Lb2XSaQsgcCpVfI7Z934R75k+AM/1er7TJ57Gj+f/gLy0vH3+4jWHbD4ub2Xj7hCrNjeyvqyNnFRfPDm7vLqh8YDfS1OPMfNOqUkm37q4mAkjkwn6DUZlBUj2m4zKCvQ6Hu6GWwjveB+3tZ5o9XbctkbCG17DzBxJ6IMV+4xD+YMkHzsPp6maQPGxBMadgL9gEmZaNmZqtrf8qNxjLYToow0bNvDwww+za5c3gfatt95i7ty5AP2a4A+VJPmBoDVO3Su889i9vLbhCrZUHE/H7HeArJQqfvmZK9EnrOKxDd/kpc2N/PHl/+xxmvnHXcrssSeQGkhlTO4YxueNQymF1pqXP25g465d7KyNkBIw2VoVoj3m8tL6BixDYbsH14MzKivA5FFeKzvmaAqzAkwsSMYXv9f7mFEpZCZbmPFbzCxT9doa11rjhpuJlq8jtKMOHYvSumY5dn0F7aUf7rs7fee67vumBY5NwW0PkTTxJMzkjIN6b0IIsaeamhqWLFnCmjXe8Gdubi7XXnsts2bNGuDI+kaS/JEU3kr9S/N4dNWtvLvlfOBnParc9NMo97/3DPPXnsXulz/f62lunPtVbp739cR+zHZ5dUMD9zy9icWrqvYbxp4J3jIU40cEmV6cyoziVCaMSMYyFT5TkRa0GJObREayddBd1W6kldDmtwlvfIP20g8Jb3wdnL23+PdkpuV6T97yB7GyCzHTskkaOwtMH8HJp0kXuhCiX7z44ossXrwY27bx+/1ceumlXHDBBf2+FO3hJEn+CLE3/4AH/5XKqk1PdSu3zBgnnA337vw6Wjm8/XDP12alZHH1CZ/imIJJnDv1HKK25q3NTTz4WgWPvLXvpH7BjBwKswOU5AVpt11K8pLITfMza2waAcvY721jB0JrTWz3J9iNFTS/sZjWtx7d72uULwkdi5By3IUYKRk4rQ2knnApgZKZ+PJKZDEWIcSAycnJwbZtTj75ZBYsWEBW1t7v2BmsZOJdP2qoi/HU3x7htfeO7XHs2OOSGT+3kjueubXX104vnMbVJ17FBdPPJ9kf5OX19Vx7z4f7vF5Oqo/Z49K57rSRXHxcTr+3cCPb3qP51QeJ1ZbRvvMD3LZ9j+EbwTSCk08nUDKT4KRTSJp4soyLCyEGje3bt7NlyxbOO+88wGu47Nq1i6Kiov28sn/JxLvBxo3ws+98wK6KZKB7gk9OUfjPXsp9nyyHZ7q/7HOnfobvXPgtwPvlWlfaymW/+ZgPy1p7vczonAB+y+CbF4zmulNHHtZWOYB2HcKbVtG+cx2Rbe8S3bUJK3ME7aUf7jehByefRqy2nJwrv0fqnCtRpvyqCSEGp6amJpYuXcrrr7+OUorJkyczevRolFIDnuAPlfzLeziFPqHi+cv4ycJHgc5u5mMKV3P1TVeyqvZ5fvXMb+CT7i/75jm38JWzvgSA7WhO+8lqtlf3/gS0Oz89kc+fWdBvrfTmN5fQsuoRYhWbe12GNVbxSY+y5BnnknzsuSRNmE2geIaMkQshjgq2bfP888/z5JNPEglNrywAACAASURBVIlEME2T888/n9zc3IEO7bCRJH842C1sfORC/t+yPwHdx6H/fJ+ClC9y3E+697QcM3ISd137G8bkeGvJ245m1E2v9jj1qKwAqUkmf/3aVKYU9s8Sp5Gta2hb9zwNT/73XuskH3sO/tHT8BdMxJc3BuULEiiagvIF+iUmIYToTx988AGLFi2iqsqb13Tcccdx7bXXMmLEiAGO7PCSJH+oXk3Cdiz+37JV3YqPmZbCdd9I4rL7bqKsvrzbsYdvfJBphVPRWvP2liZ+uHgLH+zs2SW//d7TSQkc3tWTtOti15XR+s4ThDa+Tnj9K73Wy77yDgLF0wlOPk3WTxdCDDlvvfUWVVVVFBQUcP311zN9+vSBDqlfSJI/WNFKeKuEtkg63/r7i4niy67N4odrL+ftGDzw2+4vSUtK5fXvrwTgsXeq+fp9G3qcNjlgsOPeMw5bmE5rPaH1K2l+7SEin7yFttv3Wjf1xMtJOeES0uZcediuL4QQg0EoFKK5uZmRI0cCsGDBAsaNG8fZZ5+NZQ3dVDh031l/itWjV5XwzJovseydmxLFuWPb+OHaL/SofvGMC/n5FT/FNCz+uXI3331oc7fjHQ9Z+fNXpx6Wlnt489s0PH0PoXXP77OemZ5H6uzLyDj3a/hHjj/k6wohxGDjui6vvvoqjz32GNnZ2fz4xz/GMAyys7MTs+iHMknyByq8lfLn5vPzxau7FU89UfGPplsS+ymBFJ779tOkB9PQWvPYO9Xc9LeNPU735H8cx0kTDs8Kbe1l6yn7ydxej/mLpuIvmkrm+V8nUDzde/a4EEIMYZs2bWLhwoWUlnqPgR41ahStra2kp6cPcGRHjiT5A+FGKX/2cn6+ZFG34q/eVshXnu38Rvj3L/2VE0q8JQ/3NqHurk9P5DNnFGAe4m1vbixC88v3U7voRz2Opc6eT8Z5XyNpwhyZ8S6EGDbq6upYsmQJq1d7jbHs7GyuueYaTjzxxGH3b6Ek+b4Kb4fVU/j5ks4W/PwF+Zw3P5M5vzg1UXbVCVcmEvxHZa3M+0X3x70u+/eZnDIp85DDaX13OZX/++Vej+V/8Xekn/HpQ76GEEIcbWzb5r/+679oaGjA7/dz0UUXceGFFxIIDM87gSTJ99XqKd12c/J8FM6uYM4vLk6UnTHxNH58+Q8JRx3GfOP1Hqeo/stZhyWULV/K61Fm5Ywm+/Lvkn769YflGkIIcbTQWqO1xjAMLMvi4osvZvPmzSxYsICcnJyBDm9ASZLviw8vR2u6zaL/0W8mcsqv5yT2Txl/Mv/z2d9T2xJl6ne630737UuK+d78kkMKIbLzA3b9ej66PdStPHX2fEZ8/a+yPKwQYljauXMnCxcuZObMmVx00UUAzJs3j3POOWeAIxscJMnvi9bwzjG0NDbznX90n2j3w+W3J7ZvPfcWvnzml9Bad0vwE0YEefMXczgYWmsan/sDkc1v0/Z+78+Nn/D3moM6txBCHO2am5t5/PHHefXVV9Fa09DQwPnnn49pmsNu3H1fJMnvy2tBtKZHgt9w7Ldo/rghsf+lM77Im5saueK/P0iUzZuWxaLbZhzUZSPb36f8F73f2pF9xe1kXngzhj94UOcWQoijmW3bvPTSSyxbtoxwOIxpmsybN4/LL78c05S7hvYkSb437bvgw0sB+NfK7yeKL70qn5aiF3n72c4E/+RtTzL+tjdojTjdTvHQN3s+eW5/nFAztQ/dQcuqJd3Ks+Z/l6Sxx5E84zz5hiqEGLYaGxu56667qKioAGD69Olcd911jBo1aoAjG7wkye+p/Pew7T8A2FY5ndc+7lz97aQLFOff3bm++5qfrGbUTa91e/lfvzaFy2fnH/Blm176OzX/+l63srzP/z8yzvrsAZ9LCCGGooyMDJKTk8nPz+e6665j5syZ0vDZD0nyXbVtSCR4gF8/9o/E9m/vm8qpd3aOr/9mwa96JPiDmT3fXr6Bsh+f2a3MXzSNglsfxJc7+oDPJ4QQQ0U4HOapp57irLPOIj8/H6UUN910E2lpafh8voEO76ggSb6DtmHN8QC4WvG9he8CLgCnz8uiIVaZqHry+JP43L2d91ymJplsvee0Pl/Kbq4hvPENqv701R7HCm9fTnDSKQf5JoQQ4ujnui5vvPEGS5cupbm5mcrKSr75zW8C3sI2ou8kyXco63yazNf/+A4dCR7gM18p5LN//Xxiv7nxRqAxsb/t96f36RJuLMK2G3tvnWdf8T2y5//7gcUshBBDzJYtW3jooYfYuXMnAOPHj+fSSy8d4KiOXpLkO+z8JQCvfHRVt+I/PDANw1B8tGs9AMn+NF5c35ngK//Uvau9Ny1vP07Vn7/W84DpI/fan5F5bs8WvRBCDCcNDQ088sgjvPXWWwBkZmayYMECTj75ZBl3PwSS5AEaXwHdjtbw8Kud97//eeH0Hr9cdY2dSX3tr0/C2M/a840r/tRjXXn/qGMo/mXPFfGEEGK4ikQirF69GsuyuPDCC7n44otJSkoa6LCOepLkAdZdCMAL6zqXhL3tjpJEgq9qrk6UNzZ7Sf6MyZkUZu//F7Brgs/77F2kn/U5WZ1OCDHsaa3ZuHEjkydPRilFQUEBX/ziF5k4cSJ5eT2X7hYHR5L86pkA7Kw5hkfe+HaieNrMtMT2fz75qy4v8GZ0/ukr3dey743WOrGd/5U/kH7qNYcYrBBCHP3Ky8tZuHAhGzZs4Oabb2b27NkAnHrqqft5pThQwzvJR8ogvAnbMfnPR/6VKL7tjpJu1VZu8h4V6zipAJw7PZu8dP8+T93y1mNU/eXGxH7q7MsOU9BCCHF0am1t5fHHH+eVV15Ba01KSgq2bQ90WEPa8E7yTd64+KvrOyfbXXpVfrdWfHl9eWK7vv4WAB6+dd+r2e2661OEN3TeQx8YO0uWoRVCDFuO4/DKK6/wxBNP0NbWhmEYiaVoU1NTBzq8IW14J/lNX0RrWPT6dxNF8xeM6FblX6seTmzbTvdjvalZ9KNuCb7oRytIGnv8YQhWCCGOTq+88goPPfQQAFOmTOH666+nqKhogKMaHoZvkm9+B4B/+9tLiaJ5F/Z87vDCtxcD4Lpe93x2yt4/sup//BvNrz2U2B/3h+0YQfmWKoQYfmKxWGJVujPOOIP33nuPc845h+OPP15uiTuChu8073UX8snu4wlHva55y1Jc94XuDzloDrcktltavDH1vXXVV913S7cEX/TjFyTBCyGGnUgkwtKlS7n99tsJhUIA+P1+vvvd7zJr1ixJ8EdYv7bklVIXAvcAJnCf1vrXexwvBv4JZMbr3K61fqY/YwK858S7Ie5+4i+Joj88OK1bFcd1OONXcxP7ofDJAIzN7zm2Xr/8v2l5s/PJcWN/vwkzVZZeFEIMH67r8tZbb/HII4/Q1NQEwAcffMApp8gy3QOp35K8UsoE/gCcB5QDq5VSy7XWH3ep9kNgidb6j0qpqcAzQEl/xZTQ5M2W91thonaQuedn9/h2+ceX/5zYPmnsZTxR6T2nOCPY8yNre+/pxLYkeCHEcLNt2zYWLlzI1q1bARg7diw33HAD48ePH+DIRH+25OcAW7TW2wCUUouAy4GuSV4D6fHtDGB3P8bTad0FAERtr1V+4fyeCy/srN2Z2H5i1dmJ7T1XuGvftZH20g8ByPvMbyTBCyGGleXLl/PEE08A3qNgr776ak455RQMWfRrUOjPJF8IlHXZLwdO2qPOT4EVSqlvAinAuf0Yj8f2xtlbwhmJorT0nh/DivUvAHDNiZ/mnie9srOmZPWoV/ajMxLbycddcDgjFUKIQW/ChAlYlsV5553HpZdeSjAotwsPJgP9Vet64H6tdRFwMfCgUqpHTEqpryml3lVKvVtTU3NoVyz/HQB/ff6/EkU+f/dLdl3GNi+l8wlzS/6t+6S7mod/kNhOO2UBvuzCQ4tNCCEGMa0177//Po8//niibOrUqdx5550sWLBAEvwg1J8t+V1A1+eqFsXLuvoycCGA1nqVUioJyAWqu1bSWv8F+AvA7NmzNYei8UUANpbP2WuV8+++KLG9dFXnF4Cu4/ZOawNNL3RO3Bvx1f89pLCEEGIw2717NwsXLmT9eu+JnLNmzWLMmDGA98Q4MTj1Z5JfDUxUSo3FS+7XATfsUacUOAe4Xyk1BUgCDrGpvh/Nb9FlSXmu/VxB98Ph5sT2ZcddQltrGmt3tjF9dPfb4XbdeUVie/TPV/ZPrEIIMcDa2tpYtmwZL730Eq7rEgwGueKKKygslJ7Lo0G/JXmtta2U+gbwf3i3x/1da71eKfVz4F2t9XLgO8BflVLfwpuE9wXd9aku/eT+l36S2J57QfcFcNaWvp/Y/unlP+Z7D28D4AtndX4ZiNWWES335g+qQDKBoqn9Ga4QQgyIlStXsnTpUlpbW1FKMXfuXK644grS09P3/2IxKPTrffLxe96f2aPsx122PwZO688YuqlaCEBrpHPSnWl2ny1/7wt/AOCEMbOwTIsHX6vodtwJNbPzP2Yl9sf819v9Fa0QQgyorVu30trayqRJk7jhhhsoLi4e6JDEARpey9pW3AdAUygXgPkL8ntU2Vy1BYCslO4z6WOO18FQ8dvrEmWZF9yMlTWyX0IVQogjrba2lra2tsRY+1VXXcX06dM58cQTZaW6o9TwSvJRr1VeWuM9Cz47p/vjYruOFHzxjM+zbmfnsrbXnzoSbUeJbF0NgG/EOHKv/Vl/RyyEEP2uvb2dZ599lmeffZa8vDx++tOfYlkWGRkZzJmz90nKYvAbXkk+so32WCCxWzK+++0eW2u2JbanFExm1E2vJ/aTAya1S36R2C+846l+DFQIIfqf1pp33nmHJUuW0NDQAMDo0aOJRqNY1vBKD0PV8PlbjOwAYHd95zKLo0Yndavy38/9NrFtGmZiFv6c8ek0PPN7Gp/zxuut3DFY6T1XyRNCiKPFzp07eeihh9iyxRuiLC4u5oYbbmDSpEkDHJk4nIZPkv/YG0t/4YM97+Lr9OaWVYnt6uZoYnvJbdPYffP5if0RX7m3HwIUQogjIxaL8bvf/Y6mpibS0tK46qqrOP3002Up2iFo+CT5Vu/WuNVb9r/07N3X/oaPyloT+/qTVxPbJb/9CCtjxOGPTwgh+pFt22it8fl8+Hw+rr76asrKypg/fz7JyckDHZ7oJ8MjyVf9C6DbIjif/dreF3I4Y9Lp/Pu/dgCQkWxR8burE8ckwQshjjbr1q1j0aJFnHTSSVx++eUAnHbakbt7WQyc4ZHkKx8A4Nv/eD5RdMa87k+La2tvS2ybyiRmuwBEI+2J8vSzPtefUQohxGFVWVnJokWLWLduHQBr167lsssuk275YWR4JPm29WgNbZG9r69c3tC5rL7P8rG5MgzAt2Kdz5XP++yd/RejEEIcJqFQiCeffJIXXngBx3EIBoPMnz+fc845RxL8MDM8krxdx6pNlyR2f/e3nsvQfm/JHd32O8bk2/HupR97z0aUYfZjkEIIcejq6+v5+c9/TnNzM0opzjzzTK688koyMjL2/2Ix5Az9JO/GAHhj4/xEUXJKz2S9vXZHry+fojcDYKbl9HpcCCEGk6ysLAoLC8nPz+eGG26gpKRkoEMSA2joJ/nK+wHYvNtbb37Ksak9qvzq6c5u+Dd/8Cobt1cl9ouoYsyd7/VvjEIIcZDq6+t59NFHueSSSygsLEQpxc0330xycrIsRSuGQZLf+Utsp7PlPu/Cni3yRW8vTmynBFJ44pdfB/MrAEy6+R58uaP7P04hhDgA0WiU5557jmeeeYZoNEokEuHWW28FICUlZYCjE4PF0E/ysSre335uYnfmCd0fkdh1wt2imx4C4LfGlxJlqbPnI4QQg4XWmjVr1rB48WLq6uoAmD17Ntdcc80ARyYGo6Gd5OPj8Q+88gMAeuu5enH9i4ntKQWT0Vpzsl7LKnUCFx0ztD8eIcTRpaqqin/+859s3LgRgKKiIm644QYmT548wJGJwWpoZ7G2jwCIRL1x+Fkn9ZxduqNuZ7f92O5PaFPe6k83nCNrOAshBg/DMNiyZQupqalceeWVnHnmmZim3PUj9m5oJ/kK7x73nLQK6loKOPfinuPxj615AoBzp54DQHjrO2zEe4iNrWXSihBi4Ni2zbvvvsucOXMwDIO8vDxuueUWxo8fT2pqz0nEQuxpaCf5yvtpCWdQ11IAQFp697fbEul8XvzZU+YC0LDsbpL4T6L4GZHR/XnzQghxpKxfv56FCxeye/duHMdJLEM7c+bMAY5MHE2GdpIHPtx5RmI7J6970tZdFrO/dObFaNch1FBNs5UGQFbKkP94hBCDTHV1NYsXL2bt2rUA5OXlkZ6evp9XCdG7oZvF4gncNGwAxowLYppqjypenbQkL6m3l37ETjofXDMuP3gkIhVCCMLhME8//TQrVqzAtm0CgQCXXnop559/Pj6fb6DDE0epoZvko96tcbvqJgCQltHzra4t9R4/29FtX/m/X+ZB48rEcVlIQghxpLzxxhs888wzAJx66qlcddVVZGVlDXBU4mg3dJN8ezkAjaH8vVbZVrO9274vr5gXGk8HYN40+Z9LCNG/WltbExPo5s6dy9atWzn33HMZP378AEcmhoo+P45Iqfh9ZUcLuxGA7VVTAEhK6vlWk3xJAIzP9/6HCm94LbFW/eRCWTFKCNE/GhoauO+++7jjjjtobm4GwLIsbrzxRknw4rDab0teKXUqcB+QChQrpWYCN2qtb+7v4A5JvCVf1VgCwOgxST2qLHxrEQCzxhyH2x4CwMB7jvzFx+cegSCFEMNJLBZjxYoVPPXUU7S3t2NZFlu2bGHWrFkDHZoYovrSXf9b4AJgOYDW+gOl1Jn9GtXhEPJWhEpOChGKJDN1RlqPKqX1ZQDsatjNtpvGoIH16hggMW9PCCEOmdaa999/n0WLFlFTUwPA8ccfz7XXXkt+/t6HFIU4VH0ak9dal+0xCc3pn3AOo/A2tIZQxBtlyM3vfvuc7diJ7SuPuxReWcxr6sRE2aSCo2t0QggxeC1atIjnn38egFGjRnH99dczbdq0AY5KDAd9SfJl8S57rZTyAbcBG/o3rMMgVkt1U+fT43z+7jPln1//QmL7rNxxlAHvqmMTZVkpcsuKEOLwOPHEE3nzzTe5/PLLOfvss2UpWnHE9CXJfx24BygEdgErgME9Hu9GoeUddteflSjy+7tPvFuzs/MZ8eGPVwKQFXAgBseN6dm1L4QQfeE4Dq+++iplZWV87nOfA2DChAncfffdBAKBAY5ODDd9SfLHaK0/3bVAKXUa8Eb/hHQYNHmhWab3FLr8kT2Xp81N9SbWnT35LFrfeRyAD9uLwIB50+X2OSHEgduwYQMLFy6kvNyb+HvmmWdSUlICIAleDIi+JPl7gT2nfvZWNnhsvwOALRXeGs97jscDPPruYwBMHDGJyAsPevWTpkIUaltiRyhQIcRQUFtby+LFi1mzZg0AOTk5XHvttYwZM2aAIxPD3V6TvFLqFOBUIE8p9e0uh9KBwT2g1OqtZLetfh4AtdXRbocd16GmxZvhuq38w0R5bdT7pj2jWJ7uJITom2XLlvH0009j2zZ+v59LLrmECy64AL9fHnAlBt6+WvJ+vHvjLaDrIHUzcHV/BnW4ZOTmwQ446fTMbuWbq7Yktr894zLCK5cA3lr126rDzCiWMXkhRN+0trZi2zYnn3wyV199NdnZ2QMdkhAJe03yWuuVwEql1P1a651HMKZD43bpardygAj5I7uPhW2u8la1y0nNIfav7wHgHz2NbRVhAFICg7ujQggxcHbs2EF7ezvHHOOtqXH55ZczZ84cJk6cOMCRCdFTX8bkQ0qpu4BpQGLZOK31vH6L6pB0uYXfCACRHjU61qx3HBu7YbdX6Ha+Ll+eIy+E2ENTUxNLly7ljTfeIC8vj1/84hf4fD5SU1MlwYtBqy9J/iFgMXAp3u10nwdq+jOoQ+LGx99VgFDYW6J2z4fJvbD+RQDG6M4DGd9aDrevAyDo7/OS/kKIIc62bV544QWWL19OJBLBNE1mzZqF67oDHZoQ+9WXJJ+jtf6bUuq2Ll34q/s7sINW+Xfvp27no7XeI2RTUju732NOLLGcbabZ+fYrIp2td78lSV4IAR988AGLFi2iqqoKgBkzZnDdddcxcuTIAY5MiL7pS5LvGOSuUEpdAuwGBu/MEjMdgEi08ylyeSM6E/iDbz6U2P50xHtrWZd9hx89W3qEAhRCHA1isRgPPPAADQ0NjBw5kuuuu44ZM2YMdFhCHJC+JPlfKqUygO/g3R+fDvxbv0Z1GLSmfSmx3XXi3T3P39tZ3liFAxjJGWRHvWVsZc16IYavUCiEYRgkJSXh8/m4/vrrqaur45xzzsGy+vSoDyEGlf3+1mqtn4pvNgFnQ2LFu6NCTm7va9BfO2cBzmP/DYCVnsva972u/c+eUXDEYhNCDA6u6/L666+zdOlSTj/9dBYsWADA7NmzBzgyIQ7NvhbDMYFr8Nasf05r/ZFS6lLg+0AQOP7IhHiAwt7tca5W+6w2//jLIJ7k/aOOYUdNKwANbbLanRDDySeffMLChQvZudO7U3jHjh24rothyNwccfTbV0v+b8Bo4B3g90qp3cBs4Hat9RNHIriDorxJdvXVXtKuq+1M2rs6bpcDxgazqIhvW3kl1DSvBWDCSOmuF2I4qK+vZ8mSJbzzzjsAZGVlcc011zBnzhzUnrfkCHGU2leSnw3M0Fq7SqkkoBIYr7WuOzKhHaSyuwEwUqcC3SfdLVu7PLEdefVfiW0zOT2xfeqk7qvjCSGGnpqaGn70ox8RjUbx+XxcdNFFXHTRRfIQGTHk7CvJR7XWLoDWOqKU2jboEzxA+qnQ/Ca43iI4mVmdb/HPr/wVgAn542lYfhcAypdEqL1zIZzcNHmOvBBDXV5eHpMnTyYQCLBgwQJyc3MHOiQh+sW+Bp0mK6XWxf982GX/Q6XUur6cXCl1oVJqk1Jqi1Lq9r3UuUYp9bFSar1S6uGDeRPdNL8JQDR4So9Dp004FYATS05IlI385j9xdWcduUdeiKGntLSUu+66i9LSzltlb7nlFm666SZJ8GJI21dLfsqhnDg+ce8PwHlAObBaKbVca/1xlzoTgTuA07TWDUqp/EO5pndSC7RNXX0QgMYGO3HojS3eF4CpW9YkypKnnsWWqvZDvqwQYvBpbm7miSeeYOXKlWitWbZsGd/85jcB8Pmk104Mfft6QM2hPpRmDrBFa70NQCm1CLgc+LhLna8Cf9BaN8SvWX1IV9QuaC+pO6QBEbLjt9A9u+65RDVr4xuJbWWYvLS+HoARsma9EEOCbdu8/PLLLFu2jFAohGmazJs3j/nz5w90aEIcUf25ukMhUNZlvxw4aY86kwCUUm/gPaP+p1rr5/aog1Lqa8DXAIqLi/d+xcr7E5uLHvLG5E3TmyW7fnfnd4tjMNFA/pf/B4Dla7yl+P2WzKgV4mhXWlrKn//8ZyoqvPtnpk2bxvXXX8+oUaMGODIhjryBXsLJAiYCc4Ei4FWl1LFa68aulbTWfwH+AjB79my950kSNt+c2Jw8LYWN69sSD6d5/ROv9X7u5Lno8gcASJl5PgCrtzYDcOL4jEN/R0KIAZWWlkZdXR35+flcd911zJw5U26JE8NWn5K8UioIFGutNx3AuXfh3WffoShe1lU58LbWOgZsV0p9gpf0D+4BONkXQf2zkHMZ0aj3XeCci7xJNdUtXms9v8uTo4yUTJ56r/OBerLanRBHn3A4zMqVKzn//PMxDIOsrCz+P3vnHR5F1fbhe7LphEAg0lvoBFIhFOk9dANIAKUKImKvvAqKCi8WFPRVRD8VpCWhSBOkCUgRpQakhxIkhZIQ0tvunu+PSSZZ0jZAsgmc+7py7c6ZM2ee2Z3sM+ec5zy/N954g/r168t5d8kjT5FOXlGUQcA8wBZwUxTFG/hQCFHU5NZhoImiKG6ozn0kMPquOuuBUcBiRVFcUYfvLxfvEnIRt0t9rTGey2EpABgNqrPPNKhJcVo4PaZel409iqIwcVHOMP7jTWVPXiIpLxiNRv7880/Wrl1LfHw8dnZ2dO/eHYDGjRtb2DqJpGxgTk9+FmoQ3R4AIURoluMuFCGEXlGUF4BtqPPtPwkhTiuK8iFwRAixMWtfH0VRzgAG4M37WosvsqPkrajorCMxwUClKuqTfIZe1ZmvEncdAJvqjUwOndyjthzSk0jKCZcuXWLlypVcuXIFgEaNGtGgQQPLGiWRlEHMkpoVQsTf5QALnhfPXUmILcCWu8rey/VeAK9l/d0fKblmEpw8SEyIA8DWVuFmQs6QvMu+YAAUnTWpGTlJcF70zz2zIJFIyiJxcXGsWbOGgwcPAlC5cmWefPJJ2rVrJ3PNSyT5YI6TP60oymhAl7Wu/SXgz5I16x64/VvOe7s62NrdISNd4FLVhjm/fZOzK+v5xM7Nmz/OxGnlNSrLdJYSSVknNDSUgwcPYm1tTd++fRkwYAD29vaWNksiKbOY4+RfBN4F0oGVqEPss0vSqHviclZCvcrqnJzIiq9TUEjLVJfTVbCroFWvMugNfll9f8vyJRJJySKE4ObNm1SvXh2ALl26cOPGDXr06EG1avefO0siedgxx8k3F0K8i+royy5O3pAUCigkJxnIzMyZUdh+agcA/h594dJZAKxdanD4UjgAT7R5rLStlUgkRRAREUFQUBCXL1/mv//9Ly4uLuh0OkaOHGlp0ySScoM5Tv5zRVFqAGuAECHEqRK26d6wyhqyqz+DmFsZWnG6SNLed2ryOLBA246MUwP1GlZ3KBUTJRJJ0SQlJbF+/Xp2796NEIIKFSoQFRWFi4uLpU2TSModRTp5IUT3LCc/AvhOURRnVGdftobsRXYQXU6AYN0G9sQm5QTru/8RRLbLz9TnrJfvOzqJGQAAIABJREFU3Fz+eEgklsZgMPDHH3+wbt06kpOTURSFHj168MQTT+Dk5GRp8ySScolZyXCEENeBrxRF2Q28BbxHWZuXT8zOn2M0LU7L6cknHVqnvX/555xofK/6FUvUNIlEUjQ///wz+/fvB6BFixaMGjWKOnXqWNgqiaR8Y04ynBZAIDAMiAVCgNdL2K7iIXKWwuHQhNis4XphhFtZme5aVW8CEarMpOtTc9mxSRWlcbSzwsleV7r2SiQSQA2sy16e26NHD86fP8+IESPw9fWVeSskkgeAOT35n1Ade18hRFQJ23P/2FYjPU1dGhdzK4NVh9cAcObGRa1Kpe4TiA9RewzLprUqfRslkkec9PR0Nm/ezPXr13n+eVVzokGDBsydO1eud5dIHiDmzMl3KA1D7gtDssnmiSOq4EztevZkVqnDX5f+pl7W8jn7Rn4oVjqsFDAKcK8t5/okktJCCMFff/3FmjVriItTH8avXbtG3bpqMirp4CWSB0uBTl5RlFVCiBGKovyDaYY7BTVZnWeJW2cumaaZcCtnpbI16AU6K/USe9xQh+rR6RBCYMy6IjlUL5GUDleuXGHlypVcunQJUHvuo0eP1hy8RCJ58BTWk38563VgaRjyQLCrD0DkNTX5jd/jlVh4+RAAis4WAIcm7biVmKkdYqOT834SSUkihGDZsmXs2bMHAGdnZ4YNG0bHjh1lz10iKWEKdPJCiOist88LId7OvU9RlE+At/MeZSGSjma9UbvnCXf0anGigcu3VAGLNKNaVqnXZE7dTtcOtbKSTl4iKUkURcHe3h6dTkefPn0YOHAgDg4yN4VEUhqYE3jXm7wOvV8+ZZYjXg2iI10dko+OVJ24U5WcpDju6WrvXlfBhbCzcUgkkpJBCMGJEyewsrLC01Od1Rs0aBBdu3bV0tNKJJLSobA5+anA80BDRVFO5tpVEThQ0oYVi1R1jo+qgzEac8IH7F1z1sg3z8jAtm5LFGtbNh1Vl9W5VrQpVTMlkoedqKgogoODOXXqFFWqVGHOnDnY2dnh4OAge+8SiQUorCe/EvgNmAtMz1WeKIS4XaJWFZfkM+qrjSvJSTlr5jMdbgBgp6jzfvosLfmb8WoPv6KDWbmAJBJJEaSkpLBhwwZ27dqFwWDAwcGBPn36oNPJwFaJxJIU5uWEECJcUZRpd+9QFKVKmXL0js0hIxKc2xJxNU0rttGpPfU66akA2DdQhw6PhScCEOAnhWkkkvvBaDSyd+9efvnlF5KSklAUhW7duvHEE0/g7OxsafMkkkeeonryA4GjqBFtuSPUBNCwBO0qHnd+V19ta5ORoaa1tbbJMdfZoJY5tRlMakZOT7+XR9XSs1EieQgxGAxs2bKFpKQkmjZtyujRo6lXr56lzZJIJFkUFl0/MOvVrfTMuUccmkBqGOiciItVl8e5NXIgKS3CtJp7F0L+ztGQb9NQ9jQkkuISGxuLvb09FSpUwMbGhrFjx5KSkoKfn59MRSuRlDGKXKSqKEpHRVEqZL1/WlGULxRFKVuP6qlh6qtNVXTW6o9Mpco2RMerc/C3s+YFravWJSFVXUrXrKZj6dspkZRj0tPTWb9+Pe+88w7r16/Xylu1akXbtm2lg5dIyiDmRJ59C3gpiuKFKkzzA7AM6FqShpmNyJWMzzpHMtbewYofDywFoIZBdeyKorD6LzUYz71OhdKzUSIpxwghOHz4MKtWreL2bTUUJzk52URcRiKRlE3McfJ6IYRQFGUI8LUQ4kdFUZ4pacPMRp8rpa1NFSBe27yTor6vqjfg0LwTABevpwCQrs+dqVcikeTH1atXCQoK4sKFCwDUq1eP0aNH07RpUwtbJpFIzMEcJ5+oKMp/gDFAZ0VRrICys8BcZOnHW9mDoiM+Tp2Tz+3Cn0hKoFKPiRiMQnPug1vLyHqJpDBu3LjBhx9+iBACJycnhg0bRufOnWUqWomkHGGOkw8ERgMThRDXs+bjPytZs+4BXUVAFaUBSEhK0XZVNRio0Hog3/8eqZU90UY6eYnkboxGo+bEq1evTtu2balUqRKDBw/G0VHGsUgk5Q1zpGavK4qyAvBTFGUgcEgIsbTkTbtHsuYIr6SE5hTZVUBRFH49pma6c7CxkjnrJZK7+OeffwgODmbixIk0atQIgGeffVbOu0sk5ZginbyiKCNQe+57UNfK/09RlDeFEGtK2DbzyE5pa0w3KU7XqwlwmqWnI9JVvfnUrDX0U3rVKT37JJIyzo0bNwgKCuLkSTV79Y4dOzQnLx28RFK+MWe4/l3ATwhxE0BRlMeAnUDZcPJZojQIg0nxnZQEcADftFSt7OS/ai57j3pOpWaeRFJWSU1NZdOmTezYsQODwYC9vT2DBw+mV69eljZNIpE8IMxx8lbZDj6LWMxYX19qpJxVXyt1MCm21qmXZisEFTuOJD3TqO3zaVCx1MyTSMoiYWFhfPPNNyQkJKAoCp06dWLYsGFUqlTJ0qZJJJIHiDlOfquiKNuAoKztQGBLyZlUTKyrZL1RhxUvh6kBd7cSb0FVqKXPxJiWyLXYnJz2daral7aVEkmZonr16mRmZtKoUSNGjx6Nm1vZT2wpkUiKjzmBd28qijIU6JRV9L0QYl3JmlUMUtX1uzg2B8DWVnX21gZ1SL6KwYBDk/Z8vjMi38MlkkeBuLg4duzYwdChQ7G2tsbZ2ZkZM2ZQo0YNOe8ukTzEFKYn3wSYBzQC/gHeEEJEFlTfYiQcVl/TVdNOHFUV5lJt1e2qBgN29T2Ju6yunx/atlrp2yiRWIjMzEy2bt3K5s2bycjIoHLlyvTp0weAmjVrWtg6iURS0hTWk/8JWArsBQYB/wOGloZRxSK7F+LQGADnytYk3NGTZqumr61kNGLfyI+krercfX9vV4uYKZGUJkIIjh07RkhICDExMQC0bt0aHx8fC1smkUhKk8KcfEUhxP9lvT+vKMqx0jCoWAgBSVnr4Su2BnKG69OtY6hkUCPuFWsbdp+OAyDDYMzbjkTyEBEREcHKlSs5d+4cALVr12b06NG0aNHCwpZJJJLSpjAnb68oig85OvIOubeFEJZ3+ulXc9679AQg5mamVtQxJQUrpyomGvIysl7ysHPlyhXOnTtHhQoVCAgIoGvXruiylBglEsmjRWFOPhr4Itf29VzbAuhRUkaZTcr5nPc6JzIzcnrpmdbxZGQo1Hj+J6LichLlNKouU3NKHi4MBgNXr16lYcOGAHTs2JHExES6dOmCk5PMCSGRPMoU6OSFEN1L05B7IvmM+urcEYC42zm9eKNVBolWVli71CAxVe3J13eVS+ckDxdnzpwhKCiImzdvMmfOHFxdXbGysqJ///6WNk0ikZQBzFknX3axdlZfjera+IR4vclu/+RErCvXYNo36sPA1Zg0JJKHgZs3b7Jq1SqOHVNnzVxdXYmPj8fVVQaWSiSSHMq3k8/GyRvICbRPsrsCQIt0dZg+LEtDvllNOVQvKd+kpaWxefNmtm3bhl6vx87OjoEDB9KnTx9sbMqOArREIikbPBxO/i6Eog7P2wA3knLKt0yXy4ck5ZulS5fy119/AdChQweGDx+Oi4uLha2SSCRlFXNU6BTgKaChEOLDLD35GkKIQyVuXVEkZgf4izy7BiUmYFO9ESsO3tLKKjo8lM80koccvV6PtbV67w4YMICYmBgCAwM1pTiJRCIpCHO83kLAiBpN/yGQCKwF/ErQLvOwyZp/zLgOwK0bGdquikYjtrWacfKq2pV/vKkU3pCUL+7cucPatWu5c+cOr732GoqiULt2bd555x1LmyaRSMoJ5jj5dkIIX0VRjgMIIeIURbEtYbvMJKsHX7EdAFcuqrKyTmkNaeqQjjE9mXQbdVndxG61LWKhRFJcMjMz2bFjB5s2bSI9PR1ra2uioqKoXVvewxKJpHiY4+QzFUXRkeVRs/Tky0bauNumYni3Y9UldAmOajS9fWM/dm9RM91VcpRD9ZKyjRCC0NBQQkJCuHlTVXf29vYmMDCQ6tWrW9g6iURSHjHH830FrAOqKYoyBxgOzDCncUVR/IEvAR3wgxDi4wLqDQPWAH5CiCPmtA2AdWX11UaVm719W10il2z3L1UNBkTd1lrV1g2dzW5WIilthBB8/fXXHD9+HFDFY0aNGkWrVq0sbJlEIinPmCM1u0JRlKNAT9SUtk8IIc4WdVxW7/8boDcQARxWFGWjEOLMXfUqAi8Df9+D/SqOak7ua5fVOfl0m1vUStdzo2Iz4DIATvYyraek7KIoCnXr1uX8+fMMGTKE7t27a8F2EolEcq+YE11fD0gBNuUuE0L8W8ShbYGLQojLWccEA0OAM3fV+wj4BHizGHbnISUlJz99jPOfkATnY+6nRYmk5DAajezduxcHBwfatVNjSvr160fPnj2pWFHqK0gkkgeDOV2Fzajz8QpgD7gB54GWRRxXG7iWazsCaJe7gqIovkBdIcRmRVGK7+TTcpq/fSsnsr6aQY2on79Njbp/yb9usZuWSEqK8+fPs3LlSq5du4azszNeXl7Y29tjZ2eHnZ2dpc2TSCQPEeYM13vk3s5yzM/f74kVRbFCFbwZb0bdZ4FnAerVq5ezw5CgvlrZaenu0qxv0T41BYO1A+ej1Ux3Y7vUul9zJZL7JiYmhlWrVnHkiBp2UrVqVQIDA6Vjl0gkJUaxJ/2EEMcURWlXdE0igdxd6DpZZdlUBFoBe9R8O9QANiqKMvju4DshxPfA9wBt2rTJyXyjVyPnsa3Bv5fV5XPCKhNXg4Hz+lra1dWTwjQSC5KZmcnmzZv57bffyMzMxNbWlv79++Pv74+tbRlZjSqRSB5KzJmTfy3XphXgC0SZ0fZhoImiKG6ozn0kMDp7pxAiHtDUNBRF2QO8UazoehRAgF0dkpNvA+CQUQtXg4FQxR2A2i6ylySxLIqicPjwYTIzM2nXrh1PPvkkVapUsbRZEonkEcCcnnzuKCA96hz92qIOEkLoFUV5AdiGuoTuJyHEaUVRPgSOCCE23ovB+aNgY2MFQEzFP9ErcNCuI+ihvcx0J7EAV69excXFBWdnZ6ytrRk3bhwATZs2tbBlEonkUaJQJ5+1DK6iEOKNe2lcCLEF2HJX2XsF1O12D2fQ3sUkxgJgsErFPT2dFMdKoIdu7lK8Q1J6JCQk8Msvv7Bv3z46d+7M+PHjAencJRKJZSjQySuKYp3VG+9YmgaZTcatXBsKYTcvkh0CUNEoOJv2GABe9eVyJEnJo9fr+f3339m4cSOpqanodDocHR0RQqBkayBLJBJJKVNYT/4Q6vx7qKIoG4HVQHL2TiHELyVsW+EIVSseXSVQFM5GncWeujgIwadWz2rVpIa8pKQ5efIkwcHBXL+uLtn08PBg5MiR1KxZ08KWSSSSRx1z5uTtgVhUFbrs9fICsKyTz0bnBIBLBRdSgep6PVFWagpbv0bOshclKVGioqJYsGABANWrV2fUqFF4enpa2CqJRCJRKczJV8uKrD9FjnPPJq+Au4WJiIugKn44GwwcsHEDAW8Oqm9psyQPIRkZGdrSt1q1atGjRw9cXV3p1auXTEUrkUjKFIX9IukAJ0ydezaWd/JC5HorSEhNpCqQggP/iho42evo2LSy5eyTPHQYjUb279/PL7/8wpQpU2jRQtVMePrppy1smUQikeRPYU4+WgjxYalZUlzu/K6+GpI4ce0klZNVta5Eg6q53cezKjbWVpayTvKQERYWxsqVK7l69SoAf//9t+bkJRKJpKxSmJMv25PZ+kT11boivxxdT8XUvgCkG2sABqpXkpnEJPfP7du3Wb16NX//rYokuri48OSTT2qiMhKJRFKWKczJ9yw1K+4FY1agf7WRRJ+JxtpYAYAU6xjAhd4eMqOY5P44deoUX3/9NRkZGdjY2ODv70///v1lrnmJRFJuKNDJCyFul6YhxSb1kvpqXYVDl7dq8nbRNurSuooOMgBKcn+4ublha2uLp6cnI0aMwNXVteiDJBKJpAxRfj1hppoMJ8O6OlbCRiu+ZaWK0VRyLL+XJrEM165dY+vWrYwfPx4bGxsqVKjA7NmzcXZ2trRpEolEck+UX0+Y1ZOPSEzDRp+Tnz5OUSPqa1SWQ6oS80hMTGTdunX88ccfCCGoU6cO/fr1A5AOXiKRlGvKp5PPuAmpYYDCLepjq68KgB2qvvxjzjbY28jIeknh6PV6du/ezYYNG0hJScHKyoqePXvSpUsXS5smkUgkD4Ty6eSTT6mvji2ISzfikKGmD83AAUjE0VZnOdsk5YLz58+zbNkyoqJU1eSWLVsycuRIateubWHLJBKJ5MFRPp28IUl9TTlDrD4Wo1VWHnslFYA2jeQQq6Rw4uPjiYqKolq1agQGBuLt7S1TIEskkoeO8unkk46pr5V7EHczjkpZiXCELhaoRJuG0slLTElNTSUsLEzLK+/n50dmZiZt27bFxsamiKMlEomkfFI+nXzqFfU1M5Z/b1/DOdUfAL1wAMBWZrqTZGE0Gjl48CBr1qwhKSmJDz/8kJo1a6IoCh07lk0VZYlEInlQlE8nb1dLfa3YGlenqsQa1EQ4sdYpgC21XWRkvQQuXbrEypUruXJFfSh0c3PDYDCU6DkzMzOJiIggLS2tRM8jkUgePuzt7alTp84DHV0sn04+8Yj66tCY8Jgr6HVJ6PR2XNS5Anpa1K5gUfMkluXOnTusWbOGP//8E4BKlSoxfPhwOnTogJVVyY7yREREULFiRRo0aCDn+CUSidkIIYiNjSUiIgI3N7cH1m75dPI26pI5MmM5/m8oHvpnAUhVjADUlD35R5rg4GAOHTqEtbU1ffr0YcCAATg4OJTKudPS0qSDl0gkxUZRFKpWrcqtW7ceaLvl08krWWY7eVPLPkorTrEy4mgn5+MfNYQQpKam4ujoCEBAQAAGg4Enn3ySatWqlbo90sFLJJJ7oSR+O8qnk8/FjetpZPXrMSrgYCPXyD9KREZGEhwcTHp6Ov/5z39QFIXq1aszbdo0S5smkUgkFqd8dnvTrqmvugpUyVCXz2UqmQB8MrqJpaySlCLJycmsWLGC999/n9OnTxMVFcXNmzctbVaZQKfT4e3tTatWrRg0aBB37tzR9p0+fZoePXrQrFkzmjRpwkcffYQQQtv/22+/0aZNG9zd3fHx8eH111/P9xzm1ispGjRoQExMjEnZjRs3GDhwIF5eXri7u9O/f3/++ecfvL298fb2pkqVKri5ueHt7U2vXr0IDw9HURRmzJihtRETE4ONjQ0vvPBCnnPm1z5AeHg4K1eufKDXUlzu3LnDwoULte2oqCiGDx9erDa++OILmjdvjoeHB15eXrz22mtkZmZqNnp4eODh4YG7uzszZswoMLg0NTWVrl27mgS5LliwAHt7e+Lj47WyJUuW5Pmcu3XrxpEjasxVUlISU6ZMoVGjRrRu3Zpu3bppks/3ihCCl156icaNG+Pp6cmxY8fyrRcSEoKnpyctW7bk7bff1sq/+OIL3N3d8fT0pGfPnly9ehWA0NBQOnToQMuWLfH09CQkJEQ7ZuTIkYSFhd2X3feFEKJc/bVu7SvEH45C/GEnRHq0mBx4UkwOPClGjjosHpu8R0TeThOShxe9Xi927dolXnzxRTFhwgQxceJEsXTpUpGQkGBp04QQQpw5c8bSJogKFSpo78eOHStmz54thBAiJSVFNGzYUGzbtk0IIURycrLw9/cXX3/9tRBCiH/++Uc0bNhQnD17VgihftYLFy7M07659QoiMzPz3i4sF/Xr1xe3bt0yKXv22WfFggULtO0TJ06Y7B83bpxYvXq1tn3lyhXh5uYmvL29tbKFCxcKLy8vMW3atDznLKj93bt3iwEDBjzQaykuV65cES1btrzn47/99lvRt29fERcXJ4QQIj09XcydO1fEx8fnsTExMVGMGjVKjB07Nt+2vv76a5PPSQgh2rZtKzp16iR++uknrWzx4sV5PueuXbuKw4cPCyGECAwMFNOnTxcGg0EIIcTly5fFr7/+es/XKIQQmzdvFv7+/sJoNIqDBw+Ktm3b5qkTExMj6tatK27evCmEUP+Hdu7cKYQQYteuXSI5OVkIod4rI0aMEEIIcf78eXHhwgUhhBCRkZGiRo0a2me5Z88eMWnSJLNtzO83BDgi7tFnltPhegMo1mRaVSXN5jT2mdVI0KlBdzJn/cOLEIJPPvmEixcvAtC8eXNGjRpF3bp1LWxZ/lyc+FiJtNv4J/MDczp06MDJkycBWLlyJR07dqRPnz4AODo68vXXX9OtWzemTZvGp59+yrvvvkvz5s0BdURg6tSpedosrN748eMZOHCg1ot0cnIiKSmJPXv2MHPmTFxcXDh37hxDhw6lbt262rTKrFmzcHJy4o033uCzzz5j1apVpKenExAQwAcffGDWtUZHR2vXBmiJjwrD0dGRFi1acOTIEdq0aUNISAgjRozQ0h2b0/706dM5e/Ys3t7ejBs3jqlTpzJ16lSOHDmCtbU1X3zxBd27d8dgMPD222+zdetWrKysmDx5Mi+++CIA//vf/9i0aROZmZmsXr2a5s2bc+jQIV5++WXS0tJwcHBg8eLFNGvWjNOnTzNhwgQyMjIwGo2sXbuWmTNncunSJby9venduzfTpk1j4MCBnDp1qtDzZjNnzhz27t1L5cqqwJetrS3Tp0/P9zNzcnJi0aJF1K1bl9u3b1OlShWT/StWrDAZ2bh06RJJSUksXLiQOXPmMGHChCK/l0uXLvH333+zYsUKbUWMm5vbfUedb9iwgbFjx6IoCu3bt+fOnTtER0dTs2ZNrc7ly5dp0qQJjz2m/v/26tWLtWvX0rNnT7p3767Va9++PcuXLwegadOmWnmtWrWoVq0at27donLlynTu3Jnx48ej1+uxti59l1v+nLzIzHrVozfqsTY4ARBmmwFAFSeZvexhRVEUvLy8iIuLIzAwkNatW8sgt0IwGAz8/vvvPPPMM4A6VN+6dWuTOo0aNSIpKYmEhAROnTpl1rC7ufXu5tixY5w6dQo3NzeOHz/OK6+8ojn5VatWsW3bNrZv305YWBiHDh1CCMHgwYPZu3evWaJB06ZNIzAwkK+//ppevXoxYcIEatWqVeRxI0eOJDg4mOrVq6PT6ahVq1a+Tr6g9j/++GPmzZvHr7/+CsDnn3+Ooij8888/nDt3jj59+nDhwgUWL15MeHg4oaGhWFtbc/v2ba1tV1dXjh07xsKFC5k3bx4//PADzZs3Z9++fVhbW7Nz507eeecd1q5dy6JFi3j55Zd56qmnyMjIwGAw8PHHH3Pq1ClCQ0MBdQohm++//77A8wIkJCSQlJRULAfq7OyMm5sbYWFhtGvXTivPyMjg8uXLNGjQQCsLDg5m5MiRdO7cmfPnz3Pjxg2qV69eaPunT5/G29sbna7oGKvAwEDOnz+fp/y1115j7NixJmWRkZEmnYI6deoQGRlp4uQbN27M+fPnCQ8Pp06dOqxfv56MjIw87f/444+aWmVuDh06REZGBo0aNQLAysqKxo0bc+LEiTz/f6VB+XPyxqwP26oCd1LuIBQ9ABmKwE4nCjlQUt5IT09ny5YtVKlSha5duwLQp08fevfuja2trYWtK5ri9LgfJKmpqXh7exMZGUmLFi3o3bu3Rey4m7Zt22qOxMfHh5s3bxIVFcWtW7dwcXGhbt26fPnll2zfvh0fHx9AnZcNCwszy8n37duXy5cvs3XrVn777Td8fHw4deqU1iMrCH9/f2bOnEn16tUJDAwsdvt3s3//fq2n3Lx5c+rXr8+FCxfYuXMnzz33nNaby90DHjp0KACtW7fml19+AVR9hXHjxhEWFoaiKNr8eIcOHZgzZw4REREMHTqUJk0Kj0Mq7Lz5sW3bNt5++23u3LnDypUrefzxx/OtJ0Te39uYmBhtNCCboKAg1q1bh5WVFcOGDWP16tW88MILBT6gF/fBPff894PAxcWFb7/9lsDAQKysrHj88ce5dOmSSZ3ly5dz5MgR/vjjD5Py6OhoxowZw88//2ySk6NatWpERUVZxMmXw7HtrBurYhsu3MgJZtAjcHW0kEmSB4oQgr/++ot33nmHTZs2sWbNGtLTVREiGxubcuHgLYmDgwOhoaFcvXoVIQTffPMNAO7u7hw9etSk7uXLl3FycsLZ2ZmWLVvm2Z8fhdWztrbGaFSnzoxGo0kPqEIF0yRVTz75JGvWrCEkJERzrkII/vOf/xAaGkpoaCgXL17URiLMoUqVKowePZply5bh5+fH3r17izzG1taW1q1b8/nnnxcZrHYv7ZuDnZ2a20On06HXqx2XmTNn0r17d06dOsWmTZu0QLfRo0ezceNGHBwc6N+/P7t27bqvczs7O+Pk5KRlhuzbty+hoaG0atUq3x4sQGJiIuHh4SbD1KDee7kD8v755x/CwsLo3bs3DRo0IDg4mKCgIACqVq1KXFycyfG3b9/G1dWVli1bcuLECbMyVGYLTN39t3Tp0jx1a9euzbVr17TtiIiIfJUnBw0axN9//83Bgwdp1qyZyXXu3LmTOXPmsHHjRu17A3VEZMCAAcyZM4f27dubtJc95WIJyqGTz8HayhobQ44Yja1ODt2Wd8LDw5k7dy7ff/89cXFx1K9fn5deesnkn0liHo6Ojnz11Vd8/vnn6PV6nnrqKfbv38/OnTsBtcf/0ksv8dZbbwHw5ptv8t///pcLFy4AqpNetGhRnnYLq9egQQPtAWDjxo1a7zM/AgMDCQ4OZs2aNTz55JOA6mB++uknkpJUpcnIyEizV03s2rWLlJQUQHVCly5dol69emYd+/rrr/PJJ58U2sstqP2KFSuSmJio1evcuTMrVqwA4MKFC/z77780a9aM3r17891332lO/O5h87uJj4/XHNCSJUu08suXL9OwYUNeeuklhgwZwsmTJ/PYkBtzzvuf//yHqVOOtoiGAAAgAElEQVSnaisxhBAFRs8nJSXx/PPP88QTT+Di4mKyz8XFBYPBoB0bFBTErFmzCA8PJzw8nKioKKKiorh69Sp+fn4cOHCA69evA3DkyBHS09OpW7cujRo1ok2bNrz//vvaiEF4eDibN2/OY09ISIj2UJj77+6heoDBgwezdOlSrSNRqVIlk6H6bLLvubi4OBYuXMikSZMAOH78OFOmTGHjxo0mOTgyMjIICAhg7Nix+T4oXrhwgVatWuX7eZY05W+4PhfGXA95QoHqVWU62/JKWloaQUFB7N+/HyEEzs7ODBs2jI4dO5Z4KtqHGR8fHzw9PQkKCmLMmDFs2LCBF198kWnTpmEwGBgzZoy2jMnT05MFCxYwatQoUlJSUBSFgQMH5mmzsHqTJ09myJAheHl54e/vn6f3npuWLVuSmJhI7dq1tR/aPn36cPbsWTp06ACoQV7Lly/PN6mRp6endm+MGDGCmjVr8sILL2ijCZMmTcLPz8+sz6lly5a0bNmy0DpHjx7Nt/3MzEx0Oh1eXl6MHz+e559/nqlTp+Lh4YG1tTVLlizBzs6OSZMmceHCBTw9PbGxsWHy5Mn5LtXL5q233mLcuHHMnj2bAQMGaOWrVq1i2bJl2NjYUKNGDd555x2qVKlCx44dadWqFf369TPJE2HOeadOnUpycjLt2rXDzs4OJycnOnbsqE2bAHTv3h0hBEajkYCAAGbOnJmv3X369GH//v306tWL4OBgtmzZYrI/ICCA4OBg3n77bb788kv69++P0WjEycmJoKAg7Tv94YcfeP3112ncuDEODg64urry2WefFfodFUX//v3ZsmULjRs3xtHRkcWLF2v7vL29tZiGl19+mRMnTgDw3nvvaT35N998k6SkJO2htF69emzcuJFVq1axd+9eYmNjtQeyJUuW4O3tzY0bN3BwcKBGjRr3Zfu9ouQ3r1KWaePTTBz58ipU6sof1u+z4qOKAKx3vo1rRWvOfC6VxcojRqORDz74gKioKHr37s3AgQO1DHblibNnz9KiRQtLmyGRWIxjx44xf/58li1bZmlTygTz58/H2dnZ7Gmn/H5DFEU5KoRocy/nL389eYM6XIaVXfbsPEZFnTfq61l4QImkbHHixAnq1q1LlSpVsLKyYuLEidjZ2VnsiVcikdw/vr6+2pJBc6LjH3YqV67MmDFjLHb+8ufkUYN6sK7ErWuqm1eEehnudSpayihJMYiOjiY4OJh//vmH9u3b8+yzqsBQ/fr1LWyZRCJ5EEycONHSJpQZzMkLUJKUQyefhUMTLl+JB5xRsuIHa1SWwVllmZSUFDZu3Mjvv/+OwWDAwcGBBg0aIISQ690lEomkBCi/Th5IvVkJgAwrNRK3U/PKhVWXWAij0ci+ffv45ZdfSExMRFEUunTpwtChQ3F2di66AYlEIpHcE+Xayd+JVYfu79hdp5KdCy4VZLa7skh0dLS2bKVp06aMGjVKDs1LJBJJKVAO1yZlhdspVsTowwFI18VT28loOZMkeUhISNDe165dm0GDBvHcc8/x9ttvSwdfwkgVupJRoVuyZAmKomh5BgDWr1+PoiisWbMGUJernTlzplDbFy1alG+illmzZlG7dm28vb1p3rw5U6dO1RILFcT69euLPN+ePXvyXQqZzfDhw7l8+bK2HRoaiqIobN26VSsLDw/Ps8571qxZzJs3T9ueN28ezZs3x9vbGz8/v3yvsbj8/PPPNGnShCZNmvDzzz/nW+fEiRN06NABDw8PBg0apP32HDp0SPvuvby8WLdunXbMxIkTqVatWp5reuONN+47uVCZ416VbSz117rVY6oCXeS3YtzYHWJy4EnR4dkfRPfpO81W+ZGUHGlpaWLDhg1iypQpeVTAHgWkCl3RlFcVusWLFwsPDw/xzDPPaGUjRowQXl5eJu3eK++//7747LPPhBBCGAwG0bFjR7Fr165Cj7n7mvKjMJW8U6dOiSeeeMKk7K233hKdOnUyUZnLT+Uut73ffvut6NOnj6ZaFx8fL5YsWVKoXUURGxsr3NzcRGxsrLh9+7Zwc3MTt2/fzlOvTZs2Ys+ePUIIIX788UcxY8YMIYR6f2ffa1FRUeKxxx7Ttv/44w9x9OjRPNcUHh4uevfufV923y8PWoWu/PXks3odmUYrbNOzRA70j1HFxcmCRkmEEBw6dIgZM2Zogg7nzp2ztFmPPB06dCAyMhIoWIXu448/BgpXl8tNUSp02b1aUJPZgNqb7Ny5M4MHD8bd3Z3p06dr6XbBtFf42Wef4efnh6enJ++//77Z1xodHU2dOnW07eKq0AGaCl1BdO7cmUOHDpGZmUlSUhIXL17E29tb259bD93JyYl3330XLy8v2rdvz40bN/Jca0FkZGSQlpamZZT7v//7P/z8/PDy8mLYsGGkpKTw559/snHjRt588028vb25dOkSFy9epFevXnh5eeHr66vlXE9KSmL48OE0b96cp556Shu9WbFiBUOGDNHOK4Rg9erVLFmyhB07dhSY9e5u/vvf//Ltt99qMTbOzs6MGzfOrGMLYtu2bfTu3ZsqVarg4uJC7969TUYXsrlw4YKmbdC7d2/Wrl0LqN9tdr7+tLQ0k+DeLl265JvdsH79+sTGxmpZ+B4Gyt+cvFDT3EUl5AzVRVo50KyidPKW4t9//yUoKEhTgqpbty6jR4+mWbNmFrbMslR79o+iK90DN7/valY9qUL3YFXoQBVP6dWrF9u2bSM+Pp7BgwdrOd/vJjk5mfbt2zNnzhzeeust/u///s9kaiA/5s+fz/Lly7l69Sr9+vXTHiCGDh3K5MmTAZgxYwY//vgjL774IoMHDzaR9m3Xrh3Tp08nICCAtLQ0jEYj165d4/jx45w+fZpatWrRsWNHDhw4QKdOnThw4ACjRo3Szv/nn3/i5uZGo0aN6NatG5s3b2bYsGGF2pyQkEBiYiINGzYstB6oD3DZKX9z06VLF7766iuTsoIU4+6mZcuWbNiwgSeeeILVq1eb5Kb/+++/mThxIlevXmXZsmVmSb36+vpy4MCBIq+7vFD+evKoTj7TKidncipSsMRSHD16lA8++IDz58/j5OTE2LFjef/99x95B29JslXoatSowY0bN8q8Ct2JEyc0Fbrt27drKnS+vr6cO3eOsLCwIlpWyVaJmzx5MufOncPHx4dbt4pWAvT392fHjh0EBwcXqkKXTfZDQXBwsImDvBtbW1ttLrx169Ym8q8F8eqrrxIaGsrNmzdJTk4mODgYUB+sOnfujIeHBytWrOD06dN5jk1MTCQyMpKAgAAA7O3ttayRbdu2pU6dOlhZWeHt7a3ZEh0dbaLSFxQUxMiRI7XrzBaTeVCKcW+++Wa+eebvdvDF4aeffmLhwoW0bt2axMREEwGrdu3acfr0aQ4fPszcuXPNGpnIVox7WCh/Pfkswq7qLW2CBFXZzMXFBV9fX4YMGVJorvJHDXN73A+abBW6lJQU+vbtyzfffMNLL72Eu7t7HtW0/FTovLy8Cm2/sHr3okJ3/fr1PCp0U6ZMuadrz1aJGz16NAMHDmTv3r1F9shyq9CdOXOGjRs3Flq/bdu2/PPPPzg6OuZRYcuNjY2N5gRzq8uZg42NDf7+/uzdu5eRI0cyfvx41q9fj5eXF0uWLGHPnj1mtwWYCDzltiW3apzBYGDt2rVs2LCBOXPmIIQgNjaWxMTEAhXj3NzcNBW7bOGcwihOT7527dom1xkREUG3bt3yHNu8eXO2b98OqEP3+YnYtGjRAicnJ06dOkWbNoVnh7WkYlxJUA578upc0p8ncj1pKVDX1d5C9jxanDp1innz5mk/DA4ODsyZM4fRo0dLB1/GkCp0D1aFLjcff/wx//3vf82qey8IIThw4ACNGjUC1OupWbMmmZmZJk4yt/pcxYoVqVOnDuvXrwcgPT1d+zwKokWLFly8eBGA33//HU9PT65du0Z4eDhXr15l2LBhrFu3DicnJ2rWrKlFnt++fZutW7fSqVMnQFWxmzZtmhbZnpSUlG90fXF68n379mX79u3ExcURFxfH9u3b6du3b5562feH0Whk9uzZPPfccwBcuXJFe5i5evUq586do0GDBoV+HmBZxbiSoPw5eaP6o2Fno87V6K2SAWjjJpOqlCQ3btzgq6++4osvvuDMmTPs3r1b2ydlYMsuuVXoHBwc2LBhA7Nnz6ZZs2Z4eHjg5+eXrwpdixYtaNWqlcnSqmwKqzd58mT++OMPvLy8OHjw4D2p0I0ePVpbEjV8+PACJVQ9PT2pU6cOderU4bXXXuPo0aO0adMGT09POnToUGwVuuIEivXr14/u3bubXd9c5s+fry1/NBgMPP/88wB89NFHtGvXjo4dO2oBj6AOqX/22Wf4+Phw6dIlli1bxldffYWnpyePP/54kQFkAwYM0HrLQUFB2lB/NsOGDdOG7JcuXcpHH32Et7c3PXr04P3339ceQqZOnUr37t3x8/OjVatWdO7c+b7VI6tUqcLMmTPx8/PDz8+P9957T3sImzRpkhbgGBQURNOmTWnevDm1atXS0sju378fLy8vvL29CQgIYOHChbi6ugIwatQoOnTowPnz56lTpw4//vgjAJmZmVy8eLHI3n55okRV6BRF8Qe+BHTAD0KIj+/a/xowCdADt4CJQoirhbXZpoWDOPKd4NVt/yP5UnvibK/zh70tu2a2plVdGXz3oElNTeXXX39l+/btGAwG7O3tGTRoEL169cLGRiYfuhupQicpT6SmptK9e3cOHDggxWSAdevWcezYMT766COL2VBuVOgURdEB3wC9gQjgsKIoG4UQuTM3HAfaCCFSFEWZCnwKFB35AlxPuUJF2pNmlYEirGleSw4VP2iOHTvGsmXLiI+PB6BTp04MGzaMSpUqWdgyiUTyIHBwcOCDDz4gMjLS7KmNhxm9Xl/qiZ1KmpIMvGsLXBRCXAZQFCUYGAJoTl4IsTtX/b+Ap81tvJJDZYxAigIVlVSsdVLg5EFjZWVFfHw8jRo1YvTo0VpktEQieXjIb577USU7NuRhoiSdfG3gWq7tCKBdIfWfAX4rslWhB3RkCh06QAgHqtoXHlwiMY+4uDjOnj3L448/DoCXlxevvfYa7u7u9z2/JpFIJJLSp0wsoVMU5WmgDZDvmiNFUZ4FngVo3dQK1clbkT2DVEVJyO8wiZlkZmaybds2Nm/eTEZGBvXq1aNOnTooivJQRZlKJBLJo0ZJOvlIoG6u7TpZZSYoitILeBfoKoRIz68hIcT3wPcAbZrrBIBtdE4Mgr2TDLi7F4QQHDt2jJCQEE3sw9fXF3t7uRxRIpFIHgZK0skfBpooiuKG6txHAqNzV1AUxQf4DvAXQpi3GDZrnbzR7g5W6ZVxMNqA0fAAzX40iIiIICgoiLNnzwJq4olRo0bh7u5uYcskEolE8qAosYlWIYQeeAHYBpwFVgkhTiuK8qGiKIOzqn0GOAGrFUUJVRSl8FRT2dhUJ9GgRnxH2caj2MnI+uKyefNmzp49S4UKFXjqqaeYNWuWdPAPCVJqtmSkZvNrH1QZ1pUrV97zteQWtSkudwsCFVVuSRYsWGCSIEev1/PYY48xffp0k3p3f7d3S+WWxL139OhRPDw8aNy4MS+99BL5LS2Pi4sjICAAT09P2rZty6lTp0z2GwwGfHx8TGx95pln8PLywtPTk+HDh2tJnr7++mt++umn+7bbLO5Vvs5Sf62bKUIc7SBGPLNCTA48KTwnbhXTPv+9SPm+Rx29Xm8i0xgTEyOWL18uEhMTLWjVw4eUmi2a8io1W1D7hUm5mkPXrl3F4cOH7+nYgqRmzZGgfRAYjUZhMBiKrJeZmSk8PDxMvvstW7aIxx9/XDRs2FAYjUat/O7vNvfne7/3XkH4+fmJgwcPCqPRKPz9/cWWLVvy1HnjjTfErFmzhBBCnD17VvTo0cNk/+effy5GjRplci9kS+8KIcSrr74q5s6dK4RQ//dy33e5kVKzAFa2ZC+YE8IOH28phlIYZ8+eZdasWXz11VdaXvGqVavy1FNPaVKgkocTKTVbOMWRmi2o/enTp7Nv3z68vb2ZP38+4eHhdO7cGV9fX3x9ffnzzz+1Yz755BM8PDzw8vLK04M1Go2MHz+eGTNmYDAYePPNN7XP4bvvvgPUTtkLL7xAs2bN6NWrl9kpf0FNNduzZ098fX3x8PBgw4YNALz33nssWLBAq/fuu+/y5ZdfAvl/F+Hh4TRr1oyxY8fSqlUrrl27xvjx42nVqhUeHh7Mnz8/z7l37dqFr6+viQpcUFAQL7/8MvXq1ePgwYNmXYO592hxiI6OJiEhgfbt26MoCmPHjtVSA+fmzJkz9OjRA1Dz5YeHh2vywREREWzevJlJkyaZHJMtvSuEIDU1VdMycHR0pEGDBhw6dOi+bDeHMhFdX1yEzgVdRk5CFicHmakpP27dukVISAjHjh0DwNXVldu3b2upHSUli9d7rYuudA+c+PCoWfWk1OyDlZotqP2PP/6YefPm8euvvwKQkpLCjh07sLe3JywsjFGjRnHkyBF+++03NmzYwN9//42joyO3b9/W2s7WFmjVqhXvvvsu33//PZUqVeLw4cOkp6drD2fHjx/n/PnznDlzhhs3buDu7s7EiRPN+vzt7e1Zt24dzs7OxMTE0L59ewYPHszEiRMZOnQor7zyCkajkeDgYA4dOlTgd1GvXj3CwsL4+eefad++PUePHiUyMlIbvs49PZTNgQMHTO69tLQ0du7cyXfffcedO3cICgrSlu4Whrn33u7du3n11VfzlDs6Opo8dIGqj5D74a0gSVsvLy9++eUXOnfuzKFDh7h69SoRERFUr16dV155hU8//TTfFMwTJkxgy5YtuLu78/nnn2vlbdq0Yd++fbRt27bI67kfyqWT16PDTq86KkXYUreKjAbPTVpaGps3b2bbtm3o9Xrs7OwYMGAAffv2laloHwGypWYjIyNp0aJFmZeavXXrliY1++WXX2pSs6D2PsPCwsxy8tlSs1u3buW3337Dx8eHU6dOmUip5oe/vz8zZ86kevXqhUrNFtT+3WRmZvLCCy8QGhqKTqfThHx27tzJhAkTNPnX3GI4U6ZMYcSIEbz77rsAbN++nZMnT2qjIvHx8YSFhbF3715GjRqlPYxk9yzNQQjBO++8w969e7GysiIyMpIbN27QoEEDqlatyvHjx7lx4wY+Pj5UrVrVRPYXcr6LevXqUb9+fdq3bw9Aw4YNuXz5Mi+++CIDBgzQRopyEx0dbZKq9ddff6V79+44ODgwbNgwPvroIxYsWIBOp8tXvra4krbdu3cnNDS0WMcUxfTp03n55Zfx9vbGw8MDHx8fdDodv/76K9WqVaN169b5qgMuXrwYg8HAiy++SEhIiJZbv1q1apw7d+6B2pgf5dLJA+iMqmNPsTJS0bHcXsYDx2g0MmfOHO1JtEOHDgwfPhwXFxcLW/boYW6P+0EjpWZLTmo2v/arVq1qUmf+/PlUr16dEydOYDQazVqS+vjjj7N7925ef/117O3tEULwv//9L082ui1bthTZVkGsWLGCW7ducfToUWxsbGjQoIGmJjlp0iSWLFnC9evXtZGBgr6L8PBwk+/SxcWFEydOsG3bNhYtWsSqVavyBJXllrQFdah+//79mipcbGwsu3btonfv3pqsbfaIY+7RR3Pv0eL05GvXrk1ERIS2HRERQe3atfMc6+zszOLFi7XPxs3NjYYNGxISEsLGjRvZsmULaWlpJCQk8PTTT7N8+XLtWJ1Ox8iRI/n00081J19akrblc04eULJMNyCoVVmqoGVjZWVFly5dcHNz45133mHy5MnSwT+iSKnZBys1W1D7ueVeQe1116xZEysrK5YtW4bBoC7x7d27N4sXL9bayD1c/8wzz9C/f39GjBiBXq+nb9++fPvtt9rnd+HCBZKTk+nSpQshISEYDAaio6NN1CCLIj4+nmrVqmFjY8Pu3bu5ejVHCywgIICtW7dy+PBh7cHC3O8iJiYGo9HIsGHDmD17tjY9mJvckrYJCQns27ePf//9l/DwcMLDw/nmm280tbtu3bqxbNkyQJ1yWr58uab4Z+49mt2Tv/vvbgcPULNmTZydnfnrr78QQrB06VKGDBmSp96dO3e0h9YffviBLl264OzszNy5c4mIiCA8PJzg4GB69OjB8uXLEUJo1yyEYOPGjSYKgqUlaVsuu8DJGWkYrFLRGR1IszLiUqFcXsYDIT4+nrVr11KrVi38/f0B6NmzJz179pSpaCUmUrNjxoxhw4YNvPjii0ybNg2DwcCYMWPylZpNSUlBURST5UDZFFZv8uTJDBkyBC8vL/z9/e9Javbs2bN06NABUAP3li9fTrVq1fK1I/seHzFiBDVr1uSFF17QRhOKKzXbsmXLQuscPXo03/YzMzPR6XR4eXkxfvx4nn/+eYYNG8bSpUtNPgN/f39CQ0Np06YNtra29O/f30ST/rXXXiM+Pp4xY8awYsUKwsPD8fX1RQjBY489xvr16wkICGDXrl24u7tTr1497XPKjylTpvDKK68AULduXTZt2sSgQYPw8PCgTZs2Jg7H1taW7t27U7lyZU2NrqDv4m61usjISCZMmKCN4MydOzePLf369WPMmDGAqvTWo0cPE4nqIUOG8NZbb5Gens7MmTOZOnUqXl5eCCHw9/fn6adVWRNz79HisnDhQsaPH09qair9+vWjX79+ANoDxHPPPcfZs2cZN24ciqLQsmVLTZ62IIQQjBs3joSEBIQQeHl58e2332r7Dxw4wKxZs+7b9qIoUanZkqBNcyuxblkgH33+DgChNW5yaEFPC1tV+mRmZrJz5042bdpEWloaTk5OzJs3D1tbW0ub9kgjpWYl5RGj0Yivry+rV6+mSZMmJXKOgIAAPv300xJrvzxx/PhxvvjiC23EIjflRmq2JEky1NDeV3V6tHqrQghOnDhBcHCwNnTm7e1NYGCgdPASiaTYnDlzhoEDBxIQEFCiDvjjjz8mOjpaOnnUKY7S0qwvn04+JWdtt6210YKWlC4pKSksWrRIi+itWbMmo0aNkiIyEonknnF3d+fy5cslfp5mzZrRrJnMaQKU6oqXcunk4xLUQBa9koZrpZKPTiwr2Nvbk5KSgoODA0OGDKFHjx4mySUkEolEIslNufQQu/eqT4NGq0ysH+LgMqPRyN69e3F3d6datWpYWVkxadIkHB0dtUxKEolEIpEURLl08hkp6py8raEirk6pFramZDh//jxBQUH8+++/+Pr6ahHQNWrUKOJIiUQikUhUyqeTT1eXpFyocBGvSvUtbM2DJTY2llWrVnH48GFAzTFf0mkPJRKJRPJwUi7HuvV61exUXTo9W9cponb5ID09nQ0bNvDuu+9y+PBhbG1teeKJJ5g9e7Z08pJi8ahIzXp4eGgpRrPFVoACc6DnFs+ZNGkSZ86cuW879uzZY5JgZdGiRSZyqg+C8PDwfINrCyr/66+/aNeuHd7e3rRo0YJZs2axePFiTXLX1tZW++ymT5/OkiVLUBRFS5IEsH79ehRFKVCu9pVXXjHJnpgt0Xt3Ypq7BbCWLFliIuO7dOlSTdjGx8dHEym6H7Zu3UqzZs1o3LixJr50N1evXqVnz554enrSrVs3k4x3//77L3369KFFixa4u7sTHh4OwJUrV2jXrh2NGzcmMDBQS4xTqrKx98K9ytdZ6q91U0VMDjwpJgeeFN5TvhGGlIR85frKG1FRUWLSpEliwoQJ4ttvvxUxMTGWNklyD0ip2aJ50FKz586dE/Xq1SvyGHPkV/V6fbHseP/998Vnn31WrGOKy5UrV0TLli3NLm/atKkIDQ0VQqjXc/r0aZP9d0u5Ll68WHh4eIhnnnlGKxsxYoTw8vLK9/OKiYkR7dq1MylbuHCh6NSpk+jSpYtJee57Mftc2TK+W7ZsET4+PiIyMlIIIURaWpr4/vvv834AxUCv14uGDRuKS5cuifT0dOHp6Znn+oUQYvjw4WLJkiVCCCF+//138fTTT2v7unbtKrZv3y6EECIxMVEkJycLIYR48sknRVBQkBBCiClTpmj3fGGysffCIy81qzfmCKw4KHeA4gkXlCWio6O1XlTNmjUZMWIE06dP57nnnsuTD1siuRceBanZhIQEk9TN2ecUhciyduvWTZOXdXJy4vXXX8fLy4uDBw+yfPly2rZti7e3N1OmTNHS0m7duhVfX1+8vLzo2bMn4eHhLFq0iPnz5+Pt7c2+fftMriM0NJT27dvj6elJQEAAcXFx2rnffvtt2rZtS9OmTdm3bx9AoRK1xeHmzZtaBkGdToe7u3uRx2Qrq2VmZpKUlMTFixfx9vbOt+7atWu17JrZBAUF8fnnnxMZGWnSKy6MuXPnMm/ePE0p0M7OjsmTJ5t1bEEcOnSIxo0b07BhQ2xtbRk5cqTJKE82uWVju3fvrtU5c+YMer1eW+Lm5OSEo6MjQgh27drF8OHDARg3bpwmR1uasrH3Qrlz8pmGnIQvblzDyqH86aEnJCSwZMkSZsyYoeX5BnXtZNOmTS1omeSBste+ZP7MJFtqdvDgwYB5UrN3788Pc+vdzbFjx/jyyy+5cOECgYGBrFq1Stu3atUqAgMDTeRNQ0NDOXr0aB5RnWy6d+9Oq1at6Nq1K7Nnz86zf926dZos69KlSwt0msnJybRr144TJ05QtWpVQkJCOHDggKYily3sMnnyZNauXcuJEydYvXo1DRo04LnnnuPVV18lNDSUzp07m7Q7duxYPvnkE06ePImHhwcffPCBtk+v13Po0CEWLFiglVerVo0dO3Zw7NgxQkJCeOmll4r9GQO8+uqrNGvWjICAAL777jsTYZiCUBSFXr16sW3bNjZs2KDdM/lxt2zstWvXiI6Opm3btowYMYKQkBCz7DT3PlqxYoU21ZD7L9vh5vhjFYsAABq6SURBVCYyMpK6detq20XJxoJ6nyQmJhIbG8uFCxeoXLkyQ4cOxcfHhzfffBODwUBsbCyVK1fWlizf3W62bGxZpNwF3imoPd80m5vYCUcLW1M89Ho9u3btYsOGDaSmpqLT6cwW35BIzOVRkZrdvXs3rq6uXLp0iZ49e9KtWzeTOWBzZVl1Op2mVPf7779z9OhRLed9amoq1apV46+//tKEn4BChWxA1ZS4c+cOXbt2BdSeX7YID8DQoUMBaN26tTbnW5BEbXF57733eOqpp9i+fTsrV64kKCgoXwnUuxk5ciRfffUV8fHx/H97dx4dVZUtfvy7mafHqKIQCBgCCARiMCq/hxBEJgM0yCytIoi/pkH7Z0NsfApIA4Id1JbV2jbSSLDzIwKKoD7FiERcDIoyCKIgEEUm4SUCCXPCfn/cqtuZUwmZqtiftWpRwx1OnQq165577t4vvPBCtrz6WR07dixb+d633nqL4cOHu9sYO3ZsgXM0ilo2dvTo0YwePbpI6xRm/vz5TJo0iSVLltCtWzeaNm1K5cqVycjI4PPPP2f79u00b96cESNGsGTJkjwL1mRVVmVji8PvgrzXxSr/Q60M/2n+rl27SEhI4NixYwB06NCBUaNGucNqJgB1K/wIqjRca6VmQ0JCaNy4MXv27CnWJNUaNWq4RVfUU1QkZ5GV9957r8jbLYi3OIs3sEDxStTmJyQkhAkTJjB+/Hiuv/56UlJSCj0FePvtt7Nr1y5q1apV4IhiXmVjjx8/Tnx8PABHjx7lhx9+IDQ0lJo1a3Lp0iU35XZeZWPz+/HlFR8fT2xsbK7nW7VqlWtiYNOmTfn555/dx/mVjW3SpIl7JJ+ens7bb79N/fr1CQoKIjw8nJtvvhmAQYMGsWXLFsaOHcupU6fIyMigSpUqubZbVmVji8Pvhuv/Tbihln+cj9+8eTMvvfQSx44do3HjxvzhD3/giSeesABvStW1Umr2xIkTJCcnExyc/XLa4pRl7dmzJytXrnT3mZqayk8//cSdd97Jhg0bSE5Odp8HcpWZ9apXrx4NGjRwh3DffPNN96g+P/mVqC2qDz74wJ3r88MPP1C5cmXq16/v07rz5s3L9wjeK2vZ2H379pGens6RI0fcsrFPPfWUWza2e/fubl318+fPs3z5crds7FNPPUVMTAzHjx8H4NKlSyxatCjX/kaPHp1n2di8Zv5HRkbyww8/kJyczKVLl0hISMjz1IO3PC44cwPGjh3rrn/q1ClOnjwJ4Fb8ExF69Ojh7jMuLi7b0X1ZlY0tDj8O8tDt1op7jbz3PxlAREQEQUFBDB8+nFmzZtGpU6ciD1kZUxxZS83WrFmT1atXM3v2bNq0aUNYWBiRkZF5lpq95ZZb6NChQ545zQtabvz48Xz22WfuJLbilJq9//776dKlC2FhYQwdOjTPIArOOfnw8HB69OjBvHnzaNy4cbbXvQVX2rVrx4MPPlhgWVavdu3aMXv2bHr37k3Hjh3p1auXOzy9cOFC7rvvPjp16uSOPAwYMIBVq1a5E++yiouLIyYmho4dO7Jjxw6mT59e4L5///vfExcXR6dOnfj+++8L7DuvvXv3EhQU5N5WrFjBm2++SZs2bQgPD3fL1uYsD5uffv36uUE4P9HR0e7w/7Jlyxg8eHC214cMGeIG+Zdffpl33nmH8PBw7rzzToYNG+aeern33nuZNGkS99xzD+3btyciIoIzZ8741M78VKlShb/97W/06dOHW265heHDh7slhKdPn86aNWsAZyJomzZtaN26Nb/88gtPP/004IyszJ8/n549exIWFoaqupMBn3/+eV588UVatWpFSkoK48aNc/e7cePGCnNaLCe/KzXb4eY6+n9u38zpmt/x2G8b07Vnwb+Oy9qVK1fYuHEjiYmJTJ06lVq1arnPW333wGelZs21oGvXrrz//vs+jxAEsoLKxhaHlZp1KZWqFv+cVWnYv38/8fHx/PTTT0D2X3cW4I0xgeKFF17g0KFDFuQp27KxxeGHQf7fw9xSvWJMdEhNTWXlypVs2bIFgAYNGjBs2DDuuOOOcm6ZMcaUPPtu+7eKOkzv5XdB/nKGM0uzklbj5lYtyrcxwOeff058fDyXLl2iSpUq9O3bl+joaHf2rDHGGFNe/C7IizgzIiWzHo0blX+51YYNG3Lp0iU6d+7M8OHDs10/aowxxpQnvwvyXuerpJbLfn/++We+++47NzVo+/btmTlzZrYsS8YYY0xF4IdB3jknX00yynSv6enprFq1iqSkJFSV0NBQN/uVBXhjjDEVkd8F+YxMp0DNFcrm0r/MzEySkpJ49913OXv2LJUqVeLuu++2YXljjDEVnt9d15V5xfldUq3GpUKWvHp79uzh2WefJT4+nrNnz7q1mUePHp2rTrIxFcW1UE8+q8uXLzN16lRCQ0OJiIigS5cufPjhh4CTgc+blx5g5cqVjBkzBnBqm1eqVIlvvvnGfb1Dhw5uLvniSkpKon///rmef//997n11lvp1KkT7dq14x//+Adz5sxxC654P7fw8HAWLFjAs88+i4i42eUA/vrXvyIibgW9nIYOHZotgdGOHTsQET766CP3ubzq0GetngdObve2bdsSHh5OZGQkS5cuLXZ/eMXFxREaGkpoaChxcXF5LrNz5043EdKAAQPc5Dg//vgjNWvWdPvnd7/7Xa51Bw4cmO19TZkyhU8//fSq2+3v/C7Ie9WpmlLq+9i8eTNHjhzhuuuuY9KkSUyZMoWgoKBS368xV8Obu3737t00bNjQLel6/vx5Bg4cyNSpU9m7dy87d+5k06ZNvPrqq4BTFWzSpEn861//Ys+ePXz11Ve0atUq1/Z9XS4/3lztJWXatGkcO3aM3bt3s23bNt59991sWfK+/vpr9uzZk+e6QUFBzJkzp0j7a9GiRZHbePnyZR599FHee+89du7cyfbt24mKiuLpp59207R6P7cdO3a4FejCwsJISEhwt7NixQo3g1tO3377LZmZmW7edXAy0nXt2tXNQOeL1157jcTERLcS4Lp167japGmpqanMnDmTL774gi+//JKZM2e6pXezeuSRR5g3bx67du1i8ODB2XLWh4SEuP2TM93yO++8k+vA67HHHnPLKF/L/G643qtVraqFL1REFy5c4Ndff3VTbA4ZMoQbb7yR3r17U7Vqye/PBLZHR+4qle0uTAjzedkuXbq4R6r51ZOPiopi4sSJJVZPvn///m4Z0Dp16pCenk5SUhLTpk2jQYMGfP/999x33300a9aMiRMnAs6RZJ06dZgyZQqxsbEsX76cixcvMnjw4GwlWnM6d+4cr7/+OsnJye5lq40bN3arogFMnjyZOXPmuAVUsurfvz8bNmxg7969tGnTxud+Laq0tDQyMjLcIjHVq1f3aX+DBg1i9erVPPPMMxw4cIB69erl+10UHx+fLZ+6qrJixQoSExO56667uHDhgk9Fb5577jmSkpKoW9e5eqlu3bo89NBDvrzNfK1du5ZevXq51ft69erFRx99xKhRo7Itt2/fPjftba9evejTp0+hiWbS09N58cUXWbhwYbbPPTg4mJSUFI4fP86NN954Ve33Z353JJ9+wcmwdFMJJru7cuUKmzZt4qmnnuLVV191C0PUr1+f6OhoC/DGLwV6PXlwskw2b97cDUh5GT58ONu2bcs27O1VqVIlnnzyyUKLslythg0bMnDgQIKDgxk1ahTx8fFugZSC1K1bl2bNmrF7924SEhLcnPl5yVnnfdOmTbRs2ZKQkBCioqL44IMPCt3fmTNnSEtLyzYakJ/Y2Ng867x7RyGy8rXOe/v27Vm9ejXgjFpkrSiXnJzMrbfeSvfu3bPVCZg2bRqTJ092U4hnFRERwcaNGwt9L4HMb4/kb2jSpES2c/DgQZYtW8aBAwcA5z/jmTNnaNCgQYls31y7inLEXZKulXryvqpcuTIxMTHMnTuXfv365Xr9/vvvZ86cOW6FubxMnDjRDRZHjx4lPDwccMrleoubFGbRokXs2rWLTz75hPnz55OYmMiSJUsKXW/kyJEkJCSwdu1a1q1bxxtvvJHncjnrvC9btoyRI0e621i6dClDhgzJtzhWUYtmxcTEEBMTU6R1CrN48WIef/xxZs2axcCBA90StTfddBOHDh2iUaNGfP311wwaNIhvv/2WgwcPcuDAAV566aU851LccMMNHD16tETb6G/8Nsh36VH4L82CnDp1irffftv9j1uvXj2GDh1Kly5dLM+88WvXUj35Vq1acejQIc6cOVPg0fwDDzzA3Llz8ywHWqVKFSZPnszzzz+f7/reeQ3gnJPfsWOHT+3LKSwsjLCwMB544AFatmzpU5Dv378/MTEx3HbbbQW+x6x13jMzM3n77bdZvXo1c+bMQVVJSUkhLS2NRo0a5TofnpqaSsuWLalbty516tTh4MGDhR7Nx8bG5nkKpFu3bixYsCDbc02bNnUr14FT5z0qKirXum3btuXjjz8GnKF77+hD9erV3dMxnTt3JiQkhH379rF161a++uorWrRoQUZGBidOnCAqKsrdV0Wu815mVNWvbtc1aKfjR3yjGadPanFlZmbqn/70J3344Yd1/Pjxunz5cj137lyxt2eM1549e8q7CVq7dm33/rZt27R58+Z6+fJlPXfunLZs2VITExNVVfXcuXMaHR2tCxYsUFXVnTt3akhIiO7du1dVnf8nf//733Ntv6DlZs2apU8++aSqqq5atUqdrxjV9evXa3R0dLbt7N69W7t06aKhoaF69OhRVVVdu3at3n777ZqWlqaqqocPH9ZffvmlwPcbExOjY8aM0YsXL6qq6okTJ3T58uWqqhocHKwnTzrfFa+88oo2a9ZMH3roIVVVfeONN3TixImqqnrx4kUNCQnRG264QZOTkwvcX3BwcIGv5/Ve09LSdP369e7jxMREbd++fbZlsn5uqqozZszQ2NhYVVVdtmyZfv3116qq2r17d926dWuu/Y4YMcL9bNeuXau9e/fO9vqDDz6ocXFxqqrauXNnXbdunaqqpqSkaGhoqO7fv19VnX7q27evnj592m27d73iSklJ0RYtWmhqaqqmpqZqixYtNCUlJddy3s86MzNTH3jgAf3nP/+pqs5nmpGRoaqqBw4c0CZNmuRaPzk5OVef9u/fXzdv3nxVbS9reX2HAF9pMWOmXx6yZlY5TeW61xVpHVV1jzAqVapEv379CA8PZ9asWQwbNsx+7ZmAFMj15L1mz57N9ddfT7t27ejQoQP9+/fP84h33Lhx+c7sr1atGo8//jgnTpwocF++WrduXbY679u3b+cvf/mLW+d9xowZPh3Fe40cOZKIiIgClylKnfelS5cya9YswsPDufvuu5kxYwYhISEATJgwgR49ehAZGUmHDh246667rnp0s2HDhkybNo3IyEgiIyOZPn26OwnvkUcecS8JXLZsGa1bt6Zt27Y0adKEhx9+GIANGzbQsWNHwsPDGTp0KK+99pq7fn4uX77M/v37ue22YlVoDRh+V0/++obtdVC/hbwe/58+r3PkyBESEhIIDQ11JyGpapHPQRlTGKsnb8rL+fPn6dGjBxs3bqRy5crl3Zxyt2rVKrZt21ahy8DmxerJg8+57s6ePcvq1av59NNPuXLlCkePHqVfv35UrVrVArwxJqDUrFmTmTNncuTIEZo3b17ezSl3GRkZ5ZqkqaLwyyAvWvAlbVeuXOGzzz5j1apVpKenIyJERUUxaNAguxzOGBOw+vTpU95NqDCGDRtW3k2oEPwyyFeqcibf19LS0oiNjeXw4cMAtGnThlGjRtkvW1Nm7FSQMaY4SuP0uV8GeTR30gOvOnXqULt2bRo1asSIESPo3LmzfeGaMlOjRg1SUlJo1KiR/d0ZY3ymnsscfclKWBR+GeQrVct071+8eJEPP/yQO+64g5tuugkR4dFHH6V27dpuIgVjykpQUBCHDx/m5MmT5d0UY4yfqVGjRonXRynVIC8ifYGXgcrAIlWdl+P16sBSoDOQAoxQ1R8L227NGpdRVb788kuWL1/Or7/+SnJyMk888QSAZasz5aZq1apuVjdjjClvpRbkRaQy8ArQCzgMbBWRNaqatRzUOOBXVW0lIiOB54H8kzN7VKv5M3PnznVzUQcHBxMdHV3i78EYY4zxZ6V5JH87sF9VDwKISALwGyBrkP8N8Kzn/krgbyIiWsDsA5UznEj9jBOpTvGG++67j65du1oqWmOMMSaH0gzyTYGfszw+DNyR3zKqmiEip4FGwP/ku1W5gIjQu3dvBgwYkGflIWOMMcb4ycQ7EXkUeNTz8OLixYt3L168uDybFOiuo6AfWqakWD+XPuvj0md9XPraFHfF0gzyR4BmWR4HeZ7La5nDIlIFqIczAS8bVV0ILAQQka+Km97P+Mb6uGxYP5c+6+PSZ31c+kTkq+KuW5onsrcCoSLSUkSqASOBNTmWWQM85Lk/FPi0oPPxxhhjjPFdqR3Je86xTwLW4lxCt1hVvxWRP+OUzVsD/BN4U0T2A6k4PwSMMcYYUwJK9Zy8qv438N85npue5f4FoKgJhheWQNNMwayPy4b1c+mzPi591selr9h97HelZo0xxhjjG7u43BhjjAlQFTbIi0hfEdkrIvtFZGoer1cXkbc8r38hIi3KvpX+zYc+/qOI7BGRb0RknYgEl0c7/VlhfZxluSEioiJis5SLwZd+FpHhnr/nb0Xk/5d1G/2dD98XzUVkvYhs93xn3Fse7fRnIrJYRE6IyO58XhcRWeD5DL4RkYhCN6qqFe6GM1HvAHAzUA3YCbTLsczvgdc890cCb5V3u/3p5mMf9wBqee5PsD4u+T72LPcfwAZgC3Bbebfb324+/i2HAtuBBp7HN5R3u/3p5mMfLwQmeO63A34s73b72w3oBkQAu/N5/V7gQ0CAO4EvCttmRT2Sd1PiquolwJsSN6vfAHGe+yuBnmK1PYui0D5W1fWqes7zcAtOrgPjO1/+jgFm4dRtuFCWjQsgvvTzeOAVVf0VQFVPlHEb/Z0vfaxAXc/9esDRMmxfQFDVDThXmuXnN8BSdWwB6ovITQVts6IG+bxS4jbNbxlVzQC8KXGNb3zp46zG4fyCNL4rtI89w23NVPWDsmxYgPHlb7k10FpENorIFk+FTOM7X/r4WeC3InIY56qqx8qmadeUon5v+0daW1O+ROS3wG1A9/JuSyARkUrAi8CYcm7KtaAKzpB9FM6I1AYRCVPVU+XaqsAyCliiqi+ISBecHCgdVPVKeTfsWlZRj+SLkhKXglLimnz50seIyD3A08BAVb1YRm0LFIX18X8AHYAkEfkR5xzbGpt8V2S+/C0fBtao6mVVTQb24QR94xtf+ngcsBxAVTcDNXDy2puS49P3dlYVNchbStzSV2gfi8itwD9wArydwyy6AvtYVU+r6nWq2kJVW+DMexioqsXOU32N8uX74l2co3hE5Dqc4fuDZdlIP+dLHx8CegKIyC04Qf5kmbYy8K0BHvTMsr8TOK2qxwpaoUIO16ulxC11PvZxLFAHWOGZ03hIVQeWW6P9jI99bK6Sj/28FugtInuATCBGVW3kz0c+9vFk4HUReQJnEt4YO/AqGhFZhvNj9DrP3IYZQFUAVX0NZ67DvcB+4BzwcKHbtM/AGGOMCUwVdbjeGGOMMVfJgrwxxhgToCzIG2OMMQHKgrwxxhgToCzIG2OMMQHKgrwx5UBEMkVkR5ZbiwKWTS+B/S0RkWTPvrZ5MpIVdRuLRKSd5/5/5Xht09W20bMdb7/sFpH3RKR+IcuHW7UzY/Jnl9AZUw5EJF1V65T0sgVsYwnwvqquFJHewHxV7XgV27vqNhW2XRGJA/ap6pwClh+DU7lvUkm3xZhAYEfyxlQAIlJHRNZ5jrJ3iUiuanUicpOIbMhypHuX5/neIrLZs+4KESks+G4AWnnW/aNnW7tF5P95nqstIh+IyE7P8yM8zyeJyG0iMg+o6WlHvOe1dM+/CSISnaXNS0RkqIhUFpFYEdnqqYP9f33ols14im+IyO2e97hdRDaJSBtP5rU/AyM8bRnhaftiEfnSs2xeVf+MuWZUyIx3xlwDaorIDs/9ZGAYMFhVz3jSrm4RkTU5MobdD6xV1TkiUhmo5Vn2GeAeVT0rIn8C/ogT/PIzANglIp1xMmbdgVOf+gsR+QynZvhRVY0GEJF6WVdW1akiMklVw/PY9lvAcOADTxDuCUzAyWt+WlUjRaQ6sFFEPvbkkc/F8/564mS2BPgeuMuTee0e4DlVHSIi08lyJC8iz+GkuB7rGer/UkQ+UdWzBfSHMQHLgrwx5eN81iApIlWB50SkG3AF5wi2MXA8yzpbgcWeZd9V1R0i0h1ohxM0AarhHAHnJVZEnsHJJz4OJ4iu8gZAEXkHuAv4CHhBRJ7HGeL/vAjv60PgZU8g7wtsUNXznlMEHUVkqGe5ejgFYnIGee+Pn6bAd0BiluXjRCQUJ2Vq1Xz23xsYKCJTPI9rAM092zLmmmNB3piKYTRwPdBZVS+LU5WuRtYFVHWD50dANLBERF4EfgUSVXWUD/uIUdWV3gci0jOvhVR1nzh17u8FZovIOlUtaGQg67oXRCQJ6AOMABK8uwMeU9W1hWzivKqGi0gtnDzpE4EFwCxgvaoO9kxSTMpnfQGGqOpeX9prTKCzc/LGVAz1gBOeAN8DCM65gIgEA7+o6uvAIiACp3Ldf4qI9xx7bRFp7eM+PwcGiUgtEakNDAY+F5EmwDlV/RdOkaKIPNa97BlRyMtbOKcBvKMC4ATsCd51RKS1Z595UtVzwOPAZPl3KWlvSc0xWRZNwynZ67UWeEw8wxriVFI05pplQd6YiiEeuE1EdgEP4pyDzikK2Cki23GOkl9W1ZM4QW+ZiHyDM1Tf1pcdquo2YAnwJfAFsEhVtwNhOOeyd+BUwZqdx+oLgW+8E+9y+BjoDnyiqpc8zy0C9gDbRGQ3TgnjAkcSPW35BhgF/AWY63nvWddbD7TzTrzDOeKv6mnbt57Hxlyz7BI6Y4wxJkDZkbwxxhgToCzIG2OMMQHKgrwxxhgToCzIG2OMMQHKgrwxxhgToCzIG2OMMQHKgrwxxhgToCzIG2OMMQHqfwHHqwj0aFtYKwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "linewidth = 2\n",
    "plt.figure(figsize=(8, 5))\n",
    "\n",
    "plt.plot(fprB, tprB, color='#db6114', lw=linewidth, label='ROC Curve LSTM Stochastic GD (AUC = %0.3f)' % auc(fprB, tprB))\n",
    "plt.plot(fprMB, tprMB, color='#1565c0', lw=linewidth, label='ROC Curve LSTM Mini Batch(AUC = %0.3f)' % auc(fprMB, tprMB))\n",
    "plt.plot(fpr3L, tpr3L, color='#2e7d32', lw=linewidth, label='ROC Curve LSTM Stacked Layers (AUC = %0.3f)' % auc(fpr3L, tpr3L))\n",
    "plt.plot(fprBi, tprBi, color='#ffc107', lw=linewidth, label='ROC Curve Bidirectional LSTM (AUC = %0.3f)' % auc(fprBi, tprBi))\n",
    "plt.plot(fprCNN, tprCNN, color='#6557d2', lw=linewidth, label='ROC Curve  CNN + LSTM (AUC = %0.3f)' % auc(fprCNN, tprCNN))\n",
    "plt.plot([0, 1], [0, 1], color='#616161', lw=linewidth, linestyle='--')\n",
    "\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('ROC Curve Plots')\n",
    "plt.legend(loc=\"lower right\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FbYw2w9EUAwH"
   },
   "source": [
    "As we can see, all models show approximately the same result, but so far the best is the Bidirectional LSTM model (95.8%), so we gonna use it. Here we are finished, then we move on to the development of a web application."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Part III - Training and Development of Different LSTM Models.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}