Knn on Amazon food using 4 methods.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [18] Amazon Fine Food Reviews Analysis\n",
    "\n",
    "\n",
    "Data Source: https://www.kaggle.com/snap/amazon-fine-food-reviews\n",
    "\n",
    "The Amazon Fine Food Reviews dataset consists of reviews of fine foods from Amazon.<br>\n",
    "\n",
    "Number of reviews: 568,454<br>\n",
    "Number of users: 256,059<br>\n",
    "Number of products: 74,258<br>\n",
    "Timespan: Oct 1999 - Oct 2012<br>\n",
    "Number of Attributes/Columns in data: 10 \n",
    "\n",
    "Attribute Information:\n",
    "\n",
    "1. Id\n",
    "2. ProductId - unique identifier for the product\n",
    "3. UserId - unqiue identifier for the user\n",
    "4. ProfileName\n",
    "5. HelpfulnessNumerator - number of users who found the review helpful\n",
    "6. HelpfulnessDenominator - number of users who indicated whether they found the review helpful or not\n",
    "7. Score - rating between 1 and 5\n",
    "8. Time - timestamp for the review\n",
    "9. Summary - brief summary of the review\n",
    "10. Text - text of the review\n",
    "\n",
    "\n",
    "#### Objective:\n",
    "Given a review, determine whether the review is positive (Rating of 4 or 5) or negative (rating of 1 or 2).\n",
    "\n",
    "<br>\n",
    "[Q] How to determine if a review is positive or negative?<br>\n",
    "<br> \n",
    "[Ans] We could use the Score/Rating. A rating of 4 or 5 could be cosnidered a positive review. A review of 1 or 2 could be considered negative. A review of 3 is nuetral and ignored. This is an approximate and proxy way of determining the polarity (positivity/negativity) of a review.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.1] Loading the data\n",
    "\n",
    "The dataset is available in two forms\n",
    "1. .csv file\n",
    "2. SQLite Database\n",
    "\n",
    "In order to load the data, We have used the SQLITE dataset as it easier to query the data and visualise the data efficiently.\n",
    "<br> \n",
    "\n",
    "Here as we only want to get the global sentiment of the recommendations (positive or negative), we will purposefully ignore all Scores equal to 3. If the score id above 3, then the recommendation wil be set to \"positive\". Otherwise, it will be set to \"negative\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\raj52\\Anaconda3\\lib\\site-packages\\gensim\\utils.py:1197: UserWarning: detected Windows; aliasing chunkize to chunkize_serial\n",
      "  warnings.warn(\"detected Windows; aliasing chunkize to chunkize_serial\")\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "import sqlite3\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import nltk\n",
    "import string\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn import neighbors\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics import roc_curve, auc\n",
    "from nltk.stem.porter import PorterStemmer\n",
    "from sklearn.decomposition import TruncatedSVD\n",
    "import re\n",
    "# Tutorial about Python regular expressions: https://pymotw.com/2/re/\n",
    "import string\n",
    "from tqdm import tqdm\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem import PorterStemmer\n",
    "from nltk.stem.wordnet import WordNetLemmatizer\n",
    "\n",
    "from gensim.models import Word2Vec\n",
    "from gensim.models import KeyedVectors\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# using the SQLite Table to read data.\n",
    "con = sqlite3.connect('database.sqlite') \n",
    "\n",
    "\n",
    "\n",
    "#filtering only positive and negative reviews i.e. \n",
    "# not taking into consideration those reviews with Score=3\n",
    "filtered_data = pd.read_sql_query(\"\"\" SELECT * FROM Reviews WHERE Score != 3 \"\"\", con) \n",
    "\n",
    "\n",
    "# Give reviews with Score>3 a positive rating, and reviews with a score<3 a negative rating.\n",
    "def partition(x):\n",
    "    if x < 3:\n",
    "        return 'negative'\n",
    "    return 'positive'\n",
    "\n",
    "#changing reviews with score less than 3 to be positive and vice-versa\n",
    "actualScore = filtered_data['Score']\n",
    "positiveNegative = actualScore.map(partition)\n",
    "filtered_data['Score'] = positiveNegative"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(525814, 10)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>B001E4KFG0</td>\n",
       "      <td>A3SGXH7AUHU8GW</td>\n",
       "      <td>delmartian</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>positive</td>\n",
       "      <td>1303862400</td>\n",
       "      <td>Good Quality Dog Food</td>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>B00813GRG4</td>\n",
       "      <td>A1D87F6ZCVE5NK</td>\n",
       "      <td>dll pa</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>negative</td>\n",
       "      <td>1346976000</td>\n",
       "      <td>Not as Advertised</td>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>B000LQOCH0</td>\n",
       "      <td>ABXLMWJIXXAIN</td>\n",
       "      <td>Natalia Corres \"Natalia Corres\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>positive</td>\n",
       "      <td>1219017600</td>\n",
       "      <td>\"Delight\" says it all</td>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>B000UA0QIQ</td>\n",
       "      <td>A395BORC6FGVXV</td>\n",
       "      <td>Karl</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>negative</td>\n",
       "      <td>1307923200</td>\n",
       "      <td>Cough Medicine</td>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A1UQRSCLF8GW1T</td>\n",
       "      <td>Michael D. Bigham \"M. Wassir\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>positive</td>\n",
       "      <td>1350777600</td>\n",
       "      <td>Great taffy</td>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id   ProductId          UserId                      ProfileName  \\\n",
       "0   1  B001E4KFG0  A3SGXH7AUHU8GW                       delmartian   \n",
       "1   2  B00813GRG4  A1D87F6ZCVE5NK                           dll pa   \n",
       "2   3  B000LQOCH0   ABXLMWJIXXAIN  Natalia Corres \"Natalia Corres\"   \n",
       "3   4  B000UA0QIQ  A395BORC6FGVXV                             Karl   \n",
       "4   5  B006K2ZZ7K  A1UQRSCLF8GW1T    Michael D. Bigham \"M. Wassir\"   \n",
       "\n",
       "   HelpfulnessNumerator  HelpfulnessDenominator     Score        Time  \\\n",
       "0                     1                       1  positive  1303862400   \n",
       "1                     0                       0  negative  1346976000   \n",
       "2                     1                       1  positive  1219017600   \n",
       "3                     3                       3  negative  1307923200   \n",
       "4                     0                       0  positive  1350777600   \n",
       "\n",
       "                 Summary                                               Text  \n",
       "0  Good Quality Dog Food  I have bought several of the Vitality canned d...  \n",
       "1      Not as Advertised  Product arrived labeled as Jumbo Salted Peanut...  \n",
       "2  \"Delight\" says it all  This is a confection that has been around a fe...  \n",
       "3         Cough Medicine  If you are looking for the secret ingredient i...  \n",
       "4            Great taffy  Great taffy at a great price.  There was a wid...  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print (filtered_data.shape) #looking at the number of attributes and size of the data\n",
    "filtered_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Exploratory Data Analysis\n",
    "\n",
    "## [18.1.2] Data Cleaning: Deduplication\n",
    "\n",
    "It is observed (as shown in the table below) that the reviews data had many duplicate entries. Hence it was necessary to remove duplicates in order to get unbiased results for the analysis of the data.  Following is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>150519</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A12HY5OZ2QNK4N</td>\n",
       "      <td>Elizabeth H. Roessner</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1256774400</td>\n",
       "      <td>It's a great book!</td>\n",
       "      <td>I've always loved chicken soup and rice. My la...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>150511</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A1C9K534BCI9GO</td>\n",
       "      <td>Laura Purdie Salas</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1344211200</td>\n",
       "      <td>Charming and childlike</td>\n",
       "      <td>A charming, rhyming book that describes the ci...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>150512</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A1DJXZA5V5FFVA</td>\n",
       "      <td>A. Conway</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1338249600</td>\n",
       "      <td>Must have.</td>\n",
       "      <td>I set aside at least an hour each day to read ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>150497</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A1HKYQOFC8ZZCH</td>\n",
       "      <td>Maria Apolloni \"lanarossa\"</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1334707200</td>\n",
       "      <td>The story is great, the softcover book is disa...</td>\n",
       "      <td>I give five stars to the Maurice Sendak story....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>150500</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A1IJKK6Q1GTEAY</td>\n",
       "      <td>A Customer</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1009324800</td>\n",
       "      <td>It Was a favorite!</td>\n",
       "      <td>This was a favorite book of mine when I was a ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>150522</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A1S3C5OFU508P3</td>\n",
       "      <td>Charles Ashbacher</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1219536000</td>\n",
       "      <td>Children will find it entertaining and a gener...</td>\n",
       "      <td>This book contains a collection of twelve shor...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>150507</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A1S4A3IQ2MU7V4</td>\n",
       "      <td>sally sue \"sally sue\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1191456000</td>\n",
       "      <td>chicken soup with rice months</td>\n",
       "      <td>This is a fun way for children to learn their ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>150529</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A25ACLV5KPB4W</td>\n",
       "      <td>Matt Hetling \"Matt\"</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1108425600</td>\n",
       "      <td>Nice cadence, catchy rhymes</td>\n",
       "      <td>In June&lt;br /&gt;I saw a charming group&lt;br /&gt;of ro...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>150506</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2IW4PEEKO2R0U</td>\n",
       "      <td>Tracy</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1194739200</td>\n",
       "      <td>Love the book, miss the hard cover version</td>\n",
       "      <td>I grew up reading these Sendak books, and watc...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>150514</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2ONB6ZA292PA</td>\n",
       "      <td>Rosalind Matzner</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1313884800</td>\n",
       "      <td>Chicken soup with Rice</td>\n",
       "      <td>It's a great book with adorable illustrations....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>150523</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2P4F2UO0UMP8C</td>\n",
       "      <td>Elizabeth A. Curry \"Lovely Librarian\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1096675200</td>\n",
       "      <td>MMMM chicken soup....</td>\n",
       "      <td>Summary:  A young boy describes the usefulness...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>150505</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2PTSM496CF40Z</td>\n",
       "      <td>Jason A. Teeple \"Nobody made a greater mistak...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1210809600</td>\n",
       "      <td>A classic</td>\n",
       "      <td>Get the movie or sound track and sing along wi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>150525</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2QID6VCFTY51R</td>\n",
       "      <td>Rick</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1025481600</td>\n",
       "      <td>In December it will be, my snowman's anniversa...</td>\n",
       "      <td>My daughter loves all the \"Really Rosie\" books...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>150495</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2QXYSXGQ2PKAU</td>\n",
       "      <td>Support for Providers</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1129507200</td>\n",
       "      <td>Disappointed</td>\n",
       "      <td>When I ordered a large amount of these books t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>150515</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2RTT81R6Y3R7X</td>\n",
       "      <td>Lindylu</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1303171200</td>\n",
       "      <td>One of our family's favorite books</td>\n",
       "      <td>This book is a family favorite and was read to...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>150527</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A367OXCD2R6MC1</td>\n",
       "      <td>Laurel \"I love the internet.\"</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1323820800</td>\n",
       "      <td>This book is much too small</td>\n",
       "      <td>It was a great price, but I can't read it to m...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>150509</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3CMRKGE0P909G</td>\n",
       "      <td>Teresa</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>1018396800</td>\n",
       "      <td>A great way to learn the months</td>\n",
       "      <td>This is a book of poetry about the months of t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>150499</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3E7R866M94L0C</td>\n",
       "      <td>L. Barker \"simienwolf\"</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1065830400</td>\n",
       "      <td>Can't explain why</td>\n",
       "      <td>This book has been a favorite of mine since I ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>150526</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3E9QZFE9KXH8J</td>\n",
       "      <td>R. Mitchell</td>\n",
       "      <td>11</td>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>1129507200</td>\n",
       "      <td>awesome book poor size</td>\n",
       "      <td>This is one of the best children's books ever ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>150496</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3KKR87BJ0C595</td>\n",
       "      <td>Gretchen Goodfellow \"Lover of children's lit\"</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>1111363200</td>\n",
       "      <td>You'll use it once, you'll use it twice</td>\n",
       "      <td>One of my earliest memories is of this book.  ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>150516</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3OI7ZGH6WZJ5G</td>\n",
       "      <td>Mary Jane Rogers \"Maedchen\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1293840000</td>\n",
       "      <td>Darling!</td>\n",
       "      <td>The same author wrote \"Where the Wild Things A...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>150503</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3R5XMPFU8YZ4D</td>\n",
       "      <td>Her Royal Motherliness \"Nana\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1233964800</td>\n",
       "      <td>so fun to read</td>\n",
       "      <td>This is my grand daughter's and my favorite bo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>150521</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3RMCRB2NDTDYP</td>\n",
       "      <td>Carol Carruthers</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1243468800</td>\n",
       "      <td>This book is great!</td>\n",
       "      <td>My 7 year old daughter brought this book home ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>150498</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A3SJWISOCP31TR</td>\n",
       "      <td>R. J. Wells</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1176336000</td>\n",
       "      <td>A Gem of a Book</td>\n",
       "      <td>This is a wonderful little book. I loved it 40...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>150517</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>ABW4IC5G5G8B5</td>\n",
       "      <td>kevin clark</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1291075200</td>\n",
       "      <td>good for children</td>\n",
       "      <td>Classic children's book, can't go wrong. I rea...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>150524</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>ACITT7DI6IDDL</td>\n",
       "      <td>shari zychinski</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>939340800</td>\n",
       "      <td>EVERY book is educational</td>\n",
       "      <td>this witty little book makes my son laugh at l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>150520</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>ADBFSA9KTQANE</td>\n",
       "      <td>James L. Hammock \"Pucks Buddy\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1256688000</td>\n",
       "      <td>Great Gift</td>\n",
       "      <td>This book was purchased as a birthday gift for...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>150528</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>AGQWHPNDZGUD0</td>\n",
       "      <td>Bolt81</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1296864000</td>\n",
       "      <td>Size does matter</td>\n",
       "      <td>This book was purchased to be used in a classr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>150501</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>AJ46FKXOVC7NR</td>\n",
       "      <td>Nicholas A Mesiano</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>940809600</td>\n",
       "      <td>This whole series is great way to spend time w...</td>\n",
       "      <td>I can remember seeing the show when it aired o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>150518</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>AK1L4EJBA23JF</td>\n",
       "      <td>L. M. Kraus</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1288224000</td>\n",
       "      <td>love this book</td>\n",
       "      <td>Great book, perfect condition arrived in a sho...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568424</th>\n",
       "      <td>193174</td>\n",
       "      <td>B009RSR8HO</td>\n",
       "      <td>A4P6AN2L435PV</td>\n",
       "      <td>romarc</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350432000</td>\n",
       "      <td>LOVE!!  LOVE!!</td>\n",
       "      <td>LOVE, LOVE this sweetener!!  I use it in all m...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568425</th>\n",
       "      <td>193167</td>\n",
       "      <td>B009RSR8HO</td>\n",
       "      <td>A5F9OUO3F2N7C</td>\n",
       "      <td>Jan</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350604800</td>\n",
       "      <td>The Best Ever</td>\n",
       "      <td>Zero sweetner is the BEST I have ever used.One...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568426</th>\n",
       "      <td>193176</td>\n",
       "      <td>B009RSR8HO</td>\n",
       "      <td>A76WHW051R3KV</td>\n",
       "      <td>Shawn \"Shawn\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350432000</td>\n",
       "      <td>My #1 Sweetener of choice</td>\n",
       "      <td>What a wonderful product! It's perfect to use ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568427</th>\n",
       "      <td>193166</td>\n",
       "      <td>B009RSR8HO</td>\n",
       "      <td>AD0V42PRKCDBM</td>\n",
       "      <td>Rachelle</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350777600</td>\n",
       "      <td>Best sweetner ever!!!</td>\n",
       "      <td>This sweetner is so great. A little bit goes a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568428</th>\n",
       "      <td>193171</td>\n",
       "      <td>B009RSR8HO</td>\n",
       "      <td>AH2FVNP7Z6PZH</td>\n",
       "      <td>Marty Campbell</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350432000</td>\n",
       "      <td>The BEST sugar replacement on the market!</td>\n",
       "      <td>I've been using Fat to Skinny Zero since it wa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568429</th>\n",
       "      <td>204271</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A133WGB2RLKB1T</td>\n",
       "      <td>Temple Gordon</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1321228800</td>\n",
       "      <td>Walkers smkey bacon crisps</td>\n",
       "      <td>These are amazing chips but they just cost too...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568430</th>\n",
       "      <td>204279</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A1PVBIUKEDNGVP</td>\n",
       "      <td>Steve L</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1333843200</td>\n",
       "      <td>One Word, \"YUM!\"</td>\n",
       "      <td>If you like salt and vinegar crisps (chips), b...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568431</th>\n",
       "      <td>204270</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A1V93C9YW5T0LQ</td>\n",
       "      <td>N. Hunt \"Übergeek\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1335571200</td>\n",
       "      <td>Americans beware, curious labeling</td>\n",
       "      <td>the product is listed as a 150g 6-pack.&lt;br /&gt;T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568432</th>\n",
       "      <td>204275</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A1XPE0WCC6RYVO</td>\n",
       "      <td>AnthonyT</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1344988800</td>\n",
       "      <td>Excellent</td>\n",
       "      <td>These are the best flavor chips, my daughter a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568433</th>\n",
       "      <td>204283</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A2QXG1QOV4MTVL</td>\n",
       "      <td>Wordup \"Wordup2you\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1321920000</td>\n",
       "      <td>Stale Chips</td>\n",
       "      <td>Item came promptly however the crisps were 3 m...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568434</th>\n",
       "      <td>204284</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A2SB8DPH72UOM7</td>\n",
       "      <td>Tim C.</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1317600000</td>\n",
       "      <td>Yum!</td>\n",
       "      <td>Bought these the other day while I was in Cana...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568435</th>\n",
       "      <td>204277</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A2TWDT92R8VPTI</td>\n",
       "      <td>d wilson \"Visitor from a Perpendicular Universe\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1337904000</td>\n",
       "      <td>Tastes just like bacon!</td>\n",
       "      <td>I had a bag of these during a trip to London. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568436</th>\n",
       "      <td>204285</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A2XNO53D6J6322</td>\n",
       "      <td>dragenfli254</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1317081600</td>\n",
       "      <td>Delish!</td>\n",
       "      <td>I had these wonderful chips in Ireland a few y...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568437</th>\n",
       "      <td>204282</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A373QMETEUKMS7</td>\n",
       "      <td>Rebecca Wade</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1327017600</td>\n",
       "      <td>Excellent!</td>\n",
       "      <td>The crisps are awesome. Give me English crisps...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568438</th>\n",
       "      <td>204274</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A379KV6EQ66ZJR</td>\n",
       "      <td>Craig</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1347062400</td>\n",
       "      <td>Awesome Crisps!!! Arrived in just 8 days in Te...</td>\n",
       "      <td>These crisps are my favorite.  I ordered these...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568439</th>\n",
       "      <td>204281</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A3CBCI8ZU6A9XM</td>\n",
       "      <td>Cody B.</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1328486400</td>\n",
       "      <td>Cody B.</td>\n",
       "      <td>I loved the chips they were AWESOME!!! but tha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568440</th>\n",
       "      <td>204278</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A3M922QSBYYXR</td>\n",
       "      <td>Jeannie Jordahl</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1335744000</td>\n",
       "      <td>These were amazing!</td>\n",
       "      <td>This chips kind of reminded me of bacon bits. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568441</th>\n",
       "      <td>204276</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>A3U0YIPTZX8DZ4</td>\n",
       "      <td>vee</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1339977600</td>\n",
       "      <td>Re-Rating</td>\n",
       "      <td>Okay, I jumped the gun, because they were send...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568442</th>\n",
       "      <td>204273</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>AG4YGLLIE8BWP</td>\n",
       "      <td>Miwintee</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1351123200</td>\n",
       "      <td>Makes me drool just thinking of them</td>\n",
       "      <td>The Brit's have out done us. The flavor is sup...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568443</th>\n",
       "      <td>204280</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>AI1G344L7R1TN</td>\n",
       "      <td>Brian M. Schissler</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1332979200</td>\n",
       "      <td>WOW.....</td>\n",
       "      <td>This could possibly be the best tasting chip I...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568444</th>\n",
       "      <td>204286</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>AVRU1Z8N59UZV</td>\n",
       "      <td>LIsa Fresch</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1349654400</td>\n",
       "      <td>Walkers Crisps 6 pack</td>\n",
       "      <td>I ordered this product on Amazon to get some o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568445</th>\n",
       "      <td>204272</td>\n",
       "      <td>B009SA5NNW</td>\n",
       "      <td>AWFA8N9IXELVH</td>\n",
       "      <td>No Pen Name</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1351123200</td>\n",
       "      <td>Deceptive description</td>\n",
       "      <td>On Oct 9 I ordered from a different vendor the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568446</th>\n",
       "      <td>188389</td>\n",
       "      <td>B009SF0TN6</td>\n",
       "      <td>A1L0GWGRK4BYPT</td>\n",
       "      <td>Bety Robinson</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350518400</td>\n",
       "      <td>Amazing!! Great sauce for everything!</td>\n",
       "      <td>You have to try this sauce to believe it! It s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568447</th>\n",
       "      <td>226019</td>\n",
       "      <td>B009SMKESO</td>\n",
       "      <td>A35K4XT7T1ZIFU</td>\n",
       "      <td>Inez Rivera</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1304985600</td>\n",
       "      <td>Not a bad product.</td>\n",
       "      <td>This review is for the boneless ham. A little ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568448</th>\n",
       "      <td>221795</td>\n",
       "      <td>B009SR4OQ2</td>\n",
       "      <td>A32A6X5KCP7ARG</td>\n",
       "      <td>sicamar</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1350604800</td>\n",
       "      <td>Awesome Taste</td>\n",
       "      <td>I bought this Hazelnut Paste (Nocciola Spread)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568449</th>\n",
       "      <td>191721</td>\n",
       "      <td>B009UOFTUI</td>\n",
       "      <td>AJVB004EB0MVK</td>\n",
       "      <td>D. Christofferson</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1345852800</td>\n",
       "      <td>weak coffee not good for a premium product and...</td>\n",
       "      <td>This coffee supposedly is premium, it tastes w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568450</th>\n",
       "      <td>1478</td>\n",
       "      <td>B009UOFU20</td>\n",
       "      <td>AJVB004EB0MVK</td>\n",
       "      <td>D. Christofferson</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1345852800</td>\n",
       "      <td>weak coffee not good for a premium product and...</td>\n",
       "      <td>This coffee supposedly is premium, it tastes w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568451</th>\n",
       "      <td>328482</td>\n",
       "      <td>B009UUS05I</td>\n",
       "      <td>ARL20DSHGVM1Y</td>\n",
       "      <td>Jamie</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1331856000</td>\n",
       "      <td>Perfect</td>\n",
       "      <td>The basket was the perfect sympathy gift when ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568452</th>\n",
       "      <td>5703</td>\n",
       "      <td>B009WSNWC4</td>\n",
       "      <td>AMP7K1O84DH1T</td>\n",
       "      <td>ESTY</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1351209600</td>\n",
       "      <td>DELICIOUS</td>\n",
       "      <td>Purchased this product at a local store in NY ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568453</th>\n",
       "      <td>327601</td>\n",
       "      <td>B009WVB40S</td>\n",
       "      <td>A3ME78KVX31T21</td>\n",
       "      <td>K'la</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1351123200</td>\n",
       "      <td>Tasty!</td>\n",
       "      <td>I purchased this to send to my son who's away ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>568454 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Id   ProductId          UserId  \\\n",
       "0       150519  0006641040  A12HY5OZ2QNK4N   \n",
       "1       150511  0006641040  A1C9K534BCI9GO   \n",
       "2       150512  0006641040  A1DJXZA5V5FFVA   \n",
       "3       150497  0006641040  A1HKYQOFC8ZZCH   \n",
       "4       150500  0006641040  A1IJKK6Q1GTEAY   \n",
       "5       150522  0006641040  A1S3C5OFU508P3   \n",
       "6       150507  0006641040  A1S4A3IQ2MU7V4   \n",
       "7       150529  0006641040   A25ACLV5KPB4W   \n",
       "8       150506  0006641040  A2IW4PEEKO2R0U   \n",
       "9       150514  0006641040   A2ONB6ZA292PA   \n",
       "10      150523  0006641040  A2P4F2UO0UMP8C   \n",
       "11      150505  0006641040  A2PTSM496CF40Z   \n",
       "12      150525  0006641040  A2QID6VCFTY51R   \n",
       "13      150495  0006641040  A2QXYSXGQ2PKAU   \n",
       "14      150515  0006641040  A2RTT81R6Y3R7X   \n",
       "15      150527  0006641040  A367OXCD2R6MC1   \n",
       "16      150509  0006641040  A3CMRKGE0P909G   \n",
       "17      150499  0006641040  A3E7R866M94L0C   \n",
       "18      150526  0006641040  A3E9QZFE9KXH8J   \n",
       "19      150496  0006641040  A3KKR87BJ0C595   \n",
       "20      150516  0006641040  A3OI7ZGH6WZJ5G   \n",
       "21      150503  0006641040  A3R5XMPFU8YZ4D   \n",
       "22      150521  0006641040  A3RMCRB2NDTDYP   \n",
       "23      150498  0006641040  A3SJWISOCP31TR   \n",
       "24      150517  0006641040   ABW4IC5G5G8B5   \n",
       "25      150524  0006641040   ACITT7DI6IDDL   \n",
       "26      150520  0006641040   ADBFSA9KTQANE   \n",
       "27      150528  0006641040   AGQWHPNDZGUD0   \n",
       "28      150501  0006641040   AJ46FKXOVC7NR   \n",
       "29      150518  0006641040   AK1L4EJBA23JF   \n",
       "...        ...         ...             ...   \n",
       "568424  193174  B009RSR8HO   A4P6AN2L435PV   \n",
       "568425  193167  B009RSR8HO   A5F9OUO3F2N7C   \n",
       "568426  193176  B009RSR8HO   A76WHW051R3KV   \n",
       "568427  193166  B009RSR8HO   AD0V42PRKCDBM   \n",
       "568428  193171  B009RSR8HO   AH2FVNP7Z6PZH   \n",
       "568429  204271  B009SA5NNW  A133WGB2RLKB1T   \n",
       "568430  204279  B009SA5NNW  A1PVBIUKEDNGVP   \n",
       "568431  204270  B009SA5NNW  A1V93C9YW5T0LQ   \n",
       "568432  204275  B009SA5NNW  A1XPE0WCC6RYVO   \n",
       "568433  204283  B009SA5NNW  A2QXG1QOV4MTVL   \n",
       "568434  204284  B009SA5NNW  A2SB8DPH72UOM7   \n",
       "568435  204277  B009SA5NNW  A2TWDT92R8VPTI   \n",
       "568436  204285  B009SA5NNW  A2XNO53D6J6322   \n",
       "568437  204282  B009SA5NNW  A373QMETEUKMS7   \n",
       "568438  204274  B009SA5NNW  A379KV6EQ66ZJR   \n",
       "568439  204281  B009SA5NNW  A3CBCI8ZU6A9XM   \n",
       "568440  204278  B009SA5NNW   A3M922QSBYYXR   \n",
       "568441  204276  B009SA5NNW  A3U0YIPTZX8DZ4   \n",
       "568442  204273  B009SA5NNW   AG4YGLLIE8BWP   \n",
       "568443  204280  B009SA5NNW   AI1G344L7R1TN   \n",
       "568444  204286  B009SA5NNW   AVRU1Z8N59UZV   \n",
       "568445  204272  B009SA5NNW   AWFA8N9IXELVH   \n",
       "568446  188389  B009SF0TN6  A1L0GWGRK4BYPT   \n",
       "568447  226019  B009SMKESO  A35K4XT7T1ZIFU   \n",
       "568448  221795  B009SR4OQ2  A32A6X5KCP7ARG   \n",
       "568449  191721  B009UOFTUI   AJVB004EB0MVK   \n",
       "568450    1478  B009UOFU20   AJVB004EB0MVK   \n",
       "568451  328482  B009UUS05I   ARL20DSHGVM1Y   \n",
       "568452    5703  B009WSNWC4   AMP7K1O84DH1T   \n",
       "568453  327601  B009WVB40S  A3ME78KVX31T21   \n",
       "\n",
       "                                             ProfileName  \\\n",
       "0                                  Elizabeth H. Roessner   \n",
       "1                                     Laura Purdie Salas   \n",
       "2                                              A. Conway   \n",
       "3                             Maria Apolloni \"lanarossa\"   \n",
       "4                                             A Customer   \n",
       "5                                      Charles Ashbacher   \n",
       "6                                  sally sue \"sally sue\"   \n",
       "7                                    Matt Hetling \"Matt\"   \n",
       "8                                                  Tracy   \n",
       "9                                       Rosalind Matzner   \n",
       "10                 Elizabeth A. Curry \"Lovely Librarian\"   \n",
       "11      Jason A. Teeple \"Nobody made a greater mistak...   \n",
       "12                                                  Rick   \n",
       "13                                 Support for Providers   \n",
       "14                                               Lindylu   \n",
       "15                         Laurel \"I love the internet.\"   \n",
       "16                                                Teresa   \n",
       "17                                L. Barker \"simienwolf\"   \n",
       "18                                           R. Mitchell   \n",
       "19         Gretchen Goodfellow \"Lover of children's lit\"   \n",
       "20                           Mary Jane Rogers \"Maedchen\"   \n",
       "21                         Her Royal Motherliness \"Nana\"   \n",
       "22                                      Carol Carruthers   \n",
       "23                                           R. J. Wells   \n",
       "24                                           kevin clark   \n",
       "25                                       shari zychinski   \n",
       "26                        James L. Hammock \"Pucks Buddy\"   \n",
       "27                                                Bolt81   \n",
       "28                                    Nicholas A Mesiano   \n",
       "29                                           L. M. Kraus   \n",
       "...                                                  ...   \n",
       "568424                                            romarc   \n",
       "568425                                               Jan   \n",
       "568426                                     Shawn \"Shawn\"   \n",
       "568427                                          Rachelle   \n",
       "568428                                    Marty Campbell   \n",
       "568429                                     Temple Gordon   \n",
       "568430                                           Steve L   \n",
       "568431                                N. Hunt \"Übergeek\"   \n",
       "568432                                          AnthonyT   \n",
       "568433                               Wordup \"Wordup2you\"   \n",
       "568434                                            Tim C.   \n",
       "568435  d wilson \"Visitor from a Perpendicular Universe\"   \n",
       "568436                                      dragenfli254   \n",
       "568437                                      Rebecca Wade   \n",
       "568438                                             Craig   \n",
       "568439                                           Cody B.   \n",
       "568440                                   Jeannie Jordahl   \n",
       "568441                                               vee   \n",
       "568442                                          Miwintee   \n",
       "568443                                Brian M. Schissler   \n",
       "568444                                       LIsa Fresch   \n",
       "568445                                       No Pen Name   \n",
       "568446                                     Bety Robinson   \n",
       "568447                                       Inez Rivera   \n",
       "568448                                           sicamar   \n",
       "568449                                 D. Christofferson   \n",
       "568450                                 D. Christofferson   \n",
       "568451                                             Jamie   \n",
       "568452                                              ESTY   \n",
       "568453                                              K'la   \n",
       "\n",
       "        HelpfulnessNumerator  HelpfulnessDenominator  Score        Time  \\\n",
       "0                          0                       0      5  1256774400   \n",
       "1                          0                       0      4  1344211200   \n",
       "2                          0                       0      5  1338249600   \n",
       "3                          2                       2      1  1334707200   \n",
       "4                          2                       2      5  1009324800   \n",
       "5                          0                       0      4  1219536000   \n",
       "6                          1                       1      4  1191456000   \n",
       "7                          0                       1      4  1108425600   \n",
       "8                          1                       1      4  1194739200   \n",
       "9                          0                       0      5  1313884800   \n",
       "10                         0                       0      4  1096675200   \n",
       "11                         1                       1      4  1210809600   \n",
       "12                         1                       2      5  1025481600   \n",
       "13                         3                       3      3  1129507200   \n",
       "14                         0                       0      5  1303171200   \n",
       "15                         0                       1      3  1323820800   \n",
       "16                         3                       4      5  1018396800   \n",
       "17                         2                       2      5  1065830400   \n",
       "18                        11                      18      1  1129507200   \n",
       "19                         3                       3      5  1111363200   \n",
       "20                         0                       0      5  1293840000   \n",
       "21                         1                       1      5  1233964800   \n",
       "22                         0                       0      5  1243468800   \n",
       "23                         2                       2      5  1176336000   \n",
       "24                         0                       0      5  1291075200   \n",
       "25                         0                       0      5   939340800   \n",
       "26                         0                       0      5  1256688000   \n",
       "27                         0                       1      3  1296864000   \n",
       "28                         2                       2      5   940809600   \n",
       "29                         0                       0      5  1288224000   \n",
       "...                      ...                     ...    ...         ...   \n",
       "568424                     0                       0      5  1350432000   \n",
       "568425                     0                       0      5  1350604800   \n",
       "568426                     0                       0      5  1350432000   \n",
       "568427                     0                       0      5  1350777600   \n",
       "568428                     0                       0      5  1350432000   \n",
       "568429                     1                       1      4  1321228800   \n",
       "568430                     0                       0      5  1333843200   \n",
       "568431                     1                       1      3  1335571200   \n",
       "568432                     0                       0      5  1344988800   \n",
       "568433                     0                       0      1  1321920000   \n",
       "568434                     0                       0      4  1317600000   \n",
       "568435                     0                       0      5  1337904000   \n",
       "568436                     0                       0      5  1317081600   \n",
       "568437                     0                       0      4  1327017600   \n",
       "568438                     0                       0      5  1347062400   \n",
       "568439                     0                       0      5  1328486400   \n",
       "568440                     0                       0      5  1335744000   \n",
       "568441                     0                       0      4  1339977600   \n",
       "568442                     0                       0      5  1351123200   \n",
       "568443                     0                       0      4  1332979200   \n",
       "568444                     0                       1      1  1349654400   \n",
       "568445                     0                       0      1  1351123200   \n",
       "568446                     0                       0      5  1350518400   \n",
       "568447                     0                       1      4  1304985600   \n",
       "568448                     1                       1      5  1350604800   \n",
       "568449                     0                       0      1  1345852800   \n",
       "568450                     0                       0      1  1345852800   \n",
       "568451                     0                       0      5  1331856000   \n",
       "568452                     0                       0      5  1351209600   \n",
       "568453                     0                       0      5  1351123200   \n",
       "\n",
       "                                                  Summary  \\\n",
       "0                                      It's a great book!   \n",
       "1                                  Charming and childlike   \n",
       "2                                              Must have.   \n",
       "3       The story is great, the softcover book is disa...   \n",
       "4                                      It Was a favorite!   \n",
       "5       Children will find it entertaining and a gener...   \n",
       "6                           chicken soup with rice months   \n",
       "7                             Nice cadence, catchy rhymes   \n",
       "8              Love the book, miss the hard cover version   \n",
       "9                                  Chicken soup with Rice   \n",
       "10                                  MMMM chicken soup....   \n",
       "11                                              A classic   \n",
       "12      In December it will be, my snowman's anniversa...   \n",
       "13                                           Disappointed   \n",
       "14                     One of our family's favorite books   \n",
       "15                            This book is much too small   \n",
       "16                        A great way to learn the months   \n",
       "17                                      Can't explain why   \n",
       "18                                 awesome book poor size   \n",
       "19                You'll use it once, you'll use it twice   \n",
       "20                                               Darling!   \n",
       "21                                         so fun to read   \n",
       "22                                    This book is great!   \n",
       "23                                        A Gem of a Book   \n",
       "24                                      good for children   \n",
       "25                              EVERY book is educational   \n",
       "26                                             Great Gift   \n",
       "27                                       Size does matter   \n",
       "28      This whole series is great way to spend time w...   \n",
       "29                                         love this book   \n",
       "...                                                   ...   \n",
       "568424                                     LOVE!!  LOVE!!   \n",
       "568425                                      The Best Ever   \n",
       "568426                          My #1 Sweetener of choice   \n",
       "568427                              Best sweetner ever!!!   \n",
       "568428          The BEST sugar replacement on the market!   \n",
       "568429                         Walkers smkey bacon crisps   \n",
       "568430                                   One Word, \"YUM!\"   \n",
       "568431                 Americans beware, curious labeling   \n",
       "568432                                          Excellent   \n",
       "568433                                        Stale Chips   \n",
       "568434                                               Yum!   \n",
       "568435                            Tastes just like bacon!   \n",
       "568436                                            Delish!   \n",
       "568437                                         Excellent!   \n",
       "568438  Awesome Crisps!!! Arrived in just 8 days in Te...   \n",
       "568439                                            Cody B.   \n",
       "568440                                These were amazing!   \n",
       "568441                                          Re-Rating   \n",
       "568442               Makes me drool just thinking of them   \n",
       "568443                                           WOW.....   \n",
       "568444                              Walkers Crisps 6 pack   \n",
       "568445                              Deceptive description   \n",
       "568446              Amazing!! Great sauce for everything!   \n",
       "568447                                 Not a bad product.   \n",
       "568448                                      Awesome Taste   \n",
       "568449  weak coffee not good for a premium product and...   \n",
       "568450  weak coffee not good for a premium product and...   \n",
       "568451                                            Perfect   \n",
       "568452                                          DELICIOUS   \n",
       "568453                                             Tasty!   \n",
       "\n",
       "                                                     Text  \n",
       "0       I've always loved chicken soup and rice. My la...  \n",
       "1       A charming, rhyming book that describes the ci...  \n",
       "2       I set aside at least an hour each day to read ...  \n",
       "3       I give five stars to the Maurice Sendak story....  \n",
       "4       This was a favorite book of mine when I was a ...  \n",
       "5       This book contains a collection of twelve shor...  \n",
       "6       This is a fun way for children to learn their ...  \n",
       "7       In June<br />I saw a charming group<br />of ro...  \n",
       "8       I grew up reading these Sendak books, and watc...  \n",
       "9       It's a great book with adorable illustrations....  \n",
       "10      Summary:  A young boy describes the usefulness...  \n",
       "11      Get the movie or sound track and sing along wi...  \n",
       "12      My daughter loves all the \"Really Rosie\" books...  \n",
       "13      When I ordered a large amount of these books t...  \n",
       "14      This book is a family favorite and was read to...  \n",
       "15      It was a great price, but I can't read it to m...  \n",
       "16      This is a book of poetry about the months of t...  \n",
       "17      This book has been a favorite of mine since I ...  \n",
       "18      This is one of the best children's books ever ...  \n",
       "19      One of my earliest memories is of this book.  ...  \n",
       "20      The same author wrote \"Where the Wild Things A...  \n",
       "21      This is my grand daughter's and my favorite bo...  \n",
       "22      My 7 year old daughter brought this book home ...  \n",
       "23      This is a wonderful little book. I loved it 40...  \n",
       "24      Classic children's book, can't go wrong. I rea...  \n",
       "25      this witty little book makes my son laugh at l...  \n",
       "26      This book was purchased as a birthday gift for...  \n",
       "27      This book was purchased to be used in a classr...  \n",
       "28      I can remember seeing the show when it aired o...  \n",
       "29      Great book, perfect condition arrived in a sho...  \n",
       "...                                                   ...  \n",
       "568424  LOVE, LOVE this sweetener!!  I use it in all m...  \n",
       "568425  Zero sweetner is the BEST I have ever used.One...  \n",
       "568426  What a wonderful product! It's perfect to use ...  \n",
       "568427  This sweetner is so great. A little bit goes a...  \n",
       "568428  I've been using Fat to Skinny Zero since it wa...  \n",
       "568429  These are amazing chips but they just cost too...  \n",
       "568430  If you like salt and vinegar crisps (chips), b...  \n",
       "568431  the product is listed as a 150g 6-pack.<br />T...  \n",
       "568432  These are the best flavor chips, my daughter a...  \n",
       "568433  Item came promptly however the crisps were 3 m...  \n",
       "568434  Bought these the other day while I was in Cana...  \n",
       "568435  I had a bag of these during a trip to London. ...  \n",
       "568436  I had these wonderful chips in Ireland a few y...  \n",
       "568437  The crisps are awesome. Give me English crisps...  \n",
       "568438  These crisps are my favorite.  I ordered these...  \n",
       "568439  I loved the chips they were AWESOME!!! but tha...  \n",
       "568440  This chips kind of reminded me of bacon bits. ...  \n",
       "568441  Okay, I jumped the gun, because they were send...  \n",
       "568442  The Brit's have out done us. The flavor is sup...  \n",
       "568443  This could possibly be the best tasting chip I...  \n",
       "568444  I ordered this product on Amazon to get some o...  \n",
       "568445  On Oct 9 I ordered from a different vendor the...  \n",
       "568446  You have to try this sauce to believe it! It s...  \n",
       "568447  This review is for the boneless ham. A little ...  \n",
       "568448  I bought this Hazelnut Paste (Nocciola Spread)...  \n",
       "568449  This coffee supposedly is premium, it tastes w...  \n",
       "568450  This coffee supposedly is premium, it tastes w...  \n",
       "568451  The basket was the perfect sympathy gift when ...  \n",
       "568452  Purchased this product at a local store in NY ...  \n",
       "568453  I purchased this to send to my son who's away ...  \n",
       "\n",
       "[568454 rows x 10 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display= pd.read_sql_query(\"\"\"\n",
    "SELECT *\n",
    "FROM Reviews\n",
    "ORDER BY ProductID,UserId,Time\n",
    "\"\"\", con)\n",
    "display"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen above the same user has multiple reviews of the with the same values for HelpfulnessNumerator, HelpfulnessDenominator, Score, Time, Summary and Text  and on doing analysis it was found that <br>\n",
    "<br> \n",
    "ProductId=B000HDOPZG was Loacker Quadratini Vanilla Wafer Cookies, 8.82-Ounce Packages (Pack of 8)<br>\n",
    "<br> \n",
    "ProductId=B000HDL1RQ was Loacker Quadratini Lemon Wafer Cookies, 8.82-Ounce Packages (Pack of 8) and so on<br>\n",
    "\n",
    "It was inferred after analysis that reviews with same parameters other than ProductId belonged to the same product just having different flavour or quantity. Hence in order to reduce redundancy it was decided to eliminate the rows having same parameters.<br>\n",
    "\n",
    "The method used for the same was that we first sort the data according to ProductId and then just keep the first similar product review and delelte the others. for eg. in the above just the review for ProductId=B000HDL1RQ remains. This method ensures that there is only one representative for each product and deduplication without sorting would lead to possibility of different representatives still existing for the same product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Sorting data according to ProductId in ascending order\n",
    "sorted_data=filtered_data.sort_values('ProductId', axis=0, ascending=True, inplace=False, kind='quicksort', na_position='last')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(364173, 10)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Deduplication of entries\n",
    "final=sorted_data.drop_duplicates(subset={\"UserId\",\"ProfileName\",\"Time\",\"Text\"}, keep='first', inplace=False)\n",
    "final.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "69.25890143662969"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Checking to see how much % of data still remains\n",
    "(final['Id'].size*1.0)/(filtered_data['Id'].size*1.0)*100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Observation:-</b> It was also seen that in two rows given below the value of HelpfulnessNumerator is greater than HelpfulnessDenominator which is not practically possible hence these two rows too are removed from calcualtions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>64422</td>\n",
       "      <td>B000MIDROQ</td>\n",
       "      <td>A161DK06JJMCYF</td>\n",
       "      <td>J. E. Stephens \"Jeanne\"</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1224892800</td>\n",
       "      <td>Bought This for My Son at College</td>\n",
       "      <td>My son loves spaghetti so I didn't hesitate or...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>44737</td>\n",
       "      <td>B001EQ55RW</td>\n",
       "      <td>A2V0I904FH7ABY</td>\n",
       "      <td>Ram</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>1212883200</td>\n",
       "      <td>Pure cocoa taste with crunchy almonds inside</td>\n",
       "      <td>It was almost a 'love at first bite' - the per...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Id   ProductId          UserId              ProfileName  \\\n",
       "0  64422  B000MIDROQ  A161DK06JJMCYF  J. E. Stephens \"Jeanne\"   \n",
       "1  44737  B001EQ55RW  A2V0I904FH7ABY                      Ram   \n",
       "\n",
       "   HelpfulnessNumerator  HelpfulnessDenominator  Score        Time  \\\n",
       "0                     3                       1      5  1224892800   \n",
       "1                     3                       2      4  1212883200   \n",
       "\n",
       "                                        Summary  \\\n",
       "0             Bought This for My Son at College   \n",
       "1  Pure cocoa taste with crunchy almonds inside   \n",
       "\n",
       "                                                Text  \n",
       "0  My son loves spaghetti so I didn't hesitate or...  \n",
       "1  It was almost a 'love at first bite' - the per...  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display= pd.read_sql_query(\"\"\"\n",
    "SELECT *\n",
    "FROM Reviews\n",
    "WHERE Score != 3 AND Id=44737 OR Id=64422\n",
    "ORDER BY ProductID\n",
    "\"\"\", con)\n",
    "\n",
    "display.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "final=final[final.HelpfulnessNumerator<=final.HelpfulnessDenominator]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(364171, 10)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "positive    307061\n",
       "negative     57110\n",
       "Name: Score, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Before starting the next phase of preprocessing lets see the number of entries left\n",
    "print(final.shape)\n",
    "\n",
    "#How many positive and negative reviews are present in our dataset?\n",
    "final['Score'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 18.1.3  Text Preprocessing: Stemming, stop-word removal and Lemmatization.\n",
    "\n",
    "Now that we have finished deduplication our data requires some preprocessing before we go on further with analysis and making the prediction model.\n",
    "\n",
    "Hence in the Preprocessing phase we do the following in the order below:-\n",
    "\n",
    "1. Begin by removing the html tags\n",
    "2. Remove any punctuations or limited set of special characters like , or . or # etc.\n",
    "3. Check if the word is made up of english letters and is not alpha-numeric\n",
    "4. Check to see if the length of the word is greater than 2 (as it was researched that there is no adjective in 2-letters)\n",
    "5. Convert the word to lowercase\n",
    "6. Remove Stopwords\n",
    "7. Finally Snowball Stemming the word (it was obsereved to be better than Porter Stemming)<br>\n",
    "\n",
    "After which we collect the words used to describe positive and negative reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n",
      "I set aside at least an hour each day to read to my son (3 y/o). At this point, I consider myself a connoisseur of children's books and this is one of the best. Santa Clause put this under the tree. Since then, we've read it perpetually and he loves it.<br /><br />First, this book taught him the months of the year.<br /><br />Second, it's a pleasure to read. Well suited to 1.5 y/o old to 4+.<br /><br />Very few children's books are worth owning. Most should be borrowed from the library. This book, however, deserves a permanent spot on your shelf. Sendak's best.\n"
     ]
    }
   ],
   "source": [
    "# find sentences containing HTML tags\n",
    "import re\n",
    "i=0;\n",
    "for sent in final['Text'].values:\n",
    "    if (len(re.findall('<.*?>', sent))):\n",
    "        print(i)\n",
    "        print(sent)\n",
    "        break;\n",
    "    i += 1;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'just', 'didn', 'wasn', 'did', \"didn't\", 'into', 'shouldn', 're', 'that', 'over', 'our', \"should've\", 'will', \"hasn't\", \"that'll\", 'itself', 'mustn', 'her', 'because', 'whom', \"you've\", 'herself', 'some', 'of', 'its', 'we', 'once', 'out', 'mightn', 'not', 'y', 'can', 've', 'hasn', 'more', \"hadn't\", 'him', 'too', 'wouldn', 'those', 'themselves', 'is', \"isn't\", 'they', 'had', 'all', 'my', 'yourself', 'or', \"shouldn't\", 'this', 'which', 's', 'as', 'same', 'there', \"it's\", 'through', \"mustn't\", 'd', 'i', 'theirs', 'for', 'do', 'an', 'after', 'now', 'you', 'won', 'yours', 'about', 'above', 'between', 'has', 'ourselves', 'll', 'than', \"you're\", 'aren', 'am', 'only', 'under', 'doing', 'during', 'ours', 'yourselves', 'where', 'them', 'hers', 'on', \"shan't\", 'me', 'here', 'haven', 'himself', 'he', 'she', 'nor', 'before', 'why', 'other', 'these', 'his', 'are', 'was', 'have', 'most', 'don', 'in', 'from', 'again', 'who', 'ain', \"wasn't\", 'against', \"doesn't\", \"weren't\", 'each', \"aren't\", 'to', 'but', 'then', 'and', \"needn't\", \"she's\", 'own', 'isn', 'weren', \"won't\", 'what', 'a', 'below', 'hadn', 'myself', 'their', 'until', 'does', 'while', 'it', 'no', 'both', \"wouldn't\", 'if', \"haven't\", 'were', \"don't\", \"couldn't\", 'few', 'by', 'how', 'doesn', 'o', 'at', 'needn', 'couldn', 'shan', 'your', 'any', 'further', 'up', 'being', \"you'll\", \"you'd\", 'down', 'such', 'the', \"mightn't\", 'with', 'having', 'very', 'should', 't', 'ma', 'so', 'been', 'when', 'm', 'be', 'off'}\n",
      "************************************\n",
      "tasti\n"
     ]
    }
   ],
   "source": [
    "#nltk.download('stopwords')\n",
    "stop = set(stopwords.words('english')) #set of stopwords\n",
    "sno = nltk.stem.SnowballStemmer('english') #initialising the snowball stemmer\n",
    "\n",
    "def cleanhtml(sentence): #function to clean the word of any html-tags\n",
    "    cleanr = re.compile('<.*?>')\n",
    "    cleantext = re.sub(cleanr, ' ', sentence)\n",
    "    return cleantext\n",
    "def cleanpunc(sentence): #function to clean the word of any punctuation or special characters\n",
    "    cleaned = re.sub(r'[?|!|\\'|\"|#]',r'',sentence)\n",
    "    cleaned = re.sub(r'[.|,|)|(|\\|/]',r' ',cleaned)\n",
    "    return  cleaned\n",
    "print(stop)\n",
    "print('************************************')\n",
    "print(sno.stem('tasty'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#removing stop words and html tags\n",
    "#listing all words from potive and negative reviews\n",
    "\n",
    "i=0\n",
    "str1=' '\n",
    "final_string=[]\n",
    "all_positive_words=[] # store words from +ve reviews here\n",
    "all_negative_words=[] # store words from -ve reviews here.\n",
    "s=''\n",
    "for sent in final['Text'].values:\n",
    "    filtered_sentence=[]\n",
    "    #print(sent);\n",
    "    sent=cleanhtml(sent) # remove HTMl tags\n",
    "    for w in sent.split():\n",
    "        for cleaned_words in cleanpunc(w).split():\n",
    "            if((cleaned_words.isalpha()) & (len(cleaned_words)>2)):    \n",
    "                if(cleaned_words.lower() not in stop):\n",
    "                    s=(sno.stem(cleaned_words.lower())).encode('utf8')\n",
    "                    filtered_sentence.append(s)\n",
    "                    if (final['Score'].values)[i] == 'positive': \n",
    "                        all_positive_words.append(s) #list of all words used to describe positive reviews\n",
    "                    if(final['Score'].values)[i] == 'negative':\n",
    "                        all_negative_words.append(s) #list of all words used to describe negative reviews reviews\n",
    "                else:\n",
    "                    continue\n",
    "            else:\n",
    "                continue \n",
    "    #print(filtered_sentence)\n",
    "    str1 = b\" \".join(filtered_sentence) #final string of cleaned words\n",
    "    #print(\"***********************************************************************\")\n",
    "    \n",
    "    final_string.append(str1)\n",
    "    i+=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "final['CleanedText']=final_string #adding a column of CleanedText which displays the data after pre-processing of the review \n",
    "final['CleanedText']=final['CleanedText'].str.decode(\"utf-8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "final.head(3) #below the processed review can be seen in the CleanedText Column \n",
    "\n",
    "\n",
    "# store final table into an SQlLite table for future.\n",
    "conn = sqlite3.connect('final.sqlite')\n",
    "c=conn.cursor()\n",
    "conn.text_factory = str\n",
    "final.to_sql('Reviews', conn,  schema=None, if_exists='replace', index=True, index_label=None, chunksize=None, dtype=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "con = sqlite3.connect('final.sqlite') \n",
    "\n",
    "final = pd.read_sql_query(\"\"\" SELECT * FROM Reviews \"\"\", con) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of negative points and positive points are  2174 & 17826\n"
     ]
    }
   ],
   "source": [
    "#Sampling 20K Data-Points of 364K Data-Points\n",
    "\n",
    "final=final.sort_values('Time')\n",
    "df_sampled=final[:20000]\n",
    "\n",
    "print(\"The number of negative points and positive points are \",\n",
    "      df_sampled[df_sampled['Score']=='negative'].shape[0]\n",
    "     ,\"&\",df_sampled[df_sampled['Score']=='positive'].shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.1.4] Split Dataset into Train and Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14000, 12) (6000, 12)\n"
     ]
    }
   ],
   "source": [
    "# split the data set into train and test\n",
    "\n",
    "from sklearn import model_selection as ms\n",
    "y=df_sampled['Score']\n",
    "dw_train,dw_test,y_train,y_test=ms.train_test_split(df_sampled, y, test_size=0.3, random_state=0)\n",
    "print(dw_train.shape,dw_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [18.2] Bag of Words (BoW)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Build The Vocabulary\n",
    "#Vectorize train and test data\n",
    "count_vect = CountVectorizer() #in scikit-learn\n",
    "d_train=count_vect.fit_transform(dw_train['CleanedText'].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the type of count vectorizer  <class 'scipy.sparse.csr.csr_matrix'>\n",
      "the shape of train-text BOW vectorizer  (14000, 15632)\n",
      "the shape of test-text BOW vectorizer  (6000, 15632)\n"
     ]
    }
   ],
   "source": [
    "print(\"the type of count vectorizer \",type(d_train))\n",
    "print(\"the shape of train-text BOW vectorizer \",d_train.get_shape())\n",
    "\n",
    "d_test = count_vect.transform(dw_test['CleanedText'].values)\n",
    "print(\"the shape of test-text BOW vectorizer \",d_test.get_shape())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.2.1] Bag Of Words with Bi-Grams and n-Grams.\n",
    "\n",
    "**Motivation**\n",
    "\n",
    "Now that we have our list of words describing positive and negative reviews lets analyse them.<br>\n",
    "\n",
    "We begin analysis by getting the frequency distribution of the words as shown below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most Common Positive Words :  [(b'like', 139429), (b'tast', 129047), (b'good', 112766), (b'flavor', 109624), (b'love', 107357), (b'use', 103888), (b'great', 103870), (b'one', 96726), (b'product', 91033), (b'tri', 86791), (b'tea', 83888), (b'coffe', 78814), (b'make', 75107), (b'get', 72125), (b'food', 64802), (b'would', 55568), (b'time', 55264), (b'buy', 54198), (b'realli', 52715), (b'eat', 52004)]\n",
      "Most Common Negative Words :  [(b'tast', 34585), (b'like', 32330), (b'product', 28218), (b'one', 20569), (b'flavor', 19575), (b'would', 17972), (b'tri', 17753), (b'use', 15302), (b'good', 15041), (b'coffe', 14716), (b'get', 13786), (b'buy', 13752), (b'order', 12871), (b'food', 12754), (b'dont', 11877), (b'tea', 11665), (b'even', 11085), (b'box', 10844), (b'amazon', 10073), (b'make', 9840)]\n"
     ]
    }
   ],
   "source": [
    "freq_dist_positive=nltk.FreqDist(all_positive_words)\n",
    "freq_dist_negative=nltk.FreqDist(all_negative_words)\n",
    "print(\"Most Common Positive Words : \",freq_dist_positive.most_common(20))\n",
    "print(\"Most Common Negative Words : \",freq_dist_negative.most_common(20))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Observation:-</b> From the above it can be seen that the most common positive and the negative words overlap for eg. 'like' could be used as 'not like' etc. <br>\n",
    "So, it is a good idea to consider pairs of consequent words (bi-grams) or q sequnce of n consecutive words (n-grams)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#bi-gram, tri-gram and n-gram\n",
    "#building the vocabulary\n",
    "#removing stop words like \"not\" should be avoided before building n-grams\n",
    "count_vect_bg = CountVectorizer(ngram_range=(1,2))\n",
    "d_train_bg=count_vect_bg.fit_transform(dw_train['CleanedText'].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the type of count vectorizer  <class 'scipy.sparse.csr.csr_matrix'>\n",
      "the shape of train-text BOW vectorizer  (14000, 281647)\n",
      "the shape of test-text BOW vectorizer  (6000, 281647)\n"
     ]
    }
   ],
   "source": [
    "#Vectorize train and test data\n",
    "print(\"the type of count vectorizer \",type(d_train_bg))\n",
    "print(\"the shape of train-text BOW vectorizer \",d_train_bg.get_shape())\n",
    "\n",
    "d_test_bg = count_vect_bg.transform(dw_test['CleanedText'].values)\n",
    "print(\"the shape of test-text BOW vectorizer \",d_test_bg.get_shape())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.2.2] ConVert Sparce Matrix To Dense Using TruncatedSVD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "svd = TruncatedSVD(n_components=2000, random_state=0)\n",
    "#without using bigram\n",
    "d_train_dense=svd.fit_transform(d_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4.11878231e-02 3.67717226e-02 1.55181718e-02 ... 3.41136910e-05\n",
      " 3.40277144e-05 3.37737473e-05]\n",
      "[0.04118782 0.07795955 0.09347772 ... 0.95253415 0.95256818 0.95260195]\n"
     ]
    }
   ],
   "source": [
    "print (svd.explained_variance_ratio_)\n",
    "print (svd.explained_variance_ratio_.cumsum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Variance of 95% is retained with dimension equal to 2000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "d_test_dense=svd.transform(d_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "dv_train=d_train_bg[:20000]\n",
    "svd_m = TruncatedSVD(n_components=600, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.02564133 0.02293013 0.00983019 0.00894277 0.00766993 0.00716409\n",
      " 0.00644842 0.00573583 0.00547404 0.00532305 0.00488414 0.00485649\n",
      " 0.00462404 0.00444995 0.00424964 0.00412082 0.00399722 0.00390175\n",
      " 0.00384269 0.00381227 0.00380386 0.00367986 0.00360575 0.00347496\n",
      " 0.00333333 0.00323307 0.00313258 0.00310785 0.00303052 0.00292175\n",
      " 0.0028494  0.00278125 0.00271867 0.00265883 0.0026426  0.002597\n",
      " 0.00253491 0.00250211 0.00249323 0.00240082 0.0023847  0.00233707\n",
      " 0.00227341 0.00223848 0.00216391 0.00212977 0.00207618 0.00205909\n",
      " 0.00203872 0.00201981 0.00198213 0.00196083 0.00192426 0.00192047\n",
      " 0.00189897 0.00187733 0.00186874 0.00181756 0.00179617 0.00179322\n",
      " 0.00176946 0.00174563 0.00173096 0.00171551 0.00169917 0.00166985\n",
      " 0.00164035 0.00163066 0.00159706 0.00158341 0.00157973 0.00156238\n",
      " 0.00153813 0.00153115 0.00150045 0.00148555 0.00147451 0.00147081\n",
      " 0.00145211 0.00144532 0.00143701 0.00143012 0.00140463 0.00139415\n",
      " 0.00138985 0.00137259 0.00134319 0.001338   0.00133026 0.00131956\n",
      " 0.00130866 0.00129629 0.00128946 0.00127562 0.00127048 0.00126294\n",
      " 0.00125049 0.00124474 0.00123984 0.00122649 0.00121315 0.00120386\n",
      " 0.00118978 0.00118646 0.0011815  0.00116941 0.00115876 0.00114586\n",
      " 0.00114064 0.00113312 0.00112633 0.00112082 0.00111132 0.001103\n",
      " 0.00109779 0.00109093 0.00107966 0.00106966 0.0010633  0.00105975\n",
      " 0.00105566 0.00104329 0.00103358 0.00103164 0.00102645 0.00101764\n",
      " 0.00101305 0.00100785 0.00100094 0.00099996 0.00099249 0.00098805\n",
      " 0.00098051 0.0009732  0.00096901 0.0009654  0.00095924 0.00095351\n",
      " 0.00094623 0.00094455 0.00093788 0.00093424 0.00092858 0.00092727\n",
      " 0.00091685 0.0009108  0.00090949 0.00090555 0.00089988 0.0008946\n",
      " 0.00089038 0.00088797 0.00088223 0.00087935 0.000875   0.00087414\n",
      " 0.00087298 0.00086199 0.00086189 0.00085426 0.000852   0.0008482\n",
      " 0.00084417 0.00084    0.00083614 0.00083129 0.00082782 0.00082279\n",
      " 0.00082061 0.00081651 0.00081368 0.00080892 0.00080296 0.00080032\n",
      " 0.0007929  0.0007909  0.00078435 0.00078326 0.0007773  0.00077489\n",
      " 0.00076905 0.00076679 0.00076368 0.0007569  0.00075501 0.00075314\n",
      " 0.00074958 0.00074781 0.00074179 0.00073971 0.00073787 0.00073367\n",
      " 0.0007333  0.00073036 0.00072366 0.00072134 0.00071812 0.0007159\n",
      " 0.00071085 0.00070808 0.00070601 0.0007033  0.00070142 0.00069859\n",
      " 0.00069507 0.00069353 0.00069294 0.00069129 0.00068378 0.00068251\n",
      " 0.00067974 0.00067564 0.00067192 0.00066922 0.00066818 0.00066491\n",
      " 0.00066233 0.00066021 0.00065729 0.00065016 0.00064937 0.00064841\n",
      " 0.00064585 0.00064428 0.00064201 0.00063937 0.00063774 0.00063369\n",
      " 0.00063022 0.00062786 0.00062629 0.0006235  0.00062159 0.00061911\n",
      " 0.00061858 0.00061522 0.00061458 0.00061084 0.00060748 0.0006057\n",
      " 0.00060177 0.00060116 0.00059886 0.00059637 0.00059422 0.00059208\n",
      " 0.00058948 0.00058783 0.00058631 0.00058398 0.00058242 0.00058029\n",
      " 0.00057723 0.00057537 0.00057302 0.00057085 0.00056946 0.00056825\n",
      " 0.0005671  0.00056636 0.00056451 0.00056116 0.00055952 0.00055771\n",
      " 0.00055725 0.00055375 0.00055272 0.00055114 0.00054894 0.00054654\n",
      " 0.00054392 0.00054228 0.00053977 0.00053862 0.00053772 0.00053305\n",
      " 0.00053278 0.00052981 0.00052904 0.00052791 0.00052541 0.00052447\n",
      " 0.00052254 0.0005214  0.00051807 0.00051723 0.00051598 0.00051471\n",
      " 0.0005133  0.00051147 0.00050906 0.0005089  0.00050779 0.00050717\n",
      " 0.00050551 0.00050205 0.00050073 0.00050034 0.00049955 0.00049844\n",
      " 0.00049653 0.00049425 0.00049115 0.00049081 0.00049042 0.0004878\n",
      " 0.00048685 0.00048597 0.00048364 0.00048201 0.00048156 0.0004805\n",
      " 0.00047731 0.0004762  0.00047517 0.00047392 0.00047234 0.00047147\n",
      " 0.00046936 0.00046821 0.00046777 0.00046563 0.00046517 0.00046415\n",
      " 0.0004624  0.00046123 0.00045871 0.00045669 0.00045639 0.00045574\n",
      " 0.00045475 0.00045381 0.00045297 0.00045113 0.00044975 0.00044839\n",
      " 0.00044695 0.00044613 0.00044416 0.00044337 0.00044207 0.00044031\n",
      " 0.00043962 0.00043771 0.00043641 0.00043513 0.0004346  0.00043288\n",
      " 0.0004325  0.00043122 0.00043038 0.00042906 0.00042834 0.00042477\n",
      " 0.00042424 0.00042347 0.00042237 0.00042181 0.00041972 0.00041813\n",
      " 0.00041807 0.00041751 0.0004171  0.00041632 0.00041424 0.00041346\n",
      " 0.00041143 0.00041096 0.00041037 0.00040916 0.00040862 0.00040627\n",
      " 0.00040568 0.00040527 0.00040343 0.00040206 0.00040184 0.00040039\n",
      " 0.00039939 0.00039742 0.00039707 0.00039582 0.00039544 0.00039435\n",
      " 0.00039264 0.00039194 0.00039197 0.00039039 0.00038868 0.00038839\n",
      " 0.00038718 0.00038554 0.0003846  0.0003833  0.0003824  0.00038067\n",
      " 0.00038026 0.00037929 0.00037824 0.00037763 0.00037743 0.00037665\n",
      " 0.00037599 0.00037359 0.00037294 0.00037156 0.00037124 0.00037037\n",
      " 0.00036903 0.00036889 0.00036778 0.00036616 0.00036575 0.00036436\n",
      " 0.00036349 0.00036217 0.00036143 0.00036073 0.0003591  0.00035879\n",
      " 0.00035799 0.00035765 0.00035661 0.00035461 0.00035402 0.00035369\n",
      " 0.00035241 0.00035151 0.00035089 0.0003501  0.00034996 0.0003495\n",
      " 0.00034734 0.00034657 0.00034637 0.00034534 0.00034386 0.00034332\n",
      " 0.00034236 0.00034097 0.00034024 0.00033994 0.00033972 0.00033913\n",
      " 0.00033815 0.00033758 0.000336   0.00033571 0.00033431 0.00033396\n",
      " 0.00033325 0.00033263 0.00033166 0.0003305  0.00032986 0.00032913\n",
      " 0.00032881 0.00032758 0.00032668 0.00032637 0.00032511 0.00032469\n",
      " 0.00032323 0.00032199 0.00032135 0.00032091 0.00032069 0.00032005\n",
      " 0.00031889 0.00031826 0.00031793 0.00031719 0.00031667 0.00031585\n",
      " 0.00031544 0.00031471 0.00031413 0.00031356 0.00031213 0.00031142\n",
      " 0.00031039 0.00031007 0.0003089  0.0003081  0.00030755 0.00030721\n",
      " 0.00030565 0.00030533 0.0003044  0.00030398 0.00030366 0.00030245\n",
      " 0.00030193 0.00030156 0.00030036 0.00029995 0.0002992  0.00029853\n",
      " 0.00029737 0.00029682 0.00029631 0.00029528 0.00029516 0.00029467\n",
      " 0.00029388 0.00029309 0.00029232 0.00029185 0.00029036 0.00028958\n",
      " 0.00028921 0.0002886  0.00028746 0.00028668 0.00028646 0.00028642\n",
      " 0.0002855  0.00028511 0.00028404 0.00028368 0.00028342 0.00028228\n",
      " 0.00028133 0.00028118 0.00028018 0.0002791  0.00027817 0.00027723\n",
      " 0.00027701 0.00027628 0.00027516 0.00027496 0.0002743  0.00027369\n",
      " 0.00027287 0.00027185 0.00027026 0.00027028 0.00026991 0.00026897\n",
      " 0.00026836 0.00026811 0.00026698 0.00026645 0.00026611 0.00026559\n",
      " 0.00026441 0.00026424 0.00026347 0.00026253 0.00026231 0.0002615\n",
      " 0.00026117 0.0002603  0.00025988 0.00025914 0.00025888 0.00025778\n",
      " 0.00025673 0.00025665 0.0002554  0.00025512 0.00025362 0.00025334\n",
      " 0.00025317 0.00025296 0.00025206 0.00025129 0.00025069 0.00024978\n",
      " 0.00024907 0.0002488  0.00024853 0.0002478  0.00024717 0.0002459\n",
      " 0.00024569 0.00024518 0.00024451 0.000243   0.00024257 0.00024171\n",
      " 0.00024134 0.00024062 0.00024016 0.00023956 0.00023934 0.00023827\n",
      " 0.00023633 0.00023581 0.00023506 0.00023449 0.00023389 0.00023314\n",
      " 0.00023196 0.00023119 0.00023048 0.00022902 0.00022821 0.00022791]\n",
      "[0.02564133 0.04857146 0.05840164 0.06734442 0.07501434 0.08217843\n",
      " 0.08862684 0.09436268 0.09983672 0.10515977 0.11004391 0.1149004\n",
      " 0.11952444 0.12397438 0.12822402 0.13234484 0.13634206 0.14024381\n",
      " 0.1440865  0.14789877 0.15170263 0.15538249 0.15898825 0.16246321\n",
      " 0.16579654 0.16902961 0.17216219 0.17527004 0.17830056 0.18122232\n",
      " 0.18407172 0.18685297 0.18957163 0.19223046 0.19487307 0.19747007\n",
      " 0.20000498 0.20250709 0.20500032 0.20740114 0.20978584 0.21212292\n",
      " 0.21439632 0.2166348  0.21879871 0.22092848 0.22300466 0.22506375\n",
      " 0.22710248 0.22912229 0.23110441 0.23306524 0.2349895  0.23690997\n",
      " 0.23880895 0.24068628 0.24255502 0.24437258 0.24616875 0.24796196\n",
      " 0.24973142 0.25147705 0.25320802 0.25492353 0.25662269 0.25829255\n",
      " 0.25993289 0.26156355 0.26316061 0.26474402 0.26632376 0.26788614\n",
      " 0.26942427 0.27095542 0.27245587 0.27394142 0.27541593 0.27688674\n",
      " 0.27833885 0.27978418 0.28122119 0.28265131 0.28405594 0.28545009\n",
      " 0.28683994 0.28821253 0.28955572 0.29089372 0.29222398 0.29354354\n",
      " 0.2948522  0.29614849 0.29743795 0.29871357 0.29998405 0.30124699\n",
      " 0.30249748 0.30374222 0.30498206 0.30620855 0.3074217  0.30862556\n",
      " 0.30981535 0.3110018  0.3121833  0.31335271 0.31451147 0.31565733\n",
      " 0.31679798 0.31793109 0.31905742 0.32017825 0.32128956 0.32239257\n",
      " 0.32349036 0.32458129 0.32566095 0.32673061 0.32779391 0.32885366\n",
      " 0.32990933 0.33095262 0.3319862  0.33301784 0.3340443  0.33506194\n",
      " 0.33607499 0.33708283 0.33808377 0.33908373 0.34007622 0.34106426\n",
      " 0.34204478 0.34301798 0.34398699 0.34495239 0.34591163 0.34686514\n",
      " 0.34781137 0.34875592 0.3496938  0.35062804 0.35155662 0.35248389\n",
      " 0.35340074 0.35431154 0.35522103 0.35612658 0.35702646 0.35792106\n",
      " 0.35881144 0.35969942 0.36058164 0.36146099 0.362336   0.36321014\n",
      " 0.36408312 0.36494511 0.36580701 0.36666127 0.36751327 0.36836148\n",
      " 0.36920565 0.37004565 0.37088179 0.37171308 0.37254089 0.37336369\n",
      " 0.37418429 0.37500081 0.37581449 0.37662341 0.37742637 0.37822669\n",
      " 0.37901959 0.37981048 0.38059484 0.3813781  0.3821554  0.38293028\n",
      " 0.38369933 0.38446612 0.3852298  0.3859867  0.38674171 0.38749485\n",
      " 0.38824443 0.38899224 0.38973403 0.39047374 0.39121161 0.39194528\n",
      " 0.39267858 0.39340894 0.3941326  0.39485394 0.39557206 0.39628796\n",
      " 0.39699881 0.3977069  0.39841291 0.39911621 0.39981762 0.40051621\n",
      " 0.40121129 0.40190481 0.40259775 0.40328905 0.40397283 0.40465534\n",
      " 0.40533508 0.40601072 0.40668263 0.40735185 0.40802004 0.40868495\n",
      " 0.40934728 0.41000749 0.41066477 0.41131493 0.41196431 0.41261271\n",
      " 0.41325856 0.41390285 0.41454485 0.41518423 0.41582196 0.41645565\n",
      " 0.41708587 0.41771373 0.41834002 0.41896352 0.41958511 0.42020421\n",
      " 0.4208228  0.42143801 0.42205259 0.42266343 0.42327091 0.42387661\n",
      " 0.42447837 0.42507953 0.42567839 0.42627476 0.42686898 0.42746106\n",
      " 0.42805054 0.42863837 0.42922469 0.42980867 0.43039109 0.43097138\n",
      " 0.4315486  0.43212397 0.432697   0.43326785 0.43383731 0.43440556\n",
      " 0.43497265 0.43553901 0.43610353 0.43666469 0.43722422 0.43778192\n",
      " 0.43833917 0.43889292 0.43944564 0.43999678 0.44054572 0.44109226\n",
      " 0.44163618 0.44217846 0.44271823 0.44325685 0.44379457 0.44432762\n",
      " 0.4448604  0.44539022 0.44591926 0.44644717 0.44697258 0.44749705\n",
      " 0.44801959 0.44854099 0.44905906 0.44957629 0.45009227 0.45060698\n",
      " 0.45112028 0.45163174 0.4521408  0.4526497  0.45315749 0.45366466\n",
      " 0.45417017 0.45467222 0.45517295 0.45567329 0.45617285 0.45667128\n",
      " 0.45716782 0.45766207 0.45815322 0.45864403 0.45913445 0.45962226\n",
      " 0.46010911 0.46059508 0.46107873 0.46156074 0.4620423  0.4625228\n",
      " 0.46300011 0.46347631 0.46395148 0.4644254  0.46489774 0.46536921\n",
      " 0.46583858 0.46630679 0.46677456 0.46724019 0.46770536 0.46816951\n",
      " 0.46863191 0.46909315 0.46955185 0.47000854 0.47046493 0.47092067\n",
      " 0.47137542 0.47182923 0.47228219 0.47273332 0.47318308 0.47363146\n",
      " 0.47407841 0.47452454 0.4749687  0.47541207 0.47585414 0.47629445\n",
      " 0.47673407 0.47717179 0.4776082  0.47804332 0.47847793 0.4789108\n",
      " 0.4793433  0.47977452 0.4802049  0.48063396 0.48106231 0.48148707\n",
      " 0.48191131 0.48233479 0.48275715 0.48317896 0.48359868 0.48401681\n",
      " 0.48443487 0.48485238 0.48526948 0.4856858  0.48610004 0.4865135\n",
      " 0.48692493 0.48733589 0.48774626 0.48815542 0.48856404 0.48897031\n",
      " 0.489376   0.48978126 0.4901847  0.49058676 0.4909886  0.49138899\n",
      " 0.49178838 0.4921858  0.49258288 0.4929787  0.49337413 0.49376848\n",
      " 0.49416112 0.49455306 0.49494502 0.49533541 0.49572409 0.49611248\n",
      " 0.49649966 0.4968852  0.4972698  0.49765311 0.4980355  0.49841617\n",
      " 0.49879643 0.49917572 0.49955396 0.49993159 0.50030902 0.50068568\n",
      " 0.50106167 0.50143525 0.50180819 0.50217976 0.502551   0.50292137\n",
      " 0.50329039 0.50365928 0.50402706 0.50439322 0.50475897 0.50512333\n",
      " 0.50548682 0.50584899 0.50621042 0.50657115 0.50693025 0.50728904\n",
      " 0.50764702 0.50800468 0.50836128 0.50871589 0.50906991 0.5094236\n",
      " 0.50977601 0.51012752 0.51047841 0.51082851 0.51117847 0.51152797\n",
      " 0.51187531 0.51222188 0.51256825 0.51291359 0.51325745 0.51360077\n",
      " 0.51394313 0.5142841  0.51462434 0.51496428 0.515304   0.51564313\n",
      " 0.51598128 0.51631886 0.51665487 0.51699057 0.51732488 0.51765884\n",
      " 0.51799209 0.51832472 0.51865638 0.51898688 0.51931674 0.51964587\n",
      " 0.51997468 0.52030226 0.52062894 0.52095531 0.52128042 0.52160511\n",
      " 0.52192834 0.52225033 0.52257168 0.52289259 0.52321328 0.52353332\n",
      " 0.52385221 0.52417047 0.5244884  0.5248056  0.52512227 0.52543812\n",
      " 0.52575355 0.52606826 0.52638239 0.52669595 0.52700808 0.5273195\n",
      " 0.52762989 0.52793996 0.52824885 0.52855696 0.52886451 0.52917172\n",
      " 0.52947737 0.5297827  0.5300871  0.53039108 0.53069473 0.53099718\n",
      " 0.53129911 0.53160067 0.53190102 0.53220097 0.53250017 0.5327987\n",
      " 0.53309607 0.5333929  0.53368921 0.53398449 0.53427964 0.53457431\n",
      " 0.53486819 0.53516128 0.53545359 0.53574544 0.5360358  0.53632538\n",
      " 0.5366146  0.5369032  0.53719065 0.53747733 0.5377638  0.53805022\n",
      " 0.53833572 0.53862083 0.53890487 0.53918855 0.53947197 0.53975425\n",
      " 0.54003558 0.54031676 0.54059694 0.54087604 0.54115421 0.54143144\n",
      " 0.54170845 0.54198473 0.54225989 0.54253484 0.54280915 0.54308283\n",
      " 0.5433557  0.54362755 0.54389782 0.5441681  0.54443801 0.54470698\n",
      " 0.54497534 0.54524345 0.54551043 0.54577688 0.54604299 0.54630859\n",
      " 0.546573   0.54683724 0.54710071 0.54736324 0.54762554 0.54788704\n",
      " 0.54814821 0.54840851 0.54866839 0.54892753 0.54918641 0.54944419\n",
      " 0.54970091 0.54995757 0.55021297 0.55046809 0.55072171 0.55097506\n",
      " 0.55122823 0.55148119 0.55173325 0.55198454 0.55223523 0.55248501\n",
      " 0.55273408 0.55298288 0.5532314  0.5534792  0.55372638 0.55397228\n",
      " 0.55421797 0.55446316 0.55470767 0.55495067 0.55519324 0.55543495\n",
      " 0.5556763  0.55591691 0.55615707 0.55639663 0.55663597 0.55687423\n",
      " 0.55711057 0.55734638 0.55758144 0.55781593 0.55804982 0.55828296\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0.55851492 0.55874612 0.55897659 0.55920561 0.55943382 0.55966174]\n"
     ]
    }
   ],
   "source": [
    "#using bigrams\n",
    "d_train_bg_dense=svd_m.fit_transform(dv_train)\n",
    "print (svd_m.explained_variance_ratio_)\n",
    "print (svd_m.explained_variance_ratio_.cumsum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "d_test_bg_dense=svd_m.transform(d_test_bg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.2.3] KNN (Brute-Force)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 11.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAEKCAYAAAB+AXB+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8lNX1+PHPmS3JhCSTkADZgISwr8oOFay4oCDgjnWt1i7Wn61LK9jab7Xaalu1WnFrXagLorVU3EBBUSuyiRFEVhEIhDVkIWRPzu+PmYxJSMKAhEzkvF+veTHzPM+9z5lEON7luVdUFWOMMea7wtHaARhjjDHHkiU2Y4wx3ymW2IwxxnynWGIzxhjznWKJzRhjzHeKJTZjjDHfKZbYjDHGfKdYYjPGGPOdYonNGGPMd4qrtQM4HhITE7Vr166tHYYxxrQpn3766T5VTWrtOI7UCZHYunbtyooVK1o7DGOMaVNEZGtrx3A0rCvSGGPMd4olNmOMMd8plthCUFpaytixY6murgZg/Pjx+Hw+Jk6cGFL58vJyLrnkErKyshg+fDhbtmxp9LprrrmGDh060K9fv3rH77jjDgYMGMCgQYM488wzyc3NBaCwsJBzzz2XgQMH0rdvX5555hkA9u7dy/jx44/y2xpjTNtmiS0ETz/9NOeffz5OpxOAX/3qVzz33HMhl3/qqaeIj49n06ZN3HTTTdx2222NXnf11Vczb968Q47/6le/YtWqVWRnZzNx4kTuuusuAGbMmEGfPn34/PPPWbRoEbfccgsVFRUkJSWRnJzMxx9/fBTf1hhj2jZLbCF44YUXmDx5cvDzuHHjiImJCbn8a6+9xlVXXQXAhRdeyMKFC2lsH7wxY8aQkJBwyPHY2Njg+4MHDyIiAIgIBw4cQFUpLi4mISEBl8s/H2jKlCm88MILIcdojDHfFSfErMhvo6Kigs2bN/NtHhfYsWMH6enpALhcLuLi4sjLyyMxMTHkOn7zm9/wr3/9i7i4ON5//30AbrjhBiZNmkRKSgoHDhxg9uzZOBz+/1cZMmQIv/3tb486ZmOMaausxXYY+/btw+fzfas6Gmud1ba6QnXPPfeQk5PDZZddxiOPPALA/PnzGTRoELm5uWRnZ3PDDTdQVFQEQIcOHYJjccYYcyKxxNaMOZ9t5/Uv9lFWVvat6klLSyMnJweAqqoqCgsLG+1yDMUPfvADXn31VQCeeeYZzj//fESErKwsMjIyWLduHQBlZWVERUV9q7iNMaYtssTWjDdX7WLOlwVUV1eHlNymT5/OnDlzDjk+adIkZs6cCcC///1vTjvttCNqsW3cuDH4fu7cufTq1QuAzp07s3DhQgB2797N+vXryczMBGDDhg2HzK40xpgTgSW2ZsR73RSWVnLmmWfyv//9L3j8lFNO4aKLLmLhwoWkpaUxf/58AFavXk2nTp0Oqefaa68lLy+PrKwsHnjgAe69914AcnNzOeecc4LXXXrppYwcOZL169eTlpbGU089BcC0adPo168fAwYM4J133uGhhx4C/I8BLF68mP79+zNu3Djuu+++4Ljd+++/z4QJE1rmB2OMMWFMGhv/+a4ZMmSIHs2SWve8+SXPLdnKrPM78cADDxx2iv9ZZ50VTHKtbcyYMbz22mvEx8e3dijGmDZKRD5V1SGtHceRshZbM3xeD2WVNfTuN4Dvf//7wQe0mxIuSW3v3r3cfPPNltSMMSckm+7fDJ/XDUBBSSXXXHNNK0cTuqSkJKZMmdLaYRhjTKuwFlsz4r0eAPJLKlo5EmOMMaGyxNaMui02Y4wxbYMltmb4ovwttgJrsRljTJthia0Z8dH+Flu+tdiMMabNsMTWjNoxtoJSa7EZY0xbYYmtGZFuJxEuh42xGWNMG2KJ7TDivR7yD1qLzRhj2gpLbIfh87opKLUWmzHGtBWW2A7D53XbrEhjjGlDLLEdRrzXY7MijTGmDbHEdhg+r8cmjxhjTBvSoolNRMaLyHoR2SQi0xo5P0ZEVopIlYhcWOd4FxH5VESyRWSNiPy0zrlFgTqzA68OLfkdarsiT4RdEIwx5rugxRZBFhEnMAM4A9gOLBeRuar6ZZ3LtgFXA7c2KL4TGKWq5SLSDvgiUDY3cP4yVT3yfWiOQrzXTVWNUlxeRUyk+3jc0hhjzLfQki22YcAmVd2sqhXAS8Dkuheo6hZVXQXUNDheoarlgY8RLRxns3y1D2lbd6QxxrQJLZkwUoGcOp+3B46FRETSRWRVoI776rTWAJ4JdEPeISJybMJtnC/KFkI2xpi2pCUTW2MJJ+SBKlXNUdUBQBZwlYh0DJy6TFX7A6cEXlc0enORH4vIChFZsXfv3iMM/Rvx0bZ1jTHGtCUtmdi2A+l1PqcBuU1c26RAS20N/iSGqu4I/HkAeBF/l2dj5Z5U1SGqOiQpKelIbxsUX7t1jT2kbYwxbUJLJrblQHcRyRARDzAVmBtKQRFJE5GowPt4YDSwXkRcIpIYOO4GJgJftEj0AXG2dY0xxrQpLZbYVLUKuAGYD6wFXlbVNSJyl4hMAhCRoSKyHbgIeEJE1gSK9waWisjnwAfAX1V1Nf6JJPMDY2/ZwA7gHy31HeCbzUbzD1qLzRhj2oIWm+4PoKpvAW81OPa7Ou+X4++ibFjuXWBAI8cPAoOPfaRNczsdxES4bOsaY4xpI2zlkRDEed02K9IYY9oIS2wh8K8XaS02Y4xpCyyxhcBnLTZjjGkzLLGFwL8QsrXYjDGmLbDEFoJ4r9u2rjHGmDbCElsIfFFuisoqqa6xFf6NMSbcWWILgc/rQRWKbPURY4wJe5bYQhAfbctqGWNMW2GJLQS+KFsI2Rhj2gpLbCGoXVbLZkYaY0z4s8QWgnjbbNQYY9oMS2whCC6EbInNGGPCniW2EMRGunGIdUUaY0xbYIktBA6HEBdly2oZY0xbYIktRD5bCNkYY9oES2whsoWQjTGmbbDEFqJ4r8c2GzXGmDbAEluIfFFu8g9ai80YY8KdJbYQ2dY1xhjTNlhiC1G8183BimoqqmpaOxRjjDHNaDaxiYhTRP5yvIIJZ8FltWyczRhjwlqziU1Vq4HBIiLHKZ6w5bNltYwxpk0IpSvyM+A1EblCRM6vfYVSuYiMF5H1IrJJRKY1cn6MiKwUkSoRubDO8S4i8qmIZIvIGhH5aZ1zg0VkdaDOh49X0rX1Io0xpm1whXBNApAHnFbnmAL/aa6QiDiBGcAZwHZguYjMVdUv61y2DbgauLVB8Z3AKFUtF5F2wBeBsrnAY8CPgSXAW8B44O0Qvse38s16kdYVaYwx4eywiU1Vf3iUdQ8DNqnqZgAReQmYDAQTm6puCZyrNyNDVetmjwgCLUsRSQZiVfWTwOd/AVM4jonNZkYaY0x4O2xXpIikicgcEdkjIrtF5FURSQuh7lQgp87n7YFjIRGRdBFZFajjvkBrLTVQz1HV+W1YV6QxxrQNoYyxPQPMBVLwJ5HXA8cOp7GxLw01MFXNUdUBQBZwlYh0PJI6ReTHIrJCRFbs3bs31Ns2yetx4naKbV1jjDFhLpTElqSqz6hqVeD1LJAUQrntQHqdz2lA7pEGGGiprQFOCdRZt7XYZJ2q+qSqDlHVIUlJoYTbPBGxh7SNMaYNCCWx7RORywPPtDlF5HL8k0kOZznQXUQyRMQDTMXf8jusQPdnVOB9PDAaWK+qO4EDIjIiMBvySuC1UOo8Fny2dY0xxoS9UBLbNcDFwC78sxUvDBxrlqpWATcA84G1wMuqukZE7hKRSQAiMlREtgMXAU+IyJpA8d7AUhH5HPgA+Kuqrg6c+xnwT2AT8BXHYeJIrXjbusYYY8Jes7MiA1P2L1DVSUdTuaq+hX9Kft1jv6vzfjn1uxZrj78LDGiizhVAv6OJ59vyed1szStpjVsbY4wJUSgrj0w+TrGEPZ/XbUtqGWNMmAvlAe2PReQRYDZwsPagqq5ssajClL8rshJVxVYZM8aY8BRKYhsV+POuOseU+iuRnBB8Xg8VVTWUVlbj9YTyozPGGHO8HW6MzQE8pqovH6d4wto3q49UWmIzxpgwdbgxthr8MxsN/j3ZwNaLNMaYcBbKdP93ReTWwBJXCbWvFo8sDNVuXVNoz7IZY0zYCqU/rfaZtZ/XOaZA5rEPJ7x9s8K/JTZjjAlXoazun3E8AmkLahdCtq5IY4wJX012RYrIr+u8v6jBuT+2ZFDhKi7K32IrLLUWmzHGhKvmxtim1nk/vcG58S0QS9iLdDuJcjvJP2gtNmOMCVfNJTZp4n1jn08Y8V63jbEZY0wYay6xaRPvG/t8wvB5PRTaslrGGBO2mps8MlBEivC3zqIC7wl8jmzxyMKUz1psxhgT1ppMbKrqPJ6BtBXxXg9rdxUd/kJjjDGtIpQHtE0dPq/bHtA2xpgwZontCPm3rvGv8G+MMSb8WGI7QvFeD9U1SlFZVWuHYowxphGW2I6QrRdpjDHh7bCJTUTOF5GNIlIoIkUicqDODMkTji/KVvg3xphwFsoiyH8GzlXVtS0dTFsQH22JzRhjwlkoXZG7Lal9I9gVaetFGmNMWAqlxbZCRGYD/wXKaw+q6n9aLKowFuyKtPUijTEmLIWS2GKBEuDMOscUOCETW1yU7clmjDHhLJT92H54tJWLyHjgIcAJ/FNV721wfgzwN2AAMFVV/x04Pgh4DH9SrQbuUdXZgXPPAmOBwkA1V6tq9tHGeKRcTgcxkS7rijTGmDAVyqzINBGZIyJ7RGS3iLwqImkhlHMCM4CzgT7ApSLSp8Fl24CrgRcbHC8BrlTVvvi3yPmbiPjqnP+Vqg4KvI5bUqsV7/XY5BFjjAlToUweeQaYC6QAqcDrgWOHMwzYpKqbVbUCeAmYXPcCVd2iqquAmgbHN6jqxsD7XGAPkBTCPY8L27rGGGPCVyiJLUlVn1HVqsDrWUJLMqlATp3P2wPHjoiIDAM8wFd1Dt8jIqtE5EERiWii3I9FZIWIrNi7d++R3rZZcV4PhdZiM8aYsBRKYtsnIpeLiDPwuhzIC6FcY5uRHtECiyKSDDwH/FBVa1t104FewFAgAbitsbKq+qSqDlHVIUlJx7axZy02Y4wJX6EktmuAi4FdwE7gwsCxw9kOpNf5nAbkhhqYiMQCbwK/VdUltcdVdaf6lePvEh0Wap3Hio2xGWNM+AplVuQ2YNJR1L0c6C4iGcAOYCrwg1AKiogHmAP8S1VfaXAuWVV3iogAU4AvjiK2byUuys2BsiqqqmtwOW25TWOMCSdNJjYR+bWq/llE/k4jXYiqemNzFatqlYjcAMzHP93/aVVdIyJ3AStUda6IDMWfwOKBc0XkzsBMyIuBMUB7Ebk6UGXttP4XRCQJf1dnNvDTI/zO31q81/8sW2FpJe3bNTrEZ4wxppU012KrXUZrxdFWrqpvAW81OPa7Ou+X4++ibFjueeD5Juo87WjjOVbio/3LahVYYjPGmLDTZGJT1dcDb0sa6Q68qEWjCnO1q48U2DibMcaEnVAGiKaHeOyEER9YCDn/oM2MNMaYcNPcGNvZwDlAqog8XOdULHBCbx9dm9gKbFktY4wJO82NseXiH1+bBHxa5/gB4KaWDCrcxXmtK9IYY8JVc2NsnwOfi8iLqmpNkzpiI104HWLPshljTBgKZduariLyJ/wLGUfWHlTVzBaLKsyJCL4oNwW2+ogxxoSdUBdBfgz/uNr3gX/hX+bqhBbntcRmjDHhKJTEFqWqCwFR1a2q+nug1Z8la222rJYxxoSnULoiy0TEAWwMrCSyA+jQsmGFv3ivm9yCstYOwxhjTAOhtNh+CXiBG4HBwOXAVS0ZVFsQF+WxWZHGGBOGQlkEeXngbTHww5YNp+2wrWuMMSY8HbbFJiLvioivzud4EZnfsmGFv/hoD6WV1ZRVVrd2KMYYY+oIpSsyUVULaj+oaj42xhZcL7LQVh8xxpiwEkpiqxGRzrUfRKQLR7gT9ndRcL1IG2czxpiwEsqsyN8A/xORDwKfxwA/brmQ2gZfcFkta7EZY0w4CWXyyDwRORkYgX9zz5tUdV+LRxbmfLZepDHGhKUmuyJFpFfgz5OBzvgXRd4BdA4cO6F90xVpLTZjjAknzbXYbsbf5Xh/I+eUE3z1EeuKNMaY8NRcYns38Oe1qrr5eATTlkS5nXhcDuuKNMaYMNPcrMjaXbL/fTwCaWtEJPCQtiU2Y4wJJ8212PJE5H0gQ0TmNjypqpNaLqy2wRflsa5IY4wJM80ltgnAyfi3qGlsnO2E57Ota4wxJuw02RWpqhWqugQYpaofNHyFUrmIjBeR9SKySUSmNXJ+jIisFJEqEbmwzvFBIvKJiKwRkVUickmdcxkislRENorIbBHxHOF3PmZs6xpjjAk/zU33/1vg7dMiMrfh63AVi4gTmAGcjX/37UtFpE+Dy7YBVwMvNjheAlypqn2B8cDf6qxXeR/woKp2B/KBaw8XS0vxed0U2JJaxhgTVprriqzdJfuvR1n3MGBT7YxKEXkJmAx8WXuBqm4JnKupW1BVN9R5nysie4AkESnE/5jBDwKnZwK/x7/D93Hn8/q3rlFVRKQ1QjDGGNNAk4lNVT8N/BnsdhSReCBdVVeFUHcqkFPn83Zg+JEGKCLDAA/wFdAeKFDVqjp1ph5pncdKvNdNZbVSUlFNdEQoq5MZY4xpaaFsW7NIRGJFJAH4HHhGRB4Ioe7GmjBHtHiyiCTjbzn+UFVrjqROEfmxiKwQkRV79+49ktuGrPYhbRtnM8aY8BHK6v5xqloEnA88o6qDgdNDKLcdSK/zOQ3/slwhEZFY4E3gt4FJLAD7AJ+I1DaPmqxTVZ9U1SGqOiQpKSnU2x4RX2BZLZsZaYwx4SOUxOYKtJwuBt44grqXA90Dsxg9wFTgsJNOAALXzwH+paqv1B5XVQXeB2pnUF4FvHYEMR1T8ZbYjDEm7ISS2O4C5uOfCLJcRDKBjYcrFBgHuyFQdi3wsqquEZG7RGQSgIgMFZHtwEXAEyKyJlD8Yvzb41wtItmB16DAuduAm0VkE/4xt6dC/rbHmHVFGmNM+All25pXgLqtps3ABaFUrqpvAW81OPa7Ou+X4+9ObFjueeD5JurcjH/GZauzrWuMMSb8hDJ55M+BySNuEVkoIvtE5PLjEVy480VZV6QxxoSbULoizwxMHpmIf0JID+BXLRpVG+FxOYj2OG1PNmOMCSOhJDZ34M9zgFmqur8F42lzah/SNsYYEx5Cear4dRFZB5QC14tIElDWsmG1HfHRtqyWMcaEk8O22FR1GjASGKKqlcBB/EtjGfzjbDYr0hhjwkeo60ClAmeISGSdY/9qgXjaHJ/XzY6C0tYOwxhjTMBhE5uI/B9wKv4V+t/Cv1r//7DEBvgf0rYxNmOMCR+hTB65EBgH7FLVHwIDgYgWjaoN8XndFJZWUlNzRMtgGmOMaSGhJLbSwALEVYH1G/cAmS0bVtvh83qoUSgqswkkxhgTDkIZY1sR2OTzH8CnQDGwrEWjakN8UbWrj1QGF0U2xhjTekJZUuv6wNvHRWQeEBvifmwnhPjob9aL7Ep0K0djjDGmycQmIic3d05VV7ZMSG2LbV1jjDHhpbkW2/3NnFPgtGMcS5sU7IostZmRxhgTDppMbKr6/eMZSFtVuydb/kFrsRljTDgIZXX/nwcmj9R+jheR65srcyKJjXIjYlvXGGNMuAhluv91qlpQ+0FV84HrWi6ktsXpEGIjbb1IY4wJF6EkNoeISO0HEXECNq+9jniv27auMcaYMBHKc2zzgZdF5HH8k0Z+Csxr0ajaGNu6xhhjwkcoie024MfAzwAB3gH+2ZJBtTU+r5u8YktsxhgTDkJ5QLsGeBz/A9oJQJqqVrd4ZG1IvNfDpj3FrR2GMcYYQpsVuUhEYgNJLRt4RkQeaPnQ2g6f102hjbEZY0xYCGXySJyqFgHnA8+o6mDg9JYNq23xRXk4UF5FZXVNa4dijDEnvFASm0tEkoGLgTeOpHIRGS8i60Vkk4hMa+T8GBFZKSJVInJhg3PzRKRARN5ocPxZEflaRLIDr0FHElNLqF0v0pbVMsaY1hdKYrsL/8zITaq6XEQygY2HKxR4LGAG/o1J+wCXikifBpdtA64GXmykir8AVzRR/a9UdVDglR3Cd2hRtetFFtqyWsYY0+pCmTzyCvBKnc+bgQtCqHsY/mS4GUBEXgImA1/WqWtL4NwhfXiqulBETg3hPq2udr1Ie5bNGGNaX3Or+/9aVf8sIn/H//xaPap642HqTgVy6nzeDgw/qigPdY+I/A5YCExT1fJjVO9R+Wa9SGuxGWNMa2uuxbY28OeKo6xbGjl2SII8CtOBXfhXP3kS/3N2dx1yc5Ef43/+js6dOx+D2zbN561d4d9abMYY09qaW93/9cCfM4+y7u1Aep3PaUDuUdZVN66dgbflIvIMcGsT1z2JP/ExZMiQY5FQmxRMbLb6iDHGtLrmuiLnNldQVScdpu7lQHcRyQB2AFOBHxxxhIfGlayqOwPrV04Bvvi2dX5b7SJcuBxiY2zGGBMGmuuKHIl/jGwWsJTGuxabpKpVInID/hmVTuBpVV0jIncBK1R1rogMBeYA8cC5InKnqvYFEJGPgF5AOxHZDlyrqvOBF0QkKRBPNv61K1uViATWi7TEZowxra25xNYJOAO4FH9L601glqquCbVyVX0LeKvBsd/Veb8cfxdlY2VPaeJ4WO7c7fO6rSvSGGPCQJPPsalqtarOU9WrgBHAJmCRiPy/4xZdG+LfusYSmzHGtLZmn2MTkQhgAv5WW1fgYeA/LR9W2+PzesjZX9LaYRhjzAmvuckjM4F+wNvAnara6pM0wpkvys1qG2MzxphW11yL7QrgINADuLHuJtqAqmpsC8fWpsRHe6wr0hhjwkBzz7GFso6kCYiLclNeVUNZZTWRbmdrh2OMMScsS17HSHBZLWu1GWNMq7LEdozEB1YfyT9o42zGGNOaLLEdI3HB9SKtxWaMMa3JEtsxUtsVaauPGGNM67LEdozYGJsxxoQHS2zHyDcr/FuLzRhjWpMltmMk0u0k0u2w9SKNMaaVWWI7huK9Htu6xhhjWpkltmMoLsptXZHGGNPKLLEdQ/Fej3VFGmNMK7PEdgzFR7spKLUWmzHGtCZLbMdQXJS12IwxprVZYjuG4r3+MTZVbe1QjDHmhGWJ7RiK93qoqlGKy6taOxRjjDlhWWI7huLqPKRdWlrK2LFjqa6uZuvWrQwePJhBgwbRt29fHn/88cPWtX//fs444wy6d+/OGWecQX5+fqPXjR8/Hp/Px8SJE+sdf+SRR8jKykJE2LdvX/D4G2+8wf/93/99i29pjDHhzRLbMVR3Wa2nn36a888/H6fTSXJyMosXLyY7O5ulS5dy7733kpub22xd9957L+PGjWPjxo2MGzeOe++9t9HrfvWrX/Hcc88dcnz06NEsWLCALl261Ds+YcIE5s6dS0lJyVF+S2OMCW+W2I6h+DotthdeeIHJkycD4PF4iIiIAKC8vJyamprD1vXaa69x1VVXAXDVVVfx3//+t9Hrxo0bR0xMzCHHTzrpJLp27XrIcRHh1FNP5Y033gjpOxljTFvToolNRMaLyHoR2SQi0xo5P0ZEVopIlYhc2ODcPBEpEJE3GhzPEJGlIrJRRGaLiKclv8ORqF0vck9hMZs3b66XWHJychgwYADp6encdtttpKSkNFvX7t27SU5OBiA5OZk9e/YcsziHDBnCRx99dMzqM8aYcNJiiU1EnMAM4GygD3CpiPRpcNk24GrgxUaq+AtwRSPH7wMeVNXuQD5w7bGK+dvyBboit+fuwefz1TuXnp7OqlWr2LRpEzNnzmT37t2tESIAHTp0OGxXqDHGtFUt2WIbBmxS1c2qWgG8BEyue4GqblHVVcAhfXOquhA4UPeYiAhwGvDvwKGZwJQWiP2o+KLcxEW5Wby1iLKyskavSUlJoW/fvodtMXXs2JGdO3cCsHPnTjp06HDM4iwrKyMqKuqY1WeMMeGkJRNbKpBT5/P2wLFvoz1QoKq18+mPRZ3HjMvp4MZx3VmWW0FJeWUwuW3fvp3S0lIA8vPz+fjjj+nZsycAV155JcuWLTukrkmTJjFz5kwAZs6cGRyvOxY2bNhAv379jll9xhgTTloysUkjx77tk8sh1ykiPxaRFSKyYu/evd/ytqG7YkQXMhKjcXUeyKIPPgRg7dq1DB8+nIEDBzJ27FhuvfVW+vfvD8CqVauCY2l1TZs2jXfffZfu3bvz7rvvMm2af4hyxYoV/OhHPwped8opp3DRRRexcOFC0tLSmD9/PgAPP/wwaWlpbN++nQEDBtQr8/777zNhwoQW+xkYY0xrkpZaJUNERgK/V9WzAp+nA6jqnxq59lngDVX9d4PjpwK3qurEwGcB9gKdVLWq4T2aMmTIEF2xYsW3/1IhWvDlbq78y2y67nyfxfPmNHldUVER1157La+88spxi2337t384Ac/YOHChcftnsaYtklEPlXVIa0dx5FqyRbbcqB7YBajB5gKzP02Fao/C78P1M6gvAp47VtF2QLG9e7A90cPIycyk7wDpU1eFxsbe1yTGsC2bdu4//77j+s9jTHmeGqxxBYYB7sBmA+sBV5W1TUicpeITAIQkaEish24CHhCRNbUlheRj4BXgHEisl1EaltltwE3i8gm/GNuT7XUdzhaIsJvJ/TB3fs0Hlm0ubXDqWfo0KEMGjSotcMwxpgW02JdkeHkeHdF1pr+n9W8siKH+TeNoVtSu+N+f2OM+Tbaalekq7UD+C675cwevP55Ln98cy1PXT20tcM54ZWWljJ+/Hjee+89nE4n48ePZ8mSJXzve9+rtxLLe++9x6233kpFRQUDBw4kJyeH999/n9WrV/Ozn/2MoqIinE4nv/nNb7jkkksAuOKKK5g9ezaVlZXcf//9vPbaa7z33nv1yhQXF1NdXU1ubi7Lli3j5ptvZunSpagqIkJKSgo1NTV4vV4cDgdOpxOfz8eyZcsoLy8nJSWFkpISSkpKqKysJC4ujoSEBKqqquqViYyMJDs7m8rKSqKjo6ms9O8RWFNTg8PhIC0tDYfDQVRUFNXV1RSBcqI4AAAgAElEQVQXF7N//36Ki4vp0qUL7du3Z82aNdTU1FBVVUXnzp2DddfWFxMTQ3Z2NuXl5WRkZFBQUEBBQQFerxePx0N8fDyq2mQZr9dLVZV/cnNCQgJOp5OIiIh6ZXbv3s2BAwc4ePAgPXr0ICcnh6qqKhwOBxUVFSQnJxMTE4OIEBERQWWlf43W2usSExNxuVzs37+fiooK3G436enpxMTEUF1djdPppKKigtLSUnJzc6moqKBPnz5s2rQJgHbt2hEZGUlkZCTR0dE8++yzZGVl0blzZ0pLS6moqCArK4vS0lJ27dpFdXU1LpeLtLQ0YmJigtf37t0bh8PB7t27qaysJC0tjaqqKvLy8qipqSE2Npb4+Ph692hYxuv1UllZidPpBPzDGO3atatXJjU1lfLycqqqqsjKymLHjh1UVVWhqsGff+/evZk9ezYJCQn07t2bqqoqCgoKqKio4LzzzuOTTz4hLy8PVeXss88G6Cciq4CrVXUlgIi8C4wBPPiHnBKAauBg4PM4oKS2jIjcAPwB8AEbgXnALwJ/3bYCVUBxg3v8GZiAv1fxXeAXqqoisgC4SFUbXzy3lqp+51+DBw/W1vLYok3a5bY39MMNe1otBuP3yCOP6N/+9rfg5wULFujcuXN1woQJwWPV1dWalpam69evV1XV8ePH69SpU1VVdf369bphwwZVVd2xY4d26tRJ8/PzNS8vT1NSUvSzzz7T9PR0jY+P1z/+8Y+HlFm0aJEmJSXp6NGjdcGCBdqlSxfNy8vTW265RUVEb7/9dl2wYIGmp6drVVWVrlixQh0Oh3744Yc6duxYTUxM1Pbt22tZWZlOmTJFRUQPHjzYaJmlS5dqWlqaer1e/fvf/6779+9Xh8OhUVFRWlNTo7Nnz9YBAwYEfy4ul0tHjRqlb7/9tgL66KOP6i233KKAjh8/XtetW6cZGRlaVVWl69evV6fTqe+8846OHTtWBw0apAkJCRodHa0XX3yxioju3LmzyTJDhgzR6OhoffLJJ/Wrr75Sp9OpHo9Hy8rK6pWZM2eOOp1OTUlJ0RdffFHdbrfecccdet5556nH49Frr71WP/zwQx0yZIiqqr777rvqdrt1+fLlGh0drQMGDNDMzEwtKyvTyMhIdTgcWlVVpc8995xecMEFqqr61FNPaWRkpC5ZskQ7duyogC5evFg3b96sDodDExIStKamRj/55BMdNmyY3njjjTp8+HBNSkrSLl266AcffKARERH64osvqsvlUqfTqXfeeWe968844wxt3769btq0Sbt06aI9e/bUzMxMLS0t1YSEBBURraysbLJMhw4d1Ol06mOPPaaffPKJxsTEaGJi4iFxDR06VJOSkjQ6OlpXrVqlXq9X58+fr+ecc44Ces011+isWbP04osv1htvvFFHjhyp6enp+vXXX2tqaqo6HA7NysrSsrIyTUtLU5fLpcAKYASwVP29e6OAbCAT2BFIYGcAGUABsAH/7PW6Za4K1LMF6AB8ApwKnAOsAn7TyD0+BpyB1yfAqXXq+o0e5t98Wyuyhf1wdFc6J3i5+421VFUffo1I03Lqrt8Jja+zmZeXR0REBD169AD8S6HVPijfo0cPunfvDvgftO/QoQN79+5l/vz5TJw4kUGDBuFwOHC5XERHRx9SZuzYsSQnJ1NRUcGSJUs466yzSEhI4JVXXiE+Pp7Nmzczbtw4evfuzbJly+jUqVOwBaaqlJaWMmDAACIiIli5ciUOh4NPP/200TIiQllZGfHx8fTo0QOXy0VNTQ0iQnV1NRdffDFlZWXs3LmTk08+GbfbTUVFBQsXLsThcHDqqafyyiuvkJGRwaJFi+jZsyfdu3dn2bJlVFRUBH8GqkpJSQkjRowAYMmSJbhcLhYsWNBkmX379pGYmEiXLl1IT08P7l/odDrrlcnKygr+Xt566y08Hg8dOnRg8eLFxMbGMn/+fEaOHElRURE7d+4MtgDj4+MB2LdvH5MmTWLbtm2Ul5fjcrlYtmwZU6dOZdGiRahqsExycjLFxcW4XC569uxJRkYGABUVFYgII0aMYNeuXWzZsoXTTz8d8DcK3nzzTdq3b8+gQYOoqqoiIyODZ599tt71w4YNw+FwBH9ehYWFTJo0CafTSXFxMQ6Hg+XLlzdZpqSkBJfLRfv27RkxYgQlJSWIyCFxnXnmmcGf13/+8x98Ph/p6el89NFHuFwu3nvvPS688ELmz5/P7t27GTBgQLAFX1lZicPhYOrUqURERNCpU6faqqJVdQngE5Fk/I9XVQM7AS/+ZLZdVb/G32KLC/w/Yt0ytclOgAjADezGv2DHw8CljdwjEn+LsO714J+AeOmhf7vrs8TWwiJcTqaf3Yv1uw8we0XO4QuYFlFRUXHI+p2NSUxMpLKykhUrVlBRUcHXX39NXl7eIdfV/mPdrVs3duzYQXp6OkDwH/raf8gbKxMVFcWePXtIT0/nyiuvZNu2bTidzuCzjWlpaezYsYMdO3YQHx/PqaeeyuLFi1FV1q5dS79+/cjNzaVdu3YsXry4yTJ5eXnExcVx5pln8sorryAiJCQk8Pnnn9cr43Q6iYqKYuXKlTz00EO43W4KCgrYu3cvDoeDyspK8vLy8Pl8TJ06lWHDhjFlyhSGDx8evP/IkSNRVfbu3YvP56N2TLuxMlu3biU2Npb777+fpKQkVJWkpKRgXA3L7Ny5k9mzZ+N2u3nsscfYs2cPbreb5ORkPv/8cxITExkzZgznn38+p5xyCoMHD+bgwYMkJyczePDg4O4XHo+HdevWBf/Ho1+/fvzyl7+kT58+wTJut5uioiJKSkqoqakJ7oJRU1NDUVER1113XbDbLycnhwcffJAhQ4bQrVs3AHr27ElpaSl79+495Prk5GRycnLo2LEjCxcuJCkpiaqqqmBcjd2jNuHGxsZy1113MWDAAKqrq1HVYFdmwzIHDx7knnvuCcZVXFxMt27dKCsrIz8/n/LycqZPn87QoUODP8c9e/bgcDiCXds5OTlUV1eDP7lAYDEMVf0E/8z0nfi7FrNVdW3gmmrAKSLtmyiTDqwB5gfKpALrgIhAmcbusbPO9ai/CzKizj0aZYntOBjfrxPDMhJ44J0NFJVVtnY4J6R9+/Ydsn5nY0SEl156iZtuuolhw4YRFRWFy1V/KHrnzp1cccUVPPPMMzgcjno7pldXVxMREYH/kcvGy9Q9d++99+Lz+YiJiQn+ww5QUFDAZZddRlpaGlu2bGHUqFG0a9eOyZMn8/rrrwcTTkFBQZNl4uLiqKys5Pnnn+e5554LjqvVXSc0Ly+PqVOn0rNnT0aNGsW1115LbGws11xzTXBcyuFwBBPpAw88wIoVK3jnnXdYunQpo0aNorKykuzsbFSViIgIIiMjKSoqAmi0TKdOnaioqOCKK65g5cqVwVhq42pYJjk5mTFjxtC9e3d++tOfEhUVRXFxMWVlZeTm5hIZGclLL73Eu+++y6effsrKlSuJjo5m+/btrFu3jlmzZuFyuYiIiAjua+hyufjwww+ZNWsWW7Zs4bPPPqNdu3Z4PB6mTJnCKaecEmwVHThwgEcffZT27duTnJzM7t27yc/PJz09nUsuuYQ1a9bUG6ONi4vjwQcfPOT67du3k56eTm5uLpMmTWLlypWICA6Hg/z8/EbvsX37diIjI4mJiWHGjBn8+9/+R31rfyeNlYmOjubqq68OxqWqdO7cORiX1+slLS2NzZs3IyLk5uaSnp6O0+nkb3/7G6eccgper7f2v9O6/yGriGQBvYE0/ONivURkTJ1rioGUJsrkAP2B0wJlauveU6dMw3uk1rmeRq5vlE0eOQ5EhN9N7MO5j/yPGe9tYvo5vVs7pBNOVFRUk+t3NjRy5Eg++ugj8vPz6d27d7ArEfwP1U+YMIG777472P2WlpbGokWLAP/vunaiR3NlOnTowLZt24JrdrrdbjZu3AjAli1b+N///se5555LYWEhSUlJiAj9+/cnOzsbn8+Hx+OhtLQ0uGpNY2VcLhcdOnTgvffeIzs7O9gNWXvPrVu3cuuttzJ69Gh69OgRXL2mc+fOTJw4kYceeoiamhrcbjdRUVFs376dlJQU1qxZQ6dOnVi7di0iwsknnxx8D1BcXBzsymqsTGRkJGlpaSxevJgJEyYgIhQVFQXjalgmPz+f1NRUunfvTnZ2Ng6HA4fDQWFhYb24nn/+eTIzM4PJsl+/frz//vvU1NTgdDopLS0lPT2dqqoqCgsLSUhI4J133qFv377BLtTOnTtz++23c8YZZ9C+fXtEhMjISD755BO2bt3KlClT2L9/f/B/YLp06cLOnTtZtWoV4H9OVFX54osvDrn+7rvvRkTo3bs3n376Kbfccgsul4uysjLS09P573//22gZl8tFnz59WLJkCddddx0iwsGDB4mKimo0rpqaGjIyMujatSvvvPMOIkJhYSGqyurVqykoKGDw4MHBhdjvuOMOHA4H3bt3p0+fPsyaNYsxY8awZcsWgNquhzQgF7gcWKKqxSJShX8h+xHAh/gTlRsobawM0Bd/d+XbgTLb8bfiIgNlDrlH4O/U23XuQZ3rm2QttuOkX2ocF56cxtMff83WvIOtHc4JJz4+nurq6pCSW+0WQV6vl4KCAq655hqA4BqfV155JRdddFHw+rPOOot33nmH/Pz84NjW2LFjmy0zfPhw3nzzTcDfRbZhwwaGDRvG+vXr+eijj0hLS+OSSy5hyZIlwe6w4uJicnJygrMZq6urmTBhAq+88goffPABP/nJT+qVqZ0dmZ+fz/jx46mpqWHXrl306dOHJ554gq1bt3LttdcyceJEPvjgA1SVIUOG8MUXX9CzZ0/cbjcbNmwgIiKCnTt3snr1ah599FF8Ph9bt26lc+fOABQWFrJr1y5EBI/HQ35+PhdffDF/+tOfWLJkCcOGDatXxufzsWTJErKysti3b1+w+zYrK4ubb76ZFStW1Cvjcrk4/fTTmTNnDgMHDgyOTVVWVrJ9+3aqqqp4+OGHSUhIYN26dcFuwQMHDvDFF18wdepUPB4PJSUlTJo0ieuvvz6YtBITE/n888/p0aMHkZGRrF27lszMTLZu3Rqcoelyuejfvz+JiYnk5ORw5ZVXkpSURHx8PKeffjpLly5lwIABAGzcuJGioiJ+/etf07VrVy6//PLg9XfffTc1NTXs37+fdevW4fF4qKmpobq6mnPPPZdOnTqRkpJS7x533303kZGRfPTRR/Tq1Ys1a/yP+paVldGlS5dG4/J4PJx99tksXbqU3bt34/F4+PLLLzlw4ABTp05lxIgRXHrppUycOJGuXbvypz/9Kdi1uXLlSnbt2sVXX31V2xNRLCK3ATGquhN/IhsrIi78E0eGABtFJAN/Q6k9sEVERgCFdcsE/tN3Bd6vxT9ediXQCejY8HoRcYmIu871tatPdcI/EaVph5td8l14teasyLp2F5Zq7zve1p/8a8Uh50pKSnTMmDFaVVWlqqoOh0MHDhyoAwcO1HPPPbfR+uqWmTdvnvp8PvV4POr1etXj8eicOXMOKXPFFVeo2+3WPn36aFlZmfbs2VPdbrdGRkZqjx49NCsrS08//XTdv3+/qqrec889wdleERERCqjL5VKv16vR0dE6atQozcrK0p49e2pGRoYCevvttx9SJjY2Vs8++2wdMWKEjhs3TrOysoLxejweTU1NVa/XqwMGDNC+ffsGy5x00knq9XpVRHT58uV68sknq8vlUo/Hoy6XS1NTU7VHjx46ePBgjYuL07Fjx2pqaqp6PB4F1OFwBO8DaHR0tHbr1k379++vgwYN0sTERPV4POp0OtXtdiv+rpDg5+joaL3gggu0W7duGhcXpyKiAwcO1JSUFBURBbR79+7qcrnU4XAEy0dERGiPHj20c+fOKiLaq1evYP21L7fbHYwrLi5Ou3btqp06dVJA27dvrwMHDlSfz6dOp1NFRB0Oh3o8HnW73epyuTQmJkYzMzM1Li5OAe3Tp48OHDhQY2Ji1Ol0Bu/hdDo1OTlZhw8frj6fTzMzMzUiIiL4Xbp27RqM3el0amRkpLrdbhURTUlJ0ejoaE1NTdXIyEiNjY3V/v376+DBg9XlcgV/VoEZdCoi6na7NTMzU2NjY9Xj8Wj37t3rlan9/iIS/N23b99eMzMzgzMRBwwYoMnJycH/fhwOhzocDnW73epwOLRjx47q8Xi0a9eu6nQ6tVOnTtq/f39NTU0N3qNdu3YaGRmpaWlpGh0drTExMdqtWzdNTExUt9utvXv31r59+9adARi8j8fj0c6dO2tmZqZmZmaqx+PRq666SlVVL7300uD1tX8Xan/eLpdL3W639uvXT7t06aLTpk3Tp556Sjt16lTvHlFRUcHfY+3vJCEhQbt37647duw4pExtuYiICE1JSdFOnTppZmamulwuvf7661VV9ayzzqoXV3p6urpcLo2OjtbMzEyNjo7WoUOHamZmpk6bNk0fe+wxbd++fb0ytb97n89X+/e5DP841xb83X9O/DMWKwNxlQNFwHrgRuCrwKsIGK/+5PgL/F2UGii3OnBcgJcD51YDQwLHncATgWT2JfCABv4tDyTSV/Uw/+bbA9rH2SPvbeSv72xg1nUjGNntm/HPGTNmUFVVxS9+4X+8o127dhQXFzdbV90yjz76KKtWreLxxx/nn//8Jz//+c/Jz8/H6/XWK3PTTTfhdDqZN28eY8aMYe7cuWzZsoVzzz2XDz74gF27dvHoo4+Sn5/Pfffdx4wZM1i/fj1XX3015513HjU1NfTt2xeXy8XGjRspLS1l27Zt3HjjjVRUVDBv3jx+9rOfsWPHjmCZ+Ph4/vCHP/DEE0+Qn59PWloajz32GN26deOKK67gzjvvJCUlhbFjxzJv3jxef/117rjjDv7whz9w//33079/fxYsWMCdd97JT3/6U2bMmMHHH3/M448/TlFREbt372b06NE8+uij/POf/+QPf/hDcAJFcXExXq+X7OxsRowYwcaNG8nLy2PDhg384he/8LduKqpZnr2a4vJqfv3/fsK61Z/x8NxPcLojufXCMZSXlfLEgtU898DvWfXJIh74z//YlfM1IsKzf/4NF18/jYduu47f/WMO5WWlPHDrNVSUl/O7J1/ljqsmMP2RWXTp2Tf4O6ipruaXk0fwu3/MITE5jeyP3+Ppe6fx8Ov+XR4e//0v+cEv7iA2vunx8RcfvodNqz/ld//4Dy898kdGjT+PzlmNd3GrQnlVNbP+fg+bv1jJNX9+gbf++Rd6jD6HmNRulFXWUFZZHXjVUFZVTWV1DcU7NrJ+1j10m3IjvqyT+frNJ+hw8ulEJ3drMq7NbzzGga1rGPjzR1j/0h/JnHg97nbNj23WLbPmmdvpfcWdOFzuJq8v3rGRja8+QPKIc+k07JyQ7rPuxbupPFhI/+v+EnJcq5+4BWdkFH2uujukMg1/XkcaVyjf/Wi+y7Shkdzys2uYMWMG48aN4/LLL+fBBx8kKSmpyTLnn38+f/rTn+jVq9en+Cd43KSqza4mLyIPAXPVv+VYSI60TKjX2xjbcfajUzKZtSyHu9/8krk3fA+nwz8u8cILL/Dii43tt9q0umVee+01fv/73wMEH8htbM+1pUuX8uc//5l58+axYMECzjnnHFwuF1999RXV1dW8/fbbXHXVVZx66qncd9999e5RWFhIamoq5557Lm+//TZlZWXs2rULVWX69OmceuqpALz66qu8/PLLwTK//e1vg9Pqd+zYQURERHCK/Pz583nooYcQEdxu/1/oWbNm8fOf/5yYmBjatWvHj370I1588UU++OADMjIycLvdvP7666Snp/P5558zcuRI+vfvz44dOwA46aSTKKusprikBHdUOzIGjeSBBV/x1ddbEJeb0356N4kjJvP5V7lk/vIFNCo++PPJ+Woz0q49f/3fPgCq2nWkuuRr7nz9Sxy9L6d01XqmPzGHiFR/Etm17yB/nz2PsqgkHl5WSOEnL0P30ziY/TYPLvwaTcri0YVriVj3ze+g9OuVweuhEOhIcSXc/tyHuGLbw0nX8dcPdwG7mvzdFxdGo+nf4445q5D0yXy5pgbWrGnyeoDig7FIxhheXraVqKGX8bU6idxdTKTbQaTLidfjIiHaQYTbicfpQLoMpzx7EOO+/30cDidDr5/ebP0A3r79qOnVm77pcQy97a+Hvf6QMr9//PAFugynclUvTptyES53REj3iRz2PcThoE+IcVVXVRI9/nxqqqtDLkPDn9eRxhXKd29YJoR7DBnci+HDhwf/fj7//PPNXl9RUcGUKVOCW2up6uUhBQZfHElSO8oyIV1vie04i3Q7+fX4nvzipWxe/XQ7Fw9Nb3QqellZGUOGDMHlcjFt2jSmTKm/n2rDMnWnnNc+F5WXl0diYuIhZdLS0oKfP/vsM0pKSti1axc1NTWsX7+eSy65hD179tS7x9q1azlw4AA1NTXBTU8LCwuprq4mLy8vOGU4NjaWLVu21CtzwQUXBAfXCwoKWL16NTt27KBnz568/PLLdOzYkfLyckaOHElJSQnz5s3jkUceCZbp168fRUVF7N27F6/Xy+23305OTg7JySkszv6SwpgMil2xzPrwC77els/oe98jt7CU/QUHiEjrzOcFLja+MBtnZDvcUe0ozFnHqMmXsSklhYndIuk3sBc+r5u4KA+T/1bK8FGnMHv6aXicDs5c6COnJoFnJndi0Ekn84vcVxh3ZiqTpvifZZr0cTwx7hy+f9OP+dFPTue2W95g8NBhLM5wMm60hwXFA+pdD3Djz15mQOD6Wr/cNZFxoz2cO/mbY80L9Tq/CLeTSNfZuJxHNqz+wCVHuG75JUexDulRlTnCtc+P5h6XHflqQeH686rd2zEUHo+HK6+88ojvoar/aOkyoV5via0VTBqYwrOLt/CXd9ZzzoBkivIOnYq+bds2UlJS2Lx5M6eddhr9+/cPDorDodPXa7uUawf5a5caqqthmejoaL7//e8zatQoSktLiYyMrDe1ve71CxYsIDo6Goej/j+MIkJxcXEwgVZXVxMXF0dFVQ3/fXMeUVFeNhfBp1v2s7OwjIrqGtTl4I2VWymrqgFXBCNu/gdv3nExj895jze/riEqrQ8///cGdq//kvXb8rn8qeXUKCzesJuOvYaRdfYYNv/hMnYVHOR3D88kaWsn9uUUEpMYhSoMy0ggNdbFr++r4KLTR5DeKYmVy5ewu3tX/4yxThE8etlgxj3djqtHZzB4cLfgz66qspLe3TNJjvO3dl0OwRsVRUnBPtq3iyDC7SQ2ykP7dhHB80sWf8zMZ572n3c5iIl00zk1mQP79x5yfUVFBfPffpMH/vrn4DEgeH3dY8aYo2OzIltB7fT/vQfKeWzRpkanotdOF8/MzOTUU0/ls88+q3e+YZm0tDRycnJ4+eWXmTx5MkVFRSQkJBy2zHnnnUd2djYZGRmUl5czcOBAdu7cSYcOHepd//rrrxMXF0daWlpw1mCHDh2oqakhKi6RJ+etQCNj2Xuggi17Cujx27e5c8a/KHNEcOHjn/CXd9azdmcRlZ5YDhws4evyKL7+aiPudj4c7kj/80K7t7Fl+bukDzmD6hqlRhVVqK7xDwhHxLanoqyEyKhIXE4HMd4IOkeU8voN32N4R/jtBcMZ0iWeBy8ZRPeKr4iI8PC9IQPYvnUzy5Yt46STTmL//v3BJFw7TbzWyy+/TGxsLHU3plXVelPkG5bJy8ujZ8+edOzYsd7voaysrN5U9Fpvv/02J598cvD6WrXXG2O+PUtsreSkzvFMGZTCPz76moNE1JuKXrs6APhbTR9//DF9+vQBYPr06bz871eJahdbr8ykSZOYOXMms2bNIjk5mdNOOy3YYps+fTpz5swJTnmvrXvixIk88cQTgH9JIYfDwVlnncXMmTOJj49n0aJFVFdXs3v3bpYuXUpMTAyTJk1i/vz5lFVW4/F1RJwuRv3lQ2679xGiu48gwuMiyuXgusEJOPZ+RfsYLz3XzeSe8/ozMrM9P7xoEokxUax88jbKNi3lmkum8PQPh1JTVcGUcaMp3baaX5zRg25b5/L7SX05uUs8j13Ug8gID3/8+SVUb/iAWdefSkxUBEV5ezhn3BjaVe5n3Zdr+O9//xv8+c6aNSvYIn3jjTc488wzOfPMM9mxYwcXXnghTz31VLALtW6Z1NRUli5dSn5+Pvn5+eTm5rJ3716ysrL4+uuv2bhxI8OGDQuW2bNnD2ed9c0+t5MmTeKll15i7dq1JCQkHHL9rFmzuPTSQ1cE2rBhA/369fsW/0UZY4ION23yu/AKl+n+De3IL9Gev31Lr5u5XCde+AO958mX9MWlW/WWv7+sHbt016QuPTQhrZsOvWK6nvXgBzrijws0OmuYdrr8L9rltjc04aSzdMhP79crn1qqN7+wTHsNHaMOh1N79Buo8xZna15xuW7fvl2TkpJ08eLFqqqakZGh8fHx6nK5NDk5OTi9t3aqfVZWlo4ePVoTExN18eLFes0112ivXr00MjLSP4Xf5VZx+Kc2Iw51RcVo+5TOmpreRSMiItTpdGpUVJTGxcUFF551OBzBqdfx8fGamJio3bp1C055drvdmpSUpDfeeKNecsklOmzYME1OTg5Oxa+d6t6hw/9v7/yjq6quPP755pmY8MMAgvJDBgwli9ZOZcaFQ62tQh20LUtdXVJraYQulh2nU8aOwIzOD1p16dRJgbbLtlRb6o9qKzOKdnVRLGWU2nYGgwEVtdZa6Yw/EKj80FgSEvb8cc4Ll8cLvAx5JLl3f9Z6692z7/mxd97N2/ece97ep3S2qaiosJEjR9r48eOtvr7eqqur7cQTT7Tq6urO7e/5be61tbU2atQoq6urs5qaGqurq7OxY8fauHHjrKmpyczMli9f3rnNe/DgwVZTUyUolTQAAAvHSURBVGMTJkywZcuW2ZgxYzq3r+eDKD/44IM2evRoA2zEiBE2Y8aMzs/1hhtusMrKSquvr7fVq1d3yltaWmzYsGG2e/fuQ66DtrY2mzRpku3fv7/cl5zjdAtgo/WB7/Duvny7fy+zdO1v+Pq6F2l74yX2Nj3E8JkLAKjKVTBkQGV41VRRO6CSITWVPPSvn+MLS+7iwAFj8+ZNrH/gTiZdcT3b9rTyh5ZWCj/OqhMq2LFyMRdc+zWGDayiddtLPPWTe7lq8VKG1FTF/itD/wOqGFJTyYCqkNJlzZo1rFzzOLd+5SsM/ei1vLQj/LD8zNNqufC9I3m48Vq+vrSxc/dUnk2bNrF06VLuueceFi1aRENDQ+cPWPPbiI/UpnA7crE2yfrAYeMUY9WqVTQ3N3PTTTeVVB9g2bJlnHTSScybN++I9boapxz1Hed44fnYnP8X86e/i8lja6mpnMq6h3PMnXseJw+qobqy4rDNHwCNs355sPDhiawY+jZz5pxDLpdjf8cBtr/VyrY9+3hj776D7++9nW179rF15zvs3n8yrw2YwK2rn0MVuaI6VeZE7QcXMeXmn7Hz7TZaquuoH1TJle8/gxlnnMqo2hra2toYdPllhzkoCNvtp02bRkdHB42NjZ3ywm3EXbVJbkfuqk2yfi6XO2Scrmhvb2fBgnDjUEp9CAF5GxoaSqpbbJxy1Hcc58j4jC2j7Nvfwe539rP7j23h/Z397Mkf/zGUW9s7OGfCcD486RSGDqw6eqeO46QKn7E5/Yrqyhwja3OMrK3ubVUcx3F6FN8V6TiO46QKd2yO4zhOqnDH5jiO46QKd2yO4zhOqiirY5N0kaQXJP1W0nVFzn9IUrOkdkmXFZybI+nF+JqTkD8W+9wcX6eU0wbHcRynf1G2XZGScsA3gL8kpABvkvQjM3suUe1/gLnAwoK2w4AvEpLKGfBkbLsrVpltZr5/33EcxzmMcs7YzgZ+a2a/M7M24IfAJckKZrbVzJ4GDhS0vRBYa2ZvRme2FriojLo6juM4KaGcjm0M8L+J8itR1hNtvxeXIf9FxcJzAJI+K2mjpI3JaO2O4zhOuinnD7SLOZxSw5wcqe1sM3tV0mDgAaABuPuwyma3A7cDSNoh6fdHGXM4sLNE/dJGlm2HbNvvtmeXUuwfdzwU6WnK6dheAcYmyqcBr3Wj7fkFbR8DMLNX4/tbku4jLHke5tiSmNmIow0oaWN/DB3TE2TZdsi2/W57Nm2HdNtfzqXIJmCipNMlVQGfBErNm/4IMEPSUElDgRnAI5JOkDQcQFIlMBPYUgbdHcdxnH5K2RybmbUDnyc4qeeBlWb2rKQbJV0MIGmKpFeAWcC3JT0b274J3ERwjk3AjVF2IsHBPQ1sBl4F7iiXDY7jOE7/o6xBkM1sNbC6QLY4cdxEWGYs1nYFsKJA1gKc1fOaAvF5XEbJsu2Qbfvd9uySWvszkbbGcRzHyQ4eUstxHMdJFZl3bEcL+5U2JK2QtF3SloRsmKS1MXzZ2rhhJ3VIGivpUUnPS3pW0jVRnnr7JVVLekLSU9H2G6L8dEkbou33x41eqUVSTtImST+O5UzYL2mrpGfi7383Rllqr/tMO7ZE2K+PAO8BrpD0nt7VquzcyeFRXK4D1pnZRGBdLKeRdmCBmb0bmAr8Tfy8s2B/KzDdzM4EJgMXSZoK3Aosi7bvAub1oo7Hg2sIm9nyZMn+aWY2ObHFP7XXfaYdGyWE/UobZvZz4M0C8SXAXfH4LuDS46rUccLMXjez5nj8FuELbgwZsN8Cb8diZXwZMB34jyhPpe15JJ0GfAz4TiyLDNlfhNRe91l3bMcS9itNnGpmr0P48gdSnzFB0njgz4ANZMT+uAy3GdhOiL/6ErA7/jQH0n/9fxX4ew7Gpj2Z7NhvwE8lPSnps1GW2uu+rNv9+wHHEvbL6adIGkQIx/YFM9vbRbjR1GFmHcBkSUOAVcC7i1U7vlodHyTNBLab2ZOSzs+Li1RNpf3AB8zstZjma62kX/e2QuUk6zO2Ywn7lSbekDQKIL5v72V9ykaMWPMAcK+ZPRjFmbEfwMx2E0LUTQWGSMrf4Kb5+v8AcLGkrYRHDtMJM7hM2G9mr8X37YSbmrNJ8XWfdcd2LGG/0sSPgHwy1znAw72oS9mIz1S+CzxvZksTp1Jvv6QRcaaGpBrgAsIzxkeBfJLfVNoOYGbXm9lpZjae8H/+n2Y2mwzYL2lgDBqPpIGEEIVbSPF1n/kfaEv6KOHOLQesMLObe1mlsiLpB4QA08OBNwgJXR8CVgJ/Qkj+OiuGMEsVks4FHgee4eBzln8kPGdLtf2S3kfYIJAj3NCuNLMbJdURZjDDgE3Ap82stfc0LT9xKXKhmc3Mgv3RxlWxeAJwn5ndLOlkUnrdZ96xOY7jOOki60uRjuM4Tspwx+Y4juOkCndsjuM4Tqpwx+Y4juOkCndsjuM4Tqpwx+b0WSSZpCWJ8kJJX+qhvu+UdNnRax7zOLNiNoFHC+Tjo33zE7LbJM09Sn9XS7ryKHXmSrqti3NvF5P3FNGuZOaIqyQ1pylyvNP3ccfm9GVagY9LGt7biiSJWSFKZR7wOTObVuTcduCa7qRKMbPlZnZ3N8bvMRIROkqt3wDMB2aY2a7yaOU4h+OOzenLtBPS1/9d4YnCGVd+JiLpfEnrJa2U9BtJX5Y0O+Yie0bShEQ3F0h6PNabGdvnJDVKapL0tKS/SvT7qKT7CD/wLtTnitj/Fkm3Rtli4FxguaTGIvbtIKQLmVN4QtIESWti0NrHJU2K8i9JWhiPp0Qd/yvqvCXRxejY/kVJ/1bQ95I4i1onaUSUTZb037G/VfkZlqTHJN0iaT3BCc+KNj4l6edFbMqP8QlCGpQZZrazq3qOUw7csTl9nW8AsyXVdqPNmYS8W38KNAD1ZnY2IV3J/ES98cB5hFQmyyVVE2ZYe8xsCjAFuErS6bH+2cA/mdkhOfskjSbk9ZpOyHU2RdKlZnYjsBGYbWaLutD1y8CCIrPA24H5ZnYWsBD4ZpG23wOuNrP3Ax0F5yYDl8e/weWS8jFRBwLNZvbnwHpC5BmAu4F/MLP3ERz3FxN9DTGz88xsCbAYuDDmdbu4C5vGAbcRnNq2Luo4Ttlwx+b0acxsL+FL92+70awp5l5rJaRm+WmUP0NwZnlWmtkBM3sR+B0wiRBH70qF9C4bCKlNJsb6T5jZy0XGmwI8ZmY7YgqUe4EPlWjfy8ATwKfyMoXsA+cA/x71+DYwKtkuxn0cbGa/iqL7CrpeZ2Z7zGwf8BzB2UAIJXZ/PP4+cG68aRhiZuuj/K4C/e9PHP8SuFPSVYTwXMXYQQjR9IkuDXecMpL1tDVO/+CrQDNhhpKnnXhjFoMbJ59TJWP9HUiUD3DoNV8YT84IqUzmm9kjyRMxvmBLF/oda96bWwjJLvNLexWEPGGTj9DmaGMm/wYddP2/XkpMvU67zexqSX9BmOVuljTZzP5QUP8dQlb6X0jabmb3ljCG4/QYPmNz+jwxMOtKwjJhnq3AWfH4EkJG6O4yS1JFfO5WB7wAPAL8tUJ6GyTVx4joR2IDcJ6k4XFJ8QrCMl9JmNmvCbOqmbG8F3hZ0qyogySdWdBmF/CWpKlR9MkSh6vgYDT7TwG/MLM9wC5JH4zyhq70lzTBzDaY2WJgJ4emfUrqtwO4CLhF0oUl6uY4PYLP2Jz+whLg84nyHcDDkp4gbMDoajZ1JF4gfIGfSnhWtU/SdwjLlc1xJrgDuPRInZjZ65KuJ6RAEbDazLqbAuRmQnT5PLOBb0n6Z4LT/iHwVEGbecAdkloI+dX2lDBOC3CGpCdj/cujfA7hOeMAwrLsZ7po3yhpIsHOdUV06sTMXpZ0MbBa0sfNbEMJ+jnOMePR/R2nnyJpkJnld4NeB4wys2t6WS3H6XV8xuY4/ZePxZniCcDvgbm9q47j9A18xuY4juOkCt884jiO46QKd2yO4zhOqnDH5jiO46QKd2yO4zhOqnDH5jiO46QKd2yO4zhOqvg/zl+1rGFv20sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.138 0.11  0.107 0.107 0.107 0.107 0.107 0.107 0.107 0.108 0.108 0.108\n",
      " 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108\n",
      " 0.108]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.571\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='brute')\n",
    "knn_optimal.fit(d_train,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.999\n",
      "The value of TNR is 0.012\n",
      "The value of FPR is 0.988\n",
      "The value of FNR is 0.001\n",
      "\n",
      "Test error is 11.000\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGa9JREFUeJzt3XucFeV9x/HPdxeViyhoguJC8Yb3xAtUiCaNiEUwKqZGo7ERlXbbRo2aaoLSSkRNaBO1MalGrCh4QTGRijZqECXxghFU4l1Zr6ygKCAqN9ndX/84s3jEPcvZsLvn7OP37WteO/PMMzPP0fV3fvvMM88oIjAzs46votQNMDOz1uGAbmaWCAd0M7NEOKCbmSXCAd3MLBEO6GZmiXBANzNLhAO6mVkiHNDNzBLRqdQNKKTT5lV+hNU+Y+9t+pW6CVaG/vz2o9rUc6x779WiY85mX9h5k6/XFpyhm5klomwzdDOzdtVQX+oWbDIHdDMzgPq6Urdgkzmgm5kBEQ2lbsImc0A3MwNocEA3M0uDM3Qzs0T4pqiZWSKcoZuZpSESGOXiB4vMzCB3U7TYZSMkvS7pGUnzJc3LyraRNFPSguxnz6xckq6UVCPpaUkH5J1nVFZ/gaRRG7uuA7qZGeS6XIpdijMkIvaLiIHZ9hhgVkT0B2Zl2wAjgP7ZUg1cDbkvAGAcMAg4EBjX+CVQiAO6mRnkbooWu/xlRgKTs/XJwDF55VMi5zGgh6TewOHAzIhYFhHLgZnA8OYu4IBuZgatnaEH8HtJT0iqzsq2i4jFANnPXll5FbAw79jarKxQeUG+KWpmBi169D8L0tV5RRMjYmLe9sERsUhSL2CmpBebO10TZdFMeUEO6GZm0KInRbPgPbGZ/Yuyn0skTSfXB/6OpN4RsTjrUlmSVa8F+uYd3gdYlJUfskH57Oba5S4XMzMgor7opTmSuknq3rgODAOeBWYAjSNVRgF3ZuszgJOz0S6DgRVZl8x9wDBJPbObocOysoKcoZuZQWs+WLQdMF0S5GLsLRFxr6S5wDRJo4E3geOy+r8DjgBqgFXAqQARsUzSxcDcrN74iFjW3IUd0M3MoNUm54qIV4F9myhfCgxtojyA0wucaxIwqdhrO6CbmYEf/TczS0b9ulK3YJM5oJuZgedDNzNLhrtczMwS4QzdzCwRDuhmZmkI3xQ1M0uE+9DNzBLhLhczs0Q4QzczS4QzdDOzRDhDNzNLRF3xL7goVw7oZmbgDN3MLBnuQzczS4QzdDOzRDhDNzNLhDN0M7NEeJSLmVkiIkrdgk3mgG5mBu5DNzNLhgO6mVkifFPUzCwR9fWlbsEmc0A3MwN3uZiZJcMB3cwsEe5DNzNLQzR4HLqZWRrc5WJmlgiPcjEzS4QzdGtrZ33/HznttBOJCJ599kVG/8MPWLt2bambZe2g+1ZbMu7y89l1952JCMad8xMOGjKIY086mmVLlwPwy59ew8Oz5gDQf89d+Pef/Ygtu3eloSH4zvDRfLz241J+hI7FAd3a0g47bM8Zp5/Gl/Ydwpo1a5h6y6/59vEjmXLjtFI3zdrBDy85m0ceeIxz/2EsnTbrRJcunTloyCBunHgrU66e+qm6lZWV/OS/xzH2jPG8/HwNW/fcirp1HX/2wHbVypNzSaoE5gFvRcSRknYCbgW2AZ4EvhsRH0vaApgCDACWAt+OiNezc5wPjAbqge9HxH3NXbOiVT/Bpz/MHpJ+JOlKSb/I1vdsq+ulqlOn3P/IlZWVdO3ShcWL3y51k6wddNuyKwMG78f0W+4CoG5dHR9+8FHB+l855EAWPP8KLz9fA8CK5R/QkEDG2a4aGopfinMW8ELe9n8AV0REf2A5uUBN9nN5ROwKXJHVQ9JewAnA3sBw4KrsS6KgNgnokn5E7ptIwOPA3Gx9qqQxbXHNFC1a9DaXX/FrXnvlcWrffIoVH3zAzPv/WOpmWTvo06+K5UvfZ/wvxnLbzBsYd9kYunTtDMAJp32L2x+YwkVXXED3rbsD0G/nvkQEV0+9glt/fz2nnH5SKZvfMTVE8ctGSOoDfAP4n2xbwKHAb7Iqk4FjsvWR2TbZ/qFZ/ZHArRGxNiJeA2qAA5u7bltl6KOBv46ICRFxU7ZMyBozeiPHWqZHj605+qjD2XW3wfTtdwDdunXlO9/5u1I3y9pBZadK9vjSbtx+w3S+/bensHrVGk4747tMu+EOjhx0HMcPHcW77yzl3B+fub7+/oO+zPmn/5hTRv4zh474Ogd+dUCJP0UHU19f/LJx/wX8EGhM57cF3o+Ixn6wWqAqW68CFgJk+1dk9deXN3FMk9oqoDcAOzRR3ptPPuBnSKqWNE/SvIaGlW3UtI5j6NCv8drrb/Lee8uoq6tj+v/ew1cGDyx1s6wdvLNoCe8sfpdnnnoegJl3P8geX96dZe8tp6GhgYjgjpvvZJ/99wJgyaJ3mTfnKd5ftoI1q9fy8KxH2fPLu5fyI3Q40dBQ9JIfq7KluvE8ko4ElkTEE3mnV1OX3Mi+5o5pUlsF9LOBWZLukTQxW+4FZpHrV2pSREyMiIERMbCiolsbNa3jWPjmWwwadABduuT+1D50yFd58cUFJW6VtYel7y7jnbfeod8ufwXAoK8N5NWXX+MLvbZdX+fQEV+n5sVXAXhk9p/Ybc9d6dxlCyorKxnwlf159eXXS9H0jqsFXS75sSpbJuad6WDgaEmvk+t6PpRcxt5DUuNAlD7Aomy9FugLkO3fGliWX97EMU1qk1EuEXGvpN3IdbFUkfumqQXmRkTHH73fTh6f+xR33PF/zH38Purq6pg//zmu/Z+bS90saycTxl7BT68ax2abbUbtG4u48OxLGXPJOey+T38igkULF3Pxef8JwIcrPuTGa27llnuvIwIemvUoD93/aIk/QQfTSnO5RMT5wPkAkg4Bzo2IkyTdDnyLXJAfBdyZHTIj256T7X8gIkLSDOAWSZeT6/HoT+6eZEGKMn2PXqfNq8qzYVZSe2/Tr9RNsDL057cfbap7okVWjj+p6JjT7cKbi7peXkA/UtLOfDJs8Sng7yNiraTOwI3A/uQy8xMi4tXs+LHAaUAdcHZE3NPc9TwO3cwMoK71Ow8iYjYwO1t/lSZGqUTEGuC4AsdfClxa7PUc0M3MwNPnmpklw9PnmpmlIRJ4stYB3cwMnKGbmSXDAd3MLBF+wYWZWRr8TlEzs1Q4oJuZJcKjXMzMEuEM3cwsEQ7oZmZpiHp3uZiZpcEZuplZGjxs0cwsFQ7oZmaJ6Phd6A7oZmYAUdfxI7oDupkZOEM3M0uFb4qamaXCGbqZWRqcoZuZpcIZuplZGqKu1C3YdA7oZmZAOEM3M0uEA7qZWRqcoZuZJcIB3cwsEVGvUjdhkzmgm5nhDN3MLBnR4AzdzCwJztDNzBIR0fEz9IpSN8DMrBxEQ/FLcyR1lvS4pD9Lek7SRVn5TpL+JGmBpNskbZ6Vb5Ft12T7d8w71/lZ+UuSDt/YZ3BANzMDGupV9LIRa4FDI2JfYD9guKTBwH8AV0REf2A5MDqrPxpYHhG7Aldk9ZC0F3ACsDcwHLhKUmVzF3ZANzMjd1O02KXZ8+R8lG1uli0BHAr8JiufDByTrY/Mtsn2D5WkrPzWiFgbEa8BNcCBzV3bAd3MjNYL6ACSKiXNB5YAM4FXgPcj1k8BVgtUZetVwEKAbP8KYNv88iaOaVLBm6KS7iL3rdKkiDi6uRObmXUk0YLp0CVVA9V5RRMjYuIn54p6YD9JPYDpwJ5NXbLxdAX2FSovqLlRLj9v7kAzs5S0ZBx6FrwnFlHvfUmzgcFAD0mdsiy8D7Aoq1YL9AVqJXUCtgaW5ZU3yj+mSQUDekT8YWONNTNLRWsNW5T0RWBdFsy7AIeRu9H5IPAt4FZgFHBndsiMbHtOtv+BiAhJM4BbJF0O7AD0Bx5v7tobHYcuqT/wU2AvoHNjeUTs3JIPaWZWzupbby6X3sDkbERKBTAtIu6W9Dxwq6RLgKeA67L61wE3Sqohl5mfABARz0maBjwP1AGnZ105BRXzYNH1wDhyw2mGAKfSdN+OmVmH1VoZekQ8DezfRPmrNDFKJSLWAMcVONelwKXFXruYUS5dImIWoIh4IyJ+TG74jZlZMlpzlEupFJOhr5FUASyQdAbwFtCrbZtlZta+WjLKpVwVk6GfDXQFvg8MAL5LrgPfzCwZn4sMPSLmZqsfkes/NzNLTn1Dx3/OsphRLg/SxGD2iHA/upklI4Uul2L60M/NW+8MHEtuCI2ZWTIaEpg+t5gulyc2KHpEkh86MrOkpDAfejFdLtvkbVaQuzG6fZu1yMysBD4vXS5P8MlEMXXAa3wyj69Zu5r37E2lboIl6nPR5QLsmT3JtJ6kLdqoPWZmJZHCKJdiPsGjTZTNae2GmJmVUrRgKVfNzYe+PbnJ1LtI2p9P5m/ZityDRmZmyUi9y+Vw4BRyc/BexicB/QPggrZtlplZ+0p6lEtETCY3BeSxEfHbdmyTmVm7ayh1A1pBMX3oA7LXKAEgqWc2n6+ZWTICFb2Uq2IC+oiIeL9xIyKWA0e0XZPMzNpfXajopVwVM2yxUtIWEbEWIHulkoctmllSyjnzLlYxAf0mYJak67PtU4HJbdckM7P2l0IfejFzufynpKfJvehUwL1Av7ZumJlZe/q8ZOgAb5P7Ajue3KP/HvViZklJOkOXtBu5t0+fCCwFbiP3XtEh7dQ2M7N2U594hv4i8BBwVETUAEg6p11aZWbWzsr4zXJFa27Y4rHkuloelHStpKGQwFeYmVkTGlDRS7kqGNAjYnpEfBvYA5gNnANsJ+lqScPaqX1mZu0ihcm5NvpgUUSsjIibI+JIcvO6zAfGtHnLzMzaUUMLlnJV7CgXACJiGXBNtpiZJaNB5duVUqwWBXQzs1TVl7oBrcAB3cyMNEa5OKCbmUFZj14plgO6mRnlPXqlWA7oZma4y8XMLBnlPByxWMW84MLMLHn1Kn5pjqS+kh6U9IKk5ySdlZVvI2mmpAXZz55ZuSRdKalG0tOSDsg716is/gJJozb2GRzQzcxo1QeL6oB/jYg9gcHA6ZL2IvdA5qyI6A/M4pMHNEcA/bOlGrgacl8AwDhgEHAgMK7xS6AQB3QzM1ovoEfE4oh4Mlv/EHgBqAJG8snLgSYDx2TrI4EpkfMY0ENSb+BwYGZELMte/TkTGN7ctR3QzcyAUPGLpGpJ8/KW6qbOKWlHYH/gT8B2EbEYckEf6JVVqwIW5h1Wm5UVKi/IN0XNzGjZTdGImAhMbK6OpC3JvQzo7Ij4QIWnFmhqRzRTXpAzdDMzco/+F7tsjKTNyAXzmyPijqz4nawrheznkqy8Fuibd3gfYFEz5QU5oJuZkRuHXuzSHOVS8euAFyLi8rxdM4DGkSqjgDvzyk/ORrsMBlZkXTL3AcMk9cxuhg7Lygpyl4uZGa06Dv1g4LvAM5LmZ2UXABOAaZJGA28Cx2X7fgccAdQAq4BTITe7raSLgblZvfHZjLcFOaCbmdF6AT0iHqbw292GNlE/gNMLnGsSMKnYazugm5nhuVzMzJLhuVzMzBLhF1yYmSWiIYFOFwd0MzPSmG3RAd3MDN8UNTNLhjN0M7NE1Knj5+gO6GZmuMvFzCwZ7nIxM0uEhy2amSWi44dzB3QzM8BdLmZmyahPIEd3QDczwxm6mVkywhm6mVkanKFbm6uoqOBPj93DorfeZuQ3R238AOvQhh07im5du1JRUUFlZSXTJl3JLydO4YGH51ChCrbpuTWXjv1Xen1xWx54aA6/vHYKFcrVHXNWNQfsuw8vvvwKF//8V3y0chUVlRVUn3wCIw77eqk/WtlLYdiicm8/Kj+dNq8qz4a1s7PPqmbAgC+zVffuDujA6kUPlboJbWrYsaO47bor6dlj6/VlH61cyZbdugFw0+138sprbzLuh2eyatVqunTpjCReqnmNc//9J9w19Vpef7MWSfTrW8WSd5dy/OgzmXHzRLbqvmWpPlab2+wLO2/y6yn+Zcfji445V78+rSxfh1FR6gZYYVVVvTlixFAmTZpa6qZYCTUGc4DVq9egLJR07doFZRur16yhcceOf9WHfn2rAOj1xW3ZpmcPlr+/on0b3QHVEUUv5ardu1wknRoR17f3dTuiyy+7iDHnX0L3hDMr+zRJVJ8zFkkcN3IEx408AoBfXHMDM+6dRfdu3Zj0ywnr69//h0f4xa9vYOny97nq5+M/c75nnn+Jdevq6FvVu90+Q0eVwk3RUmToFxXaIala0jxJ8xoaVrZnm8rON444jCVL3uPJp54pdVOsHd149WXcfv2vuPqyi5l6x93Mm5/773/WP53CrOk38o1hQ7jlt3etr3/Y1w/mrqnXcuWEC/nVtVM+da5331vG+eN/xiUXnENFhf8Y35iGFizlqk3+K0t6usDyDLBdoeMiYmJEDIyIgRUV3QpV+1w46KCBHHXkMGpefoybb7qKIUMOZvINV5a6WdbGen1xWwC27dmDoX9zEM88/9Kn9n9j2CHcP/uRzxw3cL8vsfCtxeu7Vj5auZLvnXchZ1aPYt999mz7hicgWvBPuWqrr+3tgJOBo5pYlrbRNZMy9t8msOPOA9l1t8Gc9Pff48EHH2HUKd8vdbOsDa1avYaVK1etX3/08Sfpv/OOvLHwrfV1HnzoMXbq1weAN2sX0Tio4fmXali3ro4eW2/FunXrOOv8izl6+FAOP/Rr7f9BOqgUMvS26kO/G9gyIuZvuEPS7Da6plmHtnTZcs664GIA6uvqOWLYIXx18EDOvuCS3MiVCrHD9r248LwzAZg5+2Fm3DOLTp060XmLzfn5+DFI4t4HHuKJ+c/y/ooP+d/f3Q/ApWN/wB677VKyz9YR1JfpiL+W8LBF61BSH7Zof5nWGLb4nX7fLDrm3PLG9LIctugHi8zMSGOUiwO6mRnl3TdeLAd0MzPSePTfAd3MDHe5mJklI4VRLn58zMyMXJdLscvGSJokaYmkZ/PKtpE0U9KC7GfPrFySrpRUkz2AeUDeMaOy+gskbXR2Pgd0MzNa/cGiG4DhG5SNAWZFRH9gVrYNMALony3VwNWQ+wIAxgGDgAOBcY1fAoU4oJuZ0bqP/kfEH4FlGxSPBCZn65OBY/LKp0TOY0APSb2Bw4GZEbEsIpYDM/nsl8SnuA/dzIx2GeWyXUQsBoiIxZJ6ZeVVwMK8erVZWaHygpyhm5kBEVH0kj8zbLZUb8Klm3rqNJopL8gZupkZUN+CDD0iJgITW3iJdyT1zrLz3sCSrLwW6JtXrw+wKCs/ZIPy2c1dwBm6mRmtO8qlgBlA40iVUcCdeeUnZ6NdBgMrsq6Z+4BhknpmN0OHZWUFOUM3MwNac6JCSVPJZddfkFRLbrTKBGCapNHAm8BxWfXfAUcANcAq4NSsPcskXQzMzeqNj4gNb7R+igO6mRmte1M0Ik4ssGtoE3UDOL3AeSYBk4q9rgO6mRl+9N/MLBkpPPrvgG5mhmdbNDNLhgO6mVkiyvV1nC3hgG5mhjN0M7NkeJSLmVki6qPjv1XUAd3MDPehm5klw33oZmaJcB+6mVkiGtzlYmaWBmfoZmaJ8CgXM7NEuMvFzCwR7nIxM0uEM3Qzs0Q4QzczS0R91Je6CZvMAd3MDD/6b2aWDD/6b2aWCGfoZmaJ8CgXM7NEeJSLmVki/Oi/mVki3IduZpYI96GbmSXCGbqZWSI8Dt3MLBHO0M3MEuFRLmZmifBNUTOzRLjLxcwsEX5S1MwsEc7QzcwSkUIfulL4VkqdpOqImFjqdlh58e+Fbaii1A2wolSXugFWlvx7YZ/igG5mlggHdDOzRDigdwzuJ7Wm+PfCPsU3Rc3MEuEM3cwsEQ7oZU7ScEkvSaqRNKbU7bHSkzRJ0hJJz5a6LVZeHNDLmKRK4L+BEcBewImS9iptq6wM3AAML3UjrPw4oJe3A4GaiHg1Ij4GbgVGlrhNVmIR8UdgWanbYeXHAb28VQEL87ZrszIzs89wQC9vaqLMw5LMrEkO6OWtFuibt90HWFSitphZmXNAL29zgf6SdpK0OXACMKPEbTKzMuWAXsYiog44A7gPeAGYFhHPlbZVVmqSpgJzgN0l1UoaXeo2WXnwk6JmZolwhm5mlggHdDOzRDigm5klwgHdzCwRDuhmZolwQLdWJ6le0nxJz0q6XVLXTTjXIZLuztaPbm7GSUk9JH3vL7jGjyWd+5e20axcOKBbW1gdEftFxD7Ax8A/5+9UTot/9yJiRkRMaKZKD6DFAd0sFQ7o1tYeAnaVtKOkFyRdBTwJ9JU0TNIcSU9mmfyWsH4O+BclPQz8XeOJJJ0i6VfZ+naSpkv6c7YcBEwAdsn+OvhZVu88SXMlPS3porxzjc3mmb8f2L3d/m2YtSEHdGszkjqRm8v9maxod2BKROwPrAT+DTgsIg4A5gE/kNQZuBY4CvgasH2B018J/CEi9gUOAJ4DxgCvZH8dnCdpGNCf3DTE+wEDJP2NpAHkplHYn9wXxl+38kc3K4lOpW6AJamLpPnZ+kPAdcAOwBsR8VhWPpjcSzsekQSwObnH2fcAXouIBQCSbgKqm7jGocDJABFRD6yQ1HODOsOy5alse0tyAb47MD0iVmXX8Pw4lgQHdGsLqyNiv/yCLGivzC8CZkbEiRvU24/WmyJYwE8j4poNrnF2K17DrGy4y8VK5THgYEm7AkjqKmk34EVgJ0m7ZPVOLHD8LOBfsmMrJW0FfEgu+250H3BaXt98laRewB+Bb0rqIqk7ue4dsw7PAd1KIiLeBU4Bpkp6mlyA3yMi1pDrYvm/7KboGwVOcRYwRNIzwBPA3hGxlFwXzrOSfhYRvwduAeZk9X4DdI+IJ4HbgPnAb8l1C5l1eJ5t0cwsEc7QzcwS4YBuZpYIB3Qzs0Q4oJuZJcIB3cwsEQ7oZmaJcEA3M0uEA7qZWSL+H9xnzAlSak23AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='Actual')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Bigrams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 9.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbwAAAEKCAYAAABpI+C3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VdW58PHfc6bknASSnBBMIANEBplRwqQyKKIo1hEtDhWvttp76/WtrWPfe7XXt70OvXVo1atWaC1apWhVRBQRFQcGCQhUxAGZEmYCBDIP53n/OIOHkOEwhJzA8/18zidnr73WXs85DE/W3nutLaqKMcYYc7xztHUAxhhjzLFgCc8YY8wJwRKeMcaYE4IlPGOMMScES3jGGGNOCJbwjDHGnBAs4RljjDkhWMIzxhhzQrCEZ4wx5oTgausA2lKnTp20W7dubR2GMca0K8uWLdulqhltHcehOqETXrdu3SgsLGzrMIwxpl0RkY1tHcPhsFOaxhhjTgiW8IwxxpwQLOEdgcrKSsaMGUN9fT0AEyZMIDU1lQsvvDCm9tXV1fzwhz+kR48eDB8+nA0bNjRa74YbbqBz587079//gPI77riDU045hYEDB3LppZeyd+/eA/Zv2rSJ5ORk/ud//geAmpoaRo8eTV1d3SF+UmOMaf8s4R2BadOmcdlll+F0OoFgApo+fXrM7adOnUpaWhpr167ltttu46677mq03vXXX88777xzUPn48eP54osvWLVqFb169eKBBx44YP9tt93G+eefH9n2eDyMGzeOGTNmxByjMcYcLyzhHYEXX3yRiy++OLI9btw4OnToEHP7N954gylTpgAwadIk5s+fT2PPJxw9ejR+v/+g8nPPPReXK3jf0YgRIyguLo7se/3118nPz6dfv34HtLnkkkt48cUXY47RGGOOF5bwDlNNTQ3r1q3jSKY1bN68mZycHABcLhcpKSmUlJQc1rGmTZsWGc2Vl5fz0EMPcd999x1Ur3///ixduvSwYzbGmPbKEt5h2rVrF6mpqUd0jMZGcyJyyMf57W9/i8vl4pprrgHgvvvu47bbbiM5Ofmguk6nE4/Hw/79+w89YGOMacdO6Hl4h+urbfv4+8JiKiqrjug42dnZFBUVkZ2dTV1dHaWlpY2eumzO888/z+zZs5k/f34kWS5ZsoRXXnmFO++8k7179+JwOEhMTOSWW24BgjfLJCYmHlHsxhjT3ljCOwwbdlUwrXAnNbV1VFVVtZg87rnnHoYNG8all156QPlFF13E888/z8iRI3nllVc4++yzD2mE98477/DQQw+xYMECfD5fpPzjjz+OvP/1r39NcnJyJNmVlJSQkZGB2+2OuR9jjDke2CnNw+BP8gAw5IwxfPLJJ5HyUaNGccUVVzB//nyys7OZO3cuAP/85z/JzMw86Dg33ngjJSUl9OjRg0ceeYQHH3wQgC1btnDBBRdE6l111VWMHDmSr7/+muzsbKZOnQrALbfcwv79+xk/fjyDBw/mpz/9aYuxf/DBBwcc2xhjThTS2HWkE0VBQYEeztJia3eUcc4jC/g/g10sf+uFFqcinHfeeZHk19Yuu+wyHnjgAXr37t3WoRhj2ikRWaaqBW0dx6GyEd5hCI/wUnN6cdZZZ0UmnjclXpJdTU0Nl1xyiSU7Y8wJyRLeYUjxuhGBPeU13HDDDZGJ5/HO4/Fw3XXXtXUYxhjTJizhHQanQ0jzeSgpr2nrUIwxxsTIEt5hSvO52VNhCc8YY9oLS3iHKT0pgZIyS3jGGNNeWMI7TGlJNsIzxpj2xBLeYfInJbDbruEZY0y7YQnvMPmT3OypqCUQOHHnMRpjTHtiCe8w+ZMSqA8o+6pq2zoUY4wxMbCEd5j8ScG1KO20pjHGtA+W8A6TPykBsIRnjDHthSW8w+T3BZcXs4RnjDHtQ6smPBGZICJfi8haEbm7kf2jRWS5iNSJyKSo8sEiskhEVovIKhH5YdS+F0PH/EJEpomIO1Q+VkRKRWRF6HVva342f3Iw4dnUBGOMaR9aLeGJiBN4Ejgf6AtcJSJ9G1TbBFwP/K1BeQVwnar2AyYAj4lI+PHiLwKnAAMAL/DjqHYfq+rg0Ov+o/l5GgqP8Gx5MWOMaR9a8wGww4C1qroOQEReBi4GvgxXUNUNoX2B6Iaq+k3U+y0isgPIAPaq6pzwPhH5DMhuxc/QJK/HidftZI8lPGOMaRda85RmV6Aoars4VHZIRGQY4AG+a1DuBn4EvBNVPFJEVorI2yLS79BDPjT+JFtA2hhj2ovWHOFJI2WHNEtbRLKA6cAUVQ002P0U8JGqfhzaXg7kqWqZiFwAvA70bOSYNwE3AeTm5h5KOAdJS3LbCM8YY9qJ1hzhFQM5UdvZwJZYG4tIR+At4D9UdXGDffcRPMX5i3CZqu5T1bLQ+zmAW0Q6NTyuqj6rqgWqWpCRkXEon+cgtryYMca0H62Z8JYCPUWku4h4gMnArFgahuq/BvxVVWc22Pdj4DzgquhRn4hkioiE3g8j+NlKjsonaYLf52a33aVpjDHtQqslPFWtA24B5gJrgL+r6moRuV9ELgIQkaEiUgxcATwjIqtDza8ERgPXR00zGBza9zRwErCowfSDScAXIrIS+AMwWVVbdaFLf1ICu+0RQcYY0y605jW88KnFOQ3K7o16v5RG7rJU1ReAF5o4ZqMxq+oTwBNHEu+h8ie5Ka+pp6q2nkS381h2bYwx5hDZSitHILy8mE0+N8aY+GcJ7wjYAtLGGNN+WMI7AraAtDHGtB+W8I6AjfCMMab9sIR3BGyEZ4wx7YclvCOQ4nUjgq22Yowx7YAlvCPgdAhpPltP0xhj2gNLeEcozee2aQnGGNMOWMI7Qum2nqYxxrQLlvCOUFqS2xKeMca0A5bwjpA/ycPu8tq2DsMYY0wLLOEdIX+Shz0VNQQCrbpOtTHGmCNkCe8Ipfk81AeU/VV1bR2KMcaYZljCO0LpyR4ASsqr2zgSY4wxzbGEd4TSfMGEZ1MTjDEmvlnCO0LpoeXFSuxBsMYYE9cs4R2htNAC0jbCM8aY+GYJ7whFRng2F88YY+KaJbwj5PU4SXQ7bAFpY4yJc5bwjoL0pAQb4RljTJxr1YQnIhNE5GsRWSsidzeyf7SILBeROhGZFFU+WEQWichqEVklIj+M2tddRJaIyLciMkNEPKHyhND22tD+bq352aKlJblthGeMMXGu2YQnIk4R+d3hHFhEnMCTwPlAX+AqEenboNom4Hrgbw3KK4DrVLUfMAF4TERSQ/seAh5V1Z7AHuDGUPmNwB5V7QE8Gqp3TPhtAWljjIl7zSY8Va0HhoiIHMaxhwFrVXWdqtYALwMXNzj+BlVdBQQalH+jqt+G3m8BdgAZoTjOBl4JVX0euCT0/uLQNqH94w4z7kPm97nZbXdpGmNMXHPFUOdz4A0RmQmUhwtV9R8ttOsKFEVtFwPDDzVAERkGeIDvgHRgr6qG1/EqDvVzQH+qWicipaH6uw61z0PlT0pgjy0gbYwxcS2WhOcHSgiOrMIUaCnhNTa6OqQVlkUkC5gOTFHVQBMjtvAxY+pPRG4CbgLIzc09lHCa5E9yU1ZdR3VdPQku51E5pjHGmKOrxYSnqv9ymMcuBnKitrOBLbE2FpGOwFvAf6jq4lDxLiBVRFyhUV70McP9FYuIC0gBdjc8rqo+CzwLUFBQcFQecZCWFFperLyWzBRLeMYYE49avEtTRLJF5DUR2SEi20XkVRHJjuHYS4GeobsqPcBkYFYsQYXqvwb8VVVnhstVVYEPgPAdnVOAN0LvZ4W2Ce1/P1S/1aUn2QLSxhgT72KZlvBngsmkC8HrZG+GypoVGoHdAswF1gB/V9XVInK/iFwEICJDRaQYuAJ4RkRWh5pfCYwGrheRFaHX4NC+u4BfiMhagtfopobKpwLpofJfAAdNg2gtkQWk7TqeMcbErViu4WWoanSC+4uI/DyWg6vqHGBOg7J7o94vJXhasmG7F4AXmjjmOoJ3gDYsryKYOI85e0SQMcbEv1hGeLtE5NrQnDyniFxL8CYWE/L9CM+mJhhjTLyKJeHdQPAU4zZgK8HrYze0ZlDtTarPgwg2+dwYY+JYs6c0Q6ulXK6qFx2jeNolp0NI9drkc2OMiWexrLRycXN1TJA/yWMjPGOMiWOx3LTyqYg8AczgwJVWlrdaVO2QJTxjjIlvsSS800M/748qUw5ceeWE50/ysH5XecsVjTHGtImWruE5gP9V1b8fo3jaLX+Sh2Ub97Z1GMYYY5rQ0jW8AMHJ46YF/iQPeypqCASOyeIuxhhjDlEs0xLmicjtIpIjIv7wq9Uja2fSfB7qA8r+qrqWKxtjjDnmYrmGF55z97OoMgXyj3447Vd4tZXdFTWk+NxtHI0xxpiGYnlaQvdjEUh7F15tZXd5Nd07JbVxNMYYYxpq8pSmiNwZ9f6KBvv+uzWDao/8SeGEZwtIG2NMPGruGt7kqPf3NNg3oRViade+T3i2gLQxxsSj5hKeNPG+se0Tno3wjDEmvjWX8LSJ941tn/B8HheJboeN8IwxJk41d9PKIBHZR3A05w29J7Sd2OqRtUN+n8dGeMYYE6eaTHiq6jyWgRwP/MkeG+EZY0ycimXiuYlRms/D7gob4RljTDyyhHcUpSfZCM8YY+KVJbyjKC3Jwx67hmeMMXGpVROeiEwQka9FZK2I3N3I/tEislxE6kRkUoN974jIXhGZ3aD8YxFZEXptEZHXQ+VjRaQ0at+9rfnZGpOe5KGsuo7quvpj3bUxxpgWtLi0mIhcBjwEdCZ4h6YAqqodW2jnBJ4ExgPFwFIRmaWqX0ZV2wRcD9zeyCF+B/iAm6MLVXVUVB+vAm9E7f5YVS9s6TO1lrTQXLw95bVkptg9P8YYE09iGeE9DFykqimq2lFVO7SU7EKGAWtVdZ2q1gAvAxdHV1DVDaq6Cgg0bKyq84H9TR1cRDoQfAjt6zHEckykhxJeiV3HM8aYuBNLwtuuqmsO49hdgaKo7eJQ2dFyKTBfVfdFlY0UkZUi8raI9DuKfcUkvIC0Xcczxpj4E8vjgQpFZAbBkVRk6KKq/2ihXWPLjx3NFVquAp6L2l4O5KlqmYhcQDDengcFJXITcBNAbm7uUQwnanmxipqjelxjjDFHLpYRXkegAjgX+EHoFct1smIgJ2o7G9hyqAE2RkTSCZ4yfStcpqr7VLUs9H4O4BaRTg3bquqzqlqgqgUZGRlHI5yISMIrs1OaxhgTb2J5Ht6/HOaxlwI9RaQ7sJng0xeuPsxjNXQFMFtVq8IFIpJJ8PSrisgwgsm85Cj1F5NUnwcRbPK5McbEoRZHeCKSLSKvicgOEdkuIq+KSHZL7VS1DrgFmAusAf6uqqtF5H4RuSh07KEiUkwwgT0jIquj+v0YmAmME5FiETkv6vCTgZcadDkJ+EJEVgJ/ACar6jFd5NrpEFK9bpt8bowxcSiWa3h/Bv5GMCkBXBsqG99Sw9CpxTkNyu6Ner+U4KnOxtqOaqw8tG9sI2VPAE+0FFNrs8nnxhgTn2K5hpehqn9W1brQ6y/A0b34dRxJT/LYtARjjIlDsSS8XSJyrYg4Q69rOcbXxtqTNJ+N8IwxJh7FkvBuAK4EtgFbCV4ru6E1g2rP0pM9lJTbtARjjIk3sdyluQm46BjEclxI83nYU1GDqiLS2FREY4wxbaHJhCcid6rqwyLyRxqZMK6qt7ZqZO2UP8lDfUDZV1lHis/d1uEYY4wJaW6EF15OrPBYBHK8iF5txRKeMcbEjyYTnqq+GXpboaozo/eJyBWNNDFEJbzyarp3SmrjaIwxxoTFctPKPTGWGaITnt2paYwx8aS5a3jnAxcAXUXkD1G7OgJ1rR1YexU9wjPGGBM/mruGt4Xg9buLgGVR5fuB21ozqPbMRnjGGBOfmruGtxJYKSJ/U1X73ztGXreTBJeDPfaIIGOMiSuxrKXZTUQeAPoCieFCVc1vtajaMREJLi9WZgnPGGPiSSw3rfwZ+F+C1+3OAv4KTG/NoNq7tCSPjfCMMSbOxJLwvKo6HxBV3aiqvwbObt2w2jd/ki0vZowx8SaWU5pVIuIAvhWRWwg+zLVz64bVvvmTPGwsqWjrMIwxxkSJZYT3c8AH3AoMIfg8vCmtGVR750/ysNtGeMYYE1diWTx6aehtGfAvrRvO8cHv81BWXUd1XT0JLmdbh2OMMYYYRngiMk9EUqO200RkbuuG1b75k4Nz8ey5eMYYEz9iOaXZSVX3hjdUdQ92Da9Zfl948rmd1jTGmHgRS8ILiEhueENE8mjkcUHme9+vtmIJzxhj4kUsCe//Ap+IyHQRmQ58RIyLR4vIBBH5WkTWisjdjewfLSLLRaRORCY12PeOiOwVkdkNyv8iIutFZEXoNThULiLyh1Bfq0TktFhibA3RjwgyxhgTH2K5aeWdUPIYAQhwm6ruaqmdiDiBJ4HxQDGwVERmqeqXUdU2AdcDtzdyiN8RvDv05kb23aGqrzQoOx/oGXoNJzhZfnhLcbaGSMIrswWkjTEmXjQ5whORU0I/TwNyCS4mvRnIjXH0NAxYq6rrVLUGeBm4OLqCqm5Q1VVAoGHj0GT3/bF+kNCx/6pBi4FUEck6hPZHTarPgwjsrrCbVowxJl40N8L7BXAT8PtG9iktr7bSFSiK2i7m6I24fisi9wLzgbtVtbqJ/roCW49SnzFzOoRUr9seEWSMMXGkuYQ3L/TzRlVddxjHlkbKjsbNLvcA2wAP8CxwF3B/rP2JyE0EEzm5ubkHNTha0pI8Ni3BGGPiSHM3rYRvTGl4rSxWxUBO1HY2wdOiR0RVt4ZOW1YTXNh62KH0p6rPqmqBqhZkZGQcaThN8vtstRVjjIknzY3wSkTkA6C7iMxquFNVL2rh2EuBniLSneC1v8nA1YcdaYiIZKnqVhER4BLgi9CuWcAtIvIywVOnpap6zE9nhtl6msYYE1+aS3gTgdMIPgqoset4zVLVutBi03MBJzBNVVeLyP1AoarOEpGhwGtAGvADEfkvVe0HICIfA6cAySJSTPDU6lzgRRHJIHgKcwXw01CXc4ALgLVABW28DJo/ycPnRXtbrmiMMeaYaO6J5zXAYhE5XVV3Hs7BVXUOwUQUXXZv1PulBE89NtZ2VBPljd4so6oK/Oxw4mwN/iQPe8prUFWCg1FjjDFtqcmEJyKPqerPgWkictDNHzGc0jyh+ZM81AWUfVV1pHjdbR2OMcac8Jo7pRl+qvn/HItAjjfRy4tZwjPGmLbX3CnNZaGfC8JlIpIG5IQmi5tmpEUlvO6dkto4GmOMMbE8HuhDEekoIn5gJfBnEXmk9UNr39JtAWljjIkrsSwenaKq+4DLgD+r6hDgnNYNq/1L84WfiWcJzxhj4kEsCc8VWpPySmB2S5VNUHroIbAllvCMMSYuxJLw7ic4l26tqi4VkXzg29YNq/3zup0kuBzssUcEGWNMXIjl8UAzgZlR2+uAy1szqOOBiJCe5KGkzBKeMcbEg1huWnk4dNOKW0Tmi8guEbn2WATX3qUleWyEZ4wxcSKWU5rnhm5auZDgAs29gDtaNarjhD/JY9fwjDEmTsSS8MKzpi8AXlLV3a0Yz3ElvLyYMcaYttfiNTzgTRH5CqgE/i20cHNV64Z1fEizRwQZY0zcaHGEp6p3AyOBAlWtBcqBi1s7sONBepKHsuo6quvq2zoUY4w54cUywgPoCowXkcSosr+2QjzHlfDyYnsrajmpo7ONozHGmBNbiwlPRO4DxgJ9CT7q53zgEyzhtSi8vFhJWQ0ndUxsobYxxpjWFMtNK5OAccA2Vf0XYBCQ0KpRHSfCIzybmmCMMW0vloRXqaoBoE5EOgI7gPzWDev4EBnh2Y0rxhjT5mK5hlcoIqnAn4BlQBnwWatGdZyIjPAs4RljTJuLZWmxfwu9fVpE3gE62vPwYpPqdSNiIzxjjIkHTSY8ETmtuX2qurx1Qjp+uJwOUrxuG+EZY0wcaG6E9/tm9ilwdksHF5EJwOOAE3hOVR9ssH808BgwEJisqq9E7XsHGAF8oqoXRpW/CBQAtQRPrd6sqrUiMhZ4A1gfqvoPVb2/pRhbmz/JJp8bY0w8aDLhqepZR3JgEXECTwLjCa7BuVREZqnql1HVNgHXA7c3cojfAT7g5gblLwLhxav/BvwY+N/Q9sfRyTEe+G21FWOMiQuxPC3hZ6GbVsLbaSLyb821CRlG8Bl661S1BniZBiu0qOqG0PXAQMPGqjof2N9I+RwNITjCy44hljZjIzxjjIkPsUxL+Imq7g1vqOoe4CcxtOsKFEVtF4fKjgoRcQM/At6JKh4pIitF5G0R6Xe0+joS/iQPu20enjHGtLlYpiU4RERCI6rwqUpPDO2kkTI9lOBa8BTwkap+HNpeDuSpapmIXAC8DvQ8KCiRm4CbAHJzc49iOI0LPzFBVRFp7CsxxhhzLMQywpsL/F1ExonI2cBLHDiqakoxkBO1nQ1sOfQQDxZa7iwD+EW4TFX3qWpZ6P0cwC0inRq2VdVnVbVAVQsyMjKORjjN8id5qAso+6rqWr0vY4wxTYtlhHcXwRHRvxIctb0LPBdDu6VATxHpDmwGJgNXH2acESLyY+A8YFxoBZhweSawXVVVRIYRTOYlR9rfkUrzBQfDu8trSPG6W6htjDGmtcTyeKCAqj6tqpMIXrtbpKotPu9GVeuAWwiOENcAf1fV1SJyv4hcBCAiQ0WkGLgCeEZEVofbi8jHwExgnIgUi8h5oV1PAycBi0RkhYjcGyqfBHwhIiuBPxCc5nA0T6EeFn/y9wnPGGNM24nlaQkfAheF6q4AdorIAlX9RbMNiZxanNOg7N6o90tp4i5LVR3VRHmjMavqE8ATLcV0rPl9tryYMcbEg1iu4aWo6j7gMuDPqjoEOKd1wzp++JNshGeMMfEgloTnEpEs4EpgdivHc9yJJDybmmCMMW0qloR3P8HrcGtVdamI5APftm5Yxw+fx0mCy2EjPGOMaWOxPC1hJsGbR8Lb64DLWzOo44mI2GorxhgTB5p7WsKdqvqwiPyRRiaMq+qtrRrZccQSnjHGtL3mRnhrQj8Lj0UgxzNLeMYY0/aae1rCm6Gfzx+7cI5P/iQPG0sq2joMY4w5oTV3SnNWcw1V9aKjH87xKc3nsXl4xhjTxpo7pTmS4NMOXgKW0Phi0CYG6Uke9lfXUV1XT4LL2dbhGGPMCam5aQmZwK+A/gSfWj4e2KWqC1R1wbEI7niRFpqLt7eito0jMcaYE1eTCU9V61X1HVWdAowA1gIfisi/H7PojhPpoYRXUmanNY0xpq00Ow9PRBKAicBVQDeCizL/o/XDOr6ER3h7bLUVY4xpM83dtPI8wdOZbwP/papfHLOojjPh5cVK7MYVY4xpM82N8H4ElAO9gFujntYtgKpqx1aO7bgRTnh2p6YxxrSd5ubhxbLOpolBaujBrzb53Bhj2o4ltWPA5XSQ6nNbwjPGmDZkCe8Y8fs89oggY4xpQ5bwjhF/kofdZTVUVlYyZswY6uvr2bhxI0OGDGHw4MH069ePp59+usXj7N69m/Hjx9OzZ0/Gjx/Pnj17Gq03YcIEUlNTufDCCw8of+KJJ+jRowciwq5duyLls2fP5r777juyD2mMMXHMEt4xkpbkYWdZNdOmTeOyyy7D6XSSlZXFwoULWbFiBUuWLOHBBx9ky5YtzR7nwQcfZNy4cXz77beMGzeOBx98sNF6d9xxB9OnTz+o/IwzzuC9994jLy/vgPKJEycya9YsKipszU9jzPHJEt4xcmpuKmt3lDHt+elcfPHFAHg8HhISEgCorq4mEAi0eJw33niDKVOmADBlyhRef/31RuuNGzeODh06HBzHqafSrVu3g8pFhLFjxzJ7tj3U3hhzfGrVhCciE0TkaxFZKyJ3N7J/tIgsF5E6EZnUYN87IrJXRGY3KO8uIktE5FsRmSEinlB5Qmh7bWh/t9b8bIfq0lO7QqCWb9d+d0DCKSoqYuDAgeTk5HDXXXfRpUuXZo+zfft2srKyAMjKymLHjh1HLcaCggI+/vjjo3Y8Y4yJJ62W8ETECTwJnA/0Ba4Skb4Nqm0Crgf+1sghfkdwLmBDDwGPqmpPYA9wY6j8RmCPqvYAHg3VixtZKV6GnOSizuUlEPj+ebo5OTmsWrWKtWvX8vzzz7N9+/Y2i7Fz584tnlI1xpj2qjVHeMOAtaq6TlVrgJeBi6MrqOoGVV0FHHQuT1XnA/ujyyQ4+/1s4JVQ0fPAJaH3F4e2Ce0fJ1Gz5ePBpQXdqamuZsn63Qft69KlC/369WtxhHXSSSexdetWALZu3Urnzp2PWnxVVVV4vd6jdjxjjIknrZnwuhJ8vFBYcajsSKQDe1W1rpFjRvoL7S8N1Y8bl488BSHAjMXfAVBcXExlZSUAe/bs4dNPP6V3794AXHfddXz22WcHHeOiiy7i+eeDef3555+PXA88Gr755hv69+9/1I5njDHxpDUTXmOjK22k7GgdM6b+ROQmESkUkcKdO3ceYTiHxutx0mfImbz29nzKq+tYs2YNw4cPZ9CgQYwZM4bbb7+dAQMGALBq1arItbpod999N/PmzaNnz57MmzePu+8OXhotLCzkxz/+caTeqFGjuOKKK5g/fz7Z2dnMnTsXgD/84Q9kZ2dTXFzMwIEDD2jzwQcfMHHixNb8Cowxpu2oaqu8CD5Adm7U9j3APU3U/QswqZHyscDsqG0BdgGuhn0Ac4GRofeuUD1pLsYhQ4bosfbXNz/QpH5n6SuFRU3WKS0t1UmTJh3DqFS3bdumZ5999jHt0xjTPgGF2kq5ozVfrTnCWwr0DN1V6QEmA7OO5IChL/oDIHxH5xTgjdD7WaFtQvvfD9WPK9dOHEOXPgXMLNzYZJ2OHTsyc+bMYxgVbNq0id///vfHtE9jjDmWWi3hafA62i0ER15rgL+r6moRuV9ELgIQkaEiUgxcATwjIqvD7UXkY2AmwZtPikXkvNCuu4BfiMhagtfopobKpwLpofJfAAdNg4gHIsLPbv4JSzbspXhP/EzyHjp0KIMHD27rMIwxptVIHA6CjpmCggItLCw85v0W7a5g1MM2mBTrAAAgAElEQVQf8Mvxvfj3cT2Pef/GGHMkRGSZqha0dRyHylZaaQM5fh8j8v384/PNnMi/cBhjzLFkCa+NXH5aNut3lbN8U+OLPxtjjDm6LOG1kfMHZOF1O3llWXFbh2KMMScES3htJDnBxfkDMpm9citVtfVtHY4xxhz3LOG1oUmnZbO/uo65q7e1dSjGGHPcs4TXhkbkp9M11curyze3dSjGGHPcs4TXhhwO4dJTu/LJtzvZVlrV1uEYY8xxzRJeG7t8SDYBhdc+t1GeMca0Jkt4bax7pySG5KXx6vJim5NnjDGtyBJeHLj8tGzW7ihjVXFpW4dijDHHLUt4cWDiwCwSXA6bk2eMMa3I1dYBGEjxujm3XyazVm7hPy7sQ4LL2dYhxaSyspIJEybw/vvv43Q6mTBhAosXL+bMM89k9uzZkXpPPPEEjz32GN999x2bNm3i2muvbbHN6NGjWbJkCapKQkICxcXFpKSkHFB/ypQp/PrXv2bNmjVccMEFLFy4kPLycgCSk5NJTU3F5XLRoUMH1q5dy5lnnonH42H27NnU1taSlJREbW0tAIFAAIfDQXZ2NklJSSQnJ/Pll19y8skns3HjRkpKSsjLyyM9PZ3Vq1cTCASoq6sjNzcXVcXn8+FwOHA6naSmpvLZZ59RXV3NKaecQklJCTt37sTn85GQkEBaWhp1dXVNtklNTaWiIriwuM/nw+fzkZiYeECbsrIySktLKS0tJT8/nz179rB//34cDge1tbV07tyZhISEA/pITExkxYoV1NbWkp6ejtfrZceOHdTU1OByucjNzcXhcOD1eqmvr8fpdOLxePjiiy8in2X9+vWoKomJiZHvV1VxOp0kJSVRUlJCaWkp+/fvJz8/n6qqKrZv3059fT0ej4eTTjoJl8sVqV9bW0uHDh1YsWIF1dXVpKen43K52L17N6pKSkoKKSkpB/RRW1uLx+Nh9erV1NbWkpiYGInX6XTi9Xrp2LEjbrcbESEhIYHNmzezf/9+ampq6NGjB5s3b6aurg5VjcSQn58fOU5NTQ2VlZVs2bKFmpoaLr30UhYtWkRJSQmBQICUlBRSU1NJSkriL3/5Cz169KBPnz44HA62b99ObW0tmZmZ7Ny5k0AgQP/+/dm1axder/eANrm5uVRWVkbi2rlzJ5WVlahq5O9Xx44dm+2jqqqKiooKamtrSUtLazSu+vp6SkpKIn+H6urqEJHId9mlSxc6derEjBkz8Pv95ObmRo7Zo0cPfD4fX331FQAOhwOgv4isAq5X1eUAIjIPGA14gD8B64EbgSzgQ6A3UNFCm5tVVUXkPeA7YFwjbR4GJhIcsM0D/k9UmytUtenlq9r6+URt+WqL5+E15YOvtmveXbN1zqotbR1KzJ544gl97LHHItvvvfeezpo1SydOnHhAveXLl+v69es1Ly9PH3zwwRbblJSUaF5enpaUlOjDDz+sCQkJeu+99x5U/8svv9SvvvpKx4wZo2PGjNHc3FytqqrSn/3sZ+p0OrW8vFzXrVunWVlZ+tprr+m4ceM0KytL16xZo9nZ2erz+fSPf/yj7t69Wx0Oh3q9Xg0EArpo0SLt3bu3zpo1SwcPHqynnnqqjh49Wt9++20F9KmnntJf/vKXCuiECRP0vffe05ycHK2rq9PCwkJ1OBz60Ucf6ZgxY/S0005Tv9+vOTk5evHFF6uI6NatW5tsc/rpp2tCQkIkLrfbrS6XS6uqqg5oM3PmTHU4HDpy5EidMWOGut1uvfnmm/VnP/uZOhwOnTJlSqN9LFmyRPPy8nTChAmakZGhJSUletJJJymgJSUlOmPGDB04cKCqqr7++uvqcrl00aJFOmLEiMhn3717t7pcLk1ISNCqqir96quvtHv37lpXV6dz5sxRp9Opw4YN0/fee08TExP1N7/5jbrdbnU4HHr11VcfUP/rr79Wp9Op7777ro4ZM0aHDBmiOTk5WllZqVlZWQpoeXl5o20WLFignTt3Vrfbrf/5n/+pixYt0tTUVE1JSdFAIKAfffSRFhQUqKrq448/rh6PR71er65atUp9Pp/OnTtXL7jgAgX0hhtu0OnTp+vll1+uqqpTp07VxMREXbx4sXbt2lUdDof26NFDq6qqNDU1VR0Oh9bW1uqiRYt02LBheuutt+r48eM1PT1d165dq1lZWZqQkKB/+9vf1OfzaWJionbq1Cny9yvcZvjw4ZqRkaF5eXm6YMEC9Xq9OnfuXL366qsV0EmTJjXZR15eniYmJmrfvn01EAjokCFD1Ol0HtTH+PHj1e/3a1FRkXq93shnX7dunYqIdu7cWVVVX3rpJb3yyiv11ltv1YEDB2peXp7m5eXpq6++qoAuXLhQp06dqiKiwBZgBLBEg/cenA6sAPKBDcA/gbVAQiiR1QDOFtosBcaG9v0e+IbgM1Abtvk0dCwnsCiqzRTg/2obPQ/PHIJRPTM4qWNCuzqt+eKLL3LxxRdHtseNG0eHDh0OqnfqqafSrVs3AF599dUW28ydO5fzzjsPv9/Pa6+9RnZ2Nt98881B9fv06UPv3r0B2LZtG2eccQYJCQm8+eab5OTksHLlSrp3786AAQPYvDl4F6zT6aRz585UVVWRlpZGr169cLlcBAIBRIT6+npGjBiBqlJTU4PP50NEUFXmz5+Pw+Fg7NixzJw5k+7du/Phhx8ybtw4+vTpw2effUZmZmZkNKWqlJWVMWLECBwOB4WFhbhcLt57770m2+zevZvU1FR69epFampq5EYmp9N5QJuRI0fidDqprq7mgw8+wOv1kpGREfnsc+fOZcyYMQf1ISIAFBcXM3ToUNxuNzt27MDj8TBnzhyuvPJKqqqq2Lp1K507d8btdpOUlMTOnTsjnz0tLQ2Xy0VdXR0JCQn07t2bnj178tlnn5GTkwMER8yffvopnTt3ZujQodTV1dG/f3/effddTj755Ej9mpoaALp06YKqsmvXLs455xycTid79uxBRFi2bNkBfYTbpKenU1FRgdvtpnPnzowYMYL9+/cDUF9fz6hRo9i3bx9bt26lV69eke/yH//4B6mpqeTk5PDxxx/jcrl4//33mTx5Mh9++GFkdAWQlZVFbW0tDoeDyZMnk5CQQG1tLSLC0qVLGTFiBNu2bWPDhg0MGzYMh8NBly5dqKioIC0tjcGDB0f+XoX/jkW3Oeecc4DgwOOtt94iLS2NnJwc3nvvPXw+HwsWLGiyj3Cc55xzDiLCpk2bIv8WGrZxOp2kp6dTV1cX+ezhEW4gEABg0qRJzJ07l+3btzNy5MjIv8dXX30Vt9tN7969WbJkCZ07dwbopKqLgVQRyQIUqAe2EkxS6cAsVa0meCaxHhjWQhs3sD3UrR/whn4HbtgmkeCIMKFBm1nAVQf9BxTFEl6ccDqES07tyoff7GTn/uq2DqdFNTU1rFu3LpLIYqGqbNiwocU2mzdvJicnh+uuu44lS5ZQX1/PgAEDmm1TVVXFBx98QP/+/dmyZQvZ2dmRJJednc2uXbtITEzk9ttvJzc3l5KSElJSUjj33HOZOXMmIoLf72flypUHtElLS+Oss85i4cKFPP7447jdbvbu3RtJALW1tZSUlET627x5M2lpaYwdO5aFCxcCMHLkSFQ1kszCj6RqrM0333xDhw4deOmllzjppJOor68/KK7oNp9//jnTpk0jISGBGTNmUFRURF1dHenp6axcubLRPoqKiiJxhT97p06dDoorfHrwzDPP5LvvvsPpdFJeXk5NTQ01NTWoKiUlJQCkpqYyefJkhg0bxiWXXMLy5ct54IEHSE1NZezYsagqPXv2JC0tjZUrVx5Uf/jw4SxcuJBAIEBhYSEZGRnU1tbSsWPHyPfYWJuysjLcbjfPPPMMAwcOpL6+nqysrMj31alTJ0aPHs1ll13GqFGjqKys5Le//S0FBQWcfPLJlJWVcfLJJ1NVVUVpaSlJSUn079+fn//85/Tt25chQ4awY8cOHA4Hfr+fiooKKisrCQQCbN68mUAgwL59+/jJT35Cjx496Nq1K1lZWZSWljJo0CD69OmDqlJfH1w+MHxKtGGboqIiHn30UQoKCrj11lvZsWMHqampOBwOdu7c2WgfRUVFuFwuZs+ezcCBA9m9ezdOpzNyyr2xNrW1tRQUFNCnTx+mT58OBJ/RWVJSgsPhoLq6mnvuuSfyy1BRUREvv/wybrebffv2Rf7u8P3lsGKgq6ouIvhw7q1ADsEktDxUJ4NgYuvdQpv3VXVNVBtEJL2ZNluBueE2GjyVmRDV5iCW8OLIpNOyqQ8ob6yI/zl5u3btIjU19ZDa1NfXk5KS0mK98G/iDz74ID179iQjIyPyH1hT+vfvz8MPP8ybb76Jw+GgqKgoMpoJq62t5Y033mD9+vWkpKRQW1vLCy+8wPTp0yPXrrZs2XJAm/LyctasWcPIkSO58cYb6dixIzfccAM1NTW43W4cDkekzd69e7nmmmvIzs5mw4YNnH766dTW1rJixYrINazExET27dsXOX7DNnl5edTW1jJu3DiWLVuGw+Ggvr7+gLii24wcOZILL7yQzMxMbrnllsjn2r9/f5Nx5eTkROKaPn06brf7oLhKSkqYPHkyvXv3ZvPmzXTr1o3k5GRuuOGGyChDRCJ9JCcn88gjj1BYWMi7777Laaedxo033khNTQ3PPPNM5LjJycls2bLloPpLlizh9NNPp66ujjPOOIPly5eHrxWxd+/eRvtYsmQJXq+X/Px87rrrLl555RUg+MtYOK7ExERefvll5s2bx7Jly/D5fFx//fWsXr2a2bNno6rk5uaSkpLCli1bcLlcfPTRR7z00kts2LCBzz//nJycHJxOJ4899hijRo2KjJSrqqp46qmnSE9PJysri+3bt7Nnzx6Ki4tJTk7m66+/5qOPPgqeSnM4SEpKYsuWLY22ycnJ4Yc//CGrV6/mX//1X0lKSiIQCOB2u3n00Ucb7SMnJweXy8Vtt90W+ezhP7um4nK5XKxevTryGUWEDh06ROLy+XxkZ2ezbt06RCQSl8fj4ZJLLmHRokX4fD4ARCR8akZFpAfQB8gGioAufJ/ghOApzbSof1qNtRklIqOj2uwJHaepNl2Bs6PaAOxo0OYAdtNKHOl5UgcGZafwyrJifjwqv63DaZbX66Wq6tBWhxERqqtbHr1mZ2fz4YcfRvro2bMn3377bbNt8vLyKC4uJjU1FY/Hw86dO+nSJfj3vri4mNNOO41du3YxePBgMjIycLlcdO7cmffff58VK1ZETjt5vd5Im06dOrFt2zauvvpq5s+fT3Z2Nrm5uVx44YU8/vjjkf+QvF4vGzZs4JNPPuEHP/gBpaWlZGRkICKcdtpprFmzJpJ8y8rKyMzMBGi0TWJiIj6fj4ULFzJx4kREhPLy8khcDdt89913ZGZm4nA4Ir8UuN1uysrKmowLgqO7hQsXsmPHDtxuN/v374/EtXHjRm6//XbOOOMMevXqRXJyMl6vl8TERK699lp+8pOf0Llz58gvCeHvq0uXLqxevZrMzEwqKirIysoiMzOTFStWALBp0yZcLhder/eg+uHvqG/fvnz++eekpaXhdrspLy8nKyur0T7WrFmD0+lk0KBBB3xfpaWlB8X1wgsvkJ+fz5dffkn37t3p1q0b7777bqS+quJ2uyktLcXv9/Puu+/Sr18/Fi9ejMPhoGfPnvTt25ennnqKrKws6urqyMvL4+2332bjxo1ccskl7N69m/r6en7zm99EbgJavHhx5JR4bW0tXq+XRYsWHdQmISGBvLw8tm7dyqpVqxARXC4X+/fv54svvmi0DxHh5JNPZuHChVxzzTW4XC5qamrIyclh1qxZTbbp1q0b//jHPyKnWMNxffrpp+zdu5chQ4awfXvwLKHH4yE/P5+TTz6ZO++8k/nz5/PWW2+F/9lVEUw8W4BrgcWqWha6xrcGCD8gtpjgacjw8LCpNvMJXq/7KNSmD1DZVBsAEXk7qg2hfsJtDmIjvDhz+ZBsvtq2n9Vb4ntOXlpaGvX19YeU9JxO5wFtPvvsM6677rqD6p177rmRf1S1tbUsWLCA008/HYDXXnuNZ5999qA2AwYM4OWXX47cYVhZWcmgQYN4/fXXWbhwIaeccgper5fFixdTUVERuWNxz549TJgwgUAgwLZt2+jbty9Tp06lpKQkcifjggULUFUKCgr44osv6N27N263m2+++YaEhASqq6v55JNP6NChAz/84Q8jfQCUlpaybds2RAS3282ePXu48sormTlzJgsWLODmm28+oE1qaipffPEFeXl5kVNh4Tv4nnjiiYPaBAIBRo0axbx58+jXrx8ul4stW7YgIiQmJjYZV/g3/gkTJkR+Qbjmmmt45pln2LhxIzfeeCMXXnghCxYsiFz3+e6778jJyaG6upq6ujrq6urIysriueee49NPP2XYsGGkpqayceNGEhMTGT58OIWFhfTq1QsR4fPPP2ffvn1UVlby+eef8/rrr0fq5+bmUl9fT1lZGUVFRXg8nkgfEydO5Be/+AWFhYUH9JGbm0tiYiJz5szhlFNOYfXq1QDs37+fvn37cs8991BaWkpWVhZ+v5+vvvoKh8PB+eefz5IlS9i+fTsej4cvv/yS/fv38+6779KjRw9+9atf0alTJ1auXBm59ldSUsLy5cuprq6mtrYWVWXkyJEMGDCATp06UVRUxHXXXUdGRga/+c1v8Hq9FBYW0qNHD4DIMfLy8hptk5aWxhlnnMGiRYsYOHAgbrebLVu2EAgEuPPOO+nSpQt33XXXAX2oKpWVlXz99deREaGqMnTo0CbjcrlcLFmyhI0bNzJx4kQAdu7cSbdu3ejSpQu9evViw4YNXHjhhXTr1o20tDTGjBnDP//5T/Lz8+nXr1949FwDDCGYQ24FNgFjRCQ8iEoGThGRBGAJwUQ0S0RGAKWqurWRNqcTTJQQvB7XBdjQVBsRcQNjwm0k+FtlJsEbYBrX3B0tx/srnu7SDNtdVq09fvWW/tes1W0dSotuuOEGnTdvXmT7zDPP1E6dOmliYqJ27dpV33nnHVUN3iHXtWtXdTqd6vV6dcKECaqqOnPmTM3MzDyoTX19vfbo0UM9Ho+KiPbp00dLS0tVVTU5OVkTExM1MTFR/X6/pqenq8fjUbfbHfnpdrv1rLPO0vz8fM3KylKXy6V+v18TExM1ISFBHQ6HAup2u9XpdGpWVpYOHz5cU1NTNT8/X5OSkjQlJUU7deoUiQFQp9OpiYmJ6na7VUS0S5cumpSUpJmZmQpoSkqKDho0SFNTU9XpdKqIqMvlUpfLpQRPx6jL5dL8/HxNSUlRQHv27HlAG4IX5VVE1O12a3p6uvr9fs3Pz1eXy6UiooMGDdJu3bpFPofT6VSn0xm5E7Jz587q8XgicaWlpemgQYO0Q4cOkT68Xq927NhRs7KyNDk5Wb1er+bn56vf7z8gruTk5MhncTgckbi6du2qubm5mp+frx6PR30+X+TOvoSEBBURdTqd6vF4In2Gv++uXbuqx+PR9PR0HTBggA4ZMuSA7yj6z7FDhw6an5+vPp9P3W639unTRwcMGKB9+/aNtAHU4XBoQkKC5ubmakpKiubn56vD4VC/368DBw6M/FkC6nK5NCcnR10ulyYlJUX+zHv16qUej0czMjK0X79+mp2dHenD5XJF/uzD321+fr5mZmZqbm6ubt68WadOnaqZmZmRNuHvK/zZw315PB6dMmWKqqpeddVVkfpOp1NdLpd6PJ5I7D6fT/v376+ZmZn65JNPHtSHiGhCQkLk+0pPT4/ElZ6erqrBO07T09MjbcLH79atmw4dOlTT09P15JNP1sTERP3pT3+qqqpXXnll5M/N5XKp1+uNfPZu3bopUEfwTsz1wH8TvGPyU6A29GdSBhQSnFqwgeDo7LtQm09DyayxNs9p8JJGQah+uE1BqNwJPBNKcl8Cj2jo//NQm1e1mf/zJVTxhFRQUKDhC/Xx5F9fWMZn63ez+FfjcDvjdxD++eef88gjj0Qufh9qmzvuuIMf/ehHDBw4MOY+rr32Wh599NHIqbmwQEDZVVZNRU09q1et5Lmn/8ijTz3Hf//6/3LpFVfRp1//Ro8vEnw24W/uvYfFixaxaNGimOO66qqrePLJJxk3blyjbVSV8pp6Sitr2VdZy/3/eTcrln7Gf019jam/v58BY35AUlY+pZW1wTpVdZH3+6tqCSjsnPcnKjevIff6R9j2xu/odM5NuJKavg5ate07drz1OClDJpIy+Dx2zp9Kx/5nk3BS9ybbRPcRS32Ara89RH1FKdnX/HdMbRrGFctniY5r84z76DLpPxCnu9m4Duf72vb6Q3Q+79/wdR8cU5uiF+7G6fHS5cr7YqoPB35fhxNXLJ//UPu4aVQ+y/7ya3bt2sW8efOa/LfV0GWXXcZrr732haoOEJEXgNtUdWdT9UXkcYJ3bM5v9sDHoE2rJjwRmQA8TjArP6eqDzbYPxp4DBgITFbVV6L2TQH+I7T5G1V9PnSR9OOoQ2QDL6jqz0XkeuB3QPiOjydU9bnm4ovXhPfel9v58V8L+dN1BYzve1Jbh9OsadOmMWXKFJzO2CfLH2qbcP2K2gBFuyvZtLuC4j0VbNpdQdHuitB2JdV1gUibslXvktR/HOKIrY+yVfNwO5S8kRNJTUqko9dNitdNx8Tgz+DLRYrPjcfpZF9VLX+49+dc9O//j7KawPdJKypx7auspS6gB/ShGiB5wDmIw4kIdEx009Hriuoj2GeHRBdOh4PCd/9BIFBPwfjLcMT4fb3y6K+4+Gf34fYkxFT/cPr47J2/43A4OW3cJXEV1+G0+fvv72bSz38bU/262hpWfTSHQH39IX32w/m+DiWuw+ljeL6ftR+9gdPpjPnfY01NDS+//DJTpkxZpqoFLTYAROQnqvqnWOq2dptWS3gi4iQ4cXA8wQuQS4GrVPXLqDrdgI7A7QQz8yuhcj/B4XABwaHuMmCINphBLyLLCP528VEo4RWo6i2xxhivCa+2PsCI/57P0G5+nv7RkLYOp9WoKhXhEVBVLaUVtZHEUVpZy66yGor2BJNa0e4K9lTUHtC+Q4KLHL+PXL+P3HQfOWlekhMP7T6sQADKqusO6DecrKJ/ltc0/lR6l0O+T1ShVyQ5NpI0O0b97JDgwuGQRo9rTDwTkZgTXjxpzbs0hwFrVXUdgIi8DFxM8LwrAKq6IbQv0KDtecA8Vd0d2j8PmAC8FK4gIj2Bzhw44jsuuJ0OLjm1K39dtIE95TWkJXnaOqSIot0VTF+88ZCf31evyv6okU84kUSPgBpyOYTsNC85fh/9B2SR6/eRkxZMcDl+Lyle90FTD1pLbX0gMnqrqQvQ0euiY6Ibn8d5zGIwxhyZ1kx4Xfn+NlQIjvKGH0Hbrg3qXAXM0AOHqJeHTpN+Q3DkV0Q7dflp2Uz9ZD03TS9kaE4yL/z6Zt6e+x5d0ny4XK7IROzc3FxmzZp1QNvoNS4/+ugjfv7zn7Np06bI2nhPP/00N91000FtunfvTn19PRkZGQQCAdatW4fL5eK+++7j9bfeYc0331HtTSfr0rvJ63IShY//lLIt3yIOJ4n+LlTu2EiiPxNXYjKVu7bg7pCG159J32vvw+PrwPZP/s7a916ktrIch8NBIFDP717/jM7+FH77b5NJTkpi59ZicvPymP7iS+RkdeaR3/8PL/7xRf4J1NXVsWbNmuC6lC4YM2YM77//Pi5X68+ucTsdpCcnkJ4c2+k4Y0z8ac07Ihr7tTfW86extJ1M1IgPeBPopqoDgfeA5xs9sMhNIlIoIoU7dzZ5nbXN9e3SkVvH9WRPRS0PPf6/FKcM4IyHP2Tob+fjcCdw7UMzePCFt3nq+ZdpeFp62rRpXHbZZTidTs4666zIbexbt27F5/Mxd+7cg/qbNm0aV155JXPnzmXbtm2UlZVRXl7Ov//ng/zqP+5lTSCLnJv/xJixZ3FB4DM+uH0sb734LMuWLqVv715Muex8unTpQmXJVsaedgrpqck89/hD3HnDJM4o/4T3bx/L6tefYt6cNylc+hldu2QxdswYfvmDIfzojB6kJydwck4mG9d/x8QJ5/K/f3gEp0O44447WLFiBStWrOCBBx5gzJgx+P1+PB4P48aNY8aMGcfqj8QY0941dwvnkbyAkQSXfQlv3wPc00TdvwCToravAp6J2n6G4PW/8PYg4Jtm+nYSnLfR7qYlNGb4iBH6xsef658/Wae//PsKdXq8mn/PW5p312zNu2u2DvqvuXrNnxbrA3PW6JsrN+tpQ4frd9+ti7Q/99xzdeHChfrMM8/o5MmTNT09XQOBwAF9jBw5UtevX6/r169Xt9ut51x6tZ736ALNveMNBfTqX/w/3VdZo1u2bNFevXpF2q1fv1779eunmZmZetddd6mqas+ePbVjx446YcKEg+qH26SkpOizzz4bKcvLy9OzzjpLVbXRNqrBW7ij26xYsULPP//8I/hmjTGHAyjUOJhadqiv1hzhLQV6ikh3EfEQHJHNaqFN2FzgXBFJE5E04NxQWdhVHDi6I7SwaNhFfD+BsV2rqalhw/r1XHTm4P/f3r2HWVWddxz//phLJoBykfulAQYhIEUID4rBVhgRSCACGm4ZjfSxtjYJoVWbStuHEBttEkOSP0xqDUExARHa2oiPxQsKCpoBBQlGUAMIXjODDjMEEBzm7R97ncmZmXOG4XI4zNnv53l45ux11tp7vYczs87ae5/1Mmd0X344/WI4foz2Tyzg/CcXMrXDu0y8qBsHjhzjFxt28/VfbuKV115nyoNvMOu+F/nuY6+xY9deaj7dkYdWrKC0tJR27drVrYGYOMbu3bu5oFtPHirbxyfHa3nuxU3UHP2YhRP6ADCwXS3nFRXQvXt3ysvL6/WxtraW8vJySktLASgvL6dTp05s2n5KX/kAAAw8SURBVLQpZf0jR45w8OBBrr322rqyAwcOsH37doCUbQ4fPsyaNWvqtRkyZAibN28+/RfZORcLGbv4YWY1kr5BNFDlAUvM7HeS7iD6dPCopJHAI0RrrH1J0nfM7CIz+0jSvxENmgB3WLiBJZgBfLHBIb8p6WqiL0R+BMzJVGxnU6o1K/ft20ePHj3YvXs3JSUlrF27luJrh3KsppYN297g+lUXMGVYD7a/W82Dv9nLBweOMOPHa6goe5kOb51P5eFjrN72PpddVEj/zm3Zsecdjhe05vP//gyV5e/SqlUrpk2awM6HvsXyp6Pv5DR1nezgwYO0bt2aDh061CsvLCysW70+2dNPP02bNm3o2LFjyvqpMi6sXr2a0aNH12uTyNeWro1zziXL6NV+M3sceLxB2YKkx5uJvkuXqu0SYEma5xotNGlm84lOm+aUVGtWJtaI7NevH2PGjGHr1q0UFxdTmN+K4f26UkAN350a3dTyyfFaxm7oT7tjv+WDy8cDUF1VzcIn96Kn9vGp/FZ8criaA9WHmDGwM5Mndmf6LwuZd2Mply2+h5qaGgoKCurSviTSxiSrqqqib9++vP322/Tq1YsuXbqwf/9+JFFZWdmo/urVqxstIt21a1cqKiooKipKeYwVK1Ywe3bjzB9Hjx6lqKjoZF9W51wMnbvLeDig8ZqVlZWVdQsw79+/n40bNzJ48GAA5s+fz7p16+rVL8hrxezp1/DSM4/xw9u/xoxO7zFt0gTW3noFI8r/j8/Zm/z1uKF0blvAomsGM6j7+bRt27ZuhftFixZRWFhYl2tr9uzZDBo0qK5/1dXVHDp0iDlz5rB0aXSf0MCBA+nWrRtdunRh+fLl9fLfVVVVUVZW1mhGdtVVV5Gfn09BQQFLly5t1Gb9+vX1yiBaFb5z584UFDS9+oZzzgG+lmZLkLxm5caNG23IkCE2dOhQGzJkiC1evLiu3qRJk+yFF15otMblzp07raioyPr162cjR460Xbt2mZnZlVdeaaNGjao7xpgxY6xbt251awDm5eVZmzZtbNmyZVZSUmJ9+vSxgoKCujUyZ82aZe3atatbV3LEiBFWXFxsw4cPtwEDBlj79u1t9OjRNm7cuLq+XHrppVZUVGT5+fnWs2fPuv4vWbLEevfubf3797eSkhL78MMP69rcf//9NnPmzEavy6pVq+yWW245Uy+zc66ZaKE3rWS9A9n811IGvC1btth11113wnrjx48/pfqn2qYp06ZNs507dzar7qnUP9U2zrnT11IHPD+l2QIMHz6csWPH1mVNTifx/bqTrX+qbdI5duwYU6dOZeDAgSeseyr1T7WNcy7ePFvCObiWpnPOncta6lqaPsNzzjkXCz7gOeeciwUf8JxzzsWCD3jOOediwQc855xzsRDruzQlVQB7T1CtE7D/LHTnXBTn2CHe8cc5doh3/M2J/TNm1vlsdOZMivWA1xySXmqJt9+eCXGOHeIdf5xjh3jHn8ux+ylN55xzseADnnPOuVjwAe/E7st2B7IozrFDvOOPc+wQ7/hzNna/hueccy4WfIbnnHMuFnzAa4KkiZJel/R7Sbdnuz+ZJGmJpHJJryaVdZT0lKQ3w88O2exjpkjqLelZSTsk/U7SvFAel/iLJG2StC3E/51Q3ldSWYj/YUmF2e5rpkjKk7RV0mNhO06xvyVpu6RXJL0UynLyve8DXhqS8oCfAl8ABgOzJQ3Obq8y6gFgYoOy24G1ZnYhsDZs56Ia4FYzGwSMAr4e/q/jEv9RoMTMLgaGARMljQK+D/w4xF8J3JjFPmbaPGBH0nacYgcYa2bDkr6OkJPvfR/w0rsE+L2Z7TazY8AKYEqW+5QxZvYc8FGD4inA0vB4KTD1rHbqLDGz981sS3h8kOgPX0/iE7+Z2R/DZkH4Z0AJ8F+hPGfjl9QLmAQsDtsiJrE3ISff+z7gpdcTeDtp+51QFiddzex9iAYFoEuW+5NxkvoAw4EyYhR/OKX3ClAOPAXsAg6YWU2oksvv/58A3wJqw/YFxCd2iD7cPCnpZUl/E8py8r2fn+0OnMOUosxvac1hktoC/w38vZlVRx/048HMjgPDJLUHHgEGpap2dnuVeZImA+Vm9rKkMYniFFVzLvYko83sPUldgKck7cx2hzLFZ3jpvQP0TtruBbyXpb5kyx8kdQcIP8uz3J+MkVRANNgtM7P/CcWxiT/BzA4A64iuZbaXlPhQnKvv/9HA1ZLeIrpsUUI044tD7ACY2XvhZznRh51LyNH3vg946W0GLgx3axUCs4BHs9yns+1R4Ibw+Abg11nsS8aEaza/AHaY2Y+SnopL/J3DzA5JnwbGEV3HfBb4cqiWk/Gb2Xwz62VmfYh+x58xs1JiEDuApDaSzks8BsYDr5Kj733/4nkTJH2R6NNeHrDEzO7McpcyRtJDwBiildL/AHwb+F9gJfBnwD5gupk1vLGlxZN0OfA8sJ0/Xcf5Z6LreHGIfyjRjQl5RB+CV5rZHZL6Ec16OgJbgevM7Gj2eppZ4ZTmbWY2OS6xhzgfCZv5wHIzu1PSBeTge98HPOecc7HgpzSdc87Fgg94zjnnYsEHPOecc7HgA55zzrlY8AHPOedcLPiA51ocSSZpUdL2bZIWnqF9PyDpyyeuedrHmR6yMzzboLxPiG9uUtk9kuacYH83S/rqCerMkXRPmuf+mKr8TAlxJWfiuEnSllxZhd+1DD7guZboKHCNpE7Z7kiykGGjuW4EvmZmY1M8Vw7MO5mUNGZ2r5k9eBLHP2OSViRpbv3rgbnAeDOrzEyvnGvMBzzXEtUA9wH/0PCJhjO0xMxF0hhJ6yWtlPSGpO9JKg154LZLKk7azThJz4d6k0P7PEl3S9os6beS/jZpv89KWk70xfWG/Zkd9v+qpO+HsgXA5cC9ku5OEV8FUUqWGxo+IalY0pqw0O/zkj4byhdKui08Hhn6+GLo86tJu+gR2r8p6QcN9r0ozLrWSuocyoZJ+k3Y3yOJGZmkdZLukrSeaHCeHmLcJum5FDEljjGDKNXMeDPbn66ec5ngA55rqX4KlEpqdxJtLibKe/bnwPXAADO7hCgtzNyken2AK4hSxtwrqYhoRlZlZiOBkcBNkvqG+pcA/2Jm9fIlSupBlFethCjP3EhJU83sDuAloNTM/jFNX78H3Jpi1ngfMNfMRgC3AT9L0fZ+4GYzuww43uC5YcDM8BrMlJRYL7YNsMXMPgesJ1ppB+BB4J/MbCjRgP7tpH21N7MrzGwRsACYEHLqXZ0mps8A9xANdh+kqeNcxviA51okM6sm+mP8zZNotjnkvjtKlP7myVC+nWiQS1hpZrVm9iawG/gs0RqDX1WUQqeMKIXMhaH+JjPbk+J4I4F1ZlYRUs0sA/6ymfHtATYBX0mUKcrm8HlgVejHfwLdk9uFNTHPM7MXQtHyBrtea2ZVZvYx8BrRIATRkmoPh8e/Ai4PHybam9n6UL60Qf8fTnq8EXhA0k1ES5SlUkG0TNWMtIE7l0GeHsi1ZD8BthDNaBJqCB/kwqLQydfBktdCrE3arqX+70LD9faMKGXMXDN7IvmJsP7ioTT9O938QncRJSFNnCJsRZSnbVgTbU50zOTX4Djp/wY0Z83BurjN7GZJlxLNil+RNMzMPmxQ/zDwBWCDpHIzW9aMYzh3xvgMz7VYYTHblUSnGxPeAkaEx1OIsnefrOmSWoXrev2A14EngL9TlEYISQPC6vJNKQOukNQpnJqcTXS6sFnMbCfRLGxy2K4G9kiaHvogSRc3aFMJHJQ0KhTNaubhWvGn7ABfATaYWRVQKekvQvn16fovqdjMysxsAbCf+qm1kvtXAUwE7pI0oZl9c+6M8Bmea+kWAd9I2v458GtJm4hu/Eg3+2rK60R/2LsSXQv7WNJiotOeW8LMsQKY2tROzOx9SfOJUs0IeNzMTjbNyp1Eq/UnlAL/IelfiQbzFcC2Bm1uBH4u6RBRbruqZhznEHCRpJdD/Zmh/Aai65itiU7v/lWa9ndLupAozrUp+lTHzPZIuhp4XNI1ZlbWjP45d9o8W4JzOUZSWzNL3J16O9DdzOZluVvOZZ3P8JzLPZPCzDIf2AvMyW53nDs3+AzPOedcLPhNK84552LBBzznnHOx4AOec865WPABzznnXCz4gOeccy4WfMBzzjkXC/8Ps3YvytnhHFIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.124 0.11  0.107 0.107 0.107 0.108 0.108 0.108 0.108 0.108 0.108 0.108\n",
      " 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108\n",
      " 0.108]\n"
     ]
    }
   ],
   "source": [
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train_bg,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.657\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='brute')\n",
    "knn_optimal.fit(d_train_bg,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train_bg)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test_bg)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 1.000\n",
      "The value of TNR is 0.011\n",
      "The value of FPR is 0.989\n",
      "The value of FNR is 0.000\n",
      "\n",
      "Test error is 10.967\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGNhJREFUeJzt3XucVdV99/HPdwAFvIEXDAKiKGo0F1Cj1iRWxYyIGkyNt6ZKlD7zNNFEfWoa1D5BMVaTmlix1YoRhXjXhEhsokXURBNNQEXxWhAVRogoFxNBRGZ+/eOsgQPOGc7IzJwzi+/b13rN3mvf1vE1/M5v1l57bUUEZmbW+dVUugFmZtY2HNDNzDLhgG5mlgkHdDOzTDigm5llwgHdzCwTDuhmZplwQDczy4QDuplZJrpWugGldN2inx9htY/Yb/uBlW6CVaFn//R7beo5PnxnXtkxp9uOgzb5eu3BGbqZWSaqNkM3M+tQjQ2VbsEmc0A3MwNoWFPpFmwyB3QzMyCisdJN2GQO6GZmAI0O6GZmeXCGbmaWCd8UNTPLhDN0M7M8RAajXPxgkZkZFG6Klls2QtLrkmZLmiVpZqrbXtI0SXPSz96pXpLGS5or6TlJ+xedZ1Taf46kURu7rgO6mRkUulzKLeU5IiKGRMSBaX0MMD0iBgPT0zrAMcDgVOqA66HwBQCMBQ4GDgLGNn0JlOKAbmYGhZui5ZaPZyQwKS1PAk4oqp8cBU8CvST1BY4GpkXE0ohYBkwDhrd0AQd0MzNo6ww9gP+W9JSkulS3c0QsAkg/+6T6fsCComPrU12p+pJ8U9TMDFr16H8K0nVFVRMiYkLR+ucjYqGkPsA0SS+3dLpm6qKF+pIc0M3MoFVPiqbgPaGF7QvTz8WSplDoA39LUt+IWJS6VBan3euBAUWH9wcWpvrDN6h/tKV2ucvFzAyIaCi7tETSVpK2aVoGaoHngalA00iVUcB9aXkqcEYa7XII8G7qknkQqJXUO90MrU11JTlDNzODtnywaGdgiiQoxNjbI+IBSTOAuyWNBuYDJ6X9fwWMAOYCK4EzASJiqaTLgBlpv3ERsbSlCzugm5lBm03OFRHzgM82U78EGNZMfQBnlzjXRGBiudd2QDczAz/6b2aWjYYPK92CTeaAbmYGng/dzCwb7nIxM8uEM3Qzs0w4oJuZ5SF8U9TMLBPuQzczy4S7XMzMMuEM3cwsE87Qzcwy4QzdzCwTa8p/wUW1ckA3MwNn6GZm2XAfuplZJpyhm5llwhm6mVkmnKGbmWXCo1zMzDIRUekWbDIHdDMzcB+6mVk2HNDNzDLhm6JmZploaKh0CzaZA7qZGbjLxcwsGw7oZmaZcB+6mVkeotHj0M3M8uAuFzOzTHiUi5lZJpyhW3vaa689uP2269euD9p9Vy659CrGX/uTCrbKOso2227N2B9fyJ57DyIiGHv+v3DoEQdz4te+zNIlywC49oobeHz6E4z4m1pGffNv1x671757cuqXzuSVF+ZUqvmdTwYBXVGlE9J03aJfdTasQmpqapj/+lMc+oXjmD//zUo3p2L2235gpZvQYS4b/888/eSzTLn9l3Tt1pUePbrztbpTWLliJZOvv6PkcXvuM4hrJv2AYw8+qQNbW1nP/un32tRzrPy3/1t2zOl53g0bvZ6kLsBM4M2IOE7S7sCdwPbA08DpEbFa0pbAZOAAYAlwSkS8ns5xITAaaAC+HREPtnTNmnI/QGtJ2kfSdyWNl3RNWv5ke10vd8OO/ALz5r2xWQfzzclWW/fkgEOGMOX2XwKw5sM1/OXP75V17DFf+RK/nvJQezYvT42N5ZfynAu8VLT+A+DqiBgMLKMQqEk/l0XEnsDVaT8k7QucCuwHDAeuS18SJbVLQJf0XQrfRAL+CMxIy3dIGtMe18zdySeP5M67flHpZlgH6T+wH8uWLGfcNRdz17RbGPujMfTo2R2AU8/6Kvc8PJlLr76Ibbbb5iPHHj3yKB74xbSObnLn1xjll42Q1B84FvhJWhdwJHBv2mUScEJaHpnWSduHpf1HAndGxAcR8RowFziopeu2V4Y+GvhcRFwZEbemcmVqzOiNHGsb6NatG8cfV8u9P7u/0k2xDtKlaxf2+fRe3HPLFE750td5f+UqzjrndO6+5eccd/BJnDxsFG+/tYQLLvnWesd9eui+rHp/FXNfnlehlndiDQ3ll437N+CfgKZ0fgdgeUQ0vUWjHuiXlvsBCwDS9nfT/mvrmzmmWe0V0BuBXZqp78u6D/gRkuokzZQ0s7FxRTs1rfMZPvwInnlmNosXv1PpplgHeWvhYt5a9Dazn3kRgGn3P8I+n9mbpe8so7GxkYjg57fdx6eG7rvecUefcBS/nuLs/OOIxsayS3GsSqWu6TySjgMWR8RTRadvrs89NrKtpWOa1V6jXM4Dpkuaw7pvmF2BPYFzSh0UEROACeCbosVOPeUEd7dsZpa8vZS33nyLgXvsyhuvzufgLx7IvP95jR377MA7i5cAcOQxf71eJi6J2uOP5MwTvlmpZndurXhStDhWNePzwJcljQC6A9tSyNh7SeqasvD+wMK0fz0wAKiX1BXYDlhaVN+k+JhmtUtAj4gHJO1FoYulH4VvmnpgRkR0/tH7HahHj+4cNewwvvHN71a6KdbBrrz4aq64bizdunWj/o2FfO+8yxnz/fPZ+1ODiQgWLljEZd/54dr9D/irIby1aDFvzm/x37yV0kZzuUTEhcCFAJIOBy6IiK9Jugf4KoX7i6OA+9IhU9P6E2n7wxERkqYCt0v6MYUej8EU7kmW5GGL1qlsTsMWrXxtMWxxxbivlR1ztvrebWVdryigHydpEOuGLT4D/F1EfCCpO/BTYCiFzPzUiJiXjr8YOAtYA5wXEb9u6Xp+sMjMDGBN23ceRMSjwKNpeR7NjFKJiFVAsw8NRMTlwOXlXs8B3cwMPH2umVk2PH2umVkeIoO5XBzQzczAGbqZWTYc0M3MMuEXXJiZ5cHvFDUzy4UDuplZJjzKxcwsE87Qzcwy4YBuZpaHaHCXi5lZHpyhm5nlwcMWzcxy4YBuZpaJzt+F7oBuZgYQazp/RHdANzMDZ+hmZrnwTVEzs1w4Qzczy4MzdDOzXDhDNzPLQ6ypdAs2nQO6mRkQztDNzDLhgG5mlgdn6GZmmXBANzPLRDSo0k3YZA7oZmY4Qzczy0Y0OkM3M8uCM3Qzs0xEdP4MvabSDTAzqwbRWH5piaTukv4o6VlJL0i6NNXvLukPkuZIukvSFql+y7Q+N23frehcF6b6VyQdvbHP4IBuZgY0NqjsshEfAEdGxGeBIcBwSYcAPwCujojBwDJgdNp/NLAsIvYErk77IWlf4FRgP2A4cJ2kLi1d2AHdzIzCTdFyS4vnKXgvrXZLJYAjgXtT/STghLQ8Mq2Ttg+TpFR/Z0R8EBGvAXOBg1q6tgO6mRltF9ABJHWRNAtYDEwDXgWWR6ydAqwe6JeW+wELANL2d4EdiuubOaZZJW+KSvolhW+VZkXEl1s6sZlZZxKtmA5dUh1QV1Q1ISImrDtXNABDJPUCpgCfbO6STacrsa1UfUktjXK5qqUDzcxy0ppx6Cl4Tyhjv+WSHgUOAXpJ6pqy8P7AwrRbPTAAqJfUFdgOWFpU36T4mGaVDOgR8ZuNNdbMLBdtNWxR0k7AhymY9wCOonCj8xHgq8CdwCjgvnTI1LT+RNr+cESEpKnA7ZJ+DOwCDAb+2NK1NzoOXdJg4ApgX6B7U31EDGrNhzQzq2YNbTeXS19gUhqRUgPcHRH3S3oRuFPS94FngJvS/jcBP5U0l0JmfipARLwg6W7gRWANcHbqyimpnAeLbgbGUhhOcwRwJs337ZiZdVptlaFHxHPA0Gbq59HMKJWIWAWcVOJclwOXl3vtcka59IiI6YAi4o2IuITC8Bszs2y05SiXSiknQ18lqQaYI+kc4E2gT/s2y8ysY7VmlEu1KidDPw/oCXwbOAA4nUIHvplZNjaLDD0iZqTF9yj0n5uZZaehsfM/Z1nOKJdHaGYwe0S4H93MspFDl0s5fegXFC13B06kMITGzCwbjRlMn1tOl8tTG1T9TpIfOjKzrOQwH3o5XS7bF63WULgx+ol2a5GZWQVsLl0uT7Fuopg1wGusm8fXrEPNfP7WSjfBMrVZdLkAn0xPMq0lact2ao+ZWUXkMMqlnE/w+2bqnmjrhpiZVVK0olSrluZD/wSFydR7SBrKuvlbtqXwoJGZWTZy73I5Gvg6hTl4f8S6gP5n4KL2bZaZWcfKepRLREyiMAXkiRHxsw5sk5lZh2usdAPaQDl96Aek1ygBIKl3ms/XzCwbgcou1aqcgH5MRCxvWomIZcCI9muSmVnHWxMqu1SrcoYtdpG0ZUR8AJBeqeRhi2aWlWrOvMtVTkC/FZgu6ea0fiYwqf2aZGbW8XLoQy9nLpcfSnqOwotOBTwADGzvhpmZdaTNJUMH+BOFL7CTKTz671EvZpaVrDN0SXtRePv0acAS4C4K7xU9ooPaZmbWYRoyz9BfBh4Djo+IuQCSzu+QVpmZdbAqfrNc2Voatngiha6WRyTdKGkYZPAVZmbWjEZUdqlWJQN6REyJiFOAfYBHgfOBnSVdL6m2g9pnZtYhcpica6MPFkXEioi4LSKOozCvyyxgTLu3zMysAzW2olSrcke5ABARS4EbUjEzy0ajqrcrpVytCuhmZrlqqHQD2oADupkZeYxycUA3M4OqHr1SLgd0MzOqe/RKuRzQzcxwl4uZWTaqeThiucp5wYWZWfYaVH5piaQBkh6R9JKkFySdm+q3lzRN0pz0s3eql6TxkuZKek7S/kXnGpX2nyNp1MY+gwO6mRlt+mDRGuAfI+KTwCHA2ZL2pfBA5vSIGAxMZ90DmscAg1OpA66HwhcAMBY4GDgIGNv0JVCKA7qZGW0X0CNiUUQ8nZb/ArwE9ANGsu7lQJOAE9LySGByFDwJ9JLUFzgamBYRS9OrP6cBw1u6tgO6mRkQKr9IqpM0s6jUNXdOSbsBQ4E/ADtHxCIoBH2gT9qtH7Cg6LD6VFeqviTfFDUzo3U3RSNiAjChpX0kbU3hZUDnRcSfVXpqgeY2RAv1JTlDNzOj8Oh/uWVjJHWjEMxvi4ifp+q3UlcK6efiVF8PDCg6vD+wsIX6khzQzcwojEMvt7REhVT8JuCliPhx0aapQNNIlVHAfUX1Z6TRLocA76YumQeBWkm9083Q2lRXkrtczMxo03HonwdOB2ZLmpXqLgKuBO6WNBqYD5yUtv0KGAHMBVYCZ0JhdltJlwEz0n7j0oy3JTmgm5nRdgE9Ih6n9NvdhjWzfwBnlzjXRGBiudd2QDczw3O5mJllw3O5mJllwi+4MDPLRGMGnS4O6GZm5DHbogO6mRm+KWpmlg1n6GZmmVijzp+jO6CbmeEuFzOzbLjLxcwsEx62aGaWic4fzh3QzcwAd7mYmWWjIYMc3QHdzAxn6GZm2Qhn6GZmeXCGbu3qxgk/4tgRR7H47XcYMvQjLzqxDNWeOIqtevakpqaGLl26cPfE8Vw7YTIPP/4ENaph+97bcfnF/0ifnXbg4cee4NobJ1Ojwr5jzq1j/89+CoDPfPFYBg/aDYC+O+/Ev//wksp9qE4ih2GLKrz9qPp03aJfdTasA33xCwfz3nsruPnmaxzQk/cXPlbpJrSr2hNHcddN4+nda7u1de+tWMHWW20FwK333Merr81n7D99i5Ur36dHj+5I4pW5r3HB//8XfnnHjQB87qivMOOhKRX5DJXQbcdBm/x6im/sdnLZMef61++uytdhOEOvYo89/gcGDuxf6WZYhTUFc4D331+FUijp2bPHuvpVq1i7wT6WNRlk6B0e0CWdGRE3d/R1zToDSdSdfzGSOGnkMZw0cgQA19xwC1MfmM42W23FxGuvXLv/Q7/5Hdf85y0sWbac664at7Z+9erVnHzWt+napYbRp5/MsMMO7fDP0tnkcFO0w7tcJM2PiF1LbKsD6gDUZbsDamq2am63zcrAgf257xeT3OWS5N7lsvjtJfTZaQeWLFvO/znvIi46/xscOOTTa7ffOPkuPli9mnP+/vT1jps5azb/efPt/OSaK9Y7z4I3FzH622P4yTVXsGv/XTr0s3SktuhyOWu3r5YdDCe+fm9V/jlU0x4nlfRciTIb2LnUcRExISIOjIgDHcxtc9Rnpx0A2KF3L4YddiizX3xlve3H1h7OQ4/+7iPHHTjk0yx4cxHLlr+73nkG9OvL54Z+hpfnvNrOLe/8ohX/Vat2CegUgvYZwPHNlCXtdE2zTm3l+6tYsWLl2uXf//FpBg/ajTcWvLl2n0cee5Ld032V+fULafoL+8VX5vLhh2votd22vPvnv7B69WoAli1/l2dmv8geuzX7R7EVaWxFqVbt1Yd+P7B1RMzacIOkR9vpmtm59af/wV8f9lfsuOP2vD5vJpeOu4qbb7mz0s2ydrJk6TLOvegyABrWNDCi9nC+cMiBnHfR93l9fj2qEbt8og/f+863AJj26ONM/fV0unbtSvctt+CqcWOQxLw3FjDuh9eiGhGNwei/O5k9dh9YyY/WKTRU6Yi/1vCwRetUcu9Dt4+nLfrQ/3bgV8qOObe/MaUq+9A9bNHMjDxGuTigm5lR3X3j5XJANzMjj0f/HdDNzHCXi5lZNnIY5dJe49DNzDqVRqLssjGSJkpaLOn5orrtJU2TNCf97J3qJWm8pLnpAcz9i44ZlfafI2nUxq7rgG5mRps/WHQLMHyDujHA9IgYDExP6wDHAINTqQOuh8IXADAWOBg4CBjb9CVQigO6mRlt++h/RPwWWLpB9UhgUlqeBJxQVD85Cp4EeknqCxwNTIuIpRGxDJjGR78k1uM+dDMzOmSUy84RsQggIhZJ6pPq+wELivarT3Wl6ktyhm5mBkRE2UVSnaSZRaVuEy7d3FOn0UJ9Sc7QzcyAhlZk6BExAZjQyku8Jalvys77AotTfT0woGi//sDCVH/4BvWPtnQBZ+hmZrTtKJcSpgJNI1VGAfcV1Z+RRrscArybumYeBGol9U43Q2tTXUnO0M3MgLacqFDSHRSy6x0l1VMYrXIlcLek0cB84KS0+6+AEcBcYCVwZmrPUkmXATPSfuMiYsMbretxQDczo21vikbEaSU2feTVY1H4Jjm7xHkmAhPLva4DupkZfvTfzCwbOTz674BuZoZnWzQzy4YDuplZJqr1dZyt4YBuZoYzdDOzbHiUi5lZJhqi879V1AHdzAz3oZuZZcN96GZmmXAfuplZJhrd5WJmlgdn6GZmmfAoFzOzTLjLxcwsE+5yMTPLhDN0M7NMOEM3M8tEQzRUugmbzAHdzAw/+m9mlg0/+m9mlgln6GZmmfAoFzOzTHiUi5lZJvzov5lZJtyHbmaWCfehm5llwhm6mVkmPA7dzCwTztDNzDLhUS5mZpnwTVEzs0y4y8XMLBN+UtTMLBPO0M3MMpFDH7py+FbKnaS6iJhQ6XZYdfHvhW2optINsLLUVboBVpX8e2HrcUA3M8uEA7qZWSYc0DsH95Nac/x7YevxTVEzs0w4Qzczy4QDepWTNFzSK5LmShpT6fZY5UmaKGmxpOcr3RarLg7oVUxSF+A/gGOAfYHTJO1b2VZZFbgFGF7pRlj1cUCvbgcBcyNiXkSsBu4ERla4TVZhEfFbYGml22HVxwG9uvUDFhSt16c6M7OPcECvbmqmzsOSzKxZDujVrR4YULTeH1hYobaYWZVzQK9uM4DBknaXtAVwKjC1wm0ysyrlgF7FImINcA7wIPAScHdEvFDZVlmlSboDeALYW1K9pNGVbpNVBz8pamaWCWfoZmaZcEA3M8uEA7qZWSYc0M3MMuGAbmaWCQd0a3OSGiTNkvS8pHsk9dyEcx0u6f60/OWWZpyU1EvSNz/GNS6RdMHHbaNZtXBAt/bwfkQMiYhPAauBfyjeqIJW/+5FxNSIuLKFXXoBrQ7oZrlwQLf29hiwp6TdJL0k6TrgaWCApFpJT0h6OmXyW8PaOeBflvQ48DdNJ5L0dUn/npZ3ljRF0rOpHApcCeyR/jr417TfdyTNkPScpEuLznVxmmf+IWDvDvu/YdaOHNCt3UjqSmEu99mpam9gckQMBVYA/wwcFRH7AzOB/yepO3AjcDzwReATJU4/HvhNRHwW2B94ARgDvJr+OviOpFpgMIVpiIcAB0g6TNIBFKZRGErhC+NzbfzRzSqia6UbYFnqIWlWWn4MuAnYBXgjIp5M9YdQeGnH7yQBbEHhcfZ9gNciYg6ApFuBumaucSRwBkBENADvSuq9wT61qTyT1remEOC3AaZExMp0Dc+PY1lwQLf28H5EDCmuSEF7RXEVMC0iTttgvyG03RTBAq6IiBs2uMZ5bXgNs6rhLherlCeBz0vaE0BST0l7AS8Du0vaI+13WonjpwPfSMd2kbQt8BcK2XeTB4Gzivrm+0nqA/wW+IqkHpK2odC9Y9bpOaBbRUTE28DXgTskPUchwO8TEasodLH8V7op+kaJU5wLHCFpNvAUsF9ELKHQhfO8pH+NiP8GbgeeSPvdC2wTEU8DdwGzgJ9R6BYy6/Q826KZWSacoZuZZcIB3cwsEw7oZmaZcEA3M8uEA7qZWSYc0M3MMuGAbmaWCQd0M7NM/C8n5TnnfP/EggAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='Actual')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.2.4] KNN (KD-Tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "3\n",
      "5\n",
      "7\n",
      "9\n",
      "11\n",
      "13\n",
      "15\n",
      "17\n",
      "19\n",
      "21\n",
      "23\n",
      "25\n",
      "27\n",
      "29\n",
      "31\n",
      "33\n",
      "35\n",
      "37\n",
      "39\n",
      "41\n",
      "43\n",
      "45\n",
      "47\n",
      "49\n",
      "\n",
      "The optimal number of neighbors is 7.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VOXZ8PHfNVuWyQ4BAknYRUA2iQgooCKISwERXNu6PdrW9vHRuuHb1qeP7Vt5batWrbU+ikVEQawotlQERauCsokissgSSCAsWUgg+8xc7x8zGRMIYUBCJnB9P5/5ZObMuc+5zgRyzX3OfV9HVBVjjDGmtXG0dADGGGPM8bAEZowxplWyBGaMMaZVsgRmjDGmVbIEZowxplWyBGaMMaZVsgRmjDGmVbIEZowxplWyBGaMMaZVcrV0ACdK27ZttUuXLi0dhjHGtCqrVq0qVNX0lo7jeJwyCaxLly6sXLmypcMwxphWRUS2t3QMx8tOIRpjjGmVLIEZY4xplSyB1VNZWcmoUaPw+/0AjBs3jpSUFK644oqI2ldXV3PNNdfQo0cPzj33XHJzcxtd75ZbbqFdu3acddZZDZb/+te/plOnTgwcOJCBAweyYMECANauXctNN9103MdljDGnIktg9UyfPp1JkybhdDoBuO+++5g5c2bE7V944QVSU1PZvHkzd999Nw888ECj691000288847jb539913s2bNGtasWcNll10GQL9+/cjPz2fHjh3HeETGGHPqsgRWz6xZs5gwYUL49ejRo0lMTIy4/VtvvcWNN94IwOTJk3nvvfdo7H5rI0eOJC0t7Zhi+973vsfs2bOPqY0xxpzKmjWBicg4EdkoIptFZGoj748UkdUi4hORyYe85xeRNaHH/OaME6CmpoatW7fyXYbi79y5k6ysLABcLhfJyckUFRUd0zaefvpp+vfvzy233EJJSUl4eU5ODh999NFxx2aMMaeaZktgIuIE/gxcCvQBrhORPoestgO4CXilkU1UqurA0GN8c8VZp7CwkJSUlO+0jcZ6WyIScfuf/OQnbNmyhTVr1pCRkcE999wTfq9du3bs2rXrO8VnjDGnkubsgQ0BNqvqVlWtAWYDE+qvoKq5qvolEGjGOJpUVlXLE4s3sbmohqqqqu+0rczMTPLy8gDw+XyUlpYe06nC9u3b43Q6cTgc3HbbbSxfvjz8XlVVFXFxcd8pPmOMOZU0ZwLrBOTVe50fWhapWBFZKSKfisjEExvatzQATyz+hk37A/j9/oiS2IMPPsi8efMOWz5+/HhmzJgBwOuvv85FF110TD2wgoKC8PN58+Y1GKW4adOmw0YtGmPM6aw5E1hjf7kPP8d2ZNmqmgNcDzwhIt0P24HI7aEkt3Lfvn3HFWRirAunQyipqGHs2LF8/PHH4fdGjBjBlClTeO+998jMzGThwoVAcFh7hw4dDtvWrbfeSlFRET169OCxxx5j2rRpAOzatSs8ohDguuuuY9iwYWzcuJHMzExeeOEFAO6//3769etH//79WbJkCY8//ni4zZIlS7j88suP6xiNMeZUJI1dtzkhGxYZBvxaVS8JvX4QQFUfaWTdvwH/UNXXj7CtJt8HyMnJ0eMtJZXz20WM7duBKV18PPbYY0cdOn/JJZeEk9nJUF1dzahRo/j4449xuU6Z6l/GmCggIqtCnYVWpzl7YCuAniLSVUQ8wLVARKMJRSRVRGJCz9sC5wFfN1egqfEeSsprGDRoEBdeeGF4IvORnMzkBbBjxw6mTZtmycsYY+pptr+IquoTkZ8BCwEnMF1V14nIw8BKVZ0vIucA84BU4Hsi8j+q2hfoDfxVRAIEk+w0VW2+BOb1UFxeAwSrZESbnj170rNnz5YOwxhjokqzfqVX1QXAgkOWPVTv+Qogs5F2S4F+zRlbfWnxHrYWHjxZuzPGGHMCWCUOINXrpri8tqXDMMYYcwwsgRG6BlZR0+hEZGOMMdHJEhiQ5vXgDyhlVb6WDsUYY0yELIER7IEBlIQGchhjjIl+lsAI9sAASiosgRljTGthCYzgMHqwBGaMMa2JJTCCw+gBG4lojDGtiCUwgsPowa6BGWNMa2IJDEiIceFyCMV2CtEYY1oNS2AEbzqZ6vVYD8wYY1oRS2AhafHf1kM0xhgT/SyBhaR63eyvsEEcxhjTWlgCC0nzeuwamDHGtCKWwELq7glmjDGmdbAEFpLmDRb0DQSsoK8xxrQGlsBCUuI9BBTKquw6mDHGtAaWwELSQpOZbSSiMca0DpbAQsIV6W0ghzHGtAqWwELqKtJbPURjjGkdLIGFWA/MGGNaF0tgIeF7gtk1MGOMaRUsgYXEe5x4XA6bzGyMMa2EJbAQESHNJjMbY0yrYQmsnpR4tw3iMMaYVsISWD111TiMMcZEP0tg9dg9wYwxpvWwBFZPWrxVpDfGmNbCElg9qV4PpZW1+K2grzHGRD1LYPWkxbtRhdJKG8hhjDHRzhJYPanhclJ2GtEYY6KdJbB6wtU47DqYMcZEPUtg9dTVQ7QemDHGRD9LYPWkWj1EY4xpNSyB1ZNW1wOzU4jGGBP1LIHVE+dxEut2WA/MGGNaAUtgh0iL91BSYcPojTEm2lkCO4SVkzLGmNahWROYiIwTkY0isllEpjby/kgRWS0iPhGZ3Mj7SSKyU0Sebs4460vzWjkpY4xpDZotgYmIE/gzcCnQB7hORPocstoO4CbglSNs5jfAh80VY2NS7Z5gxhjTKjSZwETEKSK/P85tDwE2q+pWVa0BZgMT6q+gqrmq+iUQaGTfg4H2wLvHuf/jkhrvtnlgxhjTCjSZwFTVDwwWETmObXcC8uq9zg8tOyoRcQB/BO47jv1+J6leD2VVPmr9h+VUY4wxUcQVwTqfA2+JyFygvG6hqr5xlHaNJb1Iy7zfASxQ1bymcqeI3A7cDpCdnR3hpptWV05qf0Ut6YkxJ2SbxhhjTrxIElgaUARcVG+ZAkdLYPlAVr3XmcCuCOMaBowQkTuABMAjIgdVtcFAEFV9DngOICcn54TcA6WunNT+ihpLYMYYE8WOmsBU9ebj3PYKoKeIdAV2AtcC10fSUFVvqHsuIjcBOYcmr+aSZhXpjTGmVTjqKEQRyRSReSKyV0T2iMjfRSTzaO1U1Qf8DFgIrAdeU9V1IvKwiIwPbfscEckHpgB/FZF13+1wvru6HphVpDfGmOgWySnEFwkOc58Sev390LIxR2uoqguABYcse6je8xUETy02tY2/AX+LIM4T4tsemFXjMMaYaBbJPLB0VX1RVX2hx9+A9GaOq8WkxLsB64EZY0y0iySBFYrI90Nzwpwi8n2CgzpOSbFuJ/Eep10DM8aYKBdJArsFuBrYDRQAk0PLTllWjcMYY6Jfk9fAQuWgrlLV8Scpnqhg9RCNMSb6RVKJY0JT65yKUr12SxVjjIl2kYxC/CRUDX4ODStxrG62qFpYWryb3MLyo69ojDGmxUSSwIaHfj5cb5nSsDLHKcXuCWaMMdHvaNfAHMBfVPW1kxRPVEiL93Cg2keNL4DHZff8NMaYaHS0a2ABgtU0Tisp3m/rIRpjjIlOkXQvFonIvSKSJSJpdY9mj6wFpYXKSdlIRGOMiV6RXAOrm/P103rLFOh24sOJDqneYDUOm8xsjDHRK5Jq9F1PRiDRpK4eYonVQzTGmKh1xFOIInJ/vedTDnnvd80ZVEtLs4r0xhgT9Zq6BnZtvecPHvLeuGaIJWqk1CUwO4VojDFRq6kEJkd43tjrU4rH5SAxxmWDOIwxJoo1lcD0CM8be33KscnMxhgT3ZoaxDFARMoI9rbiQs8JvY5t9shaWGq8m2Krh2iMMVHriAlMVZ0nM5Bok+r1UHTQemDGGBOtrE7SEaTFe2wemDHGRDFLYEcQvKWKJTBjjIlWlsCOIM3roaLGT1Wtv6VDMcYY0whLYEeQGl9X0NcGchhjTDQ6agITkUki8o2IlIpImYgcqDci8ZSVZvUQjTEmqkVSzPdR4Huqur65g4kmqVZOyhhjolokpxD3nG7JC4KDOMB6YMYYE60i6YGtFJE5wJtAdd1CVX2j2aKKAtYDM8aY6BZJAksCKoCx9ZYpcEonsJR4uwZmjDHRLJL7gd18MgKJNm6ng6RYl41CNMaYKBXJKMRMEZknIntFZI+I/F1EMk9GcC0tzWvVOIwxJlpFMojjRWA+0BHoBLwdWnbKs2ocxhgTvSJJYOmq+qKq+kKPvwHpzRxXVLB6iMYYE70iSWCFIvJ9EXGGHt8Hipo7sGiQEm/3BDPGmGgVSQK7Bbga2A0UAJNDy055aV633ZXZGGOiVCSjEHcA409CLFEn1euhqjZAZY2fOM9pfXs0Y4yJOkdMYCJyv6o+KiJPEZz31YCq3tmskUWBtNBk5uKKGjp54lo4GmOMMfU11QOrKx+18mQEEo3qykmVlNfQKcUSmDHGRJMjJjBVfTv0tEJV59Z/T0SmNGtUUSLNa+WkjDEmWkUyiOPBCJcdRkTGichGEdksIlMbeX+kiKwWEZ+ITK63vLOIrBKRNSKyTkR+HMn+TrS6eog2lN4YY6JPU9fALgUuAzqJyJP13koCfEfbsIg4gT8DY4B8YIWIzFfVr+uttgO4Cbj3kOYFwHBVrRaRBOCrUNtdERzTCZNW7xSiMcaY6NLUNbBdBK9/jQdW1Vt+ALg7gm0PATar6lYAEZkNTADCCUxVc0PvBeo3VNX6GSOGFrpzdHKcGxEotnqIxhgTdZq6BvYF8IWIvKKqx/MXvBOQV+91PnBupI1FJAv4J9ADuK+x3peI3A7cDpCdnX0cITbN6RCS49zWAzPGmCgUSc+mi4i8LiJfi8jWukcE7aSRZYcNxz8SVc1T1f4EE9iNItK+kXWeU9UcVc1JT2+e6lZp8R6bzGyMMVEo0mK+fyF43etC4CVgZgTt8oGseq8zCZ6WPCahntc6YMSxtj0RUr1WTsoYY6JRJAksTlXfA0RVt6vqr4GLImi3AugpIl1FxANcS7Cq/VGFbuESF3qeCpwHbIyk7YmWGu+hxK6BGWNM1IkkgVWJiAP4RkR+JiJXAu2O1khVfcDPgIUEJ0W/pqrrRORhERkPICLniEg+MAX4q4isCzXvDXwmIl8AHwJ/UNW1x3x0J0Ca166BGWNMNDpqLUTgLiAeuBP4DcHTiDdGsnFVXQAsOGTZQ/WeryB4avHQdouA/pHso7mleoPXwFQVkcYu6xljjGkJkRTzXRF6ehC4uXnDiT5p8R5qfAEqavx4YyLJ98YYY06Go55CFJFFIpJS73WqiCxs3rCih1XjMMaY6BTJNbC2qrq/7oWqlhDBNbBTRarVQzTGmKgUSQILiEh4lrCIdOYY5nO1dmleN2A9MGOMiTaRXNT5BfCxiHwYej2SUPWL00HdKUTrgRljTHSJZBDHOyJyNjCUYHWNu1W1sNkjixLfFvS1uWDGGBNNjngKUUTODP08G8gmWEVjJ5AdWnZaSIp14xDrgRljTLRpqgf2c4KnCv/YyHtKZNU4Wj2HQ0iN99g1MGOMiTJNJbBFoZ+31t0S5XSVEu+2HpgxxkSZpkYh1t11+fWTEUg0S/NaD8wYY6JNUz2wIhFZAnQVkcOK8Krq+OYLK7qkxnvYXlTR0mEYY4ypp6kEdjlwNsFbpzR2Hey0keb18Hne/qOvaIwx5qRp6o7MNcCnIjJcVfedxJiiTqrXw34r6GuMMVHliAlMRJ5Q1buA6SJyWOWN0+kUYlq8h1q/crDaR2Ksu6XDMcYYQ9OnEOvuuvyHkxFINEutN5nZEpgxxkSHpk4hrgr9rCshVXd35CxV/fIkxBY1wvUQK2rIbhPfwtEYY4yByG6n8oGIJIlIGvAF8KKIPNb8oUWPlLp6iDaU3hhjokYk1eiTVbUMmAS8qKqDgYubN6zokmb3BDPGmKgTSQJziUgGcDXwj2aOJyrZPcGMMSb6RJLAHgYWAptVdYWIdAO+ad6woktSrAunQ6wHZowxUSSS26nMBebWe70VuKo5g4o2IsGCviUVdksVY4yJFpEM4ng0NIjDLSLviUihiHz/ZAQXTdK8bhvEYYwxUSSSU4hjQ4M4rgDygTOA+5o1qiiUGu+h2K6BGWNM1IgkgdXN3L0MeFVVi5sxnqiV5vVYD8wYY6LIUa+BAW+LyAagErhDRNKBquYNK/qkxHtsFKIxxkSRo/bAVHUqMAzIUdVaoByY0NyBRZs0r5uSiloCgcPKQhpjjGkBkfTAADoBY0Qktt6yl5ohnqiVGu/BH1AOVPlIjrd6iMYY09KOmsBE5L+BC4A+wALgUuBjTrMElhaazFxcUWMJzBhjokAkgzgmA6OB3ap6MzAAiGnWqKKQVeMwxpjoEkkCq1TVAOATkSRgL9CtecOKPmlW0NcYY6JKJNfAVopICvC/wCrgILC8WaOKQuFTiJbAjDEmKkRSSuqO0NNnReQdIOl0ux8Y2ClEY4yJNkdMYCJydlPvqerq5gkpOnk9TtxOobjc6iEaY0w0aKoH9scm3lPgohMcS1QLF/S1U4jGGBMVjpjAVPXCkxlIa5DmtXqIxhgTLSKpRv/T0CCOutepInJHU21OVdYDM8aY6BHJMPrbVHV/3QtVLQFui2TjIjJORDaKyGYRmdrI+yNFZLWI+ERkcr3lA0VkmYisE5EvReSaSPbX3NK8Vg/RGGOiRSQJzCEiUvdCRJyA52iNQuv9mWDljj7AdSLS55DVdgA3Aa8csrwC+KGq9gXGAU/U7wW2lNRQPURjjDEtL5J5YAuB10TkWYKDN34MvBNBuyHA5tAdnBGR2QSLAH9dt4Kq5obeC9RvqKqb6j3fJSJ7gXRgPy0oLd7D/ooa/AHF6ZCjNzDGGNNsIklgDwC3Az8BBHgXeD6Cdp2AvHqv84FzjzVAERlCsMe35Vjbnmgp8R4CCmWVteF5YcYYY1pGJBOZA8CzBCcypwGZquqPYNuNdVGO6V4kIpIBzARuDMVx6Pu3E0yuZGdnH8umj0v9gr6WwIwxpmVFMgrxAxFJCiWvNcCLIvJYBNvOB7Lqvc4EdkUaWKju4j+BX6rqp42to6rPqWqOquakp6dHuunjFq7GYSMRjTGmxUUyiCNZVcuAScCLqjoYuDiCdiuAniLSVUQ8wLXA/EiCCq0/D3hJVedG0uZkqCvoa/UQjTGm5UWSwFyhU3lXA/+IdMOq6gN+RnAQyHrgNVVdJyIPi8h4ABE5R0TygSnAX0VkXaj51cBI4CYRWRN6DIz8sJpHqjd4H7D9NhLRGGNaXCSDOB4mmIQ+VtUVItIN+CaSjavqAoI3way/7KF6z1cQPLV4aLuXgZcj2cfJVP8amDHGmJYVySCOucDceq+3Alc1Z1DRKs7tJMblsGtgxhgTBZqqRn+/qj4qIk/RyOhBVb2zWSOLQiISrIdoCcwYY1pcUz2w9aGfK09GIK1FSryVkzLGmGjQVDX6t0M/Z5y8cKJfmtdtPTBjjIkCTZ1CbHLIu6qOP/HhRL/UeA+79pe1dBjGGHPaa+oU4jCCpaBeBT6j8coapx27BmaMMdGhqQTWARgDXAdcT7Aqxququq6JNqe81HgPZVW1+PwBXM5IptEZY4xpDkf8C6yqflV9R1VvBIYCm4EPROQ/T1p0USjN60EVSittMrMxxrSkJueBiUgMcDnBXlgX4EngjeYPK3qF6yFW1NAmIaaFozHGmNNXU4M4ZgBnAf8C/kdVvzppUUWxb+shWg/MGGNaUlM9sB8A5cAZwJ31b8oMqKomNXNsUSklPlgP0QZyGGNMy2pqHpiNUGhEWr1TiMYYY1qOJaljlGq3VDHGmKhgCewYxXmcxLmdVtDXGGNamCWw45Dm9VBi9wQzxpgWZQnsOKR63ZRU1FBZWcmoUaPw+/1s376dwYMHM3DgQPr27cuzzz571O0UFxczZswYevbsyZgxYygpKWl0vXHjxpGSksIVV1zRYPnTTz9Njx49EBEKCwsPa7dixQqcTievv/46APv27WPcuHHHccTGGBN9LIEdh9T4YDmp6dOnM2nSJJxOJxkZGSxdupQ1a9bw2WefMW3aNHbt2tXkdqZNm8bo0aP55ptvGD16NNOmTWt0vfvuu4+ZM2cetvy8885j8eLFdO7c+bD3/H4/DzzwAJdcckl4WXp6OhkZGXzyySfHeMTGGBN9LIEdh9TQLVVmzZrFhAkTAPB4PMTEBCc2V1dXEwgEjrqdt956ixtvvBGAG2+8kTfffLPR9UaPHk1iYuJhywcNGkSXLl0abfPUU09x1VVX0a5duwbLJ06cyKxZs44amzHGRDtLYMchzeuhsKycrVu3NkggeXl59O/fn6ysLB544AE6duzY5Hb27NlDRkYGABkZGezdu/eExLdz507mzZvHj3/848Pey8nJ4aOPPjoh+zHGmJZkCew4pMZ7KC0uJjklpcHyrKwsvvzySzZv3syMGTPYs2dPi8R311138f/+3//D6XQe9l67du2OemrTGGNagyZrIZrGtU30IO4YDpZXNvp+x44d6du3Lx999BGTJ08+4nbat29PQUEBGRkZFBQUHHa673itXLmSa6+9FoDCwkIWLFiAy+Vi4sSJVFVVERcXd0L2Y4wxLcl6YMdhXN8OJCUlU1pRTVVVFQD5+flUVgYTWklJCZ988gm9evUC4Ic//CHLly8/bDvjx49nxozgDa9nzJgRvp72XW3bto3c3Fxyc3OZPHkyzzzzDBMnTgRg06ZNnHXWWSdkP8YY05IsgR2HNgkx3DayG3Tqz4w33gFg/fr1nHvuuQwYMIBRo0Zx77330q9fPwC+/PLL8LWu+qZOncqiRYvo2bMnixYtYurUqUCwB/Uf//Ef4fVGjBjBlClTeO+998jMzGThwoUAPPnkk2RmZpKfn0///v0btDmSJUuWcPnll3/nz8AYY1qaqGpLx3BC5OTk6MqVK0/a/g5W+8i5+3+p/nw+25b964jrlZWVceuttzJ37tyTFltTRo4cyVtvvUVqampLh2KMiQIiskpVc1o6juNhPbDjlBDj4v7rx3Eg9Qw+3HDkwRpJSUlRk7z27dvHz3/+c0texphTgiWw7+CGodn0GjmBPyz6hkAg+nuy6enp4WthxhjT2lkC+w5iXE7uHnMGa3eW8q+vdrd0OMYYc1qxBPYdXTmoE2e0T+AP726k1n/06hvGGGNODEtg35HTIdx3yZlsKyzn9VX5LR2OMcacNiyBnQAX927H2dkpPLF4E1W1/pYOxxhjTguWwE4AEeGBcWeyp6yaGUtzWzocY4w5LVgCO0HO7daGC3ql88wHWyittJtdGmNMc7MEdgLdd0kvSitree7fW1o6FGOMOeVZAjuB+nZMZvyAjkz/OJe9ZVUtHY4xxpzSLIGdYPeMPYNaf4An3/+mpUMxxphTmiWwE6xzGy/XDclm9vI8cgvLWzocY4w5ZVkCawb/eVEP3E4Hjy3a1NKhGGPMKatZE5iIjBORjSKyWUSmNvL+SBFZLSI+EZl8yHvviMh+EflHc8bYHNolxXLL+V2Y/8Uu1u0qbelwjDHmlNRsCUxEnMCfgUuBPsB1ItLnkNV2ADcBrzSyid8DP2iu+Jrb7SO7kxzn5vcLNzZYXllZyahRo/D7gxOenU4nAwcOZODAgYwfP/6o262uruaaa66hR48enHvuueTm5ja63i233EK7du0Ou3llcXExY8aMoWfPnowZM4aSkhIgeBPOK6+8kv79+zNkyBC++uorAGpqahg5ciQ+n+9YPwJjjGlWzdkDGwJsVtWtqloDzAYa3HJYVXNV9UvgsCKCqvoecKAZ42tWyXFu7rigOx9s3MenW4vCy6dPn86kSZNwOp0AxMXFsWbNGtasWcP8+fOPut0XXniB1NRUNm/ezN13380DDzzQ6Ho33XQT77zzzmHLp02bxujRo/nmm28YPXo006ZNA+B3v/sdAwcO5Msvv+Sll17iv/7rvwDweDyMHj2aOXPmHPNnYIwxzak5E1gnIK/e6/zQstPGjcO70D4phkff2UDdjUNnzZrFhAkTjtLyyN566y1uvPFGACZPnsx7771HYzclHTlyJGlpaU22v/HGG3nzzTcB+Prrrxk9ejQAZ555Jrm5uezZE7zP2cSJE5k1a9Zxx2yMMc2hOROYNLLshN40S0RuF5GVIrJy3759J3LTJ0Ss28ldF5/B6h37Wbx+LzU1NWzdupUuXbqE16mqqiInJ4ehQ4eGk0lTdu7cSVZWFgAul4vk5GSKioqO0upbe/bsISMjA4CMjAz27t0LwIABA3jjjTcAWL58Odu3byc/P1ic+KyzzmLFihUR78MYY06G5kxg+UBWvdeZwK4TuQNVfU5Vc1Q1Jz09/URu+oSZMjiTbm29/H7hBvbs3UdKSkqD93fs2MHKlSt55ZVXuOuuu9iypekqHo31tkQa+65wbKZOnUpJSQkDBw7kqaeeYtCgQbhcLiB4nc7j8XDgQKs9o2uMOQW5mnHbK4CeItIV2AlcC1zfjPuLSi6ng3sv6cUds1azeFMxVVUNK3R07NgRgG7dunHBBRfw+eef07179yNuLzMzk7y8PDIzM/H5fJSWlh52qrCyspJx48Yxffp0AB544AH++c9/AhAfH09BQQEZGRkUFBTQrl07ANxuN1u3bmXVqlXU1tbi9Xo5++yzSUxMZM2aNVRXVxMbG8vTTz/NE088wZYtW5gyZQpvvvkmPXv2ZN26dYwbN45PP/2Uc889l0AgQG5uLl26dKG8vJxVq1ahqgQCAXr16gWAw+EgLy+PIUOGsHfvXmpra1m/fj2XXXYZS5cupbw8OI8uISGBlJQUXC4XiYmJbN68mfPPPx+Px8M//vGPcLy1tcEalIFAAIfDQWZmJl6vl4SEBL7++mu6d+/O9u3bKSoqonPnzrRp04Z169YRCATw+XxkZ2ejqsTHx+NwOHA6naSkpLB8+XKqq6s588wzKSoqYt++fcTHxxMTE0Nqaio+n++IbVJSUqioqAh/9vHx8cTGxjZoc/DgQUpLSyktLaVbt26UlJRw4MABnE4nfr8fEaFLly7cddddzJw5k7KyMvLz8ykvL8fv93PllVeydu1aduzYEf4MYmNj6dSpE3Fxcfj9/vCXkK+++or2UfnPAAAgAElEQVTq6mpGjRrFtm3b2L17Nz6fj5SUFFJSUlBVnE5n+POs//tv27YtJSUlBAIBBg8eTEFBATExMQ3aFBUVUVpayoEDB+jWrRt79wbPPKgqfr+fjh074na7G+zD4/Gwbt06amtradOmDT6fj8rKSmpra0lOTiYtLQ23242IEBMTQ21tLZWVleTl5YU/R5/Ph4iEt9exY0cSExPDx75nzx5KSkqorq6mR48exMfHs2FD8NS+z+eja9eueL1e/va3v9GjRw969+6Nw+Fgz5491NbWMmDAAHJzcykrKyM2NpbY2FhSU1ObbJOUlBT+/56QkIDH4yE+Pr5Bm+zsbCorK6mpqaF79+7s3r2bmpoaHA4H1dXVZGVlkZKS0mAffr+foqIifD4fbdu2xeVyUVxcTE1NDW63m6ysLNq2bcucOXNIS0ujd+/e+Hw+9u/fT01NDX369GHz5s0AJCYmAmSJyGagArhJVVeLyM+A3wApwDfAPqA94AdSgYOhx02quhpARBYBIwEP8L/Aj1RVRWQxsAUYXX8foTaPApcT7EwtAv6rXpspqlpyxD+GEPxG31wP4DJgUyj4X4SWPQyMDz0/h2BPrRwoAtbVa/tR6EOrDK1zSVP7Gjx4sEarQCCg45/6SHv/6l/atn1HraysVFXV4uJiraqqUlXVffv2aY8ePXTdunWqqjp16lR94403DtvW008/rT/60Y9UVfXVV1/VKVOmNLrOE088odu2bdPs7Gy9+OKLtba2Vg8ePKjt27fX//7v/1ZV1UceeUTvu+8+VVV99NFH9Q9/+IOqql5//fXaqVMnnT9/vg4aNEgnTJigZ555pqqqrl69Wrdt26adO3fW22+/Xe+55x7t27evqqouXrxY58+fr926ddNHHnlEVVV/9atfaVJSkhYVFem4cePU6XTqQw89pHv37tWEhAT9+9//rpdffrlOnDhRX3nlFR01apSOGjVKs7OztaqqSn/605+q0+nU8vJy3bp1q2ZkZOi8efN09OjRmpGRoevXr9fMzEyNj4/Xp556SouLi9XhcGhcXJwGAgFdtmyZ9urVS+fPn68DBw7UQYMG6ciRI/Vf//qXAvrMM8/oPffco4COGzdOFy9erFlZWerz+XTlypXqcDj03//+t44aNUrPPvtsTUtL06ysLJ0wYYKKiBYUFByxzfDhwzUmJiYcl9vtVpfLpVVVVQ3azJ07Vx0Ohw4bNkznzJmjbrdbf/SjH+nChQvV4XDoOeeco1u3btXMzExdv3695ufna7t27bRt27aalZWl48aN0/T0dC0qKtJLL71UAf3zn/+sc+bM0f79+6uq6ptvvqkul0uXLVumI0aMUJfLFW7TrVs3BbSgoEA3bNigXbt2VZ/Ppxs3blSn06nvvvuuDh8+XD0ej/72t79Vr9erCQkJGhcXp1VVVQ3aLFiwQJ1Opw4ZMkTnzJmjSUlJ+txzz+kdd9yhgI4cObLRfXz44YfauXNnTUxM1O7du2sgENARI0aow+HQQCCg//73vzUnJ0dVVRctWqRut1tXrFihcXFxGh8frwsXLtStW7eqiGi7du1UVXXmzJl61VVXqarqQw89pHFxcdqpUyf9+9//roAuXbpUb7jhBgX0/vvv12XLlumQIUP0zjvv1DFjxmibNm108+bN2rlzZ+3SpYtmZ2er1+vVoUOHqsPh0Nra2iO2ycjI0NjYWF24cKFu2bJFHQ6HJiYmhv9N1rU599xzNT09XTt37qwvv/yyOp1O/ctf/qJXXnmlut1u/c1vfnPYPtLS0jQvL0+9Xq/2799fu3XrplVVVRobG6sOh0N9Pp+++uqrevXVV+udd96pw4YN06ysLN22bZu2b98+fOxbt25Vl8ulQBXBSz5Dgc80+Pf3RmAlkAucH/obPQa4GagGfnnI+sOBNUC3UJsVwAWh9/4YygOH7mM48AngDD2W1WtzY13OaOrRrPPAVHWBqp6hqt1V9f+Glj2kqvNDz1eoaqaqelW1jar2rdd2hKqmq2pcaJ2FzRlrcxIRnvthDj3bJVDZri/3PjUbVWX9+vXk5OQwYMAALrzwQqZOnUqfPsGZBmvXrqVDhw6HbevWW2+lqKiIHj168Nhjj4VHEe7atYvLLrsMCA4UWbx4McOGDSM/P5/PPvuMGTNm4PV6ufjii3n99dfp2bMnixYtYurU4PS8l156iaeeeoozzzyTxYsX88ILL5CYmEhGRgbvv/8+l156KQCDBg0KX8P7/PPPmThxYji20aNHk5iYyJ49e8IDRTp06ICIkJaWxtdff43X62XTpk0cPHiQ9u3bh0+p3nTTTXz00UcA7N69m/POO4+YmBjefvttsrKy+OKLL+jatSv9+vVj586dQPDUZrt27aiqqiI1NZUzzjgDl8tFIBBARPD7/QwdOhRVpaamhvj4eEQEVeW9997D4XBwwQUXMHfuXLp27coHH3zA6NGj6d27N8uXL6dDhw44nU5iY2NRVQ4ePMjQoUNxOBysXLkSl8vF4sWLj9imuLiYlJQUzjjjjHDvpi7u+m2GDRuG0+mkurqaJUuWEBcXR3p6OqtWrcLj8bBt2zaysrLo06dPeNqDx+Ohffv2+P1+8vPzOeecc0hLS2PlypVAsEd99dVXU1VVFe5pu91uvF5vuEd0wQUXkJaWxoEDBxARlixZQq9evejZsyfLly+npqYGCJ4lKC4uJjU1lXPOOQf49ouv0+ls0Kbu+mwgEGDJkiVkZGTQuXNn3n77bWJjY/niiy/o3r37Yfto06YNEJwmcv755yMibNwYnIKSn5/PiBEjKCsro6CgIDylo673m5KSQlZWFj6fL9zLB7j22mv54IMPUFU6deoU/vz//ve/43a76dWrF4sXL6Zt27bMmDGDoUOHsnv3bnJzcxkyZAgOh4OOHTuiqlRVVTFp0iQAtmzZgsvlYsWKFUdsU1FREY4rKysrHJeINGhz8cUXhz/PBQsW4Ha7adOmDUuXLiU5OZmZM2cetg+n0xn+vAoLCxk/fjw7duyguroal8vF8uXLmTx5MgsXLmTPnj30798/fEbi4MGDuFwuevXqRdeuXYmLiwNwhb77fgqkiEgG3yYcAa4ASgh2JIYS7JHdfMj6SrB3VhBq4wb2hP40pAFxjexDgViCPbaYQ9rMB67jKKwSx0nSPimWOT8axveuu5m/zXiJO2ev4exzzmXt2rV88cUXrF27lltvvTW8fm1tLcOGDTtsO7GxscydO5fNmzezfPlyunXrBgT/yCxYsCA8UOTtt9+moKCAf/3rX/Tr14/rrruOwsJCli5dys0338w333zDe++9R1paGjU1Nezbt4/c3Fw2bNhAeno6ffsGv0uICIFAgKuvvrpBHKpKbm4umZmZh8VYXV0dHihSUVFBdXU1P/zhD8nLy6OmpoZ//OMf9OvXjz/96U84HMF/gjk5OeEEVlVVxZIlSzjrrLPYtWsXmZmZ4aSVmZlJYWEhsbGx3HvvvWRnZ1NUVERycjJjx45l7ty54YT5xRdfNGiTmprKhRdeyNKlS/nTn/6E2+1m//797Nu3D4fDQW1tLUVFReH97dy5k9TUVC644AKWLl0KwLBhw1DVcHKqSxiNtdm0aROJiYm8+uqr4WRzaFz123z++edMnz6dmJgYXnvtNf7whz9QU1ND+/bt+eKLL8Lrd+rUiSlTpvD1119TUFAQjqumpoaSkhJiY2NZt25dg304nU7i4uI4//zzWbFiRfiPYF0bgE2bgpVjUlJSuPbaaxkyZAgTJ07k3HPPDR/L2LFjw18IkpOTw8dyaJvVq1czffp00tLS+P3vf09eXh6JiYnhLyON7SMvLw9V5aOPPqJ///4UFRURHx/PBx98AEDbtm0ZOXIkkyZNYsSIEQwePJja2lpycnLo3bs3M2fODP+bLSoqwuVy4fV6Oeuss7jrrrvo06cPBQUFzJ49G7fbTWFhISUlJTidTgoLCwkEApSVlXHbbbfRo0cPOnXqREZGBnl5ebRv357BgwejqpSVleFyudiwYcMR25SWljJgwADuuusu0tPTUVU8Hg9FRUWNtsnLy2POnDkkJyfz61//mr179+J2uykvL2ffvn2N7qO8vJyMjAwGDx4cPnaPx8OGDRvCpyAffPBBzjnnHNxud7iN2+2mrKwMn89X92XAWe+/bz7QSVWXAUsIjmP4ObBKVdcTHEmeC8SLSJtG1i8ItXk/tD5Aeuj30uYI+ygIPRbWtdHgqcOYem0aZQnsJIp1O3ll6rVMvGwMb6/J4+q/LmN3aeNV6xcu/LbDWX/y85IlS8ITnwcOHEhsbGyD0YuFhYUNBoqMHTuWsWPH0qlTJ7KzsykrK6OsrKzBvura1E1+rjs/DsFv0nFxcTzwwAMNJj/7/X4CgQCXXXYZW7Zs4ayzzsLpdIa3XTf5ue5b77Rp08jIyCA5OZnLLruMNWvW8LOf/Sx8natdu3bs2hUc43PWWWfx6KOP8vbbb4evkx06UKW2tpa33nqLbdu2kZycTG1tLS+//DIzZ87E4XAQFxcX3l6d8vJy1q9fz7Bhw7j11ltJSkrilltuCV87cDgc4Tb79+/nhhtuIDMzk9zcXIYPH05tbS1r1qzB7/fj8XiIjY1t8Fke2qZz587U1tYyevRoVq1ahcPhwO/3N4irfpthw4ZxxRVX0KFDB375y19y2WWXISIcOHAg3EZEWL9+Pc8++yz//Oc/yc7ODsdVWFiIqpKUlNQgrqKiIq699lp69erFzp07GT58OF6vl3nz5oW/0TscjvAgnYSEBB577DFWrlzJu+++y2effRY+lpdffjnc84qNjQ3HdWibs88+myuuuILq6mrGjx9PcnJyuNe2a9euRveRlZVFYmIi119/Pa+//nr4319hYSEQ/PI2e/ZsFi1axKpVq1i9ejUul4t169bx73//m1dffRURITExMRyXy+UKv5ebm0vHjh255ppr8Hg8XHnllQ2uRT722GO0adOGjIyM8DWz/Px8srKy2LVrV/iamcvlwuPxUFJSwjPPPNNom4SEBDZu3MgvfvELVq9ejYjgcrnYtWtXo22ysrIYOXIkSUlJ/OY3vyEuLo6DBw/icDh4/PHHG92H1+slPz+fDRs28Oqrr+JyuYiJiQnHFR8fT2ZmJlu3bkVEwp+7x+Nh4sSJjBgxom4+qopIYr3/KioiPYDeBKdCvQr0EZGRfDu6vBTo2Mj6maE2I0LrE2pTUm/9xtp0Ai6q1wZg7yFtDmMJ7CQTEV5+9EGev/Fctuw9yPinP2ZN3v4m29Sf/HzhhReGJz6///77xMfHM3bs2PC6cXFxhw0USU9P55prrqGiooJevXqxZMmSBu/Xtamb/OxyucjLywvHW15ezrhx4xpMfhYR4uPjWbBgAd27d+eRRx5h1KhRJCUlERMTw5AhQ5gzZw5er5eYmBji4uIoKSmhW7dufPPNN/To0YOuXbuyY8cOINjrCp3OoHPnzuTn55OSkoLH42Hfvn3hwS75+fm0bduWwsJCunbtSnp6Oi6Xi3bt2vH++++zZs2a8OnDuu3Vtdm9ezdDhw7F6XSSmZlJdnY211xzDYmJiQQCAdxuN3FxceTm5vL73/+e733vewwcOJD09HREhLPPPpv169eHk+nBgwfDp3kbaxMbG0vbtm1ZunQpCQkJ4c+yLq5D27jdbjp06ECvXr1YtmwZXbt2De8nLi6O/Px8kpOTufzyyxk6dGj4tO6wYcNYv349lZWV4S8WdXFt376de++9l/POO49LL72UhIQEnE4ngwYNonPnzixZsgS32w1Ap06dwp9Xx44dWbduHR06dGD9+vUNjqWuV37oZ1y/TUVFBR06dKBbt27h34nT6aSkpCR8LIfuA6B37958+umn4d9rdXV1g999x44dWbp0Kd26dQsnhi5duvDGG2+ET9HV1tYSFxfXYJDTu+++S9++famqqqJbt250796de+65h/j4eGpqahARVq1axfbt25k4cSIPP/wwu3fv5re//S0iQu/evVm1ahUigohQVVVFVlYWy5Yta7SNy+UiOzubTz/9lNTU1Aa/+8ba7N+/n6ysLAYMGMCXX36Jw+HA4XBQXl7OV1991eg+IPhlb8mSJQQCAZxOJ5WVlWRlZfHJJ5+wf/9+Bg8ezGOPPUZeXh6/+tWvcLlcdO7cmf/zf/4Py5YtC480JngdDL4dLX4l8CnB03w7CPa6htJwdHnloeur6sFQm/dC6xNqkxBa/7B9qOrBULt/1WsDwdOLlTTBElgLubhPe9644zxi3A6u/usy3vx85xHXPdLk59dff51LL72U+Ph4AHILy1n4zQEOVtbw2qdb+OeXBbyzdhfPv/QKZ4+ewOx3PmJPYTFfrv2KzXsP8NO77uXFWXPwu+Op9fno3mcAEuMlISGRvz4/ndLKGvJ27sThdDJ+ynUUHazme5Ov5e9vzEMcTnw+P+UVwX9fTz/9NJWVweft27entraWWbNmUVBQEC6bVfdHYvjw4ezZs4c1a9awePFiIHj6qq7sVb9+/Zg9e3Z4BF1lZSUDBgzgzTffZOnSpZx55pnExcXx6aefUlFRQWxsLGvWrKGkpIRx48YRCATYvXs3ffr04YUXXqCoqIg2bdoQFxfHhx9+iKqSk5PDV199Ra9evXC73WzatImYmBiqq6v5+OOPSUxM5JprrgnvA6C0tJTdu3cjIrjdbkpKSrj66quZO3cuH374IT/60Y8atElJSeGrr76iXUYmpeU1+P1+qkOjzZ5++unD2gQCAUaMGMGiRYvo0aMHw4cPD392iYmJrF27lh/84AdMmDCBPXv2hOMqKSlh9+7dvPXWW3g8HgoLC7nhhhv46KOPKCgoIDY2liuuuIIPP/ww3Cves2cPubm5VFdXU11djd/vZ/LkyTz//PN88sknDBkyhJSUFLZv305WVhbJKSms+/pr0jsEk5zP52PPnj20ycjil//zMMs+/ZTufQbgiPGSm7sdl9tD734DWfjuu3TteSaO0EjAWp+f3cWlLF+xgg8+XhZeP7ltBv5AgKrqGnbk5VOlLvx+P36/n75nD+Gue+5jf2kpnsQ0PN4kNmzYQNuMLJwuF59++infbNnG6LHjANi7bx+JbTO45bbbSUlNo7i8Bm9SKmvWrMHpcjFoyHC+XLuW7B5n4nQ62blzJ+07ZHDTbT8hK7szV119LVdfdwNt2rTl7qm/wh9Q9peV8fXX60EEl8tNVVUVwy8cS/devWnTti1r1m9u0Mbt8bBixQraderMN9t3EggEqKyqIqFNB1LapNMho2ODNknJKeQMO59FixbTo3c/HA4nBw8epNbn48f/eXejcSlQUlrGV199xZVXXY3H46GiooLx48fTsWNHzjjjDHJzc7niiivo0qULjzzyCLGxsaxfvz78xSLU61bAJyIPAImqWkAwaY0K/alZxLfXvj4DegKJBEclltZfX0TqMuJwoO4U4nyCPalcERnaWBsRcYf2tz70t0KADgQT5xFJ3Sme1i4nJ0frrke0JsXlNfzk5VV8tq2YH4/qzn2X9MLp+PZ0WU1NDdnZ2ezevfuwthdddBG33fGf+DoNYt7nO1m9I9iTK1zwJ7x9RhHXZSDqq2HHE1fjSmqHIzaRNpf8lH1v/o4OP/gjRQueIHnY1cR06h1u407NYM/cXxOoKCVQdRBUAaXdlP/Bnd6FPXN+ia8oH+pO6YkDUAj4cXjicLg8+GsqQRX1+0jrMZCqkt1U7t+HBvw43TEkts3A4XBQuicPQUDq5rIJgYAvtM/Q9lXp2f8civbspLa6irL9xcR5E6ipqgwNzfahgQDO0OCNpNS2pLXPYPeOrSQkpVJavA+n04XT7aby4AH8vuAwc4fDGWrjJ+D3k5yWTkX5AeLiEygt3kecN5G2GZkU7s6nqqIcDSgOZ/D7XiCUVBwOJ0npGVQc2E9NxUES2mUhrhgqigvwV1eC1lVIE3A4cXjiUBRnbAK+0n2gAeI6dCVQXU71/r2gijgcqII4HAiCOJ34fTUkpGdSW15GdXkpyZ16UL5vJ77qYAJzumNIyT6Dkh0bw/uLS26D31dL1f59uGK9eNtmBNvUVIMGEIcTEQcaqi3gjInDFZtIbeUB/NUVeFLaIy4P6nBRs2cbqD/0Own9vXC4QBy4EtPwle1DXB5cye0JVJfjL98P/trg+gKIE9SPIz6VQGUZDm8ygYMlOL2pOOIS0UAAX/HO0D4Apzv0bwBwOHElpOKvPIDWVOJumw2qBKoOBvej/mAs6seZlI4zLhnf/t04YhPwle3F4YnHmdgGf0UpgcoyCPhBHIjThQb8EAjgiE9G/TW4EtMJ1FTi7T0SV1pH9n/4EoHKA6F9hC4TBUIxOty4ktoQqKlCXG46fP8PVG5b1bANBP+fOJw4YhKCzTyx+Mr2kZgzkbQLb2bf/Eep2PgxhAaeII7gQ/04YhMI1FbjTsnAX15C0jkTcXhT2L/kRQLVFaB+xB2LBgLEJibjrK3A4XDQrl07qqurcbvdfPzxx8yfP59f/vKXlJaW4vP5wtedXS4Xl156KW+99VYFsJvgabxXVPUWEfkv4P8CXsBHcJR4eej5HuBsYBtQBkwJLfs3wRKCrtC6s1X1P0QkB6irRVdBcADIylC93GcIDr1X4B1V/XnwY5Mc4EFVveqwP3z1WAKLAjW+AL9+ex2vfLaD0We244lrB5IYGzyts2vXLi666CI2bNgQXr+q1s/cf6/l9isvpONPZuAXJ2d2SOTKQZ0Y27cDm77+kmeffpJpTz5HrT/AhAvP5fEX55KS3oEaX4AfXHIuj878B3988Kfc98RL+APK9o3reGf280y6/R4ev/dWfjfr22twPxnbn7+8++W3ry8ZwJML1rB1w1renfMCvc65gH+9+DhXP/gEqVk9qfUFqPEHeP6OS7jqd6+h7jhqfAH25W5g0+LZ9L3+F9T4A3w167d0uvTHaEwS30y/h/aTfomG/qOHP5s9Wyhb8SZtr7iHkiXT8fa9EE+7rk1+nsXvP0/1zvVk/OCPEbWp2bOFffMfJW3MT4jrMjCiNsXvP49v1wb6/uQp9ix6nq5Dx5HRrRcJMS68MS4SQg9v6CHAK0/9li1rV/OzJ2bz8iP3MubW+3F7U6j1B6gJfWY1PqXGH6DWF0BRPp/7JEVb1nLx1P9lzetP02XoOFIyexwxrvrrA0dt43I4WPq/D1FTXsKVv/gry159ggEXfI/M7mfidjrwuIIPt9NBTOinCLzy5P9l89pVPPS/b/Dsr+/i+v/6FUmpR77evn3jOqZPm8pFk77PqO9dw2P33sqdj/wFl9tzxDbHug+AZx66kwP7i3ngyZcjjuuJB27j/j/NJKNz94jaHOs+Dm0TybFv37iOv/z3nfzg3ofpm3PeUffTNiGGmb/7OYWFhSxatIjvf//7PP744xytwMOkSZOYN2/eV6raT0ReBu5W1SbLGonIn4D5GqxXe1THuv4xtTnaOPvW8ojmeWCRemnpNu324D91zGMf6PbCclUNzhXr3Lmz+v0BXbq5UO+f+4We9dA7mjr6Nk3PuUx/98+v9etdpYdt64UXXlCfz6eqqmPHjtWlS5eqqmptba22adNGA4FAo202b94cntdV54wzztBdu3apququXbv0jDPOaNBmwoQJOmvWrMO216ZNG62pqTliXHX27t2r8+bNU1VVvz+glTU+La2s0eKD1Vp8sFqffOavuq+0Ivz6aI+n/vKcPvHUM8fU5trrvx/x+qWVNfr88y/oc889d9ixNGX69OnN3uZ49vHcc881+ns50fu5+eabw/Mem2sfx3os1dXVeuuttzb753U8bX74wx82+7HPmDFDgZV6bPN7b2vO9Y+ljfXAoswnmwu5Y9ZqHALP3DCYNgkehpx1Bmf+53R2VwTwepyMOyuDd353K3/646NcPPqicNsHH3yQIUOGcOWVVzbY5p///GfWrl3Ls88+y+zZs3njjTd47bXXGt1/3TnzutupANx33320adOGqVOnMm3aNIqLi3n00UeB4HWhrl27kpeXh9frDbcpKiri/PPPD1+cN8ZEJxFZpao5LR3H8bBBHFHmvB5teeun59EmIYbrn/+UsY//m0DHfiSXbeHJ6wax8pdj+M8hyezfV8BFF17QoO13mfwMcN111zFs2DA2btxIZmYmL7zwAhCsk7ho0aLDJj8DzJs3j7FjxzZIXgBLlixpsG1jjDnRrAcWpcqqann2gy2kJ8bQmb28+OzT4cmKR3LJJZc0mD/WkiZNmsQjjzwSrntojIlOrbkH1pzFfM13kBTr5v5xZ4ZedSV347pwYdIjiZbkVVNTw8SJEy15GWOalfXAjDHmNNaae2B2DcwYY0yrZAnMGGNMq2QJzBhjTKtkCcwYY0yrZAnMGGNMq2QJzBhjTKtkCcwYY0yrdMrMAxORfcD2o6zWFig8CeFEq9P5+E/nY4fT+/jt2JvWWVWbLlsfpU6ZBBYJEVnZWifsnQin8/GfzscOp/fx27GfusdupxCNMca0SpbAjDHGtEqnWwJ7rqUDaGGn8/GfzscOp/fx27Gfok6ra2DGGGNOHadbD8wYY8wp4rRJYCIyTkQ2ishmEZl69Batm4hM///tnX+wVVUVxz9fQMdSJ/LnoFgog6M2ySsHoqRAxkFKBhxHRCSEhqHoB1EjlVaD5gyMxlD+gUZpBU6Q0A/SaZyUIUDsxwNBCAqNEioH8mEhKI00wLc/9r56uN0H7w1c7rv3rs8Mc89eZ+991nrsd9dZZ5+3lqQ2SVsKsrMkLZe0LX++s5Y6VgtJF0laKWmrpD9Kmp7lDW+/pNMkrZW0Kdv+jSy/WFJrtn2JpFNrrWu1kNRd0nOSfpnbzWT7DkmbJW2U9GyWNey6bwoHJqk78ADwUeAKYJykK2qrVdVZAIwok90BrLDdD1iR243IQeB225cDg4DP5v/vZrD/AGzqR+MAAAbwSURBVDDMdn+gBRghaRBwH/DtbPseYHINdaw204GthXYz2Q5wje2WwuvzDbvum8KBAQOBv9h+0fZ/gUeB0TXWqarYfhr4d5l4NLAwHy8EbjipSp0kbO+yvSEfv0b6MruQJrDfiddz85T8z8Aw4KdZ3pC2A0jqDVwPPJzboklsPwoNu+6bxYFdCPyj0H4py5qN823vgvQlD5xXY32qjqQ+wPuAVprE/vwIbSPQBiwH/gq8avtg7tLI6/9+4MvA4dw+m+axHdLNylOS1kv6ZJY17LrvUWsFThKqIIvXLxscSWcAPwO+YHtfuhlvfGwfAlok9QSWAZdX6nZytao+kkYCbbbXSxpaElfo2nC2F7ja9k5J5wHLJT1fa4WqSbNEYC8BFxXavYGdNdKllrwsqRdA/myrsT5VQ9IpJOe1yPbPs7hp7Aew/SqwirQP2FNS6Ya1Udf/1cAoSTtI2wTDSBFZM9gOgO2d+bONdPMykAZe983iwNYB/fLbSKcCtwCP11inWvA4MDEfTwQeq6EuVSPve3wf2Gr7W4VTDW+/pHNz5IWktwHXkvYAVwI35W4NabvtO233tt2H9Dv+a9vjaQLbASSdLunM0jEwHNhCA6/7pvlDZkkfI92NdQd+YHtWjVWqKpJ+DAwlZaN+GbgL+AWwFHgX8HdgjO3yFz3qHkmDgTXAZt7aC/kqaR+soe2XdCVpo7476QZ1qe17JF1CikrOAp4DPm77QO00rS75EeIM2yObxfZs57Lc7AEstj1L0tk06LpvGgcWBEEQNBbN8ggxCIIgaDDCgQVBEAR1STiwIAiCoC4JBxYEQRDUJeHAgiAIgrokHFhQcyRZ0txCe4aku0/Q3Ask3XTsnsd9nTE5+/3KMnmfbN+0gmyepEnHmG+qpNuO0WeSpHntnHu9kvxEke0qVjqYImlDI2U6D7o+4cCCrsAB4EZJ59RakSK5ikFHmQx8xvY1Fc61AdM7U8bD9nzbj3Ti+ieMQtaKjvafAEwDhtveUx2tguD/CQcWdAUOkkqff7H8RHkEVYosJA2VtFrSUkl/lnSvpPG5FtZmSX0L01wraU3uNzKP7y5pjqR1kv4g6VOFeVdKWkz6Q+hyfcbl+bdIui/LZgKDgfmS5lSwbzepjMXE8hOS+kr6VU6+ukbSZVl+t6QZ+XhA1vF3WecthSkuyOO3Sfpm2dxzc1S0QtK5WdYi6fd5vmWliEnSKkmzJa0mOdsx2cZNkp6uYFPpGjeTynMMt/1Ke/2CoBqEAwu6Cg8A4yW9oxNj+pNqP70XmABcansgqZTGtEK/PsAQUpmN+ZJOI0VMe20PAAYAUyRdnPsPBL5m+4iacZIuINWWGkaqtTVA0g227wGeBcbb/lI7ut4L3F4hqvseMM32VcAM4MEKY38ITLX9QeBQ2bkWYGz+GYyVVMr5eTqwwfb7gdWkTCwAjwBfsX0lyUHfVZirp+0htucCM4Hrcl2xUe3Y9G5gHsl5/bOdPkFQNcKBBV0C2/tIX66f78Swdbn21wFSyZCnsnwzyWmVWGr7sO1twIvAZaQ8cbcplR1pJZXd6Jf7r7W9vcL1BgCrbO/O5TkWAR/poH3bgbXArSWZUrb8DwE/yXp8F+hVHJfzGp5p+7dZtLhs6hW299p+A/gTyalASqG1JB//CBicbw562l6d5QvL9F9SOP4NsEDSFFJaqkrsJqUmurldw4OgijRLOZWgPrgf2ECKOEocJN9o5SS9xX2kYj67w4X2YY5c2+X50kwqszHN9pPFEzmH3v529DveeiyzSYUVS4/kupFqVbUcZcyxrln8GRyi/d/pjuSMe9Nu21MlfYAUtW6U1GL7X2X9/0Oqcv6MpDbbizpwjSA4YUQEFnQZcoLRpRxZ8n0HcFU+Hk2qMNxZxkjqlvfFLgFeAJ4EPq1UdgVJl+YM3kejFRgi6Zz8KHAc6fFch7D9PClKGpnb+4DtksZkHSSpf9mYPcBrkgZl0S0dvFw33srAfivwjO29wB5JH87yCe3pL6mv7VbbM4FXOLIcUVG/3cAIYLak6zqoWxCcECICC7oac4HPFdoPAY9JWkt6EaK96OhovED6oj6ftJf0hqSHSY8ZN+TIbjfHKLVue5ekO0nlOQQ8YbuzpSlmkTKilxgPfEfS10nO+VFgU9mYycBDkvaT6nvt7cB19gPvkbQ+9x+b5RNJ+4BvJz1O/UQ74+dI6keyc0UFnd7E9nZJo4AnJN1ou7UD+gXBcRPZ6IOgiyPpDNulty/vAHrZnl5jtYKg5kQEFgRdn+tz5NcD+BswqbbqBEHXICKwIAiCoC6JlziCIAiCuiQcWBAEQVCXhAMLgiAI6pJwYEEQBEFdEg4sCIIgqEvCgQVBEAR1yf8A6gd0phvps7UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.15  0.114 0.109 0.107 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108\n",
      " 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108 0.108\n",
      " 0.108]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 2-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_kd = KNeighborsClassifier(n_neighbors=k,algorithm='kd_tree')\n",
    "    print(k)\n",
    "    scores_kd = cval_score(knn_kd, d_train_dense,y_train, cv=2, scoring='accuracy')\n",
    "    cv_scores.append(scores_kd.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.036\n"
     ]
    }
   ],
   "source": [
    "knn_kd_optimal = KNeighborsClassifier(n_neighbors=7,algorithm='kd_tree')\n",
    "knn_kd_optimal.fit(d_train_dense,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_kd_optimal.predict(d_train_dense)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_kd_optimal.predict(d_test_dense)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.996\n",
      "The value of TNR is 0.039\n",
      "The value of FPR is 0.961\n",
      "The value of FNR is 0.004\n",
      "\n",
      "Test error is 10.967\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGf5JREFUeJzt3XmUVNW59/Hvr2mN0BhBZgHFgTjm4hyvQ5bTwlniVaMmUaLkcjVexzggxuur6BvUVxONxgSVK04oanCMA6BcjdcJlSgKCg5Ii4ACGsEBuvt5/6gDFthdFNLVVb39fVxnddWuXefswl5PPf2cffZRRGBmZq1fVbkHYGZmzcMB3cwsEQ7oZmaJcEA3M0uEA7qZWSIc0M3MEuGAbmaWCAd0M7NEOKCbmSWiutwDaEr12j19Cat9wxYde5d7CFaBpsx9Tmu6j6Ufv1N0zFmr8yZrfLxScIZuZpaIis3QzcxaVEN9uUewxhzQzcwA6uvKPYI15oBuZgZENJR7CGvMAd3MDKDBAd3MLA3O0M3MEuGTomZmiXCGbmaWhkhglosvLDIzg9xJ0WK3VZD0nqTXJE2WNClrW1/SOEnTs58ds3ZJukbSDEmvSto+bz8Ds/7TJQ1c1XEd0M3MIFdyKXYrzl4RsW1E7Jg9HwJMiIi+wITsOcABQN9sGwxcD7kvAOBC4EfAzsCFy74EmuKAbmYGuZOixW7fzgBgVPZ4FPCTvPZbIuc5oIOkHsB+wLiIWBARC4FxwP6FDuCAbmYGq5WhSxosaVLeNnjlvQGPS3op77VuEfEhQPaza9beE5iV997arK2p9ib5pKiZGazWpf8RMQIYUaDLbhExW1JXYJykaQX6NrZyYxRob5IzdDMzaNaTohExO/s5DxhLrgY+NyulkP2cl3WvBfLXhe4FzC7Q3iQHdDMzIKK+6K0QSTWS1l32GOgPTAEeAJbNVBkI3J89fgA4LpvtsgvwaVaSeQzoL6ljdjK0f9bWJJdczMygOS8s6gaMlQS5GHtHRDwq6UVgjKRBwPvAkVn/vwEHAjOAz4HjASJigaRhwItZv4sjYkGhAzugm5lBsy3OFRHvAP0aaZ8P7NNIewAnN7GvkcDIYo/tgG5mBr7038wsGfVLyz2CNeaAbmYGXg/dzCwZLrmYmSXCGbqZWSIc0M3M0hA+KWpmlgjX0M3MEuGSi5lZIpyhm5klwhm6mVkinKGbmSWirvgbXFQqB3QzM3CGbmaWDNfQzcwS4QzdzCwRztDNzBLhDN3MLBGe5WJmloiIco9gjTmgm5mBa+hmZslwQDczS4RPipqZJaK+vtwjWGMO6GZm4JKLmVkyHNDNzBLhGrqZWRqiwfPQzczS4JKLmVkiPMvFzCwRztCtufXqtQE3j7yabt270NDQwI033s4fr70JgJN/fTy//vXx1NXV8cgjExhy3qVlHq2V0rrfb89FVw1lsy02gYALzriEPfbZlb33/zENDQ0s+Hgh5586jI/mfkz7dWsY/qeL6NGzG23atOHm62/nvjsfLvdHaF0c0K251dXVcfY5F/HK5Cm0b1/DC88/yvgJT9GtaxcOPWQ/ttt+X5YsWUKXLp3KPVQrsSGXnMEzTz7Hmb8aSvVa1bRtuw4zpr3DtZeNAODnv/opJ/3mBC4+53KOOeEI3n7zXf7z2LPo2KkDDz1zFw/d+xh1S1v/CoItppkX55LUBpgEfBARB0vaGLgTWB94GTg2IpZI+h5wC7ADMB84KiLey/ZxHjAIqAdOjYjHCh2zZAFd0hbAAKAnEMBs4IGImFqqY6Zgzpx5zJkzD4BFixYzbdp0em7QnUGDfs7lV1zHkiVLAPjoo/nlHKaVWE37duzwr9tx/qnDAKhbWsdnSxet0Kdtu3WWx6CIoKZ9OwDa1bTl00/+SX1d668Jt6jmz9BPA6YC38+eXwb8PiLulPRncoH6+uznwojYTNLRWb+jJG0FHA1sDWwAjJf0g4ho8n9sVXN/AgBJ55L7JhLwAvBi9ni0pCGlOGaKNtqoF9v224bnX3iFvn03Yffdd+Z///4gT4y/hx136Ffu4VkJ9dqoJwvnL+SSqy/g7vGjuOiqobRttw4Ap553IuNfvp+DDt+Pay/PZet33HQPm/ygD0+++hBjJ97O8N/+nkhgOdgW1RDFb6sgqRdwEHBj9lzA3sA9WZdRwE+yxwOy52Sv75P1HwDcGRFfRcS7wAxg50LHLUlAJ/eNs1NEDI+I27JteDaYQSU6ZlJqatox5q4bOPOsC/nss0VUV7ehQ4f12HX3Qzh3yCWMvuPP5R6ilVB1dRu2/OHm3DXqrxy570C++PwLBp1yHADX/O7P7Lv9AB6+9zF+dsIRAOy214+YNuUt9vqXgzl87+MY+ruzlmfsVqT6+qI3SYMlTcrbBq+0tz8A5wDL0v5OwCcRsawGVkuuekH2cxZA9vqnWf/l7Y28p1GlCugN5P5EWFkPvv6A35D/j9TQsLhEQ6t81dXV3H3XDYwePZb77nsEgA9qP1z++MVJk2loaKBz5/XLOUwroTmz5zF39ke89vLrADz+4BNs9cPNV+jz8F8fZ9+D9wLgsKMPZvzDEwGY9V4tH7w/m4379mnJIbd60dBQ/BYxIiJ2zNtGLNuPpIOBeRHxUt7u1dghV/Faofc0qlQB/XRggqRHJI3ItkeBCeTqSo3K/0eqqqop0dAq3w0jrmTqtBn84erlvyPc/8Bj7LXXbgD07bsJa6+9Nh9/vKBcQ7QSm//RAubMnkufTTcEYJc9duLtt95lw417L++z13578O70mQB8+MFcdtljJwA6dVmfPptuSO3MD1p+4K1Z85VcdgMOlfQeudLz3uQy9g6Slp237EXuvCLkMu/eANnr6wEL8tsbeU+jVKo6m6QqciWWnuS+aWqBFwsV9PNVr93zO1kA3G3Xnfififfx6mtv0JD94lxwwXDGT3iaG2+4kn79tmbJkqWce+4wnpz4TJlH2/K26Nh71Z0SsfnWfbn4qqGstfZazJr5ARecdgkXXTWUPpttSDQEs2vncPHZlzFvzkd06daZS6+5gM7dOiGJm665lYfufbTcH6HFTJn7XGPZ7GpZfMkvio45Nb+9rajjSdoTOCub5XI3cG/eSdFXI+JPkk4GfhgRJ2YnRf8tIn4qaWvgDnJxdANyCXHfQjG0ZAF9TX1XA7oV9l0K6Fa8ZgnoF/+8+ID+X7d/m4C+CV9PW3wF+EVEfCVpHeBWYDtymfnREfFO9v7zgROAOuD0iHik0PE8D93MDKAE0zwjYiIwMXv8Do3MUomIL4Ejm3j/pUDRVxA6oJuZgZfPNTNLhpfPNTNLQ3gtFzOzRDhDNzNLhAO6mVkifIMLM7M0+J6iZmapcEA3M0uEZ7mYmSXCGbqZWSIc0M3M0hD1LrmYmaXBGbqZWRo8bdHMLBUO6GZmiWj9JXQHdDMzgKhr/RHdAd3MDJyhm5mlwidFzcxS4QzdzCwNztDNzFLhDN3MLA1RV+4RrDkHdDMzIJyhm5klwgHdzCwNztDNzBLhgG5mloioV7mHsMYc0M3McIZuZpaMaHCGbmaWBGfoZmaJiGj9GXpVuQdgZlYJoqH4rRBJ60h6QdI/JL0u6aKsfWNJz0uaLukuSWtn7d/Lns/IXu+Tt6/zsvY3Je23qs/ggG5mBjTUq+htFb4C9o6IfsC2wP6SdgEuA34fEX2BhcCgrP8gYGFEbAb8PuuHpK2Ao4Gtgf2BP0lqU+jADuhmZuROiha7FdxPzqLs6VrZFsDewD1Z+yjgJ9njAdlzstf3kaSs/c6I+Coi3gVmADsXOrYDupkZqxfQJQ2WNClvG5y/L0ltJE0G5gHjgLeBTyKWLwFWC/TMHvcEZgFkr38KdMpvb+Q9jfJJUTMzIFZjOfSIGAGMKPB6PbCtpA7AWGDLxrplPxtL+aNAe5Mc0M3MKM089Ij4RNJEYBegg6TqLAvvBczOutUCvYFaSdXAesCCvPZl8t/TKJdczMzITVssditEUpcsM0dSW2BfYCrwJHBE1m0gcH/2+IHsOdnrT0REZO1HZ7NgNgb6Ai8UOnbRGbqk70XEV8X2NzNrTeqbby2XHsCobEZKFTAmIh6S9AZwp6RLgFeAm7L+NwG3SppBLjM/GiAiXpc0BngDqANOzko5TVplQJe0c3bA9YANJfUDfhURp3yLD2pmVpGa68KiiHgV2K6R9ndoZJZKRHwJHNnEvi4FLi322MWUXK4BDgbmZwf4B7BXsQcwM2sNmmvaYjkVU3KpioiZuWmRyxVM+83MWpvVmeVSqYoJ6LOysktkNaFTgLdKOywzs5ZVyZl3sYoJ6CeRK7tsCMwFxmdtZmbJqG9o/ZP+VhnQI2Ie2VlXM7NUfSdKLpJuoJGrkyJicCPdzcxapYYEls8tpuQyPu/xOsBhrLi+gJlZq5fCeujFlFzuyn8u6VZyi82YmSXjO1FyacTGwEbNPRCzYrzy+h3lHoIl6jtRcpG0kK9r6FXkLk0dUspBmZm1tORnuWSLrPcDPsiaGrJFY8zMkpJCYCv4lZQF77ERUZ9tKXxmM7NvaAgVvVWqYv7GeEHS9iUfiZlZGTXX8rnl1GTJJW8h9t2Bf5f0NrCY3F00IiIc5M0sGQ3lHkAzKFRDfwHYnq9vZGpmlqxo9I5vrUuhgC6AiHi7hcZiZlY2dRVcSilWoYDeRdKZTb0YEVeVYDxmZmWReobeBmhP43eeNjNLSuo19A8j4uIWG4mZWRmlnqG3/k9nZlak1DP0fVpsFGZmZVafQA7bZECPiAUtORAzs3JK4A5032q1RTOz5DSknKGbmX2XpLBQlQO6mRnpnxQ1M/vOaJBLLmZmSagv9wCagQO6mRme5WJmlgzPcjEzS4RnuZiZJcIlFzOzRKQwbbGYe4qamSWvXsVvhUjqLelJSVMlvS7ptKx9fUnjJE3PfnbM2iXpGkkzJL2afw9nSQOz/tMlDVzVZ3BANzMjl6EXu61CHfCbiNgS2AU4WdJWwBBgQkT0BSZkzwEOAPpm22Dgesh9AQAXAj8CdgYuXPYl0BQHdDMzmi+gR8SHEfFy9vgzYCrQExgAjMq6jeLr+zUPAG6JnOeADpJ6APsB4yJiQUQsBMYB+xc6tmvoZmZAKW4pKqkPsB3wPNAtIj6EXNCX1DXr1hOYlfe22qytqfYmOUM3M2P1MnRJgyVNytsGr7w/Se2Be4HTI+KfBQ7d2FdJFGhvkjN0MzNW79L/iBgBjGjqdUlrkQvmt0fEX7PmuZJ6ZNl5D2Be1l4L9M57ey9gdta+50rtEwuNyxm6mRm5eejFboVIEnATMDUirsp76QFg2UyVgcD9ee3HZbNddgE+zUozjwH9JXXMTob2z9qa5AzdzIxmnYe+G3As8JqkyVnbUGA4MEbSIOB94Mjstb8BBwIzgM+B4yF31zhJw4AXs34Xr+pOcg7oZmY0X0CPiL/TeP0bGrlXc0QEcHIT+xoJjCz22A7oZmZ4LRczs2R4LRczs0T4BhdmZoloSKDo4oBuZkYaqy06oJuZ4ZOiZmbJcIZuZpaIOrX+HN0B3cwMl1zMzJLhkouZWSI8bdHMLBGtP5w7oJuZAS65mJkloz6BHN0B3cwMZ+hmZskIZ+hmZmlwhm7NrlevDbh55NV0696FhoYGbrzxdv547U107NiB0bdfz0Yb9WbmzFkc/bMT+eSTT8s9XGtm/Q8fSE27dlRVVdGmTRvGjLyGP464hSf+/ixVqmL9jutx6fm/oWuXTrwzcxYXXHoVb7w1g1MHD+T4nx2xfD///GwRFw7/AzPemQkSw4aewbbbbFnGT1b5Upi2qNzdjypP9do9K3NgJda9e1d6dO/KK5On0L59DS88/yiHH3ECA4/7KQsWfMLlV1zHOWefTMeO63He0P9b7uG2uC9mP13uIZRU/8MHctdN19Cxw3rL2xYtXkz7mhoAbrv7ft5+930uPOcU5i/8hNlz5vLEU8/y/XXbrxDQhw77f2zfbxuOOHR/li5dyhdffsX3123f4p+npazVeZM1vj3FSX1+WnTMuf69MRV5O4yqcg/AVjRnzjxemTwFgEWLFjNt2nR6btCdQw7Zj1tuvRuAW269m0MP3b+cw7QWtCyYA3zxxZcoCyWdOnbgh1tuTnX1in9oL1q8mJf+MYXDD9kPgLXWWivpYN5c6oiit0rV4iUXScdHxH+39HFbo4026sW2/bbh+RdeoVvXzsyZMw/IBf2uXTqVeXRWCpIYfMb5SOLIAQdw5IADAbj6LzfzwKMTWLemhpF/HF5wH7UfzKFjh/X47aVX8eaMd9hq874MOf1E2rVdpyU+QquVwknRcmToFzX1gqTBkiZJmtTQsLglx1RxamraMeauGzjzrAv57LNF5R6OtZBbr7+Su//7Wq6/chij//oQkya/BsBp//FLJoy9lYP678Ud9z5YcB919fVMfWsGRx12EPfcfB1t267DTbeOaYnht2oNq7FVqpIEdEmvNrG9BnRr6n0RMSIidoyIHauqaprqlrzq6mruvusGRo8ey333PQLA3Hkf0717VyBXZ5/30fxyDtFKZNlfXp06dmCfH+/Ka2+8ucLrB/Xfk/ETnym4j+5dO9OtS2f+ZestAOi/5+688daM0gw4IbEa/1WqUmXo3YDjgEMa2RyJVuGGEVcyddoM/nD1iOVtDz34OMcdeyQAxx17JA8++Fi5hmcl8vkXX7J48efLH//vCy/Td5M+zJz1wfI+Tz79HBtv1Kvgfjp3Wp/uXbvw7sxaAJ57aTKb9tmwdANPRAoZeqlq6A8B7SNi8sovSJpYomMmYbddd+LYXxzBq6+9waQXHwfggguGc9kV13HnHX/m+F8ew6xZH3DUMf9R5pFac5u/YCGnDR0GQH1dPQf235Pdd9mR04dewnvv16IqsUH3rvzX2acA8PH8BRw16FQWLf6cqqoqbhtzH/ff/hfa19Qw9IyTOPeiy1lat5TeG/Rg2NAzyvnRWoX6Cp3xtzo8bdFaldSnLdq30xzTFn+20WFFx5w7Zo6tyGmLvrDIzIw0Zrk4oJuZUdm18WI5oJuZkcal/w7oZma45GJmlowUZrk4oJuZkUbJxYtzmZnRvBcWSRopaZ6kKXlt60saJ2l69rNj1i5J10iakV1Rv33eewZm/adLGriq4zqgm5nR7Jf+3wysvCTqEGBCRPQFJmTPAQ4A+mbbYOB6yH0BABcCPwJ2Bi5c9iXQFAd0MzNyJZdit1WJiKeABSs1DwBGZY9HAT/Ja78lcp4DOkjqAewHjIuIBRGxEBjHN78kVuAaupkZ0AJXzXeLiA+zY30oqWvW3hOYldevNmtrqr1JDuhmZkD9apwUlTSYXHlkmRERMaKp/qvaXSNtUaC9SQ7oZmas3iyXLHivbgCfK6lHlp33AOZl7bVA77x+vYDZWfueK7VPLHQA19DNzMiVXIrdvqUHgGUzVQYC9+e1H5fNdtkF+DQrzTwG9JfUMTsZ2j9ra5IzdDMzmnceuqTR5LLrzpJqyc1WGQ6MkTQIeB84Muv+N+BAYAbwOXA8QEQskDQMeDHrd3FErHyidQUO6GZmNO+l/xFxTBMv7dNI3wBObmI/I4GRxR7XAd3MDF/6b2aWjBQu/XdANzPDAd3MLBmVejvO1eGAbmaGM3Qzs2T4BhdmZomoj9Z/V1EHdDMzXEM3M0uGa+hmZolwDd3MLBENLrmYmaXBGbqZWSI8y8XMLBEuuZiZJcIlFzOzRDhDNzNLhDN0M7NE1Ed9uYewxhzQzczwpf9mZsnwpf9mZolwhm5mlgjPcjEzS4RnuZiZJcKX/puZJcI1dDOzRLiGbmaWCGfoZmaJ8Dx0M7NEOEM3M0uEZ7mYmSXCJ0XNzBLhkouZWSJ8paiZWSKcoZuZJSKFGrpS+FZKnaTBETGi3OOwyuLfC1tZVbkHYEUZXO4BWEXy74WtwAHdzCwRDuhmZolwQG8dXCe1xvj3wlbgk6JmZolwhm5mlggH9AonaX9Jb0qaIWlIucdj5SdppKR5kqaUeyxWWRzQK5ikNsB1wAHAVsAxkrYq76isAtwM7F/uQVjlcUCvbDsDMyLinYhYAtwJDCjzmKzMIuIpYEG5x2GVxwG9svUEZuU9r83azMy+wQG9sqmRNk9LMrNGOaBXtlqgd97zXsDsMo3FzCqcA3plexHoK2ljSWsDRwMPlHlMZlahHNArWETUAf8JPAZMBcZExOvlHZWVm6TRwLPA5pJqJQ0q95isMvhKUTOzRDhDNzNLhAO6mVkiHNDNzBLhgG5mlggHdDOzRDigW7OTVC9psqQpku6W1G4N9rWnpIeyx4cWWnFSUgdJv/4Wx/g/ks76tmM0qxQO6FYKX0TEthGxDbAEODH/ReWs9u9eRDwQEcMLdOkArHZAN0uFA7qV2tPAZpL6SJoq6U/Ay0BvSf0lPSvp5SyTbw/L14CfJunvwL8t25GkX0q6NnvcTdJYSf/Itl2B4cCm2V8HV2T9zpb0oqRXJV2Ut6/zs3XmxwObt9i/hlkJOaBbyUiqJreW+2tZ0+bALRGxHbAY+C2wb0RsD0wCzpS0DnADcAiwB9C9id1fA/xPRPQDtgdeB4YAb2d/HZwtqT/Ql9wyxNsCO0j6saQdyC2jsB25L4ydmvmjm5VFdbkHYElqK2ly9vhp4CZgA2BmRDyXte9C7qYdz0gCWJvc5exbAO9GxHQASbcBgxs5xt7AcQARUQ98KqnjSn36Z9sr2fP25AL8usDYiPg8O4bXx7EkOKBbKXwREdvmN2RBe3F+EzAuIo5Zqd+2NN8SwQJ+FxF/WekYpzfjMcwqhksuVi7PAbtJ2gxAUjtJPwCmARtL2jTrd0wT758AnJS9t42k7wOfkcu+l3kMOCGvNt9TUlfgKeAwSW0lrUuuvGPW6jmgW1lExEfAL4HRkl4lF+C3iIgvyZVYHs5Ois5sYhenAXtJeg14Cdg6IuaTK+FMkXRFRDwO3AE8m/W7B1g3Il4G7gImA/eSKwuZtXpebdHMLBHO0M3MEuGAbmaWCAd0M7NEOKCbmSXCAd3MLBEO6GZmiXBANzNLhAO6mVki/j+8XMxT/rWurQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Bigrams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "3\n",
      "5\n",
      "7\n",
      "9\n",
      "11\n",
      "13\n",
      "15\n",
      "17\n",
      "19\n",
      "21\n",
      "23\n",
      "25\n",
      "27\n",
      "29\n",
      "31\n",
      "33\n",
      "35\n",
      "37\n",
      "39\n",
      "41\n",
      "43\n",
      "45\n",
      "47\n",
      "49\n",
      "\n",
      "The optimal number of neighbors is 9.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VOX1+PHPmZnMZN9DFhLCjhKIqOxWECOIUpFSXHDDpVJr1apVgWqr9WsrVWtrq3XXIi4gVsVafyAKuKIsgiCbYkAIYUnCEghkP78/ZjImIYQRGTKB83695pWZ597n3jMgOT7rFVXFGGOMaW0cLR2AMcYYczgsgRljjGmVLIEZY4xplSyBGWOMaZUsgRljjGmVLIEZY4xplSyBGWOMaZUsgRljjGmVLIEZY4xplVwtHcCRkpycrO3bt2/pMIwxplVZsmRJsaqmtHQch+OYSWDt27dn8eLFLR2GMca0KiLyXUvHcLisC9EYY0yrZAmsnv379zN48GBqamoAGD58OPHx8fz0pz8NqH5FRQUXXXQRnTt3pl+/fmzYsKHJ866++mratGlDjx49Djj2z3/+k27dupGTk8Mdd9wBQElJCUOGDCE6OpobbrihwflnnXUWO3fu/AHf0hhjjg2WwOp57rnnGD16NE6nE4Dbb7+dqVOnBlz/2WefJSEhgXXr1nHLLbcwYcKEJs+78sormTVr1gHl8+bNY+bMmSxfvpyVK1dy2223ARAeHs7//d//8dBDDx1Q5/LLL+df//pXwDEaY8yxwhJYPS+99BLnn3++/3NeXh4xMTEB1585cybjxo0DYMyYMbz//vs09biaQYMGkZiYeED5448/zsSJE/F4PAC0adMGgKioKH7yk58QHh5+QJ2RI0fyyiuvBByjMcYcKyyB+VRWVpKfn8+Pmcm4efNmsrKyAHC5XMTFxVFSUhJw/a+//pqPPvqIfv36MXjwYBYtWnTIOgkJCVRUVPyg+xhjzLHgmJmF+GMVFxcTHx//o67RVGtLRAKuX11dzc6dO/nss89YtGgRF154Ifn5+Ye8Rps2bSgsLCQpKekHx2yMMa2VtcB8IiIiKC8v/1HXyMzMZNOmTYA3Ge3evbvJrsLm6o8ePRoRoW/fvjgcDoqLiw9Zr7y8nIiIiMOO2xhjWqPjPoGVllfxtzlf891eoaamJqAkNmnSJN54440DykeOHMmUKVMAeO211zjzzDN/UAts1KhRzJ07F/B2J1ZWVpKcnNxsHVVl69atP6rr0xhjWqOgJjARGS4ia0VknYhMbOL4IBH5QkSqRWRMo2PtRORdEVktIqtEpH0wYlSFR97/hsUbdjBs2DA+/vhj/7HTTz+dCy64gPfff5/MzExmz54NwIoVK0hLSzvgWtdccw0lJSV07tyZhx9+mMmTJwNQWFjIueee6z9v7NixDBgwgLVr15KZmcmzzz4LeKfX5+fn06NHDy6++GKmTJniT4Dt27fn1ltv5d///jeZmZmsWrUKgCVLltC/f39cLusNNsYcX6SpcZsjcmERJ/A1MBQoABYBY1V1Vb1z2gOxwG3AW6r6Wr1j84E/qeocEYkGalV138Hu17t3bz2cnThUlW53zeKqn7RneFo5Dz/88CGnzp999tn+ZNbSfvOb3zBy5Ejy8vJaOhRjTCskIktUtXdLx3E4gtkC6wusU9V8Va0EpgHn1z9BVTeo6nKgtn65iHQHXKo6x3fe3uaS148hIiRHuyneU8nJJ5/MkCFD/AuZDyZUkhdAjx49LHkZY45LwUxgbYFN9T4X+MoC0RXYJSKvi8hSEXnQ16ILiuQYD8V7KwBvN17dQubW4Nprr23pEIwxpkUEM4E1NXsh0P5KF3A63q7FPkBH4MoDbiAyXkQWi8jioqKiw42T5OjvE5gxxpjWIZgJrADIqvc5Eyj8AXWX+rofq4E3gVMan6SqT6lqb1XtnZJy+E8DSI52WwIzxphWJpgJbBHQRUQ6iIgbuBh46wfUTRCRuqx0JrCqmfN/lORoDyV7K6mtDc6EFmOMMUde0BKYr+V0AzAbWA28qqorReReERkJICJ9RKQAuAB4UkRW+urW4O0+fF9EVuDtjnw6WLEmR3uorlV2768K1i2MMcYcYUFdPKSq7wDvNCr7Q733i/B2LTZVdw6QG8z46iTHeDfPLd5bQUKU+2jc0hhjzI903O/EAd4xMIAiGwczxphWwxIYkBJd1wKrbOFIjDHGBMoSGN4xMIDiPdYCM8aY1sISGBAXEYbLITaV3hhjWhFLYIDDISTZWjBjjGlVLIH5eHfjsDEwY4xpLSyB+dh2UsYY07pYAvNJjvbYJA5jjGlFLIH5JMe4Kd5bSbCej2aMMebIsgTmkxLtobKmltLy6pYOxRhjTAAsgfn414LZOJgxxrQKlsB8bDGzMca0LpbAfJJjvPsh2lR6Y4xpHSyB+VgXojHGtC6WwHwSIt04xBKYMca0FpbAfJwOITHKFjMbY0xrYQmsnuRoN0V7bAzMGGNaA0tg9aTEWAvMGGNaC0tg9dh+iMYY03pYAqsn2fdIFdtOyhhjQp8lsHqSoz2UV9VSVlnT0qEYY4w5BEtg9dhuHMYY03pYAqsnOcYWMxtjTGthCaye5Oi67aQsgRljTKizBFZPiq8Lscj2QzTGmJBnCayexCg3IjYGZowxrYElsHpcTgcJkW7rQjTGmFYgqAlMRIaLyFoRWSciE5s4PkhEvhCRahEZ0+hYjYgs873eCmac9dWtBTPGGBPaXMG6sIg4gceAoUABsEhE3lLVVfVO2whcCdzWxCX2q2qvYMV3MN7dOGwMzBhjQl0wW2B9gXWqmq+qlcA04Pz6J6jqBlVdDtQGMY4fxLaTMsaY1iGYCawtsKne5wJfWaDCRWSxiHwmIqOObGgHlxztsUkcxhjTCgStCxGQJsp+yCaD7VS1UEQ6AnNFZIWqftvgBiLjgfEA7dq1O/xI60mOcVNWWcP+yhoi3M4jck1jjDFHXjBbYAVAVr3PmUBhoJVVtdD3Mx+YD5zcxDlPqWpvVe2dkpLy46L18W8nZd2IxhgT0oKZwBYBXUSkg4i4gYuBgGYTikiCiHh875OB04BVzdc6Mr5fzGwJzBhjQlnQEpiqVgM3ALOB1cCrqrpSRO4VkZEAItJHRAqAC4AnRWSlr/qJwGIR+RKYB0xuNHsxaGxDX2OMaR2COQaGqr4DvNOo7A/13i/C27XYuN6nQM9gxnYwyTF1+yHaVHpjjAllzbbARMQpIg8erWBCQVKUjYEZY0xr0GwCU9Ua4FQRaWpG4THJ7XIQFxFmCcwYY0JcIF2IS4GZIjIDKKsrVNXXgxZVC7PtpIwxJvQFksASgRLgzHplChzDCcxD8R4bAzPGmFB2yASmqlcdjUBCSXKMh9WFpS0dhjHGmGYcchq9iGSKyBsisl1EtonIf0TkgJmDx5KUaI+tAzPGmBAXyDqw5/EuQM7Au5fhf31lx6zkaDd7yqspr6pp6VCMMcYcRCAJLEVVn1fVat/r38CR2bcpRNUtZi4ps3EwY4wJVYEksGIRucy3JswpIpfhndRxzLLdOIwxJvQFksCuBi4EtgJbgDG+smNWcowtZjbGmFDX7CxE31OVf66qI49SPCEhObpuOylLYMYYE6oC2Ynj/ObOORZ9/0gVGwMzxphQFchC5k9E5FFgOg134vgiaFG1sPAwJzEeF0U2BmaMMSErkAQ20Pfz3nplSsOdOY45yTEe60I0xpgQdqgxMAfwuKq+epTiCRm2H6IxxoS2Q42B1eJ9KOVxJznaY2NgxhgTwgKZRj9HRG4TkSwRSax7BT2yFuZNYNYCM8aYUBXIGFjdmq9f1ytToOORDyd0JEd72LWviqqaWsKcgeR5Y4wxR1Mgu9F3OBqBhJrkGO9asJK9laTFhbdwNMYYYxo7aNNCRO6o9/6CRsf+HMygQsH3a8GsG9EYY0JRc31jF9d7P6nRseFBiCWk1CUwe6yKMcaEpuYSmBzkfVOfjzkptqGvMcaEtOYSmB7kfVOfjzl1Y2A2ld4YY0JTc5M4ThKRUrytrQjfe3yfj/lZDZFuF5Fup42BGWNMiDpoAlNV59EMJBTZWjBjjAldtsCpGbadlDHGhC5LYM1IjvZQvMfGwIwxJhQFNYGJyHARWSsi60RkYhPHB4nIFyJSLSJjmjgeKyKbfY9zOepsR3pjjAldQUtgvqc5PwacA3QHxopI90anbQSuBF4+yGX+D/ggWDEeSnK0hx37KqmuqW2pEIwxxhzEIROYiIwWkW9EZLeIlIrInnozEpvTF1inqvmqWglMo9HTnVV1g6ouBw7IECJyKpAKvBvQNwmClGg3qrBjn3UjGmNMqAmkBfYAMFJV41Q1VlVjVDU2gHptgU31Phf4yg7J9xyyvwK3B3J+sPh347DFzMYYE3ICSWDbVHX1YVy7qd06Al0AfT3wjqpuau4kERkvIotFZHFRUdEPDvBQkmPq9kO0FpgxxoSaQB6nslhEpgNvAv6miKq+foh6BUBWvc+ZQGGAcQ0ATheR64FowC0ie1W1wUQQVX0KeAqgd+/eR3x3kGTbTsoYY0JWIAksFtgHDKtXpsChEtgioIuIdAA2490c+JJAglLVS+vei8iVQO/GyetoSI6u207KEpgxxoSaQJ4HdtXhXFhVq0XkBmA24ASeU9WVInIvsFhV3xKRPsAbQAJwnoj8UVVzDud+wRDtceFxOSyBGWNMCDpkAhORTOCfwGl4W14fA79R1YJD1VXVd4B3GpX9od77RXi7Fpu7xr+Bfx/qXsEgIr7tpGwMzBhjQk0gkzieB94CMvDOIvyvr+y4YIuZjTEmNAWSwFJU9XlVrfa9/g2kBDmukJES7bZp9MYYE4ICSWDFInKZiDh9r8uAkmAHFiqsC9EYY0JTIAnsauBCYCuwBRjjKzsuJEd72FFWQU3tMf8MT2OMaVUCmYW4ERh5FGIJScnRbmoVdu6r9K8LM8YY0/IOmsBE5A5VfUBE/kkTO2io6k1BjSxEfL8bR4UlMGOMCSHNtcDqto9afDQCCVXf78ZRCWktHIwxxhi/gyYwVf2v7+0+VZ1R/5iIXBDUqEKIP4HZVHpjjAkpgUzimBRg2TEpxRKYMcaEpObGwM4BzgXaisg/6h2KBaqDHVioiI1w4XY6KLIEZowxIaW5MbBCvONfI4El9cr3ALcEM6hQIiIkRbu9Y2DGGGNCRnNjYF8CX4rIy6padRRjCjnexczWAjPGmFASyONU2ovI/UB3ILyuUFU7Bi2qEJMc7Wa7bSdljDEhJdDNfB/HO+41BHgBmBrMoEKNtcCMMSb0BJLAIlT1fUBU9TtVvQc4M7hhhZbkGA8leyupte2kjDEmZATShVguIg7gG98DKjcDbYIbVmhJjvZQXavs3l9FQpS7pcMxxhhDYC2wm4FI4CbgVOAyYFwwgwo1ydHepGXdiMYYEzoC2cx3ke/tXuCq4IYTmuoWMxftraBLakwLR2OMMQYCaIGJyBwRia/3OUFEZgc3rNDy/Ya+thbMGGNCRSBdiMmquqvug6ru5DgcAwMotqn0xhgTMgJJYLUi0q7ug4hk08TjVY5l8RFhOB1iY2DGGBNCApmFeCfwsYh84Ps8CBgfvJBCj8MhJEW5LYEZY0wICWQSxywROQXoDwhwi6oWBz2yEONdzGxjYMYYEyoO2oUoIif4fp4CtMO7ue9moJ2v7LiSHGO7cRhjTChprgV2K96uwr82cUw53nbjiHazbtuelg7DGGOMT3MJbI7v5zWqmn80ggllKb4uRFVFRFo6HGOMOe41Nwux7qnLrx2NQEJdcrSHyppaSsuPm2d5GmNMSGsugZWIyDygg4i81fgVyMVFZLiIrBWRdSIysYnjg0TkCxGpFpEx9cqzRWSJiCwTkZUict0P/2pHVnKMbSdljDGhpLkuxBHAKXgfndLUOFizRMQJPAYMBQqARSLylqquqnfaRuBK4LZG1bcAA1W1QkSiga98dQt/aBxHSv3FzJ1SolsqDGOMMT7NPZG5EvhMRAaqatFhXLsvsK5u/ExEpgHnA/4EpqobfMdqm7h3HQ+BLbgOKn8Cs6n0xhgTEg6awETk76p6M/CciByw84aqjjzEtdsCm+p9LgD6BRqYiGQB/wM6A7c31foSkfH4FlW3a9eu8eEj6vsEZl2IxhgTCprrQqx76vJDh3ntpqbqBbwFlapuAnJFJAN4U0ReU9Vtjc55CngKoHfv3kHd3ioxyo1DLIEZY0yoaK4LcYnvZ90WUohIApClqssDuHYBkFXvcybexdA/iKoWishK4HRacEak0yEk2nZSxhgTMgJ5nMp8EYkVkUTgS+B5EXk4gGsvArqISAcRcQMXA4HOXswUkQjf+wTgNGBtIHWDKTnaQ9EeGwMzxphQEMjkiDhVLQVGA8+r6qnAWYeqpKrVwA3AbGA18KqqrhSRe0VkJICI9BGRAuAC4ElfSwvgROBzEfkS+AB4SFVX/NAvd6R590O0FpgxxoSCQHajd4lIOnAh3p3pA6aq7wDvNCr7Q733i/B2LTauNwfI/SH3OhqSo91sKClr6TCMMcYQWAvsXrytqHWqukhEOgLfBDes0FTXAlM9rh6HZowxISmQx6nMAGbU+5wP/DyYQYWq5BgP5VW1lFXWEO0JpPFqjDEmWAKZxPGAbxJHmIi8LyLFInLZ0Qgu1NTfjcMYY0zLCqQLcZhvEsdP8U6N7wrcHtSoQlRytO2HaIwxoSKQBBbm+3ku8Iqq7ghiPCHNduMwxpjQEchAzn9FZA2wH7heRFKA8uCGFZpSYrwJrMj2QzTGmBZ3yBaYqk4EBgC9VbUKKMO7Ke9xJzHK14VoY2DGGNPiAp1K1xYYKiLh9cpeCEI8IS3M6SAhMsy6EI0xJgQcMoGJyN3AGUB3vIuSzwE+5jhMYGC7cRhjTKgIZBLHGCAP2KqqVwEn4X1G13HJm8BsDMwYY1paIAlsv6rWAtUiEgtsBzoGN6zQlRxjLTBjjAkFgYyBLRaReOBpYAmwF1gY1KhCWHK02yZxGGNMCAhkK6nrfW+fEJFZQGyAzwM7JiVHeyirrGF/ZQ0RbmdLh2OMMcetgyYwETmluWOq+kVwQgptKfUWM2clRrZwNMYYc/xqrgX212aOKXDmEY6lVUiO8a4FK7IEZowxLeqgCUxVhxzNQFoL29DXGGNCQyC70f/aN4mj7nOCiFzfXJ1j2ff7IVayf/9+Bg8eTE1NDd999x2nnnoqvXr1IicnhyeeeOKQ19qxYwdDhw6lS5cuDB06lJ07dzZ53vDhw4mPj+enP/1pg/JLL72Ubt260aNHD66++mqqqqoAeOmll8jNzSU3N5eBAwfy5ZdfAlBZWcmgQYOorq7+MX8ExhgTEgKZRn+tqu6q+6CqO4FrgxdSaEuqtyP9c889x+jRo3E6naSnp/Ppp5+ybNkyPv/8cyZPnkxhYWGz15o8eTJ5eXl888035OXlMXny5CbPu/3225k6deoB5Zdeeilr1qxhxYoV7N+/n2eeeQaADh068MEHH7B8+XJ+//vfM378eADcbjd5eXlMnz79x/wRGGNMSAgkgTlEROo+iIgTcAcvpNDmcTmJDXdRvLeCl156ifPP924L6Xa78Xi8rbOKigpqa2sPea2ZM2cybtw4AMaNG8ebb77Z5Hl5eXnExMQcUH7uueciIogIffv2paCgAICBAweSkJAAQP/+/f3lAKNGjeKll176Ad/YGGNCUyAJbDbwqojkiciZwCvArOCGFdqSYzxs27WX/Px82rdv7y/ftGkTubm5ZGVlMWHCBDIyMpq9zrZt20hPTwcgPT2d7du3H1Y8VVVVTJ06leHDhx9w7Nlnn+Wcc87xf+7RoweLFi06rPsYY0woCWQh8wRgPPArQIB3gWeCGVSoS472ULhlO/Hx8Q3Ks7KyWL58OYWFhYwaNYoxY8aQmpoa9Hiuv/56Bg0axOmnn96gfN68eTz77LN8/PHH/jKn04nb7WbPnj1NtuqMMaa1CORxKrWq+oSqjsE79rVAVWuCH1roSon2sKtSKC9v+rFoGRkZ5OTk8NFHHzV7ndTUVLZs2QLAli1baNOmzQ+O5Y9//CNFRUU8/PDDDcqXL1/OL37xC2bOnElSUlKDYxUVFYSHh2OMMa1ZILMQ54tIrIgkAsuA50Xk4UPVO5YlR7vZVeumpqbGn8QKCgrYv38/ADt37uSTTz6hW7duAFxxxRUsXHjg7lsjR45kypQpAEyZMsU/nhaoZ555htmzZ/PKK6/gcHz/V7lx40ZGjx7N1KlT6dq1a4M6JSUlpKSkEBYW1vhyxhjTqgQyBhanqqXAaOB5VT0VOCu4YYW25GgPe8qryTtrqL97bvXq1fTr14+TTjqJwYMHc9ttt9GzZ0/A2xqqG+uqb+LEicyZM4cuXbowZ84cJk6cCMDixYv5xS9+4T/v9NNP54ILLuD9998nMzOT2bNnA3Ddddexbds2BgwYQK9evbj33nsBuPfeeykpKeH666+nV69e9O7d23+tefPmce655wbnD8YYY44iUdXmTxBZAQwDpgB3quoiEVmuqrlHI8BA9e7dWxcvXnxU7vXG0gJumf4lDw6J4a0Xn25yinud0tJSrrnmGmbMmHFUYjuU0aNHc//99/tbh8aY45uILFHV3oc+M/QE0gK7F+9MxHW+5NUR+Ca4YYW2n3ROQQQ2O9MYMmQINTUHHxKMjY0NmeRVWVnJqFGjLHkZY44Jh2yBtRZHswUGMObxT9lfVcP/bjr90CcbY0yIOiZbYCJyh+/nP0XkH41fgVxcRIaLyFoRWSciE5s4PkhEvhCRahEZU6+8l4gsEJGVIrJcRC46nC8XTEO7p7KysJSCnftaOhRjjDkuNdeFuNr3czHeB1k2fjXLt2PHY8A5QHdgrIh0b3TaRuBK4OVG5fuAK1Q1BxgO/L3+foyhYGh37/qu91Zta+FIjDHm+NTcbvT/9f2ccpjX7ot33CwfQESmAecDq+rdY4PvWIN9l1T163rvC0VkO5AC7CJEdEyJplNKFHNWb+PK0zq0dDjGGHPcae6Blm81V1FVRx7i2m2BTfU+FwD9Ag/NH0dfvHsvftvEsfF4dwmhXbt2P/TSP9qwnDSe/jCf3fuqiIu0dVXGGHM0NbeV1AC8CegV4HO820j9EE2d/4NmjIhIOjAVGKeqB+yOq6pPAU+BdxLHD4zvRxvaPZXH53/LvLXbGXVy26N9e2OMOa41NwaWBvwO6AE8AgwFilX1A1X9IIBrFwBZ9T5nAs0/X6QeEYkF/gfcpaqfBVrvaOqVGU9KjIc5Ng5mjDFH3UETmKrWqOosVR0H9AfWAfNF5MYAr70I6CIiHUTEDVwMNNstWcd3/hvAC6oaGouomuBwCGed2Ib5a7dTUX1cbw9pjDFHXbMLmUXEIyKjgReBXwP/AF4P5MKqWg3cgHcR9GrgVVVdKSL3ishI3/X7iEgBcAHwpIis9FW/EBgEXCkiy3yvXofx/YJuWPc0yipr+PTbkpYOxRhjjivNTeKYgrf78P8Bf1TVr37oxVX1HeCdRmV/qPd+Ed6uxcb1XsSbNEPegE5JRLqdzFm1jSHdfvhu8sYYYw5Pcy2wy4GuwG+AT0Wk1PfaIyKlRye80Bce5mRw1xTeW7WN2tpjY1cTY4xpDZobA3OoaozvFVvvFaOqsUczyFA3tHsq2/dUsHzz7pYOxRhjjhuBbOZrDuHME9rgdAjvrtza0qEYY8xxwxLYERAf6aZv+0SbTm+MMUeRJbAjZGj3VL7ZvpcNxWUtHYoxxhwXLIEdIXWb+1orzBhjjg5LYEdIVmIkJ6bH8u4qGwczxpijwRLYETS0eypLvttJyd6Klg7FGGOOeZbAjqBh3VOpVXh/zfaWDsUYY455lsCOoJyMWDLiwm0czBhjjgJLYEeQiDC0eyoffVPE/krb3NcYY4LJEtgRNrR7GuVVtXz0TVFLh2KMMcc0S2BHWL+OicSEu6wb0RhjgswS2BEW5nQwpFsb3l+znRrb3NcYY4LGElgQDMtJZUdZJUu+29nSoRhjzDHLElgQDO6aQphTmGOLmo0xJmgsgQVBTHgYAzolM2fVNlStG9EYY4LBEliQDOueyoaSfazbvrelQzHGmGOSJbAgqdvc912bjWiMMUFhCSxIUmPDOSkzzhKYMcYEiSWwIBraPZUvN+1iW2l5S4dijDHHHEtgQTQsJw2A91ZbK8wYY440S2BB1KVNNNlJkby70hKYMcYcaZbAgkhEGHpiKgu+LWFvRTUA+/fvZ/DgwdTUeDf7dTqd9OrVi169ejFy5MhDXrOiooKLLrqIzp07069fPzZs2NDkeVdffTVt2rShR48eDcpnzJhBTk4ODoeDxYsX+8tXrFjBlVdeeXhf1BhjWoAlsCAb2j2VyppaPljr3dz3ueeeY/To0TidTgAiIiJYtmwZy5Yt46233jrk9Z599lkSEhJYt24dt9xyCxMmTGjyvCuvvJJZs2YdUN6jRw9ef/11Bg0a1KC8Z8+eFBQUsHHjxh/6FY0xpkVYAguyU7MTSIxy+3fleOmllzj//PMP+3ozZ85k3LhxAIwZM4b333+/ycXSgwYNIjEx8YDyE088kW7dujV57fPOO49p06YddmzGGHM0BTWBichwEVkrIutEZGITxweJyBciUi0iYxodmyUiu0Tk7WDGGGwup4MzT2jD3DXbKdtfTn5+Pu3bt/cfLy8vp3fv3vTv358333zzkNfbvHkzWVlZ3mu7XMTFxVFSUnJEYu3duzcfffTREbmWMcYEmytYFxYRJ/AYMBQoABaJyFuquqreaRuBK4HbmrjEg0Ak8MtgxXi0DO2eymtLCnjvi3XEx8c3OLZx40YyMjLIz8/nzDPPpGfPnnTq1Omg12qqtSUiRyTONm3aUFhYeESuZYwxwRbMFlhfYJ2q5qtqJTANaNB3pqobVHU5UNu4sqq+D+wJYnxHzeldkvG4HHy8YTfl5Q3XhGVkZADQsWNHzjjjDJYuXdrstTIzM9m0aRMA1dXV7N69u8muwsNRXl5ORETEEbmWMcYEWzATWFtgU73PBb6y406k28WwnDReX7mbveWV/iS2c+dOKioqACguLuaTTz6he/dSvkf2AAAgAElEQVTuAEyaNIk33njjgGuNHDmSKVOmAPDaa69x5plnHrEW2Ndff33ArEVjjAlVwUxgTf1WPaJbs4vIeBFZLCKLi4qKjuSlj7j7R/ekb/tEKlJ7cO/TrwOwevVqevfuzUknncSQIUOYOHGiP4GtWLGCtLS0A65zzTXXUFJSQufOnXn44YeZPHkyAIWFhZx77rn+88aOHcuAAQNYu3YtmZmZPPvsswC88cYbZGZmsmDBAkaMGMHZZ5/trzNv3jxGjBgRtD8DY0JN42UtAKWlpbRt25YbbrjBX3bnnXeSlZVFdHR0wHW6du2Kx+PB4/EwfPhwBg0adECd+Ph44uLiEBGKiorIzMzE7XYTHh6Oy+UiOzub3NxcvvjiC/890tLSCAsLQ0RISUkhPT0dj8eDiOB2u+nUqZN/iU1dndTUVP85OTk5/riSk5MBsnzzFJaLyCkAInKDiOwUERWRr0XkE985a0Vku4jk1z/fV2eOiFT46jwlvv+zFpH3ROTJxvfwHXtARFaKyGoR+UejOgmH/AtU1aC8gAHA7HqfJwGTDnLuv4ExTZSfAbwdyP1OPfVUDXX7K6t15D1TNSpniD71wbfNnjts2LCjFJVXeXm59uvXT6uqqo7qfY1pSY8++qj+/e9/b1B200036dixY/XXv/61v2zBggVaWFioUVFRAdVZuXKl5uTkaHl5uX777bcaFhaml19++QF1hg8frpdffrlmZ2fryy+/rAkJCbpw4UL92c9+puHh4VpbW6sLFizQvn376k033aRDhw7VxMRE3bRpk0ZFRWlubq527NhRy8vLNTw8XB0Oh1ZXV+srr7yiF154od500006YMAAzcrK0vXr12tqaqoC+umnn2p+fr66XC4FyvE2OPoDn6v3d+84YDGwAfgJUIZ3PsNVQAVwV6PzBwLLgI6+OouAM3zH/gp83cQ9BgKfAE7fa0G9OuOAO/UQv/eD2QJbBHQRkQ4i4gYuBg690OkYFh7mZMadl9B34Onc9/ZX/G3O1wd9Xtjs2bOPamwbN25k8uTJuFxBm9djTMhpvKxlyZIlbNu2jWHDhjU4r3///qSnpwdcZ+bMmVx66aV4PB6ysrJQ1brWToM6F110EbGxsQDMmjWLtLQ0RIRPP/2UmJgYtm7dSv/+/dm6dSsbNmygb9++OJ1OkpKSAO/Qw8iRI9m4cSMVFRW4XC4WLlzImDFjmD17Ntu2bSM3NxeHw0FmZiZ79+7F5XLRrVs3OnToUDfm7fLl1M+AeBFJ5/uEI8BPgZ14h4H6A98AVzU6X4EaYIuvThhQtwVRIhDRxD0UCAfcgKdRnbeAsYf6+wtaAlPVauAGYDawGnhVVVeKyL0iMhJARPqISAFwAfCkiKysqy8iHwEzgDwRKRCRsw+8S+vjdjmY88Q9XNgnm0fe/4Y//W91SDz0skuXLpxxxhktHYYxR01lZWWDZS21tbX89re/5cEHHzxoHVUNqE7dcpezzz6blJQUVJV+/fo1W2fLli14PB7GjRvH9u3bcblcFBQUUFtbS2lpKddeey2dO3embdu2pKenU1ZWRnp6OqeeeipTp04FwO12s2bNGhwOBxUVFUyaNIk+ffoQFhbmrxMWFkZpaSnV1dVUV1eDt/VTpwBoq6oLgHlAFnArsERVV+Odx7ABiBSRpCbO3+KrM9d3PkAKgO/8pu6xxfeaXVdHVXcCnnp1mhTU/91W1XeAdxqV/aHe+0VA5kHqnh7M2FqS0yH85ee5RHlcPPPxesoqa7hvVA+cjqYnY+zfv5/hw4czd+5cPvzwQ2655Rb/sTVr1jBt2jRGjRp10PtVVFRwxRVXsGTJEpKSkpg+fXqDtWh19+jQoQM1NTWkpqaSmZnJZ599xk9+8hNeeOEFLrroIjZs2IDT6aSyspL169dz9913+9euVVdXs3r1aoqKioiOjuass85i7ty5uFwupkyZwn333QfAXXfd5V+IXd+MGTO45557WL16NQsXLqR3797+uLp3747D4cDlchETE8O6devo1asXu3bt8tf/6quvSE1NJSUlhW+++QaXy8WOHTtwOp0MHz7c/13efvv7ZYUTJkzgkUceoba2loiICLZv387q1av51a9+RWlpKU6nk7y8PN59911Wr17NbbfdxpQpU9i+fTvZ2dmUlZURGxtLdXU1kZGROBwOqqur2bFjB3v27KGiogKHw4GI4HQ6iY+PR0SIiopqUGfnzp0UFRVRXV1Nly5d+PbbbwEICwsDvL/w2rdvz80338zUqVMpLi5mw4YNqCo1NTVERUURGRnJ7t27qaqqAiA8PJy2bdsSERFBTU0N5eXllJaW+uNyOp0kJiaye/duampqiImJITExEVXF6XQSFRVFWVkZO3bsoKyszP9dwPtL/KyzzmLFihVERkY2qLNt2zaKi4v932XdunU4HA4cDoe/FRIREdHgHsXFxezbt4/KykocDgcej4eamhqqq6uJjo4mOTnZP+bj8XjYtWsXO3bsoLy8nOrqalQVl8uF0+mkuroal8tF27ZtiYmJoaamBqfTyYYNG9i7dy81NTXk5OSwcuVKXC6X/+8hLS2N8vJyLrjgArKysli9ejX//ve/efrpp6msrMTtdnPhhReyb98+ysvLOeuss9iwYQP79u07aJ0nnniC1157jccff5zOnTvzpz/9iTvvvPOAOtOmTaO0tJSSkhLKy8t58sknufHGGykqKmLq1Kl8/vnnJCUlkZ6ezsqVK9m5cycFBQWkpaVRUFDAmjVrmD59Oi6XC4/Hw86dO/nXv/5FZGQkmZmZvPrqq4gIhYWFJCYm4nQ6GTVqlP/vAVARiVHVulnfKiKdgRPxTsSbDwwUkUF4W1cK7AYymjg/E1gJnC4ig1T1Q1+dnb7zSw5SB2BOvToA2xvVOdCh+hhby6s1jIE1Vltbqw/MWq3ZE97Wm175Qiura5o8r6k+d1XVkpISTUhI0LKysmbv89hjj+kvf/lLVVV/33hT97jxxht1yZIlmpOTo++9956+9dZbOmLECL399tv1/vvvV1XVG2+8UX/5y19qdna2FhUV+eu/9dZbOmTIEP/ne+65R1988UUtKSnRDh06aElJie7YsUM7dOigO3bsOOD+q1at0jVr1ujgwYN10aJF/vI777xTMzIytLy8XPPz8zU9PV3feOMNHTFihP+c6upqTU1N1Q0bNqiq6oknnqhpaWn+4/W/S52ioiJ1Op36+eef6wMPPKARERH6yCOP6Nq1a/Xrr79WVdXNmzdrcnKyLly4UAcPHqyPP/64ZmZmalZWlv7qV79Sp9OpW7Zs0ffee0+zsrK0urq6QZ0BAwaoiOiyZct0x44dGhYWpg6HQ8vLyxvUmT9/vqakpOhpp52mr7zyioqIPvLIIzp79mx1OBzapUsXzc/P18zMTF29erWqqs6ePVtTUlI0KytLk5KSNCsrS0tKSvSCCy5QQB977DGdPn265ubmHvBdBg4cqCKibdu21ZKSEu3evbsCumXLFl2zZo126NBBq6urdc+ePZqRkaEffPCB/7vMmTNHo6KiNCYmRt1ut5aXlzeos2jRIs3IyND+/fvrK6+8ok6nU9euXavXX3+9ApqTk3PAPdLT0/Wzzz7T7OxsdTqdesopp2htba0OHz5cHQ6H1tbW6ocffqi9e/dWVdU9e/ZoWlqafvHFFxoREaFOp1N37Nih+fn5KiIaHx+vqqpTp07Vn//856qq+vHHH2vbtm21bdu2+thjjymgK1as0KSkJHU4HDpu3DgdNmyYut1uzc7O1qSkJI2KitLrrrtOs7KyNDo6Wnv06KGRkZEaGxurTqdTq6qqDlonLi5OPR6PlpWV6Y4dO9ThcPjHtBrXiYmJ0djYWL3ooos0KipKCwoKNDo6WqOiorRLly56ySWXqMvl0szMTI2MjFSPx6MTJkzQqKgozcvL04EDB2rHjh01PDxcIyMj9dVXX9WLL75YHQ6HZmdna3h4uIaHh+uECRM0Pj5ec3Nzdfr06aqqGh8fr3iXMYWptydoLZAO3A78Hm9r6/+AOcAdwJPAl8AaoHPj833X2ABMBu7wfX4SyAc6N3UP/X7Owx/q6vg+L6mrc7BXiyeeI/VqjQmszqNzv9HsCW/rL6Ys0vKq6gOODxgwQNevX39A+ZNPPqmXXHLJIa8/bNgw/fTTT1VVtaqqSpOSkrS2trbJe6xfv15zcnJUVXXevHk6YsQI7dq1qxYWFqqqamFhoXbt2vWABDZ27Fh96qmn/J+XLVum55xzjr788ss6fvx4f/n48eP15ZdfPmisjRNYu3bt9Pbbb2/wXR599NEGyWj27Nk6cOBAVVUtLS1Vp9OpJ554om7ZssV/Tt13qfPEE09obGys/7vn5eXpSSeddEA8ubm5+vXXX+vgwYP1vvvu0/Hjx2t2dra2bdtWc3Nz/d+l/p9xXZ2srCxNTExUVe//rLhcLnW73f64Gtfp06ePnnfeeRoVFaUzZszQP//5zxoeHq4dOnTQLVu2NDi/rk5SUpJ27NjR/2eckpKigP71r39VVW3wd1cXV8eOHdXtdvvrtGnTRgGdOnVqg7iKi4s1KytLN2/e3OC7REVF+ZNY4+9SV6d///563nnnabt27VRVNSsrSyMiIjQ9Pb3Bd6l/j9TUVA0LC9P77rtPVVVTU1PV6XTqsmXLGnyX+nWcTqf/7+3rr79WQGNjY7WqqqrBf+t1ddq2bas9e/b0J7moqChNT0/XmJiYBvd4/vnn/RMykpKSNCcnR//85z/7v7vL5dIPP/xQX3/9dU1ISDigTmxsrMbExGh5ebk/roiICK2qqtJ//vOfmpKS0qBOdna2XnXVVZqamqq1tbUaFxenTqdTExMT9aOPPtLOnTvrxIkT9brrrtPMzEytqqrSqKgo7du3r8bGxurvfvc7jY2NVUDLy8v11ltv1RNOOEFVVceMGaMnnHCC1tbWalpamoaFhemiRYt06dKl6nQ668auBJgAFKo3eVwEvOdLRoPwrsn9GXA1sA/YDJwGLGx0vstX50PgPN+xEXgnirjwjqE1VScMeL9eHfHdw6XN/N63vRBDwK+HdOaPI3OYs2obv5iymH2V1f5jjfvp65s2bRpjxx44ztl4mu+SJUu48sor6dGjB//5z38O2H6qqbGAiy66iEsvvZSPP/6YLVu2+Aew09PT2b59OwDXX389brebE088kVmzZjFt2jTi4+M5++yz+e1vf8vs2bO5++67+fzzzznppJPo0aMHU6ZM4eabbyYnJ4fExETi4+M599xzGTRoUF1/PPfccw+JiYm43W42btzItm3fP44mMzOT4uJiAB599FE6d+7M2WefTUZGBl26dKFTp06ICCLCySef7J+CDN4xhq5duyIiTJgwgb179/Kf//yH/Px8Pv30U1asWEG3bt38E2gWLlxIeXk5EydOZOHChTz00EP+fzjbtm0jLi6OzZs3++PavHkzCxcupLKykk6dOrFr1y66d+/OVVddRWpqKjU1NWRkZPDJJ580WSciIsK/fOLOO+/koYceorKykvbt2/PJJ5/4z6+LrbLSu6awW7duZGVlUVlZyc6dO4mOjmbZsmUHjaukpASn04nD4aCystLf1Vr3dIL4+HguvvhisrKymDBhAhkZGf7vAt7/6a2srCQlJcX/XRrX8Xg8rFixgjPOOIOhQ4eyadMmoqOjOfHEE/nkk0+avEdZWRkul4vnnnuO3NxciouLiYuL829KnZyczKBBgxrUqa2tJS8vD8A/DpSRkcGXX36Jy+UiKiqKHj16kJWVxW2334HD6eLb/Hyqa2qY+fEyomLi2LtvP+UVFcxZuo7Y5FTeXbSK9cV72V5azqINO9izdy8lu3Yz+S8PsG/fPvbv34+I8Or/3mP+ohVExsQdUGd/eTk4nETHxNA9Jwe320NcShoPvDSbu++7H/FE85f/fMLf/v0aTz3zHBs3FfD8lBfYXlyCM8zN7tJS3DEJ7Crdy4ifj6VgWzEfbyjjO0mjZPcePOHhlJWVsWz5V+wrr+CJZ573dwXn5OTw6quvUlFRQWFhIXl5eRQVFeF2u9m6dSs1NTX069ePfv368dOf/hS8yWUd8EegbgfwNLzJJhtvYikDHsKb5BYDccATQJWIZACvARHAfl+dU/h+44pteJPRWuBp4Hpf+WvAt8AKvK26L1X1v75jpwKfqXcuxUHZlLMQMW5geyLdTib8ZznjnlvIs1f2ITY8jOLi4gO2nwLvL+MVK1Y0WMdVp/6O9//73/8oLy9n9uzZpKSkMHjwYGpraxssfm58j127dpGQkMBLL73ErbfeyldffdVkzAkJCdx0001Mnz6d0047jZtvvpl9+/Zx8803c+2117J69Wrat2/Pnj17WLBgAQ8++CC/+93vyMnJYe7cubz22mt4PB6efPJJ8vLymD59OgB79+4lJiaGJUuW0KtXL2bNmuUfy6jvtNNOY9iwYZxwwgl89tlnfPnll5x//vl8+umnlJWVMWnSJPr27cuvfvUr/vKXvxATE4PT6SQ3N5czzjiDPXv2MHnyZHbu3InD4SA8PJxZs2Zx1lln8cEHH3D55ZfTqVMnzjnnHEpKSujRowfx8fHU1NQQERHB2rVrG/w57tq1izvvvJMpU6ZQXV1NWVkZHTt25Pnnn2fTpk106NCBysrKBtt11a8zYcIECgsLufvuu5k0aRJXXHEFr7zyClu3bvXXERG2bNnC5ZdfzjPPPMMZZ5zhTyzFxcWoKklJSezZ8/0mNvmbtjDhtlu57c+P8NTsJezZW0aXPmfw4ow3mf6/91EcKLXM/+o77n9nNRt213D2NXeQ06s3f5v0SyI692Xv3jJiUjJY8t1OamsVcTjAGcbCld/Sqe8uKsTNjb/7P3JP6cOt116Gw+Fg8+bNZHTpwSU3TOCzzwdQWV3D9l17mPfFGnZVObjkprvo1ONkJt/6C2qyTqGsbB/tT8xl1HUTiY5P4Ksrz2FfRSXvLV1HzPx1FO9Xfn7bg8QmpfDg769nc3g2qkpRWBsem7eOfz37Aoiwo9LJhBfmE7+wgqKyalKu+D8y9+7n9vv/SNXuXVAD0bl5XHjhRVSW7PAtWHUw7tHZ7Ny8m4mvf4UnrTNkdOBnD/2PyspK9iR0I/zkXrTLGcLWF++gYts6XvhgFTWlxVTXRh5Qp6qqija/+AeJMd5dcra+eAfbS7bw9/9+zv7ottRW7efVJZtJGHQDuWf9Bmf5Lpb+/VrO/tObhHvcFH+3lm/em0ZtbS0pvc5k69K5VHQ6g4qIWE64fSBVNbVU1yjVtbVU1Sgnt4sn/KNHKS4uZs6cOVx22WX87W9/IyUlheuuu47rrruuyX/Do0ePBshX1Z4i8iLeBIWqPgI80lQdEXkE+KN6d0tq8E+yyZvA5cD4xuerag0H3yrwcuBfBznmZwkshFzQO4soj4vfTFvKpU9/zpSr+xIREXHA9lMAr776Kj/72c/8A/31vfTSS7z88ssArFq1ivT0dLZs2UL79u3p2bMnr732WoPtpxrfo7S0lHHjxlFRUUF6ejpffvklhYWFZGRksGXLFtq0aUNFRQVLly7l4Ycf5qmnnmLs2LHk5eUxf/58tm3bxrhx43jooYc477zz+P3vfw/A448/TkxMDKmpqYB3N/358+cDMGrUKCZNmgTA1q1bOe2004iPj8fhcBAXF8fChQsZMGAABQUFnHKKdx3kySefzMyZM3G5XJx55pmoKl999RUOh4OEhAQiIyPp378/u3bt8rc43W43kZGRtGnThvLyct599106duxImzZt2LBhA1lZWbRv3568vDx+97vfcc8993DNNdfw4osvktImla/zN1CjoAile/awZW8tH3xdxBcrv2b2+/O59Nd3sMWTyZ3/nIrL7WHld9t49uP17C3dhSIU79jJ4oK9PPNRPp8sW83/5sxj1C9u5auqNqzbtIWouEQWf7eTZz9ez05nPLWqFG4vYXHBXhav+Ia4nDJunTSUcy67kekfrcLpCmMbCSxa+BVV2auoqamhqHQ/S7fXcNrkuSxcupqPxt9I3GljeXCpUrr4aQiPoTQqk/iuEXQ//zre+8PP0epatlR6mLJgA9+tzWdjSn/e3VdMsSOFW+7+C+oMY/5Xm/j5459SXl0LtTV8V7yXZxYUMH3fJ2z7fCWfh52EZ30+xTWJlOcvg/A4npjzFS8Wt6OssgapEdas30zhF9vYt2Yty6NOxVO0jWJHCr//819Rh4vtksgLc5YQ3vFUVJyU79/H4m21fDNrLZu/28iLK/bgig6j2NWGf/z97yDCfxes4r3tkewu8+5qU1Zewd4aJ8lSi1aUcUbPjsRFunl7VU+Wzn+HLj170XfgiVz966d46Pe3Mff/zQTg4bF9uGPe3/jHtXkkt/FuIjBjypM8HRHJz0/PJTwinMuv6ceDG3oz6818xg05iQuvuo5Lzu7fZJ0nruhLfGIS4WEO/lLQl88+nMsDl/TmnBfuom+vnsz/0wX+reQeeeQRTrrkQp76zRm+f4UDea6L8vbbb3P92D6c9eykg/7eUFVqapXnw1bhdDqpqanhxRdfPOj5dSorKxk1ahRvvPFGhe86lx2yktdXTSSvI3l+4HWa619sTa/WPAbW2NzV27Trne/o0Ifn66L1JZqZman79+9vcE6/fv107ty5DcomTpyo06dP19TUVH/Z7NmztWPHjnr11VdrUVGRtmnTxj+wv6+iWueu2aZvLi3QtIy2Wrq3TNevX68ej0c3bdrkHzeKi4vTu+66S1VV77//fr399tu1Xbt2mpKSol9++aU6HA6dP3++Xn755Tpv3jx1uVxaXFysJ5xwgpaUlKjD4dALL7xQAXU6ndqvXz895ZRTdMqUKf57FJTs1biEJO1yUl+NSU5Xd0yiRqR2UEeYR53hUTr0+j/pr5/4f5qYlql3PPKi9j49Tz/+pkjPGTlaIyIi9Y4JE/Txxx/XK664Ql0ul7Zr104XLFign3/+uaalpekTTzyhp556qo4YMUKTk5O1U6dOGh0drUVFRRoZGalOp0u7nJCjD7/4X42OS9DE9Cy9/IFpmtS+u2b0Ga4SFq7hHU5VZ2wbdcYkqyPCO96QcPYNmnT+BEUcGj/kGs2e8LZmT3hbI084XR1xaeqIjNPM30zTtje9ooAiDk0dO1njz/rlAXUcEXEa3escdSVmarvfvqHJY+5RQB2xbTR51CR1RCaoIzJeE8689vt7RMRoxrVPqjMuVeOHXKPidCmIDvnN3/WCu5/TME+Etu/WU99ZXqiLN+zQk0/toz1zc3XGjBnavn173bJli4aFhSmgS5cu1aefflqjoqK0dF+5rtu4RTt06qw/GZKnWdkdNLlNms78fK2GR0R6JyVEROrfXnhTr7llkkZGx+icrwp15mdrNTO7o8bExuk5o8dqu05ddNbyjRqXkKiAdujcTX//l79pXHy83nXfX3TZN5u0Q6fOOiRvqCYkJOrf/vmYDh12tu7eu09drjAVEZ334Ud62x13aGpqqu6vrNbCbUXauXMXzenRQ91ut/bsmas333qr3njjjepwODQuLk4LCwv12muv1a5du6qq6o4dO7RLly7qcDh02rRp2r17dy0vL9dHHnlERUSTk5MbjDXV/3eWmpqqH330kebm5mp5ebk+88wzKiK6ceNGXbBggfbp0+eAf5upqakNxojffvttjY6O1s2bNx+0TuN/z6qqPXr08I9hBguwWEPgd/jhvFo8gCP1OpYSmKrqp+uKNecPszR7wtuacPLZet6kf+n0RRt16+79un79es3IyNCamoazFkeMGKEzZ87Ubt26NSi/5557NC4uTiMiIjQ+IVFHXPFrveyZz7TDTVM1vOOpmj3hbY3qOVTD2/VUT6z3F43T5dKIyCgNDw9Xp9OpPXr00M6dO2uXLl00PT1dHQ6HOp1O/yDyjBkzdPz48f4ENmPGDL311ltVVTUiIkKzs7PV4XBoTEyM3vXnh/XpOcvUExGlOef9QmO69tWUn/1OEVGcLhVXmMa37ai/fWWRxrbJ1LCoGA2LTlBPUqaGJbVTR0SsisutzuhElbAIFU+UxvYfo5HZPbXzFX9WhztSHWHheso9s7TjRb9XcbrUGR7t/+XuSc7Ubtf+Q11R8Yo4FFBXUqZGdh+sMX3OV0DF5dHwpLYKqDsyRp2uMHU4XSoivkQkGh4eoXEJiRrtGzzP7thJT8jpqW0zs7zX9M06FBENCwvTpORkjY6J0fYdOqjL5b1Wz9xcbZedXbcjgiYnp2h4eLiGhYWp2+3WqKhodTgc2qlzF01OSfHXSUtP9yY3h0PT0tJ1wGmnaXh4uEZERKjb7daOHTtqZmamiojGxcXp/2/vzOOjqrI8/vulshGyg4GwBVISEkkCdNikjRBUiIYWF2y0AcEBx6UHcKEVZsFuPy1qI7TOgG2zKNiuOEbpsfm4MQq0OIAGUBRRENqFaKDBqIhB4Dd/vJeiSCekwIRKVZ3v51Ofeve+e989p+pVnXfufe+cXr16+cqxsbFKSEhQbGys3zitlZ2drfT0dMXExKh79+4qKCjQpEmTfH1qDR0AxcbG+saJiYlRUlKSCgsLlZWVpZSUFAFQmzZtFBcXp9jYWEVFRalTp06Kjo5Wx44d5fF4fDdUXH755SKpqKgoRUdHKy4uzidXfHy8srOzlZaWJo/Ho8LCQhUUFKi4uNj32SYnJysqKkpdu3ZVcXGxEhMT5fV6FRMTo/T0dBUWFiozM1OJiYkCoMzMTKWlpfnGyMnJUevWrZWfn6+srCyfAZs5c6bvPM/MzFRRUZGys7Pl9XqVkpKi7Oxs5efna9CgQfr888/r7TNx4kRJUmVlpTIyMnx9/G9Uauj3/MUXX/yDoWsOzIC1gFe4GTBJ+urAIb2webfG3fuU2vQ6z3elXnr/at29YqvWbt+rmh+OnfTDhg3Tvs/CqpkAAA3cSURBVH37lJWV5av75vsf9NKWSs0of0eD7l6phLzByhh1h86b87ru/J/3tGpblbZWVmvuky+qd8nPdPXidUo+s0jtx85W1u0vqMuvlis6IVljFr6pWX95X+UVn+r93dWq+PATdejUWcvXbFS37rkaNeEG3f/0y/qPB59SbHwrndX/XN22aIVuW/q60jtk6Z/++JqiWyUqseA8tfL2U9btLyil1zDlXHiNvEXFWvLGTqWkpWtP9QHNmjVLs2bNkiQtWLBAcXFxx+6iPHxEe775Xtu++Fpv7tirv7yzW0kpaep7wSW6Y/kWzSh/R62S0xSfmKIZ5e9owMjxSj4jU5MXrdTVd/9JXXr2VZeefTXhd09qxJS7VFQ6WueMHKusnLNUfEGZFjz5vDp3ydIjS5aqsrLyuM9y9erVuuiiiyRJFRUVysvL08KFCzVt2jRt3ry50e/zvvvuU35+viRpzJgxqqqqOqk+gYzj3z7QPrfccosGDhwYcPu64wSiS0VFhQYPHuzz5MvKylRTU9OkY9TVJdA+U6ZM0auvvhpwn7lz52rRokWNHjcYfU6FUDZgtgbWgklJiEFZYSbKCkdjcZtvMWD4IKzZvg+rPqzCojUf46FVO5AYF41B3jYY3OMMLH7qOaSmtkLNocP4r5ffxxu7qvHWzr2o+e5bJKekITd2P9JrKrF6/i3olpGMGTNm4O8/9Me5l16K3CuHI+W7zzF+fBH6H7wGGyo24dpJE7D0scexqnAA9h34AY+8sQuHjhzLfPNl9UHcsHQd9uz/DkfblWFDxSF8/8nHOBKXjF3VR7Bs+1F8u24J0P4sbK8WYmJj4al8D+eXXozf3tAPFz6xFWd2jAbbJ2NEjyTc0y4DbZMTcPbZZ2Pq1Km4+eabsXbtWkRFRaGoqAjr16/HvHnz8Oijj6JtYtyxzympNaq2VWBqsbOe8CCPol2HdrjzZ3m4OPN6TJnyPv5z4lAcOXIEP3lyNhITE/HLkjMxf/5LKB1RjCFDhqC0tBR7P/sYhd3aI9oThXFjx8Dj8aBz587Ytm0bevTogZUrV/pumOjTpw9iYmKQl5eHSZMmBfR9dunSBdddd13AaxR1+5woQkR97T0eT0B98vPzkZubG/AYdccJRJc+ffogLy8PN910EwAc90B5U40BHK/LyfSpvZMxkD6pqakYN25cQMc+3X0ijmBb0KZ6haMHdiK+PnhIL/p5VrXeWf4dL6p1wQXKGP1bDf/9Kt35/CZ19eYoNy9PAwYM0MaNG33HKCsrO+65oloOHjyoUaNGyev1ql+/ftqxwwk8vOuTT3Xu0Au0fNPneq7iM7Xv1EVJKanyREcro30HZXXzKr1NW8XFxSs2Lk6ZmZkqKSlR//79lZ+fr9TUVHm9XuXk5Khnz55KT09XQkKC4uPjlZ6erssuu0ySVFJS4nsmKCkpSc8++6wk6ZlnnvE9/yNJDzzwgG86KiUlRUlJSfJ6vVq8eLFuvPFG39TT0qVLJUnl5eVq27atSMrj8ahDhw4+L2D69OmKi4tTTk6OVqxY4fssNm7cqKKiIhUUFGjkyJG+h7APHTqk3NxcC35shDwIYQ8s6AI01SvSDJg/R48e1UdffqOFq3fotmc2657HVmjU6Ksa7fdjI95XVFRo7Nixp9zef7rm0ksv1QcffHDC/oFOb50KJztdU15e7psOM4xQJpQNmE0hhgEkcWZGIs7MSHRrCvFwTWW9z07582Mj3vfp0wclJSWNjtNQ+9rpmtrbeXv06HHC/oFOb50KJztdc/jwYdx6663NJo9hGI1DxwCHPn379lVtJAHDMAwjMEi+LalvsOU4FSyUlGEYhhGSmAEzDMMwQhIzYIZhGEZIYgbMMAzDCEnMgBmGYRghiRkwwzAMIyQxA2YYhmGEJGHzHBjJPQD+1kiztgD2ngZxWiqRrH8k6w5Etv6m+4nJknTG6RCmqQkbAxYIJN8K1Qf2moJI1j+SdQciW3/TPXx1tylEwzAMIyQxA2YYhmGEJJFmwBYEW4AgE8n6R7LuQGTrb7qHKRG1BmYYhmGED5HmgRmGYRhhQsQYMJKlJLeR3E5yerDlaW5IPkyyiuQWv7p0kq+Q/Mh9TwumjM0Fyc4kXyO5leR7JKe69WGvP8l4kutJbnZ1/41b343kOlf3p0nGBlvW5oKkh+RGki+45UjSfRfJd0luIvmWWxe2531EGDCSHgDzAVwI4CwAV5E8K7hSNTtLAJTWqZsOYKWk7gBWuuVw5DCAWyXlARgI4Jfu9x0J+tcAGCqpF4DeAEpJDgRwL4Dfu7rvBzAxiDI2N1MBbPUrR5LuAFAiqbff7fNhe95HhAED0B/AdkkfSzoE4CkAI4MsU7MiaTWAfXWqRwJY6m4vBXDJaRXqNCGpUlKFu/0NnD+zjogA/d0s8d+6xRj3JQBDAfy3Wx+WugMAyU4AygAscstEhOh+AsL2vI8UA9YRwKd+5c/cukijnaRKwPmTB5ARZHmaHZJdAfQBsA4Ror87hbYJQBWAVwDsAPCVpMNuk3A+/+8HcBuAo265DSJHd8C5WHmZ5Nsk/9mtC9vzPjrYApwmWE+d3X4Z5pBMBPAsgJskfe1cjIc/ko4A6E0yFcBzAPLqa3Z6pWp+SI4AUCXpbZJDaqvraRp2uvvxU0m7SWYAeIXkB8EWqDmJFA/sMwCd/cqdAOwOkizB5EuSmQDgvlcFWZ5mg2QMHOP1uKRytzpi9AcASV8BeB3OOmAqydoL1nA9/38K4GKSu+AsEwyF45FFgu4AAEm73fcqOBcv/RHG532kGLANALq7dyPFArgSwJ+DLFMw+DOA8e72eADLgyhLs+GueywGsFXSXL9dYa8/yTNczwskWwE4H84a4GsARrnNwlJ3STMkdZLUFc5v/H8ljUEE6A4AJFuTTKrdBjAMwBaE8XkfMQ8yk7wIztWYB8DDku4KskjNCsknAQyBE436SwB3AHgewDIAXQB8AuAKSXVv9Ah5SJ4DYA2Ad3FsLeRf4ayDhbX+JAvhLNR74FygLpN0J8lsOF5JOoCNAMZKqgmepM2LO4U4TdKISNHd1fM5txgN4AlJd5FsgzA97yPGgBmGYRjhRaRMIRqGYRhhhhkwwzAMIyQxA2YYhmGEJGbADMMwjJDEDJhhGIYRkpgBM4IOSZGc41eeRvLXTXTsJSRHNd7yR49zhRv9/rU69V1d/Sb71c0jOaGR411P8upG2kwgOa+Bfd/WV99UuHr5Zzq4lmRFOEU6N1o+ZsCMlkANgMtItg22IP64WQwCZSKAGyWV1LOvCsDUk0njIekhSY+exPhNhl/UikDbjwMwGcAwSfubRyrD+EfMgBktgcNwUp/fXHdHXQ+q1rMgOYTkKpLLSH5I8h6SY9xcWO+S9Pod5nySa9x2I9z+HpKzSW4g+Q7J6/yO+xrJJ+A8CF1Xnqvc428hea9bNxPAOQAeIjm7Hv32wEljMb7uDpJeki+6wVfXkMx1639Ncpq73c+V8U1X5i1+h+jg9v+I5O/qHHuO6xWtJHmGW9eb5P+5x3uu1mMi+TrJWSRXwTG2V7g6bia5uh6dasf4OZz0HMMk7W2onWE0B2bAjJbCfABjSKacRJ9ecHI/FQAYByBHUn84qTQm+7XrCmAwnDQbD5GMh+MxVUvqB6AfgGtJdnPb9wfwb5KOyxlHsgOc3FJD4eTa6kfyEkl3AngLwBhJv2pA1nsA3FqPV7cAwGRJRQCmAXiwnr6PALhe0tkAjtTZ1xvAaPczGE2yNuZnawAVkn4CYBWcSCwA8CiA2yUVwjHQd/gdK1XSYElzAMwEMNzNK3ZxAzplAZgHx3h90UAbw2g2zIAZLQJJX8P5c51yEt02uLm/auCkDHnZrX8XjtGqZZmko5I+AvAxgFw4ceKuppN2ZB2ctBvd3fbrJe2sZ7x+AF6XtMdNz/E4gHMD1G8ngPUAflFbRyda/iAAz7hy/BFApn8/N65hkqS1btUTdQ69UlK1pO8BvA/HqABOCK2n3e3HAJzjXhykSlrl1i+tI//TfttvAFhC8lo4YanqYw+c0EQ/b1Bxw2hGIiWdihEa3A+gAo7HUcthuBdabpBe/3Uk/3h2R/3KR3H8uV03XprgpNmYLOkl/x1uDL0DDcj3Y/OxzIKTWLF2Si4KTq6q3ifo09iY/p/BETT8mw4kZpxPb0nXkxwAx2vdRLK3pL/Xaf8dnCznfyVZJenxAMYwjCbDPDCjxeAGGF2G41O+7wJQ5G6PhJNh+GS5gmSUuy6WDWAbgJcA3EAn7QpI5rgRvE/EOgCDSbZ1pwKvgjM9FxCSPoDjJY1wy18D2EnyClcGkuxVp89+AN+QHOhWXRngcFE4FoH9FwD+KqkawH6SxW79uIbkJ+mVtE7STAB7cXw6In/59gAoBTCL5PAAZTOMJsE8MKOlMQfAv/iVFwJYTnI9nBshGvKOTsQ2OH/U7eCsJX1PchGcacYK17Pbg0ZSrUuqJDkDTnoOAlgh6WRTU9wFJyJ6LWMA/IHkv8Mxzk8B2Fynz0QAC0kegJPfqzqAcQ4A6Enybbf9aLd+PJx1wAQ406nXNNB/NsnucPRcWY9MPiTtJHkxgBUkL5O0LgD5DONHY9HoDaOFQzJRUu3dl9MBZEqaGmSxDCPomAdmGC2fMtfziwbwNwATgiuOYbQMzAMzDMMwQhK7icMwDMMIScyAGYZhGCGJGTDDMAwjJDEDZhiGYYQkZsAMwzCMkMQMmGEYhhGS/D/HN1RNYozj0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.161 0.122 0.11  0.107 0.106 0.107 0.106 0.106 0.107 0.107 0.107 0.107\n",
      " 0.107 0.107 0.107 0.107 0.107 0.107 0.107 0.107 0.108 0.108 0.108 0.108\n",
      " 0.108]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    print(k)\n",
    "    knn_kd = KNeighborsClassifier(n_neighbors=k,algorithm='kd_tree')\n",
    "    scores_kd = cval_score(knn_kd, d_train_bg_dense,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores_kd.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 9.736\n"
     ]
    }
   ],
   "source": [
    "knn_kd_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_kd_optimal.fit(d_train_bg_dense,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_kd_optimal.predict(d_train_bg_dense)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_kd_optimal.predict(d_test_bg_dense)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.995\n",
      "The value of TNR is 0.056\n",
      "The value of FPR is 0.944\n",
      "The value of FNR is 0.005\n",
      "\n",
      "Test error is 10.900\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGatJREFUeJzt3Xt4VdWZx/HvLwmIgBUUQQSqqHitj6KgtGoHxUGwKjheqrWKlk6cVh21rdZqLeOtY+vUC606xcqI9QZeKLRSFVGqVhEQEfFW8EoEQQERAa1J3vnjbOIBk3CQJCdZ/j4++8k5a6+99zo88T1v3r323ooIzMys5Ssp9gDMzKxhOKCbmSXCAd3MLBEO6GZmiXBANzNLhAO6mVkiHNDNzBLhgG5mlggHdDOzRJQVewB1KWvdzZew2ufs1rFHsYdgzdDcxdO0qfv49P3XC445rTrtuMnHawzO0M3MEtFsM3QzsyZVXVXsEWwyB3QzM4CqymKPYJM5oJuZARHVxR7CJnNANzMDqHZANzNLgzN0M7NE+KSomVkiEsjQPQ/dzAyIqsqClw2R9KakFyTNljQza9tK0mRJ87KfHbN2SRopab6kOZL2zdvPsKz/PEnDNnRcB3QzM8idFC10KcwhEbFPRPTJ3l8ITImIXsCU7D3AYKBXtpQDN0HuCwAYARwA7A+MWPslUBcHdDMzyJVcCl2+mCHAmOz1GGBoXvttkTMN6CCpK3A4MDkilkXEcmAyMKi+Azigm5lB7qRoocuGBfCwpGcllWdtXSJiEUD2s3PW3g1YkLdtRdZWV3udfFLUzAw2KvPOgnR5XtOoiBiV9/7AiFgoqTMwWdIr9e2uttHU014nB3QzM9ioS/+z4D2qnvULs59LJI0nVwNfLKlrRCzKSipLsu4VQP5tRLsDC7P2/uu1T61vXC65mJlBg50UldRO0hZrXwMDgbnARGDtTJVhwITs9UTg1Gy2Sz9gRVaSeQgYKKljdjJ0YNZWJ2foZmZARINdWNQFGC8JcjH2zoh4UNIMYJyk4cDbwPFZ/0nAEcB8YDVwem48sUzS5cCMrN9lEbGsvgM7oJuZQYNdWBQRrwN719K+FBhQS3sAZ9axr9HA6EKP7YBuZga+OZeZWTISuPTfAd3MDKDq02KPYJM5oJuZgUsuZmbJcMnFzCwRztDNzBLhgG5mlobwSVEzs0S4hm5mlgiXXMzMEuEM3cwsEc7QzcwS4QzdzCwRlYU/4KK5ckA3MwNn6GZmyXAN3cwsEc7QzcwS4QzdzCwRztDNzBLhWS5mZomIKPYINpkDupkZuIZuZpYMB3Qzs0T4pKiZWSKqqoo9gk3mgG5mBi65mJklwwHdzCwRrqGbmaUhqj0P3cwsDS65mJklwrNczMwS4QzdGtpmm23G1Efvo/Vmm1FWVsr99z/ApZf9hqmP3k/7LdoD0HmbrZkxczbHHje8yKO1xrTFV9pz6TUXsfNuO0LAJeddwWFH9OdfBh5E5aeVLHizgp+fcwUrP/yIbx17OKf/8OSabXfZY2eOP2wYr744r4ifoIVJIKArmukNacpad2ueA2sC7dq1ZdWq1ZSVlfH41PGc96MRPDN9Vs36cWNHMfHPD3P77fcWcZTFsVvHHsUeQpO5cuQlzHrmee67YyJlrcrYfPM27NV7D5558lmqqqo47+dnAnDtFTess12v3Xdi5JhfM3j/Y4sx7KKYu3iaNnUfq687o+CY0/bc32/weJJKgZnAOxFxpKSewN3AVsAs4JSI+KekzYDbgP2ApcC3I+LNbB8/A4YDVcB/RsRD9R2zpNAPsLEk7Sbpp5JGSro+e717Yx0vJatWrQagVasyylq1Iv9Lt337dhzS/0AmTHiwWMOzJtCufVv2+3pv7rtjIgCVn1ay8sOPeOpv06nKar1znp1Ll+06f27bI475V/46fnKTjjcJ1dWFL4U5B3g57/2vgGsjohewnFygJvu5PCJ2Bq7N+iFpD+BEYE9gEHBj9iVRp0YJ6JJ+Su6bSMB0YEb2+i5JFzbGMVNSUlLCzBkPs+idOUyZ8jjTZzxXs27o0ME8+tjfWbnyoyKO0Bpb9+27sXzpcq64/hLueWQMl15zEZu3bbNOn2O+cxRPTnn6c9sOGnIYk8Y/3FRDTUd1FL5sgKTuwLeAP2TvBRwKrP2zegwwNHs9JHtPtn5A1n8IcHdEfBIRbwDzgf3rO25jZejDgb4RcVVE3J4tV2WDceF3A6qrq+nTdyDb9+xD3z692XPPXWvWnXjCEO4e+6cijs6aQllZKbvvtStjx9zP8YcNY83qNQw/+9Sa9eXnnkZVZSV/uW/dv9T22ndP1qz5mPmvvN7UQ275qqoKXiSVS5qZt5Svt7frgAuAten81sAHEbH2KRoVQLfsdTdgAUC2fkXWv6a9lm1q1VgBvRrYrpb2rnz2AT8n/x+punpVIw2t5Vix4kP+9vhTHD6wPwBbbdWRvn17M2nSlOIOzBrduwuXsHjhe7ww60UAHv7zo+yxV+6L/egTjuCb/3ogP/3hiM9tN3joYS63fEFRXV34EjEqIvrkLaPW7kfSkcCSiHg2b/e11dxjA+vq26ZWjTXL5VxgiqR5fPYN81VgZ+CsujbK/lFGwZf3pGinTlvx6aeVrFjxIW3atGHAoQdz9f/cCMBxxx7JA5Me4ZNPPinyKK2xLX1vGe8uXMwOO32VN197m34H9+W1f7zBgYf0Y/hZp3DaMT/g4zXr/h5IYuBRAzht6H8UadQtXMNdKXogcLSkI4A2wFfIZewdJJVlWXh3YGHWvwLoAVRIKgO2BJblta+Vv02tGiWgR8SDknYhV2LpRu6bpgKYEREtf/Z+I+ratQujb7mO0tISSkpKuPfeP/PApEcA+PYJR/Prq2/YwB4sFb+86Df86sZLadW6FQveeodLzrmCux8aTevWrbl53Eggd2L0sgt+DUCfr/dm8aIlVLxV7//zVpcGupdLRPwM+BmApP7ATyLiZEn3AMeRO784DJiQbTIxe/90tv7RiAhJE4E7JV1DruLRi9w5yTp52qK1KF+maYtWuIaYtrjqspMLjjntfnFHQcfLC+hHStqRz6YtPgd8NyI+kdQG+CPQm1xmfmJEvJ5tfzHwPaASODci/lrf8XxhkZkZQGXDFw8iYiowNXv9OrXMUomIj4Hj69j+SuDKQo/ngG5mBr59rplZMnz7XDOzNEQC93JxQDczA2foZmbJcEA3M0uEH3BhZpYGP1PUzCwVDuhmZonwLBczs0Q4QzczS4QDuplZGqLKJRczszQ4QzczS4OnLZqZpcIB3cwsES2/hO6AbmYGEJUtP6I7oJuZgTN0M7NU+KSomVkqnKGbmaXBGbqZWSqcoZuZpSEqiz2CTeeAbmYGhDN0M7NEOKCbmaXBGbqZWSIc0M3MEhFVKvYQNpkDupkZztDNzJIR1c7QzcyS4AzdzCwRES0/Qy8p9gDMzJqDqC58qY+kNpKmS3pe0ouSLs3ae0p6RtI8SWMltc7aN8vez8/W75C3r59l7a9KOnxDn8EB3cwMqK5SwcsGfAIcGhF7A/sAgyT1A34FXBsRvYDlwPCs/3BgeUTsDFyb9UPSHsCJwJ7AIOBGSaX1HdgB3cyM3EnRQpd695PzUfa2VbYEcChwb9Y+BhiavR6SvSdbP0CSsva7I+KTiHgDmA/sX9+xHdDNzNi4gC6pXNLMvKU8f1+SSiXNBpYAk4HXgA8iam4BVgF0y153AxYAZOtXAFvnt9eyTa18UtTMDIiNuB16RIwCRtWzvgrYR1IHYDywe23dsp+1pfxRT3udHNDNzGiceegR8YGkqUA/oIOksiwL7w4szLpVAD2ACkllwJbAsrz2tfK3qZVLLmZm5KYtFrrUR9I2WWaOpM2Bw4CXgceA47Juw4AJ2euJ2Xuy9Y9GRGTtJ2azYHoCvYDp9R274Axd0mYR8Umh/c3MWpKqhruXS1dgTDYjpQQYFxF/kfQScLekK4DngFuy/rcAf5Q0n1xmfiJARLwoaRzwElAJnJmVcuq0wYAuaf/sgFsCX5W0N/D9iDj7C3xQM7NmqaEuLIqIOUDvWtpfp5ZZKhHxMXB8Hfu6Eriy0GMXUnIZCRwJLM0O8DxwSKEHMDNrCRpq2mIxFVJyKYmIt3LTImvUm/abmbU0GzPLpbkqJKAvyMoukdWEzgb+0bjDMjNrWs058y5UIQH9B+TKLl8FFgOPZG1mZsmoqm75k/42GNAjYgnZWVczs1R9KUoukm6mlquTIqK8lu5mZi1SdQK3zy2k5PJI3us2wDGse38BM7MWL4X7oRdSchmb/17SH8ndbMbMLBlfipJLLXoC2zf0QMwK8dyLdxZ7CJaoL0XJRdJyPquhl5C7NPXCxhyUmVlTS36WS3aT9b2Bd7Km6uymMWZmSUkhsNX7lZQF7/ERUZUtKXxmM7PPqQ4VvDRXhfyNMV3Svo0+EjOzImqo2+cWU50ll7wbsR8E/Luk14BV5J6iERHhIG9myagu9gAaQH019OnAvnz2IFMzs2RFrU98a1nqC+gCiIjXmmgsZmZFU9mMSymFqi+gbyPpR3WtjIhrGmE8ZmZFkXqGXgq0p/YnT5uZJSX1GvqiiLisyUZiZlZEqWfoLf/TmZkVKPUMfUCTjcLMrMiqEshh6wzoEbGsKQdiZlZMCTyB7gvdbdHMLDnVKWfoZmZfJincqMoB3cyM9E+Kmpl9aVTLJRczsyRUFXsADcAB3cwMz3IxM0uGZ7mYmSXCs1zMzBLhkouZWSJSmLZYyDNFzcySV6XCl/pI6iHpMUkvS3pR0jlZ+1aSJkual/3smLVL0khJ8yXNyX+Gs6RhWf95koZt6DM4oJuZkcvQC102oBL4cUTsDvQDzpS0B3AhMCUiegFTsvcAg4Fe2VIO3AS5LwBgBHAAsD8wYu2XQF0c0M3MaLiAHhGLImJW9nol8DLQDRgCjMm6jeGz5zUPAW6LnGlAB0ldgcOByRGxLCKWA5OBQfUd2zV0MzOgMR4pKmkHoDfwDNAlIhZBLuhL6px16wYsyNusImurq71OztDNzNi4DF1SuaSZeUv5+vuT1B64Dzg3Ij6s59C1fZVEPe11coZuZsbGXfofEaOAUXWtl9SKXDC/IyLuz5oXS+qaZeddgSVZewXQI2/z7sDCrL3/eu1T6xuXM3QzM3Lz0Atd6iNJwC3AyxFxTd6qicDamSrDgAl57adms136ASuy0sxDwEBJHbOToQOztjo5Qzczo0HnoR8InAK8IGl21nYRcBUwTtJw4G3g+GzdJOAIYD6wGjgdck+Nk3Q5MCPrd9mGniTngG5mRsMF9Ih4ktrr31DLs5ojIoAz69jXaGB0ocd2QDczw/dyMTNLhu/lYmaWCD/gwswsEdUJFF0c0M3MSONuiw7oZmb4pKiZWTKcoZuZJaJSLT9Hd0A3M8MlFzOzZLjkYmaWCE9bNDNLRMsP5w7oZmaASy5mZsmoSiBHd0A3M8MZuplZMsIZuplZGpyhW4Pr3n07bh19PV223Ybq6mr+8Ic7+O3vbuHOO25il112AqDDll/hgxUf0qfvwCKP1hrawGOH0a5tW0pKSigtLWXc6JH8dtRtPPrk05SohK06bsmVF/+YzttsTUTw39f9L088PYM2bTbjyot/zB677gzANTfewuNP5Z5cdsZpJzH4sH8p5sdqETxt0RpcZWUl519wKc/Nnkv79u2Y/syDPDLlcb5z8g9q+lz9q1+w4sMPizhKa0yjf3sVHTtsWfP+9JOP5ezyUwG4/Z4J3PR/dzLigrN54ukZvF2xkEljb2HOi69w+f/8jrtuvo6/PTWdl159jXtvvYF/fvopp515AQd/vQ/t27Ur1kdqEVp+OIeSYg/A1vXuu0t4bvZcAD76aBWvvDKPbtttu06f4447irvHTqhtc0tQfiBes+ZjlD1Z57Enp3H0oAFIYu+v7c7KlR/x3vvLeO2Nt+nbey/Kykppu3kbdu3VkyenPVuk0bcclUTBS3PV5AFd0ulNfcyWavvtu7PP3l/jmenP1bQdfNABLF7yHvPnv1HEkVljkUT5eRdzwvfO5p4Jk2rar//9rQw45hQeePgxzvr+KQAsfm8p23buVNOnS+dOLH7vfXbduSdPTJvJmo8/ZvkHK5gxaw7vLnmvyT9LSxMb8V9zVYySy6XA/9W2QlI5UA6g0i0pKfny/onYrl1bxo29mR/9ZAQrV35U0/7tbw9lrLPzZP3xpt/QeZutWbr8A/793IvouX0P+uyzF+eccRrnnHEaN982ljvv+zNnff8Ucg+LX5ckDjxgP+a+8g++e8aP6dhhS/beczdKS0uL8GlalhROijZKhi5pTh3LC0CXuraLiFER0Sci+nyZg3lZWRn3jL2Zu+4az5/+9Nea9tLSUo4ZOphx90ws4uisMXXeZmsAtu7YgQHf/AYvvPTqOuu/NbA/j0z9OwDbdu7Eu0ver1m3eMn7dO6U2/6MYSdx35gb+MP1vySA7btv1zQfoAVLIUNvrJJLF+BU4KhalqWNdMxk3DzqN7z8ynyuu37UOu2HDTiYV1+dzzvvLCrSyKwxrV7zMatWra55/dT0WfTacQfeWvBOTZ/HnphGz+27A9D/oH5MfHAKEcHzc1+mfft2bNNpK6qqqvhgRe6k+avz3+Af89/gG/vv1/QfqIWp3oiluWqskstfgPYRMXv9FZKmNtIxk3DgN/pyynePY84LLzFzxsMAXHLJVfz1wUc54YQhPhmasKXLlnPORZcDUFVZxRED+3NQvz6ce9EVvPl2BSoR223bmV+cfzYA3/x6X554egaDT/gem7dpw+UXnQdAZWUVp/7wJwC0b9uWq35xPmVlLrlsSFUtJayWRrXV4ZqDstbdmufArKjWLHyi2EOwZqhVpx21qfv4zvbHFBxz7nxr/CYfrzF4HrqZGb7038wsGc25Nl4oB3QzM3zpv5lZMlxyMTNLRAqzXBzQzcxIo+Tim3OZmdGwFxZJGi1piaS5eW1bSZosaV72s2PWLkkjJc3PrqjfN2+bYVn/eZKGbei4DuhmZjT4pf+3AoPWa7sQmBIRvYAp2XuAwUCvbCkHboLcFwAwAjgA2B8YsfZLoC4O6GZm5EouhS4bEhGPA8vWax4CjMlejwGG5rXfFjnTgA6SugKHA5MjYllELAcm8/kviXW4hm5mBrXevbKBdYmIRdmxFknqnLV3Axbk9avI2upqr5MDupkZULURJ0Xzb/WdGRURo+rqv6Hd1dIW9bTXyQHdzIyNm+WSBe+NDeCLJXXNsvOuwJKsvQLokdevO7Awa++/XvvU+g7gGrqZGbmSS6HLFzQRWDtTZRgwIa/91Gy2Sz9gRVaaeQgYKKljdjJ0YNZWJ2foZmY07Dx0SXeRy647SaogN1vlKmCcpOHA28DxWfdJwBHAfGA1cDpARCyTdDkwI+t3WUSsf6J1HQ7oZmY07KX/EXFSHasG1NI3gDPr2M9oYHShx3VANzPDl/6bmSUjhUv/HdDNzHBANzNLRnN9HOfGcEA3M8MZuplZMvyACzOzRFRFy3+qqAO6mRmuoZuZJcM1dDOzRLiGbmaWiGqXXMzM0uAM3cwsEZ7lYmaWCJdczMwS4ZKLmVkinKGbmSXCGbqZWSKqoqrYQ9hkDuhmZvjSfzOzZPjSfzOzRDhDNzNLhGe5mJklwrNczMwS4Uv/zcwS4Rq6mVkiXEM3M0uEM3Qzs0R4HrqZWSKcoZuZJcKzXMzMEuGTomZmiXDJxcwsEb5S1MwsEc7QzcwSkUINXSl8K6VOUnlEjCr2OKx58e+Fra+k2AOwgpQXewDWLPn3wtbhgG5mlggHdDOzRDigtwyuk1pt/Hth6/BJUTOzRDhDNzNLhAN6MydpkKRXJc2XdGGxx2PFJ2m0pCWS5hZ7LNa8OKA3Y5JKgRuAwcAewEmS9ijuqKwZuBUYVOxBWPPjgN687Q/Mj4jXI+KfwN3AkCKPyYosIh4HlhV7HNb8OKA3b92ABXnvK7I2M7PPcUBv3lRLm6clmVmtHNCbtwqgR9777sDCIo3FzJo5B/TmbQbQS1JPSa2BE4GJRR6TmTVTDujNWERUAmcBDwEvA+Mi4sXijsqKTdJdwNPArpIqJA0v9pisefCVomZmiXCGbmaWCAd0M7NEOKCbmSXCAd3MLBEO6GZmiXBAtwYnqUrSbElzJd0jqe0m7Ku/pL9kr4+u746TkjpI+uEXOMZ/SfrJFx2jWXPhgG6NYU1E7BMRXwP+CfxH/krlbPTvXkRMjIir6unSAdjogG6WCgd0a2xPADtL2kHSy5JuBGYBPSQNlPS0pFlZJt8eau4B/4qkJ4F/W7sjSadJ+l32uouk8ZKez5ZvAFcBO2V/HVyd9Ttf0gxJcyRdmrevi7P7zD8C7Npk/xpmjcgB3RqNpDJy93J/IWvaFbgtInoDq4CfA4dFxL7ATOBHktoANwNHAQcD29ax+5HA3yJib2Bf4EXgQuC17K+D8yUNBHqRuw3xPsB+kr4paT9yt1HoTe4Lo28Df3Szoigr9gAsSZtLmp29fgK4BdgOeCsipmXt/cg9tOPvkgBak7ucfTfgjYiYByDpdqC8lmMcCpwKEBFVwApJHdfrMzBbnsvetycX4LcAxkfE6uwYvj+OJcEB3RrDmojYJ78hC9qr8puAyRFx0nr99qHhbhEs4L8j4vfrHePcBjyGWbPhkosVyzTgQEk7A0hqK2kX4BWgp6Sdsn4n1bH9FOAH2balkr4CrCSXfa/1EPC9vNp8N0mdgceBYyRtLmkLcuUdsxbPAd2KIiLeA04D7pI0h1yA3y0iPiZXYnkgOyn6Vh27OAc4RNILwLPAnhGxlFwJZ66kqyPiYeBO4Oms373AFhExCxgLzAbuI1cWMmvxfLdFM7NEOEM3M0uEA7qZWSIc0M3MEuGAbmaWCAd0M7NEOKCbmSXCAd3MLBEO6GZmifh/cqaG39i451gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observation :**\n",
    "The accuracy is almost same when we are using brute-foce KNN and KD-Tree KNN in both the cases i.e. while using bigrams & n-grams and when we are not using bigrams & n-grams."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [18.3] TF-IDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Build The Vocabulary\n",
    "tf_idf_vect = TfidfVectorizer()\n",
    "d_train=tf_idf_vect.fit_transform(dw_train['CleanedText'].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the type of count vectorizer  <class 'scipy.sparse.csr.csr_matrix'>\n",
      "the shape of train-text BOW vectorizer  (14000, 15632)\n",
      "the shape of test-text BOW vectorizer  (6000, 15632)\n"
     ]
    }
   ],
   "source": [
    "#Vectorize train and test data\n",
    "print(\"the type of count vectorizer \",type(d_train))\n",
    "print(\"the shape of train-text BOW vectorizer \",d_train.get_shape())\n",
    "\n",
    "d_test = tf_idf_vect.transform(dw_test['CleanedText'].values)\n",
    "print(\"the shape of test-text BOW vectorizer \",d_test.get_shape())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Bigrams ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "##Build The Vocabulary using bi-grams and ngrams\n",
    "tf_idf_vect_ng = TfidfVectorizer(ngram_range=(1,2))\n",
    "d_train_ng=tf_idf_vect_ng.fit_transform(dw_train['CleanedText'].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the type of count vectorizer  <class 'scipy.sparse.csr.csr_matrix'>\n",
      "the shape of train-text BOW vectorizer  (14000, 281647)\n",
      "the shape of test-text BOW vectorizer  (14000, 281647)\n"
     ]
    }
   ],
   "source": [
    "#Vectorize train and test data\n",
    "\n",
    "print(\"the type of count vectorizer \",type(d_train_ng))\n",
    "print(\"the shape of train-text BOW vectorizer \",d_train_ng.get_shape())\n",
    "\n",
    "d_test_ng = tf_idf_vect_ng.transform(dw_test['CleanedText'].values)\n",
    "print(\"the shape of test-text BOW vectorizer \",d_train_ng.get_shape())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.3.1] ConVert Sparce Matrix To Dense Using TruncatedSVD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "svd = TruncatedSVD(n_components=4000, random_state=0)\n",
    "#without using bigram\n",
    "d_train_dense=svd.fit_transform(d_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3.34695510e-03 1.05892243e-02 7.52687142e-03 ... 2.71112219e-05\n",
      " 2.70664296e-05 2.69988762e-05]\n",
      "[0.00334696 0.01393618 0.02146305 ... 0.93827056 0.93829763 0.93832463]\n"
     ]
    }
   ],
   "source": [
    "print (svd.explained_variance_ratio_)\n",
    "print (svd.explained_variance_ratio_.cumsum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Variance of 94% is retained with dimension equal to 4000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "d_test_dense=svd.transform(d_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "dv_train=d_train_ng[:20000]\n",
    "svd_m = TruncatedSVD(n_components=700, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.00107144 0.00337399 0.00231914 0.00162754 0.00149142 0.00142097\n",
      " 0.00129976 0.00121034 0.00116515 0.00114128 0.00110561 0.0010996\n",
      " 0.0010388  0.00099458 0.00098735 0.00097283 0.00096465 0.00094027\n",
      " 0.00091868 0.00090752 0.00088248 0.00086171 0.00083861 0.00083053\n",
      " 0.00080008 0.00078783 0.00077507 0.00076554 0.00076495 0.00076079\n",
      " 0.00074925 0.00073854 0.0007333  0.00072899 0.00071422 0.00070221\n",
      " 0.00070068 0.00069347 0.0006893  0.00068609 0.00067511 0.00067336\n",
      " 0.00066162 0.00065829 0.00064978 0.00064089 0.00063908 0.00063506\n",
      " 0.00062912 0.00062505 0.00062134 0.00061354 0.00060767 0.0006024\n",
      " 0.00059908 0.00059656 0.000589   0.00058459 0.000582   0.00057678\n",
      " 0.00057441 0.00057133 0.00056664 0.0005647  0.00055795 0.00055586\n",
      " 0.00055322 0.00055153 0.00054719 0.00054447 0.00053775 0.00053486\n",
      " 0.00053379 0.00053072 0.00052757 0.0005255  0.00052299 0.00051904\n",
      " 0.00051732 0.00051295 0.00051053 0.00050905 0.00050645 0.00050164\n",
      " 0.00050104 0.00049803 0.00049489 0.00049278 0.0004913  0.00048939\n",
      " 0.00048661 0.00048389 0.00048084 0.00047793 0.00047586 0.00047488\n",
      " 0.00046998 0.00046919 0.00046867 0.00046665 0.00046389 0.00046263\n",
      " 0.00046131 0.00045736 0.00045516 0.00045274 0.00044982 0.00044836\n",
      " 0.00044495 0.00044234 0.00044133 0.00043873 0.00043752 0.00043455\n",
      " 0.0004308  0.00042988 0.0004277  0.00042646 0.0004244  0.00042319\n",
      " 0.0004208  0.00042027 0.00041838 0.0004184  0.00041664 0.00041419\n",
      " 0.00040975 0.00040861 0.00040729 0.00040553 0.00040451 0.00040269\n",
      " 0.00040114 0.00040047 0.00039947 0.00039802 0.0003955  0.00039473\n",
      " 0.00039298 0.00039183 0.00038959 0.00038708 0.00038659 0.00038613\n",
      " 0.00038397 0.00038326 0.00038261 0.00038138 0.00037877 0.00037823\n",
      " 0.00037799 0.0003742  0.00037309 0.00037225 0.00037078 0.00037043\n",
      " 0.00036937 0.00036832 0.00036578 0.0003655  0.000364   0.0003632\n",
      " 0.00036152 0.0003613  0.00036041 0.00035971 0.00035851 0.00035781\n",
      " 0.00035562 0.00035522 0.00035447 0.00035419 0.00035274 0.00035227\n",
      " 0.00035162 0.00034998 0.0003498  0.00034956 0.00034816 0.00034689\n",
      " 0.00034647 0.0003453  0.00034427 0.00034337 0.0003431  0.00034246\n",
      " 0.00034113 0.00034087 0.00033876 0.00033825 0.0003377  0.00033643\n",
      " 0.00033572 0.00033474 0.00033423 0.00033312 0.0003322  0.00033036\n",
      " 0.00033024 0.00032911 0.00032877 0.0003279  0.00032764 0.00032693\n",
      " 0.00032618 0.00032506 0.00032442 0.00032405 0.00032353 0.00032297\n",
      " 0.00032157 0.00032079 0.00031977 0.00031955 0.00031854 0.00031757\n",
      " 0.00031649 0.00031616 0.00031506 0.00031393 0.00031358 0.00031297\n",
      " 0.00031164 0.00031094 0.0003108  0.00031029 0.00030969 0.00030882\n",
      " 0.00030845 0.00030779 0.00030723 0.00030651 0.00030567 0.00030524\n",
      " 0.00030438 0.00030386 0.00030349 0.0003023  0.00030207 0.00030178\n",
      " 0.00030121 0.00030044 0.00029958 0.00029889 0.00029838 0.00029755\n",
      " 0.00029702 0.00029677 0.00029622 0.00029548 0.00029477 0.00029461\n",
      " 0.00029377 0.0002928  0.00029264 0.00029176 0.00029083 0.00029028\n",
      " 0.00029011 0.00028929 0.00028882 0.00028817 0.00028745 0.00028653\n",
      " 0.00028625 0.00028588 0.00028486 0.00028425 0.00028392 0.00028333\n",
      " 0.00028266 0.0002822  0.00028192 0.00028071 0.00028039 0.00027936\n",
      " 0.0002789  0.00027834 0.00027793 0.00027771 0.00027739 0.00027693\n",
      " 0.00027639 0.00027597 0.00027564 0.00027489 0.00027446 0.00027382\n",
      " 0.00027319 0.00027306 0.00027259 0.00027184 0.00027166 0.00027082\n",
      " 0.00027061 0.00027025 0.00026982 0.00026901 0.00026875 0.00026827\n",
      " 0.00026778 0.00026694 0.00026673 0.00026633 0.00026592 0.00026493\n",
      " 0.00026439 0.00026388 0.00026349 0.00026301 0.00026255 0.00026228\n",
      " 0.00026171 0.00026148 0.00026127 0.0002612  0.00026038 0.00025962\n",
      " 0.00025931 0.00025908 0.00025889 0.00025866 0.00025805 0.00025688\n",
      " 0.00025664 0.00025643 0.00025593 0.00025526 0.00025509 0.00025479\n",
      " 0.00025403 0.00025347 0.00025317 0.00025294 0.0002525  0.00025208\n",
      " 0.00025176 0.00025088 0.00025075 0.00025054 0.00025012 0.0002497\n",
      " 0.00024913 0.00024887 0.00024805 0.00024757 0.00024715 0.00024653\n",
      " 0.00024632 0.00024578 0.00024534 0.00024521 0.00024508 0.00024446\n",
      " 0.00024416 0.00024378 0.00024331 0.00024267 0.00024251 0.00024237\n",
      " 0.00024168 0.00024163 0.00024095 0.00024065 0.00024061 0.00024006\n",
      " 0.0002397  0.0002393  0.00023902 0.00023876 0.00023834 0.00023797\n",
      " 0.00023761 0.00023667 0.00023623 0.00023598 0.00023585 0.0002358\n",
      " 0.00023549 0.00023524 0.00023457 0.00023395 0.00023382 0.00023341\n",
      " 0.00023299 0.00023293 0.00023244 0.00023208 0.000232   0.00023166\n",
      " 0.00023141 0.000231   0.0002301  0.0002299  0.00022919 0.00022894\n",
      " 0.00022883 0.00022857 0.00022802 0.00022774 0.00022749 0.000227\n",
      " 0.00022637 0.00022612 0.00022587 0.00022543 0.0002252  0.0002248\n",
      " 0.00022449 0.00022418 0.00022355 0.00022347 0.00022295 0.00022255\n",
      " 0.00022239 0.0002219  0.00022171 0.0002215  0.00022122 0.00022077\n",
      " 0.00022034 0.00022019 0.00021996 0.00021976 0.00021955 0.00021884\n",
      " 0.00021872 0.00021825 0.0002179  0.00021781 0.00021738 0.00021727\n",
      " 0.00021679 0.00021633 0.00021602 0.00021564 0.00021536 0.0002152\n",
      " 0.00021468 0.00021454 0.00021439 0.00021397 0.00021339 0.00021304\n",
      " 0.00021278 0.0002124  0.00021223 0.00021204 0.00021159 0.00021105\n",
      " 0.00021103 0.00021072 0.00021019 0.00021001 0.00020993 0.00020918\n",
      " 0.00020888 0.00020876 0.00020868 0.00020814 0.00020787 0.00020739\n",
      " 0.00020719 0.00020707 0.00020657 0.00020613 0.0002059  0.00020575\n",
      " 0.00020567 0.00020536 0.0002051  0.00020495 0.00020446 0.00020416\n",
      " 0.00020412 0.00020354 0.00020338 0.00020324 0.00020283 0.00020266\n",
      " 0.00020223 0.00020189 0.00020163 0.00020126 0.00020083 0.00020052\n",
      " 0.00020012 0.00019989 0.0001996  0.00019937 0.00019913 0.00019878\n",
      " 0.00019844 0.0001982  0.00019785 0.00019776 0.00019758 0.00019735\n",
      " 0.00019693 0.00019678 0.0001963  0.00019608 0.00019571 0.00019537\n",
      " 0.00019513 0.00019505 0.00019471 0.00019439 0.00019415 0.00019394\n",
      " 0.00019365 0.00019347 0.00019327 0.00019303 0.0001928  0.00019235\n",
      " 0.00019198 0.00019176 0.00019153 0.00019117 0.00019097 0.00019069\n",
      " 0.00019043 0.00019008 0.00018985 0.00018967 0.00018925 0.000189\n",
      " 0.00018891 0.00018839 0.00018824 0.000188   0.00018783 0.00018768\n",
      " 0.00018761 0.00018704 0.00018683 0.00018634 0.00018615 0.00018599\n",
      " 0.00018576 0.00018558 0.00018517 0.000185   0.00018446 0.00018429\n",
      " 0.00018425 0.00018377 0.0001835  0.00018333 0.00018294 0.00018261\n",
      " 0.0001825  0.00018208 0.00018192 0.00018175 0.00018118 0.00018107\n",
      " 0.00018086 0.0001804  0.00018012 0.00017995 0.00017987 0.00017956\n",
      " 0.00017954 0.0001793  0.00017883 0.00017867 0.00017856 0.00017808\n",
      " 0.00017778 0.00017765 0.00017731 0.00017725 0.000177   0.00017681\n",
      " 0.00017626 0.00017616 0.00017575 0.00017552 0.00017519 0.00017511\n",
      " 0.00017483 0.00017474 0.0001744  0.00017423 0.00017397 0.00017368\n",
      " 0.00017336 0.0001732  0.00017296 0.00017274 0.00017262 0.00017234\n",
      " 0.00017207 0.00017198 0.00017148 0.00017118 0.00017102 0.00017083\n",
      " 0.00017057 0.00017046 0.00017005 0.00016992 0.00016971 0.00016941\n",
      " 0.00016921 0.0001688  0.00016866 0.00016842 0.00016823 0.0001681\n",
      " 0.00016782 0.00016748 0.00016733 0.00016697 0.00016669 0.00016637\n",
      " 0.00016621 0.00016591 0.00016561 0.0001654  0.00016525 0.00016511\n",
      " 0.00016466 0.00016449 0.00016431 0.00016417 0.00016389 0.00016367\n",
      " 0.00016333 0.00016306 0.00016275 0.00016253 0.00016236 0.00016205\n",
      " 0.00016201 0.00016154 0.00016118 0.00016106 0.00016087 0.00016073\n",
      " 0.00016053 0.00016008 0.00015999 0.00015957 0.0001594  0.00015921\n",
      " 0.00015883 0.0001587  0.00015845 0.00015817 0.00015784 0.0001576\n",
      " 0.00015732 0.00015704 0.00015693 0.00015681 0.00015665 0.00015619\n",
      " 0.00015591 0.00015544 0.00015527 0.000155   0.00015489 0.00015466\n",
      " 0.00015432 0.00015404 0.00015395 0.00015351 0.00015335 0.00015324\n",
      " 0.00015278 0.00015275 0.00015234 0.0001521  0.00015164 0.00015152\n",
      " 0.00015143 0.00015105 0.00015039 0.00015002 0.0001499  0.00014983\n",
      " 0.0001496  0.00014914 0.00014897 0.00014876 0.00014821 0.00014791\n",
      " 0.00014773 0.00014746 0.00014708 0.00014653 0.000146   0.0001457\n",
      " 0.00014545 0.00014534 0.00014526 0.00014456]\n",
      "[0.00107144 0.00444543 0.00676458 0.00839211 0.00988353 0.0113045\n",
      " 0.01260426 0.0138146  0.01497976 0.01612103 0.01722664 0.01832624\n",
      " 0.01936505 0.02035963 0.02134698 0.02231981 0.02328446 0.02422472\n",
      " 0.0251434  0.02605093 0.02693341 0.02779511 0.02863372 0.02946425\n",
      " 0.03026433 0.03105216 0.03182723 0.03259276 0.03335771 0.03411851\n",
      " 0.03486775 0.03560629 0.03633959 0.03706858 0.0377828  0.03848501\n",
      " 0.0391857  0.03987917 0.04056847 0.04125456 0.04192967 0.04260303\n",
      " 0.04326465 0.04392294 0.04457272 0.04521361 0.04585268 0.04648774\n",
      " 0.04711686 0.04774191 0.04836325 0.04897679 0.04958446 0.05018686\n",
      " 0.05078593 0.0513825  0.05197149 0.05255608 0.05313808 0.05371486\n",
      " 0.05428927 0.05486059 0.05542723 0.05599193 0.05654988 0.05710574\n",
      " 0.05765896 0.05821049 0.05875768 0.05930215 0.05983989 0.06037475\n",
      " 0.06090853 0.06143926 0.06196683 0.06249233 0.06301532 0.06353436\n",
      " 0.06405167 0.06456463 0.06507516 0.06558421 0.06609066 0.06659231\n",
      " 0.06709334 0.06759138 0.06808627 0.06857905 0.06907034 0.06955973\n",
      " 0.07004634 0.07053023 0.07101107 0.071489   0.07196485 0.07243973\n",
      " 0.07290971 0.0733789  0.07384757 0.07431422 0.07477812 0.07524074\n",
      " 0.07570205 0.07615941 0.07661457 0.07706731 0.07751714 0.07796549\n",
      " 0.07841044 0.07885279 0.07929412 0.07973286 0.08017038 0.08060493\n",
      " 0.08103573 0.08146561 0.08189331 0.08231977 0.08274417 0.08316736\n",
      " 0.08358816 0.08400843 0.08442681 0.08484521 0.08526184 0.08567604\n",
      " 0.08608579 0.08649439 0.08690169 0.08730721 0.08771172 0.08811441\n",
      " 0.08851556 0.08891602 0.0893155  0.08971351 0.09010901 0.09050373\n",
      " 0.09089671 0.09128855 0.09167813 0.09206521 0.0924518  0.09283793\n",
      " 0.0932219  0.09360516 0.09398777 0.09436914 0.09474792 0.09512615\n",
      " 0.09550414 0.09587834 0.09625143 0.09662368 0.09699446 0.09736488\n",
      " 0.09773425 0.09810256 0.09846834 0.09883384 0.09919785 0.09956105\n",
      " 0.09992257 0.10028387 0.10064428 0.10100399 0.1013625  0.10172031\n",
      " 0.10207593 0.10243115 0.10278561 0.1031398  0.10349254 0.10384482\n",
      " 0.10419643 0.10454641 0.10489621 0.10524577 0.10559393 0.10594082\n",
      " 0.10628729 0.10663259 0.10697686 0.10732024 0.10766334 0.1080058\n",
      " 0.10834693 0.10868781 0.10902657 0.10936482 0.10970252 0.11003895\n",
      " 0.11037466 0.11070941 0.11104363 0.11137675 0.11170896 0.11203932\n",
      " 0.11236955 0.11269866 0.11302743 0.11335533 0.11368297 0.1140099\n",
      " 0.11433608 0.11466115 0.11498556 0.11530961 0.11563314 0.11595611\n",
      " 0.11627768 0.11659847 0.11691823 0.11723778 0.11755632 0.11787389\n",
      " 0.11819037 0.11850653 0.11882159 0.11913553 0.1194491  0.11976208\n",
      " 0.12007372 0.12038466 0.12069546 0.12100575 0.12131544 0.12162426\n",
      " 0.12193271 0.1222405  0.12254772 0.12285423 0.1231599  0.12346515\n",
      " 0.12376953 0.12407338 0.12437687 0.12467917 0.12498124 0.12528301\n",
      " 0.12558422 0.12588466 0.12618424 0.12648313 0.12678151 0.12707905\n",
      " 0.12737607 0.12767284 0.12796907 0.12826455 0.12855932 0.12885393\n",
      " 0.1291477  0.12944049 0.12973313 0.13002489 0.13031572 0.130606\n",
      " 0.13089611 0.1311854  0.13147422 0.1317624  0.13204985 0.13233638\n",
      " 0.13262263 0.13290851 0.13319337 0.13347761 0.13376153 0.13404486\n",
      " 0.13432752 0.13460972 0.13489165 0.13517235 0.13545275 0.1357321\n",
      " 0.13601101 0.13628935 0.13656727 0.13684498 0.13712237 0.1373993\n",
      " 0.13767569 0.13795166 0.1382273  0.13850219 0.13877666 0.13905047\n",
      " 0.13932366 0.13959672 0.13986931 0.14014115 0.14041281 0.14068363\n",
      " 0.14095424 0.1412245  0.14149432 0.14176332 0.14203207 0.14230035\n",
      " 0.14256813 0.14283507 0.1431018  0.14336813 0.14363405 0.14389898\n",
      " 0.14416337 0.14442725 0.14469074 0.14495375 0.1452163  0.14547858\n",
      " 0.14574029 0.14600177 0.14626305 0.14652425 0.14678463 0.14704425\n",
      " 0.14730356 0.14756264 0.14782152 0.14808018 0.14833823 0.14859511\n",
      " 0.14885175 0.14910817 0.1493641  0.14961937 0.14987446 0.15012925\n",
      " 0.15038328 0.15063675 0.15088992 0.15114286 0.15139536 0.15164744\n",
      " 0.1518992  0.15215008 0.15240083 0.15265137 0.15290148 0.15315119\n",
      " 0.15340031 0.15364919 0.15389724 0.15414482 0.15439197 0.1546385\n",
      " 0.15488482 0.1551306  0.15537594 0.15562115 0.15586623 0.15611068\n",
      " 0.15635485 0.15659863 0.15684194 0.15708461 0.15732712 0.15756949\n",
      " 0.15781117 0.1580528  0.15829374 0.15853439 0.158775   0.15901507\n",
      " 0.15925476 0.15949406 0.15973308 0.15997184 0.16021018 0.16044815\n",
      " 0.16068576 0.16092243 0.16115866 0.16139464 0.16163049 0.16186629\n",
      " 0.16210177 0.16233701 0.16257158 0.16280553 0.16303935 0.16327276\n",
      " 0.16350575 0.16373869 0.16397113 0.16420321 0.16443521 0.16466687\n",
      " 0.16489827 0.16512928 0.16535938 0.16558928 0.16581847 0.1660474\n",
      " 0.16627623 0.16650481 0.16673283 0.16696057 0.16718806 0.16741505\n",
      " 0.16764143 0.16786754 0.16809341 0.16831884 0.16854404 0.16876884\n",
      " 0.16899332 0.16921751 0.16944106 0.16966453 0.16988747 0.17011003\n",
      " 0.17033242 0.17055431 0.17077602 0.17099752 0.17121874 0.17143951\n",
      " 0.17165985 0.17188004 0.1721     0.17231976 0.17253931 0.17275815\n",
      " 0.17297687 0.17319512 0.17341302 0.17363083 0.17384821 0.17406547\n",
      " 0.17428226 0.17449859 0.17471462 0.17493026 0.17514562 0.17536083\n",
      " 0.17557551 0.17579005 0.17600444 0.17621842 0.17643181 0.17664485\n",
      " 0.17685763 0.17707003 0.17728226 0.1774943  0.17770589 0.17791694\n",
      " 0.17812797 0.17833869 0.17854888 0.17875889 0.17896882 0.17917799\n",
      " 0.17938687 0.17959564 0.17980432 0.18001246 0.18022033 0.18042772\n",
      " 0.18063491 0.18084198 0.18104855 0.18125468 0.18146059 0.18166633\n",
      " 0.18187201 0.18207736 0.18228247 0.18248741 0.18269187 0.18289604\n",
      " 0.18310016 0.18330369 0.18350707 0.18371031 0.18391314 0.1841158\n",
      " 0.18431803 0.18451992 0.18472155 0.18492281 0.18512364 0.18532416\n",
      " 0.18552428 0.18572417 0.18592378 0.18612315 0.18632227 0.18652105\n",
      " 0.1867195  0.1869177  0.18711555 0.18731331 0.18751089 0.18770824\n",
      " 0.18790517 0.18810195 0.18829825 0.18849433 0.18869004 0.18888542\n",
      " 0.18908055 0.1892756  0.18947031 0.1896647  0.18985885 0.19005279\n",
      " 0.19024645 0.19043991 0.19063318 0.19082621 0.19101901 0.19121136\n",
      " 0.19140334 0.1915951  0.19178663 0.1919778  0.19216877 0.19235946\n",
      " 0.19254989 0.19273997 0.19292982 0.19311949 0.19330873 0.19349773\n",
      " 0.19368665 0.19387504 0.19406328 0.19425128 0.19443911 0.19462679\n",
      " 0.1948144  0.19500143 0.19518826 0.1953746  0.19556076 0.19574675\n",
      " 0.19593251 0.19611809 0.19630327 0.19648826 0.19667272 0.19685701\n",
      " 0.19704126 0.19722502 0.19740852 0.19759185 0.19777479 0.1979574\n",
      " 0.1981399  0.19832198 0.1985039  0.19868565 0.19886683 0.19904791\n",
      " 0.19922876 0.19940917 0.19958929 0.19976924 0.19994911 0.20012867\n",
      " 0.20030821 0.20048751 0.20066634 0.20084501 0.20102357 0.20120165\n",
      " 0.20137943 0.20155708 0.20173438 0.20191163 0.20208864 0.20226545\n",
      " 0.20244171 0.20261787 0.20279362 0.20296914 0.20314432 0.20331943\n",
      " 0.20349427 0.20366901 0.20384341 0.20401764 0.20419162 0.2043653\n",
      " 0.20453866 0.20471186 0.20488482 0.20505756 0.20523018 0.20540251\n",
      " 0.20557458 0.20574656 0.20591804 0.20608922 0.20626024 0.20643108\n",
      " 0.20660165 0.20677211 0.20694216 0.20711209 0.2072818  0.20745121\n",
      " 0.20762042 0.20778922 0.20795788 0.2081263  0.20829453 0.20846263\n",
      " 0.20863045 0.20879794 0.20896526 0.20913223 0.20929892 0.20946529\n",
      " 0.2096315  0.20979741 0.20996303 0.21012842 0.21029367 0.21045878\n",
      " 0.21062343 0.21078792 0.21095223 0.2111164  0.21128029 0.21144396\n",
      " 0.21160729 0.21177035 0.21193311 0.21209563 0.21225799 0.21242004\n",
      " 0.21258205 0.21274359 0.21290477 0.21306583 0.2132267  0.21338742\n",
      " 0.21354795 0.21370803 0.21386802 0.21402759 0.21418699 0.2143462\n",
      " 0.21450502 0.21466372 0.21482218 0.21498035 0.21513819 0.21529578\n",
      " 0.2154531  0.21561014 0.21576708 0.21592388 0.21608053 0.21623672\n",
      " 0.21639263 0.21654807 0.21670334 0.21685834 0.21701322 0.21716788\n",
      " 0.2173222  0.21747624 0.21763019 0.2177837  0.21793705 0.21809029\n",
      " 0.21824306 0.21839581 0.21854815 0.21870025 0.21885189 0.21900342\n",
      " 0.21915485 0.2193059  0.21945629 0.21960631 0.21975621 0.21990605\n",
      " 0.22005565 0.22020479 0.22035376 0.22050251 0.22065073 0.22079864\n",
      " 0.22094638 0.22109384 0.22124091 0.22138744 0.22153344 0.22167915\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0.2218246  0.22196993 0.2221152  0.22225976]\n"
     ]
    }
   ],
   "source": [
    "#using bigrams\n",
    "d_train_ng_dense=svd_m.fit_transform(dv_train)\n",
    "print (svd_m.explained_variance_ratio_)\n",
    "print (svd_m.explained_variance_ratio_.cumsum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "d_test_ng_dense=svd_m.transform(d_test_ng)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.3.4] KNN (Brute-Force)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [05:37<00:00, 13.99s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 7.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAEKCAYAAAB+AXB+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VNX5+PHPk5nMZAeyELIRCAlrCCCLUAVUBFEoUAUFRcCltla/tm5fsC7tz6pfrVu1uNSqiLt1oSBVURAqoCwREURlERHCnoAQluzP74+ZjEPIMihDJvK8X6/7ytxzzzn3TPSVh3vvOc8VVcUYY4z5uQhr7AEYY4wxx5MFNmOMMT8rFtiMMcb8rFhgM8YY87Nigc0YY8zPigU2Y4wxPysW2IwxxvysBDWwichQEVkrIhtEZEotxweIyAoRqRCR0bUcjxORrSIy1a+sp4is9vb5qIhIML+DMcaYpiVogU1EHMBjwLlAZ2CciHSuUW0zMAl4uY5u/gL8t0bZE8BVQI53G3qchmyMMeZnwBnEvvsAG1R1I4CIvAqMBL6srqCqm7zHqmo2FpGeQDLwHtDLW5YCxKnqJ97954FRwLv1DSQxMVHbtGnzk7+QMcacTD799NNCVU1q7HEcq2AGtjRgi99+AXBqIA1FJAx4ELgUGFSjz4IafaY11F+bNm3Iz88P5NTGGGO8ROS7xh7DjxHMZ2y1PfsKNDHl74B3VHVLjfKA+xSRq0QkX0Tyd+/eHeBpjTHGNHXBvGIrADL89tOBbQG27Qf0F5HfATGAS0QOAI94+2mwT1V9CngKoFevXpbp2RhjThLBDGzLgRwRaQtsBcYCFwfSUFUvqf4sIpOAXqo6xbtfLCJ9gaXABODvx3ncxhhjmrCg3YpU1QrgWmAO8BXwL1VdIyJ3isgIABHpLSIFwBjgHyKyJoCurwaeBjYA39DAxJHj4fDhwwwcOJDKykoAhg4dSvPmzRk+fHhA7UtLS7nooovIzs7m1FNPZdOmTbXWu/zyy2nZsiW5ublHlN9+++3k5eXRvXt3hgwZwrZtnovUffv28ctf/pJu3brRpUsXpk2bBsDu3bsZOtQmixpjTk5BXcemqu+oantVbaeqd3vL7lDVWd7Py1U1XVWjVTVBVbvU0sdzqnqt336+quZ6+7xWT8AL5Z599lnOP/98HA4HADfffDMvvPBCwO2feeYZWrRowYYNG7j++uuZPHlyrfUmTZrEe++9d1T5zTffzKpVq1i5ciXDhw/nzjvvBOCxxx6jc+fOfP755yxYsIAbb7yRsrIykpKSSElJYfHixT/i2xpjTNNmmUcC8NJLLzFy5Ejf/qBBg4iNjQ24/cyZM5k4cSIAo0ePZt68edQWjwcMGEB8fPxR5XFxcb7PBw8epHpNuohQXFyMqnLgwAHi4+NxOj13l0eNGsVLL70U8BiNMebnIpjP2H4WysrK2LhxIz9lHdzWrVvJyPDMo3E6nTRr1oyioiISExMD7uPWW2/l+eefp1mzZsyfPx+Aa6+9lhEjRpCamkpxcTGvvfYaYWGef6v06tWL22677UeP2Rhjmiq7YqvHjM8K+MeclTRv3vwn9VPb1dmxZgK7++672bJlC5dccglTp3oyjM2ZM4fu3buzbds2Vq5cybXXXsv+/fsBaNmype9ZnDHGnEwssNXjP6t28O/VuykpKflJ/aSnp7Nli2dJXkVFBfv27av1lmMgLr74Yt58800Apk2bxvnnn4+IkJ2dTdu2bfn6668BKCkpITIy8ieN2xhjmiILbPVIjHGxX91UVlYGFNxuueUWZsyYcVT5iBEjmD59OgBvvPEGZ5111jFdsa1fv973edasWXTs2BGA1q1bM2/ePAB27tzJ2rVrycrKAmDdunVHza40xpiTgQW2eiTEuNhzsIzBgwezaNEiX3n//v0ZM2YM8+bNIz09nTlz5gCwevVqWrVqdVQ/V1xxBUVFRWRnZ/PQQw9x7733ArBt2zbOO+88X71x48bRr18/1q5dS3p6Os888wwAU6ZMITc3l7y8PN5//30eeeQRwLMM4OOPP6Zr164MGjSI++67z/fcbv78+QwbNiw4vxhjjAlhcgJmyze6Xr166Y/JFfnsom+5c/aXTB+RxNNP/L3BKf7nnHOOL8g1tgEDBjBz5kxatGjR2EMxxjRRIvKpqvZq7HEcK7tiq0dCjAuAtOzOnHnmmb4F2nUJlaC2e/dubrjhBgtqxpiTkk33r0dijBuAogOlXH755Y08msAlJSUxatSoxh6GMcY0Crtiq0d8tOeKrehgWSOPxBhjTKAssNWj+lZk0YHSRh6JMcaYQFlgq0d8lCewFR6wKzZjjGkqLLDVw+kIo0VUOEUH7YrNGGOaCgtsDUiIcVNkV2zGGNNkWGBrQEK0ywKbMcY0IRbYGpAY46bQbkUaY0yTYYGtAQkxdsVmjDFNSVADm4gMFZG1IrJBRKbUcnyAiKwQkQoRGe1Xnikin4rIShFZIyK/9Tu2wNvnSu/WMpjfISHazb7D5ZRXVgXzNMYYY46ToGUeEREH8BgwGCgAlovILFX90q/aZmAScFON5tuBX6hqqYjEAF9421a/YOwSVT325I8/QvVatr0Hy2gZF3EiTmmMMeYnCOYVWx9gg6puVNUy4FVgpH8FVd2kqquAqhrlZapa/WDLHeRx1ish2tayGWNMUxLMgJEGbPHbL/CWBUREMkRklbeP+/yu1gCmeW9D3i7H+irqY5RQnS/SJpAYY0yTEMzAVlvACfgdOaq6RVXzgGxgoogkew9doqpdgf7e7dJaTy5ylYjki0j+7t27j3HoP/ghrZZdsRljTFMQzMBWAGT47acD2+qoWyfvldoaPEEMVd3q/VkMvIznlmdt7Z5S1V6q2ispKelYT+uTGO25Yiu0fJHGGNMkBDOwLQdyRKStiLiAscCsQBqKSLqIRHo/twBOA9aKiFNEEr3l4cBw4IugjN4rLtKJM0wsw78xxjQRQQtsqloBXAvMAb4C/qWqa0TkThEZASAivUWkABgD/ENE1nibdwKWisjnwH+BB1R1NZ6JJHO8z95WAluBfwbrO3jH6F3LZldsxhjTFAT1RaOq+g7wTo2yO/w+L8dzi7Jmuw+AvFrKDwI9j/9I65cQbfkijTGmqbDMIwFIiHFRaLcijTGmSbDAFoDEGDd7bLq/McY0CRbYAmAZ/o0xpumwwBaA+BgXh8oqOVRW0dhDMcYY0wALbAGoXstmV23GGBP6LLAFwJd9xCaQGGNMyLPAFgBfvkhby2aMMSHPAlsAqjP8261IY4wJfRbYAlB9K7LQpvwbY0zIs8AWgCiXkyiXw67YjDGmCbDAFiDLF2mMMU2DBbYAJUS7bVakMcY0ARbYApQYY9lHjDGmKbDAFqD4aBdFNnnEGGNCngW2ACXEeF5do6qNPRRjjDH1sMAWoIRoFxVVyv7Dli/SGGNCmQW2ACV6s4/YWjZjjAltQQ1sIjJURNaKyAYRmVLL8QEiskJEKkRktF95poh8KiIrRWSNiPzW71hPEVnt7fNREZFgfodqvnyRNoHEGGNCWtACm4g4gMeAc4HOwDgR6Vyj2mZgEvByjfLtwC9UtTtwKjBFRFK9x54ArgJyvNvQoHyBGhKiLV+kMcY0BcG8YusDbFDVjapaBrwKjPSvoKqbVHUVUFWjvExVqyOIu3qcIpICxKnqJ+qZxfE8MCqI38En0ZdWy67YjDEmlAUzsKUBW/z2C7xlARGRDBFZ5e3jPlXd5m1f8GP7/ClaeBMh77FbkcYYE9KCGdhqe/YV8Fx5Vd2iqnlANjBRRJKPpU8RuUpE8kUkf/fu3YGetk7hjjCaR4XbWjZjjAlx9QY2EXGIyP0/su8CIMNvPx3YdqydeK/U1gD9vX2mB9Knqj6lqr1UtVdSUtKxnrZWCdGWfcQYY0JdvYFNVSuBnj9y5uFyIEdE2oqICxgLzAqkoYiki0ik93ML4DRgrapuB4pFpK93TBOAmT9ibD9KQrSbQps8YowxIc0ZQJ3PgJki8jpwsLpQVd+qr5GqVojItcAcwAE8q6prROROIF9VZ4lIb2AG0AL4pYj8P1XtAnQCHhQRxXP78QFVXe3t+mrgOSASeNe7nRAJMS7W7zpwok5njDHmRwgksMUDRcBZfmUK1BvYAFT1HeCdGmV3+H1ezpG3FqvLPwDy6ugzH8gNYNzHXUKMiyUb7YrNGGNCWYOBTVUvOxEDaQoSot3sPVRORWUVToclbTHGmFDU4F9n7/OuGSKyS0R2isibInLUVdbJoHot255DNoHEGGNCVSCXHdPwTPpIxbNm7G1v2UknIaY6+4gFNmOMCVWBBLYkVZ2mqhXe7Tng+Myfb2ISoi1fpDHGhLpAAluhiIz3rmlziMh4PJNJTjq+KzZbpG2MMSErkMB2OXAhsANPcuLR3rKTTqJl+DfGmJBX76xIb4b+C1R1xAkaT0iLiwjHESZ2xWaMMSEskMwjI+urczIJCxPiLa2WMcaEtEAWaC8WkanAaxyZeWRF0EYVwhKiXRRaYDPGmJAVSGD7hffnnX5lypGZSE4aiTFuuxVpjDEhrKFnbGHAE6r6rxM0npCXEONi8+ZDjT0MY4wxdWjoGVsVcO0JGkuTkBDtpsgy/BtjTMgKZLr/ByJyk/eN1vHVW9BHFqISYlwcLKvkcFllYw/FGGNMLQJ5xla9Zu0avzIFso7/cEKfby3bwVLSXVGNPBpjjDE1BZLdv+2JGEhTkRDtyT6y52AZ6S0ssBljTKip81akiPyv3+cxNY7dE8xBhbIEyz5ijDEhrb5nbGP9Pt9S49jQIIylSai+Yiu0CSTGGBOS6gtsUsfn2vZPGr4rtoN2xWaMMaGovsCmdXyubb9WIjJURNaKyAYRmVLL8QEiskJEKkRktF95dxH5RETWiMgqEbnI79hzIvKtiKz0bt0DGcvxEuVyEBEeZlP+jTEmRNU3eaSbiOzHc3UW6f2Mdz+ioY69CZQfAwYDBcByEZmlql/6VdsMTAJuqtH8EDBBVdeLSCrwqYjMUdXvvcdvVtU3GhpDMIiIdy2bXbEZY0woqjOwqarjJ/bdB9igqhsBRORVPAmVfYFNVTd5j1XVOPc6v8/bRGQXnpebfk8ISIxxUWi3Io0xJiQFskD7x0oDtvjtF3jLjomI9AFcwDd+xXd7b1E+LCLuOtpdJSL5IpK/e/fuYz1tvRJiLPuIMcaEqmAGttommAT0bM7XgUgK8AJwmTe9F3hmaHYEegPxwOTa2qrqU6raS1V7JSUlHctpG5Rgr64xxpiQFczAVgBk+O2nA9sCbSwiccB/gNtUdUl1uapuV49SYBqeW54nVII3w7/qMcVpY4wxJ0AwA9tyIEdE2oqIC8+6uFmBNPTWnwE8r6qv1ziW4v0pwCjgi+M66gAkxrgor1SKSytO9KmNMcY0oMHAJiLni8h6EdknIvtFpNhvhmSdVLUCz5sB5gBfAf9S1TUicqeIjPD23VtECoAxwD9EZI23+YXAAGBSLdP6XxKR1cBqIBG46xi/809m2UeMMSZ0BZIE+a/AL1X1q2PtXFXfAd6pUXaH3+fleG5R1mz3IvBiHX02+gtO473ZR4oOlNI2MbqRR2OMMcZfILcid/6YoPZzlhDtuWIrtCs2Y4wJOYFcseWLyGvAvwHfHHdVfStoowpxiTHeK7aDNuXfGGNCTSCBLQ5PJpAhfmUKnLSBLT7anrEZY0yoCuR9bJediIE0JS5nGHERTlukbYwxISiQWZHpIjJDRHaJyE4ReVNEjprwcbJJjHFbWi1jjAlBgUwemYZn/VkqnpRYb3vLTmoJMS67YjPGmBAUSGBLUtVpqlrh3Z7Dk5D4pJYQ7WaPXbEZY0zICSSwFYrIeBFxeLfxQFGwBxbqPFdsFtiMMSbUBBLYLseTCWQHsB0Y7S07qSXEuNlzqIzKKssXaYwxoSSQWZGbgREnYCxNSkK0C1XYe6jMt67NGGNM46szsInI/6rqX0Xk79TyuhlVvS6oIwtx/vkiLbAZY0zoqO+KrTqNVv6JGEhTk+CXLxJiG3cwxhhjfOoMbKr6tvfjoVpeHTMmqKNqAhK9V2y2ls0YY0JLIJNHbgmw7KSSEON/xWaMMSZU1PeM7VzgPCBNRB71OxQHnPRv2GweGU6YWL5IY4wJNfU9Y9uG5/naCOBTv/Ji4PpgDqopCAsT4qPdluHfGGNCTH3P2D4HPheRl1W1/ASOqclItEXaxhgTcgJ5xtZGRN4QkS9FZGP1FkjnIjJURNaKyAYRmVLL8QEiskJEKkRktF95dxH5RETWiMgqEbnI71hbEVkqIutF5DURcQX0TYMgIcZFkU0eMcaYkBJoEuQn8DxXOxN4HnihoUYi4gAeA84FOgPjRKRzjWqbgUnAyzXKDwETVLULMBT4m4g09x67D3hYVXOAvcAVAXyHoEiIdtvkEWOMCTGBBLZIVZ0HiKp+p6p/Bs4KoF0fYIOqblTVMuBVYKR/BVXdpKqrgKoa5etUdb338zZgF5AkIuI99xveqtOBUQGMJSjio+1WpDHGhJpAAluJiIQB60XkWhH5FdAygHZpwBa//QJv2TERkT6AC/gGSAC+V9XqWZl19ikiV4lIvojk7969+1hPG5DEGBfFpRWUlFcGpX9jjDHHLpDA9gcgCrgO6AmMByYG0E5qKTumjMEikoLntudlqlp1LH2q6lOq2ktVeyUlBectO9Vr2ez1NcYYEzoCSYK83PvxAHDZMfRdAGT47afjWUIQEBGJA/4D3KaqS7zFhUBzEXF6r9qOqc/jLSH6h3yRqc0jG2sYxhhj/DR4xSYiH/hN3EBEWojInAD6Xg7keGcxuoCxeN7E3SBv/RnA8/7pvFRVgfl4Xp0DnivHmYH0GQzVV2yFtpbNGGNCRiC3IhNV9fvqHVXdSwDP2LxXVNcCc/AkVP6Xqq4RkTtFZASAiPQWkQJgDPAPEVnjbX4hMACYJCIrvVt377HJwA0isgHPM7dnAvqmQZDol+HfGGNMaGjwViRQJSKtve9lQ0QyCfBZmaq+A7xTo+wOv8/L8dxOrNnuReDFOvrciGfGZaOzfJHGGBN6AglstwKLROS/3v0BwFXBG1LTEe1y4HaG2eQRY4wJIYFMHnlPRE4B+uKZlXi9qhYGfWRNgIiQGOOm0G5FGmNMyKjzGZuIdPT+PAVojWf24VagtbfMUJ1Wy25FGmNMqKjviu0GPLccH6zlmBJY9pGfPcs+YowxoaW+wPaB9+cV3gkbphYJ0W7W7Shu7GEYY4zxqm+6f/Vbst+op85JLzHGReHBMjxL7IwxxjS2+q7YikRkPtBWRI5aWK2qI4I3rKYjIcZFWUUVB0oriI0Ib+zhGGPMSa++wDYMOAVPrsbanrMZPLciwbNI2wKbMcY0vvreoF0GLBGRX6hqcNLj/wwkVGcfOVhKm8ToRh6NMcaYOgObiPxNVf8APCsiRz1AsluRHonV+SJtZqQxxoSE+m5FVr8l+4ETMZCmqvqKzbKPGGNMaKjvVuSn3p/VqbQQkRZAhvet1wbPOjawfJHGGBMqAnltzQIRiROReOBzYJqIPBT8oTUNbqeD2Ain3Yo0xpgQEchra5qp6n7gfGCaqvYEzg7usJqWhGgXRXYr0hhjQkIggc0pIil43pE2O8jjaZISYtx2K9IYY0JEIIHtTjwvC92gqstFJAtYH9xhNS0Jli/SGGNCRiCvrXkdeN1vfyNwQTAH1dQkxLhZsXlvYw/DGGMMgU0e+at38ki4iMwTkUIRGR9I5yIyVETWisgGEZlSy/EBIrJCRCpEZHSNY++JyPciMrtG+XMi8q2IrPRu3QMZSzAlxrjYc7CMyirLF2mMMY0tkFuRQ7yTR4YDBUB74OaGGomIA3gMOBfoDIwTkc41qm0GJgEv19LF/cCldXR/s6p2924rA/gOQZUQ7aJK4ftDdjvSGGMaWyCBrToB4nnAK6q6J8C+++B5LrfRm57rVWCkfwVV3eRdE1dVs7GqzgOaxPtgErzZR2yRtjHGNL5AAtvbIvI10AuYJyJJQEkA7dKALX77Bd6y4+FuEVklIg+LiPs49fmjVWcfsbVsxhjT+BoMbKo6BegH9FLVcuAgNa686iC1dXdsw6vVLUBHoDcQD0yu9eQiV4lIvojk794d3BzO1fkiiw7alH9jjGlsDc6K9EoDBotIhF/Z8w20KQAy/PbTgW3HMLZaqep278dSEZkG3FRHvaeApwB69eoV1FkdCb60WnbFZowxja3BwCYifwLOwDMB5B08k0EW0XBgWw7kiEhbYCswFrj4pwzWO54UVd0uIgKMAr74qX3+VM2jXIhYvkhjjAkFgTxjGw0MAnao6mVAN6DB51qqWgFci2dx91fAv1R1jYjcKSIjAESkt4gUAGOAf4jImur2IrIQz/q5QSJSICLneA+9JCKrgdVAInBXgN81aBxhQnyUi0KbPGKMMY0ukFuRh1W1yrvWLA7YBWQF0rmqvoPnKs+/7A6/z8vx3KKsrW3/OsrPCuTcJ1pCjMuu2IwxJgQEEtjyRaQ58E/gU+AAsCyoo2qCEqLd9ozNGGNCQCAptX7n/fikiLwHxNn72I6WEONizbb9jT0MY4w56dUZ2ETklPqOqeqK4AypaUqMcVNotyKNMabR1XfF9mA9xxQIyWddjSUh2kVxSQVlFVW4nIHMyTHGGBMMdQY2VT3zRA6kqfNPq9WqWUQDtY0xxgRLINn9r/FOHqnebyEiv6uvzcnoh7RadjvSGGMaUyD3zH6tqt9X76jqXuDXwRtS0+TLPmJr2YwxplEFEtjCvFk+AN/raFzBG1LTVH0r0tayGWNM4wpkHdsc4F8i8iSeSSO/Bd4L6qiaoOpbkbaWzRhjGlcggW0ycBVwNZ6M/e8DTwdzUE1RrNuJyxFGoWX4N8aYRhXIAu0q4Ek8C7TjgXRVrQz6yJoYEfGm1bIrNmOMaUyBzIpcICJx3qC2EpgmIg8Ff2hNj+WLNMaYxhfI5JFmqrofOB+Ypqo9gbODO6ymKSHabbMijTGmkQUS2JwikgJcCMwO8niaNLsVaYwxjS+QwHYnnpmRG1R1uYhkAeuDO6ymKTHGTdHBUlSVw4cPM3DgQCorK/nuu+/o2bMn3bt3p0uXLjz55JMN9rVnzx4GDx5MTk4OgwcPZu/evbXWGzp0KM2bN2f48OFHlE+dOpXs7GxEhMLCQl/57Nmz+dOf/vTTvqgxxoSwBgObqr6uqnnVWf5VdaOqXhD8oTU9CdEuSsqrOFRWybPPPsv555+Pw+EgJSWFjz/+mJUrV7J06VLuvfdetm3bVm9f9957L4MGDWL9+vUMGjSIe++9t9Z6N998My+88MJR5aeddhpz584lMzPziPJhw4Yxa9YsDh069OO/qDHGhLA6A5uI/K/3599F5NGa24kbYtMRH/3DWraXXnqJkSNHAuByuXC7PQu4S0tLqaqqarCvmTNnMnHiRAAmTpzIv//971rrDRo0iNjY2KPKe/ToQZs2bY4qFxHOOOMMZs+2u8rGmJ+n+q7YvvL+zMfzgtGaW4NEZKiIrBWRDSIypZbjA0Rkhfft3KNrHHtPRL4Xkdk1ytuKyFIRWS8ir4lIyGRBSfRmH9m+t5iNGzceEVi2bNlCXl4eGRkZTJ48mdTU1Hr72rlzJykpKQCkpKSwa9eu4zbOXr16sXDhwuPWnzHGhJI6A5uqvu39Ob22raGOvam3HgPOBToD40Skc41qm4FJwMu1dHE/cGkt5fcBD6tqDrAXuKKhsZwo1dlHvi3YQfPmzY84lpGRwapVq9iwYQPTp09n586djTFEAFq2bNngrVBjjGmq6rsVOau+LYC+++CZcLJRVcuAV4GR/hVUdZP3bdxH3ZtT1XlAcY0xCZ73wL3hLZoOjApgLCdEdb7IAxVhlJSU1FonNTWVLl26NHjFlJyczPbt2wHYvn07LVu2PG7jLCkpITIy8rj1Z4wxoaS+W5H9gHRgIfAAnheP+m8NSQO2+O0XeMt+igTge1WtOI59HjcJ0S7CBPZUhFNZWekLbgUFBRw+fBiAvXv3snjxYjp06ADAhAkTWLZs2VF9jRgxgunTPRfG06dP9z2vOx7WrVtHbm7ucevPGGNCSX2BrRXwRyAXeAQYDBSq6n9V9b8B9C21lOmxD/HH9SkiV4lIvojk7969+yeeNjAR4Q56ZrZgwdrdDBkyhEWLFgHw1Vdfceqpp9KtWzcGDhzITTfdRNeuXQFYtWqV71mavylTpvDBBx+Qk5PDBx98wJQpnkeU+fn5XHnllb56/fv3Z8yYMcybN4/09HTmzJkDwKOPPkp6ejoFBQXk5eUd0Wb+/PkMGzYsaL8HY4xpVKra4Aa48TwL2w38T4Bt+gFz/PZvAW6po+5zwOhays8AZvvtC1AIOGs7R11bz5499UR5YsEGzZw8W+f892MdP358vXX37duno0ePPkEj89ixY4eeddZZJ/ScxpimCcjXAP7eh9pW7zo2EXGLyPnAi8A1wKPAWwHGzOVAjncWowsYCwTybK5O3l/0fKB6BuVEYOZP6fN4O7uT51nYLlcqZ555JpWVdeeLjouL4/XXXz9RQwNg8+bNPPhgIHeSjTGmaRJPrKjlgMh0PLch3wVeVdUvjrlzkfOAvwEO4FlVvVtE7sTzr4BZItIbmAG0AEqAHaraxdt2IdARiAGKgCtUdY4388mrQDzwGTBeVevNPNyrVy/Nz88/1uH/KKrKGQ8sICsxmmmX9Tkh5zTGmGAQkU9VtVdjj+NY1ffamkuBg0B74Dr/l2jjuXiKa6hzVX0HeKdG2R1+n5fjmaBSW9v+dZRvxDPjMiSJCGd1bMlLSzdzqKyCKFcgr7wzxhhzvNS3ji1MVWO9W5zfFhtIUDuZnd0pmbKKKhZvKGrsoRhjzEknkCTI5hj1bhNPrNvJvK8abxG2McacrCywBYHLGcaA9knM+3oXVVU/dYWDMcaYY2GBLUgGdWrJ7uJSVm/d19hDMcaEKP/XW1Xbv38/aWlpXHvttb6yM844gw4dOtC9e3fy8vL4xS9+UW+b4uJiUlNTcbvduN1unE4n11133VH1BwwY4Hu91T//+U9SUlJwuVzEx8dXJ4XIFZFVInJ4b3eVAAAgAElEQVQKgIicKSIFIlIqIioi5SKy3ZsTeIGITPfmBvZv84hf/dUiUiIio8RjnYh8V8s5VvptJSIyynvsVRHJaej3aoEtSM7s0JIwgXlfH7/kxcaYnxf/11tVu/322xk4cOBRdV966SVWrlzJb37zGy666KJ622zZsoWkpCT279/PH//4RwB+9atfHVU/JSWFuXPnkp6ezm233UZ8fDzFxcV06tSJ9957D+AL4CrgCQBVnQ/8EugAbMczmbATMBToiiflYY5/GzzrlDsA3wG/Ag4B7+PJI7zH+/mIc6hqd1Xt7u2vuj7eOv/b0O/VAluQtIh20TOzhT1nM8bUyf/1VgCffvopO3fuZMiQIT+pzcyZMxk7dixut5uZM2ficDh8r87yr5+UlESbNm2oqKggKiqK8ePH43a7cblcxMTEAESr6hKguYikAKjqZ6q6CWgGfKWq36vqt3gCUDTg8G/jVx9gBPCuqh7Ckzv4UeBsPG+R8Z3Dz2i/+uBJ8Xi2iNQ73dwCWxCd1TGZNdv2s33f4cYeijEmxJSVlR3xequqqipuvPFG7r///lrrX3bZZeTl5bF69WrfC4TrarN161YyMjIoKytj/fr1tG/fnm3bttVZ3+l0smvXLiIiIqioqODrr7/G6XQCVL8WrLa8vG7gI799xXMF1q2eNr8CXvF+TsPzhpcN3ja11R/rVx9VrfKrXycLbEFUnYVk3ld2O9IYc6TCwsIjXm/1+OOPc95555GRkXFU3ZdeeonVq1fzxhtvUFVVxQsvvFBvm+rEG4WFhZSVlZGVlYWI1Fnf4XDQr18/pk6dSv/+/YmMjMS7dtl/9pvvs/fKKgz4ssZQDwCptbXBk6ijEzCnuhvvz11+bWqeo6tffWqpXytbPRxE2S1jaB0fxYdf72J838zGHo4xJoRERkYe8XqrTz75hIULF/L4449z4MABysrKiImJ4d577yUtzXMhk5SURGRkJMuWLWPChAl1tklPT2fLli188803VFVVUVJSQmpqKm+99Vat9QH69u3LoEGDuOWWWxgwYABfffUVQLl3eOmA/0scL/Qea+VXJkA4cLiONlF4cv9W91kAZAAR3ja1nWOGX/1q1fXr1tjJKk/EdiKTINf051lfaPtb39FDpRWNNgZjVFUPHTqkAwYM0IqKCv3ss8+0b9++2rlzZ+3atau++uqrvnrz5s3THj16aJcuXXTChAlaXl7eYN/PPfecZmdna1ZWlnbo0EErKip006ZNesopp2i3bt20c+fOeuWVV2rnzp1VRLRv376al5ennTt31iuuuELbtGmjWVlZ2rZtW83Ozva1efDBB/WCCy7Q9u3bq9PpVKfTqW63W51Op7722mtHnWPo0KGanp6uUVFR2q1bN01JSVGXy6Uioi6XS7OysrRbt26am5vra3POOeeo2+1WQGNjY9XlcqnL5dLExERNSUnRrKws7dq1q3766ae6b98+bdasmcbGxiqg2dnZGhUVpQ6HQwF1uVyakZFxRP3U1FTt06ePRkZGKqBRUVGalJSk4eHhCujAgQO1Xbt2vjaqqtOmTTuiTWxsrGZmZmpubq6GhYVp7969tV27dtq6dWudNGmSqqoOHz7cN5a2bduq2+3WX//61xodHa1hYWGakZGhffr00SeffFKnTJmi06ZN04suukj79u2r4eHh2rp1a+3atatu375d3W63ulwuxfPsazKwTY9MUL8ET0L6L/DckmyLZzJJMZAC9AWW1WhTCoz02x+GJ2Xjau/nmvWXAGdqjb/n3vopNcuPqFPfwZ/L1piBbeG63Zo5eba+v2ZHo43BGFXVqVOn6t/+9jdVVV27dq2uW7dOVVW3bt2qrVq10r1792plZaWmp6fr2rVrVVX19ttv16effrrefouKirRt27ZaVFSkf/3rXzU+Pl737NmjpaWlWlJSoqqqxcXFmpqaqv/973914MCBOn/+fFVVraqq0g4dOmjz5s21pKREv/76a23btq1WVFRocXGxRkdH6/3336//+c9/tEWLFjpv3jy98cYbNTU1VQcPHnzUOVq1aqUrVqzQ6OhoXbNmjebl5WlJSYnGxcVpeHi4VlRU6AsvvKAXXHCBqqru3btXnU6nPv/885qcnKxOp1M3bdqkGzdu1LCwMI2Li9Oqqir95JNPtE+fPnrdddfp0KFDNTk5WZOTk3XRokWalZWlX375pTqdTnU4HHrZZZcdUX/cuHGalJSkzz//vGZmZuorr7+hbXM66lMffqURUTEK6H3/+UJ/8+ArmpbTVae8+bkOufr/aURsCx1w1f9TV0xzdbojNTalrUYlpKiEOdQV3Ux/8/xyzRtykcbGJ+ttM1brwAt/ozHNE9Tpcuvj8zfoLyf+j4Y5HOpyubRDhw565ZVX6iuvvKKxsbE6ZcoUffrppzUqKkpbtmypDodD3W63RkREaHh4uDZr1kw7dOigeHL4bgc2AanqCSx/BiqBCmC/d1sLjAN2At94y4Z661/n7UO9V2RPe8sFeNZ7jtVAL/0heLUBtgJh6ve3HEiuGQBr2+wZW5D1aWtZSExo8J9N1759e3JyPMuBUlNTadmyJbt376aoqAi320379u0BGDx4MG+++Wa9/c6ZM4fBgwcTHx/PjBkzOOecc3jvvfdwuVy+mXilpaU4HA6ys7MBfLe/Kioq2Lx5M6effjput5sOHTqQk5PDsmXLKCwspKSkhHHjxjFz5kxatWpFXFwcH3/8MW63mx07drBnz54jzhEeHk5ycjLww8zAzZs3+6awL1u2jLFjx7JgwQJU1TfOQYMGcfDgQdLT08nMzKRt27aICCUlJYgIffv2ZceOHWzatInTTz8dESEiIoJ3332XK6+8kk6dOlFRUUF2djbvvvsuvXr18tXv2LkzVYSxJ6Uvu4tLufrh19mb0oe73llLmXdy38Mv/4f5e5qxe88e3ln6Fbvc6bgT0ojsdAaV5aXkXfA//PLOl/nlPW8S5gynvOQg63fsY/vuIsoIY8bi1RTE96AiJplKFe5772tWtTqHjJtm0qVrd9q0acPcuXMZNWoUpaWlXH/99aSkpPCrX/2KnTt3UlFRQUlJCYcPH6asrIxvv/22evLIF8A8oLeqbgNQ1T+rqkNVnfpDqsUOeN/hqartvGXvees/qqopqiqqmqqqV3rLFU9Au0ZVu6qqL1O9qm5S1TT1TBbxdzHwj4b+X7fAFmTVWUg+tCwkphHVnIHnb9myZZSVldGuXTsSExMpLy+n+m0Yb7zxBlu2bKm3b/8ZeBs3bqRTp05s3boV8KynysvLIyMjg8mTJ5Oa+sMz/3POOYekpCTKysoYM2aMr7x58+aMHTuWTp06kZaWxq233sqrr77Kzp07mTBhAsuXLwegbdu2LF68uM5zVI+reqJF9bicTifR0dHk5uYyYcIEunTpwmWXXcbBgwd9wejQoUNUVlaiqhQVFVFVVcX+/fv59a9/zc6dO3G73ezevZsnnniCRYsWcfiw55FPdnY2SUlJzJq3iMI933M4ayCPzFrO96XKH6+ZyOF9hbi3rqDF9mXse3oSLeMiiY6O5roulay961xO79aBZ8a0408D4zm9SyZRH/2NqrISMg+v5/nLevPqVf1oHhtNq5ZJ/N/AWHat+IDTunfiuYtyeOCcZAZ1a0tYVTnN3ruNESUf8snkMygrOcjZZ59Nu3btWLNmDenp6YSFhbFu3TpEhHPOOYdTTjmFv/71r77/Bi1atKC0tBQ8U/fHq2ogb2v+HpgeQL2f0iag+hbYToBBnVqyq7iUL7ZZFhLTOGrOwKu2fft2Lr30UqZNm0ZYWBgiwquvvsr1119Pnz59iI2Nrf6Xe528t4iOOEf120AyMjJYtWoVGzZsYPr06ezc+cOdizlz5rBixQrCwsJYs2aNrzwmJoaHHnqIt956i82bN3PhhRdy+umn069fP4YMGULbtm35/vvv2bt3L9u2bavzHNXjeuWVV3zfoXpcTqeTjz76iMmTJ/P555/z8MMPEx0dTVxcHOeeey79+/dHRHA6nWzbto3HH3+chIQEUlJSqKysZNu2bbRo0YIRI0awc+cu7n3wEQC+3LaPb/crEyffR5kzisKqaNokRCKH97LozWdpnZ5GBGWcNeA0dmzfjlZV4Xa72b9/v+/7iwgVFRUsXLiQBx54gIiICHbs2MFzzz3nq5ORkXHE79K/jdvt5tP85Wzdsol33noVVWXs2LEkJyezbdu2I+ovWrSIl156iUWLFjFjxgzmzZvn67Nly5bww3T/BqnqNFWtCLT+j2kTaH2bFXkCnOHNQjL3q13kpR/9x8WcGIcPH2bo0KF8+OGHOBwOhg4dypIlSzj99NOZPXu2r96HH37ITTfdRFlZGd26dWPLli3Mnz+f1atXc/XVV7N//34cDge33norF110EQCXXnopr732GuXl5Tz44IPMnDmTDz/88Ig2Bw4c8P1RXLZsGTfccANLly5FVRERUlNTqaqqIioqirCwMBwOB5s2baKsrIzS0lLi4uJwOBwcPHiQiooKoqOjSUpK8i2urW6zceNGysvLKSsrw+Xy/F1SVcrLy0lNTSUqKoqoqCimTp3KH/7wB4YNG0a/fv0AyMnJITo6mgMHDlBVVcV9991HeHg4WVlZOBwOoqOjKS8v57vvvqOiooLS0lJiYmJQVf7v//4Pp9PJSy+9xN69e3nqqad49NFHOeecc0hNTeXLL7+kd+/e7Nmzh/POO4+4OM9LQqqqqnj77be57777ACgoKCA1NZX58+cjIgwbNowJEybQpk0bPvnkE8rKyigpKWHJkiV8s3Ejeaf0oVPXblRGNGf58uWc0qs3hw4d5j9zPmBHcRkHS8qpqKxkyYpVdBp8Efs+2sDOor28vLKIbytbEJPQij9Pn0O5CuEJ6bTo3p/s/iNYcc3ZlFUqN7z5JV/Onsn2bzfxi0HnUll6iMqKSrYX7uXf6w5TdiCCr/81H4Dt27YSHx9Pi/D9fH2gkN1v3UVxcTFaVcVrLz4HQLdu3fjiiy+IiIigU6dOLF68mJSUlCO+e3l5OT169PD93jt27MiKFSu44oorfL/3yMjIWtvk5+fjdDoZNWoUS5Ys8c2QLCkpweVysW/fPuLj40lPT2fgwIEkJiYCcN5557FixQoGDRoEUD1js+atwKahoYdwP4etMSePVLvg8cV63iMfNfYwTmr+kydUVefOnauzZs3SYcOG+cpqTp4YOnSojh07VlXrnnBRVFSkqamp+tlnn2lGRoa2aNFC77nnnqPaLFiwQJOSkvS0007TuXPnamZmphYVFemNN96oIqJ//OMfde7cuZqRkaEVFRW6detWDQsL0/fff18HDhyoAwcO1NatW2tJSYlOmDBBAT148GCtbRYsWKDp6em+c+zZs0cBjYyM1KqqKv3oo480Li5OH374Yf3yyy81MjJSBw4cqHPnztXWrVtrUVGR/v73v1dAzzrrLP366681JSVFx48fr8XFxepwOHTmzJna97T+es2td2uLpBQNd0dqZGxzBfTyx+boyNuf0ZjEVB37j0X6ywfeVQlzaK9rHtHo1Bx1t2ilPf40Wzv//lkFFHFo9ztma1K/X2lYRIz2+PO7mnP5AyqOcHVGNdPM8/9Xw5sna2K/CzTMHa1ImDoTW2vzgZepo3krjes7WtN//6qCaMrlU1XCIzTl8sc0LDJO4/peqOJ0KxKmrW+eqdF5Q9TRIk0zJ8/WtN8+reII13ZX/k2dsfEqTrd2mXin9r3xae+4wnTckwu11+hrNKp5gv7h1c906G//rJGxzTQusZVe/cCLGhnTTMf+z20KaEREhHbu3Fnffvttzc7O1ilTpugzzzyjCQkJumvXLs3IyNAzzzxT09LS9MCBA9qrVy8FdPPmzXrDDTdo69atVVW1oqJC8/LydNeuXZqUlKQpKSn697//XT/55BPNycnRmJgY3bp1q37yySfau3fvI9pERUWpquqkSZN06tSpOnXqVP3Nb36jubm5+vjjj+uYMWNUVXXPnj3ao0cPPXjwoJaXl+ugQYN09uzZquqZ1JOamqp4Xgrd6H/Dj3Vr9AGciC0UAtvj8zdo5uTZuv37w409lJNWv3799Ntvvz2ibP78+UcEtl27dmm7du18+126dNGBAwfW2l9eXp6uW7dOX375Zb3qqqtUVTUzM1OTkpL0kUceqbNN79699a677vK1ad26tcbHx/sC6JAhQ/Tjjz9WVdXw8HBdsmSJDhw4UDt06KDjxo3ztXG5XPr222/X2SYxMVEnTJigpeWVum/fPt+08e3bt+sLL7yggHbu3Fm7deumYWFh2rNnT73rrrs0Ly9PO3bsqE6nU1NSUjQxMVGLig9pdscuekr/wfqH6R+phIVpu98+oe6MXG014WFNOPc6BVFxhGtYeIT2u/0NPeU3D6iEOTSyeUttltZOne4ojWqeqHiS4Wp4RJQmZrTTcHekhkdEazNvcIyKa6Et27TXlpk5etqYqzTMGa7xKZka5gzXMIdTAU3KaKtJ6W31mvueUUDjWiRo6+yOKmFhmpicoiKiSckp6nJHaHJKqkbHxKrTGa6ZbdtqfEKihoeHa8dOnbRr16566qmnqtPp6VdENCwsTF0ul2ZnZ2urVq00KytLXS6XTpw4UVVVL774Yo2IiFBAnU6nRkRE+Kb7h4eHq8Ph0NzcXM3MzPRNq8/NzT3iHNWzD+Pi4jQuLk6zsrI0Pj5ec3JydOvWraqqevXVV/vaOBwOjY2N1dzcXH333Xe1ZcuWmpWVpREREbp8+XJVVb355ps1MTHRN65u3bppaWmpHj58WIcPH65ut1t79+6t33zzje//xxdeeEE7d+6sXbp00ZtvvtlXvnz5cj3//PMtsNXauScx5lo8KVCm1HJ8ALACz7TR0TWOTQTWe7eJfuULvH2u9G4tGxpHKAS2tTv2a+bk2frikk2NPZSTUmlpqSYnJx9VXjOwVVVVaevWrXX58uVaWlqqUVFRmpube1S7pUuXaseOHbWyslLvv/9+/ctf/qKqnoBTPUW9rjYDBgzQ6667Tv/yl7/opZde6vlDnZSkd999t6qqXn755fr666/r0qVLNTw8XLt3764xMTEaHx+vrVq10i5duqjT6dTk5GS94YYbdOf+wzryovF6ywNP6S1PvqVhDqfGt26vSJjGZvfSzMmzNXHY9YrTpWHuKO026U4d88THmtKpl467+0W9/d+rNSzMoe7IKI2LT9QB547Sfy3dqOHuCM3seaZGJKRpq4l/U2eLNA2LbqHidGt4dJwmZ3VUd2SUTrr6et13uEyjo6P1wgsv1JYtW+obb7xxxHdRVW3Tpo326NFDk5KS9LLLLvP9XkaOHKmtWrXSN954Q4cNG6alpaVH/N4yMzN19+7dvv3TTz9du3fvrrfddpuqqiYnJ+v777+vqqoOh0N79uypp556qs6YMcPXZsWKFdqqVSvNz8/XSy65RHft2lXf/y6qqjp27Fg9++yzVVUDarNixQrt2bOnb3lEIG0eeughX/3avntDbQJ1rG2uu+46nTt3bpMNbEF7xiYiDuAxYDCeFebLRWSWqvqnYNkMTAJuqtE2HvgT0AvP2odPvW33eqtcon5TQ5uCnJYxZMRHMu+rXVxyqmUhOdHqmjxRk//kieLiYiIjI4+aPFE94WL69OmEhYWhqlSp8m3hQcrKK3CGuyg8UMa6ncW+Nrt27uDSiy/hvkef5MG7/szeg2VIZClXXH8r/545i/CIKOYuWsapX+1ky95DzF/1LTNunMw1dz9Bl75ncs/Vo9m6cS39zh5N9oCRfPX7kRTuO8DTc7/gzfB5FH65k4X7VrN/2Vukjr6NTqeewYJbh1G5ayMjkvfx/rx8SmLjEEc4EaXfIwJlFVV8taOYLyu2kXb1s+yadT8VzZJZ+OEHrI1/k0ocHC6rJDY2llHtI9h05gB+NfKXDBs0gOHDh/Pee+9x/vnns+iDt1m5bBQAiYmJpKamUlBQcMTvFGDx4sWkpqZy+eWXM3fuXD766CMGDBhAQkKCr43/s866xMXFkZiYyB/+8AcAunTpQnx8PACbN28mNTWVjRs3ctZZZ9G1a1fatWtHjx49yMzMZMuWLbz44osNngPgrLPOwuFwUFlZGVCbHj16kJeXx/jx4wECatO8eXMuvfRSgIC+e802gTrWNrm5ub5nbU1RMCeP9AE2qOpG8LxHB082Z19gU2/GZxGp+YDyHOADVd3jPf4Bnqu/V2iiRIRBHZN5ZdlmDpdVEulyNNzI1CnQiSD9+/enuLiYiooKNmzYwKhRo/j3v/9db9/r1q1jx44dVFZWUl5e7lvvBZ73WA0bNoxzhp7HxRMmsWnDOrqN+g3fffcdzx5cwK4DZVQdLOb+++/nwb8/Tvygq3CndWLnK7cQ13cMNy84yJb8fFZ8/S0RrbvyZkkuxQcOcEgi2f3paq6Yns+OT79k/oKFNB9wKTOKWjHjP1+xY3852iyV5fkrOND+XMQZTlVZCd07tOaKkV14eP5hvl/3Lg89ej8TLh6LiJB0TySpqakk7F/P7o1rSE1KJCoqiivP6sSVV/ajw0MHeOeWkaSkpKCqDFj1DKcNPJ3XXv6O3/SO4b7/hJMZcRgiYrmgTzvu/+9bdM1pQ1paGt27d2fhwoWEh4dz5plnsmzZMgDS09OJiYkhOjoa+GFiA+D7mZOTw3fffceyZcsYMGAABQUFtG/f3temIenp6ZxxxhkkJCTUeY6srCzOOOMMPvvsM9q1awdAWlqab/1cIH79618HXLfas88+e0z1L7vssmM+x4lo82O+eygJ5nT/NMB/AUxtmZt/bNtp3hfQ3S7V/xxsAs7ulExpRRWLNxQ29lCavJrvsbr55pt965X8LVy4kJUrV/LFF1/gcrkYPnx4vf3u2bOHO+64g6VLl/Lxxx+zf/9+xowZw75D5Tz68izS2rSjsl1/ZpV0pHTg9bjSuxCWnENVwefcMaQ1zdxhUFXBU2+8x5MvvEn5gscpfuEaJk6YwHN3XsfgqhXExUaTldoS9/aVPDgmj5joGCr37eCXZw/gkaHJVG7/it652Xz82I0s+P2prPrzEHq1bk56ZCXJWsSiW4fiCgOtquTPV19MZMFyVn/6Cf9zzdWMHjWCAwcOAOB2u/n6668pKChgyJAh7Nu3jy1bttClSxeeeeYZioqKSElJYe/evZSVleEIE/r0yKNg8yYG9umGy+Xis88+Y//+/Rw+fJi1a9fSp08fCgoKWLhwIR06dKCyspIlS5aQm5sLwIgRI/j8889p3749Tz75JEuXLqVPnz4cPHiQ4mLPFezZZ5/NkiVLaN++Pd9++y3r16+nuLjY10dDRowYwfPPP4+qsmTJEpo1a+b7Ht61VxQWFrJ48WI6d+7sa7du3Tq6dOkS0DlMExese5zAGLypU7z7lwJ/r6Puc/g9YwNuBm7z278duNH7Oc37MxbPy+cm1NHnVXjynOVXzzRqbKXlldrljvd0ypufN/ZQmryGJoKUllfq0o1F+uD7a/X8xxdr99tmqjic2uvX9+gl/1yiv30hX9M69tCouBYa7nJri6RWesujL+jk+x7Tdh06a9vs9pqamaVRzeK17dmXaJspszXmlOEKonFp2dqqbQfN6tBFu3U/RZcvX67jxo3zTSBwOBw6dOhQVVWNjo5WEdFu3bppenq6iog6nU4NDw/XyMhIXx7DsLAwbdOmjbZq1UoBTUhI0E6dOml4eLg6nc4jJjWEh4f72mdlZWmzZs18E0E6derky6mId5aew+HQlJQUfeqpp7R58+aalZWl0dHResYZZ6iq6j333OM7h8Ph0NTUVG3Xrp3GxsZqnz591O12a1pamsbExGhOTo526NBB4+PjNTw83DMJJDxc3W63ioimpaX5xpWYmOibcPHNN99oXl6eLz/k2WefrVlZWdq+fXudNWuWduzYUcvLy/Xcc8/1TZ545JFHNC0tzTf+K664QlU9z0F/97vfaVZWlubm5vomTyxevFhzc3M1Ly9Pc3Nzj3imtGPHDt/sQRM4mugztmAGtn7AHL/9W4Bb6qhbM7CNA/7ht/8PYFwt7SYBUxsaSyhMHql29Yv52vuuD7Sysqqxh9Jk1TYRpKqqSqe/OVu79jtTJz27VDvd/q5mTp6tbafM1pFTF+l5196l6Z37aFbfofqrxxbp2Q8u0FPvnqudvfWqt+ZnXK7N+o//Yf+UYRrfuoM+OnedXnzlNbo0f8UR5x04cKDvD6uq6jXXXKN/+ctfdPz48aqqmpubqw888ICqql5wwQWan59/1IQV/4kNqoFNOnjggQd8k1puuukm/fzzhv+x9NZbb/kmXATSpuZkiED4nyPQSRr+bYLlx0y4ME03sAXzGdtyIEdE2uJJZjkWT56vQMwB7hGRFt79IcAt3remNlfVQhEJB4YDc4/zuINqUMdk3lm9gy+27bPF2j9S9USQbd8fZtGGQhZvKGTxhiIKvlzF/l0HSNxziNE90zktO5G+WQk0iwzn3HPv4m93/i/79u1j4sS+vluYABWVVRworaC4pIK/PbyCAwcPM3pCL1KaRfBm5Kd8+WU0vzvj/7d37nFVlVkf/y7uJHfhIEqC4AUzE8Vr5Q01dHTMHAvNqbTLOM3oa32ysncafWum0XnNZmqyqcku5jtWdrMyzczStDHFCktLi0RTQUHEa0AB6/1jb06Hm+IFD5zzfD+f8+Hs5zxr77Vg6zrP3muvXxLThjx+Wt9UlcTERFq3bk1FRQW9e/cmMTGR5cuX43A4SEtLY+3atdVsXAsb0tLSGlR00LZtW6ZMmUJFRUW9wpQ1KS8v56677gJokE3NYogzPUZDizRcbRqLsym4MDRjGjNrAr8AvsHq9vwHe+xBYLT9vhfW/bOTQBGw3cX2ZqzHBHKAyfZYC+BT4AtgO/AoVi+zZrNiKzpRpokzl7bDy7wAABkUSURBVOsj7+10tysNwlXqRFU1IyNDw8PDq604VFX/8Y9/aHJysvNh04bY9O/f3ykR0qNHD73iiitq2fTo0cMpdbJp02bt2OUyFR8fBdQnKER9Q6I1sGW8jpn5uM5+8mUdPPQq5zNegM6YMUMff/xx9fHx0RYtWujll19eSyJk9+7dTpkPQFNTU50+tm7dWq+55hpNTk7Wjh076rvvvquqqiUlJdqrVy9t0aKFtmvXTmfNmqWq1mW9rl27Oh/Krnq+bObMmdqmTRtNSEjQ2NhYDQ4O1okTJzqPM3bsWF29evV5+7sZDOcDmumKze0OXIhXU0psqqpjn/hYRz7WPLqQNKRbh6p12So3N1cTEhJ07ty5p7UpKiqq1hUjKChIMzIyatkMHDhQ316Xpe269tL24+9XCWyhcWPu0cCQCPXx9dPPvzug3333nSYlJen777+vI0eO1A0bNjh96dq1q4aHh2tmZqbOmjVLfX19taioSBcuXKgtW7ZUVevSZnZ2tu7YsUP79Omjvr6+un37dj18+LBGRkaqw+HQ0tJS3bVrlyYlJWl5eblWVlbq8ePHdeDAgU6Jko0bN+q2bds0OTlZJ0+erLt27XJKsLhS81KkqnXJMi8v79z/YAbDeaS5JjbTBNkNDOnsYNv+Yxw4Wnr6yW7GVeoEYMiQIYSGhtaa1717d2fn+Ndee+20NqtWrSIjI4OoqCgiIiIQEfz9/Z2fd+zej/dzjrG9oIypKw6Sf7QU/4IdtGvXjocz07i0UzJxrWIpObiLpKQkDh8+zLhx41izZg2ZmZns3LkTVeXIkSP4+fkxadIk8vPz8ff3Z8WKFYSHh1NeXk5+fj4BAQF069aNTp06Ofshjhw5kl69enHllVcCVoXhQw89RExMDJs3b2bZsmWkpKSwceNGRo8ezddff42I0KVLFyZNmsTixYsZPnw4CxYsqHbZsy4OHjxIcHCws1+gwWA4N0xicwNDO1t6UWt2NG2NtlNJndSHqrJ79+7T2lRJikyePJnY2FjKysq4rEcvFm/cza/++R/6/++HvJz1PX6+Pvx5zKWktY3k0pZC6+gIHpj9R3Jycjh8+DC7d+8GYOzYsTz99NOUlJSwZ88e7r33Xvbu3cvFF1/MHXfcwfDhw8nLyyMqKootW7awadMmOnXqVEte5ZNPPmHKlCnk5uaSk5ND27ZtERGKiopYuHAhXbp0Yf/+/VxzzTXs2bOHzp0788MPP3D77bfTp08fAO6//34GDRrEkiVLGDFiRK3YBw0aVO1ZuyVLljBlypQG/44NBsOpMYnNDXRwhBAfGcwHXxe425VT0tBuHa5UVFQQHh5+2nnWVQ54/Mmn+csL7yC+fjy69H3++OZ2jpf+xN0ZnXhsfHfSEiL5dd8E/HwFVSUzM5MdO3Ywfvx4/Pz8eOONN5z7rHqk0dfXl+zsbOLj49m/f381GZOgoCCOHTvGvHnzCAkJqSWv0qdPH955551qNmFhYU65j7qOs2/fPjZv3sy2bduccxwORzWbUxEREcFNN93UoLkGg+H0GNkaNyAiDO3c9LuQBAcHV0lXNBgRcT4kW0V5RSUHjpZQdKKMZzfkknvoJB9+U0rOF5t45uQqyktOUKkQ9uMhVk7vT0qrUESEtWv3VduPw+Hg+PHjiAgJCQmEhYU5V1yu3Seq8PHxoWvXrk5hyvj4eLKysmjVqlW9NgEBASQlJbF+/XrGjRtHfHw8J0+erCUR4kpERASDBg3i3XffdT5kXFpa6rQ5HWfTScJgMNSPWbG5iSGdHdW6kJSUlDBw4EAqKioAazWQmppKamoqo0ePPu3+ysrKyMzMpH379vTp08d5ia4mN998Mw6Ho1aXh8OHDzNs2DA6dOjAsGHDKC4uJjIy0tkR4rLLLuPyyy9n69atgKWhNWDAAMrLf9b8KzxeRrkKJ0p+ZPbrn3Proix6Tv8nkanDuOPlbLbuO8qDy79iWfZ+QpJ7cDIni9t6O1h4YxpU/MToYQPoHBfGsmXLuO+++2r5PmDAABYvXkxZWRndu3cnPz+fvn37kpuby7Zt23jiiScoLCzkyJEjgLXays/PJzc3l+LiYtLT0zl06BATJ06s1rFi3759TgXk8vJytm7dSqdOnQD45S9/SUFBAXFxcc4uGb179652nJKSEt5//31SUlKcvpouFwaDG3F39cqFeDW1qkhVqzPGJX9c6exCUrP6sEWLFme0vwULFuiUKVNUVfXFF1/U6667rs5569at008//VS7dOlSbfzuu+/WOXPmqKrqnDlz9J577lFV1VGjRunrr7+uqqorVqzQkJAQjY6O1sDAIA1uEaJXjL1ZJ/xro8b/4nb1DWmpiI+KX4BelJSmGX9bp8OmzdWQyGgNjYjSwMAgjWvdRleuXKkVFRXavn17ZyeNmJgYXbZsmaqq9u7d2ymXEhAQoD4+Purn56cOh0ODg4PV399fAwICtF27dpqYmKgdO3bUpKQknThxom7dulXbtGnj7LwRGhqq/fv31+TkZE1MTNTQ0FBnx4qrr75as7Ky9L333tO2bds6O2mEhobqVVddpaqWfMcll1zi7JKxYsUKVVXdunWrpqamateuXbVLly76wAMPOH+XpsuFwVOgmVZFut2BC/FqiolNVfW3i60uJJWVlbVaRJ1pYnPV4/rpp5+0ZcuWWllZd3eT3NzcWomtY8eOznLzvLw87dixo1ZUVOryNR9r+qhf6d9W79TJT36gAWHRmjjT6soRN/kxbZHcU0f/Y73e88pWfWb9Ll3/TaG+u+4/OnGi1XmjoV0xPvvsM2e3joZ2rHClIcc5m+4TVfIdZ4LpcmHwFJprYjP32NzIkM6xrNx2gM9zD9WqPiwtLaVnz574+fkxc+ZMxowZc8p9VVUZAvj5+REeHk5RUZFT9v10HDx40FluHhXjYG/eAS574D1OlJVzoqIN363ege/2lSR1v4IpQzqQ0iqUDjH96Xvpg7w59crqO+sQzf6cr8+oK0b37t0ZPHhwgyVCatKQ45xN94mzke8wXS4MBvdiEpsbGdwpBhF4a/OOWtWH9elK1Yf15ao6ZyN8sPG7Iv6w7EvKfqrgyvbRpKc4SJl6BXlfbeHOt9ayYcMGp1wIWMUWx48fr/Wc2s0333zGxz4bmzPhbIo0zka+wxSDGAzuxRSPuJGWIYH0aBvJxj3Ha1Uf1qUrVRPXgpOLLrqI9PR0UlNT6datG7t27eKjjz6q0yYzMxNVpaysjJSUFAICAjh27BhXjP89E57+hB+KD9EmrhVP3pDGdb0uZtbUSYwckUFFRQUhISHOIpW0tDQKCgro0aOHs+AEYMeOHTgcDkTEWYEI1nNx/fr1Y+jQodWKVKps+vXrR2BgIA8//HA1m5pFKgaDwXAqTGJzM+kpDnYWK8d+KOPoiZMAp9SVuu+++5zPbrlqkk2aNIn09HSys7OZNm0a/v7+ZGRk1Dres88+y/DhwxER7rzzTk6cOMHz63bQIjmNja8t5NberRgZ/A3jrx0LWCvHrKwsFi5cSEBAAM888wyRkZHk5OQQHR2Nr68v3377LUOGDGHu3LkAREVF8ac//anWCiwgIAAfHx9atmxZp81jjz3GjBkzatkMGTKEl19++Xz9yg0Gg6fj7pt8F+LVVItHVFWLT5bpDc9s0hZdh+mlt87TFV/k6YYNG+rVlRo5cqSzSMS14KSkpETHjRunycnJmpCQoKNHj1ZV1f379+uIESOc9i1bttTo6Gj18/NTX19fjW7fTRPuXa6j/rpcRURjY2M1PT1di4qKVFX1lltu0YiICO3cubMGBQVpWFiY8/itWrXSwMBAraysdBacuDJ9+vRa8jIJCQk6ePBgVdU6bWbPnq3z5s2rNpadnV0tBoPBcGGgmRaPuN2BC/FqyomtiqffWKNxacM04d7lOu6fH2v298V1zqsqQ69Lk6yKwYMH69tvv11rvMrmh7Jynbvya/UNc2hw6w76wvqdWlBQoH5+ftXK1l2pqqTs0qWL7t27V1VV/f399eKLL9bCwkJVVY2IiKhmU1diCw8P1+joaOd2TZu6Elt5eXk1G4PBcGForonNFI80EW4dk44e2kXwJZfw9w9yuHrBx4xJbc3dw1NoE/FzB4tVq1YB9be7ys/P58svv6zzMuShQ4cIuCiUYX9bx77iEiLCQpgw7pfMn3odL8TEEBQUhJ/fqU8J61y37n35+/vj7+9/xkUq9RWc1Ievr+8Z2xgMBu/F3GNrQtx26638+vJ2fDhjEL8fnMzKbQdIf3gt81bt4ERZ9eKJ+tpdLV26lGuuuaZap3yA/KMl/HH5TvKKjhHk78tLv+lL2iXtuf66cWRnZ7Ny5Up+/PFHunXrdkof4+Pj2bt3LwEBAbRp04bi4mKioqLIz8/H4XCcNsbY2FhKSkoICgpqsA1YnVWCgoIaNNdgMHg3JrE1QUKD/Lk7I4UPZgxixKWtWPDhdwyat5Ylm76nvKISgMjISCoqKmoltyVLXmTA8Kt5e2seD6/aya2LthA/+Hq6Tf4zG74vJTTQh9d/04u+SS0ZNWoUTz31FADz588nKCjIudJzLVJxZfTo0SxatAiATp06ER8fj4iwaNGialI19TFs2DD8/Pzw9/dvsE1RURExMTG1krXBYDDUhVRdWvJkevbsqVu2bHG3G2dN9t4jPPTOV2TtLqZTbCh/GNmZ/h2imXjjZHoOGUVIUnd2HDjO59u/Yf0jv6PN755DxAdfH6FddAtyFt/Pr26exvTrR/HAPdOYMGECQ4cO5ciRI7Rt25bS0lICAgJ46aWXGDVqFHl5eaSmpvLmm2/Sr18/JkyYwNq1azl06BAOh4O4uDiOHDlCWFgYQUFBFBYWEhsbS3BwMKtXr+bAgQMkJSVRWlqKquLj48Ojjz7K1KlTee6555g9ezaBgYG0bduWV155haioKA4cOEDPnj05duwYPj4+hISE8NVXXxEWFsarr77Kxo0bmT9/vrv/FAaDVyEin6pqT3f7caaYxNZMUFXe3XaAOSt38P3hHwgN8qNoz06OZS0jetRdOEIDSYkLI6VVKJ1iQ0mJCyU5JoQgf18yMjKc9+Y+//xzHnnkERYvXnzK47nanE/Gjh3LnDlznE2GG8vGYDCcO801sTVq8YiIDAceBXyBhao6t8bnA4C/A5cB41X1VZfPbgLutzf/rKqL7PE04HkgGFgBTFcvyM4iwoiucaR3dvDvT77n24ITdBzWkZxOcOfv0okOq18ixTVBubauOpWyc2MktR9//JExY8acUYI6GxuDweDdNNqKTUR8gW+AYcA+IAuYoKpfucxJBMKAGcBbVYlNRKKALUBPQIFPgTRVLRaRzcB04BOsxPaYqq48lS+esGIzGAyGC01zXbE1ZvFIbyBHVXep6o/AS0C1SgFV3a2qXwCVNWwzgNWqelhVi4HVwHARiQPCVHWjvUp7ATh1d2CDwWAweBWNmdjaAHtdtvfZY+di28Z+f9p9ishvRGSLiGwpLCxssNMGg8FgaN40ZmKr66ndhl73rM+2wftU1X+pak9V7RkTE9PAwxoMBoOhudOYiW0fcLHLdjyQd462++z3Z7NPg8FgMHgBjZnYsoAOItJORAKA8cBbDbRdBVwlIpEiEglcBaxS1XzguIj0FauP043Am43hvMFgMBiaJ42W2FS1HJiKlaS+Bpaq6nYReVBERgOISC8R2QdcCzwlIttt28PAn7CSYxbwoD0GcDuwEMgBvgNOWRFpMBgMBu/CPKBtMBgMhjox5f4Gg8FgMDQBvGLFJiKFwJ7TTIsGDl0Ad5oi3hw7eHf8JnbvpSHxJ6hqsysr94rE1hBEZEtzXHKfD7w5dvDu+E3s3hk7eHb85lKkwWAwGDwKk9gMBoPB4FGYxPYz/3K3A27Em2MH747fxO69eGz85h6bwWAwGDwKs2IzGAwGg0fh9YlNRIaLyE4RyRGRme72p7ERkWdFpEBEtrmMRYnIahH51v4Z6U4fGwsRuVhEPhSRr0Vku4hMt8c9Pn4RCRKRzSKy1Y79AXu8nYhssmN/2W5/57GIiK+IfC4iy+1tr4hfRHaLyJciki0iW+wxjz3vvTqx2WKoC4ARwCXABBG5xL1eNTrPA8NrjM0E1qhqB2CNve2JlAN3qWpnoC/we/vv7Q3xlwHpqtoNSMXSN+wL/BX4mx17MXCLG328EEzHavFXhTfFP1hVU11K/D32vPfqxEYDxFA9DVX9CDhcY/hqYJH9fhEeKt6qqvmq+pn9/jjWf3Bt8IL41eKEvelvvxRIB161xz0y9ipEJB4YidVrFruRutfEXwcee957e2I7FzFUTyLWVk7A/ulwsz+NjogkAt2BTXhJ/PZluGygAEuV/jvgiN2wHDz//P87cA9QaW+3xHviV+A9EflURH5jj3nsee/nbgfczLmIoRqaKSISArwG3KGqx6wv7p6PqlYAqSISAbwBdK5r2oX16sIgIqOAAlX9VEQGVQ3XMdUj4weuUNU8EXEAq0Vkh7sdaky8fcV2LmKonsRBEYkDsH8WuNmfRkNE/LGS2r9V9XV72GviB1DVI8BarPuMESJS9QXXk8//K4DRIrIb65ZDOtYKziviV9U8+2cB1pea3njwee/tie1cxFA9ibeAm+z3N+Gh4q32PZVngK9V9RGXjzw+fhGJsVdqiEgwMBTrHuOHwDh7mkfGDqCq96lqvKomYv07/0BVJ+IF8YtICxEJrXqPJdy8DQ8+773+AW0R+QXWNzdf4FlVfcjNLjUqIvIiMAirs/dBYDawDFgKtAW+B651EXb1GETkSmA98CU/32f5b6z7bB4dv4hchlUg4Iv1hXapqj4oIklYK5go4HPg16pa5j5PGx/7UuQMVR3lDfHbMb5hb/oBS1T1IRFpiYee916f2AwGg8HgWXj7pUiDwWAweBgmsRkMBoPBozCJzWAwGAwehUlsBoPBYPAoTGIzGAwGg0dhEpuhySIiKiLzXbZniMj/nKd9Py8i404/85yPc62tJvBhjfFEO75pLmOPi8ik0+zvtyJy42nmTBKRx+v57ERd4+cLOy5X5YjbROQzT+ocb2j6mMRmaMqUAWNFJNrdjrhiq0I0lFuA36nq4Do+KwCmn4lUiqo+qaovnMHxzxsuHToaOv8GYBpwlaoWN45XBkNtTGIzNGXKseTr76z5Qc0VV9VKREQGicg6EVkqIt+IyFwRmWhrkX0pIskuuxkqIuvteaNse18RmSciWSLyhYhMcdnvhyKyBOsB75r+TLD3v01E/mqPzQKuBJ4UkXl1xFeIJRdyU80PRCRZRN61m9auF5EUe/x/RGSG/b6X7eNG2+dtLrtobdt/KyL/W2Pf8+1V1BoRibHHUkXkE3t/b1StsERkrYj8RUTWYSXha+0Yt4rIR3XEVHWM67BkUK5S1UP1zTMYGgOT2AxNnQXARBEJPwObbli6W12BG4COqtobS65kmsu8RGAglpTJkyIShLXCOqqqvYBewG0i0s6e3xv4g6pW0+wTkdZYul7pWFpnvURkjKo+CGwBJqrq3fX4Ohe4q45V4L+AaaqaBswAnqjD9jngt6raD6io8VkqkGn/DjJFpKonagvgM1XtAazD6jwD8AJwr6pehpW4Z7vsK0JVB6rqfGAWkGHruo2uJ6YE4HGspHagnjkGQ6NhEpuhSaOqx7D+0/2vMzDLsrXXyrCkWd6zx7/ESmZVLFXVSlX9FtgFpGD10btRLHmXTVjSJh3s+ZtVNbeO4/UC1qpqoS2B8m9gQAPjywU2A9dXjYmlPnA58Irtx1NAnKud3fcxVFX/Yw8tqbHrNap6VFVLga+wkg1YrcRett//H3Cl/aUhQlXX2eOLavj/ssv7j4HnReQ2rPZcdVGI1aLpunoDNxgaEW+XrTE0D/4OfIa1QqmiHPuLmd3c2PU+lWuvv0qX7Uqqn/M1+8kplpTJNFVd5fqB3V/wZD3+navuzV+wxC6rLu35YOmEpZ7C5nTHdP0dVFD/v/WG9NRzxq2qvxWRPlir3GwRSVXVohrzf8BSpd8gIgWq+u8GHMNgOG+YFZuhyWM3Zl2KdZmwit1Amv3+aixF6DPlWhHxse+7JQE7gVXA7WLJ2yAiHe2O6KdiEzBQRKLtS4oTsC7zNQhV3YG1qhplbx8DckXkWtsHEZFuNWyKgeMi0tceGt/Aw/nwczf764ENqnoUKBaR/vb4DfX5LyLJqrpJVWcBh6gu++TqXyEwHPiLiGQ00DeD4bxgVmyG5sJ8YKrL9tPAmyKyGasAo77V1KnYifUfeCzWvapSEVmIdbnyM3slWAiMOdVOVDVfRO7DkkARYIWqnqkEyENY3eWrmAj8U0Tux0raLwFba9jcAjwtIiex9NWONuA4J4EuIvKpPT/THr8J6z7jRViXZSfXYz9PRDpgxbmmDp+cqGquiIwGVojIWFXd1AD/DIZzxnT3NxiaKSISoqpV1aAzgThVne5mtwwGt2NWbAZD82WkvVL0A/YAk9zrjsHQNDArNoPBYDB4FKZ4xGAwGAwehUlsBoPBYPAoTGIzGAwGg0dhEpvBYDAYPAqT2AwGg8HgUZjEZjAYDAaP4v8BfpVzl/UMRJoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.138 0.11  0.102 0.101 0.101 0.103 0.103 0.103 0.104 0.105 0.105 0.105\n",
      " 0.105 0.105 0.106 0.105 0.106 0.106 0.106 0.106 0.106 0.106 0.107 0.107\n",
      " 0.107]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in tqdm(neighbors):\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 9.007\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='brute')\n",
    "knn_optimal.fit(d_train,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.995\n",
      "The value of TNR is 0.119\n",
      "The value of FPR is 0.881\n",
      "The value of FNR is 0.005\n",
      "\n",
      "Test error is 10.183\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGXFJREFUeJzt3XuYXfPZ//H3Z3JwSDQJOYhENQiqdTmTH57+kF4jiNODoi1T4hlFHYpWHEqdnqYHhzrUI0iFEgltJFWHRkjFU2mSSioIEiEygkFSPxLE7Ll/f+yVsRNz2JGZ7D1fn5drXbPWd52+O+a69z33+q61FBGYmVn7V1HqDpiZWetwQDczS4QDuplZIhzQzcwS4YBuZpYIB3Qzs0Q4oJuZJcIB3cwsEQ7oZmaJ6FjqDjSl83r9fQurfc5W3TYrdResDM2tna61Pcan7y4oOuZ06rnlWp+vLThDNzNLRNlm6GZm61R9rtQ9WGsO6GZmALm6UvdgrTmgm5kBEfWl7sJac0A3MwOod0A3M0uDM3Qzs0T4oqiZWSISyNA9Dt3MDIhcXdFTSyS9JmmOpNmSZmZtG0uaJGle9rNH1i5J10uaL+lZSbsUHKcq236epKqWzuuAbmYG+YuixU7F2S8idoqI3bLl4cDkiBgITM6WAQ4EBmZTNXAz5L8AgEuBPYE9gEtXfgk0xQHdzAzyJZdipy/mMGB0Nj8aOLyg/c7ImwZ0l9QXOACYFBFLImIpMAkY0twJHNDNzCB/UbTYqWUB/FXSPyVVZ219IuJNgOxn76y9H7CoYN+arK2p9ib5oqiZGaxR5p0F6eqCppERMbJgee+IWCypNzBJ0ovNHa6x3jTT3iQHdDMzWKNb/7PgPbKZ9Yuzn7WSxpOvgb8tqW9EvJmVVGqzzWuAzQt27w8sztr3Xa19SnP9csnFzAxa7aKopC6SNlo5D1QCzwETgZUjVaqACdn8ROCEbLTLIOD9rCTzKFApqUd2MbQya2uSM3QzMyCi1W4s6gOMlwT5GHtPRDwiaQYwTtIw4HXg6Gz7h4CDgPnAcuDEfH9iiaQrgBnZdpdHxJLmTuyAbmYGrXZjUUQsAHZspP09YHAj7QGc3sSxRgGjij23A7qZGfjhXGZmyUjg1n8HdDMzgNynpe7BWnNANzMDl1zMzJLhkouZWSKcoZuZJcIB3cwsDeGLomZmiXAN3cwsES65mJklwhm6mVkinKGbmSXCGbqZWSLqin/BRblyQDczA2foZmbJcA3dzCwRztDNzBLhDN3MLBHO0M3MEuFRLmZmiYgodQ/WmgO6mRm4hm5mlgwHdDOzRPiiqJlZInK5UvdgrTmgm5mBSy5mZslwQDczS4Rr6GZmaYh6j0M3M0uDSy5mZonwKBczs0Q4Q7fWts02W3L3H25uWB4w4Ktcdvlv+NuUv3PjjSPo2rULCxcu4oSqM/jggw9L2FNra4/NfIBlHy4nV19Pri7H0ZVVbPfNgfz8V8PpvP565OpyXH7+L5kz6wV232sXbrrzN9S8vji/71+e4HdX317iT9DOOKBba3v55QXsvscBAFRUVPDaqzOZMOER7h1zC+cPv5KpU6dRVXUM557zQ35+2W9K3Ftra1X/eSr/XvJ+w/J5l5zBTb+5jamPP823Bu/FeZecQdURpwLwz2mzOfX755Sqq+1fKz+cS1IHYCbwRkQMlTQAuBfYGHgGOD4iVkhaD7gT2BV4DzgmIl7LjnEBMAzIAWdGxKPNnbOiVT/Bqh9mO0nnS7pe0m+z+a+31flStP/++7BgwUJef/0NttlmK6ZOnQbA5MlPcsQRB5W4d1YKEdB1oy4AdP1KV2rferfEPUpIfX3xU3HOAuYWLP8SuDYiBgJLyQdqsp9LI2Jr4NpsOyRtDxwLfAMYAvwu+5JoUpsEdEnnk/8mEjAdmJHNj5E0vC3OmaLvHH0oY8dNAOD551/ikEMqATjyyKH0779ZKbtm60AE3D7uBu6fNJqjjz8cgF9cfA3nXXomj8/6Mz/9+Zlce9VNDdvvtNsOjH/ibm4Zcx1bb7tlqbrdftVH8VMLJPUHDgZuy5YF7A/cn20yGjg8mz8sWyZbPzjb/jDg3oj4JCJeBeYDezR33rYquQwDvhERnxY2SroGeB4Y0UbnTUanTp0YOrSSi3+W/6eqPuVcrrnmci668GwefHASK1Z82sIRrL377tCTeeftd9m4Zw9uv+9GXp23kMpD9mfEJdcy6cEnGHLot7nyuos56agf8cKzLzF410NZvuwjvjV4L24c/SuGDDqq1B+hfVmDUS6SqoHqgqaRETGyYPk64KfARtnyJsC/I2LlWzRqgH7ZfD9gEUBE1El6P9u+HzCt4JiF+zSqrUou9UBjKWTfbF2jJFVLmilpZn1uWRt1rX0YMmQ/Zs2eQ21t/k/ql156hYMP/h6D/s9BjB33AAsWLCxxD62tvfN2/v/9kneX8thDU9hhl+05/JiDmfTgEwA8MvExdth5ewCWfbiM5cs+AuDJyX+nY8eOdN+4W2k63k5FfX3xU8TIiNitYGoI5pKGArUR8c+Cw6uxU7awrrl9GtVWAf1sYLKkhyWNzKZHgMnk60qNKvxHqujQpY261j4c853DGDt2QsNyr16bACCJC4afxchb7ypV12wd2GDD9dmwy4YN83vvuyfz5r5C7VvvsPteuwAw6D92Z+GCRQD07L1Jw7477Lw9qqhY5WKqFaH1Si57A4dKeo186Xl/8hl7d0krqyL9gcXZfA2wOUC2vhuwpLC9kX0a1SYll4h4RNI25Os9/ch/09QAMyKi/Y/eb2MbbLA+gwd/i9NO/+xywzHHHM6pP6wC4IEHHmb06LGl6p6tA5v02pgb7vg1AB07dODBPz3KU09M45Jz/5sLrzyHDh078snHn3DJub8AoHLo/hz3gyOpy+X45KOPOfeUi0rZ/faplZ7lEhEXABcASNoXOC8ivifpPuAo8kG+CliZsU3Mlp/O1j8eESFpInBPVqreDBhI/ppkkxRl+h69zuv1L8+OWUlt1c0Xg+3z5tZOb6w8sUaWXf69omNOl0vuLup8BQF9qKQt+WzY4izg+xHxiaT1gbuAncln5sdGxIJs/4uAk4A64OyIeLi583kcupkZQF3rFw8iYgowJZtfQCOjVCLiY+DoJva/Criq2PM5oJuZgR+fa2aWDD8+18wsDeFnuZiZJcIZuplZIhzQzcwS4RdcmJmlwe8UNTNLhQO6mVkiPMrFzCwRztDNzBLhgG5mlobIueRiZpYGZ+hmZmnwsEUzs1Q4oJuZJaL9l9Ad0M3MAKKu/Ud0B3QzM3CGbmaWCl8UNTNLhTN0M7M0OEM3M0uFM3QzszREXal7sPYc0M3MgHCGbmaWCAd0M7M0OEM3M0uEA7qZWSIip1J3Ya05oJuZ4QzdzCwZUe8M3cwsCc7QzcwSEdH+M/SKUnfAzKwcRH3xU3MkrS9puqR/SXpe0mVZ+wBJ/5A0T9JYSZ2z9vWy5fnZ+q8VHOuCrP0lSQe09Bkc0M3MgPqcip5a8Amwf0TsCOwEDJE0CPglcG1EDASWAsOy7YcBSyNia+DabDskbQ8cC3wDGAL8TlKH5k7sgG5mRv6iaLFTs8fJ+zBb7JRNAewP3J+1jwYOz+YPy5bJ1g+WpKz93oj4JCJeBeYDezR3bgd0MzPWLKBLqpY0s2CqLjyWpA6SZgO1wCTgFeDfEQ2PAKsB+mXz/YBFANn694FNCtsb2adRvihqZgbEGjwOPSJGAiObWZ8DdpLUHRgPfL2xzbKfjaX80Ux7kxzQzcxom3HoEfFvSVOAQUB3SR2zLLw/sDjbrAbYHKiR1BHoBiwpaF+pcJ9GueRiZkZ+2GKxU3Mk9coycyRtAHwbmAs8ARyVbVYFTMjmJ2bLZOsfj4jI2o/NRsEMAAYC05s7d9EZuqT1IuKTYrc3M2tPcq33LJe+wOhsREoFMC4iHpT0AnCvpCuBWcDt2fa3A3dJmk8+Mz8WICKelzQOeAGoA07PSjlNajGgS9ojO2E34KuSdgROjogzvsAHNTMrS611Y1FEPAvs3Ej7AhoZpRIRHwNHN3Gsq4Crij13MSWX64GhwHvZCf4F7FfsCczM2oPWGrZYSsWUXCoiYmF+WGSDZtN+M7P2Zk1GuZSrYgL6oqzsEllN6Azg5bbtlpnZulXOmXexignop5Ivu3wVeBt4LGszM0tGrr79D/prMaBHRC3ZVVczs1R9KUoukm6lkbuTIqK6kc3NzNql+gQen1tMyeWxgvn1gSNY9fkCZmbtXgrPQy+m5DK2cFnSXeQfNmNmlowvRcmlEQOALVq7I6urT+Ff11rdsy/cW+ouWKK+FCUXSUv5rIZeQf7W1OFt2Skzs3Ut+VEu2UPWdwTeyJrqs4fGmJklJYXA1uxXUha8x0dELptS+MxmZp9THyp6KlfF/I0xXdIubd4TM7MSaq3H55ZSkyWXggex7wP8l6RXgGXk36IREeEgb2bJqC91B1pBczX06cAufPYiUzOzZEWjb3xrX5oL6AKIiFfWUV/MzEqmroxLKcVqLqD3knROUysj4po26I+ZWUmknqF3ALrS+JunzcySknoN/c2IuHyd9cTMrIRSz9Db/6czMytS6hn64HXWCzOzEsslkMM2GdAjYsm67IiZWSkl8Aa6L/S0RTOz5NSnnKGbmX2ZpPCgKgd0MzPSvyhqZvalUS+XXMzMkpArdQdagQO6mRke5WJmlgyPcjEzS4RHuZiZJcIlFzOzRKQwbLGYd4qamSUvp+Kn5kjaXNITkuZKel7SWVn7xpImSZqX/eyRtUvS9ZLmS3q28B3Okqqy7edJqmrpMzigm5mRz9CLnVpQB5wbEV8HBgGnS9oeGA5MjoiBwORsGeBAYGA2VQM3Q/4LALgU2BPYA7h05ZdAUxzQzcxovYAeEW9GxDPZ/AfAXKAfcBgwOttsNJ+9r/kw4M7ImwZ0l9QXOACYFBFLImIpMAkY0ty5XUM3MwPa4pWikr4G7Az8A+gTEW9CPuhL6p1t1g9YVLBbTdbWVHuTnKGbmbFmGbqkakkzC6bq1Y8nqSvwR+DsiPh/zZy6sa+SaKa9Sc7QzcxYs1v/I2IkMLKp9ZI6kQ/md0fEn7LmtyX1zbLzvkBt1l4DbF6we39gcda+72rtU5rrlzN0MzPy49CLnZojScDtwNyIuKZg1URg5UiVKmBCQfsJ2WiXQcD7WWnmUaBSUo/sYmhl1tYkZ+hmZrTqOPS9geOBOZJmZ20XAiOAcZKGAa8DR2frHgIOAuYDy4ETIf/WOElXADOy7S5v6U1yDuhmZrReQI+Ip2i8/g2NvKs5IgI4vYljjQJGFXtuB3QzM/wsFzOzZPhZLmZmifALLszMElGfQNHFAd3MjDSetuiAbmaGL4qamSXDGbqZWSLq1P5zdAd0MzNccjEzS4ZLLmZmifCwRTOzRLT/cO6AbmYGuORiZpaMXAI5ugO6mRnO0M3MkhHO0M3M0uAM3Vpd//6bcceo39Jn017U19dz2213c8ONtwNw+mknctppJ1JXV8fDD09m+AVXlbi31toqj6yiy4YbUlFRQYcOHRg36npuGHknjz/1NBWqYOMe3bjqonPp3WsTFixcxM+uuoYXXp7PmdVVnPjdoxqO89S0mYy47n/I1ddz5CFDOPn475TwU7UPHrZora6uro6f/PQyZs1+jq5duzD9H4/w2OQn6dO7F4cecgA77/JtVqxYQa9em5S6q9ZGRt0wgh7duzUsn/i9Izmj+gQA/nDfBG7+/T1c+tMz6PaVjRj+4x/y+JNPr7J/Lpfjyqtv4tbr/ptNe/fkmJPPYr999mSrAVus08/R3rT/cA4Vpe6Areqtt2qZNfs5AD78cBkvvjiPfpttyimnnMCvfn0TK1asAOCdd94rZTdtHerapUvD/EcffYyyN+ts0qM7O3x9Wzp2XDUvmzP3Zb7afzM279eXTp06ceDg/8vjU6etyy63S3VE0VO5WucBXdKJ6/qc7dUWW/Rnpx2/yT+mz2LgwC3ZZ589+PtTf+bxx+5nt113LHX3rA1IovrHF/Gdk87gvgkPNbT/9pY7GHzE8fzlr0/wo5OPb/YYte+8y6a9ezUs9+ndk1onAC2KNfivXJUiQ7+sqRWSqiXNlDSzvn7ZuuxT2enSZUPGjb2Vc867lA8++JCOHTvQvXs39trnEM4ffiVj7vmfUnfR2sBdN1/Nfb+/kZuvvoIxf3qQmbPnAHDWKT9g8vi7OLhyP+7545+bPUY0Em+UwPsy21r9Gkzlqk0CuqRnm5jmAH2a2i8iRkbEbhGxW0VFl6Y2S17Hjh25b+ytjBkzngceeBiAN2rebJifMXM29fX19Oy5cSm7aW2gd3ZtZJMe3Rn8rb2Y88JLq6w/uHJfHpvyv80eo0/vnrxV+07D8tu179Krp6+5tMQZetP6ACcAhzQy+W+/Ftw68mrmvjif6347sqFtwsRH2W+/vQEYOHBLOnfuzLvvLilVF60NLP/oY5YtW94w//fpzzBwy6+xcNEbDds8MXUaA7bo3+xxvrndNrxes5iaxW/x6aef8vDkv7HfPoPatO8pSCFDb6tRLg8CXSNi9uorJE1po3MmYe+9duf47x/Fs3NeYOaMvwLws5+N4Pd33Mttt17N7FmTWbHiU04adnaJe2qt7b0lSznrwisAyNXlOKhyX/YZtBtnX3glr71egyrEZpv25pKfnAHAu+8t4ZhhZ/LhsuVUVFTwh3EPMOHuW+japQsX/vhUTjnnYnK5HEcMrWTrLT3CpSW5xmpV7YyiTD9Ex879yrNjVlIfLZ5a6i5YGerUc8u1vkrw3S2OKDrm3LNwfFlelfA4dDMzfOu/mVkyyrk2XiwHdDMzfOu/mVkyXHIxM0tECqNcHNDNzEij5OKHc5mZ0bo3FkkaJalW0nMFbRtLmiRpXvazR9YuSddLmp/dUb9LwT5V2fbzJFW1dF4HdDMzWv3W/zuAIau1DQcmR8RAYHK2DHAgMDCbqoGbIf8FAFwK7AnsAVy68kugKQ7oZmbkSy7FTi2JiCeB1Z/NcRgwOpsfDRxe0H5n5E0DukvqCxwATIqIJRGxFJjE578kVuEaupkZsA7umu8TEW9m53pTUu+svR+wqGC7mqytqfYmOaCbmQG5NbgoKqmafHlkpZERMbKp7Vs6XCNt0Ux7kxzQzcxYs1EuWfBe0wD+tqS+WXbeF6jN2muAzQu26w8sztr3Xa19SnMncA3dzIx8yaXY6QuaCKwcqVIFTChoPyEb7TIIeD8rzTwKVErqkV0MrczamuQM3cyM1h2HLmkM+ey6p6Qa8qNVRgDjJA0DXgeOzjZ/CDgImA8sB04EiIglkq4AZmTbXR4Rzb4EwQHdzIzWvfU/Io5rYtXgRrYN4PQmjjMKGFXseR3Qzczwrf9mZslI4dZ/B3QzMxzQzcySUa6v41wTDuhmZjhDNzNLhl9wYWaWiFy0/7eKOqCbmeEauplZMlxDNzNLhGvoZmaJqHfJxcwsDc7QzcwS4VEuZmaJcMnFzCwRLrmYmSXCGbqZWSKcoZuZJSIXuVJ3Ya05oJuZ4Vv/zcyS4Vv/zcwS4QzdzCwRHuViZpYIj3IxM0uEb/03M0uEa+hmZolwDd3MLBHO0M3MEuFx6GZmiXCGbmaWCI9yMTNLhC+KmpklwiUXM7NE+E5RM7NEOEM3M0tECjV0pfCtlDpJ1RExstT9sPLi3wtbXUWpO2BFqS51B6ws+ffCVuGAbmaWCAd0M7NEOKC3D66TWmP8e2Gr8EVRM7NEOEM3M0uEA3qZkzRE0kuS5ksaXur+WOlJGiWpVtJzpe6LlRcH9DImqQNwE3AgsD1wnKTtS9srKwN3AENK3QkrPw7o5W0PYH5ELIiIFcC9wGEl7pOVWEQ8CSwpdT+s/Digl7d+wKKC5ZqszczscxzQy5saafOwJDNrlAN6easBNi9Y7g8sLlFfzKzMOaCXtxnAQEkDJHUGjgUmlrhPZlamHNDLWETUAT8CHgXmAuMi4vnS9spKTdIY4GlgW0k1koaVuk9WHnynqJlZIpyhm5klwgHdzCwRDuhmZolwQDczS4QDuplZIhzQrdVJykmaLek5SfdJ2nAtjrWvpAez+UObe+KkpO6STvsC5/i5pPO+aB/NyoUDurWFjyJip4j4JrAC+GHhSuWt8e9eREyMiBHNbNIdWOOAbpYKB3Rra1OBrSV9TdJcSb8DngE2l1Qp6WlJz2SZfFdoeAb8i5KeAv5z5YEk/UDSjdl8H0njJf0rm/YCRgBbZX8d/Drb7ieSZkh6VtJlBce6KHvO/GPAtuvsX8OsDTmgW5uR1JH8s9znZE3bAndGxM7AMuBi4NsRsQswEzhH0vrArcAhwH8AmzZx+OuBv0XEjsAuwPPAcOCV7K+Dn0iqBAaSfwzxTsCukr4laVfyj1HYmfwXxu6t/NHNSqJjqTtgSdpA0uxsfipwO7AZsDAipmXtg8i/tON/JQF0Jn87+3bAqxExD0DSH4DqRs6xP3ACQETkgPcl9Vhtm8psmpUtdyUf4DcCxkfE8uwcfj6OJcEB3drCRxGxU2FDFrSXFTYBkyLiuNW224nWe0SwgF9ExC2rnePsVjyHWdlwycVKZRqwt6StASRtKGkb4EVggKStsu2Oa2L/ycCp2b4dJH0F+IB89r3So8BJBbX5fpJ6A08CR0jaQNJG5Ms7Zu2eA7qVRES8A/wAGCPpWfIBfruI+Jh8ieUv2UXRhU0c4ixgP0lzgH8C34iI98iXcJ6T9OuI+CtwD/B0tt39wEYR8QwwFpgN/JF8Wcis3fPTFs3MEuEM3cwsEQ7oZmaJcEA3M0uEA7qZWSIc0M3MEuGAbmaWCAd0M7NEOKCbmSXi/wNGXHi1Lxrv9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Bigrams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 7.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAEKCAYAAAB+AXB+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8lNXVwPHfyUxmkgxkX8jOElC2hCVsIiAiSkXBBVAUAVGxVV+rFgVfq+1ra7VudcFa9wUVqrYWRJAiRetW2VXQoogRQtjDDklIct4/ZjIOIcsgJpng+X4+88k897n3zh3QHO597nMeUVWMMcaYE0VYUw/AGGOM+TFZYDPGGHNCscBmjDHmhGKBzRhjzAnFApsxxpgTigU2Y4wxJxQLbMYYY04oFtiMMcacUCywGWOMOaE4m3oAjSExMVFbt27d1MMwxphmZfny5TtUNampx3GsfhKBrXXr1ixbtqyph2GMMc2KiHzX1GP4IWwpMgiHDh1i0KBBVFRUADBs2DBiY2M555xzgmpfWlrKRRddRE5ODn369KGgoKDGepMmTSI5OZkuXbrUeP7+++9HRNixYwcAc+fO5Te/+c2xfyFjjDmBWWALwrPPPssFF1yAw+EA4Oabb2bGjBlBt3/mmWeIi4tj3bp13HjjjUydOrXGehMnTuTtt9+u8dzGjRtZuHAhWVlZ/rLhw4czZ84cDh48eAzfxhhjTmwW2ILw8ssvM3LkSP/xkCFDaNmyZdDtZ8+ezYQJEwAYNWoUixYtoqanKgwcOJD4+Pga+7jxxhu59957ERF/mYhw2mmnMXfu3KDHYowxJzoLbPUoKytj/fr1HM/mk02bNpGZmQmA0+kkJiaGnTt3Bt1+zpw5pKenk5eXd9S5/Px83n///R88NmOMOdE0aGATkWEislZE1onItBrODxSRFSJSLiKjajgfLSKbRGR6QFlPEfnc1+cjEjiFaQA7duwgNjb2uPqoaXYW7LAPHjzIXXfdxZ133lnj+eTkZIqKio5rfMYYcyJpsMAmIg7gMeBnQCdgrIh0qlZtAzAReKWWbn4HvFet7HFgMtDe9xr2Iw25RpGRkZSUlBxXHxkZGWzcuBGA8vJy9uzZU+uSY3XffPMN3377LXl5ebRu3ZrCwkJ69OjBli1bACgpKSEyMvK4xmeMMSeShpyx9QbWqep6VS0DZgEjAyuoaoGqfgZUVm8sIj2BFOCfAWWpQLSqfqzeadCLwHkN+B2Ii4ujoqIiqOB266238sYbbxxVPmLECF544QUAXn/9dU4//fSgZ2xdu3Zl27ZtFBQUUFBQQEZGBitWrKBVq1YAfPXVV7XuojTGmJ+ihgxs6cDGgONCX1m9RCQMeAC4uYY+C39Inz/EGysLeek/33HmmWfywQcf+MsHDBjA6NGjWbRoERkZGSxYsACAzz//3B9wAl1xxRXs3LmTnJwcHnzwQe655x4AioqKOPvss/31xo4dS79+/Vi7di0ZGRk888wz9Y5x8eLFDB8+/Hi/qjHGnDAa8gbtmqYkR19sqtk1wDxV3VhtZhN0nyIyGe+S5RFb5I/FW59toXDXQe6+7joefPBBzjjjDIBaN2scPnyYfv36HVUeERHBa6+9dlR5Wloa8+bN8x/PnDmz3jEF3gO3detWDh06RNeuXettZ4wxPxUNGdgKgcyA4wwg2F0O/YABInIN0AJwich+4GFfP/X2qapPAk8C5OfnBxtQjxDvCeezwjK6d+/O4MGDqaio8N/LVpOqmVtj2bBhAw888ECjfqYxxoS6hgxsS4H2ItIG2ARcDFwSTENVvbTqvYhMBPJVdZrveJ+I9AU+AcYDj/7I4/aL97jZdbAMVWXSpEkN9TE/WK9evZp6CMYYE3Ia7BqbqpYD1wELgC+BV1V1jYjcKSIjAESkl4gUAqOBJ0RkTRBd/wJ4GlgHfAPMb5AvACR4XByuUPaVljfURxhjjPmRNWgSZFWdB8yrVnZHwPulHLm0WFMfzwPPBxwvAxplG2C8xwVA8f4yoiPCG+MjjTHGHCfLPFKHqsC280BZE4/EGGNMsCyw1aEqsO2ywGaMMc2GBbY6+JciLbAZY0yzYYGtDgktbCnSGGOaGwtsdYgMd+B2hlF8oLSph2KMMSZIFtjqICIkeFwUHzjc1EMxxhgTJAts9Yhv4bIZmzHGNCMW2OoR73Hb5hFjjGlGLLDVIz4qnOKDFtiMMaa5sMBWj3iPm+L9FtiMMaa5sMBWj4QWLg6UVVByuKKph2KMMSYIFtjqERdlN2kbY0xzYoGtHpZ9xBhjmhcLbPWoyj5igc0YY5oHC2z1sBmbMcY0Lw0a2ERkmIisFZF1IjKthvMDRWSFiJSLyKiA8mwRWS4iq0RkjYj8PODcu74+V/leyQ35HeKjLF+kMcY0Jw32oFERcQCPAUOBQmCpiMxR1S8Cqm0AJgJTqjXfDJyiqqUi0gJY7Wtb5Dt/qe+Bow0uJjIcR5jYo2uMMaaZaMgnaPcG1qnqegARmQWMBPyBTVULfOcqAxuqamAUcdOES6ZhYUJcVLjN2IwxpployICRDmwMOC70lQVFRDJF5DNfH38MmK0BPOdbhrxdROTHGW7t4j2WL9IYY5qLhgxsNQUcDbaxqm5U1VwgB5ggIim+U5eqaldggO91WY0fLjJZRJaJyLLt27cf49CPFBflYpdl+DfGmGahIQNbIZAZcJwBFNVSt1a+mdoavEEMVd3k+7kPeAXvkmdN7Z5U1XxVzU9KSjrWjz1CQgsXO23GZowxzUJDBralQHsRaSMiLuBiYE4wDUUkQ0Qife/jgP7AWhFxikiirzwcOAdY3SCjD+BdirRrbMYY0xw0WGBT1XLgOmAB8CXwqqquEZE7RWQEgIj0EpFCYDTwhIis8TXvCHwiIp8C7wH3q+rneDeSLPBde1sFbAKeaqjvUCU+ysXuQ4epqAx6JdUYY0wTachdkajqPGBetbI7At4vxbtEWb3dQiC3hvIDQM8ff6R1i/e4UIXdB8tIaOFu7I83xhhzDCzzSBDifcHMliONMSb0WWALQoLHso8YY0xzYYEtCPboGmOMaT4ssAXBMvwbY0zzYYEtCDZjM8aY5sMCWxBczjBaRjgtsBljTDNggS1I8R6XbR4xxphmwAJbkOI9Lnt0jTHGNAMW2IKUYDM2Y4xpFiywBckeXWOMMc2DBbYgxXm8j65RtXyRxhgTyiywBSnB46KsopL9peVNPRRjjDF1sMAWpHiP5Ys0xpjmwAJbkOI94YDlizTGmFBngS1IVTM22/JvjDGhzQJbkCzDvzHGNA91BjYRcYjIfT+0cxEZJiJrRWSdiEyr4fxAEVkhIuUiMiqgPFtElovIKhFZIyI/DzjXU0Q+9/X5iIjIDx3fsYj3WL5IY4xpDuoMbKpaAfT8IcFDRBzAY8DPgE7AWBHpVK3aBmAi8Eq18s3AKaraDegDTBORNN+5x4HJQHvfa9ixju2HiHI5cDnDLLAZY0yIcwZRZyUwW0ReAw5UFarq3+tp1xtYp6rrAURkFjAS+CKgjwLfucrAhqoaGD3c+AKwiKQC0ar6se/4ReA8YH4Q3+O4iAgJHpcFNmOMCXHBBLZ4YCdwekCZAvUFtnRgY8BxId7ZV1BEJBN4C8gBblbVIhHJ9/UT2Gd6sH0er3gLbMYYE/LqDWyqevkP7Lum5cug03ao6kYg17cE+Q8Ref1Y+hSRyXiXLMnKygr2Y+tkGf6NMSb01bsrUkQyROQNEdkmIltF5G8ikhFE34VAZsBxBlB0rANU1SJgDTDA12fgZ9fap6o+qar5qpqflJR0rB9bI8vwb4wxoS+Y7f7PAXOANLzLfm/6yuqzFGgvIm1ExAVc7OunXr5gGul7Hwf0B9aq6mZgn4j09W1oGQ/MDqbPH4MtRRpjTOgLJrAlqepzqlruez0P1DsFUtVy4DpgAfAl8KqqrhGRO0VkBICI9BKRQmA08ISIrPE17wh8IiKfAu8B96vq575zvwCeBtYB39AIG0eqJHhc7C8tp7S8orE+0hhjzDEKZvPIDhEZB8z0HY/Fu5mkXqo6D5hXreyOgPdLOXJpsap8IZBbS5/LgC7BfP6PLTBfZGpMZFMMwRhjTD2CmbFNAsYAW/DeXzbKV/aTU5Uv0pYjjTEmdNU5Y/PdZH2hqo5opPGENMvwb4wxoS+YzCMjG2ksIc/SahljTOgL5hrbhyIyHfgrR2YeWdFgowpRVYFt534LbMYYE6qCCWyn+H7eGVCmHJmJ5CchNjKcMIFdBy2wGWNMqKrvGlsY8LiqvtpI4wlpYWFCXJRlHzHGmFBW3zW2Srz3ohmfeI+LYluKNMaYkBXMdv+FIjJFRDJFJL7q1eAjC1FxHhfFthRpjDEhK5hrbFX3rF0bUKZA2x9/OKEvwePi6237m3oYxhhjahFMdv82jTGQ5sLyRRpjTGirdSlSRG4JeD+62rk/NOSgQlmCx8Wug2VUVAb9BB5jjDGNqK5rbBcHvL+12rlhDTCWZiHO40IV9hw63NRDMcYYU4O6ApvU8r6m45+M77OPlDbxSIwxxtSkrsCmtbyv6fgnI8GXL9KyjxhjTGiqa/NInojsxTs7i/S9x3cc0eAjC1FxluHfGGNCWq2BTVUdjTmQ5qJqxmb3shljTGgK5gbtH0xEhonIWhFZJyLTajg/UERWiEi5iIwKKO8mIh+LyBoR+UxELgo497yIfCsiq3yvbg35Harzz9hsKdIYY0JSMDdo/yC+Z7k9BgwFCoGlIjJHVb8IqLYBmAhMqdb8IDBeVb8WkTRguYgsUNXdvvM3q+rrDTX2uridDlq6nZYv0hhjQlSDBTagN7BOVdcDiMgsvM928wc2VS3wnasMbKiqXwW8LxKRbUASsJsQEGc3aRtjTMhqyKXIdGBjwHGhr+yYiEhvwAV8E1B8l2+J8k8i4q6l3WQRWSYiy7Zv336sH1uneN9N2sYYY0JPvYFNRC4Qka9FZI+I7BWRfQE7JOtsWkPZMd0mICKpwAzgct+TBsB7s/jJQC8gHphaU1tVfVJV81U1Pykp6Vg+tl4JHpdt9zfGmBAVzIztXmCEqsaoarSqtlTV6CDaFQKZAccZQFGwAxORaOAt4Neq+p+qclXdrF6lwHN4lzwbleWLNMaY0BVMYNuqql/+gL6XAu1FpI2IuPCm6JoTTENf/TeAF1X1tWrnUn0/BTgPWP0DxnZc4n2PrlH9yd6nbowxISuYzSPLROSvwD8Afx4pVf17XY1UtVxErgMWAA7gWVVdIyJ3AstUdY6I9MIbwOKAc0Xk/1S1MzAGGAgkiMhEX5cTVXUV8LKIJOFd6lwF/PwYvu+PIt7joqy8kgNlFbRwN+T+G2OMMccqmN/K0Xi3358ZUKZAnYENQFXnAfOqld0R8H4p3iXK6u1eAl6qpc/Tgxhzg/Lni9xfZoHNGGNCTDDPY7u8MQbSnCS08Aa2nQdKyUqIauLRGGOMCRTMrsgMEXlDRLaJyFYR+ZuIHDXL+imJi/IGNtvyb4wxoSeYzSPP4d30kYb3PrQ3fWU/WZbh3xhjQlcwgS1JVZ9T1XLf63m8WUB+suJbVD2TzQKbMcaEmmAC2w4RGSciDt9rHLCzoQcWyjwuBy5HmAU2Y4wJQcEEtkl4t99vATYDo3xlP1kiYjdpG2NMiApmV+QGYEQjjKVZscBmjDGhqdbAJiK3qOq9IvIoNeR4VNXrG3RkIS6hhcseXWOMMSGorhlbVRqtZY0xkOYmLsrFhuKDTT0MY4wx1dQa2FT1Td/bgzXkaxzdoKNqBuI9LnuKtjHGhKBgNo/cGmTZT0qCx8W+0nJKyyuaeijGGGMC1HWN7WfA2UC6iDwScCoaKG/ogYW6qnvZdh04TKsYRxOPxhhjTJW6rrEV4b2+NgJYHlC+D7ixIQfVHMRHfX+TdquYiCYejTHGmCp1XWP7FPhURF5R1cONOKZmwZ/h33ZGGmNMSAnmmSutReRuoBPgn5qoatsGG1UzEJjh3xhjTOgINgny43ivqw0GXgRmBNO5iAwTkbUisk5EptVwfqCIrBCRchEZFVDeTUQ+FpE1IvKZiFwUcK6NiHwiIl+LyF99T9tudPG+RMg2YzPGmNASTGCLVNVFgKjqd6r6W6Deh32KiAN4DPgZ3tneWBHpVK3aBmAi8Eq18oPAeN/TtIcBD4lIrO/cH4E/qWp7YBdwRRDf4UcXExmOCOyywGaMMSElmMBWIiJhwNcicp2InA8kB9GuN7BOVderahkwCxgZWEFVC1T1M6CyWvlXqvq1730RsA1IEhHBG1Rf91V9ATgviLH86BxhQlyUZR8xxphQE0xguwGIAq4HegLjgAlBtEsHNgYcF/rKjomI9AZcwDdAArBbVatuN/hBff5YLF+kMcaEnmCSIC/1vd0PXH4MfUtN3R1De0QkFe/1vAmqWumbsQXVp4hMBiYDZGVlHcvHBi0+ygKbMcaEmnpnbCKyMOD6FiISJyILgui7EMgMOM7Ae29cUEQkGngL+LWq/sdXvAOIFZGqgFxrn6r6pKrmq2p+UlLDPBfVZmzGGBN6glmKTFTV3VUHqrqL4K6xLQXa+3YxuoCLgTnBDMpX/w3gxcA8laqqwGK8z4QD75Lo7GD6bAjxLSywGWNMqAkmsFWKiH8tT0SyCWJJ0Xcd7DpgAd4nBbyqqmtE5E4RGeHrq5eIFAKjgSdEZI2v+RhgIDBRRFb5Xt1856YCN4nIOrzX3J4J6ps2gASPi10Hy6isPKYVVmOMMQ0omBu0bwM+EJH3fMcD8V27qo+qzgPmVSu7I+D9UrzLidXbvQS8VEuf6/HuuGxycVEuKhX2HDpMnKdJbqczxhhTTTCbR94WkR5AX7wbQm5U1R0NPrJm4PvsI2UW2IwxJkTUuhQpIif7fvYAsvBu0tgEZPnKfvIsX6QxxoSeumZsN+FdcnyghnNKENlHTnTfBzbLF2mMMaGirsC20PfzCt91LVPN94HNHn5gjDGhoq5dkVVPyX69jjo/aTZjM8aY0FPXjG2niCwG2ojIUfefqeqIhhtW8+B2Omjhdlq+SGOMCSF1BbbhQA+8Ka1qus5mgDhPuGX4N8aYEFLXE7TLgP+IyCmqur0Rx9SsxHvcNmMzxpgQUmtgE5GHVPUG4FkROSq1hi1FeiV4XGzdW9LUwzDGGONT11Jk1VOy72+MgTRX8R4XX27e29TDMMYY41PXUuRy38+qVFqISByQ6Xs4qOH7DP+qSs1P1THGGNOYgnlszbsiEi0i8cCnwHMi8mDDD615iPe4KC2v5GBZRVMPxRhjDMFl949R1b3ABcBzqtoTOKNhh9V8WFotY4wJLcEENqfvSdZjgLkNPJ5mJ8HzfSJkY4wxTS+YwHYn3meqrVPVpSLSFvi6YYfVfFRl9bd72YwxJjQE89ia14DAp1ivBy5syEE1JzZjM8aY0BLM5pF7fZtHwkVkkYjsEJFxwXQuIsNEZK2IrBORaTWcHygiK0SkXERGVTv3tojsFpG51cqfF5Fva3iydpOwfJHGGBNaglmKPNO3eeQcoBDoANxcXyMRcQCPAT8DOgFjRaRTtWobgInAKzV0cR9wWS3d36yq3XyvVUF8hwbTwu3E5QizGZsxxoSIYAJbuO/n2cBMVS0Osu/eeK/Lrfel55oFjAysoKoFvnviKqs3VtVFwL4gP6vJiIjlizTGmBASTGB7U0T+C+QDi0QkCQgmh1Q6sDHguNBX9mO4S0Q+E5E/iYi7pgoiMllElonIsu3bGzbVZbzHbdv9jTEmRNQb2FR1GtAPyFfVw8ABqs28alFTGo6jck7+ALcCJwO9gHhgak2VVPVJVc1X1fykpKQf4WNrl+Bx2VKkMcaEiHp3RfqkA0NFJCKg7MV62hQCmQHHGUDRMYytRqq62fe2VESeA6Ycb5/HK87jonDXwaYehjHGGIIIbCLyG+A0vBtA5uHdDPIB9Qe2pUB7EWkDbAIuBi45nsH6xpOqqpvFm5jxPGD18fZ5vGzGZowxoSOYa2yjgCHAFlW9HMgDaryuFUhVy4Hr8N7c/SXwqqquEZE7RWQEgIj0EpFCYDTwhIisqWovIu/jvX9uiIgUishZvlMvi8jnwOdAIvD7IL9rg4n3uNhXUk5Z+VF7YIwxxjSyYJYiD6lqpe9es2hgG9A2mM5VdR7eWV5g2R0B75fiXaKsqe2AWspPD+azG1PVvWy7DpaREh1RT21jjDENKZgZ2zIRiQWeApYDK4AlDTqqZsYSIRtjTOgIJqXWNb63fxGRt4Foex7bkSywGWNM6Kg1sIlIj7rOqeqKhhlS82P5Io0xJnTUNWN7oI5zCoTcta6m4p+x7bd8kcYY09RqDWyqOrgxB9KcxUa5EIHig4ebeijGGPOTF0x2/2t9m0eqjuNE5Jq62vzUOMKE2Mhwy/BvjDEhIJhdkVep6u6qA1XdBVzVcENqnuI9Lts8YowxISCYwBbmy/IB+B9H42q4ITVPCZYI2RhjQkIwgW0B8KqIDBGR04GZwNsNO6zmJ84TboHNGGNCQDCZR6YCk4Ff4M3Y/0/g6YYcVHMU73Gz/LtdTT0MY4z5yQvmBu1K4C94b9COBzJUtaLBR9bMJHhc7Dp4mMpKJSyspif2GGOMaQzB7Ip8V0SifUFtFfCciDzY8ENrXuI8Lioqlb0ltuXfGGOaUjDX2GJUdS9wAfCcqvYEzmjYYTU/ln3EGGNCQzCBzSkiqcAYYG4Dj6fZsnyRxhgTGoIJbHfi3Rm5TlWXikhb4OuGHVbzUxXYdu63wGaMMU2p3sCmqq+pam5Vln9VXa+qFwbTuYgME5G1IrJORKbVcH6giKzwPettVLVzb4vIbhGZW628jYh8IiJfi8hfRSQk7qkLfCabMcaYplNXdv9bVPVeEXkUb9LjI6jq9XV17LuR+zFgKFAILBWROar6RUC1DcBEYEoNXdwHRAFXVyv/I/AnVZ0lIn8BrgAer2ssjcGWIo0xJjTUtd3/S9/PZT+w7954ly/XA4jILGAk4A9sqlrgO1dZvbGqLhKR0wLLfBlQTgcu8RW9APyWEAhsEeEOPC6HLUUaY0wTqyu7/5u+ny/8wL7TgY0Bx4VAnx/YV5UEYLeqlgf0mX6cff5o4lu4bCnSGGOaWF1LkXPqaqiqI+rpu6a7lI9a0jxGQfcpIpPxZkwhKyvrOD82OPFRLtvub4wxTayupch+eGdcM4FPqDmo1KUQyAw4zgCKjrGP6nYAsSLi9M3aau1TVZ8EngTIz88/3oAalHiPi+32sFFjjGlSde2KbAX8L9AFeBjvJpAdqvqeqr4XRN9Lgfa+XYwu4GKgzllgfVRVgcVA1Q7KCcDs4+nzxxTvcVNs19iMMaZJ1RrYVLVCVd9W1QlAX2Ad8K6I/E8wHftmVNfhvQfuS+BVVV0jIneKyAgAEeklIoXAaOAJEVlT1V5E3gdeA4aISKGInOU7NRW4SUTW4b3m9swxfucGE+8Jp9h3je3QoUMMGjSIiooKvvvuO3r27Em3bt3o3Lkzf/nLX+rtq7i4mKFDh9K+fXuGDh3Krl01J1geNmwYsbGxnHPOOUeUf/vtt/Tp04f27dtz0UUXUVbmHdeGDRsYPHgw3bt3Jzc3l3nz5gHw+eefM3HixOP49sYYEyJUtdYX4MabSus1vDOw24H0utqE4qtnz57aGP68eJ1mT52rB0oP6/Tp0/Whhx5SVdXS0lItKSlRVdV9+/Zpdna2btq0qc6+br75Zr377rtVVfXuu+/WW265pcZ677zzjs6ZM0eHDx9+RPno0aN15syZqqp69dVX65///GdVVb3qqqv879esWaPZ2dn+NkOGDNHvvvvuGL+1MeZEBSzTEPgdfqyvWmdsIvIC8BHQA/g/Ve2lqr9T1U0NHm2bqYSA7CMvv/wyI0eOBMDlcuF2uwEoLS2lsvKouxuOMnv2bCZMmADAhAkT+Mc//lFjvSFDhtCyZcsjylSVf/3rX4waNeqo9iLC3r17AdizZw9paWn+dueeey6zZs0K+vsaY0woqusa22VAB+CXwEcistf32iciextneM1L1U3aW3ftZ/369bRu3dp/buPGjeTm5pKZmcnUqVOPCCg12bp1K6mpqQCkpqaybdu2oMexc+dOYmNjcTq9e4MyMjLYtMn775Hf/va3vPTSS2RkZHD22Wfz6KOP+tvl5+fz/vvvB/05xhgTiuq6xhamqi19r+iAV0tVjW7MQTYXcb7A9u2mLcTGxh5xLjMzk88++4x169bxwgsvsHXr1gYbh3cF4Ujee9th5syZTJw4kcLCQubNm8dll13mn0EmJydTVHS8G1eNMaZpBZME2QSpainyYEUYJSUlNdZJS0ujc+fO9c6MUlJS2Lx5MwCbN28mOTk56HEkJiaye/duysu997EXFhb6Z4jPPPMMY8aMAaBfv36UlJSwY8cOAEpKSoiMjAz6c4wx3wvcMFZl7969pKenc9111/nLhg0bRl5eHp07d+bKK69k4MCBdbZRVTIyMnC5XERGRnLSSSfVuIJz2223kZmZSYsWLQC4++67ycnJoUOHDgwaNIicnBz69OlDQUEBAC+//DIZGRm43W5EBBFh1apVgPcSx9VXXw3QRUQ+E5EeACIyWERWBbxKROQ837lZItL+R/wj/cEssP2I4lt4A9t+IqioqPAHt8LCQg4dOgTArl27+PDDDznppJMAGD9+PEuWLDmqrxEjRvDCC96kLy+88IL/el0wRITBgwfz+uuvH9U+KyuLRYsWAfDll19SUlJCUlISAF999RVdunQ55u9tjIFnn32WCy64AIfD4S+7/fbbGTRo0BH1Xn31VT799FNWr17NsmXLaNOmTZ1t5s+fz8GDB/nwww9ZvHgxsbGxNf5D99xzz/X/Lvniiy+YNWsWa9asYdy4caxatYq1a9dy4403MnXqVAAuvfRS3nzzTdauXUtaWhpZWVl069YNgLy8PBYvXgywGm+ii8cBVHWxqnZT1W540xseBP7pG8LjwC3H8Uf4o7HA9iOKjginS3o0b6zcxNChQ/nggw8AbwDp06cPeXl5DBo0iClTptC1a1cAPvvsM/+1tEDTpk1j4cKFtG/fnoULFzJtmvfhCMuWLePKK6/01xswYACjR49m0aIG+l7cAAAgAElEQVRFZGRksGDBAgD++Mc/8uCDD5KTk8POnTu54oorAHjggQd46qmnyMvLY+zYsTz//PP+ZcrFixczfPjwhvsDMuYEFrhhDGD58uVs3bqVM88884h60dHeKznl5eVs2LCB/Pz8OtvMnj2bVq1aISL07duX3bt3+1dzAvXt29f/u2T27NlcfPHFuN1uPvzwQ04++WSWLFnCqFGjWLRokf9yRffu3WndujUHDhzgggsu8PdVXFzs/8e4qv4Hb2KM6r+oRgHzVfWg7/h94AwRqSvxR+No6m2ZjfFqrO3+qqpvrCjU7Klz9Ym/v6Pjxo2rs+6ePXt01KhRjTSyupWUlGifPn308OHDTT0UY5qd0tJSTUlJ8R9XVFTooEGDdMOGDfrcc8/ptddee0T9M888U2NiYjQiIkLLy8vrbDN8+HDt1q2bdunSRfPy8rRNmza6ZMmSWsfi8Xj02muv1RkzZqiqaufOnfWiiy7S1157TVVV27Ztq9u3bz+ijdPp1H//+9/+4+HDh2t6eroCK9UbBBcB+RrwexX4F3BOtbKFQE9thN/rdb1sxvYjO7trKq2iI/j3zhYMHjz4iLXz6qKjo3nttdcacXS127BhA/fcc49/J6UxJng7duw4YsPYn//8Z84++2wyMzNrrL9gwQJWrFiB0+nkX//6V51tVJU77riDzz//nPfff5/du3f7EyvURgM2kFW9r1qZqf7+k08+QUTo2LHjEW3i4uIAAp93qQHtU4GueBNwBNoG1L3luzE0dWRtjFdjzthUv79R+4uiPY36ucY0tYMHD+rAgQO1vLxcCwoKtEePHpqXl6edOnXSxx9/3F/vrLPO0tzcXO3UqZNeccUVOmDAgDrbVFZWanp6uoaHh6vb7dbIyEgtKio6qv6wYcM0IyNDo6KiNC8vT1NTU9XlcqmIqMvl0rZt22peXp5/9tOpUyc966yz1O12K6AtW7ZUl8ulLpdLExMTNTU1Vdu2batdu3bV5cuX6549ezQmJkZbtmypgObk5GhUVJSGhYUpoGlpaZqVlaUxMTGakpKi2dnZmpCQoG63W51OpwLavn17vfnmm7V169YKaEZGhrZr167ONq1atdJvv/1WVVXdbrdGREQcNa60tDQdP368OhwOTUhIUIfDoR07dtQWLVoooMOGDdPDhw9rQkKC7t69W9PS0vTaa6/VG264QWNjY/XJJ5/UVq1aaXh4uLpcLk1JSVHgc/UGxrVAqu/9dcB2vIHud3izUq0FzvK93wR8BvQIqF/gq/8V3keXtcabf/gr4LvA+r42WXiv3X1ZVd9XPgtor/X8zm/yoNMYr8YObLsPlOnJv56vv3p1VaN+rjFNLdiMO3v2eP/RV1lZqXl5eTp+/Pg627z11lsaFxenS5Ys0V/96lealZVVY/1WrVrpihUr1OPx6Jo1azQ3N1dLSko0Ojpaw8PDtby8XGfMmKEXXnihqqru2rVLnU6nvvjii5qSkqJOp1MLCgp0/fr1GhYWptHR0VpZWakff/yx9u7dW6+//nodNmyYpqSkaEpKin7wwQfatm1b/eKLLxRQl8ull19+ub++qupzzz2nSUlJ+uKLL2pmZqbOnDlTu3Tpovv37/cHrsOHD+tNN93kzwQU2CYxMVFPOeUU3b9/v77//vsaHh6ukZGRWllZqXfffbempqbq9ddfr2PHjtW0tDSNiIjQ1atXa5cuXbS4uFjPPvtsBfTss8/WmTNn6ujRo/31r7nmGk1PT9dWrVppcnKydurUSUtKSrRfv37qC0TL8KZUXKLfB53uwEpfcFuNN0NVG2AzsA9I9bX5JKD+f4AtQCLQAu9DpM/2lT8VWN/X5l1gqO99CyDK934Q8JTW8zvfliIbQExUOGPyM5i9ahPb9ta87d+YE1GwGXdq20BRW5vADRQfffQRDoeDzZs3H1U/PDyclJQUf5uLL76YDRs2sG/fPjp27MiSJUu4+OKLeffdd1FV3n77bVwuF0OGDOHAgQNkZGSQnZ1NmzZtEBFKSkr8mza2bNlCQUEBp556KiJCREQE8+fP58orr6Rjx46ICOnp6cyfP5/8/Hz/Jo9NmzbhcDi47LLLUFVuuukmduzYQe/evWnRogVhYWF8/PHHlJaWUlpaelSbiIgI1q5dS3x8PGeccQZhYWG43W4qKip45JFHOHToEAUFBWzcuJEtW7ZQWlrKWWedRVJSEvn5+SxatIjw8HAWLFjAAw88wDfffENBQQFnnnkm77//Ptu2bWPbtm3s3LmTqKgo3G43Z511VtVOza54A881AX/Nu4BkvE+HeUNVS1X1W6AS2KOqm/XIDSelQDlQAqCq+9W74WQk8AhwBt4AGisiqSLSCXCq6sJq9SHIDSoW2BrI5f3bUF6pvPjxd009FGMaRVlZ2TFl3Kn65Xvo0CGuueaaOtts2rQJt9vNhAkTWLp0KeC9jaauz9i0aROZmZnMmDEDgI4dO7Jp0yacTicej4cuXbowfvx4OnfuzOWXX86BAwf8wevgwYNUVFSgquzcuZODpYfZvWcPA869iI8+X0eZhrFp81bue+hR/vTSHPre9U9UoXDHXvZoJN1/+RRbyj0M/d0bvPBlBSUx2cR3HsCmzVvYXekmqv94Ui5/jPDELAhzcM/zs/lw5ZekZLXl/VVryW7XgZ49e3LBBRewdetWLrroIg4ePOhPh5eTk8Onn37K4MGDqays5KqrrqJ79+5kZmZy3nnnkZSURM+ePVm9ejXh4eFcf/31DB48mD//+c+0bNmS6dOnAzBw4EDKysrYsWMHERERXHLJJZSXlzNjxgzi4+MBClW1q6ouq/r7UdUCVU0HHHiXHatEAYH3LlU9CLoDsBtIAv4lIveJiMN3bgPe5cu86vVF5O8isjKgPqpaGVC/VhbYGkjrRA9ndkrhpU++41BZ7RtIjPkhAm8GXrVqFf369aNz587k5uby17/+1V9vwIABdOvWjW7dupGamkpiYmK9baZOnUpERAQul4uYmBhKS0uPekLFVVddRefOnQkLC2PSpEkkJCTgdrspLi5m9OjR5OTkcNJJJ/HFF1/4M+7ccMMN/puBk5OT+eabb4iLi6O8vJxzzjnnqDa33XYb1113HSLCe++9x969e1m4cCFt2rRhw4YNnH766eTm5rJ9+3b/Z1x33XV07tyZQ4cO8dZbbwHebDsA+/bt8/7ZlVUgYQ5e/Ps8Lv/FL1m16lMuvPZ/cUdGER7hoc/AIbTrmg8iVCCc+pu/kXHO/3BQInngg+38+6ttFG/fBm4PiZ1OofLgbmIKP0LChJi4eOJiY+jQ8jCxUeH0zI6nU4qHgxvWMPqaW2kZl0R4RSktdn1Nx9RowtwexOnigzUF7BpwE2u37ueXf/2UG2etYP47iyk6aRThLROY//FnXDTlbn7/t/+gEkbx4XDufv0jNruzkMhonl2xi3+uKWJj4SYOdR9LxsQ/8eI/l9B1zE2UlpYxduxYkpOTefrpp2vcoBIXF8egQYOYPn06AwYMoG3btlXJGo7lWZaVeJcoAynemd0AvDO9oUBbYCLfP+MzcMNJYP0pQK+A+tRQv2b1rVWeCK/GvsZWZcm3OzV76lyd8XFBk3y+OXEFXstau3atfvXVV6qqumnTJm3VqpXu2rXrqDZ5eXl66aWX1tlm+/bt6nA49JNPPtF7771XIyMj9eGHHz7qWlZaWpq+9957OmjQIH3kkUc0MzNTo6KitFWrVupyuXTz5s26fv16bdu2rX87e2RkpN51112an5+v7du315KSEl25cqWKiDqdTi0pKTmizYoVKzQqKkoTEhJ0/Pjx+sorr2hxcbEmJydreHi4pqamHnEt62BpuTocTr3twafUHRmlp427Xk86fbRGJqSpOJzqiIrR9Mvu16ybZ2tYZLRm3fKmJp57szpjUzVx5DQVt0ddGZ019rTLtfO01xURdboi9erH5mrP08/VMIdDk1qlacvoaHU4HBodHa233Xabjh49Wq+55hp1Op3atWtXHTRokC5cuFA7dOigRUVF+vHHH+ugQYNUVTU7O1tHjBihffr0UVXVyZMna3R0tD744J908+5Dmt02R2csWqm3/uVv2rpLL73m5eUaFd9K24y4XqN7DNfMG2apuCI1Mq2Dtp/4R03MG6IS5lB3TJI6IzwqYWHa8azLdPwzn+igyf+nrXuepk6nU1VVL7jgAj399NM1MzPTv0GlZcuWOnXqVFVV/cMf/qB/+MMfVFX1iSeeqNrc8qXW8nvVF6h+F3C8Ebgj4Hgt319vexfvBpJEvHmIHwOeAMYCf8O7HHlE/YB+LgMeCzj+G3BGbeNSu8bWsPKz48jLiOHZD76lsrJRHuJtfiICr2V16NCB9u29mYzS0tJITk5m+/btR9Tft28fq1ev9medqK3N3/72NzweD7179+aNN97glFNO4dlnnz3qWpbD4SAnJwfwpoD62c9+hohw6NAhunbtyuLFiwkPD6dNmzYsWbKEXbt2cfjwYU499VS2b9/OKaecgtvt9me6cblc7Nq1i5deegmn08mSJUto3bo1ZWVlOJ1OzjjjDJ5++mnE5WH3nr2Iw0lJOTz27wI+/e83pPQ7jw63vE5FRQXPri6lrLySzTGd+XbJQk465SzCXW4qDu7hmtFnkPHFKyQnJvDwxd158NpRyMGdPDp5KEmxLXHsXM+T1w3npYvagSpUHubRq85k1NBTSE5KYtvmTTz0pz8RGxtLTEwMw4YN45133qF9+/Y4HA7WrVvH9u3bKSkpobKykkceeYRevXqxa9cu/9/J4cOHKSwspLS0lN69e7Nv3z5GjbqQgi9XkZwQx7jTu/G7K0cSHVbKb4dmktTSTZ+obdx1+TC+e3AMCdEewvZs4t27xvHmX35Pfs8elOzexhOPPUJ8XBzvzXiAFyb1pk3Zt7SPDycqKoqSkhK++uorXnrpJR5++GHGjh3L/fffz/jx47nnnnsA6N+/P7NmzWLLli089NBDVSn5DojI+SJydw3/GR4EzhcRt4i0wTvT6ideffFdb8P7yLM4vl8hPB3vTsc5wHi8S4/u6vVFJKla/SodgDXUwW5aakAiwhUD2nL9zJUsXruNIR1TmnpI5gRQ07WsKkuWLKGsrIx27dodUf7aa6/hdDr9GW9qa/OPf/wDEeHjjz9m/fr1tGnTxp+we+PGjQwfPpx169Zx3333kZqaSnmlsvbbQsJbxFFRqcTHJ3Oo0sk7y75k/T7hk6XLOO/CMUS1aEmkpyWTrr6WjYWFvP73N3jvoyWUl5cT5nTi8sRw0yOzWLpwEQXfbWTouRfgcHvAFcW2HTu54le/pnz3NuKiPVBZTnhyW8pS2rFo/lxK9+5k7+p3id/6X0DZ/uKNaPlhdsz6X8LKS9ixciER4Q7Cwzw8fs05lJSUsGfPHv730qE4nU56dO/OZSPPory8HBHh4jFjcDqd5OTksH//fk4++WQKCwsZO3YsAIsWLeLAgQPs3LmTwYMH43Q6mTJlChUVFbjdbtauXcutt97K4cOHAXA4HPTv35+0tDTKy8vZtm0bHTt2pFOnTogI0dHRnHbaaURFRREdHU1RURFpaWlHtYmMjEREGDp0KPPnz2fAgAFERUXx3HPPARAWFkb//v0ZMmRI1UoVX3/9NWeccQZvvvkmkZGRpKam8s033/g37wR6/PHH2bx5M1lZWSQkJPDss89y9tlnA7TDG8AeVdUiEbkeb+qsFKAl3t2Rm4FJwDl4r4ElAWN8XV+Ld+kw0VfvW+Bq4DBwITAYuAe4HLwPuRaRKcAi8d5wtxzvBhZEJAU45AuAtatrOne8L2AY3unlOmBaDecHAivw7pgZVe3cBOBr32tCQPm7vj5X+V7J9Y2jqZYiVVXLyiu03x/e0Yue+KjJxmBOLJs2bdKTTjrpqPKioiLt0KGDfvzxx0edO+200zQtLa3eNvfee69eddVV2qtXL3W7IzS/b39NSG6lL//nO31gwX91yqur9Pz73tSWmSdrhxtfUXdmF23Z41yNGTBOJTxCI086VR0tkzRu8CTNnjpXPV2HauJ5t2r21Lmafs0Lmj11roZFxapExWjKJfdo9tS56ohOVoloqcln/VxjTuqjrXqepT2v+J2OffJjveSheRqdlKrXTJ+tqW1O0imPzNRO3Xtr/0Gn681Tp+kjjzyiKSkpunDhQv+fjarqN998o9nZ2bpu3TpVVV2xYoW2bt1a58yZo5deeqlu27at3j/niy++WM844wxV1aDbPPjgg/r0008H3SawfrBuuukm7du3b9D1V6xYoT179jymcVXBu1vxJSBJGyZG3Ahc8WPXb8ig5gC+wXvhzwV8CnSqVqc1kAu8GBjYgHhgve9nnO99nH4f2PKPZSxNGdhUVZ94z3vD9ueFu5t0HKZxBN6kvHLlSu3bt6926tRJu3btqrNmzfLXmzRpkubm5mrnzp01MTFRd+/eXWf9CRMmaFxcnIaHhyugjz32mP/cnj17tHv37vrqq6/qo48+qu3atVNAf37r7zWuVbqCaESLGO165kUanZyhcRk5OmTK4xqb2UHbDh6jnpTWioiGuSM1Ijlb2//qVXVEJytOl+JwqjM+XVPG/J/2uesdHTn9Az1pwLk6+ub71R0ZpWnZbfX8sRM0KsqjHTp21pTUNL13+lP63817tf+g03Xmmwv1v5v3+l/pWdnaJa+H3nzH73XDzgPatl2Opqen6xNPPKGq3nRTH330/T8Es7Ozdfv27fqb3/xG77vvPp08ebJOnjxZb7jhBn3qqaf817KqmzBhgj+NlKpqjx499O233w767/HJJ5/UZ555xn+NMBjPPvvsMaWlO9b6VW2efPLJYxrX5Zdf7r9Geixo4Cdo+2Zpzh+7fkMOuB+wIOD4VuDWWuo+Xy2wjQWeCDh+AhirzTSw7TlUpp1un683zFrZpOMwjSPYjR1VNylPnz5dBw0apHfffXed9UeOHKlZWVlaUlKiqamp2rp1ay0vL9fS0lI9/fTT9Xf33KdvfVakVz74qp7665ka1jJRHdFJmjjkSo3repoiojFtu+mZD76rXcdMUQlzaPcxv9SRv5+lo+95TT1xSZrRKV/DI6L01pkfacvYeAV06Lnn6+wF/9LU1FQdN26cFhcXa/v27XXKlCmanJysvXv31tatW6vH49Hp06drWFiYFhUV6eOPP64xMTFaXl6u+/fv171796qqavfu3TUiIkJnz56t69ev14yMDG3RooW+9957un79em3Tps0RbbKzs7WgoED79eun8+fP17lz5+qwYcN0+PDh+tRTT2mvXr1UVbW4uNj/y3v79u2ak5Oja9as8f+9dOnSpcYAaGrX0IGtoV4NGdhGAU8HHF8GTK+lbvXANgX4dcDx7cAU/T6wfe5bhrwdkPrG0tSBTVX1/+as0Xa3vqWbdx9q6qGYBtavXz9/+qPqcnNz/YErsP4ll1yi99xzT531e/TooWPHjlVV32yve3e978U5OvKXf1AJc2h4chsNT26j7pS2Ovw3M9QTHafDL7xIBw4cpPPnz9eYmBgdOXKkqqr+z//8jwLaqVMnzcvL0w4dOmhcXJzecccdmpmZ6Z8VZmdna+fOnfWf//ynOhwO9Xg82rVrVx03bpy63W4NDw/XsLAwfwYNh8OhTqdT27Ztq4mJiZqTk6NLly71Lw2Gh4eriPizZ3To0EHnzJmjiYmJ2rZtW42KivIn77399tv943A6nVr1/3FlZaX+/Oc/1/DwcO3cubMuXbpUVVU//PBD7dKli+bm5mqXLl2OWOLbsmWLPwCa4FlgOzpYja4hsD1aS93qge3mGgLbr3zv030/W+LNJTa+lj4n+9aHl1Wl32lKG3Ye0DbT5uo9879s6qGYBlQ9y3ugTz75RE8++WStqKjwl1122WUaFhamp512mh44cKDW+uUVldo2p4PGJSZrcnZ7TekxRB0xrTRx5DTtdPt8Hf/MJ/rnxet0+XfFWlbu7T82NlanTZvm7y8rK0s7dOigqqpTpkzR3r1769KlS/1Z5fv27at33HGHP6v8mDFjtH///pqTk6M7duzQLl266P3336+qqhdeeKEuW7ZMFy9erMOHDz9i3GeccYYuW7Ys6Gs5f//73/XXv/51vfWOp80PuZZlmm9ga8jt/oVA4F2AGUDR8bZV1U2+n/uAV4DeNXWgqk+qar6q5lc9SLMpZcZHMaxLK17+z3ccKC1v6uGYBlI9y3uVzZs3c9lll/Hcc88RFvb9/3b33HMP7du3p2PHjv6bpMvKK3l35VrOGz2WUy7/NaOf+A9df7uALRGZSJ9xxF9yPy0i3ES7hZvP7MCq35zJC5N684vT2tEjK45wR83/W7vdbtLT06moqOC+++6jRYsWiIg/q3zVdv4qiYmJdOzYkaSkJIqKiujduzetW7dm7ty5JCcn07Nnzxo/Jzk5maKiIl566SWC+X+vvLycX/3qV/XWO542sbGxTJgw4Zg+wzRjDRUx8d5KsB5vcsyqzSOda6n7PEdvHvkW78aRON/7eF+fib464cDrwM/rG0soLEWqqi4rKNbsqXP1+Q+/beqhNHuBGzRUvdniY2Jijpo9nHrqqZqXl6d5eXnaqlUrTUhIqLfNLbfcom63W10ul/bt29efeT5wY0dWVpamp6eriOjSpUt13Lhx/mWzyMhIbdeunXbo0EHffvtt3bNnj3br1k1TUlL8WeSvvPJKff7557V16zYqItqmY6463RHqSc7UVhfepq6Udpp4zhQNj0vVMKdLAW3bMVdvuPUOPVxeoYMHD9bIyEhNT0/3f46q6qFDh7RXr16am5urDodDc3Nz/d+rZcuWRzwjsGrTxSWXXKKZmZnqdrv9meunTp3qv2G3R48e+vXXX/s3dUybNk3T09M1OztbU1JSNDIy0n/jt6r3RuCqXYqmeaOZztgatnNv9uav8O6OvM1Xdicwwve+F97Z2QFgJ7AmoO0kvLcJrAMu95V58N7T8BneG/QeBhz1jSNUApuq6nmPfaAD7/2XlldUNvVQmrXADRqqqu+8847OmTPnqCAVKDDzRm1tAjNvFBcXa2RkpJ5//vmqeuRGkHfffVeTkpK0f//++s4772haWpquXLlSW7VqpYAuXrZG3/j3Ck1Jz9JOPfvp+b+4Va964K966f1/16T23TQyMUOdnljN+uVMFbdHAY3PH67n3vlXdTidmturv67buleXLFmq//3vf3XgwIGalZWlOTk5eujQIT311FMV0AMHDugbb7yhLVq00PLycq2srNR9+/apqmpGRoa6XC5dsGCBFhcXa1xcnKalpfmz1Ve/5jRo0KAjliJXr16tubm5mpqaql999ZV/U0egmpYibZPGicMCWwi/QimwvfVZkWZPnatvr97c1ENp1mraoFHTL9kqe/fuVYfDoZ999lmdbf7yl79odHS0/zguLk47duxYY5+5ubma262nXjj5V9pp8Pna9w/vaJjboxLRUqP7jtbsqXM1PDFbkTANT26jESlt1ZPaTmMy2mtEdIJGtIjWVq07qMvlVhHRJ554QmfMmKGAJiQk+GeaK1eu1AEDBmhUVJTGx8f7n0kWFRWlhw8f1tdee82/y1BV9eGHH9b09HR1OBwaFhamUVFR2q5dO3366ae1ZcuW2rZtW42Li9MXXnhBVb3Xq9LT09Xlcml0dLRmZmb6v+MvfvEL9Xg82qFDB503b95RfwbV//xsk8aJpbkGNkup1cjO7JRCRlwkz7z/bVMPpdmqK/NGberKvBFo7969iAjLli2jrKyM/fv3s2vXLgAqK5W1W/bxwkcFXHjnC6wt2sV/d5Twz+Vfs88RQ+828YSHQU6Hk0nc/SUvTurNOUMG8NTzM9i36Wv2b/qKnOQW7Cv6hk452dw29WY2f7uW884bicvl4osvvmDcuHG0a9eOP/7xj6xatYrly5czceJEPvzwQy688EJ27txJWVkZV155JW3atOHTTz/lk08+YdiwYZSVlQFw7bXXkpiYSEREBFOmTOHAgQOsW7eOK664gj59+vDqq69SXFzM+PHjATj//PP9KZ727NnDhg0b/H8e4eHhzJ49m7Vr1/Kzn/3sqD+v0047jblz5/qPX3nlFa6++uqg/16MaQgW2BqZ0xHG5f3bsKSgmE837m7q4TRLtW3QqMuMGTNISEgIqu6YMWO48cYb6d27Nw6Hk5Jy5RcvLSf/rnc466F/c9sr7zPv0V9z/i9/T05yS0b1SOeqgW15+OLuuJxhnJSVQn5uJ/q3iyfO4yLO48btdOBwOPxZ9bdu3epPVZWYmEhUVBS7d3v/exg0aJD/+wW2WbNmDatXrwa8Ky1V6Zfuu+8+4uPj8WYf+r5NYWEhS5Ys8beB7zd2BKtLly4MGTIk6Pq2ScOEAgtsTWBMfgYt3U6e+eD7WVvgY0jA+8up6nEjI0aMqLfP0tJSLrroInJycujTpw8FBQU11ps0aRLJycn+5LNViouLGTp0KO3bt2fo0KH+WcrLL79Mbm4uubm5nHLKKXz66aeAd9Y0cODAqkSpR6j+XYYNG0ZsbCznnHPOEfWuuOIK8vLyyM3N5fzzz+fUU0+tt03Pnj3Jyspi7dq1tGvXjlWrVh31+dOnTycnJwcRYceOHUyfPp333nuPrVu30X/QYDJbt6XdSZ146K8L+NeXW/lq41aSMtrgDHdxyy238I9FH5I94X4qz76DktJS9h4q4/lfjqDg8asYFV9E3PsP8OCd/8uWRc/x7ZefMvcfr/9/e+ceV1WZNf7vggOiCIIiimDcvKNGipZWKlh4wRRNMzVNJZucpqlm7GI3x25vk2P2zi8ra7DsatY7lWlpNy1rUikTNctL4ngjr4iiCAjr98fe53S4KZQIHp7v53M+7v3sZz17rc32rP3sZ521+OijjwDr77Z27VpXUtw9e/aUqkEG4HA4aNeunauuWEREBICrX2Uy3aN7mfcAABnwSURBVLt3Z9myZS6ZY8eOOUuLVCgTFBREv379XDIAp06dcslUhSlTplS5L8CkSZNwOEwKWkPtYhxbLRDg58P1PVuzdGM2e4/mAzB//nxGjBjhrFpLw4YNWb9+PevXr2fx4sVnHTM9PZ3g4GC2b9/OnXfe6criXpaJEyeW+qJz8sQTT9C/f3+2bdtG//79XRm/o6Oj+eKLL9iwYQMPPvggN998M4Cr6rB7HS8nZW256667XMUe3ZkzZw6ZmZls2LCB3NxcgoODzyizefNmsrKyePXVVwkLC6OkpITOnTtzOK+ATXtz+XbnEfbk5LOVVvS5/Z80atqSQbM+4va776dh+ytQn4b85+uvYdjjHI25mr9OGsULq3awK6cA6TWR6Jufxb9VOw7u2MyBvbvoHdkEtIQGWsDRfTuY97+zePq+qUy6cQKpqanMnj2bnj17kp6eTkZGBt988w1+fn4cOXKEVatWkZWVxaZNm3j22Wc5ePCga0ZWUlLCgQMHyMrKIicnh759+3L06FGuu+46srKy2LZtGz179iwns3btWjp06ADA0KFD+fnnn2nTpk2lMvn5+Xz66acuGYCtW7cSFxdX2W1kMHgGtb3Idz4+dSl4xMmenJMaM32pPrZ0s6qWD4bw9/ev1nju+fWKioq0WbNmWlJSceRlVlaWxsXFlWpzz7fnTIxbliNHjpRKpLt+/XodNGhQuX7VDewoKSnRFi1a6N13331GmQf+9ohGd+isKXc+qdG9h6h/eDuNuHG2Rt6zRBtEdFKvhoEqDl/1DmimHSc9oY2attR+afdrQHBz/fOT8zXsohiNbh+nD85+Xp949iX18vbWoOCm6ufnp+Hh4bps2TLt27evNmrUSB0Oh/r4+GhwcLD6+/trUVGRdurUSQFt27atRkVFqY+PjzocDg0NDVVfX1/18vJSQP38/NTLy0tjY2O1a9euOnz4cM3MzNSoqCh1OBwqIurv769xcXEaGxurUVFRGhQUpDExMdquXTtNTk7WjIyMCmWSk5NV1QrSCA8Pd8k4AzsyMzM1Pj5eu3TponFxcTpz5kzX9TOBHYbqwgUaPGLeGdQS4UENGdwljDfX7OKWKyPLBUOcOnWKhIQEHA4H9957L6mpqWccb+/eva6quA6HgyZNmnD48GFCQkKqpM/+/fsJCwsDICwsjAMHDpTrk56eXiqAoHPnzq7XaU6qG9gxadIkli5dytGjR5kxY0a54ycLT/PClz/z6eYDLFu8msI8Zd8r/8THC4pOHKV3aAmDrulEyxs+I6yJH2FBfoT4N8DLS4iKeo5+rX3o/5c/88Bdk1i3eD7Z2dkElhxn/47tJHTvzty5c0lISHCd77777qN58+bs2LEDLy8vRo8ezfLly8nMzGTw4MG0a9eOcePGMXLkSJfMzp076dOnD5s2bXKVA5kyZQoDBw5k9erVjB8/nq5du5KVVXHA0Jw5cwgMDCQtLa3cscpk3njjDWbOnFlOpmvXrnz/fdkixr/KmMAOQ33AOLZaJO2KaD7I3MdLn20oFwyxa9cuWrVqxY4dO0hKSqJLly7lamy5Yz1clcYZTHAuWLFiBenp6Xz11VeuNm9vb3x9fTl+/DgBAQFA9QM7XnrpJXbv3k3Xrl156623mHDjRNbtyuHTzftZ9MF6tuw4wo4Pf6JjWCBdwptwZeoEHrtvGqdPF3HRRRehO9cy8fJplY7vfl0CAwMREVSVWbNm0b9//1LXKDs7mx9//JG5c+e6soOEhIQQEhLC7t27mTVrFmlpaaVk8vLyuPbaa3n66adL1bhyBmnMmjXrrNcgKCiI8ePHV/manU8Zg+FCxKyx1SLxrYPoERXM29/v59SpU6WOOQMBYmJi6NevX6VP4U4iIiLYvXs3YKUbys3NpWnTpuX65efnM3r0aNcX/j333EPnzp3Jz8/n+eefB6wv+NDQUFf/hIQE0tLSeP/99xkzZgy+vr74+/tz6aWXcvLkSfz8/FwBG+Hh4Zw8ebJKQSpxcXHExsbSp09fThUWMfVPt+Pj24DkSXcx/+ssQhr70tzrBAf+OYrld/YlvnUQTRzFeHt70aBBA3x8fFi8eDHt27dn+fLlrnM4z+0s2Oi8Lnv27CEvL88VrOEecHHs2DFSUlKIjo4u9ZOAiIgI/Pz8aNy4cTmZoqIirr32WsaNG8eIESNK2VmdII3fEnBxvmQMhgsR49hqman9YvmlwMGh46c4fuIkADk5ORQUFADWDOjrr7+mU6dOAEyfPp1333233DhDhw5lwYIFALzzzjskJSVVOGObP38+AwcORERYunQp69atY/369dxyyy08+uijHDt2jAULFjBs2DAA/vGPf7Bz505ee+012rVrR4cOHUhKSiIxMZGbbrqJ4uJifHx8uPzyy/n000+JjIykuLiYMWPGuIJUVJVj+UUcP3WaqdMeJLhtN8Y9/grbs3bhFdkNSf0fTp08ie9FXUm4ZgLdAo4ztHAl96d0IrpVc9LS0mjfvj19+vTh1VdfpaCggOXLl5Odnc2tt97KsmXLmDx5sms24h4gk5iYyMcff0xOTg5TpkwhMzOT5ORkVq9eTZMmTQgLC6OwsJDhw4czYcKEcrkNKwvSUFXS0tLo2LEjf/nLX8pd561bt5Zz6gaD4TxR24t85+NTF4NH3ElftUP9u1ytKXfP1cLTxWcsv5GSklKqCKOT/Px8HTlypMbGxmqPHj30559/VlWrppd7gEezZs00JCREHQ6HBgYGampqqqqqHjp0SMPCwrRFixaalJSkhw8fVlXV0NBQDQgIcGXBCAwM1GeeeUZTUlJ04cKF6ufn5wpS+fnAcQ1pGa7x/YboiPuf0yGP/Z82ahGlfq3jXIEdiJc2u2aaNr9mmjr8m6iXt4+GXtRGG/k31jvuuENnzJihKSkpGhkZqSEhIern56ehoaHauHFjzcjI0OjoaPXx8VFvb2+NiYlxpY8KCAjQAQMGqKqVecOZ2iosLEyvvPJKjY2N1djYWG3cuLFGRkZq586dddiwYZqRkaGvvvqqenl5uUqqOBwO7dWrl6pWHqSxatUqBbRLly6ua7N06VJVVS0sLNQOHTpUu4CkwVDX4AINHql1Bc7Hp647NlXVh9I/UP+4RL35lQwtKCqutJ8zKu63ULakyvLly7V379564sQJPXjwoEZHR7vKklTUX1U1Li5OFy1apCkpKTp8+HBt3bq1rtmcpX9+c51G3btEvQNDNWzM49o8/iod9Og72jQiVh9fulnnf7VDP9q4TxsHBur+3Hz9+5NP6iOPPKJBQUGqaqVu6tatm86YMUMvueSSUuVOTp8+rQ6Hw1V3S1X11ltvddXtUtVSJVVUK478VFW96aab9J133qnyNfst5U5+SxkWg6EucqE6NvPCvY4wc/IQdm/7gWUb9/EnhWfGdsPXUf5NsftaUnUpG9iRnJxMRkYGvXv3pnnz5vTq1avUGkxFgSDWvW79rqpX0gA++zqDkfO+oXFgMDf3ieHZ1/347umbWLKkBX37xjPsTT+mD+7okvcWITTQj7IvScPCwmjTpg0lJSWMHTu21CtBb29vvLy8OHHiRDk9nDhLqpyN6mbe+C0BF7+lDIvBYDh3mDW2OsT8/7mHmcO68PHm/dz6xjoKT5ecsb97ho8VK1a4MpXEx8fj5+fHe++9V6p/w4YNSwWpFBQUsGHDBvLy8jh27Bh5eXm0bdu2XH/3QJCIiAg2bd/F5l+O88grH5GXe4SpyfGsujuR6YM64u0leHkJkydPdv3Y2p0WLVqQnZ1NREQEW7ZscQWp7Nmzh9TU1FK1ytwpKSkpVS/MPVjGKV8280ZFVDfzxm8JuBg1alS1U34ZDIZzh3FsdYyJl0czc2gcn1TBubln+EhMTHRlKvn8889p1KgRycnJpfoHBwdTXFzscm4vvvgiDRs2ZPv27YwYMYKVK1e6ZKZPn87KlStLBYIUni7hVKtLeDL9LQ7nFdKve2eaNWnM9MEdada4QTn9AE6ePOlKtgu/BrkMGDCApUuXMmDAAHJycvj4448ZMGBAhWMcPnwYHx+fUg5m6NChLFy4kIKCglJBHWfDBHUYDPWA2n4Xej4+F8IaW1kW/CdLI+9ZomkvV77mVlGGD1XVefPm6dixYyuUmTx5sqsIZP/+/TUqKko7duyoPXv21CZNmrgCQZxBKpMnT9aXFr2v45/+QH1CLtKGEZ3U22EFWfj4+GhkZKRmZ2frww8/rA0aNFBvb28NCwvTmJgYbdmypatsinOd6tChQ5qUlKRt2rTRDh06aHR0tMbGxuqcOXM0PDxcAwIC1MfHR5s3b665ubn673//W5s2bare3t4aGhpaao3x0UcfLRfUoap6/fXXa8uWLdXhcGh4eLjr3Caow2CoHlyga2y1rsD5+FyIjk31zM6tosAOJ4mJifrBBx+49k8UFOmHG/bpW2t36ew3lmnikGv1kx9+0ei2HXTxVxt03X+P6Ka9R7V1ZJR+v+W/mn00XxP7X6U/7M3VUY+9rv5xidrmtpe1RWQbPZJXUOpcZwvGmDZtmmZmZv6Oq6A6fPhw/emnn37XGKomqMNgqC4XqmOr0eARERmIXeUa+JeqPlHmeB/gaaArcL2qvuN27EbgAXv3UVVdYLd3B14GGgIfArfbfwCPY0KvKAR48P0f+OPr3zF3XDcaOKx1q8oyfGRnZ7Nx40auujqZL7ce5L3v97Lsh184WVjs6pNXHE7ay2vIPnKSqa+vwxFo1d/Kzj3FNXO/xrthICTcweB/rsLftymDk/tz76SeTPi8AcH+vqXOd7ZgjKpk3jgThYWFpKam0r59+981DpigDoOhvlBjjk1EvIG5wNXAHiBDRBar6ma3bruAicC0MrJNgRlAAqDAd7ZsDvAccDOwGsuxDQQ+qik7apvxvaJAhAff28QfX1vHszdYzq1sIAhYs+//fWEBUd37ceU/vuTg8QIC/BwMi2/F0IvDad20IYWnSygsvpKCohL+sDqWSYmhtOvSnZOnChkzr4Cnxl9BUbFSeNrK8DGkSxjB/gMrLYNT3WCM6uLr61tqje73MGrUqHMyjsFgqNvU5IytJ7BdVXcAiMhCYBjgcmyqutM+VjZCYgDwiaoesY9/AgwUkZVAoKp+Y7e/AqTiwY4NYPxlkQjwwHubmPraOp67oVupQJBD+SW8v34f732/ly/nvUxI4o2ktA5i+CXhJHYIZeZDD/CL9KTX8OGlxx19Ld9/9j5/GDWIhQsXMvDqqxiV0Lpaum3dutU4DIPBUKeoyajIcGC32/4eu+33yIbb22cdU0RuFpFvReTbgwcPVlnpusoNl0XyaGpnPv/pAFNfW8fB4wW0uaQ3A+99gSv+voJZy7fgk38I/9NH2TjvTl6YkMCgLmH4+XizceNGWrZsWW7MtLQ0Dh8+TJs2bXjqqadcNdj27dvH4MGDXf3GjBlDr1692LJlCxEREaSnpwNWrsTt27eXyo5vMBgMtU1NztgqSi1f1bWwymSrPKaqvgC8AJCQkOARa3A3XBaJCNz/7iZ6PPYphc0up2TVEu5/ci7D4sO5qFkjuP/acnJFRUX06tWrXLufnx9vv/12ufZWrVrx4YcfuvbffPPNCvVZsmQJI0eONIl1DQZDnaImv5H2AO7vtSKAqqZ82AP0KyO70m6P+I1jegTjLo2kcQMHm/bmcs3Fl5Ox3J+J/WIq/DG0k9+TreRMmGAMg8FQF5GaCigUEQewFegP7AUygLGq+kMFfV8GljijIu3gke+AbnaXdUB3VT0iIhnAbcAarOCR/6eqH5Yd052EhAT99ttvz4ldBoPBUF8Qke9U9YJba6ixNTZVPQ38CVgO/AgsUtUfRORhERkKICI9RGQPMAqYJyI/2LJHgEewnGEG8LAzkASYCvwL2A78jIcHjhgMBoOhetTYjK0uYWZsBoPBUH3MjM1gMBgMhjqAcWwGg8Fg8CiMYzMYDAaDR2Ecm8FgMBg8CuPYDAaDweBRGMdmMBgMBo+iXoT7i8hB4L9n6RYCHDoP6tRF6rPtUL/tN7bXX6pif6SqNj8fypxL6oVjqwoi8u2F+HuNc0F9th3qt/3G9vppO3i2/eZVpMFgMBg8CuPYDAaDweBRGMf2Ky/UtgK1SH22Heq3/cb2+ovH2m/W2AwGg8HgUZgZm8FgMBg8inrv2ERkoIhsEZHtInJvbetT04jIfBE5ICKb3NqaisgnIrLN/je4NnWsKUSktYisEJEfReQHEbndbvd4+0XET0TWikimbftMuz1aRNbYtr8lIr61rWtNIiLeIvK9iCyx9+uF/SKyU0Q2ish6EfnWbvPY+75eOzYR8QbmAoOATsAYEelUu1rVOC8DA8u03Qt8pqptgc/sfU/kNPBXVe0IXAbcav+964P9BUCSql4MxAMDReQy4O/AHNv2HCCtFnU8H9yOVR/SSX2yP1FV491C/D32vq/Xjg3oCWxX1R2qWggsBIbVsk41iqp+CRwp0zwMWGBvLwBSz6tS5wlVzVbVdfb2cawvuHDqgf1qkWfv+tgfBZKAd+x2j7TdiYhEAClYhYoREaEe2V8BHnvf13fHFg7sdtvfY7fVN1qoajZYX/5AaC3rU+OISBRwCbCGemK//RpuPXAA+ASrAv1Ru9o9eP79/zRwN1Bi7zej/tivwMci8p2I3Gy3eex976htBWoZqaDNhIl6OCLSGPg/4A5VPWY9uHs+qloMxItIEPAu0LGibudXq/ODiAwBDqjqdyLSz9lcQVePtB+4XFX3iUgo8ImI/FTbCtUk9X3Gtgdo7bYfAeyrJV1qk/0iEgZg/3uglvWpMUTEB8upva6q/7ab6439AKp6FFiJtc4YJCLOB1xPvv8vB4aKyE6sJYckrBlcvbBfVffZ/x7AeqjpiQff9/XdsWUAbe3IKF/gemBxLetUGywGbrS3bwTer0Vdagx7TSUd+FFVn3I75PH2i0hze6aGiDQErsJaY1wBjLS7eaTtAKo6XVUjVDUK6//556o6jnpgv4j4i0iAcxtIBjbhwfd9vf+BtogMxnpy8wbmq+pjtaxSjSIibwL9sDJ77wdmAO8Bi4CLgF3AKFUtG2BywSMiVwCrgI38us5yH9Y6m0fbLyJdsQIEvLEeaBep6sMiEoM1g2kKfA/coKoFtadpzWO/ipymqkPqg/22je/auw7gDVV9TESa4aH3fb13bAaDwWDwLOr7q0iDwWAweBjGsRkMBoPBozCOzWAwGAwehXFsBoPBYPAojGMzGAwGg0dhHJuhziIiKiKz3fanicjfztHYL4vIyLP3/N3nGWVXE1hRpj3Ktu82t7ZnRGTiWca7RUQmnKXPRBF5ppJjeRW1nytsu9wrR0wRkXWelDneUPcxjs1QlykARohISG0r4o5dFaKqpAF/VNXECo4dAG6vTqkUVX1eVV+pxvnPGW4ZOqrafzxwG5Csqjk1o5XBUB7j2Ax1mdNY5evvLHug7IzLORMRkX4i8oWILBKRrSLyhIiMs2uRbRSRWLdhrhKRVXa/Iba8t4jMEpEMEdkgIn9wG3eFiLyB9QPvsvqMscffJCJ/t9seAq4AnheRWRXYdxCrXMiNZQ+ISKyILLOT1q4SkQ52+99EZJq93cPW8Rtb501uQ7Sy5beJyJNlxp5tz6I+E5Hmdlu8iKy2x3vXOcMSkZUi8riIfIHlhEfZNmaKyJcV2OQ8x3VYZVCSVfVQZf0MhprAODZDXWcuME5EmlRD5mKsultdgPFAO1XtiVWu5Da3flFAX6xSJs+LiB/WDCtXVXsAPYApIhJt9+8J3K+qpWr2iUgrrLpeSVi1znqISKqqPgx8C4xT1bsq0fUJ4K8VzAJfAG5T1e7ANODZCmRfAm5R1V5AcZlj8cBo+xqMFhFnTlR/YJ2qdgO+wMo8A/AKcI+qdsVy3DPcxgpS1b6qOht4CBhg13UbWolNkcAzWE7tl0r6GAw1hnFshjqNqh7D+tL9czXEMuzaawVYpVk+tts3YjkzJ4tUtURVtwE7gA5YefQmiFXeZQ1WaZO2dv+1qppVwfl6ACtV9aBdAuV1oE8V7csC1gJjnW1iVR/oDbxt6zEPCHOXs/M+Bqjqf+ymN8oM/Zmq5qrqKWAzlrMBK5XYW/b2a8AV9kNDkKp+YbcvKKP/W27bXwMvi8gUrPRcFXEQK0XTdZUabjDUIPW9bI3hwuBpYB3WDMXJaewHMzu5sfs6lXuuvxK3/RJK3/Nl88kpVimT21R1ufsBO7/giUr0+711bx7HKnbpfLXnhVUnLP4MMmc7p/s1KKby/+tVyannsltVbxGRS7FmuetFJF5VD5fpfxKrKv1XInJAVV+vwjkMhnOGmbEZ6jx2YtZFWK8JnewEutvbw7AqQleXUSLiZa+7xQBbgOXAVLHK2yAi7eyM6GdiDdBXRELsV4pjsF7zVQlV/QlrVjXE3j8GZInIKFsHEZGLy8jkAMdF5DK76foqns6LX7PZjwW+UtVcIEdErrTbx1emv4jEquoaVX0IOETpsk/u+h0EBgKPi8iAKupmMJwTzIzNcKEwG/iT2/6LwPsishYrAKOy2dSZ2IL1Bd4Ca63qlIj8C+t15Tp7JngQSD3TIKqaLSLTsUqgCPChqla3BMhjWNnlnYwDnhORB7Cc9kIgs4xMGvCiiJzAqq+WW4XznADiROQ7u/9ou/1GrHXGRlivZSdVIj9LRNpi2flZBTq5UNUsERkKfCgiI1R1TRX0Mxh+Nya7v8FwgSIijVXVGQ16LxCmqrfXsloGQ61jZmwGw4VLij1TdAD/BSbWrjoGQ93AzNgMBoPB4FGY4BGDwWAweBTGsRkMBoPBozCOzWAwGAwehXFsBoPBYPAojGMzGAwGg0dhHJvBYDAYPIr/DwVtFsLPYyEDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.14  0.108 0.102 0.1   0.101 0.101 0.102 0.102 0.103 0.103 0.103 0.104\n",
      " 0.104 0.105 0.105 0.105 0.106 0.106 0.106 0.106 0.106 0.107 0.107 0.106\n",
      " 0.106]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train_ng,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 8.857\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='brute')\n",
    "knn_optimal.fit(d_train_ng,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train_ng)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test_ng)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.994\n",
      "The value of TNR is 0.119\n",
      "The value of FPR is 0.881\n",
      "The value of FNR is 0.006\n",
      "\n",
      "Test error is 10.283\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGYJJREFUeJzt3XucVWW9x/HPdwBBwQS5RWCGiZbWERGRtAzBBrzk5XjXo+TlTF4yLY5JWt7t2PGSKeYJkxN4kksqQmgqIiYeJQeVULwEkuiIOSpECqjMzO/8sRe4xZlhIzOz9zx+377Wa/Z61tprPRvwt3/zW896liICMzNr/cqK3QEzM2saDuhmZolwQDczS4QDuplZIhzQzcwS4YBuZpYIB3Qzs0Q4oJuZJcIB3cwsEW2L3YGGbNG+j29htY/ZYZtexe6ClaAXqiu1ucdY+9aSgmNOu247bPb5moMzdDOzRJRshm5m1qLqaovdg83mgG5mBlBbU+webDYHdDMzIKKu2F3YbA7oZmYAdQ7oZmZpcIZuZpYIXxQ1M0tEAhm6x6GbmQFRW1PwsjGSXpb0jKT5kuZlbdtKmilpUfazS9YuSTdIWixpgaQBeccZme2/SNLIjZ3XAd3MDHIXRQtdCrNfRPSPiIHZ+mhgVkT0A2Zl6wAHAP2ypQK4GXJfAMDFwF7AIODidV8CDXFANzODXMml0OWTORQYn70eDxyW1z4hcuYCnSX1AoYDMyNieUSsAGYCIxo7gQO6mRnkLooWumxcAA9IelJSRdbWMyJeB8h+9sjaewOv5r23KmtrqL1BvihqZgablHlnQboir2lsRIzNW98nIpZJ6gHMlPRCY4errzeNtDfIAd3MDDbp1v8seI9tZPuy7Ge1pKnkauBvSOoVEa9nJZXqbPcqYLu8t/cBlmXtQzZof7ixfrnkYmYGTXZRVFJHSVuvew2UA88C04F1I1VGAtOy19OBk7LRLoOBlVlJ5n6gXFKX7GJoedbWIGfoZmZARJPdWNQTmCoJcjH29oi4T1IlMEXSqcArwFHZ/vcCBwKLgdXAybn+xHJJlwOV2X6XRcTyxk7sgG5mBk12Y1FELAF2q6f9bWBYPe0BnNXAscYB4wo9twO6mRl4ci4zs2QkcOu/A7qZGUDt2mL3YLM5oJuZgUsuZmbJcMnFzCwRztDNzBLhgG5mlobwRVEzs0S4hm5mlgiXXMzMEuEM3cwsEc7QzcwS4QzdzCwRNYU/4KJUOaCbmYEzdDOzZLiGbmaWCGfoZmaJcIZuZpYIZ+hmZonwKBczs0REFLsHm80B3cwMXEM3M0uGA7qZWSJ8UdTMLBG1tcXuwWZzQDczA5dczMyS4YBuZpYI19DNzNIQdR6HbmaWBpdczMwS4VEuZmaJcIZuTW2nnXbgd/978/r1vn0/z6WXXcOfHn6MMWOuolOnjixd+ionjTybd955t4g9teY2a940Vr27mtq6OmprajiyfCRf+spOXPJfo2nfoT21NTVcev7Peebp5xi09wBumnAtVa8sA2DmPbP51bW/KfInaGWaOKBLagPMA16LiIMl9QUmAdsCTwEnRsQHktoDE4A9gLeBYyLi5ewYPwZOBWqB70fE/Y2d0wG9xPz1r0vYc9BwAMrKynj5b/OYNu0+Jk38NeePvoI5c+YycuQxjPrh6Vxy6TVF7q01t5P+9XT+sXzl+vXzLjqbm675DXMeeox9h+3NeRd9n5MOPx2AJ+c+zen/9sNidbX1a/rJuc4Bngc+k63/HPhFREyS9N/kAvXN2c8VEbGjpGOz/Y6RtAtwLLAr8DngQUk7RUSDtaGypv4E60j6kqTzJd0g6ZfZ6y831/lSNHTo11myZCmvvPIaO+30RebMmQvArFmPcPjhBxa5d1YMEUGnrTsCsPVnOlH99zeL3KOE1NUVvmyEpD7AQcBvsnUBQ4E7sl3GA4dlrw/N1sm2D8v2PxSYFBHvR8TfgMXAoMbO2ywBXdL55H61EPAEUJm9nihpdHOcM0VHH3UIk6dMA2Dhwhf59rfLATjiiIPp0+dzxeyatYCI4NYpY7hz5gSOPvFwAH72k+s47+LvM/vpGfzoknO47sqb1u/ff+BXuXv27xg78ZfsuPMOxep261UXhS8bdz3wI2Bd9O8K/CMi1k26XgX0zl73Bl4FyLavzPZf317Pe+rVXCWXU4FdI2JtfqOk64CFwFXNdN5ktGvXjoMPLucnP839UVV8dxTXXXcZF15wLjNmzOSDD9Zu5AjW2h1/8GlUv/EW23brwrjfj2HJopcZ/u2hXHXRdTwwYzYjDtmfK67/KacceRYLF7zI0D0OYfWqNew7bG/GjL+aEYOPKPZHaF02YZSLpAqgIq9pbESMzbYdDFRHxJOShqx7Sz2HiY1sa+w99WqukksduZrPhnrx4TfWx0iqkDRP0ry62lXN1LXWYcSI/Xh6/jNUV78FwIsvvsRBB53A4K8dyOQpd7NkydIi99CaW/Ubub/75W+t4MF7H+ZfBuzKYccczAMzZgNw3/QH+ZfddwFg1burWL1qDQCPzHqMdm3b0nnbbYrT8VYq6uoKXyLGRsTAvGVs3qH2AQ6R9DK5SsVQchl7Z0nrkug+wLLsdRWwHUC2fRtgeX57Pe+pV3MF9HOBWZL+KGlsttwHzCJ3oaBe+X9IZW06NlPXWodjjj6UyZOnrV/v3r0rAJL48ehzGHvLbcXqmrWALbfqQMeOW61/vc+Qwfz1+Zeo/vubDNp7AACDv7EnS5fkfiPv1qPr+vd+dfddUFnZRy6mWgGaqOQSET+OiD4R8QVyFzUfiogTgNnAkdluI4F1/4NPz9bJtj8UEZG1HyupfTZCph+5EnaDmqXkEhH3SdqJXAG/N7lfHaqAysau0FrOllt2YNiwfTnzrA8vNxxzzGGccXru7/zuu//I+PGTi9U9awFdu3dlzG//C4A2bdoy4677eHT24/x01GouvGIUbdq24f33PuCiUT8DYPjBQzn2O0dSW1vDe2veZ9R3Lyxm91un5p/L5XxgkqQrgKeBW7P2W4HbJC0ml5kfCxARCyVNAZ4DaoCzNhY/FSX6HL0t2vcpzY5ZUe2wTa9id8FK0AvVlfXVmzfJqstOKDjmdLzod5t9vubgcehmZgA1rb944IBuZgaePtfMLBmePtfMLA3hybnMzBLhDN3MLBEO6GZmifADLszM0uBnipqZpcIB3cwsER7lYmaWCGfoZmaJcEA3M0tD1LrkYmaWBmfoZmZp8LBFM7NUOKCbmSWi9ZfQHdDNzACipvVHdAd0MzNwhm5mlgpfFDUzS4UzdDOzNDhDNzNLhTN0M7M0RE2xe7D5HNDNzIBwhm5mlggHdDOzNDhDNzNLhAO6mVkiolbF7sJmc0A3M8MZuplZMqLOGbqZWRKcoZuZJSKi9WfoZcXugJlZKYi6wpfGSOog6QlJf5G0UNKlWXtfSX+WtEjSZElbZO3ts/XF2fYv5B3rx1n7i5KGb+wzOKCbmQF1tSp42Yj3gaERsRvQHxghaTDwc+AXEdEPWAGcmu1/KrAiInYEfpHth6RdgGOBXYERwK8ktWnsxA7oZmbkLooWujR6nJx3s9V22RLAUOCOrH08cFj2+tBsnWz7MEnK2idFxPsR8TdgMTCosXM7oJuZsWkBXVKFpHl5S0X+sSS1kTQfqAZmAi8B/4hYPwVYFdA7e90beBUg274S6JrfXs976uWLomZmQGzCdOgRMRYY28j2WqC/pM7AVODL9e2W/awv5Y9G2hvkgG5mRvOMQ4+If0h6GBgMdJbUNsvC+wDLst2qgO2AKkltgW2A5Xnt6+S/p14uuZiZkRu2WOjSGEnds8wcSVsC+wPPA7OBI7PdRgLTstfTs3Wy7Q9FRGTtx2ajYPoC/YAnGjt3wRm6pPYR8X6h+5uZtSa1TTeXSy9gfDYipQyYEhEzJD0HTJJ0BfA0cGu2/63AbZIWk8vMjwWIiIWSpgDPATXAWVkpp0EbDeiSBmUn3Ab4vKTdgNMi4uxP8EHNzEpSU91YFBELgN3raV9CPaNUIuI94KgGjnUlcGWh5y6k5HIDcDDwdnaCvwD7FXoCM7PWoKmGLRZTISWXsohYmhsWuV6jab+ZWWuzKaNcSlUhAf3VrOwSWU3obOCvzdstM7OWVcqZd6EKCehnkCu7fB54A3gwazMzS0ZtXesf9LfRgB4R1WRXXc3MUvWpKLlIuoV67k6KiIp6djcza5XqEpg+t5CSy4N5rzsAh/PR+QXMzFq9FOZDL6TkMjl/XdJt5CabMTNLxqei5FKPvsD2Td2RDdWl8KdrTe6Z5yZvfCezT+BTUXKRtIIPa+hl5G5NHd2cnTIza2nJj3LJJlnfDXgta6rLJo0xM0tKCoGt0a+kLHhPjYjabEnhM5uZfUxdqOClVBXyO8YTkgY0e0/MzIqoqabPLaYGSy55E7F/Hfh3SS8Bq8g9RSMiwkHezJJRV+wONIHGauhPAAP48EGmZmbJinqf+Na6NBbQBRARL7VQX8zMiqamhEsphWosoHeX9MOGNkbEdc3QHzOzokg9Q28DdKL+J0+bmSUl9Rr66xFxWYv1xMysiFLP0Fv/pzMzK1DqGfqwFuuFmVmR1SaQwzYY0CNieUt2xMysmBJ4At0nmm3RzCw5dSln6GZmnyYpTFTlgG5mRvoXRc3MPjXq5JKLmVkSaovdgSbggG5mhke5mJklw6NczMwS4VEuZmaJcMnFzCwRKQxbLOSZomZmyatV4UtjJG0nabak5yUtlHRO1r6tpJmSFmU/u2TtknSDpMWSFuQ/w1nSyGz/RZJGbuwzOKCbmZHL0AtdNqIGGBURXwYGA2dJ2gUYDcyKiH7ArGwd4ACgX7ZUADdD7gsAuBjYCxgEXLzuS6AhDuhmZjRdQI+I1yPiqez1O8DzQG/gUGB8ttt4Pnxe86HAhMiZC3SW1AsYDsyMiOURsQKYCYxo7NyuoZuZAc3xSFFJXwB2B/4M9IyI1yEX9CX1yHbrDbya97aqrK2h9gY5QzczY9MydEkVkublLRUbHk9SJ+BO4NyI+Gcjp67vqyQaaW+QM3QzMzbt1v+IGAuMbWi7pHbkgvnvIuKurPkNSb2y7LwXUJ21VwHb5b29D7Asax+yQfvDjfXLGbqZGblx6IUujZEk4Fbg+Yi4Lm/TdGDdSJWRwLS89pOy0S6DgZVZaeZ+oFxSl+xiaHnW1iBn6GZmNOk49H2AE4FnJM3P2i4ArgKmSDoVeAU4Ktt2L3AgsBhYDZwMuafGSbocqMz2u2xjT5JzQDczo+kCekQ8Sv31b6jnWc0REcBZDRxrHDCu0HM7oJuZ4blczMyS4blczMwS4QdcmJkloi6BoosDupkZacy26IBuZoYvipqZJcMZuplZImrU+nN0B3QzM1xyMTNLhksuZmaJ8LBFM7NEtP5w7oBuZga45GJmlozaBHJ0B3QzM5yhm5klI5yhm5mlwRm6Nbn27dvz8EN3skX79rRt24a77rqHSy+7lgnjb2SPPXZj7dq1VFbO54wzz6empqbY3bUmVn7ESDputRVlZWW0adOGKeNu4MaxE3jo0ccpUxnbdtmGKy8cRY/uXYkI/vP6/2bO45V06NCeKy8cxS4777j+WO+uWsUhx3+XYfvuzYWjzizip2odUhi26IdEl5j333+f/cuPZo+B32KPgeUMLx/CXoMGMHHiVHb9yr70330YW27ZgVNPOb7YXbVmMu7Gq7hz/E1MGXcDACefcARTJ9zMneNv4pv77MXN/3M7AHMer+SVqmXcO/lWLvnR97n8mjEfOc6Nt9zGwN2/2uL9b61iE5ZS5YBeglatWg1Au3ZtaduuHRHBH+97aP32ysr59OnTq1jdsxbWqWPH9a/XrHkPZU/Wmf3oXA4ZMQxJ7PaVL/POO+/y5lu5ZwgvfGERby9fwd57DihGl1ulGqLgpVS1eECXdHJLn7O1KSsrY17lA7z+2gJmzXqEJyqfXr+tbdu2nHDCEdx//+wi9tCaiyQqfnAhR59yNr+fdu/69l/++rcMO/xE7nlgNt877UQA3njzbT7bo9v6fXr26MYbb75FXV0dV4+5hVFnndbi/W/NYhP+K1XFyNAvbWiDpApJ8yTNq6tb1ZJ9Kil1dXUM3LOc7fsOZM+Bu7Prrjuv3zbmxp8xZ86fefT/nihiD6253Hbztfz+f8Zw87WXM/GuGcyb/wwA53z3O8yaehsHle/H7Xf+AYDcw+I/ShKT7prBvl/bk149u7do31u7uk1YSlWzXBSVtKChTUDPht4XEWOBsQBtt+hdul+DLWTlyn/yp0ceY3j5EBYufJGf/uQHdO/elTPOdOaVqh7duwLQtUtnhu27N8889yID+39YBz+ofAhn/sfFfO+0E/lsj278vfqt9dveqH6LHt268pdnn+fJBQuZdNcMVq95j7Vr17LVVh34wRmntPjnaU1KOfMuVHONcukJDAdWbNAu4LFmOmcSunXblrVra1i58p906NCBYUO/wdXX/IpTTj6O8m8N4VvDj6k3M7PWb/Wa94i6Ojp23IrVa97jsSee4oyTj2fpq6+x/Xa9AZg9Zy59t+8DwJCvD2binX/ggP2/yYKFL9CpU0e6d9uWn19y/vpj3n3PTBa+sMjBvAClnHkXqrkC+gygU0TM33CDpIeb6ZxJ6NWrJ+NuvZ42bcooKyvjjjv+wD33Psh7q5eydGkVj86ZDsDdd9/LFVdeX+TeWlN6e/kKzrngcgBqa2o5sHwIXx88kHMvuIKXX6lCZeJzn+3BReedDcC+X9uTOY9XcsDRp7Blhw5cfsEPitn9Vq82gURJpZrtueRi9VmzbE6xu2AlqF23HbS5xzh++8MLjjm3L5262edrDr6xyMwM19DNzJLhGrqZWSJSuPXfAd3MDJdczMySkcIoFwd0MzPSKLl4ci4zM5r21n9J4yRVS3o2r21bSTMlLcp+dsnaJekGSYslLZA0IO89I7P9F0kaubHzOqCbmdHkk3P9FhixQdtoYFZE9ANmZesABwD9sqUCuBlyXwDAxcBewCDg4nVfAg1xQDczI1dyKXTZmIh4BFi+QfOhwPjs9XjgsLz2CZEzF+gsqRe56VNmRsTyiFgBzOTjXxIf4Rq6mRn1z17ZxHpGxOvZuV6X1CNr7w28mrdfVdbWUHuDHNDNzIDaTbgoKqmCXHlknbHZbLGfRH3TCEQj7Q1yQDczY9NGueRP9b0J3pDUK8vOewHVWXsVsF3efn2AZVn7kA3aH27sBK6hm5mRK7kUunxC04F1I1VGAtPy2k/KRrsMBlZmpZn7gXJJXbKLoeVZW4OcoZuZ0bTj0CVNJJddd5NURW60ylXAFEmnAq8AR2W73wscCCwGVgMnA0TEckmXA5XZfpdFxIYXWj/CAd3MjKa99T8ijmtg07B69g3grAaOMw4YV+h5HdDNzPCt/2ZmyUjh1n8HdDMzHNDNzJJRqo/j3BQO6GZmOEM3M0uGH3BhZpaI2mj9TxV1QDczwzV0M7NkuIZuZpYI19DNzBJR55KLmVkanKGbmSXCo1zMzBLhkouZWSJccjEzS4QzdDOzRDhDNzNLRG3UFrsLm80B3cwM3/pvZpYM3/pvZpYIZ+hmZonwKBczs0R4lIuZWSJ867+ZWSJcQzczS4Rr6GZmiXCGbmaWCI9DNzNLhDN0M7NEeJSLmVkifFHUzCwRLrmYmSXCd4qamSXCGbqZWSJSqKErhW+l1EmqiIixxe6HlRb/u7ANlRW7A1aQimJ3wEqS/13YRzigm5klwgHdzCwRDuitg+ukVh//u7CP8EVRM7NEOEM3M0uEA3qJkzRC0ouSFksaXez+WPFJGiepWtKzxe6LlRYH9BImqQ1wE3AAsAtwnKRditsrKwG/BUYUuxNWehzQS9sgYHFELImID4BJwKFF7pMVWUQ8Aiwvdj+s9Digl7bewKt561VZm5nZxziglzbV0+ZhSWZWLwf00lYFbJe33gdYVqS+mFmJc0AvbZVAP0l9JW0BHAtML3KfzKxEOaCXsIioAb4H3A88D0yJiIXF7ZUVm6SJwOPAzpKqJJ1a7D5ZafCdomZmiXCGbmaWCAd0M7NEOKCbmSXCAd3MLBEO6GZmiXBAtyYnqVbSfEnPSvq9pK0241hDJM3IXh/S2IyTkjpLOvMTnOMSSf/xSftoVioc0K05rImI/hHxFeAD4PT8jcrZ5H97ETE9Iq5qZJfOwCYHdLNUOKBbc5sD7CjpC5Kel/Qr4ClgO0nlkh6X9FSWyXeC9XPAvyDpUeBf1x1I0nckjcle95Q0VdJfsmVv4Crgi9lvB1dn+50nqVLSAkmX5h3rwmye+QeBnVvsT8OsGTmgW7OR1JbcXO7PZE07AxMiYndgFfATYP+IGADMA34oqQNwC/Bt4BvAZxs4/A3AnyJiN2AAsBAYDbyU/XZwnqRyoB+5aYj7A3tI2lfSHuSmUdid3BfGnk380c2Kom2xO2BJ2lLS/Oz1HOBW4HPA0oiYm7UPJvfQjv+TBLAFudvZvwT8LSIWAUj6X6CinnMMBU4CiIhaYKWkLhvsU54tT2frncgF+K2BqRGxOjuH58exJDigW3NYExH98xuyoL0qvwmYGRHHbbBff5puimAB/xkRv97gHOc24TnMSoZLLlYsc4F9JO0IIGkrSTsBLwB9JX0x2++4Bt4/Czgje28bSZ8B3iGXfa9zP3BKXm2+t6QewCPA4ZK2lLQ1ufKOWavngG5FERFvAt8BJkpaQC7Afyki3iNXYrknuyi6tIFDnAPsJ+kZ4Elg14h4m1wJ51lJV0fEA8DtwOPZfncAW0fEU8BkYD5wJ7mykFmr59kWzcwS4QzdzCwRDuhmZolwQDczS4QDuplZIhzQzcwS4YBuZpYIB3Qzs0Q4oJuZJeL/AZAUXNuWgYr+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.3.5] KNN (KD-Tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████████| 25/25 [6:03:45<00:00, 861.37s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 7.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4VOX1B/DvmZnMJJnJvu8hC1s2dlAKUVkriqig2IIBrfwUV4oIVmutWqV1qa2KSxWKKxULBS2CqEjZZJVFVCCyJoQlIWzZl+/vj5mMSQhhQIZM4HyeZ57M3Hvfe08SzeG+y7lCEkoppVRrY2jpAJRSSqlzoQlMKaVUq6QJTCmlVKukCUwppVSrpAlMKaVUq6QJTCmlVKukCUwppVSrpAlMKaVUq6QJTCmlVKtkaukAzpfQ0FAmJia2dBhKKdWqrF+/vpBkWEvHcS4umgSWmJiIdevWtXQYSinVqojInpaO4VxpF2I9ZWVlyM7ORk1NDQBg8ODBCAwMxDXXXONS+4qKCtx8881ISUlBz549sXv37iaPu+222xAeHo709PQG2x9//HHExMSgU6dO6NSpExYsWAAAWLx4Mbp27YqMjAx07doVX375pbNN//79UVxcfA7frVJKtW6awOqZPn06brjhBhiNRgDApEmT8M4777jc/q233kJQUBByc3MxYcIETJ48ucnjxowZg4ULFza5b8KECdi4cSM2btyIq6++GgAQGhqKjz/+GFu2bMHMmTMxevRo5/GjR4/GtGnTXI5RKaUuFprA6nnvvfdw3XXXOT/369cPfn5+LrefN28ecnJyAADDhw/HF198gaaq/fft2xfBwcEun7dz586Ijo4GAKSlpaG8vBwVFRUAgKFDh+KDDz5w+VxKKXWx0ATmUFlZiZ07d+LnTATJz89HXFwcAMBkMiEgIABFRUVndY6XX34ZmZmZuO2225rsGvz3v/+Nzp07w2KxAACCgoJQUVFx1tdRSqnWThOYQ2FhIQIDA3/WOZq62xIRl9vfdddd+PHHH7Fx40ZERUVh4sSJDfZv3boVkydPxuuvv95ge3h4OPbv339uQSulVCulCczBx8cH5eXlP+scsbGx2LdvHwCguroax44dO6uuwoiICBiNRhgMBtxxxx1Ys2aNc19eXh6uv/56vP3220hOTm7Qrry8HD4+Pj8rdqWUam00gTkEBQWhpqbGpST28MMPY+7cuadsHzp0KGbOnAkA+Oijj3DVVVed1R1YQUGB8/3cuXOdsxSPHj2KIUOG4JlnnkHv3r0btCGJAwcO/KyuT6WUao0u+QR2vLwKf128HRv3HcXAgQOxfPly574+ffpgxIgR+OKLLxAbG4tFixYBALZs2YLIyMhTznX77bejqKgIKSkpeOGFFzB16lQAwP79+50zCgHglltuwWWXXYZt27YhNjYWb731FgDgoYceQkZGBjIzM7FkyRL89a9/BWAfF8vNzcWTTz7pnGJ/6NAhAMD69evRq1cvmEwXzZI+pZRyiTQ1btMadevWjeeykPlYaRWynvgMjw7pgK62Y3jhhRfOOHV+0KBBzmTW0u6//34MHToU/fr1a+lQlFKtkIisJ9mtpeM4F5f8HZi/jwkmg+BISSU6d+6MK6+80rmQ+XQ8JXkBQHp6uiYvpdQl6ZLvdxIRBFvNKDpZCcBeJaM1ueOOO1o6BKWUahFuvQMTkcEisk1EckVkShP7+4rIBhGpFpHhjfbViMhGx2u+O+MMsVlQVFLpzksopZQ6z9x2ByYiRgCvABgAIA/AWhGZT/K7eoftBTAGwINNnKKMZCd3xVdfiNWMIyUVF+JSSimlzhN33oH1AJBLcifJSgCzAFxX/wCSu0luBlDrxjjOKMRm1jswpZRqZdyZwGIA7Kv3Oc+xzVXeIrJORL4WkWHnN7SGgq1mHDmpCUwppVoTd07iaGoF79nM2Y8nuV9EkgB8KSJbSP7Y4AIi4wCMA4D4+PhzDjTUZsGJimpUVNfAYjKe83mUUkpdOO68A8sDEFfvcywAlwv2kdzv+LoTwFcAOjdxzBsku5HsFhZ27g8UDbaaAQBHtBtRKaVaDXcmsLUAUkWkjYiYAYwE4NJsQhEJEhGL430ogN4Avmu+1bmrS2BF2o2olFKthtsSGMlqAPcAWATgewAfktwqIk+IyFAAEJHuIpIHYASA10Vkq6N5BwDrRGQTgCUApjaavXhehdocCUzvwJRSqtVw60JmkgsALGi07bF679fC3rXYuN1KABnujK2+YKv92VpFJ3UqvVJKtRaXfCkpwD6NHtAxMKWUak00gQHws5jgZRQU6hiYUkq1GprAYK+HGGK1aDUOpZRqRTSBOdQv6KuUUsrzaQJz0HJSSinVumgCc7AX9NUEppRSrYUmMIcQm0Wn0SulVCuiCcwh2GpGSWUNyquafxqzUkopz6AJzCHEqtU4lFKqNdEE5hBis1fj0MeqKKVU66AJzKGuoG+hrgVTSqlWQROYQ11BX70DU0qp1kETmIPzkSp6B6aUUq2CJjAHm8UEs8mgkziUUqqV0ATmYK+HqOWklFKqtdAEVk+ITatxKKVUa6EJrJ5gq0W7EJVSqpXQBFZPqNWs5aSUUqqV0ARWT7AW9FVKqVZDE1g9wTYzSitrUFap9RCVUsrTaQKrJ9RqLyela8GUUsrzaQKrx7mYWafSK6WUx9MEVk9IXTkpHQdTSimPpwmsnhBHF2KhzkRUSimPpwmsHr0DU0qp1kMTWD2+ZiMsWg9RKaVaBU1g9YgIQm0WncShlFKtgCawRoKtZp1Gr5RSrYAmsEa0oK9SSrUOmsAaCdZHqiilVKugCayREO1CVEqpVsGtCUxEBovINhHJFZEpTezvKyIbRKRaRIY3sd9fRPJF5GV3xllfiM2C8qpalFZWX6hLKqWUOgduS2AiYgTwCoBfAugI4BYR6djosL0AxgB4/zSneRLAUnfF2BQtJ6WUUq1DswlMRIwi8uw5nrsHgFySO0lWApgF4Lr6B5DcTXIzgNomrt0VQASAz87x+uck1LGYWdeCKaWUZ2s2gZGsAdBVROQczh0DYF+9z3mObWckIgYAzwOYdA7X/VmC6yrSazkppZTyaCYXjvkGwDwRmQ2gpG4jyTlnaNdU0qOLcY0HsIDkvuZyp4iMAzAOAOLj4108dfNCrHoHppRSrYErCSwYQBGAq+ptI4AzJbA8AHH1PscC2O9iXJcB6CMi4wHYAJhF5CTJBhNBSL4B4A0A6Natm6vJsVl19RB1DEwppTzbGRMYybHneO61AFJFpA2AfAAjAfzKlYYkf133XkTGAOjWOHm5i6/ZBB8vI47oVHqllPJoZ5yFKCKxIjJXRA6JyEER+beIxJ6pHclqAPcAWATgewAfktwqIk+IyFDHubuLSB6AEQBeF5GtP+/bOT90MbNSSnk+V7oQZ8A+zX2E4/Mox7YBZ2pIcgGABY22PVbv/VrYuxabO8c/AfzThTjPmxCbWcfAlFLKw7myDiyM5AyS1Y7XPwGEuTmuFhVi1XqISinl6VxJYIUiMsqxJswoIqNgn9Rx0Qq2WnQavVJKeThXEthtAG4CcABAAYDhjm0XrVBHFyJ5XiY2KqWUcoNmx8Ac5aBuJDn0AsXjEYKtZlRU16KksgY2iyvDhEoppS40VypxXNfcMRejEJu9GscRnYmolFIey5XbixWOavD/QsNKHBvcFlULq6vGUVhSgfgQ3xaORimlVFNcSWCXO74+UW8b0bAyx0WlrhqH3oEppZTnOtMYmAHAqyQ/vEDxeATnI1W0GodSSnmsM42B1cJeTeOSElJXkV7XgimllMdyZRr9YhF5UETiRCS47uX2yFqQj9kIX7NRy0kppZQHc2UMrG7N1931thFA0vkPx3MEazUOpZTyaK5Uo29zIQLxNCE2i3YhKqWUBzttF6KIPFTv/YhG+552Z1CeIMRq1nJSSinlwZobAxtZ7/3DjfYNdkMsHkUL+iqllGdrLoHJad439fmiE2yzPxNM6yEqpZRnai6B8TTvm/p80Qm1WlBZU4uTFdUtHYpSSqkmNDeJI0tEjsN+t+XjeA/HZ2+3R9bCnIuZT1bCz9urhaNRSinV2GkTGEnjhQzE09SVkyoqqURiqLWFo1FKKdWYKwuZL0nOahw6E1EppTySJrDTcBb01ZmISinlkTSBncZPBX01gSmllCfSBHYa3l5GWLUeolJKeawzJjARuUFEdojIMRE5LiIn6s1IvKjZy0npGJhSSnkiV4r5/gXAtSS/d3cwnkYL+iqllOdypQvx4KWYvAAg1FGNQymllOdx5Q5snYj8C8B/ADj700jOcVtUHiLYasaW/GMtHYZSSqkmuJLA/AGUAhhYbxsBXPQJLMRmwZESez1EkYu+/KNSSrUqrjwPbOyFCMQThVjNqKohjpdXI8BHy0kppZQncWUWYqyIzBWRQyJyUET+LSKxFyK4lqaLmZVSynO5MoljBoD5AKIBxAD42LHtohes5aSUUspjuZLAwkjOIFnteP0TQJib4/IIIVqNQymlPJYrCaxQREaJiNHxGgWgyJWTi8hgEdkmIrkiMqWJ/X1FZIOIVIvI8HrbE0RkvYhsFJGtInKn69/S+eOsSK9T6ZVSyuO4ksBuA3ATgAMACgAMd2xrlogYAbwC4JcAOgK4RUQ6NjpsL4AxAN5vtL0AwOUkOwHoCWCKiES7EOt5VVcP8YhW41BKKY/jyizEvQCGnsO5ewDIJbkTAERkFoDrAHxX79y7HftqG12z/i2PBS1Us9FiMsLPYkKh3oEppZTHOW0CE5GHSP5FRF6Cfd1XAyTvO8O5YwDsq/c5D/a7KZeISByA/wJIATCJ5P4mjhkHYBwAxMfHu3rqsxJs03JSSinliZq7A6srH7XuHM/d1MrfUxLh6ZDcByDT0XX4HxH5iOTBRse8AeANAOjWrZvL5z4bIVoPUSmlPNJpExjJjx1vS0nOrr9PREa4cO48AHH1PscCOOUu6kxI7heRrQD6APjobNv/XMFWC/KKSy/0ZZVSSp2BK2NLD7u4rbG1AFJFpI2ImAGMhH092Rk5Fk/7ON4HAegNYJsrbc+3UO1CVEopj9TcGNgvAVwNIEZE/l5vlz+A6jOdmGS1iNwDYBEAI4DpJLeKyBMA1pGcLyLdAcwFEATgWhH5I8k0AB0APC8ihL0r8jmSW87xe/xZ6h6povUQlVLKszQ3BrYf9vGvoQDW19t+AsAEV05OcgGABY22PVbv/VrYuxYbt1sMINOVa7hbiM2C6lrieFk1Any1HqJSSnmK5sbANgHYJCLvk6y6gDF5lLpqHIUlFZrAlFLKg7gyBpYoIh+JyHcisrPu5fbIPMRPi5l1HEwppTyJq8V8X4V93OtKAG8DeMedQXmSn8pJaTUOpZTyJK4kMB+SXwAQkntIPg7gKveG5TlC6irS6x2YUkp5FFeeyFwuIgYAOxyzCvMBhLs3LM9R14WoBX2VUsqzuHIH9gAAXwD3AegKYBSAHHcG5UnMJgP8vE06BqaUUh7GlWK+ax1vTwIY695wPFOozYJCHQNTSimPcsY7MBFZLCKB9T4Hicgi94blWYK1HqJSSnkcV7oQQ0kerftAshiX0BgYoAV9lVLKE7mSwGpFxPmsEhFJwFlUlb8YhNjM+kwwpZTyMK7MQnwEwHIRWer43BeOZ3BdKkKsFhSXVqK2ljAYtB6iUkp5AlcmcSwUkS4AesFeWHcCyUK3R+ZBgq1m1NQSx8qqEOSYVq+UUqplnbYLUUTaO752ARAPe3HffADxjm2XDGc1Dh0HU0opj9HcHdhvYe8qfL6JfcSlWI3jZAVSwm0tHI1SSimg+QS22PH1dpKXTPHepmhBX6WU8jzNzUKse+ryRxciEE8Waqt7pIomMKWU8hTN3YEVicgSAG1EZH7jnSSHui8sz1I3ceOITqVXSimP0VwCGwKgC+yPTmlqHOyS4WU0IMDHC0UlWk5KKaU8RXNPZK4E8LWIXE7y8AWMySOFWM06C1EppTzIaROYiLxI8gEA00XklMobl1IXImCfSq9diEop5Tma60Kse+rycxciEE8XbDVjV2FJS4ehlFLKobkuxPWOr3UlpCAiQQDiSG6+ALF5lBCbBev3FLd0GEoppRxceZzKVyLiLyLBADYBmCEiL7g/NM9SV5G+tvaSqmOslFIey5Vq9AEkjwO4AcAMkl0B9HdvWJ4n2GpGLYGjZVUtHYpSSim4lsBMIhIF4CYAn7g5Ho8VYvupnJRSSqmW50oCewLAIgC5JNeKSBKAHe4Ny/OEWLWgr1JKeRJXHqcyG8Dsep93ArjRnUF5ImdFep1Kr5RSHsGVSRx/cUzi8BKRL0SkUERGXYjgPMlPBX21C1EppTyBK12IAx2TOK4BkAegLYBJbo3KAwX7Ogr66h2YUkp5BFcSmJfj69UAPiB5xI3xeCyT0YBAXy99pIpSSnmIM46BAfhYRH4AUAZgvIiEASh3b1ieyV4PUbsQlVLKE5zxDozkFACXAehGsgpACYDrXDm5iAwWkW0ikisiU5rY31dENohItYgMr7e9k4isEpGtIrJZRG52/VtynxCrRSdxKKWUh3DlDgwAYgAMEBHvetvebq6BiBgBvAJgAOxjZ2tFZD7J7+odthfAGAAPNmpeCuBWkjtEJBrAehFZRPKoi/G6RYjNjNxDJ1syBKWUUg5nTGAi8gcAVwDoCGABgF8CWI4zJDAAPWBfO7bTcZ5ZsN+5ORMYyd2OfbX1G5LcXu/9fhE5BCAMQIsmsGB9pIpSSnkMVyZxDAfQD8ABkmMBZAGwuNAuBsC+ep/zHNvOioj0AGAG8GMT+8aJyDoRWXf4sPsfWRZiNaO4tBI1Wg9RKaVanCsJrIxkLYBqEfEHcAhAkgvtpIltZ/WX31HC6h0AYx0xNDwZ+QbJbiS7hYWFnc2pz0mIzQISKC7VuzCllGppriSwdSISCOAfANYD2ABgjQvt8gDE1fscC2C/q4E5kuV/ATxK8mtX27nTT4uZNYEppVRLc6WU1HjH29dEZCEAfxefB7YWQKqItAGQD2AkgF+5EpSImAHMBfC2o5SVR6grJ1V4sgJtI/xaOBqllLq0nTaBiUiX5vaR3NDciUlWi8g9sBcCNgKYTnKriDwBYB3J+SLSHfZEFQTgWhH5I8k02Cvf9wUQIiJjHKccQ3Lj2Xxz51uI1T70p3dgSinV8pq7A3u+mX0EcNWZTk5yAewzF+tve6ze+7Wwdy02bvcugHfPdP4LTQv6KqWU5zhtAiN55YUMpDUI8jVDRB+popRSnsCVavR3OyZx1H0OEpHxzbW5WBkNgiBfsz7UUimlPIArsxDvqF8Bg2QxgDvcF5JnC7aadQxMKaU8gCsJzCAizjVdjhJRZveF5Nm0GodSSnkGVxLYIgAfikg/EbkKwAcAFro3LM8VatMuRKWU8gSuFPOdDGAcgLtgr67xGYA33RmUJ9MuRKWU8gyuLGSuBfAa7AuZgwHEkqxxe2QeKsRqQXFpFapramEyunIDq5RSyh1cmYX4lYj4O5LXRgAzROQF94fmmerWghWXVrVwJEopdWlz5RYigORxADcAmEGyK4D+7g3Lc9VV49AnMyulVMtyJYGZHFXhbwLwiZvj8XjOgr5ajUMppVqUKwnsCdhnIuaSXCsiSQB2uDcszxVaV9BXJ3IopVSLOmMCIzmbZGZdVXqSO0ne6P7QPNNPd2AVKCsrQ3Z2NmpqarBnzx507doVnTp1QlpaGl577bUznuvIkSMYMGAAUlNTMWDAABQXFzd53ODBgxEYGIhrrrmmwfaXX34ZKSkpEBEUFhae0m7t2rUwGo346KOPAACHDx/G4MGDz/ZbVkopj3TaBCYiDzm+viQif2/8unAhepZAXzMMjnqI06dPxw033ACj0YioqCisXLkSGzduxOrVqzF16lTs39/848+mTp2Kfv36YceOHejXrx+mTp3a5HGTJk3CO++8c8r23r174/PPP0dCQsIp+2pqajB58mQMGjTIuS0sLAxRUVFYsWLFWX7XSinleZq7A/ve8XUd7A+ybPy6JDnrIZZU4r333sN1110HADCbzbBY7BM8KioqUFt7ygOkTzFv3jzk5OQAAHJycvCf//ynyeP69esHP79Tnz/WuXNnJCYmNtnmpZdewo033ojw8PAG24cNG4b33nvvjLEppZSna64a/ceOrzMvXDitQ7DVjMNHS7Bz584GCWTfvn0YMmQIcnNz8eyzzyI6OrrZ8xw8eBBRUVEAgKioKBw6dOi8xJefn4+5c+fiyy+/xNq1axvs69atGx599NHzch2llGpJzT3Qcn5zDUkOPf/htA4hNjMKDh5EYGBgg+1xcXHYvHkz9u/fj2HDhmH48OGIiIi44PE98MAD+POf/wyj0XjKvvDw8DN2bSqlVGvQXCWOywDsg7324WrYy0gp2NeC5R80oLy8vMn90dHRSEtLw7JlyzB8+PDTniciIgIFBQWIiopCQUHBKd1952rdunUYOXIkAKCwsBALFiyAyWTCsGHDUF5eDh8fn/NyHaWUaknNjYFFAvgdgHQAfwMwAEAhyaUkl16I4DxViM2M47UW1NTUOJNYXl4eysrKAADFxcVYsWIF2rVrBwC49dZbsWbNmlPOM3ToUMycae+hnTlzpnM87efatWsXdu/ejd27d2P48OGYNm0ahg0bBgDYvn070tPTz8t1lFKqJZ02gZGsIbmQZA6AXgByAXwlIvdesOg8VLDVjGNlVeg/YACWL18OAPj+++/Rs2dPZGVlITs7Gw8++CAyMjIAAJs3b3aOddU3ZcoULF68GKmpqVi8eDGmTJkCwH4H9Zvf/MZ5XJ8+fTBixAh88cUXiI2NxaJFiwAAf//73xEbG4u8vDxkZmY2aHM6S5YswZAhQ372z0AppVqakDz9ThELgCEAbgGQCGA+gOkk8y9IdGehW7duXLdu3QW51vur9+J3c7fglYFBmDXj1SanuNc5fvw4br/9dsyePfuCxHYmffv2xbx58xAUFNTSoSilPICIrCfZraXjOBfNTeKYCXv34acA/kjy2wsWlYe7PDkEAHDAHIUrr7wSNTU1TU6YAAB/f3+PSV6HDx/Gb3/7W01eSqmLwmnvwESkFkCJ42P9gwQASfq7ObazciHvwABg8Iv/g7+3Fz6887ILdk2llDrfWvMdWHNjYAaSfo6Xf72Xn6clr5YwOD0Sa/ccwaETTc9EVEop5V76RMZzNDg9EiSw+LuDLR2KUkpdkjSBnaN2EX5oE2rFwm8PtHQoSil1SdIEdo5EBIPSIrHqxyIc06czK6XUBacJ7GcYnB6J6lriix+0G1EppS40TWA/Q2ZMAKICvLUbUSmlWoAmsJ/BYLB3Iy7dfhilldUtHY5SSl1SNIH9TIPSIlFRXYul2w63dChKKXVJcWsCE5HBIrJNRHJFZEoT+/uKyAYRqRaR4Y32LRSRoyLyiTtj/Lm6JwYh2GrGwq3ajaiUUheS2xKYiBgBvALglwA6ArhFRDo2OmwvgDEA3m/iFM8CGO2u+M4Xk9GAAR0i8OX3h1BRXdPS4Sil1CXDnXdgPQDkktxJshLALAANnhdCcjfJzQBqGzcm+QWAE26M77wZnB6JExXVWJlb1NKhKKXUJcOdCSwG9gdi1slzbLvoXJ4SAj+LSWcjKqXUBeTOBNbUE5xP/+yWc7mAyDgRWSci6w4fbrlJFBaTEVd1CMfi7w+iuuaUm0mllFJu4M4Elgcgrt7nWAD7z+cFSL5BshvJbmFhYefz1GdtcFokjpRUYu3u4haNQynlXmVlZcjOzkZNzU9j3sePH0dMTAzuuece57bBgwcjKysLaWlp+M1vfoO+ffs224YkYmNjYTab4ePjg3bt2uHQoUOnXP+RRx5BXFwcbDYbAOCZZ55BSkoK2rZti+zsbKSkpKBnz57YvXs3AKCqqgo9e/aExWKBiODRRx8FAFRWVqJPnz4AEOeYaLdZRLrUXUdE/iIiW0XkexH5u4iIY/vnIuIRz2RyZwJbCyBVRNqIiBnASNgfiHlRym4XBovJgEU6G1Gpi9r06dNxww03NHgG4O9//3tkZ2c3OO7DDz/Epk2b8O2332LdunVo06ZNs20+/fRTlJaWYsWKFViyZAkCAwMRHh5+yvWvvfZarFmzBgDw3XffYdasWdi6dStGjRqFjRs3Ytu2bZgwYQImT54MAJg9ezYCAwOxbds2xMfHY+bMmdi9ezfMZjPi4+MBwA9AKoBxAF4FABG5HEBvAJmwPxeyO4C6YN8BMP5n/AjPG7clMJLVAO4BsAjA9wA+JLlVRJ4QkaEAICLdRSQPwAgAr4vI1rr2IrIMwGwA/UQkT0QGuSvW88HXbEJ22zAs/PYAamvPa0+pUsqDvPfee7juup/mo61fvx4HDx7EwIEDGxzn729/6lR1dTX27t2Lbt26Ndtm3rx5iIyMhIigV69eOHr0KAoKCk65fq9evRAVFeVsM3LkSFgsFqxYsQLt27fHmjVrMHz4cHzxxRcgCRGB2WxGbGwsSMJsNjtjq6ioAOzPdyTJrwEEikgU7MM93gDMACwAvADU1cybD+CWn/EjPG/cug6M5AKSbUkmk/yTY9tjJOc73q8lGUvSSjKEZFq9tn1IhpH0cRyzyJ2xng+D0yNx4Hg5Nucfa+lQlFJuUFlZiZ07dyIxMREAUFtbi4kTJ+LZZ59t8vhBgwYhLCwMZWVlGD9+fLNt8vPzYbFYMHbsWHTq1AlVVVXIy8trNp78/HzExcU537dp0wb5+fkwmUwICAhAUVERhg8fDqvViqioKOTl5eHuu+9GcHAwAHt3KOwJqk4egBiSqwAsAVDgeC0i+T0AkCwGYBGREJd/cO5C8qJ4de3alS3taEklkx/+L59Z8H1Lh6I8UGlpKfv27cvq6mp+88037NWrFzt27MiMjAzOmjXLeVxOTg4TExOZlZXFjIwMdunS5Yxt2rZtS7PZTC8vLxqNRg4dOvSU4x944AF27NiRIsJp06YxISGBZrOZNpuNoaGhTEpKYps2bZiSksKsrCympKQwODiYFouFAGgwGGgwGCgiHDBgACMjI09pExkZSZPJRABMTU2liNBoNNLLy4smk4mRkZGnXCMxULLiAAAgAElEQVQwMJBms9l5DR8fH5rNZhoMBvr7+zMpKYnt2rVj+/btmZWVxYSEBPr5+dHLy4siQgA0mUy0WCw0Go20WCxMSkpiVlYW09PTmZWVxYCAABqNRgJgWloaAdDLy4teXl40GAxMTExkRkYG169fz2PHjjEkJIRWq7VBXL6+vs73/fr1Y3JyMqOionjfffeRJB966KEGbYxGIyMjI5mYmEij0dhkm/79+9NgMBAAbTYbTSYT586dywMHDhAA/f39T4krJCSEWVlZzu89NDSUWVlZBECLxcJ//etfJMmkpCTu2rWLoaGhTE1NZWVlJePi4mi1Wunl5UWz2czIyEgCqALgRxIAvgDQFcDjAEpgvxNLALAH9uS2DcB/AJQB2A6gi6PdlY79FY425QCGOfbNAjATQC6AzY3abKz3atwmlWf4u9/iied8vTwhgZHkqDe/ZvZfvmRtbW1Lh6I8zMsvv8wXX3yRJLlt2zZu376dJJmfn8/IyEgWFxeTtCew2bNnu9ymqKiIbdq0YVFREf/yl7/Qy8uL06ZNO+X40NBQrlmzhn379mVYWBhjY2NZVFTEXr16UURYUFDAH374gW3atGF1dTVPnDjB6OhoLl26lJdddhlFhIsXL6bVaqWfnx/NZjPLy8sbtFm7di2jo6PZq1cvfvDBBzQajdy2bRvHjx/vTB6NrxEVFcWvv/6aCQkJNBqN7NKlC2trazl48GAaDAbW1tbyf//7H7t160aSPHHiBCMjI7lhwwb6+PjQaDTyyJEj3LlzJ0WEgYGBJMl33nmHN954I0ly+fLljImJYUxMDF955RUC4L59+/jrX/+aIsKcnByuWrWKPXr04H333cdbbrmFN910E3ft2sW4uDj6+PiwXbt29PX1pb+/P41GI6uqqjhw4ECazWYmJCQ4k96dd97JyMhIGgwGHjlyhN988w1FhFarlbW1tQ3aBAYG0maz0d/fnzfffDOtVitLSkp411130WAwMDU19ZS47r77bpKk1Wqlt7c3H3/8cZKkl5cXvb29uXLlSlZVVTEkJIT33nsvU1NT2b9/f5JkRESE8x8wO3furPvHRg0AL9oTxzYAUQD+CuB5ALsBXA7gAICHAYwFUApgPYDhAFazLpkAnQEkwl6gohiAr2P7Q7AvqRIAveq3qdc2GMCRem2yAfyj8XGNX1oL8TwbnB6J3UWl2HawVazBVhdQ/bGTtm3bIjU1FQAQHR2N8PBwNLUUxJU2ixYtwoABAxAcHIyPPvoIAGA2m085Pjo6GoGBgaiqqkJNTQ2uvvpqBAcHY+fOnbDZbFiyZAnatWuH1NRUrFmzBhUVFTAajUhJSUFeXh6CgoLQv39/APZuMIvFguLi4gZt6k9UeP/99xETE4O2bdvi448/ho+PD44cOYKAgIAG1zCZTIiLi0N5eTkMBgNuuOEGiAi++eYbiAg2b96MPn364Pjx4ygoKEBFRQW8vLwQERGByspKpKenIygoCNXV1SCJ2tpaVFdXY+TIkfjqq69AEu3bt4fBYP9z99prryEwMBCxsbH4/PPPERYWhjlz5qBXr144cOAAdu/ejYEDByIsLAyJiYkoLS1FUlIScnJyICKoqamBiGDVqlVYtGgREhMTsWrVKjz33HMYM2YMXn31VRw7dgwpKSkICgpCeno6APtYWE1NDa699loEBARg1apV+Otf/4qcnBwEBQXB19cXNpsNPj4+mDVrFiwWC4qKitCtWzfs3LkTn376aYMxs+rqalx11VWYM2cOtm7ditraWpSXl6NLly746KOP0LlzZxw6dAhZWVnIy8sDSZw8eRImkwm9e/dGmzZtEBkZCdgTS7WI9AJwjGQBgK8BZDkuNQTASQDfwp6AdgCIh/1OrG7MDCS/IbkbgC+AL0iWOtonw95NaWTDcbb6hgP4tF6bZQD6i4ipmf+lNIGdbwM6RkAEuqhZNdB47KS+NWvWoLKyEsnJyc5tjzzyCNLT07Fp0ybngP3p2tSNg1RWVmLbtm1ITU1FcXHxaY/38vJCVVUVDAYDKisrceTIEYgI8vPzAQCBgYEYOXIk4uLiMHnyZERHR+Po0aPo2NFeCY4kKisrERYWhhUrVjTZxmKxYMuWLbjiiiswYMAA7Nu3DzabDR06dMCKFSuavEZJSQlMJhOmT5+OzMxMFBYWIiAgAAsXLgQAhIaGom/fvg3a1NbWol+/fgCAd955B4A9WW/atAkmkwlWqxXp6enONiaTCbt27UJtbS22b9+O4uJi58/h8OHDOH78OO64444GP7uSkhJER0c7x5rqks3KlSsBALGxsc6fXZ2KigoUFxcjMzMTWVlZMBqNCAoKwqZNm/DDDz8gLCzslDbfffcdysrKnP8wuPzyy9G5c2d88803KCoqQq9evQAAK1euRGxsLCoqKrB69WqEhITgF7/4BWpraxEQEICUlBQ8//zzWLt2LSZOnIh+/frh8OHDMJvNKCkpQVVVFf70pz+hurraOYkD9u69f+Cn2YUfAfgRQLRj23aSH8NejOI4gB9pn6jXVIEKK4A59T7HwN4FWZcQm2ozEsAHdR9I1jpiykIzNIGdZ+F+3uieEKwJ7DxpvOZm8ODBCAwMxDXXXNPguD59+qBTp07o1KkToqKiEBoaesY2kydPhre3N8xmMwICAlBRUYGNGzfisssuQ1paGjIzMzFhwgSkpaXBYDDgoYceQkREBEQEiYmJCAsLQ3JyMhISEtChQwdnm7vuugv+/v4QEef5bTYbCgsLkZOTg5SUFLRr1w6LFi1CQUEBhg0bhrKyMhiNRiQmJuLjjz/GyZMnUVJSgvLycoSFhTmPB4CCggIMGTIEu3fvhtFoxNSpU7F3714UFhaipKQEx44dw9SpU5GZmYkNGzagoKAAo0ePxjPPPIPBgwdj7dq1AICPP/4YPXr0QG1tLUj7bDUAsNlseOGFF5Cbm4uZM2di3759KCkpQVJSEgB7AjMajfD29sb+/fubbFNRUYH9+/ejR48emDlzJgICAkDSub2pa5SWlqJjWjqm/u01/HnaDHuirKrCltw8LNtxGGW1Btz31Mt4df5yvPjqP/Ds+4tAEse8I/GP/+3E69PfgYigHGbM+GwD/vG/nSitBn4z9W388d0v8NzLb6Do2EnUwIAeg4aj35DrUVVVjUqYQDFi+N2PQLz9MOeHEny4di+W7yjEmFcWobyiArtKzXj1q1xUVNeiwjsYJ8sqMPOrrfj1m19j6/5j+P28b7G4uj2KO43CDc99gtraWsT3G42M+95A14kzYPLxw5HjJfjDB8uwYNVmnIQvXlv6I/aEdEfM1XfjaEkFtnz3A5786Gs8NesrmL190GXYOJwQKyY++QLM3j4I6dQfS7cdQkBiOu5/6SPYAoLx5IcrcMvjb8LHLwjx8fEYMmQIpk2bhpycHPzud79D9+7d4e3tjZEjR6Kqqgrjx4/HhAkTsHv3bvTp0wfl5eUAUEH7RLsMkuscv+Makv8H+9rdOfipXq3AvqZ3Yb3/lZzTrh13VmYAX9bbL7B3KUY30yYD9hnr9R1q1OYUzd6eqXMzKD0ST37yHXYXliAx1NrS4bRqjdfcTJo0CaWlpXj99dcbHLds2TLn+06dOiE9Pb3ZNoWFhXj++eexcuVKLF26FH/4wx/w+uuvY/DgwXj77beRmpqK/fv3IysrCwsWLMCkSZOQlJQEs9mMuLg4XH311XjjjTewZcsWbN26FWPHjsWuXbtw8OBBJCUlYeLEifjqq6+wc+dO57/Uk5OT8cEHH+DEiRPYv38/rrrqKgQFBWHSpElYunQpLBYLnnzySTz11FNYvHgxli9fjuHDh8NsNmPhwoXo378/1q1bhyFDhiA+Ph4jRozAa6+9hoceeghr1qxBWVkZampqcPjwYcyYMQNJSUkYN24camtr8dRTT+HFF1/EI488gsrKSvTt2xf5+fl47rnnEBMTg4qKCkRH2/9W7NuXh+CwcHj5hSAxpR0en/o8zBYLdhUU4rOtB1BLoKamBsdLK/DjkUos2noAG3/4EelXWWArNiAwJgnffL0M/kEhWLcjHyE7j6KqlqitJn7YtQ+ffl+Eb9d+h31hvTA9/0fsqQ1Gl5ETUSsm/FDmh/FvLIZ3UlfUihEnS0rw39xSLH9rDfK378Kflx2CyVaNQgnDH595FhDBv5d/iwUF3ig6WQ4SyD9yErM3HcLHxd/iSPFRvLi8ACKCQlM4Sk7+AEt0O6w7WA3bkMeAaWNw7GQZCAM2bNyMk4f2490n7wGrK8GaauzZuQNGL2/AGoKjhw+AAGyRbVC0+weYbIGoqKpFyZFDMFiDUVFlr8Lz4+rPIUYTSo4WobyqFjW1NYDRBMCAvUerEH3T41j/XA5WHyRWH89DZU0tTpw8CZ/k7njhi52oKT+JqhpixmfrULq3EDXlJ1FRUYGXHp/oiKsKqzZshldid/z5s1xUHtqJY6UVCLN5oby8HD4+Pli1ahWWLVuGadOm4eTJk6isrITNZkNsbCwAYPXq1aisrESbNm1w4sSJsjP8b7gfPxWlyIN9bVhdsmlcoOIm2MfI6j8gMQ9AJ9gnfpyuzVySVY2u612vTdPONEjWWl6eMomDJPcdKWHC5E/42le5LR1Kq3fZZZdx165dDbYtWbKEQ4YMafL448eP02g0cvPmzc22ee211+jv7++8Rr9+/ZiVlXXK+TIzM7l9+3ZmZ2fzqaee4rhx45iQkMCYmBhmZmby/fffJ0kOHDiQK1eu5LFjx+jl5cVt27axQ4cOvP7660mStbW1NBgMtFgsLCgoYEVFBYODg3n//ffz2LFjTExMZO8+fTky5zccfvu9fHPZTo64cxJFhDb/AH65fhszO3dhUHAwn376aSYmJrK2tpYJCQncvn07ExMT+fzzz9NoNNJoNPK7775jaVk5LRYL23VM59sLlrN9p+78x/9+ZJuM7hwy4VnaQqM4+Jl5FKN91mD6xHcZOeR+wmRh/KR5jL1/Fk1B0fRO7EJjYBSNtmDG3j+L4mUhxECYLIy4ZSoD+t5KmH0atBGLjdaMATQFxzJ+4lwafPztM/9C4hh/3QM0+vgxc8T9/NXLnzMwMp7JnXvTxy+Aox98ip0uv4Lz1u2m0TGb8bUPFzDnrvsZEhbOb/OPcu0Pe5iYlML2HdNoNpuZlp7Bex+YwDvH302DwcCAgADu2LWXY8bexpTUVJ4or+LegkNMTkmhwWDgP995j+07dGDRsZMMCAx0zuRbtmwZU1JSOGXKFM6YMYN33303e/bsyYiICC5btoyZmZm0Wq188803KSLcu3cvn3nmGUZGRjb4b6Znz54MCQlhWloay8vLuXPnTkZERNBqtTI/P5+rVq1i9+7dG7Qxm8386N9zWFZZzRPl9gkYV/Xrzw4dO/LDOf9h127deKK8iq++8SbH3XkXu/Xowf8uXMwT5VWc8OAkPvzIo4yNjWXHjh25f/9+zpkzh1OmTCFJ5/dCkt9++y07duzI8vJyfvPNN3UzJrfQPnHiegDPsOHkit2wL2jeBPs41m2OBGWEfTxsTaPjv4Z9wkdovW1DYC/MHtVMmyvZ6G86gC0Aohpvb3BMcztb08uTEhhJXvP3ZRz2yvKWDqNVq6ioYERExCnbm0tgb731Fi0Wyxnb/OUvf2FAQABXrlzJiIgI9urV65Q/RKtXr2b79u1ZU1PD7Oxs3nfffXzyyScZHx9PHx8f3njjjXz22WdJkrfddhtnz57Nd955h97e3rz11ltpNpvZvn17jho1iuHh4fTy8mJoeDifenk67/7jXyki9AuNojU8juJloXjbKBYrxWKlV0g8Dd42wuhFr9AEhg57mKaQOPuU8eAYislCk48fYTAysedA9h0zhUYvMyFC75AoJo38PcOumUgAFC9vGgPCKSYz4UhWMBgpRi+K0USDyYtmXxv9gsPp7RdAg9HEsJhExiW341XXjaSI0OSYcg57149jCrqZsfGJNHl50WqzsW2HNEbFxNHPL4AAGBAUTLPZYp/ubjAwJiaGJpOJMTExNBqNjImJYUZGBm+88UaKCA0Gg3M6fN00b29vbyYlJTEoKIhGo5GZmZnMyMhgnz59nFP6/f39nVPh+/TpQ5vNxuTkZHp5eTE4OJiZmZmMioqizWYjAEZFRTEoKMg5hT4mJoZWq5Xp6elMSEhwJrDu3bvTYDDQaDQ6p57DMT2+bqp+ZGQk4+PjmZ+fT5J87LHHnG1sNhv9/PzYtm1bvvvuuwwPD2dSUhJtNhsXLFhAkvzb3/5WN5WdUVFRvP3220mSY8eO5cCBA2mxWJiens61a9eStCej0aNHMzo6mjU1NSTJNm3a8Pvvv+ctt9zCtm3bkiSfffZZPv300842dQmMJDMyMpw/38mTJxPAOtoTxvcAnna8v89x51TtuFtaB/uY2DYAHzvebwGwAkC0o83jsM9orGvzpmN7BOyLoOvadONPSSoRQD4AA+v9PXe0aZDomnq1eOI5Xy9PS2Avf7mDCZM/YcHRspYOxW3qr2siyUGDBjEgIOCU5HLbbbcxMzOTaWlpDA0N5dGjR5s9Picnx/kHRkT4yiuvNNhfl4xeeuklJicnEwBfeuklpqSk0Nvbm4GBgbz33nuZnJzsXEOzZMkSdunSxfkHODQ0lPHx8ezevTu9vb3p6+tLo9HItm3bcuHChdy1dx+TU1IZFh7B5HYd6O3jS/+gYF7/m98yMDSSRpMXE9O6cvBtk/iHed8yq9/1HPh/j9EvNJIQA4f9YQb9wmIYEJPEDleP4RV//pzibSOMZgb1/z8mTP6EfpkD2GH04/zlozMoBiPjUjvyFwOv4eXZV/HByQ9zzG/+jx0zshgRFcPxDz/FuOR2HDH+d7z/pdkUg5E3PTmT1pAoJlx+LRP6jeYv/vwFg5MyaQuLZv/bJ/O5RT8wOCKaD02dxqdfnk6bnz83bv2BlZWVvOGGG/jmm2+SJDds2MDExETOnz+fDz74IDdt2nTG3/1zzz3H9PR0kuSvf/1rHjp0qNnjN2zYwOzsbD766KMkySFDhrCiouK8XoMkf/vb37JXr15nFVe7du34+eefu9zmhRdecP7sXI2rfhtXbNiwgV27dj2rNmf7vdepl8DeBRDG8/y3GcAEALe7o02LJ57z9fK0BLbj4AkmTP6EM1fuaulQ3Kb+GiWS/Pzzzzl//vxTEtKxY8ecx2dnZ/OZZ55p9vjrrruO8fHxLCgoYExMDJOSkpxJkvwpgW3YsIHfbdvBiOhohkbG8N5XF9Di60cRA+PTe/LBD7/hr//0NqNS0nnz4/9gXMeuNHlZGNexK38x8m5m9r+R49/6H30Dw2jwstDLGsCsie/QFBBOr/Akhl43hUb/cMbe+x4tcekM6DOatqzBNPiFUsw+FLMvo4dNZsYfFtIvqRO9Q2OZcsNv6R0UwUF/XUprRAJDkrMYmX4Z73p3HcNi29Bk8uKkJ5/j3qISDhgwgCtXrmRBQQETEhKYnZ3N8ePHc9y4cbz66qv59NNPMzU1lT169ODrr7/u7KasO54kExISOH/+fF599dUkyXHjxjEjI4Ovv/46SfsC5/3793PVqlXMzs52/gzffvttjh8/3vm5S5cuXLhwocu/+w8//JAvvfRSg9/Lmdx5550sLCx06zWmT5/ON95446za3HrrrWd9jaqqKpePP9c2Y8eOZXl5+Vld42y/d/KnBOauF+xrx0zuaKOzEN0kJdyGlHDbRT0bsXFNuH79+sHPz++U4+rqrr333nuIiYlxznY73fH79u1D7969nXXh2rRpg6+/Xo09RSVY+O0BfLR+H7bkHcOEz4/i6hnbcKj4JMpC2mH2xwvg164XKIJy3zAszy3CprzjOLhvJzZs24cTNSbUiuBoWRUKfRPww9r/Yd2eI6goK0FtdRWMBgOu6JQCY2UJgi3AC5P/D8FWM17+VWekxwTg5QduRvDR7xEd6IMQfytYWYoFj9+C36cdR+meLfjTwxOw49/Po1taKv72ywhkJkWjrGAHhl/RFU/9sg3Kjh6GCHBN3+7474dvY82aNejRowciIyMRFxeH0tJS9O3bF/PmzUPbtm3RpUsX7NmzB8HBwQgNDcWOHTsaHL9t2zYA9gksdVPchw4digMHDiAjIwNff/01AgICEBUVhe7du6O4uNi51uzLL790tgHs0/wzMzNd/t2PGDEC99xzT4PitGfy6quvIiTE9epD53KNsWPH4o477jirNjNnzjzra5hMZzf/7VzaTJ8+HRaL5cwH1rvG2X7vFwLJGbRPuT//bdyZeS/ky9PuwEjy2YU/MOnh/7LoZPNdJa3R2Y5PjR49mgaDgVdccQVLSkqaPL6iqob7j5YyMTmVwaHhjE1ux9C4ZIa178aYG3/HhMmf0BLbkQYffxpMZlqDwnnnM2/R5h/Aux+YyOzsbH766af09/fntddeS5KcPXs2zWYzAwMD6e3tTQD09vZmWloaO3Xq5BzX6NChA/v3788nnniCAOjr68usrCwajUaaTCaKCC0Wi7NMkojQ29ubERERDAgIcJZOysrKYnR0tPO4umO9vLzYo0cPBgcHMykpiRaLxfl9z5kzh+Hh4faxJpOJvr6+TExMZNu2bTlq1Ch6eXmxbdu27Natm3OsZdKkSc7YLRYLR40a5fy9BAYGMikpqcHYCUl+9tlnzMjIYHp6OnNycpxdeAcOHDhlUoG6dMDNd2DufLV4AOfr5YkJbEveUSZM/oT/WrPXua3xuJHBYGBWVhazsrKcf3SbU15ezptuuonJycns0aPHKTP06owdO5ZhYWFMS0trsL2oqIj9+/dnSkoK+/fvzyNHjjTYv2bNGhoMBmcpo0OHDnHQoEGnnD8/P5/t2rU7ZXtTCaykooqfrfmOsYnJ7H/jKP7qwWf42H+2cPy763nFhJcZ0qEXMx9fxITJnzBh8if0Te/H4CG/ZfzEufRL7UEvXz+OmPxXzlqzhxv3FrO0omEXSWBgoHPGFUnGxcU5B7MffPBB9ujRg+vWrXPGnZ2dzblz5zIhIYG5ubkcMWIEe/fuzV/96lecP38+09PT+dxzzzmPJ8mDBw8yMzOTS5cuJWkfo0hPT+ff//53l8eN5syZ4xwDcnWMon4bV5zt8eTZj8+oi0trTmC6DsyN0qL9ERvkg4VbD+Cm7vZlFI3XNfn4+GDjxo0un/Ott95CUFAQcnNzMWvWLEyePBn/+te/TjluzJgxuOeee3Drrbc22D516lT069cPU6ZMwdSpUzF16lT8+c9/BmBf2zN58mQMGvTTk2vCwsIQFRWFFStWoHfv3s7tPj4+dYsgG6ioqsGxsir8c8UubMk/ji35R5F76CSqyk6i4MgJVJnTcHzuHCRbOyPMZrGvq7GYcF2naITaLAixmfF5eQ9YzSY8+shgfD8kAEOHDsV913TDL7rHN/kzMZlMzgW1AGC1WhEdHY2amho8++yzaNeunXONU93X2NhYXHHFFfjmm2/QuXNn1NTUoKysDD4+PoiOjsbll1/e4Pjw8HBcf/31WLNmDfr27YvOnTsjOjoaFovltJXIG6uursbEiRMBAO++++5Zt3HH8YC9isbo0aPPqo1SnkDHwNxIRDA4LRLLdxTiRLl9jV7jcaOzNW/ePOTk5ABAg2f+NNa3b1/nIxNO1z4nJwf/+c9/nPteeukl3Hjjjac8RG/YsGF47733GmwLCgpCdU0NVu84gHdW7cak2Zsw+MX/Yew/12L9nmI8/vF3WLr9EKIDvDGynRn/+E02Qq0mXBOQh3FDf4Etjw/CvcnFiD6wAukxAXjiunTc1y8Vv+6ZgJxhA7Fs0TwEWoC3334bRqMRl112GdasWXNKQgYAb29vfPXVVyguLna+AMBgMDQYByouLq4rnYOjR49ixYoV6NixI4YOHYrc3Fxs27YNfn5+zrGmkpISnDhhr2lZUlKCzz77zFnXDrDXGzzbcaPAwECXjz+XNudyjXMZn1HKI7T0LeD5enliFyJJrt1VxITJn3Dexvwmx42MRiO7du3Knj17cu7cuWc8X1paGvft2+f8nJSUxMOHDzd57K5du07pQgwICGjwua5yd15enrNrs3419Lp9ddOZ9xaV8A/zvuUvX/wf/TIHMPzmp5gw+RN2fuIzhqdm0RoQRLPFm1HR0fz0009ZU1PDyy+/nOnp6QwMDORVV13lnJXYpk0b+vr60tvbmzExMc5ZcFdeeSUjIiLo5eVFPz8//vvf/yZpH8+Ki4tzduv97W9/c64pCggIoJ+fH5OTk/nWW29x/PjxzvVDM2fOJEk+/fTTzvEso9HY4GczZcoUWiwWtm3b1rlG58cff2RmZiYzMzPZsWNHPvXUU87jKysr2b59+7OeWaaUp0Er7kJs8QDO18tTE1hNTS27PbWY499d3+S4Ud0f4x9//NE5JtOcjh07npLAGk9Nrhtny83NZVpaGh966CGmpaUxLS2NPj4+DY6tS2DXX389O3XqxOrqao4aNYoBAQHOqgbbt29nYFAwJ83eyNABd9IrKMo+aaHrL+hlNrNd+44NHh/TeJztlVdeYUpKCkNDQ52TI7p27coBAwbw0KFDPHToELOyspzPqqo/8YC0J57k5GQGBQVx2rRpJMmysjJ2797dmVwee+wx5/E333yz8zEirjrbcaBzGWtSyhO15gSmXYhuZjAIBnaMwJJthyAm8ynjRnVjLElJSc4xmebExsZi3759AOzjHceOHTulq7D+ONuJEyewYcMGbNy4EatXr3ZW4QbsRWHruguXLl2KPXv2IDk5GR9++CFKS0sxZcoUxMQn4upRd+JEtWDexv244ep+WPHVEiQkJOC5xybh4SlT8P/tnXt4VNW1wH+LQMAAJQYTJALySECgSFSIkMojQQQMl/JxoSBYRaJWb0tpqZaH4lvro1KvVvuwpVKLgnILtRSRAEpRkUdCQlCEgEGKIGQiyktCmJ1lYWoAABJPSURBVKz7xzkZJiGBQTMkM7N+3zdfztln733WmjmZNWs/1opqgG9pPJyaZyssLKRv377MnDmTdevWkZubS0xMDCtWrGDOnDns3r2b+Ph44uPj6dChA/feey/9+/evpMtHH33E/Pnz+fDDD8nJyeHXv/41Xq+Xxo0bs2rVKvLz88nLy2PZsmV88MEHANx55508+eST5/Q5xcbG+oZWA+GbzDUZhlG7mAE7Dwz97sUcO+Fli8eL1+v1GTH/ORmPx+ObkwGYMWMGixYtOq2vESNGMHfuXAAWLlxIRkZGJeMBlefZSktLGTBggC+1RHJyMg888ADg7H+pqNelSxdyc3PZtWsXF110ET+5azrbotpTdLQRn+Svo31SF9b8Mp0//Gw0vS/vAkBaWhq33HLLaff3n2dz8w0RFxdH27ZtGTVqFNnZ2Rw9erRSu4kTJ1YKyOvf17hx42jcuDEdOnQgKSmJ9evXIyI0a9YMgLKyMsrKynz99evXjxUrVnDyZOBbT851HuibzDUZhlG7mAE7D/Tp2JLvNGnI3z74lPRB1/Luu+8CsHXrVnr16kXPnj1JT09n+vTpPgNWUFDg+/L3Jysri5KSEpKSkpg9ezaPP/44AHv37uX666/35Z2aMWMGffv2pbi4mIcffpgXXngBj8fD4cOH2bBhA8nJyWRnZzN9+vRKuaq27z+M58vDzM0tYX3RQdq1bEqLmMZkjRlOwneaBKTv/v37fTmsjh075jPSAIcOHWLKlClkZmYyZ84cX3mvXr2qNWAVua4q8M+95PV6SUlJISEhgcGDB3P11VcDzuKNpKQk8vPzA5LXMIzQxAzYeaBRVANu69eR1duLWR+Tyqynnqf0pJe0tDQKCgrIz8+noKCArKwsX5uysjL69u0LVM6JtXbtWgoLC2nWrBknTpygW7duLF68mMTERJYuXYrH4yE2NpZXX32Vffv24fV6mTlzJvfccw/t2rXjyJEjjB8/nsLCQlauXElcXBwej4eYZt+h64ARdO3YlrLS4/xowhieG59CUkIzjh07ypIlS0hOTmbw4MG+VX4LFy5k6NCh7Nixg7S0tEoGo3///r4Muf5069aNadOmsXjxYmbNmuUrT0hIqLQUvoKq7eHUcGVUVBR5eXns2bOH9evXs2XLlrP2ZxhG+GAG7DwxeVAy//zJNVx15RUURrUj46lVLNq0h/Ly07+gAV/yQqg8p5Wenk5eXh55eXmsWrWKmJiYSmnGq9ufFR8fz9ixYzl27BhdunQhe8VKdhw4zNqdJfwzfy8z3/iY3cVfUdqhPz99cg4xTaL5r6TGtLggmtLSUlSV4cOHU1hYyKBBg3xeX7t27ViwYAFJSUnMmjWL22+/HYBWrVqRmprKggULaNq0aaVwOHv27CExMZH+/fuzc+dOPB4PgC+PUVX85/z82/sTGxvLwIEDfZl7z9SfYRjhgxmw80iPNi2Yd2sfFv3vvcQ2bcLPF+ST+dy7vLPtQLWeRgU17R1buHAhw4YNIyYmBoDDx8vY+oWXI8dP8LsVW3n8zY+ZuiCXh5/9MwUxPbn8py+yYfsePsjdzKCnVzP0xjvIevB5cj4vo3l0A9bO/hFTMq+kefPmvnm2kpISoqOjmThxIlB571hqaiotWrQAoFGjRhQUFADOPF1ZWRnz5s3j88+dWJAHDx4kJyeH5cuXM2TIEHJzczlx4oQvNt727dsr7bGqYMSIEcyfP5/S0lKKiop8e7SKi4v58ssvAcdDXbFiBZdddpmv3fbt2+nevXvgH45hGKFHXS+DrK1XfV1GXxNeb7ku3rRHr3lipV46bYmO+8Nazdt98LR6NcUcVFUdODBdZ//pFZ29fJuOeuE97TjjX3rptCXatIezPytp5r+09wNLVRo20uYJbbV1cg+d/NzfNT6xnf511WZNGzhYX/1nth4rPamTJk3S7OxsLSoq0q5du2rLli19+ZZERJctW6afffaZdu3a1bePqkmTJnrBBRf44vdV5NPyeDyanp6uDRo00IyMDH322We1U6dO2rJlS01MTNSePXtqnz59NDMz07dk/qabbtIWLVpodHS0JiQk6HXXXefT85FHHtGOHTtW2qOVn5+vKSkp2qNHD+3evbs++OCDvvoW288wAocQXkZf5wLU1ivUDFgFpWVefem9Ir3yoeV66bQleuffNurOA4d91/33jpWXl2vh/sP60ntFesMzb2pUTAttd9dibT99iY54bo0+8eZWXb3tgC5esUZ/MG68b29WTXvH/I1Ebm6u3njjjee0+bmCVatWaVxcnC9OYAWJiYl66NChgN6Hfv36nRaX8Ztisf0MI3BC2YBZ/Jg6JrphA25Oa89/X9WGF//9CS+u+YS3PtzPuN5tmTIoma+9Dfji0FHufj2f93Z42PuVM7/V4KOlfDftWh67KZW0Ti2JjYk+1WnneEo+3U55eTlRUVG+eaQ2bdpU2jvmP892xRVXkJ6ejtfrPU3GVq1asW/fPlq3bl1p7xjA5s2bufXWW1m7di2dO3eu1K60tJQmTc6+crG4uJipU6dy4YUXnuvbVy0W288wIgObA6snNGvckJ8P7szqu9OZcHU7Fmz4D9c88TaDns/hiyPHeTN/N5e3ieWRkd9l9d0DSTiQw29m/pjre7T2GS//vWOTJk3yBQwOZO9Y1Tb++Lf33zu2e/duRo0axcsvv3ya8SopKSE+Pp5GjRqdVff4+HhGjhwZ6Ft1Viy2n2FECHXtAtbWK1SHEGuiqPiI3v+PLfrcyu36/bE36rK3lp+6VlSkiYmJ6vV6K7XJzMzU999//7S+vv76ax09erR26tRJe/furTt37lRVZ3hy2LBhvnrjxo3Tiy++WBs2bKiXXHKJbxjO4/FoRkaGJiUlaUZGhpaUlKiqalZWlsbGxvrSwfh/Bq+//rpOnTq19t4QwzCCAiE8hCiO/KFPr169dOPGjXUtRlDYtGkTs2fP5uWXXz5jvSFDhlQaFqxLRo0axa9+9Su6dOlS16IYhnEGRCRHVXvVtRzfBBtCDAHOND/lT30xXidOnGDkyJFmvAzDCCrmgRmGYUQw5oHVgIgMFZFtIrJDRKZXc72/iOSKyEkRGV3l2s0iUui+Ag8TbhiGYUQEQTNgIhIFPA8MA7oBN4hItyrVdgMTgVeqtI0D7geuBlKB+0WkdtZYG4ZhGGFBMD2wVGCHqn6iqieA+UCleEiquktVNwPlVdoOAbJV9QtVPQhkA0ODKKthGIYRYgTTgF0C/MfvfI9bFuy2hmEYRgQQTAN2+k5ZCHTFSEBtReR2EdkoIhuLi4vPSTjDMAwjtAmmAdsDtPU7bwMEmqApoLaq+kdV7aWqveLj47+xoIZhGEboEbRl9CLSENgODAI+AzYA41X1w2rqvgQsUdWF7nkckANc6VbJBa5S1S/OcL9i4NOziHUR4Dk3TcKKSNY/knWHyNbfdD8zl6pqSHoAQd0HJiLXA88AUcAcVX1URB7CCV3yhoj0BhYBFwLHgc9VtbvbdhIw0+3qUVX9Sy3IszFU9zvUBpGsfyTrDpGtv+kevroHNeKpqi4FllYpu8/veAPO8GB1becAc4Ipn2EYhhG6WCgpwzAMIySJNAP2x7oWoI6JZP0jWXeIbP1N9zAlbGIhGoZhGJFFpHlghmEYRpgQMQbsbIGFww0RmSMiB0Rki19ZnIhkuwGSs8M1vqSItBWRt0Vkq4h8KCJT3PKw119EmojIehHJd3V/0C3vICLrXN0XiEh0XcsaLEQkSkQ2icgS9zySdN8lIgUikiciG92ysH3uI8KABRhYONx4idPjR04HVqpqMrDSPQ9HTgK/UNWuQB/gx+7nHQn6lwIZqtoTSAGGikgf4AngN67uB4GsOpQx2EwBtvqdR5LuAOmqmuK3fD5sn/uIMGAEEFg43FDVfwNVN35/H5jrHs8FRp5Xoc4TqrpPVXPd48M4X2aXEAH6u1nij7injdyXAhnAQrc8LHUHEJE2QCbwJ/dciBDdz0DYPveRYsAsOLBDK1XdB86XPJBQx/IEHRFpD1wBrCNC9HeH0PKAAziZHHYCX6rqSbdKOD//zwC/5FSGi5ZEju7g/FhZLiI5InK7Wxa2z31QNzLXI75NYGEjRBGRZsD/AT9T1UPOj/HwR1W9QIqIxOJEuulaXbXzK1XwEZHhwAFVzRGRgRXF1VQNO939+J6q7hWRBCBbRD6ua4GCSaR4YN8msHA4sV9EWgO4fw/UsTxBQ0Qa4Rivear6d7c4YvQHUNUvgXdw5gFj3fikEL7P//eAESKyC2eaIAPHI4sE3QFQ1b3u3wM4P15SCePnPlIM2AYg2V2NFA2MA96oY5nqgjeAm93jm4F/1KEsQcOd9/gzsFVVZ/tdCnv9RSTe9bwQkQuAa3HmAN8GRrvVwlJ3VZ2hqm1UtT3O//gqVZ1ABOgOICJNRaR5xTFwHbCFMH7uI2Yjc3WBhetYpKAiIq8CA3GiUe8H7gcWA68B7YDdwJgzRfgPVUTkGmANUMCpuZCZOPNgYa2/iFyOM1EfhfMD9TVVfUhEOuJ4JXHAJuBGVS2tO0mDizuEeJeqDo8U3V09F7mnDYFX3ADqLQnT5z5iDJhhGIYRXkTKEKJhGIYRZpgBMwzDMEISM2CGYRhGSGIGzDAMwwhJzIAZhmEYIYkZMKPOEREVkaf9zu8SkQdqqe+XRGT02Wt+6/uMcaPfv12lvL2r32S/st+KyMSz9HeHiNx0ljoTReS3NVw7Ul15beHq5Z/p4DYRyQ2nSOdG/ccMmFEfKAVGichFdS2IP24Wg0DJAv5HVdOruXYAmHIuaTxU9feq+tdzuH+t4Re1ItD6PwQmA9ep6sHgSGUYp2MGzKgPnMRJff7zqheqelAVnoWIDBSR1SLymohsF5HHRWSCmwurQEQ6+XVzrYiscesNd9tHichTIrJBRDaLyI/8+n1bRF7B2QhdVZ4b3P63iMgTbtl9wDXA70XkqWr0K8ZJY3Fz1Qsi0klElrnBV9eIyGVu+QMicpd73NuVca0r8xa/LhLd9oUi8mSVvp92vaKVIhLvlqWIyAduf4sqPCYReUdEHhOR1TjGdoyrY76I/LsanSru8QOc9BzXqaqnpnqGEQzMgBn1heeBCSLS4hza9MTJ/dQD+CHQWVVTcVJpTPar1x4YgJNm4/ci0gTHY/pKVXsDvYHbRKSDWz8VuEdVK+WME5FEnNxSGTi5tnqLyEhVfQjYCExQ1btrkPVx4BfVeHV/BCar6lXAXcAL1bT9C3CHqvYFvFWupQBj3fdgrIhUxPxsCuSq6pXAapxILAB/Baap6uU4Bvp+v75iVXWAqj4N3AcMcfOKjahBp0uB3+IYr89rqGMYQcMMmFEvUNVDOF+uPz2HZhvc3F+lOClDlrvlBThGq4LXVLVcVQuBT4DLcOLE3SRO2pF1OGk3kt3661W1qJr79QbeUdViNz3HPKB/gPoVAeuB8RVl4kTLTwNed+X4A9Dav50b17C5qr7vFr1SpeuVqvqVqh4HPsIxKuCE0FrgHv8NuMb9cRCrqqvd8rlV5F/gd/we8JKI3IYTlqo6inFCE/2gRsUNI4hESjoVIzR4BsjF8TgqOIn7Q8sN0us/j+Qfz67c77ycys921XhpipNmY7KqvuV/wY2hd7QG+b5tPpbHcBIrVgzJNcDJVZVyhjZnu6f/e+Cl5v/pQGLG+fRW1TtE5GocrzVPRFJUtaRK/WM4Wc7fFZEDqjovgHsYRq1hHphRb3ADjL5G5ZTvu4Cr3OPv42QYPlfGiEgDd16sI7ANeAu4U5y0K4hIZzeC95lYBwwQkYvcocAbcIbnAkJVP8bxkoa754eAIhEZ48ogItKzSpuDwGER6eMWjQvwdg04FYF9PPCuqn4FHBSRfm75D2uSX0Q6qeo6Vb0P8FA5HZG/fMXAUOAxERkSoGyGUSuYB2bUN54GfuJ3/iLwDxFZj7MQoibv6Exsw/miboUzl3RcRP6EM8yY63p2xZwl1bqq7hORGTjpOQRYqqrnmpriUZyI6BVMAH4nIvfiGOf5QH6VNlnAiyJyFCe/11cB3Oco0F1Ectz6Y93ym3HmAWNwhlNvqaH9UyKSjKPnympk8qGqRSIyAlgqIqNUdV0A8hnGt8ai0RtGPUdEmqlqxerL6UBrVZ1Sx2IZRp1jHphh1H8yXc+vIfApMLFuxTGM+oF5YIZhGEZIYos4DMMwjJDEDJhhGIYRkpgBMwzDMEISM2CGYRhGSGIGzDAMwwhJzIAZhmEYIcn/A14UxNzwgAvBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.152 0.114 0.104 0.102 0.103 0.103 0.105 0.105 0.106 0.106 0.106 0.106\n",
      " 0.107 0.107 0.107 0.107 0.107 0.107 0.107 0.107 0.107 0.108 0.108 0.107\n",
      " 0.107]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in tqdm(neighbors):\n",
    "    knn_kd = KNeighborsClassifier(n_neighbors=k,algorithm='kd_tree')\n",
    "    scores_kd = cval_score(knn_kd, d_train_dense,y_train, cv=2, scoring='accuracy')\n",
    "    cv_scores.append(scores_kd.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 9.171\n"
     ]
    }
   ],
   "source": [
    "knn_kd_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_kd_optimal.fit(d_train_dense,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_kd_optimal.predict(d_train_dense)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_kd_optimal.predict(d_test_dense)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.995\n",
      "The value of TNR is 0.104\n",
      "The value of FPR is 0.896\n",
      "The value of FNR is 0.005\n",
      "\n",
      "Test error is 10.400\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGeJJREFUeJzt3XuUFOWd//H3ZwCDgCuogAioKGiiyQpeWXU9KAbBu/ESdUUMmFlZNRqN91/ihfUsOW68oBsDRlY0EcE78Y5EVtyIgIooogsiyAgBESSIGmHm+/ujC2xwZmhkmu55/Lw8dab6qaernuaM3/7Ot56qUkRgZmaNX0WpB2BmZg3DAd3MLBEO6GZmiXBANzNLhAO6mVkiHNDNzBLhgG5mlggHdDOzRDigm5klommpB1CXZlt19CWs9jVdW3cs9RCsDM1aMkWbu4/VS+cWHHOa7bDbZh+vGJyhm5klomwzdDOzLaqmutQj2GwO6GZmANVrSj2CzeaAbmYGRNSUegibzQHdzAygxgHdzCwNztDNzBLhk6JmZolIIEP3PHQzMyCq1xS8bIykeZLelDRd0rSsbTtJ4yXNzn62ydolaZikOZJmSNo3bz8Dsv6zJQ3Y2HEd0M3MIHdStNClMIdHRPeI2D97fSUwISK6AROy1wD9gG7ZUgncCbkvAOBa4CDgQODatV8CdXFANzODXMml0OWbOQEYla2PAk7Ma783ciYDrSV1AI4CxkfEsohYDowH+tZ3AAd0MzPInRQtdNm4AJ6T9KqkyqytfUQsAsh+tsvaOwIL8t5blbXV1V4nnxQ1M4NNyryzIF2Z1zQiIkbkvT4kIhZKageMl/ROfburbTT1tNfJAd3MDDbp0v8seI+oZ/vC7OcSSY+Sq4EvltQhIhZlJZUlWfcqoHPe2zsBC7P2Xhu0T6xvXC65mJlBg50UldRS0jZr14E+wFvAOGDtTJUBwOPZ+jjg7Gy2S09gRVaSeRboI6lNdjK0T9ZWJ2foZmZARINdWNQeeFQS5GLs/RHxjKSpwFhJg4APgFOz/k8BRwNzgM+An+TGE8skDQGmZv1uiIhl9R3YAd3MDBrswqKImAvsU0v7x0DvWtoDOL+OfY0ERhZ6bAd0MzPwzbnMzJKRwKX/DuhmZgDVq0s9gs3mgG5mBi65mJklwyUXM7NEOEM3M0uEA7qZWRrCJ0XNzBLhGrqZWSJccjEzS4QzdDOzRDhDNzNLhDN0M7NErCn8ARflygHdzAycoZuZJcM1dDOzRDhDNzNLhDN0M7NEOEM3M0uEZ7mYmSUiotQj2GwO6GZm4Bq6mVkyHNDNzBLhk6JmZomori71CDabA7qZGbjkYmaWDAd0M7NEuIZuZpaGqPE8dDOzNLjkYmaWCM9yMTNLhDN0K4Ztt/0Hhg//T/bee08igsqfXspnn3/Of90xlFatWjBvfhVnn30BK1d+WuqhWhE9P+0xVn36GdU1NVSvqebUPgPYc+9uXHfTlbRosTUfLljEZYN/xapPV7FT5w48+dIY3n/vAwDeePUtrr9saIk/QSPjgG7FcMvNN/Dcsy9w+umVNGvWjBYttuaZp0dz+RVDmDRpMucM+DGXXjqY6667qdRDtSIb8KPBfLJsxbrXQ26+hpuuu42pL7/Oj844jkHnn8WwXw8HYMG8D/nREWeVaqiNXwPfnEtSE2Aa8GFEHCupC/AAsB3wGtA/Ir6U9B3gXmA/4GPgxxExL9vHVcAgoBr4WUQ8W98xKxr0E6z/Yb4r6QpJwyTdlq1/r1jHS8U227Ti0EMPYuR/jwZg9erVrFjxN/bYY3cmTZoMwPMTJnHSSUeXcphWIl267szUl18H4C//8wo/PPbwEo8oITU1hS+FuQiYlff618AtEdENWE4uUJP9XB4RXYFbsn5I2gs4Hdgb6Av8NvuSqFNRArqkK8h9EwmYAkzN1kdLurIYx0zFbrvtwtKlH3P3729h6pRnGf67m2jRYmtmznyX447rA8ApJx9L5047lXikVmwRcPfY23lo/ChO7X8iALPfmcsRfQ8D4Kjjj6RDx/br+nfceScennAf9z72O/Y7qHtJxtyo1UThy0ZI6gQcA/w+ey3gCOChrMso4MRs/YTsNdn23ln/E4AHIuLvEfE+MAc4sL7jFitDHwQcEBFDI+IP2TI0G8ygjbz3W61pkyb06PEDhg+/lwMOPIpVqz7j8ssv4KeVlzD4vHN4ZfLTtNqmJV9+ubrUQ7UiO/PYczn5yLOpPONizhx4Kvv37ME1Fw3hzIGn8ND4UbRs1YLVX+YeyvDR4qX03vd4Tu7dn6G/upWbfjeElq1alvgTNDLV1QUvkiolTctbKjfY263A5cDadH574JOIWPsUjSqgY7beEVgAkG1fkfVf117Le2pVrIBeA9SWQnbgqw/4Nfn/SDU1q4o0tPJW9eEiqqoWMWVq7s/qhx95kh7df8C7777H0cecyUE9+zFmzOPMnTuvtAO1ovto8VIAli1dzvNPTeQH++7F+3Pmc+5pP+OUHw7gqUee44N5VQCs/nI1nyzP1drfnvEOC+ZVsevuO5ds7I1R1NQUvkSMiIj985YRa/cj6VhgSUS8mrd71XbIjWyr7z21KlZAvxiYIOlpSSOy5RlgArm6Uq3y/5EqKr6d2cXixR9RVbWQPfbYHYAjjjiUWbP+j7ZttwdAEldfdREjRtxXymFakW3dojktWrZYt35Ir4OYPes9ttuhDZD7PTjvkoGMGfUIAG22b01FRe5/50677MQuu3Wmav6HpRl8Y9VwJZdDgOMlzSNXej6CXMbeWtLaiSidgIXZehXQGSDbvi2wLL+9lvfUqiizXCLiGUl7kCuxdCT3TVMFTI2Ixj97v8gu/vkvuXfU7Wy1VTPmvv8B5557Cf3POoXzBp8DwGOPPcU9o8aUdpBWVNu33Y7b78nNYmrapAlPPPIsL70wmf4//TFnDjwVgPFPvsAjo/8EwP7/1IOfXf6vrKmupqa6musuG8qKT/5WsvE3Sg10L5eIuAq4CkBSL+AXEfEvkh4ETiEX5AcAj2dvGZe9fjnb/ueICEnjgPsl3Uyu4tGN3DnJOinK9Dl6zbbqWJ4Ds5Lq2rreEqJ9S81aMqW28sQmWXXDvxQcc1r+6o8FHS8voB8raTe+mrb4OnBWRPxdUnPgPqAHucz89IiYm73/GmAgsAa4OCKeru94noduZgawpuGLBxExEZiYrc+lllkqEfEFcGod778RuLHQ4zmgm5mBb59rZpYM3z7XzCwN4Xu5mJklwhm6mVkiHNDNzBLhB1yYmaXBzxQ1M0uFA7qZWSI8y8XMLBHO0M3MEuGAbmaWhqh2ycXMLA3O0M3M0uBpi2ZmqXBANzNLROMvoTugm5kBxJrGH9Ed0M3MwBm6mVkqfFLUzCwVztDNzNLgDN3MLBXO0M3M0hBrSj2CzeeAbmYGhDN0M7NEOKCbmaXBGbqZWSIc0M3MEhHVKvUQNpsDupkZztDNzJIRNc7QzcyS4AzdzCwREY0/Q68o9QDMzMpB1BS+1EdSc0lTJL0haaak67P2LpJekTRb0hhJW2Xt38lez8m275q3r6uy9nclHbWxz+CAbmYG1FSr4GUj/g4cERH7AN2BvpJ6Ar8GbomIbsByYFDWfxCwPCK6Ardk/ZC0F3A6sDfQF/itpCb1HdgB3cyM3EnRQpd695PzafayWbYEcATwUNY+CjgxWz8he022vbckZe0PRMTfI+J9YA5wYH3HdkA3M2PTArqkSknT8pbK/H1JaiJpOrAEGA+8B3wSse4WYFVAx2y9I7AAINu+Atg+v72W99TKJ0XNzIDYhNuhR8QIYEQ926uB7pJaA48C36utW/aztpQ/6mmvkwO6mRnFmYceEZ9Imgj0BFpLappl4Z2AhVm3KqAzUCWpKbAtsCyvfa3899TKJRczM3LTFgtd6iOpbZaZI2lr4EhgFvACcErWbQDweLY+LntNtv3PERFZ++nZLJguQDdgSn3HLjhDl/SdiPh7of3NzBqT6oa7l0sHYFQ2I6UCGBsRT0h6G3hA0r8DrwN3Z/3vBu6TNIdcZn46QETMlDQWeBtYA5yflXLqtNGALunA7IDbAjtL2gc4NyIu/AYf1MysLDXUhUURMQPoUUv7XGqZpRIRXwCn1rGvG4EbCz12ISWXYcCxwMfZAd4ADi/0AGZmjUFDTVsspUJKLhURMT83LXKdetN+M7PGZlNmuZSrQgL6gqzsEllN6ELg/4o7LDOzLaucM+9CFRLQB5Mru+wMLAaez9rMzJJRXdP4J/1tNKBHxBKys65mZqn6VpRcJN1FLVcnRURlLd3NzBqlmgRun1tIyeX5vPXmwEmsf38BM7NGL4X7oRdSchmT/1rSfeRuNmNmloxvRcmlFl2AXRp6IBtK4N/WimDG2w+UegiWqG9FyUXScr6KrxXkLk29spiDMjPb0pKf5ZLdZH0f4MOsqSa7aYyZWVJSCGz1fiVlwfvRiKjOlhQ+s5nZ19SECl7KVSF/Y0yRtG/RR2JmVkINdfvcUqqz5JJ3I/ZDgZ9Keg9YRe4pGhERDvJmloyaUg+gAdRXQ58C7MtXDzI1M0tW1PrEt8alvoAugIh4bwuNxcysZNaUcSmlUPUF9LaSLqlrY0TcXITxmJmVROoZehOgFbU/edrMLCmp19AXRcQNW2wkZmYllHqG3vg/nZlZgVLP0HtvsVGYmZVYdQI5bJ0BPSKWbcmBmJmVUgJPoPtGd1s0M0tOTcoZupnZt0kKN6pyQDczI/2TomZm3xo1csnFzCwJ1aUeQANwQDczw7NczMyS4VkuZmaJ8CwXM7NEuORiZpaIFKYtFvJMUTOz5FWr8KU+kjpLekHSLEkzJV2UtW8nabyk2dnPNlm7JA2TNEfSjPxnOEsakPWfLWnAxj6DA7qZGbkMvdBlI9YAl0bE94CewPmS9gKuBCZERDdgQvYaoB/QLVsqgTsh9wUAXAscBBwIXLv2S6AuDuhmZjRcQI+IRRHxWra+EpgFdAROAEZl3Ubx1fOaTwDujZzJQGtJHYCjgPERsSwilgPjgb71Hds1dDMzoBiPFJW0K9ADeAVoHxGLIBf0JbXLunUEFuS9rSprq6u9Ts7QzczYtAxdUqWkaXlL5Yb7k9QKeBi4OCL+Vs+ha/sqiXra6+QM3cyMTbv0PyJGACPq2i6pGblg/seIeCRrXiypQ5addwCWZO1VQOe8t3cCFmbtvTZon1jfuJyhm5mRm4de6FIfSQLuBmZFxM15m8YBa2eqDAAez2s/O5vt0hNYkZVmngX6SGqTnQztk7XVyRm6mRkNOg/9EKA/8Kak6Vnb1cBQYKykQcAHwKnZtqeAo4E5wGfATyD31DhJQ4CpWb8bNvYkOQd0MzMaLqBHxEvUXv+GWp7VHBEBnF/HvkYCIws9tgO6mRm+l4uZWTJ8Lxczs0T4ARdmZomoSaDo4oBuZkYad1t0QDczwydFzcyS4QzdzCwRa9T4c3QHdDMzXHIxM0uGSy5mZonwtEUzs0Q0/nDugG5mBrjkYmaWjOoEcnQHdDMznKGbmSUjnKGbmaXBGbo1uE6dduKekbfRfse21NTU8Pvf/5Hb77ibf/zHvfjtHUNp2aoF8+dX0f/sC1i58tNSD9caWJ+TB9CyRQsqKipo0qQJY0cO4/YR9/Lnl16mQhVs12ZbbrzmUtq13Z6I4D9u/R2TXp5K8+bf4cZrLmWvPbsy5dU3+PWwr55f/P4HC7jp+ivpfdjBJfxk5S+FaYvKPf2o/DTdqmN5DqzIdtyxHR12bMfr09+iVauWTHnlGU4+ZSAj776VK64YwouTJnPOgB/TpcvOXHvdTaUe7hb3+cJJpR5CUfU5eQBj7h5Gm9bbrmv7dNUqWrVsCcAfHnyc997/gGsvv5AX/zKF+x/+E3f+5w3MmPkOQ28bzui7bl1vfyv+tpJ+pw1kwmP3sXXz5lv0s2xJzXbYbbMfTzF419MKjjl3zhtblo/DqCj1AGx9f/3rEl6f/hYAn366infemU3HnXZkzz1258VJkwF4fsIkTjrp6FIO07agtcEc4PPPv0BZKHnhpckc37c3ktjn+99j5cpP+Wjp+s8Qfu6FSfxzz/2TDuYNZQ1R8FKutnhAl/STLX3MxmqXXTrRfZ/v88qU15k5812OO64PAKecfCydO+1U4tFZMUii8ufXcNrAC3nw8afWtd82/B56n9SfJ597gQvO7Q/A4o8+Zsd2O6zr077dDiz+aOl6+3v6+Rfp98NeW2TsjV1swn/lqhQZ+vV1bZBUKWmapGk1Nau25JjKTsuWLRg75i4u+cW1rFz5KedWXsK/nXcOr0x+mm22acmXX64u9RCtCO678zc8+N93cOdvhjD6kSeYNv1NAC7613OY8Oh9HNPncO5/+E8A1FYulb6qBHy0dBmz577PIQftt2UG38jVbMJSrooS0CXNqGN5E2hf1/siYkRE7B8R+1dUtKyrW/KaNm3Kg2PuYvToR3nssacBePfd9+h3zJkc1LMfD4x5nLlz55V2kFYU7dpuD8D2bVrT+7CDefPtd9fbfkyfXjw/8X8B2LHdDvx1yVcZ+eIlS2m3w/brXj/z5xfpfdjBNGvquQ+FcIZet/bA2cBxtSwfF+mYybhrxG+Y9c4cbr3tq5kKbbP/0SVx9VUXMXzEfaUanhXJZ59/wapVn61b/8uU1+i2267MX/Dhuj4vTJpMl106AdDr0J6Me2YCEcEbb82iVauWtN1hu3V9nx4/kaOP7LVFP0NjlkKGXqyv7ieAVhExfcMNkiYW6ZhJOOTgA+h/1inMePNtpk19DoBf/nIoXbt2YfDgcwB47LGnuGfUmBKO0orh42XLuejqIQBUr6nm6D69OLTn/lx89b8z74MqVCF22rEdv7rsQgAO+6cDmPTyVPqdNpCtmzdnyNU/X7evDxct5q9LlrJ/jx+U5LM0RtVlOuNvU3jaojUqqU9btG+mIaYtnrnLSQXHnPvnP1qW0xZdXDMzw5f+m5klo5xr44VyQDczI41L/x3QzcxwycXMLBkpzHJxQDczI42Si2/OZWZGw15YJGmkpCWS3spr207SeEmzs59tsnZJGiZpTnZF/b557xmQ9Z8tacDGjuuAbmZGg1/6fw/Qd4O2K4EJEdENmJC9BugHdMuWSuBOyH0BANcCBwEHAteu/RKoiwO6mRm5kkuhy8ZExIvAsg2aTwBGZeujgBPz2u+NnMlAa0kdgKOA8RGxLCKWA+P5+pfEelxDNzOj9rtXNrD2EbEoO9YiSe2y9o7Agrx+VVlbXe11ckA3MwOqN+GkqKRKcuWRtUZExIi6+m9sd7W0RT3tdXJANzNj02a5ZMF7UwP4Ykkdsuy8A7Aka68COuf16wQszNp7bdA+sb4DuIZuZkau5FLo8g2NA9bOVBkAPJ7XfnY226UnsCIrzTwL9JHUJjsZ2idrq5MzdDMzGnYeuqTR5LLrHSRVkZutMhQYK2kQ8AFwatb9KeBoYA7wGfATgIhYJmkIMDXrd0NEbHiidT0O6GZmNOyl/xFxRh2betfSN4Dz69jPSGBkocd1QDczw5f+m5klI4VL/x3QzcxwQDczS0a5Po5zUzigm5nhDN3MLBl+wIWZWSKqo/E/VdQB3cwM19DNzJLhGrqZWSJcQzczS0SNSy5mZmlwhm5mlgjPcjEzS4RLLmZmiXDJxcwsEc7QzcwS4QzdzCwR1VFd6iFsNgd0MzN86b+ZWTJ86b+ZWSKcoZuZJcKzXMzMEuFZLmZmifCl/2ZmiXAN3cwsEa6hm5klwhm6mVkiPA/dzCwRztDNzBLhWS5mZonwSVEzs0S45GJmlghfKWpmlghn6GZmiUihhq4UvpVSJ6kyIkaUehxWXvx7YRuqKPUArCCVpR6AlSX/Xth6HNDNzBLhgG5mlggH9MbBdVKrjX8vbD0+KWpmlghn6GZmiXBAL3OS+kp6V9IcSVeWejxWepJGSloi6a1Sj8XKiwN6GZPUBPgvoB+wF3CGpL1KOyorA/cAfUs9CCs/Dujl7UBgTkTMjYgvgQeAE0o8JiuxiHgRWFbqcVj5cUAvbx2BBXmvq7I2M7OvcUAvb6qlzdOSzKxWDujlrQronPe6E7CwRGMxszLngF7epgLdJHWRtBVwOjCuxGMyszLlgF7GImINcAHwLDALGBsRM0s7Kis1SaOBl4E9JVVJGlTqMVl58JWiZmaJcIZuZpYIB3Qzs0Q4oJuZJcIB3cwsEQ7oZmaJcEC3BiepWtJ0SW9JelBSi83YVy9JT2Trx9d3x0lJrSX92zc4xnWSfvFNx2hWLhzQrRg+j4juEfF94EvgvPyNytnk372IGBcRQ+vp0hrY5IBulgoHdCu2SUBXSbtKmiXpt8BrQGdJfSS9LOm1LJNvBevuAf+OpJeAH63dkaRzJN2RrbeX9KikN7LlYGAosHv218FNWb/LJE2VNEPS9Xn7uia7z/zzwJ5b7F/DrIgc0K1oJDUldy/3N7OmPYF7I6IHsAr4f8CREbEvMA24RFJz4C7gOOCfgR3r2P0w4H8iYh9gX2AmcCXwXvbXwWWS+gDdyN2GuDuwn6TDJO1H7jYKPch9YRzQwB/drCSalnoAlqStJU3P1icBdwM7AfMjYnLW3pPcQzv+VxLAVuQuZ/8u8H5EzAaQ9AegspZjHAGcDRAR1cAKSW026NMnW17PXrciF+C3AR6NiM+yY/j+OJYEB3Qrhs8jont+Qxa0V+U3AeMj4owN+nWn4W4RLOA/ImL4Bse4uAGPYVY2XHKxUpkMHCKpK4CkFpL2AN4BukjaPet3Rh3vnwAMzt7bRNI/ACvJZd9rPQsMzKvNd5TUDngROEnS1pK2IVfeMWv0HNCtJCLiI+AcYLSkGeQC/Hcj4gtyJZYns5Oi8+vYxUXA4ZLeBF4F9o6Ij8mVcN6SdFNEPAfcD7yc9XsI2CYiXgPGANOBh8mVhcwaPd9t0cwsEc7QzcwS4YBuZpYIB3Qzs0Q4oJuZJcIB3cwsEQ7oZmaJcEA3M0uEA7qZWSL+Pykxv3mpszN4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Bigrams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████████| 25/25 [1:10:00<00:00, 163.68s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 23.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAEKCAYAAAB+AXB+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VNX5+PHPM5NkspCEhCSQZLIQAihLQDZBC4gsUrSIfFHABRWrVWv5dlO0/dWq/VotVmvd9asidRe/KqgoCuKOrLIURcCAEHYCyJKNIc/vj5lMJ8kkGZYhCTzv12temXvuOec+M2ienHvPPVdUFWOMMeZk4WjsAIwxxpjjyRKbMcaYk4olNmOMMScVS2zGGGNOKpbYjDHGnFQssRljjDmpWGIzxhhzUrHEZowx5qRiic0YY8xJJaKxAzgRUlJSNDc3t7HDMMaYZmXJkiW7VDW1seM4UqdEYsvNzWXx4sWNHYYxxjQrIvJDY8dwNOxUpDHGmJOKJbYQlJaWMnDgQA4fPgzA8OHDadmyJRdccEFI7cvLyxk7diz5+fmceeaZbNiwIWi9iRMnkpaWRpcuXYLu//vf/46IsGvXLgA+/vhjEhMT6d69O927d+euu+4CoKKiggEDBuDxeI7wkxpjTPNniS0Ezz77LKNHj8bpdAJw88038/zzz4fc/plnniEpKYl169bxm9/8hsmTJwetd9VVV/H+++8H3bdp0yY+/PBDsrOzq5X379+fZcuWsWzZMm6//XYAoqKiGDx4MK+++mrIMRpjzMnCElsIXnzxRS688EL/9uDBg4mPjw+5/YwZM7jyyisBGDNmDHPnziXY44IGDBhAcnJy0D5+85vfMGXKFEQkpGOOGjWKF198MeQYjTHmZGGJrQEVFRUUFhZyLLMqN2/eTFZWFgAREREkJiZSXFwccvuZM2eSmZlJt27dau2bP38+3bp146c//SmrVq3yl3fp0oVFixYddczGGNNchTWxichwEflORNaJyK1B9g8QkaUi4hGRMUH2J4jIZhF5JKDsY1+fy3yvtHB+hl27dtGyZctj6iPY6CzUkVdJSQl33323//pZoB49evDDDz+wfPlyfvWrXzFq1Cj/PqfTSVRUFPv37z/6wI0xphkKW2ITESfwKPBToBMwXkQ61ai2EbgKeKmObv4CfBKk/DJV7e577ThOIQcVExNDWVnZMfXhdrvZtGkTAB6Phx9//LHOU441ff/996xfv55u3bqRm5tLUVERPXr0YNu2bSQkJNCiRQsARowYwaFDh/wTS8A7aSU6OvqYYjfGmOYmnCO2PsA6VS1U1QrgFeDCwAqqukFVVwCVNRuLSE+gNfBBGGOs11tfb+bd7/Zx+PDhkJLbbbfdxptvvlmrfOTIkUybNg2A119/nXPPPTfkEVvXrl3ZsWMHGzZsYMOGDbjdbpYuXUqbNm3Ytm2bfzS4cOFCKisradWqFQDFxcWkpqYSGRkZ6sc1xpiTQjgTWyawKWC7yFfWIBFxAPcDN9dRZarvNOSfJNQMcRTeXbmVF776gWHDhvH555/7y/v378/FF1/M3LlzcbvdzJ49G4CVK1fSpk2bWv1cc801FBcXk5+fzwMPPMC9994LwJYtWxgxYoS/3vjx4+nXrx/fffcdbrebZ555pt74Xn/9dbp06UK3bt2YNGkSr7zyij9hzps3r1rfxhhzqgjnyiPBEk7ti03B3QjMUtVNQfLWZaq6WUTigf8DrgD+VevgItcB1wG1psiHKjXexdIf9nDPTTfxwAMPMGTIEAA+++yzoPUPHTpEv379apVHR0czffr0WuUZGRnMmjXLv/3yyy83GFPgPXA33XQTN910U9B6L730Evfcc0+D/RljzMkmnCO2IiArYNsNbAmxbT/gJhHZAPwdmCAi9wKo6mbfz/14r831CdaBqj6lqr1UtVdq6tEtdZYW76L4YAVdCroxaNAg/w3adakauTW2iooKRo0aRceOHRs7FGOMOeHCOWJbBLQXkbbAZmAccGkoDVX1sqr3InIV0EtVbxWRCKClqu4SkUjgAmDOcY/cJzXeBUDxgQomTpwYrsMcd1FRUUyYMKGxwzDGmEYRthGbqnqAm4DZwLfAa6q6SkTuEpGRACLSW0SKgIuBJ0VkVd09AuACZovICmAZ3oT5v+H6DGnx3hmFO/Yf26xIY4wxJ05YV/dX1VnArBpltwe8X4T3FGV9fTwHPOd7fxDoebzjrEvViG3n/vITdUhjjDHHyFYeqUeaL7HtsMRmjDHNhiW2eqS0sBGbMcY0N5bY6hEV4SApNtKusRljTDNiia0BqfEuG7EZY0wzYomtAWnx0XaNzRhjmhFLbA2wEZsxxjQvltgakBbvYsf+8qCPnjHGGNP0WGJrQGq8iwpPJfvKPI0dijHGmBBYYmvAf27StpmRxhjTHFhia0Cq3aRtjDHNiiW2BlStF2kTSIwxpnmwxNYAWy/SGGOaF0tsDUiIjiAqwmGJzRhjmglLbA0QEf+Uf2OMMU2fJbYQ2E3axhjTfFhiC4F3xGbT/Y0xpjmwxBYCG7EZY0zzEdbEJiLDReQ7EVknIrcG2T9ARJaKiEdExgTZnyAim0XkkYCyniKy0tfnQyIi4fwM4J3yv6fkEBWeynAfyhhjzDEKW2ITESfwKPBToBMwXkQ61ai2EbgKeKmObv4CfFKj7HHgOqC97zX8OIVcp6op/7sO2KjNGGOaunCO2PoA61S1UFUrgFeACwMrqOoGVV0B1BoKiUhPoDXwQUBZOpCgqvPVuyrxv4BRYfwMgPcaG9jqI8YY0xyEM7FlApsCtot8ZQ0SEQdwP3BzkD6LjqbPY2E3aRtjTPMRzsQW7NpXqM9+uRGYpaqbapSH3KeIXCcii0Vk8c6dO0M8bHBVy2rZzEhjjGn6IsLYdxGQFbDtBraE2LYf0F9EbgRaAFEicgD4p6+fBvtU1aeApwB69ep1TA9Ta9UiChEbsRljTHMQzsS2CGgvIm2BzcA44NJQGqrqZVXvReQqoJeq3urb3i8ifYEFwATg4eMcdy2RTgfJsVF2jc0YY5qBsJ2KVFUPcBMwG/gWeE1VV4nIXSIyEkBEeotIEXAx8KSIrAqh6xuAp4F1wPfAe2H5ADXYvWzGGNM8hHPEhqrOAmbVKLs94P0iqp9aDNbHc8BzAduLgS7HM85QpNp6kcYY0yzYyiMhSo13scsSmzHGNHmW2EJUdSrSe/ucMcaYpsoSW4jS4qOpOFzJj6WHGjsUY4wx9bDEFiK7SdsYY5oHS2whsmW1jDGmebDEFiIbsRljTPNgiS1E/xmx2bJaxhjTlFliC1ELVwTRkQ4bsRljTBNniS1EIkJafLRdYzPGmCbOEtsRsGW1jDGm6bPEdgTSbFktY4xp8iyxHQEbsRljTNNnie0IpMW7+LH0EGWHDjd2KMYYY+pQb2ITEaeI3Heigmnqqu5l23XARm3GGNNU1ZvYVPUw0FNE5ATF06SlxUcDtvqIMcY0ZaE8j+1rYIaITAcOVhWq6hthi6qJstVHjDGm6QslsSUDxcC5AWUKnHKJzdaLNMaYpq/BxKaqVx9t5yIyHPgn4ASeVtV7a+wfADwIFADjVPV1X3kO3sTpBCKBh1X1Cd++j4F0oNTXzTBV3XG0MR6J5LgoRGzEZowxTVmDsyJFxC0ib4rIDhHZLiL/JyLuENo5gUeBnwKdgPEi0qlGtY3AVcBLNcq3AmepanfgTOBWEckI2H+Zqnb3vU5IUgOIcDpoFRfFTlsv0hhjmqxQpvtPBWYCGUAm8LavrCF9gHWqWqiqFcArwIWBFVR1g6quACprlFeoatWwyBVinCdEany0jdiMMaYJCyVhpKrqVFX1+F7PAakhtMsENgVsF/nKQiIiWSKywtfH31R1S8DuqSKyTET+dKJnbNpN2sYY07SFkth2icjlvnvanCJyOd7JJA0JlnA01MBUdZOqFgD5wJUi0tq36zJV7Qr0972uCHpwketEZLGILN65c2eoh22QLatljDFNWyiJbSJwCbAN77WvMb6yhhQBWQHbbmBLHXXr5BuprcKbxFDVzb6f+/Fem+tTR7unVLWXqvZKTQ1lgBma1HgXuw6UU1kZco42xhhzAjW48gjwX6o6UlVTVTVNVUep6g8h9L0IaC8ibUUkChiH91pdg3wTVmJ875OAs4HvRCRCRFJ85ZHABcC/Q+nzeEmLd3HosLK39NCJPKwxxpgQhbLyyIX11amnrQe4CZgNfAu8pqqrROQuERkJICK9RaQIuBh4UkRW+ZqfDiwQkeXAJ8DfVXUl3okks33X3pYBm4H/PZr4jpbdpG2MMU1bKDdofyEijwCvUn3lkaUNNVTVWcCsGmW3B7xfhPcUZc12H+K9t61m+UGgZwgxh81/ltUqo2Ob+MYMxRhjTBChJLazfD/vCihTqq9EcsqwEZsxxjRt9SY2EXEAj6vqayconibPltUyxpimraFrbJV4r5MZnzhXBLFRThuxGWNMExXKdP8PReT3vhumk6teYY+sCbN72YwxpukK5Rpb1T1rvwwoUyDv+IfTPHhXH7H1Io0xpikKZXX/ticikOYkLT6ab7fta+wwjDHGBFHnqUgRuSXg/cU19v01nEE1dbZepDHGNF31XWMbF/D+thr7hochlmYjNd7F/jIPZYcON3YoxhhjaqgvsUkd74Ntn1LsXjZjjGm66ktsWsf7YNunlFT/vWw2gcQYY5qa+iaPdBORfXhHZzG+9/i2o8MeWROWZiM2Y4xpsupMbKrqPJGBNCd2KtIYY5quUG7QNjW0inPhEFtWyxhjmiJLbEfB6RBatbAp/8YY0xRZYjtKtqyWMcY0TZbYjpLdpG2MMU1Tg4lNREaLyFoR+VFE9onI/oAZkqcs74jNpvsbY0xTE8qIbQowUlUTVTVBVeNVNSGUzkVkuIh8JyLrROTWIPsHiMhSEfGIyJiA8hwRWSIiy0RklYhcH7Cvp4is9PX5kIg0ys3iqfEudh2ooLLylL6lzxhjmpxQEtt2Vf32SDsWESfwKPBToBMwXkQ61ai2EbgKeKlG+VbgLFXtDpwJ3CoiGb59jwPXAe19r0ZZ3istPprDlcrukorGOLwxxpg6hPLYmsUi8irwFuC/qKSqbzTQrg+wTlULAUTkFeBC4JuAPjb49lUGNlTVwGzhwpeARSQdSFDV+b7tfwGjgPdC+BzHVeC9bCktXCf68MYYY+oQyogtASgBhgE/870uCKFdJrApYLvIVxYS34NNV/j6+JuqbvG1LzraPo+nNP+yWjaBxBhjmpJQnsd29VH2HezaV8gXpFR1E1DgOwX5loi8fiR9ish1eE9Zkp2dHephQ2arjxhjTNMUyqxIt4i8KSI7RGS7iPyfiLhD6LsIyArYdgNbjjRA30htFdDf12fgsevsU1WfUtVeqtorNTX1SA/bIFsI2RhjmqZQTkVOBWYCGXhP+73tK2vIIqC9iLQVkSi8z3ebGUpQvmQa43ufBJwNfKeqW4H9ItLXNxtyAjAjlD6Pt9ioCFq4ImzEZowxTUwoiS1VVaeqqsf3eg5ocAikqh7gJmA28C3wmqquEpG7RGQkgIj0FpEi4GLgSRFZ5Wt+OrBARJYDnwB/V9WVvn03AE8D64DvaYSJI1VSbfURY4xpckKZFblLRC4HXvZtjweKQ+lcVWcBs2qU3R7wfhHVTy1WlX8IFNTR52KgSyjHDzdbfcQYY5qeUEZsE4FLgG147y8b4ys75VliM8aYpieUWZEbgZEnIJZmJy3exSeW2IwxpkmpM7GJyC2qOkVEHibIlHpVnRTWyJqB1HgXB8o9lFR4iI0K5ayuMcaYcKvvt3HVMlqLT0QgzVFafDTgvZctp5UlNmOMaQrq/G2sqm/73pao6vTAfSJycVijaiYCb9LOaRXXyNEYY4yB0CaP3BZi2SnHltUyxpimp75rbD8FRgCZIvJQwK4EwBPuwJoDW1bLGGOanvouDG3Be31tJLAkoHw/8JtwBtVcJMdG4XSILatljDFNSH3X2JYDy0XkJVU9dAJjajYcDiGlRZSN2IwxpgkJZSpfrojcg/dhodFVhaqaF7aompG0+Gi7xmaMMU1IqIsgP473utog4F/A8+EMqjmx1UeMMaZpCSWxxajqXEBU9QdVvQM4N7xhNR9pthCyMcY0KaGciiwTEQewVkRuAjYDaeENq/lIjXdRfKCcw5WK0xHsOajGGGNOpFBGbL8GYoFJQE/gcuDKcAbVnKTGu6hUKD5oozZjjGkKQlkEeZHv7QHg6vCG0/ykBdzLVrXEljHGmMbT4IhNRD4UkZYB20kiMju8YTUfqbb6iDHGNCmhnIpMUdW9VRuquge7xuYXuBCyMcaYxhdKYqsUkeyqDRHJIchjbIIRkeEi8p2IrBORW4PsHyAiS0XEIyJjAsq7i8h8EVklIitEZGzAvudEZL2ILPO9uocSS7jYslrGGNO0hDIr8o/A5yLyiW97AHBdQ41ExAk8CgwFioBFIjJTVb8JqLYRuAr4fY3mJcAEVV0rIhnAEhGZHTByvFlVXw8h9rCLjnQSHx1hic0YY5qIUCaPvC8iPYC+gAC/UdVdIfTdB1inqoUAIvIKcCHgT2yqusG3r7LGMdcEvN8iIjuAVGAvTZDdpG2MMU1HnaciReQ0388eQDbeRZE3A9m+soZkApsCtot8ZUdERPoAUcD3AcV3+05R/kNEXEfa5/HmvUnbFkI2xpimoL4R22/xnnK8P8g+peHVR4LdrRzStTl/ByLpeJfvulJVq0Z1twHb8Ca7p4DJwF1B2l6H75RpdnZ2zd3HVWp8NCuLmuRg0hhjTjn1JbYPfT+vqTqdeISKgKyAbTfeUV9IRCQBeBf4f6r6VVW5qm71vS0XkanUvj5XVe8pvImPXr16HVFCPVK2rJYxxjQd9c2KrHpK9tFO0lgEtBeRtiISBYwDZobS0Ff/TeBfqjq9xr50308BRgH/Psr4jpvUeBclFYc5WG7PXzXGmMZW34itWETmAW1FpFZCUtWR9XWsqh7f2pKzASfwrKquEpG7gMWqOlNEeuNNYEnAz0TkTlXtDFyCd/ZlKxG5ytflVaq6DHhRRFLxnupcBlx/JB84HNICbtJu6wploqkxxphwqe+38PlAD7zXuIJdZ2uQqs4CZtUouz3g/SK8pyhrtnsBeKGOPpvckwUC72VrmxLXyNEYY8yprb4naFcAX4nIWaq68wTG1OxUrT5iMyONMabx1ZnYRORBVf018KyI1Jp80dCpyFOJrT5ijDFNR32nIquekv33ExFIc9YyJpJIp9jMSGOMaQLqOxW5xPezaiktRCQJyFLVFScgtmbD4RBSWtjqI8YY0xSE8tiaj0UkQUSSgeXAVBF5IPyhNS+pdi+bMcY0CaGs7p+oqvuA0cBUVe0JDAlvWM1Pmq0XaYwxTUIoiS3Cd1P0JcA7YY6n2fIuhGyzIo0xprGFktjuwnuT9TpVXSQiecDa8IbV/KTGR1N8sALP4cqGKxtjjAmbUB5bMx2YHrBdCPxXOINqjlLjXahC8cEKWidEN3Y4xhhzygpl8sgU3+SRSBGZKyK7ROTyExFcc5Jm97IZY0yTEMqpyGG+ySMX4F2xvwNwc1ijaobsJm1jjGkaQklskb6fI4CXVXV3GONptv6zELJNIDHGmMYUylL0b4vIaqAUuNG3sr799q4hpYWN2IwxpilocMSmqrcC/YBeqnoIOAhcGO7AmpvoSCeJMZF2k7YxxjSyUE5FAmQC/yUiE4AxwLDwhdR8pQbcpF1aWsrAgQM5fPgwP/zwAz179qR79+507tyZJ554osG+du/ezdChQ2nfvj1Dhw5lz549QesNHz6cli1bcsEFF1Qrv+yyy+jYsSNdunRh4sSJHDp0CIB33nmHP//5z8f4SY0xpukKZVbkn4GHfa9BwBTAVvYPIi1gWa1nn32W0aNH43Q6SU9P58svv2TZsmUsWLCAe++9ly1bttTb17333svgwYNZu3YtgwcP5t577w1a7+abb+b555+vVX7ZZZexevVqVq5cSWlpKU8//TQA559/PjNnzqSkpOQYP60xxjRNoYzYxgCDgW2qejXQDXCFNapmKnDE9uKLL3Lhhd4ztlFRUbhc3q+svLycysqGb+KeMWMGV155JQBXXnklb731VtB6gwcPJj4+vlb5iBEjEBFEhD59+lBUVASAiHDOOefwzju2iIwx5uQUSmIrVdVKwCMiCcAOIC+UzkVkuIh8JyLrROTWIPsHiMhSEfGIyJiA8u4iMl9EVonIChEZG7CvrYgsEJG1IvKqiESFEsuJ4B2xlVFeXk5hYSG5ubn+fZs2baKgoICsrCwmT55MRkZGvX1t376d9PR0ANLT09mxY8dRxXTo0CGef/55hg8f7i/r1asXn3322VH1Z4wxTV0oiW2xiLQE/hdYAiwFFjbUSEScwKPAT4FOwHgR6VSj2kbgKuClGuUlwARV7QwMBx70xQDwN+Afqtoe2ANcE8JnOCFS412UHarkh83baNmyZbV9WVlZrFixgnXr1jFt2jS2b99+QmK68cYbGTBgAP379/eXpaWlNXgq1BhjmqtQZkXeqKp7VfUJYChwpe+UZEP64F1fslBVK4BXqDGbUlU3+J7tVlmjfI2qrvW934J3lJgqIgKcC7zuqzoNGBVCLCdE1U3aBw47KCsLfkdERkYGnTt3bnDE1Lp1a7Zu3QrA1q1bSUtLO+J47rzzTnbu3MkDD1R/ylBZWRkxMTFH3J8xxjQHdSY2EelR8wUk413tv0cIfWcCmwK2i3xlR0RE+gBRwPdAK2CvqnqOpc9wSYv3rhFZ4Yzl8OHD/uRWVFREaWkpAHv27OGLL76gY8eOAEyYMIGFC2sPgEeOHMm0adMAmDZtmv96XaiefvppZs+ezcsvv4zDUf2fec2aNXTp0uXIPpwxxjQT9d2gfX89+xTvyKk+Uke7kPkel/M83lFipW/EFlKfInIdcB1Adnb2kRz2qKX6Vx8pZ9iwYXz++ecMGTKEb7/9lt/97neICKrK73//e7p27QrAihUr/NfSAt16661ccsklPPPMM2RnZzN9uncd6sWLF/PEE0/4Zzn279+f1atXc+DAAdxuN8888wznnXce119/PTk5OfTr1w+A0aNHc/vttwMwb9487rnnnrB/H8YY0xjqTGyqOugY+y4CsgK23UDIF3Z8E1XeBf6fqn7lK94FtBSRCN+orc4+VfUp4CmAXr16HVFCPVpVy2pt2VvKTTfdxAMPPMCQIUMYOnQoK1asqFV/3759tG/fnqysrFr7WrVqxdy5c2uV9+rVy5/UgDpPaXo8nqDl27dvp7S01J9YjTHmZBPKfWy/DJi4gYgkiciNIfS9CGjvm8UYBYwDZoYSlK/+m8C/fI/NAUBVFZiH9xYEgCuBGaH0eSIkxkRyenoCry3eREG37gwaNIjDhw/XWT8hIcE/EjtRNm7cyP331zcYN8aY5i2UWZHXqureqg1V3QNc21Aj34jqJrwPKf0WeE1VV4nIXSIyEkBEeotIEXAx8KSIrPI1vwQYAFwlIst8r+6+fZOB34rIOrzX3J4J6ZOeACLCr87Np3DnQWat3MrEiRNxOp2NHVY1vXv3pnv37g1XNMaYZkq8g6B6KoisALr5RktV0/hX+KbiNwu9evXSxYsXn5BjVVYqwx78FKcI7/13fxyOYJcFjTGm6RORJaraq7HjOFKhjNhmA6+JyGARORd4GXg/vGE1Xw6HcNOgfL7bvp8Pvjkx96oZY4z5j1AS22RgLnAD8Evf+1vCGVRzd0FBOrmtYnn4o7U0NCI2xhhzfIVyg3alqj6hqmPwXlubr6p1z4gwRDgd3Dgon1Vb9jHvu6NbCssYY8zRCWVW5McikiAiycAyYKqIPNBQu1PdRWdk4k6K4aG562zUZowxJ1AopyITVXUfMBqYqqo9gSHhDav5i3Q6uOGcdizbtJcv1hU3djjGGHPKCCWxRfhWALkEsGedHIExPd20SYjmoY/WNnYoxhhzygglsd2Fd2bkOlVdJCJ5gP2mDoErwskvBuaxcP1uFhTaqM0YY06EUCaPTFfVAlW90bddqKr/Ff7QTg7j+2ST0sLFwx+ta+xQjDHmlFDnWpEicouqThGRhwmy0LCqTgprZCeJ6Egn1w1oy19nrWbpxj30yE5q7JCMMeakVt+I7Vvfz8V4HzBa82VCdNmZOSTFRvLwXDuDa4wx4Vbf6v5v+35OO3HhnJziXBH8vH8e983+jpVFP9LVndjYIRljzEmrvlOR9a7Er6ojj384J68J/XJ48pPveWTeWp68otktvWaMMc1GfQ8a7Yf3CdgvAwsI/uBQE6L46EiuOrstD81dy+pt+zitTUJjh2SMMSel+q6xtQH+AHQB/gkMBXap6ieq+smJCO5kM/HsXOKinDxiMySNMSZs6kxsqnpYVd9X1SuBvsA64GMR+dUJi+4k0zI2igln5fLuyq2s23Gg2r7S0lIGDhzofzCp0+mke/fudO/enZEjg5/1DWwze/ZskpKScLlcxMXF4XK5eOutt2q1mTBhAlFRUXTu3Jny8nJOO+00oqKicLlctGnTBofDQeAjfjZs2EBERASRkZHExMRw7bXXMnbsWPLz84mNjaVdu3YNtrn++uv9+7p27cppp50WtE1MTIz/Mwe2GTJkCHv27DnCb9sYc6qq9z42EXGJyGjgBbwr+z8EvHEiAjtZ/fwnbYmOcPLYvOqjtmeffZbRo0f7H0waExPDsmXLWLZsGTNnBr/cGdjm+++/Z+zYsZSXl/PPf/4TgGHDhtVq06pVKyZNmoSI8Jvf/IYDBw5QUlLCn//8Z0pLSznrrLNqtcnKymLBggW0a9eOM844g6SkJNatW8fIkSNxOp0MGDCg3jZPPPGEv/zSSy9l+PDhQdu0a9fO/5kD21xxxRU89thjQb8DY4ypqc7EJiLTgC+BHsCdqtpbVf+iqptPWHQnoVYtXFx2ZjYzlm/hh+KD/vIXX3yRCy+88Ij6CmwzY8YMrrzySgAOHToEeJNjTQsWLGDUqFEAzJkzhxEjRhAREcEtt9xCeXk5u3fvrtUmLi6O5OTkWsd5+OGHKSwsDLrIc2CbQNdffz1z5sxBEp7IAAAgAElEQVQ5os85cuRIXn755SNqY4w5ddU3YrsC6AD8N/CliOzzvfaLyL5QOheR4SLynYisE5Fbg+wfICJLRcQjImNq7HtfRPaKyDs1yp8TkfUissz36h5KLE3JdQPycDqEx+Z9D0BFRQWFhYXk5ub665SVldGrVy/69u0b9JRizTabN28mKysLgOnTp5OUlERxcXHQNm6327/99ddfU1JSwt69ezl8+DAlJSW1jrV+/XpGjBjB+vXrWbt2rf84qampiAilpaX1tvnss8/85UlJSZSXl/uTb802Z5xxBgMHDgzapubnMcaYYOq7jy2UdSTrJCJO4FG8k06KgEUiMlNVvwmothG4Cvh9kC7uA2KBXwTZd7Oqvn4s8TWmtIRoxvXO4qUFG5k0pD1SsoeWLVtWq7Nx40YyMjIoLCzk3HPPpWvXrrRr144fig8yb/UOtm7dSmyLBFQVEfGPmrZu3crKlSuJj49HpPpE1l27dlU7TlxcHIMGDeKss84iNTWV6OjoWm3S09PZuHEj+/fvZ/Dgwf73VZxOZ60kVbPNpZdeyqpVq0hI8M4ETUtLo6KiImibVq1asWTJEkaNGlWrzZYtW2jVqtXRfOXGmFPIMSWvBvTBu3ByoapWAK8A1c61qeoGVV0BVNZsrKpzgf01y08W1w9shwg88fH3xMTEUFZWVm1/RkYGAFk5uXTq2Zc/Pj2Tc+//mIH3fcwdb3/DY59vYuPOH+l3z0fc9sYKYlqmsbZwPa+99hoXXngh+/btq3UqsOZx3G43F110EcuWLeO9996joqKCFi1aVGvjcrn8ySQmJob4+Hjmz58PgMfjwePxEBkZWW+bdu3asWbNGv/+srIyHA5HnW169uwZtE2wU6vGGFNTOBNbJt774KoU+cqOh7tFZIWI/ENEXMepzxMqo2UMY3q6eXXxJiqcMRw+fNifdNZu3MZLX67jxheXUHDr63w471PmF8eS2TKGzkXvcNvp+1hy14W0jHbSpXUMby/fyvq40xj163v4n4f+lx2eGPr+ZKB/9HXbbbfx5ptvkpSUxOHDhykvLwfgggsu4MknnwTg/vvvJzo62p8Mq9rs3LnTP1OzoqKCyspK/2nC6dOn+2c+BqrZZu3ateTl5QGgqmzbto3o6Og62xQWFgZtE3iq1hhj6qSqYXkBFwNPB2xfATxcR93ngDFBys8B3qlRlo73ZnEXMA24vY4+r8O7zuXi7OxsbYp+2HVQ8257V++Y+W8dNfZyvf5vU3Xkw59p68vu08iUHI1pk6cp2fn66zvv1wNlh1RV9fzzz9cvv/xSVVUnTpyoH374oZYfOqxz/71J2/fsr4hDI9vka8YvntZz/z5Pb5k2T1smt9JPP/tcVVXbtm2rSUlJGhERoenp6RodHa2RkZHqcrk0OTlZo6KitFWrVhoZGalffvmlvv7665qQkKAREREKaHJysvbs2VPbtWunWVlZGhUV5W+TkpKiqlqrTatWrfTpp59WVdUpU6ZodHS0RkVFaVpamg4bNszfplOnTlpQUKBnnHGGzpw50/89LVq0SEePHn3C/l2MMV7AYg1TjgjnK5yJrR8wO2D7NuC2OuqGnNiOZH/Vq2fPnqH9KzaC3766THMmv6PpV/1T4zoP0lGPfq4PzVmjK4v2amVlZa36VYlAVXXp0qV6+eWX16qzfucBfeazQr386a+0/R9maXTuGdrl9vf1zaVFdbap7zh1mTRpks6ZM6fBeo3Rxhhz7JprYqtvSa1jtQhoLyJtgc3AOODSY+1URNJVdat4z7ONAv59rH02pt+f14GYKAc9srtR9JWLX/6ir/9etmBmz57tf3/GGWcwaNAgDh8+XK1NbkocE3/Slok/acvBcg9fXHoGT3zyPbe+sYJ3ftU/aJv6jlOXLl26MHjw4BA/6YltY4w5dYk3KYepc5ERwIOAE3hWVe8Wkbvw/hUwU0R6A28CSUAZsE1VO/vafgacBrQAioFrVHW2iHwEpOI9HbkMuF5VD9Q8dqBevXpp4CoXp6Id+8o478FPyWgZw5s3nk1URDgvrxpjTgYiskRVm92q7WFNbE2FJTavD1Zt47rnl/CLgXnc9tPTGzscY0wT11wTm/3ZfgoZ1rkNl56ZzVOfFvLlul2NHY4xxoSFJbZTzP87/3TapsTx29eWs7ekouEGxhjTzFhiO8XERkXw0LgzKD5Yzm1vrORUOBVtjDm1WGI7BXXJTOR3wzry3r+3MX1JUWOHY4wxx5UltlPUdf3z6JfXijtmrmLDroMNNzDGmGbCEtspyuEQ7r+kG5FOB//96jIOHa61XKcxxjRLlthOYRktY/jrRV1ZvmkvD81d29jhGGPMcWGJ7RR3fkE6F/d08+i8dSxcX/sho8YY09xYYjP8eWRnspJj+c2ry/ixtPqz1UpLSxk4cKB/5f3hw4fTsmVLLrjggmr1HnnkEfLz8xERdu0K7R65adOm0b59e9q3b8+0adOC1pk+fTqdO3fG4XAQeJN9cXExgwYNokWLFtx0003V2vzxj38kKyur1iN4HnnkEaZOnRpSbMaY5ssSm6GFK4IHx3Zn274y/t9b/652C8Czzz7L6NGj/etK3nzzzTz//PO1+jj77LOZM2cOOTk5IR1z9+7d3HnnnSxYsICFCxdy5513smfPnlr1unTpwhtvvMGAAQOqlUdHR/OXv/yFv//977Xa/OxnP2PhwoW1yidOnMhDDz0UUnzGmObLEpsB4IzsJH49uD1vL9/CW8s2+8tffPFFLrzwP8+HHTx4MPHx8bXbn3HGET0vbfbs2QwdOpTk5GSSkpIYOnQo77//fq16p59+Oh07dqxVHhcXx09+8pNaz3UD6Nu3L+np6bXKY2Njyc3NDZr0jDEnD0tsxu/GQfn0zk3iT2+tYtPuEioqKigsLAzLAz43b95MVlaWf9vtdrN58+Z6WhwfvXr18j8o1RhzcrLEZvycDuGBS7ojwK9fXca27Tto2bJlWI4VbMWTqid+h1NaWhpbtmwJ+3GMMY3HEpupJis5lv+5qAtLftjDi4u3UVZWFpbjuN1uNm3a5N8uKioiIyMjLMcKVFZWRkxMTNiPY4xpPJbYTC0Xds9kVPcMnl60g4Nlh44puS1cuJAJEybUKj/vvPP44IMP2LNnD3v27OGDDz7gvPPOO5awQ7JmzRq6dOkS9uMYYxqPJTYT1F9GdaFrZiLlrTtz37S3/OX9+/fn4osvZu7cubjdbv+Tth966CHcbjdFRUUUFBTw85//HICNGzcGHSElJyfzpz/9id69e9O7d29uv/12kpOTAfj5z3/un9r/5ptv4na7mT9/Pueff3615Jebm8tvf/tbnnvuOdxuN9988w0At9xyC263m5KSEtxuN3fccYe/zRdffMGQIUOO75dljGlS7EGjpk4Hyj1cdNcLzJ/xL56b9i/G9HQfcR8333wzV1xxBQUFBWGI8Mh8/fXXPPDAA0FvVzDG1GYPGg1CRIaLyHcisk5Ebg2yf4CILBURj4iMqbHvfRHZKyLv1ChvKyILRGStiLwqIlHh/AynshauCGb8+XK69OrH715dyksLNh5xH/fdd1+TSGoAu3bt4i9/+Utjh2GMCbOwJTYRcQKPAj8FOgHjRaRTjWobgauAl4J0cR9wRZDyvwH/UNX2wB7gmuMVs6ktNiqCj//3Ls49vQ1/eHMlU79Y39ghHbWhQ4eG5dYFY0zTEs4RWx9gnaoWqmoF8ApwYWAFVd2gqiuAWkvLq+pcYH9gmXjng58LvO4rmgaMCkPsJkB0pJMnrujJsE6tufPtb3jik+/rrR/qMlzXXHMN3bp1o6CggDFjxnDgwIEGY7nnnnvIz8+nY8eO/ut7NdW1vJeqMmnSJPLz8ykoKGDp0qX+fZMnT6ZLly506dKFV1991V8+btw41q61BaKNaU7CmdgygU0B20W+smPRCtirqp7j2KcJgSvCyaOX9eBn3TK4973V/HPO2jqfvh3qMlz/+Mc/WL58OStWrCA7O5tHHnmk3hi++eYbXnnlFVatWsX777/PjTfe6E+egepa3uu9995j7dq1rF27lqeeeoobbrgBgHfffZelS5eybNkyFixYwH333ce+ffsAuOGGG5gyZUrDX5AxpskIZ2ILdrftsc5UCblPEblORBaLyOKdO3ce42ENQKTTwYNju/NfPdz8Y84apsz+LmhyC3UZroSEBMA7kiotLW3wBu0ZM2Ywbtw4XC4Xbdu2JT8/P+jyWHUt7zVjxgwmTJiAiNC3b1/27t3L1q1b+eabbxg4cCARERHExcXRrVs3//Je/fv3Z86cOXg8nlr9GWOapnAmtiIgK2DbDRzrkg+7gJYiEtFQn6r6lKr2UtVeqampx3hYU8XpEO4bU8ClZ2bz+Mffc9c731RLbke6DNfVV19NmzZtWL16Nb/61a/qrXusy3DV1b5bt2689957lJSUsGvXLubNm+e/edzhcJCfn8/y5ctDPo4xpnGFM7EtAtr7ZjFGAeOAmcfSoXp/g84DqmZQXgnMOKYozRFzOIS7R3Xh6rNzmfrFBv741r+prPQmt127dh3RMlxTp05ly5YtnH766dWubQVzrMtw1dV+2LBhjBgxgrPOOovx48fTr18/IiIi/HVsGS5jmhlVDdsLGAGsAb4H/ugruwsY6XvfG+/I7iBQDKwKaPsZsBMo9dU5z1eeBywE1gHTAVdDcfTs2VPN8VdZWan3zPpWcya/o797bZl6Dlfq7t27NScnp1bdefPm6fnnn19nX7Nnz9bk5GT1eDyqqnreeedpYmJitTZ//etftUePHpqbm6tdu3ZVp9Opzz77bJ31c3Jy9Mwzz9SsrCyNiopSEdHc3Fz//g4dOuiWLVtUVfW1117TTp06qYho586dNSMjQ/Pz8/W5557zb3fo0EHff/99f/3WrVsroHl5eTpu3DgtLS1VVdWuXbtqdnZ2tfqqqg888IB26tRJO3fuXK3+2LFjdc2aNUf03Z8IJSUlOmDAAPV4PLphwwbt0aOHduvWTTt16qSPP/64v94f/vAHdbvdGhcXF3Lff/3rX7Vdu3a1vqNADz/8sLZr104B3blzp7/84MGDmpGRoXl5edqxY0c97bTT/HENHTpUO3XqpO3bt9eUlBTt1q2bduvWTSMiIjQ9Pb3BzzJu3DiNiorSqKgoveiii/yfv8qPP/6oiYmJmpCQ4I/rnHPOUafTqYC2atVK8/LytGvXrrpkyRL98ccfNSMjQ91ut0ZGRiqgbrdb27Rpo5GRkep0OjUpKanBNrGxsRoZGanR0dEaHR2taWlptdq0aNFCIyMjVUS0ffv2Ghsbq1FRUf4+4uPjtU+fPrp+/Xr/MdLS0jQqKkoBveiii/xxRURE6M9+9jMFyoAVQA/9z+/mD4FyvJeBqn4XfwcsBZ70bfvbADfhncGuvnzwEN7LSoJ3ZnxhkGNMAVYB31bV95XPAZK0odzTUIWT4WWJLXwqKyv1Hx9+pzmT39GbXlqqFZ7D6na7/b+0q9RMbJWVlbp27Vr/+3PPPVcHDRqkqqpvvPGGjhs3TmfOnFmtzb///W9NSkrSl156Se+4445qiXDOnDm16ufk5OiaNWu0bdu2WlxcrH369NHIyEgtLi7W+fPna+/evVVV1ePx6JdffqmrV6/W7t27a0REhG7fvl13796tbrdbXS6Xrl+/XgsLCzUvL089Ho/OmzdPMzMztX///rpo0SK9+OKLderUqbpq1Spt166dXn311dXqFxUVaW5urpaUlKiq+uurqn788cf685///Dj/yxy7Rx55RB988EFVVS0vL9eysjJVVd2/f7/m5OTo5s2bVVV1/vz5umXLlpAT26pVq7SgoEDLysqqfUc1LV26VNevX685OTnVEtsNN9ygp512mlZWVuqnn36qvXr1UlXVDz/8UF0ul27cuFE9Ho/27dtX582bp6qqycnJeuaZZ9b7Wb744guNjIzUTZs2aWFhoUZFRen1119fLaZJkybp8OHD9YorrtCcnBz9/PPPtV27drp+/XqNiIjQyMhIPXTokM6fP1/79OmjkyZN0qFDh2qrVq103bp1mpOTox07dtS8vDwtLS3V5ORkFZF626SlpanT6dTHH39c58+fr/Hx8ZqSkqKVlZXV2vTu3VtTU1M1Li5OV6xYobGxsTp79mwdMWKEAjpx4kR9+eWX9ZJLLtFJkyZpv379NCsrS9evX6+ZmZnqcDg0Pz9fy8rK1O12a0REhAKLgb7AAvUmlrOAZb4BxmagBBgKtAX2+hKX1Ghzpa+fDUAaMB84xzfwWQH8McgxvgCcvtd84JyAvv6oDfzOtyW1zDEREX49pAOTh5/G28u3cMUzC+jU+yd89PEn/jrBluFSVa688kq6du1K165dWbFiBf/85z8B+P777ykoKKg14aRz587k5uby61//mnvvvZf777/fP/Py/vvvp7y8HKi+vFefPn2IiooiMjKSNWvWkJeXx+mnn861117LY489BsChQ4e45ppruOiii1i9ejX9+/cnLS2NpKQkUlJS/M9xC5yw0r59e0SEyspKPB4PJSUlZGRkMGPGDCZOnMi8efPIysqqNsHF4/FQWlparX7V99MUJ6gETgKKiorC5XIBUF5eTmXlf+7Qqev5d3U51klAb7zxBjfccAMiQv/+/dm3bx9bt27F4/FQWVlJRUUF5eXlHDp0iNatW7N27Vr2799PcXFxvZ/lxRdfJDs7G7fbTdu2bXE6nRQWFvqPu2TJErZv387YsWP9E5/ee+89rrnmGnJzc/F4POTl5bFo0SL69u3Ltm3b2LBhA3369MHhcJCRkYGq8uOPPzJy5EicTicHDhzA4XDU26akpISIiAhatWpF3759KSkpQUT8k6Cq2gwbNqzad9SyZUuysrL47LPPiIiI4KOPPmLMmDHMnj2b7du3U1BQgMPhwO12c+jQIRwOh//fpU2bNlVdxanqV3jnNqTjHXUdBrYCsXiTWZGqrsd75i3R9zdAYJuqZCeAC4gEtuO9/eshYHyQY0QDUTXqg/dy1viG/huzxGaOixvOacf/jOrCuh0HWJnYj0tvmcLN05fz+dpdfPzJp+zcuZPS0lKKioo477zzcDgcfPHFF6xcuZKlS5fidDrp2rUrAMuWLfOvNVlTQUEBiYmJeDweli9f7k9ms2bNIiUlBYBJkyZRVFSEx+Phj3/8I5dffjlvvvkmgwcP5rLLLuPmm29m5cqV9OrlXSkoOjqab775hm+++YaMjIxqDzbdvXs37du3929XTTjJzMzk97//PfPnz2f48OEkJiYybNgwNm/eTHZ2tn/CSc362dnZpKen++tD05ygEmwS0KZNmygoKCArK4vJkycf9dMYjmUSUEVFBXv37qVHjx7+spSUFAYMGMDo0aPp378/PXv2JD09nfPOO4/TTz+d559/nsjISAB/cgv2WcrLy9m9ezcbNmygpKSE8vJyf/3Kykp+97vfcd9991WLZ+vWrWRlZVFRUQFAfn4+mzdvprKykn379nHttdeSn59PZmYm6enpbNq0idatWzN37lxSU1PxeDxERUWxevXqOtscOHCAhIQE7rrrLgoKCjh8+DCqSnFxcdA2Bw8e5O6776ZXr160a9eOAwcO0K5dO8rKytizZw/l5eXcdttt9O7dm8jISNLT09mxYwcOh4Pk5GQ8Hg+bNm2qupWmamWnIiBTVefjneewFWgJLFPVb311DgNOEWlVR5ssvKcXZ/vaZAKrAZevTbBjbA2oj6ruCahfJ0ts5ri5vG8OX902mFduG8+ZZw9g1orNXP7MAvreM5c7317Fsk17g07gqDnh5IUXXqCumaz33HMP8+bNIy8vj927d/O3v/2t3piqjvfyyy8zfrz3D70jmXDidDpp27ZttTIRYc+ePcyYMYO+ffvy/vvvc/DgQV544QX/8QInnATWX79+PVu2bPHXr9LUJqgEmwSUlZXFihUrWLduHdOmTWP79u11tK5fsP8GQv032bVrV7WJPeD9w+SVV17hww8/ZMmSJSxdupTNmzfz0Ucf8emnn/Lyyy+Tmppa7TsO9llcLhdXXXUVY8eOpX///kRERPjPCDz22GOMGDGiWkIO/CyBCwGICI899hitWrUiPT2d7du3s2fPHoqKisjKymLLli2MHDmSpUuXIiI4HA727NlTZ5vo6Gji4+N59NFHef1179oUDoeDLVu2BG0TFxfHVVddxapVq3jnnXdQVbKzs0lMTOQf//gHsbGxuN1uCgsLERG2bNlCVlYWTqeTBx98kP79+xMbG1v1bxL4D6Mikg+cjndG+gHgNBEZEFDnAJBRR5tNQFfgXF+bqr53BLSpeYzMgPoEqR9cQ+cqT4aXXWNrHKUVHn13xRa97l+LtP0fZmnO5Hd0wJSP9P7Zq3Xt9v3+ekc64aSqfs39weq/9NJLOmHCBE1OTtbdu3drenq6vvDCC6oafMLJ6aefrj/5yU/0jDPO0M6dO2tycrKmpKSox+PRr7/+WhMTE7Vt27aanZ2t55xzjg4cOFAXLVqkZ511ljocDgV0xIgRmpKSou+//7727dtXO3fu7J9ckZmZqSKid9xxh6anp/sv7ouItm7dWrOzs/W0007TTp06aYcOHTQlJUVdLpcC6nQ6NSYmRqOiotThcGh8fLzm5eXVapOUlOSfECAi6nQ6/RMioqKiNC8vT/Pz87Vr167aqVMnTUtLq7qeou3bt1eHw+GfDBEbG6tZWVnVJnmsXr1a4+Li/G1iY2N1/PjxKiKamJioiYmJmpeXp23bttX8/Hzt1q2b5ufna3Jysv+ziIhmZGRop06dNC4uTpOTk4O2yczM1MjISP/EnqrvOC4uTl9//XVV/c8koClTpmjr1q01OTlZ4+Li9M4779SzzjpLIyIiNCIiQuPi4jQ7O9s/gUJV9YUXXtDY2Fj/ZwH066+/1t27d6uIaH5+vrZr104TExO1devWmpOTowkJCepwONThcPivfz3//PMKaGJior766qt66aWXakREhLrdbo2NjVWXy6WTJ0/WnJwcHTBggA4fPlx3796tLpdLnU6nvvbaa3W2adGihV5wwQX6t7/9zR9XXFycrl27NmibyMhI/etf/6qDBw/WX/ziFyoi2qdPH+3QoYNecMEF6nA4NCcnxz8RpSqugoICHTdunKqq9u/fX0VEgdXqTd7fAenAzcCffGV7gI+BW3zbm/Bed8sP1gbvNbYU4HbgFrwTTcYDS4D8YMfw9XN71TF820uqjlHXq9GTzol4WWJrfHtLKvTVhRv10v+dr7m3vqM5k9/REf/8VJ/8ZJ2+t3Krtmqdri98vkZf+GqDPvNZoT7+8Tr95d+f19P6DNS/vL1K//TWSr1l+nK99vEP9L9fXqpJqW30kgnX6O9vvkVVVRcsWKBDhw6tldiKi4s1OTlZx40bp1OmTNHk5GQtLi7WBQsW6JAhQ2pNOOnXr586HA5duHCh/xdPQkKClpWV6Zw5c9TtdqvH49G3335bIyIitF+/fvrhhx9qTEyM3nLLLdqmTRv/L7yvvvpK3W63fvvtt/rVV19pTk6OpqSk6Nlnn61DhgzRpKQkLS4u1t/97ncK6KRJk3TOnDmalZWlHo9HN2/erCkpKbpw4UIdMGCARkZGakZGhpaVlemECRMU0IMHDwZts3z5cnW73epwOHTZsmW6e/dudTqdGhkZqZWVlfrqq69qQUGBqnonr6SmpurZZ5+tr7/+ugL6ySefaFpamgJ65pln6hdffKG5ubnq8Xj8f1jMmDFDBwwYoC1bttSsrCyNi4vTa6+9VgHdunWrPvzww5qYmKgej0f379+vGRkZ+sknn2jPnj3V4XDod999p99//71GRESo0+nUsrIyXb16tbZt29bfJicnRzMzM/WFF15QEdEZM2ZoYmKitmjRQocMGaIzZszQqv+/n376aXW5XPrBBx9oXFycnnnmmZqQkKB/+MMf/LMYy8rK9Nprr9UOHTqoqvePpOzsbH3vvfc0NTVVIyIitKysTL/++msFNCsryzv79557ND09XVVVp06dqr/85S81JydH3333XXU6nbp7924FNCoqSq+++mqdP3++5ufn66233qrXX3+9ut1uPXTokGZlZWmXLl00NzdXDxw44J+xWFpaqr/97W81OztbVbVam5SUFE1MTNQZM2boZ5995v9jpaKiQu+55x5t06ZNtTZxcXH69ddfa4sWLXTUqFHqcrm0RYsW/j/qzjrrLL311lt1zJgx/kk42dnZmpmZqR06dNCtW7dqRkZGVaJfDEwGtqg3qYzFOzMxwpfEDgAX4Z08shXvLMoIvJNBFtZoswFoA8wFfgacD7zn6+fsIPUj8F5fmwv8zLdPfPUj1BKbJbamZPuPpfr0Z4U68uHPNGeyN8nFdR2qaWP/x7/tcndSR0yCSkSURsS30rZX3K197v5QE9t115jWuSquOI3OLtD2t/yfjn3yS03LytMol0ujo6M1MzOz2hTyjh07anp6urpcLp0yZYqqqk6fPl07duyoTzzxhJ5//vn6xhtv+EcHIqIxMTHarl07zczM1LZt22peXp4mJCT4Zwm+8cYb/tGDiGhsbKyWlZVpTk6OxsTEqNPp1A4dOuisWbP8cdx+++3q8sXYrVs3veaaa1RVNTMzU51Op/+v5WHDhumXX36pqqoFBQW6Zs0aLSgo0OjoaB0/fryqqmZnZ2tUVJS+/fbbdbZJSEjQ1NRUVVXdt2+ff0SxdetWVa1+u0NBQYH27v3/2zvz8CrKq4H/zl2SkIXsiUkIhGwESEIKBGyDLG6sKqAsCuICWmulFgVF6qe0FdSyaVFcKrb0qaKpOyqtCIiUIDsYKKgosgYSCAQUs5Cc74+ZXJMYJFhj4Ob9Pc997sw775k5596598yZ951zsrVv377qdrtVVbVLly4aHx+vbrdbFy9erIGBgZ5I75lnng1tXxIAABYuSURBVPHIBQUFaUBAgOcP1+l06rx583TGjBmanJyseXl5evjwYY2Pj9f9+/drZmamRkdHa2JioqakpKiPj4+2bNlSCwoKtH///tqzZ0/Ny8vT6dOnq9PpVIfD4Yls1q1bpzfddJMGBASo2+3W+Ph4TU5O1szMTM3IyNCLLrpI09LSVEQ0JydHQ0NDdfv27RoZGamBgYGal5enY8eOVRHRzMxMTU/P0MeemKdFJ0o1qGVLDQ0LU5fbrW63W1sGB2tIWLhuLyjRtskp2jIkRFds/kQHDRmmLfz91el0alBQS/WxdQPUZZ8/ye3aa2yr1nrLHRP0odlPaERklDqdVlQoDof6+vmpy+1Wp8ulQS2DNb5NggaHhGqbtkm6YtOO78hUy/n4+GpUdIyGR0ZpfJsEdTqdet2N43R7QYn26H2pOu3I0+ly6QUxceq0I9XExEQNCAjQ7OxsTUxM1MmTJ+tTTz2l4eHhnmjV5XKpn5+fut1uDQkJ0bZt21ZP9y+wnVIs1izFVUCFrVcZcNyOtn6D9WjX53ZbP7Wc0Z22A1RbLl+/dVK59rZ8oKvd7rSjue3Af4HZWu2woCvwqp7hP7/2zWqD4ScgqqUfY3u0ZWyPtuwtPsnx0gp2XhbCgmee4Okpd+PjcuD7h4/xcTlwOuqMvUzZBMDS/3zEtEdn0rtHGut3F1MWm0X4pRPwi25L2gUtySsNpXzLAbITwtixYwfl5eW0bt2aSZMmAbBmzRpyc3MpLi5m0aJFDBkyhCFDhqCqJCQk8Oqrr5KZmUloaCiJiYnk5+fXUiMuLo7U1FS2bdvG7NmzKS0txdfXF1WlsrKSoUOHkpubW0tm4MCB5ObmEhUVRVZWFpGRkYwZM4b9+/cTGBjomTxTPZli7dq1lJeXk5SURGFhIf7+/ixfvpz09HQOHDhAeHg4H3zwAYMGDapXprS0lG7dunG8tILn//4iIkJE9AW8/PYSevW9gpbh0by9eivlFZspPnGSlq4W7NywBXG6mDr/LXxDYzi8fQcKlGgLBlw1lP79+jP62uG4nNbwfGFhIZ07d+bSSy/lnnvuISQkhOjoaFavXk1VVRUdOnRg5MiRFBUVMWPGDGJjYyksLGTMmDFs2bKFNWvWUFlZSVZWFqtWreKdd95h2IgRDBs+giNHDjPxwYcZMOJGru7ZiaoqZfDwUZRXlFNRqSR37kFmnytJ6nYJ5aeqKDtVZb9XsvNXvSlwRZNwRR/uXFxI8YmTOJw+jH78XY789wucQZEU97oXp38wc/bCnIfe58SJr4i+djpx8VaF9YMv3EP5wZ1c9vA7HPWJ41SgmxGz38U39QaiUm+w+iycQnD2YPyTu1F+6HOOr3uDyq+O8tWFN3Ji3Rsslq44DwUTcPPzBNTzWyh660+UnyzBMXIa5a/8Hh0yhTEv7QQSvlem8mQJgSOn4bdoJh8G9aHfYyuh+29p1f23tfq2Ah6/LIR7fz2WN998k0suuYTRo0dz1113ERkZyW233Vbvb3To0KE8/PDDpKWlbcWa4DFBVatzE+bUJyMijwO3qpXA3oOqPg48Xre/qqqIFABX1ZRR1Urgl/UqZlV8mXeabR6MYzM0KfFh/gB0jM3hyO5PCA9wewbsv49LelzI7k8HcMOAdjidTr6+5UI27z3Gui+LWf/lUf65YR8LVu8GoFVoC1r5lFIqfoxbYBecTR/BnzeVU7D9E/L3lXzbDmSOeZBBo26hquwk4vLh5Cn49NAJWof54+d2UlBQwPXXX8+CBQtwOBzVV5IAVFZW4na7yc7OrqVvTZnJk78tTfjII4/wyiuvEBkZyYZNmyn+upyT5af4fN9BJk2+j4dmP8XazwspOnyYrG459Bl0NUkZXbhjSG9KvvqaD/J3MeX1fFZ/cYTtb28kf/FdtB85mZ9Pe4/y8nI+Km5B5tT3OPjikyjCnpJT/P7lVbTcGcahAyX87qXVHF36LBEDJrBz2XOUHz1CQIfePDL1PiqK94P4UFVVzvj5Szn5aQHLvv6YqVuDaOF2EuBSDhUW4dO2K69s2Mu2eUupFBdHS5UV2/Zw2W9msm7+H0m+4tf8IjmDqbMmsrIsgUOFRXxcFU+ra4fh7rmbdx8YyaZdh7lz/jLu3dCCgzuO0aLraMJj05jz1EP8oyCSw0VFhPW9HVfm5TjKTnL4iev5Yv8hjm7+lM/8O1oXQy6H/e5EAadAgK+LNuH++Pq1oGV4NJmtgskaNZc/39qfW3slERYWjq/Lwd5PtjDN6eDRsf2JiIjEIfBgXgxflBdzb04ESeNe5L5xw7l5QHtSOlj1BYuLDnH7s/uYP2UsLrcb6Mx7r59i00cr6d0lhO5311eNqzb/8r0Ch9PBJVd2wjmqYUmUasuc+RgXpUbSvXt3evfuDVBr0lJ9lJeXM3jwYM/sYFUd3SDFYGtdp9YIMg3qbxyb4Zzh5ptv/sH9A3xd5CRHkJNsTfk/VVnF9oITlqPbXcxne0opLyvjwLFvau3jyFdllFZU1m6PSOFnt/+ZgsIi8h8by0EN4fI5HyICUb5VfPbXifQZfivbTkVzcschfIIj+HT9auDb2X1B4VFsO1DC4a/K2XvwMFPGXUOPq28jd48fOw6e4EDASUpWf8SCY6mUVjnYW1LOvg830PmPSzj44WZeeWcpIT2v53d5ZZxc8CRVTj8++SaQz/+1nsBDUVSJk9JvvuGLE07+vfUgBXv3sOe/G8ka/Es6du3Gofz/4HC6aBfmYFTOBdw/bzdOh4O4YF+uzEnmqpFdGfPSCfw+folJDz3EFUOG0u/n8wjOyCDrZ62Z9edF/OmRR8hd+A+KjygT+qbz8v4VDOjbhejkVE6UVrBx5RLy/fwJjY7jWFEBh79RqlT55usTBPkEceDYNxQXFhDmCqJEAvGNbMPad17A6duCE+rHweOluP2CcLjcVB4vJKttFD17JvK3Jd9wzYCudMnuzlMF7xJVuZH3oqL4v6GdGX3d5fi6HSQsHE94S5hwdWfGjbvsO+dG4H0ObrrcepzjvjFd2f1CZ44dO8Y9V+eQnd2OuRUnuX9oN8/3NeG1uQQFBjIwM9bz2Mgb7ZMo2vs5/Tq3JTk5ht8eK2RYz0zPM3uPP57LiGuu5orOrT3H7Z9xF1dfvYoe7eO4NOPMz/b1z5h4xj4/hsyCBQsa3NfHx4cxY8ac9TFU9S+NLdPg/me6V+kNLzPGZlDVBmVEUVU9dOiQqqqWlpaqj6+vPvn3f+obm/bp+Dkvqn9wuHYcOl7TH/yXZzyw1W8Wqis4Wrs98Ia6W0Yq4tC42xdom3vf1ujrHlVHQIiGXnyLpkx5V38+/X0NT87Sy+6eq4FhUfq7l/I0IDhMQfSiK67V6S++r06XS9O7/kLf2rxf/721QPsMuEpbt03WydNmaWr7jrptT5H6+fkpoMuWLdPc3Fx1Op06a9Ysjw0jRozQ0NBQbd26tc6aNUtHjhypDodDw8LCNC8vT59++ml1uVw6Z84cj0xQUJDOnDlT4+Pjtbi4WJcvX64ioq1atdKPPvpIY2NjdfTo0bWO0a5dO33//fc1ISFBi4uLPRNOFi1apKtWrdI2bdp4JpykpKRo3759NSUlRRcvXqyqqhUVFern56c+Pj66YsUKnTBhgiejTE2Z8ePH68UXX6xVVVW6cuVKDQ0N1YyMDF29erW+9tprOnny5FrfYUBAgG7dutWT4WTq1KkaFBSkp06d0oULF+qwYcM8fSsrKzUuLk7j4uJqZThZtGiR+vr6anl5ea1MNdV0795dly1b9p3zLD093TN2eb4DrNdz4D/8bF9NrsBP8TKOzaCqevPNN+uSJUs86z169NCIiIjvTDiZOHGipqWlaWpqqnbv3t0jM378eBURT/7B9IxM/cc7H2juuj3aqddAFYfTHrh3a+aFvXRx/gFtm9xORUQzMjM1ISHBk4cvMjLSM3VfRDQiIkITEhL0ggsu8OQc7NSpk7Zp08Yz287tdnve3W63tmjRQhMTEzU4OFgB7dChg3bq1EljYmI8jwcEBwer0+nUmJgYzcnJ8Uwk8PX19diSkJDgmUAQFRWl4eHhnmO0b99efX19NTU1VTMzM3XIkCGqqrpw4UIVEfXx8dGoqCjt2LGjJiUlaVBQkPr5+WliYqLGxcVpYGCgpqSkaEZGho4bN05FxPMZBAYGaocOHTQtLc3z6IK/v79GRkZ6JoPMnTtXw8LCdN++fRoZGak+Pj7q6+urw4cP13bt2mlFRYXGxMTo9OnTVVV10qRJnkcq4uLitE+fPp5JKjk5OZqUlKTZ2dnaq1cvT0qwO+64w5O3MSYmxjOpZ+3atZqYmKiJiYmanp6u69at85w7u3bt0tjYWK2srKx1jh08ePA7DvB8xji2c/hlHJtB1co9WDPiOFuZiRMn6pYtW350mbp6jRo1SgsLC79XZubMmZqenn5Wev0QmdmzZ+tzzz3XYJmax2goNWUGDhyoZWVlZ5R57bXX9P7771fVhn1eP4Tqxy/OhpqflzdgHNs5/DKOzVDN/Pnz602429QyZ9s/NzdX586d2+gyzz//vFZUVJxzeuXm5urRo0cb3P+H8Oyzz561zNl+Xuc656tjqy4F4NV07dpV169ff+aOBoPBYPAgIhtUtWtT63G2mFyRBoPBYPAqjGMzGAwGg1dhHJvBYDAYvArj2AwGg8HgVRjHZjAYDAavwjg2g8FgMHgVzWK6v4gUAbvP0C0COHyGPt5Kc7Ydmrf9xvbmS0Psb6Oq9ZezP4dpFo6tIYjI+vPxeY0fg+ZsOzRv+43tzdN28G77za1Ig8FgMHgVxrEZDAaDwaswju1bnm1qBZqQ5mw7NG/7je3NF6+134yxGQwGg8GrMBGbwWAwGLyKZu/YRKSfiHwiIjtFZHJT69PYiMjzIlIoIltrtIWJyBIR+cx+D21KHRsLEYkXkeUisl1EtonInXa719svIn4islZEtti2/95ubysia2zbXxYRn6bWtTEREaeIbBKRt+31ZmG/iHwpIvkisllE1tttXnveN2vHJiJO4EmgP9ABuFZEOjStVo3O34B+ddomA0tVNQVYaq97I6eAu1W1PXAh8Gv7+24O9pcBF6tqJyAL6CciFwKPAnNs248CY5tQx5+CO4HtNdabk/19VDWrxhR/rz3vm7VjA7oBO1X1C1UtB14CrmpinRoVVf0QKK7TfBWwwF5eAAz+SZX6iVDVAlXdaC+fwPqDi6MZ2G/XjfzKXnXbLwUuBl6x273S9mpEpBUwEHjOXheakf314LXnfXN3bHHA3hrr++y25ka0qhaA9ecPRDWxPo2OiCQAPwPW0Ezst2/DbQYKgSXA58AxVT1ld/H28/8x4B6gyl4Pp/nYr8B7IrJBRG6127z2vHc1tQJNjNTTZqaJejkiEgi8CvxWVY9bF+7ej6pWAlkiEgK8DrSvr9tPq9VPg4gMAgpVdYOI9K5urqerV9oP5KjqARGJApaIyI6mVqgxae4R2z4gvsZ6K+BAE+nSlBwSkRgA+72wifVpNETEjeXUXlDV1+zmZmM/gKoeAz7AGmcMEZHqC1xvPv9zgCtF5EusIYeLsSK4ZmG/qh6w3wuxLmq64cXnfXN3bOuAFHtmlA8wEniriXVqCt4CbrCXbwDebEJdGg17TGU+sF1VZ9fY5PX2i0ikHakhIi2AS7HGGJcD19jdvNJ2AFW9T1VbqWoC1u98maqOohnYLyIBIhJUvQxcDmzFi8/7Zv+AtogMwLpycwLPq+q0JlapURGRhUBvrMzeh4AHgTeAXKA1sAcYpqp1J5ic94hID2AlkM+34yxTsMbZvNp+EcnEmiDgxLqgzVXVP4hIIlYEEwZsAkaralnTadr42LciJ6rqoOZgv23j6/aqC3hRVaeJSDheet43e8dmMBgMBu+iud+KNBgMBoOXYRybwWAwGLwK49gMBoPB4FUYx2YwGAwGr8I4NoPBYDB4FcaxGc5ZRERFZFaN9YkiMvVH2vffROSaM/f8n48zzK4msLxOe4Jt3/gabU+IyI1n2N9tIjLmDH1uFJEnTrPtq/rafyxsu2pWjrhFRDZ6U+Z4w7mPcWyGc5kyYKiIRDS1IjWxq0I0lLHA7arap55thcCdZ1MqRVWfVtW/n8XxfzRqZOhoaP/rgfHA5ap6tHG0Mhi+i3FshnOZU1jl6yfU3VA34qqORESkt4isEJFcEflURB4RkVF2LbJ8EUmqsZtLRWSl3W+QLe8UkRkisk5EPhaRX9bY73IReRHrAe+6+lxr73+riDxqtz0A9ACeFpEZ9dhXhFUu5Ia6G0QkSUT+ZSetXSkiaXb7VBGZaC9n2zqutnXeWmMXsbb8ZyLypzr7nmVHUUtFJNJuyxKRj+z9vV4dYYnIByIyXURWYDnhYbaNW0Tkw3psqj7GcKwyKJer6uHT9TMYGgPj2AznOk8Co0Qk+CxkOmHV3coArgdSVbUbVrmS8TX6JQC9sEqZPC0iflgRVomqZgPZwC0i0tbu3w34narWqtknIrFYdb0uxqp1li0ig1X1D8B6YJSqTjqNro8Ad9cTBT4LjFfVLsBEYF49sn8FblPVnwOVdbZlASPsz2CEiFTnRA0ANqpqZ2AFVuYZgL8D96pqJpbjfrDGvkJUtZeqzgIeAPradd2uPI1NbYAnsJzawdP0MRgaDePYDOc0qnoc60/3N2chts6uvVaGVZrlPbs9H8uZVZOrqlWq+hnwBZCGlUdvjFjlXdZglTZJsfuvVdVd9RwvG/hAVYvsEigvAD0baN8uYC1wXXWbWNUHfgH809bjGSCmppyd9zFIVfPsphfr7HqpqpaoainwXyxnA1YqsZft5X8APeyLhhBVXWG3L6ij/8s1llcBfxORW7DSc9VHEVaKpuGnNdxgaESae9kaw/nBY8BGrAilmlPYF2Z2cuOa41Q1c/1V1VivovY5XzefnGKVMhmvqv+uucHOL/j1afT7X+veTMcqdll9a8+BVScs63tkznTMmp9BJaf/rTckp57HblW9TUS6Y0W5m0UkS1WP1Ol/Eqsq/X9EpFBVX2jAMQyGHw0TsRnOeezErLlYtwmr+RLoYi9fhVUR+mwZJiIOe9wtEfgE+DfwK7HK2yAiqXZG9O9jDdBLRCLsW4rXYt3maxCqugMrqhpkrx8HdonIMFsHEZFOdWSOAidE5EK7aWQDD+fg22z21wH/UdUS4KiIXGS3X386/UUkSVXXqOoDwGFql32qqV8R0A+YLiJ9G6ibwfCjYCI2w/nCLOCOGut/Ad4UkbVYEzBOF019H59g/YFHY41VlYrIc1i3KzfakWARMPj7dqKqBSJyH1YJFAHeVdWzLQEyDSu7fDWjgKdE5H4sp/0SsKWOzFjgLyLyNVZ9tZIGHOdroKOIbLD7j7Dbb8AaZ/THui1702nkZ4hICpadS+vRyYOq7hKRK4F3RWSoqq5pgH4Gw/+Mye5vMJyniEigqlbPBp0MxKjqnU2slsHQ5JiIzWA4fxloR4ouYDdwY9OqYzCcG5iIzWAwGAxehZk8YjAYDAavwjg2g8FgMHgVxrEZDAaDwaswjs1gMBgMXoVxbAaDwWDwKoxjMxgMBoNX8f8OBLYhTghAUQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.145 0.12  0.115 0.115 0.115 0.111 0.109 0.108 0.108 0.107 0.107 0.107\n",
      " 0.107 0.107 0.107 0.108 0.107 0.107 0.108 0.108 0.108 0.108 0.108 0.108\n",
      " 0.108]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in tqdm(neighbors):\n",
    "    knn_kd = KNeighborsClassifier(n_neighbors=k,algorithm='kd_tree')\n",
    "    scores_kd = cval_score(knn_kd, d_train_ng_dense,y_train, cv=2, scoring='accuracy')\n",
    "    cv_scores.append(scores_kd.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.621\n"
     ]
    }
   ],
   "source": [
    "knn_kd_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_kd_optimal.fit(d_train_ng_dense,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_kd_optimal.predict(d_train_ng_dense)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_kd_optimal.predict(d_test_ng_dense)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 1.000\n",
      "The value of TNR is 0.014\n",
      "The value of FPR is 0.986\n",
      "The value of FNR is 0.000\n",
      "\n",
      "Test error is 10.933\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGHtJREFUeJzt3XuYVdV9//H3Z0ADopGLYhAQUYlGk4iXCDUx9VbES4M20ZikipF0nl9+XtMag9qGBLXamMaKjTYYEfCOafhJ/ZlYxFg18Ua84bUgioxQMDCgAgoz8+0fZ4EHPGc4yMycM8vPy2c/55y11957HR+e7/nOd6+9tyICMzPr/OqqPQAzM2sbDuhmZplwQDczy4QDuplZJhzQzcwy4YBuZpYJB3Qzs0w4oJuZZcIB3cwsE12rPYByum7b35ew2ofs13tQtYdgNejZ//mDtnYf6/40v+KYs81Oe2z18dqDM3Qzs0zUbIZuZtahWpqrPYKt5oBuZgbQ3FTtEWw1B3QzMyCipdpD2GoO6GZmAC0O6GZmeXCGbmaWCZ8UNTPLhDN0M7M8RAazXHxhkZkZFE6KVrpshqTXJc2R9Iyk2amtt6SZkuam116pXZImSJon6TlJBxbtZ3TqP1fS6M0d1wHdzAwKJZdKl8ocERFDI+Lg9HksMCsihgCz0meAY4EhaakHrofCDwAwDhgGHAKMW/8jUI4DupkZFE6KVrp8NKOAKen9FODEovapUfAY0FNSP+AYYGZELI+IRmAmMLK1Azigm5lBW2foAfynpD9Kqk9tu0TEYoD02je19wcWFm3bkNrKtZflk6JmZrBFl/6nIF1f1DQxIiYWff5iRCyS1BeYKenl1nZXoi1aaS/LAd3MDLboStEUvCe2sn5Rel0qaTqFGvgSSf0iYnEqqSxN3RuAgUWbDwAWpfbDN2l/sLVxueRiZgZENFe8tEZSD0k7rH8PjACeB2YA62eqjAbuTu9nAKen2S7DgZWpJHMfMEJSr3QydERqK8sZupkZtOWFRbsA0yVBIcbeFhG/lfQkME3SGOAN4OTU/17gOGAesBr4NkBELJd0KfBk6jc+Ipa3dmAHdDMzaLObc0XEfGD/Eu3LgKNKtAdwVpl9TQImVXpsB3QzM/Cl/2Zm2WheV+0RbDUHdDMz8P3Qzcyy4ZKLmVkmnKGbmWXCAd3MLA/hk6JmZplwDd3MLBMuuZiZZcIZuplZJpyhm5llwhm6mVkmmip/wEWtckA3MwNn6GZm2XAN3cwsE87Qzcwy4QzdzCwTztDNzDLhWS5mZpmIqPYItpoDupkZuIZuZpYNB3Qzs0z4pKiZWSaam6s9gq3mgG5mBi65mJllwwHdzCwTrqGbmeUhWjwP3cwsDy65mJllwrNczMwy4Qzd2ts5Z49hzJhvIokbb7yNCdf+stpDsg6ywye3Z9zPLmKvvfcgIhj3vX/k0COG8dVvfYXlyxoBuPaKX/DIrEfZdeCnmP7Q7bz+6gIA5vzxBS77wVXVHH7n44Bu7Wm//fZmzJhv8meHHs/ateu4955bufc3s5g377VqD806wIWXnc/vH3iMC75zCV236Ur37t049Ihh3DzxDqZef/uH+jcseJOvH31Gxw80F218cy5JXYDZwJsRcYKkwcAdQG/gKeC0iFgr6RPAVOAgYBnw9Yh4Pe3jImAM0AycGxH3tXbMujb9Bht/mX0k/UDSBEnXpPefaa/j5WiffYbw+ONPsWbNezQ3N/PQw49x4qiR1R6WdYAe22/HQcOHMv22/wCgaV0T77z9bpVHlbmWlsqXypwHvFT0+Z+AqyNiCNBIIVCTXhsjYi/g6tQPSfsCpwL7ASOB69KPRFntEtAl/YDCL5GAJ4An0/vbJY1tj2Pm6IUXXuaww4bTu3cvunfvxrEjj2TAgF2rPSzrAAMG9adx2QrGX3MJd86czLh/Hkv37boBcOqZX+OuB6by46svZocdd9iwTf/d+nHnzMncOP3nHDBs/2oNvfNqicqXzZA0ADge+GX6LOBI4FepyxTgxPR+VPpMWn9U6j8KuCMi3o+I14B5wCGtHbe9MvQxwBci4sqIuCUtV6bBjNnMtpa8/PI8rrrq5/z2N7dz7z238uxzL9Lc1PnPxNvmdenahX0+92numjydr//FGaxZ/R5nnn0a0yb/mhOGncwpR43mrSXLuOBH5wDw1pJlHHPQSXz9L87gp+MmcOV1P6LH9ttV+Vt0Ms3NlS+b9y/AhcD6dL4PsCIi1j9FowHon973BxYCpPUrU/8N7SW2Kam9AnoLUCqV7McHX/BDJNVLmi1pdkvLqnYaWudy0+Q7OGTYSI446qs0Nq5gruvnHwtLFi1lyeK3mPP0iwDMvOd37PP5vVn+p0ZaWlqICH5969189oB9AVi3dh0rG98G4KXnXmHhgjcZtOduVRt/ZxQtLRUvxbEqLfXr9yPpBGBpRPyxaPcqdcjNrGttm5La66To+cAsSXP54BdmN2Av4OxyG0XERGAiQNdt+3f+y7bawM479+Gtt5YxcOCunHjisXzpsK9Ue0jWAZa9tZwlby5h0J67seDVNxh22MHM/+/X2KlvH/60dBkARx7758x7eT4Avfr0ZGXj27S0tNB/t10ZNHggDQverOZX6Hy24ErR4lhVwheBr0g6DugGfJJCxt5TUteUhQ8AFqX+DcBAoEFSV2BHYHlR+3rF25TULgE9In4r6dMUSiz9KfzSNABPRoRrBlvgrjtvoHefXqxb18S5517CihUrqz0k6yBXXnI1V1w3jm222YaGBYv44fmXM/ay77H3Z4cQESxauJhLv/8TAA4cPpSzLvwOTU3NtDS3cNmFP+HtFe9U+Rt0Mm10L5eIuAi4CEDS4cAFEfEtSXcBX6NwfnE0cHfaZEb6/Gha/0BEhKQZwG2Sfkah4jGEwjnJshQ1+hw9Z+hWyn69B1V7CFaDnv2fP5QqT2yRVeO/VXHM6fHDWys6XlFAP0HSHnwwbfFp4K8j4n1J3YCbgQMoZOanRsT8tP0lwJlAE3B+RPymteN5HrqZGUA7TDiIiAeBB9P7+ZSYpRIR7wEnl9n+cuDySo/ngG5mBr59rplZNnz7XDOzPITv5WJmlgln6GZmmXBANzPLhB9wYWaWBz9T1MwsFw7oZmaZ8CwXM7NMOEM3M8uEA7qZWR6i2SUXM7M8OEM3M8uDpy2ameXCAd3MLBOdv4TugG5mBhBNnT+iO6CbmYEzdDOzXPikqJlZLpyhm5nlwRm6mVkunKGbmeUhmqo9gq3ngG5mBoQzdDOzTDigm5nlwRm6mVkmHNDNzDIRzar2ELaaA7qZGc7QzcyyES3O0M3MsuAM3cwsExGdP0Ovq/YAzMxqQbRUvrRGUjdJT0h6VtILkn6c2gdLelzSXEl3Sto2tX8ifZ6X1u9etK+LUvsrko7Z3HdwQDczA1qaVfGyGe8DR0bE/sBQYKSk4cA/AVdHxBCgERiT+o8BGiNiL+Dq1A9J+wKnAvsBI4HrJHVp7cAO6GZmFE6KVrq0up+Cd9PHbdISwJHAr1L7FODE9H5U+kxaf5QkpfY7IuL9iHgNmAcc0tqxHdDNzGi7gA4gqYukZ4ClwEzgVWBFxIZbgDUA/dP7/sBCgLR+JdCnuL3ENiU5oJuZARGVL5LqJc0uWuo33lc0R8RQYACFrPozpQ6ZXkv9QkQr7WV5louZGVs2Dz0iJgITK+i3QtKDwHCgp6SuKQsfACxK3RqAgUCDpK7AjsDyovb1ircpyRm6mRmFaYuVLq2RtLOknul9d+Bo4CXgd8DXUrfRwN3p/Yz0mbT+gYiI1H5qmgUzGBgCPNHasSvO0CV9IiLer7S/mVln0tx293LpB0xJM1LqgGkRcY+kF4E7JF0GPA3cmPrfCNwsaR6FzPxUgIh4QdI04EWgCTgrIppbO/BmA7qkQ9IBdwR2k7Q/8J2IOOcjfFEzs5rUVhcWRcRzwAEl2udTYpZKRLwHnFxmX5cDl1d67EpKLhOAE4Bl6QDPAkdUegAzs86gLWe5VEslJZe6iFhQmBa5Qatpv5lZZxOtzh/pHCoJ6AtT2SVSTegc4L/bd1hmZh2rljPvSlUS0L9LoeyyG7AEuD+1mZllo7ml80/622xAj4ilpLOuZma5+liUXCTdQImrkyKivkR3M7NOqSWD2+dWUnK5v+h9N+AkNr6/gJlZp5fD/dArKbncWfxZ0s0UbjZjZpaNj0XJpYTBwKC2HohZJWY/f0u1h2CZ+liUXCQ18kENvY7Cpalj23NQZmYdLftZLukm6/sDb6amlnTTGDOzrOQQ2Fr9SUrBe3q6t2+zg7mZ5aolVPFSqyr5G+MJSQe2+0jMzKqorW6fW01lSy5FN2L/EvA3kl4FVlF4ikZEhIO8mWWjpdoDaAOt1dCfAA7kgweZmpllK0o+8a1zaS2gCyAiXu2gsZiZVU1TDZdSKtVaQN9Z0t+WWxkRP2uH8ZiZVUXuGXoXYHtKP3nazCwrudfQF0fE+A4biZlZFeWeoXf+b2dmVqHcM/SjOmwUZmZV1pxBDls2oEfE8o4ciJlZNWXwBLqPdLdFM7PstOScoZuZfZzkcKMqB3QzM/I/KWpm9rHRIpdczMyy0FztAbQBB3QzMzzLxcwsG57lYmaWCc9yMTPLhEsuZmaZyGHaYiXPFDUzy16zKl9aI2mgpN9JeknSC5LOS+29Jc2UNDe99krtkjRB0jxJzxU/w1nS6NR/rqTRm/sODuhmZhQy9EqXzWgC/i4iPgMMB86StC8wFpgVEUOAWekzwLHAkLTUA9dD4QcAGAcMAw4Bxq3/ESjHAd3MjLYL6BGxOCKeSu/fAV4C+gOjgCmp2xQ+eF7zKGBqFDwG9JTUDzgGmBkRyyOiEZgJjGzt2A7oZmZAqPJFUr2k2UVLfal9StodOAB4HNglIhZDIegDfVO3/sDCos0aUlu59rJ8UtTMjC07KRoRE4GJrfWRtD3w78D5EfG2yt9aoNSKaKW9LGfoZmYULv2vdNkcSdtQCOa3RsSvU/OSVEohvS5N7Q3AwKLNBwCLWmkvywHdzIzCPPRKl9aokIrfCLwUET8rWjUDWD9TZTRwd1H76Wm2y3BgZSrJ3AeMkNQrnQwdkdrKcsnFzIw2nYf+ReA0YI6kZ1LbxcCVwDRJY4A3gJPTunuB44B5wGrg21B4apykS4EnU7/xm3uSnAO6mRltF9Aj4hFK17+hxLOaIyKAs8rsaxIwqdJjO6CbmeF7uZiZZcP3cjEzy4QfcGFmlomWDIouDuhmZuRxt0UHdDMzfFLUzCwbztDNzDLRpM6fozugm5nhkouZWTZccjEzy4SnLZqZZaLzh3MHdDMzwCUXM7NsNGeQozugm5nhDN3MLBvhDN3MLA/O0K1d3TDxnzn+uKNZ+tafGHrAhx50Yhka8dXR9NhuO+rq6ujSpQvTJk3g2olTeeCRR6lTHb177cjll/wdfXfuwwMPP8q1N0ylToW+Y8+r58D9PwvA5w87niF77A5Av1125l9/8qPqfalOIodpiyo8/aj2dN22f20OrAMd9qVhvPvuKm666RoH9GTNooerPYR2NeKro7nzxgn06rnjhrZ3V61i+x49ALjlrrt59bU3GHfhOaxevYbu3bshiVfmvcYF//CP/MftNwDwhaNP4sn7p1flO1TDNjvtsdWPp/ju7qdUHHOuf31aTT4Owxl6DXv4kccZNGhAtYdhVbY+mAOsWfMeSqFku+26f9D+3ntsWGEfSVMGGXqHB3RJ346Imzr6uGadgSTqv3cJkjh51LGcPOo4AK75xWRm/HYWO/TowaRrr9zQ//7/+j3X/NtkljWu4Lqfjt/QvnbtWk4581y6dqljzGmncNSXD+3w79LZ5HBStMNLLpLeiIjdyqyrB+oB1GXHg+rqepTq9rEyaNAA7v5/U1xySXIvuSx9axl9d+7DssYV/M35F3Px977LwUM/t2H9DVPv5P21azn7O6dttN3sZ+bwbzfdxi+vuWKj/Sx8czFjzh3LL6+5gt0G7Nqh36UjtUXJ5czdv1ZxMJz0+q9q8s+huvbYqaTnyixzgF3KbRcREyPi4Ig42MHcPo767twHgD69enLUlw9lzouvbLT++BGHc/+Dv//QdgcP/RwL31xM44qVG+1nYP9+fOGAz/Py3FfbeeSdX2zBf7WqXQI6haB9OvCXJZZl7XRMs05t9Zr3WLVq9Yb3f3jiKYbssTsLFr65oc/vHn6Mwem8yhsNi1j/F/aLr8xj3bomeu74SVa+/Q5r164FoHHFSp6e8yJ77l7yj2Ir0rIFS61qrxr6PcD2EfHMpiskPdhOx8zOLTf/nD//8p+x0069eX3+bH48/qfcNPmOag/L2smy5Y2cd/GlADQ3NXPciMP50vCDOf/iy3j9jQZUJ3b9VF9++P1zAJj54CPM+M0sunbtSrdPbMtPx49FEvMXLGT8T65FdSJagjF/fQp7Dh5Uza/WKTTX6Iy/LeFpi9ap5F5Dt4+mLWro3xx0UsUx57YF02uyhu5pi2Zm5DHLxQHdzIzaro1XygHdzIw8Lv13QDczwyUXM7Ns5DDLpb3moZuZdSotRMXL5kiaJGmppOeL2npLmilpbnrtldolaYKkeekCzAOLthmd+s+VNHpzx3VANzOjzS8smgyM3KRtLDArIoYAs9JngGOBIWmpB66Hwg8AMA4YBhwCjFv/I1COA7qZGW176X9EPAQs36R5FDAlvZ8CnFjUPjUKHgN6SuoHHAPMjIjlEdEIzOTDPxIbcQ3dzIwOmeWyS0QsBoiIxZL6pvb+wMKifg2prVx7Wc7QzcyAiKh4kVQvaXbRUr8Vhy511Wm00l6WM3QzM6B5CzL0iJgITNzCQyyR1C9l5/2Apam9ARhY1G8AsCi1H75J+4OtHcAZupkZbTvLpYwZwPqZKqOBu4vaT0+zXYYDK1Np5j5ghKRe6WToiNRWljN0MzOgLW9UKOl2Ctn1TpIaKMxWuRKYJmkM8AZwcup+L3AcMA9YDXw7jWe5pEuBJ1O/8RGx6YnWjTigm5nRtidFI+IbZVZ96NFjUfglOavMfiYBkyo9rgO6mRm+9N/MLBs5XPrvgG5mhu+2aGaWDQd0M7NM1OrjOLeEA7qZGc7Qzcyy4VkuZmaZaI7O/1RRB3QzM1xDNzPLhmvoZmaZcA3dzCwTLS65mJnlwRm6mVkmPMvFzCwTLrmYmWXCJRczs0w4Qzczy4QzdDOzTDRHc7WHsNUc0M3M8KX/ZmbZ8KX/ZmaZcIZuZpYJz3IxM8uEZ7mYmWXCl/6bmWXCNXQzs0y4hm5mlgln6GZmmfA8dDOzTDhDNzPLhGe5mJllwidFzcwy4ZKLmVkmfKWomVkmnKGbmWUihxq6cvhVyp2k+oiYWO1xWG3xvwvbVF21B2AVqa/2AKwm+d+FbcQB3cwsEw7oZmaZcEDvHFwntVL878I24pOiZmaZcIZuZpYJB/QaJ2mkpFckzZM0ttrjseqTNEnSUknPV3ssVlsc0GuYpC7Az4FjgX2Bb0jat7qjshowGRhZ7UFY7XFAr22HAPMiYn5ErAXuAEZVeUxWZRHxELC82uOw2uOAXtv6AwuLPjekNjOzD3FAr20q0eZpSWZWkgN6bWsABhZ9HgAsqtJYzKzGOaDXtieBIZIGS9oWOBWYUeUxmVmNckCvYRHRBJwN3Ae8BEyLiBeqOyqrNkm3A48Ce0tqkDSm2mOy2uArRc3MMuEM3cwsEw7oZmaZcEA3M8uEA7qZWSYc0M3MMuGAbm1OUrOkZyQ9L+kuSdttxb4Ol3RPev+V1u44KamnpP/7EY7xI0kXfNQxmtUKB3RrD2siYmhEfBZYC/yf4pUq2OJ/exExIyKubKVLT2CLA7pZLhzQrb09DOwlaXdJL0m6DngKGChphKRHJT2VMvntYcM94F+W9AjwV+t3JOkMSf+a3u8iabqkZ9NyKHAlsGf66+Cq1O/7kp6U9JykHxft65J0n/n7gb077P+GWTtyQLd2I6krhXu5z0lNewNTI+IAYBXw98DREXEgMBv4W0ndgBuAvwQOAz5VZvcTgP+KiP2BA4EXgLHAq+mvg+9LGgEMoXAb4qHAQZK+LOkgCrdROIDCD8YX2virm1VF12oPwLLUXdIz6f3DwI3ArsCCiHgstQ+n8NCO30sC2JbC5ez7AK9FxFwASbcA9SWOcSRwOkBENAMrJfXapM+ItDydPm9PIcDvAEyPiNXpGL4/jmXBAd3aw5qIGFrckIL2quImYGZEfGOTfkNpu1sEC7giIn6xyTHOb8NjmNUMl1ysWh4DvihpLwBJ20n6NPAyMFjSnqnfN8psPwv4btq2i6RPAu9QyL7Xuw84s6g2319SX+Ah4CRJ3SXtQKG8Y9bpOaBbVUTEW8AZwO2SnqMQ4PeJiPcolFj+fzopuqDMLs4DjpA0B/gjsF9ELKNQwnle0lUR8Z/AbcCjqd+vgB0i4ingTuAZ4N8plIXMOj3fbdHMLBPO0M3MMuGAbmaWCQd0M7NMOKCbmWXCAd3MLBMO6GZmmXBANzPLhAO6mVkm/hf4ATWoJw0T/AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [18.4] Word2Vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "i=0\n",
    "train_list=[]\n",
    "test_list=[]\n",
    "for sent in dw_train['CleanedText'].values:\n",
    "    train_list.append(sent.split())\n",
    "for sent in dw_test['CleanedText'].values:\n",
    "    test_list.append(sent.split())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim.models import Word2Vec\n",
    "w2v_model=Word2Vec(train_list,min_count=5,size=50, workers=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of words that occured minimum 5 times  5352\n",
      "sample words  ['grew', 'loma', 'linda', 'product', 'excit', 'see', 'avail', 'amazon', 'bad', 'news', 'low', 'fat', 'big', 'frank', 'way', 'differ', 'normal', 'one', 'newli', 'vegetarian', 'girlfriend', 'tri', 'embarrass', 'werent', 'near', 'good', 'actual', 'made', 'littl', 'sick', 'theyr', 'dens', 'strang', 'stick', 'linkett', 'find', 'somewher', 'els', 'dont', 'think', 'peppermint', 'tea', 'help', 'upset', 'stomach', 'sleep', 'love', 'order', 'sever', 'time']\n"
     ]
    }
   ],
   "source": [
    "w2v_words = list(w2v_model.wv.vocab)\n",
    "print(\"number of words that occured minimum 5 times \",len(w2v_words))\n",
    "print(\"sample words \", w2v_words[0:50])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.4.1] Average W2V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14000\n",
      "50\n"
     ]
    }
   ],
   "source": [
    "# compute average word2vec for each review in train\n",
    "d_train = []; # the avg-w2v for each sentence/review is stored in this list\n",
    "for sent in train_list: # for each review/sentence\n",
    "    sent_vec = np.zeros(50) # as word vectors are of zero length\n",
    "    cnt_words =0; # num of words with a valid vector in the sentence/review\n",
    "    for word in sent: # for each word in a review/sentence\n",
    "        if word in w2v_words:\n",
    "            vec = w2v_model.wv[word]\n",
    "            sent_vec += vec\n",
    "            cnt_words += 1\n",
    "    if cnt_words != 0:\n",
    "        sent_vec /= cnt_words\n",
    "    d_train.append(sent_vec)\n",
    "print(len(d_train))\n",
    "print(len(d_train[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6000\n",
      "50\n"
     ]
    }
   ],
   "source": [
    "# compute average word2vec for each review in test\n",
    "d_test = []; # the avg-w2v for each sentence/review is stored in this list\n",
    "for sent in test_list: # for each review/sentence\n",
    "    sent_vec = np.zeros(50) # as word vectors are of zero length\n",
    "    cnt_words =0; # num of words with a valid vector in the sentence/review\n",
    "    for word in sent: # for each word in a review/sentence\n",
    "        if word in w2v_words:\n",
    "            vec = w2v_model.wv[word]\n",
    "            sent_vec += vec\n",
    "            cnt_words += 1\n",
    "    if cnt_words != 0:\n",
    "        sent_vec /= cnt_words\n",
    "    d_test.append(sent_vec)\n",
    "print(len(d_test))\n",
    "print(len(d_test[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.4.2] KNN (Brute-Force)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 19.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VPXV+PHPyUwyIXsCSchCAiGJJkDCEpGiQDWyKIqK4C4o1q6WotaK7a9PXfooVetStbWtsqgoVR9RKgpFRCuK7ItFXNiEQFgCgbBlP78/ZjJOIISBMmQC5/16zStz7/3ee0+SIYfvekVVMcYYY1qbkJYOwBhjjDkRlsCMMca0SpbAjDHGtEqWwIwxxrRKlsCMMca0SpbAjDHGtEqWwIwxxrRKlsCMMca0SpbAjDHGtErOlg7gZGnXrp127NixpcMwxphWZenSpWWqmtjScZyI0yaBdezYkSVLlrR0GMYY06qIyLctHcOJsiZEH4cOHWLAgAHU1dUBMGTIEOLi4rj00kv9Or+qqoprrrmG7Oxszj33XDZu3NhkuTFjxpCUlETXrl0b7b/vvvtIS0uje/fudO/enXfffReAXbt2ccEFFxAVFcXtt9/e6JyLLrqI8vLy4/xOjTGm9bME5mPixIkMHz4ch8MBwN13381LL73k9/kvvPAC8fHxrF27ljvuuIN77rmnyXI333wzs2bNavLYHXfcwYoVK1ixYgWXXHIJAOHh4Tz44IM89thjR5S/6aab+POf/+x3jMYYc7qwBOZj6tSpXH755d7t4uJioqOj/T7/7bffZvTo0QCMGDGCuXPn0tRq//379ychIcHv60ZGRnL++ecTHh5+xLFhw4bx6quv+n0tY4w5XVgC86iurmb9+vX8NwNBtmzZQocOHQBwOp3Exsaya9eu47rGM888Q0FBAWPGjPGraTA+Pp6qqqrjvo8xxrR2lsA8ysrKiIuL+6+u0VRtS0T8Pv8nP/kJ69atY8WKFaSkpHDXXXf5dV5SUhJbt271+z7GGHM6sATm0aZNGyorK/+ra6Snp7N582YAamtr2bt373E1FSYnJ+NwOAgJCeG2225j0aJFfp1XWVlJmzZtTihmY4xprSyBecTHx1NXV+dXErv33nuZPn36EfuHDRvGlClTAHjjjTe48MILj6sGVlpa6n0/ffr0I0YpNkVV2bZt23/V9GmMMa3RGZ/AKiprePL9r1mxeQ+DBg1i/vz53mP9+vVj5MiRzJ07l/T0dGbPng3A559/Tvv27Y+41q233squXbvIzs7m8ccfZ8KECQBs3brVO6IQ4LrrruN73/seX331Fenp6bzwwgsA/OpXv6Jbt24UFBQwb948nnjiCe85HTt25M4772Ty5Mmkp6fzxRdfALB06VL69OmD03naTOkzxhi/SFP9Nift4iJDgKcAB/C8qk447Hh/4EmgALhWVd/wOVYHfO7Z3KSqw5q7V1FRkZ7IROa9h2oovP9f/L+hefSK2svjjz9+zKHzgwcP9iazlvaLX/yCYcOGUVxc3NKhGGNaIRFZqqpFLR3HiQjYf9tFxAE8CwwESoDFIjJDVb/wKbYJuBn4ZROXOKSq3QMVX4OYcCcuZwg791XRo18PLrjgAurq6rxzwZoSLMkLoGvXrpa8jDFnpEC2O/UG1qrqegARmQZcDngTmKpu9ByrD2AczRIREqNd7NhXBbhXyWhNbrvttpYOwRhjWkQg+8DSgM0+2yWeff4KF5ElIvKZiFzRVAER+aGnzJKdO3eecKCJ0S52ehKYMcaY1iGQCayp4XfH0+GW4WmXvR54UkQ6H3Ex1b+papGqFiUmnvhiyknRLnbs+++G0BtjjDm1ApnASoAOPtvpgN+zbVV1q+freuBDoMfJDM6X1cCMMab1CWQCWwzkiEgnEQkDrgVm+HOiiMSLiMvzvh1wHj59ZydbUnQ45QdrqK5tsa44Y4wxxylgCUxVa4HbgdnAGuA1VV0tIg+IyDAAETlHREqAkcBfRWS15/Q8YImIrATmARMOG714UiVGuwAo22+1MGOMaS0COvtVVd8F3j1s3//4vF+Mu2nx8PM+BboFMjZfSZ4EtnNfFalxtiSTMca0Bmf8ShzwXQ1sh/WDGWNMq2EJjO8SmA3kMMaY1sMSGNAuqqEGZkPpjTGmtbAEBoQ6QkiIDLMamDHGtCKWwDySfJaTMsYYE/wsgXnYZGZjjGldLIF5WAIzxpjWxRKYR0MCC+Tz0Ywxxpw8lsA8EqNcVNfVU3GotqVDMcYY4wdLYB5JMeGADaU3xpjWwhKYR2KUTWY2xpjWxBKYR1KMLSdljDGtiSUwD1tOyhhjWhdLYB7RLifhoSHstEeqGGNMq2AJzENESIx2saPCBnEYY0xrYAnMR2KUy2pgxhjTSlgC85EUHc6OCktgxhjTGlgC85EYbTUwY4xpLSyB+UiKdrHnYA1VtXUtHYoxxphjsATmo2Eofdn+6haOxBhjzLFYAvPRMJnZ5oIZY0zwswTmIzHKsx6iDaU3xpigZwnMh3c1DhvIYYwxQc8SmI+2UWGIYEPpjTGmFbAE5iPUEUJCRJjVwIwxphWwBHYY93JSlsCMMSbYWQI7jE1mNsaY1sES2GGSosPZaaMQjTEm6FkCO0xDDUxVWzoUY4wxzbAEdpjEaBc1dcreQzUtHYoxxphmWAI7TJJnLtgOW43DGGOCmiWww3gnM1sCM8aYoBbQBCYiQ0TkKxFZKyLjmzjeX0SWiUitiIxo4niMiGwRkWcCGaev72pgNpDDGGOCWcASmIg4gGeBi4F84DoRyT+s2CbgZuCVo1zmQeCjQMXYFKuBGWNM6xDIGlhvYK2qrlfVamAacLlvAVXdqKqrgPrDTxaRXkAy8K8AxniEKJeTNqEOm8xsjDFBLpAJLA3Y7LNd4tl3TCISAvwRuDsAcR3r3jaZ2RhjWoFmE5iIOETk0RO8tjSxz9/JVT8F3lXVzc0VEpEfisgSEVmyc+fO4w7waBKjXdaEaIwxQc7Z3EFVrRORXiIievwze0uADj7b6cBWP8/9HtBPRH4KRAFhIrJfVRsNBFHVvwF/AygqKjppM4+Tol18s2P/ybqcMcaYAGg2gXksB94WkdeBAw07VfXNY5y3GMgRkU7AFuBa4Hp/glLVGxrei8jNQNHhySuQEqNdfLpu16m6nTHGmBPgTwJLAHYBF/rsU6DZBKaqtSJyOzAbcAATVXW1iDwALFHVGSJyDjAdiAcuE5H7VbXLiXwjJ1NStIu9h2qorKkjPNTR0uEYY4xpwjETmKrecqIXV9V3gXcP2/c/Pu8X425abO4ak4HJJxrDiWgYSl+2v4r0+IhTeWtjjDF+OuYoRBFJF5HpIrJDRLaLyP+JSLNJp7VLig4HbDkpY4wJZv4Mo58EzABScQ+D/6dn32nLJjMbY0zw8yeBJarqJFWt9bwmA4kBjqtFWQIzxpjg508CKxORGz1zwhwiciPuQR2nrbaRYYhYE6IxxgQzfxLYGOBqYBtQCozw7DttOR0htI0MsxqYMcYEsWZHIXoW5L1KVYedoniCRmJ0ODttRXpjjAlazdbAVLWOwxbgPVPYclLGGBPc/JnI/InneVz/oPFKHMsCFlUQSIp28c32fS0dhjHGmKPwJ4H19Xx9wGef0nhljtNOYrSLsv1V1NcrISFNrUtsjDGmJR2rDywE+IuqvnaK4gkaiVEuauqUvYdqiI8Ma+lwjDHGHOZYfWD1wO2nKJagkhTjngtmQ+mNMSY4+TOMfo6I/FJEOohIQsMr4JG1sMQom8xsjDHBzJ8+sIY5Xz/z2adA1skPJ3gkxTSsh2hD6Y0xJhj5sxp9p1MRSLCx5aSMMSa4HbUJUUR+5fN+5GHHHgpkUMEgyuUkIsxhfWDGGBOkmusDu9bn/b2HHRsSgFiCjk1mNsaY4NVcApOjvG9q+7SUGGUJzBhjglVzCUyP8r6p7dNSUozLBnEYY0yQam4QR6GIVOCubbXxvMezHR7wyIJAYpSL+fvKWjoMY4wxTThqAlNVx6kMJBglxYRTUVlLZU0d4aFn/I/DGGOCij8Tmc9YNpnZGGOClyWwZiTaclLGGBO0LIE1w2pgxhgTvCyBNSPJuxqHjUQ0xphgc8wEJiLDReQbEdkrIhUiss9nROJprW2UixCxGpgxxgQjfxbzfQS4TFXXBDqYYOMIERIiXezcbwnMGGOCjT9NiNvPxOTVICnaxY4KS2DGGBNs/KmBLRGRfwBvAd6/5Kr6ZsCiCiKJ0VYDM8aYYORPAosBDgKDfPYpcEYksKRoF19t29fSYRhjjDmMP88Du+VUBBKsEqNdlO2vor5eCQk5I9YwNsaYVsGfUYjpIjJdRHaIyHYR+T8RST8VwQWDxGgXtfVK+cHqlg7FGGOMD38GcUwCZgCpQBrwT8++M0JStHvdYusHM8aY4OJPAktU1UmqWut5TQYS/bm4iAwRka9EZK2IjG/ieH8RWSYitSIywmd/pogsFZEVIrJaRH7s93d0kiVG22ocxhgTjPxJYGUicqOIODyvG4FdxzpJRBzAs8DFQD5wnYjkH1ZsE3Az8Mph+0uBvqraHTgXGC8iqX7EetI1rMZhQ+mNMSa4+JPAxgBXA9twJ5YRnn3H0htYq6rrVbUamAZc7ltAVTeq6iqg/rD91arakDFcfsYZEN4amDUhGmNMUPFnFOImYNgJXDsN2OyzXYK7NuUXEekAzASygbtVdesJxPBfi3Q5iQxzWA3MGGOCzFETmIj8SlUfEZGncc/7akRVxx7j2k2NOT/iOkejqpuBAk/T4Vsi8oaqbj8sxh8CPwTIyMjw99LHzSYzG2NM8GmuBtawfNSSE7x2CdDBZzsdOO5alKpuFZHVQD/gjcOO/Q34G0BRUZHfyfF4JUa72FFhK9IbY0wwOWoCU9V/et4eVNXXfY+JyEg/rr0YyBGRTsAW4Frgen+C8swz26Wqh0QkHjgPeNyfcwMhKTqcNdvOiAX4jTGm1fBncMS9fu5rRFVrgduB2bhrc6+p6moReUBEhgGIyDkiUgKMBP7qqWkB5AELRWQl8BHwmKp+7kesAZEY7bJh9MYYE2Sa6wO7GLgESBORP/kcigFq/bm4qr4LvHvYvv/xeb8Yd9Pi4efNAQr8ucepkBjtYl9lLZU1dYSHOlo6HGOMMTRfA9uKu/+rEljq85oBDA58aMHDJjMbY0zwaa4PbCWwUkReUdWaUxhT0PFOZt5XSYeEiBaOxhhjDPj3OJWOIvIw7tU0wht2qmpWwKIKMlYDM8aY4OPvYr5/wd3vdQHwIvBSIIMKNoneGpglMGOMCRb+JLA2qjoXEFX9VlXvAy4MbFjBpW2kixCxGpgxxgQTf5oQK0UkBPhGRG7HPacrKbBhBRdHiNA2yobSG2NMMPGnBjYOiADGAr2AG4HRgQwqGCVFu6wJ0Rhjgog/i/ku9rzdD9wS2HCCl01mNsaY4HLMGpiIzBGROJ/teBGZHdiwgo+7BmbrIRpjTLDwpwmxnaruadhQ1XLOsD4wcNfAyvZXU18fsDWDjTHGHAd/Eli9iHifVSIimRzHY1FOF4lRLurqld0Hq1s6FGOMMfg3CvE3wHwR+ciz3R/PM7jOJEkx7jncO/dV0S7K1cLRGGOM8WcQxywR6Qn0wf2QyjtUtSzgkQUZ39U48lJaOBhjjDFHb0IUkbM9X3sCGbgX990CZHj2nVGSbDUOY4wJKs3VwO7E3VT4xyaOKWfYahy2HqIxxgSX5hLYHM/XW1V1/akIJphFhDmJcjltKL0xxgSJ5kYhNjx1+Y1TEUhrYJOZjTEmeDRXA9slIvOATiIy4/CDqjoscGEFp8QoW07KGGOCRXMJbCjQE/ejU5rqBzvjJMa4WLO1oqXDMMYYQ/NPZK4GPhORvqq68xTGFLQSo1x8ZDUwY4wJCkdNYCLypKqOAyaKyBErb5yJTYhJMS72V9VysLqWiDB/5oAbY4wJlOb+Cjc8dfmxUxFIa5DoWYGjbF81GW0tgRljTEtqrglxqedrwxJSiEg80EFVV52C2IJOw3JSO/ZVktE2ooWjMcaYM5s/j1P5UERiRCQBWAlMEpHHAx9a8GmogdlQemOMaXn+rEYfq6oVwHBgkqr2Ai4KbFjBKdGWkzLGmKDhTwJzikgKcDXwToDjCWoJkWE4QsRqYMYYEwT8SWAPALOBtaq6WESygG8CG1ZwcoQIbSPDbDkpY4wJAv48TuV14HWf7fXAVYEMKpglxdhyUsYYEwz8GcTxiGcQR6iIzBWRMhG58VQEF4wSo1zs3G8JzBhjWpo/TYiDPIM4LgVKgFzg7oBGFcSSosPZUWEJzBhjWpo/CSzU8/US4FVV3R3AeIJeYrSLXQeqqas/YnESY4wxp5A/CeyfIvIlUATMFZFE4IwdxZAY7aKuXtl9oLqlQzHGmDPaMROYqo4HvgcUqWoNcAC4PNCBBaskezKzMcYEBX9qYABpwFUiMgoYAQzy5yQRGSIiX4nIWhEZ38Tx/iKyTERqRWSEz/7uIrJARFaLyCoRucbPOAPuu8nMlRw6dIgBAwZQV1fHt99+S69evejevTtdunThueeeO+a1du/ezcCBA8nJyWHgwIGUl5c3WW7IkCHExcVx6aWXNtr/zDPPkJ2djYhQVlbm3V9eXs6VV15JQUEBvXv35j//+Q8A1dXV9O/fn9ra2hP99o0xJmj4Mwrxd8DTntcFwCPAMVeiFxEH8CxwMZAPXCci+YcV2wTcDLxy2P6DwChV7QIMAZ4Ukbhj3fNUSIp2r4e4c18VEydOZPjw4TgcDlJSUvj0009ZsWIFCxcuZMKECWzdurXZa02YMIHi4mK++eYbiouLmTBhQpPl7r77bl566aUj9p933nm8//77ZGZmNtr/0EMP0b17d1atWsWLL77IL37xCwDCwsIoLi7mH//4x4l868YYE1T8qYGNAIqBbap6C1AIuPw4rzfuyc/rPc8Wm8ZhTY+qutGzMHD9Yfu/VtVvPO+3AjuARD/uGXANNbCd+6uYOnUql1/u/pbCwsJwudzHqqqqqK+vP+o1Grz99tuMHj0agNGjR/PWW281Wa64uJjo6Ogj9vfo0YOOHTsesf+LL76guLgYgLPPPpuNGzeyfft2AK644gqmTp16zNiMMSbY+ZPADqlqPVArIjG4k0mWH+elAZt9tks8+46LiPQGwoB1TRz7oYgsEZElO3eemmdutglzEO1yUrprP+vXr2+UQDZv3kxBQQEdOnTgnnvuITU1tdlrbd++nZSUFABSUlLYsWPHSYmxsLCQN998E4BFixbx7bffUlJSAkDXrl1ZvHjxSbmPMca0JH8S2BJP893fgaXAMmCRH+dJE/uOa+y5Zw3Gl4BbPEm08cVU/6aqRapalJh46ipoidEuNm/bTlxc41bNDh06sGrVKtauXcuUKVO8tZ5Tbfz48ZSXl9O9e3eefvppevTogdPpXnTF4XAQFhbGvn37WiQ2Y4w5WfxZSuqnnrfPicgsIMbP54GVAB18ttOB5juFfHhqezOB/6eqn/l73qnQLtrFnv1VVFY2PZsgNTWVLl268PHHHzNixIgmywAkJydTWlpKSkoKpaWlJCUlnZT4YmJimDRpEgCqSqdOnejUqZP3eFVVFeHh4SflXsYY01KOWgMTkZ6Hv4AE3KvT9/Tj2ouBHBHpJCJhwLXADH+C8pSfDrzoWYsxqCRFu9irLurq6rxJrKSkhEOHDgHuUYCffPIJZ511FgCjRo1i0aIjK63Dhg1jypQpAEyZMsXbn/bf2rNnD9XV7nlqzz//PP379ycmJgaAXbt2kZiYSGhoaHOXMMaYoNdcE+Ifm3k9dqwLq2otcDvulezXAK+p6moReUBEhgGIyDkiUgKMBP4qIqs9p18N9AduFpEVnlf3E/oOAyAx2sWOikoGDRrE/PnzAVizZg3nnnsuhYWFDBgwgF/+8pd069YNgFWrVnn7unyNHz+eOXPmkJOTw5w5cxg/3j3TYMmSJfzgBz/wluvXrx8jR45k7ty5pKenM3v2bAD+9Kc/kZ6eTklJCQUFBd5z1qxZQ5cuXTj77LN57733eOqpp7zXmjdvHpdccklgfjDGGHMKierpsSRSUVGRLlmy5JTc6y8fruMPs77klSuT+Mszf2pyiHuDiooKbr31Vl5/PTgqksOHD+fhhx/21g6NMWc2EVmqqkUtHceJ8Gce2M9852CJSLyI/LS5c053uclRAFTFZnLBBRdQV1d31LIxMTFBk7yqq6u54oorLHkZY04L/oxCvE1V9zRsqGo5cFvgQgp+/XISiY8I5f+WlTBmzBgcDkdLh+SXsLAwRo0a1dJhGGPMSeFPAgsREe+QeM8KG2GBCyn4hTlDuKwwlTlfbKeisqalwzHGmDOSPwlsNvCaiBSLyIXAq8CswIYV/K7skUZVbT3vfV7a0qEYY8wZyZ8Edg8wF/gJ8DPP+18FMqjWoHuHOLLaRfLmsi0tHYoxxpyR/HmcSr2qPqeqI3D3fS1Q1aOPWjhDiAhX9khj4YbdlJQfbOlwjDHmjOPPKMQPRSRGRBKAFcAkEXk88KEFvyt6uJd2fGu51cKMMeZU86cJMVZVK4DhwCRV7QVcFNiwWocOCRH07pTAm8u3cLrMpzPGmNbCnwTm9CyqezXwToDjaXWG90hj/c4DrCrZ29KhGGPMGcWfBPYA7pGIa1V1sYhkAd8ENqzW45KCFMKcIby5rKSlQzHGmDOKP4M4XlfVgoZV6T0PqLwq8KG1DjHhoQzMT+afq0qprj32QyyNMcacHEd9nIqI/EpVHxGRp2niOV6qOjagkbUiw3ukMXNVKR99vZOB+cktHY4xxpwRmnse2BrP11OzQm4r1j83kbaRYUxfXmIJzBhjTpGjJjBV/afn65RTF07rFOpwLy31yqJN7D1YQ2yEPWvLGGMCrbkmxGYfPqmqw05+OK3XVT3TmfzpRmZ+Xsr152a0dDjGGHPaa64J8XvAZtxrHy4EpJmyZ7yuaTFkJ0UxfXmJJTBjjDkFmhuF2B74NdAVeAoYCJSp6keq+tGpCK41aVhaavHGcjbtsqWljDEm0I6awFS1TlVnqepooA+wFvhQRH5+yqJrZa7okYYITLelpYwxJuCanQcmIi4RGQ68jHsl+j8Bb56KwFqjtLg29OnUlunLS2xpKWOMCbCjJjARmQJ8CvQE7lfVc1T1QVW16kUzhvdMY+OugyzbtOfYhY0xxpyw5mpgNwG5wC+AT0WkwvPaJyIVpya81ufibimEh4YwfbktLWWMMYHUXB9YiKpGe14xPq9oVY05lUG2JlEuJ4O7tOefK0upqj3jH5tmjDEB489ivuY4Xdkjjb2Hapj35c6WDsUYY05blsAC4PzsdrSLclkzojHGBJAlsABwOkK4onsqH3y5g/ID1S0djjHGnJYsgQXIlT3TqKlT3vm8tKVDMcaY05IlsADJT4nh7PbRTLcHXRpjTEBYAguQhqWllm3aw4ayAy0djjHGnHYsgQXQ5d2PXFrq0KFDDBgwgLo69xB7h8NB9+7d6d69O8OGHXuB/6qqKq655hqys7M599xz2bhxY5PlxowZQ1JSEl27dm20//XXX6dLly6EhISwZMl3j3r7/PPPufnmm4//mzTGmBZiCSyA2seGc352u0ZLS02cOJHhw4fjcDgAaNOmDStWrGDFihXMmNHsE2wAeOGFF4iPj2ft2rXccccd3HPPPU2Wu/nmm5k1a9YR+7t27cqbb75J//79G+3v1q0bJSUlbNq06Xi/TWOMaRGWwALsyh5pbN59iCXflgMwdepULr/88hO+3ttvv83o0aMBGDFiBHPnzm1y3cX+/fuTkJBwxP68vDzOOuusJq992WWXMW3atBOOzRhjTiVLYAE2uEt7IsIcvLlsC9XV1axfv56OHTt6j1dWVlJUVESfPn146623jnm9LVu20KFDBwCcTiexsbHs2rXrpMRaVFTExx9/fFKuZYwxgRbQBCYiQ0TkKxFZKyLjmzjeX0SWiUitiIw47NgsEdkjIu8EMsZAi3Q5GdKlPe+s2kpJ6Xbi4uIaHd+0aRNLlizhlVdeYdy4caxbt67Z6zVV2xI5Oc8aTUpKYuvWrSflWsYYE2gBS2Ai4gCeBS4G8oHrRCT/sGKbgJuBV5q4xKO4FxRu9a7smca+yloWl+ynsrKy0bHU1FQAsrKy+P73v8/y5cubvVZ6ejqbN28GoLa2lr179zbZVHgiKisradOmzUm5ljHGBFoga2C9gbWqul5Vq4FpQKPOH1XdqKqrgPrDT1bVucC+AMZ3yvTt3I7kGBevLN9FTW2dN4mVl5dTVVUFQFlZGZ988gn5+e4cf++99zJ9+vQjrjVs2DCmTJkCwBtvvMGFF1540mpgX3/99RGjFo0xJlgFMoGlAZt9tks8+04aEfmhiCwRkSU7dwbvwrmOEOE3Q/NZU7qP2pSuTH7TPTpwzZo1FBUVUVhYyAUXXMD48eO9Cezzzz+nffv2R1zr1ltvZdeuXWRnZ/P4448zYcIEALZu3coll1ziLXfdddfxve99j6+++or09HReeOEFAKZPn056ejoLFixg6NChDB482HvOvHnzGDp0aMB+DsYYczJJoJ4cLCIjgcGq+gPP9k1Ab1X9eRNlJwPvqOobh+3/PvBLVb30WPcrKipS33lNwWjl5j2MevQfbJj3D57520Ru6pN51NrT4MGDmT179imLraqqigEDBjB//nycTucpu68xpmWJyFJVLWrpOE5EIGtgJUAHn+104IweIVDYIY6PJtxCwTnn8dvpqxg7bQUHqmqbLOubvHwnP8+bN8878bl79+6Eh4cfc/SiP5OfN23aRHx8PKmpqUc0I+7evZuBAweSk5PDwIEDKS93TwnYu3cvl112GYWFhXTp0oVJkyYBsHPnToYMGXI8PxpjjDlugUxgi4EcEekkImHAtcCxZ+qe5uIiwvj38w/wq4vzmblqK8Oemc8325vv6vOd/HzBBRd4Jz5/8MEHREREMGjQoGbP92fyc05ODvfee2+Tk58nTJhAcXEx33zzDcXFxd5my2effZb8/HxWrlzJhx9+yF133UV1dTWJiYmkpKTwySefHMdPxhhjjk/AEpiq1gIubCtQAAAgAElEQVS3A7OBNcBrqrpaRB4QkWEAInKOiJQAI4G/isjqhvNF5GPgdaBYREpEZPCRd2mdQkKEn12Qzcu3nsveQzUMe+YT3vJZbupwR5v8/MYbb3DxxRcTERHR7P3+28nPvuePHj3aW+MTEfbt24eqsn//fhISErzNj1dccQVTp05tNi5jjPlvBHQemKq+q6q5qtpZVf/Xs+9/VHWG5/1iVU1X1UhVbauqXXzO7aeqiaraxlPm1HUInSJ9s9sxc2w/uqbFMO4fK/jN9M+prKlrVKapyc8Npk2bxnXXXXfM+/y3k5+3b99OSkoKACkpKezYsQOA22+/nTVr1pCamkq3bt146qmnCAlxf6RsUrQxJtBsJY4WlhwTziu39eFH/bOYunATI59bwObdB73Hy8rKjpj8DFBaWsrnn3/eaBTh0QRq8vPs2bPp3r07W7duZcWKFdx+++1UVFQANinaGBN4lsCCQKgjhHsvyeOvN/ViY9kBLn16PnPXbAfci/0ePvkZ4LXXXuPKK68kNDT0mNf/byc/JycnU1rqfjBnaWkpSUlJAEyaNInhw4cjImRnZ9OpUye+/PJLwCZFG2MCzxJYEBncpT3vjD2ftLg23DplCY/M+pLomFjq6uqOSGKvvvrqEc2HgZr87Hv+lClTvP1xGRkZzJ07F3A3M3711VdkZWUBNinaGBN4lsCCTGbbSN78aV+uPacDf/5wHWOnLeeigQOZP3++t8zGjRvZvHkzAwYMaHRuoCY/jx8/njlz5pCTk8OcOXMYP969rOVvf/tbPv30U7p160ZxcTF/+MMfaNeuHWCToo0xp4CqnhavXr166enm7/9ep5n3vKMjfv+yXn/DDccsP2jQoFMQlX/69eunu3fvbukwjDnpDh48qP3799fa2lrvvr1792pqaqr+7Gc/8+4bPHiwFhQUaH5+vv7oRz9qVL4p9fX1+vOf/1w7d+6s3bp106VLlzZZ7te//rWmp6drZGRko/3l5eWamJioWVlZ2rt3b92wYYM3rmuuuUb79Omj+fn52qFDB+3atavm5+drRkaGAsv1GH9fgXuBtcBXuBeoaKrM7Z4yCrTz2S/AnzzHVgE9fY5lAP/CPVL9C6CjZ/80IOeYcR2rQGt5nY4JTPW7JPb9236rh6qqWzocv+zYsUOnT5/e0mEYExDPPPOMPvnkk432jR07Vq+77rpGCWzv3r2q6k5Mw4cP11dffbXZ686cOVOHDBmi9fX1umDBAu3du3eT5RYsWKBbt249IoFdffXV2rdvX1VVffXVV/Xqq6/WsWPH6rXXXqsJCQm6YsUKLSsr07S0NC0tLVVV1fPOO0+Bndp88soHVgIuoBOwDnA0Ua4H0BHYeFgCuwR4z5PI+gALfY59CAz0vI8CIjzvBwB/by4uVbUmxGD3g35Z/L+heWxIOJc7XltFTd0R6x4HncTERK644oqWDsOYgDh8XubSpUvZvn37EQsKxMTEAO6BU9XV1cfsd3777bcZNWoUIkKfPn3Ys2ePd/CUrz59+nintfiaPXs2d911F+Ce7zl79my2b99O+/btadeuHYWFhaxfv56zzz7b29Vw4403AsQf41u+HJimqlWqugF3Tar34YVUdbmqbjzK+S968uxnQJyIpHieTuJU1Tme8/erasMQ7I+Bi0Sk2XXtLIG1Ag1J7L3/bGPsq8tbRRIz5nR0+LzM+vp67rrrLh599NEmyw8ePJikpCSio6MZMWJEk2Ua+M7XBPfo4S1bjr7AweFxHThwgN693XklJCSEqqoq7r33XrZv3+6N5dZbb2Xx4sVs3LiR2tpa5syZAyAi0raZy/+3C7Mf7fxcYI+IvCkiy0XkUc9juFDVetyJsrDZKx+ritZaXqdrE6Kv5z9er5n3vKM/fmmJVtfWHbXc4W30v/rVr7RLly7apUsXnTZt2jHvU1lZqVdffbV27tzZ25belJtuuklDQ0M1Pz9fVd1t/rGxsTpo0CC96KKLNDs7Wy+66CK94YYbtKCgQDt27KiAdunSRQsLCzU7O1tjY2P14osv1n79+mlNTY2qqo4ePVrj4+M1LCxMXS6XPvvss03e/+mnn9bOnTsroE8//bRmZ2drdna2Tpo0qcm+hNdee01xt89rTk6OXnbZZd5rdevWTTMyMjQ3N1dnzZrl3Z+Zmaldu3bVwsJC9f2M3XXXXTp37txj/ixbA9/Py8aNG7Vnz55aWFio+fn5+pe//MVbzrfvxd9zcnJyNCwsTENDQzUhIUGrqqqOKH/NNdd4f499+/bVuLg47+++bdu2mpWVpWeddZaeffbZ3nMADQ0NVRFRl8vlLe9wONTlcmlWVpYWFhZ6f3eJiYne332XLl20c+fO3rhCQkK0Y8eO3s9KQ78RoGFhYQpop06dNDY2VgENCQnR4uJi7dy5s6akpOjYsWNVVXXSpEmNzsnOztbOnTtrdna2tm3bVuPi4jQrK6vRfRquCWhUVJTGxMRobm6uduvWTQFt165dk3EVFhaqiGhMTIympKRoaGioAtqvXz9Vdf/bSEhI0LKyMr366qvV5XJ5//0lJCRoaGiohoeH6/DhwxX346w24dM/hbs/qyG2XcAyz34B9gI7ObI/qw5YAVQDs3z2bwWu9dmeC/QCRniulQU4gf8DbvUpNxW4TJv5u9/iiedkvc6EBKbqXxLzbaN/55139KKLLtKamhrdv3+/9urVy9s2fzTPPvus/uhHP1LV79rSmzJu3Di96667tEuXLqqq+v777+uMGTM0KytLH374YVVVffjhh/UXv/iFqrr7CRISErzHGsoPHTpU77vvPn355ZdVVfXyyy/XjIwMrays1PXr12tWVlaTHeDLli3TDRs2aHp6umZmZuquXbt09+7dmpycrMXFxUf0JXzxxRfapk0bHTBggC5evNh7ndWrV2vnzp31lltuOeJ+mZmZunPnziPuvXHjRh04cGCzP8fWwvfzUlVVpZWVlaqqum/fPs3MzNQtW7aoauO+F3/OWb16tXbp0kUrKyv1vvvu09DQUJ06deoR5VNSUvSzzz7TzMxMzcvL0+9973taX1+vV155pYaHh2t9fb3++9//1qKiIu85gC5btkzbtGmjBQUF3s+KiGhcXJyqqr700kt61VVXqarq66+/rk6nU9PS0nTmzJkaGhqqmzdv1htuuEFFREePHu39rDT0ZzkcDt2wYYNmZmbqc889p3l5eRoREaExMTHqcDi0pqZGBw0apGFhYZqZmalt27ZVQH/84x9rSkqK93tfv369hoWFaVhYWKPP5NixY3XIkCHqcDg0MzNTR40apXl5ebp48WL9yU9+ok6nU6+77roj4nI6naqqGhkZqatXr9aCggItLS31JvPa2lq99tprNSQkRDMzMzUqKkqdTqc3rsTERH3ooYd0/fr1GhMT05DAsvHpn8Ldn3UAd3/Wg8C9nv2XAHuAtzmyP2u/5+tGGveBzQA+8Nn+CkjxnP+hz/6bgGd9tv8PuEib+btvTYitzK3nd+K3l+bz3n+28fNXmm5O9G2j/+KLLxgwYABOp5PIyEgKCwubXLDXl79rJy5cuLBRX1dxcTHR0dFs37690dqJ7733HgDvv/8+4eHh3r6AhvLQeO3EzZs3c9555+FyuejUqRPZ2dksWrToiPv36NGDjh07UllZyYABA0hISCA+Pp74+Hjy8vKO6EvIy8vzLnV1+Pc7ZswY5s2bR4cOHY56P1+ZmZns2rWLbdu2NVuuNfD9vISFheFyuQD3Uwzq67/7fPn2vfhzzttvv80NN9yAy+Vi1qxZREVFsW7duiPKO51Ob9PZ7t27GTp0KCLCp59+SnR0NNu2baNfv35UVFRQWlpKVVUVIkJycjK1tbVce+21uFwuamtrUVXq6+u9+z/88ENUlc6dO3s/d2+88QYZGRmkp6fz/vvvk5iYyJtvvkmfPn3Ytm0bGzduZNCgQYiIt6lw8+bN3HTTTYgIdXV1iAgLFixg9uzZdOzYkQULFvDYY4/hdDr5y1/+wsGDBykuLsblctGhQwdqa91Pnairq2PJkiV8/fXXbNy4kWuuucYb1+DBg9m2bRuqyrRp00hJSWHBggUUFRU1isvXm9PfYtBlw1m+zt1XVl1dzaeffsrll1/OVVddxcaNG5kwYQJxcXH88Y9/5NChQ4SHh1NQUEBcXByHDh0CdwLbiDupZItIiqoux137ApgFXCsiLuBGoAooAJbg6c86xkfsb0AfEXGKSB9gr6qW4l7wPV5EEj3lLsQ9ErFBLrCaZlgCa4VuPb8T/3NpPrNWH5nEDm+jLyws5L333uPgwYOUlZUxb94876ocR+PP2okN90lPTz/i/KqqqiPWThw1ahRr1qxhx44dvPzyy1x88cWsXv3dZ7Nr164sXrwYgIMHDzJv3jwKCgq44447SElJabYvoLa2ltTUVO92fX19oz+8vn0JlZWVLF26lJtvvtm7KPGWLVvIyMggOzublStXNiovIgwaNIhevXrxt7/9rdF9e/bs2epX3G9qrc3NmzdTUFBAhw4duOeeexr9bMHdauPPOQ2fo4EDB7Jw4ULi4+PJzc1t9h7V1dVMnDiRgoICysrKGv0u2rVrR//+/b2fzWHDhlFTU+Pt43nppZcASE1NZeXKld7/tHXt2pW+ffsCsGPHDt555x127NjB119/TXl5OSEhIVRXV7Nz504qKiq47bbbqK6tp66ujuz8Akq2bGXS9H/x0uf7qaypQ1xROF3hTJ0xh537qhrFWFdXR1FRERUVFbz66qsUFBRQUFCAqpKWlsbKlStZs2YN1dXV3Hbbbd5zSktLeeyxxwDo27cv5eXl9OzZk86dO/PZ4iWUlJRwzuCRfPT1DmprawlxhnLgwAF++7v7ePa1Wdz58LPk5eXhdDq54oorGs33jIyMpHv37pxzzjlUVFRQXV3N3XffzXnnndewEHi1uhdf7wyU813/VjjupPYU8Dnu5DIMeITv+qfa8l0/VRsRqQYyga9F5HnP/pm4myE3AH8Hfur5HNUBvwTmisjnuJsn/w4gIsnAIU+iO7rmqmet6XWmNCH6esHTnPijF79rTtyyZYueddZZjcr9/ve/18LCQr3ooov0+uuvP2IIsGrjfpD8/HwdMGCAxsbG6tChQzUrK0vLyspUtXG/U3Z2tm7YsMHbhKiqOm/ePG8TR4OIiAjt2LGjhoaGav/+/XXixIk6c+ZMzc7O1nnz5unQoUP1tddeU6fTqSKiQ4cO1RdffFErKyv1+uuv17CwME1KStLc3Fzt1auXNy5V1draWg0JCdHw8HAtLCz09g2kp6d773/hhRfqkiVLvD+fAQMG6PTp07Vt27aamZnp7Ue49tpr9cknn9TExERNT0/Xbt266Q9+8APNz89XEdF27dppWlqaArpp0yaNjY3Vtm3bakZGhp599tman5+v3bp102nTpmltba12795dExISvM1HcXFx2rNnT62trdXly5d75+VkZGRoWlqaioguXrxYe/bsqU6nU10ul7c/KCcnR5999lnvOampqZqcnKyADhw4UHNzczUsLExFREVE27Rpo9nZ2dqtWzdvXOPGjdOzzz5bAR0wYIBmZmZ6+04uvfRS7dy5c6M+wC1btnj34ek3zM7OVkDDw8P1zjvvPOKc5557TsPDwxXQ2NhYLSws1C1btmhubq6GhoZ6+3h873HOOefoli1bNDQ0VOPj4/Xjjz/Wr776SgFNS0vz/u4afo9btmzR7t2767Zt29TpdGrbtm117dq1mp2drSKi55xzjs6YMUNV1fvZXb9+vfbo0UPT09N12LBhGhkZqd26ddOQkBBN7ZCprjYR2n/kbRqdlKaFt/9F214yTiO7FmvmPe+oI6qthoSFa59rx6nT1UajMrsqzjCN6TNCM+95R6OzuuvQ307Wh99doxP/tVS/2b5PIyMjNT4+XteuXatbtmxRp9Op3bp10xkzZmiXLl00KytLP/lsoT7557/pldfcoO3TOuifps3ShKQUvfPpaep0hWt8Zp5GdcjTqMIh6oxrr+1HP6ltLxmncYUD9bKnP9abnpihrjYRetMv/p92ys7Vl19+WaOionT8+PFH/ZsRFRWlf/7znxtt4xlGD7wM/Bvo5dlOxV0z6+X52tmTjM7H0z+Fpz+robzna1ZDef2uOfCY/Vm+L+AOfPrDjvayGlgrNsanJnb7K8uoqatvcu3E3/zmN6xYsYI5c+agquTk5BxxLd9njqWnp3PVVVfx0ksvoaqN1k4877zzeP/990lPT6eqqqrJuFwul3f47xdffEFNTQ1z586lffv2fP3110ybNo1LLrmEmpoa9u7dC7hrYFFRUfTr14/MzExKSkpwuVyEhIRQU1PDa6+9xqxZs9i6dSuTJ0/23uupp54iNDSUtm3bep+TFh4e3qgGVlJS4v0ffsPX6Ohoqqqq+N3vfkffvn3Zs2cP+/fvJzY2ltzcXO/93nrrLSZPnkz//v3p168fo0aNIjMzkyeeeILa2loefPBBJk6cyIEDB1i1ahWzZs1i3LhxTJgwgaysLA4ePEhpaSnbtm3j4MGDbN++HYfDQUREBC+++CKrV6/mxRdfpLq6mr59+7J+/Xo2bNjAiy++yIwZM9i3bx+///3vmT17Ng8//DCTJk1i9erVTJo0if379xMREUFNTQ01NTWUlpYyZMgQVJXHHnuM//3f/0VEWL16NbNmzeLll1+msLCQxMREFi5cSF1dHV9++SUul4uZM2cyf/58Zs2axU9/+lPq6uq8TxnIy8ujXbt2bNq0iYULFxIREeGtKTVcu+Gc/v37c/HFF5OcnExiYqJ3Tcz9+/cDEB8ff8Q9unTpwr333ktYWBgRERFs3ryZxMREQkND2bZtm/d3VlJSQkpKCm2TksnvWsBbs+YiIiSmpPHE31/mQGUNIOzYe4B5a/fwxL/WsK1sN0/8eytPL97Lgah0du7Zz1fV8dS5Yqg+5ybUEcb28v1U14ewZMUq9pdtY90r91E5fzL1GxYy+MBc0ttG0eWsHLJCduByhnD94L6EO0MYXVzIby/Nx3monAPOGF6Yv57755Zy0eMfcagWatskcNef3+Lh2euoq69n484K7n/3G7Y5k1m/8Vv6DxzKnXf9kremv8n2nbt45JNdVCXlM/ndz6hXIa5DLo6qClJkD7pvJ/qvR9CFL1G3/jPyS97hxXGX0SXvbMq+XopTlBtuuMHbtHo0TqezUWuG5/FH1QCqeiOQjOfBw6rasBr3t7jna/XguwcVhwOH8HlQcUN5VV3vU75BQ3l/7QGmHKuQJbBWbsz5nfjdZfnMXr2d219ZRtRhayfW1dV5m/9WrVrFqlWrvO3ovmsn+vZpDBs2jNWrVxMdHU1paWmjtRMb+p0cDgd1dXVHJLG6eqVdYhIPPfkc/1y5lVG3301MUhrPLtnHgcpqEhLaEhoayqJFi9i/fz//+Mc/APfq9e3bt0dE6NatG9OmTaOyspLp06cTGhrK+eefz86dOwG8/wBLSkqYOXMmMTEx7Nq1i/LycjZt2kRlZSW7du2ipqaGzz77jNjYWFJSUigvL/fGO3fuXEJCQjj33HMpLS0lLCyM5cuXk5eXx9atW+nduzexsbG0b9+euLg46urqWLt2Lf379wfciyknJyfTo0cPiouLycvLY9GiRaSmphIfH88777xDXl4eSUlJJCQkEBcXh8PhYNeuXdTW1pKbm+v9j8SAAQNISUmhurqajz76iE6dOhEaGsrSpUsJDQ2lc+fOdOrUifz8fO/TsPPz8xER2rRpQ1lZGQMHDiQhIYElS5YAEBoaytVXX01lZSWlpaXeuPbt20dMTAyqyuAhF5PQPh2HwwHABx98QGhoKJ06dWLRokWUl5ezcuVKSktLiYiIIC4ujoSEBEQEh8NBVVUVDoeD999/n7q6OhYtWkT79u1ZsWIFBw8eZPjw4ZSUlBAREUFFRQWqSkVFBbsP1FBPCKN/NJa3PvuKd/81l38vXgGhLmocbbj3D88w7o011Na5m/Gue3E1qd+/jnVbdtD/mRVk3T6Z12bO4f5/l1Oj8OWaL5g882P2JRWgKN9u3Mhr6+C3d4+j2hnJu//ZxkeLVlGyZjmEOGiXkUNl+Xa+f24Pwl1h1O3fRVyki/deeIzOWZ24/Ye38IeHfs/om27kr08/Tn19Pbt37/b+bHv37k1VVRV3/2g0O+f/A5dWMe9/rmT+uHN560e9+ePIQtq0CefA9m/5qjKS+V98iyoc2LWNs/PyufjqW0jPzudPMxYw6md3c/HlI0hObMvTV2SRsG8dsx+6hfjoCA58vYAIRz2PP/gbOnXqxC2jR/HA/fdzww03MGHCBMrKyti5cyefffYZI0eOZMOGDVRWVjJixAimT5/Ovffee8TfizZt2vDWW29RVVXFhg0bGvoio8TN2z8lIvGePi+ABOA83E2IM4BRuPunXE2VF5F2PuUbHLM/y5eqTlJ3s+YxC54WrzOxCdHXxPnu5sQxkxbp0BHX6z9nuptoDh06pHl5eZqXl6fnnnuuLl++3HvO0KFD9dNPP9WqqipNTk727j906JCOGDFCU1NTNTY2VtetW6eq7uaeIUMu1m+2V2hyarq2bZ+mbaJiNMTh1DZxiRreNk1D2sQojlDFEaohUQnqiG2vzuh2Gp7UUXGGKSEOjYxN0K49emlKSopGRERoeHi4JiQk6PDhw3XAgAFaVFSkycnJ6nQ6FfAOU3799dc1MjJS77zzTh06dKgWFhZqUlKShoSEqNPp1OjoaE1KStJevXppenq6pqWlaXx8vE6ZMkVVVR966CFvM6WIaFJSkvd779evnzocDs3NzdV3331XVVXffPNNdblc2q1bN42IiNCf/OQnqqqakZGh4eHhmpKS4h36P2bMGH399dd14cKFGh0drYsWLdIf//jHmpubqzfffLMmJiZqSEiIdu7c2dsk1mDhwoV69tlna//+/XXkyJF6/vnna25uriYkJGhISIi+8sorje6hqnrBBRdox44dNSkpSXNycvTBBx/UqqoqdTqdGh4e7h35eeGFF+q8+Qt08vR/aZgrXH85cY5GJ2eohDg0qvvFmnnPOyrhUe4mv/Ov19Trfq8h4VEaGpukbZI7aURKtnb/+XMaFt1WHa4IDY9NVEQUERVnqPa/+3nN6HuZhoSGaZv4ZI1OzdLY9ByN7ZCreZf/REOcoSoO9+8xNC5FXRmFGj/wJwrivk9iR3cTXUSsgiig4nCfE+IMVWd4pEbGJ6krIlolJERTOp2l7VIz1NUmQh2e64a5XCoimpqeoUV9+mpkZKR2yursHbpfUFCg8fHxGhISooA6nU6NioryDqFPS0vTyMhI7dq1q2ZmZur48eP117/+tUZERHibWBuG3AONhuq3b99eMzIydMuWLfrJJ580GtLucrk0Ojpac3NzdcCAAd5h9FFRUd7P2MCBA1VEvNdtaNqMi4vTjIwMDQsL065du+rYsWP1oYce0l//+tfe76lr1676/PPPa3x8vKanp2vnzp01JydHVVUfffRRzc3N9Y4gfeqppzQtLU0dDodGRUV545o5c6YCO3CvrlEBDFF3E94TQA3ugRw1wL89+wWYCFTi7hcr8uzv69le6fnqOxw+GVikAfi73+KJ52S9zvQEpqo6af56zbp3pqbc/JRGdblAL/3Tx3rfjP/oOyu36ra9h44o37B2YlP9ZqqqH3zwgRYPGqIzV23Vh2Z+odf89VPN/+177n6BmCRNuvZ/NarLBXrehLk68rlPddy05fqH99boSws26gdrtuuXpRX6wP8+rA8++KDW1tXr36fP1XYZuZp40Q808553NLnnRfrwm5/p1j0H9corr9Qvv/yy0RD3LVu2aFxcnL700kuqqvrLX/5Sr7jiCr3vvvu0qKjIm1Aa+tFUVYcMGaJTp07VpKQk73JWDfPTGsqoqj7yyCN6yy23aNeuXTUsLExTU1O9Q65VVbdu3aq5ubm6YMECVVVvXJMnT1aHw6Eioj169PCWHzNmjP7973/XtLQ07d27t3euUkZGhqqqbtq0ScPDwzU2Nlbbt2/v7QfaunWr5uTkaF5enkZGRmp0dLR3OaEbb7xRRUQvueQS7z3eeOMNnTJlisbGxuqCBQs0OTlZc3Jy9IEHHtD/fLNBQ0JCNCahnXYvvlJv+PtnGpPVXRNH3q8hEbEa1eMSzfnNuxqR1EGjk9I0PilVO+UVaPuMLHe/2Ihb9A/vrdGeA4fryHue0JG/flq7Dxqp46Yt14i4dhrZtr3ePHGhjp64UGNTO6kjLFwvHPuYZvUZop3OHaTf/+nDeuHYP+pZF1ylyWf11D433aPpBefprZMXaU6fgZqR10N7XXiZjnvk79o2OUV/+/hf9U8TX9Ubb/mBlh+o0vfnftDod7Rs2TLt2LGjzpgxQ2+44QbdsWPHMf8N3HnnndqnTx9VVb/OWbZsmZ511ln6/vvv+33O448/rs8//7zf5Q+Pyx/Lli3TXr16Hdd9TiQuVVVgiR5P0vCzf+pEyx/Pq9llOkzrcvN5nbiyZzrLNhXxl7/uR53w6qJNTPpkIwAdEtpwTmYCvTrGc07HBN57zz2cvqHfbPeBalaW7GHV5r2sLNnDJx8vZfO6XayduoxQh5CfEsPwnukUpMfyi6ku3n9oFJ/OTeaWWwZ4m6EOl90pkw8//BBHiPCDKy7kpSfac9P15xPdtRv/1/EBnltYxnMLZpMa140vDkZSr98N1294nljDqMlHH32UwYMH065dO8rLy5kxYwbvvvsulZWVVFRUMHLkSBYtWsTgwYNp3749UVFRANx9990cPHiQv/71r95rp6amcv/99zNv3jxGjhyJqnofHFpRUcHQoUP5/e9/T58+fbzn7N27l/vvv5/27duze/duNmzYQHl5+f9v797jo6quBY7/VmYSkvBKYgLEBAnJh2cCIYXgCzqwQSIAABIwSURBVJsQLCBQfIIVidimUmkvpf0ghau31raI9QrS+qhcLVYRUClIG2lVFEVUEOQpoEW0INLwCASUZ4DMun+ck+kQEglCGGZmfT+ffDJnzzlz1prXnn32OXuTmJjI1q1beffdd8nPz2fp0qV4vV48Hg/bt29n+PDhPProo6gqBw4cYPbs2XTq1ImioiISExOZOHEiAwYMYNCgQXTq1ImXX36Zvy5cTFxSKxDhvfdX8NyyrSxd9wkn2l7O3MkTOHFC6TPweg7vK2dX+R5+O3Ua07a3wufzceBwJV8cjeGSo8eJOrSX6JWz6NyxAxW7PuTYzFEcq9hBVFQUmZ06sXbtWn7wgx/wzDNb+O5l2Yzt35FFUw/y82svpbS0lKUblzJ//HVUHtiHqnJ80SPMnDmTnk8k8+mR/fxiQA79/jCWfv36cd/wAkpLS3nuk/fZv2cPn5Rv5ciRIxx9/Q98f3BvVq5ciaoyMC+Dj3K7cM1lXSgtLeWtha/QrXN7/+s4fPhwZs6cSV5eHklJScTExDBz5sx6fQZycnLo2LEjVVVV9domLy+PSy+9lMLCQoB6bZOQkEBxcXG9168ZV12flZpxde3atXqopwaL6xvaDzzXgOvXX0PUisH4sxZY7SqPV+mabfv0qSWf6Y9mrNTuv12obcYv0DbjF2iXX72qtz+9XH88a5XGNE/RS8a+pG3GL9CMCQu0z5TFOuTepzT3iiJdu22fHj1+8oXENS/wXb58uRYXF5+y/71792pGRoZWVFRoRUWFZmRk6N69e/33b91zUKcs3KRX/m6Rthm/QOPbdNHhD8zSZZ/t0aoqn7Zs2VJzcnL8F4S2bdtW33jjjZN+qVe3wJ544gm97bbb9PLLL9f27dtrWVmZP67AVpqq6qZNm9Tr9WpFRYX/jL6rr75aKysrtaioSKdOnXpSHgUFBTpx4kQdOXKktmnTRlu0aOE/PDlnzhz1eDw6ZcqUk7bp3LmzxsTEaEVFhZbt2u0/3PX84nX60/sfV29MI80feKv+aMZKvfGP72nzzFxNHTxW8URry2EPavJN9zkjIUTH6sV3PKlR8YnqaZyoGQN/rNf8fone8uQyjWuepK2ze2hiyzQd8pO7NTraGQVi/fr1umTJEo2Pj/dfAFytQ4cO2r9/f83IyNAdO3Zo9+7d/WdWzp8/X5s0aXLKhePZ2dnaqFEj/+uYlpamjRo10i1btvhfl8BtCgoKdNq0af7nfMOGDf6R2d9///1T1g98HQPl5ORoWVnZKe8rc+5whi2wC+kv6AGcqz+rwOrH5/PplvKDOueDbTp+7jotmvyWXvHAIu347cH608nP6nuflutXR45pr169NDk5WWNjYzUtLc1/2nPgsfTU1FQtKSlRVad/auTIkbXuc/r06ZqVlaVZWVn69NNP+8tLSkr8hwvnzp2nKa1S1eONVk/jBI3NyNMrHlik3/rODfrdIbdoZmamtm/fXrOzs0+Jq/qLr6CgQEtLSzUlJUXz8/NPiqvml6PP59OkpCRNS0vTpk2bal5enubk5Ohzzz2nIqLt2rXT3NxczcjI0JSUFP+p8B6PRz0ej8bFxWlSUpImJCZp02bNVUS0TbvOmpbVUVu17aA3T3pem6dlaZQ3RiXKq0R5FU+0Snxz5xBs81Zuv1BLjU1spRLl8Q8pFOXxqNcbrd7oaI1v3Fi90dGa3voSTUlJURHR3NxcTUtLO2kopauuukpjY2M1NjZW4+LiNDMzU9PT01VE/KezV+fi9XrV6/X6+3WaN2+uF198sWZmZmpqaqqmp6f7X5eXXnrJv6/qvLOysnTq1KmalpammZmZ2qxZM/+lGdXrx8TEaGJioiYnJ/uf8wkTJmijRo1O6mcMVPM12rlzp/91NA0nlCswceIPfT169NDqs4TMmVuzZg0PP/yw/2LQMzVu3DiKi4vp2rXrWcdy+NgJXtu4k5fX7eCNd5ZTsfwlsofdTb/sVvTPaUXPjCS8ntpPoC0rKyMvL49JkyZRUlLij6uiooLJkyezYMECAMoPVPLCgjf4wwP3cfDL/SCCeDzc8fBfOF7l49gJ5XiVz/93rEr58O8zOH78GK2LhrPtk4/4dN6DNMkuourQfhpn9yamRVsAGsd4SGnaiI1PjaWg+Od07ppHSpNGTB51HScqD/PKe2v49dhRxEQJN910IzfeeKM//v3793P99dfz6KOPkpOTw5o1a7jhhht45JFHWLJkyWmf47Fjx7J06VKWLVtW79dlzJgxDB48mD59+tT7dZw6dSrNmjWjpKTka9c7m22+yT7MmRORVaraI9hxfBPWB2YA55h77969632Mvqa6RuP+JuJjvFyfl871eel89b1u3P1AJZVpTZmz8gtmLPucixrH0De7Jf1zUrki6yKiAyqzuLg4qqqq/ENZPfTQQxw4epwl8/7B53sPM2rmKj7c/iX/3u9ekjLgPuIE9r85naoD5Sz4cAfRHiHaE0WMJ4poTxRedzk2IYVDn64hPTGevKuvZM67KXzn21247qabSWnaiJQmsSQ3jSE+xvlYFf69GROu6USPHjkAvN21Ez6fj+zUpsx78Xn69et3yigXCQkJFBYW8uqrr5KTk3NSP1B9nuOafS313aZPnz7+56s+Avtb6utMt/km+zCRxVpgJmQcPnaCxZvKeWXDTt78eBeHjlXRLNbLdzq34pqcVvRql4wItLnkEibOfpOPdx9l3fb9fFZ+kCOff8hXK+bTY+SD5LZOIDe9Oa3jjnNV1yw8WsWAAQO45557KCoqYsWKFTz22GPMmDHjpP1XVFTQvXt3Vq9eDThDSa1atcp/kXdNhYWFTJ48mR49nB+3GzduZNiwYaxYsYKysjL69OnD5s2bqaioIDo62j82Xd++fRk/fjyDBg0CoEuXLixcuLDWOaCMOVvWAjPmPIiP8TKgSyoDuqRy9HgV72zewysbdvD6RzuZt3o78TEeTlQpB5OzuWfaXFrn9CQ3PYEtfx7L0X9vRY8c4l+PFHPP9On0uyqPcePG8bPiBfh8PkaNGkVRUREA27Zt858BGSgpKYlf/vKX5OfnA3Dvvff6K68f/vCH3HnnnfTo0YP58+czevRoysvLGThwIN26deO1114jOzuboUOH0rlzZ7xeL48//jgej4cdO3YwYsQIqqqq8Pl8DB061F957dq1i7i4OKu8jKmFtcBMyDt2wsfSz/aw6OPdxMd4aHLwCxbPe4a5L8w67Sy4tTmX/Xlny/qBTEOzFpgxQRTjjaKwQwsKO7RwSzrR+OB2fD5f0Pvzzpb1AxlTN2uBGWNMBAvlFpgN5muMMSYkWQVmjDEmJFkFZowxJiRZBWaMMSYkWQVmjDEmJFkFZowxJiRZBWaMMSYkhc11YCJSDnx+mtWSgT3nIZwLVSTnH8m5Q2Tnb7l/vTaqmnI+gjnXwqYCqw8RWRmqF+ydC5GcfyTnDpGdv+UevrnbIURjjDEhySowY4wxISnSKrAngx1AkEVy/pGcO0R2/pZ7mIqoPjBjjDHhI9JaYMYYY8JExFRgItJfRDaJyKciMiHY8TQ0EXlaRHaLyIaAsiQReV1ENrv/E4MZY0MRkdYi8paIfCwiG0VkjFse9vmLSKyIrBCRdW7uv3bL24rIcjf3F0UkJtixNhQR8YjIGhFZ4C5HUu5bRWS9iKwVkZVuWdi+7yOiAhMRD/A4cA3QGbhFRDoHN6oG9wzQv0bZBGCRqrYDFrnL4egEMFZVOwGXAT9xX+9IyL8SKFLVXKAb0F9ELgMeBKa6ue8DwnmK5zHAxwHLkZQ7QG9V7RZw+nzYvu8jogIDegKfquq/VPUY8AJwbZBjalCqugSoqFF8LfCse/tZ4LrzGtR5oqo7VHW1e/sAzpdZGhGQvzoOuovR7p8CRcBctzwscwcQkXRgIPAnd1mIkNy/Rti+7yOlAksDvghY3u6WRZqWqroDnC95oEWQ42lwIpIB5AHLiZD83UNoa4HdwOvAZ8B+VT3hrhLO7//fA78AfO7yRURO7uD8WFkoIqtEZKRbFrbve2+wAzhPpJYyO/0yzIlIE2Ae8DNV/cr5MR7+VLUK6CYiCcB8oFNtq53fqBqeiAwCdqvqKhEprC6uZdWwyz3AlapaJiItgNdF5J/BDqghRUoLbDvQOmA5HSgLUizBtEtEUgHc/7uDHE+DEZFonMprlqq+5BZHTP4AqrofWIzTD5ggItU/WMP1/X8lMFhEtuJ0ExThtMgiIXcAVLXM/b8b58dLT8L4fR8pFdgHQDv3bKQY4HtAaZBjCoZSYIR7ewTwtyDG0mDcfo/pwMeq+nDAXWGfv4ikuC0vRCQOuBqnD/At4CZ3tbDMXVX/W1XTVTUD5zP+pqreSgTkDiAijUWkafVtoC+wgTB+30fMhcwiMgDn15gHeFpV7w9ySA1KRJ4HCnFGo94F/Ar4KzAHuATYBgxR1ZoneoQ8EekFvAOs5z99IXfj9IOFdf4i0hWno96D8wN1jqr+RkQycVolScAaYLiqVgYv0oblHkK8S1UHRUrubp7z3UUvMFtV7xeRiwjT933EVGDGGGPCS6QcQjTGGBNmrAIzxhgTkqwCM8YYE5KsAjPGGBOSrAIzxhgTkqwCM0EnIioiUwKW7xKR+87RYz8jIjedfs2z3s8Qd/T7t2qUZ7j5jQ4oe0xEbj/N490pIredZp3bReSxOu47WFv5ueLmFTjTwR0isjqcRjo3Fz6rwMyFoBK4QUSSgx1IIHcWg/oqAX6sqr1ruW83MOZMpvFQ1WmqOuMM9n/OBIxaUd/1i4HRQF9V3dcwURlzKqvAzIXgBM7U5z+veUfNFlR1y0JECkXkbRGZIyKfiMjvRORWdy6s9SKSFfAwV4vIO+56g9ztPSLykIh8ICIfisiPAh73LRGZjXMhdM14bnEff4OIPOiW3Qv0AqaJyEO15FeOM43FiJp3iEiWiLzqDr76joh0dMvvE5G73Nv5bozL3Jg3BDzExe72m0Xkf2s89hS3VbRIRFLcsm4i8r77ePOrW0wislhEJonI2ziV7RA3x3UisqSWnKr3MRRneo6+qrqnrvWMaQhWgZkLxePArSLS/Ay2ycWZ+6kLUAy0V9WeOFNpjA5YLwMowJlmY5qIxOK0mL5U1XwgH7hDRNq66/cE7lHVk+aME5GLceaWKsKZaytfRK5T1d8AK4FbVXVcHbH+DhhbS6vuSWC0qnYH7gL+WMu2fwbuVNXLgaoa93UDbnafg5tFpHrMz8bAalX9FvA2zkgsADOA8araFaeC/lXAYyWoaoGqTgHuBfq584oNriOnNsBjOJXXzjrWMabBWAVmLgiq+hXOl+tPz2CzD9y5vypxpgxZ6Javx6m0qs1RVZ+qbgb+BXTEGSfuNnGmHVmOM+1GO3f9Faq6pZb95QOLVbXcnZ5jFvDteua3BVgBDKsuE2e0/CuAv7hx/B+QGridO65hU1Vd6hbNrvHQi1T1S1U9CnyEU6mAM4TWi+7tmUAv98dBgqq+7ZY/WyP+FwNuvwc8IyJ34AxLVZtynKGJhtaZuDENKFKmUzGh4ffAapwWR7UTuD+03EF6A/uRAsez8wUs+zj5vV1zvDTFmWZjtKq+FniHO4beoTriO9v5WCbhTKxYfUguCmeuqm5fs83p9hn4HFRR92e6PmPG+fNW1TtF5FKcVutaEemmqntrrH8YZ5bzd0Vkt6rOqsc+jDlnrAVmLhjuAKNzOHnK961Ad/f2tTgzDJ+pISIS5faLZQKbgNeAUeJMu4KItHdH8P46y4ECEUl2DwXegnN4rl5U9Z84raRB7vJXwBYRGeLGICKSW2ObfcABEbnMLfpePXcXxX9GYB8GvKuqXwL7ROQqt7y4rvhFJEtVl6vqvcAeTp6OKDC+cqA/MElE+tUzNmPOCWuBmQvNFOC/ApafAv4mIitwToSoq3X0dTbhfFG3xOlLOioif8I5zLjabdmVc5qp1lV1h4j8N870HAL8Q1XPdGqK+3FGRK92K/CEiPwPTuX8ArCuxjYlwFMicghnfq8v67GfQ0C2iKxy17/ZLR+B0w8Yj3M49ft1bP+QiLTDyXNRLTH5qeoWERkM/ENEblDV5fWIz5izZqPRG3OBE5Emqlp99uUEIFVVxwQ5LGOCzlpgxlz4BrotPy/wOXB7cMMx5sJgLTBjjDEhyU7iMMYYE5KsAjPGGBOSrAIzxhgTkqwCM8YYE5KsAjPGGBOSrAIzxhgTkv4fiafMJ+PZpLYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.151 0.119 0.11  0.108 0.105 0.104 0.105 0.105 0.104 0.103 0.104 0.104\n",
      " 0.104 0.104 0.104 0.105 0.105 0.105 0.105 0.105 0.105 0.106 0.105 0.105\n",
      " 0.106]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.143\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_optimal.fit(d_train,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.996\n",
      "The value of TNR is 0.060\n",
      "The value of FPR is 0.940\n",
      "The value of FNR is 0.004\n",
      "\n",
      "Test error is 10.783\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGV9JREFUeJzt3XucVWW9x/HPd0BSLgqaIA6kpHit4yU1jprHW6h4IVM7WioqnjkZeMm0UDMTtUPnHC2poy8xKbyBmJHk6aiIWloikJDglYsXRhAM8AaIzp7f+WMvxg3OntnIDHvPw/fda71m72c9a61n4/Tbv/mtZ62liMDMzNq+qnIPwMzMWoYDuplZIhzQzcwS4YBuZpYIB3Qzs0Q4oJuZJcIB3cwsEQ7oZmaJcEA3M0tE+3IPoJj2Hap9Cat9wm7depd7CFaBZi+eog3dx0f/mF9yzNnss5/f4OO1BmfoZmaJqNgM3cxso6rPlXsEG8wB3cwMIFdX7hFsMAd0MzMgor7cQ9hgDuhmZgD1DuhmZmlwhm5mlgifFDUzS4QzdDOzNEQCs1x8YZGZGeRPipa6NEPSq5JmSZopaXrWtrWkSZLmZD+7Ze2SNFLSXEnPStq3YD+Dsv5zJA1q7rgO6GZmkC+5lLqU5rCI2Dsi9sveDwMmR0RfYHL2HuAYoG+21AA3Q/4LALgK+DJwAHDVmi+BYhzQzcwgf1K01OXTGQiMyV6PAb5W0H575E0BukrqCRwFTIqIZRGxHJgEHN3UARzQzcxgvTJ0STWSphcsNevuDXhY0t8K1vWIiEUA2c/uWXs1sKBg29qsrVh7UT4pamYG63Xpf0SMAkY10eWgiFgoqTswSdKLTfRt7M6N0UR7Uc7QzcygRU+KRsTC7OcSYAL5GvjirJRC9nNJ1r0WKLwvdC9gYRPtRTmgm5kBEbmSl6ZI6iSpy5rXQH9gNjARWDNTZRBwf/Z6InBmNtulH/BOVpJ5COgvqVt2MrR/1laUSy5mZtCSFxb1ACZIgnyMvTsiHpQ0DRgvaTDwOnBK1v+PwABgLrASOBsgIpZJugaYlvUbHhHLmjqwA7qZGbTYzbkiYj6wVyPtS4EjGmkPYEiRfY0GRpd6bAd0MzPwpf9mZsnIfVTuEWwwB3QzM/D90M3MkuGSi5lZIpyhm5klwgHdzCwN4ZOiZmaJcA3dzCwRLrmYmSXCGbqZWSKcoZuZJcIZuplZIupKf8BFpXJANzMDZ+hmZslwDd3MLBHO0M3MEuEM3cwsEc7QzcwS4VkuZmaJiCj3CDaYA7qZGbiGbmaWDAd0M7NE+KSomVkicrlyj2CDOaCbmYFLLmZmyXBANzNLhGvoZmZpiHrPQzczS4NLLmZmifAsFzOzRCSQoVeVewD2SVVVVUyb+hD3TxgDwI479uavT/6BF557krvvupnNNtuszCO0jaHLlp254Vc/YeKT45j4xDj22u8LfO9HQ5n45Dh+99id3PjrEXTZsvNa22xX3YOp8x/lrPO+WaZRt2H19aUvFcoBvQJdcP65vPjinIb3//GTK/j5yFvZfc+DWb78Hc45+7Qyjs42lmHXfpe/PDaFEw4+la8ffjrzX36Vp/40lRP/5Vt8/bDTeXXeAs69YNBa2/xg+EU8MfmpMo24jYsofSmBpHaSZkh6IHvfR9LTkuZIukdSh6z9M9n7udn6HQv2cVnW/pKko5o7ZqsFdEm7SfqBpJGSbsxe795ax0tFdXVPBhxzBKNHj21oO+zQg7jvvv8F4I477mXgCc3+d7U2rlPnjnzpn/fhvrsmAlD3UR3vvfs+f/3TVHJZrffZv82mx/bdG7Y5/JhDqH3tDea99EpZxtzmtXyGfiHwQsH7nwI/i4i+wHJgcNY+GFgeETsDP8v6IWkP4FRgT+Bo4CZJ7Zo6YKsEdEk/AMYBAqYC07LXYyUNa41jpuKG669m2GXXUp/90myzTTfefvudhv8T176xiO2rtyvnEG0j6LVDNcuXLufaG6/k3kfGcPUNl7NFx83X6nPiN4/nySwb36Lj5pwz9Axu+u/byjHcNNRH6UszJPUCjgV+lb0XcDjw26zLGOBr2euB2Xuy9Udk/QcC4yJidUS8AswFDmjquK2VoQ8G9o+IERFxZ7aMyAYzuJltN1nHDjiSJUv+wTMzZjW05f+7ri0SuG+zNa19+3bs/sVduWfM7zjlyEGsWrmKweef2bC+5qKzyNXV8cB9DwIw5NJ/445bxrFq5apyDbnty+VKXiTVSJpesNSss7efA98H1qTz2wBvR8Sap2jUAtXZ62pgAUC2/p2sf0N7I9s0qrVmudQD2wOvrdPek48/4Cdk/yg1AGq3FVVVnVppeJXpwAP34/jj+nPM0Yez+eafYcstu3DD9VfTtetWtGvXjlwuR6/qnixauLjcQ7VW9ubCJSxe+BaznnkOgIf/8CjnZgH9hG8M4JCvHsS5Jw9t6P/Ffffkq8cdzsVXDqXLVp2J+npWr/6QsaN/2+j+7ZNiPU52RsQoYFRj6yQdByyJiL9JOnRNc2O7aWZdU9s0qrUC+kXAZElz+Pgb5nPAzsDQYhsV/iO171C9yaWhV/xwBFf8cAQA/3LIP3Pxd7/NmYPOZ9zYWzjppGMZP34iZ5xxChP/8HCZR2qtbelby3hz4WJ23OlzvDrvdfp9ZX/mvfwKBx3Wj8FDz+CsE8/jg1WrG/oPGvjthtffueRcVq5Y6WC+vlruStGDgBMkDQA2B7Ykn7F3ldQ+y8J7AQuz/rVAb6BWUntgK2BZQfsahds0qlUCekQ8KGkX8iWWavLfNLXAtIho+7P3N7LLLr+Ou++8ieE//j4z//4co389tvmNrM37yeXX89ObrmazDpux4LU3uPLCaxn30Gg6dOjAreNHAvkTo8O//59lHmkiWuheLhFxGXAZQJahXxIR35J0L3Ay+fOLg4D7s00mZu+fytY/GhEhaSJwt6QbyFc8+pI/J1mUKrUeuylm6Na83br1br6TbXJmL57SWHlivawY/q2SY06nH91V0vEKAvpxkj5PPphvDcwATo+I1ZI2B+4A9iGfmZ8aEfOz7a8AzgHqgIsi4v+aOp6vFDUzA6hr+eJBRDwOPJ69nk8js1Qi4gPglCLbXwdcV+rxHNDNzMC3zzUzS4Zvn2tmlob1mbZYqRzQzczAGbqZWTIc0M3MEuEHXJiZpcHPFDUzS4UDuplZIjzLxcwsEc7QzcwS4YBuZpaGyLnkYmaWBmfoZmZp8LRFM7NUOKCbmSWi7ZfQHdDNzACiru1HdAd0MzNwhm5mlgqfFDUzS4UzdDOzNDhDNzNLhTN0M7M0RF25R7DhHNDNzIBwhm5mlggHdDOzNDhDNzNLhAO6mVkiIqdyD2GDOaCbmeEM3cwsGVHvDN3MLAnO0M3MEhHR9jP0qnIPwMysEkR96UtTJG0uaaqkv0t6TtLVWXsfSU9LmiPpHkkdsvbPZO/nZut3LNjXZVn7S5KOau4zOKCbmQH1OZW8NGM1cHhE7AXsDRwtqR/wU+BnEdEXWA4MzvoPBpZHxM7Az7J+SNoDOBXYEzgauElSu6YO7IBuZkb+pGipS5P7yXs/e7tZtgRwOPDbrH0M8LXs9cDsPdn6IyQpax8XEasj4hVgLnBAU8d2QDczY/0CuqQaSdMLlprCfUlqJ2kmsASYBMwD3o5ouAVYLVCdva4GFgBk698Btilsb2SbRvmkqJkZEOtxO/SIGAWMamJ9DthbUldgArB7Y92yn42l/NFEe1EO6GZmtM489Ih4W9LjQD+gq6T2WRbeC1iYdasFegO1ktoDWwHLCtrXKNymUS65mJmRn7ZY6tIUSdtmmTmStgCOBF4AHgNOzroNAu7PXk/M3pOtfzQiIms/NZsF0wfoC0xt6tglZ+iSPhMRq0vtb2bWluRa7l4uPYEx2YyUKmB8RDwg6XlgnKRrgRnAbVn/24A7JM0ln5mfChARz0kaDzwP1AFDslJOUc0GdEkHZAfcCvicpL2AcyPi/E/xQc3MKlJLXVgUEc8C+zTSPp9GZqlExAfAKUX2dR1wXanHLqXkMhI4DliaHeDvwGGlHsDMrC1oqWmL5VRKyaUqIl7LT4ts0GTab2bW1qzPLJdKVUpAX5CVXSKrCZ0PvNy6wzIz27gqOfMuVSkB/TzyZZfPAYuBR7I2M7Nk5Orb/qS/ZgN6RCwhO+tqZpaqTaLkIulWGrk6KSJqGuluZtYm1Sdw+9xSSi6PFLzeHDiRte8vYGbW5qVwP/RSSi73FL6XdAf5m82YmSVjkyi5NKIPsENLD8SsFDOeu7vcQ7BEbRIlF0nL+biGXkX+0tRhrTkoM7ONLflZLtlN1vcC3sia6rObxpiZJSWFwNbkV1IWvCdERC5bUvjMZmafUB8qealUpfyNMVXSvq0+EjOzMmqp2+eWU9GSS8GN2A8G/k3SPGAF+adoREQ4yJtZMurLPYAW0FQNfSqwLx8/yNTMLFnR6BPf2pamAroAImLeRhqLmVnZ1FVwKaVUTQX0bSVdXGxlRNzQCuMxMyuL1DP0dkBnGn/ytJlZUlKvoS+KiOEbbSRmZmWUeobe9j+dmVmJUs/Qj9hoozAzK7NcAjls0YAeEcs25kDMzMopgSfQfaq7LZqZJac+5QzdzGxTksKNqhzQzcxI/6Somdkmo14uuZiZJSFX7gG0AAd0MzM8y8XMLBme5WJmlgjPcjEzS4RLLmZmiUhh2mIpzxQ1M0teTqUvTZHUW9Jjkl6Q9JykC7P2rSVNkjQn+9kta5ekkZLmSnq28BnOkgZl/edIGtTcZ3BANzMjn6GXujSjDvheROwO9AOGSNoDGAZMjoi+wOTsPcAxQN9sqQFuhvwXAHAV8GXgAOCqNV8CxTigm5nRcgE9IhZFxDPZ6/eAF4BqYCAwJus2ho+f1zwQuD3ypgBdJfUEjgImRcSyiFgOTAKOburYrqGbmQGt8UhRSTsC+wBPAz0iYhHkg76k7lm3amBBwWa1WVux9qKcoZuZsX4ZuqQaSdMLlpp19yepM3AfcFFEvNvEoRv7Kokm2otyhm5mxvpd+h8Ro4BRxdZL2ox8ML8rIn6XNS+W1DPLznsCS7L2WqB3wea9gIVZ+6HrtD/e1LicoZuZkZ+HXurSFEkCbgNeiIgbClZNBNbMVBkE3F/QfmY226Uf8E5WmnkI6C+pW3YytH/WVpQzdDMzWnQe+kHAGcAsSTOztsuBEcB4SYOB14FTsnV/BAYAc4GVwNmQf2qcpGuAaVm/4c09Sc4B3cyMlgvoEfEkjde/oZFnNUdEAEOK7Gs0MLrUYzugm5nhe7mYmSXD93IxM0uEH3BhZpaI+gSKLg7oZmakcbdFB3QzM3xS1MwsGc7QzcwSUae2n6M7oJuZ4ZKLmVkyXHIxM0uEpy2amSWi7YdzB3QzM8AlFzOzZOQSyNEd0M3McIZuZpaMcIZuZpYGZ+jW4nr12p7fjL6RHtttS319Pb/61V384pe3cfWPL+X44/tTXx+8teQfnHPud1m0aHG5h2strP9Jg+jUsSNVVVW0a9eO8aNH8otRt/Pok09RpSq27rYV113xPbpvuw3zX1vAldfdwPMvz+WCmkGc/c2TAVi9+kMGDbmUDz/6iFxdjq8edjBDzz2jzJ+s8qUwbVH5px9VnvYdqitzYK1su+2603O77syYOZvOnTsx9ekHOenkc6itXcR7770PwNAh57D77rswZOiwMo9241u18IlyD6FV9T9pEPfcNpJuXbdqaHt/xQo6d+oEwJ333s+8V17nqu+fz9Llb7PwzcU8+uen2LJL54aAHhGsWvUBHTtuwUd1dZx53iUMu/Df2esLu5flM20Mm3328xv8eIrzdvxGyTHn5lfHV+TjMKrKPQBb25tvLmHGzNkAvP/+Cl58cQ7V22/XEMwBOnXqSKV+EVvLWxPMAVat+gBloWSbbl354u670r792n9oS6Jjxy0AqKuro66uDqki409FqSNKXirVRi+5SDo7In69sY/bFu2wQy/23usLPD11BgDXDP8Bp3/rZN55912O/OopzWxtbZEkar57BZI4ZeAxnDJwAAA33vIbJj44mS6dOjH6FyOa3U8ul+Mb51zA628s5LSvH8c/7blbaw+9zUvhpGg5MvSri62QVCNpuqTp9fUrNuaYKk6nTh0Zf8+tXHzJVQ3Z+ZU/+il9dtqfsWMnMOQ7Z5d5hNYa7rj5eu799S+5+fprGPu7B5g+cxYAF/77WUyecAfH9j+Mu+/7Q7P7adeuHfeN+R8mT7iDWc+/zJz5r7byyNu++vVYKlWrBHRJzxZZZgE9im0XEaMiYr+I2K+qqlOxbslr3749995zK2PHTuD3v/+/T6wfO24CJ544oAwjs9bWfdttgHw55YhDDmTW8y+ttf7Y/ofyyON/KXl/W3bpzP77/hNPTpneouNMUazH/ypVa2XoPYAzgeMbWZa20jGTceuo63nhxbn8/MZRDW0779yn4fXxx/XnpZfmlWNo1opWrvqAFStWNrz+69Rn6Pv5HXltwRsNfR57Ygp9dujV5H6WLX+bd7O/6j5YvZop02bQZ4ferTfwRKSQobdWDf0BoHNEzFx3haTHW+mYSTjowP054/STeXbW80yf9jAAV145grPPPpVddtmJ+vp6Xn/9Db4zZNOb4ZK6pcuWc+Hl1wCQq8sxoP+hHNxvPy66/Fpefb0WVYntt+vOjy49H4B/LF3Gvw6+gPdXrKSqqoo7x/+e+++6hbeWLueKa/+bXH09UR8cdfhXOPSgL5fzo7UJuQQmGnjaorUpqU9btE+nJaYtfnOHE0uOOXe/NqEipw35wiIzM9KY5eKAbmZGZdfGS+WAbmZGGpf+O6CbmeGSi5lZMlKY5eKAbmZGGiUX35zLzIyWvbBI0mhJSyTNLmjbWtIkSXOyn92ydkkaKWludkX9vgXbDMr6z5E0qLnjOqCbmdHil/7/Bjh6nbZhwOSI6AtMzt4DHAP0zZYa4GbIfwEAVwFfBg4ArlrzJVCMA7qZGfmSS6lLcyLiz8CydZoHAmOy12OArxW03x55U4CuknoCRwGTImJZRCwHJvHJL4m1uIZuZgYb4xkDPSJiUXasRZK6Z+3VwIKCfrVZW7H2ohzQzcyA3HqcFJVUQ748ssaoiBhVrH9zu2ukLZpoL8oB3cyM9ZvlkgXv9Q3giyX1zLLznsCSrL0WKLwdZi9gYdZ+6Drtjzd1ANfQzczIl1xKXT6licCamSqDgPsL2s/MZrv0A97JSjMPAf0ldctOhvbP2opyhm5mRsvOQ5c0lnx2/VlJteRnq4wAxksaDLwOrHmO5B+BAcBcYCVwNkBELJN0DTAt6zc8ItY90boWB3QzM1r20v+IOK3IqiMa6RvAkCL7GQ2MLvW4DuhmZvjSfzOzZKRw6b8DupkZDuhmZsmo1Mdxrg8HdDMznKGbmSXDD7gwM0tELtr+U0Ud0M3McA3dzCwZrqGbmSXCNXQzs0TUu+RiZpYGZ+hmZonwLBczs0S45GJmlgiXXMzMEuEM3cwsEc7QzcwSkYtcuYewwRzQzczwpf9mZsnwpf9mZolwhm5mlgjPcjEzS4RnuZiZJcKX/puZJcI1dDOzRLiGbmaWCGfoZmaJ8Dx0M7NEOEM3M0uEZ7mYmSXCJ0XNzBLhkouZWSJ8paiZWSKcoZuZJSKFGrpS+FZKnaSaiBhV7nFYZfHvha2rqtwDsJLUlHsAVpH8e2FrcUA3M0uEA7qZWSIc0NsG10mtMf69sLX4pKiZWSKcoZuZJcIBvcJJOlrSS5LmShpW7vFY+UkaLWmJpNnlHotVFgf0CiapHfA/wDHAHsBpkvYo76isAvwGOLrcg7DK44Be2Q4A5kbE/Ij4EBgHDCzzmKzMIuLPwLJyj8MqjwN6ZasGFhS8r83azMw+wQG9sqmRNk9LMrNGOaBXtlqgd8H7XsDCMo3FzCqcA3plmwb0ldRHUgfgVGBimcdkZhXKAb2CRUQdMBR4CHgBGB8Rz5V3VFZuksYCTwG7SqqVNLjcY7LK4CtFzcwS4QzdzCwRDuhmZolwQDczS4QDuplZIhzQzcwS4YBuLU5STtJMSbMl3Sup4wbs61BJD2SvT2jqjpOSukr6zqc4xo8lXfJpx2hWKRzQrTWsioi9I+ILwIfAtwtXKm+9f/ciYmJEjGiiS1dgvQO6WSoc0K21PQHsLGlHSS9Iugl4Bugtqb+kpyQ9k2XynaHhHvAvSnoS+PqaHUk6S9Ivs9c9JE2Q9PdsORAYAeyU/XXwX1m/SyVNk/SspKsL9nVFdp/5R4BdN9q/hlkrckC3ViOpPfl7uc/KmnYFbo+IfYAVwA+BIyNiX2A6cLGkzYFbgeOBrwDbFdn9SOBPEbEXsC/wHDAMmJf9dXCppP5AX/K3Id4b+JKkQyR9ifxtFPYh/4Wxfwt/dLOyaF/uAViStpA0M3v9BHAbsD3wWkRMydr7kX9ox18kAXQgfzn7bsArETEHQNKdQE0jxzgcOBMgInLAO5K6rdOnf7bMyN53Jh/guwATImJldgzfH8eS4IBurWFVROxd2JAF7RWFTcCkiDhtnX5703K3CBbwHxFxyzrHuKgFj2FWMVxysXKZAhwkaWcASR0l7QK8CPSRtFPW77Qi208Gzsu2bSdpS+A98tn3Gg8B5xTU5qsldQf+DJwoaQtJXciXd8zaPAd0K4uIeAs4Cxgr6VnyAX63iPiAfInlf7OToq8V2cWFwGGSZgF/A/aMiKXkSzizJf1XRDwM3A08lfX7LdAlIp4B7gFmAveRLwuZtXm+26KZWSKcoZuZJcIB3cwsEQ7oZmaJcEA3M0uEA7qZWSIc0M3MEuGAbmaWCAd0M7NE/D/Ua3p4F2r5gQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.4.3] KNN (KD-Tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 19.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VPXV+PHPyUwyIXsCSchCAiGJJkDCEpGiQDWyKIqK4C4o1q6WotaK7a9PXfooVetStbWtsqgoVR9RKgpFRCuK7ItFXNiEQFgCgbBlP78/ZjJOIISBMmQC5/16zStz7/3ee0+SIYfvekVVMcYYY1qbkJYOwBhjjDkRlsCMMca0SpbAjDHGtEqWwIwxxrRKlsCMMca0SpbAjDHGtEqWwIwxxrRKlsCMMca0SpbAjDHGtErOlg7gZGnXrp127NixpcMwxphWZenSpWWqmtjScZyI0yaBdezYkSVLlrR0GMYY06qIyLctHcOJsiZEH4cOHWLAgAHU1dUBMGTIEOLi4rj00kv9Or+qqoprrrmG7Oxszj33XDZu3NhkuTFjxpCUlETXrl0b7b/vvvtIS0uje/fudO/enXfffReAXbt2ccEFFxAVFcXtt9/e6JyLLrqI8vLy4/xOjTGm9bME5mPixIkMHz4ch8MBwN13381LL73k9/kvvPAC8fHxrF27ljvuuIN77rmnyXI333wzs2bNavLYHXfcwYoVK1ixYgWXXHIJAOHh4Tz44IM89thjR5S/6aab+POf/+x3jMYYc7qwBOZj6tSpXH755d7t4uJioqOj/T7/7bffZvTo0QCMGDGCuXPn0tRq//379ychIcHv60ZGRnL++ecTHh5+xLFhw4bx6quv+n0tY4w5XVgC86iurmb9+vX8NwNBtmzZQocOHQBwOp3Exsaya9eu47rGM888Q0FBAWPGjPGraTA+Pp6qqqrjvo8xxrR2lsA8ysrKiIuL+6+u0VRtS0T8Pv8nP/kJ69atY8WKFaSkpHDXXXf5dV5SUhJbt271+z7GGHM6sATm0aZNGyorK/+ra6Snp7N582YAamtr2bt373E1FSYnJ+NwOAgJCeG2225j0aJFfp1XWVlJmzZtTihmY4xprSyBecTHx1NXV+dXErv33nuZPn36EfuHDRvGlClTAHjjjTe48MILj6sGVlpa6n0/ffr0I0YpNkVV2bZt23/V9GmMMa3RGZ/AKiprePL9r1mxeQ+DBg1i/vz53mP9+vVj5MiRzJ07l/T0dGbPng3A559/Tvv27Y+41q233squXbvIzs7m8ccfZ8KECQBs3brVO6IQ4LrrruN73/seX331Fenp6bzwwgsA/OpXv6Jbt24UFBQwb948nnjiCe85HTt25M4772Ty5Mmkp6fzxRdfALB06VL69OmD03naTOkzxhi/SFP9Nift4iJDgKcAB/C8qk447Hh/4EmgALhWVd/wOVYHfO7Z3KSqw5q7V1FRkZ7IROa9h2oovP9f/L+hefSK2svjjz9+zKHzgwcP9iazlvaLX/yCYcOGUVxc3NKhGGNaIRFZqqpFLR3HiQjYf9tFxAE8CwwESoDFIjJDVb/wKbYJuBn4ZROXOKSq3QMVX4OYcCcuZwg791XRo18PLrjgAurq6rxzwZoSLMkLoGvXrpa8jDFnpEC2O/UG1qrqegARmQZcDngTmKpu9ByrD2AczRIREqNd7NhXBbhXyWhNbrvttpYOwRhjWkQg+8DSgM0+2yWeff4KF5ElIvKZiFzRVAER+aGnzJKdO3eecKCJ0S52ehKYMcaY1iGQCayp4XfH0+GW4WmXvR54UkQ6H3Ex1b+papGqFiUmnvhiyknRLnbs+++G0BtjjDm1ApnASoAOPtvpgN+zbVV1q+freuBDoMfJDM6X1cCMMab1CWQCWwzkiEgnEQkDrgVm+HOiiMSLiMvzvh1wHj59ZydbUnQ45QdrqK5tsa44Y4wxxylgCUxVa4HbgdnAGuA1VV0tIg+IyDAAETlHREqAkcBfRWS15/Q8YImIrATmARMOG714UiVGuwAo22+1MGOMaS0COvtVVd8F3j1s3//4vF+Mu2nx8PM+BboFMjZfSZ4EtnNfFalxtiSTMca0Bmf8ShzwXQ1sh/WDGWNMq2EJjO8SmA3kMMaY1sMSGNAuqqEGZkPpjTGmtbAEBoQ6QkiIDLMamDHGtCKWwDySfJaTMsYYE/wsgXnYZGZjjGldLIF5WAIzxpjWxRKYR0MCC+Tz0Ywxxpw8lsA8EqNcVNfVU3GotqVDMcYY4wdLYB5JMeGADaU3xpjWwhKYR2KUTWY2xpjWxBKYR1KMLSdljDGtiSUwD1tOyhhjWhdLYB7RLifhoSHstEeqGGNMq2AJzENESIx2saPCBnEYY0xrYAnMR2KUy2pgxhjTSlgC85EUHc6OCktgxhjTGlgC85EYbTUwY4xpLSyB+UiKdrHnYA1VtXUtHYoxxphjsATmo2Eofdn+6haOxBhjzLFYAvPRMJnZ5oIZY0zwswTmIzHKsx6iDaU3xpigZwnMh3c1DhvIYYwxQc8SmI+2UWGIYEPpjTGmFbAE5iPUEUJCRJjVwIwxphWwBHYY93JSlsCMMSbYWQI7jE1mNsaY1sES2GGSosPZaaMQjTEm6FkCO0xDDUxVWzoUY4wxzbAEdpjEaBc1dcreQzUtHYoxxphmWAI7TJJnLtgOW43DGGOCmiWww3gnM1sCM8aYoBbQBCYiQ0TkKxFZKyLjmzjeX0SWiUitiIxo4niMiGwRkWcCGaev72pgNpDDGGOCWcASmIg4gGeBi4F84DoRyT+s2CbgZuCVo1zmQeCjQMXYFKuBGWNM6xDIGlhvYK2qrlfVamAacLlvAVXdqKqrgPrDTxaRXkAy8K8AxniEKJeTNqEOm8xsjDFBLpAJLA3Y7LNd4tl3TCISAvwRuDsAcR3r3jaZ2RhjWoFmE5iIOETk0RO8tjSxz9/JVT8F3lXVzc0VEpEfisgSEVmyc+fO4w7waBKjXdaEaIwxQc7Z3EFVrRORXiIievwze0uADj7b6cBWP8/9HtBPRH4KRAFhIrJfVRsNBFHVvwF/AygqKjppM4+Tol18s2P/ybqcMcaYAGg2gXksB94WkdeBAw07VfXNY5y3GMgRkU7AFuBa4Hp/glLVGxrei8jNQNHhySuQEqNdfLpu16m6nTHGmBPgTwJLAHYBF/rsU6DZBKaqtSJyOzAbcAATVXW1iDwALFHVGSJyDjAdiAcuE5H7VbXLiXwjJ1NStIu9h2qorKkjPNTR0uEYY4xpwjETmKrecqIXV9V3gXcP2/c/Pu8X425abO4ak4HJJxrDiWgYSl+2v4r0+IhTeWtjjDF+OuYoRBFJF5HpIrJDRLaLyP+JSLNJp7VLig4HbDkpY4wJZv4Mo58EzABScQ+D/6dn32nLJjMbY0zw8yeBJarqJFWt9bwmA4kBjqtFWQIzxpjg508CKxORGz1zwhwiciPuQR2nrbaRYYhYE6IxxgQzfxLYGOBqYBtQCozw7DttOR0htI0MsxqYMcYEsWZHIXoW5L1KVYedoniCRmJ0ODttRXpjjAlazdbAVLWOwxbgPVPYclLGGBPc/JnI/InneVz/oPFKHMsCFlUQSIp28c32fS0dhjHGmKPwJ4H19Xx9wGef0nhljtNOYrSLsv1V1NcrISFNrUtsjDGmJR2rDywE+IuqvnaK4gkaiVEuauqUvYdqiI8Ma+lwjDHGHOZYfWD1wO2nKJagkhTjngtmQ+mNMSY4+TOMfo6I/FJEOohIQsMr4JG1sMQom8xsjDHBzJ8+sIY5Xz/z2adA1skPJ3gkxTSsh2hD6Y0xJhj5sxp9p1MRSLCx5aSMMSa4HbUJUUR+5fN+5GHHHgpkUMEgyuUkIsxhfWDGGBOkmusDu9bn/b2HHRsSgFiCjk1mNsaY4NVcApOjvG9q+7SUGGUJzBhjglVzCUyP8r6p7dNSUozLBnEYY0yQam4QR6GIVOCubbXxvMezHR7wyIJAYpSL+fvKWjoMY4wxTThqAlNVx6kMJBglxYRTUVlLZU0d4aFn/I/DGGOCij8Tmc9YNpnZGGOClyWwZiTaclLGGBO0LIE1w2pgxhgTvCyBNSPJuxqHjUQ0xphgc8wEJiLDReQbEdkrIhUiss9nROJprW2UixCxGpgxxgQjfxbzfQS4TFXXBDqYYOMIERIiXezcbwnMGGOCjT9NiNvPxOTVICnaxY4KS2DGGBNs/KmBLRGRfwBvAd6/5Kr6ZsCiCiKJ0VYDM8aYYORPAosBDgKDfPYpcEYksKRoF19t29fSYRhjjDmMP88Du+VUBBKsEqNdlO2vor5eCQk5I9YwNsaYVsGfUYjpIjJdRHaIyHYR+T8RST8VwQWDxGgXtfVK+cHqlg7FGGOMD38GcUwCZgCpQBrwT8++M0JStHvdYusHM8aY4OJPAktU1UmqWut5TQYS/bm4iAwRka9EZK2IjG/ieH8RWSYitSIywmd/pogsFZEVIrJaRH7s93d0kiVG22ocxhgTjPxJYGUicqOIODyvG4FdxzpJRBzAs8DFQD5wnYjkH1ZsE3Az8Mph+0uBvqraHTgXGC8iqX7EetI1rMZhQ+mNMSa4+JPAxgBXA9twJ5YRnn3H0htYq6rrVbUamAZc7ltAVTeq6iqg/rD91arakDFcfsYZEN4amDUhGmNMUPFnFOImYNgJXDsN2OyzXYK7NuUXEekAzASygbtVdesJxPBfi3Q5iQxzWA3MGGOCzFETmIj8SlUfEZGncc/7akRVxx7j2k2NOT/iOkejqpuBAk/T4Vsi8oaqbj8sxh8CPwTIyMjw99LHzSYzG2NM8GmuBtawfNSSE7x2CdDBZzsdOO5alKpuFZHVQD/gjcOO/Q34G0BRUZHfyfF4JUa72FFhK9IbY0wwOWoCU9V/et4eVNXXfY+JyEg/rr0YyBGRTsAW4Frgen+C8swz26Wqh0QkHjgPeNyfcwMhKTqcNdvOiAX4jTGm1fBncMS9fu5rRFVrgduB2bhrc6+p6moReUBEhgGIyDkiUgKMBP7qqWkB5AELRWQl8BHwmKp+7kesAZEY7bJh9MYYE2Sa6wO7GLgESBORP/kcigFq/bm4qr4LvHvYvv/xeb8Yd9Pi4efNAQr8ucepkBjtYl9lLZU1dYSHOlo6HGOMMTRfA9uKu/+rEljq85oBDA58aMHDJjMbY0zwaa4PbCWwUkReUdWaUxhT0PFOZt5XSYeEiBaOxhhjDPj3OJWOIvIw7tU0wht2qmpWwKIKMlYDM8aY4OPvYr5/wd3vdQHwIvBSIIMKNoneGpglMGOMCRb+JLA2qjoXEFX9VlXvAy4MbFjBpW2kixCxGpgxxgQTf5oQK0UkBPhGRG7HPacrKbBhBRdHiNA2yobSG2NMMPGnBjYOiADGAr2AG4HRgQwqGCVFu6wJ0Rhjgog/i/ku9rzdD9wS2HCCl01mNsaY4HLMGpiIzBGROJ/teBGZHdiwgo+7BmbrIRpjTLDwpwmxnaruadhQ1XLOsD4wcNfAyvZXU18fsDWDjTHGHAd/Eli9iHifVSIimRzHY1FOF4lRLurqld0Hq1s6FGOMMfg3CvE3wHwR+ciz3R/PM7jOJEkx7jncO/dV0S7K1cLRGGOM8WcQxywR6Qn0wf2QyjtUtSzgkQUZ39U48lJaOBhjjDFHb0IUkbM9X3sCGbgX990CZHj2nVGSbDUOY4wJKs3VwO7E3VT4xyaOKWfYahy2HqIxxgSX5hLYHM/XW1V1/akIJphFhDmJcjltKL0xxgSJ5kYhNjx1+Y1TEUhrYJOZjTEmeDRXA9slIvOATiIy4/CDqjoscGEFp8QoW07KGGOCRXMJbCjQE/ejU5rqBzvjJMa4WLO1oqXDMMYYQ/NPZK4GPhORvqq68xTGFLQSo1x8ZDUwY4wJCkdNYCLypKqOAyaKyBErb5yJTYhJMS72V9VysLqWiDB/5oAbY4wJlOb+Cjc8dfmxUxFIa5DoWYGjbF81GW0tgRljTEtqrglxqedrwxJSiEg80EFVV52C2IJOw3JSO/ZVktE2ooWjMcaYM5s/j1P5UERiRCQBWAlMEpHHAx9a8GmogdlQemOMaXn+rEYfq6oVwHBgkqr2Ai4KbFjBKdGWkzLGmKDhTwJzikgKcDXwToDjCWoJkWE4QsRqYMYYEwT8SWAPALOBtaq6WESygG8CG1ZwcoQIbSPDbDkpY4wJAv48TuV14HWf7fXAVYEMKpglxdhyUsYYEwz8GcTxiGcQR6iIzBWRMhG58VQEF4wSo1zs3G8JzBhjWpo/TYiDPIM4LgVKgFzg7oBGFcSSosPZUWEJzBhjWpo/CSzU8/US4FVV3R3AeIJeYrSLXQeqqas/YnESY4wxp5A/CeyfIvIlUATMFZFE4IwdxZAY7aKuXtl9oLqlQzHGmDPaMROYqo4HvgcUqWoNcAC4PNCBBaskezKzMcYEBX9qYABpwFUiMgoYAQzy5yQRGSIiX4nIWhEZ38Tx/iKyTERqRWSEz/7uIrJARFaLyCoRucbPOAPuu8nMlRw6dIgBAwZQV1fHt99+S69evejevTtdunThueeeO+a1du/ezcCBA8nJyWHgwIGUl5c3WW7IkCHExcVx6aWXNtr/zDPPkJ2djYhQVlbm3V9eXs6VV15JQUEBvXv35j//+Q8A1dXV9O/fn9ra2hP99o0xJmj4Mwrxd8DTntcFwCPAMVeiFxEH8CxwMZAPXCci+YcV2wTcDLxy2P6DwChV7QIMAZ4Ukbhj3fNUSIp2r4e4c18VEydOZPjw4TgcDlJSUvj0009ZsWIFCxcuZMKECWzdurXZa02YMIHi4mK++eYbiouLmTBhQpPl7r77bl566aUj9p933nm8//77ZGZmNtr/0EMP0b17d1atWsWLL77IL37xCwDCwsIoLi7mH//4x4l868YYE1T8qYGNAIqBbap6C1AIuPw4rzfuyc/rPc8Wm8ZhTY+qutGzMHD9Yfu/VtVvPO+3AjuARD/uGXANNbCd+6uYOnUql1/u/pbCwsJwudzHqqqqqK+vP+o1Grz99tuMHj0agNGjR/PWW281Wa64uJjo6Ogj9vfo0YOOHTsesf+LL76guLgYgLPPPpuNGzeyfft2AK644gqmTp16zNiMMSbY+ZPADqlqPVArIjG4k0mWH+elAZt9tks8+46LiPQGwoB1TRz7oYgsEZElO3eemmdutglzEO1yUrprP+vXr2+UQDZv3kxBQQEdOnTgnnvuITU1tdlrbd++nZSUFABSUlLYsWPHSYmxsLCQN998E4BFixbx7bffUlJSAkDXrl1ZvHjxSbmPMca0JH8S2BJP893fgaXAMmCRH+dJE/uOa+y5Zw3Gl4BbPEm08cVU/6aqRapalJh46ipoidEuNm/bTlxc41bNDh06sGrVKtauXcuUKVO8tZ5Tbfz48ZSXl9O9e3eefvppevTogdPpXnTF4XAQFhbGvn37WiQ2Y4w5WfxZSuqnnrfPicgsIMbP54GVAB18ttOB5juFfHhqezOB/6eqn/l73qnQLtrFnv1VVFY2PZsgNTWVLl268PHHHzNixIgmywAkJydTWlpKSkoKpaWlJCUlnZT4YmJimDRpEgCqSqdOnejUqZP3eFVVFeHh4SflXsYY01KOWgMTkZ6Hv4AE3KvT9/Tj2ouBHBHpJCJhwLXADH+C8pSfDrzoWYsxqCRFu9irLurq6rxJrKSkhEOHDgHuUYCffPIJZ511FgCjRo1i0aIjK63Dhg1jypQpAEyZMsXbn/bf2rNnD9XV7nlqzz//PP379ycmJgaAXbt2kZiYSGhoaHOXMMaYoNdcE+Ifm3k9dqwLq2otcDvulezXAK+p6moReUBEhgGIyDkiUgKMBP4qIqs9p18N9AduFpEVnlf3E/oOAyAx2sWOikoGDRrE/PnzAVizZg3nnnsuhYWFDBgwgF/+8pd069YNgFWrVnn7unyNHz+eOXPmkJOTw5w5cxg/3j3TYMmSJfzgBz/wluvXrx8jR45k7ty5pKenM3v2bAD+9Kc/kZ6eTklJCQUFBd5z1qxZQ5cuXTj77LN57733eOqpp7zXmjdvHpdccklgfjDGGHMKierpsSRSUVGRLlmy5JTc6y8fruMPs77klSuT+Mszf2pyiHuDiooKbr31Vl5/PTgqksOHD+fhhx/21g6NMWc2EVmqqkUtHceJ8Gce2M9852CJSLyI/LS5c053uclRAFTFZnLBBRdQV1d31LIxMTFBk7yqq6u54oorLHkZY04L/oxCvE1V9zRsqGo5cFvgQgp+/XISiY8I5f+WlTBmzBgcDkdLh+SXsLAwRo0a1dJhGGPMSeFPAgsREe+QeM8KG2GBCyn4hTlDuKwwlTlfbKeisqalwzHGmDOSPwlsNvCaiBSLyIXAq8CswIYV/K7skUZVbT3vfV7a0qEYY8wZyZ8Edg8wF/gJ8DPP+18FMqjWoHuHOLLaRfLmsi0tHYoxxpyR/HmcSr2qPqeqI3D3fS1Q1aOPWjhDiAhX9khj4YbdlJQfbOlwjDHmjOPPKMQPRSRGRBKAFcAkEXk88KEFvyt6uJd2fGu51cKMMeZU86cJMVZVK4DhwCRV7QVcFNiwWocOCRH07pTAm8u3cLrMpzPGmNbCnwTm9CyqezXwToDjaXWG90hj/c4DrCrZ29KhGGPMGcWfBPYA7pGIa1V1sYhkAd8ENqzW45KCFMKcIby5rKSlQzHGmDOKP4M4XlfVgoZV6T0PqLwq8KG1DjHhoQzMT+afq0qprj32QyyNMcacHEd9nIqI/EpVHxGRp2niOV6qOjagkbUiw3ukMXNVKR99vZOB+cktHY4xxpwRmnse2BrP11OzQm4r1j83kbaRYUxfXmIJzBhjTpGjJjBV/afn65RTF07rFOpwLy31yqJN7D1YQ2yEPWvLGGMCrbkmxGYfPqmqw05+OK3XVT3TmfzpRmZ+Xsr152a0dDjGGHPaa64J8XvAZtxrHy4EpJmyZ7yuaTFkJ0UxfXmJJTBjjDkFmhuF2B74NdAVeAoYCJSp6keq+tGpCK41aVhaavHGcjbtsqWljDEm0I6awFS1TlVnqepooA+wFvhQRH5+yqJrZa7okYYITLelpYwxJuCanQcmIi4RGQ68jHsl+j8Bb56KwFqjtLg29OnUlunLS2xpKWOMCbCjJjARmQJ8CvQE7lfVc1T1QVW16kUzhvdMY+OugyzbtOfYhY0xxpyw5mpgNwG5wC+AT0WkwvPaJyIVpya81ufibimEh4YwfbktLWWMMYHUXB9YiKpGe14xPq9oVY05lUG2JlEuJ4O7tOefK0upqj3jH5tmjDEB489ivuY4Xdkjjb2Hapj35c6WDsUYY05blsAC4PzsdrSLclkzojHGBJAlsABwOkK4onsqH3y5g/ID1S0djjHGnJYsgQXIlT3TqKlT3vm8tKVDMcaY05IlsADJT4nh7PbRTLcHXRpjTEBYAguQhqWllm3aw4ayAy0djjHGnHYsgQXQ5d2PXFrq0KFDDBgwgLo69xB7h8NB9+7d6d69O8OGHXuB/6qqKq655hqys7M599xz2bhxY5PlxowZQ1JSEl27dm20//XXX6dLly6EhISwZMl3j3r7/PPPufnmm4//mzTGmBZiCSyA2seGc352u0ZLS02cOJHhw4fjcDgAaNOmDStWrGDFihXMmNHsE2wAeOGFF4iPj2ft2rXccccd3HPPPU2Wu/nmm5k1a9YR+7t27cqbb75J//79G+3v1q0bJSUlbNq06Xi/TWOMaRGWwALsyh5pbN59iCXflgMwdepULr/88hO+3ttvv83o0aMBGDFiBHPnzm1y3cX+/fuTkJBwxP68vDzOOuusJq992WWXMW3atBOOzRhjTiVLYAE2uEt7IsIcvLlsC9XV1axfv56OHTt6j1dWVlJUVESfPn146623jnm9LVu20KFDBwCcTiexsbHs2rXrpMRaVFTExx9/fFKuZYwxgRbQBCYiQ0TkKxFZKyLjmzjeX0SWiUitiIw47NgsEdkjIu8EMsZAi3Q5GdKlPe+s2kpJ6Xbi4uIaHd+0aRNLlizhlVdeYdy4caxbt67Z6zVV2xI5Oc8aTUpKYuvWrSflWsYYE2gBS2Ai4gCeBS4G8oHrRCT/sGKbgJuBV5q4xKO4FxRu9a7smca+yloWl+ynsrKy0bHU1FQAsrKy+P73v8/y5cubvVZ6ejqbN28GoLa2lr179zbZVHgiKisradOmzUm5ljHGBFoga2C9gbWqul5Vq4FpQKPOH1XdqKqrgPrDT1bVucC+AMZ3yvTt3I7kGBevLN9FTW2dN4mVl5dTVVUFQFlZGZ988gn5+e4cf++99zJ9+vQjrjVs2DCmTJkCwBtvvMGFF1540mpgX3/99RGjFo0xJlgFMoGlAZt9tks8+04aEfmhiCwRkSU7dwbvwrmOEOE3Q/NZU7qP2pSuTH7TPTpwzZo1FBUVUVhYyAUXXMD48eO9Cezzzz+nffv2R1zr1ltvZdeuXWRnZ/P4448zYcIEALZu3coll1ziLXfdddfxve99j6+++or09HReeOEFAKZPn056ejoLFixg6NChDB482HvOvHnzGDp0aMB+DsYYczJJoJ4cLCIjgcGq+gPP9k1Ab1X9eRNlJwPvqOobh+3/PvBLVb30WPcrKipS33lNwWjl5j2MevQfbJj3D57520Ru6pN51NrT4MGDmT179imLraqqigEDBjB//nycTucpu68xpmWJyFJVLWrpOE5EIGtgJUAHn+104IweIVDYIY6PJtxCwTnn8dvpqxg7bQUHqmqbLOubvHwnP8+bN8878bl79+6Eh4cfc/SiP5OfN23aRHx8PKmpqUc0I+7evZuBAweSk5PDwIEDKS93TwnYu3cvl112GYWFhXTp0oVJkyYBsHPnToYMGXI8PxpjjDlugUxgi4EcEekkImHAtcCxZ+qe5uIiwvj38w/wq4vzmblqK8Oemc8325vv6vOd/HzBBRd4Jz5/8MEHREREMGjQoGbP92fyc05ODvfee2+Tk58nTJhAcXEx33zzDcXFxd5my2effZb8/HxWrlzJhx9+yF133UV1dTWJiYmkpKTwySefHMdPxhhjjk/AEpiq1gIubCtQAAAgAElEQVS3A7OBNcBrqrpaRB4QkWEAInKOiJQAI4G/isjqhvNF5GPgdaBYREpEZPCRd2mdQkKEn12Qzcu3nsveQzUMe+YT3vJZbupwR5v8/MYbb3DxxRcTERHR7P3+28nPvuePHj3aW+MTEfbt24eqsn//fhISErzNj1dccQVTp05tNi5jjPlvBHQemKq+q6q5qtpZVf/Xs+9/VHWG5/1iVU1X1UhVbauqXXzO7aeqiaraxlPm1HUInSJ9s9sxc2w/uqbFMO4fK/jN9M+prKlrVKapyc8Npk2bxnXXXXfM+/y3k5+3b99OSkoKACkpKezYsQOA22+/nTVr1pCamkq3bt146qmnCAlxf6RsUrQxJtBsJY4WlhwTziu39eFH/bOYunATI59bwObdB73Hy8rKjpj8DFBaWsrnn3/eaBTh0QRq8vPs2bPp3r07W7duZcWKFdx+++1UVFQANinaGBN4lsCCQKgjhHsvyeOvN/ViY9kBLn16PnPXbAfci/0ePvkZ4LXXXuPKK68kNDT0mNf/byc/JycnU1rqfjBnaWkpSUlJAEyaNInhw4cjImRnZ9OpUye+/PJLwCZFG2MCzxJYEBncpT3vjD2ftLg23DplCY/M+pLomFjq6uqOSGKvvvrqEc2HgZr87Hv+lClTvP1xGRkZzJ07F3A3M3711VdkZWUBNinaGBN4lsCCTGbbSN78aV+uPacDf/5wHWOnLeeigQOZP3++t8zGjRvZvHkzAwYMaHRuoCY/jx8/njlz5pCTk8OcOXMYP969rOVvf/tbPv30U7p160ZxcTF/+MMfaNeuHWCToo0xp4CqnhavXr166enm7/9ep5n3vKMjfv+yXn/DDccsP2jQoFMQlX/69eunu3fvbukwjDnpDh48qP3799fa2lrvvr1792pqaqr+7Gc/8+4bPHiwFhQUaH5+vv7oRz9qVL4p9fX1+vOf/1w7d+6s3bp106VLlzZZ7te//rWmp6drZGRko/3l5eWamJioWVlZ2rt3b92wYYM3rmuuuUb79Omj+fn52qFDB+3atavm5+drRkaGAsv1GH9fgXuBtcBXuBeoaKrM7Z4yCrTz2S/AnzzHVgE9fY5lAP/CPVL9C6CjZ/80IOeYcR2rQGt5nY4JTPW7JPb9236rh6qqWzocv+zYsUOnT5/e0mEYExDPPPOMPvnkk432jR07Vq+77rpGCWzv3r2q6k5Mw4cP11dffbXZ686cOVOHDBmi9fX1umDBAu3du3eT5RYsWKBbt249IoFdffXV2rdvX1VVffXVV/Xqq6/WsWPH6rXXXqsJCQm6YsUKLSsr07S0NC0tLVVV1fPOO0+Bndp88soHVgIuoBOwDnA0Ua4H0BHYeFgCuwR4z5PI+gALfY59CAz0vI8CIjzvBwB/by4uVbUmxGD3g35Z/L+heWxIOJc7XltFTd0R6x4HncTERK644oqWDsOYgDh8XubSpUvZvn37EQsKxMTEAO6BU9XV1cfsd3777bcZNWoUIkKfPn3Ys2ePd/CUrz59+nintfiaPXs2d911F+Ce7zl79my2b99O+/btadeuHYWFhaxfv56zzz7b29Vw4403AsQf41u+HJimqlWqugF3Tar34YVUdbmqbjzK+S968uxnQJyIpHieTuJU1Tme8/erasMQ7I+Bi0Sk2XXtLIG1Ag1J7L3/bGPsq8tbRRIz5nR0+LzM+vp67rrrLh599NEmyw8ePJikpCSio6MZMWJEk2Ua+M7XBPfo4S1bjr7AweFxHThwgN693XklJCSEqqoq7r33XrZv3+6N5dZbb2Xx4sVs3LiR2tpa5syZAyAi0raZy/+3C7Mf7fxcYI+IvCkiy0XkUc9juFDVetyJsrDZKx+ritZaXqdrE6Kv5z9er5n3vKM/fmmJVtfWHbXc4W30v/rVr7RLly7apUsXnTZt2jHvU1lZqVdffbV27tzZ25belJtuuklDQ0M1Pz9fVd1t/rGxsTpo0CC96KKLNDs7Wy+66CK94YYbtKCgQDt27KiAdunSRQsLCzU7O1tjY2P14osv1n79+mlNTY2qqo4ePVrj4+M1LCxMXS6XPvvss03e/+mnn9bOnTsroE8//bRmZ2drdna2Tpo0qcm+hNdee01xt89rTk6OXnbZZd5rdevWTTMyMjQ3N1dnzZrl3Z+Zmaldu3bVwsJC9f2M3XXXXTp37txj/ixbA9/Py8aNG7Vnz55aWFio+fn5+pe//MVbzrfvxd9zcnJyNCwsTENDQzUhIUGrqqqOKH/NNdd4f499+/bVuLg47+++bdu2mpWVpWeddZaeffbZ3nMADQ0NVRFRl8vlLe9wONTlcmlWVpYWFhZ6f3eJiYne332XLl20c+fO3rhCQkK0Y8eO3s9KQ78RoGFhYQpop06dNDY2VgENCQnR4uJi7dy5s6akpOjYsWNVVXXSpEmNzsnOztbOnTtrdna2tm3bVuPi4jQrK6vRfRquCWhUVJTGxMRobm6uduvWTQFt165dk3EVFhaqiGhMTIympKRoaGioAtqvXz9Vdf/bSEhI0LKyMr366qvV5XJ5//0lJCRoaGiohoeH6/DhwxX346w24dM/hbs/qyG2XcAyz34B9gI7ObI/qw5YAVQDs3z2bwWu9dmeC/QCRniulQU4gf8DbvUpNxW4TJv5u9/iiedkvc6EBKbqXxLzbaN/55139KKLLtKamhrdv3+/9urVy9s2fzTPPvus/uhHP1LV79rSmzJu3Di96667tEuXLqqq+v777+uMGTM0KytLH374YVVVffjhh/UXv/iFqrr7CRISErzHGsoPHTpU77vvPn355ZdVVfXyyy/XjIwMrays1PXr12tWVlaTHeDLli3TDRs2aHp6umZmZuquXbt09+7dmpycrMXFxUf0JXzxxRfapk0bHTBggC5evNh7ndWrV2vnzp31lltuOeJ+mZmZunPnziPuvXHjRh04cGCzP8fWwvfzUlVVpZWVlaqqum/fPs3MzNQtW7aoauO+F3/OWb16tXbp0kUrKyv1vvvu09DQUJ06deoR5VNSUvSzzz7TzMxMzcvL0+9973taX1+vV155pYaHh2t9fb3++9//1qKiIu85gC5btkzbtGmjBQUF3s+KiGhcXJyqqr700kt61VVXqarq66+/rk6nU9PS0nTmzJkaGhqqmzdv1htuuEFFREePHu39rDT0ZzkcDt2wYYNmZmbqc889p3l5eRoREaExMTHqcDi0pqZGBw0apGFhYZqZmalt27ZVQH/84x9rSkqK93tfv369hoWFaVhYWKPP5NixY3XIkCHqcDg0MzNTR40apXl5ebp48WL9yU9+ok6nU6+77roj4nI6naqqGhkZqatXr9aCggItLS31JvPa2lq99tprNSQkRDMzMzUqKkqdTqc3rsTERH3ooYd0/fr1GhMT05DAsvHpn8Ldn3UAd3/Wg8C9nv2XAHuAtzmyP2u/5+tGGveBzQA+8Nn+CkjxnP+hz/6bgGd9tv8PuEib+btvTYitzK3nd+K3l+bz3n+28fNXmm5O9G2j/+KLLxgwYABOp5PIyEgKCwubXLDXl79rJy5cuLBRX1dxcTHR0dFs37690dqJ7733HgDvv/8+4eHh3r6AhvLQeO3EzZs3c9555+FyuejUqRPZ2dksWrToiPv36NGDjh07UllZyYABA0hISCA+Pp74+Hjy8vKO6EvIy8vzLnV1+Pc7ZswY5s2bR4cOHY56P1+ZmZns2rWLbdu2NVuuNfD9vISFheFyuQD3Uwzq67/7fPn2vfhzzttvv80NN9yAy+Vi1qxZREVFsW7duiPKO51Ob9PZ7t27GTp0KCLCp59+SnR0NNu2baNfv35UVFRQWlpKVVUVIkJycjK1tbVce+21uFwuamtrUVXq6+u9+z/88ENUlc6dO3s/d2+88QYZGRmkp6fz/vvvk5iYyJtvvkmfPn3Ytm0bGzduZNCgQYiIt6lw8+bN3HTTTYgIdXV1iAgLFixg9uzZdOzYkQULFvDYY4/hdDr5y1/+wsGDBykuLsblctGhQwdqa91Pnairq2PJkiV8/fXXbNy4kWuuucYb1+DBg9m2bRuqyrRp00hJSWHBggUUFRU1isvXm9PfYtBlw1m+zt1XVl1dzaeffsrll1/OVVddxcaNG5kwYQJxcXH88Y9/5NChQ4SHh1NQUEBcXByHDh0CdwLbiDupZItIiqoux137ApgFXCsiLuBGoAooAJbg6c86xkfsb0AfEXGKSB9gr6qW4l7wPV5EEj3lLsQ9ErFBLrCaZlgCa4VuPb8T/3NpPrNWH5nEDm+jLyws5L333uPgwYOUlZUxb94876ocR+PP2okN90lPTz/i/KqqqiPWThw1ahRr1qxhx44dvPzyy1x88cWsXv3dZ7Nr164sXrwYgIMHDzJv3jwKCgq44447SElJabYvoLa2ltTUVO92fX19oz+8vn0JlZWVLF26lJtvvtm7KPGWLVvIyMggOzublStXNiovIgwaNIhevXrxt7/9rdF9e/bs2epX3G9qrc3NmzdTUFBAhw4duOeeexr9bMHdauPPOQ2fo4EDB7Jw4ULi4+PJzc1t9h7V1dVMnDiRgoICysrKGv0u2rVrR//+/b2fzWHDhlFTU+Pt43nppZcASE1NZeXKld7/tHXt2pW+ffsCsGPHDt555x127NjB119/TXl5OSEhIVRXV7Nz504qKiq47bbbqK6tp66ujuz8Akq2bGXS9H/x0uf7qaypQ1xROF3hTJ0xh537qhrFWFdXR1FRERUVFbz66qsUFBRQUFCAqpKWlsbKlStZs2YN1dXV3Hbbbd5zSktLeeyxxwDo27cv5eXl9OzZk86dO/PZ4iWUlJRwzuCRfPT1DmprawlxhnLgwAF++7v7ePa1Wdz58LPk5eXhdDq54oorGs33jIyMpHv37pxzzjlUVFRQXV3N3XffzXnnndewEHi1uhdf7wyU813/VjjupPYU8Dnu5DIMeITv+qfa8l0/VRsRqQYyga9F5HnP/pm4myE3AH8Hfur5HNUBvwTmisjnuJsn/w4gIsnAIU+iO7rmqmet6XWmNCH6esHTnPijF79rTtyyZYueddZZjcr9/ve/18LCQr3ooov0+uuvP2IIsGrjfpD8/HwdMGCAxsbG6tChQzUrK0vLyspUtXG/U3Z2tm7YsMHbhKiqOm/ePG8TR4OIiAjt2LGjhoaGav/+/XXixIk6c+ZMzc7O1nnz5unQoUP1tddeU6fTqSKiQ4cO1RdffFErKyv1+uuv17CwME1KStLc3Fzt1auXNy5V1draWg0JCdHw8HAtLCz09g2kp6d773/hhRfqkiVLvD+fAQMG6PTp07Vt27aamZnp7Ue49tpr9cknn9TExERNT0/Xbt266Q9+8APNz89XEdF27dppWlqaArpp0yaNjY3Vtm3bakZGhp599tman5+v3bp102nTpmltba12795dExISvM1HcXFx2rNnT62trdXly5d75+VkZGRoWlqaioguXrxYe/bsqU6nU10ul7c/KCcnR5999lnvOampqZqcnKyADhw4UHNzczUsLExFREVE27Rpo9nZ2dqtWzdvXOPGjdOzzz5bAR0wYIBmZmZ6+04uvfRS7dy5c6M+wC1btnj34ek3zM7OVkDDw8P1zjvvPOKc5557TsPDwxXQ2NhYLSws1C1btmhubq6GhoZ6+3h873HOOefoli1bNDQ0VOPj4/Xjjz/Wr776SgFNS0vz/u4afo9btmzR7t2767Zt29TpdGrbtm117dq1mp2drSKi55xzjs6YMUNV1fvZXb9+vfbo0UPT09N12LBhGhkZqd26ddOQkBBN7ZCprjYR2n/kbRqdlKaFt/9F214yTiO7FmvmPe+oI6qthoSFa59rx6nT1UajMrsqzjCN6TNCM+95R6OzuuvQ307Wh99doxP/tVS/2b5PIyMjNT4+XteuXatbtmxRp9Op3bp10xkzZmiXLl00KytLP/lsoT7557/pldfcoO3TOuifps3ShKQUvfPpaep0hWt8Zp5GdcjTqMIh6oxrr+1HP6ltLxmncYUD9bKnP9abnpihrjYRetMv/p92ys7Vl19+WaOionT8+PFH/ZsRFRWlf/7znxtt4xlGD7wM/Bvo5dlOxV0z6+X52tmTjM7H0z+Fpz+robzna1ZDef2uOfCY/Vm+L+AOfPrDjvayGlgrNsanJnb7K8uoqatvcu3E3/zmN6xYsYI5c+agquTk5BxxLd9njqWnp3PVVVfx0ksvoaqN1k4877zzeP/990lPT6eqqqrJuFwul3f47xdffEFNTQ1z586lffv2fP3110ybNo1LLrmEmpoa9u7dC7hrYFFRUfTr14/MzExKSkpwuVyEhIRQU1PDa6+9xqxZs9i6dSuTJ0/23uupp54iNDSUtm3bep+TFh4e3qgGVlJS4v0ffsPX6Ohoqqqq+N3vfkffvn3Zs2cP+/fvJzY2ltzcXO/93nrrLSZPnkz//v3p168fo0aNIjMzkyeeeILa2loefPBBJk6cyIEDB1i1ahWzZs1i3LhxTJgwgaysLA4ePEhpaSnbtm3j4MGDbN++HYfDQUREBC+++CKrV6/mxRdfpLq6mr59+7J+/Xo2bNjAiy++yIwZM9i3bx+///3vmT17Ng8//DCTJk1i9erVTJo0if379xMREUFNTQ01NTWUlpYyZMgQVJXHHnuM//3f/0VEWL16NbNmzeLll1+msLCQxMREFi5cSF1dHV9++SUul4uZM2cyf/58Zs2axU9/+lPq6uq8TxnIy8ujXbt2bNq0iYULFxIREeGtKTVcu+Gc/v37c/HFF5OcnExiYqJ3Tcz9+/cDEB8ff8Q9unTpwr333ktYWBgRERFs3ryZxMREQkND2bZtm/d3VlJSQkpKCm2TksnvWsBbs+YiIiSmpPHE31/mQGUNIOzYe4B5a/fwxL/WsK1sN0/8eytPL97Lgah0du7Zz1fV8dS5Yqg+5ybUEcb28v1U14ewZMUq9pdtY90r91E5fzL1GxYy+MBc0ttG0eWsHLJCduByhnD94L6EO0MYXVzIby/Nx3monAPOGF6Yv57755Zy0eMfcagWatskcNef3+Lh2euoq69n484K7n/3G7Y5k1m/8Vv6DxzKnXf9kremv8n2nbt45JNdVCXlM/ndz6hXIa5DLo6qClJkD7pvJ/qvR9CFL1G3/jPyS97hxXGX0SXvbMq+XopTlBtuuMHbtHo0TqezUWuG5/FH1QCqeiOQjOfBw6rasBr3t7jna/XguwcVhwOH8HlQcUN5VV3vU75BQ3l/7QGmHKuQJbBWbsz5nfjdZfnMXr2d219ZRtRhayfW1dV5m/9WrVrFqlWrvO3ovmsn+vZpDBs2jNWrVxMdHU1paWmjtRMb+p0cDgd1dXVHJLG6eqVdYhIPPfkc/1y5lVG3301MUhrPLtnHgcpqEhLaEhoayqJFi9i/fz//+Mc/APfq9e3bt0dE6NatG9OmTaOyspLp06cTGhrK+eefz86dOwG8/wBLSkqYOXMmMTEx7Nq1i/LycjZt2kRlZSW7du2ipqaGzz77jNjYWFJSUigvL/fGO3fuXEJCQjj33HMpLS0lLCyM5cuXk5eXx9atW+nduzexsbG0b9+euLg46urqWLt2Lf379wfciyknJyfTo0cPiouLycvLY9GiRaSmphIfH88777xDXl4eSUlJJCQkEBcXh8PhYNeuXdTW1pKbm+v9j8SAAQNISUmhurqajz76iE6dOhEaGsrSpUsJDQ2lc+fOdOrUifz8fO/TsPPz8xER2rRpQ1lZGQMHDiQhIYElS5YAEBoaytVXX01lZSWlpaXeuPbt20dMTAyqyuAhF5PQPh2HwwHABx98QGhoKJ06dWLRokWUl5ezcuVKSktLiYiIIC4ujoSEBEQEh8NBVVUVDoeD999/n7q6OhYtWkT79u1ZsWIFBw8eZPjw4ZSUlBAREUFFRQWqSkVFBbsP1FBPCKN/NJa3PvuKd/81l38vXgGhLmocbbj3D88w7o011Na5m/Gue3E1qd+/jnVbdtD/mRVk3T6Z12bO4f5/l1Oj8OWaL5g882P2JRWgKN9u3Mhr6+C3d4+j2hnJu//ZxkeLVlGyZjmEOGiXkUNl+Xa+f24Pwl1h1O3fRVyki/deeIzOWZ24/Ye38IeHfs/om27kr08/Tn19Pbt37/b+bHv37k1VVRV3/2g0O+f/A5dWMe9/rmT+uHN560e9+ePIQtq0CefA9m/5qjKS+V98iyoc2LWNs/PyufjqW0jPzudPMxYw6md3c/HlI0hObMvTV2SRsG8dsx+6hfjoCA58vYAIRz2PP/gbOnXqxC2jR/HA/fdzww03MGHCBMrKyti5cyefffYZI0eOZMOGDVRWVjJixAimT5/Ovffee8TfizZt2vDWW29RVVXFhg0bGvoio8TN2z8lIvGePi+ABOA83E2IM4BRuPunXE2VF5F2PuUbHLM/y5eqTlJ3s+YxC54WrzOxCdHXxPnu5sQxkxbp0BHX6z9nuptoDh06pHl5eZqXl6fnnnuuLl++3HvO0KFD9dNPP9WqqipNTk727j906JCOGDFCU1NTNTY2VtetW6eq7uaeIUMu1m+2V2hyarq2bZ+mbaJiNMTh1DZxiRreNk1D2sQojlDFEaohUQnqiG2vzuh2Gp7UUXGGKSEOjYxN0K49emlKSopGRERoeHi4JiQk6PDhw3XAgAFaVFSkycnJ6nQ6FfAOU3799dc1MjJS77zzTh06dKgWFhZqUlKShoSEqNPp1OjoaE1KStJevXppenq6pqWlaXx8vE6ZMkVVVR966CFvM6WIaFJSkvd779evnzocDs3NzdV3331XVVXffPNNdblc2q1bN42IiNCf/OQnqqqakZGh4eHhmpKS4h36P2bMGH399dd14cKFGh0drYsWLdIf//jHmpubqzfffLMmJiZqSEiIdu7c2dsk1mDhwoV69tlna//+/XXkyJF6/vnna25uriYkJGhISIi+8sorje6hqnrBBRdox44dNSkpSXNycvTBBx/UqqoqdTqdGh4e7h35eeGFF+q8+Qt08vR/aZgrXH85cY5GJ2eohDg0qvvFmnnPOyrhUe4mv/Ov19Trfq8h4VEaGpukbZI7aURKtnb/+XMaFt1WHa4IDY9NVEQUERVnqPa/+3nN6HuZhoSGaZv4ZI1OzdLY9ByN7ZCreZf/REOcoSoO9+8xNC5FXRmFGj/wJwrivk9iR3cTXUSsgiig4nCfE+IMVWd4pEbGJ6krIlolJERTOp2l7VIz1NUmQh2e64a5XCoimpqeoUV9+mpkZKR2yursHbpfUFCg8fHxGhISooA6nU6NioryDqFPS0vTyMhI7dq1q2ZmZur48eP117/+tUZERHibWBuG3AONhuq3b99eMzIydMuWLfrJJ580GtLucrk0Ojpac3NzdcCAAd5h9FFRUd7P2MCBA1VEvNdtaNqMi4vTjIwMDQsL065du+rYsWP1oYce0l//+tfe76lr1676/PPPa3x8vKanp2vnzp01JydHVVUfffRRzc3N9Y4gfeqppzQtLU0dDodGRUV545o5c6YCO3CvrlEBDFF3E94TQA3ugRw1wL89+wWYCFTi7hcr8uzv69le6fnqOxw+GVikAfi73+KJ52S9zvQEpqo6af56zbp3pqbc/JRGdblAL/3Tx3rfjP/oOyu36ra9h44o37B2YlP9ZqqqH3zwgRYPGqIzV23Vh2Z+odf89VPN/+177n6BmCRNuvZ/NarLBXrehLk68rlPddy05fqH99boSws26gdrtuuXpRX6wP8+rA8++KDW1tXr36fP1XYZuZp40Q808553NLnnRfrwm5/p1j0H9corr9Qvv/yy0RD3LVu2aFxcnL700kuqqvrLX/5Sr7jiCr3vvvu0qKjIm1Aa+tFUVYcMGaJTp07VpKQk73JWDfPTGsqoqj7yyCN6yy23aNeuXTUsLExTU1O9Q65VVbdu3aq5ubm6YMECVVVvXJMnT1aHw6Eioj169PCWHzNmjP7973/XtLQ07d27t3euUkZGhqqqbtq0ScPDwzU2Nlbbt2/v7QfaunWr5uTkaF5enkZGRmp0dLR3OaEbb7xRRUQvueQS7z3eeOMNnTJlisbGxuqCBQs0OTlZc3Jy9IEHHtD/fLNBQ0JCNCahnXYvvlJv+PtnGpPVXRNH3q8hEbEa1eMSzfnNuxqR1EGjk9I0PilVO+UVaPuMLHe/2Ihb9A/vrdGeA4fryHue0JG/flq7Dxqp46Yt14i4dhrZtr3ePHGhjp64UGNTO6kjLFwvHPuYZvUZop3OHaTf/+nDeuHYP+pZF1ylyWf11D433aPpBefprZMXaU6fgZqR10N7XXiZjnvk79o2OUV/+/hf9U8TX9Ubb/mBlh+o0vfnftDod7Rs2TLt2LGjzpgxQ2+44QbdsWPHMf8N3HnnndqnTx9VVb/OWbZsmZ511ln6/vvv+33O448/rs8//7zf5Q+Pyx/Lli3TXr16Hdd9TiQuVVVgiR5P0vCzf+pEyx/Pq9llOkzrcvN5nbiyZzrLNhXxl7/uR53w6qJNTPpkIwAdEtpwTmYCvTrGc07HBN57zz2cvqHfbPeBalaW7GHV5r2sLNnDJx8vZfO6XayduoxQh5CfEsPwnukUpMfyi6ku3n9oFJ/OTeaWWwZ4m6EOl90pkw8//BBHiPCDKy7kpSfac9P15xPdtRv/1/EBnltYxnMLZpMa140vDkZSr98N1294nljDqMlHH32UwYMH065dO8rLy5kxYwbvvvsulZWVVFRUMHLkSBYtWsTgwYNp3749UVFRANx9990cPHiQv/71r95rp6amcv/99zNv3jxGjhyJqnofHFpRUcHQoUP5/e9/T58+fbzn7N27l/vvv5/27duze/duNmzYQHl5+f9v797jo6quBY7/VmYSkvBKYgLEBAnJh2cCIYXgCzqwQSIAABIwSURBVJsQLCBQfIIVidimUmkvpf0ghau31raI9QrS+qhcLVYRUClIG2lVFEVUEOQpoEW0INLwCASUZ4DMun+ck+kQEglCGGZmfT+ffDJnzzlz1prXnn32OXuTmJjI1q1beffdd8nPz2fp0qV4vV48Hg/bt29n+PDhPProo6gqBw4cYPbs2XTq1ImioiISExOZOHEiAwYMYNCgQXTq1ImXX36Zvy5cTFxSKxDhvfdX8NyyrSxd9wkn2l7O3MkTOHFC6TPweg7vK2dX+R5+O3Ua07a3wufzceBwJV8cjeGSo8eJOrSX6JWz6NyxAxW7PuTYzFEcq9hBVFQUmZ06sXbtWn7wgx/wzDNb+O5l2Yzt35FFUw/y82svpbS0lKUblzJ//HVUHtiHqnJ80SPMnDmTnk8k8+mR/fxiQA79/jCWfv36cd/wAkpLS3nuk/fZv2cPn5Rv5ciRIxx9/Q98f3BvVq5ciaoyMC+Dj3K7cM1lXSgtLeWtha/QrXN7/+s4fPhwZs6cSV5eHklJScTExDBz5sx6fQZycnLo2LEjVVVV9domLy+PSy+9lMLCQoB6bZOQkEBxcXG9168ZV12flZpxde3atXqopwaL6xvaDzzXgOvXX0PUisH4sxZY7SqPV+mabfv0qSWf6Y9mrNTuv12obcYv0DbjF2iXX72qtz+9XH88a5XGNE/RS8a+pG3GL9CMCQu0z5TFOuTepzT3iiJdu22fHj1+8oXENS/wXb58uRYXF5+y/71792pGRoZWVFRoRUWFZmRk6N69e/33b91zUKcs3KRX/m6Rthm/QOPbdNHhD8zSZZ/t0aoqn7Zs2VJzcnL8F4S2bdtW33jjjZN+qVe3wJ544gm97bbb9PLLL9f27dtrWVmZP67AVpqq6qZNm9Tr9WpFRYX/jL6rr75aKysrtaioSKdOnXpSHgUFBTpx4kQdOXKktmnTRlu0aOE/PDlnzhz1eDw6ZcqUk7bp3LmzxsTEaEVFhZbt2u0/3PX84nX60/sfV29MI80feKv+aMZKvfGP72nzzFxNHTxW8URry2EPavJN9zkjIUTH6sV3PKlR8YnqaZyoGQN/rNf8fone8uQyjWuepK2ze2hiyzQd8pO7NTraGQVi/fr1umTJEo2Pj/dfAFytQ4cO2r9/f83IyNAdO3Zo9+7d/WdWzp8/X5s0aXLKhePZ2dnaqFEj/+uYlpamjRo10i1btvhfl8BtCgoKdNq0af7nfMOGDf6R2d9///1T1g98HQPl5ORoWVnZKe8rc+5whi2wC+kv6AGcqz+rwOrH5/PplvKDOueDbTp+7jotmvyWXvHAIu347cH608nP6nuflutXR45pr169NDk5WWNjYzUtLc1/2nPgsfTU1FQtKSlRVad/auTIkbXuc/r06ZqVlaVZWVn69NNP+8tLSkr8hwvnzp2nKa1S1eONVk/jBI3NyNMrHlik3/rODfrdIbdoZmamtm/fXrOzs0+Jq/qLr6CgQEtLSzUlJUXz8/NPiqvml6PP59OkpCRNS0vTpk2bal5enubk5Ohzzz2nIqLt2rXT3NxczcjI0JSUFP+p8B6PRz0ej8bFxWlSUpImJCZp02bNVUS0TbvOmpbVUVu17aA3T3pem6dlaZQ3RiXKq0R5FU+0Snxz5xBs81Zuv1BLjU1spRLl8Q8pFOXxqNcbrd7oaI1v3Fi90dGa3voSTUlJURHR3NxcTUtLO2kopauuukpjY2M1NjZW4+LiNDMzU9PT01VE/KezV+fi9XrV6/X6+3WaN2+uF198sWZmZmpqaqqmp6f7X5eXXnrJv6/qvLOysnTq1KmalpammZmZ2qxZM/+lGdXrx8TEaGJioiYnJ/uf8wkTJmijRo1O6mcMVPM12rlzp/91NA0nlCswceIPfT169NDqs4TMmVuzZg0PP/yw/2LQMzVu3DiKi4vp2rXrWcdy+NgJXtu4k5fX7eCNd5ZTsfwlsofdTb/sVvTPaUXPjCS8ntpPoC0rKyMvL49JkyZRUlLij6uiooLJkyezYMECAMoPVPLCgjf4wwP3cfDL/SCCeDzc8fBfOF7l49gJ5XiVz/93rEr58O8zOH78GK2LhrPtk4/4dN6DNMkuourQfhpn9yamRVsAGsd4SGnaiI1PjaWg+Od07ppHSpNGTB51HScqD/PKe2v49dhRxEQJN910IzfeeKM//v3793P99dfz6KOPkpOTw5o1a7jhhht45JFHWLJkyWmf47Fjx7J06VKWLVtW79dlzJgxDB48mD59+tT7dZw6dSrNmjWjpKTka9c7m22+yT7MmRORVaraI9hxfBPWB2YA55h77969632Mvqa6RuP+JuJjvFyfl871eel89b1u3P1AJZVpTZmz8gtmLPucixrH0De7Jf1zUrki6yKiAyqzuLg4qqqq/ENZPfTQQxw4epwl8/7B53sPM2rmKj7c/iX/3u9ekjLgPuIE9r85naoD5Sz4cAfRHiHaE0WMJ4poTxRedzk2IYVDn64hPTGevKuvZM67KXzn21247qabSWnaiJQmsSQ3jSE+xvlYFf69GROu6USPHjkAvN21Ez6fj+zUpsx78Xn69et3yigXCQkJFBYW8uqrr5KTk3NSP1B9nuOafS313aZPnz7+56s+Avtb6utMt/km+zCRxVpgJmQcPnaCxZvKeWXDTt78eBeHjlXRLNbLdzq34pqcVvRql4wItLnkEibOfpOPdx9l3fb9fFZ+kCOff8hXK+bTY+SD5LZOIDe9Oa3jjnNV1yw8WsWAAQO45557KCoqYsWKFTz22GPMmDHjpP1XVFTQvXt3Vq9eDThDSa1atcp/kXdNhYWFTJ48mR49nB+3GzduZNiwYaxYsYKysjL69OnD5s2bqaioIDo62j82Xd++fRk/fjyDBg0CoEuXLixcuLDWOaCMOVvWAjPmPIiP8TKgSyoDuqRy9HgV72zewysbdvD6RzuZt3o78TEeTlQpB5OzuWfaXFrn9CQ3PYEtfx7L0X9vRY8c4l+PFHPP9On0uyqPcePG8bPiBfh8PkaNGkVRUREA27Zt858BGSgpKYlf/vKX5OfnA3Dvvff6K68f/vCH3HnnnfTo0YP58+czevRoysvLGThwIN26deO1114jOzuboUOH0rlzZ7xeL48//jgej4cdO3YwYsQIqqqq8Pl8DB061F957dq1i7i4OKu8jKmFtcBMyDt2wsfSz/aw6OPdxMd4aHLwCxbPe4a5L8w67Sy4tTmX/Xlny/qBTEOzFpgxQRTjjaKwQwsKO7RwSzrR+OB2fD5f0Pvzzpb1AxlTN2uBGWNMBAvlFpgN5muMMSYkWQVmjDEmJFkFZowxJiRZBWaMMSYkWQVmjDEmJFkFZowxJiRZBWaMMSYkhc11YCJSDnx+mtWSgT3nIZwLVSTnH8m5Q2Tnb7l/vTaqmnI+gjnXwqYCqw8RWRmqF+ydC5GcfyTnDpGdv+UevrnbIURjjDEhySowY4wxISnSKrAngx1AkEVy/pGcO0R2/pZ7mIqoPjBjjDHhI9JaYMYYY8JExFRgItJfRDaJyKciMiHY8TQ0EXlaRHaLyIaAsiQReV1ENrv/E4MZY0MRkdYi8paIfCwiG0VkjFse9vmLSKyIrBCRdW7uv3bL24rIcjf3F0UkJtixNhQR8YjIGhFZ4C5HUu5bRWS9iKwVkZVuWdi+7yOiAhMRD/A4cA3QGbhFRDoHN6oG9wzQv0bZBGCRqrYDFrnL4egEMFZVOwGXAT9xX+9IyL8SKFLVXKAb0F9ELgMeBKa6ue8DwnmK5zHAxwHLkZQ7QG9V7RZw+nzYvu8jogIDegKfquq/VPUY8AJwbZBjalCqugSoqFF8LfCse/tZ4LrzGtR5oqo7VHW1e/sAzpdZGhGQvzoOuovR7p8CRcBctzwscwcQkXRgIPAnd1mIkNy/Rti+7yOlAksDvghY3u6WRZqWqroDnC95oEWQ42lwIpIB5AHLiZD83UNoa4HdwOvAZ8B+VT3hrhLO7//fA78AfO7yRURO7uD8WFkoIqtEZKRbFrbve2+wAzhPpJYyO/0yzIlIE2Ae8DNV/cr5MR7+VLUK6CYiCcB8oFNtq53fqBqeiAwCdqvqKhEprC6uZdWwyz3AlapaJiItgNdF5J/BDqghRUoLbDvQOmA5HSgLUizBtEtEUgHc/7uDHE+DEZFonMprlqq+5BZHTP4AqrofWIzTD5ggItU/WMP1/X8lMFhEtuJ0ExThtMgiIXcAVLXM/b8b58dLT8L4fR8pFdgHQDv3bKQY4HtAaZBjCoZSYIR7ewTwtyDG0mDcfo/pwMeq+nDAXWGfv4ikuC0vRCQOuBqnD/At4CZ3tbDMXVX/W1XTVTUD5zP+pqreSgTkDiAijUWkafVtoC+wgTB+30fMhcwiMgDn15gHeFpV7w9ySA1KRJ4HCnFGo94F/Ar4KzAHuATYBgxR1ZoneoQ8EekFvAOs5z99IXfj9IOFdf4i0hWno96D8wN1jqr+RkQycVolScAaYLiqVgYv0oblHkK8S1UHRUrubp7z3UUvMFtV7xeRiwjT933EVGDGGGPCS6QcQjTGGBNmrAIzxhgTkqwCM8YYE5KsAjPGGBOSrAIzxhgTkqwCM0EnIioiUwKW7xKR+87RYz8jIjedfs2z3s8Qd/T7t2qUZ7j5jQ4oe0xEbj/N490pIredZp3bReSxOu47WFv5ueLmFTjTwR0isjqcRjo3Fz6rwMyFoBK4QUSSgx1IIHcWg/oqAX6sqr1ruW83MOZMpvFQ1WmqOuMM9n/OBIxaUd/1i4HRQF9V3dcwURlzKqvAzIXgBM7U5z+veUfNFlR1y0JECkXkbRGZIyKfiMjvRORWdy6s9SKSFfAwV4vIO+56g9ztPSLykIh8ICIfisiPAh73LRGZjXMhdM14bnEff4OIPOiW3Qv0AqaJyEO15FeOM43FiJp3iEiWiLzqDr76joh0dMvvE5G73Nv5bozL3Jg3BDzExe72m0Xkf2s89hS3VbRIRFLcsm4i8r77ePOrW0wislhEJonI2ziV7RA3x3UisqSWnKr3MRRneo6+qrqnrvWMaQhWgZkLxePArSLS/Ay2ycWZ+6kLUAy0V9WeOFNpjA5YLwMowJlmY5qIxOK0mL5U1XwgH7hDRNq66/cE7lHVk+aME5GLceaWKsKZaytfRK5T1d8AK4FbVXVcHbH+DhhbS6vuSWC0qnYH7gL+WMu2fwbuVNXLgaoa93UDbnafg5tFpHrMz8bAalX9FvA2zkgsADOA8araFaeC/lXAYyWoaoGqTgHuBfq584oNriOnNsBjOJXXzjrWMabBWAVmLgiq+hXOl+tPz2CzD9y5vypxpgxZ6Javx6m0qs1RVZ+qbgb+BXTEGSfuNnGmHVmOM+1GO3f9Faq6pZb95QOLVbXcnZ5jFvDteua3BVgBDKsuE2e0/CuAv7hx/B+QGridO65hU1Vd6hbNrvHQi1T1S1U9CnyEU6mAM4TWi+7tmUAv98dBgqq+7ZY/WyP+FwNuvwc8IyJ34AxLVZtynKGJhtaZuDENKFKmUzGh4ffAapwWR7UTuD+03EF6A/uRAsez8wUs+zj5vV1zvDTFmWZjtKq+FniHO4beoTriO9v5WCbhTKxYfUguCmeuqm5fs83p9hn4HFRR92e6PmPG+fNW1TtF5FKcVutaEemmqntrrH8YZ5bzd0Vkt6rOqsc+jDlnrAVmLhjuAKNzOHnK961Ad/f2tTgzDJ+pISIS5faLZQKbgNeAUeJMu4KItHdH8P46y4ECEUl2DwXegnN4rl5U9Z84raRB7vJXwBYRGeLGICKSW2ObfcABEbnMLfpePXcXxX9GYB8GvKuqXwL7ROQqt7y4rvhFJEtVl6vqvcAeTp6OKDC+cqA/MElE+tUzNmPOCWuBmQvNFOC/ApafAv4mIitwToSoq3X0dTbhfFG3xOlLOioif8I5zLjabdmVc5qp1lV1h4j8N870HAL8Q1XPdGqK+3FGRK92K/CEiPwPTuX8ArCuxjYlwFMicghnfq8v67GfQ0C2iKxy17/ZLR+B0w8Yj3M49ft1bP+QiLTDyXNRLTH5qeoWERkM/ENEblDV5fWIz5izZqPRG3OBE5Emqlp99uUEIFVVxwQ5LGOCzlpgxlz4BrotPy/wOXB7cMMx5sJgLTBjjDEhyU7iMMYYE5KsAjPGGBOSrAIzxhgTkqwCM8YYE5KsAjPGGBOSrAIzxhgTkv4fiafMJ+PZpLYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.151 0.119 0.11  0.108 0.105 0.104 0.105 0.105 0.104 0.103 0.104 0.104\n",
      " 0.104 0.104 0.104 0.105 0.105 0.105 0.105 0.105 0.105 0.106 0.105 0.105\n",
      " 0.106]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_kd = KNeighborsClassifier(n_neighbors=k,algorithm='kd_tree')\n",
    "    scores_kd = cval_score(knn_kd, d_train,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores_kd.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.143\n"
     ]
    }
   ],
   "source": [
    "knn_kd_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_kd_optimal.fit(d_train,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_kd_optimal.predict(d_test)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.996\n",
      "The value of TNR is 0.060\n",
      "The value of FPR is 0.940\n",
      "The value of FNR is 0.004\n",
      "\n",
      "Test error is 10.783\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGV9JREFUeJzt3XucVWW9x/HPd0BSLgqaIA6kpHit4yU1jprHW6h4IVM7WioqnjkZeMm0UDMTtUPnHC2poy8xKbyBmJHk6aiIWloikJDglYsXRhAM8AaIzp7f+WMvxg3OntnIDHvPw/fda71m72c9a61n4/Tbv/mtZ62liMDMzNq+qnIPwMzMWoYDuplZIhzQzcwS4YBuZpYIB3Qzs0Q4oJuZJcIB3cwsEQ7oZmaJcEA3M0tE+3IPoJj2Hap9Cat9wm7depd7CFaBZi+eog3dx0f/mF9yzNnss5/f4OO1BmfoZmaJqNgM3cxso6rPlXsEG8wB3cwMIFdX7hFsMAd0MzMgor7cQ9hgDuhmZgD1DuhmZmlwhm5mlgifFDUzS4QzdDOzNEQCs1x8YZGZGeRPipa6NEPSq5JmSZopaXrWtrWkSZLmZD+7Ze2SNFLSXEnPStq3YD+Dsv5zJA1q7rgO6GZmkC+5lLqU5rCI2Dsi9sveDwMmR0RfYHL2HuAYoG+21AA3Q/4LALgK+DJwAHDVmi+BYhzQzcwgf1K01OXTGQiMyV6PAb5W0H575E0BukrqCRwFTIqIZRGxHJgEHN3UARzQzcxgvTJ0STWSphcsNevuDXhY0t8K1vWIiEUA2c/uWXs1sKBg29qsrVh7UT4pamYG63Xpf0SMAkY10eWgiFgoqTswSdKLTfRt7M6N0UR7Uc7QzcygRU+KRsTC7OcSYAL5GvjirJRC9nNJ1r0WKLwvdC9gYRPtRTmgm5kBEbmSl6ZI6iSpy5rXQH9gNjARWDNTZRBwf/Z6InBmNtulH/BOVpJ5COgvqVt2MrR/1laUSy5mZtCSFxb1ACZIgnyMvTsiHpQ0DRgvaTDwOnBK1v+PwABgLrASOBsgIpZJugaYlvUbHhHLmjqwA7qZGbTYzbkiYj6wVyPtS4EjGmkPYEiRfY0GRpd6bAd0MzPwpf9mZsnIfVTuEWwwB3QzM/D90M3MkuGSi5lZIpyhm5klwgHdzCwN4ZOiZmaJcA3dzCwRLrmYmSXCGbqZWSKcoZuZJcIZuplZIupKf8BFpXJANzMDZ+hmZslwDd3MLBHO0M3MEuEM3cwsEc7QzcwS4VkuZmaJiCj3CDaYA7qZGbiGbmaWDAd0M7NE+KSomVkicrlyj2CDOaCbmYFLLmZmyXBANzNLhGvoZmZpiHrPQzczS4NLLmZmifAsFzOzRCSQoVeVewD2SVVVVUyb+hD3TxgDwI479uavT/6BF557krvvupnNNtuszCO0jaHLlp254Vc/YeKT45j4xDj22u8LfO9HQ5n45Dh+99id3PjrEXTZsvNa22xX3YOp8x/lrPO+WaZRt2H19aUvFcoBvQJdcP65vPjinIb3//GTK/j5yFvZfc+DWb78Hc45+7Qyjs42lmHXfpe/PDaFEw4+la8ffjrzX36Vp/40lRP/5Vt8/bDTeXXeAs69YNBa2/xg+EU8MfmpMo24jYsofSmBpHaSZkh6IHvfR9LTkuZIukdSh6z9M9n7udn6HQv2cVnW/pKko5o7ZqsFdEm7SfqBpJGSbsxe795ax0tFdXVPBhxzBKNHj21oO+zQg7jvvv8F4I477mXgCc3+d7U2rlPnjnzpn/fhvrsmAlD3UR3vvfs+f/3TVHJZrffZv82mx/bdG7Y5/JhDqH3tDea99EpZxtzmtXyGfiHwQsH7nwI/i4i+wHJgcNY+GFgeETsDP8v6IWkP4FRgT+Bo4CZJ7Zo6YKsEdEk/AMYBAqYC07LXYyUNa41jpuKG669m2GXXUp/90myzTTfefvudhv8T176xiO2rtyvnEG0j6LVDNcuXLufaG6/k3kfGcPUNl7NFx83X6nPiN4/nySwb36Lj5pwz9Axu+u/byjHcNNRH6UszJPUCjgV+lb0XcDjw26zLGOBr2euB2Xuy9Udk/QcC4yJidUS8AswFDmjquK2VoQ8G9o+IERFxZ7aMyAYzuJltN1nHDjiSJUv+wTMzZjW05f+7ri0SuG+zNa19+3bs/sVduWfM7zjlyEGsWrmKweef2bC+5qKzyNXV8cB9DwIw5NJ/445bxrFq5apyDbnty+VKXiTVSJpesNSss7efA98H1qTz2wBvR8Sap2jUAtXZ62pgAUC2/p2sf0N7I9s0qrVmudQD2wOvrdPek48/4Cdk/yg1AGq3FVVVnVppeJXpwAP34/jj+nPM0Yez+eafYcstu3DD9VfTtetWtGvXjlwuR6/qnixauLjcQ7VW9ubCJSxe+BaznnkOgIf/8CjnZgH9hG8M4JCvHsS5Jw9t6P/Ffffkq8cdzsVXDqXLVp2J+npWr/6QsaN/2+j+7ZNiPU52RsQoYFRj6yQdByyJiL9JOnRNc2O7aWZdU9s0qrUC+kXAZElz+Pgb5nPAzsDQYhsV/iO171C9yaWhV/xwBFf8cAQA/3LIP3Pxd7/NmYPOZ9zYWzjppGMZP34iZ5xxChP/8HCZR2qtbelby3hz4WJ23OlzvDrvdfp9ZX/mvfwKBx3Wj8FDz+CsE8/jg1WrG/oPGvjthtffueRcVq5Y6WC+vlruStGDgBMkDQA2B7Ykn7F3ldQ+y8J7AQuz/rVAb6BWUntgK2BZQfsahds0qlUCekQ8KGkX8iWWavLfNLXAtIho+7P3N7LLLr+Ou++8ieE//j4z//4co389tvmNrM37yeXX89ObrmazDpux4LU3uPLCaxn30Gg6dOjAreNHAvkTo8O//59lHmkiWuheLhFxGXAZQJahXxIR35J0L3Ay+fOLg4D7s00mZu+fytY/GhEhaSJwt6QbyFc8+pI/J1mUKrUeuylm6Na83br1br6TbXJmL57SWHlivawY/q2SY06nH91V0vEKAvpxkj5PPphvDcwATo+I1ZI2B+4A9iGfmZ8aEfOz7a8AzgHqgIsi4v+aOp6vFDUzA6hr+eJBRDwOPJ69nk8js1Qi4gPglCLbXwdcV+rxHNDNzMC3zzUzS4Zvn2tmlob1mbZYqRzQzczAGbqZWTIc0M3MEuEHXJiZpcHPFDUzS4UDuplZIjzLxcwsEc7QzcwS4YBuZpaGyLnkYmaWBmfoZmZp8LRFM7NUOKCbmSWi7ZfQHdDNzACiru1HdAd0MzNwhm5mlgqfFDUzS4UzdDOzNDhDNzNLhTN0M7M0RF25R7DhHNDNzIBwhm5mlggHdDOzNDhDNzNLhAO6mVkiIqdyD2GDOaCbmeEM3cwsGVHvDN3MLAnO0M3MEhHR9jP0qnIPwMysEkR96UtTJG0uaaqkv0t6TtLVWXsfSU9LmiPpHkkdsvbPZO/nZut3LNjXZVn7S5KOau4zOKCbmQH1OZW8NGM1cHhE7AXsDRwtqR/wU+BnEdEXWA4MzvoPBpZHxM7Az7J+SNoDOBXYEzgauElSu6YO7IBuZkb+pGipS5P7yXs/e7tZtgRwOPDbrH0M8LXs9cDsPdn6IyQpax8XEasj4hVgLnBAU8d2QDczY/0CuqQaSdMLlprCfUlqJ2kmsASYBMwD3o5ouAVYLVCdva4GFgBk698Btilsb2SbRvmkqJkZEOtxO/SIGAWMamJ9DthbUldgArB7Y92yn42l/NFEe1EO6GZmtM489Ih4W9LjQD+gq6T2WRbeC1iYdasFegO1ktoDWwHLCtrXKNymUS65mJmRn7ZY6tIUSdtmmTmStgCOBF4AHgNOzroNAu7PXk/M3pOtfzQiIms/NZsF0wfoC0xt6tglZ+iSPhMRq0vtb2bWluRa7l4uPYEx2YyUKmB8RDwg6XlgnKRrgRnAbVn/24A7JM0ln5mfChARz0kaDzwP1AFDslJOUc0GdEkHZAfcCvicpL2AcyPi/E/xQc3MKlJLXVgUEc8C+zTSPp9GZqlExAfAKUX2dR1wXanHLqXkMhI4DliaHeDvwGGlHsDMrC1oqWmL5VRKyaUqIl7LT4ts0GTab2bW1qzPLJdKVUpAX5CVXSKrCZ0PvNy6wzIz27gqOfMuVSkB/TzyZZfPAYuBR7I2M7Nk5Orb/qS/ZgN6RCwhO+tqZpaqTaLkIulWGrk6KSJqGuluZtYm1Sdw+9xSSi6PFLzeHDiRte8vYGbW5qVwP/RSSi73FL6XdAf5m82YmSVjkyi5NKIPsENLD8SsFDOeu7vcQ7BEbRIlF0nL+biGXkX+0tRhrTkoM7ONLflZLtlN1vcC3sia6rObxpiZJSWFwNbkV1IWvCdERC5bUvjMZmafUB8qealUpfyNMVXSvq0+EjOzMmqp2+eWU9GSS8GN2A8G/k3SPGAF+adoREQ4yJtZMurLPYAW0FQNfSqwLx8/yNTMLFnR6BPf2pamAroAImLeRhqLmVnZ1FVwKaVUTQX0bSVdXGxlRNzQCuMxMyuL1DP0dkBnGn/ytJlZUlKvoS+KiOEbbSRmZmWUeobe9j+dmVmJUs/Qj9hoozAzK7NcAjls0YAeEcs25kDMzMopgSfQfaq7LZqZJac+5QzdzGxTksKNqhzQzcxI/6Somdkmo14uuZiZJSFX7gG0AAd0MzM8y8XMLBme5WJmlgjPcjEzS4RLLmZmiUhh2mIpzxQ1M0teTqUvTZHUW9Jjkl6Q9JykC7P2rSVNkjQn+9kta5ekkZLmSnq28BnOkgZl/edIGtTcZ3BANzMjn6GXujSjDvheROwO9AOGSNoDGAZMjoi+wOTsPcAxQN9sqQFuhvwXAHAV8GXgAOCqNV8CxTigm5nRcgE9IhZFxDPZ6/eAF4BqYCAwJus2ho+f1zwQuD3ypgBdJfUEjgImRcSyiFgOTAKOburYrqGbmQGt8UhRSTsC+wBPAz0iYhHkg76k7lm3amBBwWa1WVux9qKcoZuZsX4ZuqQaSdMLlpp19yepM3AfcFFEvNvEoRv7Kokm2otyhm5mxvpd+h8Ro4BRxdZL2ox8ML8rIn6XNS+W1DPLznsCS7L2WqB3wea9gIVZ+6HrtD/e1LicoZuZkZ+HXurSFEkCbgNeiIgbClZNBNbMVBkE3F/QfmY226Uf8E5WmnkI6C+pW3YytH/WVpQzdDMzWnQe+kHAGcAsSTOztsuBEcB4SYOB14FTsnV/BAYAc4GVwNmQf2qcpGuAaVm/4c09Sc4B3cyMlgvoEfEkjde/oZFnNUdEAEOK7Gs0MLrUYzugm5nhe7mYmSXD93IxM0uEH3BhZpaI+gSKLg7oZmakcbdFB3QzM3xS1MwsGc7QzcwSUae2n6M7oJuZ4ZKLmVkyXHIxM0uEpy2amSWi7YdzB3QzM8AlFzOzZOQSyNEd0M3McIZuZpaMcIZuZpYGZ+jW4nr12p7fjL6RHtttS319Pb/61V384pe3cfWPL+X44/tTXx+8teQfnHPud1m0aHG5h2strP9Jg+jUsSNVVVW0a9eO8aNH8otRt/Pok09RpSq27rYV113xPbpvuw3zX1vAldfdwPMvz+WCmkGc/c2TAVi9+kMGDbmUDz/6iFxdjq8edjBDzz2jzJ+s8qUwbVH5px9VnvYdqitzYK1su+2603O77syYOZvOnTsx9ekHOenkc6itXcR7770PwNAh57D77rswZOiwMo9241u18IlyD6FV9T9pEPfcNpJuXbdqaHt/xQo6d+oEwJ333s+8V17nqu+fz9Llb7PwzcU8+uen2LJL54aAHhGsWvUBHTtuwUd1dZx53iUMu/Df2esLu5flM20Mm3328xv8eIrzdvxGyTHn5lfHV+TjMKrKPQBb25tvLmHGzNkAvP/+Cl58cQ7V22/XEMwBOnXqSKV+EVvLWxPMAVat+gBloWSbbl354u670r792n9oS6Jjxy0AqKuro66uDqki409FqSNKXirVRi+5SDo7In69sY/bFu2wQy/23usLPD11BgDXDP8Bp3/rZN55912O/OopzWxtbZEkar57BZI4ZeAxnDJwAAA33vIbJj44mS6dOjH6FyOa3U8ul+Mb51zA628s5LSvH8c/7blbaw+9zUvhpGg5MvSri62QVCNpuqTp9fUrNuaYKk6nTh0Zf8+tXHzJVQ3Z+ZU/+il9dtqfsWMnMOQ7Z5d5hNYa7rj5eu799S+5+fprGPu7B5g+cxYAF/77WUyecAfH9j+Mu+/7Q7P7adeuHfeN+R8mT7iDWc+/zJz5r7byyNu++vVYKlWrBHRJzxZZZgE9im0XEaMiYr+I2K+qqlOxbslr3749995zK2PHTuD3v/+/T6wfO24CJ544oAwjs9bWfdttgHw55YhDDmTW8y+ttf7Y/ofyyON/KXl/W3bpzP77/hNPTpneouNMUazH/ypVa2XoPYAzgeMbWZa20jGTceuo63nhxbn8/MZRDW0779yn4fXxx/XnpZfmlWNo1opWrvqAFStWNrz+69Rn6Pv5HXltwRsNfR57Ygp9dujV5H6WLX+bd7O/6j5YvZop02bQZ4ferTfwRKSQobdWDf0BoHNEzFx3haTHW+mYSTjowP054/STeXbW80yf9jAAV145grPPPpVddtmJ+vp6Xn/9Db4zZNOb4ZK6pcuWc+Hl1wCQq8sxoP+hHNxvPy66/Fpefb0WVYntt+vOjy49H4B/LF3Gvw6+gPdXrKSqqoo7x/+e+++6hbeWLueKa/+bXH09UR8cdfhXOPSgL5fzo7UJuQQmGnjaorUpqU9btE+nJaYtfnOHE0uOOXe/NqEipw35wiIzM9KY5eKAbmZGZdfGS+WAbmZGGpf+O6CbmeGSi5lZMlKY5eKAbmZGGiUX35zLzIyWvbBI0mhJSyTNLmjbWtIkSXOyn92ydkkaKWludkX9vgXbDMr6z5E0qLnjOqCbmdHil/7/Bjh6nbZhwOSI6AtMzt4DHAP0zZYa4GbIfwEAVwFfBg4ArlrzJVCMA7qZGfmSS6lLcyLiz8CydZoHAmOy12OArxW03x55U4CuknoCRwGTImJZRCwHJvHJL4m1uIZuZgYb4xkDPSJiUXasRZK6Z+3VwIKCfrVZW7H2ohzQzcyA3HqcFJVUQ748ssaoiBhVrH9zu2ukLZpoL8oB3cyM9ZvlkgXv9Q3giyX1zLLznsCSrL0WKLwdZi9gYdZ+6Drtjzd1ANfQzczIl1xKXT6licCamSqDgPsL2s/MZrv0A97JSjMPAf0ldctOhvbP2opyhm5mRsvOQ5c0lnx2/VlJteRnq4wAxksaDLwOrHmO5B+BAcBcYCVwNkBELJN0DTAt6zc8ItY90boWB3QzM1r20v+IOK3IqiMa6RvAkCL7GQ2MLvW4DuhmZvjSfzOzZKRw6b8DupkZDuhmZsmo1Mdxrg8HdDMznKGbmSXDD7gwM0tELtr+U0Ud0M3McA3dzCwZrqGbmSXCNXQzs0TUu+RiZpYGZ+hmZonwLBczs0S45GJmlgiXXMzMEuEM3cwsEc7QzcwSkYtcuYewwRzQzczwpf9mZsnwpf9mZolwhm5mlgjPcjEzS4RnuZiZJcKX/puZJcI1dDOzRLiGbmaWCGfoZmaJ8Dx0M7NEOEM3M0uEZ7mYmSXCJ0XNzBLhkouZWSJ8paiZWSKcoZuZJSKFGrpS+FZKnaSaiBhV7nFYZfHvha2rqtwDsJLUlHsAVpH8e2FrcUA3M0uEA7qZWSIc0NsG10mtMf69sLX4pKiZWSKcoZuZJcIBvcJJOlrSS5LmShpW7vFY+UkaLWmJpNnlHotVFgf0CiapHfA/wDHAHsBpkvYo76isAvwGOLrcg7DK44Be2Q4A5kbE/Ij4EBgHDCzzmKzMIuLPwLJyj8MqjwN6ZasGFhS8r83azMw+wQG9sqmRNk9LMrNGOaBXtlqgd8H7XsDCMo3FzCqcA3plmwb0ldRHUgfgVGBimcdkZhXKAb2CRUQdMBR4CHgBGB8Rz5V3VFZuksYCTwG7SqqVNLjcY7LK4CtFzcwS4QzdzCwRDuhmZolwQDczS4QDuplZIhzQzcwS4YBuLU5STtJMSbMl3Sup4wbs61BJD2SvT2jqjpOSukr6zqc4xo8lXfJpx2hWKRzQrTWsioi9I+ILwIfAtwtXKm+9f/ciYmJEjGiiS1dgvQO6WSoc0K21PQHsLGlHSS9Iugl4Bugtqb+kpyQ9k2XynaHhHvAvSnoS+PqaHUk6S9Ivs9c9JE2Q9PdsORAYAeyU/XXwX1m/SyVNk/SspKsL9nVFdp/5R4BdN9q/hlkrckC3ViOpPfl7uc/KmnYFbo+IfYAVwA+BIyNiX2A6cLGkzYFbgeOBrwDbFdn9SOBPEbEXsC/wHDAMmJf9dXCppP5AX/K3Id4b+JKkQyR9ifxtFPYh/4Wxfwt/dLOyaF/uAViStpA0M3v9BHAbsD3wWkRMydr7kX9ox18kAXQgfzn7bsArETEHQNKdQE0jxzgcOBMgInLAO5K6rdOnf7bMyN53Jh/guwATImJldgzfH8eS4IBurWFVROxd2JAF7RWFTcCkiDhtnX5703K3CBbwHxFxyzrHuKgFj2FWMVxysXKZAhwkaWcASR0l7QK8CPSRtFPW77Qi208Gzsu2bSdpS+A98tn3Gg8B5xTU5qsldQf+DJwoaQtJXciXd8zaPAd0K4uIeAs4Cxgr6VnyAX63iPiAfInlf7OToq8V2cWFwGGSZgF/A/aMiKXkSzizJf1XRDwM3A08lfX7LdAlIp4B7gFmAveRLwuZtXm+26KZWSKcoZuZJcIB3cwsEQ7oZmaJcEA3M0uEA7qZWSIc0M3MEuGAbmaWCAd0M7NE/D/Ua3p4F2r5gQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.4.5] TF-IDF W2V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████| 14000/14000 [00:34<00:00, 407.31it/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm import tqdm\n",
    "# TF-IDF weighted Word2Vec\n",
    "# we are converting a dictionary with word as a key, and the idf as a value\n",
    "dictionary = dict(zip(tf_idf_vect.get_feature_names(), list(tf_idf_vect.idf_)))\n",
    "# tfidf words/col-names\n",
    "tfidf_feat = tf_idf_vect.get_feature_names() \n",
    "# final_tf_idf is the sparse matrix with row= sentence, col=word and cell_val = tfidf\n",
    "\n",
    "d_train = []; # the tfidf-w2v for each sentence/review is stored in this list\n",
    "row=0;\n",
    "#for train dataset\n",
    "for sent in tqdm(train_list):\n",
    "    sent_vec = np.zeros(50) # as word vectors are of zero length\n",
    "    weight_sum =0; # num of words with a valid vector in the sentence/review\n",
    "    for word in sent: # for each word in a review/sentence\n",
    "        if word in w2v_words:\n",
    "            vec = w2v_model.wv[word]\n",
    "            # obtain the tf_idfidf of a word in a sentence/review\n",
    "            tf_idf = dictionary[word]*sent.count(word)\n",
    "            sent_vec += (vec * tf_idf)\n",
    "            weight_sum += tf_idf\n",
    "    if weight_sum != 0:\n",
    "        sent_vec /= weight_sum\n",
    "    d_train.append(sent_vec)\n",
    "    row += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████████████████████████████████████████████████| 6000/6000 [00:14<00:00, 400.14it/s]\n"
     ]
    }
   ],
   "source": [
    "#for train dataset\n",
    "d_test=[]\n",
    "for sent in tqdm(test_list):\n",
    "    sent_vec = np.zeros(50) # as word vectors are of zero length\n",
    "    weight_sum =0; # num of words with a valid vector in the sentence/review\n",
    "    for word in sent: # for each word in a review/sentence\n",
    "        if word in w2v_words:\n",
    "            vec = w2v_model.wv[word]\n",
    "            # obtain the tf_idfidf of a word in a sentence/review\n",
    "            tf_idf = dictionary[word]*sent.count(word)\n",
    "            sent_vec += (vec * tf_idf)\n",
    "            weight_sum += tf_idf\n",
    "    if weight_sum != 0:\n",
    "        sent_vec /= weight_sum\n",
    "    d_test.append(sent_vec)\n",
    "    row += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.4.7] KNN (Brute-Force)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 21.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4lOW5+PHvPZPMZN/IZA+BEIiGBMIii0tQkEWpYBEVXEBBPa312FNrK9aunJ6jP7e61KVWVKQoVhRBRRAR3FABFVT2LQqELSHsJCHJ/fsjyZwkhDAiQyZwf65rrsy87/O8750Ec/s877OIqmKMMca0No6WDsAYY4w5EZbAjDHGtEqWwIwxxrRKlsCMMca0SpbAjDHGtEqWwIwxxrRKlsCMMca0Sn5NYCIyRETWiMh6EZnQxPkCEflSRCpFZGSjc21F5F0RWSUiK0WknT9jNcYY07r4LYGJiBN4ArgEyAFGi0hOo2LfAzcALzVxiReBB1T1bKAXsNNfsRpjjGl9gvx47V7AelXdCCAi04DhwMq6AqpaWHuuun7F2kQXpKrzassdON7N4uPjtV27dicrdmOMOSN88cUXxarqaek4ToQ/E1gqsLne5y1Abx/rdgL2iMjrQHvgPWCCqlbVLyQitwC3ALRt25alS5f+6KCNMeZMIiLftXQMJ8qfz8CkiWO+LrwYBFwA3AmcA2RS09XY8GKqz6hqT1Xt6fH8+P+BOHz4MP369aOqqiZPDhkyhJiYGH7yk5/4VL+8vJyrr76arKwsevfuTWFhYZPlxo0bR0JCArm5uQ2O//nPfyY1NZX8/Hzy8/OZPXs2AEeOHGHs2LHk5eVx9tlnc++99wJQUVFBQUEBlZWVJ/gdG2NM6+XPBLYFSK/3OQ0o+gF1v1LVjapaCbwBdD/J8R3lueeeY8SIETidTgB+85vfMGXKFJ/rT5o0idjYWNavX8+vfvUr7rrrribL3XDDDcyZM6fJc7/61a9YtmwZy5Yt49JLLwXg1Vdfpby8nG+++YYvvviCf/zjHxQWFuJyuRgwYACvvPLKD/xOjTGm9fNnAlsCdBSR9iLiAkYBs35A3VgRqWtW9afeszN/mTp1KsOHD/d+HjBgAJGRkT7XnzlzJmPHjgVg5MiRzJ8/n6ZW+y8oKCAuLs7n64oIBw8epLKyksOHD+NyuYiKigLg8ssvZ+rUqT5fyxhjThd+S2C1LafbgLnAKuDfqrpCRCaKyDAAETlHRLYAVwL/EJEVtXWrqOk+nC8i31DTHflPf8UKNd1xGzdu5McMBNm6dSvp6TWNzqCgIKKjoykpKflB1/j73/9Oly5dGDduHKWlpUBNMgwPDyc5OZm2bdty5513ehNgbm4uS5YsOeGYjTGmtfLrPDBVna2qnVS1g6r+T+2xP6rqrNr3S1Q1TVXDVbWNqnauV3eeqnZR1TxVvUFVK/wZa3FxMTExMT/qGk21tkSaehTYtJ///Ods2LCBZcuWkZyczK9//WsAFi9ejNPppKioiE2bNvHQQw+xceNGAJxOJy6Xi/379/+o2I0xprWxlThqhYaGUlZW9qOukZaWxubNNQMvKysr2bt37w/qKkxMTMTpdOJwOLj55ptZvHgxAC+99BJDhgwhODiYhIQEzjvvvAYjLsvLywkJCflRsRtjTGtjCaxWbGwsVVVVPiWxu+++mxkzZhx1fNiwYUyePBmA6dOn079//x/UAtu2bZv3/YwZM7yjFNu2bcv777+PqnLw4EE+++wzzjrrLABKSkrweDwEBwf7fB9jjDkdnPEJbF/ZER55by3LN+9h0KBBfPzxx95zF1xwAVdeeSXz588nLS2NuXPnAvDNN9+QlJR01LXGjx9PSUkJWVlZPPzww9x3330AFBUVeUcUAowePZq+ffuyZs0a0tLSmDRpEgC//e1vycvLo0uXLixYsIC//e1vAPziF7/gwIED5Obmcs4553DjjTfSpUsXABYsWNDg2sYYc6aQpp7btEY9e/bUE5nIvPfwEbr+5V1+P/RsekTs5eGHHz7u0PnBgwd7k1lLGzFiBPfeey/Z2dktHYoxphUSkS9UtWdLx3EizvgWWFRIEC6ng10HyunWrRsXXXSRdyLzsQRK8qqoqODyyy+35GWMOSP5cympVkFE8ES62bW/HKhZJaO1cLlcjBkzpqXDMMaYFnHGt8AA4iPdFB/w6yh9Y4wxJ5klMMAT8X8tMGOMMa2DJTDAE+myBGaMMa2MJTBqWmC7D5ZTVX16jMg0xpgzgSUwwBPpplqh5KC1wowxprWwBEZNAgMo3m8DOYwxprWwBAbER9QksF0HrAVmjDGthSUw/q8FZgM5jDGm9bAERr0WmCUwY4xpNSyBAeHuIMJdToqtC9EYY1oNS2C16i8nZYwxJvBZAqsVb6txGGNMq2IJrJYn0m2jEI0xphWxBFbLuhCNMaZ1sQRWyxPhZu/hI5RXNr8XmDHGmMBgCaxWfO1csBLbVsUYY1oFS2C1PDYXzBhjWhVLYLVsNQ5jjGldLIHV8i7oayMRjTGmVbAEVqtNhAuwFpgxxrQWlsBquYOcRIcG21wwY4xpJSyB1WNzwYwxpvWwBFaPx5aTMsaYVsOvCUxEhojIGhFZLyITmjhfICJfikiliIxsdK5KRJbVvmb5M846nki3DeIwxphWIshfFxYRJ/AEMBDYAiwRkVmqurJese+BG4A7m7jEYVXN91d8TbEFfY0xpvXwZwusF7BeVTeqagUwDRhev4CqFqrq10C1H+PwmSfSzcGKKg6WV7Z0KMYYY47DnwksFdhc7/OW2mO+ChGRpSLymYhc3lQBEbmltszSXbt2/ZhYAZsLZowxrYk/E5g0cUx/QP22qtoTuAZ4REQ6HHUx1WdUtaeq9vR4PCcap5clMGOMaT38mcC2AOn1PqcBRb5WVtWi2q8bgYVAt5MZXFPibTKzMca0Gv5MYEuAjiLSXkRcwCjAp9GEIhIrIu7a9/HAecDK5mv9eLYeojHGtB5+S2CqWgncBswFVgH/VtUVIjJRRIYBiMg5IrIFuBL4h4isqK1+NrBURJYDC4D7Go1e9Is24W4cYgnMGGNaA78NowdQ1dnA7EbH/ljv/RJquhYb11sE5PkztqY4HUJcuNuWkzLGmFbAVuJoJD7Cxa79tqmlMcYEOktgjXgirQVmjDGtgSWwRjyRbortGZgxxgQ8S2CN1K1Ir/pDpqwZY4w51SyBNeKJcFNRVc2+MltOyhhjApklsEZsLpgxxrQOlsAa8URYAjPGmNbAElgj3haYjUQ0xpiAZgmsEetCNMaY1sESWCPRocEEO8VWpDfGmADXbAITEaeIPHCqggkEImI7MxtjTCvQbAJT1Sqgh4g0tbfXaatuLpgxxpjA5ctivl8BM0XkVeBg3UFVfd1vUbUwT4SbbXvLWjoMY4wxzfAlgcUBJUD/escUOG0TWHyEm2+27m3pMIwxxjTjuAlMVW88FYEEEk+km5KDFVRVK07HGdV7aowxrcZxRyGKSJqIzBCRnSKyQ0ReE5Gj9vA6nXgi3VRVK6WHbFsVY4wJVL4Mo38emAWkAKnAm7XHTls2F8wYYwKfLwnMo6rPq2pl7esFwOPnuFqUJTBjjAl8viSwYhG5rnZOmFNErqNmUMdpK752PUSbzGyMMYHLlwQ2DrgK2A5sA0bWHjttWQvMGGMCX7OjEEXECVyhqsNOUTwBIdzlJDTYaQnMGGMCmC8rcQw/RbEEDBGpWY3DuhCNMSZg+TKR+RMR+TvwCg1X4vjSb1EFgPgIlz0DM8aYAOZLAju39uvEeseUhitznHY8kW42FR88fkFjjDEt4njPwBzAU6r671MUT8DwRLpZvGl3S4dhjDHmGI73DKwauO0UxRJQPBEhlB46QkVldUuHYowxpgm+DKOfJyJ3iki6iMTVvfweWQuLj3QBUHLQnoMZY0wg8uUZWN2cr1/UO6ZA5skPJ3B46iYz768gOTq0haMxxhjTmC+r0bc/FYEEGu9k5gNlQHTLBmOMMeYox+xCFJHf1nt/ZaNz/+vLxUVkiIisEZH1IjKhifMFIvKliFSKyMgmzkeJyNbaYfynlK3GYYwxga25Z2Cj6r2/u9G5Ice7cO0qHk8AlwA5wGgRyWlU7HvgBuClY1zmv4EPjncvf6hbD9ESmDHGBKbmEpgc431Tn5vSC1ivqhtVtQKYRqNVPVS1UFW/Bo4a6iciPYBE4F0f7nXShQQ7iQwJoviA7QlmjDGBqLkEpsd439TnpqQCm+t93lJ77Lhq5589BPzmOOVuEZGlIrJ0165dvlz6B/FEuq0FZowxAaq5QRxdRWQfNa2t0Nr31H4O8eHaTbXSfEl8ALcCs1V1s8ixG3uq+gzwDEDPnj19vbbPPBGWwIwxJlAdM4GpqvNHXnsLkF7vcxpQ5GPdvsAFInIrEAG4ROSAqh41EMSfPJFuVhTtO35BY4wxp5wv88BO1BKgo4i0B7ZSMyjkGl8qquq1de9F5Aag56lOXlAzkMNaYMYYE5h8WYnjhKhqJTXLUM0FVgH/VtUVIjJRRIYBiMg5IrIFuBL4h4is8Fc8J8IT6eZAeSWHK6paOhRjjDGN+LMFhqrOBmY3OvbHeu+XUNO12Nw1XgBe8EN4x1U3F6z4QDnpcWEtEYIxxphj8FsL7HRQl8B2WjeiMcYEnOMmMBEZISLrRGSviOwTkf31RiSe1jw2mdkYYwKWL12I9wOXqeoqfwcTaOp3IRpjjAksvnQh7jgTkxdAXLgLEWuBGWNMIPKlBbZURF4B3gC8f8lV9XW/RRUggp0O4sJc7LIWmDHGBBxfElgUcAgYVO+YAqd9AgNbTsoYYwKVL/uB3XgqAglUNpnZGGMCky+jENNEZIaI7BSRHSLymog0O3frdOKJdNsgDmOMCUC+DOJ4HpgFpFCzmvybtcfOCHVdiKonfa1gY4wxP4IvCcyjqs+ramXt6wXA4+e4AoYnwk15ZTX7yytbOhRjjDH1+JLAikXkOhFx1r6uA0r8HVigiI90ATaU3hhjAo0vCWwccBWwHdgGjKw9dkbwRNRsfVZsCcwYYwKKL6MQvweGnYJYAlLdahw2F8wYYwLLMROYiPxWVe8XkcdpYidlVb3dr5EFCG8CsxaYMcYElOZaYHXLRy09FYEEqpjQYJwOsQRmjDEB5pgJTFXfrH17SFVfrX9ORK70a1QBxOEQ4iNclsCMMSbA+DKI424fj522bDKzMcYEnuaegV0CXAqkishj9U5FAWfUpChPhNsGcRhjTIBp7hlYETXPv4YBX9Q7vh/4lT+DCjSeSDcrt50Re3gaY0yr0dwzsOXAchF5SVWPnMKYAk58hJviAxVUVysOh7R0OMYYY/DtGVg7EZkuIitFZGPdy++RBRBPpJuqamXP4TM6jxtjTEDxdTHfp6h57nUR8CIwxZ9BBRqbC2aMMYHHlwQWqqrzAVHV71T1z0B//4YVWDwRlsCMMSbQ+LIjc5mIOIB1InIbsBVI8G9YgSXeu5xUWQtHYowxpo4vLbD/AsKA24EewHXAWH8GFWisC9EYYwKPL4v5Lql9ewC40b/hBKZIdxDuIAfFBypaOhRjjDG1jtsCE5F5IhJT73OsiMz1b1iBRUS8OzMbY4wJDL50Icar6p66D6payhn2DAxq5oJZAjPGmMDhSwKrFpG2dR9EJIMmtlc53VkLzBhjAosvCewe4GMRmSIiU4AP8XExXxEZIiJrRGS9iExo4nyBiHwpIpUiMrLe8QwR+UJElonIChH5ma/fkL/Ygr7GGBNYfBnEMUdEugN9AAF+parFx6snIk7gCWAgsAVYIiKzVHVlvWLfAzcAdzaqvg04V1XLRSQC+La2bpEv35Q/eCLc7D5UwZGqaoKdvuR9Y4wx/nTMv8Qiclbt1+5AW2oW990KtK09djy9gPWqulFVK4BpwPD6BVS1UFW/BqobHa9Q1brmjru5OE8VT6QbVdh90EYiGmNMIGiuBXYHcAvwUBPnlOOvxpEKbK73eQvQ29fARCQdeBvIAn7TVOtLRG6pjZG2bds2Pn1SxddbjSMxKsSv9zLGGHN8zSWwebVfx6vqiSze29Sy7T4P/lDVzUAXEUkB3hCR6aq6o1GZZ4BnAHr27OnXgSXeycz2HMwYYwJCc11zdQM1pp/gtbcA6fU+p1HTDfmD1La8VgAXnGAcJ0WCrcZhjDEBpbkWWImILADai8isxidVddhxrr0E6Cgi7al5djYKuMaXoEQkDShR1cMiEgucBzzsS11/ibcFfY0xJqA0l8CGAt2p2TqlqedgzVLVytrFf+cCTuA5VV0hIhOBpao6S0TOAWYAscBlIvIXVe0MnA08JCJKTVfkg6r6zQ+N4WQKdTmJcAdZAjPGmADR3I7MFcBnInKuqu46kYur6mxgdqNjf6z3fgk1XYuN680DupzIPf3JE+m2Z2DGGBMgjpnAROQRVf0v4LnallADPnQhnnY8EW6KrQVmjDEBobkuxLpdlx88FYG0Bp5IN6u272vpMIwxxtB8F+IXtV8/qDtWO6AivXby8RknPsJlz8CMMSZA+LKdykIRiRKROGA58LyItOiIwJbiiXSzv6ySsiNVLR2KMcac8XxZoilaVfcBI4DnVbUHcLF/wwpMdZOZbVFfY4xpeb4ksCARSQauAt7yczwBzWOTmY0xJmD4ksAmUjOXa72qLhGRTGCdf8MKTDaZ2RhjAocv26m8Crxa7/NG4Ap/BhWobD1EY4wJHL4M4ri/dhBHsIjMF5FiEbnuVAQXaNqEWwvMGGMChS9diINqB3H8hJoFejsBv/FrVAHKFeQgNizYBnEYY0wA8CWBBdd+vRR4WVV3+zGegOeJdFsLzBhjAsBxn4EBb4rIauAwcKuIeIAy/4YVuOIjLIEZY0wgOG4LTFUnAH2Bnqp6BDgIDPd3YIHKFvQ1xpjA4EsLDCAVGCgiIfWOveiHeAJezYK+FagqIk1tOm2MMeZUOG4CE5E/ARcCOdRsjXIJ8DFnagKLdHP4SBUHK6qIcPua/40xxpxsvgziGAkMALar6o1AV8Dt16gCmE1mNsaYwOBLAjusqtVApYhEATuBTP+GFbhsOSljjAkMvvSBLRWRGOCfwBfAAWCxX6MKYJbAjDEmMPgyCvFWVd2jqk8DA4GxtV2JZ6T6K9IfPnyYfv36UVVVxXfffUePHj3Iz8+nc+fOPP3008e91u7duxk4cCAdO3Zk4MCBlJaWNlluyJAhxMTE8JOf/KTB8WuvvZbs7Gxyc3MZN24cR44cAWDq1Kl06dKFLl26cO6557J8+XIAKioqKCgooLKy8sf8CIwxJiAcM4GJSPfGLyCOmtXpu5+6EANLbJgLh9S0wJ577jlGjBiB0+kkOTmZRYsWsWzZMj7//HPuu+8+ioqKmr3Wfffdx4ABA1i3bh0DBgzgvvvua7Lcb37zG6ZMmXLU8WuvvZbVq1fzzTffcPjwYZ599lkA2rdvzwcffMDXX3/NH/7wB2655RYAXC4XAwYM4JVXXvmRPwVjjGl5zXUhPtTMOQX6n+RYWgWnQ2hTO5n5o6lTeemll4Ca5FCnvLyc6urq415r5syZLFy4EICxY8dy4YUX8v/+3/87qtyAAQO85eq79NJLve979erFli1bADj33HO9x/v06eM9DnD55Zdz9913c+211x43PmOMCWTHTGCqetGpDKQ18US42b7nABs3bqRdu3be45s3b2bo0KGsX7+eBx54gJSUlGavs2PHDpKTkwFITk5m586dJxTPkSNHmDJlCo8++uhR5yZNmsQll1zi/Zybm8uSJUtO6D7GGBNIfFmN/he1gzjqPseKyK3+DSuweSLdbNuxk5iYmAbH09PT+frrr1m/fj2TJ09mx44dpySeW2+9lYKCAi644IIGxxcsWMCkSZMatOqcTicul4v9+/efktiMMcZffBlGf7Oq7qn7oKqlwM3+CynweSLdlJZDWVnTS0KmpKTQuXNnPvroo2avk5iYyLZt2wDYtm0bCQkJPziWv/zlL+zatYuHH364wfGvv/6am266iZkzZ9KmTZsG58rLywkJCcEYY1ozXxKYQ+qtmSQiTsDVTPnTXnyEmz1VLqqqqrxJbMuWLRw+fBiA0tJSPvnkE7KzswEYM2YMixcfPfNg2LBhTJ48GYDJkyczfPgPW2Ly2WefZe7cubz88ss4HP/3q/z+++8ZMWIEU6ZMoVOnTg3qlJSU4PF4CA4Obnw5Y4xpVXxJYHOBf4vIABHpD7wMzPFvWIHNE+nmSJVScNEAPv74YwBWrVpF79696dq1K/369ePOO+8kLy8PqGkN1T3rqm/ChAnMmzePjh07Mm/ePCZMmADA0qVLuemmm7zlLrjgAq688krmz59PWloac+fOBeBnP/sZO3bsoG/fvuTn5zNx4kQAJk6cSElJCbfeeiv5+fn07NnTe60FCxY0GPxhjDGtlahq8wVEHMAtwMWAAO8Cz6pqlf/D813Pnj116dKlp+Reyzbv4fInPuHnuQ6+nftSk0Pc6+zbt4/x48fz6quvnpLYjmfEiBHce++93tahMebMJiJfqGrP45cMPMddiaN2GamngadFJA5IC7Tkdap1TYsmo00YX5eFMuCii6iqqsLpdDZZNioqKmCSV0VFBZdffrklL2PMacGXUYgLRSSqNnktA54XkYePV+90JiIM75rCog0l/GTkNcdMXoHG5XIxZsyYlg7DGGNOCl+egUWr6j5gBPC8qvagpjvxuERkiIisEZH1IjKhifMFIvKliFSKyMh6x/NF5FMRWSEiX4vI1b5+Q6fKsPwUVOHNr7e1dCjGGHNG8iWBBYlIMnAV8JavF64drfgENfuH5QCjRSSnUbHvgRuAlxodPwSMUdXOwBDgkfpz0QJBVkIknVOimLVsa0uHYowxZyRfEthEakYirlfVJSKSCazzoV6v2jobVbUCmAY0GCeuqoWq+jVQ3ej4WlVdV/u+iJotXDw+3POUGp6fwvIteyksPtjSoRhjzBnHl9XoX1XVLqp6a+3njap6hQ/XTgU21/u8pfbYDyIivaiZd7ahiXO3iMhSEVm6a9euH3rpH+2yrimIwKzlzS/aa4wx5uQ75ihEEfmtqt4vIo9Ts3hvA6p6+3GuLU0ca37M/tExJANTqNnC5ajVcVX1GeAZqBlG/0OufTIkR4dyTrs43li2lf/sn0W9+d7GGGP8rLkW2Krar0up2ciy8et4tgDp9T6nAT43VWp3f34b+L2qfuZrvVNteH4KG3cdZEXRvpYOxRhjzijNrUb/Zu3XySd47SVARxFpD2wFRgHX+FJRRFzADOBFVQ2MSVTHcGluMn+auYJZy4vITY1u6XCMMeaM0VwX4qzmKqrqsOOcrxSR26gZAOIEnlPVFSIyEViqqrNE5BxqElUscJmI/KV25OFVQAHQRkRuqL3kDaq6zNdv7FSJDXfRr5OHWcuKmDDkLBwO60Y0xphTobmVOPpSMwjjZeBzmn6m1SxVnQ3MbnTsj/XeL6Gma7FxvX8B//qh92spw/JTmL96J4sLd9Mns83xKxhjjPnRmnsGlgT8DsgFHgUGAsWq+oGqfnAqgmstBuYkEhrsZOYyG41ojDGnyjETmKpWqeocVR0L9AHWAwtF5D9PWXStRJgriEGdE3nn221UVB41WNIYY4wfNDsPTETcIjKCmu68XwCPAa+fisBam+H5Kew5dISP1p36+WjGGHMmam4Qx2Rqug/fAf6iqt+esqhaoQs6eogNC2bmsiIGnJ3Y0uEYY8xpr7lBHNcDB4FOwO31N2UGVFWj/BxbqxLsdHBpXjKvf7mVg+WVhLuPu1ONMcaYH6G5Z2AOVY2sfUXVe0Va8mra8PxUDh+p4r1VO1o6FGOMOe35spiv8VHPjFiSo0NsNKIxxpwClsBOIodDGNY1hQ/X7mL3wYqWDscYY05rlsBOsmH5KVRWK7O/sY0ujTHGnyyBnWQ5yVFkJUTYFivGGONnlsBOMhFheNcUFm/aTdGewy0djjHGnLYsgfnBsPwUAN60VpgxxviNJTA/yGgTTn56jI1GNMYYP7IE5ifD81NYuW0f63bsb+lQjDHmtGQJzE+GdknGIdhgDmOM8RNLYH6SEBnCeVnxzFxWhKq2dDjGGHPasQTmR8O6pvD97kMs27ynpUMxxpjTjiUwPxqcm4QryGHdiMYY4weWwPwoKiSY/tkJvLl8G1XVNd2Ihw8fpl+/flRVVQHgdDrJz88nPz+fYcOGHfea5eXlXH311WRlZdG7d28KCwubLDdu3DgSEhLIzc1tcPzVV1+lc+fOOBwOli5d6j0+b948evToQV5eHj169OD999/3nrv44ospLS39od++Mcb4lSUwPxuen0LxgXI+3VACwHPPPceIESNwOp0AhIaGsmzZMpYtW8asWbOOe71JkyYRGxvL+vXr+dWvfsVdd93VZLkbbriBOXPmHHU8NzeX119/nYKCggbH4+PjefPNN/nmm2+YPHky119/vffc9ddfz5NPPunz92yMMaeCJTA/u+isBCLdQcxcthWAqVOnMnz48BO+3syZMxk7diwAI0eOZP78+U0OEikoKCAuLu6o42effTbZ2dlHHe/WrRspKTUTsDt37kxZWRnl5eUADBs2jJdffvmEYzbGGH+wBOZnIcFOBucmMefb7ew7eJiNGzfSrl077/mysjJ69uxJnz59eOONN457va1bt5Keng5AUFAQ0dHRlJSUnNSYX3vtNbp164bb7QYgNjaW8vLyk34fY4z5MWzb4FNgeH4K07/YwpufryYmJqbBue+//56UlBQ2btxI//79ycvLo0OHDse8VlOtrXq7Zf9oK1as4K677uLdd99tcDwhIYGioiLatGlz0u5ljDE/hrXAToG+mW2Ij3Az89tiysrKGpyr67bLzMzkwgsv5Kuvvmr2WmlpaWzevBmAyspK9u7d22RX4YnYsmULP/3pT3nxxRePSqJlZWWEhoaelPsYY8zJYAnsFAhyOvhZv0wWF1Ww52C5N4mVlpZ6nzMVFxfzySefkJOTA8Ddd9/NjBkzjrrWsGHDmDx5MgDTp0+nf//+J6UFtmfPHoYOHcq9997Leeed1+CcqrJ9+/YGXZ/GGNPSLIGdIuPbyZB7AAAgAElEQVTPb8/VPdOpTM7jr5NeB2DVqlX07NmTrl27ctFFFzFhwgRvAvvmm29ISko6+jrjx1NSUkJWVhYPP/ww9913HwBFRUVceuml3nKjR4+mb9++rFmzhrS0NCZNmgTAjBkzSEtL49NPP2Xo0KEMHjwYgL///e+sX7+e//7v//YO69+5cycAX3zxBX369CEoyHqcjTGBQ06XZY569uyp9ec1BaIjVdVcPvFffPjaC8x549+clxV/zLKDBw9m7ty5pzC6Y/vlL3/JsGHDGDBgQEuHYow5yUTkC1Xt2dJxnAhrgZ1CwU4HL989mozcc/iPFxeztpmV6gMleUHN3DFLXsaYQGMJ7BSLCglm7tN/JtTt4sbnl7Bzf9nxK7Wwm2++uaVDMMaYo/g1gYnIEBFZIyLrRWRCE+cLRORLEakUkZGNzs0RkT0i8pY/Y2wJqTGhPDf2HHYfrOCmyUs5VFHZ0iEZY0yr47cEJiJO4AngEiAHGC0iOY2KfQ/cALzUxCUeAK5v4vhpIS8tmsdHd+PbrXu5/eVl3rUSjTHG+MafLbBewHpV3aiqFcA0oMEaSqpaqKpfA9WNK6vqfOC03s744pxE/nRZZ95btYO/vr2ypcMxxphWxZ/jolOBzfU+bwF6n8wbiMgtwC0Abdu2PZmXPmXGntuO70oO8dwnm2gbF8aN57Vv6ZCMMaZV8GcLrKnZtSe1n0xVn1HVnqra0+PxnMxLn1L3DD2bQTmJTHxrJe+u2H7U+fpbsCxYsMA7Tys/P5+QkJCj1lBsvGXLnXfeSXR0NC6Xi6ysrCa3YDl8+DBJSUl4PB5ycnJITk7G6XQSExPDsmXLGDhwIB07duSss84iMzMTEWHkyJG43W5CQkLo3Lkzzz//PABr164lPj6ejh07MnDgQJ588kk6duxIhw4dyM/Pp2vXrg3K79q1i/z8/Ca3eQG49957ycrKIjs7u8HozL/97W907tyZ3NxcRo8e7Z0gPmrUKNatW3fivxBjTKvgzwS2BUiv9zkNsJ0dm+B0CI+O6kaX1Gh+OW0ZX29puINz/S1YLrroIu/2K++//z5hYWEMGjTomOXffvtt3n77ba6++mpKS0uprq7mjjvuOCqG5557jquuuoq5c+eyZ88eevTowRtvvEFmZiajRo1iwIABrFu3jkGDBjFo0CAyMjIIDQ1l1KhRZGVlsXDhQn79619TUVHBs88+S0ZGBi+88AJ9+/bld7/7HZ9//jnXXHMNhYWFLFy4sEF5j8dD+/bt+f3vf3/UNi8rV65k2rRprFixgjlz5nDrrbdSVVXF1q1beeyxx1i6dCnffvstVVVVTJs2DYCf//zn3H///Sf5t2SMCTT+TGBLgI4i0l5EXMAo4PgbXp2hQl1Onh17Dm0iXIx7YSlbSg95zx1rC5bp06dzySWXEBYW1uB4/fIrV66ksrKSG2+8kfDwcPr168e8efOOWhR46tSp3HHHHcTFxbFv3z7uueceIiMjSU5OZt26dYwZMwaoWeJqwYIFAGRkZHhbeQcOHCAuLo6goCBmzpzJbbfdxtSpU72ricTFxREWFkZqairvvPNOg/JQs3/ZRx99dNT3OHPmTEaNGoXb7aZ9+/ZkZWWxePFioGYtyMOHD1NZWcmhQ4e860pecMEFvPfee1RW2uhOY05nfktgqloJ3AbMBVYB/1bVFSIyUUSGAYjIOSKyBbgS+IeIrKirLyIfAa8CA0Rki4gM9lesgcIT6eb5G86hvLKKcS8sYe/hI1RUVBy1BUudadOmMXr06AbHGpfv2rUrW7duJT4+nuLiYj744ANcLleDrVEa16msrPRu2VK3zqLL5QIgOTnZu8TUTTfdxPr161mzZg15eXk8+uijOBwOduzYwaBBg/joo484dOiQd73H2267jUOHDvGzn/2sQXmAnj17NpnA6m8fAzWLGW/dupXU1FTuvPNO2rZtS3JyMtHR0d6WqMPhICsri+XLl/+gn78xppVR1dPi1aNHDz1dfLJul3a4+20d/cynun7T95qdnX1UmaKiIo2Pj9eKiooGx7du3XpUeY/Hozk5OXrxxRfrNddco23atNHi4uIm62zatEndbrdu3rxZFyxYoEOHDlWHw9GgfExMjGZkZOhTTz2liYmJmpOTo+vWrdOQkBCNjo7WoKAgraio0Li4OL3//vvV6XRqly5dNC0tTYOCgnTixIkNyg8dOtRbvl+/fjp06FCNjY1Vl8ulIqKJiYlaWVmpqqqpqakaFhamgwYN0o4dO2q7du0U0LCwMA0KClKXy6XR0dHqcDg0MTFR8/LytG/fvhodHa0dO3ZUt9utgHbs2FEdDoe3TlhYmKanp2unTp10zpw5qqq6evVqzcjI8NYJCwvT0aNHa1ZWlsbExGhBQYF26NDhuHX+9re/qarqr3/9ax0/fnyTdbp27ep9RUZGNqgzf/78E//H5INDhw5pQUGBVlZWamFhoXbv3l27du2qOTk5+tRTT3nL/e53v9O0tDQNDw/3+dr/+7//e9T329jjjz+uHTp0UEB37drlPV5dXa3/+Z//qR06dNC8vDz94osvVFX1/fffb/DzcrvdOmPGDFVVvfrqq3Xt2rUn8mM4YwFLNQD+hp/Iq8UDOFmv0ymBqaq+unSzZtz1lmb81zQNiU3SX037Sv/54Qb9dEOx7j1coY888ojefPPNR9XbvXu3ZmRkNDg2aNAgXbRokaqqjho1SiMjI7W6urrJOps2bdLw8HBdtGiRLliwQC+99FJ1OBy6detWVa1JnJ06ddKMjAzNzs7WsWPHaufOnVVVtWvXrvrggw9qeHi4rl27VlNTU/WJJ57QqKgoVVXNycnRNm3a6KhRoxqUHzp0qO7bt09TU1O1X79+2q9fP23btq2WlZXpgAEDNCwszJvAevTooffff792795dhw8frps2bdK0tDRt06aN3njjjbp7924NCgpSt9ut7777rn766aeanZ2ts2bN0n79+unq1au1X79+On36dAX0gw8+0EWLFqmIaO/evXXjxo2amZnpvd/KlSt19erVWlBQoDExMZqenq4lJSX6+uuvq4jotm3bmq0TFxenhYWFqqo6b948jYiI0LKysqPq1KmsrNTExERvncLCQh04cOAJ/Avy3d///nd95JFHVFW1vLxcy8rKVFV1//79mpGR4f3df/rpp1pUVORzAluxYoV26dKl2e9XVfXLL7/UTZs2aUZGRoME9vbbb+uQIUO0urpaP/30U+3Vq9dRdUtKSjQ2NlYPHjyoqqoLFy7Um2666Yf9AM5wrTmB2VJSAWpkjzReuqk3tw3Jx0E1H64q4q9vr2LUM5/R5c/v8oeH/sEOTw+eWLCeBWt2smt/OXfffTcLFy6kqqrKOyKvqqqK/v37M3nyZL7++ms+/vhjBg0ahIh4t2yJjY1tUCcyMtK7Zcu2bdvIysrixRdfBGDy5Mne52u7d+/2dgG+8847rF27lqysLBITE3nsscfIzc1l+/btiAilpaXs2rWLffv20alTJ3bs2MGuXbvIysoCakYu5ubmArB9+3bOO+883G43xcXFAHz88cds2rSJ3bt30717d0JDQ1m3bh0JCQmUlZURHR1Nt27dCAoKorKyErfbTW5uLn369EFVqaioICIiguzsbAD++c9/EhwcTEFBAQsXLiQ7O5svv/yS9PT0Bs/Zzj77bLKzs9mzZw+RkZFccsklxMXFsXr1akJDQ5kxY8ZRz+bq10lLSyMjIwOAJUuWEBUVRWlp6VF16syfP58OHTp462RkZFBSUsL27UePTj1Z6j8zdblc3p24y8vLqa7+vymaffr0ITk52efrNvf8sr5u3bo12UU+c+ZMxowZg4jQp08f9uzZw7Zt2xqUafwc2J5/nmFaOoOerNfp1gKrb9y4cTpv3jzdse+wvr96h/7lpQUaHuvR8+99r6aVVvuKzu6tgyb8UzsVDNPLJjyht7/8pd4+5XONS22vrtBwdQa7NKFtlk54fp7+bd4aze5+rmbm5OvMZVv12uvH6oUXXqhJSUnqdDpVRLyvhIQE7dq1q7Zr107DwsI0OTlZHQ6HAg2+Zmdn64IFC3TgwIGalZWlHo9H+/fvr4899phGRER4u9RycnK0c+fO2q9fP3366ad16NChOmbMGI2OjlaXy6VOp1NdLpd27txZw8LCND8/XxMSErRTp046e/Zsb9fmH//4R83OzlaHw6GdO3fWsrIynTRpkoqIejwe78+vf//+3vuoqvbr109TUlI0PDxclyxZov/xH/+hISEhGhwcrFOnTlWPx6NpaWmal5en06ZNU1XVpKQkzcrK0ri4OAV0zJgx6na79dprr9WvvvrqmHXS0tLU5XKpy+XS+Ph4jYqK0ldeeaVBnbZt22pqaqqKiF522WXerla3260xMTHqcDg0OTlZn3jiCe3Tp4/m5ORoUlKSt5syKytL09PTvV2uIqKhoaGalZWleXl5mpOTo3l5eTpu3DgNDw9XQBMTEzU1NVWDg4MV0IEDB3q7+qZMmaJ5eXkaGhqq1113nUZGRiqgbdq00bS0NBURHThwoCYlJWlmZqa2b99es7KytGvXrpqWlqaJiYkKaGRkpLrdbs3KytKzzjpLHQ6HRkdHNyifk5OjDz30kF5xxRUaFBSkISEhGhERoS6XSwFNTk72dm1GRkZqx44dNScnR4cMGaJpaWnqcDi83c1ut1sjIiLU4XBoamqqdu3aVXNzc7Vr166alJSksbGxCminTp28/9aCg4PV4XBou3btvN2Ue/fu1ZSUFO3Vq5eGhoYqoLGxsRoZGakiohERERoTE6OZmZnN1gkODtbg4GB1uVyalJSkCQkJR9WJjo72/nw7dOigQUFB3t+9iGhKSspR98jLy/P+7t1ut3o8Hu/PKzg4WDMzM7VXr166adMmb53u3bt760RGRnr/TXo8HgV2AuuBr4Huqgo14xcKqZn6tBbYUft5DTUjzL+rX762zh+AQ7V11gDtao9PAyY3vkftubbAu9SMk1jZqE5HPc7f/RZPPCfrdTonsC+//FKvu+66Js/tPVyhn24o1mc/2qgZXfrqsMc/0r6//qcm9RioBfe/r+feO197/nWe5v9lrnb+4xztdM9sbT+hJuGFtOvmTX7JNzyqqecM0gfmrNbPN5ZoRWVVszE1ftZ255136vLly73J5YILLtDdu3cfVf7nP/+5Pvfcc97jTZUfO3asvvjii7px40aNiorSbt266fTp04+qUycmJkYnTJigqqoXXnihOhwOveiii7znGyewvn37akhIiI4fP17PP/98jYiI0ISEBA0NDdWnnnpKr7jiCp0+fbpu3bpVk5KSdMeOHRoUFKTDhw/XO+64QzMyMrRPnz6akJCgl19+ua5Zs6bJOk6nUxMSErSkpER3796tTqdTPR6PPvbYYw3qLFy4UD0ej5577rkaERGh0dHR+tJLL+ncuXPV6XTqRRddpH/60580LS1NV61apaqqAwYM0MjISE1PT9dvv/1WMzIytKSkRK+88koF9IknntBXXnlFu3Tp4v35BwcH6+OPP64FBQUaHx+vqamp+u2332p4eHiT3aFbt27VmJgYveuuu/Tcc8/VlJQU3bhxo4aHh3v/CJaVlenq1au1ffv2WllZqUuWLNHQ0FDNzMzUkSNH6pNPPqmlpaV6zz33qMPh0PT09Abl9+/fr+Hh4frAAw9oRkaGnnXWWXrOOedodXW1Nymr1nRtXnjhhbp06VLdv3+/JiUl6dy5cxXQvLw8bzeliGhUVJTOmjVLp0yZoldccYWq1nQtBgUFaUJCgr799tsaHByshYWFeu2116qI6NixY73dlLfffruOHj1aPR6Pvvjii5qRkaGPPvqoZmdna3h4uF588cUqInrkyJFj1klOTtaOHTvq7t27dcOGDepwODQ0NLRBd+jtt9+uQ4YM0cTERE1MTNSXXnpJY2Nj9cMPP9Sf/vSn6nK5dPz48UfdIz4+Xl9++WXv/4Dl5uZqWVmZ9/dYWVmpL7/8sl511VV6++2366hRozQ6OlrffvttTUxM1KCgIC0sLNSNGzfW/Q9MBTXzdvsAn2tNAukGfAZsB84FvgFigBuBMuDZ+uVr6ywFrqtNdBlAWO3x31KzqIU0UWchMLD2fUS9Ov2Af+px/u5bF2Ir0K1bNy666CLvkPX6okKC6ZPZhvHnt6dw+SJm3nY+ix68if+5dRTv31HAJxP6s+Sei/nqj4P49i+DWfPXS9h471A2/O+llK5dwjd/HsQbvziPCdddQlJ2D55csJar/vEp3SfO45YXlzL18+/YvPvQUfcNDQ31djkCPPDAA3Tp0gWoGdV4xx13EBsbe1T5q6++mtdee63BtRqXz87OZsuWLcTExBAaGsrmzZu9Q+SbEhQURFFRESUlJSxbtgyo6cars2XLFuLj/2/vtZ07d5Kbm4vT6eSjjz4iLi6O8PBwQkJCyMrKYv/+/aSkpJCSkkJCQgKvvfYakZGRnHPOORw4cACAzZs3U11dTXx8PJ06dWqyTkhICOeddx5xcXHExMQgIlRWVhIaGtqgTr9+/byjO2NiYujQoQPBwcF88cUXBAcHEx4eTlpaGjk5OZSWlrJv3z42bNhA+/btqaqq4pNPPmHw4MHExcWxcOFCAMrKyrjqqqsoKytj27Zt7Nmzh+DgYAYPHsyRI0eorq5m6NChpKSkeEeavvfeew26+iIiIqisrKRHjx6UlpbSu3dv2revWSmmuroat9tNaWkp2dnZdOzYkcWLFxMfH09FRQUJCQkkJCSwZ88eYmJiePHFF3G73VRWVhIdHe0tX1xcTFlZmXc0bWlpKcOHD0dEqKiooKqqiuXLl+NyuSgqKiIlJYXy8nKCg4P57LPPcDgcjB492nttVSU1NdU7R3HhwoWoKtu2bcPlchEcHMz06dNp27YtGRkZvPfee3g8Hl5//XX69OnD9u3bKSwspHPnzjidTq6//nrvv5mxY8cC8PXXX+N0Ovn000+PWefw4cNcddVVxMbGkp6ejqpSXV1NVVVVgzrnn38+IkJISAhz5swhKSmJ0NBQFi1aRFRUFHPnzqVnz54N7hEUFMSoUaMAmDdvHtdccw3ff/89hw4dwul0smjRIkaOHMncuXPZsWMHSUlJJCYmcumll3Lw4EFvt3b79u0JDw8HcNb+v95nQIyIJAPlQCU1yeoS4CVV3UNNAloDXEpNwooRkeTadW7LVPVftf+ZHVTVuj8cHQB37X0+a1QnSFXnAajqgXp1PgIuFpFmV4uyBNZKjBs3DqfTedLKOx1CSLCTyJBg8tNjuH1AR5ZO+V+++tMQnrq2Oz/pmsyKon3cM+NbLrh/Af0fXMifZ61gweqdHKqoPOq5WX0ul4vLL78cqGnhr1+/3lt+xowZnHXWWUDN7tDPPPNMg/IAffv2Zdq0aYSGhrJ//34qKiro1asXixcv9s5Hqy8kJISFCxcyefJkhgwZAsCKFStQVSZNmkRJSQlt2rTxlt+5cyejRo3inXfeYceOHezfv5/CwkLCw8OJjIzk22+/5cknn2Tx4sVUVFSwcOFCEhIS6NOnD++++y5VVVUcPHiQkpIShg8fzqZNm1i3bp03xro6YWFhJCYmcuONN5KYmEh1dTUHDx6kU6dOTdbZs2cPSUlJhIWFcc899/Dggw9SUVHBli1byM3N9U4h2LhxI6GhoXz33Xfs3LmTp59+msTERCoqKigtLSUiIsKbyOvqvPXWW4gIv/3tb1m+fLk3EcfGxlJeXo6IsHTpUrZs2UJSUhJbt25l2bJlHDlyhH/9619s2LCBVatWcfDgQe8zRY/HwyeffMKMGTP4/vvvGTVqFGeffTapqals2rSJt99+mwcffJALL7zQm/AHDx7MJ598QkxMTIPy99xzD0VFRZSWlvLss8/SpUsX9u7di9PpZM6cOXz22WeEhIQwePBg0tPTueuuu5g9ezYi4p1mMWXKFAD27t3r/UMfHh5Obm4uY8aMoXPnzpSUlDBjxgyKiopYu3YtpaWlOBwOKioqvM9ob775Znbs2IHb7WbEiBEUFRUxc+ZMUlNTUVX27t1LTEwMixYtorq6usk6+/bt47PPPmPQoEF4PB5UlZSUFJYvX95knV27djFjxgyKi4sZO3YsO3fuJDg4mOTkZL766qsm73H48GH+/e9/k5qa6v3e27Rpw6JFi3A4HJSX1zwX37FjB1CzSe7Bgwe9ybCysrJuiouISN1/IFuoWQawE7AH8AA/B86vXaC9bonAXUDXxuVF5HUgGfhzbXlqz39XW/6oe4jI6yLylYg8UFdHVaup6XKsq9O04zXRWsvrdO5CbCnV1dW6bsc+ffajjTpm0ufa6Z7ZmnHXW9rxd7O11//M04SeQ7TLzQ/qxQ8t1Ese+VDjOnRRV0SMOoNdGhbr0UF3PKo3v/C5JnXsqm3Ss9QdHqltc3vphJc/0/9+c4XGJqaqyx2qwS63xsQn6i8ffF5f+GST5vToq9Fx8eoMClanM0iHj7tdn/1oo97858fVGRSsYZHRGuRya0h4pIZGxqg4nOoKi1BxODQ6zqM9e/fVyy67TDMzMzU8PFyjo6M1Pj5eXS6X95ldQkKCtmnTRoOCghTQs88+W91ut3bq1Em7dOmil1xyiXbq1EknTpyoIqIul0sTEhI0J6ezOpxODQoK0vDwCM1o1147ZHXUCwdcrC+99oZ2yOqov73nDyoi6nA6NSwsXC/sf7EuX7Ve3W63ulwubd++vfd5Xt2ozgULFni7KR9//HGtrq7W6667Tp1Op0ZGRuqRI0d03LhxOn36dH3nnXcU0Kf/8Yy2zcjQHj3P0fPOL9Blqzeow+HU1PR0vXjIUP126x7tdW6BPvbCvzU+IVFDQsN0yrufa26PPhrnSdKI6BjtlJuv0TFx6nA69boxY3Xu3LkaGxurGRkZmpWVpSKin332mWZmZmp8fLz3eQ2g8fHx+thjj+kDDzygPXr00OnTp+vs2bMV0NzcXF2yZIn27dtXo6Ki1OFwaMeOHXX06NH62GOPaVpamj7zzDPe8m3atFGn06kOh0OzsrJ0zZo1GhYW5p1qUXc91Zru0C5duni7w6ZMmaKq6o23/ijczMxMLS4u1t///vfqcrk0JSVFr7/+ek1KStIOHTqow+HQ9u3ba0REhN59993aoUMHXbp0qd56660aHBysGzZs0IyMDM3IyNDx48draGiohoeHa0pKik6YMME7BaBxnYiICO3atas+++yzumHDBhURbdeunc6aNavJOqmpqTpgwADt3bu3PvjggxoWFqaRkZGal5ent9xyS5P3CA8P18zMTB0/frxmZWVpcHCwtm3b1htXXFycFhcX61VXXaWRkZG6a9cub3f5WWedpX369Kl7Nl0F5GlN1918oAcwEthbm2yeAxYD44G3gbeAD4HLmiifSU0X4pvA+Nprvk3Nc67LjnGPTGrW5X2trk5tual1dY718udivqaVExGyEiLJSohk/PntKTtSxZLC3Xy8vpg9B4+wdeRYlr41hayCCzlSVc1PfvdPjlRVU1mlHKmu5khVNd+XlnH2fzzCkapq9m5ey9aPpjN7zV4qq0qpjO9EwlU34QyLBuCNXfDGrBVw8T3EUNPhXrFjA+99/gbLPCspXbCAhOsfxpXQnmCnEORwEOwUgp0Ogp0ORGDb3jK27tjA98vf5NZHZrFm5lP8+tab6NE9v8nvsbS0lG7dunHzzTcTFRXF+PHjuf3225k3bx5//etfueKKkYy86Zcs2lDCpxtK+HxTCTx8HdWV5QR1uRQKrqcS2FB+iBtu/TVRfa7klcretP3tmxxc+QFl33/DpnNuY+jTSyk/UoUjJILqqx6nHLht/kG2jruAxPOv5s6PjxCansPnO5VvZi9n1pEPWbtdqKqupkzc9LlvIWs+WM575Vlsf/cfOEKjuHdjCltLD1OakMah5R9x6cPzqa6uYlvpIfYUVTL0sY/ZsmodX935G8LPHoQWfsXv5+9i+84DOGIzOVKyhfKhf6V8+kSq9yxh1tpDfP5BFdXxmaT+ZDyp6e3Yee945pdEcahSiE/LJCoqkgn3PcFV/fJwhUexamcZ3yxdxCEN4stiiE2IwxUSxq59ZUz5tJDcy3/OzsNPUbFuBaGJmXy4+CscqblEJbdns7bhiHpwh4Yx7q7/od/gYfz1lzewa3sRG/c7CHaHcOjgAW674y7+6447CHM5qa6uacl0796doUOHcs0117B582ZvywagoKCgpqV0qJzSPXvZUR6EK74t8YkpHNi/j90SRWRye9LP7s7moufZve8gVTh564PP2VRYyMBLL6Ps0AGqqqq5878f4vCRKtp3ymHRkmU122Yo7N23nwPOCGbOXUBh4Xdc8pNhHDpYU+f39z0CDifhbZJ46/1PSMnvB0DJ7lK2H6hmzvsfUvjdd1w2bDgHDuyvaZHt309CUjIdIiJYuXLl/2/v3KOjqq89/tkzmZiQREAINBDeEh6iIApNay0KVmllqddbfFFA67Kl14tWxWqrV62r9loFb7lKK5RXrdjCtS1trfVZfEAVwYAgRUWQWEFN5P0MeXzvH+dMOqQBkpY4zMz+rDVrzu93fo+9Jydnn9/j7E0kEsEswo4dO1i3bh3l5eVcdNFF7NoVlJ8+fToAgwcPZunSpdTV1RGNRtm1axdFRUUsWbKk3iXcli1bqKqqYvLkyWRlZdG3b19GjRrFzTffHJ+2F7Av/JeIu/yLASuA7sAmgvWpwQQGrRTYGtY5qLykDeGU9JNh+VlhnUGH6kPShvB+szBse1ZYLiehTqO4AXOaTE4sypm9Czmzd9xx8inMLtzL+MsHNXF6czizZ2czfvyIoPzdI5FEbZ2oqRMH4savNjB+EsSiI5g/7zjGjhtBzt3nEYtGyIpY/bpNQz7csY/n1w7g4RkVPPrKe1S3/zKrf/cxX67DbTsAAA93SURBVFxbxoh+HTi7Twfa5mXXl49Pbebl5TF+/HiqqqpYvnIVp503mhcO9OK/73mOrXsOANCtXStGnVLER3nHsW9PNT+47SbatOvA228s48HvXcvosRMYednV9W3vOqszt437Mt8d3oXaujqunVrDZ784gsu/VMKyRU/yhzn3MeLCsQwZdQUHauuoyI1x8pDP88qvp9Ptq18ne/BQyp/7Bf2GDmNo+2re2vo+W/84hXMvuYY1S57lir5Z/DQ3xsDifF79637OyilnQSybmj1buXnCWFq1qua/9m+ne68SZs6czNcvHM70y/rwnZdaEzHjncrdzL2sN998oop3EBOvHkN5+Xrm/+2v9B5wKtv21qC8dsx8YjFba2J8vGYV+Sefw3XzV1JTU8vmTZv4zfo6aPd5PvnLZB5dH6Nu7Uaqa2rYsucA819dT9X6pVhOJ6pZy6rXXiKroD1Prt7M9lUreGjB0+QNGM6B6hrmvrqJ+ZXLqNy0iwMVlXzjl2+yY2cQTWnWumx+OnoS1Vs3ccLZV5Nds5f3//AsK9ucQa2Me6fNYu6fV6PiU9GGDTyx+iNOvO1PfPzHqexXDl/538VUb4+wefMmIjn5LDtQTMXqmezpNYKaOmPHlgrIKaCix0gia9dS2/MMctt2omrRHJYVnMGW3Y+zs3wL1ZUbqasRFjXq9u7i11s6s2/nOpRTQKvxM6hd9Qz7Fs3h5ZxS9tQ8yl8Wv0TbM7/G0ul/RhK7du3mrsU72PVBNbRqS/bY6cRWPYMWzWGvsnl6Wwe2L5xD689dwp7qOlS1l921EfYNORMK3mJncSlZbTuhRXN4bN9A9h6o5Y8rP2B/+bucePYlZH+ykO3bt3PxxRdTXl5OSUkJa9euZdq0adx1113ccccdPPLIIyxfvpwbbriBlStXxqeSI8BGM7sFKJD0oZlVAG3Dc08B1wLfJwhxNRZYT7CutSOxvJnFbw5nAkvC498DVwBrzKy0sTqSKoHhBOtqcUqANRyOww3PUunjU4hOQ/ZUVeuZNR/plsff0Ok/eFbdbnlCPW59Qv/+kyX6yaJ39c5HO1VXV6dLx4zTbQ/N07d/tUI9R98iIlHFOvRQq6JeKuxWovvmPaUPtu3V1KlT1blzZ0WjUeXm5mrkyJGSpIkTJ8rMDvIOsWLFCtXW1urEE0+s365dWFiohQsXSgqmtiKRSP3rCbFYrH6XXG5ubv326/z8fHXs2FElJSUqLCys7yfc/ly/lbq0tFQ5OTnKzc1Vdna2evbsWb/lvXXr1ho4cKA6duxY/4pE/FWFWCymtm3b1m8LLyoqUnFxcf103ZQpUxSLxerrHJeToy7deujUIaXKLyhQl27dlduqlTp+pkgnDThZXbp2VX643TwaTrX2799fbdq0UZ8+fZSVlaVOnTopv6BAxcVd1K//AE286TsaMPBUndinn/ILjtfxrdsoKxZTLJat1ie00y9f2aBhF1ymSDSqop591LF7iXoM+pzyT+ggzBTLaSXMlNeuSMUnDVXXfoPVrqirIlkx5R/fRj1K+qtdh47KCbe3x7epx7fQd+7cWa3y8tT/pJPUtWs3XX/jJP142gz16du/foo5Gs1SVlbwygFmimZlqXPX7mpX2EGf6VSsPy1dozvvf0i9SvopGtYBZBZRLBZT+w6fUV5+gYq6dFM0K0v/Nv5benz533TGeRcqln1cfbsWiSiaFZNZRAVtgmn0ou4lyju+rc7/+o26fNK9at+pqyLRoI/YcTnKyj5OJ3QoUkFBgXJyctSrVy8VFxerR48e2rRpk+bMmaOSkpJ6XcxMkUhE2dnZGjdunAg2aqwPv2crmL67jmCdS0A18ElY5m0C94D7CXYmLgE6hXWmEexoFMF2+nhbHQm24a8P65yuv08Tfolga/1qYC6QnVDnNR3hvp90w3O0Pm7AnMNRW1unle9v05Sn39JXpr5U//rAyXc+paIrpyrvpLM1+O5ndO281zXv1XJtqNx9kLeShiS+2hB/heBIJNYZM2aMKioqmlWnKf1MnjxZAwYMqE83pc4DDzygmTNnNrl8Y/0cibKyMg0bNky33367JOn8889XVVVVk/v4Z36vptQpKytTnz599NxzzzW5zo033qjS0tJmyZVYpym6l5WV6bTTTqv/u7SUXNddd52AtxUYjUeBQh3JaMANJKxVHe3yzamTdMNztD5uwJzmsHn7Xv3ilY2atGClZi/eoHsemKbq6upmtTFr1qxGXSMls86CBQv04IMPNquP2bNnN1v3f6afCRMmHORTsyX6kJr/G48bN67Zv9eMGTNavM5VV11V79arpfqYMWNGs11JEbwLltVS5ZtTx8LCKc/pp5+uhoEQHcdxnMNjZq9LOj3Zcvwz+HtgjuM4TkriBsxxHMdJSdyAOY7jOCmJGzDHcRwnJXED5jiO46QkbsAcx3GclMQNmOM4jpOSuAFzHMdxUpK0eZHZzCoJYs4cjvYEPr0ylUzWP5N1h8zW33U/PN0kFR6hzDFJ2hiwpmBmy1P1jfOjQSbrn8m6Q2br77qnr+4+heg4juOkJG7AHMdxnJQk0wzYjGQLkGQyWf9M1h0yW3/XPU3JqDUwx3EcJ33ItBGY4ziOkyZkjAEzs5Fm9raZvWtmtyZbnpbGzGabWYWZvZmQd4KZPWtm68LvtsmUsaUwsy5mtsjM1prZGjO7PsxPe/3NLMfMXjOzN0Ldvx/m9zCzpaHu880sO9mythRmFjWzFWb2RJjOJN03mtlqM1tpZsvDvLS97jPCgJlZFJgGfBnoD1xuZv2TK1WLMxcY2SDvVuB5Sb2B58N0OlID3CSpH1AKXBv+vTNB/ypguKSBwCBgpJmVAj8C/ifUfRtwdRJlbGmuB9YmpDNJd4CzJQ1K2D6fttd9RhgwYCjwrqQNkg4AvwIuTLJMLYqkl4CtDbIvBH4eHv8cuOhTFepTQtKHksrC410EN7POZID+YaT43WEyFn4EDAceD/PTUncAMysGzgdmhmkjQ3Q/DGl73WeKAesM/C0h/UGYl2l0lPQhBDd5oEOS5WlxzKw7cCqwlAzRP5xCWwlUAM8C64HtkmrCIul8/f8Y+A5QF6bbkTm6Q/Cw8oyZvW5m3wjz0va6z0q2AJ8S1kieb79Mc8wsH/g18G1JO4OH8fRHUi0wyMzaAL8F+jVW7NOVquUxs1FAhaTXzeyseHYjRdNO9wTOkLTZzDoAz5rZW8kWqCXJlBHYB0CXhHQxsDlJsiSTj82sCCD8rkiyPC2GmcUIjNc8Sb8JszNGfwBJ24EXCNYB25hZ/IE1Xa//M4ALzGwjwTLBcIIRWSboDoCkzeF3BcHDy1DS+LrPFAO2DOgd7kbKBi4Dfp9kmZLB74Hx4fF44HdJlKXFCNc9ZgFrJT2QcCrt9TezwnDkhZnlAucQrAEuAr4aFktL3SV9V1KxpO4E/+N/ljSGDNAdwMzyzKwgfgycC7xJGl/3GfMis5l9heBpLArMlnRPkkVqUczsl8BZBN6oPwbuBBYCC4CuwPvAaEkNN3qkPGb2BeBlYDV/Xwv5HsE6WFrrb2anECzURwkeUBdIutvMehKMSk4AVgBfk1SVPElblnAKcZKkUZmie6jnb8NkFvCYpHvMrB1pet1njAFzHMdx0otMmUJ0HMdx0gw3YI7jOE5K4gbMcRzHSUncgDmO4zgpiRswx3EcJyVxA+YkHTOTmU1JSE8ys7uOUttzzeyrRy75L/czOvR+v6hBfvdQv4kJeQ+Z2ZVHaG+CmY07QpkrzeyhQ5zb3Vj+0SLUKzHSwTVmVpZOns6dYx83YM6xQBVwsZm1T7YgiYRRDJrK1cB/SDq7kXMVwPXNCeMh6WFJjzSj/6NGgteKppYfC0wEzpW0rWWkcpx/xA2YcyxQQxD6/IaGJxqOoOIjCzM7y8xeNLMFZvaOmd1rZmPCWFirzaxXQjPnmNnLYblRYf2omd1vZsvMbJWZfTOh3UVm9hjBi9AN5bk8bP9NM/tRmHcH8AXgYTO7vxH9KgnCWIxveMLMepnZU6Hz1ZfNrG+Yf5eZTQqPh4QyvhLK/GZCE53C+uvM7L4GbU8JR0XPm1lhmDfIzF4N2/ttfMRkZi+Y2Q/N7EUCYzs61PENM3upEZ3ifVxCEJ7jXEmfHKqc47QEbsCcY4VpwBgza92MOgMJYj+dDIwFSiQNJQilMTGhXHdgGEGYjYfNLIdgxLRD0hBgCHCNmfUIyw8FbpN0UMw4M+tEEFtqOEGsrSFmdpGku4HlwBhJNx9C1nuBmxoZ1c0AJko6DZgE/KSRunOACZI+B9Q2ODcIuDT8DS41s7jPzzygTNJg4EUCTywAjwC3SDqFwEDfmdBWG0nDJE0B7gDOC+OKXXAInboBDxEYr48OUcZxWgw3YM4xgaSdBDfX65pRbVkY+6uKIGTIM2H+agKjFWeBpDpJ64ANQF8CP3HjLAg7spQg7EbvsPxrkt5rpL8hwAuSKsPwHPOALzZRv/eA14Ar4nkWeMv/PPB/oRzTgaLEeqFfwwJJfwmzHmvQ9POSdkjaD/yVwKhA4EJrfnj8KPCF8OGgjaQXw/yfN5B/fsLxEmCumV1D4JaqMSoJXBNdckjFHacFyZRwKk5q8GOgjGDEEaeG8EErdNKbuI6U6M+uLiFdx8HXdkN/aSIIszFR0tOJJ0IfensOId+/Go/lhwSBFeNTchGCWFWDDlPnSH0m/ga1HPp/uik+4+r1ljTBzD5LMGpdaWaDJG1pUH4vQZTzxWZWIWleE/pwnKOGj8CcY4bQwegCDg75vhE4LTy+kCDCcHMZbWaRcF2sJ/A28DTwLQvCrmBmJaEH78OxFBhmZu3DqcDLCabnmoSktwhGSaPC9E7gPTMbHcpgZjawQZ1twC4zKw2zLmtidxH+7oH9CmCxpB3ANjM7M8wfeyj5zayXpKWS7gA+4eBwRInyVQIjgR+a2XlNlM1xjgo+AnOONaYA/5mQ/hnwOzN7jWAjxKFGR4fjbYIbdUeCtaT9ZjaTYJqxLBzZVXKEUOuSPjSz7xKE5zDgSUnNDU1xD4FH9DhjgJ+a2e0ExvlXwBsN6lwN/MzM9hDE99rRhH72ACeZ2eth+UvD/PEE64CtCKZTrzpE/fvNrDeBns83IlM9kt4zswuAJ83sYklLmyCf4/zLuDd6xznGMbN8SfHdl7cCRZKuT7JYjpN0fATmOMc+54cjvyygHLgyueI4zrGBj8Acx3GclMQ3cTiO4zgpiRswx3EcJyVxA+Y4juOkJG7AHMdxnJTEDZjjOI6TkrgBcxzHcVKS/wffV09DTbd8sQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.158 0.122 0.112 0.108 0.108 0.108 0.107 0.107 0.107 0.107 0.106 0.106\n",
      " 0.106 0.106 0.106 0.107 0.106 0.106 0.106 0.106 0.106 0.106 0.106 0.106\n",
      " 0.106]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.393\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_optimal.fit(d_train,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.998\n",
      "The value of TNR is 0.047\n",
      "The value of FPR is 0.953\n",
      "The value of FNR is 0.002\n",
      "\n",
      "Test error is 10.733\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGFdJREFUeJzt3Xu8VVW99/HPd7NRLirgBTVAwcDr8RENjUo7Kj54PVLHOMeOJ9GsbeW10vLyqA+IT1RqZqUnfMRA8355SVopYqaeowkiXkg74JUtBgpIBWjuvX/njzXAJe619kL23mvtwffda77WmmONOcdYvfC3xv7NMcdURGBmZl1fXbU7YGZm7cMB3cwsEw7oZmaZcEA3M8uEA7qZWSYc0M3MMuGAbmaWCQd0M7NMOKCbmWWivtodKKV+kwG+hdU+ZNd+g6rdBatBzy1+XBt6jvfeeqnimNN96502uL2O4BG6mVkmanaEbmbWqVqaq92DDeaAbmYG0NxU7R5sMAd0MzMgoqXaXdhgDuhmZgAtDuhmZnnwCN3MLBO+KGpmlgmP0M3M8hAZzHLxjUVmZlC4KFrp1gZJr0h6VtJcSbNT2ZaSZkian177pXJJulLSAknPSNqn6DzjUv35ksa11a4DupkZFFIulW6VOSgihkfEiLR/DjAzIoYBM9M+wOHAsLQ1AFdD4QcAuAj4JLAfcNGaH4FSHNDNzKBwUbTS7aMZA0xN76cCnysqnxYFjwN9JW0PHArMiIhlEbEcmAEcVq4BB3QzM2jvEXoA90t6UlJDKts2It4ASK/9U/kAYGHRsY2prFR5Sb4oamYG63XrfwrSDUVFkyNictH+ZyJikaT+wAxJL5Q7XStlUaa8JAd0MzNYrztFU/CeXObzRel1iaS7KOTAF0vaPiLeSCmVJal6I1C8LvRAYFEqP3Cd8ofK9cspFzMzIKK54q0cSb0lbb7mPTAaeA6YDqyZqTIOuDu9nw4cn2a7jARWpJTMfcBoSf3SxdDRqawkj9DNzKA9byzaFrhLEhRi7I0R8VtJs4BbJZ0EvAaMTfV/DRwBLABWAScCRMQySRcDs1K9CRGxrFzDDuhmZtBui3NFxEvAXq2ULwVGtVIewCklzjUFmFJp2w7oZmbgW//NzLLR/F61e7DBHNDNzMDroZuZZcMpFzOzTHiEbmaWCQd0M7M8hC+Kmpllwjl0M7NMOOViZpYJj9DNzDLhEbqZWSY8Qjczy0RT5Q+4qFUO6GZm4BG6mVk2nEM3M8uER+hmZpnwCN3MLBMeoZuZZcKzXMzMMhFR7R5sMAd0MzNwDt3MLBsO6GZmmfBFUTOzTDQ3V7sHG8wB3cwMnHIxM8uGA7qZWSacQzczy0O0eB66mVkenHIxM8uEZ7mYmWXCI3Rrb5tuuikPPXgHm2y6KfX13bjzznsZP+EyvvH1Ezj9tK8wdOgQtt3+H1i6dHm1u2odbPMtNmP85ecxdNedIOCCb07kgFGf5uDDPktLSwvL3lrO+adfzJuL3+Kgww7gtO+eTEtLC81NzUy64AqeeuLpan+FriWDgK6o0QVp6jcZUJsd6wS9e/di5cpV1NfX8/BDd/HNb13Eu39/l+XLVzBzxu188lOHb7QBfdd+g6rdhU5zyZUXMOcPT3PHL6dT372enj170NLSwsq/rQLguK/8Cx/feTATvvMDevbqyepVqwHYefehXDp5Ikfvf2w1u9+pnlv8uDb0HKuuOLnimNPrzJ+32Z6kbsBs4PWIOErSEOBmYEtgDvCliPi7pE2BacAngKXAv0bEK+kc5wInAc3A6RFxX7k2O2yELmlXYAwwAAhgETA9Ip7vqDZzsXJl4T/Y7t3rqe/enYhg7tx5Ve6Vdabem/XiE5/am/NPvxiApvea+Ot7f/tAnZ69eqxdIHBNMF9TzkY7HNoA7T9CPwN4Htgi7X8f+FFE3CzpPygE6qvT6/KIGCrp2FTvXyXtDhwL7AF8DHhA0s4RUTLZX9fe3wBA0ncp/BIJeAKYld7fJOmcjmgzJ3V1dcyedT9vvP4MM2c+zBOznqp2l6yTDdxxAMuXLmfijy/gtgemMv7y8wqBGjj93K/xwJy7OfKYQ/npDyavPWbU4f/I9Edv5qobLuOCb06sVte7rpaofGuDpIHAkcD/T/sCDgZuT1WmAp9L78ekfdLno1L9McDNEfFuRLwMLAD2K9duhwR0Cr84+0bEpIi4IW2TUmdO6qA2s9HS0sKIfUez45AR7Dtib/bYY5dqd8k6WX19N3bbcxdumXonYw8Zx+pVqznptOMBuPJ7/8Eh+4zh3jvu49++/IW1x8z8ze85ev9jOf2E73Lqd0+uVte7rubmyre2XQF8B1gz7N8KeDsi1jxFo5FC9oL0uhAgfb4i1V9b3soxreqogN5C4U+EdW3P+1/wQyQ1SJotaXZLy8oO6lrXsWLFX/j9w//FoaMPrHZXrJP9edESFi96k2fnFFJt9//qQXbf84M/7PfeeT+HHHXQh4598vG5DBo8gL5b9umUvuYiWloq3opjVdoa1pxH0lHAkoh4suj0reXco43Pyh3Tqo7KoZ8JzJQ0n/d/YXYAhgKnljooIiYDk2HjvSi69dZb8t57TaxY8Rd69OjBqIMP4IeXXlXtblknW/rmMv68aDGDP74Dr7z4GiMP2JcX//tldhgyiNdeLvwnddChB/Dy/FcBGDR4IAtfaQRgtz13oXv3et5etqJq/e+S1uNO0eJY1YrPAEdLOgLoQSGHfgXQV1J9GoUPpHBdEQoj70FAo6R6oA+wrKh8jeJjWtUhAT0ifitpZwoplgEUfmkagVnlEvoG22+/LVOuvYJu3eqoq6vj9tt/xb2/foBTT/kyZ337G2y33TY89eQD/Oa3D3Ly186udnetA/2/8y7j+1eNp/sm3Vn46utccMZExl9+HoOH7kC0BIsa/8yEs78PwP8+6iCOHns4TU1NvPPOu5zVcEGVe98FtdNaLhFxLnAugKQDgbMi4jhJtwFfoHB9cRxwdzpketp/LH3+YESEpOnAjZIup5DxGEbhmmRJnrZoXcrGNG3RKtce0xZXTjiu4pjT+8JfVtReUUA/StJOvD9t8Sng3yPiXUk9gOuBvSmMzI+NiJfS8ecDXwaagDMj4jfl2vONRWZmAE3tnzyIiIeAh9L7l2hllkpEvAOMLXH8JcAllbbngG5mBl4+18wsG14+18wsD5HBWi4O6GZm4BG6mVk2HNDNzDLhB1yYmeXBzxQ1M8uFA7qZWSY8y8XMLBMeoZuZZcIB3cwsD9HslIuZWR48Qjczy4OnLZqZ5cIB3cwsE10/he6AbmYGEE1dP6I7oJuZgUfoZma58EVRM7NceIRuZpYHj9DNzHLhEbqZWR6iqdo92HAO6GZmQHiEbmaWCQd0M7M8eIRuZpYJB3Qzs0xEs6rdhQ3mgG5mhkfoZmbZiBaP0M3MsuARuplZJiK6/gi9rtodMDOrBdFS+VaOpB6SnpD0tKR5ksan8iGS/iBpvqRbJG2SyjdN+wvS54OLznVuKv+TpEPb+g4O6GZmQEuzKt7a8C5wcETsBQwHDpM0Evg+8KOIGAYsB05K9U8ClkfEUOBHqR6SdgeOBfYADgOuktStXMMO6GZmFC6KVrqVPU/B39Ju97QFcDBweyqfCnwuvR+T9kmfj5KkVH5zRLwbES8DC4D9yrXtgG5mxvoFdEkNkmYXbQ3F55LUTdJcYAkwA3gReDti7RJgjcCA9H4AsBAgfb4C2Kq4vJVjWuWLomZmQKzHcugRMRmYXObzZmC4pL7AXcBurVVLr60N+aNMeUkO6GZmdMw89Ih4W9JDwEigr6T6NAofCCxK1RqBQUCjpHqgD7CsqHyN4mNa5ZSLmRmFaYuVbuVI2iaNzJHUEzgEeB74HfCFVG0ccHd6Pz3tkz5/MCIilR+bZsEMAYYBT5Rru+IRuqRNI+LdSuubmXUlze23lsv2wNQ0I6UOuDUi7pH0R+BmSROBp4BrU/1rgeslLaAwMj8WICLmSboV+CPQBJySUjkltRnQJe2XGuwD7CBpL+ArEXHaR/iiZmY1qb1uLIqIZ4C9Wyl/iVZmqUTEO8DYEue6BLik0rYrSblcCRwFLE0NPA0cVGkDZmZdQXtNW6ymSlIudRHxamFa5Fplh/1mZl3N+sxyqVWVBPSFKe0SKSd0GvDfHdstM7POVcsj70pVEtC/TiHtsgOwGHgglZmZZaO5petP+mszoEfEEtJVVzOzXG0UKRdJ19DK3UkR0dBKdTOzLqklg+VzK0m5PFD0vgfweT64voCZWZeXw3rolaRcbinel3Q9hcVmzMyysVGkXFoxBNixvTtiVomn5t1Y7S5YpjaKlIuk5byfQ6+jcGvqOR3ZKTOzzpb9LJe0yPpewOupqCUtGmNmlpUcAlvZn6QUvO+KiOa05fCdzcw+pCVU8VarKvkb4wlJ+3R4T8zMqqi9ls+tppIpl6KF2PcHvirpRWAlhadoREQ4yJtZNlqq3YF2UC6H/gSwD+8/yNTMLFvR6hPfupZyAV0AEfFiJ/XFzKxqmmo4lVKpcgF9G0nfKvVhRFzeAf0xM6uK3Efo3YDNaP3J02ZmWck9h/5GREzotJ6YmVVR7iP0rv/tzMwqlPsIfVSn9cLMrMqaMxjDlgzoEbGsMztiZlZNGTyB7iOttmhmlp2WnEfoZmYbkxwWqnJANzMj/4uiZmYbjRY55WJmloXmanegHTigm5nhWS5mZtnwLBczs0x4louZWSaccjEzy0QO0xYreaaomVn2mlX5Vo6kQZJ+J+l5SfMknZHKt5Q0Q9L89NovlUvSlZIWSHqm+BnOksal+vMljWvrOzigm5lRGKFXurWhCfh2ROwGjAROkbQ7cA4wMyKGATPTPsDhwLC0NQBXQ+EHALgI+CSwH3DRmh+BUhzQzcxov4AeEW9ExJz0/q/A88AAYAwwNVWbyvvPax4DTIuCx4G+krYHDgVmRMSyiFgOzAAOK9e2A7qZGRCqfJPUIGl20dbQ2jklDQb2Bv4AbBsRb0Ah6AP9U7UBwMKiwxpTWanyknxR1MyM9bsoGhGTgcnl6kjaDLgDODMi/qLSSwu09kGUKS/JI3QzMwq3/le6tUVSdwrB/JcRcWcqXpxSKaTXJam8ERhUdPhAYFGZ8pIc0M3MKMxDr3QrR4Wh+LXA8xFxedFH04E1M1XGAXcXlR+fZruMBFaklMx9wGhJ/dLF0NGprCSnXMzMaNd56J8BvgQ8K2luKjsPmATcKukk4DVgbPrs18ARwAJgFXAiFJ4aJ+liYFaqN6GtJ8k5oJuZ0X4BPSIepfX8N7TyrOaICOCUEueaAkyptG0HdDMzvJaLmVk2vJaLmVkm/IALM7NMtGSQdHFANzMjj9UWHdDNzPBFUTOzbHiEbmaWiSZ1/TG6A7qZGU65mJllwykXM7NMeNqimVkmun44d0A3MwOccjEzy0ZzBmN0B3QzMzxCNzPLRniEbmaWB4/Qrd1dM/kyjjziEJa8+RbD9y483OSYY47iwgu+xW67DuNTnz6SJ+c8U+VeWkcZfcw4evfqRV1dHd26dePWKVfyk8nTePDRx6hTHVv268Ml53+b/ttsxT33Pci1v7wNgF49e3LBWaey67CdSp7HyvO0RWt306bdylVXXcd11/14bdm8eS8w9l++ytU/m1TFnllnmfKTSfTr22ft/onHHcNpDccDcMNtd3P1dTdy0XdOY8DHtuMXP/0BfbbYnEcem8X4H1zJTddcUfI8Vl7XD+cO6DXnkUf/wI47DvxA2QsvLKhSb6wWbNa799r3q1e/g9KTdfbec/e15f9rj11ZvOStzu5aVpoyCOmdHtAlnRgR13V2u2ZdgSQavnk+khg75nDGjjkCgB///BdM/+1MNu/dmyk/+fBfanfecx/7jxzR5nmsNF8U/WjGA60GdEkNQAOAuvWhrq53a9XMsnX91ZfRf5utWLr8bb565nkM2XEQI4bvyRknn8AZJ5/ANdNu4cY7fsWpX/nS2mOeePJp7rznfq6/+tI2z2Ol5XBRtK4jTirpmRLbs8C2pY6LiMkRMSIiRjiY28ao/zZbAbBVv76M+uynefaPf/rA50eOPpAHHvrPtft/WvAyF066gp9MupC+fbao+Dz2YbEe/6tVHRLQKQTt44F/amVb2kFtmnVpq1a/w8qVq9a+/68n5jBsp8G8uvD1tXV+98jjDEnXWN748xLOPO9ivnfh2QzeYWCb57HyWtZjq1UdlXK5B9gsIuau+4GkhzqozSzccP3P+MfPfoqtt96SV16azfgJl7Js+dv8+EcT2WabLZl+9zSefnoeRxx1XLW7au1s6bLlnHHexQA0NzVzxOgD2X/kCM48byKvvNaI6sTHtuvPhWefBsDV193Iir/8lYmX/gxg7fTEUuex8pqjdkfelVLU6Jeo32RAbXbMqmr1okeq3QWrQd233kkbeo5/2/HzFcecG1+9a4Pb6wietmhmhme5mJllo5Zz45VyQDczw7f+m5llwykXM7NM5DDLxQHdzIw8Ui4ddWORmVmX0p43FkmaImmJpOeKyraUNEPS/PTaL5VL0pWSFqQ76vcpOmZcqj9f0ri22nVANzOj3W/9/wVw2Dpl5wAzI2IYMDPtAxwODEtbA3A1FH4AgIuATwL7ARet+REoxQHdzIxCyqXSrS0R8TCwbJ3iMcDU9H4q8Lmi8mlR8DjQV9L2wKHAjIhYFhHLgRl8+EfiAxzQzcyAiKh4k9QgaXbR1lBBE9tGxBuprTeA/ql8ALCwqF5jKitVXpIvipqZAc3rcVE0IiYDk9up6daWEYgy5SV5hG5mRvumXEpYnFIppNclqbwRGFRUbyCwqEx5SQ7oZmasX8rlI5oOrJmpMg64u6j8+DTbZSSwIqVk7gNGS+qXLoaOTmUlOeViZkb7zkOXdBNwILC1pEYKs1UmAbdKOgl4DRibqv8aOAJYAKwCTgSIiGWSLgZmpXoTImLdC60f4IBuZkb73vofEV8s8dGoVuoGcEqJ80wBplTargO6mRm+9d/MLBs53PrvgG5mhgO6mVk2avVxnOvDAd3MDI/Qzcyy4QdcmJllojm6/lNFHdDNzHAO3cwsG86hm5llwjl0M7NMtDjlYmaWB4/Qzcwy4VkuZmaZcMrFzCwTTrmYmWXCI3Qzs0x4hG5mlonmaK52FzaYA7qZGb7138wsG77138wsEx6hm5llwrNczMwy4VkuZmaZ8K3/ZmaZcA7dzCwTzqGbmWXCI3Qzs0x4HrqZWSY8Qjczy4RnuZiZZcIXRc3MMuGUi5lZJnynqJlZJjxCNzPLRA45dOXwq5Q7SQ0RMbna/bDa4n8Xtq66anfAKtJQ7Q5YTfK/C/sAB3Qzs0w4oJuZZcIBvWtwntRa438X9gG+KGpmlgmP0M3MMuGAXuMkHSbpT5IWSDqn2v2x6pM0RdISSc9Vuy9WWxzQa5ikbsDPgMOB3YEvStq9ur2yGvAL4LBqd8JqjwN6bdsPWBARL0XE34GbgTFV7pNVWUQ8DCyrdj+s9jig17YBwMKi/cZUZmb2IQ7otU2tlHlakpm1ygG9tjUCg4r2BwKLqtQXM6txDui1bRYwTNIQSZsAxwLTq9wnM6tRDug1LCKagFOB+4DngVsjYl51e2XVJukm4DFgF0mNkk6qdp+sNvhOUTOzTHiEbmaWCQd0M7NMOKCbmWXCAd3MLBMO6GZmmXBAt3YnqVnSXEnPSbpNUq8NONeBku5J748ut+KkpL6SvvER2vi/ks76qH00qxUO6NYRVkfE8Ij4B+DvwNeKP1TBev/bi4jpETGpTJW+wHoHdLNcOKBbR3sEGCppsKTnJV0FzAEGSRot6TFJc9JIfjNYuwb8C5IeBf55zYkknSDpp+n9tpLukvR02j4NTAI+nv46+GGqd7akWZKekTS+6Fznp3XmHwB26bT/N8w6kAO6dRhJ9RTWcn82Fe0CTIuIvYGVwP8BDomIfYDZwLck9QCuAf4JOADYrsTprwR+HxF7AfsA84BzgBfTXwdnSxoNDKOwDPFw4BOSPivpExSWUdibwg/Gvu381c2qor7aHbAs9ZQ0N71/BLgW+BjwakQ8nspHUnhox39KAtiEwu3suwIvR8R8AEk3AA2ttHEwcDxARDQDKyT1W6fO6LQ9lfY3oxDgNwfuiohVqQ2vj2NZcEC3jrA6IoYXF6SgvbK4CJgREV9cp95w2m+JYAHfi4ifr9PGme3YhlnNcMrFquVx4DOShgJI6iVpZ+AFYIikj6d6Xyxx/Ezg6+nYbpK2AP5KYfS9xn3Al4ty8wMk9QceBj4vqaekzSmkd8y6PAd0q4qIeBM4AbhJ0jMUAvyuEfEOhRTLvemi6KslTnEGcJCkZ4EngT0iYimFFM5zkn4YEfcDNwKPpXq3A5tHxBzgFmAucAeFtJBZl+fVFs3MMuERuplZJhzQzcwy4YBuZpYJB3Qzs0w4oJuZZcIB3cwsEw7oZmaZcEA3M8vE/wBvDyY6WLrMrwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [18.4.7] KNN (KD-Tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The optimal number of neighbors is 21.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEKCAYAAABzHwA5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4lOW5+PHvPZPMZN/IZA+BEIiGBMIii0tQkEWpYBEVXEBBPa312FNrK9aunJ6jP7e61KVWVKQoVhRBRRAR3FABFVT2LQqELSHsJCHJ/fsjyZwkhDAiQyZwf65rrsy87/O8750Ec/s877OIqmKMMca0No6WDsAYY4w5EZbAjDHGtEqWwIwxxrRKlsCMMca0SpbAjDHGtEqWwIwxxrRKlsCMMca0Sn5NYCIyRETWiMh6EZnQxPkCEflSRCpFZGSjc21F5F0RWSUiK0WknT9jNcYY07r4LYGJiBN4ArgEyAFGi0hOo2LfAzcALzVxiReBB1T1bKAXsNNfsRpjjGl9gvx47V7AelXdCCAi04DhwMq6AqpaWHuuun7F2kQXpKrzassdON7N4uPjtV27dicrdmOMOSN88cUXxarqaek4ToQ/E1gqsLne5y1Abx/rdgL2iMjrQHvgPWCCqlbVLyQitwC3ALRt25alS5f+6KCNMeZMIiLftXQMJ8qfz8CkiWO+LrwYBFwA3AmcA2RS09XY8GKqz6hqT1Xt6fH8+P+BOHz4MP369aOqqiZPDhkyhJiYGH7yk5/4VL+8vJyrr76arKwsevfuTWFhYZPlxo0bR0JCArm5uQ2O//nPfyY1NZX8/Hzy8/OZPXs2AEeOHGHs2LHk5eVx9tlnc++99wJQUVFBQUEBlZWVJ/gdG2NM6+XPBLYFSK/3OQ0o+gF1v1LVjapaCbwBdD/J8R3lueeeY8SIETidTgB+85vfMGXKFJ/rT5o0idjYWNavX8+vfvUr7rrrribL3XDDDcyZM6fJc7/61a9YtmwZy5Yt49JLLwXg1Vdfpby8nG+++YYvvviCf/zjHxQWFuJyuRgwYACvvPLKD/xOjTGm9fNnAlsCdBSR9iLiAkYBs35A3VgRqWtW9afeszN/mTp1KsOHD/d+HjBgAJGRkT7XnzlzJmPHjgVg5MiRzJ8/n6ZW+y8oKCAuLs7n64oIBw8epLKyksOHD+NyuYiKigLg8ssvZ+rUqT5fyxhjThd+S2C1LafbgLnAKuDfqrpCRCaKyDAAETlHRLYAVwL/EJEVtXWrqOk+nC8i31DTHflPf8UKNd1xGzdu5McMBNm6dSvp6TWNzqCgIKKjoykpKflB1/j73/9Oly5dGDduHKWlpUBNMgwPDyc5OZm2bdty5513ehNgbm4uS5YsOeGYjTGmtfLrPDBVna2qnVS1g6r+T+2xP6rqrNr3S1Q1TVXDVbWNqnauV3eeqnZR1TxVvUFVK/wZa3FxMTExMT/qGk21tkSaehTYtJ///Ods2LCBZcuWkZyczK9//WsAFi9ejNPppKioiE2bNvHQQw+xceNGAJxOJy6Xi/379/+o2I0xprWxlThqhYaGUlZW9qOukZaWxubNNQMvKysr2bt37w/qKkxMTMTpdOJwOLj55ptZvHgxAC+99BJDhgwhODiYhIQEzjvvvAYjLsvLywkJCflRsRtjTGtjCaxWbGwsVVVVPiWxu+++mxkzZhx1fNiwYUyePBmA6dOn079//x/UAtu2bZv3/YwZM7yjFNu2bcv777+PqnLw4EE+++wzzjrrLABKSkrweDwEBwf7fB9jjDkdnPEJbF/ZER55by3LN+9h0KBBfPzxx95zF1xwAVdeeSXz588nLS2NuXPnAvDNN9+QlJR01LXGjx9PSUkJWVlZPPzww9x3330AFBUVeUcUAowePZq+ffuyZs0a0tLSmDRpEgC//e1vycvLo0uXLixYsIC//e1vAPziF7/gwIED5Obmcs4553DjjTfSpUsXABYsWNDg2sYYc6aQpp7btEY9e/bUE5nIvPfwEbr+5V1+P/RsekTs5eGHHz7u0PnBgwd7k1lLGzFiBPfeey/Z2dktHYoxphUSkS9UtWdLx3EizvgWWFRIEC6ng10HyunWrRsXXXSRdyLzsQRK8qqoqODyyy+35GWMOSP5cympVkFE8ES62bW/HKhZJaO1cLlcjBkzpqXDMMaYFnHGt8AA4iPdFB/w6yh9Y4wxJ5klMMAT8X8tMGOMMa2DJTDAE+myBGaMMa2MJTBqWmC7D5ZTVX16jMg0xpgzgSUwwBPpplqh5KC1wowxprWwBEZNAgMo3m8DOYwxprWwBAbER9QksF0HrAVmjDGthSUw/q8FZgM5jDGm9bAERr0WmCUwY4xpNSyBAeHuIMJdToqtC9EYY1oNS2C16i8nZYwxJvBZAqsVb6txGGNMq2IJrJYn0m2jEI0xphWxBFbLuhCNMaZ1sQRWyxPhZu/hI5RXNr8XmDHGmMBgCaxWfO1csBLbVsUYY1oFS2C1PDYXzBhjWhVLYLVsNQ5jjGldLIHV8i7oayMRjTGmVbAEVqtNhAuwFpgxxrQWlsBquYOcRIcG21wwY4xpJSyB1WNzwYwxpvWwBFaPx5aTMsaYVsOvCUxEhojIGhFZLyITmjhfICJfikiliIxsdK5KRJbVvmb5M846nki3DeIwxphWIshfFxYRJ/AEMBDYAiwRkVmqurJese+BG4A7m7jEYVXN91d8TbEFfY0xpvXwZwusF7BeVTeqagUwDRhev4CqFqrq10C1H+PwmSfSzcGKKg6WV7Z0KMYYY47DnwksFdhc7/OW2mO+ChGRpSLymYhc3lQBEbmltszSXbt2/ZhYAZsLZowxrYk/E5g0cUx/QP22qtoTuAZ4REQ6HHUx1WdUtaeq9vR4PCcap5clMGOMaT38mcC2AOn1PqcBRb5WVtWi2q8bgYVAt5MZXFPibTKzMca0Gv5MYEuAjiLSXkRcwCjAp9GEIhIrIu7a9/HAecDK5mv9eLYeojHGtB5+S2CqWgncBswFVgH/VtUVIjJRRIYBiMg5IrIFuBL4h4isqK1+NrBURJYDC4D7Go1e9Is24W4cYgnMGGNaA78NowdQ1dnA7EbH/ljv/RJquhYb11sE5PkztqY4HUJcuNuWkzLGmFbAVuJoJD7Cxa79tqmlMcYEOktgjXgirQVmjDGtgSWwRjyRbortGZgxxgQ8S2CN1K1Ir/pDpqwZY4w51SyBNeKJcFNRVc2+MltOyhhjApklsEZsLpgxxrQOlsAa8URYAjPGmNbAElgj3haYjUQ0xpiAZgmsEetCNMaY1sESWCPRocEEO8VWpDfGmADXbAITEaeIPHCqggkEImI7MxtjTCvQbAJT1Sqgh4g0tbfXaatuLpgxxpjA5ctivl8BM0XkVeBg3UFVfd1vUbUwT4SbbXvLWjoMY4wxzfAlgcUBJUD/escUOG0TWHyEm2+27m3pMIwxxjTjuAlMVW88FYEEEk+km5KDFVRVK07HGdV7aowxrcZxRyGKSJqIzBCRnSKyQ0ReE5Gj9vA6nXgi3VRVK6WHbFsVY4wJVL4Mo38emAWkAKnAm7XHTls2F8wYYwKfLwnMo6rPq2pl7esFwOPnuFqUJTBjjAl8viSwYhG5rnZOmFNErqNmUMdpK752PUSbzGyMMYHLlwQ2DrgK2A5sA0bWHjttWQvMGGMCX7OjEEXECVyhqsNOUTwBIdzlJDTYaQnMGGMCmC8rcQw/RbEEDBGpWY3DuhCNMSZg+TKR+RMR+TvwCg1X4vjSb1EFgPgIlz0DM8aYAOZLAju39uvEeseUhitznHY8kW42FR88fkFjjDEt4njPwBzAU6r671MUT8DwRLpZvGl3S4dhjDHmGI73DKwauO0UxRJQPBEhlB46QkVldUuHYowxpgm+DKOfJyJ3iki6iMTVvfweWQuLj3QBUHLQnoMZY0wg8uUZWN2cr1/UO6ZA5skPJ3B46iYz768gOTq0haMxxhjTmC+r0bc/FYEEGu9k5gNlQHTLBmOMMeYox+xCFJHf1nt/ZaNz/+vLxUVkiIisEZH1IjKhifMFIvKliFSKyMgmzkeJyNbaYfynlK3GYYwxga25Z2Cj6r2/u9G5Ice7cO0qHk8AlwA5wGgRyWlU7HvgBuClY1zmv4EPjncvf6hbD9ESmDHGBKbmEpgc431Tn5vSC1ivqhtVtQKYRqNVPVS1UFW/Bo4a6iciPYBE4F0f7nXShQQ7iQwJoviA7QlmjDGBqLkEpsd439TnpqQCm+t93lJ77Lhq5589BPzmOOVuEZGlIrJ0165dvlz6B/FEuq0FZowxAaq5QRxdRWQfNa2t0Nr31H4O8eHaTbXSfEl8ALcCs1V1s8ixG3uq+gzwDEDPnj19vbbPPBGWwIwxJlAdM4GpqvNHXnsLkF7vcxpQ5GPdvsAFInIrEAG4ROSAqh41EMSfPJFuVhTtO35BY4wxp5wv88BO1BKgo4i0B7ZSMyjkGl8qquq1de9F5Aag56lOXlAzkMNaYMYYE5h8WYnjhKhqJTXLUM0FVgH/VtUVIjJRRIYBiMg5IrIFuBL4h4is8Fc8J8IT6eZAeSWHK6paOhRjjDGN+LMFhqrOBmY3OvbHeu+XUNO12Nw1XgBe8EN4x1U3F6z4QDnpcWEtEYIxxphj8FsL7HRQl8B2WjeiMcYEnOMmMBEZISLrRGSviOwTkf31RiSe1jw2mdkYYwKWL12I9wOXqeoqfwcTaOp3IRpjjAksvnQh7jgTkxdAXLgLEWuBGWNMIPKlBbZURF4B3gC8f8lV9XW/RRUggp0O4sJc7LIWmDHGBBxfElgUcAgYVO+YAqd9AgNbTsoYYwKVL/uB3XgqAglUNpnZGGMCky+jENNEZIaI7BSRHSLymog0O3frdOKJdNsgDmOMCUC+DOJ4HpgFpFCzmvybtcfOCHVdiKonfa1gY4wxP4IvCcyjqs+ramXt6wXA4+e4AoYnwk15ZTX7yytbOhRjjDH1+JLAikXkOhFx1r6uA0r8HVigiI90ATaU3hhjAo0vCWwccBWwHdgGjKw9dkbwRNRsfVZsCcwYYwKKL6MQvweGnYJYAlLdahw2F8wYYwLLMROYiPxWVe8XkcdpYidlVb3dr5EFCG8CsxaYMcYElOZaYHXLRy09FYEEqpjQYJwOsQRmjDEB5pgJTFXfrH17SFVfrX9ORK70a1QBxOEQ4iNclsCMMSbA+DKI424fj522bDKzMcYEnuaegV0CXAqkishj9U5FAWfUpChPhNsGcRhjTIBp7hlYETXPv4YBX9Q7vh/4lT+DCjSeSDcrt50Re3gaY0yr0dwzsOXAchF5SVWPnMKYAk58hJviAxVUVysOh7R0OMYYY/DtGVg7EZkuIitFZGPdy++RBRBPpJuqamXP4TM6jxtjTEDxdTHfp6h57nUR8CIwxZ9BBRqbC2aMMYHHlwQWqqrzAVHV71T1z0B//4YVWDwRlsCMMSbQ+LIjc5mIOIB1InIbsBVI8G9YgSXeu5xUWQtHYowxpo4vLbD/AsKA24EewHXAWH8GFWisC9EYYwKPL4v5Lql9ewC40b/hBKZIdxDuIAfFBypaOhRjjDG1jtsCE5F5IhJT73OsiMz1b1iBRUS8OzMbY4wJDL50Icar6p66D6payhn2DAxq5oJZAjPGmMDhSwKrFpG2dR9EJIMmtlc53VkLzBhjAosvCewe4GMRmSIiU4AP8XExXxEZIiJrRGS9iExo4nyBiHwpIpUiMrLe8QwR+UJElonIChH5ma/fkL/Ygr7GGBNYfBnEMUdEugN9AAF+parFx6snIk7gCWAgsAVYIiKzVHVlvWLfAzcAdzaqvg04V1XLRSQC+La2bpEv35Q/eCLc7D5UwZGqaoKdvuR9Y4wx/nTMv8Qiclbt1+5AW2oW990KtK09djy9gPWqulFVK4BpwPD6BVS1UFW/BqobHa9Q1brmjru5OE8VT6QbVdh90EYiGmNMIGiuBXYHcAvwUBPnlOOvxpEKbK73eQvQ29fARCQdeBvIAn7TVOtLRG6pjZG2bds2Pn1SxddbjSMxKsSv9zLGGHN8zSWwebVfx6vqiSze29Sy7T4P/lDVzUAXEUkB3hCR6aq6o1GZZ4BnAHr27OnXgSXeycz2HMwYYwJCc11zdQM1pp/gtbcA6fU+p1HTDfmD1La8VgAXnGAcJ0WCrcZhjDEBpbkWWImILADai8isxidVddhxrr0E6Cgi7al5djYKuMaXoEQkDShR1cMiEgucBzzsS11/ibcFfY0xJqA0l8CGAt2p2TqlqedgzVLVytrFf+cCTuA5VV0hIhOBpao6S0TOAWYAscBlIvIXVe0MnA08JCJKTVfkg6r6zQ+N4WQKdTmJcAdZAjPGmADR3I7MFcBnInKuqu46kYur6mxgdqNjf6z3fgk1XYuN680DupzIPf3JE+m2Z2DGGBMgjpnAROQRVf0v4LnallADPnQhnnY8EW6KrQVmjDEBobkuxLpdlx88FYG0Bp5IN6u272vpMIwxxtB8F+IXtV8/qDtWO6AivXby8RknPsJlz8CMMSZA+LKdykIRiRKROGA58LyItOiIwJbiiXSzv6ySsiNVLR2KMcac8XxZoilaVfcBI4DnVbUHcLF/wwpMdZOZbVFfY4xpeb4ksCARSQauAt7yczwBzWOTmY0xJmD4ksAmUjOXa72qLhGRTGCdf8MKTDaZ2RhjAocv26m8Crxa7/NG4Ap/BhWobD1EY4wJHL4M4ri/dhBHsIjMF5FiEbnuVAQXaNqEWwvMGGMChS9diINqB3H8hJoFejsBv/FrVAHKFeQgNizYBnEYY0wA8CWBBdd+vRR4WVV3+zGegOeJdFsLzBhjAsBxn4EBb4rIauAwcKuIeIAy/4YVuOIjLIEZY0wgOG4LTFUnAH2Bnqp6BDgIDPd3YIHKFvQ1xpjA4EsLDCAVGCgiIfWOveiHeAJezYK+FagqIk1tOm2MMeZUOG4CE5E/ARcCOdRsjXIJ8DFnagKLdHP4SBUHK6qIcPua/40xxpxsvgziGAkMALar6o1AV8Dt16gCmE1mNsaYwOBLAjusqtVApYhEATuBTP+GFbhsOSljjAkMvvSBLRWRGOCfwBfAAWCxX6MKYJbAjDEmMPgyCvFWVd2jqk8DA4GxtV2JZ6T6K9IfPnyYfv36UVVVxXfffUePHj3Iz8+nc+fOPP3008e91u7duxk4cCAdO3Zk4MCBlJaWNlluyJAhxMTE8JOf/KTB8WuvvZbs7Gxyc3MZN24cR44cAWDq1Kl06dKFLl26cO6557J8+XIAKioqKCgooLKy8sf8CIwxJiAcM4GJSPfGLyCOmtXpu5+6EANLbJgLh9S0wJ577jlGjBiB0+kkOTmZRYsWsWzZMj7//HPuu+8+ioqKmr3Wfffdx4ABA1i3bh0DBgzgvvvua7Lcb37zG6ZMmXLU8WuvvZbVq1fzzTffcPjwYZ599lkA2rdvzwcffMDXX3/NH/7wB2655RYAXC4XAwYM4JVXXvmRPwVjjGl5zXUhPtTMOQX6n+RYWgWnQ2hTO5n5o6lTeemll4Ca5FCnvLyc6urq415r5syZLFy4EICxY8dy4YUX8v/+3/87qtyAAQO85eq79NJLve979erFli1bADj33HO9x/v06eM9DnD55Zdz9913c+211x43PmOMCWTHTGCqetGpDKQ18US42b7nABs3bqRdu3be45s3b2bo0KGsX7+eBx54gJSUlGavs2PHDpKTkwFITk5m586dJxTPkSNHmDJlCo8++uhR5yZNmsQll1zi/Zybm8uSJUtO6D7GGBNIfFmN/he1gzjqPseKyK3+DSuweSLdbNuxk5iYmAbH09PT+frrr1m/fj2TJ09mx44dpySeW2+9lYKCAi644IIGxxcsWMCkSZMatOqcTicul4v9+/efktiMMcZffBlGf7Oq7qn7oKqlwM3+CynweSLdlJZDWVnTS0KmpKTQuXNnPvroo2avk5iYyLZt2wDYtm0bCQkJPziWv/zlL+zatYuHH364wfGvv/6am266iZkzZ9KmTZsG58rLywkJCcEYY1ozXxKYQ+qtmSQiTsDVTPnTXnyEmz1VLqqqqrxJbMuWLRw+fBiA0tJSPvnkE7KzswEYM2YMixcfPfNg2LBhTJ48GYDJkyczfPgPW2Ly2WefZe7cubz88ss4HP/3q/z+++8ZMWIEU6ZMoVOnTg3qlJSU4PF4CA4Obnw5Y4xpVXxJYHOBf4vIABHpD7wMzPFvWIHNE+nmSJVScNEAPv74YwBWrVpF79696dq1K/369ePOO+8kLy8PqGkN1T3rqm/ChAnMmzePjh07Mm/ePCZMmADA0qVLuemmm7zlLrjgAq688krmz59PWloac+fOBeBnP/sZO3bsoG/fvuTn5zNx4kQAJk6cSElJCbfeeiv5+fn07NnTe60FCxY0GPxhjDGtlahq8wVEHMAtwMWAAO8Cz6pqlf/D813Pnj116dKlp+Reyzbv4fInPuHnuQ6+nftSk0Pc6+zbt4/x48fz6quvnpLYjmfEiBHce++93tahMebMJiJfqGrP45cMPMddiaN2GamngadFJA5IC7Tkdap1TYsmo00YX5eFMuCii6iqqsLpdDZZNioqKmCSV0VFBZdffrklL2PMacGXUYgLRSSqNnktA54XkYePV+90JiIM75rCog0l/GTkNcdMXoHG5XIxZsyYlg7DGGNOCl+egUWr6j5gBPC8qvagpjvxuERkiIisEZH1IjKhifMFIvKliFSKyMh6x/NF5FMRWSEiX4vI1b5+Q6fKsPwUVOHNr7e1dCjGGHNG8iWBBYlIMnAV8JavF64drfgENfuH5QCjRSSnUbHvgRuAlxodPwSMUdXOwBDgkfpz0QJBVkIknVOimLVsa0uHYowxZyRfEthEakYirlfVJSKSCazzoV6v2jobVbUCmAY0GCeuqoWq+jVQ3ej4WlVdV/u+iJotXDw+3POUGp6fwvIteyksPtjSoRhjzBnHl9XoX1XVLqp6a+3njap6hQ/XTgU21/u8pfbYDyIivaiZd7ahiXO3iMhSEVm6a9euH3rpH+2yrimIwKzlzS/aa4wx5uQ75ihEEfmtqt4vIo9Ts3hvA6p6+3GuLU0ca37M/tExJANTqNnC5ajVcVX1GeAZqBlG/0OufTIkR4dyTrs43li2lf/sn0W9+d7GGGP8rLkW2Krar0up2ciy8et4tgDp9T6nAT43VWp3f34b+L2qfuZrvVNteH4KG3cdZEXRvpYOxRhjzijNrUb/Zu3XySd47SVARxFpD2wFRgHX+FJRRFzADOBFVQ2MSVTHcGluMn+auYJZy4vITY1u6XCMMeaM0VwX4qzmKqrqsOOcrxSR26gZAOIEnlPVFSIyEViqqrNE5BxqElUscJmI/KV25OFVQAHQRkRuqL3kDaq6zNdv7FSJDXfRr5OHWcuKmDDkLBwO60Y0xphTobmVOPpSMwjjZeBzmn6m1SxVnQ3MbnTsj/XeL6Gma7FxvX8B//qh92spw/JTmL96J4sLd9Mns83xKxhjjPnRmnsGlgT8DsgFHgUGAsWq+oGqfnAqgmstBuYkEhrsZOYyG41ojDGnyjETmKpWqeocVR0L9AHWAwtF5D9PWXStRJgriEGdE3nn221UVB41WNIYY4wfNDsPTETcIjKCmu68XwCPAa+fisBam+H5Kew5dISP1p36+WjGGHMmam4Qx2Rqug/fAf6iqt+esqhaoQs6eogNC2bmsiIGnJ3Y0uEYY8xpr7lBHNcDB4FOwO31N2UGVFWj/BxbqxLsdHBpXjKvf7mVg+WVhLuPu1ONMcaYH6G5Z2AOVY2sfUXVe0Va8mra8PxUDh+p4r1VO1o6FGOMOe35spiv8VHPjFiSo0NsNKIxxpwClsBOIodDGNY1hQ/X7mL3wYqWDscYY05rlsBOsmH5KVRWK7O/sY0ujTHGnyyBnWQ5yVFkJUTYFivGGONnlsBOMhFheNcUFm/aTdGewy0djjHGnLYsgfnBsPwUAN60VpgxxviNJTA/yGgTTn56jI1GNMYYP7IE5ifD81NYuW0f63bsb+lQjDHmtGQJzE+GdknGIdhgDmOM8RNLYH6SEBnCeVnxzFxWhKq2dDjGGHPasQTmR8O6pvD97kMs27ynpUMxxpjTjiUwPxqcm4QryGHdiMYY4weWwPwoKiSY/tkJvLl8G1XVNd2Ihw8fpl+/flRVVQHgdDrJz88nPz+fYcOGHfea5eXlXH311WRlZdG7d28KCwubLDdu3DgSEhLIzc1tcPzVV1+lc+fOOBwOli5d6j0+b948evToQV5eHj169OD999/3nrv44ospLS39od++Mcb4lSUwPxuen0LxgXI+3VACwHPPPceIESNwOp0AhIaGsmzZMpYtW8asWbOOe71JkyYRGxvL+vXr+dWvfsVdd93VZLkbbriBOXPmHHU8NzeX119/nYKCggbH4+PjefPNN/nmm2+YPHky119/vffc9ddfz5NPPunz92yMMaeCJTA/u+isBCLdQcxcthWAqVOnMnz48BO+3syZMxk7diwAI0eOZP78+U0OEikoKCAuLu6o42effTbZ2dlHHe/WrRspKTUTsDt37kxZWRnl5eUADBs2jJdffvmEYzbGGH+wBOZnIcFOBucmMefb7ew7eJiNGzfSrl077/mysjJ69uxJnz59eOONN457va1bt5Keng5AUFAQ0dHRlJSUnNSYX3vtNbp164bb7QYgNjaW8vLyk34fY4z5MWzb4FNgeH4K07/YwpufryYmJqbBue+//56UlBQ2btxI//79ycvLo0OHDse8VlOtrXq7Zf9oK1as4K677uLdd99tcDwhIYGioiLatGlz0u5ljDE/hrXAToG+mW2Ij3Az89tiysrKGpyr67bLzMzkwgsv5Kuvvmr2WmlpaWzevBmAyspK9u7d22RX4YnYsmULP/3pT3nxxRePSqJlZWWEhoaelPsYY8zJYAnsFAhyOvhZv0wWF1Ww52C5N4mVlpZ6nzMVFxfzySefkJOTA8Ddd9/NjBkzjrrWsGHDmDx5MgDTp0+nf//+J6UFtmfPHoYOHcq9997Leeed1+CcqrJ9+/YGXZ/GGNPSLIGdIuPbyZB7AAAgAElEQVTPb8/VPdOpTM7jr5NeB2DVqlX07NmTrl27ctFFFzFhwgRvAvvmm29ISko6+jrjx1NSUkJWVhYPP/ww9913HwBFRUVceuml3nKjR4+mb9++rFmzhrS0NCZNmgTAjBkzSEtL49NPP2Xo0KEMHjwYgL///e+sX7+e//7v//YO69+5cycAX3zxBX369CEoyHqcjTGBQ06XZY569uyp9ec1BaIjVdVcPvFffPjaC8x549+clxV/zLKDBw9m7ty5pzC6Y/vlL3/JsGHDGDBgQEuHYow5yUTkC1Xt2dJxnAhrgZ1CwU4HL989mozcc/iPFxeztpmV6gMleUHN3DFLXsaYQGMJ7BSLCglm7tN/JtTt4sbnl7Bzf9nxK7Wwm2++uaVDMMaYo/g1gYnIEBFZIyLrRWRCE+cLRORLEakUkZGNzs0RkT0i8pY/Y2wJqTGhPDf2HHYfrOCmyUs5VFHZ0iEZY0yr47cEJiJO4AngEiAHGC0iOY2KfQ/cALzUxCUeAK5v4vhpIS8tmsdHd+PbrXu5/eVl3rUSjTHG+MafLbBewHpV3aiqFcA0oMEaSqpaqKpfA9WNK6vqfOC03s744pxE/nRZZ95btYO/vr2ypcMxxphWxZ/jolOBzfU+bwF6n8wbiMgtwC0Abdu2PZmXPmXGntuO70oO8dwnm2gbF8aN57Vv6ZCMMaZV8GcLrKnZtSe1n0xVn1HVnqra0+PxnMxLn1L3DD2bQTmJTHxrJe+u2H7U+fpbsCxYsMA7Tys/P5+QkJCj1lBsvGXLnXfeSXR0NC6Xi6ysrCa3YDl8+DBJSUl4PB5ycnJITk7G6XQSExPDsmXLGDhwIB07duSss84iMzMTEWHkyJG43W5CQkLo3Lkzzz//PABr164lPj6ejh07MnDgQJ588kk6duxIhw4dyM/Pp2vXrg3K79q1i/z8/Ca3eQG49957ycrKIjs7u8HozL/97W907tyZ3NxcRo8e7Z0gPmrUKNatW3fivxBjTKvgzwS2BUiv9zkNsJ0dm+B0CI+O6kaX1Gh+OW0ZX29puINz/S1YLrroIu/2K++//z5hYWEMGjTomOXffvtt3n77ba6++mpKS0uprq7mjjvuOCqG5557jquuuoq5c+eyZ88eevTowRtvvEFmZiajRo1iwIABrFu3jkGDBjFo0CAyMjIIDQ1l1KhRZGVlsXDhQn79619TUVHBs88+S0ZGBi+88AJ9+/bld7/7HZ9//jnXXHMNhYWFLFy4sEF5j8dD+/bt+f3vf3/UNi8rV65k2rRprFixgjlz5nDrrbdSVVXF1q1beeyxx1i6dCnffvstVVVVTJs2DYCf//zn3H///Sf5t2SMCTT+TGBLgI4i0l5EXMAo4PgbXp2hQl1Onh17Dm0iXIx7YSlbSg95zx1rC5bp06dzySWXEBYW1uB4/fIrV66ksrKSG2+8kfDwcPr168e8efOOWhR46tSp3HHHHcTFxbFv3z7uueceIiMjSU5OZt26dYwZMwaoWeJqwYIFAGRkZHhbeQcOHCAuLo6goCBmzpzJbbfdxtSpU72ricTFxREWFkZqairvvPNOg/JQs3/ZRx99dNT3OHPmTEaNGoXb7aZ9+/ZkZWWxePFioGYtyMOHD1NZWcmhQ4e860pecMEFvPfee1RW2uhOY05nfktgqloJ3AbMBVYB/1bVFSIyUUSGAYjIOSKyBbgS+IeIrKirLyIfAa8CA0Rki4gM9lesgcIT6eb5G86hvLKKcS8sYe/hI1RUVBy1BUudadOmMXr06AbHGpfv2rUrW7duJT4+nuLiYj744ANcLleDrVEa16msrPRu2VK3zqLL5QIgOTnZu8TUTTfdxPr161mzZg15eXk8+uijOBwOduzYwaBBg/joo484dOiQd73H2267jUOHDvGzn/2sQXmAnj17NpnA6m8fAzWLGW/dupXU1FTuvPNO2rZtS3JyMtHR0d6WqMPhICsri+XLl/+gn78xppVR1dPi1aNHDz1dfLJul3a4+20d/cynun7T95qdnX1UmaKiIo2Pj9eKiooGx7du3XpUeY/Hozk5OXrxxRfrNddco23atNHi4uIm62zatEndbrdu3rxZFyxYoEOHDlWHw9GgfExMjGZkZOhTTz2liYmJmpOTo+vWrdOQkBCNjo7WoKAgraio0Li4OL3//vvV6XRqly5dNC0tTYOCgnTixIkNyg8dOtRbvl+/fjp06FCNjY1Vl8ulIqKJiYlaWVmpqqqpqakaFhamgwYN0o4dO2q7du0U0LCwMA0KClKXy6XR0dHqcDg0MTFR8/LytG/fvhodHa0dO3ZUt9utgHbs2FEdDoe3TlhYmKanp2unTp10zpw5qqq6evVqzcjI8NYJCwvT0aNHa1ZWlsbExGhBQYF26NDhuHX+9re/qarqr3/9ax0/fnyTdbp27ep9RUZGNqgzf/78E//H5INDhw5pQUGBVlZWamFhoXbv3l27du2qOTk5+tRTT3nL/e53v9O0tDQNDw/3+dr/+7//e9T329jjjz+uHTp0UEB37drlPV5dXa3/+Z//qR06dNC8vDz94osvVFX1/fffb/DzcrvdOmPGDFVVvfrqq3Xt2rUn8mM4YwFLNQD+hp/Iq8UDOFmv0ymBqaq+unSzZtz1lmb81zQNiU3SX037Sv/54Qb9dEOx7j1coY888ojefPPNR9XbvXu3ZmRkNDg2aNAgXbRokaqqjho1SiMjI7W6urrJOps2bdLw8HBdtGiRLliwQC+99FJ1OBy6detWVa1JnJ06ddKMjAzNzs7WsWPHaufOnVVVtWvXrvrggw9qeHi4rl27VlNTU/WJJ57QqKgoVVXNycnRNm3a6KhRoxqUHzp0qO7bt09TU1O1X79+2q9fP23btq2WlZXpgAEDNCwszJvAevTooffff792795dhw8frps2bdK0tDRt06aN3njjjbp7924NCgpSt9ut7777rn766aeanZ2ts2bN0n79+unq1au1X79+On36dAX0gw8+0EWLFqmIaO/evXXjxo2amZnpvd/KlSt19erVWlBQoDExMZqenq4lJSX6+uuvq4jotm3bmq0TFxenhYWFqqo6b948jYiI0LKysqPq1KmsrNTExERvncLCQh04cOAJ/Avy3d///nd95JFHVFW1vLxcy8rKVFV1//79mpGR4f3df/rpp1pUVORzAluxYoV26dKl2e9XVfXLL7/UTZs2aUZGRoME9vbbb+uQIUO0urpaP/30U+3Vq9dRdUtKSjQ2NlYPHjyoqqoLFy7Um2666Yf9AM5wrTmB2VJSAWpkjzReuqk3tw3Jx0E1H64q4q9vr2LUM5/R5c/v8oeH/sEOTw+eWLCeBWt2smt/OXfffTcLFy6kqqrKOyKvqqqK/v37M3nyZL7++ms+/vhjBg0ahIh4t2yJjY1tUCcyMtK7Zcu2bdvIysrixRdfBGDy5Mne52u7d+/2dgG+8847rF27lqysLBITE3nsscfIzc1l+/btiAilpaXs2rWLffv20alTJ3bs2MGuXbvIysoCakYu5ubmArB9+3bOO+883G43xcXFAHz88cds2rSJ3bt30717d0JDQ1m3bh0JCQmUlZURHR1Nt27dCAoKorKyErfbTW5uLn369EFVqaioICIiguzsbAD++c9/EhwcTEFBAQsXLiQ7O5svv/yS9PT0Bs/Zzj77bLKzs9mzZw+RkZFccsklxMXFsXr1akJDQ5kxY8ZRz+bq10lLSyMjIwOAJUuWEBUVRWlp6VF16syfP58OHTp462RkZFBSUsL27UePTj1Z6j8zdblc3p24y8vLqa7+vymaffr0ITk52efrNvf8sr5u3bo12UU+c+ZMxowZg4jQp08f9uzZw7Zt2xqUafwc2J5/nmFaOoOerNfp1gKrb9y4cTpv3jzdse+wvr96h/7lpQUaHuvR8+99r6aVVvuKzu6tgyb8UzsVDNPLJjyht7/8pd4+5XONS22vrtBwdQa7NKFtlk54fp7+bd4aze5+rmbm5OvMZVv12uvH6oUXXqhJSUnqdDpVRLyvhIQE7dq1q7Zr107DwsI0OTlZHQ6HAg2+Zmdn64IFC3TgwIGalZWlHo9H+/fvr4899phGRER4u9RycnK0c+fO2q9fP3366ad16NChOmbMGI2OjlaXy6VOp1NdLpd27txZw8LCND8/XxMSErRTp046e/Zsb9fmH//4R83OzlaHw6GdO3fWsrIynTRpkoqIejwe78+vf//+3vuoqvbr109TUlI0PDxclyxZov/xH/+hISEhGhwcrFOnTlWPx6NpaWmal5en06ZNU1XVpKQkzcrK0ri4OAV0zJgx6na79dprr9WvvvrqmHXS0tLU5XKpy+XS+Ph4jYqK0ldeeaVBnbZt22pqaqqKiF522WXerla3260xMTHqcDg0OTlZn3jiCe3Tp4/m5ORoUlKSt5syKytL09PTvV2uIqKhoaGalZWleXl5mpOTo3l5eTpu3DgNDw9XQBMTEzU1NVWDg4MV0IEDB3q7+qZMmaJ5eXkaGhqq1113nUZGRiqgbdq00bS0NBURHThwoCYlJWlmZqa2b99es7KytGvXrpqWlqaJiYkKaGRkpLrdbs3KytKzzjpLHQ6HRkdHNyifk5OjDz30kF5xxRUaFBSkISEhGhERoS6XSwFNTk72dm1GRkZqx44dNScnR4cMGaJpaWnqcDi83c1ut1sjIiLU4XBoamqqdu3aVXNzc7Vr166alJSksbGxCminTp28/9aCg4PV4XBou3btvN2Ue/fu1ZSUFO3Vq5eGhoYqoLGxsRoZGakiohERERoTE6OZmZnN1gkODtbg4GB1uVyalJSkCQkJR9WJjo72/nw7dOigQUFB3t+9iGhKSspR98jLy/P+7t1ut3o8Hu/PKzg4WDMzM7VXr166adMmb53u3bt760RGRnr/TXo8HgV2AuuBr4Huqgo14xcKqZn6tBbYUft5DTUjzL+rX762zh+AQ7V11gDtao9PAyY3vkftubbAu9SMk1jZqE5HPc7f/RZPPCfrdTonsC+//FKvu+66Js/tPVyhn24o1mc/2qgZXfrqsMc/0r6//qcm9RioBfe/r+feO197/nWe5v9lrnb+4xztdM9sbT+hJuGFtOvmTX7JNzyqqecM0gfmrNbPN5ZoRWVVszE1ftZ255136vLly73J5YILLtDdu3cfVf7nP/+5Pvfcc97jTZUfO3asvvjii7px40aNiorSbt266fTp04+qUycmJkYnTJigqqoXXnihOhwOveiii7znGyewvn37akhIiI4fP17PP/98jYiI0ISEBA0NDdWnnnpKr7jiCp0+fbpu3bpVk5KSdMeOHRoUFKTDhw/XO+64QzMyMrRPnz6akJCgl19+ua5Zs6bJOk6nUxMSErSkpER3796tTqdTPR6PPvbYYw3qLFy4UD0ej5577rkaERGh0dHR+tJLL+ncuXPV6XTqRRddpH/60580LS1NV61apaqqAwYM0MjISE1PT9dvv/1WMzIytKSkRK+88koF9IknntBXXnlFu3Tp4v35BwcH6+OPP64FBQUaHx+vqamp+u2332p4eHiT3aFbt27VmJgYveuuu/Tcc8/VlJQU3bhxo4aHh3v/CJaVlenq1au1ffv2WllZqUuWLNHQ0FDNzMzUkSNH6pNPPqmlpaV6zz33qMPh0PT09Abl9+/fr+Hh4frAAw9oRkaGnnXWWXrOOedodXW1Nymr1nRtXnjhhbp06VLdv3+/JiUl6dy5cxXQvLw8bzeliGhUVJTOmjVLp0yZoldccYWq1nQtBgUFaUJCgr799tsaHByshYWFeu2116qI6NixY73dlLfffruOHj1aPR6Pvvjii5qRkaGPPvqoZmdna3h4uF588cUqInrkyJFj1klOTtaOHTvq7t27dcOGDepwODQ0NLRBd+jtt9+uQ4YM0cTERE1MTNSXXnpJY2Nj9cMPP9Sf/vSn6nK5dPz48UfdIz4+Xl9++WXv/4Dl5uZqWVmZ9/dYWVmpL7/8sl511VV6++2366hRozQ6OlrffvttTUxM1KCgIC0sLNSNGzfW/Q9MBTXzdvsAn2tNAukGfAZsB84FvgFigBuBMuDZ+uVr6ywFrqtNdBlAWO3x31KzqIU0UWchMLD2fUS9Ov2Af+px/u5bF2Ir0K1bNy666CLvkPX6okKC6ZPZhvHnt6dw+SJm3nY+ix68if+5dRTv31HAJxP6s+Sei/nqj4P49i+DWfPXS9h471A2/O+llK5dwjd/HsQbvziPCdddQlJ2D55csJar/vEp3SfO45YXlzL18+/YvPvQUfcNDQ31djkCPPDAA3Tp0gWoGdV4xx13EBsbe1T5q6++mtdee63BtRqXz87OZsuWLcTExBAaGsrmzZu9Q+SbEhQURFFRESUlJSxbtgyo6cars2XLFuLj/2/vtZ07d5Kbm4vT6eSjjz4iLi6O8PBwQkJCyMrKYv/+/aSkpJCSkkJCQgKvvfYakZGRnHPOORw4cACAzZs3U11dTXx8PJ06dWqyTkhICOeddx5xcXHExMQgIlRWVhIaGtqgTr9+/byjO2NiYujQoQPBwcF88cUXBAcHEx4eTlpaGjk5OZSWlrJv3z42bNhA+/btqaqq4pNPPmHw4MHExcWxcOFCAMrKyrjqqqsoKytj27Zt7Nmzh+DgYAYPHsyRI0eorq5m6NChpKSkeEeavvfeew26+iIiIqisrKRHjx6UlpbSu3dv2revWSmmuroat9tNaWkp2dnZdOzYkcWLFxMfH09FRQUJCQkkJCSwZ88eYmJiePHFF3G73VRWVhIdHe0tX1xcTFlZmXc0bWlpKcOHD0dEqKiooKqqiuXLl+NyuSgqKiIlJYXy8nKCg4P57LPPcDgcjB492nttVSU1NdU7R3HhwoWoKtu2bcPlchEcHMz06dNp27YtGRkZvPfee3g8Hl5//XX69OnD9u3bKSwspHPnzjidTq6//nrvv5mxY8cC8PXXX+N0Ovn000+PWefw4cNcddVVxMbGkp6ejqpSXV1NVVVVgzrnn38+IkJISAhz5swhKSmJ0NBQFi1aRFRUFHPnzqVnz54N7hEUFMSoUaMAmDdvHtdccw3ff/89hw4dwul0smjRIkaOHMncuXPZsWMHSUlJJCYmcumll3Lw4EFvt3b79u0JDw8HcNb+v95nQIyIJAPlQCU1yeoS4CVV3UNNAloDXEpNwooRkeTadW7LVPVftf+ZHVTVuj8cHQB37X0+a1QnSFXnAajqgXp1PgIuFpFmV4uyBNZKjBs3DqfTedLKOx1CSLCTyJBg8tNjuH1AR5ZO+V+++tMQnrq2Oz/pmsyKon3cM+NbLrh/Af0fXMifZ61gweqdHKqoPOq5WX0ul4vLL78cqGnhr1+/3lt+xowZnHXWWUDN7tDPPPNMg/IAffv2Zdq0aYSGhrJ//34qKiro1asXixcv9s5Hqy8kJISFCxcyefJkhgwZAsCKFStQVSZNmkRJSQlt2rTxlt+5cyejRo3inXfeYceOHezfv5/CwkLCw8OJjIzk22+/5cknn2Tx4sVUVFSwcOFCEhIS6NOnD++++y5VVVUcPHiQkpIShg8fzqZNm1i3bp03xro6YWFhJCYmcuONN5KYmEh1dTUHDx6kU6dOTdbZs2cPSUlJhIWFcc899/Dggw9SUVHBli1byM3N9U4h2LhxI6GhoXz33Xfs3LmTp59+msTERCoqKigtLSUiIsKbyOvqvPXWW4gIv/3tb1m+fLk3EcfGxlJeXo6IsHTpUrZs2UJSUhJbt25l2bJlHDlyhH/9619s2LCBVatWcfDgQe8zRY/HwyeffMKMGTP4/vvvGTVqFGeffTapqals2rSJt99+mwcffJALL7zQm/AHDx7MJ598QkxMTIPy99xzD0VFRZSWlvLss8/SpUsX9u7di9PpZM6cOXz22WeEhIQwePBg0tPTueuuu5g9ezYi4p1mMWXKFAD27t3r/UMfHh5Obm4uY8aMoXPnzpSUlDBjxgyKiopYu3YtpaWlOBwOKioqvM9ob775Znbs2IHb7WbEiBEUFRUxc+ZMUlNTUVX27t1LTEwMixYtorq6usk6+/bt47PPPmPQoEF4PB5UlZSUFJYvX95knV27djFjxgyKi4sZO3YsO3fuJDg4mOTkZL766qsm73H48GH+/e9/k5qa6v3e27Rpw6JFi3A4HJSX1zwX37FjB1CzSe7Bgwe9ybCysrJuiouISN1/IFuoWQawE7AH8AA/B86vXaC9bonAXUDXxuVF5HUgGfhzbXlqz39XW/6oe4jI6yLylYg8UFdHVaup6XKsq9O04zXRWsvrdO5CbCnV1dW6bsc+ffajjTpm0ufa6Z7ZmnHXW9rxd7O11//M04SeQ7TLzQ/qxQ8t1Ese+VDjOnRRV0SMOoNdGhbr0UF3PKo3v/C5JnXsqm3Ss9QdHqltc3vphJc/0/9+c4XGJqaqyx2qwS63xsQn6i8ffF5f+GST5vToq9Fx8eoMClanM0iHj7tdn/1oo97858fVGRSsYZHRGuRya0h4pIZGxqg4nOoKi1BxODQ6zqM9e/fVyy67TDMzMzU8PFyjo6M1Pj5eXS6X95ldQkKCtmnTRoOCghTQs88+W91ut3bq1Em7dOmil1xyiXbq1EknTpyoIqIul0sTEhI0J6ezOpxODQoK0vDwCM1o1147ZHXUCwdcrC+99oZ2yOqov73nDyoi6nA6NSwsXC/sf7EuX7Ve3W63ulwubd++vfd5Xt2ozgULFni7KR9//HGtrq7W6667Tp1Op0ZGRuqRI0d03LhxOn36dH3nnXcU0Kf/8Yy2zcjQHj3P0fPOL9Blqzeow+HU1PR0vXjIUP126x7tdW6BPvbCvzU+IVFDQsN0yrufa26PPhrnSdKI6BjtlJuv0TFx6nA69boxY3Xu3LkaGxurGRkZmpWVpSKin332mWZmZmp8fLz3eQ2g8fHx+thjj+kDDzygPXr00OnTp+vs2bMV0NzcXF2yZIn27dtXo6Ki1OFwaMeOHXX06NH62GOPaVpamj7zzDPe8m3atFGn06kOh0OzsrJ0zZo1GhYW5p1qUXc91Zru0C5duni7w6ZMmaKq6o23/ijczMxMLS4u1t///vfqcrk0JSVFr7/+ek1KStIOHTqow+HQ9u3ba0REhN59993aoUMHXbp0qd56660aHBysGzZs0IyMDM3IyNDx48draGiohoeHa0pKik6YMME7BaBxnYiICO3atas+++yzumHDBhURbdeunc6aNavJOqmpqTpgwADt3bu3PvjggxoWFqaRkZGal5ent9xyS5P3CA8P18zMTB0/frxmZWVpcHCwtm3b1htXXFycFhcX61VXXaWRkZG6a9cub3f5WWedpX369Kl7Nl0F5GlN1918oAcwEthbm2yeAxYD44G3gbeAD4HLmiifSU0X4pvA+Nprvk3Nc67LjnGPTGrW5X2trk5tual1dY718udivqaVExGyEiLJSohk/PntKTtSxZLC3Xy8vpg9B4+wdeRYlr41hayCCzlSVc1PfvdPjlRVU1mlHKmu5khVNd+XlnH2fzzCkapq9m5ey9aPpjN7zV4qq0qpjO9EwlU34QyLBuCNXfDGrBVw8T3EUNPhXrFjA+99/gbLPCspXbCAhOsfxpXQnmCnEORwEOwUgp0Ogp0ORGDb3jK27tjA98vf5NZHZrFm5lP8+tab6NE9v8nvsbS0lG7dunHzzTcTFRXF+PHjuf3225k3bx5//etfueKKkYy86Zcs2lDCpxtK+HxTCTx8HdWV5QR1uRQKrqcS2FB+iBtu/TVRfa7klcretP3tmxxc+QFl33/DpnNuY+jTSyk/UoUjJILqqx6nHLht/kG2jruAxPOv5s6PjxCansPnO5VvZi9n1pEPWbtdqKqupkzc9LlvIWs+WM575Vlsf/cfOEKjuHdjCltLD1OakMah5R9x6cPzqa6uYlvpIfYUVTL0sY/ZsmodX935G8LPHoQWfsXv5+9i+84DOGIzOVKyhfKhf6V8+kSq9yxh1tpDfP5BFdXxmaT+ZDyp6e3Yee945pdEcahSiE/LJCoqkgn3PcFV/fJwhUexamcZ3yxdxCEN4stiiE2IwxUSxq59ZUz5tJDcy3/OzsNPUbFuBaGJmXy4+CscqblEJbdns7bhiHpwh4Yx7q7/od/gYfz1lzewa3sRG/c7CHaHcOjgAW674y7+6447CHM5qa6uacl0796doUOHcs0117B582ZvywagoKCgpqV0qJzSPXvZUR6EK74t8YkpHNi/j90SRWRye9LP7s7moufZve8gVTh564PP2VRYyMBLL6Ps0AGqqqq5878f4vCRKtp3ymHRkmU122Yo7N23nwPOCGbOXUBh4Xdc8pNhHDpYU+f39z0CDifhbZJ46/1PSMnvB0DJ7lK2H6hmzvsfUvjdd1w2bDgHDuyvaZHt309CUjIdIiJYuXLl/2/v3KOjqq89/tkzmZiQREAINBDeEh6iIApNay0KVmllqddbfFFA67Kl14tWxWqrV62r9loFb7lKK5RXrdjCtS1trfVZfEAVwYAgRUWQWEFN5P0MeXzvH+dMOqQBkpY4zMz+rDVrzu93fo+9Jydnn9/j7E0kEsEswo4dO1i3bh3l5eVcdNFF7NoVlJ8+fToAgwcPZunSpdTV1RGNRtm1axdFRUUsWbKk3iXcli1bqKqqYvLkyWRlZdG3b19GjRrFzTffHJ+2F7Av/JeIu/yLASuA7sAmgvWpwQQGrRTYGtY5qLykDeGU9JNh+VlhnUGH6kPShvB+szBse1ZYLiehTqO4AXOaTE4sypm9Czmzd9xx8inMLtzL+MsHNXF6czizZ2czfvyIoPzdI5FEbZ2oqRMH4savNjB+EsSiI5g/7zjGjhtBzt3nEYtGyIpY/bpNQz7csY/n1w7g4RkVPPrKe1S3/zKrf/cxX67DbTsAAA93SURBVFxbxoh+HTi7Twfa5mXXl49Pbebl5TF+/HiqqqpYvnIVp503mhcO9OK/73mOrXsOANCtXStGnVLER3nHsW9PNT+47SbatOvA228s48HvXcvosRMYednV9W3vOqszt437Mt8d3oXaujqunVrDZ784gsu/VMKyRU/yhzn3MeLCsQwZdQUHauuoyI1x8pDP88qvp9Ptq18ne/BQyp/7Bf2GDmNo+2re2vo+W/84hXMvuYY1S57lir5Z/DQ3xsDifF79637OyilnQSybmj1buXnCWFq1qua/9m+ne68SZs6czNcvHM70y/rwnZdaEzHjncrdzL2sN998oop3EBOvHkN5+Xrm/+2v9B5wKtv21qC8dsx8YjFba2J8vGYV+Sefw3XzV1JTU8vmTZv4zfo6aPd5PvnLZB5dH6Nu7Uaqa2rYsucA819dT9X6pVhOJ6pZy6rXXiKroD1Prt7M9lUreGjB0+QNGM6B6hrmvrqJ+ZXLqNy0iwMVlXzjl2+yY2cQTWnWumx+OnoS1Vs3ccLZV5Nds5f3//AsK9ucQa2Me6fNYu6fV6PiU9GGDTyx+iNOvO1PfPzHqexXDl/538VUb4+wefMmIjn5LDtQTMXqmezpNYKaOmPHlgrIKaCix0gia9dS2/MMctt2omrRHJYVnMGW3Y+zs3wL1ZUbqasRFjXq9u7i11s6s2/nOpRTQKvxM6hd9Qz7Fs3h5ZxS9tQ8yl8Wv0TbM7/G0ul/RhK7du3mrsU72PVBNbRqS/bY6cRWPYMWzWGvsnl6Wwe2L5xD689dwp7qOlS1l921EfYNORMK3mJncSlZbTuhRXN4bN9A9h6o5Y8rP2B/+bucePYlZH+ykO3bt3PxxRdTXl5OSUkJa9euZdq0adx1113ccccdPPLIIyxfvpwbbriBlStXxqeSI8BGM7sFKJD0oZlVAG3Dc08B1wLfJwhxNRZYT7CutSOxvJnFbw5nAkvC498DVwBrzKy0sTqSKoHhBOtqcUqANRyOww3PUunjU4hOQ/ZUVeuZNR/plsff0Ok/eFbdbnlCPW59Qv/+kyX6yaJ39c5HO1VXV6dLx4zTbQ/N07d/tUI9R98iIlHFOvRQq6JeKuxWovvmPaUPtu3V1KlT1blzZ0WjUeXm5mrkyJGSpIkTJ8rMDvIOsWLFCtXW1urEE0+s365dWFiohQsXSgqmtiKRSP3rCbFYrH6XXG5ubv326/z8fHXs2FElJSUqLCys7yfc/ly/lbq0tFQ5OTnKzc1Vdna2evbsWb/lvXXr1ho4cKA6duxY/4pE/FWFWCymtm3b1m8LLyoqUnFxcf103ZQpUxSLxerrHJeToy7deujUIaXKLyhQl27dlduqlTp+pkgnDThZXbp2VX643TwaTrX2799fbdq0UZ8+fZSVlaVOnTopv6BAxcVd1K//AE286TsaMPBUndinn/ILjtfxrdsoKxZTLJat1ie00y9f2aBhF1ymSDSqop591LF7iXoM+pzyT+ggzBTLaSXMlNeuSMUnDVXXfoPVrqirIlkx5R/fRj1K+qtdh47KCbe3x7epx7fQd+7cWa3y8tT/pJPUtWs3XX/jJP142gz16du/foo5Gs1SVlbwygFmimZlqXPX7mpX2EGf6VSsPy1dozvvf0i9SvopGtYBZBZRLBZT+w6fUV5+gYq6dFM0K0v/Nv5benz533TGeRcqln1cfbsWiSiaFZNZRAVtgmn0ou4lyju+rc7/+o26fNK9at+pqyLRoI/YcTnKyj5OJ3QoUkFBgXJyctSrVy8VFxerR48e2rRpk+bMmaOSkpJ6XcxMkUhE2dnZGjdunAg2aqwPv2crmL67jmCdS0A18ElY5m0C94D7CXYmLgE6hXWmEexoFMF2+nhbHQm24a8P65yuv08Tfolga/1qYC6QnVDnNR3hvp90w3O0Pm7AnMNRW1unle9v05Sn39JXpr5U//rAyXc+paIrpyrvpLM1+O5ndO281zXv1XJtqNx9kLeShiS+2hB/heBIJNYZM2aMKioqmlWnKf1MnjxZAwYMqE83pc4DDzygmTNnNrl8Y/0cibKyMg0bNky33367JOn8889XVVVVk/v4Z36vptQpKytTnz599NxzzzW5zo033qjS0tJmyZVYpym6l5WV6bTTTqv/u7SUXNddd52AtxUYjUeBQh3JaMANJKxVHe3yzamTdMNztD5uwJzmsHn7Xv3ilY2atGClZi/eoHsemKbq6upmtTFr1qxGXSMls86CBQv04IMPNquP2bNnN1v3f6afCRMmHORTsyX6kJr/G48bN67Zv9eMGTNavM5VV11V79arpfqYMWNGs11JEbwLltVS5ZtTx8LCKc/pp5+uhoEQHcdxnMNjZq9LOj3Zcvwz+HtgjuM4TkriBsxxHMdJSdyAOY7jOCmJGzDHcRwnJXED5jiO46QkbsAcx3GclMQNmOM4jpOSuAFzHMdxUpK0eZHZzCoJYs4cjvYEPr0ylUzWP5N1h8zW33U/PN0kFR6hzDFJ2hiwpmBmy1P1jfOjQSbrn8m6Q2br77qnr+4+heg4juOkJG7AHMdxnJQk0wzYjGQLkGQyWf9M1h0yW3/XPU3JqDUwx3EcJ33ItBGY4ziOkyZkjAEzs5Fm9raZvWtmtyZbnpbGzGabWYWZvZmQd4KZPWtm68LvtsmUsaUwsy5mtsjM1prZGjO7PsxPe/3NLMfMXjOzN0Ldvx/m9zCzpaHu880sO9mythRmFjWzFWb2RJjOJN03mtlqM1tpZsvDvLS97jPCgJlZFJgGfBnoD1xuZv2TK1WLMxcY2SDvVuB5Sb2B58N0OlID3CSpH1AKXBv+vTNB/ypguKSBwCBgpJmVAj8C/ifUfRtwdRJlbGmuB9YmpDNJd4CzJQ1K2D6fttd9RhgwYCjwrqQNkg4AvwIuTLJMLYqkl4CtDbIvBH4eHv8cuOhTFepTQtKHksrC410EN7POZID+YaT43WEyFn4EDAceD/PTUncAMysGzgdmhmkjQ3Q/DGl73WeKAesM/C0h/UGYl2l0lPQhBDd5oEOS5WlxzKw7cCqwlAzRP5xCWwlUAM8C64HtkmrCIul8/f8Y+A5QF6bbkTm6Q/Cw8oyZvW5m3wjz0va6z0q2AJ8S1kieb79Mc8wsH/g18G1JO4OH8fRHUi0wyMzaAL8F+jVW7NOVquUxs1FAhaTXzeyseHYjRdNO9wTOkLTZzDoAz5rZW8kWqCXJlBHYB0CXhHQxsDlJsiSTj82sCCD8rkiyPC2GmcUIjNc8Sb8JszNGfwBJ24EXCNYB25hZ/IE1Xa//M4ALzGwjwTLBcIIRWSboDoCkzeF3BcHDy1DS+LrPFAO2DOgd7kbKBi4Dfp9kmZLB74Hx4fF44HdJlKXFCNc9ZgFrJT2QcCrt9TezwnDkhZnlAucQrAEuAr4aFktL3SV9V1KxpO4E/+N/ljSGDNAdwMzyzKwgfgycC7xJGl/3GfMis5l9heBpLArMlnRPkkVqUczsl8BZBN6oPwbuBBYCC4CuwPvAaEkNN3qkPGb2BeBlYDV/Xwv5HsE6WFrrb2anECzURwkeUBdIutvMehKMSk4AVgBfk1SVPElblnAKcZKkUZmie6jnb8NkFvCYpHvMrB1pet1njAFzHMdx0otMmUJ0HMdx0gw3YI7jOE5K4gbMcRzHSUncgDmO4zgpiRswx3EcJyVxA+YkHTOTmU1JSE8ys7uOUttzzeyrRy75L/czOvR+v6hBfvdQv4kJeQ+Z2ZVHaG+CmY07QpkrzeyhQ5zb3Vj+0SLUKzHSwTVmVpZOns6dYx83YM6xQBVwsZm1T7YgiYRRDJrK1cB/SDq7kXMVwPXNCeMh6WFJjzSj/6NGgteKppYfC0wEzpW0rWWkcpx/xA2YcyxQQxD6/IaGJxqOoOIjCzM7y8xeNLMFZvaOmd1rZmPCWFirzaxXQjPnmNnLYblRYf2omd1vZsvMbJWZfTOh3UVm9hjBi9AN5bk8bP9NM/tRmHcH8AXgYTO7vxH9KgnCWIxveMLMepnZU6Hz1ZfNrG+Yf5eZTQqPh4QyvhLK/GZCE53C+uvM7L4GbU8JR0XPm1lhmDfIzF4N2/ttfMRkZi+Y2Q/N7EUCYzs61PENM3upEZ3ifVxCEJ7jXEmfHKqc47QEbsCcY4VpwBgza92MOgMJYj+dDIwFSiQNJQilMTGhXHdgGEGYjYfNLIdgxLRD0hBgCHCNmfUIyw8FbpN0UMw4M+tEEFtqOEGsrSFmdpGku4HlwBhJNx9C1nuBmxoZ1c0AJko6DZgE/KSRunOACZI+B9Q2ODcIuDT8DS41s7jPzzygTNJg4EUCTywAjwC3SDqFwEDfmdBWG0nDJE0B7gDOC+OKXXAInboBDxEYr48OUcZxWgw3YM4xgaSdBDfX65pRbVkY+6uKIGTIM2H+agKjFWeBpDpJ64ANQF8CP3HjLAg7spQg7EbvsPxrkt5rpL8hwAuSKsPwHPOALzZRv/eA14Ar4nkWeMv/PPB/oRzTgaLEeqFfwwJJfwmzHmvQ9POSdkjaD/yVwKhA4EJrfnj8KPCF8OGgjaQXw/yfN5B/fsLxEmCumV1D4JaqMSoJXBNdckjFHacFyZRwKk5q8GOgjGDEEaeG8EErdNKbuI6U6M+uLiFdx8HXdkN/aSIIszFR0tOJJ0IfensOId+/Go/lhwSBFeNTchGCWFWDDlPnSH0m/ga1HPp/uik+4+r1ljTBzD5LMGpdaWaDJG1pUH4vQZTzxWZWIWleE/pwnKOGj8CcY4bQwegCDg75vhE4LTy+kCDCcHMZbWaRcF2sJ/A28DTwLQvCrmBmJaEH78OxFBhmZu3DqcDLCabnmoSktwhGSaPC9E7gPTMbHcpgZjawQZ1twC4zKw2zLmtidxH+7oH9CmCxpB3ANjM7M8wfeyj5zayXpKWS7gA+4eBwRInyVQIjgR+a2XlNlM1xjgo+AnOONaYA/5mQ/hnwOzN7jWAjxKFGR4fjbYIbdUeCtaT9ZjaTYJqxLBzZVXKEUOuSPjSz7xKE5zDgSUnNDU1xD4FH9DhjgJ+a2e0ExvlXwBsN6lwN/MzM9hDE99rRhH72ACeZ2eth+UvD/PEE64CtCKZTrzpE/fvNrDeBns83IlM9kt4zswuAJ83sYklLmyCf4/zLuDd6xznGMbN8SfHdl7cCRZKuT7JYjpN0fATmOMc+54cjvyygHLgyueI4zrGBj8Acx3GclMQ3cTiO4zgpiRswx3EcJyVxA+Y4juOkJG7AHMdxnJTEDZjjOI6TkrgBcxzHcVKS/wffV09DTbd8sQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the misclassification error for each k value is :  [0.158 0.122 0.112 0.108 0.108 0.108 0.107 0.107 0.107 0.107 0.106 0.106\n",
      " 0.106 0.106 0.106 0.107 0.106 0.106 0.106 0.106 0.106 0.106 0.106 0.106\n",
      " 0.106]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score as cval_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# creating odd list of K for KNN\n",
    "myList = list(range(0,50))\n",
    "neighbors = list(filter(lambda x: x % 2 != 0, myList))\n",
    "\n",
    "# empty list that will hold cv scores\n",
    "cv_scores = []\n",
    "\n",
    "# perform 10-fold cross validation\n",
    "for k in neighbors:\n",
    "    knn_brute = KNeighborsClassifier(n_neighbors=k,algorithm='brute')\n",
    "    scores = cval_score(knn_brute, d_train,y_train, cv=10, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())\n",
    "\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# determining best k\n",
    "optimal_k = neighbors[MSE.index(min(MSE))]\n",
    "print('\\nThe optimal number of neighbors is %d.' % optimal_k)\n",
    "\n",
    "# plot misclassification error vs k \n",
    "plt.plot(neighbors, MSE)\n",
    "\n",
    "for xy in zip(neighbors, np.round(MSE,3)):\n",
    "    plt.annotate('(%s, %s)' % xy, xy=xy, textcoords='data')\n",
    "\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "print(\"the misclassification error for each k value is : \", np.round(MSE,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train error is 10.393\n"
     ]
    }
   ],
   "source": [
    "knn_optimal = KNeighborsClassifier(n_neighbors=optimal_k,algorithm='kd_tree')\n",
    "knn_optimal.fit(d_train,y_train)\n",
    "#for train error\n",
    "y_true, y_pred = y_train, knn_optimal.predict(d_train)\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"Train error is %.3f\" %((fp+fn)/140))\n",
    "#for test error\n",
    "y_true, y_pred = y_test, knn_optimal.predict(d_test)\n",
    "cm=confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The value of TPR is 0.998\n",
      "The value of TNR is 0.047\n",
      "The value of FPR is 0.953\n",
      "The value of FNR is 0.002\n",
      "\n",
      "Test error is 10.733\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGFdJREFUeJzt3Xu8VVW99/HPd7NRLirgBTVAwcDr8RENjUo7Kj54PVLHOMeOJ9GsbeW10vLyqA+IT1RqZqUnfMRA8355SVopYqaeowkiXkg74JUtBgpIBWjuvX/njzXAJe619kL23mvtwffda77WmmONOcdYvfC3xv7NMcdURGBmZl1fXbU7YGZm7cMB3cwsEw7oZmaZcEA3M8uEA7qZWSYc0M3MMuGAbmaWCQd0M7NMOKCbmWWivtodKKV+kwG+hdU+ZNd+g6rdBatBzy1+XBt6jvfeeqnimNN96502uL2O4BG6mVkmanaEbmbWqVqaq92DDeaAbmYG0NxU7R5sMAd0MzMgoqXaXdhgDuhmZgAtDuhmZnnwCN3MLBO+KGpmlgmP0M3M8hAZzHLxjUVmZlC4KFrp1gZJr0h6VtJcSbNT2ZaSZkian177pXJJulLSAknPSNqn6DzjUv35ksa11a4DupkZFFIulW6VOSgihkfEiLR/DjAzIoYBM9M+wOHAsLQ1AFdD4QcAuAj4JLAfcNGaH4FSHNDNzKBwUbTS7aMZA0xN76cCnysqnxYFjwN9JW0PHArMiIhlEbEcmAEcVq4BB3QzM2jvEXoA90t6UlJDKts2It4ASK/9U/kAYGHRsY2prFR5Sb4oamYG63XrfwrSDUVFkyNictH+ZyJikaT+wAxJL5Q7XStlUaa8JAd0MzNYrztFU/CeXObzRel1iaS7KOTAF0vaPiLeSCmVJal6I1C8LvRAYFEqP3Cd8ofK9cspFzMzIKK54q0cSb0lbb7mPTAaeA6YDqyZqTIOuDu9nw4cn2a7jARWpJTMfcBoSf3SxdDRqawkj9DNzKA9byzaFrhLEhRi7I0R8VtJs4BbJZ0EvAaMTfV/DRwBLABWAScCRMQySRcDs1K9CRGxrFzDDuhmZtBui3NFxEvAXq2ULwVGtVIewCklzjUFmFJp2w7oZmbgW//NzLLR/F61e7DBHNDNzMDroZuZZcMpFzOzTHiEbmaWCQd0M7M8hC+Kmpllwjl0M7NMOOViZpYJj9DNzDLhEbqZWSY8Qjczy0RT5Q+4qFUO6GZm4BG6mVk2nEM3M8uER+hmZpnwCN3MLBMeoZuZZcKzXMzMMhFR7R5sMAd0MzNwDt3MLBsO6GZmmfBFUTOzTDQ3V7sHG8wB3cwMnHIxM8uGA7qZWSacQzczy0O0eB66mVkenHIxM8uEZ7mYmWXCI3Rrb5tuuikPPXgHm2y6KfX13bjzznsZP+EyvvH1Ezj9tK8wdOgQtt3+H1i6dHm1u2odbPMtNmP85ecxdNedIOCCb07kgFGf5uDDPktLSwvL3lrO+adfzJuL3+Kgww7gtO+eTEtLC81NzUy64AqeeuLpan+FriWDgK6o0QVp6jcZUJsd6wS9e/di5cpV1NfX8/BDd/HNb13Eu39/l+XLVzBzxu188lOHb7QBfdd+g6rdhU5zyZUXMOcPT3PHL6dT372enj170NLSwsq/rQLguK/8Cx/feTATvvMDevbqyepVqwHYefehXDp5Ikfvf2w1u9+pnlv8uDb0HKuuOLnimNPrzJ+32Z6kbsBs4PWIOErSEOBmYEtgDvCliPi7pE2BacAngKXAv0bEK+kc5wInAc3A6RFxX7k2O2yELmlXYAwwAAhgETA9Ip7vqDZzsXJl4T/Y7t3rqe/enYhg7tx5Ve6Vdabem/XiE5/am/NPvxiApvea+Ot7f/tAnZ69eqxdIHBNMF9TzkY7HNoA7T9CPwN4Htgi7X8f+FFE3CzpPygE6qvT6/KIGCrp2FTvXyXtDhwL7AF8DHhA0s4RUTLZX9fe3wBA0ncp/BIJeAKYld7fJOmcjmgzJ3V1dcyedT9vvP4MM2c+zBOznqp2l6yTDdxxAMuXLmfijy/gtgemMv7y8wqBGjj93K/xwJy7OfKYQ/npDyavPWbU4f/I9Edv5qobLuOCb06sVte7rpaofGuDpIHAkcD/T/sCDgZuT1WmAp9L78ekfdLno1L9McDNEfFuRLwMLAD2K9duhwR0Cr84+0bEpIi4IW2TUmdO6qA2s9HS0sKIfUez45AR7Dtib/bYY5dqd8k6WX19N3bbcxdumXonYw8Zx+pVqznptOMBuPJ7/8Eh+4zh3jvu49++/IW1x8z8ze85ev9jOf2E73Lqd0+uVte7rubmyre2XQF8B1gz7N8KeDsi1jxFo5FC9oL0uhAgfb4i1V9b3soxreqogN5C4U+EdW3P+1/wQyQ1SJotaXZLy8oO6lrXsWLFX/j9w//FoaMPrHZXrJP9edESFi96k2fnFFJt9//qQXbf84M/7PfeeT+HHHXQh4598vG5DBo8gL5b9umUvuYiWloq3opjVdoa1pxH0lHAkoh4suj0reXco43Pyh3Tqo7KoZ8JzJQ0n/d/YXYAhgKnljooIiYDk2HjvSi69dZb8t57TaxY8Rd69OjBqIMP4IeXXlXtblknW/rmMv68aDGDP74Dr7z4GiMP2JcX//tldhgyiNdeLvwnddChB/Dy/FcBGDR4IAtfaQRgtz13oXv3et5etqJq/e+S1uNO0eJY1YrPAEdLOgLoQSGHfgXQV1J9GoUPpHBdEQoj70FAo6R6oA+wrKh8jeJjWtUhAT0ifitpZwoplgEUfmkagVnlEvoG22+/LVOuvYJu3eqoq6vj9tt/xb2/foBTT/kyZ337G2y33TY89eQD/Oa3D3Ly186udnetA/2/8y7j+1eNp/sm3Vn46utccMZExl9+HoOH7kC0BIsa/8yEs78PwP8+6iCOHns4TU1NvPPOu5zVcEGVe98FtdNaLhFxLnAugKQDgbMi4jhJtwFfoHB9cRxwdzpketp/LH3+YESEpOnAjZIup5DxGEbhmmRJnrZoXcrGNG3RKtce0xZXTjiu4pjT+8JfVtReUUA/StJOvD9t8Sng3yPiXUk9gOuBvSmMzI+NiJfS8ecDXwaagDMj4jfl2vONRWZmAE3tnzyIiIeAh9L7l2hllkpEvAOMLXH8JcAllbbngG5mBl4+18wsG14+18wsD5HBWi4O6GZm4BG6mVk2HNDNzDLhB1yYmeXBzxQ1M8uFA7qZWSY8y8XMLBMeoZuZZcIB3cwsD9HslIuZWR48Qjczy4OnLZqZ5cIB3cwsE10/he6AbmYGEE1dP6I7oJuZgUfoZma58EVRM7NceIRuZpYHj9DNzHLhEbqZWR6iqdo92HAO6GZmQHiEbmaWCQd0M7M8eIRuZpYJB3Qzs0xEs6rdhQ3mgG5mhkfoZmbZiBaP0M3MsuARuplZJiK6/gi9rtodMDOrBdFS+VaOpB6SnpD0tKR5ksan8iGS/iBpvqRbJG2SyjdN+wvS54OLznVuKv+TpEPb+g4O6GZmQEuzKt7a8C5wcETsBQwHDpM0Evg+8KOIGAYsB05K9U8ClkfEUOBHqR6SdgeOBfYADgOuktStXMMO6GZmFC6KVrqVPU/B39Ju97QFcDBweyqfCnwuvR+T9kmfj5KkVH5zRLwbES8DC4D9yrXtgG5mxvoFdEkNkmYXbQ3F55LUTdJcYAkwA3gReDti7RJgjcCA9H4AsBAgfb4C2Kq4vJVjWuWLomZmQKzHcugRMRmYXObzZmC4pL7AXcBurVVLr60N+aNMeUkO6GZmdMw89Ih4W9JDwEigr6T6NAofCCxK1RqBQUCjpHqgD7CsqHyN4mNa5ZSLmRmFaYuVbuVI2iaNzJHUEzgEeB74HfCFVG0ccHd6Pz3tkz5/MCIilR+bZsEMAYYBT5Rru+IRuqRNI+LdSuubmXUlze23lsv2wNQ0I6UOuDUi7pH0R+BmSROBp4BrU/1rgeslLaAwMj8WICLmSboV+CPQBJySUjkltRnQJe2XGuwD7CBpL+ArEXHaR/iiZmY1qb1uLIqIZ4C9Wyl/iVZmqUTEO8DYEue6BLik0rYrSblcCRwFLE0NPA0cVGkDZmZdQXtNW6ymSlIudRHxamFa5Fplh/1mZl3N+sxyqVWVBPSFKe0SKSd0GvDfHdstM7POVcsj70pVEtC/TiHtsgOwGHgglZmZZaO5petP+mszoEfEEtJVVzOzXG0UKRdJ19DK3UkR0dBKdTOzLqklg+VzK0m5PFD0vgfweT64voCZWZeXw3rolaRcbinel3Q9hcVmzMyysVGkXFoxBNixvTtiVomn5t1Y7S5YpjaKlIuk5byfQ6+jcGvqOR3ZKTOzzpb9LJe0yPpewOupqCUtGmNmlpUcAlvZn6QUvO+KiOa05fCdzcw+pCVU8VarKvkb4wlJ+3R4T8zMqqi9ls+tppIpl6KF2PcHvirpRWAlhadoREQ4yJtZNlqq3YF2UC6H/gSwD+8/yNTMLFvR6hPfupZyAV0AEfFiJ/XFzKxqmmo4lVKpcgF9G0nfKvVhRFzeAf0xM6uK3Efo3YDNaP3J02ZmWck9h/5GREzotJ6YmVVR7iP0rv/tzMwqlPsIfVSn9cLMrMqaMxjDlgzoEbGsMztiZlZNGTyB7iOttmhmlp2WnEfoZmYbkxwWqnJANzMj/4uiZmYbjRY55WJmloXmanegHTigm5nhWS5mZtnwLBczs0x4louZWSaccjEzy0QO0xYreaaomVn2mlX5Vo6kQZJ+J+l5SfMknZHKt5Q0Q9L89NovlUvSlZIWSHqm+BnOksal+vMljWvrOzigm5lRGKFXurWhCfh2ROwGjAROkbQ7cA4wMyKGATPTPsDhwLC0NQBXQ+EHALgI+CSwH3DRmh+BUhzQzcxov4AeEW9ExJz0/q/A88AAYAwwNVWbyvvPax4DTIuCx4G+krYHDgVmRMSyiFgOzAAOK9e2A7qZGRCqfJPUIGl20dbQ2jklDQb2Bv4AbBsRb0Ah6AP9U7UBwMKiwxpTWanyknxR1MyM9bsoGhGTgcnl6kjaDLgDODMi/qLSSwu09kGUKS/JI3QzMwq3/le6tUVSdwrB/JcRcWcqXpxSKaTXJam8ERhUdPhAYFGZ8pIc0M3MKMxDr3QrR4Wh+LXA8xFxedFH04E1M1XGAXcXlR+fZruMBFaklMx9wGhJ/dLF0NGprCSnXMzMaNd56J8BvgQ8K2luKjsPmATcKukk4DVgbPrs18ARwAJgFXAiFJ4aJ+liYFaqN6GtJ8k5oJuZ0X4BPSIepfX8N7TyrOaICOCUEueaAkyptG0HdDMzvJaLmVk2vJaLmVkm/IALM7NMtGSQdHFANzMjj9UWHdDNzPBFUTOzbHiEbmaWiSZ1/TG6A7qZGU65mJllwykXM7NMeNqimVkmun44d0A3MwOccjEzy0ZzBmN0B3QzMzxCNzPLRniEbmaWB4/Qrd1dM/kyjjziEJa8+RbD9y483OSYY47iwgu+xW67DuNTnz6SJ+c8U+VeWkcZfcw4evfqRV1dHd26dePWKVfyk8nTePDRx6hTHVv268Ml53+b/ttsxT33Pci1v7wNgF49e3LBWaey67CdSp7HyvO0RWt306bdylVXXcd11/14bdm8eS8w9l++ytU/m1TFnllnmfKTSfTr22ft/onHHcNpDccDcMNtd3P1dTdy0XdOY8DHtuMXP/0BfbbYnEcem8X4H1zJTddcUfI8Vl7XD+cO6DXnkUf/wI47DvxA2QsvLKhSb6wWbNa799r3q1e/g9KTdfbec/e15f9rj11ZvOStzu5aVpoyCOmdHtAlnRgR13V2u2ZdgSQavnk+khg75nDGjjkCgB///BdM/+1MNu/dmyk/+fBfanfecx/7jxzR5nmsNF8U/WjGA60GdEkNQAOAuvWhrq53a9XMsnX91ZfRf5utWLr8bb565nkM2XEQI4bvyRknn8AZJ5/ANdNu4cY7fsWpX/nS2mOeePJp7rznfq6/+tI2z2Ol5XBRtK4jTirpmRLbs8C2pY6LiMkRMSIiRjiY28ao/zZbAbBVv76M+uynefaPf/rA50eOPpAHHvrPtft/WvAyF066gp9MupC+fbao+Dz2YbEe/6tVHRLQKQTt44F/amVb2kFtmnVpq1a/w8qVq9a+/68n5jBsp8G8uvD1tXV+98jjDEnXWN748xLOPO9ivnfh2QzeYWCb57HyWtZjq1UdlXK5B9gsIuau+4GkhzqozSzccP3P+MfPfoqtt96SV16azfgJl7Js+dv8+EcT2WabLZl+9zSefnoeRxx1XLW7au1s6bLlnHHexQA0NzVzxOgD2X/kCM48byKvvNaI6sTHtuvPhWefBsDV193Iir/8lYmX/gxg7fTEUuex8pqjdkfelVLU6Jeo32RAbXbMqmr1okeq3QWrQd233kkbeo5/2/HzFcecG1+9a4Pb6wietmhmhme5mJllo5Zz45VyQDczw7f+m5llwykXM7NM5DDLxQHdzIw8Ui4ddWORmVmX0p43FkmaImmJpOeKyraUNEPS/PTaL5VL0pWSFqQ76vcpOmZcqj9f0ri22nVANzOj3W/9/wVw2Dpl5wAzI2IYMDPtAxwODEtbA3A1FH4AgIuATwL7ARet+REoxQHdzIxCyqXSrS0R8TCwbJ3iMcDU9H4q8Lmi8mlR8DjQV9L2wKHAjIhYFhHLgRl8+EfiAxzQzcyAiKh4k9QgaXbR1lBBE9tGxBuprTeA/ql8ALCwqF5jKitVXpIvipqZAc3rcVE0IiYDk9up6daWEYgy5SV5hG5mRvumXEpYnFIppNclqbwRGFRUbyCwqEx5SQ7oZmasX8rlI5oOrJmpMg64u6j8+DTbZSSwIqVk7gNGS+qXLoaOTmUlOeViZkb7zkOXdBNwILC1pEYKs1UmAbdKOgl4DRibqv8aOAJYAKwCTgSIiGWSLgZmpXoTImLdC60f4IBuZkb73vofEV8s8dGoVuoGcEqJ80wBplTargO6mRm+9d/MLBs53PrvgG5mhgO6mVk2avVxnOvDAd3MDI/Qzcyy4QdcmJllojm6/lNFHdDNzHAO3cwsG86hm5llwjl0M7NMtDjlYmaWB4/Qzcwy4VkuZmaZcMrFzCwTTrmYmWXCI3Qzs0x4hG5mlonmaK52FzaYA7qZGb7138wsG77138wsEx6hm5llwrNczMwy4VkuZmaZ8K3/ZmaZcA7dzCwTzqGbmWXCI3Qzs0x4HrqZWSY8Qjczy4RnuZiZZcIXRc3MMuGUi5lZJnynqJlZJjxCNzPLRA45dOXwq5Q7SQ0RMbna/bDa4n8Xtq66anfAKtJQ7Q5YTfK/C/sAB3Qzs0w4oJuZZcIBvWtwntRa438X9gG+KGpmlgmP0M3MMuGAXuMkHSbpT5IWSDqn2v2x6pM0RdISSc9Vuy9WWxzQa5ikbsDPgMOB3YEvStq9ur2yGvAL4LBqd8JqjwN6bdsPWBARL0XE34GbgTFV7pNVWUQ8DCyrdj+s9jig17YBwMKi/cZUZmb2IQ7otU2tlHlakpm1ygG9tjUCg4r2BwKLqtQXM6txDui1bRYwTNIQSZsAxwLTq9wnM6tRDug1LCKagFOB+4DngVsjYl51e2XVJukm4DFgF0mNkk6qdp+sNvhOUTOzTHiEbmaWCQd0M7NMOKCbmWXCAd3MLBMO6GZmmXBAt3YnqVnSXEnPSbpNUq8NONeBku5J748ut+KkpL6SvvER2vi/ks76qH00qxUO6NYRVkfE8Ij4B+DvwNeKP1TBev/bi4jpETGpTJW+wHoHdLNcOKBbR3sEGCppsKTnJV0FzAEGSRot6TFJc9JIfjNYuwb8C5IeBf55zYkknSDpp+n9tpLukvR02j4NTAI+nv46+GGqd7akWZKekTS+6Fznp3XmHwB26bT/N8w6kAO6dRhJ9RTWcn82Fe0CTIuIvYGVwP8BDomIfYDZwLck9QCuAf4JOADYrsTprwR+HxF7AfsA84BzgBfTXwdnSxoNDKOwDPFw4BOSPivpExSWUdibwg/Gvu381c2qor7aHbAs9ZQ0N71/BLgW+BjwakQ8nspHUnhox39KAtiEwu3suwIvR8R8AEk3AA2ttHEwcDxARDQDKyT1W6fO6LQ9lfY3oxDgNwfuiohVqQ2vj2NZcEC3jrA6IoYXF6SgvbK4CJgREV9cp95w2m+JYAHfi4ifr9PGme3YhlnNcMrFquVx4DOShgJI6iVpZ+AFYIikj6d6Xyxx/Ezg6+nYbpK2AP5KYfS9xn3Al4ty8wMk9QceBj4vqaekzSmkd8y6PAd0q4qIeBM4AbhJ0jMUAvyuEfEOhRTLvemi6KslTnEGcJCkZ4EngT0iYimFFM5zkn4YEfcDNwKPpXq3A5tHxBzgFmAucAeFtJBZl+fVFs3MMuERuplZJhzQzcwy4YBuZpYJB3Qzs0w4oJuZZcIB3cwsEw7oZmaZcEA3M8vE/wBvDyY6WLrMrwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=sns.heatmap(cm,annot=True,fmt=\"d\")\n",
    "ax.set(xlabel='Predicted', ylabel='True')\n",
    "print()\n",
    "tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()\n",
    "print(\"The value of TPR is %.3f\" %(tp/(fn+tp)) +\n",
    "      \"\\nThe value of TNR is %.3f\" %(tn/(tn+fp)) +\n",
    "     \"\\nThe value of FPR is %.3f\" %(fp/(tn+fp)) +\n",
    "     \"\\nThe value of FNR is %.3f\" %(fn/(fn+tp)) )\n",
    "print()\n",
    "print(\"Test error is %.3f\" %((fp+fn)/60))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------------+---------------------------+-----------+-------------+------------+\n",
      "|       Model       |     Vectorising Method    | Optimal K | Train Error | Test Error |\n",
      "+-------------------+---------------------------+-----------+-------------+------------+\n",
      "| KNN (Brute Force) |        Bag Of Words       |     11    |    10.571   |   11.000   |\n",
      "| KNN (Brute Force) | Bag Of Words with n-grams |     9     |    10.657   |   10.967   |\n",
      "|   KNN (KD-Tree)   |        Bag Of Words       |     7     |    10.036   |   10.967   |\n",
      "|   KNN (KD-Tree)   | Bag Of Words with n-grams |     9     |    9.736    |   10.900   |\n",
      "| KNN (Brute Force) |           Tf-Idf          |     7     |    9.007    |   10.183   |\n",
      "| KNN (Brute Force) |    Tf-Idf with n-grams    |     7     |    8.857    |   10.283   |\n",
      "|   KNN (KD-Tree)   |           Tf-Idf          |     7     |    9.171    |   10.400   |\n",
      "|   KNN (KD-Tree)   |    Tf-Idf with n-grams    |     23    |    10.621   |   10.933   |\n",
      "| KNN (Brute Force) |          Avg W2V          |     19    |    10.143   |   10.783   |\n",
      "|   KNN (KD-Tree)   |          Avg W2V          |     19    |    10.143   |   10.783   |\n",
      "| KNN (Brute Force) |    Tf-Idf weighted W2V    |     21    |    10.393   |   10.733   |\n",
      "|   KNN (KD-Tree)   |    Tf-Idf weighted W2V    |     21    |    10.393   |   10.733   |\n",
      "+-------------------+---------------------------+-----------+-------------+------------+\n"
     ]
    }
   ],
   "source": [
    "from prettytable import PrettyTable\n",
    "    \n",
    "pt = PrettyTable()\n",
    "\n",
    "pt.field_names = [ \"Model\", \"Vectorising Method\", \"Optimal K\", \"Train Error\", \"Test Error\"]\n",
    "\n",
    "pt.add_row([\"KNN (Brute Force)\",\"Bag Of Words\", \"11\", \"10.571\", \"11.000\"])\n",
    "pt.add_row([\"KNN (Brute Force)\",\"Bag Of Words with n-grams\", \"9\",\"10.657\",\"10.967\"])\n",
    "pt.add_row([\"KNN (KD-Tree)\",\"Bag Of Words\", \"7\",\"10.036\",\"10.967\" ])\n",
    "pt.add_row([\"KNN (KD-Tree)\",\"Bag Of Words with n-grams\", \"9\", \"9.736\", \"10.900\" ])\n",
    "pt.add_row([\"KNN (Brute Force)\",\"Tf-Idf\", \"7\", \"9.007\" , \"10.183\"])\n",
    "pt.add_row([\"KNN (Brute Force)\",\"Tf-Idf with n-grams\", \"7\",\"8.857\",\"10.283\"])\n",
    "pt.add_row([\"KNN (KD-Tree)\",\"Tf-Idf\", \"7\",\"9.171\",\"10.400\"])\n",
    "pt.add_row([\"KNN (KD-Tree)\",\"Tf-Idf with n-grams\", \"23\",\"10.621\",\"10.933\" ])\n",
    "pt.add_row([\"KNN (Brute Force)\",\"Avg W2V\", \"19\" ,\"10.143\",\"10.783\" ])\n",
    "pt.add_row([\"KNN (KD-Tree)\",\"Avg W2V\", \"19\" ,\"10.143\",\"10.783\" ])\n",
    "pt.add_row([\"KNN (Brute Force)\",\"Tf-Idf weighted W2V\", \"21\", \"10.393\", \"10.733\"])\n",
    "pt.add_row([\"KNN (KD-Tree)\",\"Tf-Idf weighted W2V\", \"21\", \"10.393\", \"10.733\"])\n",
    "\n",
    "print(pt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1) Train error is almost same in all the cases.\n",
    "\n",
    "2) Minimum train train error is achieved when we are vectorising with *tf-idf with uni-grams and bi-grams* and using *KNN with brute force* model.\n",
    "\n",
    "3) Test error is also, almost same in all the cases.\n",
    "\n",
    "4) We have achieved the minimum test error when we are vectorising with *tf-idf without un-igrams and bi-grams* and using *KNN with brute force* model."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}