NLP Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "#### This project focuses on Natural Language Processing which is part of machine learning and a way for computers to learn and analyze the human language. The Kaggle competition can be found here: https://www.kaggle.com/c/nlp-getting-started/overview and the data was downloaded from https://www.kaggle.com/c/nlp-getting-started/data\n",
    "\n",
    "#### Twitter has become an important communication channel in times of emergency. The ubiquitousness of smartphones enables people to announce an emergency they’re observing in real-time. Because of this, more agencies are interested in programatically monitoring Twitter (i.e. disaster relief organizations and news agencies). For this challenege, we are building a machine learning model that predicts which Tweets are about real disasters and which one’s aren’t."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     /Users/johnkaiser/nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n",
      "[nltk_data] Downloading package punkt to\n",
      "[nltk_data]     /Users/johnkaiser/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn import linear_model, model_selection, metrics\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "#!pip install nltk\n",
    "import nltk\n",
    "from nltk.tokenize import word_tokenize\n",
    "from nltk.corpus import stopwords\n",
    "nltk.download('stopwords')\n",
    "nltk.download('punkt')\n",
    "from nltk.stem import PorterStemmer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import data and begin Exploratory Data Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>keyword</th>\n",
       "      <th>location</th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Our Deeds are the Reason of this #earthquake M...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Forest fire near La Ronge Sask. Canada</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>All residents asked to 'shelter in place' are ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13,000 people receive #wildfires evacuation or...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Just got sent this photo from Ruby #Alaska as ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>#RockyFire Update =&gt; California Hwy. 20 closed...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>#flood #disaster Heavy rain causes flash flood...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>13</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I'm on top of the hill and I can see a fire in...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>14</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>There's an emergency evacuation happening now ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I'm afraid that the tornado is coming to our a...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id keyword location                                               text   \n",
       "0   1     NaN      NaN  Our Deeds are the Reason of this #earthquake M...  \\\n",
       "1   4     NaN      NaN             Forest fire near La Ronge Sask. Canada   \n",
       "2   5     NaN      NaN  All residents asked to 'shelter in place' are ...   \n",
       "3   6     NaN      NaN  13,000 people receive #wildfires evacuation or...   \n",
       "4   7     NaN      NaN  Just got sent this photo from Ruby #Alaska as ...   \n",
       "5   8     NaN      NaN  #RockyFire Update => California Hwy. 20 closed...   \n",
       "6  10     NaN      NaN  #flood #disaster Heavy rain causes flash flood...   \n",
       "7  13     NaN      NaN  I'm on top of the hill and I can see a fire in...   \n",
       "8  14     NaN      NaN  There's an emergency evacuation happening now ...   \n",
       "9  15     NaN      NaN  I'm afraid that the tornado is coming to our a...   \n",
       "\n",
       "   target  \n",
       "0       1  \n",
       "1       1  \n",
       "2       1  \n",
       "3       1  \n",
       "4       1  \n",
       "5       1  \n",
       "6       1  \n",
       "7       1  \n",
       "8       1  \n",
       "9       1  "
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = 'nlp-getting-started'\n",
    "\n",
    "train = pd.read_csv(f'{path}/train.csv')\n",
    "test = pd.read_csv(f'{path}/test.csv')\n",
    "\n",
    "train.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data Frame Description: \n",
      " \n",
      "                   id     keyword location   \n",
      "count    7613.000000        7552     5080  \\\n",
      "unique           NaN         221     3341   \n",
      "top              NaN  fatalities      USA   \n",
      "freq             NaN          45      104   \n",
      "mean     5441.934848         NaN      NaN   \n",
      "std      3137.116090         NaN      NaN   \n",
      "min         1.000000         NaN      NaN   \n",
      "25%      2734.000000         NaN      NaN   \n",
      "50%      5408.000000         NaN      NaN   \n",
      "75%      8146.000000         NaN      NaN   \n",
      "max     10873.000000         NaN      NaN   \n",
      "\n",
      "                                                     text      target  \n",
      "count                                                7613  7613.00000  \n",
      "unique                                               7503         NaN  \n",
      "top     11-Year-Old Boy Charged With Manslaughter of T...         NaN  \n",
      "freq                                                   10         NaN  \n",
      "mean                                                  NaN     0.42966  \n",
      "std                                                   NaN     0.49506  \n",
      "min                                                   NaN     0.00000  \n",
      "25%                                                   NaN     0.00000  \n",
      "50%                                                   NaN     0.00000  \n",
      "75%                                                   NaN     1.00000  \n",
      "max                                                   NaN     1.00000   \n",
      " \n",
      "\n",
      "DF shape:  (7613, 5) \n",
      " \n",
      "\n",
      "Data Types: \n",
      " \n",
      " id           int64\n",
      "keyword     object\n",
      "location    object\n",
      "text        object\n",
      "target       int64\n",
      "dtype: object \n",
      " \n",
      "\n",
      "target\n",
      "0    4342\n",
      "1    3271\n",
      "Name: count, dtype: int64 \n",
      " \n",
      "\n",
      "Null values: \n",
      " id             0\n",
      "keyword       61\n",
      "location    2533\n",
      "text           0\n",
      "target         0\n",
      "dtype: int64 \n",
      " \n",
      "\n",
      "True disaster tweets (1):  3271\n",
      "False disaster tweets (0):  4342\n"
     ]
    }
   ],
   "source": [
    "print('Data Frame Description: \\n \\n', train.describe(include='all'), '\\n \\n')\n",
    "print('DF shape: ', train.shape, '\\n \\n')\n",
    "print('Data Types: \\n \\n', train.dtypes, '\\n \\n')\n",
    "print(train['target'].value_counts(), '\\n \\n')\n",
    "print('Null values: \\n', train.isnull().sum(), '\\n \\n')\n",
    "print('True disaster tweets (1): ', sum(train['target']==1))\n",
    "print('False disaster tweets (0): ', sum(train['target']==0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id keyword location                                               text\n",
      "0   0     NaN      NaN                 Just happened a terrible car crash\n",
      "1   2     NaN      NaN  Heard about #earthquake is different cities, s...\n",
      "2   3     NaN      NaN  there is a forest fire at spot pond, geese are...\n",
      "3   9     NaN      NaN           Apocalypse lighting. #Spokane #wildfires\n",
      "4  11     NaN      NaN      Typhoon Soudelor kills 28 in China and Taiwan \n",
      " \n",
      "\n",
      "(3263, 4) \n",
      " \n",
      "\n",
      "id             0\n",
      "keyword       26\n",
      "location    1105\n",
      "text           0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(test.head(), '\\n \\n')\n",
    "print(test.shape, '\\n \\n')\n",
    "print(test.isnull().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initial findings in the data:\n",
    "\n",
    "* We can see from the initial training data EDA that there are 5 columns and 7,613 observations\n",
    "* It looks like 'text' and 'target' are the features we'll focus on and can probably drop the other 3. There were also several null values in both 'keyword' and 'location'.\n",
    "* Of the 7,613 'text' observations, 7,503 are unique so there must be 110 duplicates, we can take care of that\n",
    "* More than half of the training tweets are NOT true disaster tweets\n",
    "* We also have a test set of 4 columns (just missing the 'target' feature) and 3,263 observations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create new 'tweet' data frame with only 'text' and 'target' features\n",
    "tweet = train[['text', 'target']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>Check these out: http://t.co/rOI2NSmEJJ http:/...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>Check these out: http://t.co/rOI2NSmEJJ http:/...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>320 [IR] ICEMOON [AFTERSHOCK] | http://t.co/vA...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>115</th>\n",
       "      <td>320 [IR] ICEMOON [AFTERSHOCK] | http://t.co/vA...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>118</th>\n",
       "      <td>320 [IR] ICEMOON [AFTERSHOCK] | http://t.co/TH...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7600</th>\n",
       "      <td>Evacuation order lifted for town of Roosevelt:...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7607</th>\n",
       "      <td>#stormchase Violent Record Breaking EF-5 El Re...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7609</th>\n",
       "      <td>@aria_ahrary @TheTawniest The out of control w...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7610</th>\n",
       "      <td>M1.94 [01:04 UTC]?5km S of Volcano Hawaii. htt...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7611</th>\n",
       "      <td>Police investigating after an e-bike collided ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>179 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   text  target\n",
       "40    Check these out: http://t.co/rOI2NSmEJJ http:/...       0\n",
       "48    Check these out: http://t.co/rOI2NSmEJJ http:/...       0\n",
       "106   320 [IR] ICEMOON [AFTERSHOCK] | http://t.co/vA...       0\n",
       "115   320 [IR] ICEMOON [AFTERSHOCK] | http://t.co/vA...       0\n",
       "118   320 [IR] ICEMOON [AFTERSHOCK] | http://t.co/TH...       0\n",
       "...                                                 ...     ...\n",
       "7600  Evacuation order lifted for town of Roosevelt:...       1\n",
       "7607  #stormchase Violent Record Breaking EF-5 El Re...       1\n",
       "7609  @aria_ahrary @TheTawniest The out of control w...       1\n",
       "7610  M1.94 [01:04 UTC]?5km S of Volcano Hawaii. htt...       1\n",
       "7611  Police investigating after an e-bike collided ...       1\n",
       "\n",
       "[179 rows x 2 columns]"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet[tweet.duplicated(['text'], keep=False)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "tweet = tweet.drop_duplicates(subset=['text'], keep='first')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7503, 2)"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Text(0, 0, '57.4%'), Text(0, 0, '42.6%')]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkQAAAGwCAYAAABIC3rIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAp9klEQVR4nO3de3SNZ6LH8d9OIheJHUUuUnFpjRDXcWns0TpojpRUW5fqRTGuhwkm0uI4bZW2Ji1tXYZpWmq0qyylhhly3CYkpsQtmhZF1UrFGZJQko2SkOzzx4y9uptUK5LsxPP9rLXX2M/77DfPa81uvut9371ZHA6HQwAAAAbzcPcCAAAA3I0gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxvNy9gJqgpKREZ86cUZ06dWSxWNy9HAAA8As4HA5dunRJYWFh8vC49TkggugXOHPmjMLDw929DAAAUA6nT59Wo0aNbjmHIPoF6tSpI+lff6FWq9XNqwEAAL+E3W5XeHi48/f4rRBEv8DNy2RWq5UgAgCghvklt7twUzUAADAeQQQAAIxHEAEAAOMRRLgrzJw5UxaLxeXRsmVLSdK3335batvNx5o1a37R/seNGyeLxaL58+c7xwoLCzV06FBZrVa1aNFCf//7311eM3fuXE2cOLHCjhEAUHm4qRp3jdatW7tEiZfXv/7vHR4errNnz7rMff/99zV37lz16dPnZ/e7bt067dmzR2FhYaX2kZGRofT0dG3atEnPPvuscnNzZbFYlJWVpSVLlujAgQMVcGQAgMpGEOGu4eXlpdDQ0FLjnp6epcbXrVunwYMHKyAg4Jb7/Oc//6mJEydqy5Ytio2Nddl29OhRPfbYY2rdurXuu+8+TZkyRefPn1dQUJDGjx+vN998k08lAkANwSUz3DVOnDihsLAw3XfffRoyZIiys7PLnJeRkaHMzEyNGjXqlvsrKSnR0KFDNWXKFLVu3brU9vbt2+uzzz7T1atXtWXLFjVs2FANGjTQihUr5Ovrq/79+1fIcQEAKh9niHBXiIqK0vLlyxUREaGzZ89q1qxZeuihh3T48OFSX8j1wQcfqFWrVvrNb35zy32++eab8vLy0qRJk8rcPnLkSH355ZeKjIxUgwYNtHr1al28eFEzZsxQamqqXnrpJa1atUr333+/li1bpnvvvbfCjhcAULEIItwVfngvULt27RQVFaUmTZpo9erVLmeCrl69qpUrV+rll1++5f4yMjK0YMECHTx48Ce/0KtWrVpavHixy9iIESM0adIkff7551q/fr2++OILzZkzR5MmTdLatWvv4AgBAJWJS2a4K9WtW1ctWrTQN9984zL+6aef6vvvv9ewYcNu+fp//OMfysvLU+PGjeXl5SUvLy+dOnVKzz//vJo2bVrma3bs2KEjR45owoQJSk1NVd++feXv76/BgwcrNTW1go4MAFAZOEOEu9Lly5d18uRJDR061GX8gw8+0GOPPaagoKBbvn7o0KGKjo52GYuJidHQoUM1YsSIUvOvXbumuLg4rVixQp6eniouLpbD4ZAkXb9+XcXFxXd4RACAysQZItwVXnjhBaWlpenbb7/V7t271b9/f3l6euqZZ55xzvnmm2+0c+dOjR49usx9tGzZUuvWrZMk1a9fX23atHF51KpVS6GhoYqIiCj12tdee019+/bVr3/9a0lSt27d9Je//EVffvmlFi1apG7dulXCUQMAKgpniHBX+L//+z8988wz+u677xQUFKQHH3xQe/bscTkTtGzZMjVq1Ei9e/cucx/Hjx9XQUHBbf/sw4cPa/Xq1crMzHSODRo0SKmpqXrooYcUERGhlStX3vZ+AQBVx+K4eV4fP8lutyswMFAFBQV8rwwAADXE7fz+5pIZAAAwHpfMqpFOUz5y9xKAailj7q0/FQgAd4ozRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjVZsgeuONN2SxWBQfH+8cu3btmuLi4lS/fn0FBARo4MCBys3NdXlddna2YmNjVbt2bQUHB2vKlCm6ceOGy5zU1FR17NhRPj4+at68uZYvX14FRwQAAGqKahFE+/fv13vvvad27dq5jE+ePFkbNmzQmjVrlJaWpjNnzmjAgAHO7cXFxYqNjVVRUZF2796tDz/8UMuXL9eMGTOcc7KyshQbG6uePXsqMzNT8fHxGj16tLZs2VJlxwcAAKo3twfR5cuXNWTIEC1ZskT33HOPc7ygoEAffPCB3nnnHfXq1UudOnXSn//8Z+3evVt79uyRJG3dulVfffWVPv74Y3Xo0EF9+vTRa6+9psWLF6uoqEiSlJSUpGbNmuntt99Wq1atNGHCBA0aNEjz5s1zy/ECAIDqx+1BFBcXp9jYWEVHR7uMZ2Rk6Pr16y7jLVu2VOPGjZWeni5JSk9PV9u2bRUSEuKcExMTI7vdriNHjjjn/HjfMTExzn2UpbCwUHa73eUBAADuXl7u/OGrVq3SwYMHtX///lLbcnJy5O3trbp167qMh4SEKCcnxznnhzF0c/vNbbeaY7fbdfXqVfn5+ZX62YmJiZo1a1a5jwsAANQsbjtDdPr0af3+97/XihUr5Ovr665llGn69OkqKChwPk6fPu3uJQEAgErktiDKyMhQXl6eOnbsKC8vL3l5eSktLU0LFy6Ul5eXQkJCVFRUpPz8fJfX5ebmKjQ0VJIUGhpa6lNnN5//3Byr1Vrm2SFJ8vHxkdVqdXkAAIC7l9uC6OGHH9ahQ4eUmZnpfHTu3FlDhgxx/rlWrVpKSUlxvub48ePKzs6WzWaTJNlsNh06dEh5eXnOOdu2bZPValVkZKRzzg/3cXPOzX0AAAC47R6iOnXqqE2bNi5j/v7+ql+/vnN81KhRSkhIUL169WS1WjVx4kTZbDZ17dpVktS7d29FRkZq6NChmjNnjnJycvTSSy8pLi5OPj4+kqRx48Zp0aJFmjp1qkaOHKnt27dr9erVSk5OrtoDBgAA1ZZbb6r+OfPmzZOHh4cGDhyowsJCxcTE6E9/+pNzu6enpzZu3Kjx48fLZrPJ399fw4cP16uvvuqc06xZMyUnJ2vy5MlasGCBGjVqpKVLlyomJsYdhwQAAKohi8PhcLh7EdWd3W5XYGCgCgoKKvV+ok5TPqq0fQM1WcbcYe5eAoAa6HZ+f7v9e4gAAADcjSACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAAA1xhtvvCGLxaL4+HhJ0oULFzRx4kRFRETIz89PjRs31qRJk1RQUPCz+zp69Kgee+wxBQYGyt/fX126dFF2drZze0JCgurVq6fw8HCtWLHC5bVr1qxRv379KvTY4F5e7l4AAAC/xP79+/Xee++pXbt2zrEzZ87ozJkzeuuttxQZGalTp05p3LhxOnPmjD799NOf3NfJkyf14IMPatSoUZo1a5asVquOHDkiX19fSdKGDRu0cuVKbd26VSdOnNDIkSMVExOjBg0aqKCgQC+++KL+/ve/V/oxo+oQRACAau/y5csaMmSIlixZotdff9053qZNG61du9b5/P7779fs2bP13HPP6caNG/LyKvvX3Isvvqi+fftqzpw5Lq+96ejRo+rRo4c6d+6szp07Kz4+XllZWWrQoIGmTp2q8ePHq3HjxpVwpHAXLpkBAKq9uLg4xcbGKjo6+mfnFhQUyGq1/mQMlZSUKDk5WS1atFBMTIyCg4MVFRWl9evXO+e0b99eBw4c0MWLF5WRkaGrV6+qefPm+uyzz3Tw4EFNmjSpog4N1QRBBACo1latWqWDBw8qMTHxZ+eeP39er732msaOHfuTc/Ly8nT58mW98cYbeuSRR7R161b1799fAwYMUFpamiQpJiZGzz33nLp06aLf/va3+vDDD+Xv76/x48crKSlJ7777riIiItStWzcdOXKkwo4V7sMlMwBAtXX69Gn9/ve/17Zt25z39/wUu92u2NhYRUZGaubMmT85r6SkRJL0+OOPa/LkyZKkDh06aPfu3UpKStJ//Md/SJJmzpzpsp9Zs2YpOjpatWrV0uuvv65Dhw5p48aNGjZsmDIyMu7sQOF2nCECAFRbGRkZysvLU8eOHeXl5SUvLy+lpaVp4cKF8vLyUnFxsSTp0qVLeuSRR1SnTh2tW7dOtWrV+sl9NmjQQF5eXoqMjHQZb9WqlcunzH7o2LFj+vjjj/Xaa68pNTVV3bt3V1BQkAYPHqyDBw/q0qVLFXfQcAvOEAEAqq2HH35Yhw4dchkbMWKEWrZsqWnTpsnT01N2u10xMTHy8fHR3/72t589k+Tt7a0uXbro+PHjLuNff/21mjRpUmq+w+HQf/3Xf+mdd95RQECAiouLdf36dUly/u/NMEPNRRABAKqtOnXqqE2bNi5j/v7+ql+/vtq0aSO73a7evXvr+++/18cffyy73S673S5JCgoKkqenpySpZcuWSkxMVP/+/SVJU6ZM0VNPPaXu3burZ8+e2rx5szZs2KDU1NRSa1i6dKmCgoKc3zvUrVs3zZw5U3v27NGmTZsUGRmpunXrVt5fAqoEQQQAqLEOHjyovXv3SpKaN2/usi0rK0tNmzaVJB0/ftzlyxr79++vpKQkJSYmatKkSYqIiNDatWv14IMPuuwjNzdXs2fP1u7du51jDzzwgJ5//nnFxsYqODhYH374YSUdHaqSxeFwONy9iOrObrcrMDDQ+VHOytJpykeVtm+gJsuYO8zdSwBQA93O729uqgYAAMbjkhkAVAHOAANlqy5ngDlDBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMJ5bg+jdd99Vu3btZLVaZbVaZbPZtGnTJuf2a9euKS4uTvXr11dAQIAGDhyo3Nxcl31kZ2crNjZWtWvXVnBwsKZMmaIbN264zElNTVXHjh3l4+Oj5s2ba/ny5VVxeAAAoIZwaxA1atRIb7zxhjIyMnTgwAH16tVLjz/+uI4cOSJJmjx5sjZs2KA1a9YoLS1NZ86c0YABA5yvLy4uVmxsrIqKirR79259+OGHWr58uWbMmOGck5WVpdjYWPXs2VOZmZmKj4/X6NGjtWXLlio/XgAAUD1ZHA6Hw92L+KF69epp7ty5GjRokIKCgrRy5UoNGjRIknTs2DG1atVK6enp6tq1qzZt2qRHH31UZ86cUUhIiCQpKSlJ06ZN07lz5+Tt7a1p06YpOTlZhw8fdv6Mp59+Wvn5+dq8eXOZaygsLFRhYaHzud1uV3h4uAoKCmS1Wivt2DtN+ajS9g3UZBlzh7l7CXeM9zdQtsp8f9vtdgUGBv6i39/V5h6i4uJirVq1SleuXJHNZlNGRoauX7+u6Oho55yWLVuqcePGSk9PlySlp6erbdu2zhiSpJiYGNntdudZpvT0dJd93Jxzcx9lSUxMVGBgoPMRHh5ekYcKAACqGbcH0aFDhxQQECAfHx+NGzdO69atU2RkpHJycuTt7a26deu6zA8JCVFOTo4kKScnxyWGbm6/ue1Wc+x2u65evVrmmqZPn66CggLn4/Tp0xVxqAAAoJrycvcCIiIilJmZqYKCAn366acaPny40tLS3LomHx8f+fj4uHUNAACg6rg9iLy9vdW8eXNJUqdOnbR//34tWLBATz31lIqKipSfn+9ylig3N1ehoaGSpNDQUO3bt89lfzc/hfbDOT/+ZFpubq6sVqv8/Pwq67AAAEAN4vZLZj9WUlKiwsJCderUSbVq1VJKSopz2/Hjx5WdnS2bzSZJstlsOnTokPLy8pxztm3bJqvVqsjISOecH+7j5pyb+wAAAHDrGaLp06erT58+aty4sS5duqSVK1cqNTVVW7ZsUWBgoEaNGqWEhATVq1dPVqtVEydOlM1mU9euXSVJvXv3VmRkpIYOHao5c+YoJydHL730kuLi4pyXvMaNG6dFixZp6tSpGjlypLZv367Vq1crOTnZnYcOAACqEbcGUV5enoYNG6azZ88qMDBQ7dq105YtW/Sf//mfkqR58+bJw8NDAwcOVGFhoWJiYvSnP/3J+XpPT09t3LhR48ePl81mk7+/v4YPH65XX33VOadZs2ZKTk7W5MmTtWDBAjVq1EhLly5VTExMlR8vAAConqrd9xBVR7fzPQZ3gu8pAcrG9xABdy++hwgAAKCaIIgAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYr1xB1KtXL+Xn55cat9vt6tWr152uCQAAoEqVK4hSU1NVVFRUavzatWv6xz/+cceLAgAAqEpetzP5yy+/dP75q6++Uk5OjvN5cXGxNm/erHvvvbfiVgcAAFAFbiuIOnToIIvFIovFUualMT8/P/3xj3+ssMUBAABUhdsKoqysLDkcDt13333at2+fgoKCnNu8vb0VHBwsT0/PCl8kAABAZbqtIGrSpIkkqaSkpFIWAwAA4A63FUQ/dOLECe3YsUN5eXmlAmnGjBl3vDAAAICqUq4gWrJkicaPH68GDRooNDRUFovFuc1isRBEAACgRilXEL3++uuaPXu2pk2bVtHrAQAAqHLl+h6iixcv6sknn6zotQAAALhFuYLoySef1NatWyt6LQAAAG5RrktmzZs318svv6w9e/aobdu2qlWrlsv2SZMmVcjiAAAAqkK5guj9999XQECA0tLSlJaW5rLNYrEQRAAAoEYpVxBlZWVV9DoAAADcplz3EAEAANxNynWGaOTIkbfcvmzZsnItBgAAwB3KFUQXL150eX79+nUdPnxY+fn5Zf6jrwAAANVZuYJo3bp1pcZKSko0fvx43X///Xe8KAAAgKpUYfcQeXh4KCEhQfPmzauoXQIAAFSJCr2p+uTJk7px40ZF7hIAAKDSleuSWUJCgstzh8Ohs2fPKjk5WcOHD6+QhQEAAFSVcgXR559/7vLcw8NDQUFBevvtt3/2E2gAAADVTbmCaMeOHRW9DgAAALcpVxDddO7cOR0/flySFBERoaCgoApZFAAAQFUq103VV65c0ciRI9WwYUN1795d3bt3V1hYmEaNGqXvv/++otcIAABQqcoVRAkJCUpLS9OGDRuUn5+v/Px8/fWvf1VaWpqef/75il4jAABApSrXJbO1a9fq008/VY8ePZxjffv2lZ+fnwYPHqx33323otYHAABQ6cp1huj7779XSEhIqfHg4GAumQEAgBqnXEFks9n0yiuv6Nq1a86xq1evatasWbLZbBW2OAAAgKpQrktm8+fP1yOPPKJGjRqpffv2kqQvvvhCPj4+2rp1a4UuEAAAoLKVK4jatm2rEydOaMWKFTp27Jgk6ZlnntGQIUPk5+dXoQsEAACobOUKosTERIWEhGjMmDEu48uWLdO5c+c0bdq0ClkcAABAVSjXPUTvvfeeWrZsWWq8devWSkpKuuNFAQAAVKVyBVFOTo4aNmxYajwoKEhnz56940UBAABUpXIFUXh4uHbt2lVqfNeuXQoLC7vjRQEAAFSlcgXRmDFjFB8frz//+c86deqUTp06pWXLlmny5Mml7iu6lcTERHXp0kV16tRRcHCwnnjiCee/jXbTtWvXFBcXp/r16ysgIEADBw5Ubm6uy5zs7GzFxsaqdu3aCg4O1pQpU3Tjxg2XOampqerYsaN8fHzUvHlzLV++vDyHDgAA7kLluql6ypQp+u677/S73/1ORUVFkiRfX19NmzZN06dP/8X7SUtLU1xcnLp06aIbN27of/7nf9S7d2999dVX8vf3lyRNnjxZycnJWrNmjQIDAzVhwgQNGDDAeYaquLhYsbGxCg0N1e7du3X27FkNGzZMtWrV0h/+8AdJUlZWlmJjYzVu3DitWLFCKSkpGj16tBo2bKiYmJjy/BUAAIC7iMXhcDjK++LLly/r6NGj8vPz069+9Sv5+Pjc0WLOnTun4OBgpaWlqXv37iooKFBQUJBWrlypQYMGSZKOHTumVq1aKT09XV27dtWmTZv06KOP6syZM85vz05KStK0adN07tw5eXt7a9q0aUpOTtbhw4edP+vpp59Wfn6+Nm/eXGodhYWFKiwsdD632+0KDw9XQUGBrFbrHR3jrXSa8lGl7RuoyTLmDnP3Eu4Y72+gbJX5/rbb7QoMDPxFv7/LdcnspoCAAHXp0kVt2rS54xiSpIKCAklSvXr1JEkZGRm6fv26oqOjnXNatmypxo0bKz09XZKUnp6utm3buvxTIjExMbLb7Tpy5Ihzzg/3cXPOzX38WGJiogIDA52P8PDwOz42AABQfd1REFWkkpISxcfHq1u3bmrTpo2kf32azdvbW3Xr1nWZGxISopycHOecH/+7ajef/9wcu92uq1evllrL9OnTVVBQ4HycPn26Qo4RAABUT+W6h6gyxMXF6fDhw/rss8/cvRT5+PhUyBkvAABQM1SLM0QTJkzQxo0btWPHDjVq1Mg5HhoaqqKiIuXn57vMz83NVWhoqHPOjz91dvP5z82xWq38UyMAAMC9QeRwODRhwgStW7dO27dvV7NmzVy2d+rUSbVq1VJKSopz7Pjx48rOzpbNZpMk2Ww2HTp0SHl5ec4527Ztk9VqVWRkpHPOD/dxc87NfQAAALO59ZJZXFycVq5cqb/+9a+qU6eO856fwMBA+fn5KTAwUKNGjVJCQoLq1asnq9WqiRMnymazqWvXrpKk3r17KzIyUkOHDtWcOXOUk5Ojl156SXFxcc7LXuPGjdOiRYs0depUjRw5Utu3b9fq1auVnJzstmMHAADVh1vPEL377rsqKChQjx491LBhQ+fjk08+cc6ZN2+eHn30UQ0cOFDdu3dXaGio/vKXvzi3e3p6auPGjfL09JTNZtNzzz2nYcOG6dVXX3XOadasmZKTk7Vt2za1b99eb7/9tpYuXcp3EAEAAEluPkP0S74CydfXV4sXL9bixYt/ck6TJk30v//7v7fcT48ePfT555/f9hoBAMDdr1rcVA0AAOBOBBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwHkEEAACMRxABAADjEUQAAMB4BBEAADAeQQQAAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAwnluDaOfOnerXr5/CwsJksVi0fv16l+0Oh0MzZsxQw4YN5efnp+joaJ04ccJlzoULFzRkyBBZrVbVrVtXo0aN0uXLl13mfPnll3rooYfk6+ur8PBwzZkzp7IPDQAA1CBuDaIrV66offv2Wrx4cZnb58yZo4ULFyopKUl79+6Vv7+/YmJidO3aNeecIUOG6MiRI9q2bZs2btyonTt3auzYsc7tdrtdvXv3VpMmTZSRkaG5c+dq5syZev/99yv9+AAAQM3g5c4f3qdPH/Xp06fMbQ6HQ/Pnz9dLL72kxx9/XJL00UcfKSQkROvXr9fTTz+to0ePavPmzdq/f786d+4sSfrjH/+ovn376q233lJYWJhWrFihoqIiLVu2TN7e3mrdurUyMzP1zjvvuIQTAAAwV7W9hygrK0s5OTmKjo52jgUGBioqKkrp6emSpPT0dNWtW9cZQ5IUHR0tDw8P7d271zmne/fu8vb2ds6JiYnR8ePHdfHixTJ/dmFhoex2u8sDAADcvaptEOXk5EiSQkJCXMZDQkKc23JychQcHOyy3cvLS/Xq1XOZU9Y+fvgzfiwxMVGBgYHOR3h4+J0fEAAAqLaqbRC50/Tp01VQUOB8nD592t1LAgAAlajaBlFoaKgkKTc312U8NzfXuS00NFR5eXku22/cuKELFy64zClrHz/8GT/m4+Mjq9Xq8gAAAHevahtEzZo1U2hoqFJSUpxjdrtde/fulc1mkyTZbDbl5+crIyPDOWf79u0qKSlRVFSUc87OnTt1/fp155xt27YpIiJC99xzTxUdDQAAqM7cGkSXL19WZmamMjMzJf3rRurMzExlZ2fLYrEoPj5er7/+uv72t7/p0KFDGjZsmMLCwvTEE09Iklq1aqVHHnlEY8aM0b59+7Rr1y5NmDBBTz/9tMLCwiRJzz77rLy9vTVq1CgdOXJEn3zyiRYsWKCEhAQ3HTUAAKhu3Pqx+wMHDqhnz57O5zcjZfjw4Vq+fLmmTp2qK1euaOzYscrPz9eDDz6ozZs3y9fX1/maFStWaMKECXr44Yfl4eGhgQMHauHChc7tgYGB2rp1q+Li4tSpUyc1aNBAM2bM4CP3AADAyeJwOBzuXkR1Z7fbFRgYqIKCgkq9n6jTlI8qbd9ATZYxd5i7l3DHeH8DZavM9/ft/P6utvcQAQAAVBWCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gAAIDxCCIAAGA8gggAABiPIAIAAMYzKogWL16spk2bytfXV1FRUdq3b5+7lwQAAKoBY4Lok08+UUJCgl555RUdPHhQ7du3V0xMjPLy8ty9NAAA4GbGBNE777yjMWPGaMSIEYqMjFRSUpJq166tZcuWuXtpAADAzbzcvYCqUFRUpIyMDE2fPt055uHhoejoaKWnp5eaX1hYqMLCQufzgoICSZLdbq/UdRYXXq3U/QM1VWW/96oC72+gbJX5/r65b4fD8bNzjQii8+fPq7i4WCEhIS7jISEhOnbsWKn5iYmJmjVrVqnx8PDwSlsjgJ8W+Mdx7l4CgEpSFe/vS5cuKTAw8JZzjAii2zV9+nQlJCQ4n5eUlOjChQuqX7++LBaLG1eGqmC32xUeHq7Tp0/LarW6ezkAKhDvb7M4HA5dunRJYWFhPzvXiCBq0KCBPD09lZub6zKem5ur0NDQUvN9fHzk4+PjMla3bt3KXCKqIavVyn8wgbsU729z/NyZoZuMuKna29tbnTp1UkpKinOspKREKSkpstlsblwZAACoDow4QyRJCQkJGj58uDp37qwHHnhA8+fP15UrVzRixAh3Lw0AALiZMUH01FNP6dy5c5oxY4ZycnLUoUMHbd68udSN1oCPj49eeeWVUpdNAdR8vL/xUyyOX/JZNAAAgLuYEfcQAQAA3ApBBAAAjEcQAQAA4xFEAADAeAQR8COLFy9W06ZN5evrq6ioKO3bt8/dSwJQAXbu3Kl+/fopLCxMFotF69evd/eSUI0QRMAPfPLJJ0pISNArr7yigwcPqn379oqJiVFeXp67lwbgDl25ckXt27fX4sWL3b0UVEN87B74gaioKHXp0kWLFi2S9K9vNA8PD9fEiRP13//9325eHYCKYrFYtG7dOj3xxBPuXgqqCc4QAf9WVFSkjIwMRUdHO8c8PDwUHR2t9PR0N64MAFDZCCLg386fP6/i4uJS314eEhKinJwcN60KAFAVCCIAAGA8ggj4twYNGsjT01O5ubku47m5uQoNDXXTqgAAVYEgAv7N29tbnTp1UkpKinOspKREKSkpstlsblwZAKCyGfOv3QO/REJCgoYPH67OnTvrgQce0Pz583XlyhWNGDHC3UsDcIcuX76sb775xvk8KytLmZmZqlevnho3buzGlaE64GP3wI8sWrRIc+fOVU5Ojjp06KCFCxcqKirK3csCcIdSU1PVs2fPUuPDhw/X8uXLq35BqFYIIgAAYDzuIQIAAMYjiAAAgPEIIgAAYDyCCAAAGI8gAgAAxiOIAACA8QgiAABgPIIIAAAYjyACAADGI4gA1Eg9evRQfHy8u5fhVN3WA+D2EEQAjFVUVOTuJQCoJggiADXOb3/7W6WlpWnBggWyWCyyWCw6efKkRo0apWbNmsnPz08RERFasGBBqdc98cQTmj17tsLCwhQRESFJ2r17tzp06CBfX1917txZ69evl8ViUWZmpvO1hw8fVp8+fRQQEKCQkBANHTpU58+f/8n1fPvtt1X11wGgAni5ewEAcLsWLFigr7/+Wm3atNGrr74qSbrnnnvUqFEjrVmzRvXr19fu3bs1duxYNWzYUIMHD3a+NiUlRVarVdu2bZMk2e129evXT3379tXKlSt16tSpUpe+8vPz1atXL40ePVrz5s3T1atXNW3aNA0ePFjbt28vcz1BQUFV85cBoEIQRABqnMDAQHl7e6t27doKDQ11js+aNcv552bNmik9PV2rV692CSJ/f38tXbpU3t7ekqSkpCRZLBYtWbJEvr6+ioyM1D//+U+NGTPG+ZpFixbp17/+tf7whz84x5YtW6bw8HB9/fXXatGiRZnrAVBzEEQA7hqLFy/WsmXLlJ2dratXr6qoqEgdOnRwmdO2bVtnDEnS8ePH1a5dO/n6+jrHHnjgAZfXfPHFF9qxY4cCAgJK/cyTJ0+qRYsWFXsgAKocQQTgrrBq1Sq98MILevvtt2Wz2VSnTh3NnTtXe/fudZnn7+9/2/u+fPmy+vXrpzfffLPUtoYNG5Z7zQCqD4IIQI3k7e2t4uJi5/Ndu3bpN7/5jX73u985x06ePPmz+4mIiNDHH3+swsJC+fj4SJL279/vMqdjx45au3atmjZtKi+vsv+z+eP1AKhZ+JQZgBqpadOm2rt3r7799ludP39ev/rVr3TgwAFt2bJFX3/9tV5++eVSYVOWZ599ViUlJRo7dqyOHj2qLVu26K233pIkWSwWSVJcXJwuXLigZ555Rvv379fJkye1ZcsWjRgxwhlBP15PSUlJ5R08gApHEAGokV544QV5enoqMjJSQUFBiomJ0YABA/TUU08pKipK3333ncvZop9itVq1YcMGZWZmqkOHDnrxxRc1Y8YMSXLeVxQWFqZdu3apuLhYvXv3Vtu2bRUfH6+6devKw8OjzPVkZ2dX3sEDqHAWh8PhcPciAKA6WbFihUaMGKGCggL5+fm5ezkAqgD3EAEw3kcffaT77rtP9957r7744gvndwwRQ4A5CCIAxsvJydGMGTOUk5Ojhg0b6sknn9Ts2bPdvSwAVYhLZgAAwHjcVA0AAIxHEAEAAOMRRAAAwHgEEQAAMB5BBAAAjEcQAQAA4xFEAADAeAQRAAAw3v8DzzWy09xNexcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = sns.countplot(data=tweet, x='target')\n",
    "p.bar_label(p.containers[0], fmt=lambda x: f'{(x/len(tweet))*100:0.1f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### After removing duplicates from the tweet dataframe 'text' column, 57% are now 0 or NON disaster tweets and 42% ARE classified as disaster tweets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Observing some of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['#TeamFollowBack Madhya Pradesh Train Derailment: Village Youth Saved Many Lives  #FollowBack'\n",
      " 'Firefigthers Evacuate from Northampton Township House Fire http://t.co/hPplD1jHtZ'\n",
      " \"@breakingnewslh @bree_mars watch cnn's the seventies terrorism episode. Iran has always hated the U.S. They want us obliterated.\"]\n"
     ]
    }
   ],
   "source": [
    "# First, 3 samples of True disaster tweets\n",
    "np.random.seed(10)\n",
    "print(tweet[tweet['target']==1].sample(3)['text'].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['My 2 fav worlds have collided! Thanks to @lennonparham @Jessica_StClair I found the @GilmoreGuysShow podcast!! #ihave44episodesofGG #nojoke'\n",
      " 'Not because i want to cheat or anything. Just feels good to vent to my twitter without a txt being screamed at lol'\n",
      " \"i still don't see the point of a frozen sequel like where's the wreck it ralph sequel\"]\n"
     ]
    }
   ],
   "source": [
    "# Next, 3 samples of tweets that are NOT disaster tweets\n",
    "print(tweet[tweet['target']==0].sample(3)['text'].values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text Cleaning & Preprocessing\n",
    "\n",
    "#### In natural language processing, text needs to be cleaned and preprocessed before being fed into a model. Removing punctuations, stop words (such as 'I', 'is', 'the', etc), setting text to lowercase, port stemming (leaving the root word be removing tense) and also tokenizing or using a countvectorizer helps the model process and analyze text easier.\n",
    "\n",
    "#### Below, I created the clean_text() function to take in each text line, set it to lowercase, remove punctuation, remove English stopwords, stem the words, then join as string for the return value. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "stop_words = stopwords.words('english')\n",
    "np.array(stop_words)\n",
    "stemmer = PorterStemmer()\n",
    "\n",
    "def clean_text(text):\n",
    "   # text to lowercase\n",
    "   lower = text.lower()\n",
    "   # remove punctuation\n",
    "   import string\n",
    "   trans = str.maketrans('', '', string.punctuation)\n",
    "   no_punc = lower.translate(trans)\n",
    "   # remove stopwords\n",
    "   tokens = word_tokenize(no_punc)\n",
    "   no_stop = [word for word in tokens if word not in stop_words]\n",
    "   # Stem words\n",
    "   stemmed = [stemmer.stem(word) for word in no_stop]\n",
    "   return ' '.join(stemmed)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Our Deeds are the Reason of this #earthquake M...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Forest fire near La Ronge Sask. Canada</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>All residents asked to 'shelter in place' are ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>13,000 people receive #wildfires evacuation or...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Just got sent this photo from Ruby #Alaska as ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>#RockyFire Update =&gt; California Hwy. 20 closed...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>#flood #disaster Heavy rain causes flash flood...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>I'm on top of the hill and I can see a fire in...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>There's an emergency evacuation happening now ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>I'm afraid that the tornado is coming to our a...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text  target\n",
       "0  Our Deeds are the Reason of this #earthquake M...       1\n",
       "1             Forest fire near La Ronge Sask. Canada       1\n",
       "2  All residents asked to 'shelter in place' are ...       1\n",
       "3  13,000 people receive #wildfires evacuation or...       1\n",
       "4  Just got sent this photo from Ruby #Alaska as ...       1\n",
       "5  #RockyFire Update => California Hwy. 20 closed...       1\n",
       "6  #flood #disaster Heavy rain causes flash flood...       1\n",
       "7  I'm on top of the hill and I can see a fire in...       1\n",
       "8  There's an emergency evacuation happening now ...       1\n",
       "9  I'm afraid that the tornado is coming to our a...       1"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "tweet['text'] = tweet['text'].apply(clean_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying clean_text() function\n",
    "\n",
    "#### We can see the difference from the original text above to the cleaned text below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0            deed reason earthquak may allah forgiv us\n",
       "1                 forest fire near la rong sask canada\n",
       "2    resid ask shelter place notifi offic evacu she...\n",
       "3    13000 peopl receiv wildfir evacu order california\n",
       "4    got sent photo rubi alaska smoke wildfir pour ...\n",
       "5    rockyfir updat california hwi 20 close direct ...\n",
       "6    flood disast heavi rain caus flash flood stree...\n",
       "7                            im top hill see fire wood\n",
       "8         there emerg evacu happen build across street\n",
       "9                          im afraid tornado come area\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet['text'][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SKLearn's CountVectorizer\n",
    "\n",
    "#### We can now apply CountVectorizer which transforms the text into a vector based on frequency of each word as it occurs in the text. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-5 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-5 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-5 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-5 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-5 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-5 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-5 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-5 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-5 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-5 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>CountVectorizer()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" checked><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;CountVectorizer<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.4/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html\">?<span>Documentation for CountVectorizer</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>CountVectorizer()</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "CountVectorizer()"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_vec = CountVectorizer()\n",
    "count_vec.fit(tweet['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'deed': 3790,\n",
       " 'reason': 15271,\n",
       " 'earthquak': 4384,\n",
       " 'may': 12886,\n",
       " 'allah': 1058,\n",
       " 'forgiv': 5248,\n",
       " 'us': 18270,\n",
       " 'forest': 5241,\n",
       " 'fire': 5089,\n",
       " 'near': 13609,\n",
       " 'la': 12120,\n",
       " 'rong': 15688,\n",
       " 'sask': 15943,\n",
       " 'canada': 2558,\n",
       " 'resid': 15459,\n",
       " 'ask': 1434,\n",
       " 'shelter': 16260,\n",
       " 'place': 14634,\n",
       " 'notifi': 13856,\n",
       " 'offic': 13993,\n",
       " 'evacu': 4737,\n",
       " 'order': 14163,\n",
       " 'expect': 4803,\n",
       " '13000': 138,\n",
       " 'peopl': 14472,\n",
       " 'receiv': 15287,\n",
       " 'wildfir': 18802,\n",
       " 'california': 2521,\n",
       " 'got': 5726,\n",
       " 'sent': 16142,\n",
       " 'photo': 14556,\n",
       " 'rubi': 15769,\n",
       " 'alaska': 1012,\n",
       " 'smoke': 16544,\n",
       " 'pour': 14782,\n",
       " 'school': 16010,\n",
       " 'rockyfir': 15647,\n",
       " 'updat': 18235,\n",
       " 'hwi': 11063,\n",
       " '20': 253,\n",
       " 'close': 3028,\n",
       " 'direct': 4010,\n",
       " 'due': 4313,\n",
       " 'lake': 12140,\n",
       " 'counti': 3366,\n",
       " 'cafir': 2494,\n",
       " 'flood': 5163,\n",
       " 'disast': 4025,\n",
       " 'heavi': 6121,\n",
       " 'rain': 15160,\n",
       " 'caus': 2692,\n",
       " 'flash': 5132,\n",
       " 'street': 16995,\n",
       " 'manit': 12755,\n",
       " 'colorado': 3142,\n",
       " 'spring': 16796,\n",
       " 'area': 1334,\n",
       " 'im': 11193,\n",
       " 'top': 17784,\n",
       " 'hill': 6228,\n",
       " 'see': 16091,\n",
       " 'wood': 18908,\n",
       " 'there': 17543,\n",
       " 'emerg': 4535,\n",
       " 'happen': 5998,\n",
       " 'build': 2389,\n",
       " 'across': 832,\n",
       " 'afraid': 908,\n",
       " 'tornado': 17796,\n",
       " 'come': 3160,\n",
       " 'three': 17617,\n",
       " 'die': 3979,\n",
       " 'heat': 6116,\n",
       " 'wave': 18625,\n",
       " 'far': 4932,\n",
       " 'haha': 5929,\n",
       " 'south': 16695,\n",
       " 'tampa': 17295,\n",
       " 'get': 5574,\n",
       " 'hah': 5928,\n",
       " 'wait': 18544,\n",
       " 'second': 16081,\n",
       " 'live': 12439,\n",
       " 'gon': 5696,\n",
       " 'na': 13517,\n",
       " 'fvck': 5442,\n",
       " 'florida': 5173,\n",
       " 'tampabay': 17296,\n",
       " '18': 195,\n",
       " '19': 211,\n",
       " 'day': 3718,\n",
       " 'ive': 11515,\n",
       " 'lost': 12553,\n",
       " 'count': 3361,\n",
       " 'bago': 1654,\n",
       " 'myanmar': 13495,\n",
       " 'arriv': 1383,\n",
       " 'damag': 3621,\n",
       " 'bu': 2368,\n",
       " '80': 633,\n",
       " 'multi': 13437,\n",
       " 'car': 2605,\n",
       " 'crash': 3412,\n",
       " 'break': 2255,\n",
       " 'what': 18737,\n",
       " 'man': 12740,\n",
       " 'love': 12565,\n",
       " 'fruit': 5386,\n",
       " 'summer': 17096,\n",
       " 'fast': 4948,\n",
       " 'goooooooaaaaaal': 5713,\n",
       " 'ridicul': 15558,\n",
       " 'london': 12508,\n",
       " 'cool': 3296,\n",
       " 'ski': 16449,\n",
       " 'wonder': 18903,\n",
       " 'looooool': 12530,\n",
       " 'wayi': 18628,\n",
       " 'cant': 2579,\n",
       " 'eat': 4398,\n",
       " 'shit': 16283,\n",
       " 'nyc': 13924,\n",
       " 'last': 12187,\n",
       " 'week': 18679,\n",
       " 'girlfriend': 5620,\n",
       " 'cooool': 3300,\n",
       " 'like': 12382,\n",
       " 'pasta': 14386,\n",
       " 'end': 4583,\n",
       " 'bbcmtd': 1775,\n",
       " 'wholesal': 18776,\n",
       " 'market': 12806,\n",
       " 'ablaz': 771,\n",
       " 'httptcolhyxeohy6c': 9078,\n",
       " 'alway': 1110,\n",
       " 'tri': 17917,\n",
       " 'bring': 2303,\n",
       " 'metal': 13026,\n",
       " 'rt': 15755,\n",
       " 'httptcoyao1e0xngw': 10785,\n",
       " 'africanbaz': 911,\n",
       " 'newsnigeria': 13684,\n",
       " 'flag': 5126,\n",
       " 'set': 16172,\n",
       " 'aba': 745,\n",
       " 'httptco2nndbgwyei': 6990,\n",
       " 'cri': 3451,\n",
       " 'plu': 14678,\n",
       " 'side': 16335,\n",
       " 'look': 12526,\n",
       " 'sky': 16459,\n",
       " 'night': 13730,\n",
       " 'httptcoqqsmshaj3n': 9788,\n",
       " 'phdsquar': 14538,\n",
       " 'mufc': 13428,\n",
       " 'theyv': 17568,\n",
       " 'built': 2393,\n",
       " 'much': 13424,\n",
       " 'hype': 11072,\n",
       " 'around': 1376,\n",
       " 'new': 13660,\n",
       " 'acquisit': 829,\n",
       " 'doubt': 4196,\n",
       " 'epl': 4654,\n",
       " 'season': 16069,\n",
       " 'inec': 11297,\n",
       " 'abia': 767,\n",
       " 'httptco3imaomknna': 7051,\n",
       " 'barbado': 1708,\n",
       " 'bridgetown': 2290,\n",
       " 'jamaica': 11557,\n",
       " 'ûò': 19314,\n",
       " 'two': 18069,\n",
       " 'santa': 15928,\n",
       " 'cruz': 3500,\n",
       " 'ûó': 19318,\n",
       " 'head': 6086,\n",
       " 'st': 16834,\n",
       " 'elizabeth': 4507,\n",
       " 'polic': 14711,\n",
       " 'superintend': 17130,\n",
       " 'httptcowdueaj8q4j': 10538,\n",
       " 'lord': 12539,\n",
       " 'check': 2819,\n",
       " 'httptcoroi2nsmejj': 9923,\n",
       " 'httptco3tj8zjin21': 7074,\n",
       " 'httptcoyduixefip': 10798,\n",
       " 'httptcolxtjc87kl': 9135,\n",
       " 'nsfw': 13886,\n",
       " 'outsid': 14225,\n",
       " 'your': 19152,\n",
       " 'aliv': 1056,\n",
       " 'dead': 3737,\n",
       " 'insid': 11354,\n",
       " 'awesom': 1589,\n",
       " 'time': 17670,\n",
       " 'visit': 18474,\n",
       " 'cfc': 2751,\n",
       " 'ancop': 1184,\n",
       " 'site': 16420,\n",
       " 'thank': 17472,\n",
       " 'tita': 17695,\n",
       " 'vida': 18429,\n",
       " 'take': 17274,\n",
       " 'care': 2615,\n",
       " 'soooo': 16666,\n",
       " 'pump': 15021,\n",
       " 'southridgelif': 16704,\n",
       " 'want': 18564,\n",
       " 'chicago': 2851,\n",
       " 'preach': 14820,\n",
       " 'hotel': 6378,\n",
       " 'httptcoo9qknbfofx': 9452,\n",
       " 'gain': 5463,\n",
       " 'follow': 5206,\n",
       " 'know': 12041,\n",
       " 'stat': 16882,\n",
       " 'grow': 5834,\n",
       " 'httptcotiyulif5c6': 10175,\n",
       " 'west': 18710,\n",
       " 'burn': 2422,\n",
       " 'thousand': 17607,\n",
       " 'alon': 1088,\n",
       " 'httptcovl5tbr3wbr': 10450,\n",
       " 'perfect': 14480,\n",
       " 'tracklist': 17845,\n",
       " 'life': 12354,\n",
       " 'leav': 12253,\n",
       " 'first': 5106,\n",
       " 'retain': 15488,\n",
       " 'quit': 15108,\n",
       " 'weird': 18688,\n",
       " 'better': 1923,\n",
       " 'use': 18287,\n",
       " 'wear': 18659,\n",
       " 'everi': 4751,\n",
       " 'singl': 16390,\n",
       " 'next': 13692,\n",
       " 'year': 19101,\n",
       " 'least': 12251,\n",
       " 'deputi': 3878,\n",
       " 'shot': 16306,\n",
       " 'brighton': 2299,\n",
       " 'home': 6309,\n",
       " 'httptcogwnrhmso8k': 8429,\n",
       " 'wife': 18795,\n",
       " 'six': 16428,\n",
       " 'jail': 11547,\n",
       " 'niec': 13723,\n",
       " 'httptcoev1ahoucza': 8141,\n",
       " 'lanford': 12166,\n",
       " 'salmon': 15879,\n",
       " 'httptcovplr5hka2u': 10464,\n",
       " 'httptcosxhw2tnnlf': 10088,\n",
       " 'arsonist': 1388,\n",
       " 'deliber': 3825,\n",
       " 'black': 2033,\n",
       " 'church': 2934,\n",
       " 'north': 13825,\n",
       " 'carolinaåêablaz': 2642,\n",
       " 'httptcopcxarbh9an': 9595,\n",
       " 'noch': 13781,\n",
       " 'elbestia': 4477,\n",
       " 'alexissanchez': 1039,\n",
       " 'happi': 5999,\n",
       " 'teammat': 17366,\n",
       " 'train': 17862,\n",
       " 'hard': 6008,\n",
       " 'goodnight': 5704,\n",
       " 'gunner': 5895,\n",
       " 'httptcouc4j4jhvgr': 10279,\n",
       " 'kurd': 12098,\n",
       " 'trampl': 17867,\n",
       " 'turkmen': 18031,\n",
       " 'later': 12193,\n",
       " 'other': 14192,\n",
       " 'vandal': 18349,\n",
       " 'front': 5377,\n",
       " 'diyala': 4100,\n",
       " 'httptco4izfdyc3cg': 7123,\n",
       " 'truck': 17973,\n",
       " 'r21': 15122,\n",
       " 'voortrekk': 18515,\n",
       " 'ave': 1568,\n",
       " 'tambo': 17292,\n",
       " 'intl': 11412,\n",
       " 'cargo': 2622,\n",
       " 'section': 16084,\n",
       " 'httptco8kscqkfkkf': 7384,\n",
       " 'heart': 6107,\n",
       " 'citi': 2958,\n",
       " 'gift': 5601,\n",
       " 'skylin': 16463,\n",
       " 'kiss': 12010,\n",
       " 'upon': 18243,\n",
       " 'lip': 12410,\n",
       " 'httpstcocyompz1a0z': 6506,\n",
       " 'tonight': 17765,\n",
       " 'lo': 12471,\n",
       " 'angel': 1199,\n",
       " 'ig': 11159,\n",
       " 'fb': 4972,\n",
       " 'fill': 5061,\n",
       " 'sunset': 17121,\n",
       " 'peep': 14456,\n",
       " 'httptcoicsjgz9te1': 8622,\n",
       " 'climat': 3015,\n",
       " 'energi': 4594,\n",
       " 'httptco9fxmn0l0bd': 7438,\n",
       " 'revel': 15507,\n",
       " 'wmv': 18878,\n",
       " 'video': 18430,\n",
       " 'mean': 12930,\n",
       " 'mac': 12665,\n",
       " 'farewel': 4934,\n",
       " 'en': 4573,\n",
       " 'rout': 15725,\n",
       " 'dvd': 4351,\n",
       " 'gtxrwm': 5860,\n",
       " 'progress': 14925,\n",
       " 'greet': 5800,\n",
       " 'month': 13308,\n",
       " 'student': 17031,\n",
       " 'would': 18948,\n",
       " 'pen': 14461,\n",
       " 'torch': 17792,\n",
       " 'public': 15003,\n",
       " 'httptco9fxpixqujt': 7439,\n",
       " 'rene': 15408,\n",
       " 'amp': 1151,\n",
       " 'jacinta': 11524,\n",
       " 'secret': 16083,\n",
       " '2k13': 360,\n",
       " 'fallen': 4900,\n",
       " 'edit': 4425,\n",
       " 'mar': 12777,\n",
       " '30': 372,\n",
       " '2013': 269,\n",
       " 'httpstco7mlmsuzv1z': 6458,\n",
       " 'navista7': 13583,\n",
       " 'steve': 16924,\n",
       " 'someth': 16637,\n",
       " 'els': 4517,\n",
       " 'tinderbox': 17685,\n",
       " 'clown': 3039,\n",
       " 'hood': 6342,\n",
       " 'news24680': 13675,\n",
       " 'nowplay': 13872,\n",
       " 'ian': 11096,\n",
       " 'buff': 2382,\n",
       " 'magnitud': 12702,\n",
       " 'httptcoav2jsjfftc': 7600,\n",
       " 'edm': 4430,\n",
       " 'nxwestmidland': 13918,\n",
       " 'huge': 11009,\n",
       " 'httptcorwzbfvnxer': 9955,\n",
       " 'talk': 17286,\n",
       " 'go': 5672,\n",
       " 'dont': 4164,\n",
       " 'make': 12722,\n",
       " 'work': 18918,\n",
       " 'kid': 11961,\n",
       " 'cuz': 3570,\n",
       " 'bicycl': 1951,\n",
       " 'accid': 798,\n",
       " 'split': 16773,\n",
       " 'testicl': 17452,\n",
       " 'imposs': 11228,\n",
       " 'michael': 13070,\n",
       " 'father': 4959,\n",
       " 'i24': 11079,\n",
       " 'nashvilletraff': 13557,\n",
       " 'traffic': 17853,\n",
       " 'move': 13379,\n",
       " '8m': 668,\n",
       " 'slower': 16512,\n",
       " 'usual': 18298,\n",
       " 'httpstco0ghk693egj': 6412,\n",
       " 'center': 2734,\n",
       " 'lane': 12165,\n",
       " 'block': 2080,\n",
       " 'santaclara': 15929,\n",
       " 'us101': 18271,\n",
       " 'nb': 13587,\n",
       " 'great': 5783,\n",
       " 'america': 1135,\n",
       " 'pkwi': 14632,\n",
       " 'bayarea': 1764,\n",
       " 'httptcopmlohzurwr': 9637,\n",
       " 'httptcogkye6gjtk5': 8371,\n",
       " 'personalinjuri': 14502,\n",
       " 'read': 15248,\n",
       " 'advic': 885,\n",
       " 'solicitor': 16625,\n",
       " 'help': 6154,\n",
       " 'otleyhour': 14195,\n",
       " 'stloui': 16944,\n",
       " 'caraccidentlawy': 2607,\n",
       " 'speed': 16745,\n",
       " 'among': 1149,\n",
       " 'teen': 17390,\n",
       " 'httpstcok4zomof319': 6598,\n",
       " 'httpstcos2kxvm0cba': 6702,\n",
       " 'tee': 17387,\n",
       " 'report': 15428,\n",
       " 'motor': 13364,\n",
       " 'vehicl': 18375,\n",
       " 'curri': 3557,\n",
       " 'herman': 6175,\n",
       " 'rd': 15236,\n",
       " 'stephenson': 16916,\n",
       " 'involv': 11433,\n",
       " 'overturn': 14254,\n",
       " 'pleas': 14664,\n",
       " 'httptcoybjezkurw1': 10790,\n",
       " 'bigrigradio': 1965,\n",
       " 'awar': 1585,\n",
       " 'i77': 11086,\n",
       " 'mile': 13117,\n",
       " 'marker': 12805,\n",
       " '31': 385,\n",
       " 'mooresvil': 13319,\n",
       " 'iredel': 11455,\n",
       " 'ramp': 15178,\n",
       " '86': 658,\n",
       " '118': 110,\n",
       " 'pm': 14687,\n",
       " 'sleepjunki': 16489,\n",
       " 'sleep': 16487,\n",
       " 'pill': 14590,\n",
       " 'doubl': 4193,\n",
       " 'risk': 15587,\n",
       " 'httptco7s9nm1fict': 7335,\n",
       " 'knew': 12031,\n",
       " 'httpstcoysxun5vceh': 6788,\n",
       " 'cabrillo': 2481,\n",
       " 'hwymagellan': 11067,\n",
       " 'av': 1561,\n",
       " 'mir': 13172,\n",
       " '080615': 42,\n",
       " '110358': 97,\n",
       " '40': 444,\n",
       " 'congest': 3234,\n",
       " 'pastor': 14388,\n",
       " 'scene': 16000,\n",
       " 'accidentwho': 802,\n",
       " 'owner': 14264,\n",
       " 'rang': 15189,\n",
       " 'rover': 15730,\n",
       " 'mom': 13281,\n",
       " 'didnt': 3978,\n",
       " 'wish': 18854,\n",
       " 'spilt': 16761,\n",
       " 'mayonnais': 12892,\n",
       " 'horribl': 6364,\n",
       " 'past': 14385,\n",
       " 'sunday': 17113,\n",
       " 'final': 5067,\n",
       " 'abl': 770,\n",
       " 'god': 5677,\n",
       " 'piss': 14614,\n",
       " 'donni': 4163,\n",
       " 'tell': 17401,\n",
       " 'anoth': 1235,\n",
       " 'truckcrash': 17974,\n",
       " 'fortworth': 5267,\n",
       " 'interst': 11406,\n",
       " 'httptcors22lj4qfp': 9938,\n",
       " 'click': 3011,\n",
       " 'youv': 19159,\n",
       " 'crashgthttptcold0uniyw4k': 3413,\n",
       " 'ashvil': 1426,\n",
       " '23': 310,\n",
       " 'sb': 15979,\n",
       " 'sr': 16820,\n",
       " '752': 619,\n",
       " 'httptcohylmo0wgfi': 8579,\n",
       " 'carolina': 2640,\n",
       " 'motorcyclist': 13367,\n",
       " 'i540': 11082,\n",
       " 'cross': 3472,\n",
       " 'median': 12949,\n",
       " 'motorcycl': 13366,\n",
       " 'rider': 15556,\n",
       " 'travel': 17887,\n",
       " 'httptcop18lzrlmy6': 9556,\n",
       " 'fyi': 5448,\n",
       " 'cadfyi': 2487,\n",
       " 'properti': 14946,\n",
       " 'damagenhs999': 3624,\n",
       " 'piner': 14598,\n",
       " 'rdhorndal': 15239,\n",
       " 'dr': 4218,\n",
       " 'naayf': 13519,\n",
       " 'turn': 18032,\n",
       " 'onto': 14107,\n",
       " 'chandane': 2774,\n",
       " 'magu': 12706,\n",
       " 'mma': 13235,\n",
       " 'taxi': 17331,\n",
       " 'ram': 15175,\n",
       " 'halfway': 5959,\n",
       " 'everyon': 4758,\n",
       " 'conf': 3225,\n",
       " 'left': 12262,\n",
       " 'manchest': 12742,\n",
       " '293': 346,\n",
       " 'eddi': 4418,\n",
       " 'stop': 16959,\n",
       " 'back': 1628,\n",
       " 'nh3a': 13704,\n",
       " 'delay': 3821,\n",
       " 'min': 13136,\n",
       " 'httptcooia5fxi4gm': 9489,\n",
       " 'damagewpd1600': 3625,\n",
       " '17th': 194,\n",
       " '862015209': 662,\n",
       " 'injuri': 11337,\n",
       " '2781': 337,\n",
       " 'willi': 18810,\n",
       " 'foreman': 5239,\n",
       " 'httptcovckit6edev': 10419,\n",
       " 'aashiqui': 743,\n",
       " 'actress': 843,\n",
       " 'anu': 1254,\n",
       " 'aggarw': 933,\n",
       " 'nearfat': 13611,\n",
       " 'httptco6otfp31lqw': 7259,\n",
       " 'suffield': 17083,\n",
       " 'alberta': 1018,\n",
       " 'httpstcobptmlf4p10': 6493,\n",
       " 'backup': 1635,\n",
       " 'southaccid': 16696,\n",
       " 'right': 15562,\n",
       " 'exit': 4795,\n",
       " 'langtre': 12169,\n",
       " 'rdconsid': 15237,\n",
       " 'nc': 13597,\n",
       " '115': 108,\n",
       " '150': 152,\n",
       " '16': 167,\n",
       " 'altern': 1104,\n",
       " 'chang': 2777,\n",
       " 'determin': 3914,\n",
       " 'option': 14146,\n",
       " 'financi': 5071,\n",
       " 'support': 17141,\n",
       " 'plan': 14637,\n",
       " 'ongo': 14091,\n",
       " 'treatment': 17898,\n",
       " 'deadli': 3741,\n",
       " 'hagerstown': 5927,\n",
       " 'today': 17724,\n",
       " 'ill': 11184,\n",
       " 'detail': 3910,\n",
       " 'your4stat': 19153,\n",
       " 'whag': 18733,\n",
       " 'flowri': 5180,\n",
       " 'marinad': 12800,\n",
       " 'even': 4745,\n",
       " 'fuck': 5397,\n",
       " 'mf': 13049,\n",
       " 'drive': 4257,\n",
       " 'norwaymfa': 13839,\n",
       " 'bahrain': 1657,\n",
       " 'previous': 14878,\n",
       " 'road': 15610,\n",
       " 'kill': 11977,\n",
       " 'explos': 4820,\n",
       " 'httpstcogfjfgtodad': 6547,\n",
       " 'still': 16940,\n",
       " 'heard': 6104,\n",
       " 'leader': 12241,\n",
       " 'kenya': 11926,\n",
       " 'forward': 5268,\n",
       " 'comment': 3171,\n",
       " 'issu': 11489,\n",
       " 'disciplinari': 4032,\n",
       " 'measuresarrestpastornganga': 12937,\n",
       " 'aftershockdelo': 922,\n",
       " 'scuf': 16052,\n",
       " 'ps': 14976,\n",
       " 'game': 5476,\n",
       " 'cya': 3575,\n",
       " 'effort': 4452,\n",
       " 'pain': 14288,\n",
       " 'win': 18821,\n",
       " 'roger': 15653,\n",
       " 'bannist': 1698,\n",
       " '320': 391,\n",
       " 'ir': 11448,\n",
       " 'icemoon': 11118,\n",
       " 'aftershock': 921,\n",
       " 'httptcoynxnvvkcda': 10838,\n",
       " 'djicemoon': 4104,\n",
       " 'dubstep': 4307,\n",
       " 'trapmus': 17883,\n",
       " 'dnb': 4126,\n",
       " 'danc': 3642,\n",
       " 'ices': 11120,\n",
       " 'httptcoweqpesenku': 10541,\n",
       " 'victori': 18423,\n",
       " 'bargain': 1719,\n",
       " 'basement': 1737,\n",
       " 'price': 14880,\n",
       " 'dwight': 4357,\n",
       " 'david': 3699,\n",
       " 'eisenhow': 4469,\n",
       " 'httptcovam5podgyw': 10405,\n",
       " 'httptcozevakjapcz': 10924,\n",
       " 'nobodi': 13780,\n",
       " 'rememb': 15393,\n",
       " 'came': 2536,\n",
       " 'charl': 2794,\n",
       " 'schulz': 16012,\n",
       " 'speak': 16727,\n",
       " 'someon': 16636,\n",
       " 'xb1': 19044,\n",
       " 'also': 1100,\n",
       " 'harder': 6014,\n",
       " 'conflict': 3231,\n",
       " 'gloriou': 5658,\n",
       " 'triumph': 17945,\n",
       " 'thoma': 17595,\n",
       " 'growingupspoil': 5837,\n",
       " 'clay': 2990,\n",
       " 'pigeon': 14583,\n",
       " 'shoot': 16296,\n",
       " 'guess': 5871,\n",
       " 'one': 14083,\n",
       " 'actual': 844,\n",
       " 'free': 5325,\n",
       " 'tc': 17345,\n",
       " 'terrifi': 17440,\n",
       " 'best': 1910,\n",
       " 'roller': 15666,\n",
       " 'coaster': 3072,\n",
       " 'ever': 4749,\n",
       " 'disclaim': 4033,\n",
       " 'httpstcoxmwodfmtui': 6768,\n",
       " 'httptcom4jdzmgjow': 9157,\n",
       " 'httptcon0uhasfkbv': 9290,\n",
       " 'httptcoe14epzhoth': 8024,\n",
       " 'httptco22a9d5do6q': 6956,\n",
       " 'kjforday': 12018,\n",
       " 'httptcothyzomvwu0': 10170,\n",
       " 'httptco83joo0xk29': 7352,\n",
       " 'wisdomw': 18852,\n",
       " 'bonu': 2165,\n",
       " 'minut': 13168,\n",
       " 'daili': 3605,\n",
       " 'habit': 5923,\n",
       " 'could': 3353,\n",
       " 'realli': 15264,\n",
       " 'improv': 11231,\n",
       " 'mani': 12752,\n",
       " 'alreadi': 1095,\n",
       " 'lifehack': 12357,\n",
       " 'httptcotbm9fqb8cw': 10129,\n",
       " 'protect': 14958,\n",
       " 'profit': 14920,\n",
       " 'global': 5649,\n",
       " 'meltdown': 12985,\n",
       " 'wiedem': 18794,\n",
       " 'http': 6410,\n",
       " 'httptcowztz4hgmvq': 10629,\n",
       " 'moment': 13283,\n",
       " 'scari': 15991,\n",
       " 'guy': 5907,\n",
       " 'behind': 1848,\n",
       " 'scream': 16040,\n",
       " 'bloodi': 2090,\n",
       " 'murder': 13455,\n",
       " 'silverwood': 16369,\n",
       " '2010': 266,\n",
       " 'full': 5411,\n",
       " 'stream': 16992,\n",
       " 'youtub': 19157,\n",
       " 'httptcovve3usesgf': 10483,\n",
       " 'gtgt': 5852,\n",
       " '15': 151,\n",
       " 'book': 2167,\n",
       " 'httptcof6ntuc734z': 8173,\n",
       " 'esquireattir': 4697,\n",
       " 'sometim': 16640,\n",
       " 'face': 4866,\n",
       " 'difficulti': 3992,\n",
       " 'wrong': 18988,\n",
       " 'joel': 11693,\n",
       " 'osteen': 14188,\n",
       " 'thing': 17575,\n",
       " 'stand': 16855,\n",
       " 'dream': 4240,\n",
       " 'belief': 1858,\n",
       " 'possibl': 14765,\n",
       " 'brown': 2343,\n",
       " 'prais': 14814,\n",
       " 'ministri': 13160,\n",
       " 'wdyouth': 18649,\n",
       " 'biblestudi': 1946,\n",
       " 'httpstcoujk0e5gbcc': 6736,\n",
       " 'way': 18626,\n",
       " 'avoid': 1578,\n",
       " 'trap': 17881,\n",
       " 'think': 17577,\n",
       " 'lose': 12549,\n",
       " 'job': 11687,\n",
       " 'orang': 14150,\n",
       " 'never': 13656,\n",
       " 'onfireand': 14090,\n",
       " 'bb': 1768,\n",
       " 'httpstcojv8ppkhjy7': 6592,\n",
       " 'kick': 11960,\n",
       " 'say': 15975,\n",
       " 'done': 4158,\n",
       " 'interrupt': 11402,\n",
       " 'georg': 5559,\n",
       " 'bernard': 1904,\n",
       " 'shaw': 16245,\n",
       " 'oyster': 14267,\n",
       " 'shell': 16258,\n",
       " 'andrew': 1192,\n",
       " 'carnegi': 2637,\n",
       " 'anyon': 1261,\n",
       " 'need': 13620,\n",
       " 'pu': 15001,\n",
       " 'play': 14652,\n",
       " 'hybrid': 11069,\n",
       " 'slayer': 16486,\n",
       " 'ps4': 14980,\n",
       " 'eu': 4721,\n",
       " 'hmu': 6275,\n",
       " 'cod8sandscrim': 3087,\n",
       " 'empirikgam': 4562,\n",
       " 'codawscrim': 3088,\n",
       " '4tpkotc': 498,\n",
       " '4tpfa': 497,\n",
       " 'aftershockorg': 923,\n",
       " 'expert': 4809,\n",
       " 'franc': 5307,\n",
       " 'begin': 1841,\n",
       " 'examin': 4774,\n",
       " 'airplan': 978,\n",
       " 'debri': 3762,\n",
       " 'found': 5277,\n",
       " 'reunion': 15500,\n",
       " 'island': 11483,\n",
       " 'french': 5350,\n",
       " 'air': 970,\n",
       " 'httptcoyvvpznzmxg': 10875,\n",
       " 'news': 13674,\n",
       " 'strict': 17005,\n",
       " 'liabil': 12335,\n",
       " 'context': 3273,\n",
       " 'pilot': 14593,\n",
       " 'error': 4679,\n",
       " 'common': 3179,\n",
       " 'compon': 3202,\n",
       " 'aviat': 1575,\n",
       " 'cr': 3397,\n",
       " 'httptco6cz3bohrd4': 7229,\n",
       " 'crobscarla': 3467,\n",
       " 'lifetim': 12365,\n",
       " 'odd': 13976,\n",
       " '8015': 638,\n",
       " 'wedn': 18672,\n",
       " 'httptcobkpfpogysi': 7692,\n",
       " 'alexalltimelow': 1032,\n",
       " 'awwww': 1597,\n",
       " 'theyr': 17567,\n",
       " 'cuti': 3566,\n",
       " 'good': 5699,\n",
       " 'famili': 4908,\n",
       " 'member': 12987,\n",
       " 'osama': 14182,\n",
       " 'bin': 1990,\n",
       " 'laden': 12130,\n",
       " 'iron': 11462,\n",
       " 'mhmmm': 13061,\n",
       " 'gov': 5733,\n",
       " 'suspect': 17177,\n",
       " 'goe': 5683,\n",
       " 'engin': 4597,\n",
       " 'httptcotyjxrfd3st': 10238,\n",
       " 'via': 18410,\n",
       " 'wing': 18838,\n",
       " '29072015': 344,\n",
       " 'httptcoi7kztevb2v': 8603,\n",
       " 'cessna': 2749,\n",
       " 'ocampo': 13962,\n",
       " 'coahuila': 3066,\n",
       " 'mexico': 13046,\n",
       " 'juli': 11775,\n",
       " '29': 342,\n",
       " '2015': 271,\n",
       " 'four': 5280,\n",
       " 'men': 12995,\n",
       " 'includ': 11251,\n",
       " 'state': 16883,\n",
       " 'govern': 5735,\n",
       " 'offici': 13997,\n",
       " 'watchthevideo': 18609,\n",
       " 'httptcop64xrvgjik': 9566,\n",
       " 'httptcolsmx2vwr3j': 9112,\n",
       " 'wednesday': 18674,\n",
       " 'began': 1839,\n",
       " 'kca': 11891,\n",
       " 'votejkt48id': 18519,\n",
       " 'mbataweel': 12899,\n",
       " 'rip': 15581,\n",
       " 'binladen': 1994,\n",
       " 'almost': 1085,\n",
       " 'cowork': 3386,\n",
       " 'nude': 13897,\n",
       " 'mode': 13253,\n",
       " 'mickinyman': 13079,\n",
       " 'theatlant': 17492,\n",
       " 'might': 13105,\n",
       " 'wreck': 18972,\n",
       " 'polit': 14718,\n",
       " 'httptcotagzbcxfj0': 10120,\n",
       " 'mlb': 13228,\n",
       " 'unbeliev': 18137,\n",
       " 'insan': 11350,\n",
       " 'airport': 980,\n",
       " 'aircraft': 973,\n",
       " 'aeroplan': 891,\n",
       " 'runway': 15793,\n",
       " 'freaky': 5320,\n",
       " 'httpstcocezhq7czl': 6497,\n",
       " 'airplaneåê29072015': 979,\n",
       " 'httptcowq3wjsgphl': 10590,\n",
       " 'httptcotfcdronra6': 10152,\n",
       " 'usama': 18277,\n",
       " 'ladin': 12132,\n",
       " 'natur': 13574,\n",
       " 'plane': 14638,\n",
       " 'festiv': 5022,\n",
       " 'httpstcokq9ae6ap2b': 6603,\n",
       " 'death': 3752,\n",
       " 'carfest': 2621,\n",
       " 'httptcogibyqhhkpk': 8360,\n",
       " 'dtn': 4296,\n",
       " 'brazil': 2248,\n",
       " 'exp': 4799,\n",
       " 'httptcom9ig3wq8lq': 9172,\n",
       " 'httptcov4smaeslk5': 10390,\n",
       " 'ûïairplane': 19276,\n",
       " 'wtf': 19003,\n",
       " 'can': 2556,\n",
       " 'ûªt': 19271,\n",
       " 'believ': 1859,\n",
       " 'eye': 4847,\n",
       " 'httptco6ffylajwp': 7236,\n",
       " 'nicol': 13720,\n",
       " 'fletcher': 5152,\n",
       " 'victim': 18422,\n",
       " 'ago': 938,\n",
       " 'littl': 12433,\n",
       " 'bit': 2011,\n",
       " 'trauma': 17885,\n",
       " 'although': 1105,\n",
       " 'she': 16249,\n",
       " 'omg': 14075,\n",
       " 'httptcoxdxdprcpn': 10664,\n",
       " 'bro': 2317,\n",
       " 'jetengin': 11652,\n",
       " 'turbojet': 18027,\n",
       " 'bo': 2126,\n",
       " 'g90': 5452,\n",
       " 'httptcokxxnszp6nk': 9003,\n",
       " 'phone': 14555,\n",
       " 'ship': 16277,\n",
       " 'terribl': 17438,\n",
       " 'statist': 16889,\n",
       " 'cop': 3304,\n",
       " 'hous': 6386,\n",
       " 'colombia': 3137,\n",
       " '12': 115,\n",
       " 'httpstcozhjlflbhzl': 6794,\n",
       " 'httpstcoiecc1jdoub': 6569,\n",
       " 'drone': 4269,\n",
       " 'worri': 18937,\n",
       " 'esp': 4691,\n",
       " 'vicin': 18417,\n",
       " 'httptcokz35rgngjf': 9006,\n",
       " 'earli': 4376,\n",
       " 'wake': 18546,\n",
       " 'call': 2523,\n",
       " 'sister': 16418,\n",
       " 'beg': 1838,\n",
       " 'ride': 15555,\n",
       " 'wher': 18750,\n",
       " 'ambul': 1127,\n",
       " 'hospit': 6371,\n",
       " 'rodkiai': 15651,\n",
       " 'httptcoay6zzcupnz': 7617,\n",
       " 'twelv': 18052,\n",
       " 'fear': 4983,\n",
       " 'pakistani': 14295,\n",
       " 'helicopt': 6141,\n",
       " 'httptcosc9dns41mc': 9994,\n",
       " 'seriou': 16163,\n",
       " 'lorri': 12545,\n",
       " 'httptco9pfeaqeski': 7464,\n",
       " 'httptcofntg70rnkx': 8240,\n",
       " 'emsne': 4570,\n",
       " 'reuter': 15503,\n",
       " 'httptcomdnugvubwn': 9184,\n",
       " 'yugvani': 19165,\n",
       " 'lead': 12240,\n",
       " 'servic': 16168,\n",
       " 'boss': 2194,\n",
       " 'welcom': 18690,\n",
       " 'chariti': 2793,\n",
       " 'httptcomj2jq2psv6': 9210,\n",
       " 'aberystwythshrewsburi': 765,\n",
       " 'incid': 11246,\n",
       " 'halt': 5965,\n",
       " 'shrew': 16322,\n",
       " 'httptcoxum8ylcb4q': 10726,\n",
       " 'sprinter': 16799,\n",
       " 'automat': 1557,\n",
       " 'frontlin': 5379,\n",
       " 'choic': 2886,\n",
       " '14': 141,\n",
       " 'lez': 12327,\n",
       " 'compliant': 3200,\n",
       " 'ebay': 4403,\n",
       " 'httptco4evttqpeia': 7113,\n",
       " 'nanotech': 13540,\n",
       " 'devic': 3934,\n",
       " 'target': 17314,\n",
       " 'destroy': 3907,\n",
       " 'blood': 2087,\n",
       " 'clot': 3033,\n",
       " 'httptcohfy5v3slbb': 8506,\n",
       " '20skyhawkmm20': 293,\n",
       " 'traplord29': 17882,\n",
       " 'fredosantana300': 5324,\n",
       " 'lilreese300': 12390,\n",
       " 'hella': 6144,\n",
       " 'crazi': 3421,\n",
       " 'fight': 5052,\n",
       " 'coupl': 3370,\n",
       " 'mosh': 13352,\n",
       " 'pit': 14615,\n",
       " 'run': 15783,\n",
       " 'lucki': 12605,\n",
       " 'justsay': 11814,\n",
       " 'randomthought': 15186,\n",
       " 'httptcobfes5twbzt': 7665,\n",
       " 'tilnow': 17667,\n",
       " 'dna': 4125,\n",
       " 'httptco7xglah10zl': 7339,\n",
       " 'httptcothmblaatzp': 10164,\n",
       " 'tanslash': 17303,\n",
       " 'fouseytub': 5284,\n",
       " 'ok': 14035,\n",
       " 'hahahah': 5932,\n",
       " 'httptcozsberqnn9n': 10972,\n",
       " 'httptcoq8ivrzojzv': 9721,\n",
       " 'pakistan': 14294,\n",
       " 'nine': 13743,\n",
       " 'httptco8e7ry8ebmf': 7364,\n",
       " 'thenissonian': 17537,\n",
       " 'rejectdcartoon': 15368,\n",
       " 'nissan': 13749,\n",
       " 'medic': 12950,\n",
       " 'assist': 1455,\n",
       " 'ems1': 4568,\n",
       " 'ny': 13919,\n",
       " 'emt': 4571,\n",
       " 'petit': 14521,\n",
       " '17': 180,\n",
       " 'per': 14477,\n",
       " 'hour': 6385,\n",
       " 'minimum': 13156,\n",
       " 'wage': 18538,\n",
       " 'ûª': 19263,\n",
       " 'httptco4oa6swlxmr': 7133,\n",
       " 'em': 4526,\n",
       " 'paramed': 14347,\n",
       " 'httptcofcqmkffflw': 8193,\n",
       " 'httptcovayaymbngu': 10407,\n",
       " 'httptco3brme6sn4t': 7033,\n",
       " 'httptcoujrx9kgawp': 10311,\n",
       " 'httptcokp2lf4aut': 8973,\n",
       " 'kiwikaryn': 12017,\n",
       " 'park': 14358,\n",
       " 'lot': 12554,\n",
       " 'said': 15859,\n",
       " 'john': 11700,\n",
       " 'httptcohpvodud7ip': 8540,\n",
       " 'ltlt': 12593,\n",
       " 'httptcoshzpyiqok5': 10018,\n",
       " 'httptcopwwpum6rbj': 9688,\n",
       " 'httptcoyspon4d6qo': 10861,\n",
       " 'ûïleoblakecart': 19291,\n",
       " 'dog': 4142,\n",
       " ...}"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_vec.vocabulary_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_vec = count_vec.transform(tweet['text'])\n",
    "test_vec = count_vec.transform(test['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(7503, 19325)\n",
      "(3263, 19325)\n"
     ]
    }
   ],
   "source": [
    "print(train_vec.shape)\n",
    "print(test_vec.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Architecture\n",
    "\n",
    "#### First, we will split the training data using sklearn's train_test_split into 80/20 training/test. For the model, I'll use sklearn's logistic regression since we are predicting a binary output (0 for NOT disaster tweet, 1 for disaster tweet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = model_selection.train_test_split(train_vec, tweet['target'], test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-6 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-6 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-6 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-6 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-6 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-6 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-6 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-6 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-6 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-6 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" checked><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;LogisticRegression<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.4/modules/generated/sklearn.linear_model.LogisticRegression.html\">?<span>Documentation for LogisticRegression</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>LogisticRegression()</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "LogisticRegression()"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = linear_model.LogisticRegression()\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Evaluation\n",
    "\n",
    "#### For the Kaggle competition, F1 scores are used to determine accuracy between predicted values and true values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 Score: 73.45%\n"
     ]
    }
   ],
   "source": [
    "f1 = metrics.f1_score(y_test, preds)\n",
    "print(f'F1 Score: {f1 * 100:.2f}%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x17c9e8ac0>"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAGwCAYAAACuFMx9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA40ElEQVR4nO3deXhTddr/8U+6l7ZpKdCWQkEQBSpLFRTijlYq8lMUxm1QK6LjYEGFEYFREUGpg6MoiuCCBWdE0VFQcGNxWJQiUMSHzQqIlq0tWmhtsVtyfn9goxlAGpI2JOf9uq5zXeSc7zm58zwd79z395tzLIZhGAIAAAEryNcBAACAhkWyBwAgwJHsAQAIcCR7AAACHMkeAIAAR7IHACDAkewBAAhwIb4OwBMOh0P79u1TTEyMLBaLr8MBALjJMAz9/PPPSk5OVlBQw9WflZWVqq6u9vg6YWFhioiI8EJEjcuvk/2+ffuUkpLi6zAAAB7avXu3Wrdu3SDXrqysVLu20Sostnt8raSkJO3atcvvEr5fJ/uYmBhJ0g8bTpM1mhkJBKbrzuzq6xCABlOrGn2uj5z/PW8I1dXVKiy264e802SNOflcUfazQ217fK/q6mqSfWOqa91bo4M8+n8gcCoLsYT6OgSg4fx6w/bGmIqNjrEoOubk38ch/50u9utkDwBAfdkNh+wePA3Gbji8F0wjI9kDAEzBIUMOnXy29+RcX6P3DQBAgKOyBwCYgkMOedKI9+xs3yLZAwBMwW4Yshsn34r35Fxfo40PAECAo7IHAJiCmRfokewBAKbgkCG7SZM9bXwAAAIclT0AwBRo4wMAEOBYjQ8AAAIWlT0AwBQcv26enO+vSPYAAFOwe7ga35NzfY1kDwAwBbshD596571YGhtz9gAABDgqewCAKTBnDwBAgHPIIrssHp3vr2jjAwAQ4KjsAQCm4DCObJ6c769I9gAAU7B72Mb35Fxfo40PAECAo7IHAJiCmSt7kj0AwBQchkUOw4PV+B6c62u08QEACHAkewCAKdS18T3Z3HHaaafJYrEctWVlZUmSKisrlZWVpWbNmik6OlqDBg1SUVGRyzUKCgrUv39/NWnSRAkJCRo9erRqa2vd/uy08QEApmBXkOwe1Lh2N8evW7dOdvtvZ23evFlXXHGFrr/+eknSyJEj9eGHH+qdd95RbGyshg8froEDB+qLL7448n52u/r376+kpCStXr1a+/fv12233abQ0FBNnjzZrVhI9gAAUzA8nLM33Dy3RYsWLq+ffPJJnX766brkkktUWlqqWbNmae7cubrsssskSTk5OercubPWrFmj3r17a/Hixdq6dauWLl2qxMREpaWladKkSRozZowmTJigsLCwesdCGx8AADeUlZW5bFVVVSc8p7q6Wv/+9791xx13yGKxKC8vTzU1NUpPT3eO6dSpk9q0aaPc3FxJUm5urrp27arExETnmIyMDJWVlWnLli1uxUyyBwCYgrfm7FNSUhQbG+vcsrOzT/jeCxYs0KFDh3T77bdLkgoLCxUWFqa4uDiXcYmJiSosLHSO+X2irzted8wdtPEBAKZgN4JkNzyYs//1drm7d++W1Wp17g8PDz/hubNmzVK/fv2UnJx80u/vCZI9AABusFqtLsn+RH744QctXbpU7733nnNfUlKSqqurdejQIZfqvqioSElJSc4xa9eudblW3Wr9ujH1RRsfAGAKDlnkUJAH28kt7svJyVFCQoL69+/v3NejRw+FhoZq2bJlzn35+fkqKCiQzWaTJNlsNm3atEnFxcXOMUuWLJHValVqaqpbMVDZAwBMwRe3y3U4HMrJyVFmZqZCQn5LubGxsRo6dKhGjRql+Ph4Wa1WjRgxQjabTb1795Yk9e3bV6mpqbr11ls1ZcoUFRYW6uGHH1ZWVla9pg5+j2QPAEADWbp0qQoKCnTHHXccdWzq1KkKCgrSoEGDVFVVpYyMDL344ovO48HBwVq0aJGGDRsmm82mqKgoZWZmauLEiW7HYTEMw2+f0FtWVqbY2Fgd/La9rDHMSCAwZSSn+ToEoMHUGjVarvdVWlrq1jy4O+pyxfyvz1BUTPBJX6fiZ7uu6769QWNtKFT2AABTODJn78GDcPz4qXeUwwAABDgqewCAKTg8vDe+Q347602yBwCYg+c31SHZAwBwSqv7vfzJn++/yZ45ewAAAhyVPQDAFOyGRXYPHnHrybm+RrIHAJiC3cMFenba+AAA4FRFZQ8AMAWHESSHB6vxHazGBwDg1EYbHwAABCwqewCAKTjk2Yp6h/dCaXQkewCAKXh+Ux3/bYb7b+QAAKBeqOwBAKbg+b3x/bc+JtkDAEzBzM+zJ9kDAEzBzJW9/0YOAADqhcoeAGAKnt9Ux3/rY5I9AMAUHIZFDk9+Z+/HT73z368pAACgXqjsAQCm4PCwje/PN9Uh2QMATMHzp975b7L338gBAEC9UNkDAEzBLovsHtwYx5NzfY1kDwAwBdr4AAAgYFHZAwBMwS7PWvF274XS6Ej2AABTMHMbn2QPADAFHoQDAAACFpU9AMAUDA+fZ2/w0zsAAE5ttPEBAEDAorIHAJiCmR9xS7IHAJiC3cOn3nlyrq/5b+QAAKBeqOwBAKZAGx8AgADnUJAcHjS0PTnX1/w3cgAAUC9U9gAAU7AbFtk9aMV7cq6vkewBAKbAnD0AAAHO8PCpdwZ30AMAAKcqKnsAgCnYZZHdg4fZeHKur5HsAQCm4DA8m3d3GF4MppHRxgcAIMCR7KHbzktVRnLaUdsL41pJkkYP6nDUsefGtHa5xrHOX74gzgefBjhal17lemzOLs3dsEWf7vtatitLXY5f0O+QJr+5U+9s3qxP932t9mf9ctQ1Wrat0vhZuzRv02a9l79JD838XnHNaxrrI8ALHL8u0PNkc9fevXt1yy23qFmzZoqMjFTXrl21fv1653HDMDR+/Hi1bNlSkZGRSk9P1/bt212uUVJSosGDB8tqtSouLk5Dhw5VeXm5W3HQxoemfZwvh/231tb330Ro3E0ddNHVv/0Hsd/gH3Xb6ELn6/BIx1HX+dvUAvXsU+Z8HW21N1DEgHsimjj03ZYIffpmvB597ftjHt+yNkorF8Zp5D/3HHU8PNKuyW9+p++2RmrM9adLkjIfLNTEObt03/87Q4Yf/yTLTByyyOHBvLu75x48eFAXXHCB+vTpo48//lgtWrTQ9u3b1bRpU+eYKVOmaNq0aZozZ47atWunRx55RBkZGdq6dasiIiIkSYMHD9b+/fu1ZMkS1dTUaMiQIfrLX/6iuXPn1juWUyLZT58+XU899ZQKCwvVvXt3Pf/88zrvvPN8HZZpxDVzTcrzXohVy9Oq1M322zfH8EhD8Qm1f3idaKv9hGMAX1j/X6vW/9d63OPL3o2XJCW2rj7m8bPOO6zElGpl9T1Th8uDJUlP3ddG727brLQLy/XVqhjvB41TVllZmcvr8PBwhYeHHzXuH//4h1JSUpSTk+Pc165dO+e/DcPQs88+q4cfflgDBgyQJL3++utKTEzUggULdNNNN2nbtm365JNPtG7dOvXs2VOS9Pzzz+uqq67SP//5TyUnJ9crZp+38efNm6dRo0bp0Ucf1YYNG9S9e3dlZGSouLjY16GZUk21RZ+921QZN/0ky+++xP73vaa6/qwu+kufjnptcktVHj76G+4LD7XS9Wd10YirztCnb8bL8OPFLMDvhYY5JOPI/z7q1FRZZDiks86r8GFkcEfdHfQ82SQpJSVFsbGxzi07O/uY7/fBBx+oZ8+euv7665WQkKCzzz5br7zyivP4rl27VFhYqPT0dOe+2NhY9erVS7m5uZKk3NxcxcXFORO9JKWnpysoKEhffvllvT+7zyv7Z555RnfddZeGDBkiSZo5c6Y+/PBDvfbaaxo7dqyPozOf1Z/EqrwsWH1vKHHu63PdQSW0rlazxBrt2hapWU+01J6d4Ro/63vnmNtG71faBeUKj3Qob0WMnv97a/1SEaRr7/zRB58C8K5v8qJUeThIQx/ar5wnW0oyNPSh/QoOkeITmLf3Fyc77/778yVp9+7dslp/6xQdq6qXpO+++04zZszQqFGj9Pe//13r1q3Tvffeq7CwMGVmZqqw8MjUaGJiost5iYmJzmOFhYVKSEhwOR4SEqL4+HjnmPrwabKvrq5WXl6exo0b59wXFBSk9PR057ea36uqqlJVVZXz9f+2UuC5T9+M17l9ytQs6bd2/FW3/OT8d7vOlYpPqNGYGzpo3/dhSj7tSNtz8Mgi55gOXX9R5eEgvTMjgWSPgFBaEqLH7z5NI7L3aMDQH2U4pP8uaKrt/xcpw8F8vdlYrVaXZH88DodDPXv21OTJkyVJZ599tjZv3qyZM2cqMzOzocN04dM2/o8//ii73f6H32p+Lzs726V1kpKS0lihmkLRnlB9tSpGV/75pz8c1+mcw5Kkfd8f+9ts3Zgf94epuor/ECIwbFgRoyHnd9aN3c7S9V266Kl726hZUo32F4T5OjTUk0MW5/3xT2pzc4Fey5YtlZqa6rKvc+fOKigokCQlJSVJkoqKilzGFBUVOY8lJSUdNa1dW1urkpIS55j68PmcvTvGjRun0tJS57Z7925fhxRQFr/VTHHNa9Ur/Y87Jjs3R0r64/blzi2Rio6rVVg4E/cILGUlIaooC1b3C35WXPNarVl84goPpwbj19X4J7sZbib7Cy64QPn5+S77vv32W7Vt21bSkcV6SUlJWrZsmfN4WVmZvvzyS9lsNkmSzWbToUOHlJeX5xzz2WefyeFwqFevXvWOxadt/ObNmys4OPgPv9X83vFWPMJzDoe0eF680q8vUfDv/ir2fR+m/85vqvMuL1NMU7t2bY3QSxNaqWvvcrVPrZQkrVls1cEDIerc47BCwx3asDJGb01L0J/+esBHnwZwFdHEruR2v620T0qpVvuzftHPh4J1YG+YYuJq1aJVjZolHvkCm3L6kb/tg8UhOnggVJLU98YSFWwPV+lPR/7Wh03cq/kvt9CenRGN/4FwUhr7qXcjR47U+eefr8mTJ+uGG27Q2rVr9fLLL+vll1+WJFksFt1///16/PHHdcYZZzh/epecnKxrr71W0pFOwJVXXqm77rpLM2fOVE1NjYYPH66bbrqp3ivxJR8n+7CwMPXo0UPLli1zfjCHw6Fly5Zp+PDhvgzNdL5aGaPivWHKuKnEZX9IqKGvVsVo/qstVHk4SC2Sa3ThVYd08/2/fUELDjW0cHZzvTQhXIYhJZ9Wrbsn7FO/wX88HQA0ljO7/6Kn3t3pfP3Xx/ZJkhbPa6qnR7ZR775leuDZ3zqFf595pM36r6cT9e+njxQerU+v1JBx+xUTZ1fR7lC9OS1R773cvBE/BfzNueeeq/nz52vcuHGaOHGi2rVrp2effVaDBw92jnnwwQdVUVGhv/zlLzp06JAuvPBCffLJJ87f2EvSG2+8oeHDh+vyyy9XUFCQBg0apGnTprkVi8UwfPsDqXnz5ikzM1MvvfSSzjvvPD377LN6++239c033xw1l/+/ysrKFBsbq4Pftpc1xq9mJIB6y0hO83UIQIOpNWq0XO+rtLS0XoveTkZdrrhuyRCFRp38GouaimrNvyKnQWNtKD7/6d2NN96oAwcOaPz48SosLFRaWpo++eSTEyZ6AADc0dht/FOJz5O9JA0fPpy2PQAADeSUSPYAADS0xr43/qmEZA8AMAUzt/FZ1QYAQICjsgcAmIKZK3uSPQDAFMyc7GnjAwAQ4KjsAQCmYObKnmQPADAFQ579fM6fH+tFsgcAmIKZK3vm7AEACHBU9gAAUzBzZU+yBwCYgpmTPW18AAACHJU9AMAUzFzZk+wBAKZgGBYZHiRsT871Ndr4AAAEOCp7AIAp8Dx7AAACnJnn7GnjAwAQ4KjsAQCmYOYFeiR7AIApmLmNT7IHAJiCmSt75uwBAAhwVPYAAFMwPGzj+3NlT7IHAJiCIckwPDvfX9HGBwAgwFHZAwBMwSGLLNxBDwCAwMVqfAAAELCo7AEApuAwLLJwUx0AAAKXYXi4Gt+Pl+PTxgcAIMBR2QMATMHMC/RI9gAAUyDZAwAQ4My8QI85ewAAAhyVPQDAFMy8Gp9kDwAwhSPJ3pM5ey8G08ho4wMAEOCo7AEApsBqfAAAApwhz55J78ddfNr4AAAEOip7AIAp0MYHACDQmbiPT7IHAJiDh5W9/LiyZ84eAIAGMGHCBFksFpetU6dOzuOVlZXKyspSs2bNFB0drUGDBqmoqMjlGgUFBerfv7+aNGmihIQEjR49WrW1tW7HQmUPADAFX9xB76yzztLSpUudr0NCfku7I0eO1Icffqh33nlHsbGxGj58uAYOHKgvvvhCkmS329W/f38lJSVp9erV2r9/v2677TaFhoZq8uTJbsVBsgcAmIK3FuiVlZW57A8PD1d4ePgxzwkJCVFSUtJR+0tLSzVr1izNnTtXl112mSQpJydHnTt31po1a9S7d28tXrxYW7du1dKlS5WYmKi0tDRNmjRJY8aM0YQJExQWFlbv2GnjAwDghpSUFMXGxjq37Ozs447dvn27kpOT1b59ew0ePFgFBQWSpLy8PNXU1Cg9Pd05tlOnTmrTpo1yc3MlSbm5ueratasSExOdYzIyMlRWVqYtW7a4FTOVPQDAHAyLZ4vsfj139+7dslqtzt3Hq+p79eql2bNnq2PHjtq/f78ee+wxXXTRRdq8ebMKCwsVFhamuLg4l3MSExNVWFgoSSosLHRJ9HXH6465g2QPADAFb83ZW61Wl2R/PP369XP+u1u3burVq5fatm2rt99+W5GRkScfyEmgjQ8AQCOIi4vTmWeeqR07digpKUnV1dU6dOiQy5iioiLnHH9SUtJRq/PrXh9rHcAfIdkDAMzB8MLmgfLycu3cuVMtW7ZUjx49FBoaqmXLljmP5+fnq6CgQDabTZJks9m0adMmFRcXO8csWbJEVqtVqampbr03bXwAgCk09u1yH3jgAV199dVq27at9u3bp0cffVTBwcG6+eabFRsbq6FDh2rUqFGKj4+X1WrViBEjZLPZ1Lt3b0lS3759lZqaqltvvVVTpkxRYWGhHn74YWVlZR13ncDx1CvZf/DBB/W+4DXXXONWAAAABKI9e/bo5ptv1k8//aQWLVrowgsv1Jo1a9SiRQtJ0tSpUxUUFKRBgwapqqpKGRkZevHFF53nBwcHa9GiRRo2bJhsNpuioqKUmZmpiRMnuh2LxTBOvFwhKKh+3X6LxSK73e52ECerrKxMsbGxOvhte1ljmJFAYMpITvN1CECDqTVqtFzvq7S0tF6L3k5GXa5o8/J4BUVGnPR1HL9UquAvExs01oZSr8re4XA0dBwAADQoMz/1zqNyuLKy0ltxAADQsHy8QM+X3E72drtdkyZNUqtWrRQdHa3vvvtOkvTII49o1qxZXg8QAAB4xu1k/8QTT2j27NmaMmWKy315u3TpoldffdWrwQEA4D0WL2z+ye1k//rrr+vll1/W4MGDFRwc7NzfvXt3ffPNN14NDgAAr6GNX3979+5Vhw4djtrvcDhUU1PjlaAAAID3uJ3sU1NTtWrVqqP2/+c//9HZZ5/tlaAAAPA6E1f2bt9Bb/z48crMzNTevXvlcDj03nvvKT8/X6+//roWLVrUEDECAOA5Lz31zh+5XdkPGDBACxcu1NKlSxUVFaXx48dr27ZtWrhwoa644oqGiBEAAHjgpO6Nf9FFF2nJkiXejgUAgAbjrUfc+qOTfhDO+vXrtW3bNklH5vF79OjhtaAAAPA6T+fdzZTs627s/8UXXyguLk6SdOjQIZ1//vl666231Lp1a2/HCAAAPOD2nP2dd96pmpoabdu2TSUlJSopKdG2bdvkcDh05513NkSMAAB4rm6Bniebn3K7sl+xYoVWr16tjh07Ovd17NhRzz//vC666CKvBgcAgLdYjCObJ+f7K7eTfUpKyjFvnmO325WcnOyVoAAA8DoTz9m73cZ/6qmnNGLECK1fv965b/369brvvvv0z3/+06vBAQAAz9Wrsm/atKkslt/mKioqKtSrVy+FhBw5vba2ViEhIbrjjjt07bXXNkigAAB4xMQ31alXsn/22WcbOAwAABqYidv49Ur2mZmZDR0HAABoICd9Ux1JqqysVHV1tcs+q9XqUUAAADQIE1f2bi/Qq6io0PDhw5WQkKCoqCg1bdrUZQMA4JRk4qfeuZ3sH3zwQX322WeaMWOGwsPD9eqrr+qxxx5TcnKyXn/99YaIEQAAeMDtNv7ChQv1+uuv69JLL9WQIUN00UUXqUOHDmrbtq3eeOMNDR48uCHiBADAMyZeje92ZV9SUqL27dtLOjI/X1JSIkm68MILtXLlSu9GBwCAl9TdQc+TzV+5nezbt2+vXbt2SZI6deqkt99+W9KRir/uwTgAAODU4XayHzJkiL7++mtJ0tixYzV9+nRFRERo5MiRGj16tNcDBADAK0y8QM/tOfuRI0c6/52enq5vvvlGeXl56tChg7p16+bV4AAAgOc8+p29JLVt21Zt27b1RiwAADQYizx86p3XIml89Ur206ZNq/cF77333pMOBgAAeF+9kv3UqVPrdTGLxeKTZP+nq69VSHB4o78v0Bh2Pt3M1yEADcZRWSn9/f3GeTMT//SuXsm+bvU9AAB+i9vlAgCAQOXxAj0AAPyCiSt7kj0AwBQ8vQueqe6gBwAA/AuVPQDAHEzcxj+pyn7VqlW65ZZbZLPZtHfvXknSv/71L33++edeDQ4AAK8x8e1y3U727777rjIyMhQZGamvvvpKVVVVkqTS0lJNnjzZ6wECAADPuJ3sH3/8cc2cOVOvvPKKQkNDnfsvuOACbdiwwavBAQDgLWZ+xK3bc/b5+fm6+OKLj9ofGxurQ4cOeSMmAAC8z8R30HO7sk9KStKOHTuO2v/555+rffv2XgkKAACvY86+/u666y7dd999+vLLL2WxWLRv3z698cYbeuCBBzRs2LCGiBEAAHjA7Tb+2LFj5XA4dPnll+vw4cO6+OKLFR4ergceeEAjRoxoiBgBAPCYmW+q43ayt1gseuihhzR69Gjt2LFD5eXlSk1NVXR0dEPEBwCAd5j4d/YnfVOdsLAwpaamejMWAADQANxO9n369JHFcvwViZ999plHAQEA0CA8/fmcmSr7tLQ0l9c1NTXauHGjNm/erMzMTG/FBQCAd5m4je/2avypU6e6bC+88II+//xz3X///S432QEAAEc8+eSTslgsuv/++537KisrlZWVpWbNmik6OlqDBg1SUVGRy3kFBQXq37+/mjRpooSEBI0ePVq1tbVuv7/Xnnp3yy236LXXXvPW5QAA8C4f/c5+3bp1eumll9StWzeX/SNHjtTChQv1zjvvaMWKFdq3b58GDhzoPG6329W/f39VV1dr9erVmjNnjmbPnq3x48e7HYPXkn1ubq4iIiK8dTkAALzKW7fLLSsrc9nqnhFzLOXl5Ro8eLBeeeUVNW3a1Lm/tLRUs2bN0jPPPKPLLrtMPXr0UE5OjlavXq01a9ZIkhYvXqytW7fq3//+t9LS0tSvXz9NmjRJ06dPV3V1tVuf3e1kP3DgQJftuuuuU+/evTVkyBDdfffd7l4OAAC/kpKSotjYWOeWnZ193LFZWVnq37+/0tPTXfbn5eWppqbGZX+nTp3Upk0b5ebmSjpSRHft2lWJiYnOMRkZGSorK9OWLVvcitntBXqxsbEur4OCgtSxY0dNnDhRffv2dfdyAAD4ld27d8tqtTpfh4eHH3PcW2+9pQ0bNmjdunVHHSssLFRYWJji4uJc9icmJqqwsNA55veJvu543TF3uJXs7Xa7hgwZoq5du7q0IwAAOOV5aTW+1Wp1SfbHsnv3bt13331asmTJKTHF7VYbPzg4WH379uXpdgAAv9OYj7jNy8tTcXGxzjnnHIWEhCgkJEQrVqzQtGnTFBISosTERFVXVx+VT4uKipSUlCTpyIPn/nd1ft3rujH15facfZcuXfTdd9+5exoAAKZx+eWXa9OmTdq4caNz69mzpwYPHuz8d2hoqJYtW+Y8Jz8/XwUFBbLZbJIkm82mTZs2qbi42DlmyZIlslqtbt/B1u05+8cff1wPPPCAJk2apB49eigqKsrl+IlaGwAA+Ewj3RgnJiZGXbp0cdkXFRWlZs2aOfcPHTpUo0aNUnx8vKxWq0aMGCGbzabevXtLkvr27avU1FTdeuutmjJligoLC/Xwww8rKyvruOsEjqfeyX7ixIn629/+pquuukqSdM0117jcNtcwDFksFtntdrcCAACgUZxid9CbOnWqgoKCNGjQIFVVVSkjI0Mvvvii83hwcLAWLVqkYcOGyWazKSoqSpmZmZo4caLb72UxDKNe4QcHB2v//v3atm3bH4675JJL3A7iZJWVlSk2NlaXdxylkGD3vuUA/uLbO5v5OgSgwTgqK/XD3x9WaWlpg3WG63JFhzGTFRx+8ovl7FWV2vGPvzdorA2l3pV93XeCxkzmAAB4C8+zr6c/etodAACntFOsjd+Y3Er2Z5555gkTfklJiUcBAQAA73Ir2T/22GNH3UEPAAB/QBu/nm666SYlJCQ0VCwAADQcE7fx631THebrAQDwT26vxgcAwC+ZuLKvd7J3OBwNGQcAAA2KOXsAAAKdiSt7tx+EAwAA/AuVPQDAHExc2ZPsAQCmYOY5e9r4AAAEOCp7AIA50MYHACCw0cYHAAABi8oeAGAOtPEBAAhwJk72tPEBAAhwVPYAAFOw/Lp5cr6/ItkDAMzBxG18kj0AwBT46R0AAAhYVPYAAHOgjQ8AgAn4ccL2BG18AAACHJU9AMAUzLxAj2QPADAHE8/Z08YHACDAUdkDAEyBNj4AAIGONj4AAAhUVPYAAFOgjQ8AQKAzcRufZA8AMAcTJ3vm7AEACHBU9gAAU2DOHgCAQEcbHwAABCoqewCAKVgMQxbj5MtzT871NZI9AMAcaOMDAIBARWUPADAFVuMDABDoaOMDAIBARWUPADAF2vgAAAQ62vgAAAS2usrek80dM2bMULdu3WS1WmW1WmWz2fTxxx87j1dWViorK0vNmjVTdHS0Bg0apKKiIpdrFBQUqH///mrSpIkSEhI0evRo1dbWuv3ZSfYAADSA1q1b68knn1ReXp7Wr1+vyy67TAMGDNCWLVskSSNHjtTChQv1zjvvaMWKFdq3b58GDhzoPN9ut6t///6qrq7W6tWrNWfOHM2ePVvjx493Oxba+AAAc2jkNv7VV1/t8vqJJ57QjBkztGbNGrVu3VqzZs3S3Llzddlll0mScnJy1LlzZ61Zs0a9e/fW4sWLtXXrVi1dulSJiYlKS0vTpEmTNGbMGE2YMEFhYWH1joXKHgBgGt5o4ZeVlblsVVVVJ3xfu92ut956SxUVFbLZbMrLy1NNTY3S09OdYzp16qQ2bdooNzdXkpSbm6uuXbsqMTHROSYjI0NlZWXO7kB9kewBAHBDSkqKYmNjnVt2dvZxx27atEnR0dEKDw/XX//6V82fP1+pqakqLCxUWFiY4uLiXMYnJiaqsLBQklRYWOiS6OuO1x1zB218AIA5GMaRzZPzJe3evVtWq9W5Ozw8/LindOzYURs3blRpaan+85//KDMzUytWrDj5GE4SyR4AYAre+p193er6+ggLC1OHDh0kST169NC6dev03HPP6cYbb1R1dbUOHTrkUt0XFRUpKSlJkpSUlKS1a9e6XK9utX7dmPqijQ8AQCNxOByqqqpSjx49FBoaqmXLljmP5efnq6CgQDabTZJks9m0adMmFRcXO8csWbJEVqtVqampbr0vlT0AwBwaeTX+uHHj1K9fP7Vp00Y///yz5s6dq+XLl+vTTz9VbGyshg4dqlGjRik+Pl5Wq1UjRoyQzWZT7969JUl9+/ZVamqqbr31Vk2ZMkWFhYV6+OGHlZWV9YdTB8dCsgcAmILFcWTz5Hx3FBcX67bbbtP+/fsVGxurbt266dNPP9UVV1whSZo6daqCgoI0aNAgVVVVKSMjQy+++KLz/ODgYC1atEjDhg2TzWZTVFSUMjMzNXHiRLdjJ9kDANAAZs2a9YfHIyIiNH36dE2fPv24Y9q2bauPPvrI41hI9lCXrgc06MZv1eGMg2rWvFKTxtuU+0Ur5/G4ppUactcmndOjSFHRNdr8f80184U07dsbc4yrGZqY/bl6nld01HWAU8Hdnb/S6LS1ysnvoic2XCBJmnTuSl2QuFcJkRU6XBuqDT8masrGXvru56aSpLiwSj1z/jJ1jC1R0/BK/VQZqaV7T9PTX5+n8tr639gEPsa98WFmEZG12rUzVi9OO/sYRw09MnG1Wras0MTx52vE3ekqLm6iyU+tUnjE0fdnvnbQdhmGpeGDBk5C1/hi3dRhm7YdjHfZv7mkucZ8eYkyPrpRQ5ZfJYuk2X0+UtCvfVuHYdHSPafp7lVXKn3RTXrwy0t1ftJeTTx3lQ8+BU5WY98b/1Ti02S/cuVKXX311UpOTpbFYtGCBQt8GY5prV/bUq/ndDlmFd6qdbk6p5bohWfP1vb8eO3dE6Ppz56jsDC7Lr1st8vY9qcf0sDrt+vZp3o2VuhAvTUJqdEzts/00NqLVVbturhp3s5UrTuQrL0VMdpysIWe2XSukqPK1TrqZ0lSWU245u44S5tLWmjf4RjlFrXWG9tTdW6L/b74KDhZdb+z92TzUz5N9hUVFerevfsfzlfAt0JDj1Q21dXBzn2GYVFNTZBSu/zo3BceXqsHH/pSL047WwcPRjR6nMCJTOj5uZbva6PVRa3/cFxkcI3+1C5fBeUx2n84+phjEiIrlNF6l9YeaNkQoQJe59M5+379+qlfv371Hl9VVeVyD+KysrKGCAu/s7sgRsVFTTTkzs16fuo5qqwM0bV/+lYtEn5RfHylc9xd93ytbVuaac3qZB9GCxxb/zY7dFbTH3Xdp9cdd8zgDlv0YNoaRYXWamdZnG7/b3/VOIJdxkw9f6nSW/2gyJBaLdvTVuO+vKShQ4cXeeumOv7Ir+bss7OzXe5HnJKS4uuQAp7dHqTHH7UpufXPevv9DzT/o/nq1v2A1n2Z5Oxo9bLtU/e0A3ppeppPYwWOpWWTcj3SY7VG5V6masfx65v3f+igaz75k25eerW+L4vVtAuWKizIdV3KExvO14BPBurulRlqE1Omh87Jbejw4U2GFzY/5Ver8ceNG6dRo0Y5X5eVlZHwG8GO7U014u4r1CSqRiEhDpWVhmvqC8u0/dsji5y6n12slsnleueD913O+/ujudqyqbnG/u1SH0QNHHFW0wNqHvGL3s9417kvJMjQuQn7desZW5T69p1yGEEqrwlXeU24fiiP1cafEpU3aLb6pnyvRT90cJ73Y2UT/VjZRN/93FSHqsI174oP9MLmc3SgMsoXHw2oN79K9uHh4W7fNQjec7giVJKU3OpndTjzoF7POUuS9M6bnfTpR+1cxs6YtUSvzOiuL3Np68O3cotaqd9H17vs+0ev5fquLE4vbUuTwzi6wWn5dQsLsh/3ukG/9nTDgj24SwsalZnb+H6V7NEwIiJqldyq3Pk6MalC7U8/pJ9/DtOB4ia68OI9Ki098u/T2pXp7qyNWvNFK32Vd+RBDAcPRhxzUd6B4iYqKqTigW9V1IZpe6nrT+1+qQ3RwepwbS+NV0pUmfq33alV+1urpCpCSU0q9NfOG1VpD9byfW0kSZe0LFDziMP6v5IEHa4N1RmxJRqbtkbrDyRpb8Wx7jeBU5KXnnrnj0j20BkdS/SPZ1Y6X//lnv+TJC35tK2mTjlX8c1+0V3DvlZc00odLInUssVt9Oa/3XsIA3CqqnIEq2eL/bq94yZZQ6v0U2Wk1h5oqRuWXKuSqsgjY+zBuvH0b/TQObkKC7Jr/+FoLd7TTjO3pvk2eKCefJrsy8vLtWPHDufrXbt2aePGjYqPj1ebNm18GJm5bPo6QVdd/qfjHv9g/hn6YP4Zbl3zj64H+Nrgz65x/rv4lyjdueKqPxy/priVbljK3SD9HW18H1m/fr369OnjfF23+C4zM1OzZ8/2UVQAgIBk4tvl+jTZX3rppTL8eA4EAAB/wJw9AMAUaOMDABDoHMaRzZPz/RTJHgBgDiaes/er2+UCAAD3UdkDAEzBIg/n7L0WSeMj2QMAzMHEd9CjjQ8AQICjsgcAmAI/vQMAINCxGh8AAAQqKnsAgClYDEMWDxbZeXKur5HsAQDm4Ph18+R8P0UbHwCAAEdlDwAwBdr4AAAEOhOvxifZAwDMgTvoAQCAQEVlDwAwBe6gBwBAoKONDwAAAhWVPQDAFCyOI5sn5/srkj0AwBxo4wMAgEBFZQ8AMAduqgMAQGAz8+1yaeMDABDgqOwBAOZg4gV6JHsAgDkY8uyZ9P6b60n2AABzYM4eAAAELCp7AIA5GPJwzt5rkTQ6kj0AwBxMvECPNj4AAAGOyh4AYA4OSRYPz/dTVPYAAFOoW43vyeaO7OxsnXvuuYqJiVFCQoKuvfZa5efnu4yprKxUVlaWmjVrpujoaA0aNEhFRUUuYwoKCtS/f381adJECQkJGj16tGpra92KhWQPAEADWLFihbKysrRmzRotWbJENTU16tu3ryoqKpxjRo4cqYULF+qdd97RihUrtG/fPg0cONB53G63q3///qqurtbq1as1Z84czZ49W+PHj3crFtr4AABzaOQFep988onL69mzZyshIUF5eXm6+OKLVVpaqlmzZmnu3Lm67LLLJEk5OTnq3Lmz1qxZo969e2vx4sXaunWrli5dqsTERKWlpWnSpEkaM2aMJkyYoLCwsHrFQmUPADCHumTvySaprKzMZauqqqrX25eWlkqS4uPjJUl5eXmqqalRenq6c0ynTp3Upk0b5ebmSpJyc3PVtWtXJSYmOsdkZGSorKxMW7ZsqfdHJ9kDAOCGlJQUxcbGOrfs7OwTnuNwOHT//ffrggsuUJcuXSRJhYWFCgsLU1xcnMvYxMREFRYWOsf8PtHXHa87Vl+08QEA5uClNv7u3btltVqdu8PDw094alZWljZv3qzPP//85N/fAyR7AIA5eOmnd1ar1SXZn8jw4cO1aNEirVy5Uq1bt3buT0pKUnV1tQ4dOuRS3RcVFSkpKck5Zu3atS7Xq1utXzemPmjjAwBMobF/emcYhoYPH6758+frs88+U7t27VyO9+jRQ6GhoVq2bJlzX35+vgoKCmSz2SRJNptNmzZtUnFxsXPMkiVLZLValZqaWu9YqOwBAGgAWVlZmjt3rt5//33FxMQ459hjY2MVGRmp2NhYDR06VKNGjVJ8fLysVqtGjBghm82m3r17S5L69u2r1NRU3XrrrZoyZYoKCwv18MMPKysrq17TB3VI9gAAc2jkn97NmDFDknTppZe67M/JydHtt98uSZo6daqCgoI0aNAgVVVVKSMjQy+++KJzbHBwsBYtWqRhw4bJZrMpKipKmZmZmjhxoluxkOwBAObgMCSLB8ne4X4b/0QiIiI0ffp0TZ8+/bhj2rZtq48++sit9/5fzNkDABDgqOwBAOZg4kfckuwBACbhYbKX/yZ72vgAAAQ4KnsAgDnQxgcAIMA5DHnUindzNf6phDY+AAABjsoeAGAOhuPI5sn5fopkDwAwB+bsAQAIcMzZAwCAQEVlDwAwB9r4AAAEOEMeJnuvRdLoaOMDABDgqOwBAOZAGx8AgADncEjy4LfyDv/9nT1tfAAAAhyVPQDAHGjjAwAQ4Eyc7GnjAwAQ4KjsAQDmYOLb5ZLsAQCmYBgOGR48uc6Tc32NZA8AMAfD8Kw6Z84eAACcqqjsAQDmYHg4Z+/HlT3JHgBgDg6HZPFg3t2P5+xp4wMAEOCo7AEA5kAbHwCAwGY4HDI8aOP780/vaOMDABDgqOwBAOZAGx8AgADnMCSLOZM9bXwAAAIclT0AwBwMQ5Inv7P338qeZA8AMAXDYcjwoI1vkOwBADjFGQ55Vtnz0zsAAHCKorIHAJgCbXwAAAKdidv4fp3s675l1dqrfBwJ0HAclZW+DgFoMHV/341RNdeqxqN76tSqxnvBNDKL4cd9iT179iglJcXXYQAAPLR79261bt26Qa5dWVmpdu3aqbCw0ONrJSUladeuXYqIiPBCZI3Hr5O9w+HQvn37FBMTI4vF4utwTKGsrEwpKSnavXu3rFarr8MBvIq/78ZnGIZ+/vlnJScnKyio4daMV1ZWqrq62uPrhIWF+V2il/y8jR8UFNRg3wTxx6xWK/8xRMDi77txxcbGNvh7RERE+GWS9hZ+egcAQIAj2QMAEOBI9nBLeHi4Hn30UYWHh/s6FMDr+PtGoPLrBXoAAODEqOwBAAhwJHsAAAIcyR4AgABHsgcAIMCR7FFv06dP12mnnaaIiAj16tVLa9eu9XVIgFesXLlSV199tZKTk2WxWLRgwQJfhwR4Fcke9TJv3jyNGjVKjz76qDZs2KDu3bsrIyNDxcXFvg4N8FhFRYW6d++u6dOn+zoUoEHw0zvUS69evXTuuefqhRdekHTkuQQpKSkaMWKExo4d6+PoAO+xWCyaP3++rr32Wl+HAngNlT1OqLq6Wnl5eUpPT3fuCwoKUnp6unJzc30YGQCgPkj2OKEff/xRdrtdiYmJLvsTExO98shIAEDDItkDABDgSPY4oebNmys4OFhFRUUu+4uKipSUlOSjqAAA9UWyxwmFhYWpR48eWrZsmXOfw+HQsmXLZLPZfBgZAKA+QnwdAPzDqFGjlJmZqZ49e+q8887Ts88+q4qKCg0ZMsTXoQEeKy8v144dO5yvd+3apY0bNyo+Pl5t2rTxYWSAd/DTO9TbCy+8oKeeekqFhYVKS0vTtGnT1KtXL1+HBXhs+fLl6tOnz1H7MzMzNXv27MYPCPAykj0AAAGOOXsAAAIcyR4AgABHsgcAIMCR7AEACHAkewAAAhzJHgCAAEeyBwAgwJHsAQAIcCR7wEO33367rr32WufrSy+9VPfff3+jx7F8+XJZLBYdOnTouGMsFosWLFhQ72tOmDBBaWlpHsX1/fffy2KxaOPGjR5dB8DJI9kjIN1+++2yWCyyWCwKCwtThw4dNHHiRNXW1jb4e7/33nuaNGlSvcbWJ0EDgKd4EA4C1pVXXqmcnBxVVVXpo48+UlZWlkJDQzVu3LijxlZXVyssLMwr7xsfH++V6wCAt1DZI2CFh4crKSlJbdu21bBhw5Senq4PPvhA0m+t9yeeeELJycnq2LGjJGn37t264YYbFBcXp/j4eA0YMEDff/+985p2u12jRo1SXFycmjVrpgcffFD/+3iJ/23jV1VVacyYMUpJSVF4eLg6dOigWbNm6fvvv3c+fKVp06ayWCy6/fbbJR15hHB2drbatWunyMhIde/eXf/5z39c3uejjz7SmWeeqcjISPXp08clzvoaM2aMzjzzTDVp0kTt27fXI488opqamqPGvfTSS0pJSVGTJk10ww03qLS01OX4q6++qs6dOysiIkKdOnXSiy++6HYsABoOyR6mERkZqerqaufrZcuWKT8/X0uWLNGiRYtUU1OjjIwMxcTEaNWqVfriiy8UHR2tK6+80nne008/rdmzZ+u1117T559/rpKSEs2fP/8P3/e2227Tm2++qWnTpmnbtm166aWXFB0drZSUFL377ruSpPz8fO3fv1/PPfecJCk7O1uvv/66Zs6cqS1btmjkyJG65ZZbtGLFCklHvpQMHDhQV199tTZu3Kg777xTY8eOdfv/JjExMZo9e7a2bt2q5557Tq+88oqmTp3qMmbHjh16++23tXDhQn3yySf66quvdM899ziPv/HGGxo/fryeeOIJbdu2TZMnT9YjjzyiOXPmuB0PgAZiAAEoMzPTGDBggGEYhuFwOIwlS5YY4eHhxgMPPOA8npiYaFRVVTnP+de//mV07NjRcDgczn1VVVVGZGSk8emnnxqGYRgtW7Y0pkyZ4jxeU1NjtG7d2vlehmEYl1xyiXHfffcZhmEY+fn5hiRjyZIlx4zzv//9ryHJOHjwoHNfZWWl0aRJE2P16tUuY4cOHWrcfPPNhmEYxrhx44zU1FSX42PGjDnqWv9LkjF//vzjHn/qqaeMHj16OF8/+uijRnBwsLFnzx7nvo8//tgICgoy9u/fbxiGYZx++unG3LlzXa4zadIkw2azGYZhGLt27TIkGV999dVx3xdAw2LOHgFr0aJFio6OVk1NjRwOh/785z9rwoQJzuNdu3Z1maf/+uuvtWPHDsXExLhcp7KyUjt37lRpaan279+vXr16OY+FhISoZ8+eR7Xy62zcuFHBwcG65JJL6h33jh07dPjwYV1xxRUu+6urq3X22WdLkrZt2+YShyTZbLZ6v0edefPmadq0adq5c6fKy8tVW1srq9XqMqZNmzZq1aqVy/s4HA7l5+crJiZGO3fu1NChQ3XXXXc5x9TW1io2NtbteAA0DJI9AlafPn00Y8YMhYWFKTk5WSEhrn/uUVFRLq/Ly8vVo0cPvfHGG0ddq0WLFicVQ2RkpNvnlJeXS5I+/PBDlyQrHVmH4C25ubkaPHiwHnvsMWVkZCg2NlZvvfWWnn76abdjfeWVV4768hEcHOy1WAF4hmSPgBUVFaUOHTrUe/w555yjefPmKSEh4ajqtk7Lli315Zdf6uKLL5Z0pILNy8vTOeecc8zxXbt2lcPh0IoVK5Senn7U8brOgt1ud+5LTU1VeHi4CgoKjtsR6Ny5s3OxYZ01a9ac+EP+zurVq9W2bVs99NBDzn0//PDDUeMKCgq0b98+JScnO98nKChIHTt2VGJiopKTk/Xdd99p8ODBbr0/gMbDAj3gV4MHD1bz5s01YMAArVq1Srt27dLy5ct17733as+ePZKk++67T08++aQWLFigb775Rvfcc88f/kb+tNNOU2Zmpu644w4tWLDAec23335bktS2bVtZLBYtWrRIBw4cUHl5uWJiYvTAAw9o5MiRmjNnjnbu3KkNGzbo+eefdy56++tf/6rt27dr9OjRys/P19y5czV79my3Pu8ZZ5yhgoICvfXWW9q5c6emTZt2zMWGERERyszM1Ndff61Vq1bp3nvv1Q033KCkpCRJ0mOPPabs7GxNmzZN3377rTZt2qScnBw988wzbsUDoOGQ7IFfNWnSRCtXrlSbNm00cOBAde7cWUOHDlVlZaWz0v/b3/6mW2+9VZmZmbLZbIqJidF11133h9edMWOG/vSnP+mee+5Rp06ddNddd6miokKS1KpVKz322GMaO3asEhMTNXz4cEnSpEmT9Mgjjyg7O1udO3fWlVdeqQ8//FDt2rWTdGQe/d1339WCBQvUvXt3zZw5U5MnT3br815zzTUaOXKkhg8frrS0NK1evVqPPPLIUeM6dOiggQMH6qqrrlLfvn3VrVs3l5/W3XnnnXr11VeVk5Ojrl276pJLLtHs2bOdsQLwPYtxvJVFAAAgIFDZAwAQ4Ej2AAAEOJI9AAABjmQPAECAI9kDABDgSPYAAAQ4kj0AAAGOZA8AQIAj2QMAEOBI9gAABDiSPQAAAe7/AwXMgx322NfoAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = metrics.confusion_matrix(y_test, preds)\n",
    "cm_plot = metrics.ConfusionMatrixDisplay(cm, display_labels=[0, 1])\n",
    "cm_plot.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Findings and Conclusion\n",
    "\n",
    "#### After making the predictions of the X_test data and comparing to the y_test 'target' data, the F1 Score for the model was about 73%. Not bad but could probably get better with other text cleaning or perhaps a different model for classification. We can also see in the confusion matrix above, that the model predicted 755 True Positives and 433 True Negatives correctly. For the incorrect predictions, there were 194 False Negatives and 119 False Positives."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Predictions and Submission CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_preds = model.predict(test_vec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  target\n",
       "0   0       1\n",
       "1   2       0\n",
       "2   3       1\n",
       "3   9       0\n",
       "4  11       1\n",
       "5  12       0\n",
       "6  21       0\n",
       "7  22       0\n",
       "8  27       0\n",
       "9  29       0"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission = pd.read_csv(f'{path}/sample_submission.csv')\n",
    "submission['target'] = test_preds\n",
    "submission.to_csv('nlp_submission.csv', index=False)\n",
    "submission.head(10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}