MovieLensRecommender.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5FOknBCEs4LR"
   },
   "source": [
    "# **Reccomendation System: MovieLens**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7LTtVC51vgor"
   },
   "source": [
    "**Section A: Importing libraries and configuration**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6UPxfuOPuC1n"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.stats import spearmanr\n",
    "import nltk \n",
    "import re \n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "import pandas_profiling\n",
    "#from pandas_profiling import ProfileReport\n",
    "import surprise\n",
    "from surprise import Reader, Dataset\n",
    "from surprise import SVD, SVDpp, SlopeOne, NMF, NormalPredictor, KNNBaseline, KNNBasic, KNNWithMeans, KNNWithZScore, BaselineOnly, CoClustering\n",
    "from surprise.model_selection import cross_validate\n",
    "from surprise.model_selection import train_test_split\n",
    "from surprise import accuracy\n",
    "from surprise.model_selection import GridSearchCV\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from flask import Flask, request\n",
    "import nltk \n",
    "import re \n",
    "import numpy as np \n",
    "from tqdm import tqdm\n",
    "import heapq \n",
    "%matplotlib inline\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pd.set_option('display.max_column',None)\n",
    "#pd.set_option('display.max_rows',None)\n",
    "#pd.set_option('display.max_seq_items',None)\n",
    "#pd.set_option('display.max_colwidth', 500)\n",
    "#pd.set_option('expand_frame_repr', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 1: Content-based filtering**\n",
    "1. Movies Info **->** Genre, Titles, Tags into one dataframe\n",
    "2. Genres **->** Top 15 genres as 1-0 columns using Bag of Words\n",
    "3. Tags **->** Embeddings using tf.hub model for universal sentence encoding\n",
    "4. Titles **->** Including them in the mix will allow us to bring together sequels as embeddings extract the context\n",
    "5. Applying Cosine Similarity to give out reccomendations\n",
    "\n",
    "**Note: So far, our system does not account for movies popularity and the ratings around it**<br>\n",
    "**Reason: Sorting on ratings is counter-intuitive, some people might like movies that are on average rated lower, are less popular.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AT-CkSDivezf"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>imdbID</th>\n",
       "      <th>spanishTitle</th>\n",
       "      <th>imdbPictureURL</th>\n",
       "      <th>year</th>\n",
       "      <th>rtID</th>\n",
       "      <th>rtAllCriticsRating</th>\n",
       "      <th>rtAllCriticsNumReviews</th>\n",
       "      <th>rtAllCriticsNumFresh</th>\n",
       "      <th>...</th>\n",
       "      <th>rtAllCriticsScore</th>\n",
       "      <th>rtTopCriticsRating</th>\n",
       "      <th>rtTopCriticsNumReviews</th>\n",
       "      <th>rtTopCriticsNumFresh</th>\n",
       "      <th>rtTopCriticsNumRotten</th>\n",
       "      <th>rtTopCriticsScore</th>\n",
       "      <th>rtAudienceRating</th>\n",
       "      <th>rtAudienceNumRatings</th>\n",
       "      <th>rtAudienceScore</th>\n",
       "      <th>rtPictureURL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy story</td>\n",
       "      <td>114709</td>\n",
       "      <td>Toy story (juguetes)</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTMwNDU0...</td>\n",
       "      <td>1995</td>\n",
       "      <td>toy_story</td>\n",
       "      <td>9</td>\n",
       "      <td>73</td>\n",
       "      <td>73</td>\n",
       "      <td>...</td>\n",
       "      <td>100</td>\n",
       "      <td>8.5</td>\n",
       "      <td>17</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>3.7</td>\n",
       "      <td>102338</td>\n",
       "      <td>81</td>\n",
       "      <td>http://content7.flixster.com/movie/10/93/63/10...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>113497</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMzM5NjE1...</td>\n",
       "      <td>1995</td>\n",
       "      <td>1068044-jumanji</td>\n",
       "      <td>5.6</td>\n",
       "      <td>28</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>46</td>\n",
       "      <td>5.8</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>40</td>\n",
       "      <td>3.2</td>\n",
       "      <td>44587</td>\n",
       "      <td>61</td>\n",
       "      <td>http://content8.flixster.com/movie/56/79/73/56...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpy Old Men</td>\n",
       "      <td>107050</td>\n",
       "      <td>Dos viejos gruñones</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTI5MTgy...</td>\n",
       "      <td>1993</td>\n",
       "      <td>grumpy_old_men</td>\n",
       "      <td>5.9</td>\n",
       "      <td>36</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>66</td>\n",
       "      <td>7</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>83</td>\n",
       "      <td>3.2</td>\n",
       "      <td>10489</td>\n",
       "      <td>66</td>\n",
       "      <td>http://content6.flixster.com/movie/25/60/25602...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>114885</td>\n",
       "      <td>Esperando un respiro</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTczMTMy...</td>\n",
       "      <td>1995</td>\n",
       "      <td>waiting_to_exhale</td>\n",
       "      <td>5.6</td>\n",
       "      <td>25</td>\n",
       "      <td>14</td>\n",
       "      <td>...</td>\n",
       "      <td>56</td>\n",
       "      <td>5.5</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>45</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5666</td>\n",
       "      <td>79</td>\n",
       "      <td>http://content9.flixster.com/movie/10/94/17/10...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>113041</td>\n",
       "      <td>Vuelve el padre de la novia (Ahora también abu...</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTg1NDc2...</td>\n",
       "      <td>1995</td>\n",
       "      <td>father_of_the_bride_part_ii</td>\n",
       "      <td>5.3</td>\n",
       "      <td>19</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>47</td>\n",
       "      <td>5.4</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>20</td>\n",
       "      <td>3</td>\n",
       "      <td>13761</td>\n",
       "      <td>64</td>\n",
       "      <td>http://content8.flixster.com/movie/25/54/25542...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                        title  imdbID  \\\n",
       "0   1                    Toy story  114709   \n",
       "1   2                      Jumanji  113497   \n",
       "2   3               Grumpy Old Men  107050   \n",
       "3   4            Waiting to Exhale  114885   \n",
       "4   5  Father of the Bride Part II  113041   \n",
       "\n",
       "                                        spanishTitle  \\\n",
       "0                               Toy story (juguetes)   \n",
       "1                                            Jumanji   \n",
       "2                                Dos viejos gruñones   \n",
       "3                               Esperando un respiro   \n",
       "4  Vuelve el padre de la novia (Ahora también abu...   \n",
       "\n",
       "                                      imdbPictureURL  year  \\\n",
       "0  http://ia.media-imdb.com/images/M/MV5BMTMwNDU0...  1995   \n",
       "1  http://ia.media-imdb.com/images/M/MV5BMzM5NjE1...  1995   \n",
       "2  http://ia.media-imdb.com/images/M/MV5BMTI5MTgy...  1993   \n",
       "3  http://ia.media-imdb.com/images/M/MV5BMTczMTMy...  1995   \n",
       "4  http://ia.media-imdb.com/images/M/MV5BMTg1NDc2...  1995   \n",
       "\n",
       "                          rtID rtAllCriticsRating rtAllCriticsNumReviews  \\\n",
       "0                    toy_story                  9                     73   \n",
       "1              1068044-jumanji                5.6                     28   \n",
       "2               grumpy_old_men                5.9                     36   \n",
       "3            waiting_to_exhale                5.6                     25   \n",
       "4  father_of_the_bride_part_ii                5.3                     19   \n",
       "\n",
       "  rtAllCriticsNumFresh  ... rtAllCriticsScore rtTopCriticsRating  \\\n",
       "0                   73  ...               100                8.5   \n",
       "1                   13  ...                46                5.8   \n",
       "2                   24  ...                66                  7   \n",
       "3                   14  ...                56                5.5   \n",
       "4                    9  ...                47                5.4   \n",
       "\n",
       "  rtTopCriticsNumReviews rtTopCriticsNumFresh rtTopCriticsNumRotten  \\\n",
       "0                     17                   17                     0   \n",
       "1                      5                    2                     3   \n",
       "2                      6                    5                     1   \n",
       "3                     11                    5                     6   \n",
       "4                      5                    1                     4   \n",
       "\n",
       "  rtTopCriticsScore rtAudienceRating rtAudienceNumRatings rtAudienceScore  \\\n",
       "0               100              3.7               102338              81   \n",
       "1                40              3.2                44587              61   \n",
       "2                83              3.2                10489              66   \n",
       "3                45              3.3                 5666              79   \n",
       "4                20                3                13761              64   \n",
       "\n",
       "                                        rtPictureURL  \n",
       "0  http://content7.flixster.com/movie/10/93/63/10...  \n",
       "1  http://content8.flixster.com/movie/56/79/73/56...  \n",
       "2  http://content6.flixster.com/movie/25/60/25602...  \n",
       "3  http://content9.flixster.com/movie/10/94/17/10...  \n",
       "4  http://content8.flixster.com/movie/25/54/25542...  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie = pd.read_csv('movies.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observations:**\n",
    "1. Rotten Tomatoes percentage implies an aggregate, not the rating of the movie\n",
    "2. Often, Rotten Tomatoes score of the movie does not coincide with the viewers' rating(viewers are forgiving)\n",
    "3. The audience amount of rating can act as a popularity measurement and act as a good attribute\n",
    "4. Movies with less rating counts have inflated ratings and fail to resonate with the population\n",
    "5. Based on the long head-tail plottings in the case of reccomendation systems datasets, it is essential to include the influence of popularity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     10197\n",
       "unique     5087\n",
       "top           0\n",
       "freq       2622\n",
       "Name: rtAudienceNumRatings, dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie['rtAudienceNumRatings'].describe(include=all)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. We know that there are 2500+ movies with AudienceNumRatings of 0!<br>\n",
    "2. We also see dtype as object indicating something irregular with the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>imdbID</th>\n",
       "      <th>spanishTitle</th>\n",
       "      <th>imdbPictureURL</th>\n",
       "      <th>year</th>\n",
       "      <th>rtID</th>\n",
       "      <th>rtAllCriticsRating</th>\n",
       "      <th>rtAllCriticsNumReviews</th>\n",
       "      <th>rtAllCriticsNumFresh</th>\n",
       "      <th>...</th>\n",
       "      <th>rtAllCriticsScore</th>\n",
       "      <th>rtTopCriticsRating</th>\n",
       "      <th>rtTopCriticsNumReviews</th>\n",
       "      <th>rtTopCriticsNumFresh</th>\n",
       "      <th>rtTopCriticsNumRotten</th>\n",
       "      <th>rtTopCriticsScore</th>\n",
       "      <th>rtAudienceRating</th>\n",
       "      <th>rtAudienceNumRatings</th>\n",
       "      <th>rtAudienceScore</th>\n",
       "      <th>rtPictureURL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>33</td>\n",
       "      <td>Wings of Courage</td>\n",
       "      <td>114952</td>\n",
       "      <td>Las alas del coraje</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTc3ODY1...</td>\n",
       "      <td>1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>http://content7.flixster.com/movie/10/97/83/10...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>143</td>\n",
       "      <td>Gospa</td>\n",
       "      <td>113200</td>\n",
       "      <td>Gospa</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>...</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200</th>\n",
       "      <td>210</td>\n",
       "      <td>Wild Bill</td>\n",
       "      <td>114938</td>\n",
       "      <td>Wild Bill</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTU4MDcy...</td>\n",
       "      <td>1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>...</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>347</th>\n",
       "      <td>360</td>\n",
       "      <td>I Love Trouble</td>\n",
       "      <td>110093</td>\n",
       "      <td>Me gustan los líos</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTI4MTIx...</td>\n",
       "      <td>1994</td>\n",
       "      <td>NaN</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>...</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>412</td>\n",
       "      <td>The Age of Innocence</td>\n",
       "      <td>106226</td>\n",
       "      <td>La edad de la inocencia</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BNzI4MjYz...</td>\n",
       "      <td>1993</td>\n",
       "      <td>NaN</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>...</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                 title  imdbID             spanishTitle  \\\n",
       "32    33      Wings of Courage  114952      Las alas del coraje   \n",
       "134  143                 Gospa  113200                    Gospa   \n",
       "200  210             Wild Bill  114938                Wild Bill   \n",
       "347  360        I Love Trouble  110093       Me gustan los líos   \n",
       "395  412  The Age of Innocence  106226  La edad de la inocencia   \n",
       "\n",
       "                                        imdbPictureURL  year rtID  \\\n",
       "32   http://ia.media-imdb.com/images/M/MV5BMTc3ODY1...  1995  NaN   \n",
       "134                                                NaN  1995  NaN   \n",
       "200  http://ia.media-imdb.com/images/M/MV5BMTU4MDcy...  1995  NaN   \n",
       "347  http://ia.media-imdb.com/images/M/MV5BMTI4MTIx...  1994  NaN   \n",
       "395  http://ia.media-imdb.com/images/M/MV5BNzI4MjYz...  1993  NaN   \n",
       "\n",
       "    rtAllCriticsRating rtAllCriticsNumReviews rtAllCriticsNumFresh  ...  \\\n",
       "32                   0                      4                    2  ...   \n",
       "134                 \\N                     \\N                   \\N  ...   \n",
       "200                 \\N                     \\N                   \\N  ...   \n",
       "347                 \\N                     \\N                   \\N  ...   \n",
       "395                 \\N                     \\N                   \\N  ...   \n",
       "\n",
       "    rtAllCriticsScore rtTopCriticsRating rtTopCriticsNumReviews  \\\n",
       "32                 50                  0                      2   \n",
       "134                \\N                 \\N                     \\N   \n",
       "200                \\N                 \\N                     \\N   \n",
       "347                \\N                 \\N                     \\N   \n",
       "395                \\N                 \\N                     \\N   \n",
       "\n",
       "    rtTopCriticsNumFresh rtTopCriticsNumRotten rtTopCriticsScore  \\\n",
       "32                     1                     1                50   \n",
       "134                   \\N                    \\N                \\N   \n",
       "200                   \\N                    \\N                \\N   \n",
       "347                   \\N                    \\N                \\N   \n",
       "395                   \\N                    \\N                \\N   \n",
       "\n",
       "    rtAudienceRating rtAudienceNumRatings rtAudienceScore  \\\n",
       "32                 0                    0               0   \n",
       "134               \\N                   \\N              \\N   \n",
       "200               \\N                   \\N              \\N   \n",
       "347               \\N                   \\N              \\N   \n",
       "395               \\N                   \\N              \\N   \n",
       "\n",
       "                                          rtPictureURL  \n",
       "32   http://content7.flixster.com/movie/10/97/83/10...  \n",
       "134                                                 \\N  \n",
       "200                                                 \\N  \n",
       "347                                                 \\N  \n",
       "395                                                 \\N  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie[movie['rtID'].isnull()].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. We realize that instead of eliminating on the basis \\N, we were to remove by rtID being null, that would require less code and be more efficient<br>\n",
    "2. We also remove all movies with 0 rtAudienceNumRatings\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape after null IDs removed:\n",
      "(9886, 21)\n",
      "\n",
      "Shape after non-zero remaining popularity: \n",
      "(7291, 21)\n"
     ]
    }
   ],
   "source": [
    "movie= movie[movie['rtID'].notnull()]\n",
    "movie['rtAudienceNumRatings']=movie['rtAudienceNumRatings'].astype(int)\n",
    "print(\"Shape after null IDs removed:\")\n",
    "print(movie.shape)\n",
    "movie=movie[movie['rtAudienceNumRatings']!=0]\n",
    "print(\"\\nShape after non-zero remaining popularity: \")\n",
    "print(movie.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1f7b98324c8>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEGCAYAAABLgMOSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZhcZZn38e9NEsJOFhKMBEyAgIIjASJEUURQCMgIzsAIzjtEZSagMDMO8zqGwREFfWVTFGURJBIYDCCykxAiBIJASDpkBRLSWUiaLJ2ks3e27r7fP+qpzunqWk9VdXV1fp/rqqtP3Wd7Oqmu+zzLeY65OyIiIoXaq9IFEBGR6qQEIiIisSiBiIhILEogIiISixKIiIjE0r3SBegohxxyiA8aNKjSxRARqSozZsxY6+790q3bYxLIoEGDqKmpqXQxRESqipl9kGmdmrBERCQWJRAREYlFCURERGJRAhERkViUQEREJBYlEBERiUUJREREYsmZQMxsjJnVm9m8SOxRM5sVXkvNbFaIDzKzbZF190T2OdnM5ppZrZndYWYW4n3MbJKZLQw/e4e4he1qzWyOmZ0UOdbIsP1CMxtZyn+Qclu2rpEp76+pdDFEJHhlQT116xsrXYyqlE8N5AFgRDTg7l9396HuPhT4M/BEZPWi5Dp3vzISvxsYBQwJr+QxRwMvufsQ4KXwHuDcyLajwv6YWR/geuBU4BTg+mTSqQan3zqZy8ZMq3QxRCT45h+m8+VfTql0MapSzgTi7lOAhnTrQi3iH4Bx2Y5hZgOAg9z9TU88wepB4MKw+gJgbFgemxJ/0BOmAr3Ccc4BJrl7g7uvByaRkuBERAqxbVdzpYtQlYrtA/k8sNrdF0Zig81sppm9amafD7HDgLrINnUhBnCou68ECD/7R/ZZnmafTPF2zGyUmdWYWc2aNWo2EhEppWITyKW0rX2sBI5w9xOBa4A/mtlBgKXZN9ezdDPtk/ex3P1edx/m7sP69Us7F5iIiMQUO4GYWXfg74BHkzF33+Hu68LyDGARcAyJWsLAyO4DgRVheXVomko2ddWHeB1weJp9MsVFRKQDFVMD+RIw391bm6bMrJ+ZdQvLR5LoAF8cmqY2m9nw0G9yGfB02O0ZIDmSamRK/LIwGms4sDEcZyJwtpn1Dp3nZ4eYiIh0oJzTuZvZOOAM4BAzqwOud/f7gUto33l+OnCDmTUBzcCV7p7sgP8OiRFd+wITwgvgJuAxM7scWAZcHOLjgfOAWqAR+BaAuzeY2Y3A9LDdDZFziIhIB8mZQNz90gzxb6aJ/ZnEsN5029cAn0wTXweclSbuwFUZjjUGGJOt3CIiUl66E11ERGJRAhERkViUQEREJBYlEBERiUUJREREYlECERGRWJRAREQkFiUQERGJRQlERERiUQIREZFYlEBERCQWJRAREYlFCURERGJRAhERkViUQEREJBYlEBERiUUJREREYlECERGRWJRAREQklpwJxMzGmFm9mc2LxH5sZh+a2azwOi+y7lozqzWzBWZ2TiQ+IsRqzWx0JD7YzN4ys4Vm9qiZ7R3iPcP72rB+UK5ziIhIx8mnBvIAMCJN/HZ3Hxpe4wHM7DjgEuD4sM9dZtbNzLoBdwLnAscBl4ZtAW4OxxoCrAcuD/HLgfXufjRwe9gu4zkK+7VFRKRYOROIu08BGvI83gXAI+6+w92XALXAKeFV6+6L3X0n8AhwgZkZcCbweNh/LHBh5Fhjw/LjwFlh+0znEBGRDlRMH8jVZjYnNHH1DrHDgOWRbepCLFO8L7DB3ZtS4m2OFdZvDNtnOlY7ZjbKzGrMrGbNmjXxfksREUkrbgK5GzgKGAqsBH4R4pZmW48Rj3Os9kH3e919mLsP69evX7pNREQkplgJxN1Xu3uzu7cA97G7CakOODyy6UBgRZb4WqCXmXVPibc5Vlh/MImmtEzHEhGRDhQrgZjZgMjbrwHJEVrPAJeEEVSDgSHANGA6MCSMuNqbRCf4M+7uwGTgorD/SODpyLFGhuWLgJfD9pnOISIiHah7rg3MbBxwBnCImdUB1wNnmNlQEk1HS4ErANz9HTN7DHgXaAKucvfmcJyrgYlAN2CMu78TTvED4BEz+ykwE7g/xO8HHjKzWhI1j0tynUNERDpOzgTi7pemCd+fJpbc/mfAz9LExwPj08QXk2YUlbtvBy4u5BwiIsW6+YX5HHnI/lw87PDcG+/hciYQEZE9yd2vLAJQAsmDpjIREZFYlEBERCQWJRAREYlFCURERGJRAhERKdK8DzeysXFXpYvR4ZRARESKdP5v/srX732z0sXocEogIiIlMH/V5koXocMpgYiISCxKICIiEosSiIiIxKIEIiIisSiBiIhILEogIiISixKIiEgaE99ZVekidHpKICIiaVzx0IxKF6HTUwIREZFYlEBERCQWJRAR2WO5e6WLUNVyJhAzG2Nm9WY2LxK71czmm9kcM3vSzHqF+CAz22Zms8Lrnsg+J5vZXDOrNbM7zMxCvI+ZTTKzheFn7xC3sF1tOM9JkWONDNsvNLORpfwHEZE9h/JHcfKpgTwAjEiJTQI+6e6fAt4Hro2sW+TuQ8Prykj8bmAUMCS8ksccDbzk7kOAl8J7gHMj244K+2NmfYDrgVOBU4Drk0lHRKQQyh/FyZlA3H0K0JASe9Hdm8LbqcDAbMcwswHAQe7+pifqjA8CF4bVFwBjw/LYlPiDnjAV6BWOcw4wyd0b3H09iWSWmuBERHJSE1ZxStEH8m1gQuT9YDObaWavmtnnQ+wwoC6yTV2IARzq7isBws/+kX2Wp9knU7wdMxtlZjVmVrNmzZrCf7Mqs37rzkoXQaSqKH0Up6gEYmbXAU3AwyG0EjjC3U8ErgH+aGYHAZZm91z/d5n2yftY7n6vuw9z92H9+vXLcbrq9sK8lZx44ySmL23IvbGIAOoDKVbsBBI6r88H/jE0S+HuO9x9XVieASwCjiFRS4g2cw0EVoTl1aFpKtnUVR/idcDhafbJFN+jTV2cSBxz6zZWuCQi1cNVBylKrARiZiOAHwBfdffGSLyfmXULy0eS6ABfHJqmNpvZ8DD66jLg6bDbM0ByJNXIlPhlYTTWcGBjOM5E4Gwz6x06z88OMRER6UDdc21gZuOAM4BDzKyOxAioa4GewKQwGndqGHF1OnCDmTUBzcCV7p5sU/kOiRFd+5LoM0n2m9wEPGZmlwPLgItDfDxwHlALNALfAnD3BjO7EZgetrshcg4RkbypCas4OROIu1+aJnx/hm3/DPw5w7oa4JNp4uuAs9LEHbgqw7HGAGMyl1pEJLtdzS00tyiDFCNnAhER6YqGXDeBI/rsV+liVDVNZSIie6xlDY25N5KMlEBERCQWJRAREYlFCURERGJRAsnB3fnuwzOY8n7XnwpFRKQQSiB5GD93FZeNmVbpYoiIdCpKICIiEosSiIiIxKIEIiIisSiBdDGamEFEOooSSBdh6Z6SIiJSRkogIiISixKIiIjEogQiIiKxKIGIiEgsSiAiIhKLEoiIiMSiBCIiErgekl6QvBKImY0xs3ozmxeJ9TGzSWa2MPzsHeJmZneYWa2ZzTGzkyL7jAzbLzSzkZH4yWY2N+xzh1niroY45xARkY6Rbw3kAWBESmw08JK7DwFeCu8BzgWGhNco4G5IJAPgeuBU4BTg+mRCCNuMiuw3Is45RESk4+SVQNx9CtCQEr4AGBuWxwIXRuIPesJUoJeZDQDOASa5e4O7rwcmASPCuoPc/U1P1B8fTDlWIecQEZEOUkwfyKHuvhIg/Owf4ocByyPb1YVYtnhdmnicc7RhZqPMrMbMatas0QOhRCQ7dYEUphyd6OlmZfIY8TjnaBtwv9fdh7n7sH79+uU4pIiIFKKYBLI62WwUftaHeB1weGS7gcCKHPGBaeJxziEiIh2kmATyDJAcSTUSeDoSvyyMlBoObAzNTxOBs82sd+g8PxuYGNZtNrPhYfTVZSnHKuQcIiLSQbrns5GZjQPOAA4xszoSo6luAh4zs8uBZcDFYfPxwHlALdAIfAvA3RvM7EZgetjuBndPdsx/h8RIr32BCeFFoecQjWMXKYb+egqTVwJx90szrDorzbYOXJXhOGOAMWniNcAn08TXFXqOPZWl7RYSESkf3YkuIlJB23Y2s3D15koXIxYlEBGRCvrOwzP48u1T2NnUUumiFEwJREQkqEQf4puL1gHQUoX9l0ogIiISixKIiIjEogQiIiKxKIGIiATV1wtRWUogIiISixKIiIjEogQiIiKxKIFUkZfnr2ZXc/XdbCRSDTY27tLzQAqkBFIl3qhdy7cfqOGXk96vdFFEuqRvj52eeyNpQwmkSqzbuhOAZQ2NFS6JSNc0f+WmSheh6iiBiIhILEogXYRpNneRornuBCmIEoiIiMSiBCIiIrEogYiISCxKICIige4DKUzsBGJmx5rZrMhrk5l9z8x+bGYfRuLnRfa51sxqzWyBmZ0TiY8IsVozGx2JDzazt8xsoZk9amZ7h3jP8L42rB8U9/eoOvqAi0gnETuBuPsCdx/q7kOBk4FG4Mmw+vbkOncfD2BmxwGXAMcDI4C7zKybmXUD7gTOBY4DLg3bAtwcjjUEWA9cHuKXA+vd/Wjg9rBdl6ZRViLS2ZSqCessYJG7f5BlmwuAR9x9h7svAWqBU8Kr1t0Xu/tO4BHgAjMz4Ezg8bD/WODCyLHGhuXHgbPC9iIi0kFKlUAuAcZF3l9tZnPMbIyZ9Q6xw4DlkW3qQixTvC+wwd2bUuJtjhXWbwzbt2Fmo8ysxsxq1qxZU8zvJyJSFtXcKl10Agn9El8F/hRCdwNHAUOBlcAvkpum2d1jxLMdq23A/V53H+buw/r165fxdxARkcKVogZyLvC2u68GcPfV7t7s7i3AfSSaqCBRgzg8st9AYEWW+Fqgl5l1T4m3OVZYfzDQUILfRURE8lSKBHIpkeYrMxsQWfc1YF5Yfga4JIygGgwMAaYB04EhYcTV3iSaw55xdwcmAxeF/UcCT0eONTIsXwS8HLYXEZEO0j33JpmZ2X7Al4ErIuFbzGwoiSalpcl17v6OmT0GvAs0AVe5e3M4ztXARKAbMMbd3wnH+gHwiJn9FJgJ3B/i9wMPmVktiZrHJcX8HiKyZ6nfvD1tXJehhSkqgbh7Iymd1+7+T1m2/xnwszTx8cD4NPHF7G4Ci8a3AxfHKHLV02RvIsXZ2LiLU372UqWL0SXoTvQqYWnHDYhIoTZt31XpInQZSiBdjKrgIuXV0uI0t+gPDZRAugzVT0Tys3x9+qd6mlleTcQj/zCNo/67XYv7HkkJRET2GBsbd/GN+94q6hivLVxbotJUPyUQEdljbN6h/o9SUgIRkT2GpswrLSWQKqNOcpHycHf9fRVICaRK6MJJpHj6MyotJZAuQhdOIrnpQqy0lEC6GP2BiEhHUQIRkT1GrhkdCqnJv1G7lhfmrSquQFWuqLmwRESqSSlr6N/4feJ+kqU3faV0B60yqoHkoFEZIiLpKYGIyB5DXYSlpQRSZVQjEilCjgxSkefSVfHftBJIldCVk4h0NkogXYxqKCKZdcrn6nTCIuVLCaSLqOLPoIhUKSUQEZEgtQL/wbqt3PLC/Mr0jVSBohOImS01s7lmNsvMakKsj5lNMrOF4WfvEDczu8PMas1sjpmdFDnOyLD9QjMbGYmfHI5fG/a1bOcQEcmk0PtALh9bw12vLGLpuvQPodrTlaoG8kV3H+ruw8L70cBL7j4EeCm8BzgXGBJeo4C7IZEMgOuBU4FTgOsjCeHusG1yvxE5ziEiklahTb1NzS1lKUdXUa4mrAuAsWF5LHBhJP6gJ0wFepnZAOAcYJK7N7j7emASMCKsO8jd3/REHfLBlGOlO4eISMHiPitkT27eKkUCceBFM5thZqNC7FB3XwkQfvYP8cOA5ZF960IsW7wuTTzbOVqZ2SgzqzGzmjVr1hTxK3Ye+TyzWUTSy5Uk9uBcEEsp5sI6zd1XmFl/YJKZzc+ybbr/PY8Rz4u73wvcCzBs2LCq/mholl2R+JqaWzCLP4h3T65lZFN0DcTdV4Sf9cCTJPowVofmJ8LP+rB5HXB4ZPeBwIoc8YFp4mQ5h4hIG8f9aCKfv/nlrNts2dHULqZH4GZXVAIxs/3N7MDkMnA2MA94BkiOpBoJPB2WnwEuC6OxhgMbQ/PTROBsM+sdOs/PBiaGdZvNbHgYfXVZyrHSnaND7Whq5prHZrF60/a069du2cE1j85i287mDi6ZiCTtbG5hxcbtOWvyc+s2dkyBuohim7AOBZ4MWbo78Ed3f8HMpgOPmdnlwDLg4rD9eOA8oBZoBL4F4O4NZnYjMD1sd4O7N4Tl7wAPAPsCE8IL4KYM5+hQL76zmife/pAdTS3c+Y2T2q2/5YX5PDHzQ049sg9f//QRFSihiCTlasRqaNzZQSXpGopKIO6+GDghTXwdcFaauANXZTjWGGBMmngN8Ml8zyEiUmrqAUlPd6KLiASp9RP1gGSnBFIqukQRkT2MEkiROnqQhkYTiuTW1NxCw1b1Z5SbEkiFvLaw0Bsb88tUutFQBK57ch4n3TiJHU1tRz9W8u+jcWf7YcLVTgmkQuaUeLighquL7PbcnMTtYruaS5Mwiq35PzXzQ4770URefb9rzIiRpARSIrmubNT0JLLbdU/OZdDo58t+nkLvIG93IVaiC7PvPToLgKmL15XmgJ1EKaYy2aNlGlf+/urNzFq2oc36t5et76hiiXRqD7+1rKzHjz8xYokL0sHH72hKIGVy9u1TAPj6sMQMLeu27uTv7noj7/3dnaYWp0c3VRJFKiaPL/xCkkLalooqTir6duogW9PMs5PNnZNrGXLdBDZt31WmEolIqkr2JVZjP6YSSImUumr6aE1idvsNW5VAROIq+s+y1F/qVVzbSEcJpEjlumrIlJC62OdPpCziT9te0mK0P355D9/hlEA6uWSCqsbqrUi1ifMFvzLDTNx7AiWQEpkwb1XW9V3tykOkGnTEqKfTbsr+nJFyqd+8nTF/XVKRcycpgRQpWjHYuK19f0XcmkNXG+4n0qGSf3cF/h1lvm+kVDcklu4P+6qH3+aG596ltn5LyY5ZKCWQUsry2Uj3uVm3ZUfBo7NEpOMsXrO1pMdLfg9sbNyV9oKzEMn9m1sqd7WpBFKkYv7rTv7pXzj316+xfVczt01cwPZdu+ftSV6pqO9DpOs54YYXOeEnL1a6GEVTAqmwZQ2N3DtlMb+dXMuDby4FYOSYaazYmOiY+9zNkytXOJEqtbsFq7BLvJYytx13tZZpJZAiRSsIJ9zwIovX5NceGW0L/XD9NmD3xG/pJly74qEZ8QspIkUpVV4pR36q5AzDSiAlNm/FprTxbE1RyZsG86HOdekodesbGTetvHNWdbRcfz8tLR1TjlJIzrNXye+E2AnEzA43s8lm9p6ZvWNm/x7iPzazD81sVnidF9nnWjOrNbMFZnZOJD4ixGrNbHQkPtjM3jKzhWb2qJntHeI9w/vasH5Q3N8jriVrt3LDs++S2n+Vmie27kz0a+T7n7yrubhPsBKMlMrXfzeVa5+YW5XPsYg7mWK5lbK20Bl+xWJqIE3Af7r7J4DhwFVmdlxYd7u7Dw2v8QBh3SXA8cAI4C4z62Zm3YA7gXOB44BLI8e5ORxrCLAeuDzELwfWu/vRwO1huw51xUM1jHl9Sbsmq9T/1PdWpq+RpGMGN02YH6s8nfUPRqrXmi07ANgrfLb+9jd/5ZEuViNJVe4+kJUbttNU5EViJu7OYzXL2bazOffGJRI7gbj7Snd/OyxvBt4DDsuyywXAI+6+w92XALXAKeFV6+6L3X0n8AhwgSW+Ec8EHg/7jwUujBxrbFh+HDjLOuAbNNo3kWnoXOr07sl3D725NOfxDWP60oas26zZnOhc39Xcwu2T3q/Kq0PpeF+87RWuevjtgvZJ/YzP/XAjo5+YW8piFSw6UjEfheaDctfgX3hnFT99/r3W98N+Oqlkx56ycC3/9fgcfj7hvdwbl0hJ+kBCE9KJwFshdLWZzTGzMWbWO8QOA6KN/XUhlineF9jg7k0p8TbHCus3hu1TyzXKzGrMrGbNmuKfBDZyzLTosYH2oyoypbGtJboqmF23kRUbtvHnGXX8+qWF/OovC0tyXOnalqzdyvNzVxa0TyXvL9iyo6ndxdHUxev4+P+8wBuL1ubcP/l3WOhvkKmJqZT/EtEL0bVbin9uezLpbWhMHKsjnwVfdAIxswOAPwPfc/dNwN3AUcBQYCXwi+SmaXb3GPFsx2obcL/X3Ye5+7B+/fpl/T0Klam6M/fDto+qLaReZJbfJHD1m3e0XonVLG0oW5VYKuf5OSvz+qLsqj55/UROvvEvbWLJp/lNXZT9qX5NzS1saEzcZHfSjYVd4XdEzix1W0ky6SUTfve9Oq45u6gEYmY9SCSPh939CQB3X+3uze7eAtxHookKEjWIwyO7DwRWZImvBXqZWfeUeJtjhfUHA9nbfkqs9Qon5QN39yuLSnPgHJKnfXvZBn79kmohXc1Vf3ybb9yXqNAPveFFvnjbKxUrS9xmnXN//Rof/58Jsc+7LaW5KtPTP1PdUcTfQ6bftZRfyXuVKIOktto3hQTSba+OG1xbzCgsA+4H3nP3X0biAyKbfQ2YF5afAS4JI6gGA0OAacB0YEgYcbU3iY72Zzxxo8Rk4KKw/0jg6cixRobli4CXvZSTzOShdQhdKUdVFLBd9Ledl1LrkfiufWIONzz7bqWL0eqhqR+woXEXS9aWdkqNjvDeyk1s31V47fjJmXVFnXfB6s2x9+2IeyrKVUFI1kB6dNt9gmufmMuIX00pzwkprgZyGvBPwJkpQ3ZvMbO5ZjYH+CLwHwDu/g7wGPAu8AJwVaipNAFXAxNJdMQ/FrYF+AFwjZnVkujjuD/E7wf6hvg1QOvQ33Jb3tAIZK6BAAwa/Xzrcqarpj+8vrRdLN8mLLO27XUagVU646YtZ8zrlZ3hNOp/npqXe6My68gb1d5buYn/eHR21m1ylaZ7EY+BznQZevHv3ox9zFT51qQKtbsGsvv446YtY/6q+Ak1l9jPRHf3v5L++258ln1+BvwsTXx8uv3cfTG7m8Ci8e3AxYWUt1Q+f8tkan74pby3z/Tdvi5NR1fjzmZmLd/QLv5YmhsNoxWuV99fw9H9D8i7TNK5/UMJv6w6o9smLsi4LtuowiVr2w6Zf3b2Cj4x4KB2n/0eRVziZ2rISPaplEK5rveaQ19o972MlhbnxXdXl+dEEboTPYZhP/1La1YvZf9Dpg/pfz0+p8371DbU5hbPOfxXKmP7rma++/AM6tYnaq5rt+xg0OjneWVBfcZ9pi3p2P/LD9ZtzTpyp9SNw7+dXBvrXE/NSnSBJqf8+ddxM/nSL19tt11RNZDYeyZs2r6LY344gdcWZh71mdpiELeG1zrfV9g92gcybvoyrvzf8k9/pARSpVL/0NZ34NA9yd/k+fWMn7uKG59L9KskR+mla8KslC/c+gqfv7kyD0VKlWkUVCGf72gfAMDTsz7Me99ik+V7Kzaxs6kla0d+agUpmVAKnUYltSaTTCDduxmrNnbMUxKVQKpU6lXL0nWNFSpJ9WtucQaNfp7bJ71f8mNXS/dUtvuUOnJ0yu9fW5w2vjXStJXrir17yiikf39kVt7nzzUWJ1P5kpaHiVGz9XOktiDsHWpMO5uKG44fHcbbUR+72H0gkp9COrjzHUhmpjmvSik5/9g9rxY5BLsAXf2/73+nflDQ9j94fE7WSUWjn/fxc1fy2aMOaX2/efsuDtynR+v77t2K6APJsf7mF7JPNfR//5R9AAC0v6jYEh4qt72pmYPpkWaP/DQ1R+4DSR3i29xSVNNeJqqBdCI7CrgC6epfQF1V9M/6qP8ez3cf7vzT9Kde2DTubMp5tfyTZ99pF3tq5oes2byjXfyWF+bnnJE6WoTlDdvazArxNz9u+2CmHlm+KHP93WS7kXDVxu2t/S85ZclhmS4qd8QY8hzVHNrA0t0Hsnl7eaY8UgLpRB6Znt+07nuZZayBNDSqL6RQHVGbSz2Hu9Pc4oyfu6qo474wbxUT30l/DHfnzsm1fLhhW1HnSHXcjyby93e/UdA+67bs4HuPzuLbD0xvt+6uPG6+LWSSw25lGIUFMPznL+V9nGwlyFS8HU2FTXeUeitBtA8k9RTlmiRSTVhlVo62yMR9IOk/EHOW66bCuMrTX5EyuWYJT+Lu3Pjcu/Q/qCfnHP+Rduvr1m/j1okLeG5OYXNgtTtPmljqlD25JK/c6zfH69xd1pC7j69maQP79+yetQaSS6m+Z5P/zeOmLeOLx/Zvuy7DPoW0QCSO0/Zm5nT3gSSVa4oW1UDK7N0CpnMvRKYPegfOYtBlrNzY/gr9l2XoUC+1ZQ2NfLhhG407sl+5btpW3D0MpfhSTV4Bx53G47JIk1UmF93zJuf++rWs94HkemJoqW6aNBIjoa59Yi5XPFTTZl2mf4Pzf/NXflPEbQHJPpA0XSBlq4Ho66YKFTLCQ3I78xft7yUoZj6ldMrx5/vX2sRki1sz3HyXvBJvKuNj9nY2tbB6U+5aRbEJpBDdsnSiX3RP9ps0S3WlbgYbQ+JuTBnhlu2f4BdFXLgk+0DS5QolkAoptFrZEX741FxuzXA3r6Y16VzK+d/xRm1iVtrUL6ik5GikvDt+M8my+zWPzeLU//dSu6nf2/f5xDv1y/MLv5u6RxHV8FJ9z76xaF1rc90B+7TtKZi+dH3WfVtiZrFkE1aLt7/IfOjNwkbF5UsJJIe/vFf+6QAKle0DWMQIRimjfL6Y6jdt5/RbJud1vJYWb53uPdP0H8mr/TjPh8h3QsMX5iU68LNd4ba0eGuSK/S7/dsP1OTeKMXkLHf551LKK/UnZyZuYJy5rP30RNm8ledMBKmd6M2tCcTbXbjkM1AhDiWQHI7/6EGVLkJB1ITVuaT+b2T733lm9oq8Oosh0be2vnEXg/rux/ZdLTS3OL97dRHHXBd/+vSol+fvnoqjkH6BpWu3Mnv5htarYYAv3DaZc8KMsMsbSjsiLOlHT++edPKNHM8LyaaU94bcmrEAAA1jSURBVAI98Xb+d8CneujNpTmn8J9TlxjIkPyXTtY03b3DHiqlUVg57NOjW6WLUBA1YcUXZ+rxuFIvdHc2tbR51Gkur4f+jy8fdyj3vbaExp1N/HxC9pvcUm1s3MUJN7yYdt1+MT/3Z6T50itX0oh6sERNNOW6X6IQP5/wXmtyyGTT9t0DI5JDjxu2Ju6xaXF44I2lZStflGogOVTb93FnbHKTxP/LhsadGT9Pm7cXNlLq9UXrOLr/AQw+JDETbbp+kFwzG7y3KvMIwX333p1Ash0mWdOI2/KTz9M0kxNR7imiySM5c/Gm7bsYNPp5nng70bT41MzdtZtkZW/ygjXhfcfdZqwEIpKHqYvXtV71F2LUQ7vvNL/txQWtI6dSFdL0uKOpmWlL1vG5ow9h/56JL/qtOwq/cs42dXo0gcxftbl1uo1S+8t7ufsrXlmQeWbbri45c3HyOUTXPDabv7vr9dYhuwB/f/cbXBn5nKXrgy/XhbCasEQy2L6rubUJ85J7pwKw9KavxD7e/05d1ro8LWX6/eY8rhqXNzRyeJ/9mLlsA9t3tfDZo/q2rovWQNwdM0v7zJmodVvSrz/mhxPaTFVy6X1Ty/a8mXymHP9hJ3ioViW9UbuWg/fbPUfW28s28IVj2t6c+EJkNoJ0Nc9uZcogqoHk0HHzWkpnk+4O7osKnMIjk9S5pPIZuplsmni9di17GQw/qi/790xcA0ZrIMlD/cuD7UcwRac1qU8zL9Xm7bvSznNVW5/9Brxqa+qtJqkXG9D+33tQ3/1al9M1YTWV6VZ0JRCRDB56c2m7WM0H2cfwx7GruSWvGkjyS+D12rV8amAvDtqnB/uFpqY1W3Yng+QV6Mo0z4S4acJ8pi9tYOHqzWkfdbp0bbz+hmTxTxh4cKz9JbOtO5raPSskdaaEpesaW5NKsdPCF0JNWCIZzK7byI6mZnp2L+9IvCsemsE/nnpEzu2+/6fZjP32Kcyu28iVXzgSgAEH78teBn+q2X3fRvJic5/ue7X7Mnl29gqenb0i4zkWr81e04galaaGk6wRSenc99oS7nttSc7tkkk8n21Lpar/t81sBPBroBvwe3e/qdTn2LBNs9vuyf741jIOKPOX4svz63l5fu7O5LeXbeCtxQ00tzinHZ14HsZHDt6Hr3/6cMZN2z2Tc7IJI06zxaIcTVVRqc/cfmvxurLU0KTzqtoEYmbdgDuBLwN1wHQze8bd3y3leTrDuHCpnJ88m/nj5O64J27kcvfwM3HjXblGUt7z6iL26bEXJx3RuzX2n2cfy4R5q9jQmBgKPGHeStZu3plxipNMrnr4bSZFhoF//5xjM06Zk+q2Fxdw75TsT+sDuHDoR1ufbS7Vr2oTCHAKUOvuiwHM7BHgAqCkCWSfMjdflMMXbp3c5gsseSdxm1iaL7jo6A2n/bbRO5LbHj/dcbNvm/NcGdbj7WOlKHchBo1+PuaexZtTt5FbL/5UmxtcDzmgJ8/96+f43M2JaVD+49HsT8X70icOTXu/0PNzV/LxjxzY2jdy5ReOyjuBRJPHbRefkPHJfN88bTA9uu3Fn2bkN1VKtdm3Rze27SoscZfSTy/8JDdNmF+2YdepqjmBHAZEn8BUB5wa3cDMRgGjAI44Incbczp/U2WdgqcO7sOAg/cB2t6Vbu0W2o4wS24aHdxhabZtE2szEqTIY7XZtv2QnsL2b3+sDEXFsKzTVxzWa992D2T6tzOPBkuUwixxjMTP8N52l+/dFZvyeh7HZ4/qy/L1jSxv2MbxHz2Id1a0v8nv6P4H8JtLT+QTA9pPrzOw937cetGnuO+1xdxx6Yl8tNe+PD1rBQb03X9vvvPw25z/qQGcOrgP/2f4x3hz8Tqenb2SK04/kjNue4Uh/Q/gP88+ltOPOYR3V2yicWcz3fYynrrqND5Yt5WFq7fwyPTlrG/cyZT/+iIT563is0f35bcv1/LcnJX89hsncvLHejNh7iouOnkg9Zu3M21JAwMO3ocp769l+JF9Oe3ovpww8GCGXnwC55/wUSDRZPaZo/rSs/tePDt7JXe9UsvB+/agfvMOrvnyMdzz6iJ+dP5xjJu2jGvOPpY3F63j1CP78PGPHMi/jZvJYb32Zejhveh34D68PL+ekz7Wi9Ubt1O7Zgvj567i04N689UTPsoXjunPpPdW07P7Xmzb2cyfZizn80P6cfnnBrNPj27MX7WJ375cy6cH9WHv7nvR1JyYT+qIPvvR/8CeTF5QzwE9e3D7XxKd1/v26MaZn+jP988+lidmfsgf31rGo1cM56h+B7Buyw7ufmURw4/sy+Mz6qhds4Xa+i38/UkDufrMo/nmH6bxwbq2gxWO6LNf6zQ2J3+sNzM+WN/689CDerJ60+5BEkP6H0D/g3oyp24jm7c38cOvfIJ99+7Guys28bcnfJTe++3Nq+/X870vHcOzs1fw8wnzefX7Z+T8DMZh+T6Hu7Mxs4uBc9z9n8P7fwJOcfd/Tbf9sGHDvKam8InZRET2ZGY2w92HpVtXzcN464DDI+8HAmpcFRHpINWcQKYDQ8xssJntDVwCPFPhMomI7DGqtg/E3ZvM7GpgIolhvGPc/Z0KF0tEZI9RtQkEwN3HA+MrXQ4RkT1RNTdhiYhIBSmBiIhILEogIiISixKIiIjEUrU3EhbKzNYAxTw4+RCg8EfSVV61lhtU9kqo1nKDyl4uH3P3fulW7DEJpFhmVpPpbszOrFrLDSp7JVRruUFlrwQ1YYmISCxKICIiEosSSP7urXQBYqrWcoPKXgnVWm5Q2Tuc+kBERCQW1UBERCQWJRAREYlFCSQHMxthZgvMrNbMRlewHGPMrN7M5kVifcxskpktDD97h7iZ2R2hzHPM7KTIPiPD9gvNbGQkfrKZzQ373GHpHgsYr9yHm9lkM3vPzN4xs3+vorLvY2bTzGx2KPtPQnywmb0VyvFoeJwAZtYzvK8N6wdFjnVtiC8ws3Mi8bJ9vsysm5nNNLPnqqzcS8P/5ywzqwmxTv95CcfuZWaPm9n88Jn/TLWUPRZ31yvDi8Q08YuAI4G9gdnAcRUqy+nAScC8SOwWYHRYHg3cHJbPAyaQeMrqcOCtEO8DLA4/e4fl3mHdNOAzYZ8JwLklKvcA4KSwfCDwPnBclZTdgAPCcg/grVCmx4BLQvwe4Dth+bvAPWH5EuDRsHxc+Oz0BAaHz1S3cn++gGuAPwLPhffVUu6lwCEpsU7/eQnHHgv8c1jeG+hVLWWP9ftW8uSd/RX+oyZG3l8LXFvB8gyibQJZAAwIywOABWH5d8ClqdsBlwK/i8R/F2IDgPmReJvtSvw7PA18udrKDuwHvA2cSuKO4e6pnxESz6b5TFjuHraz1M9Ncrtyfr5IPKHzJeBM4LlQjk5f7nC8pbRPIJ3+8wIcBCwhDE6qprLHfakJK7vDgOWR93Uh1lkc6u4rAcLP/iGeqdzZ4nVp4iUVmkZOJHElXxVlD81As4B6YBKJK+8N7t6U5nytZQzrNwJ9Y/xOpfAr4L+AlvC+b5WUG8CBF81shpmNCrFq+LwcCawB/hCaDn9vZvtXSdljUQLJLl37YjWMe85U7kLjpSuQ2QHAn4HvufumbJtmKEtFyu7uze4+lMQV/SnAJ7Kcr1OU3czOB+rdfUY0nOVcnaLcEae5+0nAucBVZnZ6lm07U9m7k2hmvtvdTwS2kmiyyqQzlT0WJZDs6oDDI+8HAisqVJZ0VpvZAIDwsz7EM5U7W3xgmnhJmFkPEsnjYXd/oprKnuTuG4BXSLRV9zKz5NM8o+drLWNYfzDQkKPs5fh8nQZ81cyWAo+QaMb6VRWUGwB3XxF+1gNPkkjc1fB5qQPq3P2t8P5xEgmlGsoeTyXbzzr7i8QVxWISHYjJzsLjK1ieQbTtA7mVtp1zt4Tlr9C2c25aiPch0UbbO7yWAH3Cuulh22Tn3HklKrMBDwK/SolXQ9n7Ab3C8r7Aa8D5wJ9o2xn93bB8FW07ox8Ly8fTtjN6MYmO6LJ/voAz2N2J3unLDewPHBhZfgMYUQ2fl3Ds14Bjw/KPQ7mrouyxft9KnrwaXiRGSrxPou37ugqWYxywEthF4krkchLt1C8BC8PP5IfMgDtDmecCwyLH+TZQG17fisSHAfPCPr8lpSOwiHJ/jkQ1ew4wK7zOq5KyfwqYGco+D/hRiB9JYjRMLYkv5Z4hvk94XxvWHxk51nWhfAuIjJwp9+eLtgmk05c7lHF2eL2TPHY1fF7CsYcCNeEz8xSJBFAVZY/z0lQmIiISi/pAREQkFiUQERGJRQlERERiUQIREZFYlEBERCQWJRCRCjCzNzLEHzCzizq6PCJxKIGIVIC7f7bSZRApVvfcm4hIqZnZFnc/IDzP4TckphtZQvr5jkQ6JdVARCrra8CxwN8A/wKoZiJVQwlEpLJOB8Z5YtbfFcDLlS6QSL6UQEQqT/MJSVVSAhGprCnAJeHBVQOAL1a6QCL5Uie6SGU9SaIDfS6J2W1frWxxRPKn2XhFRCQWNWGJiEgsSiAiIhKLEoiIiMSiBCIiIrEogYiISCxKICIiEosSiIiIxPL/AeQAEGDxqqO1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "movie.groupby(\"id\")['rtAudienceNumRatings'].mean().plot(logy=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Earlier, I was planning on scaling number of ratings by tens, thousands, and so on but this graphs tells a seperate story and we will have to accomodate some changes**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1f7ba01bfc8>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEGCAYAAACevtWaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2de3wU5fX/PyebO0kIlwAhXBIMVwUEIuAFL3jjqq29gf1aa1FaW9uq32qx9vutbW211p+1Kv0qtkovCqJVKwYFRBQURUAk3CFchEAg4ZoLuW3y/P7Y2c3sZmZ2ZnZmd2Zy3q9XXtl9duZ5zs7OnDlznvOcQ0IIMAzDMN4iKdECMAzDMNbDyp1hGMaDsHJnGIbxIKzcGYZhPAgrd4ZhGA+SnGgBAKBnz56isLAw0WIwDMO4ik2bNp0QQuQpfeYI5V5YWIiNGzcmWgyGYRhXQURfqn2WULcMEc0kogVnz55NpBgMwzCeI6HKXQixVAgxt2vXrokUg2EYxnPwhCrDMIwHYeXOMAzjQVi5MwzDeBBW7gzDMB6Eo2UYhmE8CEfLyPh0/0mUV9UlWgyGYQC0tQks2XgY/ta2RIviStgtI2PWgk9xzRMfJloMhmEAvLapAve/Vobn1x5ItCiuhJU7wzCO5PS55rD/jDFYuTMM4wqEEFj02SHUNrYkWhRXwMqdYRhHcq65FQBQ3+QHAGz68jQeeH0rfvnmNt197K+uQ9EDpdhX3fnm0li5x4Fmfxtq2NpgGEO8+HHA1/7S+kMAgIaWgLL/zxdHUTivVFcfy7ZWQgjg35sq7BHSwViu3IkoiYh+R0RPE9GtVvfvRr63cANGPbQi0WIwjCsQQuCpVXtR0+gPayeQ4b4yUwOJb4NPAZ0JXcqdiF4goioi2hbRPoWIdhNRORHNk5pvBFAAoAWArbfLDQdPoXBeKfYcr7VzmJj5qPxEokVgGNdQVnEWT6zc06H9i8OnDfeVJN0P2oSIVSzXoddyXwhgiryBiHwA5gOYCmAEgNlENALAUACfCCHuBXCndaJ2pLSsEgDwMStPhvEM/jZlRfzu9mOG+yIybu17BV3KXQixBsCpiObxAMqFEPuFEM0AFiNgtVcACN5iO9+zEMMwjqMTGu4x+dwLAByWva+Q2l4HcD0RPQ1gjdrORDSXiDYS0cbq6uoYxGAYhlEmaLgLmNPuNz7zEb7x7DoLJYofsZTZU3reEUKIcwDmRNtZCLGAiCoBzExNTR1nRgDhkNtxdW0TPiqvxlfH9Eu0KAzjSdrahKkJ1eAeZlXFlgr35r2KxXKvANBf9r4fgKNGOrAqt0yivWq3/30D7nllC6prmxIsCcN4k9KtlTDlPpd2coYZGF9iUe4bAAwmoiIiSgUwC8BbRjrwSlbIYzWNAIBWlYkghmFi41yz35QRl2jDL5HoDYVcBOATAEOJqIKI5ggh/ADuArAcwE4AS4QQ240M7rSskAzDJJbjNY144PUy5Q91mO5N/lZFI8shHty4osvnLoSYrdK+DMAys4MT0UwAM4uLi03t3wl/L4bxLOv2ncDNz69X/VyPFT70l+9iQlF3vPL9iwP7hHbqfNrCE/ncO3MsK8N4hX+s+1Lz8ySdl/n6A6ew42gNzja0mJqE9QpciYlhGEcQzUYzYsRNe2otbn7+09D7zuiW8YTlzjCM+0mKorwjP911rAZbDp9R3X770RpzETYeIZY4d4ZhGOuIoogjlf+UJ9cCAA4+Oj1q12y5x5lY3TLRfrAjZxpCaUOV2H70LN7Y3PlSgTKMU2hobkXhvFIs/uyQ5naPvbvbVFxjaBETT6jGl1jdMsEfTO3R69YXPsOvl+5QXVw0/amPcM8rW0yNzTBM7JyoC1ybz6wu13TLnKxvNhfnHkw/0Pl0uzeKdaj96GcbAgUynJKmgGHcwJlzzahr8kff0CR3vfw5frp4c4f2aMrbjP88GC2jpQFW76rC/NXlxjt3OK52y5jlpfVf4t1t7elDT9c34/Hlu3Xvf67Zb+vJzzCJ5MLfrMQlj6yyrf+3yyrxny86ZiqJGi0Ti19Gg9sWbsAfDVz/biGhE6pCiKUAlpaUlNxhbn9z4z74RngNxl+9tR1vbdGfFqfk4fdwrrm1w0ROZ/TrMd4ksgpSPIimh5NiMEU748O7J9wy0W7524/WaH7e2GIs7Xxkya7OvFCCYawiWhy7metsZ2Xg2o+34bXneC2+0AjTjAfeUO5RuG3hBkPbCyHw5uYjaGxpRZO/NTTpwzCMfdjhc3/x44OBFwZ0+7vbjuFUfbPxwWRc96c1+Mr8j2PqI1Y6hXIHgNW7q3Rv+8n+k7j7lS/w369uwR3/2ISSh98LfVby8MrQ6z8u3wWgPSskwzAxYOMDsF7dfrq+GT/41ybc/ndjBqETcfWEqtoPdvjUOZRXhRfNvu1F9R8r0iKolfyNpWWVWLOnvUpUY0srTtS139Hnr96HY2dZsTPx56lVez0X4RHN7RIPv3lLaxsA4NCpBvsHsxlXx7kHKTt8Bs3+ttD7SY+txjVPrIG/tU1jr3b0FtmY/PgHHdpaZGMET76G5laUV9Xp6pNhXv+8AjWNLYb2eWLlnlCEx57jtSicV4odUeaWnE40t4vdfvMnVu4JjXCirgn7qq27hssqzuDuxZvRFseaD55wy7y6qQK/fXtHh/bT58IvGLV4988P6Zv4OBrFSr/k0fcBAN//1yZc88SHXLzDo1TVNlp2ke44WoN7l2zBfa+aX0wXDOt9Z1ulJTJF4/XPKzrMQ/2udAcK55Ua6ufwqXNh76N5ZWKx3PWsdXlq1d6wJ/Wr/9+H5geM4Pa/b8SbXxxFVRyrtXlCuQOBVALReHVTfFINfFx+AgBw3i+WcTy8hdQ2tsTV8lHi2NlGjP/dKjy5ai9qGltw8SOr8N6O46b7a5AiteJ50QPAtD+vxasbD0ffMIJjZxtx75It+P4/N4W1P79WPc2HnB+9/Hno9aTHVod9FtVyj0W569yuzSbfT9DQS/a1f8lT9c2GI/WM4GrlLv8dyqvqorph7n+tDI+8szNqv1Ye8OM82WoJ9U1+jHxoBR59d1dC5Qj+nh/srsKoh1ag8mwjfvWWoQJkjmBHZQ3ue61jxaMqjfP1wz3VmCgtbjJ7XpeWqT9dRPW5q6joQyfPKbaH7atTZ9tVGyLovvXJ+h/725X4r7+qFyeJFVcrdzk1jX4UP/gO9kfxkz334f6off108Rcd2j6UPa7JuUrBDy9n9S79UTqMOsFJ7v98ccSyPoUQYa4zf2sbKk5rK4obpfC2PcdrNbdLBFYYnV97dp3qZ39auSf0WksHNra04vfLdqLe4FOr2UVKsxZ8omu7tjaBqlrtm5LPJuUePM8EgIMn6kOLJjd+edqW8QCXR8soMdlCP5mcl9crV4nxK9ZrbG97uDT6kwKTGOavLg9znf1u2U5c9ofVuibYG1v0TdbrRUkxN7a0Ys7CDTh4ol5zXyvV0WGNKBG5iHIrO1K+RZ8dwoI1+/H0+9Gjeeqb5TcAc9EyelbTCgBPvrcH43+3SjPCLZZVsFoE9USbELjuyTX4yaKOuXWsxuXRMvHzvy7fbt6vypjn0MlzmPbntTEvKlFi0WcBn/OZc4G+g5NpZxusH0sNLUNxzZ5qrNpV5RgDQa0wxqPvhLvK/K1C+h/9BnjL3z7TPb7a1R4sv/fXtepP5Uu3HMUq6Sla6+YdrWCIWeTKXR7ZZyeuLtbRGfNFdDaeXbMPOyprsGyrfZEgSufRybomdO+S6qr6vHpDBW99Qb9CBYALfrUcw/OzdW0rd8fsOlaLc81+ZKaqq5mgohVCf43USJKkHaPdBKOlIQHs87mH3H9x1Fme8bkz3iaeOra8qg7jHn4P//xUu2CzU5Afm9W7q6JahpHzR1W1jbj/tS2K80qF80pR1+THhoPhvmG132PBmv14aX2g8MZH5Sfww5c+V95Qgai/sYpibG0TeMiiSW2zNxi9xDPYi5U7w0SwX/Ihr9lzwnQfrW0CNz//qWraiw0HT2HoL98JuYSsuOY3HDyN217cgMdXGEtf+5ulO7BkYwUWrNmnex+9OnD9/lMG+jQXLVPb6MfCdQf1j6MxjN1JAOOZwIyVuw2wt8h6gq6Tsw0trii+crK+Cev2nVT1U/9ldTma/G3YrHMBna4xpYVF0SZgIwkeziYDk8Ry98W724+pbudva0Ozvw23/G09yiq0v2s8n862Hz2LSx5ZFbq5BlGz3K1acc6Wu06ceo07VS43smRDYNIzaPE0trRhwZro4ax6sFOZVNUEFK2aiySoHK04V2L1E4eOrd/6BTX+NoHyqjqs3XsC9yvE1Qc5cqYhaopcK6+rP7+3F0fPNuLTiCcLtWN5zRMfasbo68XV6QeI6EoiWktEzxLRlVb3z3QuglEG8sfl93ZaG7lkx804uMhHVbkHx7bxOe/Y2UbNRUlBgt8/8jhYoYiMHNuyCu2QaCuPVHBlcGaqL6xdy+e+65i7cvfoUu5E9AIRVRHRtoj2KUS0m4jKiWie1CwA1AFIB2Drev9XTCyftptE53D2KnYoQTst9+OS5d6iEg7YYWwL7jCRPUx8ZBXG/z56ubzg0JEyKa3hCHLgRH3UBV9WY5U77hdvbA0V3MmIUO5aI1hiuSt8Bz0rbM2g13JfCGCKvIGIfADmA5gKYASA2UQ0AsBaIcRUAD8H8GvrRHUHia6+4lUS4+pqH/TwqXP4t4HcRCHLXTXWO6BJX5YiS/TKAcBQvV89qN04o+VZufNf+iJhrLoxW3UKlFWcDSn37UfCnxYic+bI2W9wLkMJpfvl54fsWaWqK85dCLGGiAojmscDKBdC7AcAIloM4EYhRDA942kAaWp9EtFcAHMBYMCAAcakZhgLiVQ+ShETX/3Lx2G5/KMRVO5NklvmyJkGpPgIvbLTA2NIQ0RmLg2TQ+XR4hmVPO7nmgIK6/Bp/bnI39laqbpAL1pWU6NJttSeYvRi5eRzsPzeQ0s7ZpONxqn6ZviSCF0zUgzvq/T0YdcTZCyLmAoAyP0iFQAmENFNAK4HkAvgGbWdhRALACwAgJKSEp6CZHSTiJq1RhQ70K7cW6TVmpdK6aCDRdXtiKcOVgQLKq5IVmw/1qH+752yOPRtR8L303LLAMB5eVm65PqFVJA+Wn9uYexvVyLFR9j7u2mKn0dG4MiJ5yGIRbkrnZ5CCPE6gNd1dUA0E8DM4uLiGMRgOgOvxSldsxJmMiAeD0XLKEegRN6gIq/5nZU1uOMfGw2Pq8VcDZeDEtEmVHMz9VmuauGgbiQYEhm8aSuxQ+XmGkDJcrfHWIklWqYCQH/Z+34AjsYmDsMoY0e+c71PAL9eanz1YzD7oJoSiHY9/+o/+scM9pXqC1zO6SnWBMG1RnG7ENSjgbzKNU90TEwohECD7IlIK42v0v3SrufQWM6CDQAGE1EREaUCmAXgLSMdWFVmj/EmqpajRVfDIakSUOTqxg4+eIOWVUtrW8iNox7nrt2HmeyE+bkBf/6VQ3oZ31mBaD53IsJDJm58XuHT/ScBAP/69EsM/993ceRMYK4j8rAdkE3EKs1T2JWsTG8o5CIAnwAYSkQVRDRHCOEHcBeA5QB2AlgihDD0S9uR8pfxDusP6F+6HgsvfnwQgPrCIqMXnzzroHqcu3afvjCnvL7x1UIazRJduQOf7jupuz+j+d2dzqwFnwIASqWkdl8qRNMcPFEf9r2VHoYSOqEqhJit0r4MwDKzgwshlgJYWlJScofZPhjv4m/T98h/+NQ55GWnIT3FF31jHURebNGuvcgUskEfPZFGKGREp7HcUDr47y2atIuq3A0+QtmRttkJBI/Dr5fuwNiBuWGfXfn4B7isuGfovZLl7kS3DMMkBPnF0NLahkmPrcbdCtWzjPDutkrV/CHRIlsiFXhwMrVPTrppn7ReV9D/vLkNdU0t0j6mhlIlWtWr2sYWQ7HfHgmWUWX38dpQjQA58upwjrPc7YKjZRgzHD3bHscdtC7fV8m+qJcfaCzIMepzD06m9u+WidMqYXHRevSFbdCuEeZHxLjL0xJbvdDr8RV7ND+PV8F5tzO0TzaOStWflH8j50XLxAxPqDJm0CoFZwdGL73jNY3wJRF6d01XdctEm0/wKTwufHbgFP6osTo1OBHsovoiniDa8e6bmxF6reiWsen38lwNVaZz0uxvC9VCjZXH3g1XoEYt9+M1TeiVnYb05CRVt0ykn373sfCC20o+9xUaqXWB9rqutTpqijLWIIQwpJyVDHe75iLYcmdcjfzC+u8lsfndIwkaWUZXkx6vaUSvnHSkJifpXnIfaeErKfemKP774A3jo3LzRUbkKD09MOHoWXUr30TJcv8gRpeiGjyhyniGL23Krmf0sbmqpgm9s9OQ4kuKqpDVUIpzb7Ih37oW0aJlGH35cuT5ZJR87gmNc2cYN3BGIwlXLBgN+Tte24jeOelI03DLRENJCXS21aBuoMUvop4fbWHKXWERk01PSOxzZ1yNPBT+mIkcMHowslq0saUVZ861oHdOWsgts3Zvx8LT0VCyms0+BTD20dzahppGbaNi05ftKX3dkn4gZtjnzpglGIO9eIOefOjmWLUr6AvVf/kF/d69ctKR4ktCmwD+tFI7pDDIezuOh1YzKvlm2XJ3Hi2tbVFTH++rbl8LoGi52+SWSWicO8OY5aeLv8CNFxbgbIM9rhg5Rp6ag6tTe+ek47QUBaFesCOc2/+xEdeO6I2xA7qhpqE94mX38VpD/TDxo6W1DUZ+FkXL3YuLmDobgbApjkBwG0Ysq+Dq1N45aaiuDdSqOV2v/wa0csdxrNwRXjwjGNfPbhnn8cz75agxYGC88PGBDm2enFDtbD53u6I5GHvRc+0FU76GLPfsdPTOCSj3yJh2sxhR7ussCodktHl1U0UoG6QeIm/cgEcXMXU2n3u0/NhMOImouKSEHikefWcngECkTKovCbmZKeidE0jBa5U7xYjP/WaNnOKMs2CfO9PpsKqwcizcOP9j5KRHv0xOSP71qpom9MpJAxGht1Qv1SriHefOxAdPRst0NthwN4YTjteWw2ewdm90F0dpWSCn9/GaxpDFnpORjLRk6y4xjpbxJp70uTOM1wgo94CvnYhCit4KeELVm3jS584wWjjAcDdMVU0TesncMb2y0yzrW6/lPnZAbvSNGMfgxALZMdPZomUYb1Pf5Edtkz/MWrfScpcr98xU9apTVlWkYuKDXfnZOFomrrjRFmX0UlXbHuMepE9XC5V7q1y5q0/y+lv5PHMT7JZhGIcjX50aZNrIfMv6l+eb0bLcW3TWno0kK42D5xIBT6gynQ6lPBwdt4mDIDppV+7tlvu4gd1sGUtLuStZ7u8pLJ6JhNO3JwaOc2eYCArnlSZahDCULHe76KJhZUfmGG9rE7j9Hxuj9pniY1svEbBbhmEczvGaJmSm+mxxb0Q+xWi6ZSKUe32zvrJ7yT423RMBu2UYxuEEFzDZEdoWWR+2X7dM1W0jS7/VN+lb2ZpsJHE9Y5gbRvdVbHfVClUi6kJEm4hohh39M0ys6PHnG6VKKoxtB5FVpvKyUlW3jfS56y0czjVT7SVfJXIqoZWYiOgFIqoiom0R7VOIaDcRlRPRPNlHPwewxEpBvYCTJv8Y66mqbUSeTco9Mm99ty7qyj0yUVm9TuWezMrdVtSe6BLtc18IYIq8gYh8AOYDmApgBIDZRDSCiK4BsANA9Ol5hvEQtY1+dM1IsaXvSMu9u4Zy95tU7rPHDzAuGKMbtflqu7Kf6pr5EUKsIaLCiObxAMqFEPsBgIgWA7gRQBaALggo/AYiWiaE6BB4S0RzAcwFgAED+KRi3E9dkx9ZOjJImuFMQ3PYe62biFm3THGvLOOCMbpRmzh14grVAgCHZe8rABQIIR4UQtwN4GUAzyspdgAQQiwQQpQIIUry8vJiEIPxKm7zYjX525ClsXJUD4/cNFKxPdJy15q07eCW0RktU9Szi67tGHOo/WZOjJZRkih0PQohFgoh3tbsoJPllnGbsmKME6vl3kPF3SL3uffrlqHZR2QoZJ3OaJnCnl1Uby5M7PhcZLlXAOgve98PwFEjHXS+3DKM14k1xl0t1vzMuXa3jBDa4XORRZj1+twBINemOQNGXYk7MSvkBgCDiaiIiFIBzALwlpEOOpvlzhjEhY862TFa7mqP6JFuGSMYUe5cv90+1EIeE+qWIaJFAD4BMJSIKohojhDCD+AuAMsB7ASwRAix3cjgbLkzWthZZs+usNSstOiWr1ZedrWFRK9uqgitShVCGFLCeidUA7B2tws1JW7XDVVvtMxslfZlAJaZHZyIZgKYWVxcbLYLhnEUenzu//fBPtXPtBYS5Wak4Fxzq+FbnhHLnUPd7SPex5bzuTOMhWSldcz58tTsMWHvl28/prq/lnIPLlwy+tShN/0AYJ//l7HP/aI6XlxHi4B97owWblzRq+SWGd4nO/S68mwDdlTWqO6vqdwzA8q9zeCBqTXiczfUs7tYPHdiQseP932TLfc44kZlxRhDyS0jt4ZX7azS3F/TLZMZuHHY6pbxcO6wEpty6+tF7bd1VeIwhumMEAGZCvVL5Rf1+7u0lbtWfpeQchfGlqwbUe5ezgyZnOB89fGeUGW3DMNYRFZqsmK4m7zp4/ITSNHIm65lubfHrwtDFraRaJmSwsRat3bxqAMWZ9mV/VF1vLiOFkFnc8sw3katOpLcYmvyt+GS83qq9qGl3OsaA0paCGMWtrFoGW963YNPPYlEvkJ11kXt6z+duIiJMYidcduMMez4JdTCIOXXbpdUH8YXdVftQ1O5S0pawFhYnZFoGa/ihPku+W8WDyue3TIMYxFqqQfk1vDlQ/KQquH7Vcs/AgBfH9cPgPFCI/JEYtk2lABk9CE/D7R+Z8vGs30EDdgtw3gJtdQD8ov66uG9NftQc4vcd/3QkD9cS7Vr+fMB7/rUo+EAwz3MWpc/oXlyQpVhtHDCo7QR1C339tdXDs3TdM+pJQ5LIgopfiGAwb2zFbf735nna8rotkVKVlWHcsK5JP8qYcrdpmBIVu4MYxFqyl2uUHtmaZfhU/O5J1F7PLQQQrVYR2oUy91NPPtf4/A/M0ZY0pcT5rvC3DLsc2cY96AeLRN730lEoZuElppKTbb2kh7SO3HVmaZc0AcZCusG3IrcLSNX9J50y7DPnfESenzu0VDbkmSWu5Z2T/VpK0OjeuTGCwuQnpI4NWGV4nOaWyYexcjZLRNHnHCCuQm3Ha6MVGXFGqnczZwHActd2l9ju2gTqkbxJZHtse+XFavH/Vs1thPOJfl3SQrzuds0nk39MkzMGE2QZQSj4YR6UEo9AMDY1auybcDnHpxQVZfdareM3Nevh2hzCkp877JCAEB+1/QOn00d2cdwf0rY8XsbRS0U0pNuGYbxEmqWu5GLVy1yIkmmZWPxuRtVJL6kJEMRNi/fMcHYADKG9ekYAZQZY8FxJxEeLROH8ewfggniAOPBVbjteKXrnPwz87VI7pbR6MDqsDqfQctdKYqn/HdTNfdxyu/8/n9fYWv/4dEy7arXk6GQnS1axgnhWO7CXcdLTbkb8spoumUCaJ1HrZHVsQHNFbHRSEoiQ9a+UxS1GXKlfPl24QtbxNTe7km3TKzRMj262PtjMInFDkVxqr4ZL68/hJv/ut7yvtXC9ozENKt958hFTGr42zrWZ+0SVh3KmCZJIop7NkM70HMu2f0t5Uo8HgnavOPQYjyHVbr9bEMLlm8/hrfLKvFx+Qm0tglbsgSq+dyt8BsnETq4Zb4+rh8+2XcSR840hLZTsty7pCXj9LkWU+P6kuxyGrQT/D5Wr55NT0lCY0v4zW7e1GF49J1ditvbrXA71SImhtEiFsu9rsmPNzcfwe1/34CLHn4P979Whv3Vdbhj0iC8/ePLsPC28dYJKqF3wY3a97qsuKeqy4WI2qNlpG0e/8Zo/HhyeHF5v4JyV1s5KycvWznKxUexh0JarbS/VdI/+kYAcjPan+yDx+yG0X1VtyebtaFcoSeH5ZaxR9G72nJ3WZoMV/sjE4GZUMi3y47i7S2VWL27Ck3+NuR3Tcd3Lh6IGaP7YnS/rqELadsR6+d59E6oqvH8d0pQ26hsYSepTKhGHiE1yz0aGx68Rnlcmc+9Z1YaTtQ1Re3Lbi4q6o5XNh4Ovb+0uAc+Lj/ZYbvbJxXh4dKdABzolomD5e5q5c54GzPK/a6XNyMvOw2zxw/AjFH5GDugm+KFpJagKxbU3DJG9q9RVe7tr+VHJVKZK1nucuVu1CAKWJiBnYb2ycKJ8sQrdzkXFOTgpdsnonBeaYfP5lxWhO1Ha/DG5iO6+oqrW4Z97kxnxoxyX3THRIwv6h7Vp2nHxZVuwQIirQnVdsu9faPIY+Rv7TihmpVm/qZjNFomnkwe1gt/+fZY1c/l7g49p5Ldyl1+TrqyWAcRDSeiZ4noNSK60+r+5bCbw5scOFGP+avLcc8rWwzve/F5PXRNVtkxoRWr5Q6ohzmSfIWqrF2X5R7DhG4StT812DW1GpTYaO+5GSkdXGEPThse9r49fDQ6dt/E1HLL2DWsrl+diF4AMANAlRDiAln7FAB/BuAD8FchxKNCiJ0AfkBESQCet0FmxoN8ebIepVsrUVpWie1HawAoRztoMaB7pu5tjdQg1Ut6st5FTOqqRp/l3t4eqdyj+dyNKhKfbCJXS/kRBeSKR61Sre9wUWQJQ4WnHdV+bVbupBItY9e4em/pCwE8A+AfwQYi8gGYD+BaABUANhDRW0KIHUR0A4B50j4Mo8ihk+cCCn3rUWw7ElDoYwfk4n9mjMDUC/rgZF0zZj7zke7+Mg1Yzj4bfO5GH7Vz0pNR06iveHVGik9RqUUq88nDeiEz1Ydzze11U+XRMl8f1w8rdhzXLaNet8z3Li3SzL2+8ZfXYMOBU7jzpc91j22E1OQkNPvbcGH/XDw1ewwmSEo+8mkjWOc2LzsN1bXh8we2u2XkicOcUmZPCLEGwKmI5vEAyoUQ+4UQzQAWA7hR2v4tIcQlAL6t1icRzSWijUS0sbq62pz0LoPdSMDhU+fw3If7cMMzH+HyP67GH97dheSkJPxy+qXBivwAABrZSURBVHB8PG8yXv/hpZhzWRH65mbYmjgsHilX1Qh+rZsnDOz4mco+mak+RYUQ6YbpnZOOHb+Zgl6y0Ea5u+i68/tgxT2X65ZVLRQy8vBpHU1CINKmp0K45YvfvUi3LFp8eN+V+PedlwAIhDv2zglPQhY8SjnpKfjkgcn4zQ0dK1ZZrXCz05Px7H+NC70PX6HqELeMCgUADsveVwCYQERXArgJQBqAZWo7CyEWAFgAACUlJaauYqdO9KjRWdMPVJw+h3e2HsPbWyux5fAZAMDofl3xi2nDMPWCfPRXcacYVe5GNo/HIhIzqFVYSk/1KZ7vbQpuGCD8+6WpTPQW98pCeVWdpjzyfuS/RxKRrt9n98NTQk80Sof8qmG9sNLAk4Qa+V0zkN81o0O70jHL75qBsqTwUNg/z7rQciV705iCMDeVXBafw+PclSQSQogPAHygqwOimQBmFhcXR91WCbaEncvRMw1YtrUSpVsrsflQQKGPLOiKeVOHYfpIdYUuR0VvWUIiLXctstKSceeV5+H/PtgX1p6R4lNUAq0qF4HcCo1U7ka+udwtc+vFhaF48khR1PRTmmweQk2JBf3hthprEYcp8rAV5Gaojv/dSwqxcN1BU0PKuwyz3B0eClkBQL5UrB+Ao0Y6EEIsBbC0pKTkjhjkcA32L+ROLJVnG7Bs6zGUlh3F55JCv6AgBz+fElDoA3ron/AMYJ92d4LlrnZ9pyjIprb6VWkCFQiP40+LmOjVOqpypfe1sf1w8aAeoRvF0D7ZKMjNwJEzDdK5bOz3Ke4V/5J9xpK2KW9tVg8LoV5az0kTqkpsADCYiIoAHAEwC8DNRjqI1XJ3G150yxyvaQxY6GWV2PjlaQDAiPwc3Hf9UEwfmY/Cnl1M9z2kd8f83lbh5NqcSta43G9+5dC89m3V3DJyy12lTJ6WTklPScL/++bowHbShm2i/XWkjHpcCznp9kfSqBHt2tOMBDJplKUlJ4XtKX+CikclJr2hkIsAXAmgJxFVAPiVEOJvRHQXgOUIhEK+IITYbmTwWC13t/ncvUJVTSPe2XYMpWWV2PDlKQgRKLTws+uGYNrIfAzKs8ZCyzaoDIzcPJNNpsE1Gp6pF/m5PPWCfMxfHeGWkZT7unmT0V2WDVXVLaPD5x6kUPZEpXQMQ7HiQoQsTrWbSjRW/+xKXPX4ByqfGrugjcSudzxM+uU3q2fuuXYIdh2rDb2X36Dj4RbUpdyFELNV2pdBY9I0Gp3Ocnex4V5V24jl2wKZFT87GFDoQ3tn455rAgo9EY/ckXxtbD/bx0jxBZT75GG98P6uqpj6IpXXFxR0xcFHp4ctqQ8+afTNDZ80VJtQTU5Sd8t0k/KWXzEkD6U/maSovOTWatAqF+gYUTJ7fH8s+uywbrVcpPAkd0lxT5zfNwc/u34I7rl2MKY/pR3+akVlKyOY7aFLWnKYrPKnxbDj6MAJ1ZjpbD53t1Fd24R3tx/DsrJKrD9wEm0CGNwrC3dfPQTTR/VBcS/73CZm6LCAxQaCl+GfvnkhRv9mhWX9KoXhBRUnELipKKGQbQBAwPoPWo2RpffystPwyQOTkZeV1uEJRskACUbwEIAZo/Lx9PvlyE5LRm2TH1Y4FbLSklH6k0kA1G9WsRLZa8fvaY+Clf+u8oVzTg+FjJnOFi3jBnFP1gUUemlZJT7dH1Do5+V1wY8nD8b0Ufm2+sGN8tDMEXho6Y6EjG31/ImS8aYnVW+rQnEOAPjx5GI8v3Y/6pr8im4ZpbBBQJYKQCbPc7eMw9ItR1HUswvuuWYIbr9sEKY/vRa1TX5No9PM05zVRqyaW0bt17usuCc+Kj9hmUzycNHwYh3m+9SLqy13NyhLN3CqvhnLJYX+yf6TaG0TGNSzC+66qhjTR/XFkN5ZtsXixsLE83qEViYC8bnZh1wUEWM9PXuM7j6UlsIruQ/0HHMtn3t6ik9S7sYnj+Uj985Jx+2TBkkyAV0zU0LfX0tJLZOscUPjanzn9+69HHVNrdgXJTY/vL/Af70343/dPqFDhslxA7vh+bUHdI8p51Rdc9RtnBgtw7iY0/XNWLEj4ENfty+g0At7ZOLOK87D9FH5GNYn2xEKPTczBWdUqgg5KbTUSOqDIGGH1+RX0ZrYbJF8NmkpSVg8dyIaW1pVtw0SVNbds7RLWHa4QcnkD6ZViHQHBenfPQOHTzUofqZF0A1oRLnrPbBap/qUC/Ixun9uaAGeESYN6ak4RjwuLVe7ZZxzaetDT/IiOzlzrhkrth9H6dZAuTl/m8DAHpn4/uWDMH1UPkbk5zhCocvRivTITPVFPL6541ku6Noa1icn1Gb2MV2Xck9OwsRBPXT1l5majMe+NgqXDu6puV3kqPIb7cp7r0DFaXXlPWlwHl5ef0iXPEDALfTZgcjsJ8au/w5uGYOnSi+VSlXRkD81qaU3sMtIcbVbxm0kQvWcPdeCFTuOoXRrJT7aG1DoA7pn4o7LB2H6yHyc39d5Cl0Plxb3QP/umXFfO2D2UP1zzviQD/268/tg5T2XY3DvbPx40WYAwEWFHSeD9QylNqEKIOSuMuqW+eZF0cvYheqeKkjZOye9Q24XJR7+ygVRtwGA68/vg+vP7xN6H1ygpfZkIKfdLRNOItacqP2e7JZhdHO2oQXv7QhY6Gv3VqOlVaBftwzMmVSEGSP74oIC9yh0tdS8Ywd069BWnBe/yd7o0RfhTBqcF/Z+cMTEtDzBlBEmFHXHvz+vUPwsmFRMjxI0SkG3DByraQxF8Rg5nTKlkMBo8fdqTB8ZiAT6weXnRd22S6q+seJxNcivOT03v1hxtVvGbdh5AtU2tuC9ncdRWlaJNXtOoLm1DQW5Gbjt0iJMH5mPUbL6oW5i2sg+ipNZiiF7NuQSLxnYDQ0traEc8/IFPVYwrE82dh2rVa5zquPn+kZJP1wxNA8Tfr9KdRuzSlSL579Tgk/3n8SBE/WG9733uiHIyUjBV8cUmBo72ZeEn08Zpmvbe64dguz0FNwUMVbkz6cnMilW5K43+Q3Xk6GQnS1axmp565r8WLXzON4uq8SHe6rR7G9DX6kg9PRR+biwf64rFbqcaCtJ7Z7GyEj14Zmbx2LiIwHlaXX+kdKfTIoptTERRbUC7VDu3bukYtrIfMxfXR6Qw8C+manJ+MnVgy2XychYwQVFN40pwMzRfTs8SWnxy+nDQ4W3jSA/d8IXiRnuSheudsu4W22ZI6jQS8sq8YGk0PvkpOOWiQMxbWQ+xvTPjUt9xkQwc3RfLN0SyE0X9Jl2zUjByfpmDLZphSwRoU/XduU5ul9XrN5d3cHVYVY/+5IIymU4gDmXFuG5D/eb61iG2VQLekh0kIBZrh7eC7+aOQLfLOmv/NSkwQUFXcPeK8XGR0N+iY7ql2toX724W7m7TIeZvQ7qm/x4f1cVSssqsXp3FZr8beidk4ZvTxiAGaPyMaZ/N88qdDWCx3Jon2ys23cSD04frr2DSSKP6jM3j8XeqroOeW+G9rHe3x+rm8mXRKZzwBjFbdciEeG2S4ss6UspNj4a8siZ4fk5Gluah33uDuVcsx+rd1WjdOtRvL+rCo0tbcjLTsPs8QMwfVQ+xg3ofApdK9lSZgxFoLWIVFpd0pJxYf+Olpae/PTxZsU9l2NnZY2tY7jUcDeM3iyi0Vw2SRTIrhmPMnvu9rm77sTSFrihuRUf7K7C21sr8f7OKjS0tKJnVhq+WdIf00fmo6SwuyPykMcT+be999oheGPzEQDtR/KP3xiNFz46gHEDO0bPWD1+JB/87EpcqZrhMHaC57fZDILn5WXhPIsydEbDSQvK7ODXN5yP1OQk3HVVMarrmlS3u33SIE3lnpbsQ0NLa1x8yq52y3iBxhZJoZdV4v1dVTjX3IqeWan42rgCTB/ZF+OLOp9CVyMnPSVkGQUnCQtyMzQLM8eK1oR0LLnq9ZCWnIRrR/TGLRM71lp1Cq6zr0zSrUsqHv9GIL/9gO6ZuHxIHtbsqQ5LvxzkzR9dGnZDfv47JXhjcyBcNS0lCQ0trXFxl7laudsRBRAPGlta8eGeapSWVWLVzuOob25Fjy6p+MqYAswYmY/xRd1tnQRzM7dcPBA1jX784IroMc5WkMjbKhHh+e+UJFAC/bjN566XnlkdV6YmJRHuvXYI1uypRp7C55Fuu2tH9Ma1I3oDAH4xdTju/3cZuqTZXyzG1cr9gWnDQyv83MAHu6vxz0++xHs7q1DX5Ee3zBTccGEBZozKxwRW6LpIS/bh3muHJFoMRsJ9rlH9fPrA1chUUcI1DYF8R3kG0xJ886L+oRXAX7mwL7LS7VPBrlbuOSqV4p3K0++XIzczBTNG5WP6qHxMHNRDNU83E+Ciou5ARLHoeBJ0y2x48JqEyeAGvGi4y0NgIzl9LpDt0ahyl/PkLP2ZRM3g6mgZt51Qf551IaaNzGeFboCrhvaKyzhqscr3XT8UgPZFfM3w+MjoRLxYF1gP143og6+P64cHpupbKZsI3B0tY7E8dnP54DxW7LFg491cLaw0Wvz61oeuQ7qDi23HDa863VXISPWFJlidiqvdMgxjFWZXWhot4s0w8cLVZmTnshUYO/HyxKCd8HFzLq5W7m6DrwNzZAcjCmw8gBmySkpXD+u8PnSzsKHlPFi5M44nHoqjh2wxSjfp9bdKohet6OywweJcWLnbgNnl4kzikLsX+NfTz1fHFCA1Ocl0bnbGPmyZUCWirwCYDqAXgPlCiBV2jMMwViEP6WsvzcZ2aTSKenbBnoenJloMRgHdljsRvUBEVUS0LaJ9ChHtJqJyIpoHAEKIN4UQdwD4LoBvWSoxw9hAuOXOtjvjfoy4ZRYCmCJvICIfgPkApgIYAWA2EcmzOP1S+pyBewsbdAbkv0xvjZWJDOMWdCt3IcQaAKcimscDKBdC7BdCNANYDOBGCvAHAO8IIT63TtxwnLpuglW4+5Dfdwt7BHKzpyXz4iTGvcTqcy8AcFj2vgLABAA/BnANgK5EVCyEeDZyRyKaC2AuAAwYMCBGMdyB2+uZehm5f/2G0X2xv7oe379iUAIlYpjYiFW5K2krIYR4CsBTWjsKIRYQUSWAmampqePMDO7UZd/sfnEfXWVJ6JJ9SfiZlFOGYfTw7t2T0MWmamBmiTUUsgKAPBi4H4CjencWQiwVQszt2rVr9I0VKLGp+o4VbH3oOkwc1D3RYjA6uf965yaAYpzPsD45jiu1GKty3wBgMBEVEVEqgFkA3tK7MxHNJKIFZ8+eNTW4U90cPbLSkJ2ewnlHXIR8hSrDeAEjoZCLAHwCYCgRVRDRHCGEH8BdAJYD2AlgiRBiu94+Y7Xcncrrd14CoGPeDXbXmCNUysyZ93KGcSS6nURCiNkq7csALDMzeKz53K3ke5cW4YWPD1jSl9Mez9zOS3dMxJo91WF+cTsY2CMTw6Kk+GUYt5DQ9ANOstytWI1468UDMbhXe7X53jnmq7Qw7RTkZmD2eOsjqiLzcX9431V47hZ31CxlmGgkVLnH6nO3Ep8F/vuvj+uPlfdeEXo/OqJQLuMsvj6uX6JFYBzI8PycRItgCa6uxBQLT37rQtz9yheh91bMzUb2wS5i5/PyHRPgb+W5EKadV74/EcfONiZajJhxVmBmHMnNTMGfvjUa9U2tuGpYLyy0wN9+fl9v3PE7E5ec1zPRIjAOIyc9BTkeiHRzdYHsWPnqmPbH8lgDWS4r7tkhNDP4fkD3TAzPz0a3zFSlXRmGYSyn002o/uXbYwEEFh2o8ePJ1txsgmndxxd1x3O3lKgWYWYYhrEaT7plJhR1x/oDkTnOAkwbmY+Dj07X3N+qCZUZo/pi86EzuOfaIZb0xzAMoxdPRsu88v2LLe3PLKnJSfjtVy5oX4TDMDax/hdXY9lPJiVaDMZBdDq3jBo+H7tMGPfSOycdI3hCn5HhmRqqj940Mqb977qq3c/Oap5hGLfjGeU+a/wA5HdNx60XDzS1f2SSr55ZvLqUYRJBycBuuHpYr0SL4Xo8NaH6yQNXh16/eNtFuO3FDab6EQCW/eQyjP/9KoskYxhGL69JifeY2HD9hOo7P52EdfMmd2i/amgvHHx0Ou67fijGFxnPq94rJ3odzWF9svHPOeMN980wDGM3rp9QHZ6fg765Gaqf/+iqYiyxKXqmR1Yqzu8bkH3OpCJbxmAYhjGDp9wyVqF3QjXVl4TuXVKjxs0zDMPEG89MqDIMwzDtdBrlPn1kvun86uvmTeYiDgzDuIpO45aZL+WUMUPf3Az0756JXcdqLZSIYRjGPlwfLRNv5lxWhD98LbBgyqkFuhmGYVwfLWMHWjp7fFF3XuDEMIzj6TQ+d6uINe87wzBMPGDlrhN2wDAM4yZYuTMMw3iQThMtY4ZHbhqJrLTIQyQQtOMF+2gYhnEorNwVCSjv2eMHtLfI/DIcJMMwjNOx3C1DRIOI6G9E9JrVfTMMwzD60KXciegFIqoiom0R7VOIaDcRlRPRPAAQQuwXQsyxQ9hEEkwQ1ltHtkiGYZhEo9dyXwhgiryBiHwA5gOYCmAEgNlENMJS6RzEj64qxlt3XYoxA7olWhSGYZio6FLuQog1AE5FNI8HUC5Z6s0AFgO4Ue/ARDSXiDYS0cbq6mrdAicKXxJhVL/cRIvBMAyji1h87gUADsveVwAoIKIeRPQsgDFE9IDazkKIBUKIEiFESV5eXgxiWA9PmDIM43ZiiZZRUoFCCHESwA90dUA0E8DM4uLiqNsyDMMw+onFcq8A0F/2vh+Ao7GJ4w6G9ckBANw0tl+CJWEYhlEmFuW+AcBgIioiolQAswC8ZaQDpyYOi0bf3AwcfHQ6Zo7um2hRGIZhFNEbCrkIwCcAhhJRBRHNEUL4AdwFYDmAnQCWCCG2GxncjSl/GYZh3IAun7sQYrZK+zIAy8wOLoRYCmBpSUnJHWb7sAOeT2UYxu1w4jCGYRgPwpWYGIZhPAhXYmIYhvEgbLkrMKJvTqJFYBiGiQm23BXo1y0z0SIwDMPEBE+oMgzDeBBW7gzDMB6Efe4MwzAehH3uDMMwHoTdMgzDMB6ElTvDMIwHiSWfe8w4LZ/7C98tQbO/LdFiMAzDxAwJIRItA0pKSsTGjRsTLQbDMIyrIKJNQogSpc/YLcMwDONBWLkzDMN4EFbuDMMwHoSVO8MwjAfhFaoMwzAehFeoMgzDeBB2yzAMw3gQVu4MwzAexBGLmIioGsCXJnfvCeCEheLEE5Y9/rhVboBlTxROln2gECJP6QNHKPdYIKKNaiu0nA7LHn/cKjfAsicKt8rObhmGYRgPwsqdYRjGg3hBuS9ItAAxwLLHH7fKDbDsicKVsrve584wDMN0xAuWO8MwDBMBK3eGYRgP4mrlTkRTiGg3EZUT0bwEyfACEVUR0TZZW3ciWklEe6X/3aR2IqKnJHnLiGisbJ9bpe33EtGtsvZxRLRV2ucpIiILZe9PRKuJaCcRbSein7pFfiJKJ6LPiGiLJPuvpfYiIlovyfEKEaVK7WnS+3Lp80JZXw9I7buJ6HpZu23nFxH5iGgzEb3tMrkPSr/nF0S0UWpz/Pki9Z1LRK8R0S7pnL/YLbKbQgjhyj8APgD7AAwCkApgC4ARCZDjcgBjAWyTtT0GYJ70eh6AP0ivpwF4BwABmAhgvdTeHcB+6X836XU36bPPAFws7fMOgKkWyp4PYKz0OhvAHgAj3CC/1F+W9DoFwHpJpiUAZkntzwK4U3r9QwDPSq9nAXhFej1COnfSABRJ55TP7vMLwL0AXgbwtvTeLXIfBNAzos3x54vU998B3C69TgWQ6xbZTX3fRA4e4w91MYDlsvcPAHggQbIUIly57waQL73OB7Bbev0cgNmR2wGYDeA5WftzUls+gF2y9rDtbPge/wFwrdvkB5AJ4HMAExBYSZgceY4AWA7gYul1srQdRZ43we3sPL8A9AOwCsBkAG9Lcjhebqm/g+io3B1/vgDIAXAAUhCJm2Q3++dmt0wBgMOy9xVSmxPoLYSoBADpfy+pXU1mrfYKhXbLkR73xyBgAbtCfsm18QWAKgArEbBYzwgh/ArjhWSUPj8LoIeJ72QFTwK4H0CwGnsPl8gNAALACiLaRERzpTY3nC+DAFQDeFFyh/2ViLq4RHZTuFm5K/mznB7XqSaz0XZLIaIsAP8GcLcQokZrUxV5EiK/EKJVCHEhApbweADDNcZzhOxENANAlRBik7xZYyxHyC3jUiHEWABTAfyIiC7X2NZJsicj4D79PyHEGAD1CLhh1HCS7KZws3KvANBf9r4fgKMJkiWS40SUDwDS/yqpXU1mrfZ+Cu2WQUQpCCj2l4QQr7tNfgAQQpwB8AECvtFcIkpWGC8ko/R5VwCnoshux/l1KYAbiOgggMUIuGaedIHcAAAhxFHpfxWANxC4qbrhfKkAUCGEWC+9fw0BZe8G2c2RSJ9QjD60ZAQmM4rQPnF0foJkKUS4z/2PCJ+keUx6PR3hkzSfSe3dEfAHdpP+DgDoLn22Qdo2OEkzzUK5CcA/ADwZ0e54+QHkAciVXmcAWAtgBoBXET4x+UPp9Y8QPjG5RHp9PsInJvcjMClp+/kF4Eq0T6g6Xm4AXQBky16vAzDFDeeL1PdaAEOl1w9JcrtCdlPfN5GDW/BjTUMgwmMfgAcTJMMiAJUAWhC4e89BwCe6CsBe6X/wxycA8yV5twIokfXzPQDl0t9tsvYSANukfZ5BxIRQjLJfhsCjYxmAL6S/aW6QH8AoAJsl2bcB+F+pfRACUQvlCCjMNKk9XXpfLn0+SNbXg5J8uyGLcLD7/EK4cne83JKMW6S/7cG+3XC+SH1fCGCjdM68iYBydoXsZv44/QDDMIwHcbPPnWEYhlGBlTvDMIwHYeXOMAzjQVi5MwzDeBBW7gzDMB6ElTvDREBE61TaFxLR1+MtD8OYgZU7w0QghLgk0TIwTKwkR9+EYToXRFQnhMiS8nE/jUCKgANQzh/CMI6ELXeGUeerAIYCGAngDgBs0TOugZU7w6hzOYBFIpB98iiA9xMtEMPohZU7w2jD+TkYV8LKnWHUWQNgllQUJB/AVYkWiGH0whOqDKPOGwhMpm5FIMvih4kVh2H0w1khGYZhPAi7ZRiGYTwIK3eGYRgPwsqdYRjGg7ByZxiG8SCs3BmGYTwIK3eGYRgPwsqdYRjGg/x/yWzb8USO7AcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "movie.groupby(\"id\")['rtAudienceNumRatings'].mean().plot(logy=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "With Quantile at 75%:  14738.5\n",
      "The average:  75169.56335710367\n",
      "The standard deviation:  133036.72772066196\n",
      "\n",
      "With Quantile at 90%:  52006.0\n",
      "The average:  145987.99315068492\n",
      "The standard deviation:  188954.1347706887\n"
     ]
    }
   ],
   "source": [
    "popfilter1=movie[movie['rtAudienceNumRatings']>=movie['rtAudienceNumRatings'].quantile(0.75)]\n",
    "mean_pop=popfilter1['rtAudienceNumRatings'].mean()\n",
    "std_pop=popfilter1['rtAudienceNumRatings'].std()\n",
    "print(\"With Quantile at 75%: \",movie['rtAudienceNumRatings'].quantile(0.75))\n",
    "print(\"The average: \", mean_pop)\n",
    "print(\"The standard deviation: \", std_pop)\n",
    "popfilter2=movie[movie['rtAudienceNumRatings']>=movie['rtAudienceNumRatings'].quantile(0.90)]\n",
    "mean_pop=popfilter2['rtAudienceNumRatings'].mean()\n",
    "std_pop=popfilter2['rtAudienceNumRatings'].std()\n",
    "print(\"\\nWith Quantile at 90%: \",movie['rtAudienceNumRatings'].quantile(0.90))\n",
    "print(\"The average: \", mean_pop)\n",
    "print(\"The standard deviation: \", std_pop)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observations:** <br>\n",
    "**These observations give us insight that if we filter based on popularity, we will have less but more relevant data to filter through**<br>\n",
    "1. The quantile range tells us: specific amount required to make the cut\n",
    "2. The insight to do this comes from the graph right this cell block\n",
    "3. We note that the popularity is still in need of standardization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>imdbID</th>\n",
       "      <th>spanishTitle</th>\n",
       "      <th>imdbPictureURL</th>\n",
       "      <th>year</th>\n",
       "      <th>rtID</th>\n",
       "      <th>rtAllCriticsRating</th>\n",
       "      <th>rtAllCriticsNumReviews</th>\n",
       "      <th>rtAllCriticsNumFresh</th>\n",
       "      <th>...</th>\n",
       "      <th>rtAllCriticsScore</th>\n",
       "      <th>rtTopCriticsRating</th>\n",
       "      <th>rtTopCriticsNumReviews</th>\n",
       "      <th>rtTopCriticsNumFresh</th>\n",
       "      <th>rtTopCriticsNumRotten</th>\n",
       "      <th>rtTopCriticsScore</th>\n",
       "      <th>rtAudienceRating</th>\n",
       "      <th>rtAudienceNumRatings</th>\n",
       "      <th>rtAudienceScore</th>\n",
       "      <th>rtPictureURL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy story</td>\n",
       "      <td>114709</td>\n",
       "      <td>Toy story (juguetes)</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTMwNDU0...</td>\n",
       "      <td>1995</td>\n",
       "      <td>toy_story</td>\n",
       "      <td>9</td>\n",
       "      <td>73</td>\n",
       "      <td>73</td>\n",
       "      <td>...</td>\n",
       "      <td>100</td>\n",
       "      <td>8.5</td>\n",
       "      <td>17</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>3.7</td>\n",
       "      <td>102338</td>\n",
       "      <td>81</td>\n",
       "      <td>http://content7.flixster.com/movie/10/93/63/10...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>113497</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMzM5NjE1...</td>\n",
       "      <td>1995</td>\n",
       "      <td>1068044-jumanji</td>\n",
       "      <td>5.6</td>\n",
       "      <td>28</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>46</td>\n",
       "      <td>5.8</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>40</td>\n",
       "      <td>3.2</td>\n",
       "      <td>44587</td>\n",
       "      <td>61</td>\n",
       "      <td>http://content8.flixster.com/movie/56/79/73/56...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Heat</td>\n",
       "      <td>113277</td>\n",
       "      <td>Heat</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTM1NDc4...</td>\n",
       "      <td>1995</td>\n",
       "      <td>1068182-heat</td>\n",
       "      <td>7.7</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "      <td>...</td>\n",
       "      <td>86</td>\n",
       "      <td>7.2</td>\n",
       "      <td>17</td>\n",
       "      <td>14</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>3.9</td>\n",
       "      <td>42785</td>\n",
       "      <td>92</td>\n",
       "      <td>http://content9.flixster.com/movie/26/80/26809...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>GoldenEye</td>\n",
       "      <td>113189</td>\n",
       "      <td>GoldenEye</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BNTE1OTEx...</td>\n",
       "      <td>1995</td>\n",
       "      <td>goldeneye</td>\n",
       "      <td>6.8</td>\n",
       "      <td>41</td>\n",
       "      <td>33</td>\n",
       "      <td>...</td>\n",
       "      <td>80</td>\n",
       "      <td>6.2</td>\n",
       "      <td>11</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>63</td>\n",
       "      <td>3.4</td>\n",
       "      <td>28260</td>\n",
       "      <td>78</td>\n",
       "      <td>http://content9.flixster.com/movie/26/66/26669...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>Casino</td>\n",
       "      <td>112641</td>\n",
       "      <td>Casino, de Scorsese</td>\n",
       "      <td>http://ia.media-imdb.com/images/M/MV5BMTQ0NDk5...</td>\n",
       "      <td>1995</td>\n",
       "      <td>1067987-casino</td>\n",
       "      <td>7.2</td>\n",
       "      <td>58</td>\n",
       "      <td>47</td>\n",
       "      <td>...</td>\n",
       "      <td>81</td>\n",
       "      <td>6.3</td>\n",
       "      <td>16</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>3.9</td>\n",
       "      <td>66463</td>\n",
       "      <td>91</td>\n",
       "      <td>http://content8.flixster.com/movie/10/87/61/10...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    id      title  imdbID          spanishTitle  \\\n",
       "0    1  Toy story  114709  Toy story (juguetes)   \n",
       "1    2    Jumanji  113497               Jumanji   \n",
       "5    6       Heat  113277                  Heat   \n",
       "9   10  GoldenEye  113189             GoldenEye   \n",
       "15  16     Casino  112641   Casino, de Scorsese   \n",
       "\n",
       "                                       imdbPictureURL  year             rtID  \\\n",
       "0   http://ia.media-imdb.com/images/M/MV5BMTMwNDU0...  1995        toy_story   \n",
       "1   http://ia.media-imdb.com/images/M/MV5BMzM5NjE1...  1995  1068044-jumanji   \n",
       "5   http://ia.media-imdb.com/images/M/MV5BMTM1NDc4...  1995     1068182-heat   \n",
       "9   http://ia.media-imdb.com/images/M/MV5BNTE1OTEx...  1995        goldeneye   \n",
       "15  http://ia.media-imdb.com/images/M/MV5BMTQ0NDk5...  1995   1067987-casino   \n",
       "\n",
       "   rtAllCriticsRating rtAllCriticsNumReviews rtAllCriticsNumFresh  ...  \\\n",
       "0                   9                     73                   73  ...   \n",
       "1                 5.6                     28                   13  ...   \n",
       "5                 7.7                     58                   50  ...   \n",
       "9                 6.8                     41                   33  ...   \n",
       "15                7.2                     58                   47  ...   \n",
       "\n",
       "   rtAllCriticsScore rtTopCriticsRating rtTopCriticsNumReviews  \\\n",
       "0                100                8.5                     17   \n",
       "1                 46                5.8                      5   \n",
       "5                 86                7.2                     17   \n",
       "9                 80                6.2                     11   \n",
       "15                81                6.3                     16   \n",
       "\n",
       "   rtTopCriticsNumFresh rtTopCriticsNumRotten rtTopCriticsScore  \\\n",
       "0                    17                     0               100   \n",
       "1                     2                     3                40   \n",
       "5                    14                     3                82   \n",
       "9                     7                     4                63   \n",
       "15                   11                     5                68   \n",
       "\n",
       "   rtAudienceRating rtAudienceNumRatings  rtAudienceScore  \\\n",
       "0               3.7               102338               81   \n",
       "1               3.2                44587               61   \n",
       "5               3.9                42785               92   \n",
       "9               3.4                28260               78   \n",
       "15              3.9                66463               91   \n",
       "\n",
       "                                         rtPictureURL  \n",
       "0   http://content7.flixster.com/movie/10/93/63/10...  \n",
       "1   http://content8.flixster.com/movie/56/79/73/56...  \n",
       "5   http://content9.flixster.com/movie/26/80/26809...  \n",
       "9   http://content9.flixster.com/movie/26/66/26669...  \n",
       "15  http://content8.flixster.com/movie/10/87/61/10...  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "popfilter1.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Popularity:**\n",
    "1. Logarithmic representation was better\n",
    "2. convert to log\n",
    "3. standardize via Column Transformer\n",
    "4. Merge it with final dataframes to observe the difference<br>\n",
    "\n",
    "Note: Thought process whilst implementing this part was to utilize this as weights and add them to each row of selected columns \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>popularity</th>\n",
       "      <th>delet</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.929319</td>\n",
       "      <td>102338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.045244</td>\n",
       "      <td>44587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.093635</td>\n",
       "      <td>42785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.580120</td>\n",
       "      <td>28260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.423016</td>\n",
       "      <td>66463</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   popularity   delet\n",
       "0    0.929319  102338\n",
       "1   -0.045244   44587\n",
       "2   -0.093635   42785\n",
       "3   -0.580120   28260\n",
       "4    0.423016   66463"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ss = StandardScaler()\n",
    "popfilter=popfilter1[['rtAudienceNumRatings']].copy(deep=True)\n",
    "popfilter.columns=['popularity']\n",
    "popfilter['logarithm_base2'] = np.log2(popfilter[['popularity']])\n",
    "idfilter=popfilter1[['id','rtAudienceNumRatings']].copy(deep=True)\n",
    "idfilter.columns=['movieID','delet']\n",
    "ct = ColumnTransformer([('popuser', StandardScaler(), ['logarithm_base2'])], remainder='passthrough')\n",
    "scaled_values_popularity=ct.fit_transform(popfilter)\n",
    "df_scaled = pd.DataFrame(scaled_values_popularity)\n",
    "df_scaled.columns=['popularity','delet']\n",
    "df_scaled['delet']=df_scaled['delet'].astype(int)\n",
    "#df_scaled=df_scaled.drop(columns=['delet'])\n",
    "df_scaled.head()\n",
    "#gives us popularity standardized based on log transformation of popularity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>popularity</th>\n",
       "      <th>movieID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.929319</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.045244</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.093635</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.580120</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.423016</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   popularity  movieID\n",
       "0    0.929319        1\n",
       "1   -0.045244        2\n",
       "2   -0.093635        6\n",
       "3   -0.580120       10\n",
       "4    0.423016       16"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idfilter=idfilter.reset_index(drop=True)\n",
    "popularity = pd.concat([df_scaled, idfilter], axis=1, join='inner')\n",
    "popularity = popularity.drop('delet', axis=1)\n",
    "popularity.head()\n",
    "#gives us popularity standardized w.r.t movieID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mS1ZMfLqylJs"
   },
   "source": [
    "**We will join the scattered data into a single csv first and foremost, then begin our EDA\n",
    "But even before that, we will review all data by repeating the above code block for all .dat files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Czbv7H5Dxt3K"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>earth</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>police</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>boxing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>painter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>whale</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id    value\n",
       "0   1    earth\n",
       "1   2   police\n",
       "2   3   boxing\n",
       "3   4  painter\n",
       "4   5    whale"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tags = pd.read_csv('tags.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "tags.head()#data1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XjkuIXuDzxvB"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorID</th>\n",
       "      <th>actorName</th>\n",
       "      <th>ranking</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>annie_potts</td>\n",
       "      <td>Annie Potts</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>bill_farmer</td>\n",
       "      <td>Bill Farmer</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>don_rickles</td>\n",
       "      <td>Don Rickles</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>erik_von_detten</td>\n",
       "      <td>Erik von Detten</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>greg-berg</td>\n",
       "      <td>Greg Berg</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID          actorID        actorName  ranking\n",
       "0        1      annie_potts      Annie Potts       10\n",
       "1        1      bill_farmer      Bill Farmer       20\n",
       "2        1      don_rickles      Don Rickles        3\n",
       "3        1  erik_von_detten  Erik von Detten       13\n",
       "4        1        greg-berg        Greg Berg       17"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_actors = pd.read_csv('movie_actors.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie_actors.head()#data2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JUTbgwS_0A-y"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID country\n",
       "0        1     USA\n",
       "1        2     USA\n",
       "2        3     USA\n",
       "3        4     USA\n",
       "4        5     USA"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_countries = pd.read_csv('movie_countries.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie_countries.head()#data3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cy-tIycx0FoQ"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>directorID</th>\n",
       "      <th>directorName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>john_lasseter</td>\n",
       "      <td>John Lasseter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>joe_johnston</td>\n",
       "      <td>Joe Johnston</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>donald_petrie</td>\n",
       "      <td>Donald Petrie</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>forest_whitaker</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>charles_shyer</td>\n",
       "      <td>Charles Shyer</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID       directorID     directorName\n",
       "0        1    john_lasseter    John Lasseter\n",
       "1        2     joe_johnston     Joe Johnston\n",
       "2        3    donald_petrie    Donald Petrie\n",
       "3        4  forest_whitaker  Forest Whitaker\n",
       "4        5    charles_shyer    Charles Shyer"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_directors = pd.read_csv('movie_directors.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie_directors.head()#data4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "G_IqgSAY0L7b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>genre</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Adventure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Children</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Fantasy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID      genre\n",
       "0        1  Adventure\n",
       "1        1  Animation\n",
       "2        1   Children\n",
       "3        1     Comedy\n",
       "4        1    Fantasy"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_genres = pd.read_csv('movie_genres.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie_genres.head()#data5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CY-nEKRs0QwK"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>location1</th>\n",
       "      <th>location2</th>\n",
       "      <th>location3</th>\n",
       "      <th>location4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Canada</td>\n",
       "      <td>British Columbia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Canada</td>\n",
       "      <td>British Columbia</td>\n",
       "      <td>Delta</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>Canada</td>\n",
       "      <td>British Columbia</td>\n",
       "      <td>Delta</td>\n",
       "      <td>Tsawwassen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>Canada</td>\n",
       "      <td>British Columbia</td>\n",
       "      <td>Maple Ridge</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID location1         location2    location3   location4\n",
       "0        1       NaN               NaN          NaN         NaN\n",
       "1        2    Canada  British Columbia          NaN         NaN\n",
       "2        2    Canada  British Columbia        Delta         NaN\n",
       "3        2    Canada  British Columbia        Delta  Tsawwassen\n",
       "4        2    Canada  British Columbia  Maple Ridge         NaN"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_locations = pd.read_csv('movie_locations.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie_locations.head()#data6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aEKzjBGp0kRT"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>tagID</th>\n",
       "      <th>tagWeight</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>55</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID  tagID  tagWeight\n",
       "0        1      7          1\n",
       "1        1     13          3\n",
       "2        1     25          3\n",
       "3        1     55          3\n",
       "4        1     60          1"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_tags = pd.read_csv('movie_tags.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "movie_tags.head()#data7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vFCmZOh_0tbs"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>movieID</th>\n",
       "      <th>rating</th>\n",
       "      <th>date_day</th>\n",
       "      <th>date_month</th>\n",
       "      <th>date_year</th>\n",
       "      <th>date_hour</th>\n",
       "      <th>date_minute</th>\n",
       "      <th>date_second</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>29</td>\n",
       "      <td>10</td>\n",
       "      <td>2006</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>75</td>\n",
       "      <td>32</td>\n",
       "      <td>4.5</td>\n",
       "      <td>29</td>\n",
       "      <td>10</td>\n",
       "      <td>2006</td>\n",
       "      <td>23</td>\n",
       "      <td>23</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>75</td>\n",
       "      <td>110</td>\n",
       "      <td>4.0</td>\n",
       "      <td>29</td>\n",
       "      <td>10</td>\n",
       "      <td>2006</td>\n",
       "      <td>23</td>\n",
       "      <td>30</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75</td>\n",
       "      <td>160</td>\n",
       "      <td>2.0</td>\n",
       "      <td>29</td>\n",
       "      <td>10</td>\n",
       "      <td>2006</td>\n",
       "      <td>23</td>\n",
       "      <td>16</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>75</td>\n",
       "      <td>163</td>\n",
       "      <td>4.0</td>\n",
       "      <td>29</td>\n",
       "      <td>10</td>\n",
       "      <td>2006</td>\n",
       "      <td>23</td>\n",
       "      <td>29</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID  movieID  rating  date_day  date_month  date_year  date_hour  \\\n",
       "0      75        3     1.0        29          10       2006         23   \n",
       "1      75       32     4.5        29          10       2006         23   \n",
       "2      75      110     4.0        29          10       2006         23   \n",
       "3      75      160     2.0        29          10       2006         23   \n",
       "4      75      163     4.0        29          10       2006         23   \n",
       "\n",
       "   date_minute  date_second  \n",
       "0           17           16  \n",
       "1           23           44  \n",
       "2           30            8  \n",
       "3           16           52  \n",
       "4           29           30  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_ratedmovies = pd.read_csv('user_ratedmovies.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "user_ratedmovies.head()#data8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vHC7lrDh1OI_"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>movieID</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1162160236000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>75</td>\n",
       "      <td>32</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1162160624000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>75</td>\n",
       "      <td>110</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1162161008000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75</td>\n",
       "      <td>160</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1162160212000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>75</td>\n",
       "      <td>163</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1162160970000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID  movieID  rating      timestamp\n",
       "0      75        3     1.0  1162160236000\n",
       "1      75       32     4.5  1162160624000\n",
       "2      75      110     4.0  1162161008000\n",
       "3      75      160     2.0  1162160212000\n",
       "4      75      163     4.0  1162160970000"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_ratedmovies_timestamps = pd.read_csv('user_ratedmovies-timestamps.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "user_ratedmovies_timestamps.head()#data9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0nZp7Q1T1WBw"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>movieID</th>\n",
       "      <th>tagID</th>\n",
       "      <th>date_day</th>\n",
       "      <th>date_month</th>\n",
       "      <th>date_year</th>\n",
       "      <th>date_hour</th>\n",
       "      <th>date_minute</th>\n",
       "      <th>date_second</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75</td>\n",
       "      <td>353</td>\n",
       "      <td>5290</td>\n",
       "      <td>29</td>\n",
       "      <td>10</td>\n",
       "      <td>2006</td>\n",
       "      <td>23</td>\n",
       "      <td>20</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>78</td>\n",
       "      <td>4223</td>\n",
       "      <td>5264</td>\n",
       "      <td>16</td>\n",
       "      <td>4</td>\n",
       "      <td>2007</td>\n",
       "      <td>4</td>\n",
       "      <td>43</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>127</td>\n",
       "      <td>1343</td>\n",
       "      <td>1544</td>\n",
       "      <td>28</td>\n",
       "      <td>8</td>\n",
       "      <td>2007</td>\n",
       "      <td>3</td>\n",
       "      <td>42</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>127</td>\n",
       "      <td>1343</td>\n",
       "      <td>12330</td>\n",
       "      <td>28</td>\n",
       "      <td>8</td>\n",
       "      <td>2007</td>\n",
       "      <td>3</td>\n",
       "      <td>42</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>127</td>\n",
       "      <td>2080</td>\n",
       "      <td>1451</td>\n",
       "      <td>28</td>\n",
       "      <td>8</td>\n",
       "      <td>2007</td>\n",
       "      <td>3</td>\n",
       "      <td>42</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID  movieID  tagID  date_day  date_month  date_year  date_hour  \\\n",
       "0      75      353   5290        29          10       2006         23   \n",
       "1      78     4223   5264        16           4       2007          4   \n",
       "2     127     1343   1544        28           8       2007          3   \n",
       "3     127     1343  12330        28           8       2007          3   \n",
       "4     127     2080   1451        28           8       2007          3   \n",
       "\n",
       "   date_minute  date_second  \n",
       "0           20           15  \n",
       "1           43           45  \n",
       "2           42           27  \n",
       "3           42           27  \n",
       "4           42           47  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_taggedmovies = pd.read_csv('user_taggedmovies.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "user_taggedmovies.head()#data10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wjCwdpSM117e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>movieID</th>\n",
       "      <th>tagID</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75</td>\n",
       "      <td>353</td>\n",
       "      <td>5290</td>\n",
       "      <td>1162160415000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>78</td>\n",
       "      <td>4223</td>\n",
       "      <td>5264</td>\n",
       "      <td>1176691425000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>127</td>\n",
       "      <td>1343</td>\n",
       "      <td>1544</td>\n",
       "      <td>1188265347000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>127</td>\n",
       "      <td>1343</td>\n",
       "      <td>12330</td>\n",
       "      <td>1188265347000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>127</td>\n",
       "      <td>2080</td>\n",
       "      <td>1451</td>\n",
       "      <td>1188265367000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID  movieID  tagID      timestamp\n",
       "0      75      353   5290  1162160415000\n",
       "1      78     4223   5264  1176691425000\n",
       "2     127     1343   1544  1188265347000\n",
       "3     127     1343  12330  1188265347000\n",
       "4     127     2080   1451  1188265367000"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_taggedmovies_timestamps = pd.read_csv('user_taggedmovies-timestamps.dat', sep='\\t', header=0, skipinitialspace=False,encoding='latin-1')\n",
    "user_taggedmovies_timestamps.head()#data11"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XUDJz6hK2YNB"
   },
   "source": [
    "From the above descriptions od the data and exploring readme, we have learned the following:\n",
    "\n",
    "\n",
    "1.   user_taggedmovies exists twice; once with timestamp and once with seperate columns\n",
    "2. user_ratedmovies exists the same way as user_taggedmovies in the provided data\n",
    "3. movie_locations has NaN in lots of places and might not be very relevant to the work due to multiple reasons i.e locations like USA might overwhelm all other locations.\n",
    "\n",
    "\n",
    "\n",
    "We can choose which we want to take ahead based on what makes most sense through EDA, literature review and utilizing knowledge of the world - knowledge of the head.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We will utilize a dataframe named ExtractedData for building the dataset that contains only the most relevant data**\n",
    "\n",
    "\n",
    "1.   Initially, we will take movieID, rtID(Title) from movies.dat for now\n",
    "2.   Merge it with user_ratedmovies-timestamps to achieve the following attributes in our dataset: UserID, MovieID, User Rating, Movie Titles\n",
    "3.   Because one movie can have multiple genres, group them together and merge it with ExtractedData"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LQwHVhaU22uV"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy story</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpy Old Men</td>\n",
       "      <td>1993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                        title  year\n",
       "0        1                    Toy story  1995\n",
       "1        2                      Jumanji  1995\n",
       "2        3               Grumpy Old Men  1993\n",
       "3        4            Waiting to Exhale  1995\n",
       "4        5  Father of the Bride Part II  1995"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ExtractedData=movie[['id','title','year']]\n",
    "ExtractedData.columns = ['movieID', 'title','year']\n",
    "ExtractedData.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Duplicates Detected: movie titles - 500+**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(643, 21)\n"
     ]
    }
   ],
   "source": [
    "duplicateRowsDF = movie[movie.duplicated(['title'])]\n",
    "#duplicateRowsDF.head()\n",
    "print(duplicateRowsDF.shape)\n",
    "#duplicateRowsDF.to_csv('DuplicateTitles.csv',index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6648, 21)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Drop Duplicate Titles\n",
    "movie=movie.drop_duplicates(subset='title', keep=\"first\")\n",
    "movie.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                         0\n",
       "title                      0\n",
       "imdbID                     0\n",
       "spanishTitle               0\n",
       "imdbPictureURL            64\n",
       "year                       0\n",
       "rtID                       0\n",
       "rtAllCriticsRating         0\n",
       "rtAllCriticsNumReviews     0\n",
       "rtAllCriticsNumFresh       0\n",
       "rtAllCriticsNumRotten      0\n",
       "rtAllCriticsScore          0\n",
       "rtTopCriticsRating         0\n",
       "rtTopCriticsNumReviews     0\n",
       "rtTopCriticsNumFresh       0\n",
       "rtTopCriticsNumRotten      0\n",
       "rtTopCriticsScore          0\n",
       "rtAudienceRating           0\n",
       "rtAudienceNumRatings       0\n",
       "rtAudienceScore            0\n",
       "rtPictureURL               0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking if any null values are present anywhere important\n",
    "movie.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>genre</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Adventure, Animation, Children, Comedy, Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Adventure, Children, Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Comedy, Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Comedy, Drama, Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                                            genre\n",
       "0        1  Adventure, Animation, Children, Comedy, Fantasy\n",
       "1        2                     Adventure, Children, Fantasy\n",
       "2        3                                  Comedy, Romance\n",
       "3        4                           Comedy, Drama, Romance\n",
       "4        5                                           Comedy"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#mapping genre w.r.t movieID\n",
    "moviemapping = movie_genres.groupby(['movieID'])['genre'].apply(', '.join).reset_index()\n",
    "moviemapping.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "      <th>genre</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy story</td>\n",
       "      <td>1995</td>\n",
       "      <td>Adventure, Animation, Children, Comedy, Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>1995</td>\n",
       "      <td>Adventure, Children, Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpy Old Men</td>\n",
       "      <td>1993</td>\n",
       "      <td>Comedy, Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>1995</td>\n",
       "      <td>Comedy, Drama, Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>1995</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                        title  year  \\\n",
       "0        1                    Toy story  1995   \n",
       "1        2                      Jumanji  1995   \n",
       "2        3               Grumpy Old Men  1993   \n",
       "3        4            Waiting to Exhale  1995   \n",
       "4        5  Father of the Bride Part II  1995   \n",
       "\n",
       "                                             genre  \n",
       "0  Adventure, Animation, Children, Comedy, Fantasy  \n",
       "1                     Adventure, Children, Fantasy  \n",
       "2                                  Comedy, Romance  \n",
       "3                           Comedy, Drama, Romance  \n",
       "4                                           Comedy  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Extracting Relevant Data as part of Feature Selection\n",
    "ExtractedData=pd.merge(ExtractedData, moviemapping, on='movieID')\n",
    "ExtractedData=ExtractedData.drop_duplicates(subset='title',keep='first')\n",
    "ExtractedData.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6648, 4)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ExtractedData.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "GenreEncoding=movie_genres['genre'].values.tolist()\n",
    "GenreEncoding\n",
    "GenreEncodingString=\" \".join(GenreEncoding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Genre:**\n",
    "1. Drama is the most popular\n",
    "2. Comedy is a close second\n",
    "3. Top 15 will do more than enough to capture the nature with regards to the impact Genre has"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAACJ4AAAW/CAYAAADnjqsKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdT6hnZR3H8c+3hlpElOUoockNclOLIgYLXFXQf7JFghA0hOCmRbv+rIJKqFXSokAqsCAsgkgqMrFaVs5Q9D8cykqMtLQ2UVA9LeYYt7gz987H6V6HXi/48TvPc57z+z1n/+acWWsFAAAAAAAAAAAu1FOOegMAAAAAAAAAAFyahCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVI4d9QbO5/LLL187OztHvQ0AAAAAAAAAgP9bp0+f/sNa6/he557U4cnOzk5OnTp11NsAAAAAAAAAAPi/NTO/Ptc5r9oBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgcuyoNwAAAHBYdt771aPewiXrgQ+/8ai3AAAAAAA8CXniCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAlQOFJzPzwMz8aGZ+MDOntrnnzMw9M3P/9n3ZNj8z87GZOTMzP5yZl+36nZPb+vtn5uT/5pYAAAAAAAAAADgMF/LEk1eutV661jqxjd+b5N611rVJ7t3GSfL6JNdun1uSfCI5G6okeX+Slye5Lsn7H49VAAAAAAAAAAC49DyRV+3ckOSO7fiOJG/ZNf+ZddZ3kjx7Zp6X5LVJ7llrPbrWeizJPUle9wT+HwAAAAAAAACAI3TQ8GQl+cbMnJ6ZW7a5K9dav0uS7fuKbf6qJL/dde2D29y55gEAAAAAAAAAuAQdO+C669daD83MFUnumZmfn2ft7DG3zjP/nxefDVtuSZJrrrnmgNsDAAAAAAAAAOCwHeiJJ2uth7bvh5N8Kcl1SX6/vUIn2/fD2/IHkzx/1+VXJ3noPPP//V+3r7VOrLVOHD9+/MLuBgAAAAAAAACAQ7NveDIzz5iZZz5+nOQ1SX6c5K4kJ7dlJ5N8eTu+K8nb56xXJPnz9iqeu5O8ZmYum5nLtt+5+6LeDQAAAAAAAAAAh+Ygr9q5MsmXZubx9Z9ba319Zu5L8oWZuTnJb5LcuK3/WpI3JDmT5C9J3pEka61HZ+aDSe7b1n1grfXoRbsTAAAAAAAAAAAO1b7hyVrrl0lessf8H5O8eo/5leSd5/itTyf59IVvEwAAAAAAAACAJ5t9X7UDAAAAAAAAAAB7EZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUDlweDIzT52Z78/MV7bxC2bmuzNz/8x8fmaets0/fRuf2c7v7PqN923zv5iZ117smwEAAAAAAAAA4PBcyBNP3pXkZ7vGH0ny0bXWtUkeS3LzNn9zksfWWi9M8tFtXWbmRUluSvLiJK9L8vGZeeoT2z4AAAAAAAAAAEflQOHJzFyd5I1JPrmNJ8mrknxxW3JHkrdsxzds42znX72tvyHJnWutv621fpXkTJLrLsZNAAAAAAAAAABw+A76xJPbkrw7yT+38XOT/Gmt9fdt/GCSq7bjq5L8Nkm283/e1v97fo9rAAAAAAAAAAC4xOwbnszMm5I8vNY6vXt6j6Vrn3Pnu2b3/90yM6dm5tQjjzyy3/YAAAAAAAAAADgiB3niyfVJ3jwzDyS5M2dfsXNbkmfPzLFtzdVJHtqOH0zy/CTZzj8ryaO75/e45t/WWrevtU6stU4cP378gm8IAAAAAAAAAIDDsW94stZ631rr6rXWTpKbknxzrfW2JN9K8tZt2ckkX96O79rG2c5/c621tvmbZubpM/OCJNcm+d5FuxMAAAAAAAAAAA7Vsf2XnNN7ktw5Mx9K8v0kn9rmP5XkszNzJmefdHJTkqy1fjIzX0jy0yR/T/LOtdY/nsD/AwAAAAAAAABwhC4oPFlrfTvJt7fjXya5bo81f01y4zmuvzXJrRe6SQAAAAAAAAAAnnz2fdUOAAAAAAAAAADsRXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABARXgCAAAAAAAAAEBFeAIAAAAAAAAAQEV4AgAAAAAAAABA5V/s3bGr72UdwPHPw71lTQlxh1BBIYdqyEDE2YKEBhsK7uYgtPQP1Bgk5OTWEDhIi0lLYlNQrYqgBhrRzYKkoQtp0JCgPA33K1ziXM65b67nZrxecDjf7+d5fr/zffY33yM8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJBcvN0PAAD8/7j3e7+83Y/wsfaXH33jdj8CAAAAAADATfHGEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACARngAAAAAAAAAAkAhPAAAAAAAAAABIhCcAAAAAAAAAACTCEwAAAAAAAAAAEuEJAAAAAAAAAACJ8AQAAAAAAAAAgER4AgAAAAAAAABAIjwBAAAAAAAAACA5NTxZa31qrfXyWuv1tdYba60fHPP71lovrbX+uNb62Vrrk8f8juP+yrF+73Xf9f1j/oe11tc/qkMBAAAAAAAAAPDRO8sbT96bmUf23l+emQdm5tG11sMz89TMPL33vn9m3pmZJ479T8zMO3vvz8/M08e+WWt9cRXwTNUAACAASURBVGYuz8yXZubRmfnxWuvCrTwMAAAAAAAAAADn59TwZF/zr+P2E8fPnplHZubnx/zZmfnmcf3YcT/H+lfXWuuYP7f3fm/v/eeZuTIzD92SUwAAAAAAAAAAcO7O8saTWWtdWGu9NjN/n5lfzcyfZubdvff7x5a3Z+au4/qumfnrzMyx/s+Z+ez18xM+c/3f+s5a65W11itXr169+RMBAAAAAAAAAHAuzhSe7L0/2Hs/MDN3z7W3lHzhpG3H73WDtRvN//tv/WTv/eDe+8FLly6d5fEAAAAAAAAAALgNzhSefGjv/e7M/HZmHp6ZO9daF4+lu2fmb8f12zNzz8zMsf6ZmfnH9fMTPgMAAAAAAAAAwMfMqeHJWuvSWuvO4/rTM/O1mfn9zPxmZr51bHt8Zn5xXL9w3M+x/uu99z7ml9dad6y17puZ+2fm5Vt1EAAAAAAAAAAAztfF07fM52bm2bXWhbkWqjy/935xrfXmzDy31vrhzLw6M88c+5+ZmZ+uta7MtTedXJ6Z2Xu/sdZ6fmbenJn3Z+a7e+8Pbu1xAAAAAAAAAAA4L6eGJ3vv383MV06YvzUzD50w//fMfPsG3/XkzDx5848JAAAAAAAAAMD/mlP/1Q4AAAAAAAAAAJxEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAAAAAAAASIQnAAAAAAAAAAAkwhMAAAAAAAAAABLhCQAAAAAAAAAAifAEAAAAAAAAAIBEeAIAAAAAAAAAQCI8AQAAAAAAAAAgEZ4AAAAAAAAAAJAITwAAAADgP+zdP6jleXnH8c8DN10KRxxF1oXbbGG6yKILaQLC+ucWa2ORIg4ibLNFAmkuaQa0uVUKG0HIkBVCQFBQuIIMSyCkMLiIGGWL3eKig4s7MGICVoFvivkpN7szc+98MjNnz+7rBZdz7nN+59znVLd58/0BAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAABXhCQAAAAAAAAAAFeEJAAAAAAAAAAAV4QkAAAAAAAAAAJULw5OZeXpm/nVmXpuZX8zM32zzD87MzZl5fXu8ss1nZr4+M2/MzM9m5hPnPuvadv3rM3Pt8X0tAAAAAAAAAAAet8ucePI/Sf5urfXxJM8leWlm/izJcZJX1lrPJHll+z1JPpfkme3nxSTfSO6GKkmuJ/lUkk8muf6HWAUAAAAAAAAAgP1zYXiy1npzrfWT7fl/J3ktyVNJXkjy8nbZy0m+sD1/Icm31l0/SvKBmfloks8kubnWurPW+m2Sm0k++0i/DQAAAAAAAAAAT8xlTjz5o5k5TPLnSf4jyUfWWm8md+OUJB/eLnsqya/Ove3WNrvf/O1/48WZeXVmXr19+/bDrAcAAAAAAAAAwBN06fBkZv40yXeS/O1a678edOk9ZusB8/87WOuba61n11rPXr169bLrAQAAAAAAAADwhF0qPJmZP8nd6OSf11rf3ca/2W6hk+3xrW1+K8nT597+sSS/fsAcAAAAAAAAAIA9dGF4MjOT5B+TvLbW+odzL30/ybXt+bUk3zs3/9Lc9VyS32234vlhkudn5srMXEny/DYDAAAAAAAAAGAPHVzimr9I8tdJ/nNmfrrN/j7JSZJvz8xXkvwyyRe3136Q5PNJ3kjy+yRfTpK11p2Z+VqSH2/XfXWtdeeRfAsAAAAAAAAAAJ64C8OTtda/J5n7vPzpe1y/krx0n8+6keTGwywIAAAAAAAAAMC704W32gEAAAAAAAAAgHsRngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFA52PUCAAAAAAAAwHvD4fHprlfYa2cnR7teAeChOfEEAAAAAAAAAICK8AQAAAAAAAAAgIrwBAAAAAAAAACAivAEAAAAAAAAAICK8AQAAAAAAAAAgMrBrhcAAAAAAAAu7/D4dNcr7K2zk6NdrwAA8J7jxBMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACoHu16A/XB4fLrrFfbW2cnRrlcAAAAAAAAAgMfCiScAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUDna9AAAAj97h8emuV9hrZydHu14BAAAAAAD2ghNPAAAAAAAAAACoCE8AAAAAAAAAAKgITwAAAAAAAAAAqAhPAAAAAAAAAACoCE8AAAAAAAAAAKgITwAAAAAAAAAAqAhPAAAAAAAAAACoCE8AAAAAAAAAAKgITwAAAAAAAAAAqAhPAAAAAAAAAACoCE8AAAAAAAAAAKgITwAAAAAAAAAAqAhPAAAAAAAAAACoCE8AAAAAAAAAAKgITwAAAAAAAAAAqAhPAAAAAAAAAACoCE8AAAAAAAAAAKgITwAAAAAAAAAAqAhPAAAAAAAAAACoXBiezMyNmXlrZn5+bvbBmbk5M69vj1e2+czM12fmjZn52cx84tx7rm3Xvz4z1x7P1wEAAAAAAAAA4Em5zIkn/5Tks2+bHSd5Za31TJJXtt+T5HNJntl+XkzyjeRuqJLkepJPJflkkut/iFUAAAAAAAAAANhPF4Yna61/S3LnbeMXkry8PX85yRfOzb+17vpRkg/MzEeTfCbJzbXWnbXWb5PczDtjFgAAAAAAAAAA9shB+b6PrLXeTJK11psz8+Ft/lSSX5277tY2u98ceEiHx6e7XmFvnZ0c7XoFAAAAAAAAgPeUy9xq52HMPWbrAfN3fsDMizPz6sy8evv27Ue6HAAAAAAAAAAAj04bnvxmu4VOtse3tvmtJE+fu+5jSX79gPk7rLW+udZ6dq317NWrV8v1AAAAAAAAAAB43Nrw5PtJrm3PryX53rn5l+au55L8brslzw+TPD8zV2bmSpLntxkAAAAAAAAAAHvq4KILZuZfkvxlkg/NzK0k15OcJPn2zHwlyS+TfHG7/AdJPp/kjSS/T/LlJFlr3ZmZryX58XbdV9dadx7h9wAAAAAAAAAA4Am7MDxZa/3VfV769D2uXUleus/n3Ehy46G2AwAAAAAAAADgXau91Q4AAAAAAAAAAO9zF554AgAAAMDjd3h8uusV9trZydGuVwAAAID3JSeeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUBGeAAAAAAAAAABQEZ4AAAAAAAAAAFARngAAAAAAAAAAUDnY9QIAAAAAAAD76PD4dNcr7K2zk6NdrwAAPCJOPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiFrIjYwAAIABJREFUPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKAiPAEAAAAAAAAAoCI8AQAAAAAAAACgIjwBAAAAAAAAAKBysOsFAAAAeP85PD7d9Qp76+zkaNcrAAAAAMAfOfEEAAAAAAAAAICK8AQAAAAAAAAAgIrwBAAAAAAAAACAivAEAAAAAAAAAIDKwa4XAID/r8Pj012vsLfOTo52vQIAAAAAAAB7zIknAAAAAAAAAABUhCcAAAAAAAAAAFSEJwAAAAAAAAAAVIQnAAAAAAAAAABUhCcAAAAAAAAAAFQOdr0AAAAAAADvfofHp7teYW+dnRztegUA3qf8/+75/w2X58QTAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACrCEwAAAAAAAAAAKsITAAAAAAAAAAAqwhMAAAAAAAAAACoHu14AAADe6w6PT3e9wt46Ozna9QoAAAAAADyAE08AAAAAAAAAAKg48QQAAAAAeNdyelzP6XEAAMCT4MQTAAAAAAAAAAAqwhP+t707D7ctG+8F/PtUaUpXQkmCiOPmBkGkLoXoiyCaIIJQESFuInKjzRWRK48UrqSkIQlBpK4UoieEKtE7utKVUi1Ke0SR6KOJXo37xxj7nHV27bWbeZq9zt7v+zzr2WvNNddcc49vzjnGHPObYwEAAAAAAAAATCLxBAAAAAAAAACASSSeAAAAAAAAAAAwicQTAAAAAAAAAAAmkXgCAAAAAAAAAMAkEk8AAAAAAAAAAJhE4gkAAAAAAAAAAJNIPAEAAAAAAAAAYBKJJwAAAAAAAAAATCLxBAAAAAAAAACASSSeAAAAAAAAAAAwicQTAAAAAAAAAAAmkXgCAAAAAAAAAMAkEk8AAAAAAAAAAJhE4gkAAAAAAAAAAJNIPAEAAAAAAAAAYBKJJwAAAAAAAAAATCLxBAAAAAAAAACASSSeAAAAAAAAAAAwicQTAAAAAAAAAAAmkXgCAAAAAAAAAMAkEk8AAAAAAAAAAJhE4gkAAAAAAAAAAJNIPAEAAAAAAAAAYBKJJwAAAAAAAAAATCLxBAAAAAAAAACASSSeAAAAAAAAAAAwicQTAAAAAAAAAAAmkXgCAAAAAAAAAMAkEk8AAAAAAAAAAJhE4gkAAAAAAAAAAJNIPAEAAAAAAAAAYBKJJwAAAAAAAAAATCLxBAAAAAAAAACASSSeAAAAAAAAAAAwicQTAAAAAAAAAAAmOXyzVwDgULXjMads9iocsnadcOfNXgUAAAAAAABgPzDiCQAAAAAAAAAAk0g8AQAAAAAAAABgEoknAAAAAAAAAABMIvEEAAAAAAAAAIBJDt/sFQAAAAAAAACArWzHY07Z7FU4ZO064c6bvQqswYgnAAAAAAAAAABMIvEEAAAAAAAAAIBJJJ4AAAAAAAAAADCJxBMAAAAAAAAAACaReAIAAAAAAAAAwCQSTwAAAAAAAAAAmETiCQAAAAAAAAAAk0g8AQAAAAAAAABgEoknAAAAAAAAAABMIvEEAAAAAAAAAIBJJJ4AAAAAAAAAADCJxBMAAAAAAAAAACaReAIAAAAAAAAAwCQSTwAAAAAAAAAAmETiCQAAAAAAAAAAk0g8AQAAAAAAAABgEoknAAAAAAAAAABMIvEEAAAAAAAAAIBJJJ4AAAAAAAAAADCJxBMAAAAAAAAAACaReAIAAAAAAAAAwCQSTwAAAAAAAAAAmETiCQAAAAAAAAAAk0g8AQAAAAAAAABgEoknAAAAAAAAAABMIvEEAAAAAAAAAIBJJJ4AAAAAAAAAADCJxBMAAAAAAAAAACaReAIAAAAAAAAAwCQSTwAAAAAAAAAAmETiCQAAAAAAAAAAkxy+2SsAAAAAbJ4djzlls1fhkLXrhDtv9ioAAAAAbDojngAAAAAAAAAAMInEEwAAAAAAAAAAJpF4AgAAAAAAAADAJBJPAAAAAAAAAACYROIJAAAAAAAAAACTSDwBAAAAAAAAAGASiScAAAAAAAAAAEwi8QQAAAAAAAAAgEkkngAAAAAAAAAAMInEEwAAAAAAAAAAJpF4AgAAAAAAAADAJBJPAAAAAAAAAACYROIJAAAAAAAAAACTSDwBAAAAAAAAAGASiScAAAAAAAAAAEwi8QQAAAAAAAAAgEkkngAAAAAAAAAAMInEEwAAAAAAAAAAJpF4AgAAAAAAAADAJBJPAAAAAAAAAACYROIJAAAAAAAAAACTSDwBAAAAAAAAAGASiScAAAAAAAAAAEwi8QQAAAAAAAAAgEkkngAAAAAAAAAAMInEEwAAAAAAAAAAJpF4AgAAAAAAAADAJBJPAAAAAAAAAACYROIJAAAAAAAAAACTSDwBAAAAAAAAAGASiScAAAAAAAAAAExy+GavAAAAAAAskh2POWWzV+GQtuuEO2/2KgAAAHAQGfEEAAAAAAAAAIBJJJ4AAAAAAAAAADDJQU88qao7VNV5VfXxqnrMwf5+AAAAAAAAAAD2j4OaeFJVhyX5uyR3THLtJMdV1bUP5joAAAAAAAAAALB/HOwRT26U5OOttU+21r6X5MVJ7naQ1wEAAAAAAAAAgP3gYCeeXCXJZ2Zenz+mAQAAAAAAAABwiKnW2sH7sqp7JfnF1tpvjdf3S3Kj1tpDZ+Z5UJIHjZfXTHLeQVtBDlVHJfnSZq8EScRi0YjH4hCLxSEWi0MsFodYLA6xWCzisTjEYnGIxeIQi8UhFotDLBaLeCwOsVgcYrE4xGJxiMXiEAvW42qttSuu9MbhB3lFzk9y1ZnXP5Hkc7MztNaeneTZB3OlOLRV1WmttWM2ez0Qi0UjHotDLBaHWCwOsVgcYrE4xGKxiMfiEIvFIRaLQywWh1gsDrFYLOKxOMRicYjF4hCLxSEWi0Ms2FcH+6d23p/kp6vq6lV1sST3SfLqg7wOAAAAAAAAAADsBwd1xJPW2g+q6iFJXp/ksCTPaa2dezDXAQAAAAAAAACA/eNg/9ROWmuvTfLag/29bGl+mmlxiMViEY/FIRaLQywWh1gsDrFYHGKxWMRjcYjF4hCLxSEWi0MsFodYLBbxWBxisTjEYnGIxeIQi8UhFuyTaq1t9joAAAAAAAAAAHAIushmrwAAAAAAAAAAAIcmiScslKq6e1W1qrrWGvM9oKquPPP6xKq69oFfw0PbWuVbVSdV1T3383ceW1U33Z/L3CrWu72PeffLNl5VO6rq12ZeH1NVf7uvy91qqurHq+rFVfWJqvpQVb22qh5UVSfPmX93fKpqV1UdtcI8x1fVow70um9Fc+JxjYP03SvGk72tN0ZVdepmrN8iq6ofVtUZVXVuVZ1ZVb9fVYdEG72qjq6qO232eiyaQzmmW8FM+S89dkxcziOq6pL7d+22vpXKf7a9Oc7jnr4Pyz92tJ/vMjPt5Ko6do3POV+cUVXfXPZ6n+LC/rEUl7HftKp64sx7R1XV95fHadQzL1o27dVVdb+Z1/9QVX9woNf/UDTK+fkzrw+vqi/OO+9bx/ImHWvGsW3Sd24V+zMW2qgbM1N3n1NVr6mqy232OrG2qnrsON84a8TvxnPmm9vvV1U7q+q8mXbbPcd05+0bsFIs9rUvyXFsdVV1hZnt9j+q6rPj+X9W1YfWuYwHV9VvjOcnzWz/O6vqmAO5/ttNVT21qh4x8/r1VXXizOu/qqrf35y1O/QdiPLVH8K+0AHKojkuyTuT3GeN+R6QZHfiSWvtt1pr62pUbHPrLd/96dgkG0o8qarDDsyqLJx1x2M/buM7kuxOPGmtndZae9h+WO6WUVWV5JVJdrbWfqq1du0k/yfJj837zL7Ep6oOn7am28OUeHBwrSdGS8f11ppExAv7dmvt6NbadZLcLsmdkvzJJq/Teh2dvr7rtk2OeeuK6TYpi82wVP5Lj10Tl/OIJDpaNu5C5X8A2pvnJ3nsRj4wr622jc479qvlx6/1Hs8c9zbkk0l+aeb1vZKcOztDVf1Mer/eLavqUjNvPSzJE6rqctVvArlxkr8+wOt7qPqvJNetqiPG69sl+ezUhemb2if7MxbaqBuzVHdfN8lXkvzeZq8Qq6uqm6TXEddvrV0vyW2TfGaledfRDrvvTLvt5eMzztvXaSOx2MAyD8+E49h20lr78tJ2m+RZSZ46nh+d5IK1Pl9Vh7fWntVae96+rovziXU5NeP6UPWbco5Kcp2Z92+a5F1rLaQ617QvbL+U7zIb7g+xL7DETsrCqKpLJ7lZkv+ZmQvxVfXoqjp73Mlzwsg+PSbJC0Ym6xGzmahVddyY/5yqevLMcr5ZVU8ay3lPVW2ri5Yrle+orJ9e/e70U5L86Jh+x6p66cxnj62q14znt6+qd1fV6VX1srHcpVEBHj+mn11V16p+h+mDkzxyxOoWtWxUldpzV9exVfXWqnphkrPHtF+vqveNz/79Vqq85sTj2LEtv7yqPlJVLxgXdPfKth7b8pOr6gNV9aaqutF4/5NVddcxz46qeseIx+m1Z9SZE5LcYpTpI2vmzqqqunxVvap6hv57qup6Y/rxVfWcme/Y6okqt07y/dbas5YmtNbOSPKOJJdeKz6zqt/1cF5VvSnJNWem76yqP62qtyV5eFVdsapeUVXvH4+bjfm2W9mvZF483llVfzGO9WdX1b2T3fvR26rqpVX10VFv3HccS86uqp8a880r8ytU1Ruq6oNV9fdJlmL8xKp6+NI6jPpkO8ZjJfNidNgKx/XZY/7kOG1VrbUvJHlQkodUd4mq+sdRJh+sqlsn/WSuqv5yTD+rqh46pu++q6r6nW07x/Pjq+q5Y9veVVW/UlV/Pj7/uqq66JjvBiMuH6h+h8SVxvSd1eud94143aKqLpbkCUnuPeqUe1evj04d63pqVV1zfP4B1dsMr0nyhqp6flXdben/HsfTux6scj6YVojp8rK4dFW9ufa0n+6W7K7HP1L9zulzRhndtqreVVUfq6objflWLHP2VnPaRTWn7TWO71dO8taqeuuY95lVdVr1uxkfP7PsE6q3pc8a++VlqupTM/vVZcd+d9HN+N8XQc25k7/6ecEzq9cVn6yqW1Vv93y4qk5aZZFnJvlaVd1uhWX+wtgfzh7LuviYvrwt/YSqem+Sm+yf/3LrqKqrjePSWePvT47pJ1XVU8Y+8eRRtzy7qt6Q5Hk1v87a67i3if/aoebbST5ce84x7p3kpcvm+bUkz08v19316Ei4e3aSP0/yjCQPaa19/0Cv8CHsX5PceTw/LsnuEWRq2aiVo07eUVWXqqpTqvcvnVN7zkVmjzV3GHXOmVX15jFNvb26FWNRVRcZ7Z8rzrz+ePWRgO41YnBmVb29Vm6jXmrUCe8fZb/U3tq2bdRVvDvJVZLdfYb7cs59l6p67yjzN9Xoh61V+jmq6jdG/XNmjRFwapudE67TlZJ8qbX23SRprX2ptfa5qrrhOLacOeJxmXntsNXUspHRWNWKsRjvPbRm+smTNfted7ersuw4tgn/16HssOqjvZ1bvQ/kiGTFvti96viV1OrXQR5XVe9MTw5mde/KnhuTr5PknCTfqKofqX6+9jPp7d55fSMfrqpnJDk9yVU34x9YcOsp3w9W1R+MevSsGn0atUKbtlbuD1nXvlAr9B8e5LJgEbTWPDwW4pHk15P8v/H81CTXT3LH8fySY/rlx9+dSY6Z+ezO9GSUKyf5tyRXTHJ4krck+eUxT0tyl/H8z5P88Wb/zwtQvr+S5I1JDhtl959J7jnK7t+SXGrM/8zx+aOSvH1m+h8medx4vivJQ8fz/5XkxPH8+CSPmlmPk5Lcc+b1N8ffY9PvcLn6eP0zSV6T5KLj9TOS/MZml+MBjsexSb6W5CfSEwPfneTmy7f5sS3fcTx/ZXpH40WT/FySM8b0Sya5xHj+00lOmynnk2fWY/frJE9L8ifj+W1mlnX8WMeLj23gy0tx2YqP9DsEn7rC9PXGZ9copxukX2y/ZJLLJvn40r4w5n/GzLJfOLOsn0zy4e1Y9huMxz2y5/j1Y+nHrCuNOP3neH7x9LvkHj8+8/Akf71Gmf9t9hzX7jz2t6PSRws6fUy/SJJPJLnCZpfPIjzW2Gd2H9fHtNlj/uQ4baXHUpksm/bVsV3/7yT/OKZda2znl0jyu0lekeTw8d5S+2hXkqPG82PSR6FZOpa8M3vqim9l73rkl8d7pya54ph+7yTPGc93Jvmr8fxOSd40nj8gydNn1vuyM+t02ySvmJnv/Jn1vFWSV43nRyb51NLntsJjjZguL4vDk1x2PD8qva6occz5QZKfHcecDyR5znjvbjPlt2KZb+dHkh8mOWM8XjmmrdYumle3796fxuulmB029onrJbl8kvOS1HjvcuPvP2bPOciDlvaf7fCYU/7HZk97c/dxI/284MUz2/XXl23zR6+w/GOTnJzkFkneNqadPKZfIv0O02uM6c9L8ojxfGf2bkv/6maX1QLF6Yz0+mUpLq9Jcv/x/IEzx5uTRlkfNl4fP+J0xHg9r856QGaOex6rxmWpnbQjvcP4rkn+chyj3pwL17sfTXK1JLdP8uply7roiMELNvv/WuRHkm+O4/nLx/Z6xrJj1vHZuz/jnBGfeyT5h5npR46/O9PbYFccx6Ol/o2lOmReW2n3d27Xxzpi8Sczx/Tbz5Td2UmuMp4v1cPL95U/TfLrS/OMfedSy49P2eJt1NXKfvw9LMnLktxhvN7Xc+4fyZ420m9lz/nE8VmhnyP9Ytl52XM+sxSXLX9OOCFmlx77yEfT+0tvleRi6aNl3XDMc9n0c425x5dxzDove9oDV5jdJjymxWJM35WV+8lX63udbVftdRzzWDUGx2dPf+uO9PPoo8frl84c/3dm777Y2c+dlHG9Invq8rWugzx6s//3Q+kxyuwnk/xO+o3KT0zvX7rZKOfV+kYuSPLzm/0/LPJjHeV7+/TE9Eo/3z45yS0zv027K3vq43XvC5nTf+ixvR7beRhBFs9x2TP864vH64ukd159K0laa19ZYxk3TL/I8sWk352QfgB9VZLvpR9Qk96Qu9AdclvcSuV70SQvaq39MMnnquotSdJa+0FVvS7JXarq5ekXXx+dfiJz7STvqj7Qw8XSO+iX/PP4+4H0pJaNel9r7VPj+S+kX7h///iuI5J8YcIyF9VK8TglvQzOT5KqOiO9cfXOZZ/9XpLXjednJ/lua+37VXX2mD/psX16VR2d3rF8jXWs083TGxtprb2l+sgPR473Tmk9e/+7VfWF9E6H89f/724Z64nPklukX3D51pj/1cvef8nM89smufbY1pPkslV1mfFc2a/s5tlz/Pr8uGPhhukXrd7fWvv3JKmqT2TP3bVnp4/Okcwv81tmHL9aa6dU1VfH811V9eWq+h/pMfhga+3LB/qf3AJmj+vLTY5Ta+0bB2qFF8TSP3zz9I6ptNY+UlWfTj+e3zbJs1prPxjvrdU+SpJ/nakrDsve9ciO9FGZrpvkjaO8D0vy7zOfn63jd8z5jiOTPLeqfjr9wu7sCA9vXFrP1trbqurvqupH0/e3Vyz9L1tYzTx/40zMKsmfVtUt0ztTrpI9P1P1qdba0mhB5yZ5c2utLavvVyvz7erbrQ+zPGu1dtF66/ZfraoHpXeIXSm9TfyhJN9JcmL10QOXzjVOTG87vyrJbyb57f3wfx0qVir/1bxmZrv+/LJtfkd6R/6FtNbeUVVZdgfVNdP3m4+O189N/7mA5T8x8sP05L3tbK84VdUD0jvYkz4KzNK53PPTb9pY8rLR9lry6tbat8fzeXVWsvdxj/V7XXqn8eez97lDquqGSb7YWvt0VZ2f5DlV9SOtta+OWa6XXsdcq6ou0lpbc9j57aq1dlb10VqPS/LadX7s7CR/WX2U3ZNba+9Y9v7PJ3n7Ujt4ZvtXb69ijVg8J8m/pB/TH5ie5Jn0u3xPqj5q7z9nZbdPcteZO9svkX5xJtFGTZIjZtpAH0hPNkn2/Zz7J5K8pPooihdLT+RZslI/x22SvLy19qVkr/1mu54TztVa+2ZV3SC93+nW6XXEk5L8e2vt/WOeryfJTLnNc9/W2mkHcHW3tJViUVWPGW+v1E++Wt/rbLuK6T7V+ii8yYX7L15y4dnn+vmsfh1kI8tiz6gcN03ylPS+j5um3whyalbvG/l0a+09B32NDy1rle/tx+ODY/5Lp9+U846s3qZNNr4vrKf/kC1M4gkLoaqukH6Ccd2qaukXO1p6h2DbyKJWee/7rbWlZf0w22j7X6V8X5n55fuS9I7ar6SfUH6jes3yxtbacXM+893xd7Xy/UHGz3yN5V1s5r3/ml3tJM9trf3Rav/boWiVeLw2e8owmV+Os9vyBUufaa1dUHt+l/iR6R2UP5de3t9Zz6qtMG3pe9azXlvFuekj/6xko+Ww2vFrdnu/SJKbLD/BHI257VT2K5kXj9WO97NldsHM6wuyp/xWK/N5cTsx/a6TH0/v+KRbbZ/5rznTk32I01ZWVf8tfV//QuZv55WVt9PddWx6h/qs2bpieT1y+Fjmua21eT87sZ46/olJ3tpau/u4aLBz5r3l28Lzk9w3/efmHjhneVvCspgme5fFfdPvir7BSAzalT2xW88+slqZs8dq7aI169mqunqSR6XfQfrV6j8Dc4mRrH2j9ITp+yR5SJLbtNbeVX1I4Fuljw5xzoH4p7aI2e16+TZ/eFXdPf0u96TfLT3rSUkem37sS1ZvG8z6zrLkCVY3W98sP5YvP3+bZ7X2AHO01r5XVR9IH03mOknuMvP2celJJbvG68umX8g6sfpvuz8jyf3S73j83SR/d7DW+xD16vTRZY5NcoWZ6bNtq2TU0a21j44LjXdK8mdV9YbW2hNm5pvXVlNvr23FWLTWPlNVn6+q2yS5cXobKq21B1fVjdNvmDpjJJkuV0nu0Vo7b6+J/XPbto0649uttaPHxe+T0/sC/zb7fs79tCRPaa29uqqOTR9dYKXPL7W/5u032+6ccD1GW2Znkp0jgff3skYfelW9Pv1C7mmtteXtKiZaIRb3H2+tdA69Wt+r9tL+sfz4csTM642U8VrXQcRrY05NT4T42fQR5D6T3sb9enof62p9I8p6bWuV77FJ/qy19vfLP7hGmzbZ+L6wnv5DtrCLrD0LHBT3TPK81trVWms7WmtXTc+E/0qSB1bVJZP+O4hj/m8kucwKy3lvkltV/53Xw9I7Y9524Fd/4a1WvvepqsPGHQi3nvnMzvSff/nt7MlafE+Sm1XVf0+SqrpkVa01ksbyWO1KH8kk6UNqz7vD581J7jnuNFn6DcyrrfmfHhrmxePm+/E7jky/0+GC9A7Hw8b0eftO0odMu2/Sf7M3/TdKv74f1+lQ8ZYkF6+q3XcnjzsKb7XB5bw9yd2r6ogxksZdVpn3DekXq5a+byN3Cm918+Lx1fTfuz2s+m993zLJ+zaw3HllPrsf3DF9eOAlr0xyh/S7vF6/8X9ly9pf+8xKttW+MbblZ6UPqduy9/Z4jfQ7M89LL5cHLyUbzrSPdmVPHXuPDX79eUmuWFU3Gcu8aFVdZ43PLK9TjkwfajvpSVqrOSnJI5KktXbuBtf1kLFCTJc7MskXRsfKrdN/MmEjNlLm29m8dtFqZrfvy6Z3pnytqn4s/edAU/13jY9srb02fXuePUY9L8mLsueObCZorb2ytXb0eJy27L03pNfTPzcmfSTJjqVzlfRYOxfcuFPTL7gmvQ6aN7rfcvPqLPbNXyX5wzYz0t5ILLlXkuuN88kd6efWS53Bv5PkY621nUl+P8mjR33EfM9J8oSlUZdm7ErvF0lVXT/J1cfzKyf5Vmvtn9KTJK6/7HPvTu+bWpp/qa2m3l7bvFgk/UaAf0ry0qUEwqr6qdbae1trj0vypSRXzYXbqK9P8tBx81Oqj2I5z0nZBm3UlbTWvpb+M6qPqqqLph/X9+Wce3Z7v/9qMw5vTh9h7grJXvvNtjonXI+quuYYOWnJ0Uk+nOTK41w8VXWZmZvTkiSttV8cbSpJJ/vJnFh8epWPrLfvdbX+Ww6OKddBmO9dSX4pyVdaaz8co1pdLn20xXdn3/tGtru1yvf16ddZL50kVXWVqvrRVdq0s8cg+wIbIvGERXFc+gW9Wa9IcuX0ux1Oqz7s49KwmCcleVZVnVFVu7NWxzCPf5TkrUnOTHJ6a+0NmNkUAAADiElEQVRfDvC6Hwrmle+PJ/lY+lCYz8xMx+w4iT85vWP95DHti+mdIy+qqrPSK51rrfHdr0m/+H5G9aGw/yG9A+Z96XeprJix2lr7UJI/TvKG8V1vTB/WfCuYF49f24/f8Ywk96+q96QPcb1Uzmcl+UFVnVlVj1z2meOTHDPK+4Ssr2NgyxkXBu+e5HZV9YnqQ70fn+RzG1zO6elJW2ekx3eloeqWPCyj7KvqQ+l3JpJV4/HC9O35zPTEh0e31v5jA4ueV+aPT3LLqjo9fQjCf5tZl++l1y+7OzrZf/vMHNth3zhi1JHnJnlTesfq48d7z0hy2Lhr6iVJHjCGpD4xfds8q6rOzJ764/FJ/qaq3pF+Z8G6je37nkmePJZ5RvrdEqt5a/qw12dU1b3Tf47hz6rqXVnjwn5r7fPpnaNb8aL8ajFd7gXp2/hp6R2QH9ngd627zLe5ee2i1Tw7yb9W1Vtba2emD0l7bvoFsXeNeS6T5OTRdnpb+sgqS16QnhTxov3zLzDHk9KH8k9r7TvpP230snHcvCA98YuNeViS3xzb9f2SPHydn5tXZ7EPWmvnttaeu2zyLZN8trX22Zlpb0+vk6+W/rvrjxqf/1ySv8neP5nEMq2181trf7PCW69IcvnRH/W7SZZ+yutnk7xvTH9skv+7bHlfTPKgJP882lVLN/Oot9ewSiyS3j946ezdfvyLqjq7qs5J3w/OzIXbqE9Mv+nprDHfE1f5/q3cRl1Ta+2D6WV4n/R+q3055z4+vU5+R3pS0FrffW56vf62sd88Zby1Hc4JN+rS6T/b9aFRX187yeOS3DvJ00b5vTEXHgWT/W+lWBy/yvzHZ319r8uPYxxkE6+DMN/ZSY5KL8fZaV9r/SfW9rVvZLtbtXzHTRsvTPLucb728vT+jHlt2tn+EPsCG1Ir33gHAMCscXfp6Unu1Vr72GavDxzKqo9md3aS64+7K2FLqap7Jrlba+1+m70uALAVVNUxSZ7aWrvFAfwObVQAAJjIiCcAAGuoqmsn+XiSN0s6gX1TVbdNv3vlaTr02Yqq6mnpdzDOvaMaAFi/qnpM+gg0f3QAv0MbFQAA9oERTwAAAAAAAAAAmMSIJwAAAAAAAAAATCLxBAAAAAAAAACASSSeAAAAAAAAAAAwicQTAAAAAAAAAAAmkXgCAAAAAAAAAMAkEk8AAAAAAAAAAJjk/wNQ+k8/8ldzDgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels,counts=np.unique(GenreEncoding,return_counts=True)\n",
    "fig = plt.figure()\n",
    "ax = fig.add_axes([5,5,5,5])\n",
    "ax.bar(labels,counts)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['Action', 'Adventure', 'Animation', 'Children', 'Comedy',\n",
       "        'Crime', 'Documentary', 'Drama', 'Fantasy', 'Film-Noir',\n",
       "        'Horror', 'IMAX', 'Musical', 'Mystery', 'Romance', 'Sci-Fi',\n",
       "        'Short', 'Thriller', 'War', 'Western'],\n",
       "       ['1', '1003', '1086', '1445', '145', '1644', '1664', '25', '261',\n",
       "        '279', '3566', '421', '430', '494', '497', '5076', '519', '535',\n",
       "        '740', '978']], dtype='<U21')"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels,counts=np.unique(GenreEncoding,return_counts=True)\n",
    "array=np.array([labels,counts])\n",
    "array=np.sort(array)\n",
    "array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 1.2 Genre Matrix**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  7.87it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████| 6648/6648 [00:10<00:00, 662.41it/s]\n"
     ]
    }
   ],
   "source": [
    "dataset = nltk.sent_tokenize(GenreEncodingString) \n",
    "word2count = {} \n",
    "for data in tqdm(dataset): \n",
    "    words = nltk.word_tokenize(data) \n",
    "    for word in words: \n",
    "        if word not in word2count.keys(): \n",
    "            word2count[word] = 1\n",
    "        else: \n",
    "            word2count[word] += 1\n",
    "freq_words = heapq.nlargest(15, word2count, key=word2count.get)\n",
    "BoFGenre = [] \n",
    "for data in tqdm(ExtractedData['genre']): \n",
    "    vector = [] \n",
    "    for word in freq_words: \n",
    "        if word in nltk.word_tokenize(data): \n",
    "            vector.append(1) \n",
    "        else: \n",
    "            vector.append(0) \n",
    "    BoFGenre.append(vector) \n",
    "BoFGenre = np.asarray(BoFGenre) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "DF=pd.DataFrame(BoFGenre,columns=freq_words)\n",
    "DF.head()\n",
    "ExtractedData=ExtractedData.reset_index()\n",
    "DF=DF.reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "GenreEncodedData=pd.merge(ExtractedData, DF, left_index=True, right_index=True)\n",
    "GenreEncodedData=GenreEncodedData.drop(['index_x','index_y'],axis=1)\n",
    "#doesnt work the way its supposed to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "      <th>genre</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Romance</th>\n",
       "      <th>Action</th>\n",
       "      <th>Crime</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Children</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>War</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>Musical</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy story</td>\n",
       "      <td>1995</td>\n",
       "      <td>Adventure, Animation, Children, Comedy, Fantasy</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>1995</td>\n",
       "      <td>Adventure, Children, Fantasy</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpy Old Men</td>\n",
       "      <td>1993</td>\n",
       "      <td>Comedy, Romance</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>1995</td>\n",
       "      <td>Comedy, Drama, Romance</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>1995</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                        title  year  \\\n",
       "0        1                    Toy story  1995   \n",
       "1        2                      Jumanji  1995   \n",
       "2        3               Grumpy Old Men  1993   \n",
       "3        4            Waiting to Exhale  1995   \n",
       "4        5  Father of the Bride Part II  1995   \n",
       "\n",
       "                                             genre  Drama  Comedy  Thriller  \\\n",
       "0  Adventure, Animation, Children, Comedy, Fantasy      0       1         0   \n",
       "1                     Adventure, Children, Fantasy      0       0         0   \n",
       "2                                  Comedy, Romance      0       1         0   \n",
       "3                           Comedy, Drama, Romance      1       1         0   \n",
       "4                                           Comedy      0       1         0   \n",
       "\n",
       "   Romance  Action  Crime  Adventure  Horror  Sci-Fi  Fantasy  Children  \\\n",
       "0        0       0      0          1       0       0        1         1   \n",
       "1        0       0      0          1       0       0        1         1   \n",
       "2        1       0      0          0       0       0        0         0   \n",
       "3        1       0      0          0       0       0        0         0   \n",
       "4        0       0      0          0       0       0        0         0   \n",
       "\n",
       "   Mystery  War  Documentary  Musical  \n",
       "0        0    0            0        0  \n",
       "1        0    0            0        0  \n",
       "2        0    0            0        0  \n",
       "3        0    0            0        0  \n",
       "4        0    0            0        0  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GenreEncodedData.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 1.3: Embedding Titles**\n",
    "1. Titles are converted to 1x20 columns of embeddings\n",
    "2. This allows us to extract context from the titles i.e sequels should come together\n",
    "3. We will use the same logic for Tags, Actors, Directors later on\n",
    "4. We will calculate Cosine Similarity and attempt to retrieve reccomendations on Genre and Titles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow_hub as hub\n",
    "embed = hub.load(\"C:\\\\Users\\\\amk_m\\\\Downloads\\\\tf2-preview_gnews-swivel-20dim_1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['topic_F0', 'topic_F1', 'topic_F2', 'topic_F3', 'topic_F4', 'topic_F5', 'topic_F6', 'topic_F7', 'topic_F8', 'topic_F9', 'topic_F10', 'topic_F11', 'topic_F12', 'topic_F13', 'topic_F14', 'topic_F15', 'topic_F16', 'topic_F17', 'topic_F18', 'topic_F19']\n"
     ]
    }
   ],
   "source": [
    "EmbeddingTitles=embed(GenreEncodedData['title']).numpy()\n",
    "names=[\"topic_F\"+str(x) for x in range(20)]\n",
    "print(names)\n",
    "EmbeddedTitles=pd.DataFrame(EmbeddingTitles,columns=names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TitleGenreEncodings=pd.concat([GenreEncodedData,EmbeddedTitles],axis=1)\n",
    "#TitleGenreEncodings=TitleGenreEncodings.drop(['index_x','index_y'],axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "      <th>genre</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Romance</th>\n",
       "      <th>Action</th>\n",
       "      <th>Crime</th>\n",
       "      <th>...</th>\n",
       "      <th>topic_F10</th>\n",
       "      <th>topic_F11</th>\n",
       "      <th>topic_F12</th>\n",
       "      <th>topic_F13</th>\n",
       "      <th>topic_F14</th>\n",
       "      <th>topic_F15</th>\n",
       "      <th>topic_F16</th>\n",
       "      <th>topic_F17</th>\n",
       "      <th>topic_F18</th>\n",
       "      <th>topic_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy story</td>\n",
       "      <td>1995</td>\n",
       "      <td>Adventure, Animation, Children, Comedy, Fantasy</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.391694</td>\n",
       "      <td>-0.799809</td>\n",
       "      <td>0.115686</td>\n",
       "      <td>-0.128886</td>\n",
       "      <td>0.118368</td>\n",
       "      <td>-0.812238</td>\n",
       "      <td>-1.202401</td>\n",
       "      <td>0.807139</td>\n",
       "      <td>0.170525</td>\n",
       "      <td>-0.211800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>1995</td>\n",
       "      <td>Adventure, Children, Fantasy</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpy Old Men</td>\n",
       "      <td>1993</td>\n",
       "      <td>Comedy, Romance</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.897274</td>\n",
       "      <td>-0.604493</td>\n",
       "      <td>0.518706</td>\n",
       "      <td>-0.408605</td>\n",
       "      <td>-0.069823</td>\n",
       "      <td>-0.959486</td>\n",
       "      <td>-1.041266</td>\n",
       "      <td>0.863840</td>\n",
       "      <td>-0.060412</td>\n",
       "      <td>0.575644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>1995</td>\n",
       "      <td>Comedy, Drama, Romance</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.180029</td>\n",
       "      <td>-0.041831</td>\n",
       "      <td>-0.074353</td>\n",
       "      <td>0.054048</td>\n",
       "      <td>-0.216928</td>\n",
       "      <td>0.096300</td>\n",
       "      <td>0.588923</td>\n",
       "      <td>-0.215821</td>\n",
       "      <td>-0.093493</td>\n",
       "      <td>0.001815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>1995</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.608391</td>\n",
       "      <td>-0.679735</td>\n",
       "      <td>0.414188</td>\n",
       "      <td>-0.200275</td>\n",
       "      <td>0.236952</td>\n",
       "      <td>-0.800664</td>\n",
       "      <td>-0.994438</td>\n",
       "      <td>0.609718</td>\n",
       "      <td>0.150760</td>\n",
       "      <td>0.433787</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                        title  year  \\\n",
       "0        1                    Toy story  1995   \n",
       "1        2                      Jumanji  1995   \n",
       "2        3               Grumpy Old Men  1993   \n",
       "3        4            Waiting to Exhale  1995   \n",
       "4        5  Father of the Bride Part II  1995   \n",
       "\n",
       "                                             genre  Drama  Comedy  Thriller  \\\n",
       "0  Adventure, Animation, Children, Comedy, Fantasy      0       1         0   \n",
       "1                     Adventure, Children, Fantasy      0       0         0   \n",
       "2                                  Comedy, Romance      0       1         0   \n",
       "3                           Comedy, Drama, Romance      1       1         0   \n",
       "4                                           Comedy      0       1         0   \n",
       "\n",
       "   Romance  Action  Crime  ...  topic_F10  topic_F11  topic_F12  topic_F13  \\\n",
       "0        0       0      0  ...  -1.391694  -0.799809   0.115686  -0.128886   \n",
       "1        0       0      0  ...   0.000000   0.000000   0.000000   0.000000   \n",
       "2        1       0      0  ...  -0.897274  -0.604493   0.518706  -0.408605   \n",
       "3        1       0      0  ...  -0.180029  -0.041831  -0.074353   0.054048   \n",
       "4        0       0      0  ...  -0.608391  -0.679735   0.414188  -0.200275   \n",
       "\n",
       "   topic_F14  topic_F15  topic_F16  topic_F17  topic_F18  topic_F19  \n",
       "0   0.118368  -0.812238  -1.202401   0.807139   0.170525  -0.211800  \n",
       "1   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000  \n",
       "2  -0.069823  -0.959486  -1.041266   0.863840  -0.060412   0.575644  \n",
       "3  -0.216928   0.096300   0.588923  -0.215821  -0.093493   0.001815  \n",
       "4   0.236952  -0.800664  -0.994438   0.609718   0.150760   0.433787  \n",
       "\n",
       "[5 rows x 39 columns]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Titles Embeddings via Universal Sentence Encoder and Genre 1-0 values \n",
    "TitleGenreEncodings.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "TitleGenreEncodings.to_csv('ReccomendationSystemTask.csv',index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The Similarity Matrix based on Genre and Title**\n",
    "1. Lacks an evaluation metric as of yet\n",
    "2. Through a user study that does not represent the population space, we have recieved a good approval rate (People have either already watched the movies reccomended, and/or understand why these reccomendations were made)\n",
    "3. In our first run, dataset was 10719 movies huge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np; import pandas as pd\n",
    "from sklearn.metrics.pairwise import cosine_similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['movieID', 'title', 'year', 'genre', 'Drama', 'Comedy', 'Thriller',\n",
       "       'Romance', 'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi',\n",
       "       'Fantasy', 'Children', 'Mystery', 'War', 'Documentary', 'Musical',\n",
       "       'topic_F0', 'topic_F1', 'topic_F2', 'topic_F3', 'topic_F4', 'topic_F5',\n",
       "       'topic_F6', 'topic_F7', 'topic_F8', 'topic_F9', 'topic_F10',\n",
       "       'topic_F11', 'topic_F12', 'topic_F13', 'topic_F14', 'topic_F15',\n",
       "       'topic_F16', 'topic_F17', 'topic_F18', 'topic_F19'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TitleGenreEncodings.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "similarity = cosine_similarity(TitleGenreEncodings[['year', 'Drama', 'Comedy', 'Thriller',\n",
    "       'Romance', 'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi',\n",
    "       'Fantasy', 'Children', 'Mystery', 'War', 'Documentary', 'Musical',\n",
    "       'topic_F0', 'topic_F1', 'topic_F2', 'topic_F3', 'topic_F4', 'topic_F5',\n",
    "       'topic_F6', 'topic_F7', 'topic_F8', 'topic_F9', 'topic_F10',\n",
    "       'topic_F11', 'topic_F12', 'topic_F13', 'topic_F14', 'topic_F15',\n",
    "       'topic_F16', 'topic_F17', 'topic_F18', 'topic_F19']])\n",
    "similaritywithgenre=cosine_similarity(TitleGenreEncodings[['year', 'Drama', 'Comedy', 'Thriller',\n",
    "       'Romance', 'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi',\n",
    "       'Fantasy', 'Children', 'Mystery', 'War', 'Documentary', 'Musical']])\n",
    "#similarity['index_col'] = similarity.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.99999987, 1.        , 0.99999937, ..., 0.99999975, 0.99999922,\n",
       "       0.99999938])"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similaritywithgenre[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([   0, 1490, 2418, 1785, 3226,  759, 6645, 5251, 1709, 3566],\n",
       "      dtype=int64)"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Top_n = 10\n",
    "similarity[0].argsort()[-Top_n:][::-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    }
   ],
   "source": [
    "title = \"Toy story\" \n",
    "movie_id = TitleGenreEncodings[TitleGenreEncodings[\"title\"]==title][\"movieID\"].values[0]\n",
    "print(movie_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([   0, 1490, 2418, 1785, 3226,  759, 6645, 5251, 1709, 3566],\n",
       "      dtype=int64)"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity[0].argsort()[-Top_n:][::-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    Toy story\n",
      "Name: title, dtype: object\n",
      "1490    Blackbeard's Ghost\n",
      "Name: title, dtype: object\n",
      "2418    We're Back! A Dinosaur's Story\n",
      "Name: title, dtype: object\n",
      "1785    The Garbage Pail Kids Movie\n",
      "Name: title, dtype: object\n",
      "3226    The Wizard\n",
      "Name: title, dtype: object\n",
      "759    The Love Bug\n",
      "Name: title, dtype: object\n",
      "6645    Bedtime Stories\n",
      "Name: title, dtype: object\n",
      "5251    The SpongeBob SquarePants Movie\n",
      "Name: title, dtype: object\n",
      "1709    A Bug's Life\n",
      "Name: title, dtype: object\n",
      "3566    Stir Crazy\n",
      "Name: title, dtype: object\n"
     ]
    }
   ],
   "source": [
    "for index in similarity[0].argsort()[-Top_n:][::-1]:\n",
    "    print(TitleGenreEncodings['title'].iloc[[index]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Realization:**<br>\n",
    "1. We can see from the reccomendations that these are all \"Genre-Induced\" movies \n",
    "2. We should add more flavor to our suggestions as Toy story can retrieve a wider array of results\n",
    "3. Retrieving back the same movie might act as a hint that the system is indeed working w.r.t distances\n",
    "4. Evaluation Metric: Qualitative -> User Satisfaction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 1.4: The Next Step**\n",
    "1. Include tags in the mix in the same way as Titles i.e Embeddings\n",
    "2. Calculate Cosine Similarity \n",
    "3. Retrieve Reccomendations\n",
    "\n",
    "**Fun Fact: Inception Model does intermediate predictions to see if it's on the right track too!**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>time travel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>time travel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>32</td>\n",
       "      <td>time travel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>316</td>\n",
       "      <td>time travel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>379</td>\n",
       "      <td>time travel</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID        value\n",
       "0        1  time travel\n",
       "1        2  time travel\n",
       "2       32  time travel\n",
       "3      316  time travel\n",
       "4      379  time travel"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tagtoid=pd.merge(movie_tags[movie_tags['tagWeight']>1], tags, left_on='tagID',right_on='id')\n",
    "tagtoid.drop(['id','tagWeight','tagID'],axis=1,inplace=True)\n",
    "tagtoid.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>time travel, tim allen, comedy, pixar, tom han...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>time travel, animals, fantasy, game, robin wil...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>old</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>family, wedding, pregnancy, childhood classics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>too long, cant remember, recommendz top pick</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                                              value\n",
       "0        1  time travel, tim allen, comedy, pixar, tom han...\n",
       "1        2  time travel, animals, fantasy, game, robin wil...\n",
       "2        3                                                old\n",
       "3        5     family, wedding, pregnancy, childhood classics\n",
       "4        6       too long, cant remember, recommendz top pick"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "moviesandtags = tagtoid.groupby(['movieID'])['value'].apply(', '.join).reset_index()\n",
    "moviesandtags.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Tag_F0', 'Tag_F1', 'Tag_F2', 'Tag_F3', 'Tag_F4', 'Tag_F5', 'Tag_F6', 'Tag_F7', 'Tag_F8', 'Tag_F9', 'Tag_F10', 'Tag_F11', 'Tag_F12', 'Tag_F13', 'Tag_F14', 'Tag_F15', 'Tag_F16', 'Tag_F17', 'Tag_F18', 'Tag_F19']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tag_F0</th>\n",
       "      <th>Tag_F1</th>\n",
       "      <th>Tag_F2</th>\n",
       "      <th>Tag_F3</th>\n",
       "      <th>Tag_F4</th>\n",
       "      <th>Tag_F5</th>\n",
       "      <th>Tag_F6</th>\n",
       "      <th>Tag_F7</th>\n",
       "      <th>Tag_F8</th>\n",
       "      <th>Tag_F9</th>\n",
       "      <th>Tag_F10</th>\n",
       "      <th>Tag_F11</th>\n",
       "      <th>Tag_F12</th>\n",
       "      <th>Tag_F13</th>\n",
       "      <th>Tag_F14</th>\n",
       "      <th>Tag_F15</th>\n",
       "      <th>Tag_F16</th>\n",
       "      <th>Tag_F17</th>\n",
       "      <th>Tag_F18</th>\n",
       "      <th>Tag_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.312752</td>\n",
       "      <td>-0.989468</td>\n",
       "      <td>1.589768</td>\n",
       "      <td>-1.281532</td>\n",
       "      <td>-1.910704</td>\n",
       "      <td>0.796300</td>\n",
       "      <td>-0.003591</td>\n",
       "      <td>-0.998706</td>\n",
       "      <td>1.406521</td>\n",
       "      <td>-1.957082</td>\n",
       "      <td>-2.305140</td>\n",
       "      <td>1.395004</td>\n",
       "      <td>-2.442963</td>\n",
       "      <td>0.191763</td>\n",
       "      <td>-1.348063</td>\n",
       "      <td>-1.315639</td>\n",
       "      <td>-0.387860</td>\n",
       "      <td>0.767467</td>\n",
       "      <td>-0.641286</td>\n",
       "      <td>-0.789096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.062346</td>\n",
       "      <td>-0.810401</td>\n",
       "      <td>0.826755</td>\n",
       "      <td>-0.839619</td>\n",
       "      <td>-1.097751</td>\n",
       "      <td>-0.621831</td>\n",
       "      <td>0.295618</td>\n",
       "      <td>-0.719280</td>\n",
       "      <td>-0.783297</td>\n",
       "      <td>-0.175457</td>\n",
       "      <td>-1.305755</td>\n",
       "      <td>1.597886</td>\n",
       "      <td>-0.426935</td>\n",
       "      <td>0.531539</td>\n",
       "      <td>-1.583068</td>\n",
       "      <td>0.790853</td>\n",
       "      <td>1.668663</td>\n",
       "      <td>-0.273516</td>\n",
       "      <td>-1.036308</td>\n",
       "      <td>0.179430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.430629</td>\n",
       "      <td>-0.688265</td>\n",
       "      <td>-0.656174</td>\n",
       "      <td>-0.665727</td>\n",
       "      <td>-0.374031</td>\n",
       "      <td>-0.431358</td>\n",
       "      <td>-0.325455</td>\n",
       "      <td>0.029263</td>\n",
       "      <td>0.655504</td>\n",
       "      <td>-0.298397</td>\n",
       "      <td>-0.430264</td>\n",
       "      <td>0.918680</td>\n",
       "      <td>-0.669077</td>\n",
       "      <td>-0.373421</td>\n",
       "      <td>-1.102863</td>\n",
       "      <td>0.264218</td>\n",
       "      <td>-0.688483</td>\n",
       "      <td>-0.140395</td>\n",
       "      <td>-0.196410</td>\n",
       "      <td>0.008075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.776832</td>\n",
       "      <td>-0.550043</td>\n",
       "      <td>0.194645</td>\n",
       "      <td>-1.342958</td>\n",
       "      <td>0.245670</td>\n",
       "      <td>0.752209</td>\n",
       "      <td>-0.917581</td>\n",
       "      <td>1.287840</td>\n",
       "      <td>2.572019</td>\n",
       "      <td>-0.306152</td>\n",
       "      <td>-1.888412</td>\n",
       "      <td>0.467941</td>\n",
       "      <td>-1.613994</td>\n",
       "      <td>-0.352356</td>\n",
       "      <td>-2.164545</td>\n",
       "      <td>-1.088279</td>\n",
       "      <td>0.236017</td>\n",
       "      <td>-1.008116</td>\n",
       "      <td>-0.895948</td>\n",
       "      <td>-0.566081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.148579</td>\n",
       "      <td>-1.191658</td>\n",
       "      <td>-0.130787</td>\n",
       "      <td>0.049383</td>\n",
       "      <td>-2.072951</td>\n",
       "      <td>-0.534014</td>\n",
       "      <td>-1.227999</td>\n",
       "      <td>0.463436</td>\n",
       "      <td>0.273136</td>\n",
       "      <td>-0.119534</td>\n",
       "      <td>-1.022909</td>\n",
       "      <td>0.657264</td>\n",
       "      <td>0.741594</td>\n",
       "      <td>0.132725</td>\n",
       "      <td>-1.462465</td>\n",
       "      <td>0.829166</td>\n",
       "      <td>2.123933</td>\n",
       "      <td>-0.633606</td>\n",
       "      <td>-1.581831</td>\n",
       "      <td>-0.732403</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Tag_F0    Tag_F1    Tag_F2    Tag_F3    Tag_F4    Tag_F5    Tag_F6  \\\n",
       "0  2.312752 -0.989468  1.589768 -1.281532 -1.910704  0.796300 -0.003591   \n",
       "1 -0.062346 -0.810401  0.826755 -0.839619 -1.097751 -0.621831  0.295618   \n",
       "2  0.430629 -0.688265 -0.656174 -0.665727 -0.374031 -0.431358 -0.325455   \n",
       "3  0.776832 -0.550043  0.194645 -1.342958  0.245670  0.752209 -0.917581   \n",
       "4  0.148579 -1.191658 -0.130787  0.049383 -2.072951 -0.534014 -1.227999   \n",
       "\n",
       "     Tag_F7    Tag_F8    Tag_F9   Tag_F10   Tag_F11   Tag_F12   Tag_F13  \\\n",
       "0 -0.998706  1.406521 -1.957082 -2.305140  1.395004 -2.442963  0.191763   \n",
       "1 -0.719280 -0.783297 -0.175457 -1.305755  1.597886 -0.426935  0.531539   \n",
       "2  0.029263  0.655504 -0.298397 -0.430264  0.918680 -0.669077 -0.373421   \n",
       "3  1.287840  2.572019 -0.306152 -1.888412  0.467941 -1.613994 -0.352356   \n",
       "4  0.463436  0.273136 -0.119534 -1.022909  0.657264  0.741594  0.132725   \n",
       "\n",
       "    Tag_F14   Tag_F15   Tag_F16   Tag_F17   Tag_F18   Tag_F19  \n",
       "0 -1.348063 -1.315639 -0.387860  0.767467 -0.641286 -0.789096  \n",
       "1 -1.583068  0.790853  1.668663 -0.273516 -1.036308  0.179430  \n",
       "2 -1.102863  0.264218 -0.688483 -0.140395 -0.196410  0.008075  \n",
       "3 -2.164545 -1.088279  0.236017 -1.008116 -0.895948 -0.566081  \n",
       "4 -1.462465  0.829166  2.123933 -0.633606 -1.581831 -0.732403  "
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TagEmbeddings=embed(moviesandtags['value']).numpy()\n",
    "names=[\"Tag_F\"+str(x) for x in range(20)]\n",
    "print(names)\n",
    "TagExtractedData=pd.DataFrame(TagEmbeddings,columns=names)\n",
    "TagExtractedData.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2963, 20)"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TagExtractedData.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2963, 20)"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TagEmbeddings.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>value</th>\n",
       "      <th>Tag_F0</th>\n",
       "      <th>Tag_F1</th>\n",
       "      <th>Tag_F2</th>\n",
       "      <th>Tag_F3</th>\n",
       "      <th>Tag_F4</th>\n",
       "      <th>Tag_F5</th>\n",
       "      <th>Tag_F6</th>\n",
       "      <th>Tag_F7</th>\n",
       "      <th>...</th>\n",
       "      <th>Tag_F10</th>\n",
       "      <th>Tag_F11</th>\n",
       "      <th>Tag_F12</th>\n",
       "      <th>Tag_F13</th>\n",
       "      <th>Tag_F14</th>\n",
       "      <th>Tag_F15</th>\n",
       "      <th>Tag_F16</th>\n",
       "      <th>Tag_F17</th>\n",
       "      <th>Tag_F18</th>\n",
       "      <th>Tag_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>time travel, tim allen, comedy, pixar, tom han...</td>\n",
       "      <td>2.312752</td>\n",
       "      <td>-0.989468</td>\n",
       "      <td>1.589768</td>\n",
       "      <td>-1.281532</td>\n",
       "      <td>-1.910704</td>\n",
       "      <td>0.796300</td>\n",
       "      <td>-0.003591</td>\n",
       "      <td>-0.998706</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.305140</td>\n",
       "      <td>1.395004</td>\n",
       "      <td>-2.442963</td>\n",
       "      <td>0.191763</td>\n",
       "      <td>-1.348063</td>\n",
       "      <td>-1.315639</td>\n",
       "      <td>-0.387860</td>\n",
       "      <td>0.767467</td>\n",
       "      <td>-0.641286</td>\n",
       "      <td>-0.789096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>time travel, animals, fantasy, game, robin wil...</td>\n",
       "      <td>-0.062346</td>\n",
       "      <td>-0.810401</td>\n",
       "      <td>0.826755</td>\n",
       "      <td>-0.839619</td>\n",
       "      <td>-1.097751</td>\n",
       "      <td>-0.621831</td>\n",
       "      <td>0.295618</td>\n",
       "      <td>-0.719280</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.305755</td>\n",
       "      <td>1.597886</td>\n",
       "      <td>-0.426935</td>\n",
       "      <td>0.531539</td>\n",
       "      <td>-1.583068</td>\n",
       "      <td>0.790853</td>\n",
       "      <td>1.668663</td>\n",
       "      <td>-0.273516</td>\n",
       "      <td>-1.036308</td>\n",
       "      <td>0.179430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>old</td>\n",
       "      <td>0.430629</td>\n",
       "      <td>-0.688265</td>\n",
       "      <td>-0.656174</td>\n",
       "      <td>-0.665727</td>\n",
       "      <td>-0.374031</td>\n",
       "      <td>-0.431358</td>\n",
       "      <td>-0.325455</td>\n",
       "      <td>0.029263</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.430264</td>\n",
       "      <td>0.918680</td>\n",
       "      <td>-0.669077</td>\n",
       "      <td>-0.373421</td>\n",
       "      <td>-1.102863</td>\n",
       "      <td>0.264218</td>\n",
       "      <td>-0.688483</td>\n",
       "      <td>-0.140395</td>\n",
       "      <td>-0.196410</td>\n",
       "      <td>0.008075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>family, wedding, pregnancy, childhood classics</td>\n",
       "      <td>0.776832</td>\n",
       "      <td>-0.550043</td>\n",
       "      <td>0.194645</td>\n",
       "      <td>-1.342958</td>\n",
       "      <td>0.245670</td>\n",
       "      <td>0.752209</td>\n",
       "      <td>-0.917581</td>\n",
       "      <td>1.287840</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.888412</td>\n",
       "      <td>0.467941</td>\n",
       "      <td>-1.613994</td>\n",
       "      <td>-0.352356</td>\n",
       "      <td>-2.164545</td>\n",
       "      <td>-1.088279</td>\n",
       "      <td>0.236017</td>\n",
       "      <td>-1.008116</td>\n",
       "      <td>-0.895948</td>\n",
       "      <td>-0.566081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>too long, cant remember, recommendz top pick</td>\n",
       "      <td>0.148579</td>\n",
       "      <td>-1.191658</td>\n",
       "      <td>-0.130787</td>\n",
       "      <td>0.049383</td>\n",
       "      <td>-2.072951</td>\n",
       "      <td>-0.534014</td>\n",
       "      <td>-1.227999</td>\n",
       "      <td>0.463436</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.022909</td>\n",
       "      <td>0.657264</td>\n",
       "      <td>0.741594</td>\n",
       "      <td>0.132725</td>\n",
       "      <td>-1.462465</td>\n",
       "      <td>0.829166</td>\n",
       "      <td>2.123933</td>\n",
       "      <td>-0.633606</td>\n",
       "      <td>-1.581831</td>\n",
       "      <td>-0.732403</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                                              value    Tag_F0  \\\n",
       "0        1  time travel, tim allen, comedy, pixar, tom han...  2.312752   \n",
       "1        2  time travel, animals, fantasy, game, robin wil... -0.062346   \n",
       "2        3                                                old  0.430629   \n",
       "3        5     family, wedding, pregnancy, childhood classics  0.776832   \n",
       "4        6       too long, cant remember, recommendz top pick  0.148579   \n",
       "\n",
       "     Tag_F1    Tag_F2    Tag_F3    Tag_F4    Tag_F5    Tag_F6    Tag_F7  ...  \\\n",
       "0 -0.989468  1.589768 -1.281532 -1.910704  0.796300 -0.003591 -0.998706  ...   \n",
       "1 -0.810401  0.826755 -0.839619 -1.097751 -0.621831  0.295618 -0.719280  ...   \n",
       "2 -0.688265 -0.656174 -0.665727 -0.374031 -0.431358 -0.325455  0.029263  ...   \n",
       "3 -0.550043  0.194645 -1.342958  0.245670  0.752209 -0.917581  1.287840  ...   \n",
       "4 -1.191658 -0.130787  0.049383 -2.072951 -0.534014 -1.227999  0.463436  ...   \n",
       "\n",
       "    Tag_F10   Tag_F11   Tag_F12   Tag_F13   Tag_F14   Tag_F15   Tag_F16  \\\n",
       "0 -2.305140  1.395004 -2.442963  0.191763 -1.348063 -1.315639 -0.387860   \n",
       "1 -1.305755  1.597886 -0.426935  0.531539 -1.583068  0.790853  1.668663   \n",
       "2 -0.430264  0.918680 -0.669077 -0.373421 -1.102863  0.264218 -0.688483   \n",
       "3 -1.888412  0.467941 -1.613994 -0.352356 -2.164545 -1.088279  0.236017   \n",
       "4 -1.022909  0.657264  0.741594  0.132725 -1.462465  0.829166  2.123933   \n",
       "\n",
       "    Tag_F17   Tag_F18   Tag_F19  \n",
       "0  0.767467 -0.641286 -0.789096  \n",
       "1 -0.273516 -1.036308  0.179430  \n",
       "2 -0.140395 -0.196410  0.008075  \n",
       "3 -1.008116 -0.895948 -0.566081  \n",
       "4 -0.633606 -1.581831 -0.732403  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TagEmbedDF=pd.concat([moviesandtags,TagExtractedData],axis=1)\n",
    "TagEmbedDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boolean = TitleGenreEncodings['title'].duplicated().any()\n",
    "boolean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>value</th>\n",
       "      <th>Tag_F0</th>\n",
       "      <th>Tag_F1</th>\n",
       "      <th>Tag_F2</th>\n",
       "      <th>Tag_F3</th>\n",
       "      <th>Tag_F4</th>\n",
       "      <th>Tag_F5</th>\n",
       "      <th>Tag_F6</th>\n",
       "      <th>Tag_F7</th>\n",
       "      <th>...</th>\n",
       "      <th>Tag_F10</th>\n",
       "      <th>Tag_F11</th>\n",
       "      <th>Tag_F12</th>\n",
       "      <th>Tag_F13</th>\n",
       "      <th>Tag_F14</th>\n",
       "      <th>Tag_F15</th>\n",
       "      <th>Tag_F16</th>\n",
       "      <th>Tag_F17</th>\n",
       "      <th>Tag_F18</th>\n",
       "      <th>Tag_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>time travel, tim allen, comedy, pixar, tom han...</td>\n",
       "      <td>2.312752</td>\n",
       "      <td>-0.989468</td>\n",
       "      <td>1.589768</td>\n",
       "      <td>-1.281532</td>\n",
       "      <td>-1.910704</td>\n",
       "      <td>0.796300</td>\n",
       "      <td>-0.003591</td>\n",
       "      <td>-0.998706</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.305140</td>\n",
       "      <td>1.395004</td>\n",
       "      <td>-2.442963</td>\n",
       "      <td>0.191763</td>\n",
       "      <td>-1.348063</td>\n",
       "      <td>-1.315639</td>\n",
       "      <td>-0.387860</td>\n",
       "      <td>0.767467</td>\n",
       "      <td>-0.641286</td>\n",
       "      <td>-0.789096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>time travel, animals, fantasy, game, robin wil...</td>\n",
       "      <td>-0.062346</td>\n",
       "      <td>-0.810401</td>\n",
       "      <td>0.826755</td>\n",
       "      <td>-0.839619</td>\n",
       "      <td>-1.097751</td>\n",
       "      <td>-0.621831</td>\n",
       "      <td>0.295618</td>\n",
       "      <td>-0.719280</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.305755</td>\n",
       "      <td>1.597886</td>\n",
       "      <td>-0.426935</td>\n",
       "      <td>0.531539</td>\n",
       "      <td>-1.583068</td>\n",
       "      <td>0.790853</td>\n",
       "      <td>1.668663</td>\n",
       "      <td>-0.273516</td>\n",
       "      <td>-1.036308</td>\n",
       "      <td>0.179430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>old</td>\n",
       "      <td>0.430629</td>\n",
       "      <td>-0.688265</td>\n",
       "      <td>-0.656174</td>\n",
       "      <td>-0.665727</td>\n",
       "      <td>-0.374031</td>\n",
       "      <td>-0.431358</td>\n",
       "      <td>-0.325455</td>\n",
       "      <td>0.029263</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.430264</td>\n",
       "      <td>0.918680</td>\n",
       "      <td>-0.669077</td>\n",
       "      <td>-0.373421</td>\n",
       "      <td>-1.102863</td>\n",
       "      <td>0.264218</td>\n",
       "      <td>-0.688483</td>\n",
       "      <td>-0.140395</td>\n",
       "      <td>-0.196410</td>\n",
       "      <td>0.008075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>family, wedding, pregnancy, childhood classics</td>\n",
       "      <td>0.776832</td>\n",
       "      <td>-0.550043</td>\n",
       "      <td>0.194645</td>\n",
       "      <td>-1.342958</td>\n",
       "      <td>0.245670</td>\n",
       "      <td>0.752209</td>\n",
       "      <td>-0.917581</td>\n",
       "      <td>1.287840</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.888412</td>\n",
       "      <td>0.467941</td>\n",
       "      <td>-1.613994</td>\n",
       "      <td>-0.352356</td>\n",
       "      <td>-2.164545</td>\n",
       "      <td>-1.088279</td>\n",
       "      <td>0.236017</td>\n",
       "      <td>-1.008116</td>\n",
       "      <td>-0.895948</td>\n",
       "      <td>-0.566081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>too long, cant remember, recommendz top pick</td>\n",
       "      <td>0.148579</td>\n",
       "      <td>-1.191658</td>\n",
       "      <td>-0.130787</td>\n",
       "      <td>0.049383</td>\n",
       "      <td>-2.072951</td>\n",
       "      <td>-0.534014</td>\n",
       "      <td>-1.227999</td>\n",
       "      <td>0.463436</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.022909</td>\n",
       "      <td>0.657264</td>\n",
       "      <td>0.741594</td>\n",
       "      <td>0.132725</td>\n",
       "      <td>-1.462465</td>\n",
       "      <td>0.829166</td>\n",
       "      <td>2.123933</td>\n",
       "      <td>-0.633606</td>\n",
       "      <td>-1.581831</td>\n",
       "      <td>-0.732403</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID                                              value    Tag_F0  \\\n",
       "0        1  time travel, tim allen, comedy, pixar, tom han...  2.312752   \n",
       "1        2  time travel, animals, fantasy, game, robin wil... -0.062346   \n",
       "2        3                                                old  0.430629   \n",
       "3        5     family, wedding, pregnancy, childhood classics  0.776832   \n",
       "4        6       too long, cant remember, recommendz top pick  0.148579   \n",
       "\n",
       "     Tag_F1    Tag_F2    Tag_F3    Tag_F4    Tag_F5    Tag_F6    Tag_F7  ...  \\\n",
       "0 -0.989468  1.589768 -1.281532 -1.910704  0.796300 -0.003591 -0.998706  ...   \n",
       "1 -0.810401  0.826755 -0.839619 -1.097751 -0.621831  0.295618 -0.719280  ...   \n",
       "2 -0.688265 -0.656174 -0.665727 -0.374031 -0.431358 -0.325455  0.029263  ...   \n",
       "3 -0.550043  0.194645 -1.342958  0.245670  0.752209 -0.917581  1.287840  ...   \n",
       "4 -1.191658 -0.130787  0.049383 -2.072951 -0.534014 -1.227999  0.463436  ...   \n",
       "\n",
       "    Tag_F10   Tag_F11   Tag_F12   Tag_F13   Tag_F14   Tag_F15   Tag_F16  \\\n",
       "0 -2.305140  1.395004 -2.442963  0.191763 -1.348063 -1.315639 -0.387860   \n",
       "1 -1.305755  1.597886 -0.426935  0.531539 -1.583068  0.790853  1.668663   \n",
       "2 -0.430264  0.918680 -0.669077 -0.373421 -1.102863  0.264218 -0.688483   \n",
       "3 -1.888412  0.467941 -1.613994 -0.352356 -2.164545 -1.088279  0.236017   \n",
       "4 -1.022909  0.657264  0.741594  0.132725 -1.462465  0.829166  2.123933   \n",
       "\n",
       "    Tag_F17   Tag_F18   Tag_F19  \n",
       "0  0.767467 -0.641286 -0.789096  \n",
       "1 -0.273516 -1.036308  0.179430  \n",
       "2 -0.140395 -0.196410  0.008075  \n",
       "3 -1.008116 -0.895948 -0.566081  \n",
       "4 -0.633606 -1.581831 -0.732403  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TagEmbedDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "TitleGenreTagsEncoding=pd.merge(TitleGenreEncodings, TagEmbedDF, on='movieID',how='left')\n",
    "#TitleGenreTagsEncoding.to_csv(\"Test.csv\",index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "#as join is left-based, our tag embeds arent really present for all movies(a hint at popularity being a factor), we replace:\n",
    "TitleGenreTagsEncoding = TitleGenreTagsEncoding.replace(np.nan, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "      <th>genre</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Romance</th>\n",
       "      <th>Action</th>\n",
       "      <th>Crime</th>\n",
       "      <th>...</th>\n",
       "      <th>Tag_F10</th>\n",
       "      <th>Tag_F11</th>\n",
       "      <th>Tag_F12</th>\n",
       "      <th>Tag_F13</th>\n",
       "      <th>Tag_F14</th>\n",
       "      <th>Tag_F15</th>\n",
       "      <th>Tag_F16</th>\n",
       "      <th>Tag_F17</th>\n",
       "      <th>Tag_F18</th>\n",
       "      <th>Tag_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6643</th>\n",
       "      <td>64993</td>\n",
       "      <td>Byôsoku 5 senchimêtoru</td>\n",
       "      <td>2007</td>\n",
       "      <td>Animation, Drama, Romance</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.711675</td>\n",
       "      <td>0.266793</td>\n",
       "      <td>-0.783149</td>\n",
       "      <td>-0.00431</td>\n",
       "      <td>-0.493212</td>\n",
       "      <td>-0.535885</td>\n",
       "      <td>-1.059188</td>\n",
       "      <td>0.381155</td>\n",
       "      <td>-0.002112</td>\n",
       "      <td>0.414874</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6644</th>\n",
       "      <td>65037</td>\n",
       "      <td>Ben X</td>\n",
       "      <td>2007</td>\n",
       "      <td>Drama</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6645</th>\n",
       "      <td>65088</td>\n",
       "      <td>Bedtime Stories</td>\n",
       "      <td>2008</td>\n",
       "      <td>Adventure, Children, Comedy</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6646</th>\n",
       "      <td>65091</td>\n",
       "      <td>Manhattan Melodrama</td>\n",
       "      <td>1934</td>\n",
       "      <td>Crime, Drama, Romance</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6647</th>\n",
       "      <td>65126</td>\n",
       "      <td>Choke</td>\n",
       "      <td>2008</td>\n",
       "      <td>Comedy, Drama</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 60 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      movieID                   title  year                        genre  \\\n",
       "6643    64993  Byôsoku 5 senchimêtoru  2007    Animation, Drama, Romance   \n",
       "6644    65037                   Ben X  2007                        Drama   \n",
       "6645    65088         Bedtime Stories  2008  Adventure, Children, Comedy   \n",
       "6646    65091     Manhattan Melodrama  1934        Crime, Drama, Romance   \n",
       "6647    65126                   Choke  2008                Comedy, Drama   \n",
       "\n",
       "      Drama  Comedy  Thriller  Romance  Action  Crime  ...   Tag_F10  \\\n",
       "6643      1       0         0        1       0      0  ... -0.711675   \n",
       "6644      1       0         0        0       0      0  ...  0.000000   \n",
       "6645      0       1         0        0       0      0  ...  0.000000   \n",
       "6646      1       0         0        1       0      1  ...  0.000000   \n",
       "6647      1       1         0        0       0      0  ...  0.000000   \n",
       "\n",
       "       Tag_F11   Tag_F12  Tag_F13   Tag_F14   Tag_F15   Tag_F16   Tag_F17  \\\n",
       "6643  0.266793 -0.783149 -0.00431 -0.493212 -0.535885 -1.059188  0.381155   \n",
       "6644  0.000000  0.000000  0.00000  0.000000  0.000000  0.000000  0.000000   \n",
       "6645  0.000000  0.000000  0.00000  0.000000  0.000000  0.000000  0.000000   \n",
       "6646  0.000000  0.000000  0.00000  0.000000  0.000000  0.000000  0.000000   \n",
       "6647  0.000000  0.000000  0.00000  0.000000  0.000000  0.000000  0.000000   \n",
       "\n",
       "       Tag_F18   Tag_F19  \n",
       "6643 -0.002112  0.414874  \n",
       "6644  0.000000  0.000000  \n",
       "6645  0.000000  0.000000  \n",
       "6646  0.000000  0.000000  \n",
       "6647  0.000000  0.000000  \n",
       "\n",
       "[5 rows x 60 columns]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TitleGenreTagsEncoding.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['movieID', 'title', 'year', 'genre', 'Drama', 'Comedy', 'Thriller',\n",
       "       'Romance', 'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi',\n",
       "       'Fantasy', 'Children', 'Mystery', 'War', 'Documentary', 'Musical',\n",
       "       'topic_F0', 'topic_F1', 'topic_F2', 'topic_F3', 'topic_F4', 'topic_F5',\n",
       "       'topic_F6', 'topic_F7', 'topic_F8', 'topic_F9', 'topic_F10',\n",
       "       'topic_F11', 'topic_F12', 'topic_F13', 'topic_F14', 'topic_F15',\n",
       "       'topic_F16', 'topic_F17', 'topic_F18', 'topic_F19', 'value', 'Tag_F0',\n",
       "       'Tag_F1', 'Tag_F2', 'Tag_F3', 'Tag_F4', 'Tag_F5', 'Tag_F6', 'Tag_F7',\n",
       "       'Tag_F8', 'Tag_F9', 'Tag_F10', 'Tag_F11', 'Tag_F12', 'Tag_F13',\n",
       "       'Tag_F14', 'Tag_F15', 'Tag_F16', 'Tag_F17', 'Tag_F18', 'Tag_F19'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TitleGenreTagsEncoding.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Cosine Similarity with respect to Tags, Genres, and Titles**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "similarity = cosine_similarity(TitleGenreTagsEncoding[['year', 'Drama', 'Comedy', 'Thriller',\n",
    "       'Romance', 'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi',\n",
    "       'Fantasy', 'Children', 'Mystery', 'War', 'Documentary', 'Musical',\n",
    "       'topic_F0', 'topic_F1', 'topic_F2', 'topic_F3', 'topic_F4', 'topic_F5',\n",
    "       'topic_F6', 'topic_F7', 'topic_F8', 'topic_F9', 'topic_F10',\n",
    "       'topic_F11', 'topic_F12', 'topic_F13', 'topic_F14', 'topic_F15',\n",
    "       'topic_F16', 'topic_F17', 'topic_F18', 'topic_F19', 'Tag_F0',\n",
    "       'Tag_F1', 'Tag_F2', 'Tag_F3', 'Tag_F4', 'Tag_F5', 'Tag_F6', 'Tag_F7',\n",
    "       'Tag_F8', 'Tag_F9', 'Tag_F10', 'Tag_F11', 'Tag_F12', 'Tag_F13',\n",
    "       'Tag_F14', 'Tag_F15', 'Tag_F16', 'Tag_F17', 'Tag_F18', 'Tag_F19']])\n",
    "#similarity['index_col'] = similarity.index\n",
    "similaritywithGenreTags=cosine_similarity(TitleGenreTagsEncoding[['year', 'Drama', 'Comedy', 'Thriller',\n",
    "       'Romance', 'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi',\n",
    "       'Fantasy', 'Children', 'Mystery', 'War', 'Documentary', 'Musical', 'Tag_F0',\n",
    "       'Tag_F1', 'Tag_F2', 'Tag_F3', 'Tag_F4', 'Tag_F5', 'Tag_F6', 'Tag_F7',\n",
    "       'Tag_F8', 'Tag_F9', 'Tag_F10', 'Tag_F11', 'Tag_F12', 'Tag_F13',\n",
    "       'Tag_F14', 'Tag_F15', 'Tag_F16', 'Tag_F17', 'Tag_F18', 'Tag_F19']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.        , 0.99999388, 0.99999437, ..., 0.99999421, 0.99999167,\n",
       "       0.99999265])"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "0    Toy story\n",
      "Name: title, dtype: object\n",
      "5759    Wallace & Gromit in The Curse of the Were-Rabbit\n",
      "Name: title, dtype: object\n",
      "1542    The Lord of the Rings: The Return of the King\n",
      "Name: title, dtype: object\n",
      "6235    The Simpsons Movie\n",
      "Name: title, dtype: object\n",
      "2026    A Christmas Story\n",
      "Name: title, dtype: object\n",
      "1709    A Bug's Life\n",
      "Name: title, dtype: object\n",
      "474    The Nightmare Before Christmas\n",
      "Name: title, dtype: object\n",
      "503    Beauty and the Beast\n",
      "Name: title, dtype: object\n",
      "129    Batman Forever\n",
      "Name: title, dtype: object\n",
      "5674    Batman Begins\n",
      "Name: title, dtype: object\n"
     ]
    }
   ],
   "source": [
    "Top_n = 10\n",
    "similarity[0].argsort()[-Top_n:][::-1]\n",
    "title = \"Toy story\" \n",
    "movie_id = TitleGenreTagsEncoding[TitleGenreTagsEncoding[\"title\"]==title][\"movieID\"].values[0]\n",
    "print(movie_id)\n",
    "for index in similarity[0].argsort()[-Top_n:][::-1]:\n",
    "    print(TitleGenreTagsEncoding['title'].iloc[[index]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Realization:**<br>\n",
    "1. Now, more factors have definetly come into play\n",
    "2. More Popular titles coming together because 0 for less famous title pushes them away\n",
    "3. As acquired with domain knowledge, I have watched 90% of movies suggested here w.r.t Toy Story and love this!\n",
    "4. Suggestions have been borrowed from friends and they prefer this over the previous one already! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 1.5 Content Based Movie Reccomendations: with Metadeta**\n",
    "1. Actor, Director, Country **->** merged with Titles, Genres, Tags \n",
    "2. Countries: USA **-** Not USA matrix\n",
    "3. Actor, Director **->** embeddings\n",
    "4. Cosine Similarity \n",
    "5. Update: With and Without Title Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorID</th>\n",
       "      <th>actorName</th>\n",
       "      <th>ranking</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>tom_hanks</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>2</td>\n",
       "      <td>robin_williams</td>\n",
       "      <td>Robin Williams</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>3</td>\n",
       "      <td>jack_lemmon</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>4</td>\n",
       "      <td>whitney_houston</td>\n",
       "      <td>Whitney Houston</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>5</td>\n",
       "      <td>steve_martin</td>\n",
       "      <td>Steve Martin</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     movieID          actorID        actorName  ranking\n",
       "22         1        tom_hanks        Tom Hanks        1\n",
       "41         2   robin_williams   Robin Williams        1\n",
       "49         3      jack_lemmon      Jack Lemmon        1\n",
       "76         4  whitney_houston  Whitney Houston        1\n",
       "100        5     steve_martin     Steve Martin        1"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Filter out the actors based on their apperance rank\n",
    "df=movie_actors[movie_actors['ranking']<=1]\n",
    "df['actorName'].unique().shape\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Robin Williams</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Whitney Houston</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Steve Martin</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID        actorName\n",
       "0        1        Tom Hanks\n",
       "1        2   Robin Williams\n",
       "2        3      Jack Lemmon\n",
       "3        4  Whitney Houston\n",
       "4        5     Steve Martin"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#join movies and actors\n",
    "movie_actors['actorName']=movie_actors['actorName'].astype(str)\n",
    "moviesandactors = df.groupby(['movieID'])['actorName'].apply(', '.join).reset_index()\n",
    "moviesandactors.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>directorName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>John Lasseter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Joe Johnston</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Donald Petrie</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Charles Shyer</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID     directorName\n",
       "0        1    John Lasseter\n",
       "1        2     Joe Johnston\n",
       "2        3    Donald Petrie\n",
       "3        4  Forest Whitaker\n",
       "4        5    Charles Shyer"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_directors['directorName']=movie_directors['directorName'].astype(str)\n",
    "moviesanddirectors = movie_directors.groupby(['movieID'])['directorName'].apply(', '.join).reset_index()\n",
    "moviesanddirectors.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "actorsanddirectors=pd.merge(moviesandactors,moviesanddirectors,on='movieID',how='inner')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10134,)"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actorsanddirectors['movieID'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName</th>\n",
       "      <th>directorName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>150</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Ron Howard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>343</th>\n",
       "      <td>356</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Robert Zemeckis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>487</th>\n",
       "      <td>508</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Jonathan Demme</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>518</th>\n",
       "      <td>539</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Nora Ephron</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1618</th>\n",
       "      <td>1827</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Penny Marshall</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1812</th>\n",
       "      <td>2028</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Steven Spielberg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1856</th>\n",
       "      <td>2072</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Joe Dante</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1884</th>\n",
       "      <td>2100</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Ron Howard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2189</th>\n",
       "      <td>2418</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Garry Marshall</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2195</th>\n",
       "      <td>2424</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Nora Ephron</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2551</th>\n",
       "      <td>2797</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Penny Marshall</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2857</th>\n",
       "      <td>3114</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2872</th>\n",
       "      <td>3130</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Brian DePalma</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2889</th>\n",
       "      <td>3147</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Frank Darabont</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3100</th>\n",
       "      <td>3385</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Nicholas Meyer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3233</th>\n",
       "      <td>3525</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Neal Israel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3706</th>\n",
       "      <td>4022</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Robert Zemeckis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4342</th>\n",
       "      <td>4677</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Roger Spottiswoode</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4584</th>\n",
       "      <td>4929</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4946</th>\n",
       "      <td>5303</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Patrick Shanley</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5102</th>\n",
       "      <td>5464</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Sam Mendes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5232</th>\n",
       "      <td>5602</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Joel Coen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6919</th>\n",
       "      <td>7367</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Joel Coen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7495</th>\n",
       "      <td>8529</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Steven Spielberg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7800</th>\n",
       "      <td>8884</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Stan Dragoti</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7876</th>\n",
       "      <td>8965</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Robert Zemeckis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9078</th>\n",
       "      <td>45447</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Ron Howard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9667</th>\n",
       "      <td>56788</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>Mike Nichols</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      movieID  actorName          directorName\n",
       "0           1  Tom Hanks         John Lasseter\n",
       "141       150  Tom Hanks            Ron Howard\n",
       "343       356  Tom Hanks       Robert Zemeckis\n",
       "487       508  Tom Hanks        Jonathan Demme\n",
       "518       539  Tom Hanks           Nora Ephron\n",
       "1618     1827  Tom Hanks        Penny Marshall\n",
       "1812     2028  Tom Hanks      Steven Spielberg\n",
       "1856     2072  Tom Hanks             Joe Dante\n",
       "1884     2100  Tom Hanks            Ron Howard\n",
       "2189     2418  Tom Hanks        Garry Marshall\n",
       "2195     2424  Tom Hanks           Nora Ephron\n",
       "2551     2797  Tom Hanks        Penny Marshall\n",
       "2857     3114  Tom Hanks         John Lasseter\n",
       "2872     3130  Tom Hanks         Brian DePalma\n",
       "2889     3147  Tom Hanks        Frank Darabont\n",
       "3100     3385  Tom Hanks        Nicholas Meyer\n",
       "3233     3525  Tom Hanks           Neal Israel\n",
       "3706     4022  Tom Hanks       Robert Zemeckis\n",
       "4342     4677  Tom Hanks    Roger Spottiswoode\n",
       "4584     4929  Tom Hanks         John Lasseter\n",
       "4946     5303  Tom Hanks  John Patrick Shanley\n",
       "5102     5464  Tom Hanks            Sam Mendes\n",
       "5232     5602  Tom Hanks             Joel Coen\n",
       "6919     7367  Tom Hanks             Joel Coen\n",
       "7495     8529  Tom Hanks      Steven Spielberg\n",
       "7800     8884  Tom Hanks          Stan Dragoti\n",
       "7876     8965  Tom Hanks       Robert Zemeckis\n",
       "9078    45447  Tom Hanks            Ron Howard\n",
       "9667    56788  Tom Hanks          Mike Nichols"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actorsanddirectors[actorsanddirectors['actorName']=='Tom Hanks']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**This goes without saying Tom Hanks has starred in multiple movies alongside multiple directors and might be some users favorite**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "print(movie_countries['country'].isnull().values.any())\n",
    "movie_countries['country'].isnull().sum()\n",
    "movie_countries=movie_countries.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_countries[\"country\"]= movie_countries[\"country\"].replace(\"USA\", 1)\n",
    "movie_countries[\"country\"] = movie_countries[\"country\"].mask(movie_countries[\"country\"] !=1, 0)\n",
    "#movie_countries.to_csv(\"TestMovieData.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "#movie_countries.apply(LabelEncoder().fit_transform(movie_countries['country']))\n",
    "#movie_countries['country'] =LabelEncoder().fit_transform(movie_countries['country'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     10189\n",
       "unique        2\n",
       "top           1\n",
       "freq       6831\n",
       "Name: country, dtype: int64"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_countries['country'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Here, we are breaking down the countries data to USA-NOTUSA as nearly 70% of movies are from USA**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>USA</th>\n",
       "      <th>NUSA</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10184</th>\n",
       "      <td>65088</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10185</th>\n",
       "      <td>65091</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10186</th>\n",
       "      <td>65126</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10187</th>\n",
       "      <td>65130</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10188</th>\n",
       "      <td>65133</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       movieID USA  NUSA\n",
       "10184    65088   1     0\n",
       "10185    65091   1     0\n",
       "10186    65126   1     0\n",
       "10187    65130   1     0\n",
       "10188    65133   0     1"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_countriesUSA=movie_countries.copy(deep=True)\n",
    "movie_countriesNUSA=movie_countries.copy(deep=True)\n",
    "movie_countriesUSA.columns=['movieID','USA']\n",
    "movie_countriesNUSA.columns=['movieID','NUSA']\n",
    "movie_countriesNUSA['NUSA'] = movie_countriesNUSA['NUSA'].replace({0:1, 1:0})\n",
    "#movie_countriesNUSA\n",
    "movie_countriesFinale=pd.merge(movie_countriesUSA,movie_countriesNUSA,on='movieID')\n",
    "movie_countriesFinale.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName</th>\n",
       "      <th>directorName</th>\n",
       "      <th>USA</th>\n",
       "      <th>NUSA</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Robin Williams</td>\n",
       "      <td>Joe Johnston</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "      <td>Donald Petrie</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Whitney Houston</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Steve Martin</td>\n",
       "      <td>Charles Shyer</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID        actorName     directorName USA  NUSA\n",
       "0        1        Tom Hanks    John Lasseter   1     0\n",
       "1        2   Robin Williams     Joe Johnston   1     0\n",
       "2        3      Jack Lemmon    Donald Petrie   1     0\n",
       "3        4  Whitney Houston  Forest Whitaker   1     0\n",
       "4        5     Steve Martin    Charles Shyer   1     0"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actorsdirectorscountries=pd.merge(actorsanddirectors,movie_countriesFinale,on='movieID')\n",
    "actorsdirectorscountries.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['actors_F0', 'actors_F1', 'actors_F2', 'actors_F3', 'actors_F4', 'actors_F5', 'actors_F6', 'actors_F7', 'actors_F8', 'actors_F9', 'actors_F10', 'actors_F11', 'actors_F12', 'actors_F13', 'actors_F14', 'actors_F15', 'actors_F16', 'actors_F17', 'actors_F18', 'actors_F19']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actors_F0</th>\n",
       "      <th>actors_F1</th>\n",
       "      <th>actors_F2</th>\n",
       "      <th>actors_F3</th>\n",
       "      <th>actors_F4</th>\n",
       "      <th>actors_F5</th>\n",
       "      <th>actors_F6</th>\n",
       "      <th>actors_F7</th>\n",
       "      <th>actors_F8</th>\n",
       "      <th>actors_F9</th>\n",
       "      <th>actors_F10</th>\n",
       "      <th>actors_F11</th>\n",
       "      <th>actors_F12</th>\n",
       "      <th>actors_F13</th>\n",
       "      <th>actors_F14</th>\n",
       "      <th>actors_F15</th>\n",
       "      <th>actors_F16</th>\n",
       "      <th>actors_F17</th>\n",
       "      <th>actors_F18</th>\n",
       "      <th>actors_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.333418</td>\n",
       "      <td>-1.779091</td>\n",
       "      <td>-0.791577</td>\n",
       "      <td>-1.531474</td>\n",
       "      <td>-0.559442</td>\n",
       "      <td>1.256425</td>\n",
       "      <td>0.351360</td>\n",
       "      <td>1.007785</td>\n",
       "      <td>1.632603</td>\n",
       "      <td>0.742198</td>\n",
       "      <td>-1.286969</td>\n",
       "      <td>-0.999410</td>\n",
       "      <td>0.823705</td>\n",
       "      <td>-1.223432</td>\n",
       "      <td>0.703854</td>\n",
       "      <td>-0.451580</td>\n",
       "      <td>-0.522701</td>\n",
       "      <td>1.188622</td>\n",
       "      <td>0.308879</td>\n",
       "      <td>0.418487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.452748</td>\n",
       "      <td>-1.825097</td>\n",
       "      <td>-1.065131</td>\n",
       "      <td>-1.501005</td>\n",
       "      <td>-0.056070</td>\n",
       "      <td>1.312636</td>\n",
       "      <td>0.039013</td>\n",
       "      <td>1.149850</td>\n",
       "      <td>1.328850</td>\n",
       "      <td>0.688528</td>\n",
       "      <td>-1.196976</td>\n",
       "      <td>-0.594049</td>\n",
       "      <td>1.343878</td>\n",
       "      <td>-0.937165</td>\n",
       "      <td>0.532715</td>\n",
       "      <td>-0.023929</td>\n",
       "      <td>-0.391274</td>\n",
       "      <td>0.367733</td>\n",
       "      <td>0.447520</td>\n",
       "      <td>0.602501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.238248</td>\n",
       "      <td>-0.835818</td>\n",
       "      <td>-0.619902</td>\n",
       "      <td>-0.712091</td>\n",
       "      <td>-0.254227</td>\n",
       "      <td>0.812728</td>\n",
       "      <td>-0.021525</td>\n",
       "      <td>0.532269</td>\n",
       "      <td>0.944292</td>\n",
       "      <td>0.483457</td>\n",
       "      <td>-0.566314</td>\n",
       "      <td>-0.027771</td>\n",
       "      <td>0.682769</td>\n",
       "      <td>-0.451713</td>\n",
       "      <td>0.286310</td>\n",
       "      <td>-0.068402</td>\n",
       "      <td>-0.179913</td>\n",
       "      <td>0.461644</td>\n",
       "      <td>0.268155</td>\n",
       "      <td>0.312862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.309986</td>\n",
       "      <td>-1.368361</td>\n",
       "      <td>-1.024832</td>\n",
       "      <td>-1.427086</td>\n",
       "      <td>0.640924</td>\n",
       "      <td>1.010270</td>\n",
       "      <td>0.199718</td>\n",
       "      <td>-0.213346</td>\n",
       "      <td>0.198137</td>\n",
       "      <td>0.806633</td>\n",
       "      <td>-1.763526</td>\n",
       "      <td>-0.104799</td>\n",
       "      <td>1.243923</td>\n",
       "      <td>-0.252106</td>\n",
       "      <td>-0.023417</td>\n",
       "      <td>0.112558</td>\n",
       "      <td>-0.930885</td>\n",
       "      <td>0.633923</td>\n",
       "      <td>0.499440</td>\n",
       "      <td>0.959052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.473044</td>\n",
       "      <td>-1.874066</td>\n",
       "      <td>-0.490820</td>\n",
       "      <td>-1.567631</td>\n",
       "      <td>0.331746</td>\n",
       "      <td>1.166213</td>\n",
       "      <td>0.413818</td>\n",
       "      <td>0.737748</td>\n",
       "      <td>1.598423</td>\n",
       "      <td>1.162301</td>\n",
       "      <td>-0.782808</td>\n",
       "      <td>-0.529417</td>\n",
       "      <td>1.301883</td>\n",
       "      <td>-0.975605</td>\n",
       "      <td>1.191963</td>\n",
       "      <td>-0.206265</td>\n",
       "      <td>-0.197906</td>\n",
       "      <td>1.027958</td>\n",
       "      <td>0.721508</td>\n",
       "      <td>0.806618</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   actors_F0  actors_F1  actors_F2  actors_F3  actors_F4  actors_F5  \\\n",
       "0  -0.333418  -1.779091  -0.791577  -1.531474  -0.559442   1.256425   \n",
       "1  -0.452748  -1.825097  -1.065131  -1.501005  -0.056070   1.312636   \n",
       "2  -0.238248  -0.835818  -0.619902  -0.712091  -0.254227   0.812728   \n",
       "3  -0.309986  -1.368361  -1.024832  -1.427086   0.640924   1.010270   \n",
       "4  -0.473044  -1.874066  -0.490820  -1.567631   0.331746   1.166213   \n",
       "\n",
       "   actors_F6  actors_F7  actors_F8  actors_F9  actors_F10  actors_F11  \\\n",
       "0   0.351360   1.007785   1.632603   0.742198   -1.286969   -0.999410   \n",
       "1   0.039013   1.149850   1.328850   0.688528   -1.196976   -0.594049   \n",
       "2  -0.021525   0.532269   0.944292   0.483457   -0.566314   -0.027771   \n",
       "3   0.199718  -0.213346   0.198137   0.806633   -1.763526   -0.104799   \n",
       "4   0.413818   0.737748   1.598423   1.162301   -0.782808   -0.529417   \n",
       "\n",
       "   actors_F12  actors_F13  actors_F14  actors_F15  actors_F16  actors_F17  \\\n",
       "0    0.823705   -1.223432    0.703854   -0.451580   -0.522701    1.188622   \n",
       "1    1.343878   -0.937165    0.532715   -0.023929   -0.391274    0.367733   \n",
       "2    0.682769   -0.451713    0.286310   -0.068402   -0.179913    0.461644   \n",
       "3    1.243923   -0.252106   -0.023417    0.112558   -0.930885    0.633923   \n",
       "4    1.301883   -0.975605    1.191963   -0.206265   -0.197906    1.027958   \n",
       "\n",
       "   actors_F18  actors_F19  \n",
       "0    0.308879    0.418487  \n",
       "1    0.447520    0.602501  \n",
       "2    0.268155    0.312862  \n",
       "3    0.499440    0.959052  \n",
       "4    0.721508    0.806618  "
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actorembeds=embed(actorsanddirectors['actorName']).numpy()\n",
    "names=[\"actors_F\"+str(x) for x in range(20)]\n",
    "print(names)\n",
    "actorExtractedData=pd.DataFrame(actorembeds,columns=names)\n",
    "actorExtractedData.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['directors_F0', 'directors_F1', 'directors_F2', 'directors_F3', 'directors_F4', 'directors_F5', 'directors_F6', 'directors_F7', 'directors_F8', 'directors_F9', 'directors_F10', 'directors_F11', 'directors_F12', 'directors_F13', 'directors_F14', 'directors_F15', 'directors_F16', 'directors_F17', 'directors_F18', 'directors_F19']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>directors_F0</th>\n",
       "      <th>directors_F1</th>\n",
       "      <th>directors_F2</th>\n",
       "      <th>directors_F3</th>\n",
       "      <th>directors_F4</th>\n",
       "      <th>directors_F5</th>\n",
       "      <th>directors_F6</th>\n",
       "      <th>directors_F7</th>\n",
       "      <th>directors_F8</th>\n",
       "      <th>directors_F9</th>\n",
       "      <th>directors_F10</th>\n",
       "      <th>directors_F11</th>\n",
       "      <th>directors_F12</th>\n",
       "      <th>directors_F13</th>\n",
       "      <th>directors_F14</th>\n",
       "      <th>directors_F15</th>\n",
       "      <th>directors_F16</th>\n",
       "      <th>directors_F17</th>\n",
       "      <th>directors_F18</th>\n",
       "      <th>directors_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.252181</td>\n",
       "      <td>-0.927361</td>\n",
       "      <td>-0.231777</td>\n",
       "      <td>-0.912130</td>\n",
       "      <td>0.491651</td>\n",
       "      <td>0.575678</td>\n",
       "      <td>0.241689</td>\n",
       "      <td>0.588149</td>\n",
       "      <td>0.929212</td>\n",
       "      <td>0.433252</td>\n",
       "      <td>-0.460064</td>\n",
       "      <td>-0.590464</td>\n",
       "      <td>0.607007</td>\n",
       "      <td>-0.687870</td>\n",
       "      <td>0.456539</td>\n",
       "      <td>-0.032492</td>\n",
       "      <td>-0.275420</td>\n",
       "      <td>0.655479</td>\n",
       "      <td>0.474778</td>\n",
       "      <td>0.476021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.569218</td>\n",
       "      <td>-1.734962</td>\n",
       "      <td>-0.950825</td>\n",
       "      <td>-1.385349</td>\n",
       "      <td>0.299962</td>\n",
       "      <td>1.170124</td>\n",
       "      <td>0.415418</td>\n",
       "      <td>1.246468</td>\n",
       "      <td>1.056857</td>\n",
       "      <td>1.239762</td>\n",
       "      <td>-1.058670</td>\n",
       "      <td>-0.525210</td>\n",
       "      <td>1.422491</td>\n",
       "      <td>-1.090790</td>\n",
       "      <td>0.839541</td>\n",
       "      <td>0.237497</td>\n",
       "      <td>-0.201687</td>\n",
       "      <td>1.191358</td>\n",
       "      <td>0.624746</td>\n",
       "      <td>0.626180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.803633</td>\n",
       "      <td>-0.218671</td>\n",
       "      <td>0.856281</td>\n",
       "      <td>-0.667001</td>\n",
       "      <td>0.490485</td>\n",
       "      <td>0.219580</td>\n",
       "      <td>0.444163</td>\n",
       "      <td>0.654473</td>\n",
       "      <td>0.823934</td>\n",
       "      <td>0.569644</td>\n",
       "      <td>0.225105</td>\n",
       "      <td>-0.653097</td>\n",
       "      <td>0.581999</td>\n",
       "      <td>-0.285077</td>\n",
       "      <td>0.914893</td>\n",
       "      <td>0.175597</td>\n",
       "      <td>0.381039</td>\n",
       "      <td>-0.408570</td>\n",
       "      <td>-0.618443</td>\n",
       "      <td>0.390962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.346459</td>\n",
       "      <td>0.382251</td>\n",
       "      <td>-0.218554</td>\n",
       "      <td>-0.576592</td>\n",
       "      <td>0.749572</td>\n",
       "      <td>0.149732</td>\n",
       "      <td>0.199274</td>\n",
       "      <td>0.004454</td>\n",
       "      <td>-0.746907</td>\n",
       "      <td>0.819507</td>\n",
       "      <td>-0.559638</td>\n",
       "      <td>-0.032697</td>\n",
       "      <td>0.288008</td>\n",
       "      <td>-0.119868</td>\n",
       "      <td>0.228856</td>\n",
       "      <td>0.073922</td>\n",
       "      <td>-0.703598</td>\n",
       "      <td>0.887784</td>\n",
       "      <td>0.457597</td>\n",
       "      <td>0.433813</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.363227</td>\n",
       "      <td>-0.742808</td>\n",
       "      <td>-0.522909</td>\n",
       "      <td>-0.814757</td>\n",
       "      <td>0.765352</td>\n",
       "      <td>0.604321</td>\n",
       "      <td>-0.022693</td>\n",
       "      <td>0.456844</td>\n",
       "      <td>0.883219</td>\n",
       "      <td>0.565462</td>\n",
       "      <td>-0.327141</td>\n",
       "      <td>-0.493852</td>\n",
       "      <td>0.371466</td>\n",
       "      <td>-0.576322</td>\n",
       "      <td>0.274291</td>\n",
       "      <td>-0.325437</td>\n",
       "      <td>-0.502540</td>\n",
       "      <td>0.383853</td>\n",
       "      <td>0.547358</td>\n",
       "      <td>0.301073</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   directors_F0  directors_F1  directors_F2  directors_F3  directors_F4  \\\n",
       "0     -0.252181     -0.927361     -0.231777     -0.912130      0.491651   \n",
       "1     -0.569218     -1.734962     -0.950825     -1.385349      0.299962   \n",
       "2     -0.803633     -0.218671      0.856281     -0.667001      0.490485   \n",
       "3     -0.346459      0.382251     -0.218554     -0.576592      0.749572   \n",
       "4     -0.363227     -0.742808     -0.522909     -0.814757      0.765352   \n",
       "\n",
       "   directors_F5  directors_F6  directors_F7  directors_F8  directors_F9  \\\n",
       "0      0.575678      0.241689      0.588149      0.929212      0.433252   \n",
       "1      1.170124      0.415418      1.246468      1.056857      1.239762   \n",
       "2      0.219580      0.444163      0.654473      0.823934      0.569644   \n",
       "3      0.149732      0.199274      0.004454     -0.746907      0.819507   \n",
       "4      0.604321     -0.022693      0.456844      0.883219      0.565462   \n",
       "\n",
       "   directors_F10  directors_F11  directors_F12  directors_F13  directors_F14  \\\n",
       "0      -0.460064      -0.590464       0.607007      -0.687870       0.456539   \n",
       "1      -1.058670      -0.525210       1.422491      -1.090790       0.839541   \n",
       "2       0.225105      -0.653097       0.581999      -0.285077       0.914893   \n",
       "3      -0.559638      -0.032697       0.288008      -0.119868       0.228856   \n",
       "4      -0.327141      -0.493852       0.371466      -0.576322       0.274291   \n",
       "\n",
       "   directors_F15  directors_F16  directors_F17  directors_F18  directors_F19  \n",
       "0      -0.032492      -0.275420       0.655479       0.474778       0.476021  \n",
       "1       0.237497      -0.201687       1.191358       0.624746       0.626180  \n",
       "2       0.175597       0.381039      -0.408570      -0.618443       0.390962  \n",
       "3       0.073922      -0.703598       0.887784       0.457597       0.433813  \n",
       "4      -0.325437      -0.502540       0.383853       0.547358       0.301073  "
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "directorembeds=embed(actorsanddirectors['directorName']).numpy()\n",
    "names=[\"directors_F\"+str(x) for x in range(20)]\n",
    "print(names)\n",
    "directorExtractedData=pd.DataFrame(directorembeds,columns=names)\n",
    "directorExtractedData.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName</th>\n",
       "      <th>directorName</th>\n",
       "      <th>actors_F0</th>\n",
       "      <th>actors_F1</th>\n",
       "      <th>actors_F2</th>\n",
       "      <th>actors_F3</th>\n",
       "      <th>actors_F4</th>\n",
       "      <th>actors_F5</th>\n",
       "      <th>actors_F6</th>\n",
       "      <th>...</th>\n",
       "      <th>actors_F10</th>\n",
       "      <th>actors_F11</th>\n",
       "      <th>actors_F12</th>\n",
       "      <th>actors_F13</th>\n",
       "      <th>actors_F14</th>\n",
       "      <th>actors_F15</th>\n",
       "      <th>actors_F16</th>\n",
       "      <th>actors_F17</th>\n",
       "      <th>actors_F18</th>\n",
       "      <th>actors_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "      <td>-0.333418</td>\n",
       "      <td>-1.779091</td>\n",
       "      <td>-0.791577</td>\n",
       "      <td>-1.531474</td>\n",
       "      <td>-0.559442</td>\n",
       "      <td>1.256425</td>\n",
       "      <td>0.351360</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.286969</td>\n",
       "      <td>-0.999410</td>\n",
       "      <td>0.823705</td>\n",
       "      <td>-1.223432</td>\n",
       "      <td>0.703854</td>\n",
       "      <td>-0.451580</td>\n",
       "      <td>-0.522701</td>\n",
       "      <td>1.188622</td>\n",
       "      <td>0.308879</td>\n",
       "      <td>0.418487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Robin Williams</td>\n",
       "      <td>Joe Johnston</td>\n",
       "      <td>-0.452748</td>\n",
       "      <td>-1.825097</td>\n",
       "      <td>-1.065131</td>\n",
       "      <td>-1.501005</td>\n",
       "      <td>-0.056070</td>\n",
       "      <td>1.312636</td>\n",
       "      <td>0.039013</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.196976</td>\n",
       "      <td>-0.594049</td>\n",
       "      <td>1.343878</td>\n",
       "      <td>-0.937165</td>\n",
       "      <td>0.532715</td>\n",
       "      <td>-0.023929</td>\n",
       "      <td>-0.391274</td>\n",
       "      <td>0.367733</td>\n",
       "      <td>0.447520</td>\n",
       "      <td>0.602501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "      <td>Donald Petrie</td>\n",
       "      <td>-0.238248</td>\n",
       "      <td>-0.835818</td>\n",
       "      <td>-0.619902</td>\n",
       "      <td>-0.712091</td>\n",
       "      <td>-0.254227</td>\n",
       "      <td>0.812728</td>\n",
       "      <td>-0.021525</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.566314</td>\n",
       "      <td>-0.027771</td>\n",
       "      <td>0.682769</td>\n",
       "      <td>-0.451713</td>\n",
       "      <td>0.286310</td>\n",
       "      <td>-0.068402</td>\n",
       "      <td>-0.179913</td>\n",
       "      <td>0.461644</td>\n",
       "      <td>0.268155</td>\n",
       "      <td>0.312862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Whitney Houston</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "      <td>-0.309986</td>\n",
       "      <td>-1.368361</td>\n",
       "      <td>-1.024832</td>\n",
       "      <td>-1.427086</td>\n",
       "      <td>0.640924</td>\n",
       "      <td>1.010270</td>\n",
       "      <td>0.199718</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.763526</td>\n",
       "      <td>-0.104799</td>\n",
       "      <td>1.243923</td>\n",
       "      <td>-0.252106</td>\n",
       "      <td>-0.023417</td>\n",
       "      <td>0.112558</td>\n",
       "      <td>-0.930885</td>\n",
       "      <td>0.633923</td>\n",
       "      <td>0.499440</td>\n",
       "      <td>0.959052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Steve Martin</td>\n",
       "      <td>Charles Shyer</td>\n",
       "      <td>-0.473044</td>\n",
       "      <td>-1.874066</td>\n",
       "      <td>-0.490820</td>\n",
       "      <td>-1.567631</td>\n",
       "      <td>0.331746</td>\n",
       "      <td>1.166213</td>\n",
       "      <td>0.413818</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.782808</td>\n",
       "      <td>-0.529417</td>\n",
       "      <td>1.301883</td>\n",
       "      <td>-0.975605</td>\n",
       "      <td>1.191963</td>\n",
       "      <td>-0.206265</td>\n",
       "      <td>-0.197906</td>\n",
       "      <td>1.027958</td>\n",
       "      <td>0.721508</td>\n",
       "      <td>0.806618</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID        actorName     directorName  actors_F0  actors_F1  actors_F2  \\\n",
       "0        1        Tom Hanks    John Lasseter  -0.333418  -1.779091  -0.791577   \n",
       "1        2   Robin Williams     Joe Johnston  -0.452748  -1.825097  -1.065131   \n",
       "2        3      Jack Lemmon    Donald Petrie  -0.238248  -0.835818  -0.619902   \n",
       "3        4  Whitney Houston  Forest Whitaker  -0.309986  -1.368361  -1.024832   \n",
       "4        5     Steve Martin    Charles Shyer  -0.473044  -1.874066  -0.490820   \n",
       "\n",
       "   actors_F3  actors_F4  actors_F5  actors_F6  ...  actors_F10  actors_F11  \\\n",
       "0  -1.531474  -0.559442   1.256425   0.351360  ...   -1.286969   -0.999410   \n",
       "1  -1.501005  -0.056070   1.312636   0.039013  ...   -1.196976   -0.594049   \n",
       "2  -0.712091  -0.254227   0.812728  -0.021525  ...   -0.566314   -0.027771   \n",
       "3  -1.427086   0.640924   1.010270   0.199718  ...   -1.763526   -0.104799   \n",
       "4  -1.567631   0.331746   1.166213   0.413818  ...   -0.782808   -0.529417   \n",
       "\n",
       "   actors_F12  actors_F13  actors_F14  actors_F15  actors_F16  actors_F17  \\\n",
       "0    0.823705   -1.223432    0.703854   -0.451580   -0.522701    1.188622   \n",
       "1    1.343878   -0.937165    0.532715   -0.023929   -0.391274    0.367733   \n",
       "2    0.682769   -0.451713    0.286310   -0.068402   -0.179913    0.461644   \n",
       "3    1.243923   -0.252106   -0.023417    0.112558   -0.930885    0.633923   \n",
       "4    1.301883   -0.975605    1.191963   -0.206265   -0.197906    1.027958   \n",
       "\n",
       "   actors_F18  actors_F19  \n",
       "0    0.308879    0.418487  \n",
       "1    0.447520    0.602501  \n",
       "2    0.268155    0.312862  \n",
       "3    0.499440    0.959052  \n",
       "4    0.721508    0.806618  \n",
       "\n",
       "[5 rows x 23 columns]"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ActorEmbedDF=pd.concat([actorsanddirectors,actorExtractedData],axis=1)\n",
    "ActorEmbedDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName</th>\n",
       "      <th>directorName</th>\n",
       "      <th>directors_F0</th>\n",
       "      <th>directors_F1</th>\n",
       "      <th>directors_F2</th>\n",
       "      <th>directors_F3</th>\n",
       "      <th>directors_F4</th>\n",
       "      <th>directors_F5</th>\n",
       "      <th>directors_F6</th>\n",
       "      <th>...</th>\n",
       "      <th>directors_F10</th>\n",
       "      <th>directors_F11</th>\n",
       "      <th>directors_F12</th>\n",
       "      <th>directors_F13</th>\n",
       "      <th>directors_F14</th>\n",
       "      <th>directors_F15</th>\n",
       "      <th>directors_F16</th>\n",
       "      <th>directors_F17</th>\n",
       "      <th>directors_F18</th>\n",
       "      <th>directors_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "      <td>-0.252181</td>\n",
       "      <td>-0.927361</td>\n",
       "      <td>-0.231777</td>\n",
       "      <td>-0.912130</td>\n",
       "      <td>0.491651</td>\n",
       "      <td>0.575678</td>\n",
       "      <td>0.241689</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.460064</td>\n",
       "      <td>-0.590464</td>\n",
       "      <td>0.607007</td>\n",
       "      <td>-0.687870</td>\n",
       "      <td>0.456539</td>\n",
       "      <td>-0.032492</td>\n",
       "      <td>-0.275420</td>\n",
       "      <td>0.655479</td>\n",
       "      <td>0.474778</td>\n",
       "      <td>0.476021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Robin Williams</td>\n",
       "      <td>Joe Johnston</td>\n",
       "      <td>-0.569218</td>\n",
       "      <td>-1.734962</td>\n",
       "      <td>-0.950825</td>\n",
       "      <td>-1.385349</td>\n",
       "      <td>0.299962</td>\n",
       "      <td>1.170124</td>\n",
       "      <td>0.415418</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.058670</td>\n",
       "      <td>-0.525210</td>\n",
       "      <td>1.422491</td>\n",
       "      <td>-1.090790</td>\n",
       "      <td>0.839541</td>\n",
       "      <td>0.237497</td>\n",
       "      <td>-0.201687</td>\n",
       "      <td>1.191358</td>\n",
       "      <td>0.624746</td>\n",
       "      <td>0.626180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "      <td>Donald Petrie</td>\n",
       "      <td>-0.803633</td>\n",
       "      <td>-0.218671</td>\n",
       "      <td>0.856281</td>\n",
       "      <td>-0.667001</td>\n",
       "      <td>0.490485</td>\n",
       "      <td>0.219580</td>\n",
       "      <td>0.444163</td>\n",
       "      <td>...</td>\n",
       "      <td>0.225105</td>\n",
       "      <td>-0.653097</td>\n",
       "      <td>0.581999</td>\n",
       "      <td>-0.285077</td>\n",
       "      <td>0.914893</td>\n",
       "      <td>0.175597</td>\n",
       "      <td>0.381039</td>\n",
       "      <td>-0.408570</td>\n",
       "      <td>-0.618443</td>\n",
       "      <td>0.390962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Whitney Houston</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "      <td>-0.346459</td>\n",
       "      <td>0.382251</td>\n",
       "      <td>-0.218554</td>\n",
       "      <td>-0.576592</td>\n",
       "      <td>0.749572</td>\n",
       "      <td>0.149732</td>\n",
       "      <td>0.199274</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.559638</td>\n",
       "      <td>-0.032697</td>\n",
       "      <td>0.288008</td>\n",
       "      <td>-0.119868</td>\n",
       "      <td>0.228856</td>\n",
       "      <td>0.073922</td>\n",
       "      <td>-0.703598</td>\n",
       "      <td>0.887784</td>\n",
       "      <td>0.457597</td>\n",
       "      <td>0.433813</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Steve Martin</td>\n",
       "      <td>Charles Shyer</td>\n",
       "      <td>-0.363227</td>\n",
       "      <td>-0.742808</td>\n",
       "      <td>-0.522909</td>\n",
       "      <td>-0.814757</td>\n",
       "      <td>0.765352</td>\n",
       "      <td>0.604321</td>\n",
       "      <td>-0.022693</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.327141</td>\n",
       "      <td>-0.493852</td>\n",
       "      <td>0.371466</td>\n",
       "      <td>-0.576322</td>\n",
       "      <td>0.274291</td>\n",
       "      <td>-0.325437</td>\n",
       "      <td>-0.502540</td>\n",
       "      <td>0.383853</td>\n",
       "      <td>0.547358</td>\n",
       "      <td>0.301073</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID        actorName     directorName  directors_F0  directors_F1  \\\n",
       "0        1        Tom Hanks    John Lasseter     -0.252181     -0.927361   \n",
       "1        2   Robin Williams     Joe Johnston     -0.569218     -1.734962   \n",
       "2        3      Jack Lemmon    Donald Petrie     -0.803633     -0.218671   \n",
       "3        4  Whitney Houston  Forest Whitaker     -0.346459      0.382251   \n",
       "4        5     Steve Martin    Charles Shyer     -0.363227     -0.742808   \n",
       "\n",
       "   directors_F2  directors_F3  directors_F4  directors_F5  directors_F6  ...  \\\n",
       "0     -0.231777     -0.912130      0.491651      0.575678      0.241689  ...   \n",
       "1     -0.950825     -1.385349      0.299962      1.170124      0.415418  ...   \n",
       "2      0.856281     -0.667001      0.490485      0.219580      0.444163  ...   \n",
       "3     -0.218554     -0.576592      0.749572      0.149732      0.199274  ...   \n",
       "4     -0.522909     -0.814757      0.765352      0.604321     -0.022693  ...   \n",
       "\n",
       "   directors_F10  directors_F11  directors_F12  directors_F13  directors_F14  \\\n",
       "0      -0.460064      -0.590464       0.607007      -0.687870       0.456539   \n",
       "1      -1.058670      -0.525210       1.422491      -1.090790       0.839541   \n",
       "2       0.225105      -0.653097       0.581999      -0.285077       0.914893   \n",
       "3      -0.559638      -0.032697       0.288008      -0.119868       0.228856   \n",
       "4      -0.327141      -0.493852       0.371466      -0.576322       0.274291   \n",
       "\n",
       "   directors_F15  directors_F16  directors_F17  directors_F18  directors_F19  \n",
       "0      -0.032492      -0.275420       0.655479       0.474778       0.476021  \n",
       "1       0.237497      -0.201687       1.191358       0.624746       0.626180  \n",
       "2       0.175597       0.381039      -0.408570      -0.618443       0.390962  \n",
       "3       0.073922      -0.703598       0.887784       0.457597       0.433813  \n",
       "4      -0.325437      -0.502540       0.383853       0.547358       0.301073  \n",
       "\n",
       "[5 rows x 23 columns]"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DirectorEmbedDF=pd.concat([actorsanddirectors,directorExtractedData],axis=1)\n",
    "DirectorEmbedDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName_x</th>\n",
       "      <th>directorName_x</th>\n",
       "      <th>actors_F0</th>\n",
       "      <th>actors_F1</th>\n",
       "      <th>actors_F2</th>\n",
       "      <th>actors_F3</th>\n",
       "      <th>actors_F4</th>\n",
       "      <th>actors_F5</th>\n",
       "      <th>actors_F6</th>\n",
       "      <th>...</th>\n",
       "      <th>Tag_F10</th>\n",
       "      <th>Tag_F11</th>\n",
       "      <th>Tag_F12</th>\n",
       "      <th>Tag_F13</th>\n",
       "      <th>Tag_F14</th>\n",
       "      <th>Tag_F15</th>\n",
       "      <th>Tag_F16</th>\n",
       "      <th>Tag_F17</th>\n",
       "      <th>Tag_F18</th>\n",
       "      <th>Tag_F19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "      <td>-0.333418</td>\n",
       "      <td>-1.779091</td>\n",
       "      <td>-0.791577</td>\n",
       "      <td>-1.531474</td>\n",
       "      <td>-0.559442</td>\n",
       "      <td>1.256425</td>\n",
       "      <td>0.351360</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.305140</td>\n",
       "      <td>1.395004</td>\n",
       "      <td>-2.442963</td>\n",
       "      <td>0.191763</td>\n",
       "      <td>-1.348063</td>\n",
       "      <td>-1.315639</td>\n",
       "      <td>-0.387860</td>\n",
       "      <td>0.767467</td>\n",
       "      <td>-0.641286</td>\n",
       "      <td>-0.789096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Robin Williams</td>\n",
       "      <td>Joe Johnston</td>\n",
       "      <td>-0.452748</td>\n",
       "      <td>-1.825097</td>\n",
       "      <td>-1.065131</td>\n",
       "      <td>-1.501005</td>\n",
       "      <td>-0.056070</td>\n",
       "      <td>1.312636</td>\n",
       "      <td>0.039013</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.305755</td>\n",
       "      <td>1.597886</td>\n",
       "      <td>-0.426935</td>\n",
       "      <td>0.531539</td>\n",
       "      <td>-1.583068</td>\n",
       "      <td>0.790853</td>\n",
       "      <td>1.668663</td>\n",
       "      <td>-0.273516</td>\n",
       "      <td>-1.036308</td>\n",
       "      <td>0.179430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "      <td>Donald Petrie</td>\n",
       "      <td>-0.238248</td>\n",
       "      <td>-0.835818</td>\n",
       "      <td>-0.619902</td>\n",
       "      <td>-0.712091</td>\n",
       "      <td>-0.254227</td>\n",
       "      <td>0.812728</td>\n",
       "      <td>-0.021525</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.430264</td>\n",
       "      <td>0.918680</td>\n",
       "      <td>-0.669077</td>\n",
       "      <td>-0.373421</td>\n",
       "      <td>-1.102863</td>\n",
       "      <td>0.264218</td>\n",
       "      <td>-0.688483</td>\n",
       "      <td>-0.140395</td>\n",
       "      <td>-0.196410</td>\n",
       "      <td>0.008075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Whitney Houston</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "      <td>-0.309986</td>\n",
       "      <td>-1.368361</td>\n",
       "      <td>-1.024832</td>\n",
       "      <td>-1.427086</td>\n",
       "      <td>0.640924</td>\n",
       "      <td>1.010270</td>\n",
       "      <td>0.199718</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Steve Martin</td>\n",
       "      <td>Charles Shyer</td>\n",
       "      <td>-0.473044</td>\n",
       "      <td>-1.874066</td>\n",
       "      <td>-0.490820</td>\n",
       "      <td>-1.567631</td>\n",
       "      <td>0.331746</td>\n",
       "      <td>1.166213</td>\n",
       "      <td>0.413818</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.888412</td>\n",
       "      <td>0.467941</td>\n",
       "      <td>-1.613994</td>\n",
       "      <td>-0.352356</td>\n",
       "      <td>-2.164545</td>\n",
       "      <td>-1.088279</td>\n",
       "      <td>0.236017</td>\n",
       "      <td>-1.008116</td>\n",
       "      <td>-0.895948</td>\n",
       "      <td>-0.566081</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 104 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID      actorName_x   directorName_x  actors_F0  actors_F1  actors_F2  \\\n",
       "0        1        Tom Hanks    John Lasseter  -0.333418  -1.779091  -0.791577   \n",
       "1        2   Robin Williams     Joe Johnston  -0.452748  -1.825097  -1.065131   \n",
       "2        3      Jack Lemmon    Donald Petrie  -0.238248  -0.835818  -0.619902   \n",
       "3        4  Whitney Houston  Forest Whitaker  -0.309986  -1.368361  -1.024832   \n",
       "4        5     Steve Martin    Charles Shyer  -0.473044  -1.874066  -0.490820   \n",
       "\n",
       "   actors_F3  actors_F4  actors_F5  actors_F6  ...   Tag_F10   Tag_F11  \\\n",
       "0  -1.531474  -0.559442   1.256425   0.351360  ... -2.305140  1.395004   \n",
       "1  -1.501005  -0.056070   1.312636   0.039013  ... -1.305755  1.597886   \n",
       "2  -0.712091  -0.254227   0.812728  -0.021525  ... -0.430264  0.918680   \n",
       "3  -1.427086   0.640924   1.010270   0.199718  ...  0.000000  0.000000   \n",
       "4  -1.567631   0.331746   1.166213   0.413818  ... -1.888412  0.467941   \n",
       "\n",
       "    Tag_F12   Tag_F13   Tag_F14   Tag_F15   Tag_F16   Tag_F17   Tag_F18  \\\n",
       "0 -2.442963  0.191763 -1.348063 -1.315639 -0.387860  0.767467 -0.641286   \n",
       "1 -0.426935  0.531539 -1.583068  0.790853  1.668663 -0.273516 -1.036308   \n",
       "2 -0.669077 -0.373421 -1.102863  0.264218 -0.688483 -0.140395 -0.196410   \n",
       "3  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
       "4 -1.613994 -0.352356 -2.164545 -1.088279  0.236017 -1.008116 -0.895948   \n",
       "\n",
       "    Tag_F19  \n",
       "0 -0.789096  \n",
       "1  0.179430  \n",
       "2  0.008075  \n",
       "3  0.000000  \n",
       "4 -0.566081  \n",
       "\n",
       "[5 rows x 104 columns]"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ActorDirectorEmbeds=pd.merge(ActorEmbedDF, DirectorEmbedDF, on='movieID')\n",
    "TitleGenreActorDirectorTag=pd.merge(ActorDirectorEmbeds,TitleGenreTagsEncoding,on='movieID')\n",
    "TitleGenreActorDirectorTag.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6645, 104)"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TitleGenreActorDirectorTag.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Important Note: Due to default merge being inner, intersection occurs and only the titles with common details make the cut!**\n",
    "1. This helps us filter out automatically\n",
    "2. This haunted me for about 3 days as the final dataframe was at 2200 movies without the popularity filter!\n",
    "3. The issue was in one of the merges made during tags as they were not available for all movies\n",
    "4. Where it wasn't beneficial to us, I have applied left/outer joins and dealt with NaN values accordingly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieID</th>\n",
       "      <th>actorName_x</th>\n",
       "      <th>directorName_x</th>\n",
       "      <th>actors_F0</th>\n",
       "      <th>actors_F1</th>\n",
       "      <th>actors_F2</th>\n",
       "      <th>actors_F3</th>\n",
       "      <th>actors_F4</th>\n",
       "      <th>actors_F5</th>\n",
       "      <th>actors_F6</th>\n",
       "      <th>...</th>\n",
       "      <th>Tag_F12</th>\n",
       "      <th>Tag_F13</th>\n",
       "      <th>Tag_F14</th>\n",
       "      <th>Tag_F15</th>\n",
       "      <th>Tag_F16</th>\n",
       "      <th>Tag_F17</th>\n",
       "      <th>Tag_F18</th>\n",
       "      <th>Tag_F19</th>\n",
       "      <th>USA</th>\n",
       "      <th>NUSA</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Tom Hanks</td>\n",
       "      <td>John Lasseter</td>\n",
       "      <td>-0.333418</td>\n",
       "      <td>-1.779091</td>\n",
       "      <td>-0.791577</td>\n",
       "      <td>-1.531474</td>\n",
       "      <td>-0.559442</td>\n",
       "      <td>1.256425</td>\n",
       "      <td>0.351360</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.442963</td>\n",
       "      <td>0.191763</td>\n",
       "      <td>-1.348063</td>\n",
       "      <td>-1.315639</td>\n",
       "      <td>-0.387860</td>\n",
       "      <td>0.767467</td>\n",
       "      <td>-0.641286</td>\n",
       "      <td>-0.789096</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Robin Williams</td>\n",
       "      <td>Joe Johnston</td>\n",
       "      <td>-0.452748</td>\n",
       "      <td>-1.825097</td>\n",
       "      <td>-1.065131</td>\n",
       "      <td>-1.501005</td>\n",
       "      <td>-0.056070</td>\n",
       "      <td>1.312636</td>\n",
       "      <td>0.039013</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.426935</td>\n",
       "      <td>0.531539</td>\n",
       "      <td>-1.583068</td>\n",
       "      <td>0.790853</td>\n",
       "      <td>1.668663</td>\n",
       "      <td>-0.273516</td>\n",
       "      <td>-1.036308</td>\n",
       "      <td>0.179430</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Jack Lemmon</td>\n",
       "      <td>Donald Petrie</td>\n",
       "      <td>-0.238248</td>\n",
       "      <td>-0.835818</td>\n",
       "      <td>-0.619902</td>\n",
       "      <td>-0.712091</td>\n",
       "      <td>-0.254227</td>\n",
       "      <td>0.812728</td>\n",
       "      <td>-0.021525</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.669077</td>\n",
       "      <td>-0.373421</td>\n",
       "      <td>-1.102863</td>\n",
       "      <td>0.264218</td>\n",
       "      <td>-0.688483</td>\n",
       "      <td>-0.140395</td>\n",
       "      <td>-0.196410</td>\n",
       "      <td>0.008075</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Whitney Houston</td>\n",
       "      <td>Forest Whitaker</td>\n",
       "      <td>-0.309986</td>\n",
       "      <td>-1.368361</td>\n",
       "      <td>-1.024832</td>\n",
       "      <td>-1.427086</td>\n",
       "      <td>0.640924</td>\n",
       "      <td>1.010270</td>\n",
       "      <td>0.199718</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Steve Martin</td>\n",
       "      <td>Charles Shyer</td>\n",
       "      <td>-0.473044</td>\n",
       "      <td>-1.874066</td>\n",
       "      <td>-0.490820</td>\n",
       "      <td>-1.567631</td>\n",
       "      <td>0.331746</td>\n",
       "      <td>1.166213</td>\n",
       "      <td>0.413818</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.613994</td>\n",
       "      <td>-0.352356</td>\n",
       "      <td>-2.164545</td>\n",
       "      <td>-1.088279</td>\n",
       "      <td>0.236017</td>\n",
       "      <td>-1.008116</td>\n",
       "      <td>-0.895948</td>\n",
       "      <td>-0.566081</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 106 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieID      actorName_x   directorName_x  actors_F0  actors_F1  actors_F2  \\\n",
       "0        1        Tom Hanks    John Lasseter  -0.333418  -1.779091  -0.791577   \n",
       "1        2   Robin Williams     Joe Johnston  -0.452748  -1.825097  -1.065131   \n",
       "2        3      Jack Lemmon    Donald Petrie  -0.238248  -0.835818  -0.619902   \n",
       "3        4  Whitney Houston  Forest Whitaker  -0.309986  -1.368361  -1.024832   \n",
       "4        5     Steve Martin    Charles Shyer  -0.473044  -1.874066  -0.490820   \n",
       "\n",
       "   actors_F3  actors_F4  actors_F5  actors_F6  ...   Tag_F12   Tag_F13  \\\n",
       "0  -1.531474  -0.559442   1.256425   0.351360  ... -2.442963  0.191763   \n",
       "1  -1.501005  -0.056070   1.312636   0.039013  ... -0.426935  0.531539   \n",
       "2  -0.712091  -0.254227   0.812728  -0.021525  ... -0.669077 -0.373421   \n",
       "3  -1.427086   0.640924   1.010270   0.199718  ...  0.000000  0.000000   \n",
       "4  -1.567631   0.331746   1.166213   0.413818  ... -1.613994 -0.352356   \n",
       "\n",
       "    Tag_F14   Tag_F15   Tag_F16   Tag_F17   Tag_F18   Tag_F19  USA  NUSA  \n",
       "0 -1.348063 -1.315639 -0.387860  0.767467 -0.641286 -0.789096    1     0  \n",
       "1 -1.583068  0.790853  1.668663 -0.273516 -1.036308  0.179430    1     0  \n",
       "2 -1.102863  0.264218 -0.688483 -0.140395 -0.196410  0.008075    1     0  \n",
       "3  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000    1     0  \n",
       "4 -2.164545 -1.088279  0.236017 -1.008116 -0.895948 -0.566081    1     0  \n",
       "\n",
       "[5 rows x 106 columns]"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TitleGenreActorDirectorTagCountry=pd.merge(TitleGenreActorDirectorTag,movie_countriesFinale,on='movieID')\n",
    "TitleGenreActorDirectorTagCountry.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6644, 106)"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TitleGenreActorDirectorTagCountry.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [],
   "source": [
    "similarity1 = cosine_similarity(TitleGenreActorDirectorTagCountry[['actors_F0', 'actors_F1',\n",
    "       'actors_F2', 'actors_F3', 'actors_F4', 'actors_F5', 'actors_F6',\n",
    "       'actors_F7', 'actors_F8', 'actors_F9', 'actors_F10', 'actors_F11',\n",
    "       'actors_F12', 'actors_F13', 'actors_F14', 'actors_F15', 'actors_F16',\n",
    "       'actors_F17', 'actors_F18', 'actors_F19', 'directors_F0', 'directors_F1', 'directors_F2',\n",
    "       'directors_F3', 'directors_F4', 'directors_F5', 'directors_F6',\n",
    "       'directors_F7', 'directors_F8', 'directors_F9', 'directors_F10',\n",
    "       'directors_F11', 'directors_F12', 'directors_F13', 'directors_F14',\n",
    "       'directors_F15', 'directors_F16', 'directors_F17', 'directors_F18',\n",
    "       'directors_F19', 'year', 'Drama', 'Comedy',\n",
    "       'Thriller', 'Romance', 'Action', 'Crime', 'Adventure', 'Horror',\n",
    "       'Sci-Fi', 'Fantasy', 'Children', 'Mystery', 'War', 'Documentary',\n",
    "       'Musical', 'topic_F0', 'topic_F1', 'topic_F2', 'topic_F3', 'topic_F4',\n",
    "       'topic_F5', 'topic_F6', 'topic_F7', 'topic_F8', 'topic_F9', 'topic_F10',\n",
    "       'topic_F11', 'topic_F12', 'topic_F13', 'topic_F14', 'topic_F15',\n",
    "       'topic_F16', 'topic_F17', 'topic_F18', 'topic_F19', 'Tag_F0',\n",
    "       'Tag_F1', 'Tag_F2', 'Tag_F3', 'Tag_F4', 'Tag_F5', 'Tag_F6', 'Tag_F7',\n",
    "       'Tag_F8', 'Tag_F9', 'Tag_F10', 'Tag_F11', 'Tag_F12', 'Tag_F13',\n",
    "       'Tag_F14', 'Tag_F15', 'Tag_F16', 'Tag_F17', 'Tag_F18', 'Tag_F19','USA','NUSA']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "0    Toy story\n",
      "Name: title, dtype: object\n",
      "1709    A Bug's Life\n",
      "Name: title, dtype: object\n",
      "6231    The Simpsons Movie\n",
      "Name: title, dtype: object\n",
      "474    The Nightmare Before Christmas\n",
      "Name: title, dtype: object\n",
      "933    Evil Dead II\n",
      "Name: title, dtype: object\n",
      "5695    Happy Endings\n",
      "Name: title, dtype: object\n",
      "2796    The Emperor's New Groove\n",
      "Name: title, dtype: object\n",
      "1553    Charlotte's Web\n",
      "Name: title, dtype: object\n",
      "1542    The Lord of the Rings: The Return of the King\n",
      "Name: title, dtype: object\n",
      "503    Beauty and the Beast\n",
      "Name: title, dtype: object\n"
     ]
    }
   ],
   "source": [
    "Top_n = 10\n",
    "similarity1[0].argsort()[-Top_n:][::-1]\n",
    "title = \"Toy story\" \n",
    "movie_id = TitleGenreActorDirectorTagCountry[TitleGenreActorDirectorTagCountry[\"title\"]==title][\"movieID\"].values[0]\n",
    "print(movie_id)\n",
    "for index in similarity1[0].argsort()[-Top_n:][::-1]:\n",
    "    print(TitleGenreActorDirectorTagCountry['title'].iloc[[index]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Final Note:**\n",
    "1. Movies like Batman have been pushed away and more children friendly titles have come into play\n",
    "2. Embeddings extract the context -> push things that belong together ever so close\n",
    "3. We can filter movies out of the loop based on a popularity factor to bring forward more \"good\" reccomendations\n",
    "4. These reccomendations when shown made more 'sense' to a few movie buff friends of mine\n",
    "\n",
    "**Towards User-Item Based collaborative filtering we go, where the customer, you, are our priority!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2: Collaborative Filtering** <br>\n",
    "\n",
    "**Purpose:**\n",
    "1. Effectively model the user-item relationship \n",
    "2. Evaluate the effectiveness through a quantitative measure \n",
    "3. Connect the dots "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    855598.000000\n",
       "mean          3.437945\n",
       "std           1.002561\n",
       "min           0.500000\n",
       "25%           3.000000\n",
       "50%           3.500000\n",
       "75%           4.000000\n",
       "max           5.000000\n",
       "Name: rating, dtype: float64"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_ratedmovies_timestamps['rating'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Average User Rating: 3.5\n",
    "\n",
    "Min User Rating: 0.5\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>movieID</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>rtAudienceNumRatings</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1162160236000</td>\n",
       "      <td>10489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>783</td>\n",
       "      <td>3</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1162488835000</td>\n",
       "      <td>10489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>788</td>\n",
       "      <td>3</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1188564581000</td>\n",
       "      <td>10489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1160</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1216125522000</td>\n",
       "      <td>10489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1174</td>\n",
       "      <td>3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1111555400000</td>\n",
       "      <td>10489</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID  movieID  rating      timestamp  rtAudienceNumRatings\n",
       "0      75        3     1.0  1162160236000                 10489\n",
       "1     783        3     2.0  1162488835000                 10489\n",
       "2     788        3     3.5  1188564581000                 10489\n",
       "3    1160        3     4.0  1216125522000                 10489\n",
       "4    1174        3     2.5  1111555400000                 10489"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userandpopularity=pd.merge(user_ratedmovies_timestamps,movie,left_on='movieID',right_on='id')\n",
    "userandpopularity=userandpopularity[['userID', 'movieID', 'rating', 'timestamp','rtAudienceNumRatings']]\n",
    "userandpopularity.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>movieID</th>\n",
       "      <th>rtAudienceNumRatings</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Romance</th>\n",
       "      <th>Action</th>\n",
       "      <th>Crime</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Children</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>War</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>Musical</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75</td>\n",
       "      <td>3</td>\n",
       "      <td>10489</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>783</td>\n",
       "      <td>3</td>\n",
       "      <td>10489</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>788</td>\n",
       "      <td>3</td>\n",
       "      <td>10489</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1160</td>\n",
       "      <td>3</td>\n",
       "      <td>10489</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1174</td>\n",
       "      <td>3</td>\n",
       "      <td>10489</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID  movieID  rtAudienceNumRatings  Drama  Comedy  Thriller  Romance  \\\n",
       "0      75        3                 10489      0       1         0        1   \n",
       "1     783        3                 10489      0       1         0        1   \n",
       "2     788        3                 10489      0       1         0        1   \n",
       "3    1160        3                 10489      0       1         0        1   \n",
       "4    1174        3                 10489      0       1         0        1   \n",
       "\n",
       "   Action  Crime  Adventure  Horror  Sci-Fi  Fantasy  Children  Mystery  War  \\\n",
       "0       0      0          0       0       0        0         0        0    0   \n",
       "1       0      0          0       0       0        0         0        0    0   \n",
       "2       0      0          0       0       0        0         0        0    0   \n",
       "3       0      0          0       0       0        0         0        0    0   \n",
       "4       0      0          0       0       0        0         0        0    0   \n",
       "\n",
       "   Documentary  Musical  rating  \n",
       "0            0        0     1.0  \n",
       "1            0        0     2.0  \n",
       "2            0        0     3.5  \n",
       "3            0        0     4.0  \n",
       "4            0        0     2.5  "
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userpopularitygenre=pd.merge(userandpopularity,GenreEncodedData,on='movieID')\n",
    "userpopularitygenre.columns\n",
    "userpopularitygenre=userpopularitygenre[['userID', 'movieID','rtAudienceNumRatings','Drama', 'Comedy', 'Thriller', 'Romance',\n",
    "       'Action', 'Crime', 'Adventure', 'Horror', 'Sci-Fi', 'Fantasy',\n",
    "       'Children', 'Mystery', 'War', 'Documentary', 'Musical', 'rating']]#dropping timestamp for the time being\n",
    "userpopularitygenre.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2.1: Applying a model to predict a user's rating**<br>\n",
    "1. Using Surprise Library: Apply Model, Evaluate\n",
    "2. Split dataset to: Train/Test\n",
    "3. Evaluation Criteria: RMSE, MAE\n",
    "4. Because of many reasons, misclassification is reported on multiple papers as a signficant issue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm SVD on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.7684  0.7715  0.7680  0.7740  0.7691  0.7702  0.0023  \n",
      "MAE (testset)     0.5818  0.5837  0.5803  0.5865  0.5827  0.5830  0.0021  \n",
      "Fit time          43.27   43.74   43.86   44.37   43.79   43.81   0.35    \n",
      "Test time         1.64    1.57    1.61    1.54    1.85    1.64    0.11    \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'test_rmse': array([0.76835122, 0.77153952, 0.76799514, 0.77397514, 0.76906668]),\n",
       " 'test_mae': array([0.58175047, 0.58372328, 0.58030365, 0.58648295, 0.58270518]),\n",
       " 'fit_time': (43.27178192138672,\n",
       "  43.73742175102234,\n",
       "  43.85758137702942,\n",
       "  44.37079095840454,\n",
       "  43.793063163757324),\n",
       " 'test_time': (1.644254446029663,\n",
       "  1.5735843181610107,\n",
       "  1.6084952354431152,\n",
       "  1.5400381088256836,\n",
       "  1.8503365516662598)}"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reader = Reader()\n",
    "algo = SVD()\n",
    "data = Dataset.load_from_df(user_ratedmovies_timestamps[['userID','movieID','rating']], reader)\n",
    "cross_validate(algo, data, measures=['RMSE', 'MAE'], cv=5, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2.2 : Train-Test split**\n",
    "1. We split dataset with 25% of it reserved for testing \n",
    "2. We train and measure accuracy on RMSE and MAE\n",
    "3. RMSE gives us: concentration of data w.r.t best-fit line\n",
    "4. MAE gives us: how big of an error can we expect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = Reader()\n",
    "data = Dataset.load_from_df(user_ratedmovies_timestamps[['userID','movieID','rating']], reader)\n",
    "trainset, testset = train_test_split(data, test_size=.25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2.3: Benchmarking**\n",
    "1. Benchmark will give us a list of models that work best on default\n",
    "2. We will then optimize the top-3 and aggregate their rating for our predictions\n",
    "3. We will use the best models for collaborative filtering\n",
    "4. We will benchmark and select top-3 and optimize them for our rating predictions\n",
    "\n",
    "**Fun Fact: Netflix competition winner used SVD and since then, they have been very popular for the job**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 12%|██████████▍                                                                        | 1/8 [03:57<27:42, 237.46s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm SVD on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.7724  0.7722  0.7704  0.7691  0.7692  0.7707  0.0014  \n",
      "MAE (testset)     0.5851  0.5849  0.5824  0.5823  0.5818  0.5833  0.0014  \n",
      "Fit time          43.25   43.87   44.94   43.90   43.95   43.98   0.54    \n",
      "Test time         1.75    1.62    1.77    1.62    1.69    1.69    0.07    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 25%|████████████████████▊                                                              | 2/8 [04:21<17:21, 173.51s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm NormalPredictor on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    1.3779  1.3762  1.3768  1.3759  1.3775  1.3769  0.0007  \n",
      "MAE (testset)     1.0981  1.0980  1.0985  1.0966  1.0985  1.0979  0.0007  \n",
      "Fit time          1.13    1.44    1.51    1.50    1.62    1.44    0.16    \n",
      "Test time         1.68    1.63    1.75    1.99    1.81    1.77    0.12    \n",
      "Estimating biases using als...\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Estimating biases using als...\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Estimating biases using als...\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Estimating biases using als...\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Estimating biases using als...\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 38%|███████████████████████████████▏                                                   | 3/8 [12:40<22:35, 271.03s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm KNNBaseline on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.7933  0.7889  0.7901  0.7895  0.7919  0.7907  0.0016  \n",
      "MAE (testset)     0.6014  0.5986  0.5993  0.5989  0.6008  0.5998  0.0011  \n",
      "Fit time          23.63   24.35   23.81   22.99   23.42   23.64   0.45    \n",
      "Test time         75.17   72.60   75.86   73.21   75.04   74.37   1.25    \n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 50%|█████████████████████████████████████████▌                                         | 4/8 [20:08<21:36, 324.25s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm KNNBasic on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.8181  0.8220  0.8183  0.8181  0.8174  0.8188  0.0017  \n",
      "MAE (testset)     0.6177  0.6219  0.6185  0.6179  0.6169  0.6186  0.0017  \n",
      "Fit time          19.63   20.09   20.71   20.40   20.76   20.32   0.42    \n",
      "Test time         64.70   68.70   67.23   69.25   68.11   67.60   1.60    \n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 62%|███████████████████████████████████████████████████▉                               | 5/8 [27:49<18:15, 365.10s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm KNNWithMeans on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.8056  0.8077  0.8101  0.8091  0.8069  0.8079  0.0016  \n",
      "MAE (testset)     0.6154  0.6175  0.6184  0.6178  0.6161  0.6170  0.0011  \n",
      "Fit time          20.10   20.01   20.92   20.44   20.43   20.38   0.32    \n",
      "Test time         69.22   68.88   71.51   70.74   69.57   69.98   0.99    \n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 75%|██████████████████████████████████████████████████████████████▎                    | 6/8 [35:25<13:05, 392.57s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm KNNWithZScore on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.8047  0.8058  0.8053  0.8041  0.8063  0.8052  0.0008  \n",
      "MAE (testset)     0.6105  0.6116  0.6107  0.6114  0.6122  0.6112  0.0006  \n",
      "Fit time          20.79   20.68   20.43   20.30   20.46   20.53   0.18    \n",
      "Test time         69.49   68.89   68.87   69.30   69.04   69.12   0.24    \n",
      "Estimating biases using als...\n",
      "Estimating biases using als...\n",
      "Estimating biases using als...\n",
      "Estimating biases using als...\n",
      "Estimating biases using als...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 88%|████████████████████████████████████████████████████████████████████████▋          | 7/8 [35:58<04:44, 284.48s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm BaselineOnly on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.8078  0.8100  0.8127  0.8105  0.8081  0.8098  0.0018  \n",
      "MAE (testset)     0.6145  0.6170  0.6190  0.6161  0.6141  0.6161  0.0018  \n",
      "Fit time          3.07    3.29    3.31    3.29    3.30    3.25    0.09    \n",
      "Test time         1.46    1.43    2.08    1.48    1.44    1.58    0.25    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████████████| 8/8 [38:00<00:00, 285.06s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating RMSE, MAE of algorithm CoClustering on 5 split(s).\n",
      "\n",
      "                  Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     \n",
      "RMSE (testset)    0.8373  0.8389  0.8398  0.8388  0.8375  0.8385  0.0009  \n",
      "MAE (testset)     0.6511  0.6526  0.6538  0.6536  0.6521  0.6526  0.0010  \n",
      "Fit time          20.68   21.92   21.19   20.90   20.96   21.13   0.43    \n",
      "Test time         1.69    1.72    1.74    2.36    1.63    1.83    0.27    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_rmse</th>\n",
       "      <th>test_mae</th>\n",
       "      <th>fit_time</th>\n",
       "      <th>test_time</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Algorithm</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.770680</td>\n",
       "      <td>0.583283</td>\n",
       "      <td>43.982725</td>\n",
       "      <td>1.690108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KNNBaseline</th>\n",
       "      <td>0.790740</td>\n",
       "      <td>0.599798</td>\n",
       "      <td>23.641201</td>\n",
       "      <td>74.373301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KNNWithZScore</th>\n",
       "      <td>0.805236</td>\n",
       "      <td>0.611247</td>\n",
       "      <td>20.531046</td>\n",
       "      <td>69.117370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KNNWithMeans</th>\n",
       "      <td>0.807878</td>\n",
       "      <td>0.617037</td>\n",
       "      <td>20.379594</td>\n",
       "      <td>69.982161</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BaselineOnly</th>\n",
       "      <td>0.809805</td>\n",
       "      <td>0.616130</td>\n",
       "      <td>3.252750</td>\n",
       "      <td>1.576678</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KNNBasic</th>\n",
       "      <td>0.818782</td>\n",
       "      <td>0.618583</td>\n",
       "      <td>20.317105</td>\n",
       "      <td>67.598210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CoClustering</th>\n",
       "      <td>0.838467</td>\n",
       "      <td>0.652649</td>\n",
       "      <td>21.130931</td>\n",
       "      <td>1.830551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NormalPredictor</th>\n",
       "      <td>1.376870</td>\n",
       "      <td>1.097919</td>\n",
       "      <td>1.437893</td>\n",
       "      <td>1.771484</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 test_rmse  test_mae   fit_time  test_time\n",
       "Algorithm                                                 \n",
       "SVD               0.770680  0.583283  43.982725   1.690108\n",
       "KNNBaseline       0.790740  0.599798  23.641201  74.373301\n",
       "KNNWithZScore     0.805236  0.611247  20.531046  69.117370\n",
       "KNNWithMeans      0.807878  0.617037  20.379594  69.982161\n",
       "BaselineOnly      0.809805  0.616130   3.252750   1.576678\n",
       "KNNBasic          0.818782  0.618583  20.317105  67.598210\n",
       "CoClustering      0.838467  0.652649  21.130931   1.830551\n",
       "NormalPredictor   1.376870  1.097919   1.437893   1.771484"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "benchmark = []\n",
    "# Iterate over all algorithms\n",
    "reader = Reader()\n",
    "data = Dataset.load_from_df(user_ratedmovies_timestamps[['userID','movieID','rating']], reader)\n",
    "for algorithm in tqdm([SVD(), NormalPredictor(), KNNBaseline(), KNNBasic(), KNNWithMeans(), KNNWithZScore(), BaselineOnly(), CoClustering()]):\n",
    "    results = cross_validate(algorithm, data, measures=['RMSE','MAE'], cv=5, verbose=True)\n",
    "    tmp = pd.DataFrame.from_dict(results).mean(axis=0)\n",
    "    tmp = tmp.append(pd.Series([str(algorithm).split(' ')[0].split('.')[-1]], index=['Algorithm']))\n",
    "    benchmark.append(tmp)\n",
    "    \n",
    "ModelData=pd.DataFrame(benchmark).set_index('Algorithm').sort_values('test_rmse')\n",
    "ModelData"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2.4 Selected Models:**\n",
    "1. SVD\n",
    "2. KNNBaseline\n",
    "3. KNNWithZScore\n",
    "\n",
    "**Note: SVD will be hyperparameter-tuned and all other models will remain in their default state**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8160323823780523\n",
      "{'n_epochs': 15, 'lr_all': 0.01, 'reg_all': 0.2}\n"
     ]
    }
   ],
   "source": [
    "param_grid = {'n_epochs': [5, 10, 15], 'lr_all': [0.002, 0.005,0.001,0.01],\n",
    "              'reg_all': [0.4, 0.6, 0.2]}\n",
    "\n",
    "gs = GridSearchCV(SVD, param_grid, measures=['rmse', 'mae'], cv=5)\n",
    "gs.fit(data)\n",
    "# best RMSE score\n",
    "print(gs.best_score['rmse'])\n",
    "# combination of parameters that gave the best RMSE score\n",
    "print(gs.best_params['rmse'])\n",
    "algo = gs.best_estimator['rmse']\n",
    "algo.fit(trainset)\n",
    "predictions = algo.test(testset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE: 0.8198\n",
      "MAE:  0.6265\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.6264692701334962"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy.rmse(predictions)\n",
    "accuracy.mae(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimating biases using als...\n",
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "RMSE: 0.7951\n",
      "MAE:  0.6030\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.6030378688936903"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "algo1=KNNBaseline()\n",
    "algo1.fit(trainset)\n",
    "predictions1 = algo1.test(testset)\n",
    "accuracy.rmse(predictions1)\n",
    "accuracy.mae(predictions1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing the msd similarity matrix...\n",
      "Done computing similarity matrix.\n",
      "RMSE: 0.8097\n",
      "MAE:  0.6147\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.6147069355241408"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "algo2=KNNWithZScore()\n",
    "algo2.fit(trainset)\n",
    "predictions2 = algo2.test(testset)\n",
    "accuracy.rmse(predictions2)\n",
    "accuracy.mae(predictions2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2.5: Performing heterogenous ensemble**\n",
    "1. Using UserID and Movie ID, predict on top-3 models\n",
    "2. Extract estimated score\n",
    "3. Aggregate the result "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "def EnsembleCall(UID,IID):\n",
    "    predictor1=algo.predict(uid=UID,iid=IID)\n",
    "    predictor2=algo1.predict(uid=UID,iid=IID)\n",
    "    predictor3=algo2.predict(uid=UID,iid=IID)\n",
    "    est1=predictor1.est\n",
    "    est2=predictor2.est\n",
    "    est3=predictor3.est\n",
    "    mean=(est1+est2+est3)/3\n",
    "    return mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 2.6: Reccomender as functions**<br>\n",
    "<br>**Content-Based Reccomender**\n",
    "1. Title to MovieID mapping\n",
    "2. Index to Title mapping\n",
    "3. Fetch top 10 movies based on title \n",
    "4. Display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "TitleGenreEncodings= TitleGenreEncodings.reset_index()\n",
    "titles = TitleGenreEncodings['title']\n",
    "indices = pd.Series(TitleGenreEncodings.index, index=TitleGenreEncodings['title'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [],
   "source": [
    "TitleGenreActorDirectorTagCountry= TitleGenreActorDirectorTagCountry.reset_index()\n",
    "titles = TitleGenreActorDirectorTagCountry['title']\n",
    "indices = pd.Series(TitleGenreActorDirectorTagCountry.index, index=TitleGenreActorDirectorTagCountry['title'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6644,)"
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titles.shape ##if 2262 -> TitleGenreActorDirectorTagCountry dataframe is in play! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [],
   "source": [
    "def CBReccomender(title): #based on Titles, Genres, Tags, Country, Actor, and Director \n",
    "    Top_n = 10\n",
    "    idx=indices[title]\n",
    "    #title='Live Free or Die Hard'\n",
    "    scores=similarity1[idx].argsort()[-Top_n:][::-1]\n",
    "    movie_id = TitleGenreActorDirectorTagCountry[TitleGenreActorDirectorTagCountry[\"title\"]==title][\"movieID\"].values[0]\n",
    "    print(movie_id)\n",
    "    reccomendations=[]\n",
    "    for index in scores:\n",
    "        reccomendations.append(TitleGenreActorDirectorTagCountry['title'].values[index])\n",
    "    reccomendations.pop(0)\n",
    "    return reccomendations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "592\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Batman Begins',\n",
       " 'Superman Returns',\n",
       " 'Batman Returns',\n",
       " 'Superman III',\n",
       " 'Teenage Mutant Ninja Turtles II: The Secret of the Ooze',\n",
       " 'Strange Days',\n",
       " 'Shakespeare in Love',\n",
       " 'Repo Man',\n",
       " 'Midnight Cowboy']"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CBReccomender(\"Batman\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Oh Look! our Content-Based Reccomender is looking really good with the reccomendations!**\n",
    "1. Look at all the relevant movie titles being together\n",
    "2. Look at all the Genre-Wise hints being present in the retrieved result\n",
    "3. The first movie being the same as the movie required acts as a verification that it works<br>\n",
    "<br>\n",
    "\n",
    "**Note: Other fun tests were done to note how movies like The Godfather retrieved movies with \"The\" in their title hinting at the fact that before embedding, we should have filtered out the stop words.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "858\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Casablanca',\n",
       " 'The Treasure of the Sierra Madre',\n",
       " 'The Godfather: Part II',\n",
       " 'The Grapes of Wrath',\n",
       " 'The Maltese Falcon',\n",
       " 'The Shape of Things',\n",
       " 'Lola rennt',\n",
       " 'Il postino',\n",
       " 'Persepolis']"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CBReccomender(\"The Godfather\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Results like Lola rennt, and other Non-USA titles show that we could have increased the influence of USA-NUSA by duplicating the columns**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With all the intermediate predictions done above and multiple similarity matrices made and stored above, tested out, this one seemed the most generalizable across many titles and was kept for display<br>\n",
    "**Note: Anyone with access to the notebook can try out the many combinations w.r.t Titles-Genre, Genre-Tags, Actor-Director etc** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>**Section 2.7: Hybrid Reccomender**\n",
    "1. Take UserID -> Last 3 movies with respect to timestamps\n",
    "2. On the titles extracted -> Apply Content-Based Reccommender from above\n",
    "3. Predict User's rating via Ensembler \n",
    "4. Sort on estimates -> Display!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [],
   "source": [
    "def HybridReccomender(UserID):\n",
    "    UID=UserID\n",
    "    user_ratedmovies_timestamps['userID']=user_ratedmovies_timestamps['userID'].astype(int)\n",
    "    rslt_df = user_ratedmovies_timestamps[user_ratedmovies_timestamps['movieID'].isin(TitleGenreActorDirectorTagCountry['movieID'])] \n",
    "    #rslt_df filters out movies w.r.t to qualified movies\n",
    "    movietimestamped=rslt_df[rslt_df[\"userID\"]==UID].sort_values('timestamp')['movieID'].values[:3]\n",
    "    IID=movietimestamped\n",
    "    movielist=[]\n",
    "    title=[]\n",
    "    for i in range(3):\n",
    "        #print(i)\n",
    "        moviename=TitleGenreActorDirectorTagCountry[TitleGenreActorDirectorTagCountry['movieID']==IID[i]]['title'].values\n",
    "        #print(moviename[i])\n",
    "        title=moviename[0]\n",
    "        print(title)\n",
    "        movielist.extend(CBReccomender(title))\n",
    "    SelectedMovies=[]\n",
    "    for xyz in range(27):\n",
    "        x=(TitleGenreActorDirectorTagCountry[TitleGenreActorDirectorTagCountry[\"title\"]==movielist[xyz]][\"movieID\"].values[0])\n",
    "        SelectedMovies.append(x)\n",
    "    SelectedMovies=[filler for filler in SelectedMovies if filler not in IID] #filter to remove movie titles that CB built upon\n",
    "    SelectedMovies=[x for x in SelectedMovies if x not in IID]\n",
    "    #SelectedMovies\n",
    "    movieuser=list(zip(SelectedMovies,movielist))\n",
    "    #movieuser\n",
    "    #movieuser[1][0] # gives movieID\n",
    "    #movieuser[0][1] # gives movie title w.r.t movieID\n",
    "    testframe=pd.DataFrame(movieuser,columns=['movieID','Title'])\n",
    "    Length=len(testframe['movieID'])\n",
    "    #est=[]\n",
    "    for x in range(Length):\n",
    "        loopover=testframe.at[x,'movieID']\n",
    "        est=EnsembleCall(UID,loopover)\n",
    "        #print(testframe.loc[loopover])\n",
    "        testframe.loc[[x],'estimate']=est\n",
    "    #testframe=testframe[testframe['movieID']] #filter out movies already watched by the user\n",
    "    testframe[~testframe[\"movieID\"].isin(user_ratedmovies_timestamps[user_ratedmovies_timestamps[\"userID\"]==UID][\"movieID\"])]\n",
    "    #\n",
    "    testframe=testframe.sort_values('estimate',ascending=False)\n",
    "    #return testframe\n",
    "    #ReccomendUser=[]\n",
    "    ReccomendUser=testframe['Title'].values\n",
    "    return ReccomendUser\n",
    "    #for index in range(30):\n",
    "     #   ReccomendUser.extend(testframe['Title'].values[0])\n",
    "      #  print(ReccomendUser)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Back to the Future Part II\n",
      "2011\n",
      "Beverly Hills Cop III\n",
      "420\n",
      "Congo\n",
      "160\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['Back to the Future', '20000 Leagues Under the Sea',\n",
       "       \"Three O'Clock High\", 'The Naked Spur', 'Beverly Hills Cop',\n",
       "       'American Pimp', 'Summer School', 'Treasure Planet',\n",
       "       'Best of the Best', 'Lone Wolf McQuade', 'Code of Silence',\n",
       "       'The Distinguished Gentleman', 'The Rocketeer',\n",
       "       'Flight of the Intruder', 'Hudson Hawk', 'China Moon',\n",
       "       'Back to the Future Part III', 'The King of Marvin Gardens',\n",
       "       'Mutant Chronicles', 'Return of the Living Dead Part II',\n",
       "       'Beverly Hills Ninja', 'Best of the Best 2', 'Double Impact',\n",
       "       'Nutty Professor II: The Klumps', \"Big Momma's House 2\",\n",
       "       'The Crocodile Hunter: Collision Course', 'Double Dragon'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HybridReccomender(75)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Section 3: Finale**\n",
    "\n",
    "**Objective: Flask API**\n",
    "1. Pass UserID as extension to the url -> ?=idhere\n",
    "2. Result will be displayed to user - Personalized!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * Serving Flask app \"__main__\" (lazy loading)\n",
      " * Environment: production\n",
      "   WARNING: This is a development server. Do not use it in a production deployment.\n",
      "   Use a production WSGI server instead.\n",
      " * Debug mode: off\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)\n"
     ]
    }
   ],
   "source": [
    "app = Flask(__name__)\n",
    "@app.route('/<int:key>/',methods=[\"GET\"])\n",
    "def APICall(key):\n",
    "    movies = HybridReccomender(key).tolist()\n",
    "    result = {\"recommendations\":movies}\n",
    "    print(result)\n",
    "    return result\n",
    "    #UserID=int(input())\n",
    "    #print(request.data)\n",
    "    #return request.data\n",
    "app.run()\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**References:** <br>\n",
    "Acknowledging other people work is neccassary for without everyone assisting each other through the world wide web, we will have to reinvent the wheel, everytime.\n",
    "\n",
    "**Articles:**<br>\n",
    "https://towardsdatascience.com/a-simple-movie-recommendation-system-d135cfd0a22d<br>\n",
    "\n",
    "https://medium.com/data-science-101/movie-recommendation-system-content-filtering-7ba425ca0920<br>\n",
    "\n",
    "https://towardsdatascience.com/evaluation-metrics-for-recommender-systems-df56c6611093<br>\n",
    "\n",
    "https://www.geeksforgeeks.org/bag-of-words-bow-model-in-nlp/<br>\n",
    "\n",
    "https://www.mygreatlearning.com/blog/masterclass-on-movie-recommendation-system/<br>\n",
    "\n",
    "https://www.geeksforgeeks.org/python-implementation-of-movie-recommender-system/<br>\n",
    "\n",
    "https://medium.com/@cfpinela/how-to-evaluate-recommender-systems-874a313d7724<br>\n",
    "\n",
    "https://www.sciencedirect.com/topics/computer-science/cosine-similarity<br>\n",
    "\n",
    "https://blog.cambridgespark.com/tutorial-practical-introduction-to-recommender-systems-dbe22848392b<br>\n",
    "\n",
    "https://towardsdatascience.com/building-and-testing-recommender-systems-with-surprise-step-by-step-d4ba702ef80b<br>\n",
    "\n",
    "and many more!<br>\n",
    "\n",
    "**Kernels(Kaggle):**<br>\n",
    "\n",
    "https://www.kaggle.com/flaviobossolan/simple-efficient-movie-recommender<br>\n",
    "\n",
    "https://www.kaggle.com/myonin/music-recommendation-random-forest-xgboost<br>\n",
    "\n",
    "https://www.kaggle.com/rajmehra03/cf-based-recsys-by-low-rank-matrix-factorization<br>\n",
    "\n",
    "https://www.kaggle.com/rounakbanik/movie-recommender-systems<br>\n",
    "\n",
    "\n",
    "**Research Papers:**<br>\n",
    "\n",
    "Halder, Sajal & Sarkar, A.M.. (2012). Movie Recommendation System Based on Movie Swarm. 804-809. 10.1109/CGC.2012.121. <br>\n",
    "\n",
    "M. A. Shahjalal, Z. Ahmad, M. S. Arefin and M. R. T. Hossain, \"A user rating based collaborative filtering approach to predict movie preferences,\" 2017 3rd International Conference on Electrical Information and Communication Technology (EICT), Khulna, 2017, pp. 1-5, doi: 10.1109/EICT.2017.8275158.<br>\n",
    "\n",
    "Stanescu, Ana & Nagar, Swapnil & Caragea, Doina. (2013). A Hybrid Recommender System: User Profiling from Keywords and Ratings. 1. 73-80. 10.1109/WI-IAT.2013.11. <br>\n",
    "\n",
    "Urbanowicz, R., Meeker, M., La Cava, W., Olson, R. and Moore, J., 2020. Relief-Based Feature Selection: Introduction And Review.<br>\n",
    "\n",
    "**Books:** <br>\n",
    "\n",
    "Chapter 09: Reccomendation Systems - \"Mining of Massive Datasets\" - Jure Leskovec, Stanford Univ.\n",
    "<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1><center>Thank You!</center></h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "Daraz Task.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}