Mod3_lab3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Rushikasrithamaddula11/FMML-lab-1/blob/main/Mod3_lab3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kAKrrEmgLZ_J"
      },
      "source": [
        "# **FOUNDATIONS OF MODERN MACHINE LEARNING, IIIT Hyderabad**\n",
        "### MODULE: CLASSIFICATION-1\n",
        "### LAB-3 : Using KNN for Text Classification\n",
        "#### Module Coordinator: Jashn Arora\n",
        "\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Section 1: Understanding NLP tools**"
      ],
      "metadata": {
        "id": "_jeRAsurrdLY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this lab we will be using KNN on a real world NLP application i.e. is text classification. But first look at some NLP techniques for text classification and tools that we use when we want to use python for NLP."
      ],
      "metadata": {
        "id": "iB9N2m-HrgWY"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WMJhTF89MmT_"
      },
      "source": [
        "## Section 1.2: Data Cleaning and Preprocessing step"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nRXVjJHk3urf"
      },
      "source": [
        "Raw text must be processed and converted into a form so that it is suitable to use with various machine-learning algorithms.  \n",
        "In case of text, there are lots of things that need to be taken into account.  \n",
        "\n",
        "\n",
        "1.   Removing numbers from the text\n",
        "2.   Handling capitalization and punctuation.\n",
        "3.   Stemming and Lemmatizing text.  \n",
        "\n",
        "And most importantly, one can't just use words or images directly in algorithms; they need to be converted into vectors- a form that algorithms can understand.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **NLTK**\n",
        "NLTK (or Natural Language Tool Kit) is a commonly used library for processing text. We will use this tool in this lab. Lets first install it.\n"
      ],
      "metadata": {
        "id": "IYq4np0xqUBr"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wFOZHWV3_ABt",
        "outputId": "34624d55-6330-4533-d035-63efeb7ec8f1"
      },
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "nltk.download('punkt')\n",
        "nltk.download('averaged_perceptron_tagger')\n",
        "nltk.download('wordnet')\n",
        "nltk.download('omw-1.4')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
            "[nltk_data]     /root/nltk_data...\n",
            "[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data] Downloading package omw-1.4 to /root/nltk_data...\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gpgq3SQK5IOr"
      },
      "source": [
        "import re\n",
        "import numpy\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "from nltk.stem import SnowballStemmer\n",
        "from nltk.tokenize import word_tokenize\n",
        "from nltk import pos_tag\n",
        "from nltk.corpus import wordnet\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
        "from bs4 import BeautifulSoup\n",
        "\n",
        "def cleanText(text, lemmatize, stemmer):\n",
        "    \"\"\"Method for cleaning text from train and test data. Removes numbers, punctuation, and capitalization. Stems or lemmatizes text.\"\"\"\n",
        "\n",
        "    if isinstance(text, float):\n",
        "        text = str(text)\n",
        "    if isinstance(text, numpy.int64):\n",
        "        text = str(text)\n",
        "    try:\n",
        "        text = text.decode()\n",
        "    except AttributeError:\n",
        "        pass\n",
        "\n",
        "    soup = BeautifulSoup(text, \"lxml\")\n",
        "    text = soup.get_text()\n",
        "    text = re.sub(r\"[^A-Za-z]\", \" \", text)\n",
        "    text = text.lower()\n",
        "\n",
        "\n",
        "    if lemmatize:\n",
        "        wordnet_lemmatizer = WordNetLemmatizer()\n",
        "\n",
        "        def get_tag(tag):\n",
        "            if tag.startswith('J'):\n",
        "                return wordnet.ADJ\n",
        "            elif tag.startswith('V'):\n",
        "                return wordnet.VERB\n",
        "            elif tag.startswith('N'):\n",
        "                return wordnet.NOUN\n",
        "            elif tag.startswith('R'):\n",
        "                return wordnet.ADV\n",
        "            else:\n",
        "                return ''\n",
        "\n",
        "        text_result = []\n",
        "        tokens = word_tokenize(text)  # Generate list of tokens\n",
        "        tagged = pos_tag(tokens)\n",
        "        for t in tagged:\n",
        "            try:\n",
        "                text_result.append(wordnet_lemmatizer.lemmatize(t[0], get_tag(t[1][:2])))\n",
        "            except:\n",
        "                text_result.append(wordnet_lemmatizer.lemmatize(t[0]))\n",
        "        return text_result\n",
        "\n",
        "    if stemmer:\n",
        "        text_result = []\n",
        "        tokens = word_tokenize(text)\n",
        "        snowball_stemmer = SnowballStemmer('english')\n",
        "        for t in tokens:\n",
        "            text_result.append(snowball_stemmer.stem(t))\n",
        "        return text_result"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sample_text = \"Troubling\"\n",
        "sample_text_result = cleanText(sample_text, lemmatize=False, stemmer=True)\n",
        "sample_text_result = \" \".join(str(x) for x in sample_text_result)\n",
        "print(sample_text)\n",
        "print(sample_text_result)\n",
        "sample_text_result = cleanText(sample_text, lemmatize=True, stemmer=False)\n",
        "sample_text_result = \" \".join(str(x) for x in sample_text_result)\n",
        "print(sample_text_result)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZuBxfzbuzrs1",
        "outputId": "5fef0725-2bad-42f0-a6be-817d908b7219"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Troubling\n",
            "troubl\n",
            "trouble\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QcqQi34UoPvq"
      },
      "source": [
        "## Section 1.2: BAG OF WORDS\n",
        "\n",
        "A bag-of-words model, or BoW for short, is a way of extracting features from text for use in modeling, such as with machine learning algorithms.\n",
        "\n",
        "The approach is very simple and flexible, and can be used in many ways for extracting features from documents.\n",
        "\n",
        "A bag-of-words is a representation of text that describes the occurrence of words within a document.\n",
        "It is called a “bag” of words, because any information about the order or structure of words in the document is discarded. The model is only concerned with whether known words occur in the document, not where in the document."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "5*12"
      ],
      "metadata": {
        "id": "FN8sZqXaNLe7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "485bbb67-2688-45e0-d71a-1101875ffdbe"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "60"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3YYSpQzIM05l"
      },
      "source": [
        "# Functions to convert document(s) to a list of words, with the option of removing stopwords. Returns document-term matrix.\n",
        "\n",
        "def createBagOfWords(train, test, remove_stopwords, lemmatize, stemmer):\n",
        "    if remove_stopwords:\n",
        "        vectorizer = CountVectorizer(analyzer='word', input='content', stop_words=stopwords.words('english'))\n",
        "    else:\n",
        "        vectorizer = CountVectorizer(analyzer='word', input='content')\n",
        "\n",
        "    clean_train = []\n",
        "    for paragraph in train:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_train.append(paragraph)\n",
        "\n",
        "    clean_test = []\n",
        "    for paragraph in test:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_test.append(paragraph)\n",
        "\n",
        "    bag_of_words_train = vectorizer.fit_transform(clean_train).toarray()\n",
        "    bag_of_words_test = vectorizer.transform(clean_test).toarray()\n",
        "    return bag_of_words_train, bag_of_words_test\n"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-v0iCUBqoX82"
      },
      "source": [
        "## Section 1.3: TF-IDF\n",
        "TF-IDF technique is used to find meaning of sentences consisting of words and cancels out the incapabilities of Bag of Words technique which is good for text classification or for helping a machine read words in numbers.\n",
        "\n",
        "The number of times a term occurs in a document is called its Term frequency (TF).\n",
        "\n",
        " Document frequency is the number of documents in which the word is present.  Inverse DF (IDF) is the inverse of the document frequency which measures the informativeness of term *t*.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L6_-DDMQoXEX"
      },
      "source": [
        "def createTFIDF(train, test, remove_stopwords, lemmatize, stemmer):\n",
        "    if remove_stopwords:\n",
        "        vectorizer = TfidfVectorizer(analyzer='word', input='content', stop_words=stopwords.words('english'))\n",
        "    else:\n",
        "        vectorizer =  TfidfVectorizer(analyzer='word', input='content')\n",
        "\n",
        "    clean_train = []\n",
        "    for paragraph in train:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_train.append(paragraph)\n",
        "\n",
        "    clean_test = []\n",
        "    for paragraph in test:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_test.append(paragraph)\n",
        "\n",
        "    tfidf_train = vectorizer.fit_transform(clean_train).toarray()\n",
        "    tfidf_test = vectorizer.transform(clean_test).toarray()\n",
        "    return tfidf_train, tfidf_test"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7g0jS45epcC5"
      },
      "source": [
        "# **Section 2: UNDERSTANDING THE DATA : A REVIEWS DATASET**\n",
        "\n",
        "Sentiment analysis is the interpretation and classification of emotions (such as positive, negative and neutral) within text data using text analysis techniques.  \n",
        "Given below is a dataset consisting of reviews along with sentiment class (positive or negative)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jU6875-NxrHw",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "3fcef5dc-9c42-4df9-c5d6-6273aff81fd5"
      },
      "source": [
        "# Upload the Reviews CSV file that has been shared with you.\n",
        "# Run this cell, click on the 'Choose files' button and upload the file.\n",
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-a1a997c7-5fcb-4209-90b9-2a65ce539ecd\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-a1a997c7-5fcb-4209-90b9-2a65ce539ecd\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving reviews.csv to reviews.csv\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('reviews.csv')"
      ],
      "metadata": {
        "id": "HILJMpa_y26e"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df = df.dropna()"
      ],
      "metadata": {
        "id": "WXD0iAR5k62v"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df"
      ],
      "metadata": {
        "id": "MTCScT7uOdUy",
        "outputId": "6d1ef68c-38dc-4302-c30d-6308bafff899",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        }
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                              sentence sentiment\n",
              "0    Not sure who was more lost - the flat characte...         0\n",
              "1    Attempting artiness with black & white and cle...         0\n",
              "2           Very little music or anything to speak of.         0\n",
              "3    The best scene in the movie was when Gerardo i...         1\n",
              "4    The rest of the movie lacks art, charm, meanin...         0\n",
              "..                                                 ...       ...\n",
              "994  I just got bored watching Jessice Lange take h...         0\n",
              "995  Unfortunately, any virtue in this film's produ...         0\n",
              "996                     In a word, it is embarrassing.         0\n",
              "997                                 Exceptionally bad!         0\n",
              "998  All in all its an insult to one's intelligence...         0\n",
              "\n",
              "[955 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-915a1616-fd34-403d-b123-7ec5c5c8c7df\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentence</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Not sure who was more lost - the flat characte...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Attempting artiness with black &amp; white and cle...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Very little music or anything to speak of.</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>The best scene in the movie was when Gerardo i...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>The rest of the movie lacks art, charm, meanin...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>994</th>\n",
              "      <td>I just got bored watching Jessice Lange take h...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>995</th>\n",
              "      <td>Unfortunately, any virtue in this film's produ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>996</th>\n",
              "      <td>In a word, it is embarrassing.</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>997</th>\n",
              "      <td>Exceptionally bad!</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>998</th>\n",
              "      <td>All in all its an insult to one's intelligence...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>955 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-915a1616-fd34-403d-b123-7ec5c5c8c7df')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-915a1616-fd34-403d-b123-7ec5c5c8c7df button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-915a1616-fd34-403d-b123-7ec5c5c8c7df');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-cb9cbfa4-51a3-48cf-9b56-98781548dfd5\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-cb9cbfa4-51a3-48cf-9b56-98781548dfd5')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-cb9cbfa4-51a3-48cf-9b56-98781548dfd5 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.to_csv('reviews.csv', index=False)"
      ],
      "metadata": {
        "id": "7j9rNGkQpn7y"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-QXo11Bxvytu"
      },
      "source": [
        "# **Section 3: KNN MODEL**\n",
        "\n",
        "Given below are two KNN models; in the first case we are using Bag-of-Words and in the second case we are using TF-IDF.\n",
        "Note the different metrics and parameters used in each."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R5yZXboZ92Z4"
      },
      "source": [
        "from sklearn import metrics, neighbors\n",
        "from sklearn.model_selection import train_test_split, cross_val_score, cross_val_predict\n",
        "\n",
        "## TASK - 1: Tweak the models below and see results with different parameters and distance metrics.\n",
        "\n",
        "def bow_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using bag-of-words and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('reviews.csv')\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"sentence\"], training_data[\"sentiment\"], test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createBagOfWords(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    # print(X_train)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='euclidean', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with BOW accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    print('\\n')\n",
        "    return predicted, y_test\n",
        "\n",
        "\n",
        "def tfidf_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using tf-idf and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('reviews.csv')\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"sentence\"], training_data[\"sentiment\"],\n",
        "                                                        test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createTFIDF(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    # print(X_train)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='distance', algorithm='brute', leaf_size=30, p=2,\n",
        "                                         metric='cosine', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with TFIDF accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    return predicted, y_test"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Note: Cross-validation will be discussed in detail in the upcoming lab session."
      ],
      "metadata": {
        "id": "uPuI3wrL-8ZJ"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xzno1rRNHoFT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3bb94c1f-36bf-41d2-e886-6d391871e997"
      },
      "source": [
        "## KNN accuracy after using BoW\n",
        "predicted, y_test = bow_knn()"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with BOW accuracy = 62.30366492146597%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/model_selection/_split.py:700: UserWarning: The least populated class in y has only 1 members, which is less than n_splits=3.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross Validation Accuracy: 0.62\n",
            "[0.60784314 0.58431373 0.66141732]\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FI5NMP8L-4eW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b722a499-ff5c-4bf3-cd14-672a6fbe425a"
      },
      "source": [
        "## KNN accuracy after using TFIDF\n",
        "predicted, y_test = tfidf_knn()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with TFIDF accuracy = 70.15706806282722%\n",
            "Cross Validation Accuracy: 0.73\n",
            "[0.7254902  0.74117647 0.72834646]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/model_selection/_split.py:700: UserWarning: The least populated class in y has only 1 members, which is less than n_splits=3.\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vm2vYDVfa5AP"
      },
      "source": [
        "# Section 4: SPAM TEXT DATASET\n",
        "Now let's use what we've learnt to classify texts as spam or not spam."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "id": "c1GZgvXCsKX1",
        "outputId": "200765de-db6d-4ab3-c558-96f55649ac9a"
      },
      "source": [
        "# Upload the spam text data CSV file that has been shared with you. You can also download the file from https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset\n",
        "# Run this cell, click on the 'Choose files' button and upload the file.\n",
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-00f98624-ee7b-4a5b-b78d-c90d8aef4a79\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-00f98624-ee7b-4a5b-b78d-c90d8aef4a79\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving spam.csv to spam.csv\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qRiS7dT7piTE",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 513
        },
        "outputId": "b436388c-5224-4fe4-8fef-323902151a00"
      },
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('spam.csv', error_bad_lines=False)\n",
        "df"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-16-3b52591eaf2b>:2: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.\n",
            "\n",
            "\n",
            "  df = pd.read_csv('spam.csv', error_bad_lines=False)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     Category                                            Message\n",
              "0         ham  Go until jurong point, crazy.. Available only ...\n",
              "1         ham                      Ok lar... Joking wif u oni...\n",
              "2        spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3         ham  U dun say so early hor... U c already then say...\n",
              "4         ham  Nah I don't think he goes to usf, he lives aro...\n",
              "...       ...                                                ...\n",
              "5567     spam  This is the 2nd time we have tried 2 contact u...\n",
              "5568      ham              Will Ã¼ b going to esplanade fr home?\n",
              "5569      ham  Pity, * was in mood for that. So...any other s...\n",
              "5570      ham  The guy did some bitching but I acted like i'd...\n",
              "5571      ham                         Rofl. Its true to its name\n",
              "\n",
              "[5572 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-46894108-e848-4763-8750-657dfda3af61\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Category</th>\n",
              "      <th>Message</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5567</th>\n",
              "      <td>spam</td>\n",
              "      <td>This is the 2nd time we have tried 2 contact u...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5568</th>\n",
              "      <td>ham</td>\n",
              "      <td>Will Ã¼ b going to esplanade fr home?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5569</th>\n",
              "      <td>ham</td>\n",
              "      <td>Pity, * was in mood for that. So...any other s...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5570</th>\n",
              "      <td>ham</td>\n",
              "      <td>The guy did some bitching but I acted like i'd...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5571</th>\n",
              "      <td>ham</td>\n",
              "      <td>Rofl. Its true to its name</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5572 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-46894108-e848-4763-8750-657dfda3af61')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-46894108-e848-4763-8750-657dfda3af61 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-46894108-e848-4763-8750-657dfda3af61');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-f12492a1-e72b-447c-9c38-66b3b82f4ddc\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f12492a1-e72b-447c-9c38-66b3b82f4ddc')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-f12492a1-e72b-447c-9c38-66b3b82f4ddc button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OsvBm4luNCME"
      },
      "source": [
        "df['Category'] = df['Category'].map({'ham': 0, 'spam': 1})"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KiEHuMypWyb6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "0290f6e4-fec2-4fee-e399-09acc486876a"
      },
      "source": [
        "df.head(5)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Category                                            Message\n",
              "0         0  Go until jurong point, crazy.. Available only ...\n",
              "1         0                      Ok lar... Joking wif u oni...\n",
              "2         1  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3         0  U dun say so early hor... U c already then say...\n",
              "4         0  Nah I don't think he goes to usf, he lives aro..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-63ba7c58-0c59-41a9-85f3-b1473f38c86b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Category</th>\n",
              "      <th>Message</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-63ba7c58-0c59-41a9-85f3-b1473f38c86b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-63ba7c58-0c59-41a9-85f3-b1473f38c86b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-63ba7c58-0c59-41a9-85f3-b1473f38c86b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-243268f3-1126-47d8-b554-31296d7af3d2\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-243268f3-1126-47d8-b554-31296d7af3d2')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-243268f3-1126-47d8-b554-31296d7af3d2 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(df)"
      ],
      "metadata": {
        "id": "WRJU9rFy1XQR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "75f1ff0f-7cfa-468b-8f58-b5e020fb0bac"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5572"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wnv0v4T6sqJQ"
      },
      "source": [
        "from sklearn import metrics, neighbors\n",
        "from sklearn.model_selection import train_test_split, cross_val_score, cross_val_predict\n",
        "\n",
        "## TASK - 2: Tweak the models below and see results with different parameters and distance metrics.\n",
        "\n",
        "def bow_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using bag-of-words and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('spam.csv')\n",
        "    training_data['Category'] = training_data['Category'].map({'ham': 0, 'spam': 1})\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"Message\"], training_data[\"Category\"], test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createBagOfWords(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='euclidean', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with BOW accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    print('\\n')\n",
        "    return predicted, y_test\n",
        "\n",
        "\n",
        "def tfidf_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using tf-idf and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('spam.csv')\n",
        "    training_data['Category'] = training_data['Category'].map({'ham': 0, 'spam': 1})\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"Message\"], training_data[\"Category\"], test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createTFIDF(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='distance', algorithm='brute', leaf_size=30, p=2, metric='cosine', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with TFIDF accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    return predicted, y_test"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W8PwydHYs1h_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5b612beb-c80b-4ca3-8f92-4f8d7566c4e0"
      },
      "source": [
        "# This cell may take some time to run\n",
        "predicted, y_test = bow_knn()"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with BOW accuracy = 92.19730941704036%\n",
            "Cross Validation Accuracy: 0.91\n",
            "[0.90713324 0.90040377 0.91245791]\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# This cell may take some time to run\n",
        "predicted, y_test = tfidf_knn()"
      ],
      "metadata": {
        "id": "zf8i2P1nxpl8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c78d0b5e-9a37-43d2-ce73-ff8186d09b52"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with TFIDF accuracy = 98.56502242152466%\n",
            "Cross Validation Accuracy: 0.97\n",
            "[0.96837147 0.96769852 0.96363636]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Questions to Think About and Answer\n",
        "1. Why does the TF-IDF approach generally result in a better accuracy than Bag-of-Words ?\n",
        "2. Can you think of techniques that are better than both BoW and TF-IDF ?\n",
        "3. Read about Stemming and Lemmatization from the resources given below. Think about the pros/cons of each."
      ],
      "metadata": {
        "id": "f9xvbzS4yLTr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Answer for question number:1**"
      ],
      "metadata": {
        "id": "D2bAQb3m-Z46"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "TF-IDF vs. Bag-of-Words:\n",
        "TF-IDF (Term Frequency-Inverse Document Frequency) and Bag-of-Words (BoW) are both techniques used in natural language processing for text representation, but they have distinct differences.\n",
        "\n",
        "TF-IDF:\n",
        "\n",
        "Pros:\n",
        "\n",
        "Takes into account the importance of words by considering their frequency in the document as well as their rarity across multiple documents.\n",
        "Penalizes common words that may not carry much meaning.\n",
        "Useful for tasks like information retrieval, where distinguishing the importance of words is crucial.\n",
        "Cons:\n",
        "\n",
        "Complexity: Calculating TF-IDF scores can be computationally expensive.\n",
        "Requires a large enough corpus to generate meaningful IDF scores.\n",
        "May not capture the semantic meaning of words.\n",
        "Bag-of-Words:\n",
        "\n",
        "Pros:\n",
        "\n",
        "Simple and computationally efficient.\n",
        "Easy to understand and implement.\n",
        "Can capture the presence of words in a document.\n",
        "Cons:\n",
        "\n",
        "Ignores word order and semantics.\n",
        "Treats all words equally, regardless of their importance or rarity.\n",
        "May result in a high-dimensional and sparse representation.\n"
      ],
      "metadata": {
        "id": "Sg1JpU78-kae"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Answer for question number:2**"
      ],
      "metadata": {
        "id": "1d9jVbIY-6I6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Techniques Better than BoW and TF-IDF:\n",
        "\n",
        "Word Embeddings:\n",
        "\n",
        "Techniques like Word2Vec, GloVe, and FastText capture semantic relationships between words. They represent words as vectors in a continuous vector space, preserving contextual and semantic information.\n",
        "Transformer-based Models:\n",
        "\n",
        "Models like BERT (Bidirectional Encoder Representations from Transformers) have shown remarkable performance in various NLP tasks by considering the context of words in both directions.\n",
        "Doc2Vec:\n",
        "\n",
        "An extension of Word2Vec, Doc2Vec represents entire documents as vectors, capturing the document's semantic meaning.\n",
        "Subword Embeddings:\n",
        "\n",
        "Approaches like Byte Pair Encoding (BPE) and SentencePiece tokenize words into subword units, capturing morphological information and handling out-of-vocabulary words.\n"
      ],
      "metadata": {
        "id": "baeBbos0-9N0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Answer for question number:3**"
      ],
      "metadata": {
        "id": "3X9o19kF_HRu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Stemming vs. Lemmatization:\n",
        "\n",
        "Stemming:\n",
        "\n",
        "Pros:\n",
        "Reduces words to their root or base form, simplifying the representation.\n",
        "Generally faster and computationally less expensive.\n",
        "Cons:\n",
        "May result in words that are not actual words.\n",
        "Over-stemming can lead to loss of meaning.\n",
        "Lemmatization:\n",
        "\n",
        "Pros:\n",
        "\n",
        "Generates valid words, preserving the semantic meaning.\n",
        "Better suited for tasks where word form matters.\n",
        "Cons:\n",
        "\n",
        "Can be computationally more intensive than stemming.\n",
        "Requires a lexicon or a language-specific resource for accurate lemmatization.\n",
        "The choice between stemming and lemmatization depends on the specific requirements of the task. Stemming is often used in information retrieval or when computational efficiency is crucial, while lemmatization is preferred when maintaining valid words and semantic meaning is essential.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "GEMG0k5a_Nmg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Useful Resources for further reading\n",
        "1. Stemming and Lemmatization: https://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html\n",
        "2. TF-IDF and BoW : https://www.analyticsvidhya.com/blog/2020/02/quick-introduction-bag-of-words-bow-tf-idf/\n",
        "3. TF-IDF: https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html\n"
      ],
      "metadata": {
        "id": "T6xPL6smyWG-"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "-uUw16_tkGdV"
      },
      "execution_count": 22,
      "outputs": []
    }
  ]
}