Project 8 - Data Cleaning on Feedback Dataset.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyN3AZ0pyKzG3H5+bR+1owjH"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["**Project 8 - Data Cleaning on Feedback Dataset**"],"metadata":{"id":"5QWK5n_DJDmx"}},{"cell_type":"markdown","source":["**Data cleaning** is the process of correcting or removing corrupt, incorrect, or unnecessary data from a data set before data analysis.\n","\n","Expanding on this basic definition, data cleaning, often grouped with data cleansing, data scrubbing, and data preparation, serves to turn your messy, potentially problematic data into clean data. Importantly, that’s ‘clean data’ defined as data that the powerful data analysis engines you spent money on can actually use."],"metadata":{"id":"pCq5eX06Qiy6"}},{"cell_type":"code","execution_count":63,"metadata":{"id":"8nHQXTq6BdHS","executionInfo":{"status":"ok","timestamp":1673849489544,"user_tz":-330,"elapsed":469,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}}},"outputs":[],"source":["#Let’s get Pandas and NumPy up and running on your Python script\n","import pandas as pd\n","import numpy as np"]},{"cell_type":"code","source":["#Input Customer Feedback Dataset\n","data = pd.read_csv('feedbackTable.csv')\n","data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"IMd1EK3DBwG6","executionInfo":{"status":"ok","timestamp":1673851217709,"user_tz":-330,"elapsed":544,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"ab36c6b8-dc96-418e-a64f-eb68e8ad4457"},"execution_count":71,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                Works well   \n","1        3               good enough   \n","2        5  Everyone should buy this   \n","3        5           Amazing product   \n","4        1          this is terrible   \n","5        2               Doesnt work   \n","6        4              cool product   \n","7        5           BEST THING EVER   \n","8        5           Amazing product   \n","9        5               Love this!!   \n","10     100                 Hate this   \n","11       3                OK product   \n","\n","                                               Review Customer Name  \\\n","0   \\nThe product works fine, it is maybe a little...       Phillip   \n","1                                                 NaN         elena   \n","2                                You should buy this.        Olivia   \n","3    Love everything about this product,it works g...          John   \n","4                    The product never worked for me.         Paula   \n","5                     This doesnt work as advertised.         Ellie   \n","6    This worked well for me,Not 5 stars because t...          DAVE   \n","7            Go and buy this right now, it's amazing.         Pablo   \n","8    Love everything about this product,it works g...          John   \n","9              Iwould 100% recommend this to everone.          CARA   \n","10                    This doesnt do anylhing for me.         Helen   \n","11  It does what it has to do, but the user experi...          emma   \n","\n","                Date Review ID  \n","0    October 10 2021     #7653  \n","1     October 5 2021       NaN  \n","2                NaN       NaN  \n","3          10/5/2021       NaN  \n","4          44,491.00     #8563  \n","5                NaN       NaN  \n","6   September 5 2021     #4162  \n","7                NaN       NaN  \n","8          10/5/2021     #5675  \n","9                NaN       NaN  \n","10  September 5 2021       NaN  \n","11               NaN     #7553  "],"text/html":["\n","  <div id=\"df-b6c5e474-9bfb-408b-9d92-2cb9461a3be5\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","      <th>Date</th>\n","      <th>Review ID</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>Works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","      <td>October 10 2021</td>\n","      <td>#7653</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>NaN</td>\n","      <td>elena</td>\n","      <td>October 5 2021</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>Everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","      <td>NaN</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","      <td>10/5/2021</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","      <td>44,491.00</td>\n","      <td>#8563</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>Doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","      <td>NaN</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>DAVE</td>\n","      <td>September 5 2021</td>\n","      <td>#4162</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>BEST THING EVER</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","      <td>NaN</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","      <td>10/5/2021</td>\n","      <td>#5675</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>Love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>CARA</td>\n","      <td>NaN</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>100</td>\n","      <td>Hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","      <td>September 5 2021</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>OK product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>emma</td>\n","      <td>NaN</td>\n","      <td>#7553</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b6c5e474-9bfb-408b-9d92-2cb9461a3be5')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-b6c5e474-9bfb-408b-9d92-2cb9461a3be5 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-b6c5e474-9bfb-408b-9d92-2cb9461a3be5');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":71}]},{"cell_type":"markdown","source":["**Locate Missing Data**"],"metadata":{"id":"CSle1CzqJla_"}},{"cell_type":"code","source":["data.isnull()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"plw8fylxByoI","executionInfo":{"status":"ok","timestamp":1673851266034,"user_tz":-330,"elapsed":539,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"37bbae94-6548-43ff-abb5-42bf73d065ea"},"execution_count":72,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating  Review Title  Review  Customer Name   Date  Review ID\n","0    False         False   False          False  False      False\n","1    False         False    True          False  False       True\n","2    False         False   False          False   True       True\n","3    False         False   False          False  False       True\n","4    False         False   False          False  False      False\n","5    False         False   False          False   True       True\n","6    False         False   False          False  False      False\n","7    False         False   False          False   True       True\n","8    False         False   False          False  False      False\n","9    False         False   False          False   True       True\n","10   False         False   False          False  False       True\n","11   False         False   False          False   True      False"],"text/html":["\n","  <div id=\"df-37e8f6bc-fecb-4219-9479-548198d96142\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","      <th>Date</th>\n","      <th>Review ID</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>False</td>\n","      <td>True</td>\n","      <td>False</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-37e8f6bc-fecb-4219-9479-548198d96142')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-37e8f6bc-fecb-4219-9479-548198d96142 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-37e8f6bc-fecb-4219-9479-548198d96142');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":72}]},{"cell_type":"markdown","source":["Our output result is a list of boolean values. \n","\n","There are several insights the list can give us. First and foremost is where the missing data is – any ‘True’ reading under a column indicates missing data in that column’s category for that data file.\n","\n","So, for example, datapoint 1 has missing data in its Review section and its Review ID section (both are marked true). \n","\n","We can further expand the missing data of each feature by coding:"],"metadata":{"id":"7d_FU9v4JuXq"}},{"cell_type":"code","source":["data.isnull().sum()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"GyeoSz03B1Ee","executionInfo":{"status":"ok","timestamp":1673851378977,"user_tz":-330,"elapsed":534,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"08cce07c-05a2-47b8-e6c6-234273263484"},"execution_count":73,"outputs":[{"output_type":"execute_result","data":{"text/plain":["Rating           0\n","Review Title     0\n","Review           1\n","Customer Name    0\n","Date             5\n","Review ID        7\n","dtype: int64"]},"metadata":{},"execution_count":73}]},{"cell_type":"markdown","source":["From here, we use code to actually clean the data. This boils down to two basic options. \n","\n","1) Drop the data or, \n","\n","2) Input missing data. If you opt to:"],"metadata":{"id":"sJH3WNxpJ0Jj"}},{"cell_type":"markdown","source":["**Drop the data**\n","\n","You’ll have to make another decision – whether to drop only the missing values and keep the data in the set, or to eliminate the feature (the entire column) wholesale because there are so many missing datapoints that it isn’t fit for analysis.\n","\n","If you want to drop the missing values you’ll have to go in and mark them void according to Pandas or NumPy standards.\n","But if you want to drop the entire column, here’s the code:"],"metadata":{"id":"Lo9B6vcyJ_yP"}},{"cell_type":"code","source":["remove = ['Review ID','Date']\n","data.drop(remove, inplace =True, axis =1)"],"metadata":{"id":"ZDnXcmtXB4dY","executionInfo":{"status":"ok","timestamp":1673851486713,"user_tz":-330,"elapsed":6,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}}},"execution_count":74,"outputs":[]},{"cell_type":"code","source":["data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"Bb7uKe2fHWkL","executionInfo":{"status":"ok","timestamp":1673851490503,"user_tz":-330,"elapsed":538,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"f8ff0d52-11b1-4c81-8f41-5afdce79fbb5"},"execution_count":75,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                Works well   \n","1        3               good enough   \n","2        5  Everyone should buy this   \n","3        5           Amazing product   \n","4        1          this is terrible   \n","5        2               Doesnt work   \n","6        4              cool product   \n","7        5           BEST THING EVER   \n","8        5           Amazing product   \n","9        5               Love this!!   \n","10     100                 Hate this   \n","11       3                OK product   \n","\n","                                               Review Customer Name  \n","0   \\nThe product works fine, it is maybe a little...       Phillip  \n","1                                                 NaN         elena  \n","2                                You should buy this.        Olivia  \n","3    Love everything about this product,it works g...          John  \n","4                    The product never worked for me.         Paula  \n","5                     This doesnt work as advertised.         Ellie  \n","6    This worked well for me,Not 5 stars because t...          DAVE  \n","7            Go and buy this right now, it's amazing.         Pablo  \n","8    Love everything about this product,it works g...          John  \n","9              Iwould 100% recommend this to everone.          CARA  \n","10                    This doesnt do anylhing for me.         Helen  \n","11  It does what it has to do, but the user experi...          emma  "],"text/html":["\n","  <div id=\"df-bed64a1f-f255-4b39-bc14-1052c2582165\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>Works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>NaN</td>\n","      <td>elena</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>Everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>Doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>DAVE</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>BEST THING EVER</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>Love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>CARA</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>100</td>\n","      <td>Hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>OK product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>emma</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bed64a1f-f255-4b39-bc14-1052c2582165')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-bed64a1f-f255-4b39-bc14-1052c2582165 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-bed64a1f-f255-4b39-bc14-1052c2582165');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":75}]},{"cell_type":"markdown","source":["Now, let’s examine our other option.\n","**Input missing data**\n","\n","Technically, the method described above of filling in individual values with Pandas or NumBy standards is also a form of inputting missing data – we call it adding ‘No Review’. When it comes to inputting missing data you can either add ‘No Review’ using the code below, or manually fill in the correct data.\n","\n"],"metadata":{"id":"UyGhwYyCKQ-s"}},{"cell_type":"code","source":["data['Review'] = data['Review'].fillna('No review')\n","data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"6XVI5kZjB6lE","executionInfo":{"status":"ok","timestamp":1673851576446,"user_tz":-330,"elapsed":573,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"1f398d95-003a-4d29-f292-4b869d94eae3"},"execution_count":77,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                Works well   \n","1        3               good enough   \n","2        5  Everyone should buy this   \n","3        5           Amazing product   \n","4        1          this is terrible   \n","5        2               Doesnt work   \n","6        4              cool product   \n","7        5           BEST THING EVER   \n","8        5           Amazing product   \n","9        5               Love this!!   \n","10     100                 Hate this   \n","11       3                OK product   \n","\n","                                               Review Customer Name  \n","0   \\nThe product works fine, it is maybe a little...       Phillip  \n","1                                           No review         elena  \n","2                                You should buy this.        Olivia  \n","3    Love everything about this product,it works g...          John  \n","4                    The product never worked for me.         Paula  \n","5                     This doesnt work as advertised.         Ellie  \n","6    This worked well for me,Not 5 stars because t...          DAVE  \n","7            Go and buy this right now, it's amazing.         Pablo  \n","8    Love everything about this product,it works g...          John  \n","9              Iwould 100% recommend this to everone.          CARA  \n","10                    This doesnt do anylhing for me.         Helen  \n","11  It does what it has to do, but the user experi...          emma  "],"text/html":["\n","  <div id=\"df-4cd88df2-6d4e-4063-af69-c4ed8c0b2e4c\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>Works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>No review</td>\n","      <td>elena</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>Everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>Doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>DAVE</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>BEST THING EVER</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>Love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>CARA</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>100</td>\n","      <td>Hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>OK product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>emma</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4cd88df2-6d4e-4063-af69-c4ed8c0b2e4c')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-4cd88df2-6d4e-4063-af69-c4ed8c0b2e4c button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-4cd88df2-6d4e-4063-af69-c4ed8c0b2e4c');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":77}]},{"cell_type":"markdown","source":["**Check for Duplicates**\n","\n","Duplicates, like missing data, cause problems and clog up analytics software. Let’s locate and eliminate them.\n","\n","To locate duplicates we start out with:"],"metadata":{"id":"B4whVF8NKZ-t"}},{"cell_type":"code","source":["data.duplicated()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"o6h3FQqeB9q8","executionInfo":{"status":"ok","timestamp":1673851585848,"user_tz":-330,"elapsed":738,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"02410e58-9d73-495a-9662-d97c97a0cf08"},"execution_count":78,"outputs":[{"output_type":"execute_result","data":{"text/plain":["0     False\n","1     False\n","2     False\n","3     False\n","4     False\n","5     False\n","6     False\n","7     False\n","8      True\n","9     False\n","10    False\n","11    False\n","dtype: bool"]},"metadata":{},"execution_count":78}]},{"cell_type":"markdown","source":["list of boolean values where a ‘True’ reading indicated duplicate values.\n","\n","Let’s go and get ahead and get rid of that duplicate (datapoint 8)."],"metadata":{"id":"s4fA6YlxKifM"}},{"cell_type":"code","source":["data.drop_duplicates()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":394},"id":"vbVPaadOB_nY","executionInfo":{"status":"ok","timestamp":1673851635492,"user_tz":-330,"elapsed":569,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"d1808d64-48d3-4ec4-cf79-a76b48ba372c"},"execution_count":79,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                Works well   \n","1        3               good enough   \n","2        5  Everyone should buy this   \n","3        5           Amazing product   \n","4        1          this is terrible   \n","5        2               Doesnt work   \n","6        4              cool product   \n","7        5           BEST THING EVER   \n","9        5               Love this!!   \n","10     100                 Hate this   \n","11       3                OK product   \n","\n","                                               Review Customer Name  \n","0   \\nThe product works fine, it is maybe a little...       Phillip  \n","1                                           No review         elena  \n","2                                You should buy this.        Olivia  \n","3    Love everything about this product,it works g...          John  \n","4                    The product never worked for me.         Paula  \n","5                     This doesnt work as advertised.         Ellie  \n","6    This worked well for me,Not 5 stars because t...          DAVE  \n","7            Go and buy this right now, it's amazing.         Pablo  \n","9              Iwould 100% recommend this to everone.          CARA  \n","10                    This doesnt do anylhing for me.         Helen  \n","11  It does what it has to do, but the user experi...          emma  "],"text/html":["\n","  <div id=\"df-fb269554-75e3-40b6-b8fe-82afc88993a8\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>Works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>No review</td>\n","      <td>elena</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>Everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>Doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>DAVE</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>BEST THING EVER</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>Love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>CARA</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>100</td>\n","      <td>Hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>OK product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>emma</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fb269554-75e3-40b6-b8fe-82afc88993a8')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-fb269554-75e3-40b6-b8fe-82afc88993a8 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-fb269554-75e3-40b6-b8fe-82afc88993a8');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":79}]},{"cell_type":"markdown","source":["**Detect Outliers**\n","\n","Outliers are numerical values that lie significantly outside of the statistical norm. Cutting that down from unnecessary science garble – they are data points that are so out of range they are likely misreads. \n","\n","They, like duplicates, need to be removed. Let’s sniff out an outlier by first, pulling up our dataset."],"metadata":{"id":"eR06ZkgOPkPQ"}},{"cell_type":"code","source":["data['Rating'].describe()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"agkdTe2sCBwT","executionInfo":{"status":"ok","timestamp":1673851686788,"user_tz":-330,"elapsed":524,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"4a00cc8d-7680-4460-d830-5a3ba97bd374"},"execution_count":80,"outputs":[{"output_type":"execute_result","data":{"text/plain":["count     12.000000\n","mean      11.833333\n","std       27.797427\n","min        1.000000\n","25%        3.000000\n","50%        4.500000\n","75%        5.000000\n","max      100.000000\n","Name: Rating, dtype: float64"]},"metadata":{},"execution_count":80}]},{"cell_type":"markdown","source":["Take a look at that ‘max’ value - none of the other values are even close to 100, with the mean (the average) being 11. Now, your solution to outliers will depend on your knowledge of your dataset. In this case, the data scientists who input the knowledge know that they meant to put a value of 1 not 100. So, we can safely remove the outlier to fix our data."],"metadata":{"id":"RnS2gk_BPtxD"}},{"cell_type":"code","source":["data.loc[10,'Rating'] = 1\n","data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"U4EaMq_ACDxv","executionInfo":{"status":"ok","timestamp":1673851778043,"user_tz":-330,"elapsed":631,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"61f6ff97-dbe4-4508-9bba-a5f6d7d619e3"},"execution_count":82,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                Works well   \n","1        3               good enough   \n","2        5  Everyone should buy this   \n","3        5           Amazing product   \n","4        1          this is terrible   \n","5        2               Doesnt work   \n","6        4              cool product   \n","7        5           BEST THING EVER   \n","8        5           Amazing product   \n","9        5               Love this!!   \n","10       1                 Hate this   \n","11       3                OK product   \n","\n","                                               Review Customer Name  \n","0   \\nThe product works fine, it is maybe a little...       Phillip  \n","1                                           No review         elena  \n","2                                You should buy this.        Olivia  \n","3    Love everything about this product,it works g...          John  \n","4                    The product never worked for me.         Paula  \n","5                     This doesnt work as advertised.         Ellie  \n","6    This worked well for me,Not 5 stars because t...          DAVE  \n","7            Go and buy this right now, it's amazing.         Pablo  \n","8    Love everything about this product,it works g...          John  \n","9              Iwould 100% recommend this to everone.          CARA  \n","10                    This doesnt do anylhing for me.         Helen  \n","11  It does what it has to do, but the user experi...          emma  "],"text/html":["\n","  <div id=\"df-93113399-3f95-4e43-95bb-0b3ed26a58d7\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>Works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>No review</td>\n","      <td>elena</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>Everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>Doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>DAVE</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>BEST THING EVER</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>5</td>\n","      <td>Amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>Love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>CARA</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>1</td>\n","      <td>Hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>OK product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>emma</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-93113399-3f95-4e43-95bb-0b3ed26a58d7')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-93113399-3f95-4e43-95bb-0b3ed26a58d7 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-93113399-3f95-4e43-95bb-0b3ed26a58d7');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":82}]},{"cell_type":"markdown","source":["Now our dataset has ratings ranging from 1 to 5, which will save major skew from if there was a rogue 100 in there.\n","\n"],"metadata":{"id":"adIoVwimP0vx"}},{"cell_type":"markdown","source":["**Normalize Casing**\n","\n","Last but not least we are going to dot our i’s and cross our t’s. Meaning we are going to standardize (lowercase) all review titles so as not to confuse our algorithms, and we are going to capitalize Customer Names, so that our algorithms know they are variables (you’ll see this in action below).\n","\n","Here’s how to make every review title lowercase:\n","\n"],"metadata":{"id":"c9TNyacDP4wo"}},{"cell_type":"code","source":["data['Review Title'] = data['Review Title'].str.lower()\n","data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"wnpK3W1ACF42","executionInfo":{"status":"ok","timestamp":1673851833260,"user_tz":-330,"elapsed":752,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"ba288fcf-570b-449b-a6c2-fe54c9e19f1c"},"execution_count":83,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                works well   \n","1        3               good enough   \n","2        5  everyone should buy this   \n","3        5           amazing product   \n","4        1          this is terrible   \n","5        2               doesnt work   \n","6        4              cool product   \n","7        5           best thing ever   \n","8        5           amazing product   \n","9        5               love this!!   \n","10       1                 hate this   \n","11       3                ok product   \n","\n","                                               Review Customer Name  \n","0   \\nThe product works fine, it is maybe a little...       Phillip  \n","1                                           No review         elena  \n","2                                You should buy this.        Olivia  \n","3    Love everything about this product,it works g...          John  \n","4                    The product never worked for me.         Paula  \n","5                     This doesnt work as advertised.         Ellie  \n","6    This worked well for me,Not 5 stars because t...          DAVE  \n","7            Go and buy this right now, it's amazing.         Pablo  \n","8    Love everything about this product,it works g...          John  \n","9              Iwould 100% recommend this to everone.          CARA  \n","10                    This doesnt do anylhing for me.         Helen  \n","11  It does what it has to do, but the user experi...          emma  "],"text/html":["\n","  <div id=\"df-fc9e73c6-f235-45a2-8404-8ebd2129810f\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>No review</td>\n","      <td>elena</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>DAVE</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>best thing ever</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>5</td>\n","      <td>amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>CARA</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>1</td>\n","      <td>hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>ok product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>emma</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fc9e73c6-f235-45a2-8404-8ebd2129810f')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-fc9e73c6-f235-45a2-8404-8ebd2129810f button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-fc9e73c6-f235-45a2-8404-8ebd2129810f');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":83}]},{"cell_type":"markdown","source":["Looks great! On to making sure our high-powered programs don’t get tripped up and miscategorize a customer name because it isn’t capitalized. Here’s how to ensure Customer Name capitalization:"],"metadata":{"id":"P2rbFZOcP-mj"}},{"cell_type":"code","source":["data['Customer Name'] = data['Customer Name'].str.title()\n","data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":425},"id":"TBQzxtSXCHwb","executionInfo":{"status":"ok","timestamp":1673851855564,"user_tz":-330,"elapsed":540,"user":{"displayName":"Sunil Kumar Gupta","userId":"00545443935213036838"}},"outputId":"1ec045da-a6d6-437d-b616-3d97aef11a14"},"execution_count":84,"outputs":[{"output_type":"execute_result","data":{"text/plain":["    Rating              Review Title  \\\n","0        4                works well   \n","1        3               good enough   \n","2        5  everyone should buy this   \n","3        5           amazing product   \n","4        1          this is terrible   \n","5        2               doesnt work   \n","6        4              cool product   \n","7        5           best thing ever   \n","8        5           amazing product   \n","9        5               love this!!   \n","10       1                 hate this   \n","11       3                ok product   \n","\n","                                               Review Customer Name  \n","0   \\nThe product works fine, it is maybe a little...       Phillip  \n","1                                           No review         Elena  \n","2                                You should buy this.        Olivia  \n","3    Love everything about this product,it works g...          John  \n","4                    The product never worked for me.         Paula  \n","5                     This doesnt work as advertised.         Ellie  \n","6    This worked well for me,Not 5 stars because t...          Dave  \n","7            Go and buy this right now, it's amazing.         Pablo  \n","8    Love everything about this product,it works g...          John  \n","9              Iwould 100% recommend this to everone.          Cara  \n","10                    This doesnt do anylhing for me.         Helen  \n","11  It does what it has to do, but the user experi...          Emma  "],"text/html":["\n","  <div id=\"df-50f1207d-16bb-4da2-8142-3ba2afc49bf4\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Rating</th>\n","      <th>Review Title</th>\n","      <th>Review</th>\n","      <th>Customer Name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>4</td>\n","      <td>works well</td>\n","      <td>\\nThe product works fine, it is maybe a little...</td>\n","      <td>Phillip</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3</td>\n","      <td>good enough</td>\n","      <td>No review</td>\n","      <td>Elena</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5</td>\n","      <td>everyone should buy this</td>\n","      <td>You should buy this.</td>\n","      <td>Olivia</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5</td>\n","      <td>amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>this is terrible</td>\n","      <td>The product never worked for me.</td>\n","      <td>Paula</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>2</td>\n","      <td>doesnt work</td>\n","      <td>This doesnt work as advertised.</td>\n","      <td>Ellie</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>cool product</td>\n","      <td>This worked well for me,Not 5 stars because t...</td>\n","      <td>Dave</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>best thing ever</td>\n","      <td>Go and buy this right now, it's amazing.</td>\n","      <td>Pablo</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>5</td>\n","      <td>amazing product</td>\n","      <td>Love everything about this product,it works g...</td>\n","      <td>John</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>5</td>\n","      <td>love this!!</td>\n","      <td>Iwould 100% recommend this to everone.</td>\n","      <td>Cara</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>1</td>\n","      <td>hate this</td>\n","      <td>This doesnt do anylhing for me.</td>\n","      <td>Helen</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>3</td>\n","      <td>ok product</td>\n","      <td>It does what it has to do, but the user experi...</td>\n","      <td>Emma</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-50f1207d-16bb-4da2-8142-3ba2afc49bf4')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-50f1207d-16bb-4da2-8142-3ba2afc49bf4 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-50f1207d-16bb-4da2-8142-3ba2afc49bf4');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":84}]},{"cell_type":"markdown","source":["And there you have it – our data set with all the fixins’. Or, rather, with all the fix-outs: We’ve made good use of intuitive Python libraries to locate and eliminate bad data, and standardize the rest. We are now ready to make the most of the Data in our AI and ML models."],"metadata":{"id":"jrEEhDelQGq8"}}]}