MLT-4/MLT4_DecisionTrees.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b3788227-721c-42cf-b5f9-5536f3d35646",
   "metadata": {},
   "source": [
    "<img src=\"https://cellstrat2.s3.amazonaws.com/PlatformAssets/bluewhitelogo.svg\" alt=\"drawing\" width=\"200\"/>\n",
    "\n",
    "# ML Tuesdays - Session 4\n",
    "## Machine Learning Track\n",
    "### Decision Trees Exercise\n",
    "\n",
    "### Guidelines\n",
    "1. The notebook has been split into multiple steps with fine-grained instructions for each step. Use the instructions for each code cell to complete the code.\n",
    "2. You can refer the Logistic Regression Module in the Machine Learning Pack from CellStrat Hub.\n",
    "3. Make use of the docstrings of the functions and classes using the `shift+tab` shortcut key.\n",
    "4. Refer the internet for the explanation of any algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "335a44b5-48a3-488e-bfb6-66259383cb88",
   "metadata": {
    "tags": []
   },
   "source": [
    "## About the Dataset\n",
    "The data set consists following Input variables :\n",
    "1 - fixed acidity  \n",
    "2 - volatile acidity  \n",
    "3 - citric acid  \n",
    "4 - residual sugar  \n",
    "5 - chlorides  \n",
    "6 - free sulfur dioxide\n",
    "7 - total sulfur dioxide  \n",
    "8 - density  \n",
    "9 - pH   \n",
    "10 - sulphates   \n",
    "11 - alcohol\n",
    "\n",
    "Output variable gives the quality of th wine based on the input variables: \n",
    "\n",
    "12 - quality (score between 0 and 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f9b8fa9",
   "metadata": {},
   "source": [
    "Decision tree algorithm is one of the most versatile algorithms in machine learning which can perform both classification and regression analysis. It is very powerful and works great with complex datasets. Apart from that, it is very easy to understand and read. That makes it more popular to use. When coupled with ensemble techniques – which we will learn very soon- it performs even better.\n",
    "As the name suggests, this algorithm works by dividing the whole dataset into a tree-like structure based on some rules and conditions and then gives prediction based on those conditions.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7350f57",
   "metadata": {},
   "source": [
    "### Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7d0806f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn import tree\n",
    "from sklearn.model_selection import train_test_split,GridSearchCV\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix, roc_curve, roc_auc_score\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7438cb5d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.700</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>0.99780</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.880</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.6</td>\n",
       "      <td>0.098</td>\n",
       "      <td>25.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>0.99680</td>\n",
       "      <td>3.20</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.760</td>\n",
       "      <td>0.04</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0.092</td>\n",
       "      <td>15.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.99700</td>\n",
       "      <td>3.26</td>\n",
       "      <td>0.65</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.2</td>\n",
       "      <td>0.280</td>\n",
       "      <td>0.56</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.075</td>\n",
       "      <td>17.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.99800</td>\n",
       "      <td>3.16</td>\n",
       "      <td>0.58</td>\n",
       "      <td>9.8</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.700</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>0.99780</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1594</th>\n",
       "      <td>6.2</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.08</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.090</td>\n",
       "      <td>32.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>0.99490</td>\n",
       "      <td>3.45</td>\n",
       "      <td>0.58</td>\n",
       "      <td>10.5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1595</th>\n",
       "      <td>5.9</td>\n",
       "      <td>0.550</td>\n",
       "      <td>0.10</td>\n",
       "      <td>2.2</td>\n",
       "      <td>0.062</td>\n",
       "      <td>39.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>0.99512</td>\n",
       "      <td>3.52</td>\n",
       "      <td>0.76</td>\n",
       "      <td>11.2</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1596</th>\n",
       "      <td>6.3</td>\n",
       "      <td>0.510</td>\n",
       "      <td>0.13</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0.076</td>\n",
       "      <td>29.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.99574</td>\n",
       "      <td>3.42</td>\n",
       "      <td>0.75</td>\n",
       "      <td>11.0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1597</th>\n",
       "      <td>5.9</td>\n",
       "      <td>0.645</td>\n",
       "      <td>0.12</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.075</td>\n",
       "      <td>32.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>0.99547</td>\n",
       "      <td>3.57</td>\n",
       "      <td>0.71</td>\n",
       "      <td>10.2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1598</th>\n",
       "      <td>6.0</td>\n",
       "      <td>0.310</td>\n",
       "      <td>0.47</td>\n",
       "      <td>3.6</td>\n",
       "      <td>0.067</td>\n",
       "      <td>18.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0.99549</td>\n",
       "      <td>3.39</td>\n",
       "      <td>0.66</td>\n",
       "      <td>11.0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1599 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0               7.4             0.700         0.00             1.9      0.076   \n",
       "1               7.8             0.880         0.00             2.6      0.098   \n",
       "2               7.8             0.760         0.04             2.3      0.092   \n",
       "3              11.2             0.280         0.56             1.9      0.075   \n",
       "4               7.4             0.700         0.00             1.9      0.076   \n",
       "...             ...               ...          ...             ...        ...   \n",
       "1594            6.2             0.600         0.08             2.0      0.090   \n",
       "1595            5.9             0.550         0.10             2.2      0.062   \n",
       "1596            6.3             0.510         0.13             2.3      0.076   \n",
       "1597            5.9             0.645         0.12             2.0      0.075   \n",
       "1598            6.0             0.310         0.47             3.6      0.067   \n",
       "\n",
       "      free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "0                    11.0                  34.0  0.99780  3.51       0.56   \n",
       "1                    25.0                  67.0  0.99680  3.20       0.68   \n",
       "2                    15.0                  54.0  0.99700  3.26       0.65   \n",
       "3                    17.0                  60.0  0.99800  3.16       0.58   \n",
       "4                    11.0                  34.0  0.99780  3.51       0.56   \n",
       "...                   ...                   ...      ...   ...        ...   \n",
       "1594                 32.0                  44.0  0.99490  3.45       0.58   \n",
       "1595                 39.0                  51.0  0.99512  3.52       0.76   \n",
       "1596                 29.0                  40.0  0.99574  3.42       0.75   \n",
       "1597                 32.0                  44.0  0.99547  3.57       0.71   \n",
       "1598                 18.0                  42.0  0.99549  3.39       0.66   \n",
       "\n",
       "      alcohol  quality  \n",
       "0         9.4        5  \n",
       "1         9.8        5  \n",
       "2         9.8        5  \n",
       "3         9.8        6  \n",
       "4         9.4        5  \n",
       "...       ...      ...  \n",
       "1594     10.5        5  \n",
       "1595     11.2        6  \n",
       "1596     11.0        6  \n",
       "1597     10.2        5  \n",
       "1598     11.0        6  \n",
       "\n",
       "[1599 rows x 12 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(\"winequality_red.csv\")\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76b4fad2-3019-4dcd-a43c-89e74d94f527",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "1. Split to X and y data\n",
    "2. Perform Train Test Split\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32900a9d-cf6e-4119-b7e8-90e4502f5cf9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ffd297f-a54e-4868-b00f-9b6387fb8f9b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "5f34e6e3",
   "metadata": {},
   "source": [
    "### Create a Decision Tree without Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3da4f407",
   "metadata": {},
   "source": [
    "Make optimal use of the scikit-learn documentation and google to understand Decision Tree algorithm and apply it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20c78e59-de66-4c52-a123-8436e294eebf",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a625cfc-786d-456e-bfcb-86bd9bffa6f5",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "cd497fb2-17e7-438c-a7c8-eaadb26b75d9",
   "metadata": {},
   "source": [
    "#### Visualise the tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b035b194-ac5c-48bc-be45-bdaa733ab053",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1ff83e2-5640-45f7-931e-421a08d40197",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "f87ebba5",
   "metadata": {},
   "source": [
    "### Make predictions on test data and check accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d780bb0f-4d09-41dd-b926-2512e903cebe",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45bd22db-11ac-48be-ae81-5950c97ad33f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20f08b45-f4da-4223-9922-c658888db4c4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6898a390-f174-4804-b7a0-43eb2975c0e9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "5346c21f",
   "metadata": {},
   "source": [
    "### Try to tune some hyperparameters using the GridSearchCV algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10069900",
   "metadata": {},
   "source": [
    "When we do hyperparameter tuning, we basically try to find those sets and values of hyperparameters which will give us a model with maximum accuracy. Let's go ahead and try to improve our model.\n",
    "\n",
    "\n",
    "GridSearchCV is a method used to tune our hyperparameters. We can pass different values of hyperparameters as\n",
    "parameters for grid search.\n",
    "It does a exhaustive generation of combination of different parameters passed.\n",
    "Using cross validation score, Grid Search returns the combination of hyperparameters for which the model is performing the best. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c997d711-a702-4b93-8c25-1dac1d8328bf",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3addec4-d34e-4eab-8e04-55bf633a4fc7",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "486dd252-6b2d-45b6-859e-8c598757c69f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a3e846d-35d6-4db0-8d5f-11d693abcc0f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2604e458-817e-4d9f-891b-37b019602c84",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c3167ba-008f-4d1f-8167-6e5b36b140c6",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb1d0062-c154-44f3-9813-3b07d7c6ee58",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "4a9f42b9-0e8a-4d73-8eab-b21cb6a70b82",
   "metadata": {},
   "source": [
    "### Compare the Score has improved after using Gridsearch or not.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37942297-92a9-4059-892a-4d2b5f105a6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualizing the tree\n",
    "from sklearn import tree\n",
    "plt.figure(figsize=(25,10))\n",
    "tree.plot_tree(clf,filled=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9de9d242",
   "metadata": {},
   "source": [
    "#### Note : we must understand that giving all the hyperparameters in the gridSearch doesn't gurantee the best result. \n",
    "#### We have to do hit and trial with parameters to get the perfect score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "780ebde9",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}