CSIT5210 Assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CSIT5210 Assignment\n",
    "\n",
    "## Fall 2019\n",
    "\n",
    "### Instructor: Dr. Kenneth Leung (kwtleung@cse.ust.hk)\n",
    "\n",
    "### TA: Dr. Kevin Wang (kevinw@ust.hk)\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "## [Group Signup link](https://docs.google.com/spreadsheets/d/1pD-LTpKU51jMQ9DymGbQsSfMNJ8JcjGO02sQaXBJzdU/edit#gid=0)\n",
    "\n",
    "## Description\n",
    "\n",
    "In this assignment, you will have an opportunity to apply some data mining techniques that you learned in the class to a problem.\n",
    "\n",
    "To get started on this assignment, you need to download the given dataset and read the description carefully written on this page. Please note that all implementation of your program should be done with Python.\n",
    "\n",
    "You are required to form a team with at most 4 students. You and your groupmates should evenly divide the project tasks yourself. All team members are going to graded with the same score.\n",
    "\n",
    "There are two parts in this assignment. Part 1 is a programming exercise that your groups are required to complete certain tasks using Python. You should complete your Part 1 in this jupyter notebook (.ipynb file) directly. Part 2 is a group presentation that you need to present your findings on the Task 5 of Part 1.\n",
    "\n",
    "## Submission\n",
    "\n",
    "| Part | Due Date | Submission by | Files to Submit |\n",
    "|---|-----|------|----|\n",
    "| Part 1 | 15/11/2019 (Fri) 23:59 | [email to TA](mailto:kevinw@ust.hk) | this `ipynb` file. |\n",
    "| Part 2 | 21/11/2019 (Thu) during lecture | Printed hardcopy | Presentation slides (4 slides per page) |\n",
    "\n",
    "## Prerequesite\n",
    "\n",
    "You are recommended to install the following packages\n",
    "\n",
    "* pandas\n",
    "* geohash\n",
    "* matplotlib\n",
    "* sklearn\n",
    "\n",
    "To install these packages, you shall type in your terminal\n",
    "\n",
    "```\n",
    "> pip install pandas\n",
    "> pip install geohash\n",
    "> pip install matplotlib\n",
    "> pip install sklearn\n",
    "```\n",
    "\n",
    "The package geohash is a little tricky to install.\n",
    "\n",
    "## About the data and the context\n",
    "\n",
    "We are working on some data related to Taxi. In Task 1 to 4 we use the data set `taxi_train.csv`. In Task 5 we use another set of data. \n",
    "You can [download the data here](./tostudent.zip)\n",
    "\n",
    "\n",
    "### Taxi Data\n",
    "\n",
    "The data contains the following: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pickup_datetime</th>\n",
       "      <th>passenger</th>\n",
       "      <th>pickup_geohash</th>\n",
       "      <th>dropoff_geohash</th>\n",
       "      <th>fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2009-06-15 17:26:21</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5rxth4yu4c</td>\n",
       "      <td>dr5rxeqnjy8v</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2010-01-05 16:52:16</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5reer0t0fm</td>\n",
       "      <td>dr72h81uqhe9</td>\n",
       "      <td>16.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2011-08-18 00:35:00</td>\n",
       "      <td>2</td>\n",
       "      <td>dr5rukr7t72n</td>\n",
       "      <td>dr5ru63jpp74</td>\n",
       "      <td>5.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2012-04-21 04:30:42</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5rsrhpybjq</td>\n",
       "      <td>dr5ru7bcpe0c</td>\n",
       "      <td>7.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2010-03-09 07:51:00</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5ruvd4f7wb</td>\n",
       "      <td>dr72j06qth30</td>\n",
       "      <td>5.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2011-01-06 09:50:45</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5rsnfnuw1v</td>\n",
       "      <td>dr5ruey9ftmf</td>\n",
       "      <td>12.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2012-11-20 20:35:00</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5rud97th2k</td>\n",
       "      <td>dr5rutjbmmr6</td>\n",
       "      <td>7.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2012-01-04 17:22:00</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5rvnwq9f4p</td>\n",
       "      <td>dr5ru6d01cfq</td>\n",
       "      <td>16.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2012-12-03 13:10:00</td>\n",
       "      <td>1</td>\n",
       "      <td>dr5reyn4k3w6</td>\n",
       "      <td>dr5rsnzyumf1</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2009-09-02 01:11:00</td>\n",
       "      <td>2</td>\n",
       "      <td>dr5rsx2uj8r1</td>\n",
       "      <td>dr5ru7c125sg</td>\n",
       "      <td>8.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       pickup_datetime  passenger pickup_geohash dropoff_geohash  fare\n",
       "0  2009-06-15 17:26:21          1   dr5rxth4yu4c    dr5rxeqnjy8v   4.5\n",
       "1  2010-01-05 16:52:16          1   dr5reer0t0fm    dr72h81uqhe9  16.9\n",
       "2  2011-08-18 00:35:00          2   dr5rukr7t72n    dr5ru63jpp74   5.7\n",
       "3  2012-04-21 04:30:42          1   dr5rsrhpybjq    dr5ru7bcpe0c   7.7\n",
       "4  2010-03-09 07:51:00          1   dr5ruvd4f7wb    dr72j06qth30   5.3\n",
       "5  2011-01-06 09:50:45          1   dr5rsnfnuw1v    dr5ruey9ftmf  12.1\n",
       "6  2012-11-20 20:35:00          1   dr5rud97th2k    dr5rutjbmmr6   7.5\n",
       "7  2012-01-04 17:22:00          1   dr5rvnwq9f4p    dr5ru6d01cfq  16.5\n",
       "8  2012-12-03 13:10:00          1   dr5reyn4k3w6    dr5rsnzyumf1   9.0\n",
       "9  2009-09-02 01:11:00          2   dr5rsx2uj8r1    dr5ru7c125sg   8.9"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "taxi = pd.read_csv('taxi_train.csv', sep=',', nrows= 10)\n",
    "taxi.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* `pickup_datetime` is the starting time for the ride.\n",
    "* `passenger` means the number of passenger.\n",
    "* `pickup_geohash` and `dropoff_geohash` are the position where the passenger is picked up and dropped off respectively. They are encoded using geohash.\n",
    "* `fare` is the total amount of paid in the trip.\n",
    "\n",
    "To decode a geoash you might want to use a geohash package (follow hint on Task 1.2 to install it) and it gives you the longitude and latitude of the position. We all know that earth is a sphere (oh please), but for simplicity, when we calculate the distance between two points, we would use Euclidean distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('40.721319', '-73.844311')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import Geohash as gh\n",
    "gh.decode('dr5rxth4yu4c')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1 - Programming Task\n",
    "\n",
    "Your team are required to complete the following tasks on this Jupyter notebook. \n",
    "\n",
    "1. Data Preprocessing & Statistics \n",
    "2. Data Clustering \n",
    "3. Simple Data Visualization \n",
    "4. Frequent Pattern Mining \n",
    "5. Prediction \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Task 1 - Data Preprocessing and Statistics\n",
    "#\n",
    "#\n",
    "# Task 1.1 Read Taxi Data in using the API Pandas.read_csv so that the column 'pickup_datetime' is read as datetime64.\n",
    "#                 Hint: use the parameter parse_dates\n",
    "#                  ref: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html\n",
    "#                 To make sure your code works you might want to read only the first 1000 rows and expand it later\n",
    "\n",
    "\n",
    "\n",
    "# Task 1.2 Convert the field pickup_geohashed and dropoff_geohashed into x-y coordinate using the API Geohash.decode\n",
    "#                  The package Geohash can be found from pip. You might encounter the problem \n",
    "#                   'python3.5.2 can't find the module '  ref: https://github.com/vinsci/geohash/issues/4\n",
    "#                    This can be fixed very easily. Or\n",
    "#                   You might directly use the fixed version of Geohash in our project package.\n",
    "#                   The precision of each coordinate is with 6 decimal places\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Task 1.2.1 Create unpack the decoded pickup_geohashed and dropoff_geohashed into the pair of columns \n",
    "#                  pickup_x pickup_y  and dropoff_x, dropoff_y respectively. \n",
    "#                  Namely: if pickup is [40.712278, -73.84161]; pickup_x should contain 40.712278 and pickup_y should contain -73.84161\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Task 1.3 Create the column 'distance' based on the Euclidean distance that the ride has traveled.\n",
    "\n",
    "\n",
    "# Task 1.4.1  Check the memory you have spent by the API .info()\n",
    "\n",
    "# Task 1.4.2  Fetch the first 10 lines of your data to preview it.\n",
    "\n",
    "# Task 1.5 Remove rows with invalid geohashed. Count the number of rows removed.\n",
    "\n",
    "# Task 1.6  Display the count, mean, standard derviation of the int type variable and \n",
    "#                  display the earliest and latest pickup_time.\n",
    "\n",
    "# Task 1.7 Find the number of order between (8am to 9am)   and the order between (1am to 2am)\n",
    "#                 Note: Instead of using only the first 1000 rows, expand your selection of rows to collect enough data.\n",
    "#                Hint: try the API between_time of DataFrame. \n",
    "#           ref:  https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.between_time.html#pandas.DataFrame.between_time\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Task 2 - Data Clustering \n",
    "#\n",
    "#\n",
    "# Task 2.1 Create a DataFrame that contains two columns. The first column (the index) is a time series 0:00, 0:15, 0:30, 0:45, \n",
    "#                  1:00,... 23:00, 23:15, 23:30, 23:45\n",
    "#                  The second column is an integer that counts the number of ride between in the interval. For example, 0:00 should contains all order happens on or after 0:00 to 0:15.\n",
    "#  This task is less straight forward, at least in our solution. So let's break down a little bit.\n",
    "#  Task 2.1.1 Create a list of string containing the series '0:00', '0:15', '0:30', ... '23:45' \n",
    "#                     Hint: A double loop with if-else can do the job.\n",
    "\n",
    "\n",
    "# Task 2.1.2 Count the number of orders. You might use between_time again.\n",
    "\n",
    "\n",
    "# Task 2.2.1  Use K-mean algorithm to find 30 cluster centers of the coordinates obtained from  Task 1.2.1\n",
    "#                  You may implement your own K-mean algorithm or simply adopt the API sklearn.cluster.KMeans\n",
    "#                   ref: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html\n",
    "                   \n",
    "\n",
    "    \n",
    "# Task 2.2.2 Describe how many % of order has started from a cluster centers and ends at the same cluster centers.  \n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Task 3 - Simple Data Visualization \n",
    "#\n",
    "#\n",
    "# Task 3.1 Using data obtained from Task 1.6. Plot a curveof the volume of order in different times of a day.\n",
    "#                    Hint: try the API DataFrame.plot\n",
    "\n",
    "\n",
    "# Task 3.2 Using data obtained from Task 2.2.1. Plot a histogram of the volume of order in different cluster centers\n",
    "\n",
    "\n",
    "# Task 3.3 Scatter plot the 100 random location of the pickups in blue, 100 random location of dropoffs in red, plot also the cluster centers in black"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Task 4 - Frequent Pattern Mining \n",
    "#\n",
    "#\n",
    "# Task 4.1 Apply FPGrowth algorithm, either using existing API or write your own, to identify which set of users are likely to go-together. \n",
    "#                 Definition of go-together: they starts at the same cluster centers and their start time is in the same 15-minutes timeslot.\n",
    " #                This task may not be straight forward as you may need to build the list of transaction first."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 5 - Prediction\n",
    "\n",
    "You are given another set of Taxi data while this time there are more columns to play with. With these data you are going to help a Taxi driver, Daniel, to optimize his profit. \n",
    "\n",
    "![](http://file1.telestar.fr/var/telestar/storage/images/3/0/9/9/3099879/samy-naceri-dans-taxi-2_width1024.jpg)  - From the movie *Taxi* - src: telestar.fr\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>taxi_id</th>\n",
       "      <th>trip_start_timestamp</th>\n",
       "      <th>trip_end_timestamp</th>\n",
       "      <th>trip_seconds</th>\n",
       "      <th>trip_miles</th>\n",
       "      <th>fare</th>\n",
       "      <th>tips</th>\n",
       "      <th>tolls</th>\n",
       "      <th>extras</th>\n",
       "      <th>trip_total</th>\n",
       "      <th>payment_type</th>\n",
       "      <th>pickup_latitude</th>\n",
       "      <th>pickup_longitude</th>\n",
       "      <th>dropoff_latitude</th>\n",
       "      <th>dropoff_longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2302</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:15:00</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.25</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.25</td>\n",
       "      <td>Cash</td>\n",
       "      <td>173.0</td>\n",
       "      <td>203.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>708.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4196</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.85</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.85</td>\n",
       "      <td>Cash</td>\n",
       "      <td>170.0</td>\n",
       "      <td>351.0</td>\n",
       "      <td>170.0</td>\n",
       "      <td>351.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6847</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:15:00</td>\n",
       "      <td>780.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>13.45</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14.45</td>\n",
       "      <td>Cash</td>\n",
       "      <td>754.0</td>\n",
       "      <td>410.0</td>\n",
       "      <td>423.0</td>\n",
       "      <td>586.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8753</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>420.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>6.85</td>\n",
       "      <td>3.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>10.85</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>604.0</td>\n",
       "      <td>668.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>668.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1283</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>300.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>6.25</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.25</td>\n",
       "      <td>No Charge</td>\n",
       "      <td>744.0</td>\n",
       "      <td>605.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>405.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>4120</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:15:00</td>\n",
       "      <td>480.0</td>\n",
       "      <td>2.7</td>\n",
       "      <td>8.85</td>\n",
       "      <td>2.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.85</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>244.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7694</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.45</td>\n",
       "      <td>2.09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.54</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8096</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>300.0</td>\n",
       "      <td>2.1</td>\n",
       "      <td>6.85</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.85</td>\n",
       "      <td>Cash</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>4728</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:30:00</td>\n",
       "      <td>1260.0</td>\n",
       "      <td>7.2</td>\n",
       "      <td>21.75</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22.75</td>\n",
       "      <td>Cash</td>\n",
       "      <td>385.0</td>\n",
       "      <td>478.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>708.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>7042</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>720.0</td>\n",
       "      <td>1.6</td>\n",
       "      <td>8.05</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.05</td>\n",
       "      <td>Cash</td>\n",
       "      <td>294.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>395.0</td>\n",
       "      <td>408.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   taxi_id trip_start_timestamp   trip_end_timestamp  trip_seconds  \\\n",
       "0     2302  2016-01-01 00:00:00  2016-01-01 00:15:00        1080.0   \n",
       "1     4196  2016-01-01 00:00:00  2016-01-01 00:00:00          60.0   \n",
       "2     6847  2016-01-01 00:00:00  2016-01-01 00:15:00         780.0   \n",
       "3     8753  2016-01-01 00:00:00  2016-01-01 00:00:00         420.0   \n",
       "4     1283  2016-01-01 00:00:00  2016-01-01 00:00:00         300.0   \n",
       "5     4120  2016-01-01 00:00:00  2016-01-01 00:15:00         480.0   \n",
       "6     7694  2016-01-01 00:00:00  2016-01-01 00:00:00           0.0   \n",
       "7     8096  2016-01-01 00:00:00  2016-01-01 00:00:00         300.0   \n",
       "8     4728  2016-01-01 00:00:00  2016-01-01 00:30:00        1260.0   \n",
       "9     7042  2016-01-01 00:00:00  2016-01-01 00:00:00         720.0   \n",
       "\n",
       "   trip_miles   fare  tips  tolls  extras  trip_total payment_type  \\\n",
       "0         0.0  12.25  0.00    0.0     0.0       12.25         Cash   \n",
       "1         0.0   3.85  0.00    0.0     0.0        3.85         Cash   \n",
       "2         5.1  13.45  0.00    0.0     1.0       14.45         Cash   \n",
       "3         0.1   6.85  3.00    0.0     1.0       10.85  Credit Card   \n",
       "4         0.5   6.25  0.00    0.0     1.0        7.25    No Charge   \n",
       "5         2.7   8.85  2.00    0.0     0.0       10.85  Credit Card   \n",
       "6         0.0  10.45  2.09    0.0     0.0       12.54  Credit Card   \n",
       "7         2.1   6.85  0.00    0.0     0.0        6.85         Cash   \n",
       "8         7.2  21.75  0.00    0.0     1.0       22.75         Cash   \n",
       "9         1.6   8.05  0.00    0.0     0.0        8.05         Cash   \n",
       "\n",
       "   pickup_latitude  pickup_longitude  dropoff_latitude  dropoff_longitude  \n",
       "0            173.0             203.0             109.0              708.0  \n",
       "1            170.0             351.0             170.0              351.0  \n",
       "2            754.0             410.0             423.0              586.0  \n",
       "3            604.0             668.0             604.0              668.0  \n",
       "4            744.0             605.0              30.0              405.0  \n",
       "5             18.0             610.0              97.0              244.0  \n",
       "6              NaN               NaN               NaN                NaN  \n",
       "7              NaN               NaN               NaN                NaN  \n",
       "8            385.0             478.0             109.0              708.0  \n",
       "9            294.0             113.0             395.0              408.0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('training.csv', sep=',', nrows=10)\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 5.1: \n",
    "It is Monday 8am, the prime time for every Taxi Driver. Daniel have received three orders at the same time. Assume Daniel are equally far from these three orders. Please help Daniel to pick the one with largest income (fare + tips). The fields `fare`, `tips` and `best` are masked in testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>order1_pickup_longitude</th>\n",
       "      <th>order2_pickup_latitude</th>\n",
       "      <th>order2_pickup_longitude</th>\n",
       "      <th>order3_pickup_latitude</th>\n",
       "      <th>order3_pickup_longitude</th>\n",
       "      <th>trip_start_timestamp</th>\n",
       "      <th>order1_fare</th>\n",
       "      <th>order1_tips</th>\n",
       "      <th>order2_fare</th>\n",
       "      <th>order2_tips</th>\n",
       "      <th>order3_fare</th>\n",
       "      <th>order3_tips</th>\n",
       "      <th>best</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>order1_pickup_latitude</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>175.0</th>\n",
       "      <td>675.0</td>\n",
       "      <td>660.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>532.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>686.0</th>\n",
       "      <td>500.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>370.0</td>\n",
       "      <td>688.0</td>\n",
       "      <td>206.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173.0</th>\n",
       "      <td>203.0</td>\n",
       "      <td>779.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>744.0</td>\n",
       "      <td>605.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>660.0</th>\n",
       "      <td>120.0</td>\n",
       "      <td>419.0</td>\n",
       "      <td>615.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>675.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167.0</th>\n",
       "      <td>754.0</td>\n",
       "      <td>395.0</td>\n",
       "      <td>408.0</td>\n",
       "      <td>744.0</td>\n",
       "      <td>605.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>660.0</th>\n",
       "      <td>120.0</td>\n",
       "      <td>173.0</td>\n",
       "      <td>203.0</td>\n",
       "      <td>173.0</td>\n",
       "      <td>203.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>556.0</th>\n",
       "      <td>288.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>675.0</td>\n",
       "      <td>618.0</td>\n",
       "      <td>407.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64.0</th>\n",
       "      <td>231.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>708.0</td>\n",
       "      <td>754.0</td>\n",
       "      <td>410.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173.0</th>\n",
       "      <td>203.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767.0</th>\n",
       "      <td>733.0</td>\n",
       "      <td>166.0</td>\n",
       "      <td>725.0</td>\n",
       "      <td>225.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2016-02-01 08:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        order1_pickup_longitude  order2_pickup_latitude  \\\n",
       "order1_pickup_latitude                                                    \n",
       "175.0                                     675.0                   660.0   \n",
       "686.0                                     500.0                    37.0   \n",
       "173.0                                     203.0                   779.0   \n",
       "660.0                                     120.0                   419.0   \n",
       "167.0                                     754.0                   395.0   \n",
       "660.0                                     120.0                   173.0   \n",
       "556.0                                     288.0                   175.0   \n",
       "64.0                                      231.0                   109.0   \n",
       "173.0                                     203.0                    18.0   \n",
       "767.0                                     733.0                   166.0   \n",
       "\n",
       "                        order2_pickup_longitude  order3_pickup_latitude  \\\n",
       "order1_pickup_latitude                                                    \n",
       "175.0                                     120.0                   130.0   \n",
       "686.0                                     370.0                   688.0   \n",
       "173.0                                      81.0                   744.0   \n",
       "660.0                                     615.0                   175.0   \n",
       "167.0                                     408.0                   744.0   \n",
       "660.0                                     203.0                   173.0   \n",
       "556.0                                     675.0                   618.0   \n",
       "64.0                                      708.0                   754.0   \n",
       "173.0                                     610.0                    18.0   \n",
       "767.0                                     725.0                   225.0   \n",
       "\n",
       "                        order3_pickup_longitude trip_start_timestamp  \\\n",
       "order1_pickup_latitude                                                 \n",
       "175.0                                     532.0  2016-02-01 08:00:00   \n",
       "686.0                                     206.0  2016-02-01 08:00:00   \n",
       "173.0                                     605.0  2016-02-01 08:00:00   \n",
       "660.0                                     675.0  2016-02-01 08:00:00   \n",
       "167.0                                     605.0  2016-02-01 08:00:00   \n",
       "660.0                                     203.0  2016-02-01 08:00:00   \n",
       "556.0                                     407.0  2016-02-01 08:00:00   \n",
       "64.0                                      410.0  2016-02-01 08:00:00   \n",
       "173.0                                     610.0  2016-02-01 08:00:00   \n",
       "767.0                                       6.0  2016-02-01 08:00:00   \n",
       "\n",
       "                        order1_fare  order1_tips  order2_fare  order2_tips  \\\n",
       "order1_pickup_latitude                                                       \n",
       "175.0                             0            0            0            0   \n",
       "686.0                             0            0            0            0   \n",
       "173.0                             0            0            0            0   \n",
       "660.0                             0            0            0            0   \n",
       "167.0                             0            0            0            0   \n",
       "660.0                             0            0            0            0   \n",
       "556.0                             0            0            0            0   \n",
       "64.0                              0            0            0            0   \n",
       "173.0                             0            0            0            0   \n",
       "767.0                             0            0            0            0   \n",
       "\n",
       "                        order3_fare  order3_tips  best  \n",
       "order1_pickup_latitude                                  \n",
       "175.0                             0            0     0  \n",
       "686.0                             0            0     0  \n",
       "173.0                             0            0     0  \n",
       "660.0                             0            0     0  \n",
       "167.0                             0            0     0  \n",
       "660.0                             0            0     0  \n",
       "556.0                             0            0     0  \n",
       "64.0                              0            0     0  \n",
       "173.0                             0            0     0  \n",
       "767.0                             0            0     0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monday8am = pd.read_csv('5_1_testing.csv', sep=',', nrows=10, index_col=0)\n",
    "monday8am.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 5.2:\n",
    "It is Daniels holiday and he receives a call from his friend to ask him work as a substitute. His friend will pay him a few hundred to Daniel plus all the tips he earned during the period. Please help Daniel to optimize his profit by picking order. This time, Daniel are given three order at a time. These orders will have the location of the pick up and drop off, and the payment method of the order.  The fields `tips` and `best` are masked in testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trip_start_timestamp</th>\n",
       "      <th>order1_pickup_latitude</th>\n",
       "      <th>order1_pickup_longitude</th>\n",
       "      <th>order2_pickup_latitude</th>\n",
       "      <th>order2_pickup_longitude</th>\n",
       "      <th>order3_pickup_latitude</th>\n",
       "      <th>order3_pickup_longitude</th>\n",
       "      <th>order1_dropoff_latitude</th>\n",
       "      <th>order1_dropoff_longitude</th>\n",
       "      <th>order2_dropoff_latitude</th>\n",
       "      <th>...</th>\n",
       "      <th>order1_fare</th>\n",
       "      <th>order1_tips</th>\n",
       "      <th>order1_payment_type</th>\n",
       "      <th>order2_fare</th>\n",
       "      <th>order2_tips</th>\n",
       "      <th>order2_payment_type</th>\n",
       "      <th>order3_fare</th>\n",
       "      <th>order3_tips</th>\n",
       "      <th>order3_payment_type</th>\n",
       "      <th>best</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>686.0</td>\n",
       "      <td>500.0</td>\n",
       "      <td>395.0</td>\n",
       "      <td>408.0</td>\n",
       "      <td>225.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>173.0</td>\n",
       "      <td>203.0</td>\n",
       "      <td>564.0</td>\n",
       "      <td>...</td>\n",
       "      <td>10.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>15.00</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>45.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>728.0</td>\n",
       "      <td>277.0</td>\n",
       "      <td>210.0</td>\n",
       "      <td>470.0</td>\n",
       "      <td>294.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>728.0</td>\n",
       "      <td>...</td>\n",
       "      <td>10.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>4.50</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>11.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>527.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>744.0</td>\n",
       "      <td>605.0</td>\n",
       "      <td>527.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>411.0</td>\n",
       "      <td>...</td>\n",
       "      <td>3.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>4.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>9.50</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>419.0</td>\n",
       "      <td>615.0</td>\n",
       "      <td>411.0</td>\n",
       "      <td>545.0</td>\n",
       "      <td>411.0</td>\n",
       "      <td>545.0</td>\n",
       "      <td>225.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>754.0</td>\n",
       "      <td>...</td>\n",
       "      <td>40.50</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>9.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>688.0</td>\n",
       "      <td>206.0</td>\n",
       "      <td>660.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>355.0</td>\n",
       "      <td>333.0</td>\n",
       "      <td>454.0</td>\n",
       "      <td>...</td>\n",
       "      <td>19.50</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>7.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>6.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>173.0</td>\n",
       "      <td>203.0</td>\n",
       "      <td>754.0</td>\n",
       "      <td>410.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>708.0</td>\n",
       "      <td>695.0</td>\n",
       "      <td>...</td>\n",
       "      <td>11.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>6.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>15.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>167.0</td>\n",
       "      <td>754.0</td>\n",
       "      <td>294.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>161.0</td>\n",
       "      <td>649.0</td>\n",
       "      <td>744.0</td>\n",
       "      <td>605.0</td>\n",
       "      <td>210.0</td>\n",
       "      <td>...</td>\n",
       "      <td>6.00</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>4.50</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>7.00</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>754.0</td>\n",
       "      <td>410.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>744.0</td>\n",
       "      <td>...</td>\n",
       "      <td>6.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>7.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>6.00</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>18.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>225.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>532.0</td>\n",
       "      <td>294.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>6.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>42.00</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>4.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2016-02-04 19:00:00</td>\n",
       "      <td>158.0</td>\n",
       "      <td>270.0</td>\n",
       "      <td>419.0</td>\n",
       "      <td>615.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>163.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>668.0</td>\n",
       "      <td>754.0</td>\n",
       "      <td>...</td>\n",
       "      <td>19.25</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>5.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Credit Card</td>\n",
       "      <td>9.75</td>\n",
       "      <td>0</td>\n",
       "      <td>Cash</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  trip_start_timestamp  order1_pickup_latitude  order1_pickup_longitude  \\\n",
       "0  2016-02-04 19:00:00                   686.0                    500.0   \n",
       "1  2016-02-04 19:00:00                    18.0                    610.0   \n",
       "2  2016-02-04 19:00:00                   527.0                     24.0   \n",
       "3  2016-02-04 19:00:00                   419.0                    615.0   \n",
       "4  2016-02-04 19:00:00                    18.0                    610.0   \n",
       "5  2016-02-04 19:00:00                   173.0                    203.0   \n",
       "6  2016-02-04 19:00:00                   167.0                    754.0   \n",
       "7  2016-02-04 19:00:00                    18.0                    610.0   \n",
       "8  2016-02-04 19:00:00                    18.0                    610.0   \n",
       "9  2016-02-04 19:00:00                   158.0                    270.0   \n",
       "\n",
       "   order2_pickup_latitude  order2_pickup_longitude  order3_pickup_latitude  \\\n",
       "0                   395.0                    408.0                   225.0   \n",
       "1                   728.0                    277.0                   210.0   \n",
       "2                    18.0                    610.0                   744.0   \n",
       "3                   411.0                    545.0                   411.0   \n",
       "4                   688.0                    206.0                   660.0   \n",
       "5                   754.0                    410.0                    18.0   \n",
       "6                   294.0                    113.0                   161.0   \n",
       "7                   754.0                    410.0                    18.0   \n",
       "8                   225.0                      6.0                   130.0   \n",
       "9                   419.0                    615.0                    45.0   \n",
       "\n",
       "   order3_pickup_longitude  order1_dropoff_latitude  order1_dropoff_longitude  \\\n",
       "0                      6.0                    173.0                     203.0   \n",
       "1                    470.0                    294.0                     113.0   \n",
       "2                    605.0                    527.0                      24.0   \n",
       "3                    545.0                    225.0                       6.0   \n",
       "4                    120.0                    355.0                     333.0   \n",
       "5                    610.0                    109.0                     708.0   \n",
       "6                    649.0                    744.0                     605.0   \n",
       "7                    610.0                     18.0                     610.0   \n",
       "8                    532.0                    294.0                     113.0   \n",
       "9                    163.0                    604.0                     668.0   \n",
       "\n",
       "   order2_dropoff_latitude  ...  order1_fare  order1_tips  \\\n",
       "0                    564.0  ...        10.25            0   \n",
       "1                    728.0  ...        10.25            0   \n",
       "2                    411.0  ...         3.25            0   \n",
       "3                    754.0  ...        40.50            0   \n",
       "4                    454.0  ...        19.50            0   \n",
       "5                    695.0  ...        11.25            0   \n",
       "6                    210.0  ...         6.00            0   \n",
       "7                    744.0  ...         6.75            0   \n",
       "8                      NaN  ...         6.75            0   \n",
       "9                    754.0  ...        19.25            0   \n",
       "\n",
       "   order1_payment_type  order2_fare  order2_tips order2_payment_type  \\\n",
       "0          Credit Card        15.00            0                Cash   \n",
       "1          Credit Card         4.50            0         Credit Card   \n",
       "2                 Cash         4.25            0                Cash   \n",
       "3                 Cash         9.00            0                Cash   \n",
       "4                 Cash         7.75            0                Cash   \n",
       "5                 Cash         6.25            0         Credit Card   \n",
       "6                 Cash         4.50            0                Cash   \n",
       "7                 Cash         7.25            0         Credit Card   \n",
       "8                 Cash        42.00            0         Credit Card   \n",
       "9          Credit Card         5.75            0         Credit Card   \n",
       "\n",
       "   order3_fare  order3_tips order3_payment_type  best  \n",
       "0        45.75            0         Credit Card     0  \n",
       "1        11.25            0         Credit Card     0  \n",
       "2         9.50            0         Credit Card     0  \n",
       "3         9.75            0         Credit Card     0  \n",
       "4         6.25            0                Cash     0  \n",
       "5        15.75            0         Credit Card     0  \n",
       "6         7.00            0                Cash     0  \n",
       "7         6.00            0         Credit Card     0  \n",
       "8         4.75            0                Cash     0  \n",
       "9         9.75            0                Cash     0  \n",
       "\n",
       "[10 rows x 23 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_tips = pd.read_csv('5_2_testing.csv', sep=',', nrows=10)\n",
    "best_tips.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the data given above, predict which order will give you the best result. The accurarcy of your prediction for 5.1 and 5.2 are measured by:\n",
    " * `total_numbers_of_correct_prediction` / `number_of_rows_of_test` * 100%.  \n",
    "An empty prediction would be counted as one wrong prediction. \n",
    "\n",
    "\n",
    "* **A higher accuracy does not immediate imply a higher mark for the project. We value more on how you choose your algorithm , how you fine tune your parameters, and explain why your algorithm did/did not work.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2 - Group Presentation\n",
    "\n",
    "Prepare a 5 minutes presentation on your Task 5 prediction work. You will be given the ground truth after the due day. Compute the accuracy yourself and include it in your presentation. Your presentation should be focus on why would your choose that particular algorithm and what optimization/fine tuning you have done to improve the accuracy. You can also comment on your accuracy and suggest how could that be improved. \n",
    "\n",
    "You may use the following line to measure the accurarcy\n",
    "```\n",
    "groundtruth['predict'] = output['predict']\n",
    "groundtruth[ groundtruth['best'] == groundtruth['predict']].count()\n",
    "```\n",
    "\n",
    "Since this is a short presentation, you may assign any member to present the work.\n",
    "\n",
    "**Note: Again, a higher accuracy does not immediate imply a higher mark.**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}