04 Filtering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "naked-burke",
   "metadata": {},
   "source": [
    "# Filtering data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ongoing-definition",
   "metadata": {},
   "source": [
    "In the last section we looked at how to act on entire columns at once. For example when we did:\n",
    "\n",
    "```python\n",
    "tips[\"total_bill\"] * 100\n",
    "```\n",
    "\n",
    "it applied the multiplication to every row, multiplying each number by 100.\n",
    "\n",
    "Sometimes we don't want to have to deal with entire columns at once, we might only want to grab a subset of the data and look in just that part. For example, with the tips data, we might think that the day of the week will affect the data so we just want to grab the data for Saturdays.\n",
    "\n",
    "In Pandas there are two steps to asking a question like this.\n",
    "\n",
    "1. create a *filter* which describes the question you want to ask\n",
    "2. *apply* that filter to the data to get just the bits you are interested in\n",
    "\n",
    "You create a filter by performing some operation on your `DataFrame` or a column within it. To ask about only those rows which refer to Saturday, you grab the `day` column and compare it to `\"Sat\"`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "vietnamese-entity",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "tips = pd.read_csv(\"https://milliams.com/courses/data_analysis_python/tips.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "regulated-poker",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      False\n",
       "1      False\n",
       "2      False\n",
       "3      False\n",
       "4      False\n",
       "       ...  \n",
       "239     True\n",
       "240     True\n",
       "241     True\n",
       "242     True\n",
       "243    False\n",
       "Name: day, Length: 244, dtype: bool"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tips[\"day\"] == \"Sat\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unavailable-consciousness",
   "metadata": {},
   "source": [
    "This has created a filter object (sometimes called a *mask* or a *boolean array*) which has `True` set for the rows where the day is Saturday and `False` elsewhere.\n",
    "\n",
    "We could save this filter as a variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "welcome-intention",
   "metadata": {},
   "outputs": [],
   "source": [
    "sat_filter = tips[\"day\"] == \"Sat\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "widespread-access",
   "metadata": {},
   "source": [
    "We can use this to filter the `DataFrame` as a whole. `tips[\"day\"] == \"Sat\"` has returned a `Series` containing booleans. Passing it back into `tips` as an indexing operation will use it to filter based on the `day` column, only keeping those rows which contained `True` in the filter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "vertical-commitment",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20.65</td>\n",
       "      <td>2.34</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>17.92</td>\n",
       "      <td>2.86</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>20.29</td>\n",
       "      <td>1.92</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>15.77</td>\n",
       "      <td>1.56</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>39.42</td>\n",
       "      <td>5.31</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>35.83</td>\n",
       "      <td>3.27</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239</th>\n",
       "      <td>29.03</td>\n",
       "      <td>4.14</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>27.18</td>\n",
       "      <td>1.40</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>241</th>\n",
       "      <td>22.67</td>\n",
       "      <td>1.40</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>17.82</td>\n",
       "      <td>1.22</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>87 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     total_bill   tip  day    time  size\n",
       "19        20.65  2.34  Sat  Dinner     3\n",
       "20        17.92  2.86  Sat  Dinner     2\n",
       "21        20.29  1.92  Sat  Dinner     2\n",
       "22        15.77  1.56  Sat  Dinner     2\n",
       "23        39.42  5.31  Sat  Dinner     4\n",
       "..          ...   ...  ...     ...   ...\n",
       "238       35.83  3.27  Sat  Dinner     3\n",
       "239       29.03  4.14  Sat  Dinner     3\n",
       "240       27.18  1.40  Sat  Dinner     2\n",
       "241       22.67  1.40  Sat  Dinner     2\n",
       "242       17.82  1.22  Sat  Dinner     2\n",
       "\n",
       "[87 rows x 5 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tips[sat_filter]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "behind-chart",
   "metadata": {},
   "source": [
    "Notice that it now says that the table only has 87 rows, down from 244. However, the index has been maintained. This is because the row labels are connected to the row, they're not just row numbers.\n",
    "\n",
    "It is more common to do this in one step, rather than creating and naming a filter object. So the code becomes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "governing-recording",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20.65</td>\n",
       "      <td>2.34</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>17.92</td>\n",
       "      <td>2.86</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>20.29</td>\n",
       "      <td>1.92</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>15.77</td>\n",
       "      <td>1.56</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>39.42</td>\n",
       "      <td>5.31</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>35.83</td>\n",
       "      <td>3.27</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239</th>\n",
       "      <td>29.03</td>\n",
       "      <td>4.14</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>27.18</td>\n",
       "      <td>1.40</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>241</th>\n",
       "      <td>22.67</td>\n",
       "      <td>1.40</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>17.82</td>\n",
       "      <td>1.22</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>87 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     total_bill   tip  day    time  size\n",
       "19        20.65  2.34  Sat  Dinner     3\n",
       "20        17.92  2.86  Sat  Dinner     2\n",
       "21        20.29  1.92  Sat  Dinner     2\n",
       "22        15.77  1.56  Sat  Dinner     2\n",
       "23        39.42  5.31  Sat  Dinner     4\n",
       "..          ...   ...  ...     ...   ...\n",
       "238       35.83  3.27  Sat  Dinner     3\n",
       "239       29.03  4.14  Sat  Dinner     3\n",
       "240       27.18  1.40  Sat  Dinner     2\n",
       "241       22.67  1.40  Sat  Dinner     2\n",
       "242       17.82  1.22  Sat  Dinner     2\n",
       "\n",
       "[87 rows x 5 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tips[tips[\"day\"] == \"Sat\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "convenient-livestock",
   "metadata": {},
   "source": [
    "This has given us back our subset of data as another `DataFrame` which can used in exactly the same way as the previous one (further filtering, summarising etc.)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "played-article",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "### Exercise 1\n",
    "\n",
    "- Select the data for only Thursdays.\n",
    "- Calculate the mean of the `tip` column for Thursdays\n",
    "- Compare this with the mean of the `tip` column for Saturdays\n",
    "\n",
    "[<small>answer</small>](answer_filtering_tip_day.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "surprised-treat",
   "metadata": {},
   "source": [
    "## Other filters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "looking-velvet",
   "metadata": {},
   "source": [
    "As well as filtering with the `==` operator (which only checks for exact matches), you can do other types of comparisons. Any of the standard Python comparisons will work (i.e. `==`, `!=`, `<`, `<=`, `>`, `>=`).\n",
    "\n",
    "To grab only the rows where the total bill is less than £8 we can use `<`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "photographic-jacob",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>3.07</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>5.75</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Fri</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>7.25</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>7.51</td>\n",
       "      <td>1.40</td>\n",
       "      <td>Thur</td>\n",
       "      <td>Lunch</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>7.25</td>\n",
       "      <td>3.60</td>\n",
       "      <td>Sun</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>7.56</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Thur</td>\n",
       "      <td>Lunch</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>218</th>\n",
       "      <td>7.74</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     total_bill   tip   day    time  size\n",
       "67         3.07  0.70   Sat  Dinner     1\n",
       "92         5.75  0.70   Fri  Dinner     2\n",
       "111        7.25  0.70   Sat  Dinner     1\n",
       "149        7.51  1.40  Thur   Lunch     2\n",
       "172        7.25  3.60   Sun  Dinner     2\n",
       "195        7.56  1.01  Thur   Lunch     2\n",
       "218        7.74  1.01   Sat  Dinner     2"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tips[tips[\"total_bill\"] < 8]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "intellectual-assembly",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "### Exercise 2\n",
    "\n",
    "Filter the data to only include parties of 5 or more people.\n",
    "\n",
    "[<small>answer</small>](answer_filtering_party_size.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "restricted-journalism",
   "metadata": {},
   "source": [
    "## Combining filters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "blocked-bundle",
   "metadata": {},
   "source": [
    "If you want to apply multiple filters, for example to select only \"Saturdays with small total bills\" you can do it in one of two different ways. Either split the question into multiple steps, or ask it all at once.\n",
    "\n",
    "Let's do it multiple steps first since we already have tools we need for that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "young-pathology",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>3.07</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>7.25</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>218</th>\n",
       "      <td>7.74</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     total_bill   tip  day    time  size\n",
       "67         3.07  0.70  Sat  Dinner     1\n",
       "111        7.25  0.70  Sat  Dinner     1\n",
       "218        7.74  1.01  Sat  Dinner     2"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sat_tips = tips[tips[\"day\"] == \"Sat\"]  # First grab the Saturday data and save it as a variable\n",
    "sat_tips[sat_tips[\"total_bill\"] < 8]  # Then act on the new DataFrame as use it as before"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eligible-fitting",
   "metadata": {},
   "source": [
    "Or, you can combine the questions together using the `&` operator with a syntax like:\n",
    "\n",
    "```python\n",
    "df[(filter_1) & (filter_2)]\n",
    "```\n",
    "\n",
    "so in our case filter 1 is `tips[\"day\"] == \"Sat\"` and filter 2 is `tips[\"total_bill\"] < 8` so it becomes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "touched-accent",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>3.07</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>7.25</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>218</th>\n",
       "      <td>7.74</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Sat</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     total_bill   tip  day    time  size\n",
       "67         3.07  0.70  Sat  Dinner     1\n",
       "111        7.25  0.70  Sat  Dinner     1\n",
       "218        7.74  1.01  Sat  Dinner     2"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tips[(tips[\"day\"] == \"Sat\") & (tips[\"total_bill\"] < 8)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cleared-dairy",
   "metadata": {},
   "source": [
    "If you want to do an \"or\" operation, then instead of `&` you can use `|`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cardiac-provision",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "### Exercise 3\n",
    "\n",
    "Filter the data to only include parties of 4 or more people which happened at lunch time.\n",
    "\n",
    "Hint: The `size` and `time` columns are what you want to use here.\n",
    "\n",
    "[<small>answer</small>](answer_filtering_combined.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d53c568-ea93-400c-a11a-b2900774cb7b",
   "metadata": {},
   "source": [
    "## DataFrame indexing\n",
    "\n",
    "When we use the square bracket syntax on a `DataFrame` directly there are a few different types of object that can be passed:\n",
    "\n",
    "<dl>\n",
    "<dt>A single string</dt>\n",
    "<dd>This will select a single column form the <code>DataFrame</code>, returning a <code>Series</code> object.</dd>\n",
    "<dt>A list of strings</dt>\n",
    "<dd>This will select those columns by name, returning a <code>DataFrame</code>.</dd>\n",
    "<dt>A filter (a <code>Series</code> of <code>True</code>/<code>False</code>)</dt>\n",
    "<dd>This will filter the table as a whole, returning a <code>DataFrame</code> with only the rows matching <code>True</code>included.</dd>\n",
    "</dl>\n",
    "\n",
    "These are provided as shortcuts as they are the most common operations to do an a `DataFrame`. This is why some of them operate on columns and other on rows.\n",
    "\n",
    "If you want to be explicit about which axis you are acting on, you can pass these same types of objects to the `.loc[rows, columns]` attribute with one argument per axis. This means that\n",
    "\n",
    "```python\n",
    "tips[sat_filter]\n",
    "```\n",
    "\n",
    "is equivalent to\n",
    "```python\n",
    "tips.loc[sat_filter]\n",
    "```\n",
    "\n",
    "and that\n",
    "```python\n",
    "tips[\"size\"]\n",
    "```\n",
    "\n",
    "is equivalent to\n",
    "```python\n",
    "tips.loc[:, \"size\"]\n",
    "```\n",
    "\n",
    "The full set of rules for [`DataFrame.loc` are in the documentation](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.loc.html)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}