lefty_longevity_workbook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Do Left-Handed People Really Die Young?\n",
    "\n",
    "Use pandas and Bayesian statistics to see if the urban legend is true that lefties die earlier than righties.\n",
    "\n",
    "The content of this project is nearly identical to a DataCamp project I created at the end of 2018, but this version is free!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to use this notebook\n",
    "\n",
    "This notebook contains a series of cell blocks that fall into one of five types:\n",
    "\n",
    "* **Context:** a text cell following a numbered title that contains background information about a task.\n",
    "* **Instructions:** a text cell that contains the specific instructions for a task.\n",
    "* **Hint:** a hint about how to accomplish the coding task.\n",
    "* **Sample code:** scaffolding code to get you started on completing a task.\n",
    "* **Tests:** tests to check solutions for the task.\n",
    "\n",
    "If you'd just like to see the entire project, you can read **[the full notebook](https://nbviewer.jupyter.org/github/mbonsma/lefty-longevity/blob/master/lefty_longevity_with_solutions.ipynb)** from top to bottom, glossing over the **hint**, **sample code**, and **tests** sections. \n",
    "\n",
    "If you'd like to work on this project as an exercise, you are looking at the right notebook! Just follow the instructions below. If you have any issues at all getting set up, please let me know and I will be very happy to troubleshoot and make the instructions more clear.\n",
    "\n",
    "### Working through the exercises\n",
    "\n",
    "#### Get set up with Jupyter and Python\n",
    "1. Install [Jupyter notebook](https://jupyter.org/install) and Python. I recommend installing [Anaconda](https://www.anaconda.com/distribution/#download-section), which comes with both Jupyter and Python (and lots of other useful stuff). \n",
    "2. Download this notebook onto your computer. You can download the entire GitHub repository or just this notebook, your choice.\n",
    "3. Run Jupyter Notebook: \n",
    "  * Open Anaconda Navigator and select Jupyter Notebook, or\n",
    "  * Open a terminal window or anaconda prompt and type `jupyter notebook`. Jupyter Notebook will launch in your default browser.\n",
    "4. In the Jupyter Notebook interface, navigate to the folder in which you saved this notebook, and click on it (`lefty_longevity_workbook.ipynb`) to open the notebook.\n",
    "\n",
    "#### Using the notebook\n",
    "1. Run the first cell below to load the testing module (optional)\n",
    "2. Read the **context** and **instructions** cells below each numbered heading.\n",
    "3. In the **sample code** cell, try coding your solution and running the cell. (To run a cell, click `Run` in the top menu or hit `SHIFT-ENTER`.)\n",
    "4. To test your solution for each task, run the **tests** cell that directly follows the **sample code** cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this is the module to run the tests\n",
    "%load_ext ipython_nose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 1. Where are the old left-handed people?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "![Barack Obama signs the Patient Protection and Affordable Care Act at the White House, March 23, 2010](img/Obama_signs_health_care-20100323.jpg)\n",
    "\n",
    "Barack Obama is left-handed. So are Bill Gates and Oprah Winfrey; so were Babe Ruth and Marie Curie. A [1991 study](https://www.nejm.org/doi/full/10.1056/NEJM199104043241418) reported that left-handed people die on average nine years earlier than right-handed people. Nine years! Could this really be true? \n",
    "\n",
    "In this notebook, we will explore this phenomenon using age distribution data to see if we can reproduce a difference in average age at death purely from the changing rates of left-handedness over time, refuting the claim of early death for left-handers. This notebook uses `pandas` and Bayesian statistics to analyze the probability of being a certain age at death given that you are reported as left-handed or right-handed.\n",
    "\n",
    "A National Geographic survey in 1986 resulted in over a million responses that included age, sex, and hand preference for throwing and writing. Researchers Avery Gilbert and Charles Wysocki analyzed this data and noticed that rates of left-handedness were around 13% for people younger than 40 but decreased with age to about 5% by the age of 80. They concluded based on analysis of a subgroup of people who throw left-handed but write right-handed that this age-dependence was primarily due to changing social acceptability of left-handedness. This means that the rates aren't a factor of *age* specifically but rather of the *year you were born*, and if the same study was done today, we should expect a shifted version of the same distribution as a function of age. Ultimately, we'll see what effect this changing rate has on the apparent mean age of death of left-handed people, but let's start by plotting the rates of left-handedness as a function of age.\n",
    "\n",
    "This notebook uses two datasets: [death distribution data](https://www.cdc.gov/nchs/data/statab/vs00199_table310.pdf) for the United States from the year 1999 (source website [here](https://www.cdc.gov/nchs/nvss/mortality_tables.htm)) and rates of left-handedness digitized from a figure in this [1992 paper by Gilbert and Wysocki](https://www.ncbi.nlm.nih.gov/pubmed/1528408). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Load the handedness data from the National Geographic survey and create a scatter plot.\n",
    "\n",
    "- Import `pandas` as `pd`  and `matplotlib.pyplot` as `plt`.\n",
    "- Load the data into a pandas DataFrame named `lefthanded_data` using the provided `data_url_1`. Note that the file is a CSV file.\n",
    "- Use the `.plot()` method to create a plot of the \"Male\" and \"Female\" columns vs. \"Age\".\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Good to know\n",
    "\n",
    "This project requires the use of `pandas`, Bayesian statistics and probability, and data visualization. Some specific skills focused on here are reading data, creating new columns, creating `matplotlib` visualizations, using `numpy` arrays, constructing probabilities from frequency data, and Bayes' theorem. \n",
    "\n",
    "Helpful links:\n",
    "- pandas [cheat sheet](https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf)\n",
    "- pandas `read_csv()` function [documentation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html)\n",
    "- numpy [tutorial](https://docs.scipy.org/doc/numpy/user/quickstart.html)\n",
    "- matplotlib [tutorial](https://matplotlib.org/tutorials/introductory/pyplot.html)\n",
    "- [short intro to Python functions](https://www.w3schools.com/python/python_functions.asp)\n",
    "- [Bayes' theorem on Wikipedia](https://en.wikipedia.org/wiki/Bayes%27_theorem)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "You can read `https://data_url.csv` into a DataFrame named `my_data` like this after importing the `pandas` library:\n",
    "\n",
    "```python\n",
    "import pandas as pd\n",
    "my_data = pd.read_csv(\"https://data_url.csv\")\n",
    "```\n",
    "\n",
    "You can plot columns from a `pandas` DataFrame like this after importing the `matplotlib` library:\n",
    "\n",
    "```python\n",
    "import matplotlib.pyplot as plt\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(\"x_column\", \"y_column\", data = my_data)\n",
    "ax.set_xlabel(\"Label for the x axis of the plot\")\n",
    "ax.set_ylabel(\"Label for the y axis of the plot\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# import libraries\n",
    "# ... YOUR CODE FOR TASK 1 ...\n",
    "\n",
    "# load the data\n",
    "data_url_1 = \"https://gist.githubusercontent.com/mbonsma/8da0990b71ba9a09f7de395574e54df1/raw/aec88b30af87fad8d45da7e774223f91dad09e88/lh_data.csv\"\n",
    "lefthanded_data = ...\n",
    "\n",
    "# plot male and female left-handedness rates vs. age\n",
    "%matplotlib inline\n",
    "fig, ax = plt.subplots() # create figure and axis objects\n",
    "ax.plot(..., ..., ..., marker = 'o') # plot \"Female\" vs. \"Age\"\n",
    "ax.plot(..., ..., ..., marker = 'x') # plot \"Male\" vs. \"Age\"\n",
    "ax.legend() # add a legend\n",
    "ax.set_xlabel(...)\n",
    "ax.set_ylabel(...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_data_shape():\n",
    "    assert (lefthanded_data.shape == (77, 3)), \\\n",
    "    'The lefthanded_data you loaded is not the right shape. It should be 77, 3.'\n",
    "    \n",
    "def test_num_lines():\n",
    "    assert (len(ax.lines) == 2), \\\n",
    "    'Did you plot lefthanded rates for both men and women?'\n",
    "    \n",
    "def test_plot_dimensions():\n",
    "     assert ((ax.get_yticks()[-1] < 20) and (ax.get_xticks()[-1] > 20)), \\\n",
    "     'Did you plot \"Female\" vs. \"Age\" and \"Male\" vs. \"Age\"?'\n",
    "\n",
    "def test_plot_labels():\n",
    "    assert ax.get_xlabel() != '' and ax.get_ylabel() != '', \\\n",
    "    'Please add x and y labels to your plot.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 2. Rates of left-handedness over time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "Let's convert this data into a plot of the rates of left-handedness as a function of the year of birth, and average over male and female to get a single rate for both sexes. \n",
    "\n",
    "Since the study was done in 1986, the data after this conversion will be the percentage of people alive in 1986 who are left-handed as a function of the year they were born. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Add two new columns, one for birth year and one for mean left-handedness, then plot the mean as a function of birth year.\n",
    "- Create a column in `lefthanded_data` called `Birth_year`, which is equal to 1986 - `Age` (since the study was done in 1986).\n",
    "- Create a column in `lefthanded_data` called `Mean_lh` which is equal to the mean of the `Male` and `Female` columns.\n",
    "- Use the `.plot()` method to plot `Mean_lh` vs. `Birth_year`.\n",
    "\n",
    "<hr>\n",
    "\n",
    "Helpful links:\n",
    "- [Creating a column in `pandas`](https://coderwall.com/p/dt4yja/adding-columns-to-a-pandas-dataframe)\n",
    "- matplotlib [tutorial](https://matplotlib.org/tutorials/introductory/pyplot.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "You can create a new column called \"Integer_temp\" in the DataFrame `my_data` from an existing column called \"Temp\" like so:\n",
    "\n",
    "```python\n",
    "my_data[\"Integer_temp\"] = int(my_data[\"Temp\"])\n",
    "\n",
    "```\n",
    "\n",
    "You can calculate the mean of two columns row-wise by using the `.mean()` function and setting the parameter `axis = 1`. Try something like this:\n",
    "\n",
    "```python\n",
    "my_data[\"means\"] = my_data[[\"column_1\", \"column_2\"]].mean(axis = 1)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# create a new column for birth year of each age\n",
    "# ... YOUR CODE FOR TASK 2 ...\n",
    "\n",
    "# create a new column for the average of male and female\n",
    "# ... YOUR CODE FOR TASK 2 ...\n",
    "\n",
    "# create a plot of the 'Mean_lh' column vs. 'Birth_year'\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(..., ..., ...) # plot 'Mean_lh' vs. 'Birth_year'\n",
    "ax.set_xlabel(...) # set the x label for the plot\n",
    "ax.set_ylabel(...) # set the y label for the plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_new_columns():\n",
    "    assert 'Birth_year' and 'Mean_lh' in lefthanded_data.columns, \\\n",
    "    'Did you create two new columns called \"Birth_year\" and \"Mean_lh\"?'\n",
    "    \n",
    "def test_birth_year_column():\n",
    "    import numpy as np\n",
    "    assert np.all(lefthanded_data[\"Birth_year\"] >= 1900), \\\n",
    "    'The values in the \"Birth_year\" column should be years >= 1900.'\n",
    "    \n",
    "def test_mean_lh_column():\n",
    "    import numpy as np\n",
    "    assert not np.any(np.isnan(lefthanded_data[\"Mean_lh\"])), \\\n",
    "    'Make sure you calculate the mean lefthandedness for each row of the DataFrame.'\n",
    "\n",
    "def test_plot_contents():\n",
    "    assert (len(ax.lines) == 1), \\\n",
    "    'Did you plot the mean lefthandedness data?'\n",
    "    \n",
    "def test_plot_labels():\n",
    "    assert ax.get_xlabel() != '' and ax.get_ylabel() != '', \\\n",
    "    'Please add x and y labels to your plot.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 3. Applying Bayes' rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "The probability of dying at a certain age given that you're left-handed is **not** equal to the probability of being left-handed given that you died at a certain age. This inequality is why we need **Bayes' theorem**, a statement about conditional probability which allows us to update our beliefs after seeing evidence. \n",
    "\n",
    "We want to calculate the probability of dying at age A given that you're left-handed. Let's write this in shorthand as P(A | LH). We also want the same quantity for right-handers: P(A | RH). \n",
    "\n",
    "Here's Bayes' theorem for the two events we care about: left-handedness (LH) and dying at age A.\n",
    "\n",
    "$$P(A | LH) = \\frac{P(LH|A) P(A)}{P(LH)}$$\n",
    "\n",
    "P(LH | A) is the probability that you are left-handed *given that* you died at age A. P(A) is the overall probability of dying at age A, and P(LH) is the overall probability of being left-handed. We will now calculate each of these three quantities, beginning with P(LH | A).\n",
    "\n",
    "To calculate P(LH | A) for ages that might fall outside the original data, we will need to extrapolate the data to earlier and later years. Since the rates flatten out in the early 1900s and late 1900s, we'll use a few points at each end and take the mean to extrapolate the rates on each end. The number of points used for this is arbitrary, but we'll pick 10 since the data looks flat-ish until about 1910. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Create a function that will return P(LH | A) for particular ages of death in a given study year.\n",
    "\n",
    "- Import the `numpy` package aliased as `np`.\n",
    "- Use the last ten `Mean_lh` data points to get an average rate for the early 1900s. Name the resulting DataFrame `early_1900s_rate`.\n",
    "- Use the first ten `Mean_lh` data points to get an average rate for the late 1900s. Name the resulting DataFrame `late_1900s_rate`.\n",
    "- For the early 1900s ages, fill in `P_return` with the appropriate left-handedness rates for `ages_of_death`. That is, input `early_1900s_rate` as a fraction, i.e., divide by 100.\n",
    "- For the late 1900s ages, fill in `P_return` with the appropriate left-handedness rates for `ages_of_death`. That is, input `late_1900s_rate` as a fraction, i.e., divide by 100.\n",
    "\n",
    "<hr>\n",
    "\n",
    "When calculating `early_1900s_rate` and `late_1900s_rate`, remember that because the original data was from youngest age to oldest age, that means that the data is organized from latest birth year to earliest birth year. You will use the first ten `Mean_lh` data points to get an average rate for the late 1900s and the last ten for the early 1900s. \n",
    "\n",
    "Helpful links:\n",
    "- [Python functions](https://www.w3schools.com/python/python_functions.asp)\n",
    "- [Conditional probability](https://en.wikipedia.org/wiki/Conditional_probability)\n",
    "- [numpy array subsetting and indexing](https://docs.scipy.org/doc/numpy-1.10.0/user/basics.indexing.html)\n",
    "- numpy `logical_and()` [documentation](https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.logical_and.html)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "You can get the mean of the last ten rows of the \"column\" column in `my_data` like so:\n",
    "\n",
    "```python\n",
    "last_ten_mean = my_data[\"column\"][-10:].mean()\n",
    "```\n",
    "\n",
    "For using array indexing to fill in `P_return`, since `ages_of_death` includes some ages that are older than the oldest age represented in `lefthanded_data`, `P_return` can be filled in with `early_1900s_rate` for the correct ages like this:\n",
    "\n",
    "```python\n",
    "P_return[ages_of_death > oldest_age] = early_1900s_rate / 100\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# import library\n",
    "# ... YOUR CODE FOR TASK 3 ...\n",
    "\n",
    "# create a function for P(LH | A)\n",
    "def P_lh_given_A(ages_of_death, study_year = 1990):\n",
    "    \"\"\" P(Left-handed | ages of death), calculated based on the reported rates of left-handedness.\n",
    "    Inputs: numpy array of ages of death, study_year\n",
    "    Returns: probability of left-handedness given that subjects died in `study_year` at ages `ages_of_death` \"\"\"\n",
    "    \n",
    "    # Use the mean of the 10 last and 10 first points for left-handedness rates before and after the start \n",
    "    early_1900s_rate = ... \n",
    "    late_1900s_rate = ...\n",
    "    middle_rates = lefthanded_data.loc[lefthanded_data['Birth_year'].isin(study_year - ages_of_death)]['Mean_lh']\n",
    "    youngest_age = study_year - 1986 + 10 # the youngest age is 10\n",
    "    oldest_age = study_year - 1986 + 86 # the oldest age is 86\n",
    "    \n",
    "    P_return = np.zeros(ages_of_death.shape) # create an empty array to store the results\n",
    "    # extract rate of left-handedness for people of ages 'ages_of_death'\n",
    "    P_return[ages_of_death > oldest_age] = ...\n",
    "    P_return[ages_of_death < youngest_age] = ...\n",
    "    P_return[np.logical_and((ages_of_death <= oldest_age), (ages_of_death >= youngest_age))] = middle_rates / 100\n",
    "    \n",
    "    return P_return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_output_type():\n",
    "    test_input = np.array([80])\n",
    "    assert (type(P_lh_given_A(test_input)) == float or \n",
    "            type(P_lh_given_A(test_input)) == np.float64 or \n",
    "            type(P_lh_given_A(test_input)) == np.ndarray), \\\n",
    "    'Does the function P_lh_given_A return a number?'\n",
    "\n",
    "def test_late_1900s_rate():\n",
    "    assert round(float(P_lh_given_A(np.array([10]))), 2) == 0.13, \\\n",
    "    'Did you calculate a left-handedness probability for the late 1900s?'\n",
    "\n",
    "def test_early_1900s_rate():\n",
    "    assert round(float(P_lh_given_A(np.array([95]))), 2) == 0.06, \\\n",
    "    'Did you calculate a left-handedness probability for the early 1900s?'\n",
    "\n",
    "def test_middle_rate():\n",
    "    assert P_lh_given_A(np.array([80])) > 0.06 and P_lh_given_A(np.array([80])) < 0.13, \\\n",
    "    'Make sure that P_lh_given_A returns the correct left-handedness rate as a fraction (< 1).'\n",
    "    \n",
    "def test_middle_rates_are_different():\n",
    "    assert np.any(P_lh_given_A(np.array([20])) != P_lh_given_A(np.array([50]))), \\\n",
    "    'P_lh_given_A should return different rates for different ages between youngest_age and oldest_age.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 4. When do people normally die?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "To estimate the probability of living to an age A, we can use data that gives the number of people who died in a given year and how old they were to create a distribution of ages of death. If we normalize the numbers to the total number of people who died, we can think of this data as a probability distribution that gives the probability of dying at age A. The data we'll use for this is from the entire US for the year 1999 - the closest I could find for the time range we're interested in. \n",
    "\n",
    "In this block, we'll load in the death distribution data and plot it. The first column is the age, and the other columns are the number of people who died at that age. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Load death distribution data for the United States and plot it.\n",
    "- Load death distribution data in the provided `data_url_2` into `death_distribution_data`, setting `sep = '\\t'` and `skiprows=[1]` to account for the dataset's format.\n",
    "- Drop the NaN values from the `Both Sexes` column.\n",
    "- Use the `.plot()` method to plot the number of people who died as a function of their age.\n",
    "\n",
    "<hr>\n",
    "\n",
    "Helpful links:\n",
    "- pandas `read_csv()` function [documentation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html)\n",
    "- pandas `dropna()` [documentation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.dropna.html)\n",
    "- matplotlib [tutorial](https://matplotlib.org/tutorials/introductory/pyplot.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "You can read `https://data_url.csv` into a DataFrame named `my_data` like this after importing the `pandas` library:\n",
    "\n",
    "```python\n",
    "import pandas as pd\n",
    "my_data = pd.read_csv(\"https://data_url.csv\")\n",
    "```\n",
    "\n",
    "You can drop `NaN` values in just one column of a DataFrame by using the `subset()` argument like this:\n",
    "\n",
    "```python\n",
    "my_data = my_data.dropna(subset = [\"column_name\"])\n",
    "```\n",
    "\n",
    "You can plot columns from a `pandas` DataFrame like this:\n",
    "\n",
    "```python\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(\"x_column\", \"y_column\", data = my_data, marker = 'o')\n",
    "ax.set_xlabel(\"Label for the x axis of the plot\")\n",
    "ax.set_ylabel(\"Label for the y axis of the plot\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Death distribution data for the United States in 1999\n",
    "data_url_2 = \"https://gist.githubusercontent.com/mbonsma/2f4076aab6820ca1807f4e29f75f18ec/raw/62f3ec07514c7e31f5979beeca86f19991540796/cdc_vs00199_table310.tsv\"\n",
    "\n",
    "# load death distribution data\n",
    "# ... YOUR CODE FOR TASK 4 ...\n",
    "\n",
    "# drop NaN values from the `Both Sexes` column\n",
    "# ... YOUR CODE FOR TASK 4 ...\n",
    "\n",
    "# plot number of people who died as a function of age\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(..., ..., data = ..., marker='o') # plot 'Both Sexes' vs. 'Age'\n",
    "ax.set_xlabel(...) \n",
    "ax.set_ylabel(...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_skiprows(): \n",
    "    assert death_distribution_data.loc[0][\"Age\"] == 0, \\\n",
    "    'Make sure to include `skiprows=[1]` as an argument in `pd.read_csv`.'\n",
    "    \n",
    "def test_data_shape(): # this will also test if the dropna worked\n",
    "    assert (death_distribution_data.shape == (120, 4)), \\\n",
    "    'Make sure you drop NaN values in the \"Both Sexes\" column only. The resulting DataFrame should have 120 rows.'\n",
    "    \n",
    "def test_plot_contents():\n",
    "    assert (len(ax.lines) == 1), \\\n",
    "    'Did you plot the death distribution data?'\n",
    "    \n",
    "def test_plot_dimensions():\n",
    "    assert ((ax.get_xticks()[-1] < 200) and (ax.get_yticks()[-1] > 200)), \\\n",
    "    'Did you plot \"Both Sexes\" vs. \"Age\"?'\n",
    "    \n",
    "def test_plot_labels():\n",
    "    assert ax.get_xlabel() != '' and ax.get_ylabel() != '', \\\n",
    "    'Please add x and y labels to your plot.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 5. The overall probability of left-handedness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "In the previous code block we loaded data to give us P(A), and now we need P(LH). P(LH) is the probability that a person who died in our particular study year is left-handed, assuming we know nothing else about them. This is the average left-handedness in the population of deceased people, and we can calculate it by summing up all of the left-handedness probabilities for each age, weighted with the number of deceased people at each age, then divided by the total number of deceased people to get a probability. In equation form, this is what we're calculating, where N(A) is the number of people who died at age A (given by the dataframe `death_distribution_data`):\n",
    "\n",
    "<img src=\"https://i.imgur.com/gBIWykY.png\" alt=\"equation\" width=\"220\"/>\n",
    "\n",
    "<!--- $$P(LH) = \\frac{\\sum_{\\text{A}} P(LH | A) N(A)}{\\sum_{\\text{A}} N(A)}$$ -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Create a function called `P_lh()` which calculates the overall probability of left-handedness in the population for a given study year.\n",
    "- Create a series, `p_list`, by multiplying the number of dead people in the `Both Sexes` column with the probability of their being lefthanded using `P_lh_given_A()`. \n",
    "- Set the variable `p` equal to the sum of that series.\n",
    "- Divide `p` by the total number of dead people by summing `death_distribution_data` over the `Both Sexes` column. Return result from the function.\n",
    "\n",
    "<hr>\n",
    "\n",
    "P(LH | A) was defined in Task 3. N(A) is the value of `Both Sexes` in the `death_distribution_data` DataFrame where the `Age` column is equal to A. The denominator is total number of dead people, which you can get by summing over the entire dataframe in the `Both Sexes` column. \n",
    "\n",
    "Helpful links:\n",
    "- [Python functions](https://www.w3schools.com/python/python_functions.asp) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "You can calculate the numerator (P(LH | A) N(A)) in one line by multiplying the \"Both Sexes\" column of `death_distribution_data` by `P_lh_given_A` for all the ages in `death_distribution_data['Age']`. Try something like this:\n",
    "\n",
    "```python\n",
    "p_list = death_distribution_data['Both Sexes']*P_lh_given_A(death_distribution_data['Age'], study_year)\n",
    "```\n",
    "\n",
    "To normalize P(LH | A) N(A) by the total number of deceased people (\\sum_A N(A)), try this:\n",
    "\n",
    "```python\n",
    "p/np.sum(death_distribution_data['Both Sexes'])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "def P_lh(death_distribution_data, study_year = 1990): # sum over P_lh for each age group\n",
    "    \"\"\" Overall probability of being left-handed if you died in the study year\n",
    "    Input: dataframe of death distribution data, study year\n",
    "    Output: P(LH), a single floating point number \"\"\"\n",
    "    p_list = ... # multiply number of dead people by P_lh_given_A\n",
    "    p = ... # calculate the sum of p_list\n",
    "    return ... # normalize to total number of people (sum of death_distribution_data['Both Sexes'])\n",
    "\n",
    "print(P_lh(death_distribution_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_output_type():\n",
    "    assert type(P_lh(death_distribution_data)) == np.float64 or type(P_lh(death_distribution_data)) == float, \\\n",
    "    'Have you defined a function called P_lh that returns the overall probability of left-handedness?'\n",
    "\n",
    "def test_study_year():\n",
    "    assert round(P_lh(death_distribution_data, 2018), 2) == 0.11, \\\n",
    "    'Make sure to include `study_year` as the second argument for the function P_lh()'\n",
    "\n",
    "def test_P_lh():\n",
    "    assert round(P_lh(death_distribution_data), 2) == 0.08, \\\n",
    "    'The overall probability of left-handedness should be approximately 0.08.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 6. Putting it all together: dying while left-handed (i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "Now we have the means of calculating all three quantities we need: P(A), P(LH), and P(LH | A). We can combine all three using Bayes' rule to get P(A | LH), the probability of being age A at death (in the study year) given that you're left-handed. To make this answer meaningful, though, we also want to compare it to P(A | RH), the probability of being age A at death given that you're right-handed. \n",
    "\n",
    "We're calculating the following quantity twice, once for left-handers and once for right-handers.\n",
    "\n",
    "$$P(A | LH) = \\frac{P(LH|A) P(A)}{P(LH)}$$\n",
    "\n",
    "\n",
    "First, for left-handers.\n",
    "\n",
    "<!--Notice that I was careful not to call these \"probability of dying at age A\", since that's not actually what we're calculating: we use the exact same death distribution data for each. -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Write a function to calculate `P_A_given_lh()`.\n",
    "- Calculate `P_A`, the overall probability of dying at age A, which is given by `death_distribution_data` at age A divided by the total number of dead people (the sum of the `Both Sexes` column of `death_distribution_data`).\n",
    "- Calculate the overall probability of left-handedness P(LH) using the function defined in Task 5.\n",
    "- Calculate P(LH | A) using the function defined in Task 3.\n",
    "\n",
    "<hr>\n",
    "\n",
    "Helpful links:\n",
    "- [Python functions](https://www.w3schools.com/python/python_functions.asp)\n",
    "- [Bayes' theorem](https://en.wikipedia.org/wiki/Bayes%27_theorem)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "To calculate P(A) for particular ages of death A, divide `death_distribution_data['Both Sexes'][ages_of_death]` by the sum of the \"Both Sexes\" column. To calculate the sum, try something like this:\n",
    "\n",
    "```python\n",
    "death_distribution_data['Both Sexes'].sum() \n",
    "# or\n",
    "np.sum(death_distribution_data['Both Sexes'])\n",
    "```\n",
    "\n",
    "For the function `P_A_given_lh()`, return `P_lh_A*P_A/P_left`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "def P_A_given_lh(ages_of_death, death_distribution_data, study_year = 1990):\n",
    "    \"\"\" The overall probability of being a particular `age_of_death` given that you're left-handed \"\"\"\n",
    "    P_A = ...\n",
    "    P_left = ... # use P_lh function to get probability of left-handedness overall\n",
    "    P_lh_A = ... # use P_lh_given_A to get probability of left-handedness for a certain age\n",
    "    return P_lh_A*P_A/P_left"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_output_type():\n",
    "    test_input = np.array([60])\n",
    "    assert (type(P_A_given_lh(test_input, death_distribution_data)) == pd.core.series.Series), \\\n",
    "    'Have you defined a function called P_A_given_lh that returns a pandas Series?'\n",
    "\n",
    "def test_output_is_probability():\n",
    "    test_input = np.array([60])\n",
    "    assert (P_A_given_lh(test_input, death_distribution_data) < 1).all(), \\\n",
    "    'Make sure the function returns numbers that are less than 1.'\n",
    "    \n",
    "def test_output_sums_to_1():\n",
    "    test_input = np.arange(0,115)\n",
    "    assert (round(np.nansum(P_A_given_lh(test_input, death_distribution_data)), 2) == 1), \\\n",
    "    'P_A_given_lh(np.arange(0,115), death_distribtuion_data) should sum up to 1.'\n",
    "\n",
    "def test_study_year():\n",
    "    test_input = np.array([45])\n",
    "    assert (P_A_given_lh(test_input, death_distribution_data) > \n",
    "            P_A_given_lh(test_input, death_distribution_data, 2018)).all(), \\\n",
    "    'Make sure to include `study_year` as the third argument for the function P_A_given_lh()'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 7. Putting it all together: dying while left-handed (ii)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "And now for right-handers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Write a function to calculate `P_A_given_rh()`.\n",
    "- Calculate `P_A`, the overall probability of dying at age A, which is given by `death_distribution_data` at age A divided by the total number of dead people. (This value is the same as in task 6.)\n",
    "- Calculate the overall probability of right-handedness P(RH), which is 1-P(LH).\n",
    "- Calculate P(RH | A), which is 1 - P(LH | A).\n",
    "\n",
    "<hr>\n",
    "\n",
    "Helpful links:\n",
    "- [Python functions](https://www.w3schools.com/python/python_functions.asp)\n",
    "- [Bayes' theorem](https://en.wikipedia.org/wiki/Bayes%27_theorem)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "P(A) is the same for both `P_A_given_lh()` and `P_A_given_rh()`.\n",
    "\n",
    "For the function `P_A_given_rh()`, return `P_rh_A*P_A/P_right`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "def P_A_given_rh(ages_of_death, death_distribution_data, study_year = 1990):\n",
    "    \"\"\" The overall probability of being a particular `age_of_death` given that you're right-handed \"\"\"\n",
    "    P_A = ...\n",
    "    P_right = ... # either you're left-handed or right-handed, so P_right = 1 - P_left\n",
    "    P_rh_A = ... # P_rh_A = 1 - P_lh_A \n",
    "    return P_rh_A*P_A/P_right"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_output_type():\n",
    "    test_input = np.array([60])\n",
    "    assert (type(P_A_given_rh(test_input, death_distribution_data)) == pd.core.series.Series), \\\n",
    "    'Have you defined a function called P_A_given_rh that returns a pandas Series?'\n",
    "\n",
    "def test_output_is_probability():\n",
    "    test_input = np.array([60])\n",
    "    assert (P_A_given_rh(test_input, death_distribution_data) < 1).all(), \\\n",
    "    'Make sure the function returns numbers that are less than 1.'\n",
    "\n",
    "def test_output_sums_to_1():\n",
    "    test_input = np.arange(0,115)\n",
    "    assert (round(np.nansum(P_A_given_rh(test_input, death_distribution_data)), 2) == 1), \\\n",
    "    'P_A_given_rh(np.arange(0,115), death_distribtuion_data) should sum up to 1.'\n",
    "    \n",
    "def test_correct_trend():\n",
    "    assert (P_A_given_lh(np.array([80]), death_distribution_data) < \n",
    "    P_A_given_rh(np.array([80]), death_distribution_data)).all(), \\\n",
    "    'Did you mix up any components of P_A_given_lh and P_A_given_rh?'\n",
    "\n",
    "def test_study_year():\n",
    "    test_input = np.array([45])\n",
    "    assert (P_A_given_rh(test_input, death_distribution_data) < \n",
    "            P_A_given_rh(test_input, death_distribution_data, 2018)).all(), \\\n",
    "    'Make sure to include `study_year` as the third argument for the function P_A_given_rh()'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 8. Plotting the distributions of conditional probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "Now that we have functions to calculate the probability of being age A at death given that you're left-handed or right-handed, let's plot these probabilities for a range of ages of death from 6 to 120. \n",
    "\n",
    "Notice that the left-handed distribution has a bump below age 70: of the pool of deceased people, left-handed people are more likely to be younger. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Plot the probability of being a certain age at death given that you're left- or right-handed for a range of ages.\n",
    "- Calculate `P_A_given_lh` and `P_A_given_rh` using the functions defined in Task 6.\n",
    "- Use the `.plot()` method to plot the results versus age.\n",
    "\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "The functions `P_A_given_lh()` and `P_A_given_rh()` take an array of ages and the death distribution DataFrame as input. You can calculate the overall probability for each age with a single function call. Try something like this:\n",
    "\n",
    "```python\n",
    "left_handed_probability = P_A_given_lh(ages, death_distribution_data)\n",
    "```\n",
    "\n",
    "You can plot two lists or arrays of the same length (one for `x` values and one for `y` values) like this:\n",
    "\n",
    "```python\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(x_list, y_list, label = \"label for legend\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "ages = np.arange(6, 115, 1) # make a list of ages of death to plot\n",
    "\n",
    "# calculate the probability of being left- or right-handed for each \n",
    "left_handed_probability = P_A_given_lh(...)\n",
    "right_handed_probability = P_A_given_rh(...)\n",
    "\n",
    "# create a plot of the two probabilities vs. age\n",
    "fig, ax = plt.subplots() # create figure and axis objects\n",
    "ax.plot(ages, left_handed_probability, label = \"Left-handed\")\n",
    "ax.plot(..., ..., label = ...)\n",
    "ax.legend() # add a legend\n",
    "ax.set_xlabel(\"Age at death\")\n",
    "ax.set_ylabel(r\"Probability of being age A at death\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_list_length():\n",
    "    assert len(left_handed_probability) > 1 and len(right_handed_probability) > 1, \\\n",
    "    'Make sure you append values to each list for every age A in ages.'\n",
    "    \n",
    "def test_list_values():\n",
    "    assert np.max(left_handed_probability) < 1 and np.max(right_handed_probability) < 1, \\\n",
    "    'All the values in each list should be probabilities (less than 1)'\n",
    "\n",
    "def test_lists_are_different():\n",
    "    assert np.max(left_handed_probability) < np.max(right_handed_probability), \\\n",
    "    'Did you calculate separate values for left- and right-handed probabilities?'\n",
    "    \n",
    "def test_num_lines():\n",
    "    assert (len(ax.lines) == 2), \\\n",
    "    'Did you plot lefthanded rates for both men and women?'\n",
    "\n",
    "def test_plot_labels():\n",
    "    assert ax.get_xlabel() != '' and ax.get_ylabel() != '', \\\n",
    "    'Please add x and y labels to your plot.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 9. Moment of truth: age of left and right-handers at death"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "Finally, let's compare our results with the original study that found that left-handed people were nine years younger at death on average. We can do this by calculating the mean of these probability distributions in the same way we calculated P(LH) earlier, weighting the probability distribution by age and summing over the result.\n",
    "\n",
    "$$\\text{Average age of left-handed people at death} = \\sum_A A P(A | LH)$$\n",
    "\n",
    "$$\\text{Average age of right-handed people at death} = \\sum_A A P(A | RH)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Find the mean age at death for left-handers and right-handers.\n",
    "- Multiply the `ages` list by the left-handed probabilities of being those ages at death, then use `np.nansum` to calculate the sum. Assign the result to `average_lh_age`.\n",
    "- Do the same with the right-handed probabilities to calculate `average_rh_age`.\n",
    "- Print `average_lh_age` and `average_rh_age`.\n",
    "- Calculate the difference between the two average ages and print it.\n",
    "\n",
    "<hr>\n",
    "\n",
    "To make your printed output prettier, try using the `round()` function to round your results to two decimal places.\n",
    "\n",
    "Helpful links:\n",
    "\n",
    "* [`np.nansum()` documentation](https://docs.scipy.org/doc/numpy-1.10.4/reference/generated/numpy.nansum.html)\n",
    "* [`round()` documentation](https://www.programiz.com/python-programming/methods/built-in/round)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "To calculate the average age of death for each group, multiply the list of `ages` by the list of probabilities stored in `left_handed_probability` and `right_handed_probability`. The result should be a single number. Try something like this:\n",
    "\n",
    "```python\n",
    "average_rh_age =  np.nansum(ages*np.array(right_handed_probability))\n",
    "```\n",
    "\n",
    "To round a number to `n` decimal places and print it to the screen, try something like this:\n",
    "\n",
    "```python\n",
    "print(round(my_variable, 2))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# calculate average ages for left-handed and right-handed groups\n",
    "# use np.array so that two arrays can be multiplied\n",
    "average_lh_age =  np.nansum(...*np.array(...))\n",
    "average_rh_age =  np.nansum(...*np.array(...))\n",
    "\n",
    "# print the average ages for each group\n",
    "# ... YOUR CODE FOR TASK 9 ...\n",
    "\n",
    "# print the difference between the average ages\n",
    "print(\"The difference in average ages is \" + str(round(... - ..., 1)) + \" years.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "def test_result_type():\n",
    "    assert ((type(average_lh_age) == np.float64 or type(average_lh_age) == float) and\n",
    "            (type(average_rh_age) == np.float64 or type(average_rh_age) == float)), \\\n",
    "    'average_lh_age and average_rh age should each be a single number.'\n",
    "\n",
    "def test_average_ages():\n",
    "    assert average_lh_age < average_rh_age, \\\n",
    "    'You should get a smaller number for average_lh_age than average_rh_age.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "type:NotebookTask"
    ]
   },
   "source": [
    "## 10. Final comments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@context"
    ]
   },
   "source": [
    "We got a pretty big age gap between left-handed and right-handed people purely as a result of the changing rates of left-handedness in the population, which is good news for left-handers: you probably won't die young because of your sinisterness. The reported rates of left-handedness have increased from just 3% in the early 1900s to about 11% today, which means that older people are much more likely to be reported as right-handed than left-handed, and so looking at a sample of recently deceased people will have more old right-handers.\n",
    "\n",
    "Our number is still less than the 9-year gap measured in the study. It's possible that some of the approximations we made are the cause: \n",
    "\n",
    "1. We used death distribution data from almost ten years after the study (1999 instead of 1991), and we used death data from the entire United States instead of California alone (which was the original study). \n",
    "2. We extrapolated the left-handedness survey results to older and younger age groups, but it's possible our extrapolation wasn't close enough to the true rates for those ages. \n",
    "\n",
    "One thing we could do next is figure out how much variability we would expect to encounter in the age difference purely because of random sampling: if you take a smaller sample of recently deceased people and assign handedness with the probabilities of the survey, what does that distribution look like? How often would we encounter an age gap of nine years using the same data and assumptions? We won't do that here, but it's possible with this data and the tools of random sampling. \n",
    "\n",
    "<!-- I did do this if we want to add more tasks - it would probably take three more blocks.-->\n",
    "\n",
    "To finish off, let's calculate the age gap we'd expect if we did the study in 2018 instead of in 1990. The gap turns out to be much smaller since rates of left-handedness haven't increased for people born after about 1960. Both the National Geographic study and the 1990 study happened at a unique time - the rates of left-handedness had been changing across the lifetimes of most people alive, and the difference in handedness between old and young was at its most striking. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@instructions"
    ]
   },
   "source": [
    "Redo the calculation from Task 8, setting the `study_year` parameter to 2018.\n",
    "- In the call to `P_A_given_lh`, set `age_of_death` to `ages`, `death_distribution_data` to `death_distribution_data`, and `study_year` to 2018.\n",
    "- Do the same for `P_A_given_rh`.\n",
    "\n",
    "<hr>\n",
    "\n",
    "You can read more about this interpretation of the study in this [BBC article](www.bbc.com/news/magazine-23988352)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "@hint"
    ]
   },
   "source": [
    "The first function call should look like this:\n",
    "\n",
    "```python\n",
    "P_A_given_lh(ages, death_distribution_data, study_year = 2018)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Calculate the probability of being left- or right-handed for all ages\n",
    "left_handed_probability_2018 = ...\n",
    "right_handed_probability_2018 = ...\n",
    "\n",
    "# calculate average ages for left-handed and right-handed groups\n",
    "average_lh_age_2018 = np.nansum(ages*np.array(left_handed_probability_2018))\n",
    "average_rh_age_2018 = np.nansum(ages*np.array(right_handed_probability_2018))\n",
    "\n",
    "print(\"The difference in average ages is \" + \n",
    "      str(round(average_rh_age_2018 - average_lh_age_2018, 1)) + \" years.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "@tests"
    ]
   },
   "outputs": [],
   "source": [
    "%%nose\n",
    "\n",
    "# def test_current_year():\n",
    "#     import datetime\n",
    "#     now = datetime.datetime.now()\n",
    "#     assert current_year >= now.year, \\\n",
    "#     'Did you set the \"current_year\" variable to the current year?'\n",
    "\n",
    "def test_list_length():\n",
    "    assert len(left_handed_probability_2018) > 1 and len(right_handed_probability_2018) > 1, \\\n",
    "    'Make sure you append values to each list for every age A in ages.'\n",
    "    \n",
    "def test_list_values():\n",
    "    assert np.max(left_handed_probability_2018) < 1 and np.max(right_handed_probability_2018) < 1, \\\n",
    "    'All the values in each list should be probabilities (less than 1).'\n",
    "\n",
    "def test_lists_are_different():\n",
    "    assert (left_handed_probability_2018 != right_handed_probability_2018).all(), \\\n",
    "    'Did you calculate separate values for left and right-handed probabilities?'\n",
    "    \n",
    "def test_average_ages():\n",
    "    assert (average_rh_age_2018 - average_lh_age_2018 < 4), \\\n",
    "    'The difference in ages between the left-handed and right-handed groups should be less than the value you calculated for the 1990 study.'"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}