statistics/Probability.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J5I4B-Eqn9CZ"
   },
   "source": [
    "# Overview\n",
    "\n",
    "Probability is the liklihood of an event occurring. \n",
    "\n",
    "Examples\n",
    "*   Flipping a heads\n",
    "*   Rolling a 6 on a die\n",
    "*   Rolling two 4's and a 6 on three dice\n",
    "*   Running a mile in under 8 minutes\n",
    "\n",
    "We can represent an event using a capital letter. For example, we can say that $A$ is the event where we roll a 4 on a six-sided die.\n",
    "\n",
    "$$ A \\rightarrow \\text{ rolling a 4 } $$\n",
    "\n",
    "We typically represent the probability using the function notation $p()$. \n",
    "\n",
    "The probability of an event is the ratio of the number of *preferred* or *favorable* outcomes to the number of *possible outcomes*.\n",
    "\n",
    "$$ p( A ) = \\frac{ \\text { # favorable outcomes } }{ \\text { # possible outcomes } } $$\n",
    "\n",
    "Though probabilies can be written as percentages or fractions, we usually represent them as a decimal between 0 and 1.\n",
    "\n",
    "The probability of 1 indicates absolute certainty that an event will occur.\n",
    "$$ p( B ) = 1 $$\n",
    "\n",
    "The probability of 0 indicates absolute certainty that an event will *not* occur.\n",
    "$$ p( C ) = 0 $$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Statistical Frameworks\n",
    "\n",
    "We can ask probability questions about many different things.\n",
    "\n",
    "What is the probability that:\n",
    "* You roll a 4 on a fair six-sided die - $P(X=4)$\n",
    "* A given die is a fair die - $P(\\text{fair})$\n",
    "* It will rain tomorrow - $P(\\text{rain})$\n",
    "* A router drops a data packet - $P( \\text{drops packet} )$\n",
    "* A product performs better than another - $P( A > B )$\n",
    "* The universe expands forever - $P( \\text{universe expands} )$\n",
    "\n",
    "There are several frameworks under which we can define probabilities:\n",
    "* Classical\n",
    "* Frequentist\n",
    "* Bayesian\n",
    "\n",
    "### Classical\n",
    "Under this framework, outcomes that are equally likely have equal probabilities. So this framework can be used with questions such as \"What is the probability of rolling a 4 on a six-sided die?\". Or, it could be used to calculate the probability of getting a sum of four on two rolls. \n",
    "\n",
    "This framework works with any situation where you can calculate a ratio of *possible outcomes* to *total equally likely outcomes*. \n",
    "\n",
    "But, this framework doesn't work when you don't have equally likely outcomes, such as for probability questions like \"What is the probability the universe will expand forever?\"\n",
    "\n",
    "### Frequentist\n",
    "This framework requires a hypothetical infinite sequence of trials and then we look at the relevant frequency in that sequence. \n",
    "\n",
    "For example, with the question \"What is the probability of rolling a 4 on a six-sided die?\" we can imagine roll a die over and over and count the ratio of times we get a 4. If we roll it a sufficiently large number of times we can approximate what it would be if we rolled it an infinite number of times.\n",
    "\n",
    "In this model, there is an assumption that the probability we're trying to calculate is fixed.\n",
    "\n",
    "This framework works when we can experimentally do a large number of trials.\n",
    "\n",
    "This framework doesn't work so well when trying to figure out one-off events such as \"What is the probability it will rain tomorrow?\". There is only one tomorrow, so it would be strange to think about an infinite sequence of tomorrows. \n",
    "\n",
    "Similarly, using this approach to get the probability that a die is fair doesn't work so well. The frequentist model requires us to assume the probability is fixed, so we say that the probability is either 0 (not fair) or 1 (fair). That is not particularly useful, because based on some evidence, we'd rather know something like \"there is an 82% probability it is fair\".\n",
    "\n",
    "### Bayesian\n",
    "The Bayesian approach embraces a synthesis of evidence and personal judgment regarding that evidence.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qWHFCJ3GuoAG"
   },
   "source": [
    "## Experiments vs. Trials\n",
    "\n",
    "\n",
    "From [Wikipedia](https://en.wikipedia.org/wiki/Experiment_(probability_theory)):\n",
    "\n",
    "> In probability theory, an experiment or trial is any procedure that can be infinitely repeated and has a well-defined set of possible outcomes, known as the sample space. An experiment is said to be random if it has more than one possible outcome, and deterministic if it has only one. A random experiment that has exactly two (mutually exclusive) possible outcomes is known as a Bernoulli trial.\n",
    "\n",
    "> Random experiments are often conducted repeatedly, so that the collective results may be subjected to statistical analysis. A fixed number of repetitions of the same experiment can be thought of as a composed experiment, in which case the individual repetitions are called trials. For example, if one were to toss the same coin one hundred times and record each result, each toss would be considered a trial within the experiment composed of all hundred tosses.\n",
    "\n",
    "For something like a coin flip, or a roll of a die, we know the ratio of favorable to total outcomes, so we can easily calculate the ***Theoretical Probability*** without having to run an experiment.\n",
    "\n",
    "But, if we we are trying to calculate something where we might not know the ratio of favorable to total outcomes, we might have to run an experiment and calculate an ***Experimental Probability***.\n",
    "\n",
    "For example, we may have an ad on Facebook, and we may be trying to calculate the probability of someone clicking on the ad. For each trial, someone will see the ad and either click or not click on it. \n",
    "\n",
    "$$ p( X ) = \\frac{ \\text { # successful trials } }{ \\text { # total trials } } $$\n",
    "\n",
    "The **Law of Large Numbers** tells us that as the total number of trials increases our experimental probability will approach our theoretical probability. For example, if we run a simulation of a fair coin toss experiment, after several hundred trials, our experimental probability will approach $.5$. This makes sense intuitively because as our total trials increases, each incremental new trial can have much less impact on changing the ratio.\n",
    "\n",
    "I'll discuss this further under the Expected Value section.\n",
    "\n",
    "References:\n",
    "* [Experimental versus theoretical probability simulation - Khan Academy](https://www.khanacademy.org/math/statistics-probability/probability-library/randomness-probability-and-simulation/v/experimental-versus-theoretical-probability-simulation?modal=1)\n",
    "* [Law of Large Numbers - Wikipedia](https://en.wikipedia.org/wiki/Law_of_large_numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sample Spaces\n",
    "\n",
    "The set of all possible outcomes for an event is also called the ***sample space***, so the number of possible outcomes is the size of the sample space.\n",
    "\n",
    "The sample space is often indicated with the symbol $\\Omega$ (capital omega).\n",
    "\n",
    "The sample space for a roll of a six-sided die is:\n",
    "\n",
    "$$\\Omega = \\{ 1, 2, 3, 4, 5, 6 \\} $$\n",
    "\n",
    "So, the probability of rolling a 4 on a six-sided die would be:\n",
    "\n",
    "$$ p( A ) = \\frac{1}{6} = 0.167 $$\n",
    "\n",
    "This is because there is one favorable outcome (rolling a 4) and six possible outcomes (the number of elements in the sample space). \n",
    "\n",
    "A sample space can be compound, meaning that it can depend on multiple factors. For example the outcome of drawing a single card from a standard deck of cards can be considered compound because each card has a suit (e.g. heart or club) and a rank (e.g. 10, jack, queen).\n",
    "\n",
    "When analyzing probabilities for compound sample spaces, it can be useful to write out the possible outcomes using different methods:\n",
    "- A tree diagram\n",
    "- A two-way grid\n",
    "- A table\n",
    "\n",
    "\n",
    "References:\n",
    "- [Compound Sample Spaces - Khan Academy](https://www.khanacademy.org/math/statistics-probability/probability-library/multiplication-rule-independent/v/compound-sample-spaces?modal=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "beO1ZDLSy74z"
   },
   "source": [
    "## Random Variables\n",
    "\n",
    "A Random Variable is the set that represents the sample space (all possible outcomes) for a experiment or trial. \n",
    "\n",
    "Here are some examples:\n",
    "\n",
    "Tossing a coin (where Heads = 0 and Tails = 1):\n",
    "\n",
    "$$ X = \\{ 0, 1 \\} $$\n",
    "\n",
    "$$ p( X = 0 ) = \\frac{1}{2} $$\n",
    "\n",
    "$$ p( X = 1 ) = \\frac{1}{2} $$\n",
    "\n",
    "Rolling a six-sided die:\n",
    "\n",
    "$$ X = \\{ 1, 2, 3, 4, 5, 6 \\} $$\n",
    "\n",
    "$$ p( X = 1 ) = \\frac{1}{6} $$\n",
    "\n",
    "$$ p( X = 2 ) = \\frac{1}{6} $$\n",
    "\n",
    "$$ p( X = 3 ) = \\frac{1}{6} $$\n",
    "\n",
    "$$ p( X = 4 ) = \\frac{1}{6} $$\n",
    "\n",
    "$$ p( X = 5 ) = \\frac{1}{6} $$\n",
    "\n",
    "$$ p( X = 6 ) = \\frac{1}{6} $$\n",
    "\n",
    "Number of heads from three coin tosses:\n",
    "\n",
    "$$ X = \\{ 0, 1, 2, 3 \\} $$\n",
    "\n",
    "$$ p( X = 0 ) = \\frac{1}{8} $$\n",
    "\n",
    "$$ p( X = 1 ) = \\frac{3}{8} $$\n",
    "\n",
    "$$ p( X = 2 ) = \\frac{3}{8} $$\n",
    "\n",
    "$$ p( X = 3 ) = \\frac{1}{8} $$\n",
    "\n",
    "\n",
    "\n",
    "References:\n",
    "*  https://www.mathsisfun.com/data/random-variables.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iro_3VE76ddn"
   },
   "source": [
    "## Probabilities Add to 1\n",
    "\n",
    "Across a sample space, the sum of probabilities must equal one. \n",
    "\n",
    "$$ \\sum_{i} p( X = i ) = 1  $$\n",
    "\n",
    "So, for the number of heads from three coin tosses, we have:\n",
    "\n",
    "$$ \\sum_{i=0}^3 p( X = i ) $$\n",
    "\n",
    "$$ = p( X = 0 ) + p( X = 1 ) + p( X = 2 ) + p( X = 3 ) $$\n",
    "\n",
    "$$ = \\frac{1}{8} + \\frac{3}{8} + \\frac{3}{8} + \\frac{1}{8}$$\n",
    "\n",
    "$$ = 1 $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ak7_JAmR9jhe"
   },
   "source": [
    "## Complements\n",
    "\n",
    "The complement of an event represents the event *not* occurring. Given an event $A$ we can represent the complement as either of the following:\n",
    "\n",
    "$$A^c \\text{ or } A'$$\n",
    "\n",
    "An event and its complement are mutually exclusive. \n",
    "\n",
    "Since probabilities must add to 1, we know that:\n",
    "\n",
    "$$ p( A ) + p( A' ) = 1$$ \n",
    "\n",
    "$$ p( A' ) = 1 - p( A )$$ \n",
    "\n",
    "$$ p( A ) = 1 - p( A' )$$ \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Addition Rule\n",
    "\n",
    "When we're considering the OR of two events, we must use the addition rule.\n",
    "\n",
    "$$P\\big(A \\text{ OR } B\\big) = P(A) + P (B) - P( A \\text{ AND } B )$$\n",
    "\n",
    "The subtraction term removes any double-counted items that are overlapping in set A and B. For example, if you want to find the probability of finding a Jack or a Heart in a deck of cards, it would be:\n",
    "\n",
    "$$P\\big(\\text{Jack} \\text{ OR } \\text{Heart}\\big) = \\frac{4}{52} + \\frac{13}{52} - \\frac{1}{52} = \\frac{16}{52} = \\frac{4}{13}$$\n",
    "\n",
    "We have 4 Jacks, 13 Hearts, and one card that is both a Jack and a Heart. If we don't subtract the overlapping card that is in both sets, it will get double counted in our calculation.\n",
    "\n",
    "If the events have no overlap, they are called *mutually exclusive*. In this case, the subtraction term is 0 and the sum is just $P(A) + P(B)$.\n",
    "\n",
    "References:\n",
    "- [Addition Rule of Probability - Khan Academy](https://www.khanacademy.org/math/statistics-probability/probability-library/addition-rule-lib/v/addition-rule-for-probability)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiplication Rule for Independent Events\n",
    "\n",
    "You can use the multiplication rule to calculate the probability of getting an exact set of outcomes for a series of independent events.\n",
    "\n",
    "For example, what is the probability of flipping a fair coin three times and getting heads three times in a row?\n",
    "\n",
    "$$ P(THT) = \\frac{1}{2} \\cdot \\frac{1}{2} \\cdot \\frac{1}{2} = \\Big(\\frac{1}{2}\\Big)^3 = \\frac{1}{8} = 12.5\\%$$\n",
    "\n",
    "Now, let's say the coin is unfair:\n",
    "$$ P(H) = 0.6 $$\n",
    "$$ P(T) = 0.4 $$\n",
    "\n",
    "Then, we'd have:\n",
    "$$ P(THT) = 0.4 \\cdot 0.6 \\cdot 0.4 = 0.096 = 9.6\\% $$\n",
    "\n",
    "References:\n",
    "- [Multiplication Rule for Independent Events - Khan Academy](https://www.khanacademy.org/math/statistics-probability/probability-library/multiplication-rule-independent/)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3nNuvn4s60ZS"
   },
   "source": [
    "# Discrete and Continuous Distributions\n",
    "\n",
    "A discrete distribution is one where there are a finite set of outcomes. Examples include rolling a die, drawing three aces in a row from a deck of cards, or clicking on a button on a website.\n",
    "\n",
    "A continuous distribution is one where there are an infinite number of possible outcomes (given infinite precision in measurement). Examples including the time it takes to run a mile, or the height to weight ratio of female athletes participating in the Olympics.\n",
    "\n",
    "References:\n",
    "- [Probability: Types of Distributions\n",
    "](https://www.youtube.com/watch?v=b9a27XN_6tg&ab_channel=365DataScience)\n",
    "- [Probability Distribution Functions (PMF, PDF, CDF)](https://www.youtube.com/watch?v=YXLVjCKVP7U&t=375s&ab_channel=zedstatistics)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c4ecue-78RnW"
   },
   "source": [
    "## Discrete Distributions\n",
    "\n",
    "Discrete distributions are represented using a ***Probability Mass Function*** (PMF). \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "id": "tdYnZr6lUr-h",
    "outputId": "d0f8deec-d5a7-49f0-de5c-d6c9bb508abe"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcfklEQVR4nO3de5gkdX3v8feHRUUugsomyC63KCroiYqraNRIvIIKmBOjoKgoQjxPiHqCGKOJUYyK5gRIFDUoeEGjEq+roAbFG15ZxBtwICuiu1zCsgIColz85o+qxaad2eme7emp6Xm/nmceuquqq749Ox++Xb+qrkpVIUlS12w23wVIkjQVG5QkqZNsUJKkTrJBSZI6yQYlSeokG5QkqZNsUB2T5LNJnj/idT4myUWjXKfUJUnumuTTSa5L8h8DLH9+kn1GXMNzkvznKNe52Nmg5lCSS5PclOSGnp8dN/aaqtqvqt43xDZem+SWJNe3PxcneVuSe/Ws82tVdb9NeS8b2f6hSW5r39svknwvydPaefskqSSf6HvNg9rpX+6ZVklu7Pk9XTsX9aqb2n//+/RNe22SDwy4imcAvw/cs6r+fKaFq+oBVfXlIep7b5Kbe3L2oyRvSrJtzzo/WFVPGnSdw+jJ+Q1Jrk3yjSSPbOcd2v7+ju97zYHt9Pe2z3dtn/f+/+j7c1HvqNig5t7+VbV1z8/ls11Rks2nmfWRqtoGuAfwp8AOwLm9TWqOfbOqtga2A04GTkty93beOuCRSe7Zs/zzgYunWM+Den5P281lwZo4uwAXV9Wtm7qijeTsLW3OlgIvAB4BfD3JVpu6zQF9pM3ZUuBs4ONJ0s77MfDMvtqny9l2PTl70NyWvGlsUGOW5O5JPpNkXZJr2sfLe+Z/OcmL2seHJvl6kuOTrAdeu7F1V9UtVXU+8CyaxnBUu559kqzt2caOST7W1vCTJC8ZxXurqt8ApwB3Be7dTr4Z+CRwULvtJW19HxzFNrU4bPgbTnJUkquSXJHkBe281wGvAZ7V7hUcluTeSc5Ksj7J1Uk+mGS7nvVdmuQJ7ePXJvlokg8k+QVw6MZqqapfVdU5wAHAPWma1Ya8nt2zjfsnOTPJz5NclOSZo/hdVNUtwPtoPohu+OB3JfBD4Mnttu8B/BGwchTbnC82qPHbDHgPzSe+nYGbgLdtZPm9gUtohi/eMMgGquo24FPAY/rnJdkM+DTwfWAZ8HjgZUmePPhbmFr76e1FwA3Af/XMej/wvPbxk4EfAbPek9SitQOwLc3f7WHAiUnuXlX/ALyRdg+jqk4GArwJ2BHYA9iJjX/AOxD4KM0owEAfnqrqeuBMps7ZVu28fwd+j+YD2tuT7DnIujcmyV1omuiaqrq6Z1Zvzg6i+X/Arzd1e/PJBjX3PtmOGV+b5JNVtb6qPlZVv2z/wN8APHYjr7+8qt5aVbdW1U1DbPdymiG/fg8DllbVMVV1c1VdAryLdg9nlh7RHjO6EjgY+NOqum7DzKr6BnCPJPejCdD7p1nPd3t+V/+6CfVoMt0CHNOOFJxB80FoymOrVbW6qs6sql9X1TrgODaes29W1Ser6jcjytnTgEur6j1tds8DPgbMeHxsI57Z5mwN8FCa4fxenwD2aY+LbSxnV/fk7OWbUM+cm26sVaPz9Kr6woYnSbYEjgf2BTYcp9kmyZJ2z6ffmlludxnw8ymm7wLs2HcSwhLga/0LJtkZuGDD83b8eyrfqqpHz1DPqcCRwJ8ALwSePcUye1XV6hnWo8l0G3Cnvml3omlKG6zvO8b0S2DKv8kkvw/8C83ezTY0H8av2cj25yJne/flbHOaHPTX+hjgs+3Tn1bVA6bZ1mlVdch0hVTVTUlOB/6O5mSRryfZb4pFtx/FsbpxsEGN31E0n/r2rqorkzwYOI9mSGIqQ19uvh3G2x/4whSz1wA/qardZ1pPVf2Maf4HMAunAquB91fVL397bFcC4GfArsCFPdN2Y+qD/IN4I012/ldV/TzJ09n4UPpscrY18ASmHnpfA3ylqp4403qq6muMLmfvB84CXjei9c0rh/jGbxua407Xtgcy/2FUK06yeZI9gA/RjNcfN8Vi3wGuT/I3ab47siTJA5M8bFR1TKWqfkIzxPLqudyOFqyPAH+XZHmSzdoTGPanOS40G9vQDAFel2QZcPSI6iTJXZI8lObkn2tojin3+wxw3yTPTXKn9udhbT7n0leAJwJvnePtjIUNavxOoDnL7WrgW8DnRrDOZyW5AbiO5qyd9cBDpzqlvR1GfBrwYOAnbR3vpjn4PKeq6uxNOc1eE+0Y4Bs0p09fA7wFeE5V/WiW63sdsBdNJk4HPj6CGl+R5HqafL0fOBf4o6q6sX/B9vjyk2iO7V5Oc3z2zcBdRlDHtKrxxaqaathxwYk3LJQkdZF7UJKkThpLg0pySvvluil319P41ySrk/wgyV7jqEtaaMySFpNx7UG9l+a06unsB+ze/hwBvGMMNUkL0XsxS1okxtKgquqrTP1dgQ0OpDn9uKrqW8B2Gd915KQFwyxpMenK96CWcccvyq1tp13Rv2CSI2g+GbLVVls99P73v/9YCpT6nXvuuVdX1dL5rqOPWdKCM12WutKgBlZVJwEnAaxYsaJWrVo1zxVpsUry0/muYVOYJXXFdFnqyll8l9FczHGD5e00ScMxS5oYXWlQK4HntWcgPQK4rqp+Z0hC0ozMkibGWIb4knwI2AfYPs19if6B9sKQVfVO4AzgKTTXavsl7f1VJN2RWdJiMpYGVVUHzzC/gL8cRy3SQmaWtJh0ZYhPkqQ7sEFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjrJBiVJ6iQblCSpk2xQkqROskFJkjppbA0qyb5JLkqyOskrp5i/c5IvJTkvyQ+SPGVctUkLiVnSYjGWBpVkCXAisB+wJ3Bwkj37Fvs74LSqeghwEPD2cdQmLSRmSYvJuPagHg6srqpLqupm4MPAgX3LFHC39vG2wOVjqk1aSMySFo3Nx7SdZcCanudrgb37lnkt8J9J/grYCnjCeEqTFhSzpEWjSydJHAy8t6qWA08BTk3yO/UlOSLJqiSr1q1bN/YipQXALGkijKtBXQbs1PN8eTut12HAaQBV9U1gC2D7/hVV1UlVtaKqVixdunSOypU6yyxp0RhXgzoH2D3JbknuTHPgdmXfMj8DHg+QZA+aUPmxTrojs6RFYywNqqpuBY4EPg9cSHOG0flJjklyQLvYUcDhSb4PfAg4tKpqHPVJC4VZ0mIyrpMkqKozgDP6pr2m5/EFwKPGVY+0UJklLRZdOklCkqTb2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkdZINSpLUSTYoSVIn2aAkSZ1kg5IkddLADSrJPeeyEGmxMEvSYIbZg/pZkk8leUaSO89ZRdLkM0vSAIZpULsCXwT+BrgyyUlJHj0nVUmTbVfMkjSjgRtUVa2rqn+tqocBjwSuAk5NckmSY5LsMmdVShPELEmDme1JEju0P3cDfgwsA85L8srpXpBk3yQXJVk93XJJnpnkgiTnJ/n3WdYmLSRDZckcaTHZfNAFkzwAOAR4NnAj8D7gQVW1tp3/euAHwLFTvHYJcCLwRGAtcE6SlVV1Qc8yuwN/Czyqqq5J8nuzfldSh802S+ZIi83ADQr4KvAh4M+r6jv9M6vq0iQnTPPahwOrq+oSgCQfBg4ELuhZ5nDgxKq6pl3fVUPUJi0ks82SOdKiMswQ359W1ZH9gUry8A2Pq+o107x2GbCm5/nadlqv+wL3TfL1JN9Ksu9UK0pyRJJVSVatW7duiPKlzphtlkaWo3Z7ZkmdNkyD+sw00z83ikJo9uZ2B/YBDgbelWS7/oWq6qSqWlFVK5YuXTqiTUtjNZdZGihHYJbUfTM2qCSbtWPfaW3W87M7cOsA27kM2Knn+fJ2Wq+1wMqquqWqfgJcTBM0aSKMIEvmSIvKIHtQtwI3A1u2j2/p+bkAePsA6zgH2D3Jbu0XEw8CVvYt80maT30k2Z5mqOKSAdYtLRSbmiVzpEVlkJMkdgMCfAX4457pBayrqptmWkFV3ZrkSODzwBLglKo6P8kxwKqqWtnOe1KSC4DbgKOrav1wb0fqtE3KkjnSYpOqmu8aZm3FihW1atWq+S5Di1SSc6tqxXzXMQpmSfNpuixtdA8qyUlVdUT7+P3TLVdVz9v0EqXJZZak4c00xPeTnsc/nstCpAlnlqQhbbRBVdWbeh6/bu7LkSaTWZKGN9MQ3+MGWUlVnTWacqTJZJak4c00xHfyAOso4A9GUIs0ycySNKSZhvh2G1ch0iQzS9LwZnu7DUmS5tRMx6AurKo92sdraIYgfkdV7TwHtUkTwyxJw5vpGNThPY8PmctCpAlnlqQhzXQM6uyex1+Z+3KkyWSWpOENfAwqyZ2THJPkv5Lc2P739Um2mMsCpUljlqTBDHNH3XcA9wNeAvwU2AV4Fc0N0144+tKkiWWWpAEM06CeDty7qq5tn1+Q5NvAagyVNIynY5akGQ1zmvmVNPex6XVX4IrRlSMtCmZJGsAwlzo6FfhckrfS3LVzJ+AvgWmvzCypYZak4c3mUkev6nv+F8CbR1OONLHMkjQkL3UkjYFZkobnpY4kSZ008Fl8Se4GvBZ4LLA9kA3zvDyLNDizJA1mmD2otwN7AccA9wD+CvgZcPwc1CVNMrMkDWCY70E9CdijqtYnua2qPpVkFfBpDJY0DLMkDWCYPajNgOvaxzck2Zbmexv3GXlV0mQzS9IAhtmD+j7NmPkXga/RDFPcAFw8B3VJk8wsSQMYZg/qcODS9vFLgV8B2wHPG21J0sQzS9IABt6DqqpLeh5fBRw2JxVJE84sSYMZ6ntQSV6Y5Mwk57f/PSxJZn6lpF5mSZrZMN+DegtwIHACv71FwMtpbhvwirkoTppEZkkazDAnSRwK7FVVazdMSPIZ4LsYKmkYh2KWpBkNM8R3ffvTP+0XoytHWhTMkjSAmW638Qc9T08APp7kWH57i4Cj8YuF0ozMkjS8mYb4VgNFz7XCgD/pW+ZxwNtGWZQ0gcySNKSZbrfh1c6lETBL0vCGOUkCgCQ7A8uAtVW1ZvQlSYuDWZI2buBPdUnuleQrNEMVHwd+nOSrSXacs+qkCWSWpMEMM+zwDppriN29qu4F3B04D3jnIC9Osm+Si5KsTvLKjSz3Z0kqyYohapMWErMkDWCYIb5HA/eqqlsAqurGJK8ALpvphUmWACcCT6Q5a+mcJCur6oK+5bahuTbZt4eoS1pozJI0gGH2oK4B9uybdj/g2gFe+3BgdVVdUlU3Ax+m+SZ9v9cDb6a5eKY0qcySNIBhGtRbgC8kOTbJ/2m/w3FmO30my4Deg8Br22m3S7IXsFNVnb6xFSU5IsmqJKvWrVs3RPlSZ5glaQADN6iqehfwLGB7YP/2v8+uqpM2tYgkmwHHAUcNUMdJVbWiqlYsXbp0UzctjZ1ZkgYz0DGodtz7YmDPqjprFtu5jObb8hss547j7dsADwS+3F7QeQdgZZIDqmrVLLYndZJZkgY30B5UVd0G3AZsMcvtnAPsnmS3JHcGDgJW9qz/uqravqp2rapdgW8BBkoTxyxJgxvmGNQJwGlJHpvk3kn+YMPPTC+sqluBI4HPAxcCp1XV+UmOSXLArCqXFq4TMEvSjFJVgy2Y/GaaWVVVS0ZX0uBWrFhRq1b5wVDzI8m5VTX0d4zMknRH02Vpxj2oJFsmeSNwOs2pq1tW1WY9P/MSKGmhMUvScAYZ4juR5kyjC4E/A/5pTiuSJpdZkoYwSIPaF3hSVb0C2A942tyWJE0ssyQNYZAGtVVVXQHQXnF527ktSZpYZkkawiDfg9o8yZ/w2xut9T9nlt/nkBYbsyQNYZAGdRVwSs/z9X3PC5jx9FhJZkkaxowNqv2yn6RNZJak4XgbaklSJ9mgJEmdZIOSJHWSDUqS1Ek2KElSJ9mgJEmdZIOSJHWSDUqS1Ek2KElSJ9mgJEmdZIOSJHWSDUqS1Ek2KElSJ9mgJEmdZIOSJHWSDUqS1Ek2KElSJ9mgJEmdZIOSJHWSDUqS1Embz3cBkrpt11eePtL1XXrsU+dlG5NiMf17uAclSeokG5QkqZNsUJKkTrJBSZI6yQYlSeokG5QkqZPG1qCS7JvkoiSrk7xyivl/neSCJD9I8sUku4yrNmmhMEdaTMbSoJIsAU4E9gP2BA5OsmffYucBK6rqD4GPAm8ZR23SQmGOtNiMaw/q4cDqqrqkqm4GPgwc2LtAVX2pqn7ZPv0WsHxMtUkLhTnSojKuBrUMWNPzfG07bTqHAZ+d04qkhcccaVHp3KWOkhwCrAAeO838I4AjAHbeeecxViYtHDPlqF3GLKnTxrUHdRmwU8/z5e20O0jyBODVwAFV9eupVlRVJ1XViqpasXTp0jkpVuqokeUIzJK6b1wN6hxg9yS7JbkzcBCwsneBJA8B/o0mVFeNqS5pITFHWlTG0qCq6lbgSODzwIXAaVV1fpJjkhzQLvZPwNbAfyT5XpKV06xOWpTMkRabsR2DqqozgDP6pr2m5/ETxlWLtFCZIy0mXklCktRJNihJUid17jTzUZmUu06Oehvj2s4kb0PSeLgHJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOskGJUnqJBuUJKmTbFCSpE6yQUmSOmlsDSrJvkkuSrI6ySunmH+XJB9p5387ya7jqk1aSMySFouxNKgkS4ATgf2APYGDk+zZt9hhwDVVdR/geODN46hNWkjMkhaTce1BPRxYXVWXVNXNwIeBA/uWORB4X/v4o8Djk2RM9UkLhVnSopGqmvuNJM8A9q2qF7XPnwvsXVVH9izzo3aZte3zH7fLXN23riOAI9qn9wMumvM3sOm2B66ecamFYVLeyyjexy5VtXQUxQzKLPn31zGjeh9TZmnzEax4rKrqJOCk+a5jGElWVdWK+a5jFCblvUzK+9gUZmn++D4GM64hvsuAnXqeL2+nTblMks2BbYH1Y6lOWjjMkhaNcTWoc4Ddk+yW5M7AQcDKvmVWAs9vHz8DOKvGMf4oLSxmSYvGWIb4qurWJEcCnweWAKdU1flJjgFWVdVK4GTg1CSrgZ/TBG9SLKhhlBlMyntZkO/DLC3Mf7cp+D4GMJaTJCRJGpZXkpAkdZINSpLUSTaoOZRkpyRfSnJBkvOTvHS+a9oUSZYkOS/JZ+a7ltlKsl2Sjyb5/0kuTPLI+a5JM5ukLE1CjmA8WVpw34NaYG4Fjqqq7ybZBjg3yZlVdcF8FzZLLwUuBO4234Vsgn8BPldVz2jPgttyvgvSQCYpS5OQIxhDltyDmkNVdUVVfbd9fD3NH+Wy+a1qdpIsB54KvHu+a5mtJNsCf0xzlhtVdXNVXTuvRWkgk5KlScgRjC9LNqgxaa8o/RDg2/NcymydALwC+M0817EpdgPWAe9ph1jenWSr+S5Kw1ngWTqBhZ8jGFOWbFBjkGRr4GPAy6rqF/Ndz7CSPA24qqrOne9aNtHmwF7AO6rqIcCNwO/crkLdtZCzNEE5gjFlyQY1x5LciSZQH6yqj893PbP0KOCAJJfSXD37cUk+ML8lzcpaYG1Vbfjk/VGakGkBmIAsTUqOYExZskHNofYWBycDF1bVcfNdz2xV1d9W1fKq2pXmqgRnVdUh81zW0KrqSmBNkvu1kx4PLMSD7IvOJGRpUnIE48uSZ/HNrUcBzwV+mOR77bRXVdUZ81fSovdXwAfbs44uAV4wz/VoMGape+Y8S17qSJLUSQ7xSZI6yQYlSeokG5QkqZNsUJKkTrJBSZI6yQY1B5K8M8nfz8N2K8l92sfvTfKP465BGiWztLjZoIaU5NIkNyW5Psm1Sb6R5MVJbv9dVtWLq+r1s1x/JbkxyQ1JLktyXJIlo3sHt29neZIPJlnfbu877aVYBn39oUnOHnVdWjzM0u2vN0vTsEHNzv5VtQ2wC3As8De0V/UdkQdV1dbAY4FnAS8c4bpJcg/gbOBm4AHA9sDxwL8necYotyXNwCxpWjaoTVBV11XVSpo//OcneSD87pBAkqcl+V7Pp8Q/HHD9q4GvAw/uWdfhSVYn+XmSlUl2nEXp/xe4ATisqq6sqpuq6kPAG4B/TmPX9hPo7VcbSfLlJC9KsgfwTuCR7afTa9v5d03yz0l+muS6JGcnuWs774A0N5q7tl3PHj3rvTTJ0Ul+0H4CPTnJ7yf5bPvp+gtJ7t6z/CPa3+O1Sb6fZJ9Z/A7UIWbJLE3FBjUCVfUdmosnPqZ/XpKHAKcAfwHcE/g3YGWSu8y03iT3b9e5un3+OOBNwDOBewE/pbno5LCeCHysqvov+X8asDNw3429uKouBF4MfLOqtq6q7dpZ/w94KPBHwD1obyuQ5L7Ah4CXAUuBM4BPp7lEygZ/1tZ1X2B/4LPAq9rlNwNeApBkGXA68I/tNl4OfCzJ0qF+A+oks2SWetmgRudymn/kfkcA/1ZV366q26rqfcCvgUdsZF3fTXIjzU3Zvgy8vZ3+HOCUqvpuVf0a+FuaT167Dlnr9sAVU0y/omf+UNIcN3gh8NKquqx9r99o63wWcHpVnVlVt9CE76404dvgrVX131V1GfA14NtVdV5V/Qr4BM39fwAOAc6oqjOq6jdVdSawCnjKsDWrs8ySWQJsUKO0DPj5FNN3AY5qd6GvbXfhdwI2NpywF7A1zR/j3sCGG4HtSPNJD4CqugFYz/B3Fr2a5lNjv3v1zB/W9sAWwI+nmNdf92+ANdyx7v/ueXzTFM+3bh/vAvx53+/z0Uz9frQwmSWzBNigRiLJw2j+QKY6E2cN8Iaq2q7nZ8t2nHpa1TgN+Cbwmnby5TR/VBu2uxXNUMdlQ5b8BeB/p+dsqdYz23ovprkBGcCWPfN36C2x77VXA78C7j3F9vrrDs3/WIatm7a+U/t+n1tV1bGzWJc6xiwBZul2NqhNkORuaU4n/TDwgar64RSLvQt4cZK92wOmWyV5apJtBtzMscDhSXagGXt+QZIHt+Pub6TZfb90yNKPB7YFTk6yQ5ItkhwMvBo4ug30Opo/+kOSLEnyQu4YmP8Glm8Y+24/yZ0CHJdkx/Y1j2zrPA14apLHp7np3FE0QzPfGLJugA8A+yd5cruNLZLsk2T5LNaljjBLZmkqNqjZ+XSS62k+gbwaOI5p7oVSVauAw4G3AdfQHKQ9dNANtUH9Ks0f+xeAv6e5q+gVNH/kBw1bfFWtp9mV34LmJmPrgb8GnltVH+lZ9HDg6Hb+A7hjCM4CzgeuTLJhGOPlwA+Bc2iGaN4MbFZVF9GMd7+V5tPh/jSnF988i9rXAAfSHPRdR/NvcDT+LS9UZsksTcv7QUmSOqkznVKSpF42KElSJ9mgJEmdZIOSJHWSDUqS1Ek2KElSJ9mgJEmdZIOSJHXS/wC6Ev6TT2l9SgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# https://www.kdnuggets.com/2020/02/probability-distributions-data-science.html\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "face = [1,2,3,4,5,6]\n",
    "\n",
    "probs_fair = np.full((6), 1/6)\n",
    "plt.subplot(1,2,1)\n",
    "plt.bar(face, probs_fair)\n",
    "plt.ylabel('Probability', fontsize=12)\n",
    "plt.xlabel('Die Roll Outcome', fontsize=12)\n",
    "plt.title('Fair Die - PMF', fontsize=12)\n",
    "axes = plt.gca()\n",
    "axes.set_ylim([0,1])\n",
    "\n",
    "probs_unfair = [1/4, 1/4, 0, 0, 1/4, 1/4]\n",
    "plt.subplot(1,2,2)\n",
    "plt.bar(face, probs_unfair)\n",
    "plt.ylabel('Probability', fontsize=12)\n",
    "plt.xlabel('Die Roll Outcome', fontsize=12)\n",
    "plt.title('Unfair Die - PMF', fontsize=12)\n",
    "axes = plt.gca()\n",
    "axes.set_ylim([0,1])\n",
    "\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kVZ9jouIWLkW"
   },
   "source": [
    "Let's say we want to find the probability of rolling a 4 or less:\n",
    "\n",
    "$$ p( X \\leq 4 ) $$\n",
    "\n",
    "We can use the ***Cumulative Distribution Function*** (CDF) of the discrete distribution to figure that out. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "id": "wJaKmGp8U3Bn",
    "outputId": "9e272de9-1a04-4cea-a943-f4232dbe9561"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeVElEQVR4nO3dfdxlc73/8dfbSG5mGMycMIOR3HaviXRHRcdNqF8KpVLi1OPo5pBSnSOmU6FzUNGNIpJIdDRluiGplGQQNePwG2M0M8gYRkgxep8/1rq0bdfN3nv23tfae7+fj8d+zNrr5rs+65r92Z+1vmvttWSbiIiIqlltvAOIiIgYTgpURERUUgpURERUUgpURERUUgpURERUUgpURERUUgpUxUj6oaR3tLnNV0i6pZ1tRlSJpLUkfV/SA5K+08D88yTt2uYY3irpJ+1sc9ClQHWQpEWSHpH0UM1rk9GWsb2n7XOaWMdxkh6T9GD5ulXSaZI2rmnzl7a3WZVtGSOGjSWdKemuMob/lXS8pHXK6Zb0cLn9yyX9VNIBdW1cKemvdX+rnTsVc1RL+Rl5Vt244yR9s8Em9geeAWxo+01jzWz72bavbCK+syU9WpNnf5D0GUnr1bR5nu3XNtpmsyRtLek7ku4tC/FNko6UNEHSjPJvOJQ7f5L0A0m717XR9HfSeEqB6rx9bE+sed3ZakOSVh9h0rdtTwI2AN4AbARcV1ukOkXSBsDVwFrAzmUcuwOTgS1rZn2+7YnANsDZwGmSPlHX3BF1f6urOx1/9I3NgVttr1zVhkbJs5PKz/dU4J3AS4BfDe2IdZKkLYFrgMXAc22vB7wJmAlMqpl1cplnzwcuA/5H0iF1zbXtO6nTUqC6TNL65Z7NMkn3l8PTa6ZfKend5fAhkn4l6RRJy4HjRmvb9mO25wEHAMuAo8p2dpW0pGYdm0i6uIzhdknvX4VNOhJ4EDjY9qIyjsW2P2D7pmFivNf2ucB7gY9K2nAV1h0DYugzLOkoSfeUR+vvLKcdDxwLHFAeERwqaUtJV5RH7PdKOk/S5Jr2FknarRw+TtJFkr4p6c/AIaPFYvuvtq8F9gU2pChWQ/l6Vc06tpV0maT7JN0i6c2r8Cc4Hvi17SNt31XGcYvtt9heMUyMd9v+HMV3xomSevK7vieD7nGrAV+n2OPbDHgEOG2U+XcCFlJ0X3yqkRXYfhz4HvCK+mnlB/X7wI3ANOA1wAcl/XPjm/AkuwHftf33Jpf7HrA6sGOL643BsxGwHsXn9lDgdEnr2/4E8GmKnoSJts8EBHwG2ATYDtiU0Xfw9gMuojjyP6+RYGw/SHGUMlyerVNO+xbwT8CBwBclbd9I28PYrYyvWd8t19+xLv5OSoHqvEskrShfl9hebvti238pP+CfAnYZZfk7bX/B9krbjzSx3jspuvzqvRiYanuW7UdtLwS+SpFArdgQuKvZhWw/BtxbF+Pna/5W17cYT/Svx4BZZU/BHOAhRvjitb3A9mW2/2Z7GXAyo+fZ1bYvsf33NuXZ64BFtr9e5u4NwMUU3XKtaCnPyvioi/FJ30ktxtMVI/W1Rvu83vblQ28krQ2cAuwBrF+OniRpQnnkU29xi+udBtw3zPjNgU0kragZNwH4Zf2MkjYD5g+9L/u26y0Hmj7XJelpFH35tTG+3/bXmm0r+sLjwNPqxj2NoigNWV53jukvwHCfSSQ9A/gcxdHNJIqd8ftHWX8n8mynujxbHTh3mFhfAfywfHuH7WcP015LeVbGR12MT/pOqrIcQXXfURR7fTvZXhd4ZTleI8zf9O3my268fRim6FAk4u22J9e8Jtne6ykrtv9YezJ1hNVdDryhhT7u/YCVwG+bXC760x+BGXXjtgDuaLG9T1PkznPLPDuYkXMMWsuziRRdbyPl2c/r8myi7fc+ZcXFVbZDeTZccYIiz97YbIwUF03dA/Tkz0xSoLpvEsV5pxXlFXD1V7K1TNLqkrYDzqforz95mNl+Czwo6SMqfjsyQdJzJL24xdWeDKwLnCNp8zKOaZJOlvS8YWLcQNJbgdOBE20vb3G90V++Dfy7pOmSVisvYNiH1s67QJFnDwEPSJoGHN2mOJH0dEkvAi6hOCr7+jCz/QDYWtLbJD2tfL24zM9WfAJ4qaTPStqojONZ5YUdk4eJ8RmSjiiX+2gL54grIQWq+06luCT7XuA3wI/a0OYBkh4CHgBmU3QHvGi4y0fLbsTXAS8Abi/j+BrFyeem2b4PeClFV8w1kh4EflrGsqBm1hvLGBcA7wb+zfaxrawz+tIs4NfAVRRf+icBb7X9hxbbOx7YgeJzeCnFxQKr6sPl53s58A3gOuClth+un7E8v/xainO7dwJ3AycCT29lxbZvA3amOMqcJ+kBinNacymuoh2yQtLDwO+BvYA32T6rlXVWgfLAwoiIqKIcQUVERCV1pUBJOqv8cd2wh+sqfF7SAhW379ihG3FF9JrkUgySbh1BnU1xWfVI9gS2Kl+HA1/qQkwRvehskksxILpSoGz/guF/KzBkP+AbLvwGmKwu3Ecuotckl2KQVOWHutN48g/llpTjnvLLaUmHU+wZss4667xo22237UqA0Rt+v/SBtrb33GkjX9x43XXX3Wt7altXuOqSS7HKuplHMHIuVaVANcz2GcAZADNnzvTcuXPHOaKokhnHXNrW9uaesPeI0yS1+iPSSkguxUi6mUcwci5V5Sq+pRQ3cxwyvRwXEc1JLkXfqEqBmg28vbwC6SXAA0O3lI+IpiSXom90pYtP0vnArsAUFc8l+gTljSFtfxmYQ/Gr5wUUN4B8Zzfiiug1yaUYJF0pULYPGmO6gX/tRiwRvSy5FIOkKl18ERERT5ICFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlZQCFRERlbT6eAcQg2PGMZe2tb1FJ+zd1vYiolpyBBUREZWUAhUREZWUAhUREZXUtQIlaQ9Jt0haIOmYYaZvJulnkm6QdJOkvboVW0QvSS7FoOhKgZI0ATgd2BPYHjhI0vZ1s/07cKHtFwIHAl/sRmwRvSS5FIOkW0dQOwILbC+0/ShwAbBf3TwG1i2H1wPu7FJsEb0kuRQDo1uXmU8DFte8XwLsVDfPccBPJL0PWAfYrTuhRfSU5FIMjCpdJHEQcLbt6cBewLmSnhKfpMMlzZU0d9myZV0PMqIHJJeiL3SrQC0FNq15P70cV+tQ4EIA21cDawJT6huyfYbtmbZnTp06tUPhRlRWcikGRrcK1LXAVpK2kLQGxYnb2XXz/BF4DYCk7SiSKrt1EU+WXIqB0ZUCZXslcATwY+BmiiuM5kmaJWnfcrajgMMk3QicDxxi292IL6JXJJdikHTtXny25wBz6sYdWzM8H3hZt+KJ6FXJpRgUVbpIIiIi4gkpUBERUUkpUBERUUkpUBERUUkpUBERUUkpUBERUUl55HtERA+ZccylbW1v0Ql7t7W9dsoRVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFIKVEREVFKeqBvAYD2lMyJ6Q46gIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKikhouUJI27GQgEYMiuRTRmGaOoP4o6XuS9pe0Rsciiuh/yaWIBjRToGYAPwU+Atwt6QxJL+9IVBH9bQbJpYgxNVygbC+z/XnbLwZ2Bu4BzpW0UNIsSZt3LMqIPpJcimhMqxdJbFS+1gVuA6YBN0g6ZqQFJO0h6RZJC0aaT9KbJc2XNE/St1qMLaKXNJVLyaMYJA3fi0/Ss4GDgbcADwPnAM+3vaSc/kngJuCEYZadAJwO7A4sAa6VNNv2/Jp5tgI+CrzM9v2S/qnlrYqosFZzKXkUg6aZm8X+AjgfeJPt39ZPtL1I0qkjLLsjsMD2QgBJFwD7AfNr5jkMON32/WV79zQRW0QvaTWXkkcxUJrp4nuD7SPqE0rSjkPDto8dYdlpwOKa90vKcbW2BraW9CtJv5G0x3ANSTpc0lxJc5ctW9ZE+BGV0WoutS2PyvUll6LSmilQPxhh/I/aEQjF0dxWwK7AQcBXJU2un8n2GbZn2p45derUNq06oqs6mUsN5REkl6L6xixQklYr+75VWq3mtRWwsoH1LAU2rXk/vRxXawkw2/Zjtm8HbqVItIi+0IZcSh7FQGnkCGol8Ciwdjn8WM1rPvDFBtq4FthK0hblDxMPBGbXzXMJxV4fkqZQdFUsbKDtiF6xqrmUPIqB0shFElsAAn4OvLJmvIFlth8ZqwHbKyUdAfwYmACcZXuepFnAXNuzy2mvlTQfeBw42vby5jYnotJWKZeSRzFoxixQtu8oB1fpx4O25wBz6sYdWzNs4MjyFdF32pFLyaMYJKMWKEln2D68HP7GSPPZfnu7A4voJ8mliOaNdQR1e83wbZ0MJKLPJZcimjRqgbL9mZrh4zsfTkR/Si5FNG+sLr5XN9KI7SvaE05Ef0ouRTRvrC6+Mxtow8Az2xBLRD9LLkU0aawuvi26FUhEP0suRTSv1cdtREREdNRY56Butr1dObyYogviKWxv1oHYIvpGcimieWOdgzqsZvjgTgYS0eeSSxFNGusc1FU1wz/vfDgR/Sm5FNG8hs9BSVpD0ixJ/1/Sw+W/n5S0ZicDjOg3yaWIxjTzRN0vAdsA7wfuoLif2McoHpj2rvaHFtG3kksRDWimQL0e2NL2ivL9fEnXAAtIUkU04/UklyLG1Mxl5ndTPMem1lrAXe0LJ2IgJJciGtDMrY7OBX4k6QsUT+3cFPhXYMQ7M0dEIbkU0bxWbnX0sbr3/wKc2J5wIvpWcimiSbnVUcXNOObStre56IS9295mjC65NLp2f86H+4z3yzoGSW51FBERldTwVXyS1gWOA3YBpgAampbbs0Q0LrkU0ZhmjqC+COwAzAI2AN4H/BE4pQNxRfSz5FJEA5r5HdRrge1sL5f0uO3vSZoLfJ8kVkQzkksRDWjmCGo14IFy+CFJ61H8buNZbY8qor8llyIa0MwR1I0UfeY/BX5J0U3xEHBrB+KK6GfJpYgGNHMEdRiwqBz+APBXYDLw9vaGFNH3kksRDWj4CMr2wprhe4BDOxJRRJ9LLkU0pqnfQUl6l6TLJM0r/z1UksZeMiJqJZcixtbM76BOAvYDTuUfjwj4EMVjAz7cieAi+lFyKaIxzVwkcQiwg+0lQyMk/QC4niRVRDMOIbkUMaZmuvgeLF/14/7cvnAiBkJyKaIBYz1u45k1b08FvivpBP7xiICjyQ8LI8aUXIpo3lhdfAsAU3OvMOBVdfO8GjitnUFF9KHkUkSTxnrcRu52HtEGyaWI5jVzkQQAkjYDpgFLbC9uf0gRgyG5FDG6hvfqJG0s6ecUXRXfBW6T9AtJm3Qsuog+lFyKaEwz3Q5foriH2Pq2NwbWB24AvtzIwpL2kHSLpAWSjhllvjdKsqSZTcQW0UuSSxENaKaL7+XAxrYfA7D9sKQPA0vHWlDSBOB0YHeKq5aulTTb9vy6+SZR3Jvsmibiiug1yaWIBjRzBHU/sH3duG2AFQ0suyOwwPZC248CF1D8kr7eJ4ETKW6eGdGvkksRDWimQJ0EXC7pBEnvLX/DcVk5fizTgNqTwEvKcU+QtAOwqe1LR2tI0uGS5kqau2zZsibCj6iM5FJEAxouULa/ChwATAH2Kf99i+0zVjUISasBJwNHNRDHGbZn2p45derUVV11RNcllyIa09A5qLLf+1Zge9tXtLCepRS/lh8ynSf3t08CngNcWd7QeSNgtqR9bc9tYX0RlZRcimhcQ0dQth8HHgfWbHE91wJbSdpC0hrAgcDsmvYfsD3F9gzbM4DfAEmo6DvJpYjGNXMO6lTgQkm7SNpS0jOHXmMtaHslcATwY+Bm4ELb8yTNkrRvS5FH9K5TSS5FjKmZy8yH7hG2e914AxPGWtj2HGBO3bhjR5h31ybiiug1yaWIBox5BCVpbUmfBi6luHR1bdur1bzGTKiISC5FNKuRLr7TKa40uhl4I/DZjkYU0b+SSxFNaKRA7QG81vaHgT2B13U2pIi+lVyKaEIjBWod23cBlHdcXq+zIUX0reRSRBMauUhidUmv4h8PWqt/T4u/54gYNMmliCY0UqDuAc6qeb+87r2BMS+PjYjkUkQzxixQ5Y/9ImIVJZcimpPHUEdERCWlQEVERCU1cyeJqDPjmFGfZtC0RSfs3db2IiJ6WY6gIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKiklKgIiKikvJE3YgYVZ4cHeMlR1AREVFJKVAREVFJKVAREVFJKVAREVFJKVAREVFJKVAREVFJXStQkvaQdIukBZKOGWb6kZLmS7pJ0k8lbd6t2CJ6RfIoBklXCpSkCcDpwJ7A9sBBkravm+0GYKbt5wEXASd1I7aIXpE8ikHTrSOoHYEFthfafhS4ANivdgbbP7P9l/Ltb4DpXYotolckj2KgdKtATQMW17xfUo4byaHADzsaUUTvSR7FQKncrY4kHQzMBHYZYfrhwOEAm222WRcji+gdY+VROU9yKSqtW0dQS4FNa95PL8c9iaTdgI8D+9r+23AN2T7D9kzbM6dOndqRYCMqqm15BMmlqL5uFahrga0kbSFpDeBAYHbtDJJeCHyFIqnu6VJcEb0keRQDpSsFyvZK4Ajgx8DNwIW250maJWnfcrbPAhOB70j6naTZIzQXMZCSRzFounYOyvYcYE7duGNrhnfrViwRvSp5FIMkd5KIiIhKSoGKiIhKqtxl5u2Sp4BGRPS2HEFFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlpUBFREQlda1ASdpD0i2SFkg6ZpjpT5f07XL6NZJmdCu2iF6SXIpB0ZUCJWkCcDqwJ7A9cJCk7etmOxS43/azgFOAE7sRW0QvSS7FIOnWEdSOwALbC20/ClwA7Fc3z37AOeXwRcBrJKlL8UX0iuRSDAzZ7vxKpP2BPWy/u3z/NmAn20fUzPOHcp4l5fvbynnurWvrcODw8u02wC0d34BVNwW4d8y5ekO/bEs7tmNz21PbEUyjkkv5/FVMu7Zj2FxavQ0Nd5XtM4AzxjuOZkiaa3vmeMfRDv2yLf2yHasiuTR+sh2N6VYX31Jg05r308txw84jaXVgPWB5V6KL6B3JpRgY3SpQ1wJbSdpC0hrAgcDsunlmA+8oh/cHrnA3+h8jektyKQZGV7r4bK+UdATwY2ACcJbteZJmAXNtzwbOBM6VtAC4jyLx+kVPdaOMoV+2pSe3I7nUm/9vw8h2NKArF0lEREQ0K3eSiIiISkqBioiISkqB6iBJm0r6maT5kuZJ+sB4x7QqJE2QdIOkH4x3LK2SNFnSRZL+V9LNknYe75hibP2US/2QR9CdXOq530H1mJXAUbavlzQJuE7SZbbnj3dgLfoAcDOw7ngHsgo+B/zI9v7lVXBrj3dA0ZB+yqV+yCPoQi7lCKqDbN9l+/py+EGKD+W08Y2qNZKmA3sDXxvvWFolaT3glRRXuWH7UdsrxjWoaEi/5FI/5BF0L5dSoLqkvKP0C4FrxjmUVp0KfBj4+zjHsSq2AJYBXy+7WL4maZ3xDiqa0+O5dCq9n0fQpVxKgeoCSROBi4EP2v7zeMfTLEmvA+6xfd14x7KKVgd2AL5k+4XAw8BTHlcR1dXLudRHeQRdyqUUqA6T9DSKhDrP9nfHO54WvQzYV9Iiirtnv1rSN8c3pJYsAZbYHtrzvogiyaIH9EEu9UseQZdyKQWqg8pHHJwJ3Gz75PGOp1W2P2p7uu0ZFHcluML2weMcVtNs3w0slrRNOeo1QC+eZB84/ZBL/ZJH0L1cylV8nfUy4G3A7yX9rhz3Mdtzxi+kgfc+4LzyqqOFwDvHOZ5oTHKpejqeS7nVUUREVFK6+CIiopJSoCIiopJSoCIiopJSoCIiopJSoCIiopJSoOpI+rKk/xiH9VrSs8rhsyX9Z7djqCJJn5H0wQ6v42JJe3ZyHdGa5ONgG6gCJWmRpEckPShphaRfS3qPpCf+DrbfY/uTLbZvSQ9LekjSUkknS5rQvi14Yj3TJZ0naXm5vt+Wt1FpdPlDJF3V7rjaTdJU4O3AVzq8qhOBfAF1WfLxieV7Ih/Hw0AVqNI+ticBmwMnAB+hvCNvmzzf9kRgF+AA4F1tbBtJGwBXAY8CzwamAKcA35K0fzvXVQGHAHNsP9LJldj+LbCupJmdXE8MK/kYIxrEAgWA7Qdsz6b40L5D0nPgqYfzkl4n6Xc1e3jPa7D9BcCvgBfUtHWYpAWS7pM0W9ImLYT+b8BDwKG277b9iO3zgU8B/63CjHLv8Yk7hUi6UtK7JW0HfBnYudyzXFFOX0vSf0u6Q9IDkq6StFY5bV8VD4lbUbazXU27iyQdLemmcu/xTEnPkPTDcs/4cknr18z/kvLvuELSjZJ2HWVb9wR+PtofQ9I2kj4o6SeShr1tzNAeqqT/knS/pNuH6dK7kuIxCDEOko89kY9dN7AFaki597wEeEX9NEkvBM4C/gXYkKKrabakp4/VrqRtyzYXlO9fDXwGeDOwMXAHxQ0jm7U7cLHt+tv1XwhsBmw92sK2bwbeA1xte6LtyeWk/wJeBLwU2IDykQCStgbOBz4ITAXmAN9XcXuTIW8s49oa2Af4IfCxcv7VgPcDSJoGXErRnbYB8CHgYhVdecN5LnBL7QhJa0vaW9JpkhYCPwG2BU4DRruB6E5lW1OAk4AzJalm+s3A80dZProg+VjpfOy6gS9QpTsp/oPqHQ58xfY1th+3fQ7wN+Alo7R1vaSHKb7wrgS+WI5/K3CW7ett/w34KMVe04wmY50C3DXM+LtqpjdFRZ//u4AP2F5abuuvyzgPAC61fZntxygSZy2KxBnyBdt/sr0U+CVwje0bbP8V+B+KZ/cAHEzRZTfH9t9tXwbMBfYaIbTJwIM1cU4DlgOXlKP2tb15eZ5itu2/jLKZd9j+qu3HgXMovpSeUTP9wXJ9Mf6Sj9XMx65LgSpMA+4bZvzmwFHl4e+K8vB7U2C0roAdgIkUH6SdgKGHeG1CsZcGgO2HKL5sm30q6L0UX671Nq6Z3qwpwJrAbcNMq4/778Binhz3n2qGHxnm/cRyeHPgTXV/z5cz/PYA3A9Mqnl/D/AO4DyKvcQLJH1W0qtUPIphNHfXbMNQIZtYM30SsGKMNqI7ko/VzMeuG/gCJenFFP+5w11Fsxj4lO3JNa+1yz7mEblwIXA1cGw5+k6KD8TQeteh6KZY2mTIlwP/TzVXOpXeXMZ7K8XDwwDWrpm+UW2IdcveC/wV2HKY9dXHLYovhWbjpozv3Lq/5zq2Txhh/puo6SKx/ZjtC20fQpGo7wAeoOiiuEfSPi3ENGQ74MZVWD7aIPkIVDcfu25gC5SkdVVcCnoB8E3bvx9mtq8C75G0U3myc53y/MekYeYdzgnAYZI2oug3fqekF5R95p+mOPRe1GTopwDrUZxD2UjSmpIOAj4OHF0m4zKKD+zBkiZIehdP/rD/CZg+1G9d7oWdBZwsaZNymZ3LOC8E9pb0mvIo5SiKbpVfNxk3wDeBfST9c7mONSXtKmn6CPPPobj66inK7bzO9n/aflm5fXNbiGnILhR99TEOko89kY9dN4gF6vuSHqTYe/g4cDIjPMfE9lzgMIoT8PdTnGA9pNEVlUn2C4oP6uXAf1A8EfQuig/ogc0Gb3s5xWH4mhQPCFsOHAm8zfa3a2Y9DDi6nP5snvwBvgKYB9wtaagL4kPA74FrKbpXTgRWs30LRV/1Fyj27PahuDT40RZiXwzsR3HCdhnF/8HRjPw5/Aawl/5x9dLQ1VBPeZXb2dJVeOVe+0PlCfroruRj7+Rj1+V5UFFpkj4N3GP71A6u42LgTOfhdxGVkgIVERGVVJlDuYiIiFopUBERUUkpUBERUUkpUBERUUkpUBERUUkpUBERUUkpUBERUUn/B6gb5yUr8WtMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "face = [ 1,2,3,4,5,6 ]\n",
    "\n",
    "plt.subplot(1,2,1)\n",
    "probs = [ 1/6, 2/6, 3/6, 4/6, 5/6, 1 ]\n",
    "plt.bar( face, probs )\n",
    "plt.ylabel( 'Probability', fontsize=12 )\n",
    "plt.xlabel( 'Die Roll Outcome (≤ n)', fontsize=12 )\n",
    "plt.title( 'Fair Die - CDF', fontsize=12 )\n",
    "axes = plt.gca()\n",
    "axes.set_ylim( [0,1] )\n",
    "\n",
    "probs_unfair = [1/4, 1/2, 1/2, 1/2, 3/4, 1]\n",
    "plt.subplot(1,2,2)\n",
    "plt.bar(face, probs_unfair)\n",
    "plt.ylabel('Probability', fontsize=12)\n",
    "plt.xlabel('Die Roll Outcome', fontsize=12)\n",
    "plt.title('Unfair Die - CDF', fontsize=12)\n",
    "axes = plt.gca()\n",
    "axes.set_ylim([0,1])\n",
    "\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NdB74osfWxX2"
   },
   "source": [
    "The CDF for a discrete distribution shows us to cumulative probability for every discrete value less than and including the value we are interested in."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q2CtaJTg8TQ_"
   },
   "source": [
    "## Continuous Distributions\n",
    "\n",
    "Continuous distributions are represented using a ***Probability Density Function*** (PDF). \n",
    "\n",
    "Similar to its discrete counterpart, the cumulative function is called the Cumulative Distribution Function (CDF).\n",
    "\n",
    "It is common to see the function for the PDF represented as:\n",
    "\n",
    "$$ f(x) $$\n",
    "\n",
    "While the function for the CDF is often represented as \n",
    "\n",
    "$$ F(x) $$\n",
    "\n",
    "This is because the CDF is the integral of the PDF. In other words:\n",
    "\n",
    "$$ \\frac{ dF(x) }{ dx } = F'(x) = f(x)$$\n",
    "\n",
    "$$ \\int_{-\\infty}^x f(x)dx = F(x) $$\n",
    "\n",
    "References:\n",
    "- [Probability density functions - Khan Academy](https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/random-variables-continuous/v/probability-density-functions)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6M0VvLPGyls_"
   },
   "source": [
    "# Joint, Marginal, and Conditional Distributions\n",
    "\n",
    "Consider the following table of data that shows the joint distribution between the two dimensions *Grade on Exam* and *Time Studied*:\n",
    "\n",
    "\\begin{array} \n",
    "{|r|r|}\n",
    "\\hline  & \\text{0-20} & \\text{21-40} & \\text{41-60} & \\text{> 60} & \\text{TOTAL} \\\\ \n",
    "\\hline \\text{80-100} & 0 & 4 & \\color{blue}{16} & 20 & \\color{orange}{40} \\\\ \n",
    "\\hline \\text{60-79} & 0 & \\color{red}{20} & \\color{blue}{30} & 10 & \\color{orange}{60} \\\\ \n",
    "\\hline \\text{40-59} & 2 & 4 & \\color{blue}{32} & 32 & \\color{orange}{70} \\\\ \n",
    "\\hline \\text{20-39} & 10 & 2 & \\color{blue}{8} & 0 & \\color{orange}{20} \\\\ \n",
    "\\hline \\text{0-19} & 2 & 0 & \\color{blue}{0} & 8 & \\color{orange}{10} \\\\ \n",
    "\\hline \\text{TOTAL} & 14 & 30 & 86 & 70 & \\color{green}{200} \\\\ \n",
    "\\hline  \n",
    "\\end{array}\n",
    "\n",
    "We define two random variables to represent these segments.\n",
    "\n",
    "$$ A \\rightarrow \\text{ Grade on Exam (rows) }$$\n",
    "\n",
    "$$ B \\rightarrow \\text{ Time Studied (columns) }$$\n",
    "\n",
    "The bottom right cell (green) tells us that 200 people took the exam.\n",
    "\n",
    "References:\n",
    "*  [Marginal and conditional distributions - Khan Academy\n",
    "](https://www.khanacademy.org/math/ap-statistics/analyzing-categorical-ap/distributions-two-way-tables/v/marginal-distribution-and-conditional-distribution)\n",
    "* [A Gentle Introduction to Joint, Marginal, and Conditional Probability\n",
    "](https://machinelearningmastery.com/joint-marginal-and-conditional-probability-for-machine-learning/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XFo74NCcF05E"
   },
   "source": [
    "## Joint Distribution\n",
    "\n",
    "Consider the two-way table comparing *Grade on Exam* and *Time Studied*. A given value, such as the red 20 shows the joint probability of meeting conditions on both dimensions. In this case, the 20 out of 200 students got between 60 - 79 and studied 21 - 40 minutes. \n",
    "\n",
    "$$ p( A = \\text{'60-79'}, B = \\text{'21-40'} ) = \\frac{20}{200}$$\n",
    "\n",
    "The joint distribution is typically notated as follows:\n",
    "\n",
    "$$p(A, B)$$\n",
    "\n",
    "But, it is also often notated as any of the following:\n",
    "\n",
    "$$p(A \\cap B)$$\n",
    "\n",
    "$$p(A \\land B)$$\n",
    "\n",
    "$$p(A \\text{ AND } B)$$\n",
    "\n",
    "The joint distribution is the logical **AND** of the distributions of the two random variables $A$ and $B$. It is also often referred to as the **intersection** of the two sets.\n",
    "\n",
    "### Properties\n",
    "The joint distribution of two random variables is commutative:\n",
    "\n",
    "$$p(x, y) = p(y, x) $$\n",
    "\n",
    "The joint distribution of two random variables is associative:\n",
    "\n",
    "$$p(x, y, z) = p( (x, y), z) = p( x, (y, z)) $$\n",
    "\n",
    "References:\n",
    "* [Probability & Statistics Primer](https://atmos.washington.edu/~hakim/591/prob_primer.pdf)\n",
    "\n",
    "\n",
    "### Mutually exclusive events\n",
    "If $A$ and $B$ are **mutually exclusive**, then:\n",
    "\n",
    "$$p(A, B) = 0$$\n",
    "\n",
    "For example, the probability of rolling a 4 *and* a 6 on a single roll of a die is $0$ because it is not possible.\n",
    "\n",
    "### Independent events\n",
    "If $A$ and $B$ are **independent**, then:\n",
    "\n",
    "$$p(A, B) = p(A) \\cdot p(B)$$\n",
    "\n",
    "$$p(A | B) = p(A)$$\n",
    "\n",
    "\n",
    "For example, if you want to find the probability of rolling a 6 on two separate rolls of a die, then you would just multiply the probabilities of each roll, because they are independent.\n",
    "\n",
    "$$p(A) = \\frac{1}{6}$$\n",
    "\n",
    "$$p(B) = \\frac{1}{6}$$\n",
    "\n",
    "$$p(A, B) = \\frac{1}{6} \\cdot \\frac{1}{6} = \\frac{1}{36}$$\n",
    "\n",
    "### Dependent events\n",
    "If $A$ and $B$ are **dependent**, then:\n",
    "\n",
    "$$p(A, B) = p(A) \\cdot  p(B|A)$$\n",
    "\n",
    "For example, let's say you have a bag with 10 marbles. Of the 10, there are 7 black marbles and 3 blue marbles. You want to find the probability of drawing a black marble on the second turn, given that you already drew a blue marble on the first turn.\n",
    "\n",
    "The probability of drawing a blue marble is:\n",
    "$$p(A) = \\frac{3}{10}$$\n",
    "\n",
    "The probability of drawing a black marble is:\n",
    "$$p(B) = \\frac{7}{10}$$\n",
    "\n",
    "The probability of drawing a black marble, given that a blue one was already drawn is:\n",
    "$$p(B | A) = \\frac{7}{9}$$\n",
    "\n",
    "So:\n",
    "$$p(A, B) = p(A) \\cdot  p(B|A) = \\frac{3}{10} \\cdot \\frac{7}{9} = \\frac{7}{30}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MOYU4haFb2cU"
   },
   "source": [
    "## Marginal Distribution\n",
    "\n",
    "Consider the two-way table at the top of this section comparing Grade on Exam and Time Studied. \n",
    "\n",
    "The marginal probability gives you a summary from the perspective of just one of the dimensions. To see the marginal distribution, you can just look at the *margin* of the table, or the TOTAL column for the dimension you care about.\n",
    "\n",
    "So, if we want to know about the marginal distribution with respect to Grade on Exam, we can just look at the orange values in the total column on the right.\n",
    "\n",
    "We could represent these in terms of their probabilities, which gives us the **marginal probability distribution** denoted as $p(A)$:\n",
    "\n",
    "\\begin{array} \n",
    "{|r|r|}\n",
    "\\hline \\text{ Grade } & \\text{ Count } & \\text{ Probability } \\\\ \n",
    "\\hline 80-100 & 40 & .2 \\\\ \n",
    "\\hline 60-79 & 60 & .3 \\\\ \n",
    "\\hline 40-59 &  70 & .35 \\\\ \n",
    "\\hline 20-39 &  20 & .1 \\\\ \n",
    "\\hline 0-19 & 10 & .05 \\\\ \n",
    "\\hline \\text{ TOTAL } & 200 & 1 \\\\ \n",
    "\\hline  \n",
    "\\end{array}\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEZCAYAAABmTgnDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkS0lEQVR4nO3de7xVVb338c9XCMy8IeynjIuAUSesc7CHMJ9TWl7xWOIpLdTUSg9ZUj3HOolZqJQd0p6yC2WcpDyWoWkmKWWlkXpKBcI0MAwQBQRF8B6iwO/5Y4wNk+Xae67F3nvtC9/367Vfe84x55jrN9Zl/tYcc64xFRGYmZm1ZpfODsDMzLo+JwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4W3YikD0m6szD/nKThray/UNI7OzimoZJCUu+OfJz8WO+UtHIH67Yap6TPSfp+tXUl/VLS6Tse+XaP80pJv5D0tKSftsc2O0o9z3d7Pkc9VdnntatzsmgHkk6WNC+/GVbnD87bO/pxI2L3iFiWY/ihpC9VLD8gIuZ0dBw9QUR8OSLObGHZMRFxJbw8Ye+AE4BXA/0j4sQ2bKeUpOWSjujIx2hWfI46iqTJOYkfUSh7v6Q/SPq7pDk1bucISX+S9LyklZLeX1g2StL8vL35kka1V/zFz2t35GTRRpLOAS4DvkzaCQwBvgOM68SwuqVGHJ10AfsBD0bEpnor7iTPT1WS9gdOBFZXLFpP+vxNrXE7I4GrgfOBvYB/AubnZX2AG4EfAf2AK4Ebc7lFhP928I/0ZnsOOLGVdfqS3syP5r/LgL552TuBlcCngcdJH4QPF+r2B2YBzwD3AF8E7iwsD+B1wATgJeDFHM8v8vLlwBHtEMexwIIcxwrgwsKyoTmO3i20fzlwHrAIeBL4AbBrxeOeC6wBrqoxzs8BT+Rtn1JnnBPydlcDnyksvxD4UbU2AXOAM4E3Ai8Am/Pz/BTwVuAxoFdhW+8F/lzlubgov0Yv5fpnkL6wfR54OD/3/w3sVRHHGcAjwO1VtjkAuCnHsh64I2/zKmALsCE/1mebn78qr0/ze+SVwA/z67QI+I/m9fP09RV1vwl8o/gc5ekPAXcCX83begg4plBvGHA78CzwW2Ba83PfyufoV8C/FOOtWH4mMKeGz+zVwBdbWHYUsApQoewRYGwL688BvgT8IT/HvyB9Zn9Meg/OBYZWfl7z9A9zu2/Oz8PdwP55mYCv5/fDM8D9wJs6e3/nI4u2ORjYFbihlXXOB94GjCJ9ixlD2jk0ew0p6Qwk7RSmSeqXl00j7Zz2BT6S/14mIqaT3qCXRDrUfU87x/E8cBqwN2mH/DFJx7fS5kqnAEcD+wOvr/K4+5C+cU+oMc4BOc7TgemS3lBHnO8CRpB2DOfW000TEQ8AZwF/zM/z3hExF1iXt9fsVNJOv7L+BaQj0Gty/StIO9YP5biGA7sD366oeigpUR1dJaxPkxJoE+nI9nPpoeJU0o7uPfmxLqmhiReQXqP982MVz0H8CBgraW/YepQzvlo7s4OAxaTX6hLgCknKy64mffnpT0rSp7YWlKQTgY0RMbuGNpR5W97m/bnL+EeS9snLDgDui7zHzu7L5S0ZT4p/IOl5+yPpC9E+wAOk57S1uheRjmKWABfn8qOAQ0iflb2A95PeY53KyaJt+gNPROtdCqcAUyLi8YhYS3pzFD8cL+XlL+UPw3PAGyT1At4HTI6I5yPiL6TD4h21Q3EARMSciLg/IrZExH3AT0g7sFp9OyJWRMR60gfipMKyLcAFEbExIjbUECfAF/L6vyd9M3t/HXFelJ/P+0kf6pNouyuBDwLkHc/RpB1iLU4BvhYRyyLiOdJR2PiKLqcLc8wbqtR/ifRlYr/82t1RsbOrx/uBiyNifUSsIB05ABARq0lHA83nWcaS3vvzW9jWwxHxXxGxmfT87Au8WtIQ0tHY5Ih4MSLuJB09VyVpD1KC/dQOtqnSINL76X2kLw2vBL6Vl+0OPF2x/tPAHq1s7wcRsTQingZ+CSyNiN/mfcJPgQNbqXtDRNyT1/0x6QsSpNd0D+AfSEc5D+Tnv1M5WbTNOmBASV/ya0ldDM0ezmVbt1GRbP5OetM2Ab1J3SnFujtqR+NA0kGSfidpraSnSd+uB9Tx2JVtKD7u2oh4oY44n4yI56strzHO1mLZUT8C3iPpVaQd7h11fLirtbc36Sih2QpadinpW+mvJS2TNKn2sKvG0tr7bWtSzP+vamVba5onIuLveXL3/BjrC2XQevsuBK6KiOWtrFOVpMvzRSfPSfpcLt5A2sE/mJPzl0ndW5C+IO1ZsZk9Sd1ELXmsML2hyvzurdRdU5je+nmLiNtIR5fTgMclTZdUGVfDOVm0zR+BjcDxrazzKKmLpdmQXFZmLbAJGFxRtyVl3yZ3NA5I35JnAYMjYi/gclK/aq0q21B83Mq4y+Lsl3fK1ZbXEmdrsdTiZc9zRKwivRfeS/rW2tpOtFK19m5i+51Oi69tRDwbEZ+OiOHAccA5kg5vod7zwG7NM/notamwfDWtv99+DvyjpDcB7yZ9G67XamAfSbsVyga3tDJwOPBJSWskrcnrXivp3LIHioizchfc7hHx5Vx8H9s/L8XphaT2Fd8z/5jLGyoivhkR/xsYSeqO+o9Gx1DJyaIN8qHnZFL//vGSdpP0CknHSGruI/4J8HlJTZIG5PV/VMO2NwM/Ay7M2x3J9n3IlR4j9Xm3ZIfiyPYgfRt8QdIY4OQa6zU7W9Kg3EVzPnBNG+O8SFIfSe8g7bSaf69QS5xfyM/nAcCHS2Kp5jFgUJUrZP6bdBL5zaTXrVY/Af5d0jBJu7PtnEZNV0tJerek1+Ud3NOkk+9bCrEW3xMPArtKOlbSK0jngvoWll8LnCepn6RBwCeKj5WPAK8jn3OIiEfqaGfzNh4G5pHe130kHQxUO8fW7HDgTaQumlGk5PpR0rduJPWStCvpaGwXSbvmtrXkB8CHJQ3PCWsS6QIBSCesN5OSU19JE3P5bfW2sy0kvTUfJb+ClOBfYNtr2mmcLNooIv4fcA7pg7eWdEg9kfQtDNLVEvNI32juB/6Uy2oxkXRouoZ09cQPWln3CmCkpKck/bzK8rbE8XFgiqRnSTvva2us1+xq4NfAMmBpyeOWxbmGdIXNo6RvtmdFxF/riPP3pG6bW4GvRsSv62zLbaRvmmskPVEov4F0hHBDRRdLmRmkI5HbSVcNvUDFTrrECNIVRc+Rjm6+ExG/y8v+k5R4n5L0mfzl5uPA90lX/TxPOjne7CJS19NDpNer2hHSlaSEWM/RU6VTSBeHrCO9tteQjtBfJiLWRcSa5j/SzvzJ3IUE6UhuA/Bd4B15+r9aeuCImEFK7HeT2roR+GRe9iKpl+A00tVlHwGOz+WNtCepDU/mGNeRuhs7lXb8XJhZOUnLSZdU/razY+lokpYCH+3Jbc0nqP8KvCYinmmnbV4D/DVfLWZdlI8szNqBpPeR+r8b2mXRSJJ2IR1Fz2xLosjdLPtL2kXSWNIPWH/eTmFaB9lpfxFq1l6UhpkYCZwaEZ3et9wR8kUFj5G6Rca2cXOvIZ3X6U/qBvtYRCxo4zatg7kbyszMSrkbyszMSvXIbqgBAwbE0KFDOzsMM7NuZf78+U9ERFO1ZT0yWQwdOpR58+Z1dhhmZt2KpBZHiXA3lJmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvVsGQhaaykxZKWVLtBi6Sz8q0O75V0Zx6SG0lDJW3I5fdKurxRMZuZWdKQ31nkm6xMA44kjQUzV9KsiFhUWO3qiLg8r38c8DW2jUGzNCJGNSJWMzN7uUYdWYwBluT7DL8IzCSNNLlVxSiWr6L8zm9mZtYgjfoF90C2v8/uSuCgypUknU0aArkPcFhh0TBJC4BngM9HxB1V6k4AJgAMGdLa3UfNqhs66ebODqHdLJ96bGeHYD1MlzrBHRHTImJ/4FzSnecg3bN3SEQcSEokV1e7eXlETI+I0RExuqmp6tAmZma2gxqVLFax/U3ZB+Wylswk3d6QiNgYEevy9HzSbTlf3zFhmplZNY1KFnOBEfmm9H2A8cCs4gqSRhRmjwX+lsub8glyJA0n3XN4WUOiNjMzoEHnLCJik6SJwC1AL2BGRCyUNAWYFxGzgImSjgBeIt2o/PRc/RBgiqSXgC3AWRGxvhFxm5lZ0rAhyiNiNjC7omxyYfpTLdS7Hri+Y6MzM7PWdKkT3GZm1jU5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpRqWLCSNlbRY0hJJk6osP0vS/ZLulXSnpJGFZefleoslHd2omM3MLGlIspDUC5gGHAOMBE4qJoPs6oh4c0SMAi4BvpbrjgTGAwcAY4Hv5O2ZmVmDNOrIYgywJCKWRcSLwExgXHGFiHimMPsqIPL0OGBmRGyMiIeAJXl7ZmbWIL0b9DgDgRWF+ZXAQZUrSTobOAfoAxxWqHtXRd2BVepOACYADBkypF2CNtuZDJ10c2eH0G6WTz22s0PocbrUCe6ImBYR+wPnAp+vs+70iBgdEaObmpo6JkAzs51Uo5LFKmBwYX5QLmvJTOD4HaxrZmbtrFHJYi4wQtIwSX1IJ6xnFVeQNKIweyzwtzw9Cxgvqa+kYcAI4J4GxGxmZllDzllExCZJE4FbgF7AjIhYKGkKMC8iZgETJR0BvAQ8CZye6y6UdC2wCNgEnB0RmxsRt5mZJY06wU1EzAZmV5RNLkx/qpW6FwMXd1x0ZmbWmi51gtvMzLomJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlWpYspA0VtJiSUskTaqy/BxJiyTdJ+lWSfsVlm2WdG/+m9WomM3MLOndiAeR1AuYBhwJrATmSpoVEYsKqy0ARkfE3yV9DLgE+EBetiEiRjUiVjMze7lGHVmMAZZExLKIeBGYCYwrrhARv4uIv+fZu4BBDYrNzMxKNCpZDARWFOZX5rKWnAH8sjC/q6R5ku6SdHy1CpIm5HXmrV27ts0Bm5nZNg3phqqHpA8Co4FDC8X7RcQqScOB2yTdHxFLi/UiYjowHWD06NHRsIDNzHYCjTqyWAUMLswPymXbkXQEcD5wXERsbC6PiFX5/zJgDnBgRwZrZmbba1SymAuMkDRMUh9gPLDdVU2SDgS+R0oUjxfK+0nqm6cHAP8MFE+Mm5lZB2tIN1REbJI0EbgF6AXMiIiFkqYA8yJiFnApsDvwU0kAj0TEccAbge9J2kJKblMrrqIyM7MO1rBzFhExG5hdUTa5MH1EC/X+ALy5Y6MzM7PW+BfcZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqVqThaS+ndkIGZm1nXVc2TxiKQbJZ2Qf1hnZmY7iXqSxVDgVuBcYI2k6ZLe3iFRmZlZl1JzsoiItRHxzYh4K3Aw8DhwlaRlkqYUb1ZkZmY9y46e4H5N/tsTWEoabnxBtTvgmZlZ91fzcB+SDgA+CJwMPA9cCfxTRKzMy78I3AdM7YA4zcysE9UzNtTtwE+AEyPinsqFEbFc0mXtFZiZmXUd9SSLf42I2ysLJY1pTh7FgQHNzKznqOecxU0tlP+qPQIxM7Ouq/TIQtIugNKklKeb7Q9s6qDYzMysi6ilG2oTEIXpoi3Axe0akZmZdTm1JIthpKOJ3wOHFMoDWBsRGzoiMDMz6zpKk0VEPJwn/aM7M7OdVKvJQtL0iJiQp/+7pfUi4rT2DszMzLqOsiOLhwrTSzsyEDMz67paTRYR8Z+F6Yva8kCSxgLfAHoB34+IqRXLzwHOJJ1EXwt8pLkLTNLpwOfzql+KiCvbEouZmdWnrBvqsFo2EhG3lWynFzANOBJYCcyVNCsiFhVWWwCMjoi/S/oYcAnwAUn7ABcAo0kn1efnuk/WEpuZmbVdWTfUFTVsI4DhJeuMAZZExDIASTOBccDWZBERvyusfxdpHCqAo4HfRMT6XPc3wFjS0CNmZtYAZd1Qw9rpcQYCKwrzK4GDWln/DOCXrdQdWFlB0gRgAsCQIUPaEquZ7WSGTrq5s0NoN8unHtsh2+1y9+CW9EFSl9Ol9dSLiOkRMToiRjc1NXVMcGZmO6mycxYPRMQb8/QKtv2SezsRUfZVfhUwuDA/KJdVPt4RwPnAoRGxsVD3nRV155Q8npmZtaOycxb/Vpj+YItrlZsLjJA0jLTzH0+6L8ZWkg4EvgeMjYjHC4tuAb4sqV+ePwo4rw2xmJlZncrOWdxZmP79jj5IRGySNJG04+8FzIiIhZKmAPMiYhap22l34KdpvEIeiYjjImJ9vrHS3Ly5Kc0nu83MrDHquVNeH9JvHU4CXgs8CswELo6IF8rqR8RsYHZF2eTC9BGt1J0BzKg1VjMza1/13Pzou8AbgE8CD5PGivoc6cqkj7R/aGZm1lXUkyyOB/aPiKfy/CJJdwNLcLIwM+vR6rl0dg2wW0XZK4HV7ReOmZl1RfUM93EV8CtJ3yL9MG4wcDbQ4mi0ZmbWM+zIcB+fq5j/KPCV9gnHzMy6okYN92FmZt1Ylxvuw8zMup56fmexJ3AhcCgwgHRfbqCm4T7MzKwbq+fI4jvAW4ApwD7AJ4BHgK93QFxmZtaF1PM7i6OAN0bEOkmbI+JGSfOAX+CEYWbWo9VzZLEL8HSefk7SXqTfWLyu3aMyM7MupZ4jiz+TzlfcCtxB6pZ6DniwA+IyM7MupJ4ji38DlufpTwEvAHsDp7VvSGZm1tXUfGTRfP/sPP046danZma2E6jrdxaSPiLpN5IW5v9nKN98wszMeq56fmdxCTAOuIxtQ5R/hjRs+Wc7IjgzM+sa6jnB/SHgLRGxsrlA0k3An3CyMDPr0erphno2/1WWPdN+4ZiZWVdUNkT58MLsZcDPJE1l2xDl/4F/kGdm1uOVdUMtAYLCOFDAuyrWOQz4dnsGZWZmXUvZEOUeldbMzOofolzSEEkHSxpcZ72xkhZLWiJpUpXlh0j6k6RNkk6oWLZZ0r35b1a9MZuZWdvUc+nsvsBM4GBgHdBf0l3A+Ih4tKRuL2AacCTpfMdcSbMiYlFhtUdIV1x9psomNkTEqFpjNTOz9lXPkcV3SeND9YuIfYF+wALg8hrqjgGWRMSyiHiRlHTGFVeIiOURcR+wpY6YzMysAer5ncXbgX0j4iWAiHhe0meBVTXUHQisKMyvBA6q47F3zcOhbwKmRsTPK1eQNAGYADBkiO/FZGbWnuo5sngSGFlR9gbgqXaLpmX7RcRo4GTgMkn7V64QEdMjYnREjG5qampASGZmO496jiwuAX4r6Qq2DffxYeALNdRdRfpdRrNB1HZEAkBErMr/l0maAxwILK21vpmZtU3NRxYR8V/AB0j3335P/n9yREyvofpcYISkYZL6AOOBmq5qktRPUt88PQD4Z2BR67XMzKw91XRkka9mehAYGRG31fsgEbFJ0kTgFqAXMCMiFkqaAsyLiFmS3grcQDpx/h5JF0XEAcAbge9J2kJKblMrrqIyM7MOVlOyiIjNkjYDuwIbd+SBImI2MLuibHJhei6pe6qy3h+AN+/IY5qZWfuo55zFZcC1kr5MupopmhcUb4xkZmY9Tz3Jonn8pyMryoPUtWRmZj1U6QluSbvlo4mbgS8Cu0XELoU/Jwozsx6ulquhppGufnoAeB9waYdGZGZmXU4tyWIscFREfBY4Bnh3x4ZkZmZdTS3J4lURsRogIlYAe3VsSGZm1tXUcoK7t6R3se0GSJXz7MhvL8zMrPuoJVk8DswozK+rmA9gOGZm1mOVJouIGNqAOMzMrAvzbVPNzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1INSxaSxkpaLGmJpElVlh8i6U+SNkk6oWLZ6ZL+lv9Ob1TMZmaWNCRZSOpFuuPeMcBI4CRJIytWewT4EHB1Rd19gAuAg4AxwAWS+nV0zGZmtk2jjizGAEsiYllEvAjMBMYVV4iI5RFxH7Clou7RwG8iYn1EPAn8hnT3PjMza5BGJYuBwIrC/Mpc1tF1zcysHdRy86NuQdIEYALAkCFDOjma7mvopJs7O4R2s3zqsZ0dglmP0agji1XA4ML8oFzWbnUjYnpEjI6I0U1NTTscqJmZvVyjksVcYISkYZL6AOOBWTXWvQU4SlK/fGL7qFxmZmYN0pBkERGbgImknfwDwLURsVDSFEnHAUh6q6SVwInA9yQtzHXXA18kJZy5wJRcZmZmDdKwcxYRMRuYXVE2uTA9l9TFVK3uDGBGhwZoZmYt8i+4zcyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo1LFlIGitpsaQlkiZVWd5X0jV5+d2ShubyoZI2SLo3/13eqJjNzCzp3YgHkdQLmAYcCawE5kqaFRGLCqudATwZEa+TNB74CvCBvGxpRIxqRKxmZvZyjTqyGAMsiYhlEfEiMBMYV7HOOODKPH0dcLgkNSg+MzNrRUOOLICBwIrC/ErgoJbWiYhNkp4G+udlwyQtAJ4BPh8Rd1Q+gKQJwASAIUOGtCnYoZNublP9rmL51GM7OwQz6yG6wwnu1cCQiDgQOAe4WtKelStFxPSIGB0Ro5uamhoepJlZT9aoZLEKGFyYH5TLqq4jqTewF7AuIjZGxDqAiJgPLAVe3+ERm5nZVo1KFnOBEZKGSeoDjAdmVawzCzg9T58A3BYRIakpnyBH0nBgBLCsQXGbmRkNOmeRz0FMBG4BegEzImKhpCnAvIiYBVwBXCVpCbCelFAADgGmSHoJ2AKcFRHrGxG3mZkljTrBTUTMBmZXlE0uTL8AnFil3vXA9R0eoJmZtag7nOA2M7NO5mRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1INSxaSxkpaLGmJpElVlveVdE1efrekoYVl5+XyxZKOblTMZmaWNCRZSOoFTAOOAUYCJ0kaWbHaGcCTEfE64OvAV3LdkcB44ABgLPCdvD0zM2uQRh1ZjAGWRMSyiHgRmAmMq1hnHHBlnr4OOFyScvnMiNgYEQ8BS/L2zMysQXo36HEGAisK8yuBg1paJyI2SXoa6J/L76qoO7DyASRNACbk2eckLW6f0DvMAOCJjnwAfaUjt94mHd522LnbvzO3HXbu9rex7fu1tKBRyaLDRcR0YHpnx1ErSfMiYnRnx9EZdua2w87d/p257dC929+obqhVwODC/KBcVnUdSb2BvYB1NdY1M7MO1KhkMRcYIWmYpD6kE9azKtaZBZyep08AbouIyOXj89VSw4ARwD0NitvMzGhQN1Q+BzERuAXoBcyIiIWSpgDzImIWcAVwlaQlwHpSQiGvdy2wCNgEnB0RmxsRdwfrNl1mHWBnbjvs3O3fmdsO3bj9Sl/ezczMWuZfcJuZWSknCzMzK+Vk0c7KhjXJ6/xK0lOSbqooP0zSnyT9RdKV+aqwLk3SYEm/k7RI0kJJn8rl+0j6jaS/5f/9Wqh/haQ/S7pP0nWSds/l+0m6NZfPkTSoke2qh6RekhY0v575Qo6783vgmnxRR7V6c/J75d78979yeXdq+975dfurpAckHVzHa39Hoe2PSvp5Lu8n6Ybc/nskvamhjWqFpH/P7/O/SPqJpF3reL0vlrRC0nMV5d1jqKOI8F87/ZFO3i8FhgN9gD8DI6usdzjwHuCmQtkupB8lvj7PTwHO6Ow21dDmfYG35Ok9gAdJQ7pcAkzK5ZOAr7RQf8/C9NcKdX4KnJ6nDwOu6uy2tvIcnANc3fx6AtcC4/P05cDHWqg3Bxhdpbw7tf1K4Mw83QfYu9bXvmI71wOn5elLgQvy9D8At3Z2O3MsA4GHgFcWXucP1fF6vy1/Xp6rKP84cHmeHg9ck6dH5n1IX2BY3rf06rT2d/YL0JP+gIOBWwrz5wHntbDuOyuSRROwtDD/DmB2Z7dpB56DG4EjgcXAvrlsX2BxST0B3wXOzfMLgcGFZc90dttaiHsQcGveqd+UY30C6F3tPVFRt6Vk0V3avlfeeaqivN7Xfk/gSfIXB+Bm4B2F5UuBV3eB9jaPMrEP6UrSm4Cja329C9upTBa3AAfn6d55e6rcfxTX64w/d0O1r2rDmrxsaJIWPAH0ltT8684T2P7HiF1ePnw+ELib9OFenRetAV7dSr0f5HX+AfhWLv4z8N48/a/AHpL6d0DYbXUZ8FlgS57vDzwVEZvyfNl74Ae5G+YLeSw06D5tHwasJbVhgaTvS3oVdbz22fGko4dn8vzW9ksaQxqCotO74iJiFfBV4BFgNfA0MJ/6Xu9qthvqKG+3P23bn7Q7J4suItJXh/HA1yXdAzwLdJvfk+RzDdcD/7fwoQe2tq3Fa7Qj4sPAa4EHgA/k4s8Ah0paABxK+tV+l3o+JL0beDwi5u/gJk6JiDeTjiLfAZyay7t827PewFuA70bEgcDzpG6nrcpe++wk4CeF+anA3pLuBT4BLKALtD+fexlHSpKvBV5FGgl7p+Bk0b6qDU2ytnAS77jWKkfEHyPiHRExBrid1P/f5Ul6BSlR/DgifpaLH5O0b16+L/B4nr4lPxffL24j0g8tZwLvy/OPRsR7807o/Fz2VCPaU4d/Bo6TtJwU+2HAN0g7uuaLEwYBq/JJ8Ob3wRTY+k2ViHiWdM5jTJ7vDm2H9E13ZUTcneevIyWPml97SQNI7b65uSwinomID0fEKOA0Uhftsga0p8wRwEMRsTYiXgJ+RnoP1PR6t6JbDHXkZNG+qg1rcl1EjMp/lUOcbKdwNUxf4FzSybIuLXedXAE8EBFfKywqDt9yOulcBhFxdH4uzlTyusJ2jgP+mucHSGp+f54HzOj41tQnIs6LiEERMZT0Wt8WEacAvyN1I0Jue0RsLrwPJkvqnXeUzcn23cBf8nyXbztARKwBVkh6Qy46nDTSQulrX9jMCaRzdy80F+QrrJqvKDoTuL3yaLWTPAK8TdJu+f3a3N7S17tku91jqKPOOlnSU/+AfyEdESwFzm9hnTtIfb0bSN/Ojs7ll5K6YhaTunM6vT01tPftpG6G+4B789+/kPpcbwX+BvwW2KdK3V2A/wHuJ+0of8y2k5wn5LoPAt8H+nZ2W0ueh3ey7Wqo4aQP9RLSlU0vi53UhTE/P28LSUckvbpb24FRwLzcjp8D/Wp57Qv15wBjK8oOzm1fTPr23q+z21mI7SLSF5q/AFeRrlQqfb1z3Uvy531L/n9hLt8111uStzO8UOf8vC9ZDBzTmW33cB9mZlbK3VBmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszLoYScslHdHZcZgVOVmY1UnS+DyU9POSHs/THy+M7WTW4zhZmNVB0qdJP6C7FHgNaZC8s0jDPrzsPgaSejU0QLMO4mRhViNJe5HuM/LxiLguIp6NZEFEnBIRGyX9UNJ3Jc2W9DzwLknH5lFZn8k3v7mwYrunSnpY0jpJ51cs20XSJElL8/JrJe3TuFabJU4WZrU7mDS8w40l650MXEy6GdSdpNFYTyPdGOhY4GOSjgeQNJJ0H49TSSOZ9mf74bg/QRrC+9C8/ElgWju0xawuThZmtRsAPBHb7l2ApD8o3SJ3g6RDcvGNEfE/EbElIl6IiDkRcX+ev480HPehed3mgfRuj4iNwBfYdm8MSF1c50fEyrz8QuAEdYNb7lrP4jecWe3WAQMk9W5OGBHxfwAkrWTbl6/iDWuQdBDpHg1vIp3X6EsaOA7S0cLW9SPieUnrCtX3A26QVEwgm0nnSjptuGrb+fjIwqx2fwQ2km6A05rK0TmvJg03PTgi9iINPd985dRqCvcskLQbqSuq2QrSaKN7F/52jXwvDLNGcbIwq1GkGxBdBHxH0gmS9sgnoEeRhhxvyR7A+oh4Id8m9OTCsuuAd0t6e76HwxS2/1xeDlwsaT8ASU2SypKVWbtzsjCrQ0RcApxDuu/2Y/nve6SbVf2hhWofB6ZIehaYDFxb2N5C4GzS0cdq0gnslYW63yAdlfw6178LOKgdm2RWE9/PwszMSvnIwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVur/A03rb5kjl6E/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "face = [ '0-19','20-39','40-59','60-79','80-100' ]\n",
    "probs = [ 10/200, 20/200, 70/200, 60/200, 40/200 ]\n",
    "\n",
    "plt.bar( x=face, height=probs )\n",
    "plt.ylabel( 'Probability', fontsize=12 )\n",
    "plt.xlabel( 'Grade', fontsize=12 )\n",
    "plt.title( 'Conditional probability for studying 41-60 mins', fontsize=12 )\n",
    "\n",
    "axes.set_ylim( [0,.5] )\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, if we have a joint distribution:\n",
    "\n",
    "$$p(X, Y)$$\n",
    "\n",
    "If we know the values of $X$ we can find the probability of Y:\n",
    "\n",
    "$$p(Y)$$\n",
    "\n",
    "In this context, $p(y)$ is called the *marginal distribution* and we can find it using a process called *marginalization*.\n",
    "\n",
    "We can perform marginalization on a discrete random variable $x$ as follows:\n",
    "\n",
    "$$p(Y) = \\sum_{X} p(X, Y) = \\sum_{X} p(Y | X) p(X)$$\n",
    "\n",
    "We can perform marginalization on a continuous random variable $x$ as follows:\n",
    "\n",
    "$$p(Y) = \\int p(X, Y)dx = \\int p(Y|X) p(X)dx$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "odwJiLAcm0o-"
   },
   "source": [
    "## Conditional Distribution\n",
    "\n",
    "A **conditional distribution** (also referred to as a *conditional probability* or a *conditional probability distribution*) is notated as:\n",
    "\n",
    "$$p(A | B)$$\n",
    "\n",
    "This is read as \"*the probability of A given B*\". It is the probability of event A, given that we know that B has happened. \n",
    "\n",
    "It is defined as:\n",
    "\n",
    "$$p(A | B) = \\frac{ p(A, B) }{ p(B) }$$\n",
    "\n",
    "This is the foundation for Bayes Theorem, which is discussed below.\n",
    "\n",
    "Consider the two way table comparing Grade and Time Studied at the top of this section. We can look at the the grades, given a specific range of study time. \n",
    "\n",
    "For example, if we want to look at the distribution of grades for all students who studied for 40-61 minutes (blue column), we could look at:\n",
    "\n",
    "$$ p( A | B = \\text{'40-61'} ) $$\n",
    "\n",
    "In this case, the total is 86, not 200. This is because we're only looking at the column where *Time Studied* was between 40 - 61.\n",
    "\n",
    "This results in the following conditional probability distribution:\n",
    "\n",
    "\\begin{array} \n",
    "{|r|r|}\n",
    "\\hline \\text{ Grade } & \\text{ Count } & \\text{ Probability } \\\\ \n",
    "\\hline 80-100 & 16 & .186 \\\\ \n",
    "\\hline 60-79 & 30 & .349 \\\\ \n",
    "\\hline 40-59 &  32 & .372 \\\\ \n",
    "\\hline 20-39 &  8 & .093 \\\\ \n",
    "\\hline 0-19 & 0 & 0 \\\\ \n",
    "\\hline \\text{ TOTAL } & 86 & 1 \\\\ \n",
    "\\hline  \n",
    "\\end{array}\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEZCAYAAABmTgnDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkKUlEQVR4nO3df7xVVZ3/8ddbCMz8hcK3DFDAqAmrwYYwv5M/UkscS53SBjW10mEsqb5jTWIWKmZjOt+yGsyYpBzN0DSTlDJT0ZxSuYY/QsMAUSDQK+AvQhT8zB9rXd0ez737nHvvOfdyeT8fj/u4e6+9196fdX7sz1lr77OPIgIzM7OObNXTAZiZWe/nZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysliMyLpE5LuKMw/J2lUB+svkLR/g2MaISkk9W/kfvK+9pe0vJN1O4xT0pcl/aDaupJ+KemEzkf+qv28XtIvJD0t6afdsc1Gqefx7s7HqK8qe7/2dk4W3UDSMZJa8othZX7jvK/R+42IbSNiSY7hR5K+VrF8j4iY2+g4+oKI+HpEnNTOskMi4lJ4bcLuhCOBNwI7R8RRXdhOKUlLJR3UyH20KT5GjSJpak7iBxXKPibpd5L+Kmlujds5SNIfJK2TtFzSxwrLxkq6J2/vHkljuyv+4vt1c+Rk0UWSTgUuBL5OOgjsClwEHN6DYW2WmtE76QV2Ax6OiI31VtxCHp+qJO0OHAWsrFi0hvT+O6/G7YwBrgDOAHYA/ha4Jy8bAFwHXA4MAi4FrsvlFhH+6+Qf6cX2HHBUB+sMJL2Y/5L/LgQG5mX7A8uBLwBPkN4InyzU3RmYDTwD3A2cA9xRWB7AW4BJwIvACzmeX+TlS4GDuiGOQ4H5OY5lwFmFZSNyHP3baf9S4HTgQWAt8ENg64r9ngasAi6rMc4vA0/mbR9bZ5yT8nZXAl8sLD8LuLxam4C5wEnA24HngU35cX4KeA/wONCvsK2PAPdVeSzOzs/Ri7n+iaQPbF8BHs2P/X8DO1TEcSLwGHB7lW0OBq7PsawBfpu3eRnwErA+7+tLbY9fleen7TXyeuBH+Xl6EPi3tvXz9DUVdb8DfLv4GOXpTwB3AP+Rt/UIcEih3kjgduBZ4DfA9LbHvoP30a+AfyjGW7H8JGBuDe/ZK4Bz2ln2QWAFoELZY8CEdtafC3wN+F1+jH9Bes/+mPQanAeMqHy/5ukf5XbfkB+Hu4Dd8zIB38qvh2eAB4B39PTxzj2Lrtkb2Bq4toN1zgDeC4wlfYoZTzo4tHkTKekMJR0UpksalJdNJx2cdgE+lf9eIyJmkF6g50fq6n64m+NYBxwP7Eg6IH9a0hEdtLnSscDBwO7AW6vsdyfSJ+5JNcY5OMd5AjBD0tvqiPP9wGjSgeG0eoZpIuIh4GTg9/lx3jEi5gGr8/baHEc66FfWP5PUA70y17+EdGD9RI5rFLAt8J8VVfcjJaqDq4T1BVICHULq2X457SqOIx3oPpz3dX4NTTyT9BztnvdVPAdxOTBB0o7wci9nYrV2ZnsBC0nP1fnAJZKUl11B+vCzMylJH9dRUJKOAjZExJwa2lDmvXmbD+Qh48sl7ZSX7QHcH/mInd2fy9szkRT/UNLj9nvSB6KdgIdIj2lHdc8m9WIWAefm8g8C+5LeKzsAHyO9xnqUk0XX7Aw8GR0PKRwLTIuIJyKilfTiKL45XszLX8xvhueAt0nqB3wUmBoR6yLij6RucWd1Kg6AiJgbEQ9ExEsRcT/wE9IBrFb/GRHLImIN6Q1xdGHZS8CZEbEhItbXECfAV/P6t5E+mX2sjjjPzo/nA6Q39dF03aXAxwHygedg0gGxFscC34yIJRHxHKkXNrFiyOmsHPP6KvVfJH2Y2C0/d7+tONjV42PAuRGxJiKWkXoOAETESlJvoO08ywTSa/+edrb1aET8V0RsIj0+uwBvlLQrqTc2NSJeiIg7SL3nqiRtR0qwn+9kmyoNI72ePkr60PB64Lt52bbA0xXrPw1s18H2fhgRiyPiaeCXwOKI+E0+JvwU2LODutdGxN153R+TPiBBek63A/6G1Mt5KD/+PcrJomtWA4NLxpLfTBpiaPNoLnt5GxXJ5q+kF+0QoD9pOKVYt7M6GweS9pJ0q6RWSU+TPl0PrmPflW0o7rc1Ip6vI861EbGu2vIa4+wols66HPiwpDeQDri/rePNXa29/Um9hDbLaN8FpE+lv5a0RNKU2sOuGktHr7eXk2L+f1kH21rVNhERf82T2+Z9rCmUQcftOwu4LCKWdrBOVZIuzhedPCfpy7l4PekA/3BOzl8nDW9B+oC0fcVmticNE7Xn8cL0+irz23ZQd1Vh+uX3W0TcQupdTgeekDRDUmVcTedk0TW/BzYAR3Swzl9IQyxtds1lZVqBjcDwirrtKfs02dk4IH1Kng0Mj4gdgItJ46q1qmxDcb+VcZfFOSgflKstryXOjmKpxWse54hYQXotfIT0qbWjg2ilau3dyKsPOu0+txHxbER8ISJGAYcBp0o6sJ1664Bt2mZy73VIYflKOn69/Rx4l6R3AB8ifRqu10pgJ0nbFMqGt7cycCDwOUmrJK3K614l6bSyHUXEyXkIbtuI+Houvp9XPy7F6QWk9hVfM+/K5U0VEd+JiL8DxpCGo/6t2TFUcrLogtz1nEoa3z9C0jaSXifpEEltY8Q/Ab4iaYikwXn9y2vY9ibgZ8BZebtjePUYcqXHSWPe7elUHNl2pE+Dz0saDxxTY702p0galodozgCu7GKcZ0saIGkf0kGr7fsKtcT51fx47gF8siSWah4HhlW5Qua/SSeR30l63mr1E+BfJY2UtC2vnNOo6WopSR+S9JZ8gHuadPL9pUKsxdfEw8DWkg6V9DrSuaCBheVXAadLGiRpGPDZ4r5yD/Bq8jmHiHisjna2beNRoIX0uh4gaW+g2jm2NgcC7yAN0YwlJdd/IX3qRlI/SVuTemNbSdo6t609PwQ+KWlUTlhTSBcIQDphvYmUnAZKmpzLb6m3nV0h6T25l/w6UoJ/nlee0x7jZNFFEfH/gVNJb7xWUpd6MulTGKSrJVpIn2geAP6Qy2oxmdQ1XUW6euKHHax7CTBG0lOSfl5leVfi+AwwTdKzpIP3VTXWa3MF8GtgCbC4ZL9lca4iXWHzF9In25Mj4k91xHkbadjmZuA/IuLXdbblFtInzVWSniyUX0vqIVxbMcRSZiapJ3I76aqh56k4SJcYTbqi6DlS7+aiiLg1L/t3UuJ9StIX84ebzwA/IF31s450crzN2aShp0dIz1e1HtKlpIRYT++p0rGki0NWk57bK0k99NeIiNURsartj3QwX5uHkCD15NYD3wP2ydP/1d6OI2ImKbHfRWrrBuBzedkLpFGC40lXl30KOCKXN9P2pDaszTGuJg039ih1/lyYWTlJS0mXVP6mp2NpNEmLgX/py23NJ6j/BLwpIp7ppm1eCfwpXy1mvZR7FmbdQNJHSePfTR2yaCZJW5F60bO6kijyMMvukraSNIH0Bdafd1OY1iBb7DdCzbqL0m0mxgDHRUSPjy03Qr6o4HHSsMiELm7uTaTzOjuThsE+HRHzu7hNazAPQ5mZWSkPQ5mZWak+OQw1ePDgGDFiRE+HYWa2WbnnnnuejIgh1Zb1yWQxYsQIWlpaejoMM7PNiqR27xLhYSgzMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvVJ7/BbdYZI6bc0NMhdJul5x3a0yFYH+OehZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrFTTkoWkCZIWSlokaUqV5SdLekDSvZLukDQml4+QtD6X3yvp4mbFbGZmSVO+wS2pHzAd+ACwHJgnaXZEPFhY7YqIuDivfxjwTWBCXrY4IsY2I1YzM3utZvUsxgOLImJJRLwAzAIOL64QEc8UZt8ARJNiMzOzEs1KFkOBZYX55bnsVSSdImkxcD7wucKikZLmS7pN0j7VdiBpkqQWSS2tra3dGbuZ2RavV91IMCKmA9MlHQN8BTgBWAnsGhGrJf0d8HNJe1T0RIiIGcAMgHHjxrlXYlYn30jROtKsnsUKYHhhflgua88s4AiAiNgQEavz9D3AYuCtjQnTzMyqaVaymAeMljRS0gBgIjC7uIKk0YXZQ4E/5/Ih+QQ5kkYBo4ElTYnazMyAJg1DRcRGSZOBG4F+wMyIWCBpGtASEbOByZIOAl4E1pKGoAD2BaZJehF4CTg5ItY0I24zM0uads4iIuYAcyrKphamP99OvWuAaxobnZmZdcTf4DYzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjUtWUiaIGmhpEWSplRZfrKkByTdK+kOSWMKy07P9RZKOrhZMZuZWdKUZCGpHzAdOAQYAxxdTAbZFRHxzogYC5wPfDPXHQNMBPYAJgAX5e2ZmVmTNKtnMR5YFBFLIuIFYBZweHGFiHimMPsGIPL04cCsiNgQEY8Ai/L2zMysSfo3aT9DgWWF+eXAXpUrSToFOBUYABxQqHtnRd2hVepOAiYB7Lrrrt0StJmZJb3qBHdETI+I3YHTgK/UWXdGRIyLiHFDhgxpTIBmZluoZiWLFcDwwvywXNaeWcARnaxrZmbdrFnJYh4wWtJISQNIJ6xnF1eQNLoweyjw5zw9G5goaaCkkcBo4O4mxGxmZllTzllExEZJk4EbgX7AzIhYIGka0BIRs4HJkg4CXgTWAifkugskXQU8CGwETomITc2I28zMkmad4CYi5gBzKsqmFqY/30Hdc4FzGxedmZl1pFed4DYzs97JycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWammJQtJEyQtlLRI0pQqy0+V9KCk+yXdLGm3wrJNku7Nf7ObFbOZmSX9m7ETSf2A6cAHgOXAPEmzI+LBwmrzgXER8VdJnwbOB/4pL1sfEWObEauZmb1Ws3oW44FFEbEkIl4AZgGHF1eIiFsj4q959k5gWJNiMzOzEs1KFkOBZYX55bmsPScCvyzMby2pRdKdko6oVkHSpLxOS2tra5cDNjOzVzRlGKoekj4OjAP2KxTvFhErJI0CbpH0QEQsLtaLiBnADIBx48ZF0wI2M9sC1NyzkLRzF/azAhhemB+Wyyr3cRBwBnBYRGxoK4+IFfn/EmAusGcXYjEzszrVMwz1mKTrJB0paUCd+5kHjJY0MtedCLzqqiZJewLfJyWKJwrlgyQNzNODgb8HiifGzcyswepJFiOAm4HTgFWSZkh6Xy0VI2IjMBm4EXgIuCoiFkiaJumwvNoFwLbATysukX070CLpPuBW4LyKq6jMzKzBaj5nERGtwHeA70h6G3AccJmkAC4HLomIRzuoPweYU1E2tTB9UDv1fge8s9Y4zcys+3X2aqg35b/tgcWkK5vmV/uynZmZbf5q7llI2gP4OHAMsA64FPjbiFiel58D3A+c14A4zcysB9Vz6eztwE+AoyLi7sqFEbFU0oXdFZiZmfUe9SSLf4yI2ysLJY1vSx7FcxBmZtZ31HPO4vp2yn/VHYGYmVnvVdqzkLQVoDQp5ek2uwMbGxSbmZn1ErUMQ20EojBd9BJwbrdGZGZmvU4tyWIkqTdxG7BvoTyA1ohY34jAzMys9yhNFoUv2u3W4YpmZtZndZgsJM2IiEl5+r/bWy8iju/uwMzMrPco61k8Uphe3O5aZmbWp3WYLCLi3wvTZzc+HDMz643KhqEOqGUjEXFL94RjZma9Udkw1CU1bCOAUd0Qi5mZ9VJlw1AjmxWImZn1Xp29RbmZmW1Bys5ZPBQRb8/Ty3jlm9yvEhG7NiA2MzPrJcrOWfxzYfrjjQzEzMx6r7JzFncUpm/ryo4kTQC+DfQDfhAR51UsPxU4iXT/qVbgU23fHpd0AvCVvOrXIuLSrsRiZmb1qfmchaQBkqZJ+rOkdfn/OZK2rqFuP2A6cAgwBjha0piK1eYD4yLiXcDVwPm57k7AmcBewHjgTEmDao3bzMy6rp4T3N8DDgA+B7wn/98fuKiGuuOBRRGxJCJeAGYBhxdXiIhbI+KvefZOYFiePhi4KSLWRMRa4CZgQh1xm5lZF9XzS3lHALtHxFN5/kFJdwGLgE+V1B0KLCvMLyf1FNpzIvDLDuoOrawgaRIwCWDXXX2+3cysO9XTs1gFbFNR9npgZfeFA5I+DowDLqinXkTMiIhxETFuyJAh3RmSmdkWr57bfVwG/ErSd0mf7ocDpwDt3o22YEVev82wXFa5v4OAM4D9ImJDoe7+FXXn1rBPMzPrJp253ceXK+b/BfhGyXbmAaMljSQd/CcCxxRXkLQn8H1gQkQ8UVh0I/D1wkntDwKnl+zPzMy6UVNu9xERGyVNJh34+wEzI2KBpGlAS0TMJg07bQv8NP3UN49FxGERsUbSOaSEAzAtItZ0R1xmZlabek5wd0lEzAHmVJRNLUwf1EHdmcDMxkVnZmYdqTlZSNoeOAvYDxhM+l1uwLf7MDPr6+q5Guoi4N3ANGAn4LPAY8C3GhCXmZn1IvUMQ30QeHtErJa0KSKuk9QC/AInDDPbjI2YckNPh9Btlp53aEO2W0/PYivg6Tz9nKQdSN+xeEu3R2VmZr1KPT2L+0jnK24GfksalnoOeLgBcZmZWS9ST8/in4GlefrzwPPAjsDx3RuSmZn1NjX3LCJiSWH6CdL9m8zMbAtQ18+qSvqUpJskLcj/T1T+Bp2ZmfVd9XzP4nzSbcUvBB4FdgO+CLwN+FIjgjMzs96hnhPcnwDeHRHL2wokXQ/8AScLM7M+rZ5hqGfzX2XZM90XjpmZ9UZltygfVZi9EPiZpPN45Rbl/4a/kGdm1ueVDUMtAoLCfaCA91escwDwn90ZlJmZ9S5ltyiv62opMzPrm+q+RbmkXUm/gb08IpaVrW9mZpu/mnsOknaRdBtpaOpnwGJJt0t6c8OiMzOzXqGeYabvke4PNSgidgEGAfOBixsRmJmZ9R71DEO9D9glIl4EiIh1kr5E+k1tMzPrw+rpWawFxlSUvQ14qtuiMTOzXqmeZHE+8BtJ50n6dP6+xU25vJSkCZIWSlokaUqV5ftK+oOkjZKOrFi2SdK9+W92HTGbmVk3qOeus/8laTFwDPAu4C/AMRFxc1ldSf2A6cAHSF/omydpdkQ8WFjtMdItRb5YZRPrI2JsrbGamVn3qilZ5IP9w8CYiLilE/sZDyxqu825pFmkmxK+nCwiYmle9lIntm9mZg1U0zBURGwCNgFbd3I/Q4HidzKW57JabS2pRdKdko6otoKkSXmdltbW1k6GaWZm1dRzNdSFwFWSvk462EfbguIPIzXIbhGxIt+r6hZJD0TE4uIKETEDmAEwbty4qLYRMzPrnHqSRdv9nz5QUR5Av5K6K0g3HmwzjDouuY2IFfn/EklzgT2BxR1WMjOzblM6DCVpm9ybuAE4B9gmIrYq/JUlCoB5wGhJIyUNACYCNV3VJGmQpIF5ejDw9xTOdZiZWePVcs5iOvBh4CHgo8AF9e4kIjYCk4Eb83auiogFkqZJOgxA0nskLQeOAr4vaUGu/nagRdJ9wK3AeRVXUZmZWYPVMgw1gfQLeSslfRe4HfhsvTuKiDnAnIqyqYXpeaThqcp6vwPeWe/+zMys+9TSs3hDRKwEyHeZ3aGxIZmZWW9TS8+iv6T388oPIFXO08nvXpiZ2WailmTxBDCzML+6Yj6AUZiZWZ9VmiwiYkQT4jAzs17MP5tqZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZqaYlC0kTJC2UtEjSlCrL95X0B0kbJR1ZsewESX/Ofyc0K2YzM0uakiwk9QOmA4cAY4CjJY2pWO0x4BPAFRV1dwLOBPYCxgNnShrU6JjNzOwVzepZjAcWRcSSiHgBmAUcXlwhIpZGxP3ASxV1DwZuiog1EbEWuAmY0IygzcwsaVayGAosK8wvz2XdVlfSJEktklpaW1s7HaiZmb1WnznBHREzImJcRIwbMmRIT4djZtanNCtZrACGF+aH5bJG1zUzs27QrGQxDxgtaaSkAcBEYHaNdW8EPihpUD6x/cFcZmZmTdK/GTuJiI2SJpMO8v2AmRGxQNI0oCUiZkt6D3AtMAj4sKSzI2KPiFgj6RxSwgGYFhFrmhH3lmjElBt6OoRus/S8Q3s6BLM+oynJAiAi5gBzKsqmFqbnkYaYqtWdCcxsaIBmZtauPnOC28zMGsfJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZqaYlC0kTJC2UtEjSlCrLB0q6Mi+/S9KIXD5C0npJ9+a/i5sVs5mZJf2bsRNJ/YDpwAeA5cA8SbMj4sHCaicCayPiLZImAt8A/ikvWxwRY5sRq5mZvVazehbjgUURsSQiXgBmAYdXrHM4cGmevho4UJKaFJ+ZmXWgWcliKLCsML88l1VdJyI2Ak8DO+dlIyXNl3SbpH2q7UDSJEktklpaW1u7N3ozsy3c5nCCeyWwa0TsCZwKXCFp+8qVImJGRIyLiHFDhgxpepBmZn1Zs5LFCmB4YX5YLqu6jqT+wA7A6ojYEBGrASLiHmAx8NaGR2xmZi9rVrKYB4yWNFLSAGAiMLtindnACXn6SOCWiAhJQ/IJciSNAkYDS5oUt5mZ0aSroSJio6TJwI1AP2BmRCyQNA1oiYjZwCXAZZIWAWtICQVgX2CapBeBl4CTI2JNM+I2M7OkKckCICLmAHMqyqYWpp8HjqpS7xrgmoYHaGZm7docTnCbmVkPc7IwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlaqaclC0gRJCyUtkjSlyvKBkq7My++SNKKw7PRcvlDSwc2K2czMkqYkC0n9gOnAIcAY4GhJYypWOxFYGxFvAb4FfCPXHQNMBPYAJgAX5e2ZmVmTNKtnMR5YFBFLIuIFYBZweMU6hwOX5umrgQMlKZfPiogNEfEIsChvz8zMmqR/k/YzFFhWmF8O7NXeOhGxUdLTwM65/M6KukMrdyBpEjApzz4naWH3hN4wg4EnezqIHtKUtusbjd5DpzW8/Vty22HLbn8X275bewualSwaLiJmADN6Oo5aSWqJiHE9HUdP2JLbDlt2+7fktsPm3f5mDUOtAIYX5oflsqrrSOoP7ACsrrGumZk1ULOSxTxgtKSRkgaQTljPrlhnNnBCnj4SuCUiIpdPzFdLjQRGA3c3KW4zM6NJw1D5HMRk4EagHzAzIhZImga0RMRs4BLgMkmLgDWkhEJe7yrgQWAjcEpEbGpG3A222QyZNcCW3HbYstu/JbcdNuP2K314NzMza5+/wW1mZqWcLMzMrJSTRTcru61JXudXkp6SdH1F+QGS/iDpj5IuzVeF9WqShku6VdKDkhZI+nwu30nSTZL+nP8Paqf+JZLuk3S/pKslbZvLd5N0cy6fK2lYM9tVD0n9JM1vez7zhRx35dfAlfmijmr15ubXyr357//k8s2p7Tvm5+1Pkh6StHcdz/1vC23/i6Sf5/JBkq7N7b9b0jua2qgOSPrX/Dr/o6SfSNq6juf7XEnLJD1XUb553OooIvzXTX+kk/eLgVHAAOA+YEyV9Q4EPgxcXyjbivSlxLfm+WnAiT3dphravAvw7jy9HfAw6ZYu5wNTcvkU4Bvt1N++MP3NQp2fAifk6QOAy3q6rR08BqcCV7Q9n8BVwMQ8fTHw6XbqzQXGVSnfnNp+KXBSnh4A7Fjrc1+xnWuA4/P0BcCZefpvgJt7up05lqHAI8DrC8/zJ+p4vt+b3y/PVZR/Brg4T08ErszTY/IxZCAwMh9b+vVY+3v6CehLf8DewI2F+dOB09tZd/+KZDEEWFyY3weY09Nt6sRjcB3wAWAhsEsu2wVYWFJPwPeA0/L8AmB4YdkzPd22duIeBtycD+rX51ifBPpXe01U1G0vWWwubd8hHzxVUV7vc789sJb8wQG4AdinsHwx8MZe0N62u0zsRLqS9Hrg4Fqf78J2KpPFjcDeebp/3p4qjx/F9Xriz8NQ3avabU1ec2uSdjwJ9JfU9u3OI3n1lxF7vdx93hO4i/TmXpkXrQLe2EG9H+Z1/gb4bi6+D/hInv5HYDtJOzcg7K66EPgS8FKe3xl4KiI25vmy18AP8zDMV/O90GDzaftIoJXUhvmSfiDpDdTx3GdHkHoPz+T5l9svaTzpFhQ9PhQXESuA/wAeA1YCTwP3UN/zXc2rbnWUt7szXTuedDsni14i0keHicC3JN0NPAtsNt8nyecargH+X+FND7zctnav0Y6ITwJvBh4C/ikXfxHYT9J8YD/St/Z71eMh6UPAExFxTyc3cWxEvJPUi9wHOC6X9/q2Z/2BdwPfi4g9gXWkYaeXlT332dHATwrz5wE7SroX+Cwwn17Q/nzu5XBSknwz8AbSnbC3CE4W3avarUlaCyfxDuuockT8PiL2iYjxwO2k8f9eT9LrSInixxHxs1z8uKRd8vJdgCfy9I35sfhBcRuRvmg5C/honv9LRHwkH4TOyGVPNaM9dfh74DBJS0mxHwB8m3Sga7s4YRiwIp8Eb3sdTIOXP6kSEc+SznmMz/ObQ9shfdJdHhF35fmrScmj5ude0mBSu29oK4uIZyLikxExFjieNES7pAntKXMQ8EhEtEbEi8DPSK+Bmp7vDmwWtzpysuhe1W5rcnVEjM1/lbc4eZXC1TADgdNIJ8t6tTx0cgnwUER8s7CoePuWE0jnMoiIg/NjcZKStxS2cxjwpzw/WFLb6/N0YGbjW1OfiDg9IoZFxAjSc31LRBwL3EoaRoTc9ojYVHgdTJXUPx8o25Lth4A/5vle33aAiFgFLJP0tlx0IOlOC6XPfWEzR5LO3T3fVpCvsGq7ougk4PbK3moPeQx4r6Rt8uu1rb2lz3fJdjePWx311MmSvvoH/AOpR7AYOKOddX5LGutdT/p0dnAuv4A0FLOQNJzT4+2pob3vIw0z3A/cm//+gTTmejPwZ+A3wE5V6m4F/A/wAOlA+WNeOcl5ZK77MPADYGBPt7XkcdifV66GGkV6Uy8iXdn0mthJQxj35MdtAalH0m9zazswFmjJ7fg5MKiW575Qfy4woaJs79z2haRP74N6up2F2M4mfaD5I3AZ6Uql0uc71z0/v99fyv/PyuVb53qL8nZGFeqckY8lC4FDerLtvt2HmZmV8jCUmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknC7NeRtJSSQf1dBxmRU4WZnWSNDHfSnqdpCfy9GcK93Yy63OcLMzqIOkLpC/QXQC8iXSTvJNJt314ze8YSOrX1ADNGsTJwqxGknYg/c7IZyLi6oh4NpL5EXFsRGyQ9CNJ35M0R9I64P2SDs13ZX0m//jNWRXbPU7So5JWSzqjYtlWkqZIWpyXXyVpp+a12ixxsjCr3d6k2ztcV7LeMcC5pB+DuoN0N9bjST8MdCjwaUlHAEgaQ/odj+NIdzLdmVffjvuzpFt475eXrwWmd0NbzOriZGFWu8HAk/HKbxcg6XdKP5G7XtK+ufi6iPifiHgpIp6PiLkR8UCev590O+798rptN9K7PSI2AF/lld/GgDTEdUZELM/LzwKO1Gbwk7vWt/gFZ1a71cBgSf3bEkZE/F8ASct55cNX8QdrkLQX6Tca3kE6rzGQdOM4SL2Fl9ePiHWSVheq7wZcK6mYQDaRzpX02O2qbcvjnoVZ7X4PbCD9AE5HKu/OeQXpdtPDI2IH0q3n266cWknhNwskbUMaimqzjHS30R0Lf1tH/i0Ms2ZxsjCrUaQfIDobuEjSkZK2yyegx5JuOd6e7YA1EfF8/pnQYwrLrgY+JOl9+TccpvHq9+XFwLmSdgOQNERSWbIy63ZOFmZ1iIjzgVNJv7v9eP77PunHqn7XTrXPANMkPQtMBa4qbG8BcAqp97GSdAJ7eaHut0m9kl/n+ncCe3Vjk8xq4t+zMDOzUu5ZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSv0vPBheekrbYscAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "face = [ '0-19','20-39','40-59','60-79','80-100' ]\n",
    "probs = [ 0, 8/86, 32/86, 30/86, 16/86 ]\n",
    "\n",
    "plt.bar( x=face, height=probs )\n",
    "plt.ylabel( 'Probability', fontsize=12 )\n",
    "plt.xlabel( 'Grade', fontsize=12 )\n",
    "plt.title( 'Conditional probability for studying 41-60 mins', fontsize=12 )\n",
    "\n",
    "axes.set_ylim( [0,.5] )\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that *time studied* and *grade* and not independent because the conditional probabilities and the marginal probabilities are different. If the distriutions has very similar probabilities, we could say that they are independent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O5k98pyDNnmZ"
   },
   "source": [
    "## Venn Diagram Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 239
    },
    "id": "f7FETEIKU5LG",
    "outputId": "b6506950-4b13-4763-c32f-da66d9d5df6a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Relationship of A to B')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADNCAYAAAAbmaNAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjA0lEQVR4nO2deZxdVZXvv6uqUnNmMlcGphACiYwyqjxFEQGhHZpuFejPp2lfv8+zfTxF6X7dj2s5D61P1Nb22W1jA07wEBkVsE2UoARImJIQMo9knio1D+v9sc9tKpUablXds/c5567v53M+VXXvuXevOvee31ln7bXXElXFMAzD8ENZaAMMwzBKCRNdwzAMj5joGoZheMRE1zAMwyMmuoZhGB4x0TUMw/CIiW6JIiJLROTmEb52jogcFZHyYtvVa4zPiMjdgzy/SkQui2HcaSLyOxFpEpGvF/v9DcNEN8WIyGYRaY0EcJeI3Cki9TGNc3n+b1Xdqqr1qtpd7LEKRVXPUNUlMbz1R4F9wDhV/eRAO0UXBRWRCwZ7s6EuHkMRfaYd0WfcJCLPi8jbRvp+RnhMdNPPNapaD5wFnA38XVhzUs9cYLUOsmpIRAS4ETgQ/Yybr0af8Tjge8D9cd5lGPFiopsRVHUX8Guc+AIgIheKyNMickhEXhzodlxEThaR/xCR/SKyT0TuEZEJ0XN3AXOAhyJv69MiMi/y8iqifWaKyIMickBE1ovIX/V678+IyM9F5N8jT22ViJzX6/nbRGRH9NxaEXlHL9MqB3ndf3rf0Rj3icjPon1XiMibBjpWInKxiDwrIoejnxdHj98J3AR8OvpfLx/gLd4CzAA+DvyZiFQOMM67gf8FXB+934tDHa/BiC4EPwYmAdMKeY2RPEx0M4KINABXAuujv2cBjwCfx52ktwL/T0Sm9Pdy4EvATOB0YDbwGQBVvQHYSuRRq+pX+3n9T4Ht0es/AHxRRN7e6/n3RvtMAB4EvhPZeBrwMeB8VR0LXAFsHup1A3AtcG/0v/4YeEBExhz3j4pMwh2XbwGTgW8Aj4jIZFX9C+AeIs9SVZ8cYKybgIeAn0d/X9PfTqr6K+CLwM+i98tfCIY6Xv0Sebc3ApuA3UPtbyQTE93084CINAHbgD1ALnr8I8Cjqvqoqvao6hPAc8B7+r6Bqq5X1SdUtV1V9+KEqKC4oYjMBi4BblPVNlV9AfgXjr3tfiqyoxu4C8iLTzdQBSwUkTGqullVNxTwuv54XlXvU9XOyP5q4MJ+9rsKWKeqd6lql6r+BHiVAYSzn/+3Fvgg8ONorPsYRoihwOPVl1tF5BBwFPgm8L9DxtON0WGim36ui7zEy4AFwAnR43OBD0ahhUPRSXsp7rb4GKIZ+59Gt/lHgLt7vc9QzAQOqGpTr8e2ALN6/b2r1+8tQLWIVKjqeuAWnFe9J7Jh5lCvG8CObflfVLWHNzzJ/uzd0uexvvYOxp8AXcCj0d/3AFcOcAfRH4Ucr778o6pOAGqB84CviciVBY5nJAwT3YygqkuBO4F/jB7aBtylqhN6bXWq+uV+Xv5FQIFFqjoO5yVL77cfZOidwCQRGdvrsTnAjgLt/rGqXoq7SCjwlUJe1w+z87+ISBnQENnWn71z+zxWsL240EI9sFVEduFCGmOADw2wf99jN+LjpY5XgGU4j91IISa62eKbwDujSaS7gWtE5AoRKReRahG5LIr99mUs7tb1cBQL/lSf53cDJ/U3oKpuA54GvhSNsRj4y2j8QRGR00Tk7SJSBbQBrUBPQf/p8ZwrIu+LPOFbgHbgj/3s9ygwX0Q+JCIVInI9sBB4uAB7ZwHvAK7GTViehQt5fIWBwwO7gXnRhWBUxyuyYQHujmVVIfsbycNEN0NE8dh/B26PTu5rcbPne3Ge76fo/zNvBM4BDuMmme7v8/yXgH+IwhS39vP6Pwfm4by4XwC5QSahelMFfBmXF7sLmMrIU95+CVwPHARuAN4XxVyPQVX340Tzk8B+4NPA1aq6r4AxbgBeUNXHVXVXfsNNyi0WkTP7ec290c/9IrIi+n24xyufTdEMPA78G/D9Auw1EohYEXMj7YjIZ4BTVPUjoW0xjKEwT9cwDMMjJrqGYRgesfCCYRiGR8zTNQzD8IiJrmEYhkdMdA3DMDxiomsYhuERE13DMAyPmOgahmF4xETXMAzDIya6hmEYHjHRNQzD8IiJrmEYhkdMdA3DMDxiomsYhuERE13DMAyPmOgahmF4xETXMAzDIya6hmEYHjHRNQzD8IiJrmEYhkdMdA3DMDxSEdoAI3tIo4wBJgLjgPpoq4t+VgL5xnz9/cz/3gocBg5FPw8DTZqzpn5GurHGlMaoiAR2FjADJ7QTcQIbBz3AEd4Q4YPADs3p0ZjGM4yiY6JrDBtplAnAnGibTvgw1QFgK7AF2GPesJFkTHSNIZFGqcB5s7OjbWxYiwalDSfAW4HtmtOOwPYYxjGY6BoDIo0yA1iEE9rywOaMhB5gF7AeWKc57Q5sj2GY6BrHIo1SBpwILAamBDanmLQBrwKrNKfNoY0xShcTXQMAaZRKYAFwJi7LIKv0AJuAlZrTA6GNMUoPE90SRxqlHie0C3DpXKXEZmCF5nRfaEOM0sFEd6SICC41ahxQjYt5lvPGTH5Pr60daAKa0GTEFaVRqoALgPmEzz4IzVZguXm+hg9MdIdCpBaYihPX/DY22kYiVi24XNOmaDsC7EP9nfDSKKcAFwE1vsZMAT3ACzjPtyewLUaGMdHti0gFLve0IdomeRq5BdgJ7AC2o8Wf7JFGGQtcistGMPrnELBUc7o7tCFGNjHRBRCZhEv0bwCmkYz0qMM4Ad4GbENH7n1Jowgu9es8bOl3ISiwChdy6AptjJEtSld0RcqBk4GFuPBBkmnBpTu9ig5vyas0ygnAW4ET4jAs4xwFfqc53R7aECM7lJ7oiozFCe1puAmwNKG4SZ/VqG4bbMco3/Z8XL6teLAty7wG/EFz2h7aECP9lI7oiszBie1ssiFCR4A1wBr02KWu0ijVwOXAzBCGZZQm4DHN6aHQhhjpJvuiKzIduJDkhxBGShuwAuf99kijTAKuINn1EdJKO/C45vT10IYY6SW7oisyHpeHOi+wJb44cse7WX3LhTZZFjM9wBLN6frQhhjpJHui61K+zgbeRAkl/b92Bs2vLaZuQxVtX59B2baqkltd5ptnNacrQxthpI9sia6L215Cid1av3ABzdtPeqNweDforyfQ8i9TqOksK50LTwBeBZ6yxRTGcMiG6IqU4VZYnRHaFN88dyktu2ZT299zu8bQfnsDZa9XMsa3XSXENuBJzWlnaEOMdJB+0RWpA95JdifKBuSFC2nZfmL/gpunXei+Yzodvx9nS35jZD/wkBVMNwoh3aIr0gC8nfTl246aV86hefNphfcie3w8zd+dRm23ZCJdLonsBB61UIMxFOmN94mcA1xJCQru2kXDE1yAdx2m7pubaZ/UhS1rjYeZuJV/hjEo6fN0RSqBd1CiRVs2nkbL6nMGDykMRnMZXV+YRdfLtaV3sfLEc5rTFaGNMJJLukTXCe57KMH4LcD2ebS8cCE1jDJE0Ak9n2ugY2WdCW9M/FZzui60ET4QoQaYgKstXRttNbiiUfnvaQ/Qgashkt+OAgdVKblwTHpEt8QF98h4On5/BRVaXpyQkAlvrPQAj2Rt5ZoIZcAMXCW+KbgiSsMKc/WhGzgA7AX2AdtVGVZBpzSSDtEtccHtKqdnyVV0t9UVN/XLhDdW2oFfpr1WgwhVuFDePFzp07gX3ewHtgBbVNkb81hBSL7olrjgAix/Ky17Zo08jjsYJryxcgR4QHPaFtqQ4SLCNFyBqJMIV1/6ELAaeE2VzKTjJVt0TXDZNJ+WVefGI7h5THhjZavm9FehjSiEKHxwKm6RUZLqL3cB64EXVTkc2pjRklzRdavMrsa1zilJDk2kfdm7GKMelvJ2Qs9tc+hcV0NV3GOVIEs0p6+FNmIwRDgJV395fGhbBqEHWAs8r0pLaGNGSpJF92Jca/CSpKucnt9eTVd7rb/CNQfL6fzrEylrKU9Eu6Is0QHcq7ni970bLSLMwJU+nRLalmHQBbwCrFQldcuvk7k4QuQkSlhwAV49i1afggswsZsxt+/ITuwsQVQCbwttRG9EGCPCpcA1pEtwwZUuPQv4oAizAtsybJInuiITSNgX1DdHx9Kx+ZR447gDcUYrNTfsJXEeWQZokEaZH9oIgEioPoibKEsz9cBVIlwqkp6iTskSXVcL93JIzwGMgxcuopuycDUSPnCA2rObSd2Mewq4QBolWJ1jEUSENwNX4QQrKywE3i/CpNCGFEKyRBfeAuk4cHGxYy6thyaHrQhWBnLbTiqsTkPRqQHeHGJgESpxbZzOCjG+B8YB14okv1NMckRXZAEuXaVk6SqnZ9U5yWi1U9dDReM2E90YOF0axWs6lgjjgeuAOT7HDcAY4F0inBPakMFIhuiK1OBmUEuaV8+itaM6OaGVeR1Uv/cAraHtyBgCXOxtMGEycC2uPkKpcJ4Ib5WEljFNhug6wS3pnl5tNXRtOSV5hcZv2EfluC66Q9uRMaZLo0yLexARTsDlupfiopcFwNuSKLzhRVdkBiUeVgBYu4h2H4sghku1Uv6x3bSHtiODLI7zzSMP9yoo6cUu83HzRIki7Eku4vVWK6m0V9G9fV7yvNw8Fx2l9tRWE94iM08aZVwcbyxCPW75fCkLbp4FUcZGYgjtWZ0KTA5sQ3DWL6StWCUb4+K/7yahSxdTixCDtytCBS5LIbEX8QCcJcIpoY3IE+5EFykHzgs2fkLoLqNn68nJj7md3E71RU02qVZk5kujFPuzvwxzZPrjbSLJWHkX0rs6g2wlaI+IrafQ1j0mHbUO/nJPOuxMERUUcVWYCGfjSjEax1OOSycLfgcQRnRdLLekayvk2XhaMvJyC2FaF5W2Uq3onCmNMuqLWZSpcG4R7MkydcCloY0I5enOwbxcDk2ivbU+Xaly7ztgsd0iU42bZR8xUR3cywg/R5MGThTh5JAGhPqQ0l5ooyhsPSl9+a+LWqi25cFF54xRvv4cSnz5/DC5JGSYwb/oioyjRNun92XX7OSsPiuUcpA/OWDpY0VmkjTKiO78RJhAduspxEU1cFGowUN4uublAgcn056kJb/D4fLDVJWrhRmKzEjrIrwZCyuMhFOiBSTe8fthuTSxRNQUDc22FIYW8tT3UPHWI5Y+VmSGLboiTIXkV9VKMEEWTfi+Qp5Maa4DP45dDen0cvNce9C8qyIzcwRZDIlaaZVCZkftirzi+8SxHELgwAnpDS3kObGdKiuEU1QqgJmF7izC9OHsbwyI9zQ7f6Lruvt6v6okkV0N6Z/9LwM5t9km1IrMcEIMo814MBwzo8lIb/j0dKdS4m148hw8IRu35hccDW1B5ihIdKN0pxNjtqWU8Dq57/Pkb/A4VqI5MiFdCyIG4oxWu4gWmbHSKBML2O90LGOhmMyPCgV5wecHl7pWyXFwdCwdaam1MBQTuhkzo4PO0HZkjEJy2E+L3YrSohKP801+RFekEpJR4Sc0+6dla/LpwqMmukVm0P5pUcfbsZ5sKSXm+hrIl6c7w+NYiWb/lGwtKjj/aPLaoaScoQqbexOHEqNBxM8dqC8htNBCxKHJ2Qgt5Jnflo34dIIYP8TzJrrxMAZPKXi+RHeCp3ESjYK21GVLpKqU8glWAKeYVEmj9NtmR4RqXBaQEQ9easL4El2LQQFtNXRTlr3b8Zkd2YpTJ4CBvF2bF4kXLxe0+EXXFSw30QVa67IpTrNMdIvNQHHdQSfZjFEzOapNHCs+PN06T+MkntY6ekLbEAcNHdmaHEwA5umGoRwoJE96VPgQQ/NyI1rqsilOMzrtolpkBhJd83TjJ/Zj7ONkGSoFpmRorQttQTxM68xenDowx50z0W1vybe48kDsemWerkdaa7MpTpO7spUGlwD6O/FrvVtRmsR+nH2sN/Z+dT4XbloDi2qhaR80AlwB710BZwloPRy5B+68CA77tKu9Jn7R/eN2pn1nOR/N/93cyQkXNvDgbZfwm7jGHNtNhSiqkrCLyve5ib0sYgxN3Oa+B7xIA0/yYbqpopr93MC/MjFxHY77Oy9TILrn3gRrFkFtE+xzx5tPnAs/ugYOTIcffAlu3hLWxiGJ/Tj78HS9txj/MDz9T/Ct3o/9Mzy+Fz67Bz53Abx8C1zt266uivhF6cIGdt/9Pj539/v43A+v5fPlQseVp7AyzjHLQCqTGK1ezNNcdez3gCe4kYu4n0/zWeaxkod5VyDrBqO/O4cUiO6Hn4Z/OvZ4c8kO+Nfvwex1YWwaNpkQXe+TLJ+AdbOhufdjJ/KGN9MClYL/SS3fnuAv1nB6fSV7F0/jQNxjlWsCMzMuYh3jj/0e0MpULsQJwGLWsJNzQpg2BCKN0ve7koKOK59YB7OPPd68fxdctzuMPSMi9uOcSdEdiHfCdWPhy8vggu/Dg77H9y26f9jO+WdOZbmPsSqS6On2Ry2v85uoe+5yzqU9sa3L+543iTmPMk4m8nQT82V5Ah5ogr+9BJ65Df6LdwM8Sm5zB+VbD7P4ugU872O8igB3DiPiPfyIVbyNr/H3dFCNJHYJc9/zJlnx8uwS+3H2EW9N3G3n/4TlH4C/AR7yOrBHWbpvNWdOqmHrKZNo8jFeV1pE4XR2cTp3ALCWqexiUWCLBqLveZOOi1r6if04+/BCEyG6D/daV/1/4U1TYZdvG0T9nTjLd/LmN03nWV/jdaVDcuH1KIWxG+G3XMV8lga2aCD6njeJOI9KgNiPcyY93UVw80aY3wb19fCV6+HBp2HRX8A0AZ0I+/8N7vFtly/R3d9C5etNnP4Pb+FuH+MBdHtYsz5svsvNHGQ+XdTzBb7CmTxIB1VsiEJLM1jB1Twd2Mr+UM1p3+9K0tLa+mHRzbBxPrTVQ/1X4PoHYXIzfPfPobUePv438MVtsPGO0JYOQuzHWfS4z7bYI8hlwPx4B0kHv3s3bUcmpmEWenj0gF433/9EYYbp0pz+sPcDIkwFrgtjTkmxTZXH4hzAh3diPWMjqlqzGZdrKqfLBLeo9De51+LditIk9uPsQ3S9TOSkgZqWbIruvgqLNxaZI/08ZqLrh+ahdxkdPkS3vy9QSVIT+8cZhj1jrJ5ukTnunFG3+MTuGuMndifRPF2P1DZn8xZ855jQFmSOgWqC7PNqRWmyN+4BfIhuM5buAkBNcwJn+IvAjkx1fUsEA4lu7IJQ4nQDh+IeJH4RcOkR5u0CNc3+i//4YEellXYsMgOF5MzTjZf96qGGiC/Py0QXqG6ljJ7sTabtNNEtNubphmGPj0F8ie5BT+MkGgGpbaYjtB3FpE3oPlSRTQ8+EO2a0/b+nlClDU/CUKJs8zGIL9Hd6WmcxDNhf7Zm+tfWZOsikgCGKqyf9CLgaaUT2OFjIF+i+zo2mQbA5Iz5Kc9ntNlmQEx0w7DdRzwXfImuagd2WwTA5D1kKsHqj/XZ+n8SwP7BnlTlAJb7HgebfQ3kM4XJi+uedOqbGFPemY0Qw6FyOl+vNNEtMlsL2Gdt7FaUFh3AJl+DmegGYNyhbMRBX6lNbAHwtNKkOT1UwH6vYuG6YvKaqr/vsk/R3YMLVpc8E/dlIw663OK5xaYQLxdVWvHomZUAq30O5k90VXuwLAYAZmxLf15rD+iz9VSFtiNjFCS6Eatis6K02KEa/yq03vhelrrR83iJZOJ+qirb0u31b6qi/Wh5+i8eCaKLYTglquwazv7GgHjpIdibEKKbggr48TN9e7rjur+YZKGFIrNTczrcCdZnYrGkdNgaXby84ld0VbuxmVcAZm9M76x/UxldT43NXgeMwAwntACAKnux2O5IUWB5iIFDVL1ag3U2ZeJ+Kqta0xlieHI8Hd3WKaLYDFt0I57FMhlGwvoo59k7/kVX9Qiw3fu4CSSNIYZu0AcmYcUci8t+zemICpRHk0Ari2tO5mkD/hBq8FD1XW3mFZi9MX2FYl6qpfWAFbgpNqNNWVrJECvZjGN4KioeFIRQorsNaz3ChANU1RxNl7f7i0nZLMQekFbgtdG8QVQzYAkWZiiEjaphs6jCnECusPkrQcZOGCe9mp5VXbsr6FhZZxNoRWbVCLIWjkOV/cBzRbAnyzQDT4U2IqTXsgrzdpmzgeqKjnQI7w+mZqNmRILoooiroVR5AdhQrPfLGF3Ar0OGFfKEE12XPvZssPETQnkPZbM30m/R6iSxroq2Z8ZSE9qOjLFWc1psEViKtfXpj6WqyTguYeNzquuwLwinrqK6rDvZ8bjvTbMUsSKjwMtFf1NXuOVxXKzYcKxUTc4dQBImRZ4ObUBoKjsob9iU3JNkWT0t62qszkKR2aw5jaUuripHgUew1Z8Aa1STdUcdXnRVdzHK2dsscNrLVEkCvd0Wofu700xwY+DFON88Svx/FJIfuoqRtar8PrQRfQkvuo5nIF2pU8Wmqo2KueuT5+3ePYX2IxVW2KbIvK45jb2TShTDfIjSDDWsVmVpaCP6Ixmiq9pKwBUiSWHBi9QkqfrYxiraHppIbWg7Mobi8bseeby/pHQ6ciuwXDV8athAJEN0AVTXUuJhhopuys54PhnpY0fL6GpssJVnMbBac+p18liVI8ADZL+pZSfweJQ6l1iSI7qOpyBMEYqkMGsrNRP2hb0d7AH98ky6bLlv0WklUJqkqhMkslun4QjwgGryLyzJEl3VLuAJSjy+e/YfqJCecJNqP59My4u28iwO/qg5DfbdVkWjmfyHgaZQdsTAKuA+1XSEUJIlugCqhyGZAXBf1B1lzNx1Ybzdl2tovecE6kKMnXG2aU7XhTYCQJWdwL2kv/BUE/CwKst8NpYcLckTXQDVTcSQOJ4mFrxITXWLX4//YDmdn59lZRtjoAP4XWgjeqNKlyrLgAdxTWPTRBcuTHJvdAFJFeJqzyQQkTLgamB6aFNCcXgCHU+9iwotj//i2Ak9n5pL54Zqy8mNgSWa00RPEoswD3gzMCGsJYPSg2s/v0KVltDGjJTkii6ASCVwJTAttCmh2HQqLavOizdtqxN6PtdgFcRiYovm9NehjSgEcd1ATgXOAKYENqc3ncA64KUoEyPVJFt0wYQXePYttOxuiEd4O6GnsYEOmziLBTejXvyiNrEjwhSc+J4MwRbHHMTFnddF2ReZIPmiC3nhfQ8wNbQpIegqp2fpe+hqrS9uvNUEN1bacYJ7OLQho0GESmA2MDf6GXf4aR+wGdepN5PFsNIhulDywts0jo7fv5uKniLFd01wY6UbeERz6r29d5yIUIa745yGCz9MAepH8ZZduDZD+4C9wA5VmkdrZ9JJj+hCyQvvjrm0rryIakbZidcEN3b+Q3O6PrQRPhChGjf5VttnK4s2xU2AdQAtvbajwCHV0usMni7Rhbzwvh2YE9qUEGyaT8uqc0ce320uo+sLs+h6udYENyae05yuCG2EkVzSJ7p5RM4GzoPSK6792hk0v7Z4+AsYNlbR1thAhS3vjY21mtOSXthjDE0yF0cUgupKXKHmkitbN38VdSe+OrzY12PjabllLlUmuLGxA5JXu9VIHun1dPOI1AKXU4KLKF64gObtJw3u8bYL3f9nBh3LrL9ZnOwDHg5ZV8FID+kXXcivXrsAWBTaFN88dwktu+b0H+PdOYb22xso213JGN92lRBbgd9oTjOTR2rESzZEN4/IbOASYFxoU3zy0vk0bz3lDY+3G/SxCbT8cAo1nWUpDiEln9XAMs1l6SQy4iZbogsgUg6cFW0l02Zm3UJa1i6m5rUa2r4xnfIdVVa4Jmae0ZzG2ufMyCbZ84JUu1F9Hle6bmNoc3xx6mo61x/imU/OpcIEN1a6ceGEVAmuiGwWkT0iUtfrsZtFZEmBr18iIjcXsF+9iBwVkcdGYW5BRP9TazTeQRF5RNzdbqLJnujmUT2C6pO4NiWZWhnUh1ZgGXDvLY/pS7h+WKkvCpJQ2nErzTaENmSElAP/I+Yx3o87Tu8UkaJMbotjIK26RlXrgRnAbuDbxRgzTrIrunlU96D6IK4d9WbIzAqYw7gGhz9FdRWqPQCa0wPAL3ApTEbxyBevSfMF/GvArSIyob8nReRiEXlWRA5HPy+OHv8C8BbgO5FX+Z1BxrgJ+GfgJeAjgxkz0HjRc0tE5Asisgy3gu2kwd5LVduA+4CFg+2XBLIvunlUt6P6OPATXAHkNOb39gCbgEdQ/RmqL6PHz5prTttxF5kXotcYo+NV4P60F68BngOWALf2fUJEJuHy3r8FTAa+ATwiIpNV9e9xOcgfU9V6Vf1Yf28uInOBy4B7ou3GgQwZbLxeu90AfBQYyxBNNcWljl4P/HGw/ZJA6SXKqx4FnkXkedzVcyHJz/Ftxp34a1AtqHhzNKO+XBplA/BWklUfNS0cAX6vOc3SXcPtwDIRuaPP41cB61T1rujvn4jIx4FrgDsLfO8bgJdUdbWIHAa+KiJnq1vI1JdCxrtTVYdqKfSAiHQBdbiiOVcUaGswSk9087jb8fXAekQm4srWNeAEOAnH5SAuRLAd2J4PHwwXzel+aZQHcLVRzwfL2S0ABV4BntWcpqb3ViGo6isi8jDwt8CaXk/N5HhvcgswaxhvfyPwg2icHSKyFBdu6E90CxlvWwFjXqeqT4rLWroWWCoiC1WTGwZKgriER/UgTuReilLOpuM+/AbgBE9WNONE1m0FerSFEHm9r0ijbAYupUSLBRXIQWCp5jRtfcOGQw5YAXy912M7cTVzezMH+FX0+6BzIVE89lTg70Tkk9HDY4EzReRW1eMuXkONN+SYvVHVbuB+Efk+7jt+X6Gv9Y2Jbl/ch5cXv+WIVONKSY7rtY2NtuEeP8WJa1O0HYl+7kX1UDHMH3TwnB4FfiWNchJwMcTbBihl9OA8spWaG9ldRVpQ1fUi8jPg47zRAPZR4Nsi8iHg57gshIW4du3gMgMGm8y6CXiCY+O4NbgJtSuBh/rsP9R4w0JEBHgvMJFjPfjEYaI7FG5WdGu/z7ng/Tjcl6sMl5JThqt81oPL6czXEj0CHB1pmKCYaE43SqNsxzUiXEApTaj2zxZgueb0YGhDPPJZXAwWAFXdLyJXA3cA38OF3q5W1Xz3hjuAH4nIfwPuUtWP518rzjH5U+DGvrf1InIXTpCPEd0CxiuUh0SkG+fQbAFuKiAOHJTsrUgzhoU0Sh0u3rsQSm5RxSZgheZ0f2hDjNLBRNcAQBplDHAarmjQ2MDmxEkPbqXiyhLzbI2EYKJrHIM0igAnAovJVlukVlysb7XmijdJaRjDxUTXGBBplGk4z3cO6Yz/dwOv4+KFGzSn3YHtMQwTXWNopFHKcXmVs3ECnOTSmS24/M4twA6rc2skDRNdY9hIo4znDQGeQfgSmvtwGSZbM55fa2QAE90iE5XKexMwXVXbA5sTO9IoFTgveCauFfdE4puI68YV+slvh3De7LD6xRlGSEx0i4iIzAM24AThv6rqvWEtCkMkxBOA8bg18fXRVgdU4XIq81+83r/3/ruVPgIbLe4wjFRjoltEROR2XMGNZ4D5qnp1YJMMw0gYJrpFRETW40rUPYMrMdegqrvDWmUYRpIo9eWfRUNELsUV8Pi5unZBG4APhbXKMIykYaJbPG4CHu+1dvzH0WOGYRj/iYUXioCI1OD6sJUD+cmeKtxk0lmq6WpiaBhGfJinWxyuw6UzLeSN9u+n41qcDNiyxDCM0sM83SIgIr8CVqnqJ/s8/qe4HlAN/RRxNgyjBDHRNYIgIpuBabg7hE7gaeCvVbWQFi2GkVosvGCE5BpVrcctJd4NfDuwPYYROya6RnDUdee4DxcTN4xMY6JrBEdc26PrcQtKDCPTpLFGqpEdHhCRLlxNhr24JdSGkWnM0zVCcp2qTgCqgY8BS0VkeliTDCNeTHSN4Khqt6rej8tkuDS0PYYRJxZeMIIjIgK8F1eLd01gcwwjVkx0jZA8JCLduPq5W4CbVHVVYJsMI1ZscYRhGIZHLKZrGIbhERNdwzAMj5joGoZheMRE1zAMwyMmuoZhGB4x0TUMw/CIia5hGIZHTHQNwzA8YqJrGIbhkf8PCZX7EnB3Pp4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Example problem from: https://www.slideserve.com/lucine/conditional-probability-in-venn-diagrams\n",
    "\n",
    "#Import libraries\n",
    "from matplotlib_venn import venn3\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "# defining sets for both subjects\n",
    "A = {'A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A12', 'A13', 'AB1', 'AB2', 'AB3', 'AB4', 'AB5', 'AB6', 'AB7'}\n",
    "B = {'B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B9', 'B10', 'B11', 'B12', 'B13', 'B14', 'B15', 'B16', 'B17', 'B18', 'B19', 'AB1', 'AB2', 'AB3', 'AB4', 'AB5', 'AB6', 'AB7'}\n",
    "C = {'C1', 'C2', 'C3', 'C4', 'C5', 'C6', 'C7', 'C8', 'C9', 'C10', 'C11'}\n",
    "     \n",
    "# using Science instead of Math\n",
    "venn3([A, B, C], set_labels = ('A', 'B','Not A or B'))\n",
    "\n",
    "# plot title\n",
    "plt.title('Relationship of A to B')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Vh0gt9ZcW94S"
   },
   "source": [
    "Based on the previous Venn Diagram above, here are the various probabilities:\n",
    "\n",
    "$\\text{Total Entries} = 50$\n",
    "\n",
    "$p\\big(A\\big) = \\frac{13+7}{50} = \\frac{20}{50}$\n",
    "\n",
    "$p\\big(B\\big) = \\frac{19+7}{50} = \\frac{26}{50}$\n",
    "\n",
    "$p\\big(A \\cap B\\big) = \\frac{7}{50}$\n",
    "\n",
    "$p\\big(A \\cup B\\big) = \\frac{13+7+19}{50} = \\frac{39}{50}$\n",
    "\n",
    "$p\\big(A | B\\big) = \\frac{7}{19+7} = \\frac{7}{26}$\n",
    "\n",
    "$p\\big(B | A\\big) = \\frac{7}{13+7} = \\frac{7}{20}$\n",
    "\n",
    "<br/>\n",
    "\n",
    "And here are some \"not\" scenarios:\n",
    "\n",
    "$p\\big(B'\\big) = \\frac{13+11}{50} = \\frac{24}{50}$\n",
    "\n",
    "$p\\big(A \\cap B'\\big) = \\frac{13}{50}$\n",
    "\n",
    "$p\\big(A' \\cap B'\\big) = \\frac{11}{50}$\n",
    "\n",
    "$p\\big(A | B'\\big) = \\frac{13}{13+11} = \\frac{13}{24}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Y3QlZbPkfcM"
   },
   "source": [
    "# Bayes Theorem\n",
    "\n",
    "There are numerous ways to write Bayes Theorem, but the most common is:\n",
    "\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(y | x\\big) \\cdot p\\big(x\\big)}{p\\big(y\\big)}$$\n",
    "\n",
    "But, another popular form is:\n",
    "\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(y | x\\big) \\cdot p\\big(x\\big)}{p\\big(y | x\\big)p\\big(x\\big) + p\\big(y | x'\\big)p\\big(x'\\big)}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-mWKkDG3rRQl"
   },
   "source": [
    "## Example\n",
    "\n",
    "In the early days of HIV testing, an ELISA test was used to determine if someone tested positive for HIV by testing for antibodies.\n",
    "\n",
    "The probability that they tested positive given that they had HIV was:\n",
    "$$p( \\text{ + } | \\text{ HIV } ) = .977$$\n",
    "\n",
    "The probability that they tested negative given that they didn't have HIV was:\n",
    "$$p( \\text{ - } | \\text{ HIV' } ) = .926$$\n",
    "\n",
    "So, the test would give you an accurate result 90%+ of the time. This is often referred to as the *sensitivity* of the test. \n",
    "\n",
    "A study at during the same time showed that the probability of someone in North America having HIV was:\n",
    "\n",
    "$$p( \\text{ HIV } ) = .0026 $$\n",
    "\n",
    "***Using this data as context, if we randomly select someone from North America, have them take the test, and they test positive for HIV, what's the probability that they actually have HIV?***\n",
    "\n",
    "$$ p( \\text{ HIV } | \\text{ + } ) = \\text{?}$$\n",
    "\n",
    "We can use Bayes Theorem to calculate:\n",
    "\n",
    "$$ p( \\text{ HIV } | \\text{ + } ) = \\frac{ p( \\text{ + } | \\text{ HIV } ) \\cdot p( \\text{ HIV } ) }{ p( \\text{ + } | \\text{ HIV } ) \\cdot p( \\text{ HIV } ) + p( \\text{ + } | \\text{ HIV' } ) \\cdot p( \\text{ HIV' } )}$$\n",
    "\n",
    "We can derive:\n",
    "\n",
    "$$ p( \\text{ + } | \\text{ HIV' } ) = 1 - p( \\text{ - } | \\text{ HIV' } ) = 1 - .926 = .074 $$\n",
    "\n",
    "$$ p( \\text{ HIV' } ) = 1 - p( \\text{ HIV } ) = 1 - .0026 = .9974$$\n",
    "\n",
    "Then, we can calculate:\n",
    "\n",
    "$$ p( \\text{ HIV } | \\text{ + } ) = \\frac{.977 \\cdot .0026}{ (.977 \\cdot .0026 ) + ( .074 \\cdot .9974 )} = .033$$\n",
    "\n",
    "So, if the randomly selected person tested positive, they only have a ***3.3% chance*** of actually having HIV!\n",
    "\n",
    "This is common with tests for rare diseases. The reason is because the rate of false positives (the part after the plus sign in the denominator) far outweighs the rate of true positives.\n",
    "\n",
    "Now, let's say they take the test again. \n",
    "\n",
    "We could consider the joint distribution of the two separate tests, which are each conditional distributions:\n",
    "\n",
    "$$p( \\text{ + } | \\text{ HIV },  \\text{ + } | \\text{ HIV })$$\n",
    "\n",
    "Assuming the tests are independent, we can assume:\n",
    "\n",
    "$$p( \\text{ + } | \\text{ HIV },  \\text{ + } | \\text{ HIV }) = p( \\text{ + } | \\text{ HIV } ) \\cdot p( \\text{ + } | \\text{ HIV } ) $$ \n",
    "\n",
    "To simplify the notation, we'll define $++$ as getting a positive result on two separate tests. So the probability of having HIV, given that you tested positive twice is:\n",
    "\n",
    "$$ p( \\text{ HIV } | \\text{ ++ } )$$\n",
    "\n",
    "And the probability of testing positive twice, given that you have HIV is:\n",
    "\n",
    "$$ p( \\text{ ++ } | \\text{ HIV } )$$\n",
    "\n",
    "Applying Bayes Theorem to the two positive tests, we can calculate the probability of having HIV, given that two independent tests were both positive as follows:\n",
    "\n",
    "$$ p( \\text{ HIV } | \\text{ ++ } ) = \\frac{ p( \\text{ ++ } | \\text{ HIV } ) \\cdot p( \\text{ HIV } ) }{ p( \\text{ ++ } | \\text{ HIV } ) \\cdot p( \\text{ HIV } ) + p( \\text{ ++ } | \\text{ HIV' } ) \\cdot p( \\text{ HIV' } )}$$\n",
    "\n",
    "Because we assumed independence, we know:\n",
    "\n",
    "$$p( \\text{ ++ } | \\text{ HIV } ) = p( \\text{ + } | \\text{ HIV } ) \\cdot p( \\text{ + } | \\text{ HIV } ) $$ \n",
    "\n",
    "$$= .977 \\cdot .977 $$ \n",
    "\n",
    "$$= .955$$\n",
    "\n",
    "And:\n",
    "\n",
    "$$p( \\text{ ++ } | \\text{ HIV' } ) = p( \\text{ + } | \\text{ HIV' } ) \\cdot p( \\text{ + } | \\text{ HIV' } ) $$ \n",
    "$$= .074 \\cdot .074 $$ \n",
    "$$= .0055$$\n",
    "\n",
    "So, with two positive tests, the probability increases:\n",
    "\n",
    "$$ p( \\text{ HIV } | \\text{ ++ } ) = \\frac{.955 \\cdot .0026}{ (.955 \\cdot .0026 ) + ( .0055 \\cdot .9974 )} = .3124$$\n",
    "\n",
    "\n",
    "References:\n",
    "- [Bayesian Statistics: From Concept to Data Analysis](https://www.coursera.org/learn/bayesian-statistics/lecture/oANY8/lesson-2-2-bayes-theorem)\n",
    "- [Bayes, two tests in a row](https://math.stackexchange.com/questions/837768/bayes-two-tests-in-a-row)\n",
    "- [Bayes rule with multiple tests](https://math.stackexchange.com/questions/2054128/bayes-rule-with-multiple-tests)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "id": "Taq2bUF43WWJ",
    "outputId": "2cc7d1a1-a581-40c7-cf78-5f90c533f0cb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "p( HIV | + ) = 0.0333\n",
      "p( HIV | ++ ) = 0.3124\n",
      "p( HIV | +++ ) = 0.8571\n",
      "p( HIV | ++++ ) = 0.9875\n"
     ]
    }
   ],
   "source": [
    "positive_hiv = .977\n",
    "negative_nohiv = .926\n",
    "positive_nohiv = 1 - negative_nohiv\n",
    "hiv = .0026\n",
    "nohiv = 1 - hiv\n",
    "\n",
    "p_hiv_positive = (positive_hiv * hiv) / ( (positive_hiv * hiv) + (positive_nohiv * nohiv) )\n",
    "print(\"p( HIV | + ) = %6.4f\" % p_hiv_positive)\n",
    "\n",
    "p_hiv_positive_twice = (positive_hiv ** 2 * hiv) / ( (positive_hiv ** 2 * hiv) + (positive_nohiv ** 2 * nohiv) )\n",
    "print(\"p( HIV | ++ ) = %6.4f\" % p_hiv_positive_twice)\n",
    "\n",
    "p_hiv_positive_thrice = (positive_hiv ** 3 * hiv) / ( (positive_hiv ** 3 * hiv) + (positive_nohiv ** 3 * nohiv) )\n",
    "print(\"p( HIV | +++ ) = %6.4f\" % p_hiv_positive_thrice)\n",
    "\n",
    "p_hiv_positive_four = (positive_hiv ** 4 * hiv) / ( (positive_hiv ** 4 * hiv) + (positive_nohiv ** 4 * nohiv) )\n",
    "print(\"p( HIV | ++++ ) = %6.4f\" % p_hiv_positive_four)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TyyRlXmFnGVv"
   },
   "source": [
    "## Discrete\n",
    "\n",
    "The equation I defined in the overview defines Bayes Theorem as it applies to two discrete events. \n",
    "\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(y | x\\big) \\cdot p\\big(x\\big)}{p\\big(y\\big)}$$\n",
    "\n",
    "Which exands to:\n",
    "\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(y | x\\big) \\cdot p\\big(x\\big)}{p\\big(y | x\\big)p\\big(x\\big) + p\\big(y | x'\\big)p\\big(x'\\big)}$$\n",
    "\n",
    "If you notice, the denominator is just the marginalization of $p(y)$, assuming that there are only two outcomes $x$ and $x'$.\n",
    "\n",
    "But, let's say there are three or more possible outcomes:\n",
    "$$x_1,x_2,...,x_m$$\n",
    "\n",
    "If we want to calculate the probability of one of those outcomes (say $x_1$ or $x_7$), we can generalize using the marginalized summation for discrete events in the denominator, as follows:\n",
    "\n",
    "$$p\\big(x_1 | y\\big) = \\frac{p\\big(y | x_1\\big) \\cdot p\\big(x_1\\big)}{ \\sum_{i=1}^{m} p( y | x_i ) p( x_i ) }$$\n",
    "\n",
    "As a concrete example, imagine the time studied vs. grade example from above. \n",
    "\n",
    "\n",
    "\\begin{array} \n",
    "{|r|r|}\n",
    "\\hline  & \\text{0-20} & \\text{21-40} & \\text{41-60} & \\text{> 60} & \\text{TOTAL} \\\\ \n",
    "\\hline \\text{80-100} & 0 & 4 & \\color{blue}{16} & 20 & \\color{orange}{40} \\\\ \n",
    "\\hline \\text{60-79} & 0 & \\color{red}{20} & \\color{blue}{30} & 10 & \\color{orange}{60} \\\\ \n",
    "\\hline \\text{40-59} & 2 & 4 & \\color{blue}{32} & 32 & \\color{orange}{70} \\\\ \n",
    "\\hline \\text{20-39} & 10 & 2 & \\color{blue}{8} & 0 & \\color{orange}{20} \\\\ \n",
    "\\hline \\text{0-19} & 2 & 0 & \\color{blue}{0} & 8 & \\color{orange}{10} \\\\ \n",
    "\\hline \\text{TOTAL} & 14 & 30 & 86 & 70 & \\color{green}{200} \\\\ \n",
    "\\hline  \n",
    "\\end{array}\n",
    "\n",
    "\n",
    "Let's say x is grade and y is study time equals 41-60 minutes. We want to find the probability of a student scoring a grade of 80-100, given they studied for 41-60 minutes.\n",
    "\n",
    "$$p(x=\\text{80-100} | y)$$\n",
    "\n",
    "Since we have all the data, off the bat, we know the answer is $\\frac{16}{86} \\approx 0.186$, but let's use the discrete Bayes formula to calculate it, assuming we didn't have all the information.\n",
    "\n",
    "We calculate that the conditional and marginal probabilities and their products:\n",
    "\n",
    "\\begin{array} \n",
    "{|r|r|}\n",
    "\\hline \\text{ Grade } & p(y|x) & p(x) & p(y|x) \\cdot p(x) \\\\ \n",
    "\\hline 80-100 & \\frac{16}{40} & \\frac{40}{200} & 0.08 \\\\ \n",
    "\\hline 60-79 & \\frac{30}{60} & \\frac{60}{200} & 0.15 \\\\ \n",
    "\\hline 40-59 & \\frac{32}{70} & \\frac{70}{200} & 0.16 \\\\ \n",
    "\\hline 20-39 & \\frac{8}{20} & \\frac{20}{200} & 0.04 \\\\ \n",
    "\\hline 0-19 & \\frac{0}{10} & \\frac{10}{200} & 0 \\\\ \n",
    "\\hline \\text{TOTAL} & 1 & 1 & 0.43 \\\\ \n",
    "\\hline  \n",
    "\\end{array}\n",
    "\n",
    "So, if we want to find the probability of a student scoring a grade of 80-100, given they studied for 41-60 minutes, we can calculate:\n",
    "\n",
    "$$ p(x=\\text{80-100} | y) = \\frac{ 0.08 }{ .43 } \\approx 0.186 $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "p( x = 80-100 | y = 41-60 ) = 0.186046511627907\n"
     ]
    }
   ],
   "source": [
    "p80_given_y = 16/40\n",
    "p60_given_y = 30/60\n",
    "p40_given_y = 32/70\n",
    "p20_given_y = 8/20\n",
    "p0_given_y = 0/10\n",
    "\n",
    "p80 = 40/200\n",
    "p60 = 60/200\n",
    "p40 = 70/200\n",
    "p20 = 20/200\n",
    "p0 = 10/200\n",
    "\n",
    "p80_x = p80_given_y * p80\n",
    "p60_x = p60_given_y * p60\n",
    "p40_x = p40_given_y * p40\n",
    "p20_x = p20_given_y * p20\n",
    "p0_x = p0_given_y * p0\n",
    "\n",
    "sum_x = ( p80_x + p60_x + p40_x + p20_x + p0_x )\n",
    "\n",
    "total = p80_x / sum_x\n",
    "print( \"p( x = 80-100 | y = 41-60 ) =\", total )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4Fo0gIVrnBgd"
   },
   "source": [
    "## Continuous\n",
    "\n",
    "Based on the marginal distribution and the joint distribution rules, we know:\n",
    "\n",
    "$$p\\big(y\\big) = \\int p\\big(x, y\\big)dx = \\int p\\big(y|x\\big)p\\big(x\\big)dx $$\n",
    "\n",
    "So:\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(y | x\\big)p\\big(x\\big)}{\\int p\\big(y|x\\big)p\\big(x\\big)dx}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Daz0-46Eq7FI"
   },
   "source": [
    "## Proof / Derivation\n",
    "We know that the joint distribution of two random variables is commutative:\n",
    "\n",
    "$$p\\big(x, y\\big) = p\\big(y, x\\big) $$\n",
    "\n",
    "We also know that joint distributions that are dependent can be written as:\n",
    "$$p\\big(x, y\\big) = p\\big(x\\big) \\cdot  p\\big(y|x\\big)$$\n",
    "\n",
    "$$p\\big(y, x\\big) = p\\big(x, y\\big) = p\\big(y\\big) \\cdot  p\\big(x|y\\big)$$\n",
    "\n",
    "We can rewrite the third equation as:\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(x, y\\big)}{p\\big(y\\big)}$$\n",
    "\n",
    "Then we can substitute in the second equation into the numerator of the right hand side:\n",
    "\n",
    "$$p\\big(x | y\\big) = \\frac{p\\big(y | x\\big) \\cdot p\\big(x\\big)}{p\\big(y\\big)}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xEIF0l2We12N"
   },
   "source": [
    "# Expected Value\n",
    "\n",
    "The expected value is the average outcome we expect if we run an experiment many times. \n",
    "\n",
    "Another way to think about it is that the expected value is the *mean* of some random variable.\n",
    "\n",
    "For a discrete random variable $X$ we calculate the expected value as follows:\n",
    "\n",
    "$$E\\big(X\\big) = \\sum_{x} x p\\big(x\\big)$$\n",
    "\n",
    "We can perform marginalization on a continuous random variable $X$ as follows:\n",
    "\n",
    "$$E\\big(X\\big) = \\int x p\\big(x\\big)dx$$\n",
    "\n",
    "### Conditional Expectation\n",
    "\n",
    "For a discrete random variable $x$ we calculate as follows:\n",
    "\n",
    "$$E\\big(X | Y\\big) = \\sum_{x} x p\\big(x|y\\big)$$\n",
    "\n",
    "For a continuous random variable $x$ we calculate as follows:\n",
    "\n",
    "$$E\\big(X|Y\\big) = \\int x p\\big(x|y\\big)dx$$\n",
    "\n",
    "### Expected Value of a Function\n",
    "\n",
    "If we have a function of a random variable:\n",
    "\n",
    "$$g\\big(X\\big)$$\n",
    "\n",
    "If $x$ is discrete, we can calculate the expected value as follows:\n",
    "\n",
    "$$E\\big[g\\big(X\\big)\\big] = \\sum_{x} g\\big(x\\big) p\\big(x\\big)$$\n",
    "\n",
    "If $x$ is continuous, we can calculate the expected value as follows:\n",
    "\n",
    "$$E\\big[g\\big(X\\big)\\big] = \\int g\\big(x\\big) p\\big(x\\big)dx$$\n",
    "\n",
    "### Expansion\n",
    "\n",
    "Because the expected value is a linear operator, the expectation of the sum of random variables is equal to the sum of the expectation of the individual values:\n",
    "\n",
    "$$E\\big(cX+Y\\big) = cE\\big(X\\big) + E\\big(Y\\big)$$\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Variance\n",
    "\n",
    "$$ \\text{Var}(X) = E[(X-\\mu)^2] $$\n",
    "\n",
    "$$ \\text{Var}(X) = E(X^2) - \\big( E(X) \\big )^2 $$\n",
    "\n",
    "$$ \\text{Var}(A + B) = V(A) + V(B) + 2 \\text{ COV}(A,B) $$\n",
    "\n",
    "Where:\n",
    "- COV() is the covariance\n",
    "- V() is the variance\n",
    "\n",
    "References:\n",
    "- [Variance - Wikipedia](https://en.wikipedia.org/wiki/Variance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J8BIbwhMBhvF"
   },
   "source": [
    "# References\n",
    "\n",
    "- [Conditional Probability in Venn Diagrams](https://www.slideserve.com/lucine/conditional-probability-in-venn-diagrams)\n",
    "- [Probability Calculator](https://www.calculator.net/probability-calculator.html)\n",
    "- [Visualizing Set Diagrams With Python](https://monstott.github.io/visualizing_set_diagrams_with_python)\n",
    "- [Probability & Statistics Primer](https://atmos.washington.edu/~hakim/591/prob_primer.pdf)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Probability.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}