Confidence Intervals - 4 - Building Confidence Intervals.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confidence Intervals - Part I\n",
    "\n",
    "First let's read in the necessary libraries and the dataset.  You also have the full and reduced versions of the data available.  The reduced version is an example of you would actually get in practice, as it is the sample.  While the full data is an example of everyone in your population."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "np.random.seed(42)\n",
    "\n",
    "coffee_full = pd.read_csv('../data/coffee_dataset.csv')\n",
    "coffee_red = coffee_full.sample(200) #this is the only data you might actually get in the real world."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`1.` What is the proportion of coffee drinkers in the sample?  What is the proportion of individuals that don't drink coffee?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.59499999999999997"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coffee_red['drinks_coffee'].mean() # Drink Coffee"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.40500000000000003"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 - coffee_red['drinks_coffee'].mean() # Don't Drink Coffee"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`2.` Of the individuals who do not drink coffee, what is the average height?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "66.78492279927877"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coffee_red[coffee_red['drinks_coffee'] == False]['height'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`3.` Simulate 200 \"new\" individuals from your original sample of 200.  What are the proportion of coffee drinkers in your bootstrap sample?  How about individuals that don't drink coffee?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "bootsamp = coffee_red.sample(200, replace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.60499999999999998"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bootsamp['drinks_coffee'].mean() # Drink Coffee and 1 minus gives the don't drink"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`4.` Now simulate your bootstrap sample 10,000 times and take the mean height of the non-coffee drinkers in each sample.  Plot the distribution, and pull the values necessary for a 95% confidence interval.  What do you notice about the sampling distribution of the mean in this example?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.hist(boot_means); # Looks pretty normal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD8CAYAAAB6paOMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEb5JREFUeJzt3W+snOdZ5/HvD6fNRqWhiWws47hyKrlITiRc4jVBFCgN\nENO+cMqLyHlBjNSNgWQritiVHFaC7gtL2YWCFGkTyd1m4+yyTQ20xCLJZtOogJBIwmlJ49hpNl7i\nyj44sQGtAvsiKO61L+Y2Ozmd4zPn74y5vx9pdJ65nud+5ppbln/n+TNzUlVIkvr1XZNuQJI0WQaB\nJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXNXTLqBhaxfv762bt066TYk6bKxfv16\nnnrqqaeqavc42099EGzdupWZmZlJtyFJl5Uk68fd1lNDktQ5g0CSOmcQSFLnDAJJ6tyCQZDkXyR5\nPsk3khxP8u9b/dokTyd5tf28ZmjMvUlOJnklya1D9ZuSHGvr7k+S1XlbkqRxjXNE8Bbw0ar6AWAH\nsDvJzcAB4Jmq2gY8056TZDuwF7gB2A08kGRd29eDwF3AtvYY69YmSdLqWTAIauAf2tN3tUcBe4DD\nrX4YuK0t7wEeraq3quo14CSwK8km4OqqerYGfxbtkaExkqQJGesaQZJ1SV4AzgFPV9VzwMaqOts2\neR3Y2JY3A6eHhp9ptc1teW5dkjRBYwVBVV2oqh3AdQx+u79xzvpicJSwIpLsTzKTZOb8+fMrtVtJ\n0giL+mRxVf2fJF9lcG7/jSSbqupsO+1zrm02C2wZGnZdq8225bn1Ua9zCDgEsHPnzhULGP3ztPXA\n4xN53VP3fXwiryuttHHuGtqQ5H1t+Srgp4BvAkeBfW2zfcBjbfkosDfJlUmuZ3BR+Pl2GunNJDe3\nu4XuHBojSZqQcY4INgGH250/3wUcqao/SvLnwJEknwS+BdwOUFXHkxwBTgBvA/dU1YW2r7uBh4Gr\ngCfbQ5I0QQsGQVW9CHxoRP1vgVvmGXMQODiiPgPc+J0jJEmT4ieLJalzBoEkdc4gkKTOGQSS1DmD\nQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgk\nqXMGgSR1ziCQpM4ZBJLUuQX/eL2k0bYeeHxir33qvo9P7LX1z49HBJLUOYNAkjpnEEhS5wwCSeqc\nQSBJnVswCJJsSfLVJCeSHE/yy63+mSSzSV5oj48Njbk3yckkryS5dah+U5Jjbd39SbI6b0uSNK5x\nbh99G/jVqvp6kvcCX0vydFv3O1X1W8MbJ9kO7AVuAL4P+EqSD1bVBeBB4C7gOeAJYDfw5Mq8FUnS\nUix4RFBVZ6vq623574GXgc2XGLIHeLSq3qqq14CTwK4km4Crq+rZqirgEeC2Zb8DSdKyLOoaQZKt\nwIcY/EYP8KkkLyZ5KMk1rbYZOD007EyrbW7Lc+uSpAkaOwiSfDfwB8Cnq+pNBqd5PgDsAM4Cn12p\nppLsTzKTZOb8+fMrtVtJ0ghjBUGSdzEIgd+tqi8BVNUbVXWhqr4NfA7Y1TafBbYMDb+u1Wbb8tz6\nd6iqQ1W1s6p2btiwYTHvR5K0SOPcNRTg88DLVfXbQ/VNQ5t9AnipLR8F9ia5Msn1wDbg+ao6C7yZ\n5Oa2zzuBx1bofUiSlmicu4Z+BPg54FiSF1rt14A7kuwACjgF/AJAVR1PcgQ4weCOo3vaHUMAdwMP\nA1cxuFvIO4YkacIWDIKq+jNg1P3+T1xizEHg4Ij6DHDjYhqUJK0uP1ksSZ0zCCSpcwaBJHXOIJCk\nzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5wwCSeqc\nQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwsGQZItSb6a5ESS\n40l+udWvTfJ0klfbz2uGxtyb5GSSV5LcOlS/Kcmxtu7+JFmdtyVJGtc4RwRvA79aVduBm4F7kmwH\nDgDPVNU24Jn2nLZuL3ADsBt4IMm6tq8HgbuAbe2xewXfiyRpCRYMgqo6W1Vfb8t/D7wMbAb2AIfb\nZoeB29ryHuDRqnqrql4DTgK7kmwCrq6qZ6uqgEeGxkiSJmRR1wiSbAU+BDwHbKyqs23V68DGtrwZ\nOD007EyrbW7Lc+ujXmd/kpkkM+fPn19Mi5KkRRo7CJJ8N/AHwKer6s3hde03/FqppqrqUFXtrKqd\nGzZsWKndSpJGGCsIkryLQQj8blV9qZXfaKd7aD/PtfossGVo+HWtNtuW59YlSRM0zl1DAT4PvFxV\nvz206iiwry3vAx4bqu9NcmWS6xlcFH6+nUZ6M8nNbZ93Do2RJE3IFWNs8yPAzwHHkrzQar8G3Acc\nSfJJ4FvA7QBVdTzJEeAEgzuO7qmqC23c3cDDwFXAk+0hSZqgBYOgqv4MmO9+/1vmGXMQODiiPgPc\nuJgGJUmry08WS1LnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM6N8zkCaUFbDzw+6RYkLZFHBJLUOYNA\nkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSp\ncwaBJHXOIJCkzhkEktQ5g0CSOrdgECR5KMm5JC8N1T6TZDbJC+3xsaF19yY5meSVJLcO1W9Kcqyt\nuz9JVv7tSJIWa5wjgoeB3SPqv1NVO9rjCYAk24G9wA1tzANJ1rXtHwTuAra1x6h9SpLW2IJBUFV/\nCvzdmPvbAzxaVW9V1WvASWBXkk3A1VX1bFUV8Ahw21KbliStnOVcI/hUkhfbqaNrWm0zcHpomzOt\ntrktz61LkiZsqUHwIPABYAdwFvjsinUEJNmfZCbJzPnz51dy15KkOZYUBFX1RlVdqKpvA58DdrVV\ns8CWoU2va7XZtjy3Pt/+D1XVzqrauWHDhqW0KEka05KCoJ3zv+gTwMU7io4Ce5NcmeR6BheFn6+q\ns8CbSW5udwvdCTy2jL4lSSvkioU2SPIF4CPA+iRngN8APpJkB1DAKeAXAKrqeJIjwAngbeCeqrrQ\ndnU3gzuQrgKebA9J0oQtGARVdceI8ucvsf1B4OCI+gxw46K6kyStOj9ZLEmdMwgkqXMGgSR1ziCQ\npM4ZBJLUuQXvGpI0fbYeeHwir3vqvo9P5HW1ujwikKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0z\nCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNA\nkjpnEEhS5xYMgiQPJTmX5KWh2rVJnk7yavt5zdC6e5OcTPJKkluH6jclOdbW3Z8kK/92JEmLNc4R\nwcPA7jm1A8AzVbUNeKY9J8l2YC9wQxvzQJJ1bcyDwF3AtvaYu09J0gQsGARV9afA380p7wEOt+XD\nwG1D9Uer6q2qeg04CexKsgm4uqqeraoCHhkaI0maoKVeI9hYVWfb8uvAxra8GTg9tN2ZVtvclufW\nJUkTtuyLxe03/FqBXv5Jkv1JZpLMnD9/fiV3LUmaY6lB8EY73UP7ea7VZ4EtQ9td12qzbXlufaSq\nOlRVO6tq54YNG5bYoiRpHEsNgqPAvra8D3hsqL43yZVJrmdwUfj5dhrpzSQ3t7uF7hwaI0maoCsW\n2iDJF4CPAOuTnAF+A7gPOJLkk8C3gNsBqup4kiPACeBt4J6qutB2dTeDO5CuAp5sD0nShC0YBFV1\nxzyrbpln+4PAwRH1GeDGRXUnSVp1frJYkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS\n1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmd\nMwgkqXMGgSR1ziCQpM5dMekGtLK2Hnh80i1Iusx4RCBJnVtWECQ5leRYkheSzLTatUmeTvJq+3nN\n0Pb3JjmZ5JUkty63eUnS8q3EEcFPVNWOqtrZnh8AnqmqbcAz7TlJtgN7gRuA3cADSdatwOtLkpZh\nNU4N7QEOt+XDwG1D9Uer6q2qeg04CexahdeXJC3CcoOggK8k+VqS/a22sarOtuXXgY1teTNwemjs\nmVaTJE3Qcu8a+nBVzSb5XuDpJN8cXllVlaQWu9MWKvsB3v/+9y+zRUnSpSzriKCqZtvPc8CXGZzq\neSPJJoD281zbfBbYMjT8ulYbtd9DVbWzqnZu2LBhOS1Kkhaw5CBI8p4k7724DPw08BJwFNjXNtsH\nPNaWjwJ7k1yZ5HpgG/D8Ul9fkrQylnNqaCPw5SQX9/Pfq+p/JPkL4EiSTwLfAm4HqKrjSY4AJ4C3\ngXuq6sKyupckLduSg6Cq/gr4gRH1vwVumWfMQeDgUl9TkrTy/GSxJHXOIJCkzhkEktQ5g0CSOufX\nUEsa26S+5vzUfR+fyOv2wiMCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknq\nnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6px/oWwVTOqvOEnSUnhEIEmd84hA\n0tSb5FF2D38v2SMCSercmgdBkt1JXklyMsmBtX59SdI7rWkQJFkH/CfgZ4DtwB1Jtq9lD5Kkd1rr\nI4JdwMmq+quq+kfgUWDPGvcgSRqy1heLNwOnh56fAX5otV7M2zglLdek/h9Zy4vUU3nXUJL9wP72\n9B+SvDLJfi5hPfA3k25iGex/8i7392D/qyT/YazN5ut/Ue9prYNgFtgy9Py6VnuHqjoEHFqrppYq\nyUxV7Zx0H0tl/5N3ub8H+5+slep/ra8R/AWwLcn1Sd4N7AWOrnEPkqQha3pEUFVvJ/nXwFPAOuCh\nqjq+lj1Ikt5pza8RVNUTwBNr/bqrZOpPXy3A/ifvcn8P9j9ZK9J/qmol9iNJukz5FROS1DmDYB5J\n3pfk95N8M8nLSX44yWeSzCZ5oT0+Ns/YU0mOtW1m1rr31sN39N/qn2q140n+4zxjJ/41IMvsfyrn\nP8kXh/7tnErywjxjp3L+F9H/tM7/jiTPXuwrya55xk7r/I/b/+Lnv6p8jHgAh4F/1ZbfDbwP+Azw\nb8YYewpYP4X9/wTwFeDKVv/eEePWAf8b+EAb9w1g++XS/zTP/5z1nwV+/XKa/3H6n+b5B/4n8DOt\n9jHgjy+n+R+n/6XO/1R+oGzSknwP8GPAzwPU4Osw/jHJJNsa2yX6/yXgvqp6q9XPjRj+T18D0vZ1\n8WtATqxB67TXXE7/Ezdf/0PrA9wOfHTE8Kmd/6H1l+p/4i7x76eAq9tm3wP89YjhUzv/Y/a/JJ4a\nGu164DzwX5L8ZZL/nOQ9bd2nkryY5KEk18wzvoCvJPlaBp+SXmvz9f9B4EeTPJfkT5L8yxFjR30N\nyObVb/kdltM/TO/8X/SjwBtV9eqIsdM8/xddqn+Y3vn/NPCbSU4DvwXcO2LsNM//OP3DEubfIBjt\nCuAHgQer6kPA/wUOAA8yOGTcAZxlcHg8yoerageDb1m9J8mPrX7L7zBf/1cA1wI3A/8WOJLpPMxZ\nbv/TOv8X3QF8YY17Wozl9j+t8/9LwK9U1RbgV4DPr3Ff41pu/4uef4NgtDPAmap6rj3/feAHq+qN\nqrpQVd8GPsfgMPI7VNVs+3kO+PJ8262ikf23+pdq4Hng2wy+q2TYWF8DssqW0/80zz9JrgB+Fvji\nPGOnef7H6X+a538f8KVW+715+prm+R+n/yXNv0EwQlW9DpxO8v2tdAtwIsmmoc0+Abw0d2yS9yR5\n78Vl4KdHbbea5usf+EMGF1xJ8kEGF6HmfjnVxL8GZDn9T/n8A/wk8M2qOjPP8Gmef1ig/ymf/78G\nfrzVPgqMOrU1zfO/YP9Lnv+1vhp+uTwYnP6ZAV5k8B/QNcB/BY612lFgU9v2+4An2vIHGNxp8A3g\nOPDvpqj/dwP/rf3D+Drw0bn91/+/I+F/Mbh74rLqf5rnv9UfBn5xzraXxfyP0/80zz/wYeBrrbfn\ngJsup/kfp/+lzr+fLJakznlqSJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktS5/wfD\n+8nk+uXujwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114404748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "boot_means = []\n",
    "for _ in range(10000):\n",
    "    bootsamp = coffee_red.sample(200, replace = True)\n",
    "    boot_mean = bootsamp[bootsamp['drinks_coffee'] == False]['height'].mean()\n",
    "    boot_means.append(boot_mean)\n",
    "    \n",
    "plt.hist(boot_means); # Looks pretty normal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(65.992913281575198, 67.584027382815734)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.percentile(boot_means, 2.5), np.percentile(boot_means, 97.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`5.`  Did your interval capture the actual average height of coffee drinkers in the population?  Look at the average in the population and the two bounds provided by your 95% confidence interval, and then answer the final quiz question below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "66.44340776214705"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coffee_full[coffee_full['drinks_coffee'] == False]['height'].mean() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Captured by our interval, but not the exact same as the sample mean"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}