0_S_Prework_myname_template.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Topic:      Prework Solutions\n",
    "#### Cohort:   03_Spring2015\n",
    "#### Date:        4/1/2015\n",
    "#### Author:     Reshama"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prework: Required Exercises\n",
    "\n",
    "https://github.com/datascopeanalytics/metis-data-science-bootcamp-prework/blob/master/exercises.md\n",
    "\n",
    "Source:     Think Stats (section Using the Code), there is some accompanying code and data. You can get these from the Think Stats repository.\n",
    "https://github.com/AllenDowney/ThinkStats2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------- "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Table of Contents \n",
    "[Exercise01](#1) \n",
    "\n",
    "(Think Stats Ex 2.4) \n",
    "\n",
    "Using the variable totalwgt_lb, investigate whether first babies are lighter or heavier than others. Compute Cohen’s d to quantify the difference between the groups. How does it compare to the difference in pregnancy length?\n",
    " \n",
    "\n",
    "[Exercise02](#2)\n",
    "\n",
    "(Think Stats Ex 3.1) \n",
    "\n",
    "Something like the class size paradox appears if you survey children and ask how many children are in their family. Families with many children are more likely to appear in your sample, and families with no children have no chance to be in the sample.\n",
    "\n",
    "Use the NSFG respondent variable NUMKDHH to construct the actual distribution for the number of children under 18 in the household.\n",
    "Now compute the biased distribution we would see if we surveyed the children and asked them how many children under 18 (including themselves) are in their household.\n",
    "\n",
    "Plot the actual and biased distributions, and compute their means. As a starting place, you can use chap03ex.ipynb. This is an ipython notebook from the ThinkStats2 repository.\n",
    "\n",
    "\n",
    "[Exercise03](#3)\n",
    "\n",
    "(Think Stats Ex 4.2) \n",
    "\n",
    "The numbers generated by random.random are supposed to be uniform between 0 and 1; that is, every value in the range should have the same probability.\n",
    "Generate 1000 numbers from random.random and plot their PMF and CDF. Is the distribution uniform?\n",
    "\n",
    "\n",
    "\n",
    "[Exercise04](#4)\n",
    "\n",
    "(Think Stats Ex 7.1) \n",
    "\n",
    "Using data from the NSFG, make a scatter plot of birth weight versus mother’s age. Plot percentiles of birth weight versus mother’s age. Compute Pearson’s and Spearman’s correlations. How would you characterize the relationship between these variables?\n",
    "\n",
    "[Exercise05](#5)\n",
    "\n",
    "(Think Stats Ex 8.2)\n",
    "\n",
    "Suppose that you draw a sample with size n = 10 from an exponential distribution with λ = 2. Simulate this experiment 1000 times and plot the sampling distribution of the estimate L. Compute the standard error of the estimate and the 90% confidence interval.\n",
    "\n",
    "Repeat the experiment with a few different values of n and make a plot of standard error versus n.\n",
    "\n",
    "\n",
    "[Exercise06](#6)\n",
    "\n",
    "(Think Stats Ex 2.1 Bayes)\n",
    "\n",
    "The cookie problem is a problem discussed in sections 1.3, 2.2 and 2.3 of Think Bayes. Solve the following problem. In Section 2.3 I said that the solution to the cookie problem generalizes to the case where we draw multiple cookies with replacement. But in the more likely scenario where we eat the cookies we draw, the likelihood of each draw depends on the previous draws.\n",
    "\n",
    "Modify the solution in this chapter to handle selection without replacement. Hint: add instance variables to Cookie to represent the hypothetical state of the bowls, and modify Likelihood accordingly. You might want to define a Bowl object.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='1'></a> Exercise 1 \n",
    "(Think Stats Ex 2.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "a) Using the variable totalwgt_lb, investigate whether first babies are lighter or heavier than others. \n",
    "\n",
    "b) Compute Cohen’s d to quantify the difference between the groups. \n",
    "\n",
    "c) How does it compare to the difference in pregnancy length?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note:  Cohen's d is an effect size used to indicate the standardised difference between two means. It can be used, for example, to accompany reporting of t-test and ANOVA results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Note:\n",
    "\n",
    "Step1.  Create a directory on your computer.  Below is an example:\n",
    "\n",
    "/Users/reshamashaikh/ds/metis/metisgh/\n",
    "\n",
    "Step2.  use GitHub to pull this repo to your computer\n",
    "\n",
    "git clone https://github.com/AllenDowney/ThinkStats2.git\n",
    "\n",
    "Step3.  Put your ipython notebook in this directory (that way, it can pull the needed dependencies):  \n",
    "\n",
    "https://github.com/AllenDowney/ThinkStats2/tree/master/code\n",
    "\n",
    "Step4.  Call your prework solutions notebook:  \n",
    "\n",
    "prework_solutions_myname.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# set working directory\n",
    "path_data = \"/Users/reshamashaikh/_ds/metis/metisgh/ThinkStats2\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Answer:  chap01soln.ipynb\n",
    "####First babies are lighter than others (7.20 vs 7.33)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Answer:  \n",
    "####Cohen's D is:  -0.0887"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute Cohen’s d to quantify the difference between the groups. \n",
    "How does it compare to the difference in pregnancy length? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cohen's D for birth weight is:     -0.0887\n",
      "Cohen's D for pregnancy length is:  0.0289\n"
     ]
    }
   ],
   "source": [
    "print \"Cohen's D for birth weight is:     -0.0887\"\n",
    "print \"Cohen's D for pregnancy length is:  0.0289\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Interpretation\n",
    "This site has a good interpretation of Cohen's D, effect size:\n",
    "http://www.uccs.edu/lbecker/effect-size.html\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='2'></a> Exercise 2 \n",
    "(Think Stats Ex 3.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Something like the class size paradox appears if you survey children and ask how many children are in their family. Families with many children are more likely to appear in your sample, and families with no children have no chance to be in the sample.\n",
    "\n",
    "Use the NSFG respondent variable NUMKDHH to construct the actual distribution for the number of children under 18 in the household.\n",
    "\n",
    "Now compute the biased distribution we would see if we surveyed the children and asked them how many children under 18 (including themselves) are in their household.\n",
    "\n",
    "Plot the actual and biased distributions, and compute their means. As a starting place, you can use chap03ex.ipynb. This is an ipython notebook from the ThinkStats2 repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='3'></a> Exercise 3 \n",
    "(Think Stats Ex 4.2)\n",
    "\n",
    "The numbers generated by random.random are supposed to be uniform between 0 and 1; that is, every value in the range should have the same probability.\n",
    "Generate 1000 numbers from random.random and plot their PMF and CDF. Is the distribution uniform?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import thinkstats2\n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Answer:\n",
    "Yes, distribution looks uniform"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='4'></a> Exercise 4 \n",
    "(Think Stats Ex 7.1)\n",
    "\n",
    "Using data from the NSFG, make a scatter plot of birth weight versus mother’s age. Plot percentiles of birth weight versus mother’s age. Compute Pearson’s and Spearman’s correlations. How would you characterize the relationship between these variables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import nsfg\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Answer:\n",
    "For the Pearson r correlation, both variables should be normally distributed.  Other assumptions include linearity and homoscedasticity.  Linearity assumes a straight line relationship between each of the variables in the analysis and homoscedasticity assumes that data is normally distributed about the regression line.\n",
    "\n",
    "Spearman rank correlation test does not make any assumptions about the distribution. \n",
    "\n",
    "####Conclusions:\n",
    "Data seem to be  ...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='5'></a> Exercise 5\n",
    "(Think Stats Ex 8.2)\n",
    "\n",
    "Suppose that you draw a sample with size n = 10 from an exponential distribution with λ = 2. Simulate this experiment 1000 times and plot the sampling distribution of the estimate L. Compute the standard error of the estimate and the 90% confidence interval.\n",
    "\n",
    "Repeat the experiment with a few different values of n and make a plot of standard error versus n."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import thinkstats2\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='6'></a> Exercise 6\n",
    "(Think Stats Ex 2.1 Bayes)\n",
    "\n",
    "The cookie problem is a problem discussed in sections 1.3, 2.2 and 2.3 of Think Bayes. Solve the following problem. In Section 2.3 I said that the solution to the cookie problem generalizes to the case where we draw multiple cookies with replacement. But in the more likely scenario where we eat the cookies we draw, the likelihood of each draw depends on the previous draws.\n",
    "\n",
    "Modify the solution in this chapter to handle selection without replacement. Hint: add instance variables to Cookie to represent the hypothetical state of the bowls, and modify Likelihood accordingly. You might want to define a Bowl object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# get this file from:  https://github.com/AllenDowney/ThinkBayes/blob/master/thinkbayes.py\n",
    "from thinkbayes import Pmf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}