Quiz_01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab 2\n",
    "\n",
    "In this lab we will become familiar with distributions, histograms, and functional programming. Before that, lets submit your Quiz from last lecture.\n",
    "\n",
    "### Quiz Submission\n",
    "\n",
    "The instructions here aim to have our course repository be public (enabling things like the colab links), while student repositories are private. You should be familiar with git from the first semester of this course. I assume that you all have github accounts and have setup things to be able to [push to github using ssh](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh). The instuctions here lead you to:\n",
    "\n",
    "1. Create a directory on your personal system where you will keep all course materials.\n",
    "1. Clone the [course repository](https://github.com/UTA-DataScience/DATA3402.Spring.2023).\n",
    "1. Create a new private repository for your solutions. \n",
    "1. Copy lab notebooks from course repository to your solutions repository.\n",
    "1. Commit / push your solutions.\n",
    "1. Grant access to course instructiors.\n",
    "\n",
    "Ideally, you should be able to do all of these steps yourself. Below are step by step instructions with examples (including example directory naming convention). Feel free to modify things as you see fit. The goal here to have a private repo where you can submit your solutions. \n",
    "\n",
    "1. Open a shell on your personal computer.\n",
    "1. If you have not done so already, create a new directory/folder where you will keep all course material to navigate to it. For example: `mkdir Data-3402` and `cd Data-3402`.\n",
    "1. Clone the course repository: `git clone https://github.com/UTA-DataScience/DATA3402.Spring.2023`.\n",
    "   1. Note that after the this clone, you simply need to navigate to this directory and do `git pull` to get updates (e.g. new labs).\n",
    "1. Make a new directory for your solutions and navigate to it: `mkdir DATA3402.Solutions.$USER` and `cd  DATA3402.Solutions.$USER`\n",
    "1. Make new directory for Labs and Lab 1: `mkdir Labs ; mkdir Labs/Labs.1`.\n",
    "1. Place your Quiz 1 solutions it this directory: `cp <path_to_solution> Labs/Labs.1 `\n",
    "   1. For Lab 1 and the future labs, just copy labs from the class repo, for example here: \n",
    "      * `mkdir Labs/Lab.2`\n",
    "      * `cp ../DATA3402.Spring.2023/Labs/Lab.2/Lab.2.ipynb Labs/Lab.2`\n",
    "1. Create a new git repository with same name are your directory above (i.e. `DATA3402.Solutions.$USER`). \n",
    "   * Navigate to your page on github.\n",
    "   * Use the \"+\" button to create a new repository.\n",
    "   * Make sure your repository is private.\n",
    "   * github will give you instructions on \"create a new repository on the command line\"\n",
    "1. After creating the repository, go to the repository settings tab, select \"Manage access\" on left, and press \"Invite a collaborator\" to invite course instructor (usernames `afarbin` and `anushka30`).\n",
    "1. To submit your labs, navigate to your private repo and \n",
    "   1. Make sure you have added the lab. For example: `git add Labs/Lab.1/Lab.1.ipynb`\n",
    "   1. Commit any changes: `git commit -a -m \"Lab 1 updates\"`\n",
    "   1. Push your changes: `git push`\n",
    "   1. Check on github that your solutions have been properly submitted.\n",
    "   \n",
    "   \n",
    "After submitting Quiz 1, continue work on lab 1, or start the remainder of this lab..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Uniform Distribution\n",
    "Lets start with generating some fake random data. You can get a random number between 0 and 1 using the python random module as follow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Value of x is 0.856071552562594\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "x = random.random()\n",
    "print(\"The Value of x is\", x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Everytime you call random, you will get a new number.\n",
    "\n",
    "*Exercise 1:* Using random, write a function `generate_uniform(N, mymin, mymax)`, that returns a python list containing N random numbers between specified minimum and maximum value. Note that you may want to quickly work out on paper how to turn numbers between 0 and 1 to between other values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jt -t solarizedl -T -N -kl -ofs 12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: jt [-h] [-l] [-t THEME] [-f MONOFONT] [-fs MONOSIZE] [-nf NBFONT]\r\n",
      "          [-nfs NBFONTSIZE] [-tf TCFONT] [-tfs TCFONTSIZE] [-dfs DFFONTSIZE]\r\n",
      "          [-ofs OUTFONTSIZE] [-mathfs MATHFONTSIZE] [-m MARGINS]\r\n",
      "          [-cursw CURSORWIDTH] [-cursc CURSORCOLOR] [-cellw CELLWIDTH]\r\n",
      "          [-lineh LINEHEIGHT] [-altp] [-altmd] [-altout] [-P] [-T] [-N] [-kl]\r\n",
      "          [-vim] [-r] [-dfonts]\r\n",
      "\r\n",
      "options:\r\n",
      "  -h, --help            show this help message and exit\r\n",
      "  -l, --list            list available themes\r\n",
      "  -t THEME, --theme THEME\r\n",
      "                        theme name to install\r\n",
      "  -f MONOFONT, --monofont MONOFONT\r\n",
      "                        monospace code font\r\n",
      "  -fs MONOSIZE, --monosize MONOSIZE\r\n",
      "                        code font-size\r\n",
      "  -nf NBFONT, --nbfont NBFONT\r\n",
      "                        notebook font\r\n",
      "  -nfs NBFONTSIZE, --nbfontsize NBFONTSIZE\r\n",
      "                        notebook fontsize\r\n",
      "  -tf TCFONT, --tcfont TCFONT\r\n",
      "                        txtcell font\r\n",
      "  -tfs TCFONTSIZE, --tcfontsize TCFONTSIZE\r\n",
      "                        txtcell fontsize\r\n",
      "  -dfs DFFONTSIZE, --dffontsize DFFONTSIZE\r\n",
      "                        pandas dataframe fontsize\r\n",
      "  -ofs OUTFONTSIZE, --outfontsize OUTFONTSIZE\r\n",
      "                        output area fontsize\r\n",
      "  -mathfs MATHFONTSIZE, --mathfontsize MATHFONTSIZE\r\n",
      "                        mathjax fontsize (in %)\r\n",
      "  -m MARGINS, --margins MARGINS\r\n",
      "                        fix margins of main intro page\r\n",
      "  -cursw CURSORWIDTH, --cursorwidth CURSORWIDTH\r\n",
      "                        set cursorwidth (px)\r\n",
      "  -cursc CURSORCOLOR, --cursorcolor CURSORCOLOR\r\n",
      "                        cursor color (r, b, g, p)\r\n",
      "  -cellw CELLWIDTH, --cellwidth CELLWIDTH\r\n",
      "                        set cell width (px or %)\r\n",
      "  -lineh LINEHEIGHT, --lineheight LINEHEIGHT\r\n",
      "                        code/text line-height (%)\r\n",
      "  -altp, --altprompt    alt input prompt style\r\n",
      "  -altmd, --altmarkdown\r\n",
      "                        alt markdown cell style\r\n",
      "  -altout, --altoutput  set output bg color to notebook bg\r\n",
      "  -P, --hideprompt      hide cell input prompt\r\n",
      "  -T, --toolbar         make toolbar visible\r\n",
      "  -N, --nbname          nb name/logo visible\r\n",
      "  -kl, --kernellogo     kernel logo visible\r\n",
      "  -vim, --vimext        toggle styles for vim\r\n",
      "  -r, --reset           reset to default theme\r\n",
      "  -dfonts, --defaultfonts\r\n",
      "                        force fonts to browser default\r\n"
     ]
    }
   ],
   "source": [
    "!jt --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.3577028029753838,\n",
       " 0.1947633592061564,\n",
       " 0.3028639914732534,\n",
       " 0.591423619537068,\n",
       " 0.49735596113490177,\n",
       " 0.5939147231584829,\n",
       " 0.8835345864393203,\n",
       " 0.886562391442666,\n",
       " 0.40562736346383366,\n",
       " 0.9548356177776232]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "N = 10\n",
    "rand_test = [random.random() for _ in range(N)]\n",
    "rand_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-2.845943940492324, -6.104732815876872, -3.9427201705349324, 1.8284723907413607, -0.052880777301965054, 1.8782944631696594, 7.670691728786405, 7.73124782885332, -1.8874527307233269, 9.096712355552462]\n"
     ]
    }
   ],
   "source": [
    "rand_test_out = []\n",
    "test_min = -10\n",
    "test_max = 10\n",
    "for i in range(len(rand_test)):\n",
    "    rand_test_out.append((test_max-test_min)*rand_test[i]+test_min)\n",
    "print(rand_test_out)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Skeleton\n",
    "def generate_uniform(N,x_min,x_max):\n",
    "    out = []\n",
    "    ### BEGIN SOLUTION\n",
    "    randoms = [random.random() for _ in range(N)]\n",
    "    for i in range(len(randoms)):\n",
    "        out.append((x_max - x_min)*randoms[i]+x_min)\n",
    "    ### END SOLUTION\n",
    "    return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Max is : 9.461076111766694\n",
      "Min is : -9.064442184169572\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[5.011684839427291,\n",
       " -8.240585162948124,\n",
       " 0.27269465487309574,\n",
       " 3.04819100694443,\n",
       " 9.461076111766694,\n",
       " 5.213260157905541,\n",
       " -4.358751676761809,\n",
       " -9.064442184169572,\n",
       " -7.398288247677847,\n",
       " 3.205812667907807]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_1 = generate_uniform(N=10,x_min=-10,x_max=10)\n",
    "\n",
    "print(\"Max is :\",max(test_1))\n",
    "print(\"Min is :\",min(test_1))\n",
    "test_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data Type: <class 'list'>\n",
      "Data Length: 1000\n",
      "Type of Data Contents: <class 'float'>\n",
      "Data Minimum: -9.912564241064583\n",
      "Data Maximum: 9.991425932838574\n"
     ]
    }
   ],
   "source": [
    "# Test your solution here\n",
    "data=generate_uniform(1000,-10,10)\n",
    "print (\"Data Type:\", type(data))\n",
    "print (\"Data Length:\", len(data))\n",
    "if len(data)>0: \n",
    "    print (\"Type of Data Contents:\", type(data[0]))\n",
    "    print (\"Data Minimum:\", min(data))\n",
    "    print (\"Data Maximum:\", max(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 2a:* \n",
    "Write a function that computes the mean of values in a list. Recall the equation for the mean of a random variable $\\bf{x}$ computed on a data set of $n$ values $\\{ x_i \\} = \\{x_1, x_2, ..., x_n\\}$  is ${\\bf\\bar{x}} = \\frac{1}{n} \\sum_i^n x_i$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Skeleton\n",
    "def mean(Data):\n",
    "    m=0.\n",
    "    \n",
    "    ### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "    ### END SOLUTION\n",
    "    \n",
    "    return m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test your solution here\n",
    "print (\"Mean of Data:\", mean(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 2b:* \n",
    "Write a function that computes the variance of values in a list. Recall the equation for the variance of a random variable $\\bf{x}$ computed on a data set of $n$ values $\\{ x_i \\} = \\{x_1, x_2, ..., x_n\\}$  is ${\\bf\\langle x \\rangle} = \\frac{1}{n} \\sum_i^n (x_i - {\\bf\\bar{x}})$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Skeleton\n",
    "def variance(Data):\n",
    "    m=0.\n",
    "    \n",
    "    ### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "    ### END SOLUTION\n",
    "    \n",
    "    return m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test your solution here\n",
    "print (\"Variance of Data:\", variance(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Histogramming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 3:* Write a function that bins the data so that you can create a histogram. An example of how to implement histogramming is the following logic:\n",
    "\n",
    "* User inputs a list of values `x` and optionally `n_bins` which defaults to 10.\n",
    "* If not supplied, find the minimum and maximum (`x_min`,`x_max`) of the values in x.\n",
    "* Determine the bin size (`bin_size`) by dividing the range of the function by the number of bins.\n",
    "* Create an empty list of zeros of size `n_bins`, call it `hist`.\n",
    "* Loop over the values in `x`\n",
    "    * Loop over the values in `hist` with index `i`:\n",
    "        * If x is between `x_min+i*bin_size` and `x_min+(i+1)*bin_size`, increment `hist[i].` \n",
    "        * For efficiency, try to use continue to goto the next bin and data point.\n",
    "* Return `hist` and the list corresponding of the bin edges (i.e. of `x_min+i*bin_size`).    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution\n",
    "def histogram(x,n_bins=10,x_min=None,x_max=None):\n",
    "    ### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "    ### END SOLUTION\n",
    "\n",
    "    return hist,bin_edges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test your solution here\n",
    "h,b=histogram(data,100)\n",
    "print(h)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 4:* Write a function that uses the histogram function in the previous exercise to create a text-based \"graph\". For example the output could look like the following:\n",
    "```\n",
    "[  0,  1] : ######\n",
    "[  1,  2] : #####\n",
    "[  2,  3] : ######\n",
    "[  3,  4] : ####\n",
    "[  4,  5] : ####\n",
    "[  5,  6] : ######\n",
    "[  6,  7] : #####\n",
    "[  7,  8] : ######\n",
    "[  8,  9] : ####\n",
    "[  9, 10] : #####\n",
    "```\n",
    "\n",
    "Where each line corresponds to a bin and the number of `#`'s are proportional to the value of the data in the bin. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution\n",
    "def draw_histogram(x,n_bins,x_min=None,x_max=None,character=\"#\",max_character_per_line=20):\n",
    "    ### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "    ### END SOLUTION\n",
    "\n",
    "    return hist,bin_edges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test your solution here\n",
    "h,b=histogram(data,20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functional Programming\n",
    "\n",
    "*Exercise 5:* Write a function the applies a booling function (that returns true/false) to every element in data, and return a list of indices of elements where the result was true. Use this function to find the indices of entries greater than 0.5. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def where(mylist,myfunc):\n",
    "    out= []\n",
    "    \n",
    "    ### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "    ### END SOLUTION\n",
    "    \n",
    "    return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test your solution here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 6:* The inrange(mymin,mymax) function below returns a function that tests if it's input is between the specified values. Write corresponding functions that test:\n",
    "* Even\n",
    "* Odd\n",
    "* Greater than\n",
    "* Less than\n",
    "* Equal\n",
    "* Divisible by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def in_range(mymin,mymax):\n",
    "    def testrange(x):\n",
    "        return x<mymax and x>=mymin\n",
    "    return testrange\n",
    "\n",
    "# Examples:\n",
    "F1=inrange(0,10)\n",
    "F2=inrange(10,20)\n",
    "\n",
    "# Test of in_range\n",
    "print (F1(0), F1(1), F1(10), F1(15), F1(20))\n",
    "print (F2(0), F2(1), F2(10), F2(15), F2(20))\n",
    "\n",
    "print (\"Number of Entries passing F1:\", len(where(data,F1)))\n",
    "print (\"Number of Entries passing F2:\", len(where(data,F2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "### END SOLUTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test your solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 7:* Repeat the previous exercise using `lambda` and the built-in python functions sum and map instead of your solution above. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "### END SOLUTION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Monte Carlo\n",
    "\n",
    "*Exercise 7:* Write a \"generator\" function called `generate_function(func,x_min,x_max,N)`, that instead of generating a flat distribution, generates a distribution with functional form coded in `func`. Note that `func` will always be > 0.  \n",
    "\n",
    "Use the test function below and your histogramming functions above to demonstrate that your generator is working properly.\n",
    "\n",
    "Hint: A simple, but slow, solution is to a draw random number test_x within the specified range and another number p between the min and max of the function (which you will have to determine). If p<=function(test_x), then place test_x on the output. If not, repeat the process, drawing two new numbers. Repeat until you have the specified number of generated numbers, N. For this problem, it's OK to determine the min and max by numerically sampling the function.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_function(func,x_min,x_max,N=1000):\n",
    "    out = list()\n",
    "    ### BEGIN SOLUTION\n",
    "\n",
    "    # Fill in your solution here        \n",
    "    \n",
    "    ### END SOLUTION\n",
    "    \n",
    "    return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A test function\n",
    "def test_func(x,a=1,b=1):\n",
    "    return abs(a*x+b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 8:* Use your function to generate 1000 numbers that are normal distributed, using the `gaussian` function below. Confirm the mean and variance of the data is close to the mean and variance you specify when building the Gaussian. Histogram the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "def gaussian(mean, sigma):\n",
    "    def f(x):\n",
    "        return math.exp(-((x-mean)**2)/(2*sigma**2))/math.sqrt(math.pi*sigma)\n",
    "    return f\n",
    "\n",
    "# Example Instantiation\n",
    "g1=gaussian(0,1)\n",
    "g2=gaussian(10,3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise 9:* Combine your `generate_function`, `where`, and `in_range` functions above to create an integrate function. Use your integrate function to show that approximately 68% of Normal distribution is within one variance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def integrate(func, x_min, x_max, n_points=1000):\n",
    "    \n",
    "    return integral"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}