2-Fundamentals-2-PropertiesOfGaussians.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.2. Properties of the Gaussian distribution\n",
    "\n",
    "Once we have acquired a certain amount of familiarity with the gaussian distribution, we can go along some of its principal properties, which are the main reason of this distribution's wide usage in robotics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imports\n",
    "\n",
    "import numpy as np\n",
    "from numpy import random\n",
    "from scipy import stats\n",
    "from scipy import signal\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from ipywidgets import interact, interactive, fixed, interact_manual\n",
    "import ipywidgets as widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.1 Central limit theorem\n",
    "\n",
    "**Property.** The sum of N independent and identically distributed (i.i.d.) random variables, i.e. that belong to the same distribution and are independant to each other, becomes increasingly Gaussian the larger is N.\n",
    "\n",
    "This property holds true regardless of the probability distribution was used to create the samples. It is one of the key concepts in probability, as it allows the generalization of many problems.\n",
    "\n",
    "You can see a video demonstration of this by running the cell bellow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%HTML\n",
    "<center>\n",
    "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/dlbkaurTAUg?autoplay=0&mute=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Assigment.**\n",
    "\n",
    "We ask you to create a similar demonstration as the example above.\n",
    "\n",
    "- Complete the following `plot_sum_demo` function. This function returns a vector of length `v_length`, which results from the sum of `N` randomly generated vectors using an uniform distribution $[0, 1)$. Each random vector should have the same length (for example `v_lenght=100`).\n",
    "- Inside the function, plot the corresponding histogram.\n",
    "- Finally, check that the resulting figure has the shape of a gaussian. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_sum(v_length, N):\n",
    "    \n",
    "    #create the vector for storing the sums\n",
    "    sum_samples = np.zeros(None)\n",
    "    \n",
    "    # Generate N vectors of samples and sum them within sum_samples\n",
    "    for _ in range(0, None):\n",
    "        sum_samples += random.rand(None)\n",
    "        \n",
    "    # Plot the resultant histogram\n",
    "    plt.hist(None,\n",
    "             bins=25, density=True,\n",
    "             stacked=True, edgecolor='black')    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "v_length = 1000\n",
    "N = 10\n",
    "plot_sum(v_length, N)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now play a bit with the number of randomly generated vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interact(plot_sum, v_length=fixed(v_length), N=(0, 25, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.2 Product of gaussians\n",
    "\n",
    "The weighted sum of two gaussians, results in a random variable which its the product of both. This product of 2 gaussians is defined as:\n",
    "\n",
    "$$\n",
    "     N\\left(\n",
    "        \\frac{\\sigma_2^2\\mu_1+\\sigma_1^2\\mu_2}\n",
    "        {\\sigma_1^2+\\sigma_2^2},\n",
    "         \\frac{\\sigma_1^2 \\sigma_ 2^2}\n",
    "         {\\sigma_1^2 + \\sigma_ 2^2}\n",
    "     \\right)\n",
    "$$\n",
    "\n",
    "**Assignment.**\n",
    "\n",
    "Complete the following function to compute the product of two gaussians distributions.\n",
    "\n",
    "Draw the result and check that corresponds to the formula above playing with different distributions.\n",
    "\n",
    "![](images/fig2-2-3.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gaussians_product(mu1,mu2,sig1,sig2,x):\n",
    "    \n",
    "    var1, var2 = None, None # Get the variances from the standar deviations\n",
    "\n",
    "    X = np.arange(-12, 12, 1/x)\n",
    "    pdf1 = stats.norm(loc=None, scale=None).pdf(None)\n",
    "    pdf2 = stats.norm(loc=None, scale=None).pdf(None)\n",
    "\n",
    "    plt.plot(X, pdf1, label='N({},{})'.format(mu1, sig1))\n",
    "    plt.plot(X, pdf2, label='N({},{})'.format(mu2, sig2))\n",
    "    \n",
    "    # Get the parameters defining the gaussian distribution resulting from their product\n",
    "    mu3 = None\n",
    "    sig3 = None\n",
    "    c = stats.norm(loc=None, scale=None).pdf(X)\n",
    "\n",
    "    plt.plot(X, c, label='Avg. = N({},{})'.format(mu3, sig3))\n",
    "    plt.legend()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mu1, sig1 = 1, 1\n",
    "mu2, sig2 = 4, 2\n",
    "x = 1000    \n",
    "\n",
    "gaussians_product(mu1,mu2,sig1,sig2,x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.3 Linear transformation of gaussian random variables.\n",
    "\n",
    "**Property.** The gaussian distributions are closed under linear transformations, i.e. when we apply a sum or product to normal random variables, the result is also a normal random variable.\n",
    "\n",
    "This is also a remarkable property, for example in the field of robotics we can *operate normally over random distributions* as long as we only use linear functions.\n",
    "Otherwise, if we are in need to apply a *non-linear transformation*(e.g. sine, cosine, ...), the resulting probability distribution *it will not correspond to any Gaussian pdf*, causing additional complications in the process.\n",
    "\n",
    "**Assignment.**\n",
    "\n",
    "- Generate a number `n_samples` of random samples from the dist. $N(1,1)$.\n",
    "- Then transform it following the expression $y = a*x + b$ and plot the result for $a=b=2$.\n",
    "- Finally, draw on top the pdf of $N(4,4)$ and check that both are the same.\n",
    "\n",
    "![](images/fig2-2-4.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_transformation(n_samples, a, b):\n",
    "    \"\"\"Apply lineal transform. Generating n_samples samples from N(1,1)\"\"\"\n",
    "    \n",
    "    # Generates n_samples from N(1,1)\n",
    "    mu = None\n",
    "    stdv = None\n",
    "    samples = stats.norm(loc=None, scale=None).rvs(None)\n",
    "    \n",
    "    samples_2 = None # Apply the linear transformation to the samples\n",
    "\n",
    "    # Plot histogram (blue bars)\n",
    "    n, bins, patches = plt.hist(samples_2, bins=90, density=True, stacked=True)\n",
    "\n",
    "    delta = 1/samples.size \n",
    "    X = np.arange(bins[0], bins[-1], delta)\n",
    "    A = stats.norm(loc=None, scale=None).pdf(None) # Evaluate N(1,1) in X\n",
    "    B = stats.norm(loc=None, scale=None).pdf(None) # Evaluate the resultant distribution in X\n",
    "    \n",
    "    # Show results\n",
    "    plt.plot(X, A, color='green', label='N({},{})'.format(mu, stdv))\n",
    "    plt.plot(X, B, color='red', label='N({},{})'.format(a*mu+a, stdv*a))\n",
    "    plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_samples = 3000\n",
    "a = 2\n",
    "b = 2\n",
    "linear_transformation(n_samples, a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now play a bit with different values for $a$ and $b$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interact(linear_transformation, n_samples=fixed(n_samples), b=(-5, 5, 1), a=(1, 10, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Student discussion\n",
    "In the cell below, discuss what has been done in the notebook, what you have found interesting, or any other relevant thought."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style=\"color:green\">***Write your answer here***</span>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}