SVMs.ipynb

{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "SVMs"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Preface - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "These are notes I wrote for myself while studying for PhD qualifying exams at Stanford. \n",
      "\n",
      "I wanted to published them so that others could benefit.\n",
      "\n",
      "They draw from two sources:\n",
      "\n",
      "* CS229, an excellent machine learning class at Stanford taught by Andrew Ng.\n",
      "* An Introduction to Statistical Learning (ISL), an excellent text by G. James, D. Witten,  T. Hastie and R. Tibshirani (Springer, 2013).\n",
      "\n",
      "Some of the figures are taken from ISL with permission from the authors."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Quick cheatsheet - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "$\\textbf{Algorithm type} -$\n",
      "\n",
      "* Classification: It will compute a class label.\n",
      "* Discriminatative.\n",
      "\n",
      "$\\textbf{Model} -$\n",
      "\n",
      "* Classes for $y \\in \\{-1,1\\}$.\n",
      "* $h(z) = g(z)$ where $g(z)$ is a yet undefined step-funtion.\n",
      "* $z=w^Tx + b$ is a linear combination of features and parameters.\n",
      "* If $y^i=1$, we train to generate $w^Tx + b >> 0$ and our classifier picks 1.\n",
      "\n",
      "$\\textbf{Assumptions} -$\n",
      "\n",
      "$\\textbf{Likelihood estimate} -$   \n",
      "\n",
      "* We did not use a probabilistic framing for SVMs.\n",
      "\n",
      "$\\textbf{Strategies for training our model parameters} -$  \n",
      "\n",
      "(1) Use convex optimization:\n",
      "\n",
      "* Our objective is to maximize the geometric margin.\n",
      "* We set this up as an optimization problem and set it up in dual form.\n",
      "* $f(w) = \\|w\\|^{2}$\n",
      "* $g_i(w) = -y^i (w^T x^i + b) + 1 $ with $g_i(w) \\leq 0$ for the optimal margin (meaning all points are correctly classified).\n",
      "* $ L(w,b,\\alpha) = \\|w\\|^{2} - \\sum\\limits_{i = 1}^{k} \\alpha_i [y^i (w^T x^i + b) -1] $\n",
      "* In the dual formulation, we first minimize the Lagrangian with respect to $\\omega$: $ \\theta(\\alpha,\\beta) = min_{\\omega,b} L(\\omega,b,\\alpha) $\n",
      "* This leads to the optimal parameter value, $\\omega$.\n",
      "* We further use this to find the values of $\\alpha$: $ max_{\\alpha; \\alpha \\geq 0} W(\\alpha)$.\n",
      "\n",
      "$\\textbf{Implementing the model} -$\n",
      "\n",
      "* Now, if we want to make a prediction for a new point, x: \n",
      "* $ \\omega^T x + b = (\\sum_{i=1}^m \\alpha_i y^i x^i)^T x + b = \\sum_{i=1}^m \\alpha_i y^i <x^i x> + b $\n",
      "* Where $\\alpha_i > 0$ only for points closest to the decision boundary (the support vector)!\n",
      "* Further, it involoves an inner product, meaning that we can Kernelize it to make this computation efficient.\n"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Convex optimization - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Convex optimization: \n",
      "    \n",
      "* Find a global solution for objective function within polynomial time.\n",
      "\n",
      "Convex sets:\n",
      "\n",
      "* Think of the set as a circle.\n",
      "* We draw a line between any two points.\n",
      "* If all points on the line are within the set, the set ($C$) is convex.\n",
      "* All points on a straight line that passes between $x_1,x_2$ are described by: $ x_3 = \\theta x_1 + (1-\\theta) x_2 \\in C $.\n",
      "* If this statement is true for any $x,y$, then the set is convex.\n",
      "* Examples include: All real numbers, non-negative numbers, norm ball ($\\| 1 \\| \\leq 1$). \n",
      "* Importaintly, positive semi-definite matricies are a convex set.\n",
      "\n",
      "Convex functions:\n",
      "\n",
      "* Convex funtion: $ f(\\theta x_1 + (1-\\theta) x_2) \\leq f(x_1) + (1-\\theta)f(x_2) $\n",
      "* Will always have a global optimum.\n",
      "\n",
      "First order condition of convexity: \n",
      "\n",
      "* The intution here is that we show that a function looks like a quadratic.\n",
      "* $f(x_2) \\geq f(x_1) +\\nabla f(x_1)^T (x_2-x_1)$\n",
      "* Simply, $x_2-x_1$ is the $x$ distance between points.\n",
      "* $\\nabla f(x_1)^T$ is the gradient of the function at $x_1$.\n",
      "* $\\nabla f(x_1)^T (x_2-x_1)$ gives an estimate for $f(x_2)$.\n",
      "* We simply show that the actual value of $f(x_2)$ is always greater than this estimate.\n",
      "\n",
      "Second order condition of convexity: \n",
      "\n",
      "* The second derivative is $\\geq 0$.\n",
      "* For example, proove that $y=x^2$ is convex: the second derivative is 2, which is $\\geq 0$.\n",
      "* Note that this can easily be extended to more complex functions: if working with a matrix, simply take the Hessian.\n",
      "* Moreover, if Hessian matrix is positive semi-definite, then the function is convex.\n",
      "\n",
      "The convex set and function come together to solve problems with a specific form:\n",
      "\n",
      "* Minimize a convex function, $f(x)$.\n",
      "* Subject to $x \\in C$.\n",
      "* Also subject to inequality ($g_i(x) \\leq 0$) and equality ($h_i=0$) constraints.\n",
      "* Where $f(x)$ and $g_i(x)$ are both convex differentiable.\n",
      "* Where $h_i(x)$ is affine, meaning that it is both convex and concave. \n",
      "\n",
      "Special cases:\n",
      "\n",
      "* Linear programming: Objective is affine, inequality condition is affine.\n",
      "* Quadratic programming: Objective is quadratic (e.g., least squares cost function)\n",
      "\n",
      "Use of quadratic programming for least squares minimization:\n",
      "\n",
      "* Typical problem will be constrained least squares.\n",
      "* We want to minimize our model error.\n",
      "* Objective is $\\frac{1}{2} \\| Ax - b \\|^2$\n",
      "* With constraints: (1) matrix format, (2) equality and / or inequality constraints."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Linear decision boundary - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Previously, we showed that logistic regression is a classification algorithm. \n",
      "\n",
      "Given $h_{\\theta}(x)$ and the fact that $y \\in \\{0,1\\}$, $h_{\\theta}(x)=0.5$ is the decision boundary.\n",
      "\n",
      "This is produced when $\\theta^Tx = \\theta_1 x_1 + \\theta_2 x_2 + \\theta_3 = 0$.\n",
      "\n",
      "In turn, we can define a line of values for $\\theta_{1,2}$ over which the decision is ambiguous.\n",
      "\n",
      "In this case, the decision boundary is linear:\n",
      "\n",
      "$x_2 = -\\frac{\\theta_1}{\\theta_2} x_1 + \\theta_3 $"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "import matplotlib.pyplot as plt\n",
      "import pandas as pd\n",
      "import seaborn as sns\n",
      "%matplotlib inline\n",
      "\n",
      "# Value for theta\n",
      "theta=np.array([4,-4,5],dtype=float)\n",
      "\n",
      "# Decision boundary\n",
      "plt.subplot(2,2,2)\n",
      "x_min=0\n",
      "x_max=10\n",
      "x=np.arange(x_min, x_max, .1 )\n",
      "y=-(1/theta[2])*(theta[1]*x+theta[0])\n",
      "plt.plot(x,y,lw=2,color='black',alpha=0.75)\n",
      "plt.xlim([0,10])\n",
      "plt.ylim([0,10])\n",
      "plt.title('Decision boundary',fontsize=10)\n",
      "plt.xlabel('$x_1$',fontsize=10)\n",
      "plt.ylabel('$x_2$',fontsize=10)\n",
      "\n",
      "plt.tight_layout()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAOoAAACmCAYAAADUDhNZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGo5JREFUeJzt3XlUU3fex/F3WASrMMoibtBFa923QaS4sysESAIc/cOe\n1pljbadjp9rjGbt5Thc9djpOn047M7VzbO3pdBGSACqiIIpWQcWx7o5LcQG1KKgssoTkPn/wmEc7\nKmKBm8D39ReG3JsPyCe/5P7u/UWjKIqCEMKhuagdQAjRMimqEE5AiiqEE5CiCuEEpKhCOAEpqhBO\nQIraToYNG0ZSUhLx8fEkJiby+eef87AzYR999BGFhYX3/P63335LRkbGw0a127NnDwsWLPjF+2mN\nP/7xj2zevLlDH9MZuakdoLPy9PS0l6eyspLFixdTU1PD73//+1bva+HChff9/uzZsx8qoyPQaDRo\nNJoHvr/VasXV1bUdEzkmGVE7gI+PD2+//TZfffUV0PzHtnLlSpKTk0lISOC7776z33f16tVotVoS\nExNZtWoVcOeo88EHHxAXF0dCQgLvv/8+AH/9619Zs2YNAMePHyc1NZWEhAReeuklqqqqAJg7dy4f\nfPABKSkpxMTEUFxc/F85NRoNNTU1PP/888TGxrJs2TL7q4ANGzag1WrRarV88MEH9m3GjRtn/zon\nJ4elS5faM7/77rvMnj2byMhIe35FUXj77beJjY3lueeeo6Kiwr79xx9/THJyMlqtlrfeest++9y5\nc1m+fDkGg4G///3vRERE0NTUBEBNTQ0RERFYrdbW/8c4ERlRO0hgYCA2m42Kigry8vLw9vYmPT2d\nxsZG5syZw6RJkzhz5gz5+fmkp6fj4eFhL9mtUefatWvk5eWRk5MDNP+R3v59gCVLlrBs2TKCg4P5\n6KOP+Pjjj3nttdeA5ieItLQ0CgoK+OSTT/j888/vyKgoCocOHSI7O5v+/fvz29/+li1btjB27Fj+\n/Oc/YzKZ8Pb2Zt68eeTl5REZGXnH9j8fGa9evcq3337LmTNneOGFF4iJiSE3N5ezZ8+yadMmrly5\nQlxcHMnJyUBzIV966SX7z7Ft2zZmzJgBQFNTE0ajEYCysjK2b99OZGQkGzduJDo6utOPsjKiqmDX\nrl1kZGSQlJREamoqN27c4Ny5cxQVFWEwGPDw8ADA29v7ju28vb3x8PDgtddeIzc3136/W2pqaqip\nqSE4OBgAnU53x8gZHR0NwIgRIygrK7trttGjRzNw4EBcXFyIi4tj//79HDlyhJCQEHr37o2rqyta\nrfauI/LtNBqNvciDBg3i6tWrAOzbt4/4+Hg0Gg19+vQhNDTUvk1RURGpqalotVqKioo4ffq0/Xuz\nZs2yf52SkoLJZALAbDZjMBjum6UzkBG1g1y4cAEXFxd8fX0BeOutt5g0adId9/n+++/vub2iKLi6\nupKWlkZhYSGbN2/mq6++Yu3atffd5nbdunUDwMXFxf7S8eduHxUVRbnr+8fbb7/9+/X19Xfcz93d\n/b+yaDSaux5Ua2ho4O2338ZkMhEQEMDHH39MQ0OD/fvdu3e3fz1+/HjKysrYs2cPVquVwYMH3/Vn\n6UxkRO0AlZWVLFu2jLlz5wIwefJkvv76a3tZSkpKqKurIywsDKPRaP+Dv3Hjxh37uXnzJtXV1Uyb\nNo2lS5dy4sQJoLkEiqLQs2dPvL297aNdZmYmISEhrcp66NAhSktLsdlsbNq0ieDgYEaPHs2+ffu4\ndu0aVquV7OxsJkyYAICfnx9nzpzBZrORl5fX4v4nTJhAdnY2NpuN8vJy9uzZA2AvZa9evaitrbW/\nvL+XpKQkXn311S4xmoKMqO2moaGBpKQkmpqacHV1JSkpiWeffRZofulWVlaGTqcDmg82ffLJJ0yZ\nMoXjx49jMBhwd3dn2rRpvPLKK0DzSFRbW8uLL75IQ0MDiqLYD9zc/h515cqVLFu2jLq6OoKCglix\nYsVd891tpNRoNIwaNYp33nmHc+fOERoaSlRUFACLFy/mmWeeAWD69OmEh4fbb1+wYAE+Pj6MHDmS\nmzdv3vUxbn0dFRVFUVERs2bNon///vaDUd7e3qSkpBAfH4+fnx+jR4++7+83Pj6eDz/8kPj4+Pve\nr7PQtPdlbkuXLqWgoABfX1/Wr18PwPXr13nllVe4ePEiAwYM4MMPP/yv92NC3E9OTg7btm1j5cqV\nakfpEO3+0tdgMPDPf/7zjttWr15NWFgYmzdvJjQ0lNWrV7d3DNGJvPPOO/zlL3/hxRdfVDtKh2n3\nERWgtLSUF154wT6ixsbG8tVXX+Hn58eVK1eYO3dui+9JhOjKVDmYVFFRgZ+fH9B8MOL2SW8hxH9T\n/ajvg55CJivGiK5MlaO+vr6+XLlyBX9/f8rLy/Hx8WlxG41Gw5Ur1R2Q7sH4+3tJnvuQPPfn7+/V\nqvurMqKGh4djNpsByMjI+K9T0YQQd2r3oi5atIjZs2dTUlLCtGnTMBqNzJ8/n927dxMTE0NRURHz\n589v7xhCOLUOOerbVhztpYvkuTfJc39O8dJXCNE6UlQhnIAUVQgnIEUVwglIUYVwAlJUIZyAFFUI\nJyBFFcIJSFGFcAJSVCGcgBRVCCcgRRXCCai6CuGnn35KVlYWLi4uDBkyhBUrVtjXnhVC/D/VRtTS\n0lLWrVuH2Wxm/fr1WK1WNm7cqFYcIRyaaiNqz549cXNzo66uDhcXF+rr6wkICFArjhAOTbWi9urV\ni3nz5jF9+nQ8PT2ZPHkyYWFhasURwqGp9tL3/PnzrF27lvz8fHbu3MnNmzfJyspSK44QDk21EfXI\nkSOMGzeO3r17A80fdXDgwAESEhLuuU1rr4pvb5Ln/iRP21GtqE888QR/+9vfqK+vx8PDg8LCwhY/\nb8TRltKQPPcmee6vtU8aqhV16NChJCYmYjAYcHFxYfjw4aSmpqoVRwiHJoubPSRHfIaWPPfmiHla\nQ85MEsIJSFGFcAJSVCGcgBRViA5isViwWCwPta2qJ+UL0RVcu1bJhg1Z5ORsxM/Pn//5n7+1eh9S\nVCHayeXLl8jMNLF1a659JH3ssccfal9SVCHaWEnJj2RkGPn++x3YbDYAJk4MRa9PZciQpx5qn1JU\nIdrI0aNHMJvT2b9/HwAuLq7MmBFBUpKBoKBHf9G+pahC/AKKolBcvBezOZ3jx48B0K1bN6KjY9Fq\ndfTp06dNHkeKKsRDsFqtfP/9DkymNM6fPwc0X2MdF5fAzJlx/OpXvdr08aSoQrRCQ0MDW7fmkplp\npLy8HAAfH18SEpKIiorlkUceaZfHlaIK8QBqamrIydnI+vWZVFXdAKBfv/7o9clMmxaOu7t7uz6+\nFFWI+6isrGD9+kw2b86mrq4OgEGDnkSvTyY0NAwXl445Z0jVolZVVfHGG29w6tQpNBoNy5cvZ+zY\nsWpGEgJongM1m43k5+fS1NQEwKhRY9DrkxkzZhwajaZD86ha1Pfee4+pU6fy0Ucf0dTUZH/GEkIt\nP/54BpMpjcLCXdhsNjQaDaGhYej1KTz55BDVcqlW1OrqaoqLi1m5cmVzEDc3vLycd6kM4bwUReHo\n0SOYTGkcOLAfAFdXN8LDI9DpDAwcGKhyQhWLWlpaio+PD0uXLuXEiROMGDGC119/ne7du6sVSXQx\nNpuNffv2YjKt4+TJ/wDg4eFBVFQsiYl6/Pz8VE74/1Rb4eHw4cPMnj2bb775htGjR/Pee+/Rs2dP\nXn75ZTXiiC6kqamJ/Px8vvnmG86ePQuAt7c3er0enU6Ht7e3ugHvQrURtW/fvgQEBNgXNIuJieGz\nzz677zaOtpSG5Lk3R8xz4cIV8vK2kJVl4sqVKwD4+vqRmKgjKioWT09PGho65u/MaRY38/f3p1+/\nfpSUlPD4449TWFjI4MGD1YojOrGammpycjL4+uvvqK6uAmDAgIHodAamTp3R7nOgbUHVo75vvvkm\nr776KhaLhaCgIFasWKFmHNHJVFRUkJVlZsuWTVitFiwWK08+OQS9PoWQkNAOmwNtC6oWdejQoRiN\nRjUjiE7o4sUyzGYj27dvtc+BPv30RGbOTGTkyNEdPgfaFuTMJNFpnDlzCqMxjaKi3SiKgkaj4emn\nJ2EwpBAaOt6h3jO3lhRVODVFUThy5BBGYxoHDx4AmudAp08PR6czMGDAQJUTtg0pqnBKNpuNvXuL\nMJnSOHXqJACenp5ER88kIUGHr6+vygnbVquKWltbS48ePbBYLLi4uODq6tpeuYS4K4vFws6d2zGZ\n0ikrKwXAy8ubuDgts2bF4+XleHOgbeGBi/rZZ59x7do1rFYrzz//PKtWreLdd99tz2xC2NXX15Ob\nu5msLDNXrzbPgfr7+5OQoCcyMhpPT0+VE7avBy7qmDFjGDNmDG5ubmRnZ9sXbRKiPVVXV7Fx43qy\nszfY50ADA4PQ6ZKZMmUabm5d491biz/lhQsX8Pf3p3v37phMJubMmYNWq7Uf9haiPVy9epWsLDO5\nuTnU19cDMGTIU+j1qUyYEOJUc6BtocWirlmzhtjYWCZOnEh9fT3FxcUEBwej0+k6Ip/oYsrKSv9v\nDjQfq7V5MBg7djwGQyojRox0yjnQttBiUUePHk1paSn9+/dnwoQJ5ObmdkQu0cWcOnUSkymNPXsK\nURQFFxcXJk2agl6fzBNPyKmlLRb10qVLBAYG8sUXX3Dy5EnGjx9PVFRUR2QTnZyiKBw+fJD09HUc\nPnwQaL4uOTw8iqQkPf369Vc5oeNosaiBgYHExMSg1WqprKyUEVX8YjabjT17CjEa0zhz5hQA3bt3\nJzY2jvj4BHx8OtccaFtosaizZs2yX9hdWlrK1atXOyKX6IQsFgsFBfmYzUYuXiwDwNv7V2i1icTG\nxtGzZ0+VEzquFovq6urKiBEjgOb3q7euH20LVqsVg8FA3759+cc//tFm+xWOpa6ujtzcHLKyMqio\naH6i79OnDwkJeiIiojr9HGhbUHUS6ssvv2TQoEHU1taqGUO0k6qqG2Rnb2DjxixqamoACAp6FJ0u\nmcmTp3aZOdC2oNpv6vLlyxQUFLBgwQK++OILtWKIdlBeXs53363FbM6koaEBgKFDh6HXpxAcHNJl\np1h+CdWKunz5cpYsWWJ/phXO7/z5c2RkGCko2I6rK1gsVn796wnodMmMGDFS7XhOTZWibtu2DV9f\nX4YPH86ePXseeLvWrjPT3iRPs+PHj/Ovf/2LXbt2AeDu7sKMGTOYM2cOgwYNUiXT3Tja/1drqLIK\n4apVq8jMzMTV1ZXGxkZqamqIjo7m/fffv+92jnThryMu3tWReRRF4eDBAxiN6zhy5DAA7u7uhIdH\nkpRkYNSoIV3699OS1j5pqLZc6C179+5lzZo1D3TU19F+0V0xj81mo7BwFyZTOj/+eBqARx55hJiY\nWWi1ifTu7dOheR6UI+ZpDTnsJh6IxWJh+/atmM1GLl26CECvXr2Jj08gNjaOHj16qJywc1O9qCEh\nIYSEhKgdQ9zDzZs32bJlE+vXZ1JZWQFAQEAAiYkGwsMj8fDwUDlh16B6UYVjunHjOhs2ZLFp0wb7\nPPdjjz2OTpfMpElTZHWPDiZFFXcoLy8nM9NIXt4WGhsbARg2bDgGQyrjxwfLHKhKpKgCgHPnzpKR\nYWTHjgJsNisAv/71BAyGVIYNG65yOiFF7eJOnDiO2ZzG3r3N89kuLq5MmzYDnS6ZRx99TN1wwk6K\n2gUpisK//12M2ZzO0aNHgOY50MjIaBITDQQEBKicUPycFLULsVqt7N79PWZzOiUlPwLQo0cPZs6M\nJy5OS69evVVOKO5FitoFNDY2sm3bVszmdH766TLQPAeakJBEdPRMmQN1AlLUTqy2tpbNm7NZvz6T\n69evAdC3bz90OgPTp0fQrVs3lROKByVF7YSuX7/Ghg1Z5ORstM+BPv74E+j1KYSFTe5yS212BlLU\nTuSnny6TkWFk69ZcLBYLACNHjkKvT2Hs2PEyB+rEpKidwNmzJaxevZ7Nm/Psc6AhIRPR6VIYOnSY\nyulEW5CiOrHjx49hNK5j//59uLs3n9I3Y0YESUkGgoIeVTmdaEuqFfXSpUssWbKEyspKNBoNqamp\nPPPMM2rFcRqKorB//z5MpjSOHz8GQLdu3dDrk5gxYyZ9+vRROaFoD6oV1c3Njddee41hw4ZRW1uL\nXq9n0qRJDrUigCOxWq3s2rUTkymNc+fOAtCzZ0/7HOjgwYEOdb2laFuqFdXf3x9/f3+gedJ90KBB\nlJeXS1F/pqGhgfz8PDIzjfz0008A+Pj4kpCQRFRULI888ojKCUVHcIj3qKWlpRw/frxN1wx2djU1\nNeTkbGTDhixu3LgOQL9+/e1zoO7u7ionFB1J9aVYamtrmTt3Li+++CKRkZFqRnEIlZWVpKenk5WV\nZZ8DHTJkCHPmzGHq1KkyB9pFqVpUi8XCggULmDJlCs8++2yL93ek92BtvQbP5cuXyMgwkp+fd9sc\n6GgMhhTGjBnX4hyoI64JJHnuzWnWTFIUhddff51BgwY9UEk7q5KSHzGZ0ti9+3v7p7hPnPg0en0K\nQ4Y8pXI64ShUK+r+/fvJysriqaeeIikpCYBFixYxdepUtSJ1GEVROHr0CGZzOv/+dzHQfB3ojBmR\n6HQGAgODVE4oHI1qRQ0ODubEiRNqPbwqbDYbxcX7MJnW8Z//NP/sHh4eREXFkJCgtx8FF+LnHOKo\nb2fX1NTEzp0FZGQYOX/+HABeXl7MmqUlLk6Ll5e3ygmFo5OitqP6+nq2bs0lK8tEeXk5AL6+fiQm\n6oiMjKF79+4qJxTOQoraDmpqqtm0qXkOtKrqBgADBgxEpzMwdeoMmQMVrSZFbUMVFRWsX5/B5s3Z\n1NfXAzB48BAMhhRCQkJlDlQ8NClqG7h4sYyMDBPbtuXR1NQEwOjRYzEYUhg1aoxcByp+MSnqL3Dm\nzCnMZiOFhbuw2WxoNBpCQ8MwGFIZPPhJteOJTkSK2kqKonDkyGFycjLZvbsIAFdXNyIimudABwwY\nqHJC0RlJUR+QzWZj3769mEzrOHnyP7i7u+Lp6Ul09EwSEnT4+vqqHVF0YlLUFjQ1NbFjx3bM5nRK\nSy8A4OXlzZw5qUyeHCFzoKJDSFHvob6+nry8LWRmmrh69QoAfn7+JCToiIqKITDQ36FO8hadmxT1\nZ6qrq8jO3sDGjeuprq4CYODAQHQ6A1OmTJc5UKEKKer/qaioICvLzJYtm+xzoE8+OQS9PpWQkIky\nBypUpWpRd+zYwfLly7HZbCQnJzN//vwOz1BWVorZbGT79nys1uY50LFjx6PXpzBy5CiZAxUOQbWi\nWq1W3nnnHT7//HMCAgJITk4mIiKiw9ZMOn36FEbjOvbsKURRFFxcXAgLm4xen8ygQTIHKhyLakU9\ndOgQQUFBDBzYPO8YFxfH1q1b27WoiqJw+PBB0tPXcfjwQaB5NcQZMyJJStLTv/+AdntsIX4J1Yr6\n008/0a9fP/u/AwICOHToULs8ls1mY+/eIozGNE6fPgmAp6cnMTGz0GqTZA5UODzVivow7/1au87M\n7bTaGLTamIfe/m5+SZ72IHnuz9HytIZqhzIDAgK4dOmS/d+XL1+WT7oW4h5UK+rIkSM5d+4cpaWl\nNDY2kp2dTUREhFpxhHBoqn6kxZtvvslvfvMb+/SMrJIvxN2pvgC3EKJlcrqNEE5AiiqEE5CiCuEE\nHL6oO3bsIDY2lujoaFavXq1qlkuXLjF37lzi4uKIj4/nyy+/VDXPLVarlaSkJBYsWKB2FKqqqli4\ncCEzZ85k1qxZ/PDDD6rm+fTTT4mLi0Or1bJ48WIaGxs79PGXLl1KWFgYWq3Wftv169d57rnniImJ\nYd68eVRVVbW8I8WBNTU1KZGRkcqFCxeUxsZGJSEhQTl9+rRqecrLy5Vjx44piqIoNTU1SnR0tKp5\nblmzZo2yaNEi5fnnn1c7irJkyRIlLS1NURRFsVgsSlVVlWpZLly4oISHhysNDQ2KoijKyy+/rJhM\npg7NsG/fPuXo0aNKfHy8/baVK1cqq1evVhRFUT799FPlT3/6U4v7cegR9fbzgd3d3e3nA6vF39+f\nYcOGAXd++LKaLl++TEFBASkpKarmAKiurqa4uJjk5GSgeQrOy0u9s4F69uyJm5sbdXV1NDU1UV9f\n3+En1QQHB+PtfecqIPn5+eh0OgB0Oh15eXkt7sehi3q384Fvfeq22hzlw5eXL1/OkiVLHOJ62dLS\nUnx8fFi6dCk6nY433niDuro61fL06tWLefPmMX36dKZMmYKXlxdhYWGq5bmloqICPz8/APz8/Kio\nqGhxG/X/d+/DUa8Fra2tZeHChbz++uv06NFDtRzbtm3D19eX4cOHozjAdHhTUxPHjh1jzpw5mM1m\nunfvrupxhfPnz7N27Vry8/PZuXMnN2/eJCsrS7U8d6PRaB7o79yhi+qI5wNbLBYWLlxIQkKC6p+Q\nfuDAAfLz8wkPD2fx4sUUFRWxZMkS1fL07duXgIAA+6uMmJgYjh07plqeI0eOMG7cOHr37o2bmxtR\nUVEcOHBAtTy3+Pr6cuVK8zpc5eXl+Pj4tLiNQxfV0c4HVhzsw5cXLVpEQUEB+fn5rFq1itDQUN5/\n/33V8vj7+9OvXz9KSkoAKCwsZPDgwarleeKJJzh48CD19fUoiqJ6nlvCw8Mxm80AZGRkPNATvkOv\nmeRo5wN35Q9fflBvvvkmr776KhaLhaCgIFasWKFalqFDh5KYmIjBYMDFxYXhw4eTmpraoRkWLVrE\n3r17uX79OtOmTWPhwoXMnz+fP/zhDxiNRgYMGMCHH37Y4n7kXF8hnIBDv/QVQjSTogrhBKSoQjgB\nKaoQTkCKKoQTkKIK4QSkqEI4ASmqEE7Aoc9MEu3LarWSnZ3NhQsX6NevH4cOHWLevHkEBgaqHU38\njIyoXdiJEyeIiYkhMDAQm81GbGws/v7+ascSdyFF7cJGjBhBt27d+OGHHwgJCWHixIl4enqSl5fn\nMNf9imZS1C7s0KFDVFZWcvLkSQIDAykuLubq1atkZGSoHU38jLxH7cJ27tyJn58f48ePJzc3l969\ne+Pn58fQoUPVjiZ+Rorahf3ud79TO4J4QPLSV9yhoqKCkpISioqK1I4ibiPXowrhBGREFcIJSFGF\ncAJSVCGcgBRVCCcgRRXCCUhRhXACUlQhnIAUVQgn8L+rLBRByopx0QAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1075b0390>"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Of course, we train our model to obtain parameter values ($\\theta_{1,2,n}$) that seperate our data.\n",
      " \n",
      "Yet, in some cases, a linear decision boundary will not nicely seperate our data.\n",
      "\n",
      "Thus, we need to define a classification algorithm that implement decision boundaries with different shapes."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Functional and geometric margin - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Assume our training set is linearly seperable.\n",
      "\n",
      "$g(z)$ is the logistic funtion where $z=\\theta^T x$:\n",
      "    \n",
      "* If $y^i=1$, we train to generate $\\theta^T x >> 0$ and our classifier picks 1.\n",
      "* If $y^i=0$, we train to generate $\\theta^T x << 0$ and our classifier picks 0.\n",
      "\n",
      "The concept of $\\textbf{funtional margin}$ simply tells us whether a given training example is properly classified.\n",
      "\n",
      "Given a training dataset that is linealy seperable, some bounaries come closer to training examples (less favorable) than others.\n",
      "\n",
      "Moreover, we might choose the decision bounary that stays far away from our training examples.\n",
      "\n",
      "The concept of $\\textbf{geometric margin}$ tells us how close the decision boundary is to each training example.\n",
      "\n",
      "Useful review:\n",
      "\n",
      "* http://stackoverflow.com/questions/20058036/svm-what-is-a-functional-margin"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "The functional margin for the SVM classifier indicates whether training examples are properly classified -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For logistic regression:\n",
      "\n",
      "* Classes: $y \\in \\{0,1\\}$.\n",
      "* Hypothesis: $h(z) = g(z)$ where $g(z)$ is the logistic funtion.\n",
      "* Input: $z=\\theta^T x$ where a linear combination of features and parameters is used.\n",
      "* Funtional margin: if $y^i=1$, we train to generate $\\theta^T x >> 0$ and our classifier picks 1.\n",
      "\n",
      "For a new classifier that sets the stage for SVMs:\n",
      "\n",
      "* Classes: $y \\in \\{-1,1\\}$.\n",
      "* Hypothesis: $h(z) = g(z)$ where $g(z)$ is a yet undefined step-funtion.\n",
      "* It picks 1 if $z \\geq 0$ or it picks 0.\n",
      "* Input: $z=w^Tx + b$, again a linear combination of features and parameters is used.\n",
      "* Funtional margin: if $y^i=1$, we train to generate $w^Tx + b >> 0$ and our classifier picks 1.\n",
      "\n",
      "The funtional margin with respect to a single training example:\n",
      "\n",
      "$\\hat{\\gamma}_i = y^i (w^T x^i + b)$\n",
      "\n",
      "The funtional margin with respect to a full training set is defined as the worst-case functonal margin:\n",
      "\n",
      "$\\hat{\\gamma} = min (\\hat{\\gamma}^i)$\n"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Decision boundary -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As with logistic regression, the decision boundary is produced when $\\omega^Tx + b = \\omega_1 x_1 + \\omega_2 x_2 + b = 0$.\n",
      "\n",
      "The linear decision boundary is:\n",
      "\n",
      "$x_2 = -\\frac{\\omega_1}{\\omega_2} x_1 - \\frac{b}{\\omega_2} $\n",
      "\n",
      "Note that the slope of the decision boundary is $-\\frac{\\omega_1}{\\omega_2}$, which is orthogomal to the slope defined by the parameters, $w$."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Value for theta\n",
      "w=np.array([4,4],dtype=float)\n",
      "b=5\n",
      "\n",
      "# Vals\n",
      "x_min=-10\n",
      "x_max=10\n",
      "\n",
      "# Ensure equal figure dimentions to see Orthogonality\n",
      "fig=plt.figure(figsize=(5,5))\n",
      "\n",
      "# Decision boundary\n",
      "plt.subplot(2,2,2)\n",
      "x=np.arange(x_min, x_max, .1 )\n",
      "y=-w[0]/w[1]*x-b/w[1]\n",
      "plt.plot(x,y,lw=2,color='black',alpha=0.75)\n",
      "plt.xlim([x_min,x_max])\n",
      "plt.ylim([x_min,x_max])\n",
      "plt.title('$\\omega$ and the decision boundary',fontsize=10)\n",
      "plt.xlabel('$x_1$',fontsize=10)\n",
      "plt.ylabel('$x_2$',fontsize=10)\n",
      "\n",
      "y_o=w[1]/w[0]*x\n",
      "plt.plot(x,y_o,lw=2,color='red',alpha=0.75)\n",
      "\n",
      "plt.tight_layout()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAMgAAADLCAYAAAArzNwwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlAlHXix/H3HJyDF0h4oJaaeWepabaaoeIJq5a7dpp0\nrWl5/TC5hpkBhjTzx6bperdpurrrVZqtyc/WsjXNVsW8jwwSQVEU5oA5nt8fJJsJCMnMMwzf118y\nzPP9fnjGD88zw3MoJEmSEAShQkq5AwiCJxMFEYQqiIIIQhVEQQShCqIgglAFURBBqIIoiCBUQRRE\nEKogClINCxYsYOXKlbc9XlRUxNq1a8u/zsnJISoqyqVzVsf48ePv6vvVVZs/b3XdzXr5LURBqkGh\nUFT4+PXr11m3bp1b56yOv/3tb3f1fU9Wk/UiSRJ3e6CI+q6WrkMmT55Mbm4upaWlvPDCC/zhD38g\nJyeHV155hV69evGf//yHsLAwFi1ahJ+fH4sXL2bLli2EhITQvHlzunTpctuY7777Lj/++COjR4/m\nscce45lnnsHhcJCUlHTbeFu3bmXNmjXYbDa6d++OTqdDqbz191NVc1a2/JYtW1i5ciUKhYKOHTsy\nZ84cHnroIf7zn/9gNpuZNm0aeXl5OJ1OXn/9dYYPH17+fYBVq1axadMmAJ566ikmTJhQ5Xr5Nbvd\nzv/8z/9w7Ngx2rdvz9y5c/H396903EmTJvHJJ58AsGLFCiwWC1OmTKnRa9G1a9cqX9OXXnqJHj16\n8P3339OtWzc6duzIhAkTAPjf//1fQkJCeOGFF6r3H0eq4/Ly8qQjR47c8thzzz0n2e32Wx4rLCyU\nJEmSLBaLNGrUKKmwsFDKzs6WOnfuLB0/flySJEmaOnWqtHXrVikrK0saNWqUZLVapaKiImnIkCHS\nypUrb5s7JydHGjVqVPnXlY135swZ6bXXXivPlJycLG3evPmWsaqas7LlT506JUVGRkrXrl2TJEmS\nrl+/LkmSJPXo0UOSJEn67LPPpMTExPI5ioqKbvn+zTktFotkMpmkkSNHSseOHav05/i17Oxs6YEH\nHpC+++47SZIkKS4uTlqxYkWV4/5yfa1YsUJasGBBleuuqvVS2WvasWNH6fDhw+Wv0ZgxYyRJkiSH\nwyENHjy4fLnqqPNbkCNHjjB48GAsFgtXrlyhVatWPProo7dtij/88EN27doFQG5uLhcuXCA4OJjw\n8HA6duwIQJcuXfjpp5+4du0akZGR+Pn54efnR0RERIWb6ooeq2i8oqIivv/+e5588kkArFYrTZs2\nvWW5b7/9ttI5//3vf9+yfElJCU2bNqW4uJjhw4fTuHFjABo2bHjLmA888ABz5sxh3rx5DBw4kF69\net3y/YMHDxIZGYm/vz8AQ4YM4dtvvyUiIqLCn6MizZs356GHHgIgOjqa1atXo1arKx23KjV9LSp7\nTVu0aEH37t0BaNmyJY0bN+b48eNcvnyZzp0706hRoypz/FKdL0hpaSkAe/fupXHjxrRq1Yrw8PBb\ndl+++eYb/v3vf7Nhwwb8/Px4/vnnKSkpAcDX17f8eSqVqvzxX/7nr6gIlaloPEmSGDNmDDNmzKh0\nOYVCcducvyx5RcuvWbOmymz33nsvW7Zs4YsvvuDPf/4zffv2ZfLkydWas7L1UlHuipavaFy1Wo3T\n6Sx/3Gq13jJWTV6Lql7TwMDAW8Z96qmn2LhxIwUFBeW/ZKqrzr9JP3fuHJIk8dlnn9GuXTsOHDhA\nUFDQLc8pLi6mYcOG+Pn5cfbsWQ4fPlzlmL1792bXrl2UlJRQXFzMF198UeGbQ41Gg8lkumPGRx99\nlH/+859cvXoVgMLCQi5evHjHOe+0fN++ffnss88oLCwEyj40+KX8/Hz8/PyIjo4mJiaG48eP3/L9\nnj17smvXLqxWK2azmczMTHr16lWjXwgXL17k0KFDAGzbto1evXrRq1evCscNDg6moKCAwsJCSktL\nb/kZK1PZeqnJazpkyBC+/PJLjh49Sv/+/av9s4EXbEFMJhNRUVE8++yzjB49msjISBISEm55Tv/+\n/fnb3/7GiBEjuO++++jRowdQ+ScinTt3ZsSIEURHRxMSEkK3bt0qfF6TJk146KGHiIqKYsCAATzz\nzDMVPq9du3ZMnTqVmJgYnE4narUanU5HixYtqjVnZct3796dSZMm8fzzz6NUKuncuTPp6enlP9fJ\nkyd55513UCqVqNVq9Hr9LT93ly5dGDt2LOPGjQNg3LhxdOzYkZycnDuu95vj3HfffXz00UfEx8fT\nvn17nn76afz8/CocF8reWI8bN46wsDDatWt3xzkqWi8KhaJGr6mPjw99+/alUaNGNf50UCHV5NeF\nC8TFxfGvf/2LkJCQ8k83CgsLmT59OhcvXqRly5ZkZGTctn8tCNXldDoZO3Ys7733Hq1bt67RsrLv\nYj355JMsX778lseWLl1Kv379+Oc//0nfvn1ZunSpTOmEuu7MmTNERkby6KOP1rgcgGd8zPvrj/+G\nDh0qXb58WZIkScrPz5eGDh0qVzShnpN9C1KRgoKC8o9BmzZtSkFBgcyJhPrKIwvySwqFolpvrCRx\n7QmhKl99BWPH1ngxj/wUKyQkhMuXLxMaGkp+fj7BwcF3XEahUHD5cpEb0t1ZaGgDj8jiKTlA3izq\n/d+gmWsEhx3fOz/9Fh65BYmIiGDz5s0AbNmyhcGDB8ucSKir1AcPoHmnrBwl0WNqvLzsBZkxYwbj\nx4/n/PnzPP7442zcuJFXX32Vr7/+mqFDh7Jv3z5effVVuWMKdZD60Hdo5qSB3U7JqGisL75U4zFk\n/ztIbRK7E56ZA9yfRZ11GE1KMthslA4fieWVSaBQEBraoEbjyL4FEYTapjqahSZVV1aOIcOwvPwn\n+I3n14iCCF5Fdex7NGk6KC2lNGIIlj9NBuVv/28uCiJ4DdXJE2hStCisVkoHRmCZ/OZdlQO8qCCJ\niYnVOrJW8E6q06fQGJJQWK3Y+j+O5Y3pd10O8KKC7N27l5QULWazWe4ogpupzp5Go09EYTZj6/c7\nzFNn1ko5wIsKEhYWxsmTJ0hNTcZiscgdR3AT5flzaPRJKEwmbH0exTw9FlSq2hu/1kaS2fz58wkJ\nacrx48cwGvW3na0meB/lhR8ISk5AUVSErXcfzDPfAnXtHhziNQVp0aIFBoORJk2COXo0C6PRUOlp\nokLdp8z+8edy3MDeszfm2Djw8an9eWp9RBm1aNGSlJR0GjduQlbWYd5+O7X8nHXBeyh/yiFIG4/i\neiH2Hg9jmhXvknKAlxUEoGXLcPT6NBo2bMShQ98xd64Rm80mdyyhlihzL5aVo/Aa9m4PYpqdCL41\nPQSxBvO5bGQZtW7dBoPBSIMGDTl48ADz5qWLkngBZd4lNNp4FFcLsHfpiileCxVczK5W53Tp6DJq\n0+ZedLpUgoKC2L//G+bPn4vdbpc7lvAbKfLz0WjjUF65jL1TZ0wJOvj5uluu5LUFAWjbth06XSoa\njYZ9+74mI2MeDodD7lhCDSkuXyZIOxtlfj6ODg+UlSMgwC1ze3VBANq1ux+tNoXAwED27v2S996b\nf8vFywTPpigoIEgbhzIvD0f7DhRrU0Cjcdv8Xl8QgA4dHiApyYC/vz979nzBwoUZoiR1gOLa1bJy\nXMrF0bY9pmSDW8sB9aQgAB07diovye7dmSxatECUxIMpCq8RpI1HefEnHPe1xaRLQQqq2bkctaHe\nFASgc+cuJCTo8PX1JTNzJ0uWLBIXe/BAihvX0egSUeZk42jdBlNyClIDeS4cWK8KAtC1azcSEpLx\n8fFh584dLFu2WJTEgyiKbqDRJaK68APOluGY9GlIjRrLlqfeFQSge/cexMUloVar2bFjOytXLhMl\n8QCK4iI0uiRU58/hbNGSYoMRqXETWTPVy4IAPPRQT2bPTkSlUrNt21Y+/HCVKImcTCY0ei2qc2dw\nNmtOsSEdKThE7lT1tyAAPXv2ZtaseJRKFVu2bGTt2tWiJHIwmwlK0aI6cwpnWFhZOULkLwfU84IA\nPPJIH2JjZ6NUqvjHP9azfv3aOy8k1B6LBU1qMqqTJ3CGhlKsT0cKDZU7Vbl6XxCAvn37MWNGLEql\nkvXr17Jhg2vuXCv8itWKxqhHffwYzpCmmAzpSGFhcqe6hSjIzx57rD9Tp85EqVSybt0aNm36u9yR\nvFtJCRqjAfXRLKQmwZgMRpzNmsud6jaiIL8wYMBApkyZhkKhYPXqD/j4481yR/JOpaVo3k5FnXUY\nqXETilPScbZoKXeqComC/MoTTwxi8uQ3AVi1ajnbtn0scyIvY7OhmWtEfeg7pIaNKNan4WwZLneq\nSnnk1d1vioiIQKPRoFKpUKvV/OMf/3DLvIMGReJwOFi8eCErVixBpVIxfPhIt8zt1Ww2Aueloz54\nAKlBQ4oNRpyt28idqkoeXRCA1atXl98H3J0iI4djtztYtmwxS5cuQqVSEhk53O05vIbdTuD8ufjs\n/wYpKIhiXSrONvfKneqOPH4XS86/S4wYMYqYmLIryy9evJDMzJ2yZanTHA4wGvHZ9zWSRoNJl4qz\n7Z3vcOsJPLogCoWCiRMnMnbsWDZs2CBLhqio3zNhQgwA77//Hrt3Z8qSo85yOgl8bz7s3o0UGIhJ\nm4Kj3f1yp6o2j97FWrduHffccw9Xr15l4sSJtG3bll69erk9x+jRT2K32/noow9ZuDADlUrFgAED\n3Z6jznE6CViYgc+eL6BhEKY4HY4OD8idqkbqzP1BFi5cSGBgIDExMbJl+PDDD1m1ahUqlYrExEQG\nDhwoWxaP53TCu+/Cp5+WnR47Zw506yZ3qhrz2C2IxWLB4XAQFBSE2Wzmq6++YsqUKVUu4+obtAwf\nPobCQhMbNqxDq9UTG1tC3779bnuep9y4RrYckkTAX97Hd+cO8PWlODaR4G7dPGad1ITHFuTKlSvl\nhXA4HERFRfG73/1O5lQwfvyz2O12Nm36O/PmzWHWrHgeeaSP3LE8hyQRsGxxWTl8fDAlJOPoWve2\nHDfVmV2s6nDXbyhJkvjggxV8/PFm1Go1s2cn0rNn7/Lv19stiCThv3IZftu2glqNKV6L/aGe8mSp\nhLgFmxsoFApefPElRo2Kxm63M2dOGocOfSd3LHlJEv5/XVlWDpUa0+zE8nLUZaIgv5FCoSAm5lWG\nDx+JzWbDaDSQlXVY7ljykCT8P/oQv62bQKnCNCse+y+2qHWZKMhdUCgUvPzyn4iMHI7NZiMtTc/R\no1lyx3I7v/Vr8du4AZQqzLGzsXvRezJRkLukVCp57bXXGTQokpKSEtLSdBw9elTuWG7jt2Ed/uvX\nglKJeUYstgo+1avLREFqgVKp5PXX32DgwAisViuzZs3i5MkTcsdyOb9Nf8d/3Zqyckydie2x/nJH\nqnWiILVEqVTyxhvT6d//cSwWCwZDEqdPn5I7lsv4frwZ/9UfgEKBeco0bF56ZIEoSC1SKpVMnTqT\ngQMHYjab0esTOXv2tNyxap3vto8JWLUcAMvkN7E9MUjmRK4jClLLVCoVCQkJ9OnzKCaTCb0+ifPn\nz8kdq9b47thOwIolAFgmTaF0UKTMiVxLFMQF1Go1M2e+Re/efSgqKiI5OYELF36QO9Zd8925g4Cl\niwCwvDKJ0npwfowoiIv4+PgQGxtHz569KSq6QXJyAtnZP8od6zfzzdxJwOKFAFhiXqV0xCiZE7mH\nKIgL+fj4MGtWPD16PMz164VotfH89FOO3LFqzGd3JgHvvweAdUIMpVG/lzmR+4iCuJivry+zZyfS\nrduDFBZeQ6uNJzf3otyxqs1nzxcELswAScL67AuUjH5S7khuJQriBn5+fsTHa+nSpStXrxag1caT\nl3dJ7lh35LP3SwL/PB+cTqzjn6XkqT/KHcntREHcxN/fn4QEHZ06debKlctotXHk5+fLHatSPvu+\nJnD+O+B0YP3D05T88Rm5I8lCFMSNAgICSEjQ0aHDA+Tn56PVxnH58mW5Y91Gvf8bAufNAaeDkrHj\nKBn/rNyRZCMK4mYajQatNoX27TuQl3cJrTaOgoICuWOVUx88gOYdIzjslESPwfrcBFAo5I4lG1EQ\nGWg0GpKTDbRt255Ll3LRauO4du2q3LFQH/oOzZw0sNspGRWN9cWX6nU5QBRENkFBDdDpUrjvvrZc\nvPgTWm08hYXXZMujzjqMxmgAm43S4SOxxrxa78sBoiCyatCgIcnJKbRu3YacnGx0ukRu3Lju9hyq\no1lo0vRl5RgyDMvLfxLl+JkoiMwaNWqMXp9GeHgrLlz4AZ0ukaKiG26bX3X8GJo0HZSUUBoxBMuf\nJoNS/Le4SawJD9C4cRP0+jRatGjJ+fPn0OmSKC52/QUOVCdPoDEkobBaKR0YgWXym6IcvyLWhocI\nDg7BYEinWbPmnDt3Br1ei8lkctl8qtOnysth6/84ljemi3JUQKwRDxISUlaSsLAwzpw5RUqKFrPZ\nXOvzqM6eRqNPRGE2Y+v3O8xTZ4pyVEKsFQ8TGhqKXp9OaGgoJ0+eIDU1GYvFUmvjK8+fQ6NPQmEy\nYevzKObpsaBS1dr43kYUxAOFhYVhMKQTEtKU48ePYTTqsVqtdz2u8sIPBCUnoCgqwta7D+aZb4Ha\nYy+u6RFEQTxUs2bNMRiMBAeHcPRoFkajgZKSkt88njL7x5/LcQN7z96YY+PAx6cWE3snURAP1qJF\nSwwGI40bNyEr6zBvv51KaWlpjcdR/pRDkDYexfVC7D0exjQrXpSjmjy6IHv27GHYsGFERkaydOlS\nuePIomXLcPT6NBo2bMShQ98xd64Rm81W7eWVuRfLylF4DXu3BzHNTgRfXxcm9i41KsjNjx1tNhsO\nh8MlgW5yOBykpKSwfPlytm/fzvbt2zl79qxL5/RUrVu3wWAw0qBBQw4ePMC8eenVK0luLhptPIqr\nBdi7dMUUrwU/P9cH9iLVLsiyZct4//33SU9P//lCBMmuzMWRI0do3bo14eHh+Pj4MHLkSDIz6+/t\nz9q0uRedLpWgoCD27/+G+fPnYrfbK32+Ij8fZsxAeeUy9k6dMSXowN/ffYG9RLUL8uCDDzJ16lRm\nzZrF3r17cTqdrsxFXl4ezZs3L/86LCyMvLw8l87p6dq2bYdOl4pGo2Hfvq/JyJhX4ZZcceUKQdo4\nuHQJR4cHysoREOD+wF7gjp/xZWdnExoaSkBAAJs2beLpp58mKiqqyt9etUHxGw6Wq+m9H1zJVVlC\nQx8mI2M+sbGx7N//NStWvE9cXBzKm3/ou3IF0pLg6mXo2JGAefMI0GhckqWmPOn1qa47FmTlypUM\nGzaMPn36YLVa+fbbb+nVqxdjxoxxabCwsDByc3PLv7506RJhYWFVLuMJN2gB198spmnTcN56S4te\nn8iOHf/EarUzZco0VNcLCUqcjfLiTzjatidg7lwum51gln+9eO0NdLp3705OTg7Z2dn07t2ba9fc\nc85C165duXDhAjk5OZSWlvLpp58yaJD3XuKypjp27ERSkgF/f392785k5btz0STFlZXjvraYdCnQ\noO79xvY0dyxIbm4uvr6+fPDBBzz//PNuu7S/Wq0mKSmJl156iZEjRzJixAjatasbN593l86du5CQ\noKOJUkm3v64gb/8+HK3aYEpOQWrQUO54XuGOu1itWrVi6NChREVFcfXqVT7//HN35ALg8ccf5/HH\nH3fbfHVRtzZt+EtQA/JLSzguSWxs15bnGzZCnO5UO+64BRkxYgSnT5ddoTwnJ4crV664PJRQTcXF\naPRaQm9cJ6xnbxbf34HNX+xm5cpleNG9WWV1x4KoVCq6dOkClL0fmTx5sstDCdVgMhFkSEJ19jTO\nZs3xXbiU15P0qFRqtm3byl//ulKUpBZ49KEmQiXMZoJStKhOn8IZFkaxIR0pJISePXsza1Y8SqWK\nrVs3sWLFClGSuyQKUtdYLGhSk1GdPIEzNJRifTpSaGj5tx95pA+xsbNRKlV89NFHrF+/VsawdZ8o\nSF1itaIx6lEfP4YzpCkmQzpSBX8b6tu3HzNmxKJUKlm/fi0bNqyTIax3EAWpK0pK0BgNqI9mITUJ\nxmQw4mzWvNKnP/ZYf+Lj41Eqlaxbt4ZNm/7uxrDeQxSkLigtRfN2Kuqsw0iNm1Ccko6zRcs7LjZo\n0CCmTJmGQqFg9eoP+PjjzW4I611EQTydzYZmrhH1oe+QGjaiWJ+Gs2V4tRd/4olBTJ78JgCrVi1n\n27aPXZXUK4mCeDKbjcB56agPHkBq0JBigxFn6zY1HmbQoEgmTZoCwIoVS9ixY3ttJ/VaoiCeym4n\ncP5cfPZ/gxQURLEuFWebe3/zcJGRw3nllUkALF26iJ07d9RSUO8mCuKJHA4CM+bhs+9rJI0Gky4V\nZ9u7Pw5txIhRxMS8CsDixQvJzNx512N6O1EQT+N0EvjefHz2fokUGIhJm4Kj3f21NnxU1O+ZMCEG\ngPfff4/du+vvWZrVIQriSZxOAhZm4LPnCyR/f0xJBhwdHqj1aUaPfpLnnpuAJEksXJjBnj1f1Poc\n3kIUxFM4nQQsWoDv7sz/lqNjJ5dN9+STf2D8+GdxOp38+c/z2bv3S5fNVZeJgngCSSJgySJ8M3eC\nry+mBB2Ozl1cPu0f//gMf/jD0zidDubPf4d9+752+Zx1jSiI3CSJgGWL8d25A3x8MCUk4+jazW3T\njx//LGPHjsPpdDBv3hz27//GbXPXBaIgcpIk/Fcuw3fHdlCrMcUlYe/ew60RFAoFzz03gejoMTgc\ndt55x8jBgwfcmsGTiYLIRZLw/3AVftu2gkqNaXYi9od6yhJFoVDw4osvMWpUNHa7nTlz0jh06DtZ\nsngaURA5SBL+a1fjt2UjKFWYZsVj79lb1kgKhYKYmFcZPnwkNpsNo9FAVtZhWTN5AlEQGfitX4vf\nP9aDUoU5djb2R/rIHQkoK8nLL/+JyMjh2Gw20tL0HD2aJXcsWYmCuJnfhnX4r18LSiXmGbHY+vaT\nO9ItlEolr732OoMGRVJSUkJamo5jx76XO5ZsREHcyG/T3/Fft6asHFNnYnusv9yRKqRUKnn99TcY\nODACq9VKSoqWkydPyB1LFqIgbuL78Wb8V38ACgXmKdOwDRgod6QqKZVK3nhjOv37P47VasVgSOL0\n6VNyx3I7URA38N32MQGrlgNgmfwmtifqxhUilUolU6fOpF+/32E2m9HrEzl79rTcsdxKFMTFfHds\nJ2DFEgAsk6ZQOihS5kQ1o1KpmD49lj59HsVkMqHXJ3H+/Dm5Y7mNKIgL+e7cQcDSRQBYXplEaeRw\nmRP9Nmq1mpkz36J37z4/3xsmgQsXfpA7llt4ZEEWLFjAgAEDGD16NKNHj2bPnj1yR6ox38ydBCxe\nCIAl5lVKR4ySOdHd8fHxITY2jp49e1NUdIPk5ASys3+UO5bLeWRBFAoFEydOZMuWLWzZsoUBAwbI\nHalmdu4k4P33ALBOiKE06vcyB6odPj4+zJoVT48eD3P9eiFabTw//ZQjdyyX8siCAHX2ioA+e76A\nOXNAkrA++wIlo5+UO1Kt8vX1ZfbsRLp1e5DCwmtotfHk5l6UO5bLeGxB1qxZQ3R0NPHx8dy4cUPu\nONXis/dLAv88H5xOrOOfpeSpP8odySX8/PyIj9fSpUtXrl4tQKuNJy/vktyxXEIhyfSreuLEiRVe\nKX7atGn06NGD4OBgADIyMrh8+TJGo9HdEWvmq69ApwOHA154ASZOlDuRy1ksFt566y2ysrJo1qwZ\nGRkZd7wLWF0jW0GqKycnh0mTJvHJJ5/c8bly3eJLfeAbNHOM4LBTMnYcDaZP4fKVYlmy/JI7bnt2\n8+8jp06dJCwsjJSUOYT+4lrB7sxSHbV+CzY55Ofnl/97165ddOjQQcY0VVMfPIBm7s/liB6D9bkJ\n8BtuQFpXBQYGotWm0L59B/Ly8tBq4ygoKJA7Vq3xyILMmzePqKgooqOj2b9/P3FxcXJHqpD60Hdo\n5qSB3U7JqGisL75Ur8pxk0ajITnZQNu27bl0KRetNo5r167KHatWePwuVk24cxOuzjqMJiUZbDZK\nh4/E8sqk8nJ40u6EO3Pc/PvI+fPnCA9vRUpKOo0bN5ElS2W8YhfL06mOZqFJ05eVI3I4lpf/VC+3\nHL/WoEFDkpNTaN26DTk52eh0idy4cV3uWHdFFKSGVMePoUnTQUkJpYMisbz2OijFarypUaPG6PVp\nhIe34sKFH9DpEikqqhsf01dEvLI1oDp5Ak2KFoXVSunACCyvvyHKUYHGjZug16fRokVLzp8/h06X\nRFGR/LtXv4V4datJdfoUGkMSCosFW//HsbwxXZSjCsHBIRgM6TRr1pxz584wa9YsTCaT3LFqTLzC\n1aA6exqNPhGF2Yyt3+8wT50pylENISFlJQkLC+PEiROkpGgxm81yx6oR8SrfgfL8OTT6JBQmE7Y+\nj2KeHgsqldyx6ozQ0FD0+rKSnDx5gtTUZCwWi9yxqk0UpArKCz8QlJyAoqgIW+8+mGe+BWq13LHq\nnLCwMObPn09ISFOOHz+G0ajHarXKHataREEqocz+8edy3MDeszfm2Djw8ZE7Vp3VokULUlLSCQ4O\n4ejRLIxGAyUlJXLHuiNRkAoof8ohSBuP4noh9h4PY5oVL8pRC5o3b4HBYKRx4yZkZR3m7bdTKS0t\nlTtWlURBfkWZe7GsHIXXsHd7ENPsRPD1lTuW12jZMhy9Po2GDRtx6NB3zJ1rxGazyR2rUqIgv6DM\nu4RGG4/iagH2Ll0xxWvBz0/uWF6ndes2GAxGGjRoyMGDB5g3L91jSyIK8jNFfj4abRzKK5exd+qM\nKUEH/v5yx/Jabdrci06XSlBQEPv3f8P8+XOx2+1yx7qNKAiguHKFIG0cyvx8HB0eKCtHQIDcsbxe\n27bt0OlS0Wg07Nv3NRkZ83A4HHLHukW9L4iioICgpNko8y7haN+BYm0KaDRyx6o32rW7n+TkVAID\nA9m790vee28+TqdT7ljl6nVBFNeuEpQcj/JSLo627TElG0Q5ZHD//R1ISjLg7+/Pnj1fsHBhhseU\npN4WRFEZHt/pAAAHlUlEQVR4jaDkBJQ/5eC4ry0mXQpSUM3OFRBqT8eOncpLsnt3JosWLfCIktTL\ngihuXEejS0SZ/SOO1m0wJacgNWgod6x6r3PnLiQk6PD19SUzcydLliyS/fJP9a4giqIbaHSJqC78\ngLNlOCZ9GlKjxnLHEn7WtWs3EhKS8fHxYefOHSxbtljWktSvghQXo9FrUZ0/h7NFS4oNRqSfTwkV\nPEf37j2Ij9fi4+PDjh3bWblyqWwlqT8FMZkIMiShOnsaZ7PmFBvSkYJD5E4lVKJHj4d5660E1Go1\n27Z9zF//ulKWktSPgpjNBKVoUZ0+hTMsrKwcIaIcnq5nz97ExsajVKrYunUTH330odtL4v0FsVjQ\npCajOnkCZ2goxfp0pAoubCZ4pkce6UNs7GyUShUbN25g/fq1bp3fuwtitaIx6lEfP4YzpCkmQzqS\nl10asz7o27cfM2bEolQqWb9+LRs2rHPb3N5bkJISNEYD6qNZSE2CMRmMOJs1lzuV8Bs99lh/pk6d\niVKpZN26NWza9He3zOudBSktRfN2Kuqsw0iNm1Ccko6zRUu5Uwl3acCAgUyZMg2FQsHq1R+wdesm\nl8/pfQWx2dDMNaI+9B1Sw0YU69NwtgyXO5VQS554YhCTJ78JwAcfrOCTT7a6dD5ZC7Jjxw5GjhxJ\np06d+P77W29Wv2TJEiIjIxk2bBhfffVV9Qa02wmcl4764AGkBg0pNhhxtm7jguSCnAYNimTSpCkA\nrFy5lB07trtsLlkL0qFDBxYuXEivXr1uefzMmTN8+umnbN++neXLl6PX6+98XI7dTuC7c/DZ/w1S\nUBDFulScbe51XXhBVpGRw3nllUkALF26iJ07d7hkHlkL0q5dO+67777bHs/MzGTkyJH4+PgQHh5O\n69atOXLkSNWDGY347PsaSaPBpEvF2badi1ILnmLEiFHExLwKwOLFC8nM3Fnrc3jke5D8/HyaNWtW\n/nWzZs3Iy8ureqHdu5ECAzFpU3C0u9/FCQVPERX1e1588SUA3n//PXbvzqzV8V1+kafKbrU2ffp0\nIiIiqj2O4k5XT9+9Gz/AU84gr+ll9l3FU3KA67K8/PIEXn55gkvGdnlBVq1aVeNlwsLCuHTpvzeF\nvHTpktfd+06oGzxmF+uXx9hERESwfft2SktLyc7O5sKFC3Tv3l3GdEJ9Jesdpj7//HNSU1O5du0a\nDRo0oFOnTixfvhyAv/zlL2zcuBGVSkVCQgL9+/eXK6ZQj3nVLdgEobZ5zC6WIHgiURBBqIIoiCBU\noU4XpNaP5aolCxYsYMCAAYwePZrRo0ezZ88et84PsGfPHoYNG0ZkZCRLly51+/w3RUREEBUVxejR\no3nqqafcOndcXBz9+vUjKiqq/LHCwkImTpzI0KFDiYmJ4caNO9xgVKrDzpw5I507d0567rnnpKNH\nj5Y/fvr0aSk6OloqLS2VsrOzpcGDB0sOh8NtuRYsWCCtXLnSbfP9mt1ulwYPHixlZ2dLpaWlUnR0\ntHTmzBlZsjzxxBPStWvXZJn7wIED0vfffy+NGjWq/LE5c+ZIS5culSRJkpYsWSK98847VY5Rp7cg\ntXosVy2TZPxw8MiRI7Ru3Zrw8HB8fHwYOXIkmZm1ewhGTci1Lnr16kXDhrde7+z//u//GDNmDABj\nxoxh165dVY5RpwtSmd90LFctW7NmDdHR0cTHx995M17L8vLyaN78v2dPhoWFuf3nv0mhUDBx4kTG\njh3Lhg0bZMnwSwUFBTRt2hSApk2bUlBQUOXzPf6Ge247lquGKss1bdo0nn76aSZPngxARkYGb7/9\nNkajsVbnr0pt/6x3Y926ddxzzz1cvXqViRMn0rZt29tOb5CLQqG447ry+IJ46rFc1c01btw4Jk2a\nVKtz30lYWBi5ubnlX8t5LNs999wDQHBwMEOGDOHIkSOyFiQkJITLly8TGhpKfn4+wcHBVT7fa3ax\nJA86lis/P7/837t27aJDhw5umxuga9euXLhwgZycHEpLS/n0008ZNGiQWzMAWCwWiouLATCbzXz1\n1VduXxe/FhERwebNmwHYsmULgwcPrvL5dfpQE089lmvWrFkcP34chUJBeHg4BoOhfL/XXf71r39h\nNBpxOp089dRTvPbaa26dHyA7O5spU8pOjXU4HERFRbk1x4wZM9i/fz+FhYWEhITw5ptvMmjQIKZN\nm0Zubi4tW7YkIyPjtjfyv1SnCyIIruY1u1iC4AqiIIJQBVEQQaiCKIggVEEURBCqIAoiCFUQBRGE\nKoiCCEIVPP5YLKH6HA4Hn376KdnZ2TRv3pwjR44QExNDq1at5I5WZ4ktiBc5ceIEQ4cOpVWrVjid\nToYNG0aouN3cXREF8SJdunTB19eXQ4cO8cgjj9CnTx/8/f3ZtWuXbOeD1HWiIF7kyJEjXL16lVOn\nTtGqVSu+/fZbrly5wpYtW+SOVmeJ9yBe5Msvv6Rp06Y8/PDDfP755zRp0oSmTZvSsWNHuaPVWaIg\nXuTmWYxC7RG7WF6uoKCA8+fPs2/fPrmj1EnifBBBqILYgghCFURBBKEKoiCCUAVREEGogiiIIFRB\nFEQQqiAKIghVEAURhCr8PyAgP0ZsQAgTAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1095c9e10>"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We know that $\\omega$ (red) is orthogonal to the decision boundary:\n",
      "\n",
      "* We define the geometric margin, $\\gamma_i$, as the distance between a point and the boundary.\n",
      "* We also introduce the norm of $\\omega$: $\\sqrt(\\sum \\omega_i^2)$.\n",
      "* A unit vector in the direction of $\\omega$: $\\frac{\\omega}{\\|\\omega\\|}$\n",
      "\n",
      "The geometric margin ($\\gamma_i$)for any point, $x^i$, is derived knowing that $x^i - \\gamma_i \\frac{\\omega}{\\|\\omega\\|}$ is on the decision boundary.\n",
      "\n",
      "Any point on the decision boundary evaluates to zero: \n",
      "\n",
      "$$ w^T (x^i - \\gamma_i \\frac{\\omega}{\\|\\omega\\|}) + b $$\n",
      "\n",
      "Moreover, you can solve for $\\gamma_i$:\n",
      "\n",
      "$$ \\gamma_i = y^i ((\\frac{\\omega}{\\|\\omega\\|})^T x^i + \\frac{b}{\\|\\omega\\|}) $$\n",
      "\n",
      "Recall that the functional margin is:\n",
      "\n",
      "$\\hat{\\gamma}_i = y^i (\\omega^T x^i + b)$\n",
      "\n",
      "Thus the two margins are equal for model parameters with $\\|w\\|=1$.\n",
      "\n",
      "$\\gamma_i = \\frac{\\hat{\\gamma_i}}{\\|w\\|}$\n",
      "\n",
      "And, finally, the geometric margin for a set of training data is the minimum computed for the training set:\n",
      "\n",
      "$\\gamma = min (\\gamma_i) $"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "The geometric margin for the SVM classifier indicates how close training data are to the decision boundary -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Intuitively, we want to find parameters ($\\omega,b$) that maximize the geometric margin for a training set:\n",
      "\n",
      "$max (\\frac{\\hat{\\gamma}}{\\|\\omega\\|})$\n",
      "\n",
      "The challenge is that we have a non-convex objective.\n",
      "\n",
      "We do several things to simply this:\n",
      "\n",
      "* Enforce that the funtional margin, $\\hat{\\gamma} = 1 $.\n",
      "* Thus, the geometric margin is $\\gamma = \\frac{1}{\\|w\\|}$.\n",
      "* Observe that maximizing $\\frac{1}{\\|w\\|}$ is the same as minimizing $\\frac{1}{2}(\\|w\\|)^2$\n",
      "\n",
      "We now want to find $w$ and $b$ such that:\n",
      "\n",
      "$max (\\frac{1}{\\|w\\|})$ or $min_{\\gamma,w,b} (\\|w\\|^{2})$, which is a quadratic constraint ($w^Tw$).\n",
      "\n",
      "Solving this problem (e.g., using QP solvers) gives us the optimal margin classifier!"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "The Lagrangian is a useful tool for solving optimization problems - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Given a convex function that we want to minimize with inequality and equality constraints:\n",
      "\n",
      "* $min_x$ $f(x)$ \n",
      "* $g_i(w) \\leq 0$ \n",
      "* $h_i(w) = 0$\n",
      "\n",
      "We can solve it using the Lagrangian, which is passed x (the primal variable) as well as $\\alpha$ and $\\beta$ where $L$ is convex in x:\n",
      "\n",
      "$$ L(x,\\alpha,\\beta) = f(x) + \\sum\\limits_{i=1}^m \\alpha_i g_i(x) + \\sum\\limits_{i=1}^m \\beta_i h_i(x) $$\n",
      "\n",
      "Importaintly:\n",
      "\n",
      "* The first term captures the inequality constrains: $g_i(w) \\leq 0$\n",
      "* The second term captures the equality constraints: $h_i(w) = 0$"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "The primal problem -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Recall our objective is $min_x$ $f(x)$.\n",
      "\n",
      "In the primal problem, we first maximize the Lagrangian function over all possible values of $\\alpha$ and $\\beta$.\n",
      "\n",
      "$$ \\theta_p(x) = max_{\\alpha, \\beta, \\alpha \\geq 0} L(x,\\alpha,\\beta) = f(x) + max_{\\alpha, \\beta, \\alpha \\geq 0} [ ... ] $$\n",
      "\n",
      "This is subject to the constraint that $\\alpha_i \\geq 0$.\n",
      "\n",
      "The problem is primal feasible if the initial constraints are satisfied: \n",
      "\n",
      "* $h_i=0$ \n",
      "* $g_i \\leq 0$.\n",
      "\n",
      "This is because:\n",
      "\n",
      "* The second summation drops due to $h_i=0$.\n",
      "* The first summation drops: $\\alpha_i = 0$) will maximize the funtion by knocking out the negative terms ($g_i \\leq 0$).\n",
      "\n",
      "Therefore, for values of $x$ that are primal feasible, the objective is:\n",
      "\n",
      "* $ min_x \\theta_p(x) = f(x) $\n",
      "* $P^* = min_x \\theta_p(x)$ is the optimal value of the objective.\n",
      "\n",
      "If constraints are violated (e.g., $g_i \\geq 0$):\n",
      "\n",
      "* Then the summation blows up and $ \\theta_p(x) = \\infty $."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "The dual problem -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Again, recall our objective is $min_x$ $f(x)$.\n",
      "\n",
      "Another way of solving this is the dual problem, which reverses the order of operations.\n",
      "\n",
      "Previously we:\n",
      "\n",
      "* We maximized the Lagrangian function over all possible values of $\\alpha$ and $\\beta$.\n",
      "* For all values of $x$ that were primal feasible, we then got a simple objective that we then minimized over $x$.\n",
      "\n",
      "For the dual problem, we:\n",
      "\n",
      "* We minimize the Lagrangian function over all possible values of $x$.\n",
      "* Then, we get an objective that we maximize over all possible values of $\\alpha$ and $\\beta$!\n",
      "\n",
      "Thus, the dual objective, which is dual feasible if $\\alpha_i \\geq 0$.\n",
      "\n",
      "$$ \\theta(\\alpha,\\beta) = min_{x} L(x,\\alpha,\\beta) $$\n",
      "\n",
      "$$ D^* = max_{\\alpha,\\beta} \\theta(\\alpha,\\beta)$$\n"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Connecting the dual and primal problems - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Key points connecting the primal and dual problems:\n",
      "\n",
      "* Weak duality: The optimal value for the dual problem, d^*,  is $\\leq$ than that of the primal problem p^*. \n",
      "* And strong duality: $d^* = p^*$\n",
      "\n",
      "Therefore:\n",
      "\n",
      "* The role of the Lagrangian is to connect the primal and dual problems.\n",
      "* There are a set of parameters that solve both problems."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Karush-Kuhn-Tucker (KKT) conditions are parameters that simultanously solve the primal and dual problem - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "These set of parameters that solve both dual and primal framing of the problem:\n",
      "\n",
      "* There is a value $\\omega^*$ that is a solution to the primal problem \n",
      "* There are values $\\alpha^*, \\beta^*$ that are solutions to the dual problem.\n",
      "\n",
      "These parameters therefore satisfy the KKT conditions, the most importaint of which is the dual complementarity condition:\n",
      "\n",
      "* $\\alpha^* g_i(\\omega^*) = 0$\n",
      "* $\\alpha^* \\geq 0$\n",
      "\n",
      "This implies that $\\alpha^* > 0$ only when $g_i(\\omega^*) = 0$.\n",
      "\n",
      "In the context of SVMs, recall that $g_i$ will be:\n",
      "\n",
      "* $g_i(w) = -y^i (w^T x^i + b) + 1 $\n",
      "* $g_i = 0$ for points with a funtional margin of 1.\n",
      "* Points with a funtional margin of 1 are definitionally closest to the decision boundary."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Set up and solve the Dual problem to find optimal parameter values $w,\\alpha$ that maximize the geometric margin -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Previously, we defined a generalized Lagrangian:\n",
      "    \n",
      "$$ L(x,\\alpha,\\beta) = f(x) + \\sum\\limits_{i=1}^m \\alpha_i g_i(x) + \\sum\\limits_{i=1}^m \\beta_i h_i(x) $$\n",
      "\n",
      "We can use this to maximize the geometric margin for the optimal margin classifier:\n",
      "\n",
      "* $f(w) = \\|w\\|^{2}$\n",
      "* $g_i(w) = -y^i (w^T x^i + b) + 1 $\n",
      "\n",
      "Perform the substitution, noting that there are no equality ($h_i$) constraints in our problem:\n",
      "\n",
      "$$ L(w,b,\\alpha) = \\|w\\|^{2} - \\sum\\limits_{i = 1}^{k} \\alpha_i [y^i (w^T x^i + b) -1] $$\n",
      "\n",
      "We proceed to find the dual form of this problem, which has the classic form:\n",
      "\n",
      "$ \\theta(\\alpha,\\beta) = min_{x} L(x,\\alpha,\\beta) $\n",
      "\n",
      "We re-write with $x=\\omega+b$ and also note that $\\beta$ term is not present:\n",
      "\n",
      "$ \\theta(\\alpha,\\beta) = min_{\\omega,b} L(\\omega,b,\\alpha) $\n",
      "\n",
      "We can find the minimum by setting the derivative of $\\theta(\\alpha,\\beta)$ to zero:\n",
      "\n",
      "$$ \\nabla L(\\omega,b,\\alpha) = \\omega  - \\sum_{i=1}^m \\alpha_i y^i x^i = 0 $$\n",
      "\n",
      "We do this and substitue back into the inital Lagrangian to get:\n",
      "\n",
      "$$ min_w [ L(w,b,\\alpha) ] = \\sum \\alpha_i - 0.5 \\sum_i \\sum_j y^i y^j \\alpha_i \\alpha_j - <x^i , x^j > - b \\sum \\alpha_i y^i $$\n",
      "\n",
      "Critically, our problem meets conditions required for $p^* = d^*$ and the KKT conditions.\n",
      "\n",
      "This implies that $\\alpha^* > 0$ only when $g_i(\\omega^*) = 0$, the points close to our boundary.\n",
      "\n",
      "Moreover, we have re-framed the optimization problem:\n",
      "\n",
      "$$ max_{\\alpha; \\alpha \\geq 0} W(\\alpha) = \\sum \\alpha_i - 0.5 \\sum_i \\sum_j y^i y^j \\alpha_i \\alpha_j - <x^i , x^j > - b \\sum \\alpha_i y^i $$\n",
      "\n",
      "By solving the dual problem, we found: \n",
      "\n",
      "* A set of optimal $\\alpha_i$ values.\n",
      "* From KKT, we know that the $\\alpha_i>0$ only for points closest to the decision boundary, the support vector.\n",
      "* The optimal value of our model parameters: $ \\omega  = \\sum_{i=1}^m \\alpha_i y^i x^i $\n",
      "\n",
      "Now, if we want to make a prediction for a new point, x: \n",
      "\n",
      "$$ \\omega^T x + b = (\\sum_{i=1}^m \\alpha_i y^i x^i)^T x + b = \\sum_{i=1}^m \\alpha_i y^i <x^i x> + b $$\n",
      "\n",
      "Note two critical points:\n",
      "\n",
      "* This can be Kernelized, because only inner products between new point and the support vectors appear. \n",
      "* The only values of $\\alpha > 0$ are those for the support vectors."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Apply the Dual problem to a different objective - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Recall the general form of our constraints for the primal problem:\n",
      "    \n",
      "* $min_x$ $f(x)$ \n",
      "* $g_i(w) \\leq 0$ \n",
      "* $h_i(w) = 0$\n",
      "    \n",
      "We were given the constraint that the functional margin is:\n",
      "\n",
      "* $y^i (w^T x^i + b) \\geq 1 $\n",
      "\n",
      "We re-wrote this constraint to fit the primary problem:\n",
      "\n",
      "* $g_i(w) = -y^i (w^T x^i + b) + 1 $    \n",
      "* $g_i(w) \\leq 0$.\n",
      "\n",
      "We can use this to maximize the geometric margin for a new problem, written in primal form:\n",
      "\n",
      "* $min_{w}$ $f(w) = \\frac{1}{2} w^T w $.\n",
      "* Subject to: $w^T x^i \\geq 1$.\n",
      "\n",
      "For this new problem, we should similarly re-write the constraint:\n",
      "\n",
      "* $g_i(w) = 1-(w^T x^i)$\n",
      "* $g_i(w) \\leq 0$\n",
      "\n",
      "And we write the Lagrangian:\n",
      "\n",
      "$$ L(w,b,\\alpha) = \\frac{1}{2} w^T w + \\sum\\limits_{i = 1}^{k} \\alpha_i [1-(w^T x^i)] $$\n",
      "\n",
      "In the dual form of the problem, we first minimize the Lagrangian with respect to $w,b$.\n",
      "\n",
      "Since there is no $b$ parameter, we simply take the gradient of the Lagrangian with respect to $w$:\n",
      "\n",
      "$$ \\nabla_w L(w,b,\\alpha) = w + \\sum\\limits_{i=1}^k - a_i x_i $$\n",
      "\n",
      "We use this to find our optimal value of $w$:\n",
      "\n",
      "$$ w = \\sum\\limits_{i=1}^k a_i x_i $$\n",
      "\n",
      "Recall that our initial framing of the dual problem:\n",
      "\n",
      "$$ D^* = max_{\\alpha,\\beta,\\alpha \\geq 0} min_{x} L(w,\\alpha,\\beta) $$\n",
      "\n",
      "As shown above, we have done:\n",
      "\n",
      "$min_{x} L(w,\\alpha,\\beta)$\n",
      "\n",
      "This has given us the optimal value of $w$.\n",
      "\n",
      "We plug it back into the Lagrangian:\n",
      "\n",
      "$$ D^* = max_{\\alpha \\geq 0} L(w,b,\\alpha) = \\frac{1}{2} ( \\sum\\limits_{i=1}^k a_i x_i)^T ( \\sum\\limits_{i=1}^k a_i x_i) + \\sum\\limits_{i = 1}^{k} \\alpha_i [1-(( \\sum\\limits_{i=1}^k a_i x_i)^T x^i)] $$\n",
      "\n",
      "Distribute terms and simplify:\n",
      "\n",
      "$$ D^* = max_{\\alpha \\geq 0} L(w,b,\\alpha) = \\sum\\limits_{i = 1}^{k} \\alpha_i - \\frac{1}{2} \\sum\\limits_{i=1}^k \\sum\\limits_{j=1}^k a_i a_j (x_i)^T x^j $$\n",
      "\n",
      "Note that this can be Kernalized, because only inner products of the data appear!\n",
      "\n",
      "To evaluate a point, $z$, we simply evaluate $w^Tz$ with $w^T = (\\sum\\limits_{i=1}^k a_i x_i)^T $."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Inner products -"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Define a distance between vectors (inner product). It is essential to have a positive inner product.\n",
      "\n",
      "Furthmore, you can think of the inner product as a measure of distance.\n",
      "\n",
      "* If two vectors are orthogonal, their inner product will be small.\n",
      "* Conversely, similar vectors will have a large inner product.\n"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "The inner product can be represented as a Kernel function - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We have previously been discussing linear models, which lead to a linear decision boundary:\n",
      "\n",
      "As with logistic regression, the decision boundary is produced when $\\omega^Tx + b = \\omega_1 x_1 + \\omega_2 x_2 + b = 0$.\n",
      "\n",
      "The linear decision boundary is:\n",
      "\n",
      "$x_2 = -\\frac{\\omega_1}{\\omega_2} x_1 - \\frac{b}{\\omega_2} $\n",
      "\n",
      "However, it may be useful to learn in a higher dimentional space (e.g., with respect to $x^2$, etc).\n",
      "\n",
      "For this, the idea of a feature mapping, $\\phi$, is a way to re-cast the attributes in a higher dimentional space.\n",
      "\n",
      "$$ \\phi(x) = \\begin{bmatrix} x_i x_j \\\\   \\\\   \\end{bmatrix} $$\n",
      "\n",
      "If we want to learn in this higher dimensional space using our new funtion (above), we need to replace $x$ with $\\phi(x)$.\n",
      "\n",
      "In turn, we need to take the dot product of features, $\\phi(x)$.\n",
      "\n",
      "$$ \\phi(x)^T \\phi(z) = \\sum\\limits_i \\sum\\limits_j x_i x_j z_i z_j = \\sum\\limits_i x_i z_i \\sum\\limits_j x_j z_j = (x^T z)^2 $$\n",
      "\n",
      "In turn, this is the Kernel given the mapping specified above. \n",
      "\n",
      "Work a more complex example to further highlight the principle:\n",
      "\n",
      "$$ \\phi(x) = \\begin{bmatrix} x_i x_j \\\\ ...  \\\\ x_3 x_3 \\\\ \\sqrt{2c}x_i \\\\ ... \\\\ \\sqrt{2c}x_3 \\\\ c \\end{bmatrix} $$\n",
      "\n",
      "$$ \\phi(x)^T \\phi(z) = \\sum\\limits_i x_i z_i \\sum\\limits_j x_j z_j +  \\sum\\limits_i \\sqrt{2c}x_i \\sqrt{2c}z_i + c^2 = (x^T z)^2 + \\sqrt{2c} (x^T z) + c^2 $$\n",
      "\n",
      "Note:\n",
      "$$ A^2 + 2AB + B^2 = (A+B)^2 $$\n",
      "\n",
      "Thus:\n",
      "\n",
      "$$ K(\\phi(x),\\phi(z)) = \\phi(x)^T \\phi(z) = (x^T z + C)^2 $$\n",
      "\n",
      "There are more interesting mapping functions, which may lead to a Kernel definition.\n",
      "\n",
      "Intuitively, the Kernel mapping shows that the inner product in a higher dimentional space is equal:\n",
      "\n",
      "* To the inner product computer in the lower dimentional space with $x,z$.\n",
      "* This reduces the number of computations considerably.\n",
      "* Moreover, this is a measure of the similarly of two attribute vectors mapped by $\\phi$ into a higher dimentional space.\n",
      "\n",
      "So, a Kernel funtion, $K$, is intuitively a measure of distance."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Kernels - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If an algorithm inclues inner products $<x, z>$ between attribute vectors, then we can replace this with K(x,z) where K is a kernel. This allows the algorithm to work efficiently in the high dimensional feature space corresponding to K at low computational cost.\n",
      "\n",
      "Example:\n",
      "\n",
      "* Seperate two classes that are represented in a 2D space as concentric circles.\n",
      "* Define $\\phi$ as a mapping that will transform x,y into a (higher) dimentional space in which the classes are seperated."
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Implementation - "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, in short:\n",
      "    \n",
      "* SVM constructs a hyper-plan in a (high) dimensional space.\n",
      "* This cuts the training data.\n",
      "* It can be used for classification, regression.\n",
      "* A good separation is achieved by the hyper-plane that has the largest distance to the nearest training data points of any class (functional margin)\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os\n",
      "from IPython.display import Image\n",
      "i = Image(filename=os.getcwd()+'/Images/svm.png') # from the ISL text.\n",
      "i"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAyAAAAJYCAIAAAAVFBUnAAAAA3NCSVQICAjb4U/gAAAACXBIWXMA\nAA9hAAAPYQGoP6dpAABorUlEQVR42u3dBXRU1/r38TszySSZOHEnRnB39+DuXpxS3B2KU9whWHF3\ndw92cYI7IQRCjHgyc9/d5v3nctvb3hYCzJ58P4vFCtM0s88+zznPLzOzz/nHvwAAAJCp/sEUAAAA\nELAAAAAIWAAAAAQsAAAAELAAAAAIWAAAAAQsAAAAELAAAAAIWAAAAAQsAAAAELAAAAAIWAAAAAQs\nAAAAAhYAAAAIWAAAAAQsAAAAAhYAAAAIWAAAAAQsAAAAAhYAAAAIWAAAAAQsAAAAAhYAAAAIWAAA\nAAQsAAAAAhYAAAABCwAAAAQsAAAAAhYAAAABCwAAAAQsAAAAAhYAAAABCwAAAAQsAAAAAhYAAAAB\nCwAAAAQsAAAAAhYAAAABCwAAgIAFAAAAAhYAAAABCwAAgIAFAAAAAhYAAAABCwAAgIAFAAAAAhYA\nAAABCwAAgIAFAAAAAhYAAAABCwAAgIAFAABAwAIAAAABCwAAgIAFAABAwAIAAAABCwAAgIClt969\ne7d27dqzZ8/+EwAA/B2ie4oeKjopAQu/JSrjHwAA4FOJTkrAwm+dO3cuvTiy7O8f5cqV45cw5pMp\nZUrBfH6C9BcpRCclYOG3RH2I4hB/Z9kZqFu3LmXAfDKlTCmYT3ooAYvi4NTAfDKlYEqZT3ooAYvi\n4NTAfIIpZUqZT3ooAYvi4NQA5pMpZUrBfNJDCVgUxydav349ZcB8MqVMKZhPeigBi+IAAIAeSsCi\nOAAAoIcSsCgOAABADyVgURwAANBDCVgUBwAA9FACFsUBAAAIWKA4AACghxKwKA4AAOihBCyKAwAA\neigBCxQHAAD0UAIWxQEAAD2UgEVxAABADyVgURwELAAA6KEELIoDAAB6KAGL4gAAgB5KwKI4Psuk\nSZOuXbvGDAMA6KEELIojc8TFxeXNm1elUg0ePDg+Pp55BgDQQwlYFEcmSE5OnjBhgomJibe396FD\nh5hqAAA9lIBFcWSO+/fvV6xYUTxFmzZt3r59y4QDAOihBCyKIxPodLoVK1bY2tra2dkdOXKEOQcA\n0EMJWBRH5ggPD+/QocPTp0+ZcwAAPZSARXEAAAB6KAGL4gAAgB5KwKI4AACghxKwKI7MNGbMmN27\nd7MvAAD0UAIWxZE5UlNTa9asKcbQtGnT169fs0cAAPRQAhbFkQl0Ot26descHBysra2XLFmi1WrZ\nLwAAeigBi+LIBBEREd99950YTNmyZUNCQtg1AAB6KAGL4sgcx44d8/PzU6vVa9euZe8AAOihBCyK\nI3MkJCSMHDnywYMH7B0AAD2UgEVxAABADyVggYAFAAA9lIBFcQAAQA8lYFEcmWnEiBFz585NS0tj\nxwEA6KEELIojE+h0uh9++EGMtnjx4jdu3GDfAQDooQQsiiNznD17Nnfu3EZGRsOGDUtISGAPAgDo\noQQsiiMTJCcnjx8/Xq1W+/r6Hj16lJ0IAKCHErAojsxx79698uXLi8FPmzaN/QgAoIcSsCiOzKHV\naoOCgu7cucN+BADQQwlYFAcAAPRQAhbFAQAA6KEErCxeHDqdjp0LAKCHErAojsw0bNiw/v37f/jw\ngV0MAKCHErAojswxY8YMMzMzLy+vffv2sZcBAPRQAhbFkTkeP35crVo1sXUtWrR48+YN+xoAQA8l\nYFEcmUCn061evdre3t7W1nb58uV8MAsAQA8lYFEcmePdu3ft2rUTm9m7d2/2OACAHkrAojgyzZEj\nR27evMkeBwDQQwlYFAcAAPRQAhbFAQAAPZSARXFk8YCVlpZGGQAA6KEELIojM40aNapBgwavXr2i\nGAAA9FACFsWRObZt2+bs7GxpablgwQKtVktJAADooQQsiiMTREZGdu3aVcxDqVKlbt++TVUAAOih\nBCx9LI7nz5+fO3dOrqk4ffp0zpw5jY2NR44cmZiYSG0AAAhYBCz9Ko6BAweKH9ujR4/o6GiJZiMp\nKWnMmDEiYzVp0oTaAAAQsAhY+lUcaWlpc+fOtbCwcHV13b59u1xzEhIScuPGDWoDAEDAImDpY3G8\nePGibt264uezTA8AQA8lYFEcmUan023ZsoVlegAAeigBi+LIZFFRUd26dUtfpnfr1i1J5yolJSU1\nNZWaAQAQsKBHxZG+TM/IyGjEiBEyLtObMGFCkSJFrl69StkAAAhY0KPiSEpKGjt2rFqt9vf3P3Hi\nhFxzdeHChXz58qlUqkGDBsXHx1M8AEAPJWBBj4ojJCSkbNmy4nk7duz4/v17iaYrJSVl8uTJpqam\n3t7ehw4don4AgB5KwIIeFYdWq12yZIm1tbWjo+P69et1Op1Ek/bw4cMqVaqIeWvTps3bt2+pIgCg\nhxKwoEfF8fr166ZNm4oB1KxZ8+nTpxLNm0iEq1atypYtW/ny5akiAKCHErCgd8Wxe/dud3d3jUYz\nY8YMuZbpvX37lnsXAgA9lIAFPS2O2NjY3r17KxSKwoULZ/E7TwMA6KEELIojM128eDF//vwqlWrg\nwIFxcXHsHQAAPZSARXFkgoxletmzZz948KCks5qamhobG0t1AQA9lIBFceiRjGV6rVq1Cg8Pl25W\nZ8yY4e7uvnv3bgoMAOihBCyKQ49kLNMTVq5cKdd1HJ4+fVqzZk0xsU2aNHn9+jVlBgD0UAIWxaFH\n3r5926ZNGzHCSpUqPXjwQKKJFYlww4YNjo6O1tbWixcv5i7XAEAPJWBRHPrl0KFD3t7eJiYmEydO\nTElJkWh6379/36lTJzHDZcuWDQkJod4AgB5KwKI49EhcXNygQYNUKlXevHmDg4PlmuQTJ07kyJFD\njFyuNzoBAAQsAlaWKI6rV68WKVJEoVD88MMPMTExEs1zYmKiXG9xAgAIWASsLFQcqamps2bNMjc3\nd3Nz27lzJ3sQAEAPJWBRHJnj2bNn6cv0GjVqFBoayn4EANBDCVgURybQ6XQbN250dHS0srJatGiR\npMv00tLSXr16RR0CAD2UgEVx6JH379937txZbEKZMmXu3Lkj3fiXLFliYWExZ84ckbSoRgCghxKw\nKA49kr5Mz9jYePTo0YmJiRKNPDo6+vvvv1coFMWLF79x4wYFCQD0UAIWxaFHRK4aNWqUyFgBAQGn\nTp2Sa/Dnzp3LkyePkZHR0KFDExISKEsAoIcSsCgOPXL79u3SpUuLLercuXNkZKREI09OTp4wYYKJ\niYmvr++RI0eoTACghxKwKA49otVqFy5caGVl5eTktGnTJrmu7Xn//v2KFSt6e3uLvEVxAgA9lIBF\nceiXV69eNWrUSGxanTp1nj9/LtHIRSJ8+fIllQkA9FACFsWhp3bs2OHm5mZubj579myW6QEA6KEE\nLIojc8TExPTs2VOhUBQrVuz69evscQAAPZSARXFkjvPnz+fJk0elUg0ZMiQ+Pl7GTdBqtbdv36Zi\nAYAeSsCiOPRIxjI9Hx+fw4cPSzf+jRs3KpXKvn37fvjwgboFAHooAYvi0CPpy/TEJrdt2/bdu3cS\njTw1NfWnn34yMzPz9PTcu3cvpQsA9FACFsWhR3Q63YoVK2xtbe3t7desWSPXdRyePHkSGBgodlnz\n5s3DwsIoYACghxKwKA49Eh4e3rJlS7Ht1apVe/z4sVwBcd26dQ4ODjY2NsuWLZMrIAIAPZSARXEY\nvv3793t5eZmZmU2dOjUlJUWikUdERHTo0MHd3Z2PZAEAPZSARXHonbi4uP79+yuVygIFCly6dEmu\nwct1OyAAoIcSsCiOrOXKlSuFChVimR4AgB5KwNLr4ti6devbt28lmorU1NTp06drNBqW6QEACFgE\nLH0sjri4OCcnp2zZsq1atUrSZXrNmjWTdJmemPCTJ09S2wBAwCJgGWBxhIeHt27dWvzwKlWqPHz4\nUK6AkrFMLygoSKvVyrVPjxw5Iqa9QYMGr169osIBgIBFwDLA4jhw4ED27NlNTU0nT54s3TK97777\nTkxO+fLl7969K1dA3Lp1q7Ozs6Wl5YIFC6QLiABADyVgURz/W1xc3MCBA5VKZf78+a9evSrX/Bw7\ndszPz0+tVo8bNy45OVmikUdFRXXr1k3s3FKlSt26dYtSBwACFgHLAItDPEXx4sWvXLki3RQlJCQM\nHz7cyMgod+7cZ8+elWvwp0+fzpkzpxj8iBEjEhMTKXgAIGARsAytOKS+5vjNmzdLlCgh5qp79+7R\n0dESjTwpKWns2LGenp5cNAsACFgELIpD76Slpc2fP9/S0tLFxWXr1q1y5UVevgIAeigBi+LQXy9f\nvqxXr56YNPH3ixcvmBAAoIcSsKCPxbF27dqQkBCJJk2n023bts3FxcXCwmLevHlpaWkUEgDQQwlY\n0KPiSE1NzZcvn1qtHjt2bFJSkkRTFx0d3b17dzF7JUqUuHnzpox7P/2CDmIXcCAAAAGLgGVoxZGY\nmDhy5EgjI6OcOXOePn1argk8c+ZMrly5xOCHDRuWkJAg1+CvXr2qUCgKFy7Me8QAQMAiYBlmcdy6\ndatkyZJiMF27do2KipJoDpOSksaNG6dWq/38/I4dOyZXAVy8eDF//vwqlWrgwIFxcXEcEQBAwCJg\nGVpxaLXa9GV6zs7OBw8elGsm7969W65cOTGZHTp0iIiIkGjkKSkpkydPNjU1zZ49u3TTDgD0UALW\n13bixAnF71y8eFHPi+Ply5cNGzY8f/68dBMuAmJQUJCNjY2Dg8O6devkuo7Dw4cPq1SpIoqhVatW\n79694/ABAAIWAevPAtaUKVMufuQ3bwNxmYZMFxYW1qxZMzGrgYGBT548kWjkIhGuWrUqICCAgAUA\nBCwC1v8IWNu2baM4vr49e/Z4eHhoNJrp06fLtUyPm0MDAAGLgPW/A9bWrVspjm8iNja2T58+YhcU\nKlRIxpswAgAIWASsPwxYTk5ORkZGVlZWgYGBv79XsVzFsWLFiiNHjsi1F9KX6SmVyv79+7NMDwAI\nWAQs6V27dq1fv367du0SuWrlypW5c+cWSevQoUOSFodOp6tZs6YYbfv27eX6kFBKSsqUKVNMTU29\nvLz2798vaTkFBQXFxMRwWAEAAQv/ITo62sPDo2DBgr8vjnLlytX9yPr16/VzE7Ra7bJly2xsbOzt\n7desWSPXMr1Hjx5VrVpVzHbLli3fvHkjV/E8ffrU3Nzczc1t586dHEoAsizRHz9ul+lX5yFg4V/d\nu3dXKBQf35FGxvQdFhbWvHlzMezq1as/fvxYopGLRLh69Wo7OztbW9vly5fLFRCfPXuW/gpi48aN\nQ0NDOZoAgFew8B8BKzk52QCKY+/evZ6enmZmZitXrpRr5G/fvm3btq2Y9ooVK96/f1+ugLhx40ZH\nR0crK6tFixax3hAAAYuAhX9FRka6ubkVLlzYYIrjw4cPffv2PXnypIyDP3z4sI+Pj4mJyYQJEz6O\nvPrv/fv3nTt3FmVTpkwZuV5BBAACFgErE7Rq1WrEiBHbtm07ceLE0qVLAwIC1Gr1b26Wx2UavqH4\n+PghQ4aoVKq8efNKd+V6kWtLliwZFhbGfgRAwCJgZS1TpkwpVKiQjY2NkZGRo6Nj48aNf381JgLW\nN3ft2rWiRYsqFIqePXvKtUxPrs+QAQABi4BFcWQtaWlps2fPTl+mt2PHDiYEAOihBCyKQx8FBQUt\nW7ZMrpdYnj9/Xrt2bbE7GjZs+OrVK4oTAOihBCyKQ7/07NlT0mV6mzZtcnJysrKyWrhwoaTL9GbM\nmCHCIgcXAHooAYviMMDiOHr0qK+vr4mJyfjx4+VaphcZGdmlSxexX0qXLn379m25pj0iIsLNzc3c\n3Hz27NlpaWkcYgDooQQsisPQJCQkDB06VKVS5cmTR7pleqdOnQoICDA2Nh41alRiYqJEI4+JienZ\ns6dCoShWrNj169c5ygDQQwlYFIcBEj1edHrR7ydMmCDXyEWuGj16tMhYImlJd7kvkWjz5s0r0u2Q\nIUPi4+M51gDQQwlYFIehSUtLmzNnzpEjR2Qc/J07d8qUKSN2U+fOnSMjIyUaeXJysgi1JiYmPj4+\nv79ECADQQwlYFAe+Ja1Wu2jRIisrKycnp40bN8q1OvL+/ft169ZlXSQAeigBi+KAPgoNDW3cuLHY\nX7Vr13727BkTAgD0UAIWxYHMsXPnzvRlerNmzWKZHgDQQwlYFIc+WrZs2ciRI6VbpvfDDz8oFIqi\nRYteu3aNMgYAeigBi+LQLz/99JNarfb39z9x4oRcIw8ODk5fpjd48GBJl+mNHz/+0qVLHIYA6KEE\nLIrDAIWEhJQtW1bMQ8eOHd+/fy/RyFNSUiZNmmRiYuLt7X3o0CG5pl2EwsKFCyuVyj59+sTGxnIw\nAqCHErAoDkOj1WqXLFlibW3t6Oi4fv16uZbpPXjwoHLlymI/tmnT5u3btxKNPDU1dfr06RqNxsPD\nY+/evRyPAOihBCyKwwC9fv26SZMm6VeckmvkIhGuXLkyW7ZsdnZ2P//8s1wB8cmTJ4GBgWLamzVr\nFhYWRh0CoIcSsCgOA7Rr1y7p3m5LFx4e3rp1a7FDq1Sp8vDhQ7kC4rp16xwcHGxsbA4cOEARAqCH\nErAoDugXEVCyZ89uamo6efLklJQUiUYeERHRtWvXp0+fshMB0EMJWBQH9E5cXNzAgQOVSmX+/Pkv\nXrzIhAAAPZSARXEg03Zu4cKFFQpF7969WaaXRRw6dKh6tWruri7lypbZtGkTEwLQQwlYFIe+W7Vq\nVatWrcLDwyUac2pq6owZM9KX6e3Zs4edaPAlKg7nAAfzprntCrhYiq8nTpzItAD0UAIWxaHXNm7c\nmO1XK1eulGuZ3tOnT2vUqCF2dNOmTV+/fi3j5I8cOXLr1q1yTftXlpSUZG+XrbyX1c4WAbta5hR/\nGuXKZqJWv3v3jskB6KEELIpDr719+7ZNmzZixipVqvTgwQOJRi6iyYYNGxwdHa2trZcuXarVaiUa\nfFpaWoMGDcS016tX78WLF9Thf3XhwgUxRT9V90pPV+LPyvq+4pEtW7YwOQA9lIBFcUjg0KFD3t7e\nJiYmEydOlGuZ3vv37zt27Cj2eLly5e7evStXQNy6dauLi4uFhcW8efO4y/XvXb16VezZHyt5ZASs\nhbW9xSO7d+9mcgB6KAGL4pBDXFzcoEGDVCpV9erVpRv88ePH/f391Wr12LFjk5KSJBp5VFRU9+7d\nRcWWKFHi5s2b1OHHtFqtn69PHifzjU1yiHS1tVlAaQ8rG2srSe9TCdBDCVgUR5Z+zeDIkSMyjjwx\nMXHEiBFGRka5cuU6c+aMXIMXAxbDFoNfuXIlRfix06dPm2vMLEyNi7ha2JqbGKlUW7duZVoAeigB\ni+LAV3Xr1q2SJUuKAujWrVtUVJREI09KSvrxxx/l+hjc1/H06dOhQ4fWr1+/T58+d+7cYUIAeigB\ni+LAN5CWljZ//nxLS0tnZ+ctW7awTA8A6KEELIpDT4mYItcyvZcvX9avX59legBADyVgURz6a8OG\nDWXKlJHuDZrt27enL9ObO3cuy/QAgB5KwKI49Mvp06dz5MhhbGw8evRouZbpRUdH9+jRQ5RE8eLF\nb9y4IePkDx069KeffkpNTaUOAdBDCVgUh6FJTEwcOXKkkZFRQEDAqVOn5Br82bNnc+fOLQY/bNiw\nhIQEiUau0+n69++vUCgKFSp05coV6hAAPZSARXEYoFu3bpUqVUrMcJcuXSIjIyUaefoyPbVa7evr\ne/ToUbmm/eLFi/nz51cqlSJsxcXFUYcA6KEELIrD0Gi12oULF1paWubLl0+6NXr37t0rX768qJD2\n7dtHRERINPKUlJQpU6aYmpp6eXnt37+fOgRADyVgURwG6NWrVydPnpQ0IAYFBdnY2Njb269du1au\njPjo0aOqVauKCp84cSJFCIAeSsCiOKBfwsLCmjdvLkqlevXqjx8/lmjkIhGuXr2aS24CoIcSsCgO\n6Km9e/d6enqamZlNmzaNZXoAQMACxaG/5LqOw4cPH/r27atUKgsWLHj58mV2HwB6KAELFIfe2b59\nu4+Pz+HDh+Ua9qVLlwoUKCBiVr9+/UTkknTyuS8QAHooAYviMEz379+vWLGi2AVt27Z99+6dRCNP\nSUmZNm2amZmZl5fXvn37ZJz8oUOH9uzZMyYmhjoEQA8lYFEchkan0y1fvtzW1tbOzm716tXSLdOr\nVq2aKKEWLVq8efNGrpmfP3++ubm5m5vbjh07qEMA9FACFsVhgEQ6admypdgXVatWFalFroC4Zs0a\ne3t7GxubZcuWyRUQnz9/Xrt2bTHtDRs2DA0NpQ4B0EMJWBSHAdq/f7+Xl5ebm1tycrJcI3/37l27\ndu1ELVWsWPH+/ftyBcTNmzc7OTlZWVktXLhQq9VShwDooQQsisPQfPjwITg4WNLBHzlyxMfHx8TE\nZPz48XJlxMjIyC5duohjoUePHhQhAHooAYvigH6Jj48fOnSoSqXKkyfP+fPn5Rr86dOnb926xU4E\nQA8lYFEc0EfXr18vVqyYQqH4/vvvWaYHgB5KwKI4oL+io6MlGm1aWtrs2bNZpgeAHkrAojigv/bv\n329jYxMUFCTXp7CfP39ep06d9GV6r169knTyRVjkqqQA6KEELIrDAEVERHTo0EHsrPLly9+9e1ei\nkRvAMr1Ro0bVqlXr2bNn1CEAeigBi+IwQMeOHfPz81Or1T/++KN0y/S6du0qiq106dK3b9+Wa9r3\n7Nnj7u6u0WhmzpyZde5ynZKSEhQUVKVy5TKlS4l6e//+PQcgQA8lYFEcBishIWH48OFGRka5c+c+\ne/asXIM/ffp0zpw5jY2NR44cmZiYKNHIY2JifvjhB4VCUaRIkWvXrhl8mYkcWbtWLYXiH4VcLcp4\nWJoYq3x9vMPDwzkAAXooAYviMGQ3btwoXry4jY1NbGysXCNPSkoaM2aMyFg5cuQ4efKkXIMPDg7O\nmzevSqUaPHhwfHy8ARfY5s2bxZlhVAX3XS1zij9L6vpYmqr79OnDoQfQQwlYFIeBS0tLu3nzpqSD\nDwkJKVu2rKi9Tp06yfXeU0pKyqRJk0xMTBo0aGDA1dW6dWs/e/P0dJX+p04OWy8Pd447gB5KwKI4\noNe0Wu3ixYutra0dHR03btwo1zK9Bw8eGPYlSVu1auX/nwGrbg5bT3c36haghxKwKA5I4PXr102a\nNBFFyDI9vbJhwwaxU8ZW9EhPV8vq+VqbqXv16sXMAPRQAhbFkXW9fPlSrgHv2rUrCy7T02diL1Sv\nXk2pUBRzs6zgZWWmNsru5RkWFsbMAPRQAhbFkUWdOnXK2Nh4+PDhCQkJEg07Nja2d+/e6cv0rl69\nKunkp/zKMAopOTl5wYIF5cuVK1akyKhRo96+fcvBBdBDCVgUR9aVlJQ0btw4tVrt5+d37NgxuQZ/\n4cKFfPnyqVSqgQMHxsXFSTf5EyZMyJ8//8WLF/Uks86ZM6devXqtWrXas2cPF6MH6KEELIoDn+vu\n3bvlypUTO/e7776LiIiQaOQZy/S8vb0PHjwo17RfvXq1SJEiCoWid+/e3/YKGu/evcsZkEOlVBZw\nNvfJphGV0LVLl6ycsZKTk+fPn1+6VMmC+fMNGjTo9evXnCVADyVgURz4FFqtNigoyMbGxsHBYd26\nddIt06tcubIoztatW8v15lRqaurMmTM1Go27u/vu3bu/1TB69eplaapeUNs7/VPq3xdzFpP51V7R\nFMW2a9euRg0bli9XbtiwYaGhod98p9QIDFQpFcXdLat4W1uYGru6OL948YKzBOihBCyKA58oLCys\nWbNm5ubm0n1CWTTpVatWZfvVypUr5QqIT58+rVmzpji4mjZt+k1eLHFysG+QM1vGRRZ2tghwtTbr\n3r3713n2/v37i23P4WBexsPS3MTYwd7u3r1733B3bN26VYxnzP9dN3VlfV9bjUnXrl05P4AeSsCi\nOPBZnj9/LunIw8PDW7duLaq0cuXKDx8+lCsgbtiwwdHRsUyZMl//2W1trJvntfv4QlbeduYdO3b8\nCk99/fr1X96bLuSY/ryrG/q5WpvWq1vnG+6L9u3b+9j9x2W9GuXK5mCXbfr06T169Jg9ezZ3WgQ9\nlIBFcSArOnjwYPbs2U1NTSdNmiTXMj3RuUNCQr7+87Zq2dLJ0nRdY//0PDG+soc40rds2fIVnnrM\nmDFWZurtzQMy0kznwo5KpfIb3p78u+++y55N83HAKutpqVIq1EYqH3sLI5Uym63NlStXONBADyVg\nURzIcuLi4gYNGqRSqfLly3fhwgUm5M/dv3/f1sbaztykXoBtxexWxiplxQoVvs41xkaNGmWt+Y+A\n1a2ok0KhSEpK+lazsWvXLnGiG1zGNX08i+t4GykVuRw0axr5iX+uauDnb2+eMyAHCy1BDyVgURzI\nHLdv35ZrwBnL9Hr16iXdja6/skePHnXp0sXbyzNv7lyTJk36atdFu3jxojir9CzunJ5mNjbx97Q1\nCwys/g2nQqvVNmncSIwqn7NFKQ9LI5VSfD2tmldGBBxX6ZdX+HgRC/RQAhbFgcwJK2Lvt2zZ8s2b\nNxIN++Nlert27ZJ08sVWREVFGWRd6XS6jt99J0qrkKtloK9NNnMTSwuL69evf9tRpaWlrV69ulbN\nGhXKl2/Tpo0Y3rJ6vhkBa34tb/HI8ePHv3TOe/r0KZdspYcSsCgOGDjRCEXLsbOzs7W1Xb58uaTL\n9Bo3bizjNY1ERnR2dt6yZYtBvi0lksSqVasqlC+XL0/u7t2769vqhPDwcGMjo5Z57TMCVoOc2czN\nzKKjo7/ck27cuDG7p+c/flWtWtXHjx9zCqKHErAoDhgy8ft027ZtRRlUrFjx/v37cgXE9GV6VlZW\nixcvFk1dosG/evWqQYMGYtrr1q1rkBdkio+Pv3v3bmRk5BeNcZcuXTp+/PgnvFk8ZMgQMfnlvay6\nFXUq42klvp4wYcKXG+revXvFU5R0txxV3r1XcWcXK1MPdzfe46aHErAoDhi+w4cP+/j4mJiYiDbz\nDVd7fYL379936tRJlHGZMmXu3Lkj17Rv377d1dXVwsJi7ty5aWlphlFLqampw4YNszA3FzvF2Mio\nfbt2MTExmf4swcHBfr4+6S8ImWs006ZN+7vpfN68eX4+3kqlMmeOHEuXLv2iLyWWKlkin7PFzhb/\n/4P/S+v6qJSKhQsXcuahhxKwKA4Yvvj4ePFrvaWlpYxvXpw4cSJHjhzGxsajR4/+hqvVPkF0dHSP\nHj3EYVi8ePEbN24YQCENHjxYpIfGubNNquLZsZCjuYlxjcDAzH2KN2/eWFla5nI0n1DZc14t77o5\nbMUErlmzRm/nxNRE3fH/LgyW/sfP3rxz586cduihBCyKA1mFvFdcTExMHDlypJGRUUBAwOnTp+Ua\n/NmzZ3P/Sq43On8vJiZGY2baPM+/r3E6uIyrOMlcu3YtE59l8uTJJsaqNY38M56lmJtlwQL59XZa\nsnt6VPWxzhjt5qY5LEyNR40axQmHHkrAojgAOdy6datUqVK/3Nu4a1e5luklJycbwAefr1y5IiZ/\nevV/XwFha7Mc4pGVK1dm4rN06NAhp6PFxy8I/fJSmZmZ3k7LpEmTjJTKXsWdtzUP+LmhXzkvKyMj\nFZ9zp4cSsCgOQCZarXbhwoWWlpbOzs6bN2/m6pFfU2hoqEKh+OH/LoIl/kyr5iVOMocPH87EZxk/\nfryZ2ijjOvXiTykPq3x5cuvttKSkpLRq1eqXD6WplArFP8xMTTZu3Ei10EMJWBQH8K8zZ85Iukyv\nTp06BrlMT2/VrVPHVqMeV8ljZ4uAWTWyZ8+m8fHOnrkf4Rcxzlxjlt/ZYnp1r6B6vk1z24kdvXz5\ncj2fmRs3bsyZM2fFihVfdHEl6KEELIoD0nj69KlSqSxdurR0V35PX6Znbm4+e/ZsSZfpiWF/zr26\nHz16tGbNmr179361xaFv374tUbyYOLGolArxt5enx5comxMnTri5uqSvIjRRq8eMGcNLlaCHErAo\nDsjn5MmT6cv0Ro0alZiYKNHI05fpKRSKYsWKffPriX+CpUuXmpmZTZs27e/eSVB8f/oV1dM5OTqc\nOnXq64xZZB3xXPPnz9+9e/eXuwGi+MkiZomniIiI4AgFPZSARXFAViJXiXQlMlZAQIDIW3IN/ty5\nc3ny5FGpVEOGDPlqd+XLFB8+fOjXr59SqSxYsODly5f/+v84btw4lVLZrYjThib+c2t653O2sLSw\nIIsA9FACFsUBfXT79u3SpUuLsunUqZNcHyVJTk4eP368Wq329fU9cuSIXNMuopUIWCJmibAlItf/\n/H6dTufoYF/T3ybjY+A/N/QzUinnzJlDDQP0UAIWxQF9lL5Mz8HB4e7du9IN/t69exUqVBBl365d\nu3fv3kk08pSUlGnTppmZmXl6eu7bt+9/pkmxjb1LuHx8LQM3G03//v31aos2bdrUo0cPsS+6d+8u\n4u+tW7c4vkAPJWBRHMjS5Hqj7WM6nW758uW2trb29vZr1qyR68PRjx8/rlatmre3959fsF5slK+3\nd1lPq4x0taiOj0LxjxUrVujJhsTExKR/Ft7d2sRSrRJfqFVK8fegQYO+1R5JS0sTyXXUqFGzZ89+\n8+YNxzjooQQsigP420QHbdGihah/kVfkuuSjyB+vX7/+n9+2ZMkSsXW1/W1/qu41uIyru7WZu5tr\nfHy8nmzFgAEDzNRGU6p6ivC3vXlAszy/XGqhbg4b8feuXbvENp48ebLfr8QXXyFyRUVFpQc+OwtT\ntZHKRK3eunUrhwnooQQsigP4FPv27fP09DQzM5s6dWpKSoohbZoIJdOmTbO2skxfRVi6VMkHDx7o\nz/CcHBzqBdhmvMAmMpatmVH9ANscDuYNGtRv17atGLOjpan4I75o26bNl74MW+fOnS1MjSdU9tzZ\nImBdY/8ynlYiY/E6FuihBCyKA/pi7969+vMyyV+RsUyvQIECly5dMrDdIfbF5cuXHz16pG8DMzcz\na1fA4eOPiGW3Manma13O0zIgh784L/Up4SKyjvjTt6TLl76Xs8jWZqYmrfLZZwxGZCy1kWr27Nkc\n0aCHErAoDnx7ERER5ubm3t7ehw4dkmvkGcv0+vbt+1eW6ekhrVabufdU/qJq1qyR3VazpVmO9EAz\nvfovd9T5vpiThalx9uxeBVz+41aDBZwtqlWr+uUGExMTI569f6l/rwkQwS6bucnIkSM5qEEPJWBR\nHNALDx48qFSpkqirNm3avH37VqKRp6am/vTTT+nL9Pbu3SvdzG/evFmhUHz//ffR0dH6MJ7du3eX\nKF7MRK328vCYNGnSb644eunSJVMTdXZbsw4FHRrmymZmpHTQGLlZm2aztSlYoEDF7FYfByzxz+JF\ni365oep0upwBOYq6WYpclf6MEyp7ihr+n6s1QQ8lYIHiwNcj2tXKlSttbW3t7Ox+/vlnuZbpPXny\npHr16uK4aNasWVhYmEQjT0tLmzNnjoWFhaur6/bt27/tYNatWyfmMJ+zRefCjtV8bYyUytatW//m\ney5evFgjMFBtbGxibJz+QTHxzzt37gwePNjCxPjnhn7pWWd1Qz8LU2Px4Bcd8KZNm8QARMbqX8ql\nZT578YwiHUp6eyXQQwlYFAcMWXh4eKtWrUSBVa1aVaQWuQLi2rVr7e3tbWxsli1bJldAfPHiRd26\ndcW0N2zY8NWrV98q6nl6uJfy+PcLQj8UdxZD+pPLXMXHx2dc+CM0NNTRwd7R0rRNfnvx55ePujvY\niwe/9LBFxsoVECDGaWZq0qVLl6ioqD/6zocPH544cUKu/A16KAGL4oBBOXDggL+//82bN6UbeURE\nRIcOHcQBUqFChXv37skVELds2eLs7GxpablgwYIvvf7u9x4/fizmbXQF94z3+LY1DzBSKefNm/cX\nf8KjR49aNG9uamIi/ogvvtqH9MXUxcTE/Ml60jdv3lStUjn99TaVStmpY0e5bs0JeigBi+KA4ZD6\nfZajR4/6+vqq1eoff/wxOTlZopFHRkZ27drVw8MjNjb2Kz91eHj4b64jv6qBn0Lxj1WrVv3duKNX\nLx+KwZQoXszO3KR/KZeFtb07FXZUG6m6d+vGMU4PJWCB4gD+toSEhGHDhhkZGeXOnfvcuXNyDT4m\nJuabPG/VKlVcrc2W1vUR6Wpz0xxlPa3MNWZ/8qabFMTeFyfMMRX//cpcuwIOJmq1XDflBD2UgEVx\nAHrkxo0bxYsXF8dLjx499GSZnj579OiRh7ubSqnwsze3MDE2NjLavHmz7Bu1fPlyUQDbmwdkBKyf\nfr26xNWrV9nj9FACFigO6ItNmzZJt0xv7ty5FhYWLi4u27ZtYw/+udjY2IULF3bp0mXMmDFyrXL4\nIydOnBAnzElVPDMCVqfCjkYqlVx3DQc9lIBFccCQxcfHu7m5WVtbL1269Ot/CvtzZCzTa9CgwcuX\nL2WcfJ1Od/jwYblWR+oDUaj58uZxtjIdU8F9dUO/viVdNCbGbdq0YWbooQQsUBzQIxERER07dhQV\nWL58+bt378oVULZu3Zq+TG/+/PlyBUTh+PHjYtpr1qz59OlT6vBvefbsWeFCBf/xfxrUry/pdf9B\nDyVgURwwcKLZ+/n5qdXqcePGJSUlSTTyqKiobt26iSOoVKlSf3KFJ/20e/dud3d3jUYzc+bM31xj\nHf8zXl+6dGnbtm1yXbwD9FACFsWBLCchIWHEiBFGRka5cuWS7nbLZ86cEcMWgxebINclkWJjY3v3\n7q1QKIoUKcLHtAF6KAGL4oBhunXrVtmyZa9cuSLdyJOSksaNG6dWq/39/Y8fPy7X4C9cuJAvXz6V\nSjVw4MD4+HjqEKCHErAoDhgaqT92fffu3XLlyokDqmPHju/fv5do5CkpKZMnT/b29pb9OlUAPZSA\nRXEABkir1S5dutTa2trR0XH9+vVy5UW5rlMP0EMJWBQHkLW8fv26WbNmLNP7HCKbBgcHjxo1aty4\ncbdv32ZCQA8lYFEcQOb7+eefpbtd9O7duz08PDQazYwZM1im97ekpaW1ad1anJqsNWoLU2PxxfDh\nw5mWjOj5/Pnz0NBQpoIeSsCiOIDPbbeFCxc2MjIaNmxYQkKCRCOPjY3t06ePQqEQ45f0KBPtfMOG\nDSkpKV/zSZcsWSImrXcJlx0tArY2C2iT316cpg4dOsSxcPTo0Ty5c6Vfgqt4saKXL19mTuihBCyK\nA/h0ycnJP/74o1qt9vPzO3bsmFyDv3jxYv78+dOX6cXFxck1+Bs3boiR58uX78KFC7//r8+ePZs3\nb57YNdOnTxebmVmfOStZonhxd8uM29HsbBHgnU3TsmXLLH4UXL16VW1snNfJfGhZt4GlXX3tNFaW\nlmIXcH6ghxKwKA7gs9y7d698+fKiYjt06BARESHRyFNSUqZMmWJqapo9e/aDBw9K19eLFCmiUCh6\n9eoVGxub/uD169fr1q2rVCpF6nVycjI3Nxf7pXDhwplyi0Z/X+86OWwzApb4U9zNIrB69Sxe/61a\ntnS1Ntva7P/fanp9Y39LU+NBgwZxZqCHErAoDuBzabXaoKAgGxsbe3v7vXv3yjX4R48eVa1aVRxx\nrVq1Cg8Pl2jkqampM2fO1Gg07u7uu3fvPnbsmEhUuXLlWrp0afprcmlpafv27atevbrYuqlTp37m\n04kA7WhpuqlpjvQksaqBn6mx0dixY7N48Qf4+/0md5b2sKxcuRKnBXooAYviADJHWFhYixYtzp8/\nL93IdTrd6tWr7ezssmXLtnLlSrmu4/D06dOaNWuKM4aRkVGFChXSr0p65cqV5cuXHzhwQIQwsTkj\nR44U37Bly5bPeaKQkBBzjZl3Nk2Pok6dCjs6W5k6OznKFUm/hEoVKxRwsfj4nVN3GzNuNU0PJWBR\nHAD+v7dv37Zt21YcepUqVXrw4IFcAbFGjRoqlUqErZiYmKpVKmfc9tjby+vWrVvp31CgQIHPzI6X\nL1+uXKmS+LHiuRrUr//w4UPKZsOGDWJC2uZ32Nw0x7rG/vUCbMU/T506xczQQwlYFAeAfzt8+LCP\nj4+JicnEiRNluchnXFyctbX1kCFDxNdtWre2MDEeWtZtW/OAGdW9vLNpvDw9UlJSDh48KM4qZ8+e\n/fynS0xM5PKnH6fbwYMHKxQKlVKpUiqMjFQzZ85kWuihBCyKA8BvxcfHi5apUqny5s0bHBys/wM+\nf/68OGNcvXo1MjLSSKXqVNgx4x2r2TWyi/+0Z88erVZrY2MzadIk9u+X8Pjx4wULFixZsuT169fM\nBj2UgEVxAF9JUFDQ/v375RrztWvXihYtqlAoevbsGRMTo89DPXLkiDhjiB4fEhIivphUxfPjjwSp\nlArR+8W3eXp6cmlQ0EMJWBQHAQsGQqfT1a9fX5R0ixYt3rx5I9HI09LSZs2aZW5u7ubmtnPnTr0d\n55UrV8T0njlzJj4+3sLcvFGubBkBa0xFd/GfRLpNTEzUaDQ//fQTBQl6KAGL4iBgwXAy1po1a+zt\n7W1tbZcvX/6bj1qHhYWNHz++UqVKxYoVq1q16uzZs6OiovRn8M+ePatVq5Y4JBs1aqSf90JJTU31\n8PDo1KmT+Hr48OEqpaJpbrtp1bx6FHOyMjVWqVTOzs69evUSm3Dv3j2qEfRQAhbFQcCCQXn37l27\ndu1EbVesWPH+/fv/+vVSn6LxGxkZmZmZifgiIkLt2rXFPzUajYhc+nO5BDGSTZs2OTk5WVlZLVy4\nUKvV6tvcTpgwQczh06dP09LShg4dam5mJuZZoVA0aFD/5s2b6a8gioD74sUL6hD0UAIWxUHAggE6\ncuRI+jK9JUuWNGzYUMSpKVOmfPySVVhY2KBBg8Qh0KdPH726JFVkZGTnzp3FwEqXLn379m29mtX3\n79+LWc2RI0f6TVpiY2OvXbsmZlJ8nZCQ0KBBAzHPImBZWFjMmTNHhLD01HjmzJlRo0aJcJaedwF6\nKAGL4gAkFh8fP3To0J49eyqVyt27d4tH7ty507dvX5EDhg8fnh4RFi5cKI6C7du369vgT548KXKM\nsbGxiCaJiYn6M7CHDx+KjGVubt6tW7cLFy6EhoaGhIT8+OOPbm5uZmZme/bsiY6O/v777xUKRfHi\nxcV/atK4sZhhW42JuYmx+GLixIlUJuihBCyKA5BbWlqat7d3u3btxNdbt241MjKysXcoULqiuaWV\nuYVF+uWaypcvX6FCBT0cvMhVIl2JjBUQECDylv4MLCIiYuzYsa6urhkXGtVoNF26dBFxKuN7zp07\nV65cuXHjxqmUigGlXXe0CNjSLEfTPHaZdZUsgB5KwKI4gG/m0KFDosgvXrwYHx9vbWNTvEqtny88\nWn/15bLTIQEFigYE5NTpdBs3btTnz2Xfvn27dOnSYoSdO3eOjIzUn4GlpKScP39+7969J06ciI6O\n/q/fky9P7nJeVhmLDUXMcrU2ExtCZYIeSsCiOACJzZkzx8zMTKSoXbt2iWqfufO0SFfpf4YuWCse\nuXHjxps3b8QX4hv0diu0Wu3ChQutrKycnJxEHJToJoZuLs5Nctt9fIviAs7mDRs2pDJBDyVgSezD\nhw99+vRxdXU1NTUtWLCgOC9THMhqZsyYYWFhIb7YvHmzqPaFh/+Znq46j5xap1339Be33r17J77Y\ntm2bnm/Lq1evGjVqJIZaq1at9A+Q6b8mTZq425htaZYjPV0tq+drrFJylSwQsAhYcqtWrZqtre3S\npUtPnjzZpUsXhUKxfv16igNZSnquevToUXh4uFqtrt/xh/SAVatNF/G4sbHxtWvXjh49Kr6+dOmS\nFFu0Y8cONzc3c3PzWbNmpS/T02dXr141NVH72Wl6FnP+rpCjg6Wph7vbuHHjHj9+THGCgEXAktK+\nfftEovr4Vavq1auL8/LHV9YhYMHgJSQkiF8zBg0aJL4WfV0UfP6S5Rt27u2fr5D42sXFRWSsXLly\n5cyZU6L33WJiYnr27CkO8KJFi4qAqOej/eXT7mXLitk2MlI1adz4+vXrXl5eZmZm06ZNS01NpURB\nwCJgSaZz585WVlYfx6kNGzaIM/L58+cJWMhSBgwYYGlpefv2bRGh1q9fX7p0GRdXt6rVqh08eDAp\nKal9+/biKHBwcDhx4oRc2yWO5Tx58qhUqsGDB8fHx+t/0k1OTk7/+sOHD/3791cqlQULFpTlhUOA\ngEXA+v9KlixZokSJjx8RDUYErKCgIAIWspSYmJh8+fI5OTkdO3bs45epxK8fmzdvNjc3FwdL+jK9\n0aNHy7VpIrJMmDDBxMTEx8fn8OHDcg3+8uXLImCJmNW3b18RuShUELAIWHLw9/evWbPmx4+8fv1a\nBKwpU6b8pjjKlStX9yO/+ZwWYADevn2bHqHy588/adKkxYsXjxkzRoQS8Uj9+vXj4uJE2BIPHj16\nVMatu3//fsWKFcW2tG3bVmypRCNPTU2dNm2amZmZp6fnhQsXKFToOdEfP26XonsSsAhYfxaweAUL\nWYGIUIcOHapXr56FhYU4EGxsbNq0aXP+/HmJPnr1J8RWrFixwtbW1s7O7ueff5Zro548edKoUaNX\nr15RpZALr2BlUSVLlixevPjHj/AWIZARRwxyu8LDw1u2bCkO6qpVqz569IgdDRCwCFiZr2vXrpaW\nlr//kHtwcDABCzBg+/fv9/LyMjU1nTJlSkpKChMCELAIWJnpwIEDIk5t2rQp45HAwEB3d/ePf3cn\nYAH/U1BQ0KBBg+Li4iQasxjtgAEDlEpl/vz5L168yE4ECFgErMxUvXr1bNmyifZw/PhxLjQKfJo5\nc+aYmJhkz5794MGDco38ypUrhQoVEgd+nz59YmNjZZz8sWPHcmdoELAIWPr4W6w4sbq4uIj2ULBg\nwY9fzSJgAX/dgwcPKleuLA6W1q1bS7dMb/r06RqNxsPDY+/evXJNe0JCQokSJcS0d+/e/Y9uJg0Q\nsAhYFAcgMZ1Ot3Llymy/WrVq1Z98TF4PP0H/5MmTwMBAcbA3a9YsLCxMomlPS0ubO3euhYWF+EVx\n27Zthro6AfRQAhbFAWRp4eHhrVu3FkdN+/btf/Ofbty40b17dwcHB6VSaW5uXq1atR07dujPrWBE\nNFm3bp0Yno2NzdKlSz9e/qL/Xrx4Ubdu3fSLlr18+ZI6BD2UgEVxAAbo4MGDR44cyfhnSkpK586d\nxaHk6uo6dOjQRYsW/fTTT6VKlRKP5MmT5+nTp/oz8oiIiO+++04MrHz58nfv3pVozkVA3Lp1q7Oz\ns6Wl5Z49eyhC0EMJWBQHYMhE42/durWRkdHixYt/c02Ey5cv+/j4eHp6vn79Wq/GfOzYMT8/P7Va\nPW7cuKSkJIlmOyoq6ocfftCrzAp6KAELFAe+jZiYmGXLlo0dO3b79u2Gd1mmHTt2iINow4YN4uvg\n4OD6DRq4uXsULVpsyZIlWq325cuXzs7OrVq10rdhJyQkDB8+XOTCXLlynTlzhioF6KEELIoDMjl7\n9qyNra1SqbS1dxDFlitXbgO7I0rVqlVLlSr1r1/fN1SpVB5+AfU7/lC0YvX0tW/i8RkzZhgbG4eH\nh+vh4G/evJmxTC8qKopyBeihBCyKAxJISkpydHLKVbjEvAOX1l99OWHtPntn1xr/eYtMqYmwKI6g\n9Hv/5c6dx9Xbr3iVWgsOXREb237wj+I/hYSEvH//3sTEZPbs2fq5CWlpafPnz7e0tHRxcdm6dSvL\n9AB6KAGL4oC+S3/7bPq2EyJwpP/pNnaGeCQ0NNQwNjA4OFhszo0bN9KTVp32Payy2WssrDqNmLzy\n/ANjtcmsWbP+9eud1wcMGKDPG/Ly5ct69eqJTRB/v3jxQsZ9MWzYsHXr1hEQQQ8lYFEcMHxBQUGi\nwFZffJIRsMYs3yYeuXnzpmFs4KVLl8TmXL169e3bt7+80TZu5tITtyo2aCG+9stXWKlSLVq0SHyb\nj4/P4MGD9XxbRDTZtm2bi4uLhYXFvHnz0tLSJNoRYrTNmzcX0x4YGPjkyRMOPdBDCVgUBwzZjRs3\nRIF9P35ORsCq3ryDbbZsiYmJhrGBERERxsbGM2fOFF9XqFDR3cd/0dFrYjNHLN6osbQS2z5gwIB7\n9+4pFIoVK1ZIsUXR0dE9evQQIy9RooR0OXj37t0eHh4ajWb69On6cwUy0EMJWBQHkPkaNGxoYmpW\nv+MPfaYtLl+vmai3qVOnyjJ4nU6XkJDw5+86tWzZ0s/PT6vV3r59287e3tRMk79UBUdXd7GlNWrU\nMDIy8vLysra2lute0WfPns2VK5cY/LBhw8QMSDTy2NjYPn36iERbqFChK1eucACCHkrAojhgmER7\n7tu3r4Wlpag0Nzf3uXPnyvIpmRUrVvj5+YthZ7OzGz58eHJy8n/9tvPnz4vvGTVqlPj6zZs3P/74\nY/369bt163bp0iXxyNq1a9VqtZ5/AOu/SkpKGjdunBi8iI/Hjh2Ta/AXL17Mnz+/UqkMCgriGAQ9\nlIBFccBgpaSkRERESHR7lnnz5olDo2S1Ot3HzazZuouxWt2sefM/+uYpU6aIb+7SpcuzZ88yHoyP\nj1+yZIm5uXnZsmXlfUv07t275cuXF1vXoUMHsQflKjmxXx48eMDRB3ooAYviAPRCcnKyvYNDxQYt\n/r34ccx0caTcunXrj/6XhQsXWllZKZXK6tWri6TVokULGxsbhULRqlUrud4c/D0Ri4OCgsTmODg4\nsEwPoIcSsCgO4BPdvn37l3f9grZkBKzVFx+LtLRkyZI/+b9EkBJBJDAwsFixYuXLlx8yZMifLGcT\nMUWuZXphYWEs0wPooQQsigP4dKGhoeK46PHjrIyANWvXGfHIli1bMusp1q9fL3LY9evX5ZqZvXv3\nenp6skwPIGCB4gA+RdVq1RzdPKZtOSrS1eJj1/OVLG+bLVt8fHxm/fzg4OA8efKoVKohQ4bItUzv\nw4cPffv2VSqV8i7TGzx48IQJE/5o1QJADyVgURzAl/Ls2TNfXz9xdDh7eBmr1eYWFkePHs3cpxAN\nfvz48Wq12tfX98iRI3LNz6VLlwoUKCBiVv/+/eX6kJlOpxs2bJiItnnz5hUxl1IHPZSARXEAX1Vi\nYuLatWsHDRo0Z86cd+/efaFnuXfvXoUKFcRh2K5duy/3LF9CSkrK1KlTzczMvLy89u/fL9fOvXbt\nWtGiRRUKRc+ePWNiYqh20EMJWBQHYGh0Ot2yZctsbGxy584t0ZUs0j169KhatWriNNKyZcs3b95I\nNPK0tLRZs2aZm5u7ubnt3LmTOgQ9lIBFcQAGKCws7PTp05IGxDVr1tjb29va2i5fvlyu6zg8e/as\nVq1a4jQ4duxYihD0UAIWxQFAv7x7965du3bifFKxYsX79+/LFRA3bdoUEhLCTgQ9lIBFcQDQR0eO\nHPHx8TExMWGZHuihBCyKg4AF6DW5ruMQHx8/dOjQ9GV658+fZ/eBHkrAojgA6J1t27Z5enru3btX\nrmFfv369WLFiCoXi+++/l3eZHvcFAj2UgGWAxZGWljZz5kx//xwac/PSpcvs2bOH/YUs6PHjx+nL\n9Jo1axYWFibRyMUhPHv27PRlejt27JBx8ocMGdK5c+fIyEjqEAQsApbhFEeXLl2USmW52o1b9R2Z\nu2gpMc41a9awy5AF6XS6tWvX2tvb29jYBAUFyXUph+fPn9epU0ccvw0bNnz16pVcM7906VIrKysn\nJ6eNGzfyahYIWAQsQyiO+/fvi4F1GDIh/U5w6/75okS1Ou7uHtz+DFnWu3fv2rdvL46L8uXL37t3\nT66AuHnzZhFTRFhZuHChXAExNDS0UaNGYtpr164twiJ1CAIWAUvu4liyZIlCoVgV/DDjbrvDFq0X\nQ5WrrwCZ7ujRo76+vi4uLklJSXKNPDIyskuXLuIoLl269O3bt+Ua/I4dO9zc3MzNzWfNmpWWlkYd\ngoAFWYtj3bp1YmDzD17OCFg9J84Tj0j3FgOQ6RISEi5duiTp4E+dOhUQEGBsbDxy5MjExESJRh4T\nE9OzZ0/xi5+IiRQhCFiQtTjEuczcwqJktTorzz0Q6WregUtu3n5ly5VjlwGyS0pKGj16tMhY/v7+\nJ06ckGvwwcHBt27dYieCgAWJi2PLli3iFGxhZe2fr7CRsbGzi8vdu3fZZfrjxo0bPXr0qFGjRt++\nfR88eMCE/HU6ne7MmTMzZsxYvXr1hw8fsuYk3Llzp0yZMuL806lTp/fv31MVoIcSsCiOr+fRo0cj\nRozo0KHDrFmzWCmtVzZt2qRSqeydXYtWCrTOZmdqanrkyJE/+ua3b98OGDDAx9dP/Onfv394eHhW\nnrrY2NjKlauI487UTCP+traxOXbsWGb9cLmSilarXbRokZWVlaOj44YNG1imB3ooAYviQJYWHx9v\nY2tbslqd1RefrL/6ctX5h3mLl8me3fu/rg4TydjfP4e5pVXVpm2rNWtnYWXt6+sXERHx509x8ODB\n5s2bV6pcediwYQb2wbuuXbtqzC0Gzl659srzufsu5CtRztLKKjo6+vN/8v79+y0tLefPny/Xp7BD\nQ0MbN24sTkQ1a9Z8+vSpdDs0PRempqYSEOmhBCwQsPBZ9u7dK8pm+vaTGUsQRgVtEY9cvnz59988\nfvx4E1OzWbvOpH/n7D3nTM0048aN+5OfP3r0aPHTsgfkLl6llkhm2ezspFt09kcSExPNzMya9BiY\nMXULDl1WKpVBQUGf/8NFlhXpTUxdyZIlb968KdfM7Ny5083NTaPRzJw5U66rseh+NXLkyKpVqz56\n9IjzAz2UgAUCFj7Rjh07RNnM3XchIyWMX/NL5PqvN54rXaZMiWp1Mr5T/CkVWK9EiZJ/9MPTL4HW\npPuAdf98Ib55yfGbbt5+1QMDDWPq3r59K7au7/SlGbMhNtPKJpuIoZn1FKdPn86ZM6eRkdGIESOk\nW6b3ww8/KBSKIkWKXL16Va49e+DAAS8vL1NT0ylTpqSkpHCWoIcSsCgOAhb+tsjISDMzsxqtOqVn\noLVXnpcOrOfg6JicnPz7b65UqXLBMpU+DliFylUpX6HCH/3wqVOnmmo0H18CrdOIyaJKDePD4Fqt\n1sPTs2ytRhlbN3rZVrF1+/fvz8RnSUpKGjt2rFqt9vf3P378uFxTFBwcnDdvXpVKNXDgwLi4OIlG\nLkY7YMAApVJZoEABea+jAXooAYviwJcioszVq1dDQ0P/5HtmzJghKiegYLHa7bp558wrvl69evV/\n/c5FixaJljNs4br0PDFi8UbxzwULFvzRT544caLGwjL9013pf7qNmS5+fqZ8SkkfLF68+Jersddt\nOnje6nYDx1pnsytcuMiX+NRUSEhI2bJlra2tpZu6lJSUSZMmmZiYeHt7Hzx4UK7BX7lypXDhwgqF\nok+fPrGxsZxP6KEELIoD+OX1lVGjRplbWPzjV7Vq1/6Tuwvv2rWratVq3j6+tevU+ZN1cMnJydUD\nA8VP889fOEeBIuKLKlWr/slVyK9fv/7xXZJWnnvgkyvfX7wEWmpq6smTJ/fu3avPi+l0Op3Il84u\nLmIzjYyMmjVv/u7duy+3Q0XMkrQaHzx4ULlyZTFLrVu3lmvlqahD8RuIRqOpW7cuZxV6KAGL4gB+\n+UC6+M27bvseP/68u+uY6dkcnAoVLvz5d44T/WbdunXNmjVr2rSp+OJ/fkKlR48eoizzlyxfrVk7\ne+dfPvh84cKF//ksp06dcvfwSI+GpqamY8eO1edVXWJOnj17ZjAvy325MLpy5cpsvxJfyLVM7+nT\npwazOAP0UAIWxYFPl5SUZJstW2DLjv9eG/jrx4MOHTr09dvq6tWrK1SomCt3no4dO965c+d//i+h\noaHmFha5i5T6cfWe2XvO1fuupxj58uXL2a0GIDw8vHXr1mKHVq5c+eHDh0wI6KEELIoDMhG/cIti\nGDJ/7ccL3Ew1munTp+v/4CdMmCCGGnTqdsbgi1UKzJcvP7v1jzx79kyuAR84cCB79uympqaTJ09m\nmR7ooQQsigPS+PDhg+hezXoOzsgo07edEOWxadMm/R98hw4d/PMV/nitYpv+ozTm5uzW/+r06dOS\nLtMTY1Yqlfny5fsr7xrrJ5EO/+QziKCHErAoDhigrl27mpmb/zB5/s8XHk1af8And34XV1cprqX0\ny6IzU7OlJ25lBKwiFaoVLFiIffpHPX7y5MkiT2fPnl26ZXrifJW+TK9Xr14yLtMbP358rly5zpw5\nQx3SQwlYFAeyiri4uNp16vzj/3h4espyvcc3b95YWlkFFCg6atnWn7Ydr9mq859cOQLpHj58WKXK\nL3dFbNWqlaTL9Nzd3Xfv3i3XtN+8ebNkyZJi2rt16xYVFUUd0kMJWBQHsorr16+vWLFi//79ct2x\n5Pz58z6+vunR0NzCYsqUKb9fdPb+/XuxXadPn5brbn1fjpiin3/+OX2Zntjp0i3Tq1GjhtjdTZs2\nff36tUQjF+U3f/58S0tLZ2fnLVu2cBNDeigBi+IA/pLnz5+vXr1669atX/kjPlqt9sKFC0ePHo2J\nifl9khg7dqypqWl6AvPyys57NBnevn3bpk0bc3PzP7+6rH4GxA0bNjg6OlpbWy9ZsuTzLynyNb18\n+bJevXqiGsXfL168oA7poQQsigP4s4bXr18/pVKZnmOsrK137NihDwNbtmzZL53su56zdp8dt2pn\nrsIlLK2s3rx5k1k/X7T5EiVK2ts7lCpVWiRLGfed6PeSVt379+87duwo9m+5cuXu3r0r1/Gybds2\nFxeXkiVL8joWPZSARXEAf2jRokWiolr0HrbsdIiIMsUq1TA2Nn706NE3H1iePHmLV6mV8RH4pSdu\nmZiaTZ48OVN++JQpU8RWFyhdoen3g/KWKCu+nj17NsXwlR0/ftzf31+tVo8dO1auZXrR0dH37t1j\nD9JDCVgUBwxTamrqnj175s2bd+zYsU9+qyVv3nwlqtbOyDErzz2wtLYZNmzYN986U1PTdgPHfnwd\nB7+8BTt16vT5PzkqKsrMzKxm6y4ZP7lq07aWVlaGcXdquSQkJIwYMcLIyIhleqCHErAoDuiFu3fv\nevv4/HJ3PGNj8XehwoU/7VPD1jY2LXoP+zjH5CxYrHXr1t98A/Ply1+sUo2MUS05flNtYjplypTP\n/8n79u0TMzZr15mMHz5ty1HxyJ/cn1Eu169fl2vAt27dYpke6KEELIoD355Wq82ZK5eHb45J6w+s\n++eLUUFbsjk6V6tW/RN+VJmyZXMXLSV+SHrUWHT0WmblmM+0YsUKUeq123advv3kqGVbAwoUtbK2\nzpTPYB0/flz85Alr92UErDHLt4lHzp8/bwC1cePGDbEtjRs3luvz74axTC81NTUiIoITFD2UgEVx\nQFanTp0SZTB62daMiPD9+DnikSdPnvzdH7Vr1y7xP5ap2UCEjP4zl2XPmSebnd3bt2+/+TaKFjtx\n4kQzjSb90/c+vr6ZFYBSUlJcXF3zlSy/4tx9MXXLz9wVEdPLK7thXAlCzNumTZucnJysrKwWLVok\n3TK9+vXry7tMb+bMmfb29mvXruXz7/RQAhbFASlt2LBBlMGyU3cyAtaUTUfEI+fOnfuEn7ZixQon\nZ+f0HFO4cJFr167pz5ZGRUUdPnw4ODg4c4OC+JlmZmYWVtZ5ipXWWFhqzM1PnDhhSBUSGRnZufMv\n124tU6bMX7n3tl7Zvn27i4uLhYXF3Llz5Uq9YWFhzZs3F9MeGBj4Cb/tgB5KwKI48I2FhISIMugx\nfnZGwGr6/SATE5P3799/2g9MTk4WuerBgwdZ5zfvp0+fDh8+vFmzZqNGjXr+/LlBbuPJkydz5Mhh\nbGw8evRo6Zbp9ejRQxR58eLFb9y4Ide0792719PTUyT4n376Sa5r/9JDCVgUBwEL/2rQsKGpRtOi\n97CRSzfX79hLZWTUp08fpgW/kZiYKBKktbX148ePpRv82bNnc+fObWRkNGzYsISEBIlG/uHDh759\n+yqVykKFCl25coU6pIcSsCgOSCMuLq5z584mJibp95kZPHgwvyvjj0RHR0s68uTk5PHjx6vVal9f\n36NHj8o1+EuXLhUoUCBXrlxyfRKOHkrAojiAXxrn/fv3v/L9bYCv7N69e+XLlxenvvbt28u1TC8l\nJeXp06fsQXooAYviAAB9pNVqly1bZmNjwzI90EMJWBQHAL124sQJSZfpVa9eXcZPlYEeSsCiOIBv\nQ6fTvfoVL1F8ac+fP1epVMWKFZPuyu8Zy/SmTZsm6UcPRa4lINJDCVgUB/CVnD17tlDhwhlX5Pq0\ny3rhrwsODs6bN6+IWUOGDImPj5do5B8+fOjXr59SqSxYsODly5elm/mgoCATE5MJEyYkJydTh/RQ\nAhbFAXxBd+/eNdNo/PMX7j11kfjjn6+QRqO5d+8eM/NFiQY/ceJE0ex9fHwOHz4s1+BFtBIBS8Qs\nEbbkule3iLNDhw4V0VYEXMO4RxM9lIBFcQCfKyEh4eTJk6IrZO67M926dcvm6Lzq/MP0S6euPPfA\n1t6xR48eTPhX8ODBg0qVKolzS9u2bd+9eyfRyFNSUqZNm2ZmZubl5bVv3z65pv369evFihVTKBQ9\ne/aMiYmhDumhBCyKA1lXUFCQja1t+rt47h4ehw4dyqyfXKRI0Qr1mmVcm178KV+3afHiJZjzr0On\n061YscLFxeXu3bvSDf7x48fVqlUTNdmiRYtMuSP4V5OWljZnzhxzc3M3N7cdO3ZQh/RQAhbFgazo\n4MGDovxE7pm88dC4VTvzlSxvamr64MGDTPnhdevV88tb8OOA5ZM7f4MGDZj2r0muO+r8JiCuXbvW\n3t7e1tZ22bJlci2SeP78eZ06dby9veWdf3ooAYviAD5d9cBAkYHW/fNFegBadf6hdTa7fv36ZcoP\n37Nnj6jthl36rDh7b/mZuw069RL/3L9/v97ORnJycmhoKB1Rr7x79659+/aicipUqHD//n25AmJ4\neDh7kB5KwKI4kBV5Zfeu2+H7j19kKly+Wq1atTLr548bN06pVKqMjFQqlfhi/Pjx+jkPaWlpY8aM\nsc2WTRyMllZWAwcOZC2YXjly5IiPj4+JiYkoIXYN6KEELIoD+q5y5So5CxXPeAXr5wuPbO0de/fu\nnYlP8fz584W/evHihd7OQ79+/UQErNm6y8DZK+t3/MFYrW7foYNh7/qdO3fGxsZKNOCMZXq5c+fm\neh8gYIHigF7btWvXLxfRbt5h1q4zUzYdKVqxurGx8Z07d7LUJERERKjV6qbfD8p4Ga/DkAkKhcKA\nbzn3/v17S0tLDw+PvXv3yjXyjGV6PXr0kPSO11qt9tKlS5x86KEELIoDBm727Nkac/P0VYT2Dg47\nd+7MajNw4sQJse3Tt53ICFiLjlwVj2zfvt2At/rJkyeBgYFiM5s1axYWFibRyDOW6bm6usq4j7Zu\n3SqmvVOnTiLmcv6hhxKwKA4Ysujo6L179x4+fDhrfr47JCREHIP9ZyzLCFijlv3SAs+ePWvYG67T\n6datW+fg4GBjY7N06VKtVivR4NOX6Ynd1KBBg1evXkk0cjHPixcvtrKycnR03LhxI7eQoocSsCgO\nwGCVLFnK2SP7pA0HRbr6adtxL/9cATlzyhU4PllERESHDh1+uVpH+fKZdYWOrxYQN2/e7OTkZGlp\nuWDBArn2V2hoaJMmTcS016pV69mzZxyD9FACFsUBGKAnT574+fn/soTQ2kb87ebufvv27Sw1A8eO\nHcuVK9fNmzelG3lkZGTXrl3FXitVqtStW7fkGvyuXbvc3Nw0Gs3MmTPT0tI4EumhBCyKAzA0KSkp\n27dvnzRp0saNG7PmW6VSv2J3+vTpnDlzGhkZjRgxIjExUaKRx8TE9OrVy8vLS65FnfRQAhbFAQBZ\ngojFY8aMMTY29vf3P378uFyDl+vO1vRQAhbFAQBZS0hISNmyZcVJtWPHjizTo4cSsCgOAhYAPbV+\n/fqXL19KNGCtVrtkyRJra2tHR0cxeJbp0UMJWBQHAOiXxMRELy8vCwuLuXPnyvUp7NevX6cv06tZ\ns6ak14wV0XDfvn0ERHooAYviALK0lJSU2bNnFylS1NfPv3PnznJd8uBPREVFdevWTZymSpQoId0y\nw127drm7u2s0mhkzZqSmpso1+FOnTolpr1KlysOHDzm+6KEELIoDyIq0Wm3devVUKlWJqrUDW3a0\nc3KxsraW7qoBf+LMmTO5cuUyMjIaNmxYQkKCRCOPjY3t3bu3QqEoUqTI1atX5Zr2gwcPZs+e3dTU\ndPLkySLBc6DRQwlYFAeQtYhG+PHl4JedDnHx8m7YsKEhbWNSUtK4cePUarWfn9/58+flGvyFCxfy\n5csnEvDAgQPj4uIkGrkY7aBBg8TI8+fPf/HiRY41eigBi+IAspDevXs7e3it++eLjPvttOozQmQR\nw/sAzd27dytVqiTjKSslJWXy5Mmmpqbe3t4iEMs1+KtXrxYpUkShUIhK45oO9FACFsUBZBUDBgyw\nc3Ree+V5RsBq3K2/xtycmdE3Dx8+rFKlijjrtm7dOjw8XKKRp6amzpw508/PLyoqiv1IwALFAWQJ\nwcHB4kBuN2hc+otYc/YF29o7dujQgZnRQzqdbtWqVdl+tXLlSrleZeSTWPRQAhbFAWQtvXv3Fsdy\n9oDcBctWVqtNvLyyv3r1imnRW2/fvm3Tpo3YZZUrV2aZHj2UgEVxANBT6VctatmyZa1ataZOnRoZ\nGZnVZmDFihVXrlyRa8yHDh3y9vY2NTWdNGkSLw7RQwlYFAcA6Je0tLQSJUoolcr+/ftLukwvX758\nFy5ckDTcr169OmvepJyABYoDgIFLSUmZOnWqqampl5fX/v375Rp8xjK9Xr16xcbGyjX4W7duGRkZ\n5cyZ8/Tp0/RQAhYIWAAM0KNHj6pWrSpOay1btnzz5o1EI09NTZ01a5a5ubm7u/vu3buly1glS5YU\n0961a9cstcyQgAWKA0BWkf6OlZ2dna2t7Y4dO+Qa/LNnz2rWrClOy02aNHn9+rVEI9dqtQsWLLC0\ntHR2dt6yZUsWuYkhAQsUB4Cs5d27d+3atZPumu/pAXHDhg2Ojo7W1taLFy8WwUWiwb969apBgwai\nrdStW1eugEgPJWBRHPhFSkrKgQMHxFlY/L7LbACG5/379507dxbn57Jly4aEhMg1+O3btxcoUEBk\nXHooAYuARcCSyZkzZ1xcXf/xK4VC0a1bt7S0NKYFMDwnT57MkSOHsbHxmDFj5Fqmx1uEBCwCFgFL\nMhEREVbW1rmKlJy04eDSE7faDRyrUqkmTpzIzAAGKTExcdSoUSJjZc1levRQAhbFga9kzpw5RsbG\ni45ey7jrXOVGrV1cXbPI74vA51u6dKl0H36/fft26dKls+AyPXooAYviwFfSr18/t+y+GelK/Ok2\nZrrYiVwGGvgrxK8izZo1E4dMo0aNQkNDJRq5VqtduHChlZWVs7Pz5s2bJf2dat68eREREfRQAhYB\nC3onKChIqVTO2n02I2CVDqzn6+vHzAB/PWOJgOLk5CTCiogs0i3TE9FQnLfr1Knz4sULuWb+5cuX\ntra29vb2a9euNYwX3QlYoDgMx4cPH1zd3Nx9/PtND5q4bn+NVp3EHly8eDEzA/wtkZGRXbp0EYdP\n6dKlb9++Ldfgd+zY4ebmZmFhMWfOHLnWuLx586ZFixZi2qtXr/748WN6KAGLgAU9cvfu3WLFiqev\nIrS0spo8eTIfwAI+zalTpwICAoyNjRcsWCDXyGNiYnr27KlQKIoVK3b9+nW5Br9v3z5PT08zM7Np\n06alpqbSQwlYBCzokXv37gUHB8t1X1tADyUlJY0ZM+bkyZMyDv7cuXN58uRRqVRDhgxJSEiQaOQf\nPnzo16+fUqksWLCg+KWRHkrAImABAPRIcnLy+PHjTUxMfH19jxw5ItfgL1++XLlyZbnuGkkPJWBR\nHACQVdy/f79ixYrifN6uXbuscAl1eigBi+IAAHwNOp1u+fLl6cv01qxZw0cz6aEELIoDAPTR4sWL\nZ82aJd0yvZYtW4oTe7Vq1QxgmR49lIBFcQCAoRk6dKhCoShatOi1a9fkGnnGMr2pU6dKehXiyZMn\n37t3jx5KwCJgAYABCg4Ozps3r0qlGjx4cHx8vEQjz1imV6BAgUuXLsk17VFRUb6+viYmJuPHj09O\nTqaHErAIWABgaFJSUiZNmiSavbe396FDh+Qa/OXLlwsWLChiVt++fUXkkmjkCQkJQ4cOFdE2d+7c\n586do4cSsAhYAGCAHjx4ULlyZXHCFF1frpGnpqb+9NNPZmZmnp6ee/fulWvwN27cKF68uEKh6NGj\nR3R0ND2UgEXAAgBDo9PpVq5ceezYMRkH/+TJk+rVq4sTfrNmzcLCwiQaeVpa2ty5cy0sLFxdXfXt\npSwCFigOACAg6tauXWtvb29jYxMUFCTXXa5fvHjRokWLV69e0UMJWAQsAIDeiYiI6NChgzjzly9f\nXt571NBDCVgUBwBA7xw9etTX11etVo8bNy4pKYkJoYcSsCgOANA7S5cu7d27d2xsrERjTkhIGDZs\nmJGRUa5cuc6cOcNOpIcSsCgOANAvixYt0mg07u7uu3fvlmvk6cv0RCPo3r27Hi7T+ytGjx594sQJ\neigBi4AFAAboyZMnNWrUEGfUJk2avH79WqKRZyzTc3Fx2bZtm1w3MUxKSipXrpyY9o4dO75//54e\nSsD6GkSiV/zOxYsXKQ4A+BJENFm/fr2Dg4O1tfWSJUukW6ZXt25d0RHq16//8uVLiUYu5lnMtphz\nR0dHMf9fMyASsLJ0wJoyZcrFj8TFxVEcAPDlvH//vmPHjuLU2rJlS+kC4tatW52dnS0tLefNmyfX\nXa5fv37dtGlTMe01atR4+vQpAYuA9cUD1rZt2ygOAPjKjh8/LuklSaOiorp16yZaQ8mSJW/evCnX\n4Hfv3u3u7q7RaHbs2EHAImB92YAlfh2hOAAAf8uZM2dy5cplZGQ0fPjwhIQEiUYeGxvbv3//r/Mi\nFgErSwcsJycncYRYWVkFBgaePXuW4gAA/BVJSUljx45Vq9V+fn6SvhpHwCJgfRHXrl3r16/frl27\nRK5auXJl7ty5RdL6za3gCVgA8DXpfiXRgENCQtKX6X333XcRERHsQXqoIQes/7o88GM3btz4/f8V\nHR3t4eFRsGDB3xeHOHjqfmT9+vUcNgDwJaxYsUKcZl+8eCHRmLVa7dKlS62trR0cHNatWydXQMxc\noj9+3C7ToycBy3CEhYUt/1ORkZH/9X/s3r27iF8f3xWBV7AA4GvavXu3i4uLhYXF3Llz5VqmJ1pP\ns2bNRMsIDAx88uSJjJM/atSo3yyl/0y8goX/CFjJyckELAD4VqKjo3v06CHOvcWLF/+vbzjosz17\n9nh4eGg0munTp6empko0cq1Wu3379sz9mQQs/CIyMtLNza1w4cIUBwB8c2fPnk1fpjds2DDplun1\n6dNH/LpeqFChK1euZOWdSMDKolq1ajVixIht27adOHFi6dKlAQEBarX6NytBCFgA8K0kJSX9+OOP\n4sxcrlw56T7YdPHixfz58yuVyv79+2fu+24ELAKWvpsyZYr49cLGxkb8huTo6Ni4cePf/6pBwAKA\nb+vevXvf6l7FnyklJUU0GlNTUy8vr/379xOwCFggYAEAMsejR4+qVq2afmugN2/e0EMJWCBgAQAy\ngU6nW716tZ2dna2t7fLly7POdRwIWKA4AEC+1CLXMr23b9+2bdtW9JSKFSvev3+fHkrAImARsABA\n76xfv75gwYKXL1+Wa9iHDx/28fExMTGZMGHCx1cFoocSsAhYAIBvT0SrAgUKKJXKfv36ffjwQaKR\nx8fHDxkyRKVS5c2bNzg4mB5KwCJgAQD0SEpKyrRp08zMzDw9Pfft2yfX4K9du1a0aFGFQtGzZ8+Y\nmBh6KAGLgAUA0COPHj2qVq2aOFc3b948LCxMopGnpaXNmjXL3Nzczc1t586d9FACFgELAKBHdDrd\nmjVr7O3tc+TIodVq5Rr8s2fPatWqJXpNo0aNQkND6aEELAIWAECPvHv37uzZs5IGxE2bNjk5OVlZ\nWS1cuFC6jEgPJWBRHAAAPRUZGdmlSxfRdEqXLn379m16KAGLgAUAQOY4depUQECAsbHxqFGjEhMT\n6aEELAIWAEAf6XQ6uW63LHLV6NGjRcYSSUvkLXooAYuABQDQO9u2bXN1dd2+fbtcw75z506ZMmVE\nD+rcuXNkZCQ9lIBFwAIA6JHnz5/XqVNHnMwbNGjw6tUriUau1WoXLVpkZWXl5OS0adMm6W5iSMAC\nxQEAhkxEk82bN4uYYmlpuWDBArmW6YWGhjZu3Fg0o9q1a4uwSA8lYBGwAAB6JDIysmvXruKsXqpU\nqVu3bsk1+J07d7q5uZmbm8+ePTstLY0eSsAiYAEA9Mjp06dz5szp7OyclJQk18hjYmJ++OEHhUJR\ntGjRa9eu0UMJWAQsAIAeEdHq6tWrkg4+ODg4b968KpVq8ODB8fHx9FACFgELAIBMkJycPGHCBBMT\nEx8fn8OHD9NDCVgELAAAMseDBw8qVaokmlTbtm3fvn1LDyVgEbAAAHpKP5PKH9HpdCtWrLC1tbWz\ns/v555/17ToOBCxQHACAf+3fv1+j0cyYMSM1NVWiYYeHh7dq1Up0q6pVqz569IgeSsAiYAEA9EhM\nTEyvXr0UCkXhwoWlO/OLdOjl5WVqajplypSUlBR6KAGLgAUA0CMXLlzIly+fSqUaOHCgXDcxFKMd\nMGCAUqnMnz//xYsX6aEELAIWAECPpKSkTJo0ydTUNHv27AcPHpRr8FeuXClUqJBCoejTp09sbCw9\nlIBFwAIA6JGHDx9WrlzZ2to6KipKrpGnpqZOnz5do9F4eHjs2bOHHkrAImABAPSITqe7f/++pIN/\n8uRJYGCg6GJNmzZ9/fo1PZSARcACACBzAuK6descHBysra2XLl36le9yTcACxQEAMFgRERHfffed\naGflypULCQmhhxKwCFgAAD2lV1ec+iuOHTvm5+enVqvHjh37dW50TcACxQEA+BvOnDmjVCp79er1\nbZfp/V0JCQnDhw83MjLKmTPn6dOn6aEELAIWAECPpKamzpw5U6PRuLu779q1S67B37x5s0SJEqK7\nde3a9YsukCRggeIAAPxtz549q1mzpmgTjRs3/ibL9D5ZWlra/PnzLS0tnZ2dt2zZ8oVuYkjAAsUB\nAPgUIpps2LDB0dHR2tp68eLFX3mZ3md6+fJlvXr1RJsTf7948YIeSsAiYAEA9Mj79+87depkYWHx\n6tUr6QLitm3bXFxcxODnzp2blpZGDyVgEbAAAHokLCxM0pFHR0f36NFD9LtBgwbRQwlYBCwAADLN\n2bNnnz9/Tg8lYBGwAACghxKwKA4AQNZz+fLlL7RMjx5KwKI4AABZ0a1bt0QfqVOnzpdYpkcPJWBR\nHACALGr79u2urq4WFhZz5szJ3GV69FACFsUBAMi6oqOjv//+e4VCUaxYsevXr9NDCVgELAIWACBz\nnDt3Lk+ePCqVasiQIQkJCfRQAhYBi4AFAMgEycnJ48ePt7GxefToET2UgEXAImABADJNbGwsPZSA\nRcAiYAEAQA8lYFEcAADQQwlYFAcAAEeOHElJSaGHErAIWAAAZI6XL18aGxsXKFDg0qVL9FACFgEL\nAIDMceXKlUKFCimVyr59+3748IEeSsAiYAEAkAlSU1OnT5+u0Wg8PT337t1LDyVgEbAAAMgcT548\nCQwMFA2oWbNmb968oYcSsAhYAABkAp1Ot27dOnd397t379JDCVgELAAAMo3UiwoJWKA4AACghxKw\nKA4AAOihBCyKAwCAzLJt27bo6Gh6KAGLgAUAQOaIioqysbFxcXERMYseSsAiYAEAkDlevHhRr149\n0aEaNGjw8uVLeigBi4AFAEAm0Ol0W7dudXZ2trS0nD9/flpaGj2UgEXAAgAgE0RFRXXr1k20qpIl\nS4aEhNBDCVgELAAAMseZM2fy589/8+ZNeigBi4AFAECm0el09FACFgELAAB6KAGL4gAAAPRQAhbF\nAQDAmjVrnj59Sg8lYBGwAADIHElJSb6+vhqNZvr06ampqfRQAhYBCwCATPDhw4c+ffoolcrChQt/\n5XZGwALFAQAwZJcuXSpQoICIWQMGDIiLi6OHErAIWAAAZIKUlJSpU6eamppmz5799OnT9FACFgEL\nAIDM8ejRoxo1anydvkbAAsUBAAA9lIBFcQAAQA8lYFEcAADQQwlYIGABALKQZcuWZe4CQwIWKA4A\nQJam1WpnzpxJDyVgEbAAAKCHErAoDgAA6KEELBCwAACghxKwKA4AAOihBCyKAwAAeigBi+IAAAD0\nUAIWxQEAAD2UgEVxAABADyVgURwAAIAeSsCiOAAAoIcSsCgOAADooQQsigMAAHooAQsUBwAA9FAC\nFsUBAAA9lIBFcUhn/fr1lAHzyZQypWA+6aEELIojM9WtW5cyYD6ZUqYUzCc9lIBFcXBqYD6ZUjCl\nzCc9lIBFcXBqYD7BlDKlzCc9lIBFcXBqAPPJlDKlYD7poQSsT3Tu3DlRHGvXrv1nVlWuXLl/gvlk\nSplSMJ9/n+ieooeePXuWgIXfSi8OAADwaUQnJWDht969eycqQ6RvfgkDAOBvOXfunOihopMSsAAA\nAEDAAgAAIGABAAAQsAAAAEDAwpcUFBSkUCgsLCyYis9x9OjRdu3a+fv7azQaNze3+vXrc0/xv+XD\nhw99+vRxdXU1NTUtWLDgxo0bmRMKkjMnCFiQ1atXr6ytrcX519LSktn4HE2bNq1YseLChQtPnTq1\ndevWUqVKGRsbHz9+nJn5i6pVq2Zra7t06dKTJ0926dJFtC5uUUxBcuYEAQuyqlOnToMGDTp06MDv\nYZ8pPDz843/GxcU5OztXrVqVmfkr9u3bJxLVx69aVa9eXXQvrVbL5FCQnDlBwIJk1qxZI34JCw0N\nbd++PaeJTFepUqWcOXMyD39F586draysPo5TGzZsEJHr/PnzTA4FyZkTBCzI5M2bN3Z2dosWLRJf\nc5rIdNHR0eIU3LhxY6biryhZsmSJEiU+fuT27dsiYAUFBTE5FCRnThCwIBNxqi1Xrlz615wmMl3r\n1q3VavXVq1eZir/C39+/Zs2aHz/y+vVrEbCmTJnC5FCQnDlBwMK3d+LECcWfunHjhvi2LVu2mJiY\n3Lt3j9NEZk3px0aOHCkeX7BgAbNHwNIHFGRm4cxJwELWFRYWtvxPRUVFffjwwcnJadCgQVH/p2XL\nluI0ER0dHRcXxxz+3SmNjIz8+PvHjh0rmtnkyZOZur+uZMmSxYsX//gR3iLMLBRkZuHMScAC/oen\nT5/+0YsxDRs2ZH4+v5n9+OOPTMXf0rVrV0tLy99/yD04OJjJoSA5c4KABTkkJSWdPHny1P8RX9eo\nUcPMzEx8fefOHebnk4k2Jk61o0ePZir+rgMHDoip27RpU8YjgYGB7u7uOp2OyaEgOXOCgAVZ8UmC\nzzd9+nTRzGrWrHnhwoXgjzAzf1H16tWzZcsWFBR0/PhxLjRKQXLmBAELhqBDhw5cj/gzVaxYUalU\n/uaNA/EIM/MXxcXF9enTx8XFxcTEpGDBgh+/mgUKkjMnCFgAAAAELAAAABCwAAAACFgAAAAELAAA\nABCwAAAACFgAAAAELAAAAAIWAAAACFgAAAAELAAAAAIWAAAACFgAAAAELAAAAAIWAAAACFgAAAAE\nLAAAAAIWAAAAAQsAAAAELAAAAAIWAAAAAQsAAAAELAAAAAIWAAAAAQsAAAAELAAAAAIWAAAAAQsA\nAAAELAAAAAIWAAAAAQsAAICABQAAAAIWAAAAAQsAAICABQAAAAIWAAAAAQsAAICABQAAAAIWAAAA\nAQsAAICABQAAAAIWAAAAAQsAAEAO/w+aplmPUuc/JQAAAABJRU5ErkJggg==\n",
       "prompt_number": 1,
       "text": [
        "<IPython.core.display.Image at 0x10481c350>"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "import matplotlib.pyplot as plt\n",
      "from sklearn import svm, datasets\n",
      "%matplotlib inline\n",
      "\n",
      "# Training data\n",
      "iris = datasets.load_iris()\n",
      "X = iris.data[:, :2]  # we only take the first two features\n",
      "y = iris.target"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Linear and Polynomial kernels.\n",
      "\n",
      "For example, the polynomial and RBF are especially useful when the data-points are not linearly separable.\n",
      "\n",
      "http://scikit-learn.org/stable/auto_examples/svm/plot_svm_kernels.html"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# We create an instance of SVM and fit out data. We do not scale our data since we want to plot the support vectors\n",
      "C = 1.0  # SVM regularization parameter\n",
      "clf_1 = svm.SVC(kernel='linear', C=C).fit(X, y)\n",
      "clf_2 = svm.SVC(kernel='poly', C=C).fit(X, y)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for i,clf in enumerate([clf_1,clf_2]):\n",
      "    \n",
      "    # Step size in the mesh\n",
      "    h = .02  \n",
      "    \n",
      "    # create a mesh to plot in (test data)\n",
      "    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n",
      "    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n",
      "    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n",
      "                         np.arange(y_min, y_max, h))\n",
      "    \n",
      "    # Class prediction\n",
      "    plt.subplot(1,2,i+1)\n",
      "    Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])\n",
      "    Z = Z.reshape(xx.shape)\n",
      "    plt.contourf(xx, yy, Z, cmap=plt.cm.Paired, alpha=0.8)\n",
      "    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Paired)\n",
      "    plt.xlabel('Sepal length')\n",
      "    plt.ylabel('Sepal width')\n",
      "    plt.xlim(xx.min(), xx.max())\n",
      "    plt.ylim(yy.min(), yy.max())\n",
      "    plt.xticks(())\n",
      "    plt.yticks(())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAD8CAYAAABTjp5OAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd0VNUWh7/pM+m9F5KQQIDQDb036SKgiIqigoDYQAUb\nT0HBroBPAfGJnSLSe+9Nei+hpZBeJpPpM/f9MSa0kFASQpL7rZW1CLn3nDOTPTv77nP2b0sEQRAQ\nEREREXngkVb0AkREREREbg/RYYuIiIhUEkSHLSIiIlJJEB22iIiISCVBdNgiIiIilQTRYYuIiIhU\nFoRyIii2iQCIX+JXuXy1a9euvExXtG3xq0K/SrJtiSCUzzlsiUTCy3MPlsfQIveB5+TzAAiM8q3g\nlRRPQKOxlJPplopo21WLEXNG4+cZDkBS/7AKt/mSbFtMiYjcRIPa24AH11mLiJQVI+aMRoIEgFmm\nlhW8mtIRHbbITTQ5l1LRSxARKXcKAxNfzzAAegxOeuCDFHlFL0DkweRBN1wRkXshSphBi/HHkEsU\ngCMV8m+g/UAjOmyR64gSZgCeFb0MEZFypcvPx4ry1oUERj74QYqYEhG5jnYKz0oRaYiI3C2OoOQq\nlSW6BtFhi1zDiDmjgcoRaYiI3C3XRtduXW1A5bF50WGLFGF+oQ1IKkmoISJyFxQGJYWccI2oVDYv\nOmwR4Nro2qeCVyIiUr4URtfmziFA5bJ50WGLABDe36+ilyAiUq7cGF2nu0srVXQNosMW+ZeznrWQ\nyBUVvQwRkXKh0FlX5ugaRIctAozT/QFAQLhHBa9ERKT8uPYYX7q7tFIGKKLDFiHDQyYWyohUWa4t\nP4erJeiVMUARHXY1J2D+jopegohIuXFj+TlUjhL0WyE67GpOZTZeEZHSaDF+XlH5OVQOgaeSEB22\niIhIlaQwuvbyCCr6v8oeoIgOuxpTqHktIlIVuTG6rkwl6LdCdNjVnMocbYiI3IpCvZBro2uoPCXo\nt0J02NUUMboWqapECTOKVeOr7NE1iA67WiJ2lBGpyhTnrJP6h93i6sqF6LCrIWJHGZGqyo3l51D5\nFPlKQnTY1RQxuhapqtwYXVc2Rb6SEB12NeNG8XYRkapCydF15dIMuRWiw65miB1lRKoyVTm6BtFh\nVyvEjjIiVZUb9UKg8irylYTosKsRYkcZkapIcXohUHkV+UpCdNjVhCn5E4GqFW2IiMDNFY1wNbqu\njIp8JSE67GrC1FWdqly0ISJSmOa7saKxKkbXIDrsakOPwUkVvQQRkTKl8MTTjRuNlVnvujREh10N\nEDvKiFRFii0/p/Ir8pWE6LCrAWJHGZGqRnWtJxAddhVH7CgjUhW5VXRd2RsUlIbosKs4VfnxUKR6\nUlxFYyFV3d5Fh12Fqa6PjSJVn+Ki66rQoKA0RIddhWmn8KzoJYiIlCklRddQ9at4RYddxanKj4ci\n1ZPqGl2D6LCrLGJHGZGqRnF6IVC19K5LQ3TYVRCxo4xIVeVGvRCATHt4tdHIkVf0AkTKHrGjjEhV\nokHtbbQYP6/Y6Br+LUOXVQ9XVj1eZTVEjK5FqgqF4k436oXA1XRIdaniFVMiVQzxKJ9IVaIwvVec\ns4aq16CgNESHXcUQO8qIVCWKk04tpCo2KCgN0WFXIcSOMiJVicKnxVtF11VVQrUkRIddhRA7yohU\nJW6lF3It1SV3XYjosKsIYkcZkapEaXsx5s4h1TL1JzrscsJqNpKdlIBBm3Nf5jvrWavaPR6KVAz6\n3CyykxKwWczlNkdp0bXjKF/1s3fxWF85kJZwnFVfjUHj5Iw2O5P4gSNo8PDgcp+3uj0eitx/9v41\nk8OrfsfV0xuL2ULPcdPwCo4s0zlK0wuZZWpJD5Kqpb2LEXYZIwgCa6a+xfPjJ/HNkq18vmA9h5f/\nQsaFU+U2Z2FHGRGR8iTx2F4Sdq3km6XbmLp0G/2HjWbDd++Vy1wlRddVXUK1JESHXcZYDAXotTnE\nd+oBgHdAELUaxZOVeK7c5hQ7yojcD7Iun6Nhy/a4eXoD0KbXANIvnkUQhDKb41Z6IYUUHuWrrogO\nu4xRaJxRqp04tmc7ANqcbM4e2Y9H4M0aCGWB2FFG5H7hERjK8X92YSjQAXBg63q8gsKQlPHJpOL0\nQgqZsyKsWp+EEnPYZYxEIqHry1OYOv4l/ENrkJ58iTod+xMQXb9c5qvOj4ci95fwhq1JPLyT1x9p\nh3dgMBnJifR445syG7+06Br+tfdqXGcgOuxyIKTuQzzx+V9kJ5/H2cMHj8CSz5LeLVPyJ3LWs1a5\njC0iciMSiYQ2z46jbpeBGLS5+IRFo3J2LZOxC0vQS4quq4vmdUmIDruc0Lh5EuzWpFznOOtZq1o/\nHopUDF7BkRBctmOWVIJ+LdU5ugYxh13pEQtlRCo7pQk8iVxFdNiVFLGjjEhVoFDrurToWkyHOBAd\ndiVE7CgjUlUobExQUnRdnVqAlYaYw75HTmxewu650zHpdUQ2bkuH4RNQOrmU65xiRxmR8sZs1LN5\n1kQS/tmMysmFZo+Nom7HR8t0jkK9kJI2GqH6aV6XhBhh3wPJJ/7hn7++5/0ZfzJzwyF8vdzYOueT\n+zK3GF2LlCfb53yKp6uaGesPMmHWXA4unk3isb1lOsftqPEVIu7VOBAd9j2QeHwfHfo+TlhMLBpn\nFwaNfoukMjbqGxE7yojcDxKP7WXQ6LdwcnEltGZtOvZ7okxtuzS9kEIK0yEiDkSHfQ9oXD24fO5U\nUWluYsJpNK7lK0gjdpQRuR9oXD1IPHcacOjjXD53GnUZ2/btRNdiOuR6xBz2PVCn/SMs3racyaOe\nwi8kjL3rV9B51MflNt+IOaMxv9BG3HwRKXdaDH6NGR+MJb5jDzJTk0lPTeWRp98pk7HvNLoW0yFX\nER12CQiCwNE1czm2fgEIArEd+tGw59NF2gkKtYZ+//mRs7vWYtbreGTC7DKXmrwWsaOMSFlhyM9l\n209TSD17FBdvf1o/8xZ+EbFFPw+Na8YjE34k8ehu/ANiaNWsCwq1pszmv53oOtMeLj5N3oDosEvg\n9LYVnNr4F69/Mh2pVMa3772KQu1Mvc79i66RK9XEtutT7mspLEMXow2RsmDt1LeIqVOHEeMncPrg\nXn777GUemzIXZ4+r9uUVHIFXcESZzns7eiGFpLtLET329Yg57BK4dGAzA0eOIbJOA2rUrscTL4/j\n0oHNFbIWsaOMSFlh1utIPXecZ9+aiH9IOG17D6RmXGOunDp4X+Yv7RgfVM+O6LeD6LBLQK5yIiv1\n6pnnzNQUFCqnCltPdeywIVL2yBRK7HYb2uxMAOx2OznpqSjU5WvbdxJdA2L6rxjElEgJNOz1DEs+\nGk5uZgYyuYyNi+fRZ/x3930d43R/kOEhu+/zilRNZAolTfsO5cNhj9Gu56OcOvQPglxNSL34cpvz\ndtT4rsXRs1F0TzciviMl4B0axaMf/sTpbSsRsNNvwo93nNPLS09m959T0WWl4hdZl2aDXkZ5h5GM\n2FFGpKx5qP+LeIXFkHDuCB4x8bTu9CiyO0y5ndm5mhMb/gKgTqcBxLR8+JbX3q4aH1xNh4hPlDcj\nOuxS8AgIo9nAEXd1r6kgnyUfDafbwKeo07QFq+fOYd308fR8c9ptjxEwfweUf/9ekWpI1EMdiHqo\nw13de27PevbOm84L70wGYPbkd5DK5NRs1vmma+9GjU/cryke0WGXI8kn9xMUHskjz78MQGSdBgzr\nEIdRp0Xt4nZbY4gdZUQeRM7tWMWTr75NozadAHjy1bdZs3hRsQ77TqJr+DcdUmYrrVqIm47liEyu\nwKgvKKqENJuM2G02pPLb+zspSqiKPKhIZXIMBQVF3xsKCoq160IphduNrmeZWgJiOuRWiBF2ORJc\npyl7F3zH9xPGUKdJMzYunkds2953lMMWo2uRB5F63QYx9+uxGA0Op734x295+PUvb7ruTgSeChHT\nIbdGdNh3gS47gzVT38CozcHJw5fuY79E7XJzRCBXquj77kwOLv+Z7Vu2EtykM3FdBlbAikVEbo8T\nm5dwePnPCHY74U070mrwK8VeF1S7ET3emMqRLUsA6PHmNAJq1rvumrsRKusxOImAcDFIuRWiw75D\nrGYj898eRIMWbYjv9BLbli9k3rjHeXr6KqTSmzNMSicXmj320h3PI6ZDRO43p7avZPucTxn0yjs4\nubjy29eTsJoNtHt2XLHXB0THERAdd8vx7ia6FikZMYd9h5zbswGVWsWoj6YR36kHr30+E5vFTNKx\nPWU2h9hRRqQiOLziV/oNe5Vug56lTa/+jJz4NRf3bbyrsW5X4OlaCvPXIrdGdNh3iGC3I5XKigSg\nJBIJEqkEwS6U2RxiRxmRikFAKrtaoCWVSkG4e7u+0+haPBFVOmJK5A6JataZXb9/zQ8T3yK+cw+2\nLf8LiURGaBlViTnyfp6i4Yrcd+p1GcTCmZ/j7OaBs6sbcz59n5AGre94nLuJrgFR5+k2EB12MaQl\nnGD11LcQrBa8wqLpM/7bop8p1Rr6T/qVNVPf4NCuLajdvBk45c/rjjQJgsDpbStIO3sEJw8fGvR4\nEqXG+bbm7vLzMYeM6gNKanoes//YijbfQNf2cXRuE4sgCCxYvp+9BxIIDPDgxafa4eKsruilihTD\ntl++IGH3WpBIiR84kjrt+xb9rG7HR7CajSycNQ1BsBPWpAPtho6/7n5Dfi5HVv2BMT+X4HrxxZ67\nhjuPrpP6317JenmyaecpVm04gpurhucHtyXQz53s3AJm/b6FzMx82reKpVfn+hW6Rokg3MMzT0kD\nSyS8PPf+qH+VJTkpl1jw7mDiO3Unsk4DVvw6E0GqZPCXf9/2GLvmTif12C46PTqYM0f2c+HMafp9\n8D/kytKd2HPyeSB5MDtEZ2Tl0+XxL3jIV4mfRs7y81rGv9aHhIvprFq5j05hLpzJNZMhUbL819fR\nqMvveFZAo7GUk+mWSmW17XX/fZ+L+zfRb9hrFGhzWfXHj7R55i3qdux3W/eb9Pn89d4Q6se3IDwm\nllV//kRMu0do1GtI0TWFAk+3qxlSSFL/sAp9qpy/7B8++mIRvaPcyDTa2J1mZOGPo3nqpVnUcoYw\nNwWrL+gY+nQHRj3bsVzXUpJtixH2DWz8YRI16zVi5MRvAGjctjNjHmmHzWZDJitdgMlmtXBoxW98\nu3ovbp7edH38GT54rj+Xj+wismnJZcAPekeZ+cv/ob6Xghca+QFQy0fD1zNWk5qlY3bvSNzVcnoI\nAu9vu8KWXad5uEO9UkYUuZ8kHdnBiA+/Jr5TdwBkcjnrF864bYedsGcD4dExDJ/wGQANWrbn3ad7\nFznsOxV4epD4ZsZqxsT7EevrqJEw7k9n8vQVBKlgVFN/AJoEujBu9vpyd9glIW463oDNYsLN06vo\nexd3TwS7DWy31wzUbrMC4OTiCjiiMRd3T6xmU6n3PugdZYwmCy6KqybjppJhNP/7ehWOP2YSiQRX\nlQyjyVIhaxS5NQICrh5X6wVcPbzgDh5SrBYzru6eRd+7eDjsujAavNMS9ELcutoqPH9tslhxVV0N\nyFwVEozG6+3dVSXDZLZW2JMdiA77Jhr1fob9W9axafFcLpw8yrTxo9C4eSBTKm/rfoVKQ3jDlnz/\nn7FcOHmUtfN+5uzRA4TUeajE+6bkTwQebMH27u3rsfGyju2XtZzNMvDdwUz6PdyYtvHRfLs/nYRs\nIyvP5nAmy0Srh2pW9HJFbsDJ058fJo3j9KF9HNy+kb9mfElIw9s/ShfesBX/bFnL1mULuHDyKN+9\n/zoxLbshkUjuSuCpkBOuZdvV5m54pHtjvj+YyZksAzsua1l/Ucezj7diX4qOjefzSMg2Mu2fdPp0\naVB0QqwiEHPYxbB7wfecWDcPwS4gVSgZ+PFvuHj5k3L6EEnH9qJ2cUOu0pCfmYp3aBRR8Z2u+yWa\njXp2/f4NqWcP4+zpS4vBr+MdGlXinM/J5yGRKx54DYWd/yQwZepS8vONdOsYx5ujumMyW/nP54vY\ne+A8gX7ufDi+P7WjAsp1HWIO+86x2Wz8+eYAzPk5AHhF1OWRd/6LzWrh1Nbl6LLT8QgMoyAnA5vF\nQlR8h5t6lKYlHGfP3GmOTce68TR7fDRypYoRc0Yjlyju2GG7dbVxwjWiwk9FWa02vpq5hpXrDuPi\nombcyz1p0yyGg8cvM/HzxWTn6mjbvDbvvt4btap8S+dLsm3RYd+AzWJm8UfD8fHxpkatOmxd9heN\n+w1HrlSx+8+ptOszkMunT3Du+EHaPzKIQ9s341erMa2HvHlP8z4nn1fhRluZEB32nXPlzGFWffk6\nbXr2R6/TcmjnVvp98D+2/vgxKqlArQZN2LJ0Ht4BQUTXb8q2FQvpPvZrAmMalDhulDDjrqsazZ1D\nSPeQPdBPlvebe9p0NBqNLFy4kIsXL2K1OvKVEomECRMmlO0qHxAS9m7A1dmJcdPmIJFIaN2jH/95\nth9ylYZxU38isk59BEHgs5eHEBweRb/nXuaVXq1o2HMILt7+dzWn2FGmYqhutv3PwpkMGfsf2vRy\nNJH+7atJ7Pj1Syy6HCb9sRKpTEaXx57m9b5t+M+PC4moXY9VC2fS++2SuyzdSwm6o9GuyO1SqsPu\n27cvHh4eNGnSBLW66p+tNRXk4x8SVpTi8AsOw2w0YLNa8Atx7H5LJBL8Q2ug1+Xj5OqGi4cnJr3u\nrh222FGmYqhutm3W5+MfetWx+oeGc/TQQXwDQ4oqHD18/JFKZZiNRvxDwzHr80sc824Enm5EjK5v\nn1IddnJyMmvWrLkfa3kgCKkXz6IPZ3BoxybCY2KZ/92X1GjYEplSxZxP/8OTr71N0vmz7Fi1iNc+\nn8nyX2ZitdnxCAi9q/nEjjIVR3Wz7dAGrflj6hRGTvwKfb6WZT/PoH6voeyZ9y171q8gpkFTlv38\nPYHhkeTn5fD71MmENSy50vFeomu3rrd38krkKqU67JYtW3LkyBHq16/YCp+7JePiadLOHcPZ05ca\njVojKUZRr5DUs0fIvHSG+j2f5qfPJ6LPyyakblM6jZyERCJh609TeOvxh1E5OaN28+brN0fiE1aT\nXm9NR6a4vVMkDWpvQ3fyOAkSR9uxitRP2H/kEsfPpFAjxJs2zaIrdPe7IqjMtm23WbmwfysGbQ5B\nsY1u2hy8FqvZxIX9W9B4eFPgHcKEoY7+jQ17PUPdjv3wColk7oxP0WWl4ertj16bw4ShjxLT8mGa\nPPJ8ub2GTHt4uRxjNZosrNlyHJ3OSOv4aMJDvMt8jorilpuOcXEO2USbzcbZs2eJiIhApVI5bpJI\nOHLkSMkDPwAbM6e2Lmf3n1Np2LojF04exdk/nC6jJxfrmA6u+JVjq/+gfot2nD60j8C6zWjzzFtl\nvqZCnYUZz35bJKFaEQ77+5838v3/NtAw0JlTmQY6d2zA5HcG3Pd13C33sulY2W3bbrOy8ovXwFRA\ncGQ0B7dvoMOwCUQ0aXfTtRaTgSUfDcfN1QXvgCAO79jEw69/SVDtRve8jkJbvtsI29w5hAxvVZme\njNIbzPQbOg0KCvDRyNl/pYCfp71As0a3/oP2oHFXm47Lli0DHMZ5482VIRKz221s/ekTJv26lJDI\nGCxmE+Of6E7i0T2E1W9+3bVGnZZ9C2fyxcJNePsHotflM/bRDsR26IdPWHSZram4fF9FOOtcrZ4v\nZqxhWrdwfJwU6C02XllzkKcHtiI2OvC+r+d+U9lt+9yeDUhtRj78eTFSmYzTh/bx9VujinXYxzcs\nIjAoiLFfzUYikbB3w0rmzviCgR//XiZruRe96/Lo3fj7ot04mQyMax2IRCJhZ6KWdz/+i/V/lX3w\nVRHcMj9Qo0YNatSowXvvvVf072v/70HHajJit9sIjnA4XIVSRUhULfR5WTdda9Tl4uzmibe/w1k5\nubjiHxpR7LX3wrX5vsJCg4ogJ1ePm1qBj5PjPKmTQkawu4qMrJI3mKoKld22DXlZ1IipU7RRGFG7\nHgV5WcVGZYa8LCJq1yv6Q1Sjdj30edn3vIa7VuS7gbKuO0jPzKeGq7zo9UZ5qsnI0ZXpHBVJqWdq\njh07dt33VquV/fv3l9uCygqlxhmv4AiW/PRf7DYbZw7v58S+HTe1MQJw9QnEbhfYtHgudrudwzs2\nkXz+DD5hMWW2nmujawmSCtW8Dg70QKpQsD4hF7sgcCBFx6VcE7ExVT+6vpbKatuBtRqyd8MqLp85\nic1qZf73XxIS26TYp4OgOk3YtGQ+aYkXsZhNLJz1DcGxjctkHfcSXZfXhmPLplFsSizgSr4Zi83O\nglM5tGhSctFaZeKWOezJkyczZcoUDAYDGo2m6P8VCgXDhw/nk08+KXngByCHrU1PYe308aQlnEDj\n5kGHYe8X+9gIkJWYwIrPXiEvIwWNqzux7R9BJlcQ1rA1VpMBBIHA2g1RqDTX3SfY7aScPoTZUEBA\nzbpo3LyKHX/EnNHXGXhFq5OdSkhl2Jj/kZCYjb+3C9998nSlMux7yWFXBds+vX0l2+Z8ismgJyS2\nEZ1HT8bZo/jjcQeX/cKued9it9kIiI4jKLYJahd3whu1Ji/1Mi7e/vhFxN50n0GbQ+q5Yyg1zgTW\naoBU6ojo71aR71pmmVrS48nkcjnSN+v3LXwyfSVmq5328TX57tMhuLlqSr/xAeGeKh3Hjx9fqgEX\nO/ADYNSF2KwWZKV0Yt40+yPO7VxNRGwcl86cwG63ExoVw+Wzp/APi0ShVJGXk0Pf92bh5OHYdbbb\nrKz+5k0K0pPw8g/k8tmT9HprOr4Rta8bu7jNmYp22IVYLDYUispXtFMWlY6V3bYFQcBus5Zo22aj\ngblvPopSocDDx49LZ07g4eOHxWxCr80jumE8SefPENWsCy2ffL3ovsxLZ1j+6WhComqRk5GGk1cA\nD4/5EplccVPwcTfMMrWk55C0cpNiEAQBm82OXF61bPuWm44HDhwAYODAgUX/vpbGjcvmsep+UJqz\n1mWlcXrrMib/sZrgyGjysjIY068drp4+tO75KM+97ThZ8utXk9i74DvaD3sfcJxCkduMfDZ/LXKF\ngq3LFrD0pyn0n/jzTXPcaOAr/wgptwjjTqiMzvpeqSq2LZFISg9EfpiEf3Ao78+aj0wuZ9OiP5n/\n38+wWi2898N8ouo2pCA/j3GPP0xkfCcCoh1HHLfN+YRBo9+iwyODsFmtTB71FCc3L+HbpM1IymCr\nsMfgJKD8NDkkEkmldNalcUuHPWbMGCQSCQaDgf379xedVT1y5AhNmzZl165d922R5U128nk0Lm4E\nRzo2KN29fQkMjyT10nna9upflBusF9+KU/+bWXSfNiOFOk2aIVc4DK9ufGt++2bydWPfanPm2Z6X\nSRfFEiuE6mTb2oxkWnfqguzfjkj1mrXm1y8/RCKVElW3IQDOru5ExNZDm3GlyGFr01OoF98KcOhm\n132oOXrTXqDs9K4fdKGzB5FbeozNmzezadMmgoKCOHDgAPv372f//v0cPHiQoKA7l1B8kPGLrItR\nX8DBbRsAOH/iMEkJZ4ht0owNC3/HbDJitZjZ8Pcf+NSofc19ddi5djnaHMcO/br5v+AXWefm8Yt5\nfFSuT7ojLWKRsqM62XZgTAO2Ll2ANicbQRBYO28OLu6euLi5s33lIgCSL5zj9MF9+IRfPcLqF1mH\ntfN/QRAEtDnZ7Fi9lC4HLt+V3vWNmDuH3PMY1ZVSc9h16tThxIkTpf7fTQM/IHk+s15HzpVLOHv4\nFGl92O12Lh3ajsVQQI0m7VCqnTi6bgE7f/8auVyO2WTCarWgVKpAIsUu2JFKpYTWi6fry58gV6rI\nSkwgLeEYmRdOcmLLUpQqNa4+gXQf+zUuXo6OLCVtzhhtNrY11RAeE0BkmE9RFJ+Ykk1mjo6oMF+S\n03IR7AK1ogKQycRo/FrKIodd2W1bm56CIT8Hz+AIlGpHp5SC3EwuH96Fu18IQbGNsNvtLJ74PKnn\njqNQKgEJFrMJQbAjVyiRK5RYLWbaPvc2sW17Oz4bB7aiy8nk1JbF5GekYDYZ6TK4LZ8l5+PvVaPU\ndaUbDCTpC6jh4orXvwVJZpuN01otSqkU22M1SczQUbd+MIF+7uX4DlVO7kmtr379+rzwwgs89dRT\nCILAH3/8QYMGJcstPiiknD7E6q/fwNM3gOzUZBr2epq4bk8wb9wArEYDKo0T2+Z8Qp/3fqAgNwu7\nzYba3ROL2YzayRkv/2CyU5PQOLmjUKkw5udis5rZ9svnnNm2Ai//ILLTUojt+ChN+zyLk4fPTUer\ninPWl3U6HtuyEck6OflWE21a1mLqx0/yyfQV/LJgJ76uSlJyDDipFKgVUvwCvJg7Y0Sl2umuDFRW\n2xYEgV1/TuXUlqV4+gWSl51Fz7emkpNyic2zPsTDx4+8rAy8QqPp9/4PmAq0KFQqVCoNBr0OL/8g\nzCYDZqMBT79ActKvYNTmYDbqmfvmAOwWE0qVGn2Bjodf+wK/yFhemT8eubR0+YU/zifwn0MHCHd1\n4nK+nqnxzWni7cOAzRvQ280UWGwY1q8m0ENDus7Mqy905qWhne7Du1Y1KDXCNhgMfP/992zb5ij0\naNu2LSNHjixV3ayioxBBEPh5dHdGTPiURm06kZuZzjtP9sTJJxg3jYJ3vv8dmVzBn1Mns2PNMnQ5\nmbw3az5mo4EfPhrHR78uw8Xdk52rF/P3rG/47K+NzJr4JqlZeVzcv5WPf1tBcGQ0F04e5YOh/Rj8\n5d+4+V59nC4pun5iy0Za1nBlbHwkBquNvksP0rh9PZYu38en7YNxVcnYelHL3GOZfNszgu/2ZxDa\nsFalKh0vb8oiwq6stn3p8E72/P4lk35ZgoubB1uX/8XfP36HLjudIW9+QNteA9Bpc3n78W7InD1Q\nywQ+nLOE2R+Nw83Lh6fGvI/NauWL14ZSN741rXs8wjtP9sLZJwhPNyfGTf8VmVzOb19NZO+WDbz/\nedfbEnlKKiig05pVrBsUT01PZ/an5vHo3/tpE+CP3MnAMw19sAnw0ZYk6gc40S7cjbc2JTN31ijq\n1Qq+T++4lk6KAAAgAElEQVTeg09Jtl3qc7ZGo2HMmDEsWrSIRYsW8frrr1cKKUqLoQBjgZZGbRx/\nvT18/KjVKB5DThotuvVGrlAikUho0a0PxgKHTGp0/cYkXzhLXLM2uPzbuy6+U0+uXL4AQPMuvci+\nfA7/kPCiDcqI2DjcvHxISzheNHdpzUjPaLX0i3F0ZNHIZTwc6sWxk8nU99MU9ZVrGebKFZ0ZgBZB\nTpw+d6Ws36JqT2W17Zzki8Q1b4OLm2PTrnmXXqRfTsCgy6d5l14AuLh50KBVB7TpyTTr0hOVRkPy\nhXO06NbbcYJCoSC+Uw9SLpzF0zeAmnGN0Wen0bJbX+QKRdFnw6TT3rYi30WdjtreLtT0dAagSYA7\nPholp/JyaRnm4phXKqFlqCtJeWa8nRTE+jpx7kJ6+b1ZVYxbOuyBAwcCUK9ePeLi4q77qgzqZgqN\nM2pnt6KNxNzMdE4f3IuTVwC71izDajEjCAI71yxB7eyKPl/LmcP7CY6I5ujureTnOtoo7d2wgsBw\nh3DM7nXL8QqrSVrSJZLOnwHg/IkjaLMz8Y+qWzR3ac1Io93cWHQmFQCD1cbqxGzqxQZzJE2P1uSo\nANt5OZ8gV8cj6K4UPbVqVq8qxPKkstu2V3AER3dvQ6fNBWD32mX4hUWhcXFl97rlAOi0uRzesQk3\nv2D2rFuByWAgJDKaXauXIggCVouZvRtWEhQRTXb6Fc4dPYCTlz871yy5+tlYvQSVi9ttr6uGiwun\nsnScyykAYH9qHllGM7HuHuy4nI8gCFhsAjsT8wlxV5Klt3AyQ0/NCL+yf5OqKLdMiaSkpBAUFMTF\nixeLvbFGjRolD/wAbMxcOX2I1d+8gYePH9mpV2jUewj1ug5i3riBWI16VGoN+gIdfd//gYS9Gzm0\nbA4ePn7kZmUgk8uLctgqJxdHDtDVm15vTWPHn9M4vXUZ3v6BZKVdIa7rIFo9+SrgiK5bjJ9XYkSS\nWFDAwM0bUMgkZJpsNG9cg1lThzJlmiOH7eOi5EquAY1ShlIuIyjIm7kzR+LqUnz0Z7fbOX0+DbVK\nQURo9RCDv5eUSFWw7Z1/TOXUliV4+AaQn5tDjze/ceSwZ36Iu48v2uxMvMNieOS9Wcx7exAF2emo\nNU7oC/Jx8/LBbDJhNurxCggmO+0KTfs+R53OA5j75kBsFgNKlRqjXs/L73dlyMaU2y6U+fN8AhMO\nHSDMVUNSvpGp8c1o4uNL/03rKbCZ0VttGGx2fN1UZBVYeH14V0Y92/GW4+kNZk6eSyE6IgC3W9h/\nVeOeKh1nz55Nu3btiI6+M9W6B8GooaRTIjuwGPXUaNwGpdqJK6cPserrsUiQYDbqsVmtqJycUTq5\n0nrIG7j6BOIVEllUnpuVmED6+eMExDTAM/CqMd9uM1KjzcaZvDz820sx14wjqKYjykhMySbpSjaj\nx/9KWrYOCeDr6czGv8fj4eZ00zipGXl0Gfg5Wp0RuwBhQZ5sWjgOpbLU/eRKTVnksCu7bWszUjDm\n5+IZFIFC7diQ1udlc/nILtz8ggmq1RBBENg9dzqHV/+JQqlCIpVhNZuw221ENetMXNfHcfHyKzrZ\nVHSCymSkRqPWvDL3rTuuakw3GEjW6wl3cbnulMgZrZZtqal8fOQgEokEi13gib7xfPGfx4sd58e5\n25n4xWLkUglmm8BLQzsyfnSPe3jHKgf3lMO+fPkyL774IhEREQwcOJDp06dz6NChMl9keaF0csE/\nqu517bukUikRjdsQ07IbSrUTVrOJ1d+8yagPv2LWxkNMmP0XGmcXPpu/lt5PPc8/C2fiExZd5KwB\nvEOjiG3X5zpnXSjwdDudo9UyGfW9vPA/4nHdyZLQIC++mrEGH5mdPwfE8OeAWgSrJDz90qxixxk8\nYga13eX8OSCG3/pHIzPoeXHcL3f8PlVHKrttu/kG4RdZp8hZAzi5e1G7TU+CajmKYhL2biDl6A6+\nXbWXHzYfpV3vAdSLb8nMDYew5qWSfv5EkbOGws9GW2JadOWVuXcnSeqn0dDI27vIWQMoZTLquLsz\n5fhhRjULZN5jtZjWI4KFy/9h7dabj1Fm5ej48IvFjG8TzB8DYpjcOYzv52zkxJmKE017ECjVYU+c\nOJGNGzdy4sQJWrduzWeffUaTJk3ux9ruG7qsVFRqddEGZWSd+oRFx5J8/ixdBg4h7cIpBLu91HHu\npV1S6qXcon+fv5BO92hPlDIpCpmEh6M9SEzKLPa+K2m59IzxRCaVoJZL6VbTg9Nnku9qDdWN6mDb\n6edP0OrhPrh6eCKRSOg26FkunDqG2smZdr0GkHXxZIn336tmyLVcKijAZhdoX8Nx9jrETUVdPye2\n7j5907X/HL6Iq1JG40AXAGK8NQS5Kdm272yZracyUqrDnjRpEt27d6dr166cO3eOL7/8ksTExPux\ntvuGk4cPBdo8ks87jCEvK4Ok86fxDgjm+L4duPsGlthaDO6tGalfnh3Bain63tvLhYNXCooeiw6n\nFuDu7lzsva4uag6lOjZ5BEHg4JUCfP3Ekt/boTrYtptvEMf27sL2b1f4Y3u24xMYgt1u59i+nTh7\nBxR7X1npXV9LsEaDIMDZLAMAOrONc9kGYovZUK8dHYjWZCNZ6zgplaW3kJpvoW501apEvVNKTXT+\n/fffKBQKevbsSdu2bWnZsmVRO6UHjVupl9msFqQyh6i52WhAqb6+AEWpcabNM2/xwQsDiKgdx4WT\nR5BKpfz06QQunDpKt1c+K3Xu242u7YKAXRCQX/MHQLk+CfpfPQI488uhdH3sc05lGZBKJKTrrfz9\n42jsdkfF5bUKe999+gwDh/2XfSkFGK12dFaBDTOeRBAER57w32vtdjt2u3CTIE5lVesrCyqTbV9r\nw4Vca+9Wq8OxyeXXF7fEtu/LpQNbefOxLnh4+3HhxCECwqN4+4ke2GUKeo8ff8s57zS6ttjtyCWS\nW3btUcrlDOnbjHeX7aGml4bLuSZqRvrTr7tDbOtaGw0P9qZ/ryaMWXmAKC81F3KMtIqvWdTqy2Zz\nPPHKZI7Pg1wuvem9sVrtVc62S910BNBqtezYsYNt27axYMEC/P392b59e8kD3+eNmSNr5rF73nSs\nZjPh9ZvTadRHmPU61k4fR1rCSZQaR67aZrWgcXWny+jJhNVvUXR/VuI5Vn/1BjmpiTi5e/JQ/xdx\n8fLHLzIWZ8+SZVBvp7edIAh8efwY00+ewGoX6BkawtT45mj+FeW5UW41J1fP74t2k5ap5e9le8nO\nNyGXSlCr5BQYrYQHeTDri2eJqx1CakYec5fsxWC0sH7LMU6dT8dFo0AACgwW/DydydEasAsCPdrV\n4+uPBnMxKZPhY3/ifFI2vp7OfPfJEFo2rR562NfyoNu2SZ/Phu/e5+KhncgUCpo/9hINug/m1LYV\nbP/5M0wGPU6ubhRoc5FIJPiGx/DoxDnXOe5DK35z6GFbrYTWbUqdzgNRu7gRWKthsWp/d6p3nWYw\nMHzXdvamZ+KikDO5cRMGRhTfQzGpfxjJugI27DjFmfOprN98DLNNwEkpw2IXEIBeHerx9aTBqFUK\ntu45w65/EtAZTMxbvJcCg5kALxfSc3RIJBL8PJ25kqXDVaPko/GPMqBX0yqth11qSuTo0aP89ttv\n/Pzzz8yfP5/g4GA6drz1MZyK4PLR3Rxd9StT/ljFTztOER4eytb/fczqb96g7cO9+GVPAm//9zdU\nGic+X7iJ/sNfY+3Ucdj/zUvbrBZWfP4qA4a/zK/7LjD6o2/Y99cM/GvWK9VZF1JaNPL3pYssTb7I\ngWdbc2lkR2xyE5MOX7/BdW0e29PDidFDO7J05X5aBTmx8PFafPVwDaR2O1M6h9IvTM1TL83CYLQQ\n4OvOay90Yd2mozR1sbNgYAzvtgzEbrEyrXsNeoQ5EeCk5OLwDtiTMpj4xWKeHDWThwMV/PVYDC/G\nefD8mP+RkV09WoQVUhlse+v/PiE4yJ//bT/Jp/PWcnztXA6v+oPdf07lPz/+xc+7ztK21wBqN27G\njA2HcNGoWP3VG0X3Xzq0gxPr5/P5Xxv4aedpwiPCufjPRkLqPlSiNOudKPKN2r2D+BAX0l7uzMrH\nHuLDwwc5nH3rNmRNG9SgTXxNNmw5xocdQvl7UC0GxHrh5yTn10eiSD1zicnfOPputm0WQ7f29fh7\n6V4mtQ1kwcAYmnrLqeWt5qe+UbgIVt5tHsXKfk2Y9PliZv+5je9nr+PrrmHM7V8TSWYmb02af9uv\n5UGnVIf99ttvk5+fzyuvvMLJkyfZvHkzEydOvB9ru21STh2kXa8BBIRFIFcoeXT4aySd+Ifs5Av0\neXYUUpmM6PqNiWvehvPHD9N98PPY7TZykh0VjPmZV5BJpbTv+zhSqZQGLdsTHBlD5uUzpc59u7m+\nXRnpPN8ghEAXNU4KGWPiI9iVkVb08xvz2IVk5ul5Is4HmVRCmLuKtuFunMow0K6GO2opXEx0bEbq\nCowkJGbRr7YXMqmEWj4aGgY4cy7LSJ/aXuQYHWdgxzQOZ9ueMwgWK50jPZBKJDQOdCHcQ8XJM9Wr\nmrJy2PYB+g9/DYVShX9IOO37DOTC/q3Ed3yYsOhYZHI5j416g7OH9+Ps6k7f50eTm3z+6v0nD9Ch\n72P4h4SjUKp4dNirJJ+8WQP8bhEEgV3pGYxvEYVcKqWujyu9a/qzJyOjxPtWbjxKPT9nYn2dkEok\n9Iv1IktvxSZAvxgPdl6zufjP4Ys0D3ahhocamVTCk/V9OZVpxFkh5dFYL3YkZzvmjfJj7ZbjdAh1\nJsBFiUImZWBtD3btTyiz11vRlOqwly9fzrhx42jZsiUKRfkJjt8LTu7eJJw8WvQYcfHUUZw9fJFK\nZSRfcPziLWYTSedO4+7jS+rlC1gtZlz/PeqncfWkQJtDVqrjyJBel09a4gWc3L1va/7byfX5qjUc\nStMWfX8oTYvfNe2plOuTir1PJZeRkG0EwGYXSMgx4qGRk2Owkl1gxsfLsYuuUSuRy6Qk/rtJY7HZ\nuZRrwkMj50q+GbNNwE2p4FC6lgBfN7RGCxkFjj8QeouN5DwTvt6ut/V6qwqVw7a9uHDyKOBwjudP\nHsXZy4+Lp09gtzmqYs+fPIq7t0N47Pzxw8ivaWPn5OFNwokj13w2juHscWu7LkyH3C4SiQRftbrI\ntm12gSMZ+fhpSi5yCQ324lKeCfO/uejkfDMCoJFLScgx4utztcLSx9uFi3lmbHbHa0jIMeKhliGR\nSDiXbSTAWe2YN0tHoL87F7SWotebkGPE99/PSFXgtnLYdzXwfczzWc0mlk4egZNaiW9QCId3bKLb\nq5+Rn5XG7j++oUGrDpw9/A8F+bnUb96OQzs2EtaoLV1GTSoa49DK3zi66nfimrfl9KF9BMe1oPWQ\nN0uc905yfXlmM702rCXYTYmnWsGWy9ksaN+Rup6eRdck9Q9DIldcJ+z+xYy1fPvjWuKDXbmUayJT\nb6FFDQ+OZRh49om2vDa8a9G1C5bv4z+fLqJJkAsn03TozDaahrix+1Ieoe5O1PFzY2tSDvNmjWTr\n7tN89+N6GgY4czLTQNfODfn47f538rZXKGWVw74b7qdtp5w6yKqvx9KgZQeyUpPRFejp/fZ3rJ02\nHsGkI6hGFPs3rSYwPBJ3Lx9OHdxLn3dmEBAdB4DFZGDpxy/i4uyEd0Awh3duovuYr4rOaV/L7VTp\nFsea5CRe27ubrhG+nM4qwEOu4bc27a7bWAdH490TrhEERvlit9tp3XsyBq2OGG8N+1J0+DgrifBx\n4liGgYU/jqZ2lOMEi9Vq48lRM0lLTCfEVcH2C7lEeGtwU8k5nJJPt5r+JOpMeAZ7M+OLZxj4wnfY\ndTp8nRTsv6Ljl+nDiG8YcZe/gfvPPVU63i33e9PRZjFzYf8WTHodIXWa4h4Qij4vm7XTxpGWcAK1\nqwfBdeOxW83IFEpSTuzDbDQgEexYzEakUhnugTWo06Ef7v4hhNSLv+VuN9ydcessFtYmJ2O02+gQ\nEEig0/WVi8X1ucvXGXn2lR84eDwRJ7WSoU+2xc/blVpRAcQ3jMBgMNO672Qys3VIJBAc5MXIZzqy\nZfdp1m86itUOKoWU5wa3JSLcj/YtahVpEB84eonjZ1KoEeJN6/joEl/vg0Z1cdgAeenJJB3bi9LJ\nhYjGbZErVVw4sJUtP07GVJCPX0QdXHwDEQCLXkvqmSNIpFLsVjNWsxmFSkV06154h9YkpF487n7F\nK+PdbpVucZzJy2NPZgbeKhXdgoKRFXMM1q2rjRNuEQRGOvaF9h46z4tvzCE330hkqA9PP9YKtVpB\n++a1CPBzZ/Hqg7z5wZ+YrHbkUgl9ezShcVwYX89YR1aOY7/Fx9uVV1/ojK+PG13b1kUmk2I0WVi7\n9QQ6neNkSXjw7T0pPyhUC4ddHIsnvkDdho3o8+xIzh49wMyJb9Fi0CscXPIjY7+cxcwPxuLm7cOL\nE74gOz2Vz14ZQkSzbnR4/u1Sx74X474V5s4hpLtLrzst8sKY/2FITOGpet4k5pn4el86C38cTZ0Y\nx7yten+MRK/nzVZBGC12Jm5JIiwigOOnk3m1eSD1fJ1Ycjqb1edyObvrE2SyqnHMqTo57BvJTkpg\n8aRhvDLlW0KiajH3209JTc8CIMDPm66PPcNHwwYy5M0PaNy2C1uWzmfR7Gk8PX01Gpfi0153G13f\nCdc67CvpeXQc8BnDG3hRx8eJJWdzuSioWPn7GADy8g006DiBx+p60SnCg73JOmYfSCMi3A9tRjbv\ntnV0rfl4axJ+oX6s/ve+qsBdNTDo3bv3LQeUSCQsXbr03ldWjpiNeq6cO8aUX/5GKpXyUIeH2bZy\nEed2r6XX08OIiI0j40oSL3/yHd4BQXgHBNH9yWGsWfArULLDLpRPLUtnDTefxwbYsPM0s3tF4KKU\n4aWR0yrEhW17zxY57IyMPMa1DibAxXGMa1CcD/87mEItbw2twxx5wCENfFl+JoeDxy/TtH7leTQs\nLyq7bSce20t8xx7ENW8LwNBxkxjWPg6JRMLbm46wefFc/ELD6dBvMAB9hr7Eqt9nk7BnHfU6PVrs\nmKUpTJY1ew9dINZXQ8vQf200zpsn/j6HNt+Am6uGJWsOopZL6F/HIWbWtaYHy8/kkJycyUsPBRDi\n5jgvP6SBHzP2p91ynqrGLR322LFj7+c6ypxCvevstCv4BAZjt9lIT7qMyiuQtKRLjmvkCtKTLhEc\nUROAK5cSkMpLL5wob+O+cj6zqJu6m5OKVJ2Fml4yBEEgTW+97kypRCIhNd9MPT9HeiVFa0YqlZJe\nYMFmF5BJJWQbrNjsAsH+XuW25spEZbdtlZMryScuFRVHpSVdQuPqhiBAWtIlPHz9yM1MLyoS02lz\n0RfocPMvuZdiWQcgJeHuqiFdd9VGswxW7IJj8xwgMswXvcWOzmzDRSnDZLWTY7QCElLyzUXjpOSb\nkVaj9nm3dNjt27e/j8soe6QyOc0fH80Hzw+gVfe+nDt6CInKhVZPjmHRh89RoNXiHxbBtPGj6NT/\nKTKvJHF4xyb6vj+7xHHvRODpbvDLs5PucTVt8f7Yvnzw6ULahbqQordhUGro2/XqhtHQp9rz3U8b\nOJ9jwmC1s+OylulTnmbshD8Yt+4S9QOc2HRBS40QLwL9xf55UPltu2bzLhxbN4/PXhlKWExtti77\ni+aDXgHgk9FDaN3zUQS7wPvP9KFR607sXrsMJzdPwurFFzvevcgq3AnatTL4d1+7TXw0weH+fLj9\nCjEeCnak6HlrZLeiysTW8dF4ezgzds1FWoe58k9KAXKFnDde6s4HXywh7d/mHpsuavnwreKfGqoi\npeawz5w5wzvvvMPx48cxGh3HyyQSCefPny/ptvuS59NlpbH9l8/JS72MysXj300WI2EN2/BQ/+HI\n5AouH91N6pnDOHv6UbttL2RyBVt+/oyEnatBsGMTJJj0+UhlcvxrxGA1GXHzC6bl028UuzkzYs5o\nrCpf3jnwD+fztdR29+CjRk3wUakYumMbezMzEBAIdnLGKtgJ0DjxQYPGxHp48NHhQ/x6/ix2QcBH\nqUYulaCQSckwGNHbrLgqFPz8cgu8mza7Lo+979AFtu49S3qGlkUr9mGz2pHLpQT6eyIRBLLzjejy\n9QB4+7ihVsjw9XYh32glJ09P88aRTJ80GJvNzlcz17By3SEUCjnubk6kpucSHuLDx+8OuOXmTEpa\nLu98vICEi+n4+rhh0JvRG8106xDHm6O6V0j5b1nksB9k2z63dwNHVvyK1WxC7e6FMS8LpcaZJv2G\nExrXDKvZyMktyzBocwip+xBBtRth1GlZ9OFz6HMzkUilmI1GbFYLSicXPANCsVvNhDdux0OPDkMq\nuxqrjZgzuih3vTwxkf+eOo7JbmdAWA1G1o7ltDaPZ7dvIcdsRiGREuzkjNFuo42fP+81aITRZmPA\npvVc1OWjkMoI0Ggw2W1oZHKSCnTYgLrunsxr14HMxyOLbNtisfHXyv2kpOawcedpTp9KAgQ0Tmq8\n3TTI5VIupeRgs9mRSiWEh3hjsdgIDfbmQnK2IwU0uju9OjcgMSWbdyf/xYXLGfj4uGHUm9AbLfTo\n3ICxI7rdJMlQyIoNR5j+w1qMJgv+vh6kpufi6qJm3Ms9adMsplx/x7finiodhw4dyogRI1AoFGze\nvJlnnnmGJ598sswXeadYzUaWTh5JXIP6PD7ydTIvnaL/cyN4edKX5F8+zu653wIQFtec+P4vUrdj\nP2RyBQdX/M65bSsYNfFr3pz6M94+PsTENSEgOIzY+g157ZPpNG7WjGWTR2A26q+bM0qYgdFmo/+m\nDcQFqZjVoy6h3lIGbd3ECzu3cyo/h9/7NCTa05lIbxUzutelZy1PBm7ewBdHjzD77Cleivfn7bbB\nGCVmovxk9KrrQr7VwvNN/OgQ5UKfL7aQmaO7bt6HGkYwuG8z5i3eTfdIdz7qFEb7MFcuJWXibbUi\nNZt4v10Ib7QKIjc7n/Y+MuLkJlJSclj5y6tMn+TIZX7235WsXr6HZ6OdyEnLxjkvm1F13Qgy5DLg\n+f9SoDfd9D4bjBb6P/8tnnlZPBKm5tCxy3TxgWG1nNmybj8fff1g53tL4kG17ctHd7Pzl895cvRY\nouvURWYpYPTEL+g/9EXW/fcdMi6cQq5UE9dlIPH9hxNUuxEAiz4cSlBoKO/O+JOnx05AgkCf50Yj\nFew8Pnw0oyd+QW7CIfbM/65ormuj6y2pqbxzYB/jW9Xgy84xLEg8z3cnT9B3wzo6R3jzfdd6GGw2\nhjcJ4vvudbhszuXNf/bSfd1qZCoL/+kQypMNvTmvy+fxBu6EeINGKeHdtkFk2vLpt2n9da9ToZDx\nRN94klPzOHUykTEtAnmvbQgKmwVzXgGXkrJpF+bKx53C6BXtQWpKNs/FOFOQkkaTuqHsWDyeXp0b\nUGAwMeD5/+Kry6FPqIojxy7T1VfCsFrOrF+1l8nTVhT7Pm/dfYZxE+fRJ0BGAGZSEpIYVsuZTp52\nhr8xh6Oniq+NqEhKddgGg4HOnTsjCALh4eF88MEHrFhR/BtwP8m8dBaNk4YBI8Zw5VICnfo/SfMu\nvYis04AXJ3zGud1ri73v7I7l9Bv+Ko3adCKmQROGTficlEvn0Ou0PPf2x9SoVZdHnhuNp48v6df0\naQSHwFMGbmiUEt5uUZP6fm5MbBNDvtXMnswMvu5Uh/q+bhxM1zL74fo08HPjhQZhNA304M8L53m8\nng9Ng12o7aNhdHwAp7KMNA9xpVeMJ4l5ZgbW9cHPRcGq7Se4cv56OdU5C3bgqZYzuL4vkZ5qhjby\nQyWXciQjn5EPBRDr60TjQBcGx/mQkGOie00PYrxV7Nh3rmiMxasO8GJDHzzVcgosdkY09SfSU03/\nWC9cZQIHj9+sVHfybAoyq5VBdb1J1prpFuVBqzA3or01jGzkw+I1FS/kf7c8qLZ9fs96HnnuJRq2\n6sDpg3sZOfEbouo2JL5TDzo/OpiEfRtvusdut5OTcpmXp3xLRGwcbXsNIL5zT/7ZsJIujw0hvlMP\nouo2ZNj7n1732bhWtGxp4iVej69BlwhfmgV58mmHWvx58TwSKXzavjaJ+Ub6RvvzRJ1gGvq5MaNb\nPRZfvMSFfB1jWwYR5aWmY4Q7LUNdydBbGd7EH7NNwFUl59XmgZzIy71p3QBrNx5mcH0fmgS5EOvr\nxEvxgaQbzEgk8EJjPyI91TwR54uXRo4AvNzUj2XrrxYDHTuVjEZi57E6XiTmmeke7UHLUIeNjmjk\nw5LVxVd2Llm1n37RbjQJcuFEhoHXWgQS7e3YCO0U7sLKDUfv8TdZ9pSq1qdWq7HZbNSsWZNvv/2W\noKAgCgoK7sfaSkSmVGLQ5WO1WFAo1WSkXP1rmJ+bg1xZ/OahRConPyf7mmuzkUplmE1GTAY9aidn\n7DYbBVrtdWMUlqCrZDK0JisWmx2FTIrRZqfAYkUmkZJlMCOXOs4y55ms+DgpEQTh3/+XkmeyFo2n\nNdlQyRzX5ptteKjl2OwCBWYbAdy8oenmosZgtWO1C0UdOExWAZVMUtQHEiDPaEUplyAIAlqTDbXq\n6lgqpRytyYqbSoXZZsdoFdAoJNjsAjqTDY3q5nlVSgUFJitWu4BCJiFNd3UurcmGuhJ3tnlgbVuh\nRJvjOKanUKrQ5eUU/Uybk41cdfPmsVQqRSqTkZ+TjbOrY69Cm52JXKki/5r783Ozi+z6RlkFlVRG\npv6qPEKWwYxaJsNotWOw2lHLpWQbr/m50YxGIcdmsZBvcmwOgsMulDKHjRqtAkqZhPQCK7JbnPOX\nyqTkGa+1KysSwGyzY7YJqOQOGy2w2FDKpOSbbKiUV5ULVUoFOpNjY10pk5BRcMPn7BY2qlIryTc7\nKi2VMgn513yO8i0CavWDV/1a6qftm2++Qa/XM23aNN5//320Wi0///zz/VhbifiERuMRHMlnrw6l\nTnpPgGkAACAASURBVJPm7F2/EqVKTUBoDZb/9gON+w0v9r4Wg15mxeevYrfbcfXwYvHsqUTFNSI/\nJ5uJzw+gXd/HObxrC2pPP/yuaawLjhJ0H0GgtpsHg5YcpFukD0vOptPWP4A6bp68sv4Yl7UGmvi7\n0WX+HkY0DGPflTwMJpjcqAnP7NiKIICrSsaC41m0DHVl9v40Nl7Io3+sN5O2JCERpIxs7cq5G3JY\nwwa35puZa/hg02VahLqx+WIeMinE+7vz7d4rpBdY0FtsLDudQ5/annyzLx00TrRvWatojNdHPsyE\nKQvpHeWGr7OCdzZconOkB4czjYTV8Kdh3dCb3q/Y6ADqxoYyZWcqsZ5KdiXlo9ovJdBZwfLzWsa/\n1qcMfpsVw4Nq2/W6PM6iic9jtVoJi6nDl68/zyPPv0zGlWT2b9vIgI9+Lfa+0AYt+OjFx+k1ZAQX\nTx3j9KF9dB44hA0Lf0OuUOIXFMryX2fx0GNXHfW1566fj46h94Z1WO12/t/emcdHVV5/+LmzJzPZ\nF7IRSCAJ+xJAdoKCbIoIlrpVRVSwWlu1KmrrT7vY2qpttWpbWxQ31ALuUGXfA4GwJmFNAtn3ZWYy\nmf3+/hgSAgmQQCCZ5H345I/MzL33neHMybnnvOd7AnRq3ko/zd+vG8sL+/dy88o0bk3owY6CKh7+\n4TDDwv1590A+TwwYyM6yUn69MY9b+wWTXW0lq9zCgDAfnt+QR7CPih15JlZkVjKvZ+8W1/3sYzfz\n7MsrcLllDBol/82sJCnYQIGpnl9vyGNyXACp+SZcbsgqq+OH03U8/tCNjccP6R9N375R/DG1hKRA\nNVvzTKiVCsL1ar7LNvLC03NbvO4Dd01i9r37cLplegdqeXlrIfMHhlBR7+RghZ1X57RcpO1IWt04\nYzR6tAL8/Vs3RflaFGZcTgeHfvgcY0keKBQUZu7F7bITM3AMKQ88d8HOvYKMNHZ9/hZul5PwvoNR\nKBQoNTq0vn7UVZbgFx7D0Bl3oFR7thidL59qd7l478Rxsk0m+gUEsKBvAkqFgifTdrOmIB/Z7SbC\nV49VdhKg1vDW6HEkBgSwraSE3x88gF124a/WUmgxo0Si3uXG6naglVQMDw2m3uXCLyGMOpsVWZYJ\nDw+ktLgavV5LVm4ZploLhgBfVLIbp8NNZHQIJpMFlUrJ2FEJmE0WIiOCWHT3JAz6czUdtu46zpoN\nB9H7avH39yW/oJLAQD2n88upqjIzaWw/Hr3/hnOKNHaHk/c+28HJnBJiokMwm62YzBamXz+EG8b3\nuxr/tZekPRtnOqNt1xSfJnPjF7idDkyVZVTlHUeh1jD+7ifpNWzcBY/b/vFfKMzcg0rnS2TiUBwW\nM/rgHtitdTitFnonpxA71CMr3LTY2EC2ychHJ09id7uZE9uL0WFhVNlszFr3AzUOO0pZIsRHi83t\nYmRwGG+OHoNSoeDF/fvYXFKMj1KF3eWixmnDV6Gm3FqPLMnE+hgI9dXh6uGLwaCjylRHSLAfkgQ1\nVXXofLUcPJKPLMuEhQVgrrWgVisJ6xFAdZWZyIggBiRGYTLVkzKuHzdNOXe6vd3hZOln28nOKaVn\nTAhGUz1msxWDwYeMzNNotWp+ev8Uxo44V0I453Q5H67cid3uJDYmhNzT5fj7+bDwzomNHcHXmivq\ndNyzZw8LFy5sNOrAwECWLl3KyJEjL3rRa9kNVlOSxxcv3s9ti35BeHQsn7/zGjHDJzNy7oPtcv6W\nDPt8Vp7K5Y8ZB/j9pET+eSAPCXhiZBxpxbV8llXMhumzCNR4/gD84+gRPj19kv8b35dCs5WXth9n\nyeh4XtuTy2PJvQH4W3oui0aEo5Ak3k0v5eGhvfDTqnnzUD7/+cv9PPDEezyd3Ive/j68vCeXWbeO\nPkdXpLWUV5qYMv9VpsfqiQvU8NWJWpLHDeSVX81v87muJe3hsL3BttO/eZ/Tu9dyx8+eoaKkkBX/\n/AvzXnyPoKjeV3Te1mi4g0dw6kebNxARoGRaXChPbjzCE6PiGBTqx+tpuQzzD+d3yZ6xajaXi5nr\nf2BMjD8z4kP5IKOAE1V1pMQG89WJMl69vj9v7M3FLjuY2y+Yw2UWNucaeWPKQD46VoIhLoL+SVF8\ntXInL1wXT77Jyit7c/n6w1/Qt3f4RdfZEp99tZs/v/Et9w4Kps7h4uPMaj55ZzHJg69eN2d7cFmd\njg0sXLiQd955h4kTJwKwfft2Fi5cyKFDh9p3lVfAidR1TJh1KzPuXAhAeEwsf3jknnZx2K1VL/so\n5ySv39CfST2DefD7Q5x+eAq+aiU3xoWxv9TIlpIS5sTGNr7237MGMvzMvuhCk5XthdVM7RXK06P7\ncM93+7l/eBgpZ2bfycD+olpW3jqSozUW3lq2iR/1DefhYR7D6x3gy49Wpl6Ww163NYv+wVp+NMCT\nF00I8eHBr/fwh+duQ3GJsWjejjfY9pFNX7Lkr0uJTewPQHlRAcd3/sDoHy2+4nO3pg39dF0dJ4xG\nvvrxJD7IKGBGfBhPjvIMJxgW7s+wZdv57fBkJEniUHUVsuTm1ev7IUkS18eGMnDpFnYX1fKPaYMY\nHRXIg/87yEfzEtCqFCRHGThdY0chSXw4YzC9/7WJvQdP8emNAxkU5mmhzzPW89X3+3nq4eltfn8f\nfL6dxcNCGRbpGa9nsrn49Mtdnd5hX4xLfiNVKlWjQQNMmDABlapzFZokCdyus0Ny3U6X58F2ojVq\nfBLgkmWkM/9cTf5CutzyOS5fwjMqrAGn240EOM/IRyokaDrz1+2WG/9oOGUZSQJXkz/ATrd8+W9X\n4py1ut2e7jlvEoK6XLzCtpFwuc4W0dxO5xWbdlskVCXAjWesnSR57KMBp3y+XUuNNgweG3fJMgqp\n4TsgnXn87DGuM/bWIJ0qKTznPXuNy/8qKxTnfQ9lUHi5XV/SOlNSUli8eDF33nknAJ9//jkpKSns\n2+fZKpOcnHx1V9gKEsfPZOUL9xIUFk54TCyr/vUGg2788RWfty2GvbBvIk9sSOf/xvdlZEQAc1bt\n5bERvdlTXMOpWivXR54dNPpAQhIP/S+D58f2ocBcz/uHC3hyVBxvpp/ixW3H6Rfsx5v7cpHxOO/3\n95exeFgv/rDrJFuKanh/yW3c+7N/E+mroZe/D39OP8X9d026rPc4PWUgr779Pz4+XEFcgIZvsk0s\nmD+uWzhsb7DtQTfezpvPP8b8xY9TUVzItjVfcttvr7ww2tqJMrF6PUOCglmw+hDT40NYd7qC3+44\nzuAwf97Ye4oHEhIbbWVocDAGpYZH1mYyPT6UjzIKCfXRMKJHAIvXHubliUkMCffnpc353JIUxOEy\nC/lGOya7g7v+d4i504cxICmaBz/czLMjepNvtrLiZCmrf3PXZb3HB++ZzIuvrOJOm8tTkM82suLZ\nC+f/vYFL5rAnT5580S/vpk2bWj7xNVY0qy46xf5vl+GoNxM7bBL9UmZfkdO5HPWyv2Zm8NmpbBQy\n2FxubLILJQpSIiJwyG5GhoSxMCERhSSx8lQuqwvz0UoK9pZXYJGdSG4I1HkKOsFqHZUqJ8gykkaF\nqaYOpUrJpHH9ULhlQsP8qaw0UWe2cuMNQ7h9zihkWeajlamk78+hR49AHl04hUB/30uuu7Ckmr/8\n43vKK4zExUVgMnoahu6cN4ZR5+kI5xdV8Y8PNmEyWpg2ZQizpw5t2wfbTrRHDtsbbFuWZY5u+468\nfVtQ6/QMm30fwdEtz0tsLa2pyTTF5HCwYNsWTteZ8FGoqbZbkSWJUI2OYSFBSJLEPfEJjAgNxeRw\n8HrmYXJMRhRIpFWUgwQGpQYkGQVgUwaBsgYfg47qWgsuu5PgUD/GDIvD5XITEuLPqdxSDH4+PPrA\nVBLje1BWYeSdZRupqjQxYVw/5t88slXf7/9tOsyKr3ajViuJjg6hpKSaqKgQHls4hYDz5jyu3nCI\n79cfxM/fh8X3Xt9hsqzdVl71SmirfOrawkJ+nraT2wYGU+9wsSKriseSe7Eso4B+oTqGRerZlGMi\nOSCC10eNbjxuzOpvCDeomJcYye9SjzOupx9+GgXfHq/mzkGhKCSJTw6XM7NvEE63zNrsGpaM7sPO\nUhOa6BCW/m1ho+G++OcvSd+WxYL+kewpNZJmtPLd8ifQ+7RuEnj6odPc/ei/mNvXH4Uk8cWJWpb+\n9YHG4bwlZbVMu+N1JkXp6OGr5quTtTy2eAYLbp/Qxk/3yunO8qpXQlsH7MqyzMId2yh21DKul57U\nfBOlJidzEyN492A+dw32iJR9kVXNRxNTGBPmKQ6mV1Qwd9N6Hhneixqbk8+OFHL3kDC2na7F6pS5\nOSmIQ6UWDpbUcefgUL4/UYPNKfPkqD68uu8Uf3rxDmZcPwiA6loL029/jZtighgQpOedwwXMmTem\nTTWbZ19eQdqOTKbE6jlaZSffqeJ/y5/E58xe649WpvKXt1czNyGACouTjQUWfvjsSaIjgi5x5vbn\nilrTS0pKeOCBB5gxYwYAWVlZLF26tH1X2Mm4HPnUt45l8tDIcG5KDOJHA0OZPzCEg2UmwnzV/HJc\nFFPjA/lVShSf5+ZidniaDw5VVlJcX8+X80aiVysZGObLT0dFUFrn5L5h4dzaP4Rb+gXz0IgeFJvt\nPDiiB8Mj9WRVmlk+cwjpB3LJK/Q0AdkdTpatSGXlzcO4Z2AMb1zfn1AJtqReei5lA+9+uInb+wU2\nXvcnA4P457INjc+vWJ1OcpiWe4aEMa1vIE9e14O3399wkTN2brqbbTfYdVsG7BZYLOwoK+XZSZFM\njQ/k+YkxKBSwv9TIA8PDmdMvmDn9grlrSAj/On6k8biXDx3kzv7R/N/4RDIqjPxiTCRT4wPIrrbx\n8pRYpsYH8sSYSCINGkJ81Lw8JZaSOjvzkyL426R+/KuJXa3ZcIjhwXpemZjEvYNi+HzWUP7x4eZW\nv4d6q4NPv97Dr8dFMDU+kEdHhKGyW9mWdnZu5NvvreeJUeFM7xvI3UNCGR2h47/f7m31Na4Vl3TY\nCxYsYNq0aRQVeeYdJiQk8Ne//vWqL6wjuRz5VKfb0wnWgI9Kgf3MYw0RsEapQCVJjUWVercbtUJC\no1DgaHK80y03O1dDMcdXrcTudKNWSmhVSuwOT0HKdaYK6XPmOEmS0KvPPt8a7A5ns+s6mhzvcLrQ\nKaXznnfjrXQ3274cu3a4PbbW0MGrkECrVGB3tWzvDTjdbvzOdD7aXW58VIrGYqNGedZGfdUe29Yo\nFUh4Uol6jRKH42zXocPpRt+kL8BPo8TpdLf6DsvtbuhmPHvd823b6XThoz77fnRK6ZznOwuXdNgV\nFRXcfvvtjZNK1Gp1p6uktyeXK596Z1xflqaXs7+4jp35Rj49XMHkniHk1lj57HA5R8otvLmrhLHh\nYY37sUeEhKBVKHl0XQahPhr2FJn57lgVfYK0vL+/jN0FJvYWmXk3vZS+wTq+P1HN9jwjk2JDeHrL\nMULDA4iP9Sif+ejUTJ/YnwfXZZJaWM2b6ac4WGlm4nUJrX8P88byaVZ143U/zKjijnljG5+/ecoQ\nNueb2ZBTQ0aZhbf2lTP/llFt+pw6E93NtqHtdt3bYCDW18C/9pZxpNzC+wfKMNpcpPQM5j/7StlT\naGZPoZmPDlZwR++zTSmLk/rx74P5fJpVyJjIIN5KK+FYRT1xgVr+vKOQrHILKzIrOFFlRa2EV3cU\nEqRTk1lh5qltx/nRrWe7DG+c1J91eZX8+2AeOwurWbguk9tmJbe6RqX31ZIyOoE39pSSVW7hiyNV\n5JudjBvZt/E1P55zHW+nl3O4tI6NubVsyDMze1rzuZcdTauKjqtWrWLq1Kns37+fXbt2sWTJErZs\n2XLxE3tpnu/8gky2ycjKU6eQZZnbevcmwb/l7idZlvnT4UOsyMtFAchuiXq3E4UsU+t0Issy/moN\nO2feRIBOx56Kcn4oLMTpdrO+uJBquw2tQkmMXk+9y4m/WktenQm3LFOvkrBbPULt8b17gNNFUkIU\nLzw1h6CAs0XFequDV978jr37c4kID+DXv5xDXGxom97/N2sPsPTjzciyzL13TOJHN4045/n0Q6f5\n81urMZrqmTFlCD+7fwrKDhCQb6+iY3ex7ZaK6N/l55NeUU6UXs89ffqiu8D4uBqbjft3bCXXbMJX\nqabO4cAlyTidburObDmcERnNvydOwul2szwnh1yTkQqbjW2lxThkNyEaHXqNCqXk2fpXUm9B5afF\naK7H5XSjN+iIjQxCkmHu7FHce95OpSMnivnj376lqsrMxHFJPPnwjDbJ+lrq7bz8xrek788lKiKI\nF5++lV4xZ4uKbrebt9/fyJr1B/Ez6Hj60VnNCu7XiisqOqanp/PYY4+RmZnJwIEDKS8vZ+XKlQwd\nevHdAd5o1H3kf56jXna0toa5Gzdwx4BIlBJ8nFnE5yk3MDS4ufjOrvIyFmzfyoLBMZgdTr48Vsry\nCSncsmk9fUJ0DOuhZ212DXYnvHHdWJ7au5uFQ2IoNtvYeKqKH6bNIEyna7zuLRvWkRLnhwSsOVHN\nDXEB1MsShQ4F/1v+ZLOW8+5Gezjs7mTb5xfRX8s4zFcFufy4fyRpRTWY62HF5CmoW2iWemZvGgdq\ny5ndN5wNpyoJUvoSqfPlg5wTzErwDIxec6KG++ITKLZaqHZZmNI7hG9PljEsIIw/jzwbLf854xD/\nPX2Sib0NHC6tJ99oY0bfAHYU1TN/3jieemTmtflAOjFXvEvE4XBw7NgxAJKSklCrL50H81ajbhqB\nPLprJ4OjNfx8pOcv7bsH8tiWY2bp+InNjr19y0ZuHxzGHf09X4jf7TzBfzNLcCocvDUrHqVCotbq\n5P6vTtLHz8ArU5KY0ssT/f58XSZRymCeHOipiv80dQeGQAtz+3sigNXHq8gsq+eZCdH8MbWE+ffc\nyN3zxlzVz6Kz0167RLqDbZ8fiNhcLvquWsHhhZMI12txyzI3LN/NkgHDmRJ1bsqkxGJh4verObxw\nEv5aFTanm1Ef7KDSZuXHA0OY089jo18freS/GZWE6HTsXTAejVKB0eZk8Htb2TbjJiJ8fbG5XPRZ\ntYJ3Z8cT6KPCLcs8vfY0dw8OJS5Ix+LVuZzY8Qc06q6dlroUl7VLJC0tjeLiYsCT20tPT+f555/n\nl7/8JVVVVRc6zGs5X2oSwOJ0EOV3NpKN9tNhcTqavQ6gzuEg2nD2tTF+OupdToJ1KpRnCjZ+WiVK\nhYTR6SSmyXmj/LTUOc6e1+x0EOJ71nGE+qqxOj2FkxCdkrr6szPtBG2nu9l2U2cNHoetlCRCfDy1\nFIUkEaHXUudsXmSrczrx16gaC4halYIwH41na6D+XBuVJIlwX01jcc9Po8Rfo2o8r83lQiF5vgcN\n1w32UVHvdOOvVaKQwGbrfIW+zsQFHfbixYvRaj37d7du3cqzzz7Lfffdh7+/P4sWtSxd6u2c30ww\nM7onr6Rmk15Sy4EyI7/bcZKZ0c0lSBte+8K242SUm9hdVM1f0nL5SXxfjldZWZddQ7HJzrvppSgl\niVtjevL0xqMcrzKzKa+S9w4WMC367IDUm2NiWXG4ihOV9WRXWXl/fxlJoT7sLjCxo8DM5LFJLa5B\n0Dq6o203xV+jYVhIMEs2HyWnxsInmYWkl9YyOiys2Wt7GQwYlBr+kJpNTo2Fd/adprTOTohax/v7\nyzhZZeVklZVlB8oIVesoMdt5Z99pcmosvJx6EoNSQy+DofG6Q4ODWbqvjGKTnfU5NWSVWwjxUbH0\nQDnD+sfgZ+jeqb5LccGUyNChQzl48CAAjz76KGFhYbz00kvNnrvgib3otvFCymWyLPOfE8d578Qx\nZODe+L78tF//c4ohFqeTr/PyqLXbOV1nYmNxEWqlgplRsYRotWwqLmZXZSluWUYpSSwdN5FJEZH8\n/uB+1hQWoFepuCk6Fj+1mt4GA9OjPXMk/33iOP85fhQ3nvmPhW4rIcF+/OaZuUxow86P9sZSb+eb\ndQcwm21MHJ1AUp+IDlnHlaREupNtn58OaaDKZmNJehrplZVE+fjwh+RRDDmvNnO8tpbNJSW4ZJkt\npUUcq60lzmBgSmQ0uN38/egRLG7P9jt/lZoDs+dw2mLhmfQ0ckwmEv0DuD4iEoWkYHJEBIkBAVTZ\nbDy1dzfplRUEaTS4ZaiWXAxOiuGNP95NcKD+mn0253M8p5Stu45jMGiZfePQVjectTeXpdbncrlw\nOByo1WrWr1/Pu+++2/ics4VbJ2+npVZdSZJ4KDGJhxJbjmjrnE5u2bCWcIOKXgE+fJVXyj/GjKeg\nzsyrmYe5qU8YR0xV9PLzZWLPYFZnl5FjNjNFoeA3w0fwm+EjeD3zMB/mHGdElJ73c+v5viifv44a\nw6LEJBY1ue67tnHMuruQyPi27fpoT+rqbdxyzxv4OKyE+6p47Z3/8c9X7/O6iL872XZLzhogWKvl\n3+Oa12Ia2FpSwqLU7cxJ6EGB0UppvZ2N02Zy/85trC/PI9qgpd7t5qb4Higkia35VeTX19PH359V\n10/F7HAwa/0PfFt6klBfFa9mHuK9cROZGBHBe+PP1b2xT42hLEDRoc562+7jPPTUMsbFGKiwuPjn\nsk2s/vhx9L4d47QvxAUd9p133klKSgqhoaH4+vo2qpqdOHGCwMDAa7bAq01bBJ7OZ3lONrGBWj6e\nPRRJkpjVJ4xfb04nz1zHtrvHUlJnY2tBFVvuGotWpeCxEb0Z/eGOxi1U1TYbf8/K4u2b4wjyUWF1\nuvn56lNk1NQwOOjcltgFN+VRJnes3OmnX6UR4LLxzLgIJEliVA8fXnhlFdu+fr5D19VWuottXwkv\nHdzH29MGMjM+HFmWWbD6EM/v2wdKF9/+aCSPrs3giVFxPDPas/f6z7uzeT3rMG+N9ogrfZR9khA/\neGp8FJIkMTzKl1/t38vWmTc3u5ZmfQHcFktxTkWHBST/96cveDQ5lOui/ZBlmdd2l/LxF7tY/JOU\nDlnPhbigw/7Vr37FDTfcQElJCdOmTWvURpZlmb///e/XbIHXgra06jalymajX6i+MUXSL8RAlc2G\nS5aJD/Qls9JM3yBftGc6wnr66dAoFZgdDo/Dttvx06oI8vH8N+hUCiIMGqpszaeXNxh1R1JVXUeM\nXtn4fnsGaKkxel+RrrvYdkuF9NZSabXRP8STe5YkiX4henadrqNfmB6FJFFpdXDTmecBBoQY2JNX\n2vh7hc1Gz0BNo63EBmiptlde8HoDTLlk+cVRnFNOZHzzXPrVpqrWQq9BHg1uSZLoaVBRWWW+5uu4\nFBcN2caOHcvcuXPR68/eqiQmJnYK2cn24Eqia4CJPSL4JLOIw+VGjDYnv91xkpSICAYGBfJy6kn6\nh+hJLaphXW45FoeL19Nyifb1JeRMwaunXo9WUvLd8SpsTjc7840UGO3NouumlJxuefL0tWDi6AQ2\n5deRW23F4nCxPLOKiaM7Lp9+JXR1226gLap8TZkUEcHvdpyk1uYgs8LER5mFzO4Zyzcny0grqmFs\nVCB/2p1NkdlKkdnKX/ecYlKPsxLCk3tEsDHbyKkaK3V2F8sPVZLS48L1DuNaJTGr8jzTOjqASaMT\n+SSzmjq7i1M1VjbkmZk0JrFjFnMRuq1aX0Pnl6wLZ1d5OXqVipSICNQKBbIss720lAqbleSQ0MYq\nd0t8kn2S3x48QJ3TyYTwcOb16o3TLbPs5HEOV9fgp1bho1JSYbWTHBrMO2PG07OJk8g2Glm8aztZ\n1bXEGvS8PXocI0JDKa2v591jR5FleDApiShfX0+uL1DZ7LaxutbCjj0nUasUTBqT1KhAdjX47Kvd\n/P5v32G22JgyPok3fn93hzTxCLW+i/Pwsp9xuNZELQb6BQYy4Eyqp8pmY3tpKVqlgpSIyAt2N5od\nDh7dtZMNxcXoVSoW9EkgKSCAgro63jl2hFq7nWi9L1U2zxbTBQkJ/HrIsHMGBHyUfZKXDx2gzuFk\nenQ0f7tuDAa1mk1FRfxQVEhCQAD39+l7zmSjgttiQaIxys46XsSx7BJ6x4YyfODVu8Oss9h44oXl\nrN12BL2vhuceu4mf3Db20gdeBYS8ags8vOxnHDdaeXDfYZJCdFTWOwlQ+rBi8hQeT9vFEWMVicEG\ndhRW8c6YcdwQ2VyDwe5ycfe2zVQ764ky6NicX8HgUH9qrE7yzRZGRPpxqsZGSngkfx01ptXaB0dr\na5i+7nt6+mtRSpBbY+ObG25kwO1DKQtQENnn7C3jqfwKbr3/78T6qah3uHGotXz94S+aaf12NYTD\nvjiuX9/CqqJSkkJ8yCy38PSgIUyOiGTuxg0MDjNQa3Nisct8fcON+LXQLHSgspK7tm7muqhATtVa\nyDfVMykmmK0FlfhpVcQF6sgos/CfcRNJiWj9TqHn0/fwUU42g8N9ya62EqHTs2nazHOc9ru2ccy6\nq4D/7TnK6+/8jwHheo5VWLhn/niefnRWu3w+nZkrmunYFWmQmXz5RC53Dw1mSnwgblnmlW1F/Co9\nney6Grbe7SkUbi+o4qE1uzk4Z26z83yWm4ukcrJp/hiUColVx4p5Mz2XEks9L0+JpU+wDpvTzVPf\n57GzrIzxPXq0an0P79zBDXEBPDTC8/oPD5TxyO4d7AwJaZbH/s1rXzG9py/z+gcjyzLvpJfz9/c2\n8OtfNC/uCLoH1UWnWZ1fzBuzehOgU1FqtvPE9wdYV1TEoyNieWxEb2RZ5uEfMnjn6BGWDB7S7BzP\n7tvDHyYn8uN+UbhlmR99mU6wj5pIPw0vT4lFrZQ4UFLH42mp7L+l+XejJexOJ++fPMnr03rRO0hH\nvcPNI6tz+CA7m/sTzqbWFml38rdlQ/nbuu/467RYehg01Fqd/OKz7dx28yjie137HHdnoWtPWb0A\nDTKTxZZ6BoR5xJMUkkRiiI78OjPJPfwbC4WjIwMprbeeM4OxgSJLHaMiAxs7GcdEBVFosmJzA5If\nCAAAF6dJREFUycQHefLUWpWC+CAdRRZLq9dX67QxMPxshDwg3BdTk07IpnnsouJq+od6UhKSJJEU\nrKGo6MLFHUHX54ZPf0W0n4YAnSce62HQEKxTU1BXx9goT2pEkiTGRAdSVF/X4jmKLBZGR3pqKQpJ\nYkx0EHnGehKCdajPSOz2D/WhpN7a6judwvp6FBL0DvLYq49aQXygjhPG2mavneneSqCvmh4GTzdm\ngE5FTKCO4vLmr+1OdDuH3VQ+dWRoKN8crcbllqmpd7LtlImUiEi+zy0nu7rOs2sg/RTJocEtDu8c\nERrKymPFFJk9Dv2NvbmMjAggxk/LmhPVAJyusXGwtK5FwagLkeQXyLdHq7E63dicbr45WkW8wR+A\n8Fo3suus8x6VHM932UYcLhmz3cWGvDquS+5zoVMLugF9Db4UmhxklnmChNR8EzanzJiwMP6xPw+H\ny0211cFHhwsZGdzyNrqRoaG8te8UblmmpM7Gp1mFTOkVSmqBiRKzHVmW+fpoNcNDglud6uul16NW\nSPxw0hNw5FZbySi3cGML6cZYvR67E3YVmADILLOQX2MlMb51d6ldlW6Xw24q8FRts3Hvti0cqKrC\nLcv8fMAAlgweykfZJ3lhXzqS5NED/mBCCrEXKDy+kZnBnzMyUEgQoFFTa3PgQsZP49lXLQGvj7qO\n23q3XqrR6nRy/Q9ryDV5thXFGvRsnDYTg0aD/zQXWX5xjXlsS72dny75gM27TiDLMnfdeh1/eO62\nc3KCXRGRw74wDy/7GVlWDQ/u3I7D7cKgVvPRxBSSAgJZvHM7W0tKcSOzoG9ffjd8RIsOt9Jm5Z6t\nW8iorsYly/iqlLhkQJaxn+najfcz8PHEyUTrW9/wsrawgEWp27G7ZGTgwYQEfpc8ssXX7qus4L7t\nWzE5najVSv74y7nMneu9+uutReSwz3D+vtRqu52iegv9gvyosto5WluL0+3mnj59uTMuHrPTSYBa\nfcEIwul28+mpHPw0SkJ9NBSYrbw1ZizTY2LQKhTUOhz4qVQo2+g8dSoVqTfdQqXVihsaZVfBs/2J\n286+1tdHwwdvPkSdxYZSqUCnvXo7RASdn4atqrlmEwpgQIgf2dUW8urqSA4J5eNJkzE7HKgUigvu\nEAE4UFlFVm0NPf18qLLZ8VNp+HbKNMJ0Otyy7BGFush340JMi44hZ96PybdYiNTp0FxkYERySCiH\nbplLjd2Oz8x4KoKUHdpc0xno2mFYCzTdl7okPY3Fw3uy5SdjSL9/AjVuC8tzcgBQKRQEajQXNcgl\ne/fiq5XIejCFtPsm8NsJiTy/fy86pae5JFCjabOzbkqITneOs27K+fux9b5a4awFANSrg/lTxiG2\n3D2WDXeO4bv5o3hqTxpGu2cLnkGtvqizBvjFnlT+b3wCexZMIOuByQT6KPn9wQMoJAmVQkHAJb4b\nF0OhUNDLYLios25AkiSCtFp0GwsJr3Wz5pNoirPLKc6puKxrezvdxmG31PWVYzIx80xqQaNUMKV3\nCDkmY6vPedxYy8y4MHRn5s3N6hPeOGD3ahJe60a+gMyroPvSsPspr66OxCADvQI8hetBYX6E+mgo\nrq9v9bnMDiez4j0T0LUqBbP6hHPS2PrvxtVAs76ARdqdhNe6QZYpzqno0EayjqDbOGxo3vXVPyCQ\nT7OKkGUZk93JdyfKGBDY+rH2w4JD+OJ4CTVWj/P8JLOQgDPzGq8mmvUFV/0aAu9j7LOfIyHRx8+P\nY1VmMis8BbudhdXU2BxE+/pe4gxnCdCo+SSzAFmWMdqcrDxW3KbC+dVEs76A8Fo34TUuZKeD4pyK\nbhNxd8kctux2YywvQqlSow8O56cfPNZiC/qrI6/jji2b+PxIMSa7k1nRMQwLDsbkcLTYTHA+vxk2\njN3rS+n3n82NRcZPJk5un/cgyxRaLLhlmZ56/WXffgq6Fg5rPabKEgzB4Wh8mhf7GnRxXhkxkpn/\n3UOYj4Yqq4NXRoyi1FqPTqlE1Yo03XvjJnH7lo0sPVxAnd1JHz9//pA84pLHtQary0VBXR1hOt1l\nBzgNQUv4VI+OfFmAojHajujVdQW8upzDttWZWPP645jKCnE67MQMvI4HDW6iQ5rv0oj09WXD9Jmc\nNpvJqK7m2X17Sa0oo8Jq59URI5l3iZ0dkiQxJiycY7W1uN0yPbQ6YtpQMb8QVpeLB3ZsY39VBQpJ\non9AEMsmTEJ/Xs6vuxdguht5h3ax7q3n0fv5Y6qtZvKDvyZhzI3A2XRIA3N79WZKZBRFFgtLTxzn\n6b1pnskxKg2fplxP1CWi7UhfXyJ1PlTYrUh47Lw9goaGnR8+KgUV9Xb+b+gwFiRcvmZHU8e9bHUP\nZt1VcE6apKs57y6XEkld/jf6Jibx9vdpvP19GlprDR81URE7H5VCQZSvL8/u28t7Nw1m/8IJ/PDj\nUTy3L53CupabChr4Nj+freVFZD04mZOLr+f2QRE8uWfXFb+Hv2VloNG6OPJgClkPTiLUT+LVjEPn\nvGaAKRc6aFub4Npjt1pY99bz/PL1f/HGt9t58T8r2PreHzBXlQFnm8Ga4q/RcKS2hn01ZRx+YCKH\nH5jI7KRQfrln9yWv9+SeXczpH87Jxddz5KEUdleV8FXe6St6D7Iss3DHNl67oR8HFk5k20/G8mrm\nYY7UXHkeuml+e/WHPVj9YY8z6ZJySk7XNP54O13OYVflnyTllvkoFAo0Wh1jZs8nx3XxG4miegt6\ntZKUnp6BogNC/RgY6seJSxRZsmqqualPOEE6z/amuwdGk9kOxpdZU83tAyJRKxWoFAru6B9JZk31\nOa8xrr14lV/QtTBXFGMICKT/CI8gUa/EAUTFJVBTfNaJNkxEb0pGdTW3JIQToD1jowOim9lSS2TV\n1PCTgdFIkkSAVs3shHAyqi993MWosdsx2h3M7utpfokL8GVsVBBHa9vPkTY47kXancSsymPNJzHn\nOPCmztsbHXiXc9j+PWLYt20DAG63m8Nb19HH5+JCSD10PtTYHBwu9zjoQpOVI5WmCzbLNBDn58fm\nvCqsTs+YpLW55cRd4pjWEGfwY21OBbIsI8syP+RWEGfwa/G13mh0grajDw7HWF1J/smjAJQXFVB8\n6iT+YdHN0iFNifPzZ+Npj3wvnLFRv5Zt6dzj/PghpxwAm9PNptNVxPn5X9F7CNBo0CgUpBZ6HH9l\nvZ29pTX0voBttwcNznuRdidrlp913hdy4B3lyFt77S7X6VhXXc7XLy/G398HZ42RKAn+O24cvpfY\n8/l13mme2buH/iEGjlWZeXzAIBYn9bvoMS63m5/u2sm+qgoi9FryjVY+n3w9/QJalzcrra/HT60+\nZ20ut5sSq5X7t2/FJjs8OiUuBSuvn0Kw9txxRZ1hbFhH0R07HY/v/J5ty/5EdHwSRbnHGXnbYoZM\nu52Hl/0MlaRuMcJ2ud08lLqdwzVVhPtqKTRZWTH5BhL8Ay56rRPGWuZv3ki0n44yi43BgcH8e+yE\nVvUVuN1uiuvrifTxOafj1uJ0sq20hF+k7SIp2MDJ6jru65PAMy2IT10L3rWNu+Bzs+4qQFJdu74G\n2elgzXJPAfVXX/yse8mrOu1Wxr71EDqlhomxia2qigMUWyycMBrpqde3KgoBT17ucHU1JoeDwUFB\n+Lei6n3abGbW+u+ptHoaGcaHh7Pqhqksz8nm+fR0ZGTi/Qz8cuAQwnQ6hgUHo22h0eH8NvXuRHd0\n2ADmylKqi07h3yOGgPDoRl33iw0qkGWZQ9XVmB0OhgQHt2oHFIDJ4eBQVRUGtZohQUGtKjq+lZXF\nHzMO4pZBpZB4bcR13BwbyyOpO9hYXIwM3BkXx+yesUT56unrf2VR+9XiYs78arFIuxOAsE+Xd6/W\n9CT1MkYEB7R52kakry+RbdirCp4v7/nTpi/F/M3rGRrpw09HxWG0uXhm3WkeTd3J1rJittw1hr5B\nvryWlsM/jmWxeur0C57n/DZ1QdfHENIDQ8hZASTzkcxLTk2SJOmy9lD7qdWtlgQGj477KxkHeSEl\nhiE99OzMN/JU6m52V5Sh83GR/8gUzA4n877Yx5C6ECZFRF76pB1Eg/PsbHS5HDZceFJ0Z6HUauW2\nASEoFRJBPiqm9wlkV3kZM+LCSAj27Ll+fGQc6RVVLcq6nk93aRoQNOfGDzIueyZpe7OpuJgYfy1D\neni2to7r6Y+vRsnO8lIeTe6FRqkgWKfhvsHR7K0s7+DVeiddzmE3yKd2ZnRKJUfKPW3Cblkms9xC\nqM6H/WVG7C5PcSituIZwH12Lsq5NCa91X/X1CjonFys2dgSJ/gEUm+0YbZ4ifFmdA5PNRbSvnt3F\nnmKaLMukFdUSqWvbnazAQ5dLiXT26Brg98NG8sSeXWzPM1JpcVJrdZM6azrPpO9h0ie76BesZ2tB\nFX8ffemZcqGK05TJrZduFXQdzEcym+297kimREUxNCiYn63OoV+oDxllFmbGRPPc4GHM3bSerXnV\nGO1OjPUufndDx8xL9Ha6lMNuSeCpMzI/Lo5+gYEsz8kmMFjDo/36YdBo+Pe4CWwrLaXCauW5fqNa\nVfgUeezuy40fZBDcyYKTb6ZM45Psk+yvrOLBUT24tZdnfZtnzGJbaSlahZLJERH4tEKpT9CcLvOp\nNdwedvbouoHBQUH8ccS5wu0KSWrTQNOmiDZ1QWfh7j59ufu8oUchWh23xnrHd7Mz02Vy2A1KZd0R\nTx5btKl3J7zlblLQvnQJh90wZaOzVMuvNZr1BcJfd0O85W5S0H50CYcNdFtnLRAIug9en8NuiK6v\nlNL6epakp5FRXU0vg4E/jbiu03ZhXYjinHIi47tf12N3o63pkLWFhfwp4yBmh4MZ0TH8asgwNJcY\nESbonHh1hN1QaLzS6NrldnPX1k0k9tDw5W3JzEoKZv7mjZiuwbiv9mKAKbejlyC4hrQ2HbKvsoLH\n9+zixUnxfHrrMI7UVfC7gweu8uoEVwuvdtgtaQBfDgUWC+U2Ky+OT6BPkJ6fDu9FtJ+Wg1VV7bDK\na4NxrVLksQXN+KGwkAWDo5naO4x+IQZeu6E/qwvzO3pZgsvEax12Q3TdkkJZW/FVqahzODHanQA4\nXG4q6u34qsRto6Bz0dYUoK9KRbHZ1vh7sdnWbHKRwHvw2v+59oquAcJ0Oub3jmPOqnTmJISzOa+K\nvoYAhgWHtMv5rxVrlscw6+4Ckcfu4rQlBXh3fB+mrzvBz9dl0tNfx38O5vP74SMvfaCgU+KVEXaD\nXkh7RNcN/DF5JA/FD6CsUsmsHvG8P2HSJXU8OhuLtDtFWkRwDqE6Hd/fOIMoZTC1NWreHTuROaKB\nxWvxygj7auiFSJLE/Lg45iN0OQRdizCdjicHDuroZQjaAa+LsL1BjU8guBq01xZWgffidQ7bG9T4\nOpI1y2MozhZaw10V0SDWvfEqhy30Ey5NZ52UIRAIrhyvctgg9BNai5im3rUY2m+bSIcIvMdhi/xd\n62mYvizoOox99vOOXoKgE+A1DhtE/q4tyE7vaasXtA5h/wKvcNgium4bIo/dtehssxsFHUend9jt\nJfDUHRF57K5DZ5rdKOg4Or3Dbs8W9O5EeK0b2eXs6GUI2gGRvxY00KkddnsKPHVLZNGn3lUQ3wEB\ndHKHLaLry0ezvqCjlyBoB0T+WtCUTuuwr4bAU3dE5LG9G/ORTBG0CBrptA5btKBfOSKP7f3c+EFG\nRy9B0InolA5bCDy1IyKP7fWIu0xBA53SYYvoun0QeWzvRgQugvPpdA5bCDy1P8U5FR29BMFlIAIX\nwfl0OocNQuCpPfFMUxdpEYGgK9CpHLaIrq8Swl8LBF2CTuWwQUTX7Y1xrZj87o2I/LWgJTqNwxYC\nT1cXkcf2LkT+WtASncZhgxB4uloMMOWK7X0CQRegUzhsEV1fXURaRCDoGnS4wxbyqQKBQNA6Otxh\nC4Gna4fIY3sHouAouBAd6rCFfOq1Q+SxvQdRcBRciA512CK6vnaIPLZA4P10mMMW8qkdg5Bb7dwI\n/WvBxegwhy1u+649a5bHILvENPXOzNhnPxc7pgQXpEMctiiqdAyLtDtFm7oXIHZMCS5EhzhsEV0L\nBAJB27nmDlsIPHU8YnufQOCddEiELaLrjiO81o3Ii3RORMFRcCmuqcMW0XXHo1lfIPx1J0VscxVc\nimseYYvoWiC4MGKbq+BiXDOHLQSeOg9rlseIPLZA4IVcE4ctBJ46FwtuykPkRQQC7+OaOGzRDNC5\nEHnszofoTRC0hqvusEV0LRC0DlFwFFyKq+6wReW7c+LJY5d39DIEZ7jxg4yOXoLAC7iqDlsIPHVe\nnpq9TaRFOhnieyK4FFfVYYsW9M6LkFsVCLyPDp84IxAIBILWcVUdtoiuBYJLIzqABa1FRNjdmDXL\nYyjOFoXHzoAIbgStQTjsbswi7c6OXoJAIGgDwmELBAKBlyActkAgEHgL8lUiJSVFxrPTV/yIn3b/\nSUlJuVqmK2xb/HToz8VsW5JlWUYgEAgEnR6REhEIBAIvQThsgUAg8BKEwxYIBAIvQTjs83j55ZcZ\nNGgQQ4cOZfjw4aSlpbXr+Tdv3szs2bNb/fiV8vXXX3PkyJHG3ydPnkx6enq7X0fQ+RG27f2oOnoB\nnYnU1FRWr17N/v37UavVVFVVYbPZOnpZV8SXX37J7Nmz6d+/PwCSJAZJdEeEbXcNRITdhJKSEkJD\nQ1GrPfrdwcHBREZGApCens7kyZMZOXIkM2bMoKSkBPD8VX/88ccZPnw4gwcPZs+ePQCkpaUxbtw4\nkpOTGT9+PMePH2/1Ourq6li4cCGjR48mOTmZb775BoBly5Yxb948Zs6cSWJiIkuWLGk8ZunSpSQl\nJTF69GgWLVrEY489RmpqKt9++y1PP/00ycnJ5OTkALBixQpGjx5NUlIS27dvv/IPTtDpEbbdRbhW\ne1e9AbPZLA8bNkxOTEyUH3nkEXnLli2yLMuy3W6Xx44dK1dUVMiyLMufffaZvHDhQlmWZXny5Mny\nokWLZFmW5a1bt8qDBg2SZVmWjUaj7HQ6ZVmW5XXr1sm33XabLMuyvGnTJvnmm29udu2mjz/33HPy\nxx9/LMuyLFdXV8uJiYlyXV2d/P7778vx8fGy0WiUrVar3KtXL7mgoEAuLCyUe/fuLVdXV8sOh0Oe\nOHGi/Nhjj8myLMsLFiyQV61a1XidyZMny0899ZQsy7K8Zs0aeerUqe34CQo6K8K2uwYiJdIEvV5P\neno627ZtY9OmTdx+++288sorjBgxgszMTKZOnQqAy+UiKuqs2Pydd94JwMSJEzEajRiNRmpra7n3\n3ns5efIkkiThcDhavY61a9fy7bff8tprrwFgs9nIy8tDkiSmTJmCn58fAAMGDODUqVOUl5eTkpJC\nYGAgAPPnzz8n6pHP22o/b948AJKTkzl16lQbPyWBNyJsu2sgHPZ5KBQKUlJSSElJYfDgwXzwwQeM\nGDGCgQMHsnNn68WSXnjhBaZMmcKXX37J6dOnmTx5cpvW8cUXX5CQkHDOY7t370ar1Tb+rlQqcTqd\nzXJ35xvx+c83nKPheEH3QNi29yNy2E04fvw4J06caPx9//799O7dm6SkJMrLy9m1axcADoeDrKys\nxtd9/vnnAGzfvp3AwED8/f0xGo2Nkcr777/fpnVMnz6dN99885x1QHNjBY/Bjho1ii1btlBTU4PT\n6WTVqlWNhuzn54fRaGzT9QVdD2HbXQPhsJtgNptZsGABAwcOZOjQoRw9epSXXnoJtVrNypUrWbJk\nCcOGDWP48OGkpqY2HqfT6UhOTuaRRx5h6dKlADzzzDM899xzJCcn43K5zokEWqpmS5LU+PgLL7yA\nw+FgyJAhDBo0iBdffLHZa5oSFRXF888/z3XXXceECROIi4sjICAAgDvuuINXX32VESNGNBZmzr+u\noOsjbLtrILRErpDrr7+e119/neTk5A5dR11dHXq9HqfTybx583jggQeYM2dOh65J4N0I2+58iAi7\ni/DSSy81br+Kj4/vtgYt6HoI2z6LiLAFAoHASxARtkAgEHgJwmELBAKBlyActkAgEHgJwmELBAKB\nlyActkAgEHgJwmELBAKBl/D/o5lQZwgfG5IAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x107a9f6d0>"
       ]
      }
     ],
     "prompt_number": 20
    }
   ],
   "metadata": {}
  }
 ]
}