ch15-autoencoders.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import numpy.random as rnd\n",
    "import tensorflow as tf\n",
    "import sys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Representations\n",
    "* Much easier to remember *sequence patterns* than to remember exact lists. First studied as chess game positions (1970s).\n",
    "* Autoencoder converts inputs to internal shorthand, then returns best-guess similarity. Two parts: *encoder* (recognizer) & *decoder* (generator, aka *reconstructor*).\n",
    "* Reconstruction loss - penalizes model when reconstructions /= inputs.\n",
    "* Internal representation = lower dimensionality, so AE is forced to learn most important features in inputs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PCA with Undercomplete Linear Autoencoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXmQJFl+1/l5fseRd2WdPdXHjJgZScugY0ESSBy7LItY\n2S7CTGJhgQVsgd3FQDIME7aAWANWmJDAmEXIFjGDkAToGEADMiE0Olpznz19zPT09N1VlVlXHnFH\n+Pne/vHe83CPjMjKqq7q6uqOr9lYT2VGur9wf/717/v+jieUUiyxxBJLLPHWgXO/B7DEEkssscTd\nxZLYl1hiiSXeYlgS+xJLLLHEWwxLYl9iiSWWeIthSexLLLHEEm8xLIl9iSWWWOIthiWxL7HEEku8\nxbAk9iWWWGKJtxiWxL7EEkss8RaDdz9OeurUKfXII4/cj1MvscQSSzyweOKJJ/aVUtu3+tx9IfZH\nHnmEL3zhC/fj1EssscQSDyyEEJdO8rmlFbPEEkss8RbDktiXWGKJJd5iWBL7EkssscRbDK+b2IUQ\n7xBCPC6E+IoQ4lkhxF+9GwNbYokllljiznA3gqc58NeUUl8UQqwATwghfk0p9ZW7cOwlllhiiSVu\nE69bsSulrimlvmj+/wB4Drjweo+7xBJLLLHEneGueuxCiEeAbwA+ezePu8QSSyyxxMlx14hdCNEG\n/j3wfUqp/pzf/wUhxBeEEF/Y29u7W6d9oJEUCR9+6cMstydcYokl7ibuCrELIXw0qf8bpdR/mPcZ\npdRPKKW+WSn1zdvbtyycelvgE7uf4G9/8m/zQueF+z2U28LB5IDv+sXv4rXea/d7KEssscQc3I2s\nGAF8EHhOKfWPX/+Q3j6Y5BMAxvn4Po/k9rAz3OG1/mu82H3xfg9liSWWmIO7odh/N/CngD8ghHjK\n/O8778Jx3/JIixSAOI/v80huD4UsgAdv3Ess8XbB6053VEp9AhB3YSxvO1hiT4rkPo/k9lAoTex2\nxbHEEku8ubCsPL2PsIT+oCnfTGbAgzfuJZZ4u2BJ7PcRpRVTPFgEWVoxD9i4l1ji7YIlsd9HpNJY\nMfmDacUsFfsSS7w5sST2+4jSinnAlG8uc2DpsS+xxJsVS2K/j8iKB9OrzpUm9gfthbTEEm8XLIn9\nPsIq9gcuK8Z47A+ahbTEEm8XLIn9PsIGTx80S8NaMUvFvsQSb04sif0+YpnHvsQSS9wLLIn9PqLM\ninnAiL1U7A9YbGCJJd4uWBL7fcSDWqC0JPYllnhzY0ns9xFlVswD5lWXeewP2LiXWOLtgiWx30eU\nWTEPWHaJzYq5lce+M9hZ+vBLLHEfsCT2+4gHtkBJ3dqKkUryPb/0Pfz0sz/9Rg1riSWWMFgS+33E\ng9pM6yTpjoN0wCAbsDvcfaOGtcQSSxgsif0+4oEtUKr0ilm0rV836QJwGB++YeN6I3EYH3JlcOV+\nD2OJJeZiSez3EQ96d8dCFaV6n0Un7gBvXWJ//xffz1/5zb9yv4exxBJzsST2+4gHdQelKplPivnB\n0bc6sR9MDt6y322JBx9LYp/BB770AZ47eO4NOde9KlD64f/yVX78t166q8eswgZPYfFL6a1uxYyy\n0TLjZ4k3LZbEXoFUkvd/8f18+KUPvyHnqxYoLfKq7wSPf/UmP/e5e+f/WisGFhN7J9GKfZJPGGf3\nfrPuQhZ8/vrn7/l5LMb5mEk+QSr5hp1ziSVOiiWxV2CJ9urw6j0/l1SSXOb4jo9ClRky85AW6W0R\nf1pILh+OuTm4NxZPzYpZoFqtFQNvjGr/5NVP8ud+9c/x7MGz9/xcQPmyOqmNtjPY4fsf//4HznZ7\ns+C13mv8vU//vZqouN/4yLPX+Ys/84X7PYy5WBJ7BbZQaOcupOiluaQ7Thf/3vjrq8EqsJggR9mI\n7/j57+CjOx+9rXMDfPFS98R/czuwWTGwOPD7RhO7Pd8Lhy/c83OBvi+glftJ8OTNJ/n1y7++zKS5\nQ3zy6if5hRd+gZvjm/d7KCU+/NQuv/rsDeLszfOysXjbEPsoyfm+n3uSg+FiP3uYanK91Hv9xP7B\nT7zKd77/4wt/b1cHq+Fq7d+z6Cd9RtmIncHOic+dFZrYn7zcucUn7wxVxT5bNSulXll0ky6O0NPr\njoi98xpc+vSJP26J9pXeK7d/rjuAJfST2kz2mtm4yhK3B7vSsff5zYCnr/QA6E0Wr7bvF942xP7c\ntT4ffuoqX7y8WMXudPWNytSYftpf+Ll//Gsv8GO/+eKx57vRj7nai0uim4W1XqxiX9RWwH7udgKs\nVrE/cekeEXs1eFpR7Df7Mb/zh36dX37mGp2kw8WVi8AdEvtHfwQ+9GdO/HFLtG8EsSulSkI/qWK3\n99H2B1ri9mDn2TAb3ueRaOwPE3a7Wgh2x2++e/q2IfbEkJ0lvXm40p2S+XE++0df2ONjL+7f4ny2\niGf+Mq1U7NaKWZA2aAnBWjcngf2Oz+z2Fn7f672YJy7dmUVS9TmrFtKP/9bL7A9TXro5pBN3eGzt\nMeAOib2/A8ObUMzPk5/FMNUP/Cvde0/sk3yCQr+wT6rY7+Q+LjGFVexvRCD+JHhmZyoQj7Nc7xfe\nNsRufbBkAdEC7J6Q2NNclnbHItgXyTidf77SYw+PV+x3sltRVigePdUizSXPXu3N/cw//9jL/MWf\neeLEx6yiUAUNr6HHZR643e6Ef/vZywAM4oxu3OVc+xxNr8nB5OD2TzK4ASgYH/8CtbBL9N3h7j0P\nUFZV+qxij7OCK4dHyefNaMU8daXL9//8UwtXlW8mWAFxLxX7j/3mi+UcvhWsDQPQXVox9w8nUezX\nBoPp/x9dW/i5NC+OPU71PJNbEPvKlS8Ci4n7dpWeUoq0kHzLY5vAYjumP8kZJidTw/PG1PJbwJTY\n/+lvaGuqFbj0JjGDbMB6uM5mtHlnin14w/z3ZMEyS7AKxWuHz8Mn3w/ZvSH4qs87G/T+15+5xB9+\n/8ePkOWdKPYPfOkDXOnfu2DrJ17c4xef3GUQ39k8eCNhV7j30mP/8FNX+ZeffPVEn31mp8tK5AHQ\nm7ViupfhP/91uI+229uI2DXBpsco7esVYr/UWxysTHJ5S2I/sWK/+VVgcdrc7XrsWaEJ5cJ6g4c2\nGjy5IKYwyXKSXN5R/nwhC9p+G9AvpNf2R3zoiR3+xO+6yNm1iI4pTtoIN9hs3AGx5ylMzN+MTkbs\no2yE7/gAvPLlX4Bf+0F49WO1z6S55I/9xC/zhUsnWwUcdy6LWWtgpzNhmORH5pn11k+q2AfpQNdU\nvHzvaipG6a1XsW8WvBHB00la8NLN4S2tFaUUz+z0+D3vOgVAdzLz+Sf/DXzuJ2D/jcnQmoe3D7Fn\nsvbfedgb6mWeUoLXjiH2NJfHviDsZwDG6Xw1VHrsUtb+PQtLCCcldjuuwHP4xosbCxX7JC1QCvLZ\nZbhSMDk+TbJQRanYJ/mE9//Gi/iu4P/4/e9kJfLpmuKk9egOFbtV6wDDvYUf+9yrh/z+H/0tDkcp\no2zEu9bfhSMcXt412TQzNs6lzgHP+3+Ln332P93eeGZQJfNhWieavlmWz86z2w2eztZUxFlxx5ZJ\nIdVcITI2K7b4mGfizYKTEHsmM/7JE//k2MSHY89h7Nov3iKbbKcz4WCU8m3v3MJzxNHg6eVP6f9W\n5/EbjLcPsVsr5hhC3h8ZYs822D3OYy9OotjNZhSLFLtRbpbY75YVY8cVuA7f9PAG1/sxV7tHA7Pj\nUq3NfI8vfQh+9Gvg8mcWniOXOaEb4js+ncmIDz+1y5/+1kc4vRKxEnkMMu0/bkabbEVbr4/Yj1Hs\nz98Y8Or+iF999jrjbMxGtMHF9kO8OjQv5VGd2A8nPYSTsx+/vgeu6qt3JnXP16a+JUX9vufdSwCk\n45PFGyyRXR1epZCK3/cjv8VPfuq1Oxrv//sbL/JHf/yTR37+QCl283yM8sXE/kLnBT745Q/ymauL\n5+5xmBhi/8JrxxP7Mzt6fr/vHeusN/26x15ksGOKlo4RJfcabxtiL4OnC4oJ8kLSi/UDK5PT7E+u\nLzxWegIrJj2pFWMV+y3SHU8aPLVBXd/TxA7zfXZ7PY58j94VKFL40P9qAphHkcsc13GJvIjD8RCl\nKM+1EnmMcj3xrcfeiTu3V3o/qFz7Yzx2O/b//KVrjLIRLb/Fo26LVzyhPzBDosP07iznq3/fievE\nbh/y2euaGWspPWEw2M6P3eEur+wNud6PuXRwZ+O+dDDitf2jf2tXk0de7q8T/+OPfYIPfuJkXvVJ\nEZtYxihdfA3sNbuT3ktKqfKZ+MIt0oSf2ekSuA7vObvKWsOve+zXnga7olsq9nsPO3mTBYr9xiBB\nom+QTLcZF33GcQ/k0c/flse+4EUyS+z3QrG/5+wKDd+dS+zjRWotHQFC2zH/7s/ODQAVqsATHg23\nwcAUda03tL+9EvqMC03sG9EGm9EmhSroJ7exPLYPhNeA0WLVY8f+qZcPGKRDml6Tx8Z9Lvk+WWv7\niBUzyiyxv77MiqoV04uHsP8SfEXbO71FxG7v4wkbh9n5cHN8kyev6GswvMMg5ygtGM+xckbJ3Vfs\nSim+fLXPizcGt/7wbSA2c2I0WfxitIR+J83Z0kIiFbiO4Okr3WOz3p7e6fLe86sEnsN6M6BT9eQv\nGxtQuCeOD90LvI2I3Sr2+Tds53AMQj98XnEGgKv/6g/Bx/9R7XOFVNqzPKHHPrmVx26Osyh4atPk\nTqpC7Asl8Bw81+HrL6zOTXmcZAuuRzqCcBW+6/28evWz/JUPfeeRsRUyx+1eJnIDRqkmubWmJvZ2\n5JFI/VCvhWtsRjo757bsmOENQMD2u0+k2Aup6CcjWl6Dx26+RC4EV1ZOwbh+zpFR7HFxdxS7kh6D\ndAyf+Wfwi38JmBarzM6P3LwgsxOSjn2RKxSf39Hqd3CHWUzjNNehkxmRUSr2W3nsc8TNIozSgkKq\nhSvVW+EffeR5vnuObRQb+2vUW5wl9HraYMep/o7f8I51klzy9M7h3L0GCqn40k6P9z20BmhBU/PY\nL30aNh+DtQtH5+5oHz71Y7qq+h7j7UPs2fEe+253gnD0DdoILgBwtfcqdOpLyrTi1R+XUXLSrJgV\ndYvg6W0qdqs0Alff2o1mMDedrbRiZq9HOoKgCe/7Xj78rm/l8eQ6V68/WftIngzw9l8kLHLGRgWv\nNwL9fSKPnCErwQq+47PV2ALgIL6NXPbBdWidgtXzt1DsEt8VXNiIiIsRrfEh7xzpl9irjdYRj90S\neyJfX5GL9XlVvqKDp8ObkI1QeXJM8NTksZ8wBbO6gnv25mvA61DsRpmPZl4MU8V+DHEfvgo/dA6u\nPbPwI0op/uR//pN85LWPlN9/UdLAcfjYC3v80998iS9fPbq6i82LcXSMCn49Vox96f1uk+nyD77w\ng/zAx37gyOde2RsySgt++0PrgBY0ZUsBKbViv/ht0Dp9lNj3vgof+ZtweO+L6O4KsQsh/qUQ4qYQ\n4st343j3ArYCdJGFstOZgMhxhceZhiF2zzPWxBT27+dmlMz53EJiN8HTUCpC4XKt3+c7/uHjR4pb\nbBbFST32tKLYAVqhx2jOQ1ZaMfMUe6AzXj5tXnRZXM+SyVWOCzSEWy57141iX4l8hDtiLdAT/44V\ne/sstLaPVexJJgk9lz/09VsgFOH+qzyqXABe9r0jHvsk0w98Kl9fH/VJNkFJDyUbjLJxeZ5J/6B8\nUR5Jdyxf0Ce7j9WYi029vdO6A0uyo3SBYj/OiuldgTw+NpieFAnP7D3DkzefLEnudhX7wTDhr33o\naUDP4WLm2YqVIfZkfsGdHQfcmRVjhc7DW00e2mhwafiVuUWK44++n58P/m5FsQfT9Mj9F3Sa7sVv\ngfaZo3PX9qBafei2x3e7uFuK/V8B//1dOtY9QZnuuJDYxzRDSeSFXFg5jVCOJvaZPOVqtsNxPvst\ns2KMugiVIhQeX766z+XDMS/v1f3f2/bYizqxNwOXcVIfg1JqasXM89iDFp24w1fHukgrz+oPSiEL\nPKWIhENcxISeQ+RrQl0JPYQ7ou3riX9HxD64DitnoH1a++QLWrWmRUHoOXz7u1cAiG48R/OxP8DZ\n1llecY5WrdrVRc7JFftTV7o8d62uIIfZEGSIkoHOkDErg2Fver4jHrvpr5MuCJLPIqnEJDLnAM8R\nDOI7K3ixJHtEsS/KjKoit6T1/JFf/fSnX+PmIC6JtJN0Kop9zj370r+D53/lyI+VUvzAv/8SvXHG\n//Q7zpsx1f8+Nh1FRypbaGXYwHTSv/0mfvZ5aPgu33CxRaIO59YcyBtf4X3iZR7b1nUc602fUWoK\nFm2a48PfpufubPC0b7K11i7c9vhuF3eF2JVSHwPe1FvlTCtP55PEbndCK1KEbsjZtSaNrMFVz4UZ\nUqsq3OOI/aRZMYFShMLhlYNubZwWt12gZP7edyvEPjMGXZi04DukIwjafPb6Z8t+KNnMua1ij5Qg\nKWLWTOAUtBUjvBFN1yiacB2BuE3FftMo9tOg5BGvvPwemST0HB49pSsAN5IevOeP8M61d/KKSiDu\n1YK/E0OqBSdXdH/nPz3LD/+Xr9Z+NkhHKBmCDEjySfkCGXenK4TZ61q2FDhprCTVcQpXgeN3eN87\n1l+HYtf3f/bvJ4tWbRXIbMwvtZrke/VrcHMQ84P/8Vn+45NXS0+7G3dLxT5X0Dz+/8Cn/9mRH3/o\niR1+/bkb/MAffg/fcHHjyN8rpYjNXBwJB16d3zU1MUQaj24/G8USexS4PHYuAaGY9OdUn2cxkchw\nzX20K9XeJNP+euu09tjbp/VKrtrrqLcL0Xq5Ir6XeMM8diHEXxBCfEEI8YW9vTc+v7MMnh5jxTRC\nSeRGnFmNWM08rs2zYipL7OMCqPY8k+z44KkPeFIT5Lzx3a5iT44odo/JTEZE9aE5cj3SIQStWi5w\nPqMyS8WOIJNpObnBWjFjmp7ugeM6LhvRBoeTExK7lDqboH0a2tv6Zwt81SSXBJ7DpNAKvCUlozPf\nyKNrj/Jq1kdC7aUQm++hxMmDa9/d/xl+38Ev1H42SscoGaBkSCInMNFZR/FgSuxH76OeB/lJid2s\nFN+RZTSDG3zDO9bvuPTfKvWq762UKi26RY3qAL7Uf4X/6/QpPterb7Vox9IZp2UDu8P4kL75+Xh2\n3mcxu4MrXJ9TPPSJF/c5vxbxZ7/tESLfMWOaXr+qqBm5zpGKYgubcXQkeCoL+Mk/Ai/9xsLvGVcU\n++a6fqlmc7KnHGnGYuxJK2p6k1TbVQ9/Kwih5+9sr6P+LqzdexsG3kBiV0r9hFLqm5VS37y9vf1G\nnbaEra6bp7KlVFztTgh9SeiFnFkNOZ0rdudYMdW/X6TYC6lK//04jz1AIAByEI7NUKh//rYrTyvp\njqAVO9QzIqopmEeJfYTym3zm2mfY9FfMGOoPSq4kHhApRa6SMnAK0ApdhDsiECvlz46tPv3sT8Dn\n/sX03+MDkDmsGMUOC/OB01x77DZLpSUlLw4bPLb+GLHKuea5NZ/dEjtOTLzghTuLb0s/zbdM6kQy\nSLUVgwwoKn59Mpx+x1krITNWwklbCiTmOz2aZQTBIetN/0RptrMopCrv8bBiycXZdNV2nGK3qZ2D\npFd7SdqXRXeSlUTaSTqLFfvhK/ytrQ3+DvNrKlYbPo4jSkuv+vf2+OvCYyIExasfgzmJCyWxz8Yx\n4h5c+gTsfnHh97TEHvku0tPCM5lzDtcKrNjUajT13B/tXYLeZR04hcrcnYqSordDtnJ+4RjuJt4+\nWTHHBE9vDhKyQuF7OZEbcXY14kKeceC5xMcR+wLFXv3McVZMiCmkySRbK8KMczZNLjE/v70CpVKx\nh9qmqAZQ64r9qMd+xXPZHe7ye7Z+ux5DPmvFFLhK0VCKQiWsVqwY388QTo7PClz/MiSD44n98x+o\nL8+HpjipbTx2WFjBl+QFoe+UlaCBFFxPQh5eeRiAK55XU0yW2IWQHIxOlvLoq4wtOZsPP0YZj71Q\n0/tSjKakddRjt1lIJ/PJ7bx7LMsYuQmRr48365PfClWVPq78bXU+HBc8tWQ5dpxa7xNr63TH6Vwr\n5si833+BQ9flgKPnSnJtqQElsVd3JbJEfcrVHUVH4xtwcHSzdvuMxLMiyD7Dx4ijiUl3bPguVwa6\nw2Mq5hC7Veym7Yat33CufFb//OFv1f9t65TpKrE/N77GNxZf5WM781ccdxNvI2JfHDzd6egb77oF\noRtyZjXiMTOhr830SU9OoNjTXOK2nyM696Fjg6eBIXZfwqkVZ+74MqMMclXMzaudd26oKPZ5CugW\niv0zJrj47ae/UZ975oEoVKE9dqmQ1K0YiV6+OrIFH/zv4Of/1GJiV0p3wuu8Wiqgstp1xWTFwPFW\njOuUij2TLfZG6bTzpHDqir3yPW4OF2dXVOGRsak6Na90nGtiR4XkYnrMYlwh9tk8dmkVe53YlVL8\n8iu/fKSHjCXURqpXPkLalsi3S+zTe1312KsB9eOCp5Ysx47Q6XoGNlWyM8rK4GlcxByOtY0xyYp6\nOvD+C4wcwXAO4+gXtJ6njXnEbmyPLU8HLMfCgVePbhVpx5rM2pam1oJjAtfV4Onlvr7W817B3owV\ns2EUu7Bp0dvv1f+dtRHTMT2TJmv3YLiXuFvpjj8LfBp4txBiRwjx5+/Gce8mkmOsmJ2OnpiOkxF6\nIadXQ95TaC/wmpp94CTC30f4BwuJPSkKvNZL+OtPMErnL72TIiEw874pBIE/P/WsGrg8ic9+NN1R\nPyijyoNcfdhr30EpSId8Ou9yrnWOx1YfMWOYKVBSUnvsskCJjDXTvhQgM8TuZy5kI3jlcTb71+fn\nsY/2wKamXTeZstZ2aZ+BaA3cYGHKY5pLQt8pe3SnRYv9QULohgDEjqjlslev3974ZMQeqAwXWbOD\nJvkIZEDLb6CEnBJAPD3mEcWOmX8zxP5853n+xsf/Bp+8Wi/KifMYXyleTTRRhB2t8gbJ7WXGVBV+\ndQ7UFfsxSQA2hdBxYW+q2GtWTGVVe2DsGqVmmovtv8BYOIwcjtgo8xX79G9jcw83A/2SG62enRtA\nLcc6uzG8bQFxzGppasU4XOqbvj6OOFKr4ikzh4xit4V5anwIwQp4xpactRH7u3Qd/R3tHgz3Encr\nK+Z/VkqdU0r5SqmHlFIfvBvHvZuIy+DpUQVtt7hSIiN0Q0KZ8O5cK49dkdcmYloUROd+kejsf1po\nxSSZLKtYh9n80uo4T/CND7/mOeWkPFLYcpvEXvaKMYq9EWjSrQZxJ4sUe5FSqILPJnt8y7lvwfcj\nYFo1aZEriacgLDKEULQb09+NCj3h/cRcM7/J5qufYpAOjnY27Fya/v/rpgDGWDF7rPP8jaF+QBYU\nKSXGY7c+cMYae8MpsSdC1HzhqpLbPyGx+4a2VSWFbmIU+6rJbhg7AtwQJ+6Wwb9FVkw2s+oamxXF\naFDPmU6LmFApLuVfp//d1Y2lbrdIqfoSr9kyVWI/ZjNmm3c/bp2qKfaqFTOppGZ20umqpRas3X+e\nsSMYOA5qpkgrNvUIMFXs1Tkaj/X93zKps8PzvwNe+/iRitiyQGmW2NNbWzGW2IWTcX18ncgce3aV\n7Mu6x74SejgCnPgQmhvTD4Zt8FtTG7G3Q88Q+3q4vnAcdwtvHyvmWMU+5lQ7IJWGFPq7bBcFrkIH\n4Copj2kuEe4Y4UwWWzGFRBhiH+f1LIAb/Zi/+nNP8pGv7IJ5yWx4jlbwrnPUiqmQ0UkCqMmsYg+O\nKvbJIn81HfGVIGAgU771/LfimV2SZq2YXElcFEGuv2Mrmo65ZwpIQkso/83fYdO8wA5n+3yYjodK\nONPKxsENCFd5/8d2+PM/9Xm9pF2g2JO8qFkx0tvQit0zxB40ax57Jqff42By6941eSEJDLHnXZ2D\nrJQiKSYoGbLe0NbARDiw8Qhe2uf0SmTGNkvsZv6pOlHEB3qDksnec/Wf5zGhVDy0dh4PQXf4Eh63\nvzlKVbEPF6j3YxW7sS9Grc2ax26P2xmnxJVMl14yLWYrXypSkuy/RCEEhRCMxvUXtY2VANOsmJoV\no1/OWw1tb4xOv1tbbDNV4fbFHc9alvb5zRcLI2tVHiQ6xfGxbH7SQqnYjRXjOIK1ho+XdKGxUfts\nLZe9v0vPiK0Hxop5EHBc296dzoQLG03SIiVyI+jt4AEt6XDo1ok9ySWIHJx0sRWTSTBZLpOinjL1\nkWev8x+fusrWikPb0R57S1AW+hy1Ym6P2O33s0vbhiH2qnqqqqHad0iHfCnUS8lvOvNN+F7zyBgA\nCoxiN5O/EU6P0Ym1YotsMc3Z/4rN9/0JAA6f+un6YE2hyVf9r4XrX9I/G16H9hk6o4zDUWoU+/FW\nzCgb0ZCKPNyqK/agVfPYqyue3rgDH/4/de7xAiS5JLTE3tHEnhQJEgkyYLNhrIFoDVqnCPI+m60A\nR8zJYzd52Kmq39/U5KvHs/USeUykFA+f3uJcuMF1UfBN4sXbJvZqBlSV5McntGIsWY7Dtq5CTYa1\nY8WZZFh5SQ6zHsLkBJTzrL/LuGLn9Uf1zqlJJomMYo/mKXaTKrvVOqvP7YfmD+urYZtxFM9c46kV\nc4xizws8R7A70r1o3pXa1hD15IlA1YOnoDNjgrQLDb2i6IxS/vlHX0a1z0yJvbdL33Fo+y08x+Ne\n4+1D7MWI6MK/JZFHN5HY6Ux4aKOhVZKnFTuAkD6xENOJgX4IhJMhxNFdciyqij2RdWVo83wf3Q6J\nzMMeSUmSJ4T+61fsWa6Paa2YlrFiqktymwFgv8904COd/QBkaYQ3x4opZIECXBShUXNRMD12J+mA\ncghjQ1SNdba+9o8BcHhpxhftXqIr1njGeS/sPaeDW4MbsHKWUZozTgtk+/QxWTEmeJoOaElJHm2x\nXyP2Zs1jzypL9JWdX4an/rXuP78ASZrhCdNorKvnxLQBWMiplib2QbgG0TpRPmC96RN4ztGWAqVi\nr/88NptwxzNl8OM8JlCKM5trnF9/jB0/4P/2fwp1cHt9RmyQ1HNEraWAVezBHDFRhSXLcWD8NqPa\nq6mT3YrzXpAjAAAgAElEQVStNS56bLVCcw7z8jA2jMVgpiI4MS9omBJ7Mi94alIFR/YazmSKlYp9\nNvPmJMHTVBJVAqdfY4g9m3l52BUclTYbaw2fRt6Dpib2//DkLv/gV77KyN+c2oj9Hbphk7U3wIaB\ntxGxp+4l/NVnyIL68k1KxW53wkPrDZIiMYpdP8S59LVPm07f2mlu/PNjFXtRKvZUDWsBmP4kI/Qc\nMpkSGB8vlIVR7O4Rjz2vkNGJgqdFgesIXPMg2Tz20QKvNZ0h9okQoGB/UOAbKyar5F4XRg15CiKz\ntA39CrHHHVzaeLanR2ODTdMI7LA3s1Fw5xJXxWm+oh7Rues3nzN9Yk6XpJBGp/TDsaB9cug7jCeH\ntJREtE+xN0gIHL3qSPxGzWPPZYpQ+mW1cfAJ/cOZisra8ZPpfVd97YGXe6vKkDNtvaTuBSvQWKcp\nB6w3fALXmRM8NVW8s4rdKMLZ/iZxnhIphRc2eGj1Ya62NzgnDvjOT30vPHvy7fJskPRUO5yr2Lda\nwbE7KNn0zLFrMp/2dGuB6rEG8QBHKTwFsexzbk1f4zITa/9FXTFq0J9nxcysMKtjmhg/+9TqRX1u\n5hO7DUwfyT8vFfsxVkxWaGIfXGbDa7NlWockaX3FHVD32EFXnzaLQanYbVuQONyqKfae32AtXFs4\nhruJB5fY8/TEO5RIqSgc/eYt1LhOtHFGmktOr0aaXN0Q+jtMgi1UqdjrxC6EUezHeOw2eIo7rqni\nfpyx2vB1uqNRHpGUZDIj8OZkxVSI/URWjFGxFjaPveqrW/9yJfJmPPYhsSMQyuN6P8E3aYPVl4sN\nJnkomiYIZjN6ALpJl0Cs4KcVYq/2i6lW8nYv8SJbPCX1Epvrz5QNwKyijIMtUEVZ3Tn7XUPPZZT0\naEqF194mziTjTBK6IYkf1jz2XKX4SqvsnhfCe78Lbn5lbrELQBZPeC7wedn3cExws+zFLgPOreiH\ntOs1IVqjrYasNXwCzz268rJWDDPNrQzpxLJ+b+MiJVAK3w853z7PQTbkf8j/LvuNR+BDfwa+OGNr\nLYDNXT+9Wid2+6LfaAbHKvbEKnYBOH75IhymVe9+SKQU6wpyhpw1xF6uEvdfYBBOC9a6M1XISSV4\nGhmCrwX4jV21ufoOPXabqTYThLXxi1io+sYuJ1DscVbQCBwu9y/zjmiT0MyJuLqPgJQEmO896TJI\nBzx/+DwbkUNbDUvF/tJNTexjf0vP2zzVHrvnsxYsif14fP4D8OO/a+FDWUVaSIRr3rzupLZMnuav\nmi3fvBB6u6Stc6COEnuSG//cSY/NihFCTwDhjms2SH+Ssxp5ukDJ5DZHJkfa9/LXb8UUCt+dLntt\nlsFsuqPnCNqhV18hpCMSIRDS43ovxvOOWjG5eXhcBQ3jm3peXbGHYoUw64Ebgt+g5bcIhMeB604V\nsiygt8O/ODXk5Y1f06lilz6lr/XKmVJRjn39sMzz2ZO8IPCmVky4rotC9gYJgRsQe4H22M0cyVXG\nukyIpOTplW+CR75dP3iL0imTmL+/tck/3NzAG+qgWtWKOb+ul9Udt4EM12gRsxE582Ml9pgzVoz1\ncOMZ0kmkVux+2OR8W1sQg1bIB975z3Qq6KVPzR3zLKzHvt0O66u2JEcIbSMcV3lqVfAkj2HrnaUV\nM0ryUkCM0zGRUmwUBcIbcXY1qp2bvRfotKcVl71KTYM0exvYoKnnOviuqAdPDbGvhmv4js/INoWb\nWeUklaBp7VkpC5QWK/Y4K2j4Lpf6l3jYXytTkUdxxYqprhDiLj/17E/xx3/5j7Pq7+GgpordEPug\nnLt7WrE7LBX7LTG4Ni0/vwWSTCI8/UAKd1IjT7tcdF1Dsm6kPfbVh1AyOGLFxFmKEBLh5CQLytLT\nQoJpeauJffq5UrHLFN9M0NB8h8CXc3uMBCar5KRZMYFRP6B3hIl8p6aAJllBI3AJZ73g0orxud6P\n8czyu7pqKMyYXRQt86A4zvSB6SQdmu4qUT4oswSEEGw3trjuudpuAehfBZlzPYxRwS7qzNfDi7+m\nf9c+W3q4A89kGsyQr5SKrFCEnsMoG9JSiua6Vv77Q22ppa6vH2brk6qYM/kBgfK47J6C7ffon9/8\nytxrmaUTxo5gxwvwxzdAytKKcVXE+bZWoV0RkJjeONv+RF/XI8FTc4lnzpHY4h45S+wZoVJ4QciF\ntu4GGDV6dDOhG0nNBPUWYZwUOAI2W8ERxd70XaI5cZ0qUjM3x/lYb3xiXsyjJOf8uiXwMQ2p2Mgz\nHHeq2MtV4v4LHDZOl8fsTir5/mWwfzpnI8+dma/62Q3dkLbfZmStwZmXYTXjqLbVpF0l3qJAKfAL\nboxvcNFtERgxMIqnil1WVwiTLpf7l8llzs3itwAoonU6o5SDkR5fzzFz9+AlSAf0VLEk9lvCvn1P\nUGof5wXCNcTuxLWHznp5rqsnReiG0NvB33wIqXxd5FJ5iKobGY+z+RMlyYuaYpc7T8AvfR8oRT/O\nWYl8HSw1kycyaYO+lx/tFSMz2rfYjKP2+WJa7GHRCrzaQz1JtToJPGdGsQ+JhUDJgOu9GEc4uErV\ncq+tx+4raJjxJxVS6sZd2v4aTTlANaaBootrj3I5CKbE3r3EWAgSL0V4fYbb753aJu3T5cuw55hj\nzOSyVwlhlE9oSsnaqXOAVuyhGxLb7AOTGRPIAZEqkKwQyzGc/lr9+wU+e57EZEKw57o4KofRXqnY\nQ6fBaVd//w4+I0enPp5yJzp4WpljhSyQtnvETJn6wOyz24lnAoEyJ1SKIGyUxO6HXZ3H7jeOdB09\ngk/8E/il72OU5rQCj1bo1StP05xm6BH57vFWjLJNvcZw6t06kymLGSYFFzZ0DCYpYhpKslEUOG5F\nsafGQhvdZD+YpgJ2K+mRdv5V52wUuDWPPcnGBEo3lGv6zSmxz2YSVdo71/rFnECxT9ICJ9AriYed\nkMBYZpNkah2mJuaSOSHEPa6N9CruK+mnkcDYW+OlStvtjjBz9+oXkUBfpktivyXs2/eY3FQLrdj1\nBRduPf/cKgPH1eQaygLSIdHWwygVHrFiJpW3/nhBQ/80ryv28KVfgSd+EpIBg0nGauSRyUSrAuEQ\nGavD94ujvWJkTkvaVM2TVZ5WrRjQAalJWvDcwXN8fOfjTLKCZuDqYG3NYx8TOw5SBlzr6e/mMd9j\nd5Uisj6kebkWsqCX9lgJ1llVQ1RUIfbVh7nsB6gbRh13LuleLoAQildXpz2qZetMaV91LLHPKHZL\nCIHnMCoSWgo2N/XuNzYzJjEZPpbYfUbmmq/qXZTa29DcWqjY8zQmFYKJC2MhdNqemQuB26RlSL6j\nXAZCxyM2nPGRrBi74vGUOqLYR4acxjP3NlV6pRaGEacap/AdHzfo6JYCfvPWxH7pk/DK44yTgmbo\n0gp1+2YbXxolBS2zajtWsasZxa4kHLzEKMk51Q6JfIe4SLQVI/XK+FzVY9/Xefr7ZkUjlGKQTu0N\nWzhos2JA57LXrJg8JjTB17bfZmhJezZ4isSdmZP6FyezYoSnhcVF5RLKecRu7lV4GtIB10bXWAvX\nOJQdPheF9FkpbRiAg5LYn2ToCCRq6bHfEgsUe5zHfPRKvY9Ekhc47pTYk5piN9aCyWIJzSRw1i/g\nOg1txVSJvbIcmyxYLST5NN1RuNONGMjGpRWTmOAY4SqheTl5bl6b0KA3aGjfhhWTmla2VbQCvYvS\nj33mh/j7n/ibTFKdAXAkLS8dEQuBVCE3+vpcnqpXS5bEju4VA1N11Ek6SCVZDzdZFyMyf1qI8fDq\nwwyEorNv1HH3Epf9aY+ZF4JpcG0cTbt/doqWDtrNdHi0L6TQcxjLjLbbYLMd4gij2L1Q3zsoid0T\nExwR4DorZHZ7vNNfCzcXKPZ0Qmr6+ey7LvSvliu2yGvgxz08peji0FNm4wUxOpIVY4m9KSVSiFo1\no72n8WzrClXgKUHouzjC4Xz7PMrraNXtN25txaRjSIY1xV7t9DhOc5qBNzcTq3YYIyrG2Ri19TX6\nh4bYW6HHeiMgkQmRVGwWBbgxGy0XIYzNaTz5A0fXRGwXBf3Kqneq2KdWTMN3jzQBawj9+5bfYpzP\nJ/ZEFdPN4SspymVWzLHBU0nhafFwUYrSiokrwf7crK7i6BQZsDfe47vf9d20RMi/X2nTYYWXbg6J\nfB0n2FNm/u8+Sc/R418q9lvB3qQZsnv8yuP85d/8y7zam6Y1JnnFYz9ixVjFbjx2GyxZfQghIq3Y\nqx57USX2YxR7hdgdU3Gp0rEJnuqsmFApiNaILFm6RzNtMlmUiv2kVswssTfMZhuvHTxHJz6seeyz\nVszYcVBGsSul8BFlAyuopjseVex2K7GzzXOsihFpRZ08vKo7Ll5O9vXyvHOJ11rT5fmrSmoCdwPG\nol3+fJgWuhnYaDZFzrZOUMRImn4D1xFstsLSY0/swsW8WB1SCneF0G1NN9vYfo+2YuYE4fM0JjPH\nuOlpYrdWTNNrwGifppT0JHSkJq4VhiY3fHpdLZE3zTnSasGbESizFakJBb6atrE93zpP7hxUiP0W\nij0bQTpkkmrF3jbZUdaOGSUFrdA1tRPHWDEmJ1yhmIRmg4i4xzDJaYce602fTGU6K8aIBOGOaQWe\nUewvgBtwqDyUctgoYJBXC/6mPVosIn/GY5cZkaNFQMtvMbLP4GxWDIo1uzl8dQu9Eyj2SVaQiT02\no01Wsri0YqqFY5mxYtLoNHuei0Ty8OrD/LfRo/xGq8nlVPLS3pDHTrVphx7d1NMbw/cu03P1fXwj\n2gnAg0zsluRm3sKWbK8Np7ufJHlRy4qpTmQ7gexG1qHNT127gOM0dYfAWlbM9HyLdkOPsxzhmOO6\nYzxD7Gk8Ii0k7UiQq0KrgmitVL6uezQrJily2rdB7Gkhy+Iki1boMkoTdlXCRAhGydhYMfOCpy5K\necSZpDfJtBVTIR0bPLX92KvX4epIE/uF9nnWGRK7U8V+cUXnIF/yfa2Qu5d5LWwj8xaqCLkyugqn\n3wvtM5rMDQZxPretgL1OytHXpOXrl8H2SjjNirE54+MDSEdIUSC9dRpuEynMA3v6vZD0y6K0KopM\ne+wA170A+ruMshFCuTT9CMb7NJWiLyUHhtjbangkeDpV7CaXvZJClxqlnjC7UitwlSgJ78LKBRL2\nK1bMCRR7HjNJEpq+R9MWqpmg9FSx38qKmf7OFhkVyYgkl7QCj41mQEpOw1gxALkYavsvy3XjsK13\nMS4mIAOaBQwqMRl77lrwtKrYZaEDyVViz0Y642rWilFyqtiraYr2Wt0ieJqIG3qeJgN8ZQulptc5\nN5uhp41trrn6ep5rneOP+ufIhOA39z7GSzeHvPN0m3ZkYhqm9XSvpW3C7//Z5/nkSzOtNe4BHlxi\nz+dHxm2jqRvj6dK9H08QrunFPdPjpQzSGIUdjbsgHGifxXMiUkcgK8uxauvX2RQ1CxtUbXothJMh\nTWfD0UhPNlOYNyV2Q5COm83doMEqvRNnxcwQe8P36GU3SupQ2dUFwdOR7nsidYHP9X6Mj6gV1ViS\nchE4gIdb7qBjFfvF9mnaImbsTu2VCysXcIXDJd/Tnnb3Epc8F5VuIbMtro934Xf+b/BNf8b46xIo\n9MMxp61AeZ1M29yWWeKeagfsDXVriFQWmgDG++TXvkQqBNLfpOW1USLWu0qdNm1WbVC3ApklpIbY\nd4JVbcVkYxwi3WZ2dEBTKoZSspfrQGKjGBwJnmaGgEITQU2r+5naohqmn1dKkQqFq9yS8C60L5Cq\nPsN0dELFbgqpkqFR7Po4pWJPC97h7PHo+MvEsy12K6gSuz1jNtGr2lbostHyyShoSKmtGGCc96Zb\nMh68BFvvKhunRdKZ5qEzXTHXgqe+y8TOy7hHLAQNU01cErsX1YhdKUUCJbEn84j9Fh77RO1zYeUC\nJANco6yr+yDkxmMvmtt620zgbPssXwd8bZzz2f1fYbc75l3bbVqBZ0SJTsHtmdVpd+gfEV73Ag8u\nsRcLiF0eJXbbMjZ0Wgg3nhs8VYbYw/EhrJwD1yNwTGl6pfqsSq7Jgh3nrfe+bVK8klyXH09G+jgN\nkyQbKCBaK7NjHCc/2t1RFTSUwlXqxN0dj3jsoctYTVcwfnGVhvVXZwqUJkLQMPnr13ox3gIrxjVV\nqT5ezYpZDVY562kCGzlTYvcdn/PtC1wOGnDtaehfZVfkyHQLmW5xM74K3/in4Tv+us6R3v4IzUd+\nXGeBzGkrYO+hNNvctUygdnslLBuBxUWsg6PjA4rdp0iFoAhP0wpa4KQMkrSS8niU2PN0TGGIfddv\nlh67UKGuDxjvEylIVEInEUxUgJf0CDy3Hjy1AVdpOj9WgoeJuZ5VYrdzzJVTxX6+pfPAUw4ovBMQ\nuxEjKhmUHjvAKE7guV/i7w9+kL/72p/ke579SzTVuNzx68hhKuMaqxzcoCT2duix1gjIREGEU1ox\nnaRDwzfEPrwJq+dJ5MQQu8uASoyhVOzTOdvwnWl2WNxl4ggiM99afku3afaj2jXIZY4SU2Kvdpw8\nWYFSRiwPOds8C8kAJ9w045ueo0j1/89bZ7huAv9nm2cJ0i7fMhR0811w+7zrdJuVyGOYZOWeAt2G\nXr2qoslqY9krZjEsyc12HjR+5s3xVOEdmqrFreAhhJMxTKeEHKfWQzTEPtoDk6ERupH5TJXY47n/\nvwpL7Gdb+m09NI2DJiP9QEShfoisxx4aVSSc7GhWjNL7i4bqzipPQbcVmDB90bnyBg3fOaIsSUck\nDqwZL/VGL8YT2jayKPPYjfURijqxX2hfYFXp79mveOUAF1cvcrnRhJd+nUQoDoiRmSb2TnK9vHej\nNMdrP48TXaUfx9AybQUqqrK0Ygp9rqYpDtluh+wNdVuBpEigtQWjA9TVp0mEwA3XWQ1WEEKxN+rr\nasH22bnEnlaW4dddv7RikIbYR/uEeOQqoTNKGYg2xN0jwdPckIMntcpLK/MpMyQ3l9iVO/XYTYGP\nCDqkIry1FWN+76QjmoFbWjGbT/9/8PP/C4+qy/z49tfxB99xli1xuNCOSVC0TeBynI8haJHHVrF7\nbDR9MiGJvAabZm504g7NwNXpgUkPWqdI5QQhQ4LCY1CxnaYee92KKT32SYdE1Il9kk8ovLBG1Paa\nrQpt2VSf2doOSnNWJlkhyRkiKTjTOgPJAK+p22Bklaw7S+yqdZZrnsu6G9H0mziTQ9bNik34Xd51\nWnvso6SYKnbzTKmiUdv8/V7hwSX2fL7HPk+xdxOdn3qmoUuSe5XGPnYpKE0iWjS8CWua2APbTKoS\nYc9khdjlfKKdJXbbrjOZ6MkW+aZRlyF2mw8unIy0kLWNpzMkvlKEJ1Ts87JimoFH7kw76nlqf76/\nmo5IBKw3mgihFbuPU7NiSr/dtBuIcGvEfr59nhWlr1dX1Yn94ZWHuSQUqr/LruehAJme0nYMBddN\n17/D8RAnvI4QisP0hi7IkVlt6V0SZ6rvc6uhPcztlZA0lzgYYm9uwXgfceNpUkcQuAGroR5XuYvS\nadOEbAZZJXtjzxGlFaNkqHuajPcJhQ9OwtXehLHThkn3SGOtzChLV2pyrRK7VcSJoLRDLEkJ5ZYv\n6YdW9CbIjm+IXeaLN46QRXmtnGxEK/TK4Kk7uArRGr83/xF+pp2x77msuQcLe7KnKNaFtubG2RiC\nNtJ0eGyHHusNn0woQjckMhZeJ+nQDDx80+mT1jaZjPFEhCt9MjGNy5RZMX5VsVc89kmHWAhCM9/s\n7lhjL6pVntpmZWtGjNV6vFTbWMy5ZnFWIHw9F840z0A6xDMtgtOKeCvMfVQrZ7jmeZxzW+UYW1Kv\nTt2gwyOnmrQj33js+jh9LyAQTcBlNVoS+2KUwdOZjZaN6rsxmhJ7xxD7Q21L7NOI+STT5fW56bMc\nDm6Uij0yfb2rqVPVzYjTBYrdeu+nmyZwYvKp04k+ju2GWCp2OSV2fdyK2kPhKwiVXBisrSKbEzxt\nBnqD3sdMpawjukS+zWOvnCsdUghoeA1OtUPdVkAI8orPaq+vdHWwMBLaY1dKcXV0lXOtczQKfX27\nqlUbx8XVi4wpOHAdLnt6cst0C5lptX3Z7DX5cv85hCnk6abX9U5KUGu8ZMctUx2Iapkda0619T2T\n0jPEfgr61+BAN6+KvIg107ek3Gzj9Ht1c6uZRmN55XofOhKKhFHSQ8lAWySjA0InRIiUy4djJu4K\nxL0jL0xL7EKZSt6qUMBaShW/vaLYHROw3Iq28ESA43eYYII0VcKqoqLmnXxEI3DLnbRI+qhwDbn2\nWcbGNV9zOwsVewpsGIFjFbsl9lbo0W6AEhAKn1i0aEiHTtyhEbhEiWmZ3NomVxMCp4FXaPK3ueyL\ngqdlA7FJl1g4NAL9MrbEPvLrit2KnlVfz8t6uuNYx81grs8eZxLH03PhbMtYMa1T+ErVnndlVvpu\nY53rnsc502yO8SEtoefwxuqQ0NNZSFWPves6+KKF54iyMd+9xINL7IuCp+bhqFox/VR73A+brd5q\nBRKZpOG75cMUZTGYisnIbJ5bbamaV1R6tkCx28yZs6Z/dNcQbRbrByL0rcdeD55an7/qs+dK4mEU\n+wmIfZFiF8E+Xx/HuEoh3EEZPK0Fko3KibyIc2uRbiuAQ16xCazHLo1aaQqHOI/pJB0m+YQL7Qu4\n5sV5UDRr47Apj5c8n8uBJgtPnsKTWtXsDHTP81cHU/U8kjcXELseR57pl3bLtHTdXtHHzXNX34fm\nFvR3ylVH5IVsNvTx9sfGhz39Xv3wdys7OgGZCQq7KmLgJChgnPSRRaCtg/E+DbcBTsJuZ0Lir8Kk\nezQrxih/YUitrtinqzOb0WXnj8vUixVCsBWdQfgdJkTmwAt89kp6biTHtAK3bN/spEP6QZtg82ME\nQh+n7fbmE7tSpALW7CbS2Uhn5JiinVbo0jSbrPjCo69arBlibwYuDXNvaJ0mJyHyGohC35++qT5d\nFDyN7XgmHe2xmzqHtrEAx55f3yfBPL9poq9xbPYXtauXD2+c4obrLiD2AuFrjjgTbUM6xIlWCZQi\nqwR6pV15BQ2ueh7nTOYMk0PwN5B5m3Zbf6926GqPfetrAEHPcfFos9bwEaJeQHgv8OAS+4J0R6so\nO0mnvNmDrIOSPg+taqKtblc3yQpdwmxIM1BKT16gaYKIVWKvtrBdROw21/1M07ytTXGCLXDwXDk9\nV7iKB7g42FZRlrRsKbqvFIFSCz39KtI5wVPXTXC8AY9lGWtSgltPd7TWj/2eDa/BmdWI670YX7hl\nL3GYXt+cgER5NIQgzuMyvfR8+3y5CcERYl8xuey+x+XWGj4tmt4KLXcTB7/shb07/ioy3UTgMpF7\nFWKvpAmaBz83W7G12rqdgFXsWe6Sq5zcdNyzxUoNL+RUUweyOnaDCLsB8UxrAavYQ06Ri5yREIyz\nEXke0PAcGO3T8FsIJ0UqyP1V7bGb62qtFXscJfXYyjx2Q5wWpT1h5q2j6kG2041zOP4hE6P8F/rs\nFbXaIqZZCZ466YB/HQmEN+b3bOk++U2nPz+XXeYkQrBhiF1bMS2EOX479GiY1aerAnqqwbqyVoxL\nM7NWzCkkMU2vBVI/U5bY7Qul7rHrF2MhFUy6JEIQmn1Cm+bZHHp+bbVur9m1noOn1DT/PBszFIK/\nvRbxS+3W3ADqJCsQXh9X+GyYtEqnsYqv6k347JZ+qadTP88VskwrLcINVLaBa14Q7dAnziT5Q78L\nvv9ZeuQ4qsnqG+Cvw4NM7Pn84Gm1YZVV7cO8i8pbnG2bPRPTusce+c5UsSul08mAhlGVtZQndWti\nt/uXroVruMoprZgiHelyf5szbxS7AELHRwqzZ6MlLeNn+0pXeSYLCqJq584lX4p/gh/+3A+XP5ug\nSfeRLGejKCjcmChwS1/TWj9WMVrFfq03wRNOzYqxwdNCukwIaaJfZLtDnQd+oX2hbLF7M4tqYzvX\nPocnPC41V7gSRjTEadqhTyv0aYhtrgyuoJTievoCKn6ElrNNKg6OtWKSTP+staZtNqvYk8wEKk02\nwtgs5RteyKmWIXZbjLb9bv3fmdYChXmRNh1t89z0XEbFBFkErDkxyIxGuAJOCiiKcE177K6DUpSZ\nJjbdsTCklubTSshECBo297qYIXZRJ4EL7QsIv8PQeNknUewtEdMK9eoscB3ifMC/cUdkg/fy2zb+\na31NnMH86tNct1TYMGSqrZh2Seyt0CMKTHGdCOgUTTak0laM77GSTz12RELLb6IK/WwNzL2sVhBb\nlJtt5AVqfEgsBNGMxz5y68SemsBmqiIipaa2ZTqmb7tQOmLuLkpxVuD4PdaDUzjG3vKiFXxVr+FQ\n5phds1nP2Twv57psbCKzDVKhrcG22eB9lBSwdoF+0jcZMUtiPx4LFHuV2K3PPsq7qKLN6da6+fd0\nKWzbdSZFgu94+oKYydM0kfhJUSd2gYtQXo3kq7B+X+AGNKXLoasrKlUyYjXyy1x7a8UARI5XZubY\nyW4/ZxV7eky61vTckk7xMr/40i+W4xhKHZR8OMtZLySJm9H0p4G5JJe6EKSwxT5asffjHFc4ZS9x\n/f2NYlcuIyIaaKVpc9jPtc9B3GUkWvSSOll4jsdDKw9x+dzXczlsEqjTOhUv8AjUaa4Mr3BtdI1Y\ndvGzR1j1TlO4+yi7q3tl1xob7EvSPp5SBGYzj/WGj+sI4tTsnWny2ztr7wKg4YflPOjbIHq0qjtR\n9usbSlvbre1qYt/zfEZFovc7VZqY2tG6jgeIXFt46YDQkdPrypTYc2k2LqmoyUQIvYpiqtgtwXuq\nTgIPrVzA8cbs2xjMImKvKPkWcbmheTN0+dVwzEBI0r0/yFqk53ngjuZaMTbdc8Vt4ginVOyusZba\noUdgFHtR+PRoslXkpRWzKrsor0HmBuBktPwWygTUexOdvjqvCVi5oXVakMcdCiFomGfRWjEj161V\nntPjUP8AACAASURBVKaZfqYnskUkFXFhr/GIgRFWEyGO9pZKBsSTMcLvshlqGwbAa67hSWpWjMoT\nMuXSyXTs4FwSaxsGEM1NZLpBP9tDKsmKWSENEv333aSLfIMyYuBBJvYFHnu1D4fNjBkXPRy5Unqr\no3zGijHEHtlgiFHsrcB67NNzFCrBJcARPrlaoNgNIURuxIoU7Lsh+E1UNjF9YvTvZ4m9MJk5tmiq\nbB5lPPYTWTG5pCBllI34/PXPA9DNdkHBxSxjXfiMXalbClSUEdlYt08Amn6jbOTkKLfcr7N6fbPC\nJSakqRSTfMLucJcVf0Vv1DvpMHJW5u7PeXH1Iq+IgmtpF1dumwZVHm5xip3BDk/vPQ1AQz7GRnhW\ne8quya6Jj7Z7neRDmgqEeXgdR3CqHTAx/QTSSHuzhyuPAtDyI7ab+ppXYy26mrN+fQtzH9d8beFd\nbW/payFDVpW2EtZa+oUinATHxGZWGJf3AqbEnklNpOkMsbdtnNCsmNKyHXJYG88jazozZqfy93NR\nCaq2xKTc0LwVeLziFZwlRCbn2TDXxhfjuVZMajtZeiFNr6nHF7Tw8jGuIwg9B990Rc1yn75qspUn\ndJIOke+wJXSqY9dYkC2/hTDE3p9ocoxzveOX51YVu3kp57Lc7zQytqi1YkaOW8uKsVkwE9kkVGr6\nzKbjkthjMUex/+Qf5uwTP4rj9dmKTpctnkW4eqQ4T2QTEnwOYs0r5+JBuUPX2uYZArVFrjL2J/ul\nYh8mOVJJ+mmfPG2wGt37HHZ4kIn9GMVudwG3Vkws+ziqTctxUNKrbTBtG2LFeVyWLVuPvWUIPqlu\nDUeKKwJcQopFit0QQuiFrEvFoeOB30RkE73JhjmeLlDSYw2Fi1R1K8YSu6/0SyC5hWIvpEIqyuM8\nfuVxAA7TXfysTQCseG0GDjQ9SWgepjSXuurUZGC0g0bZU1soUSN2a8WkhUvqNIjMtn7XRtfKXGsm\nXWJvRWcFzODiykVe7r2sg7DZ1rR4Jj/FJJ/w+OXHcfBZcd/BdnQOxxtx0x4mOdrudZKPac1M41Pt\nkHFiHmbjsd9sPwZAM4hohy1QgmE1c8JvHNm4wV7HjVDHSl5taFJSMmDVLMdXTak4Torb0udqmXTP\nMg5gA3uW2M0LWiUjUkfQKOyuQaY3u1XsJs3QwhL7VStMTqDY28ZjB62wdz3FWVNfsGE243bdeO72\neLaQKnA1sVsrxiv0y0IIQWprNFKPPi228oRc5vhewin6FM3tMkjd9psooee7JXa9kXX9/kUVxR6b\nlEm7j61V7EMh6lkx5l7mKtRFY5YfsgqxO85Rxd7bIey+gPB6nGmenc6xsI2nBHmlmIoiISZgL76O\nj2Br3CsV+7e/77fxo3/09wM67dfGNIZxzjAbIpUkSaOlYj8WSi0sUMpkxnq4Tttvc2N8Q5caqx6b\nUsAPXcCTPpOqFZNPs2JC27/bEPpKYFKnZFYWNkiV4YkAVwQUzCfa3BB36Iasy5yeI8BvIPJJuS0e\noFuRVvLBi5LYi/K7AGUe+60KlCyRWOX/+JXHUUqxF+/QyPQD0fQ26LkOrbxTeuyJIXar2Ft+VPbU\nVqpuxRRGsSeFS+ZENGRBnGuPfUrsHWJvjUF8NGfYZsYAFMkWrdClFbgUyWY55oZ6hHYYcqapj/fy\nYF83CJvx2B0Bo2JCy6k/LNsrIQPDeUl7G/73T3Fp/Rv09/cjHOGAChlnlVznOdWchbQpdGuoIuTV\ntlbnv5sXaBea2JumslA4KX7bEvugdj8yQ0CTomX+bbz0RB8jMvntcWZTAE1nTVFX7BdWdBru9dyQ\nz0LFXrdibKrjaiC54rlsO5pcN5ua2B0nnq/YjfIPPF2IY62YQE5oB5Z89bmGsacVu9kNTLkjtkSf\nLNriwFaqBi0Kt01TSgbGVtP71tbT/yyxx1lBbO55o1KgBMYvr6ywEjPWQgYECCZWjKWj4xV7FjOe\n3EA4tjjJzIlwBU85ZToqgMgTEnxujq9z1mvhxL2yc6jb3OS3bel+SLvD3bJuYJDk9Mx3mMTh0mM/\nFtWUpTlWjO/4nG6e5ub4JoNsgKLgETUEmREqj0RWer+k0+BpJAyxG0JfCY1iFwLyGKUUUqT4Togn\ngpJAZ2GDqqEbslVkDFzAb+IWE1bMtnhgCqBMrnwonNLaKYOn0gZPTbqjnH8+i1IhqpTtxjY3xzd5\n9uBZro13WEn1d4mCbQohcOKd0tdMMllusgHQDpulYlfSqWoWcrsTfO6Qe02iIiMpEnaHu+WGEEw6\n5MGqbgcwg4tmQ2KANNkoFXsa614acRHjZ4/QCrzyeJf6O9qymrFiQs9lVKQ0nbqyPdUOGUz0d0mK\nBM58HROzmUk7sBZTxKSoEOOc/iuF8VebQYjMV3nNlJF/r/gUD+3+iv6dKULDSYhWNLE3zYowLWys\nRN/X0oox/07M9/ELWy05mI4ZcEQ9+LwVbYHy2Td1AosVu1GvXpOWmCr2df+QvuuyZXb2WY8aeOhg\nfpIeJfaktGIaNLxGmcfuoNgMTVzAZLcMEo8BTdbNdy7EkC3RJw03ORzr67Eatsm9FqtS0k+mwdPZ\njWEaFWK3rQGsFRO4gd4eT1DPijG+f65CAiVIbKwtGzMwK9HJjMrXEe4J3Vyr7nOm6lSfqI2r6qm+\nokhI8bk+us45fxVQ0DEpss1NHV9CK/aVMnia0zP7/+b5UrEfj+rNmWPFeI7HmeYZboxucGiWSo+Y\nhy2UHomsKnYdPK0287dWzEpoqthM69600H3WQwShcpBqfuWfDapGTsh2PmHsFKj/n703jdUty8v7\nfmvY4zucc+4Z7r01UlU90BN0u4FmaJuYNm1wgolRIgxSFAUSx1JsQsiH8DGyEseO7C9JHFsmke1E\nOBaKUGwR7DYkARsDthEx0Li76eqa7607n+Gd9rhWPqy19t7vcM6thqJQRbW+VN1z73mHPTz7Wc//\n+T//OEO3K6bpQGOXkQszAlJk93tBZgjF09ZGTjd8TOep050Nra35zDOfQQrJT33ppyjaJXuVex+d\nOhbcFG90tsiq9Yzd3wCTOCePNXtZhDGS4bcM7fHLWmF11s1rXTUrbo5u+oN6RpPss6haZ1kbrMDY\nM52xWuVOY48Vy+UU5VvXRfkseax4xg8vfmO+Dexl7eadLk3N2LO5sI4nCRces8OxDt3AY39OtcjX\nHvC7gD2c33GcYpsJb8xfB+ChOeLw7i+DzsjC+D9ZkU+PeEVr/unsn7j3Dozdf4aueOqv2dKDovLe\n7tBHEFxYWq0zdiEE2hxy6mWgxzH2VXzEiFXXEBNrVxzeE04+yhNFLmMKaWnL+fbL1LsZO8BhXPvP\n7HcnJuXC5lzztY+GCw45p4gPOfPuo710TKNzJsas2R03gb1n7KY7RsngWIyiEXOsY9++8BzyeGqb\nEFlJEWpt9aovnkqx3nnqz8Opl86enj7RA3syRbHO2GVbUIuYNxdvcsNnyXD6stt164RMZ1xLr60x\n9nnRdA2Rth29B+xXrjXGvl7wqk3dMfa7y7s88oNz3984G1JmNZXpb4igsZdtSYrfEnbF0xSsvyDq\nZTfI+qh4k+vVHazYDbSO6Ql0NWffNBgBC50SmYJpFnUSS6xTUBEgSBHUHWNfl2JaG3vGfkkLuV9V\na0C4370+us4nTj7Bz7z0MwAc1e6CsrEDy6J4s+v2K+u2T3YEpol7sN2YprStoBG4Rg+g9cd+2ShM\nNCIdaJZPjp90LGh1ivHpeJsF1Bv5DSIZ8czkGT/FxzH2Zdk3dDWrpxknmhujQ2wb8+bitqtFDHzs\nZWOYqpqFsIyidb/80Tihbv13C8DuEzfHkQP2SGRr18Gu4RWGIbBPO0fQf1f+AHV2BJPr5Nq9txAV\n4/1D/pe9Cf/r/BeAdgDs7hg1Ptq3Y+yejQpvAVx13ZjuAROp9QcWOE/93Hob4WMY+yK65uyOnrFb\n5dxRGc7lk0eKkUxYCYFYbkfJhqycWGfkOncNSt42ei3ywO6/Q20yLhhxYELj2H1i0bKKDjj3jXn7\n2ZhWj5i2hpmXwQpvXhiuwNjL1YLCH/Ns8PAeRSOWAXCDrOWPWWMTtJXd71EtmF8mxfjfueOz0p+a\n3IRQUE/GaKvWRhnKtmRJxP3VfW7mfhjMo5dd3pBfT46f5Pb89lrxtOt0b7N3JE4A3q3APmTpGyy2\nMY1j7KPrPFg94N7KFVA/1Dgvd2YUte2ZWnDFFG1B7KfldHbHWCOt6qYoVX4yUt7W5Kbt8mU2V2sr\nFDFi+bBLvHukNImtmKa6A5tERiAE6JSEwQ2/4YppbOSKp48D9sGAj1SlfMfT39G9xonvhGkS5w5Z\nFnc6ptQxdi/FTL0N7sZeStNKanqmE4B9UUlEnJMNzsUT4yecI8M0CO8Q2dTZlVR86NqH+ODBBykb\n0zXPNMby9OQZrufXKYoJeaKYZhGmvsa91e1tKaYxnOg5CynJB9OXwHvZjbuBgl4djnnupZhEjqht\nD4y/JCveaDeA3YPDKE6wTZ8t/8Ac88qf/Gn4vv+pc2kgK8b7x/xG4pmlrAfFUw/srfex+z+XHkRa\nr70v/CCHIIEovQ3suTyiEL6j8yofu9Ss1IQxhcu1ASpxD2ktmuskWqKVJIsyFlKifDFzuEIWeaIz\n8qh3xQAc6FA0dQBdm5wZOQf+eq99hs88usa53w1cSyfY2EsxXjrZzdjdn83ykZNP8EPm/XKMfR3Y\ng9OoNinaKorQezEsngq5jh1eo7+rFVjJjfGhY+wqBp2gWHeEqbbivo4w1nDTp23y6KVuaDu4e+D2\n4nb3MJ0VDWe+lmLb/D3GfuVae+puMPa2JlIR1/PrtLbly6du5uKTvmA6QlHTA3tZO+tf2ZSkHbCH\nBiWFNOvAjqzITE1mWxD1ltQArnipRQyL+51H+R6KlHKteKq9DINOSK3otPlNxl7ZmNRaattiBs1C\nm6tuTT8w5Iv/kD/6jKvSpyrlhrU0Isb4bfi8fNBJMU5jX3TdmXupA6ubeylV7Rl7kBPCKLdWIZMx\nab0B7L5hQ3oWs8sZ89e/86/zI5/4L9z58FIMwJ/5yH/KX/4jf5m5H7s2TjSmPuBReWdbimkMJ/KC\nhZCM4una6x+NY6z3gAeHSdH0BW13THKMz0pZ1Av+fPkVflKvy3rG734mSYap+/ewJkEfvw+e/saO\nsR+MDYUwvBi79xWiB/bQrVzZZC1/pPSA17Te6REYe70kMQap1zV2gIm+jhFLZvqKhMd6CdGIpXAa\nezjPCx5ws2lZ2nHn2hhHY5ZSoP3OdrgCWMZR7lhyvezqT/vaXWereo6wlsKmtPGUzFqX+Fk5n/pM\n7TPzhc1r+RgTjZwUE+ITarOWEwO9FGOWj7prMtXrwL4MjLzr1vUAbzKk1f3gkkHx1Ekxw92++wx3\ntSZtUpRUrnjqs4SU0FQDxq5MyT1/rd70MiH1cidjF8KSx8ox9ipIMdk7EtkL71pgH+pkG4zdNmih\nu3b+Lz5ybeJhCMDIShrrQqtaY6laQ6q9Kya8iL+IskghbORYQ9Uz9qytyUyLkDV1uw601lpaapQH\n9sDY7xrIRNUVTxMEogP2lBTb2SQ37Y61SboZjFclPK4x9vPXeXryNO/bfx/PTp9lX9aUMsM2jnGd\nl6e9FDO0O1rBxHfcXp+mVK10U4T8MW/a8JkiZDLqBnGPo7GzmXq3g/KDBXZ52afxFOtzU0ZJ3+5+\nI3uOrzv6BJWfzjNKNLa+xll91zUpbWTFHMkZSykYpQdrr7+fxRCSFAM7DnUN5d4306Muy/3X7vwa\nDZaVWf+sxpeNJ0m6xti72F56X/W//21P8FsPfgsTckAGjL3y12hNTGRFJ6kFTd2oKZkxLH0GS9ms\niC1Iva6xA+xH7rq+nV4x0LpaQJyzJGUseuJzzinPNDX36sHnjycshEQX24y9antgH9odAfb84Jqi\nXrpOT9Kui/pI55x7YL+Q+244CHCUTSEw9tBl27RryY4wiBdYnXZ1n03GvrC9hu5ep0BYS0mMNIpV\nYNr1ho99B2O/oxRTfz1SznpgRztS45c2JffDgI295/q/yNaBvTbey55oFmXDRXlBIl2y43uM/arV\nPJ6xh2TFLzz8AonRRABCMrECRMuqWXUBRFnsXDFJyInxF0IaKTC6Y+xlY1DSzSrNTePcBBsde6HA\nqmVg7O497lvIKbriaYzoHiDohMT0dsZem/VgSoqPcL/S8li1BhGGcvtt/V/6w3+Jv/Btf4GpLClk\nRtNqYgNn1WxQPG07V4w1cdepeHMvxVpFIwTWH/MgxTQ2IkrHpH5HcnN804UbecYeT5w1cJflEVzm\nOrjkyW4IRNX46Unu57GWyPaQxhacxum6j70xHIgzWiG6LPawppmGDcbeOZG8g2YUjcDb/H71zV91\nr7mxGzKe9U3TdWC3A2APgCNVxW/c+43u3wjRdE1UZVs5pk7sh4N7tuuBXWgXBLf0ILWqV6TWIHYA\n+1Hq6hC3kisGWtdLiHIWZIz8rsRay0N7zjN1wxsL3VkgR+keSymIy9Otlyn95/mVlxck0rlijJeH\npio00y3JrKWwMco3AJ7IhEc+w+dM7LHwn/NwNCGKEkZGsLQNjWkodjD2IB1RnHV1n03GPg+ypL8u\na3//VsQIqykCsFdLLvwYO6ex72LsikPfS0A5gzgw9mgty0ebivuecN84eKH/iwFjD5bfoLPPvMae\nSP9AfA/Yr1jDk7PBYEPx9Lq3od1f3WfawsPoJsRjpv58X1QXXZh/8LEPc2LAAbu10YbGXpFZS2Zq\nhKi2hk+7AmtNJGNYPOiG695rDSmVk2JM5ZqTtGcJephvYbt2+WFEbmDsVwL7kLF7lvTBax/kw4cf\nZiwLCpGxrFpyozhtFht2R1c8tTbqfr6fR1jPfNvQFh8kGSKibNJlyT856q2OAPE4APs2YwdnAwPW\np/uUbffz4CpIcUWqW0o5wGr6Jq6xcCwzRPaGtZdFnRQTNPbKrEsxk2iCkDVny6IHdtq1QQxBipmm\nKaZxN7tAgtUdy1RSkaqUZb3sumYB/9APdscKbS01GmUFlT+vq+ATT8akvoMX3ADl2IKMt6WYk8w5\nj25F8dUae5wztwkpFbRO511S80zd8NqiH7zhGLsiqXZIMf6c/61fvsubZwZjDTNvCZ74kYRFsyQz\nliUpeZZDlHMiIh7UFxgreMSEZbPwUcduBkDqB47Mqtluxu6vP1WcdXWfIbCPozHLboKaZ+xt6epQ\nNsKYiFoI10w3jBSQsgvycifGwf9drbneBumml2I0/t735EXbilNt2Uv2yEcn4F1cQ8YegP3W/BaT\nRDP3Gnvkm8Im7xVPr1iBsUu9M49dS81BckDkG1eO25o72ftApx2wz6pZl/mcBLujMZ3VERzgWxOx\nktLbHVuQjWfs7v+rDSnGgX/jxuot7pNHPiLWGBLRMI2Fl2JYY+ypMVgsSWS3pBghR934vMcBu/DD\nnZPBMBGAsShZkbpicRtzasoe2L0Us1QRwsZdrOgo0RjvFKqD68DUKGs9sI+6yOFh1ylAvvc4YG+7\n9wga+6JsWAYm74F9JD2we5ANrL1sDJnwyY4bjH0U6y4ZsYtzbSuwAu2b0CZeUvjSw5d58exF928H\njK41FiMMwsI4iTvGHokUIcRawS+PnEzxmw9+k5tsa+xlWxNZGCXxGrCHBMI0npCZHtgDY98lxRxl\n17Am9g+6K1wx0YiZTbs/h6z7J2rD7bntGHsejVhISVpvM/ZQzK9tyrxw3/dh4873yEs8q6YgtYYV\nsWOj6R4nVnK/XfKIMctGsGpWCOtrG5Ei8mThorrYWTzVShIpgSrPeo19IMXkUc48BPANiuOJtZRE\nmHZw7geRAtD73d3vrjiVTmp8Ojwoygvwg1iUiGmEwIQJWLZiqQSTaOJMDyGcbsjYRz1jHyW609iV\nHTFJNEr+/kf2wtsE7EKI7xJCfEkI8aIQ4sffjte8crUuG9sm0y2NPTB2IUQnx9xsV9zP3w9Ryp4H\nosAWYMDYTbvG2CMlsDbxjN0FJVnRklhLai1CtCyr9fcvPWuOvBQj0kNsm/DI38x7UUPVVo6BhxtX\npyResomj3iYXbiwRjTtgv0pjr1tDKtxFmDTl2o0/YsWClFXVEtuMMyyxGWSaVHOWQiFs3+wzSjTG\nA2TjX6tta5SFCk2UjUnNJrB7sN1zgPzrr53y9/7Fa/yPv/AiL97rvdIBwEdJL8Usq6YHfA/208id\nwzdCfIPX2avGkAr3EAndiGFJKZhmKYI+tbM2JQLdPbT2Unfz/uKt/9u9hux3ZuH1jTBECMdubUQi\nJyiRudrLIFM70xlfePgFzstzvkk7NxADYK/aBm0t+6MIaSWVzx8JEQKjZExqTZdvUjQlsQW1g7FP\nswjbjDiV6mofe5xz4V04lPMuEvla7eSBjrFHOUspyOuzrZcJBcnaplwsPLDX7nznInTTFm63YRMn\ngSVTTlrDipZbYo9l1VK0S6QH9kTLtWEbDti3B0+kWpEW91mpCCUUrRF8/pZP8oxGLFs/TLDu83Vi\nCxURrXdEFfUSWy2YCxcDMTzm7osV3PHv/WwA/IHGrn09Jgwfj21JIQbXm3d+DRl7qlMO08POyx40\ndvEORvbC2wDsQggF/DXgu4EPAz8ghPjw7/V1r1xNxf8+GfHdx+NLGTv0eeiHbcvDyftBp+z7m8ox\ndj8kQFla224xdnfzBmBfsawqrLCkxnaAtthgTY41B8b+AJMfYducM1/smai62zauM/awe2i7B07h\nJ7boKCc2b42xp9JdoKm1HXsGp+/PrWPsmjGnSpJ4J0TZOB/7UiokPbCPE40NwB4Ye1uhsNRWE6cT\nnmgaPnv8Sf7IU3/E/VJxBjIizcdMEs1P//otfvynf4v/9h99ib/xi1/pXnvRaem6s4bNB1JMAPtp\nMkbZCW+03sk0iHtVPohrE9gBpmmEJO409trUCPob68CzrV+58wtM4ykfyW505zm8vgN22Rca5QGK\ndMt3nUc5v/3wtwH4lJdKhvWX2tRo4CCPkaYH9tCINM6mDhw7x5FjwbuAfZJqrIlZSHEFY3ca+1ko\nCFZzXr14FQlMKwewXTCYHtEIiJptKabuZKyEs4V7kD2oSowV5IQkytKlKQ4Ze+PO4SvxmGXVUrUr\npB8OkkYK5TttL8oLinq78xQgjRWT4jZFNiXVKX//X93me/6HX+K1h8t+2IbvCAeX5xR7xt74712U\nZxT1nEaA9a6m4WwFmoK7vpv4eR//PHTFhEiHyruXIltRSttbXANjz9aL90Mve7A7vpORvfD2MPZv\nAl601r5kra2Avwd879vwupevtuS3k4RbCpodnadBggnAfq01nE8+CDrlwOtlQ41d+YS6tG3WgB1c\n61Bwxcx95nNg7ADzcv3mKhtng4yVk2IYHWPbnPPQnk7pNfZ1xh46OKOo7XzswSWRxhnCrjfc7FpV\na0jlovuMw5jbjIKFTVhVLZo9TpUiWt1DiEEImJAodjP2IMW0pkF7xp7mE2Lgr37g3+O54BJYnUJ2\ngJCSf/DnP83/8Z98G//sx7+Djz455e5F/xBelkPG7oclV00H+KMuuCpCtYfcCsNRArDXBukzWUZ6\nG9j3sghh+/iGxlTIAbBf8wFYry++zKdufopMpRvAbmiFRQvVJRmO5A1i9jqgDyvXORbLJJ7wtb7W\nIEXVyXS1aVHWA7vtZ8gGRjzN98iM7cLmgqygo21gHycRmIQF4grGvoB4xJnvaKWc89rsNW7YiAI/\nRCbpGTuAMhdbLxNm+lY25YH/67NyyZKEzAZgr0it5Znrh3z0yT0H7J6Q3IpGrKqGyqzQPh4h0RIG\nU5TKxmw9KMF52ffLNymTMalKuX1WYC38s688GCQ8yt7H3tbE1tKgMH53UKzOu6C3UCMZRnBTr9xU\nJeC5Zu6OWznrnD/K12OqagbGENNQSjMAds/Y83UpMHjZJ4lmVlZcVBe0dcbeO2R1hLcH2J8EXh/8\n+Q3/s7UlhPgzQohfE0L82v37939v79hU3QmpN4BujbH7AuqolRSTpyHK2PeSyEV10blilPfkxm29\nJsWAi04tpIR60bkWUp/dAjCv1m+uoLEnHtjF2AH7wncxinrlto3GQGiT1gmJB/ZYNx3TK6qwVc+Q\n1uvhj2HssZdiUrPO2FOz4tykLOsWqa6xkJJ6dptY+fmc9ZKVkMhBouA43pZiGlPjkuM16cg3Bg0B\nZnXWMZjnjkZ8/Ol9ntzPuDHNuD/rP3uwQQ6n+8zLpmPsuQf7SaqxzTVu+bm1nRTTGqz1vQm7GLt3\nxoSM88bWqMHgiqPRXvf/33zzm0lUsg7stcEIS+RvkTxWfDT5IT4g/2zXQBNW6Ir8uqOvI/VsL5ZF\n32hmG7QVXBvFCCu7cXhBapuOnCumDJESbU1yiRQz9ox9KexjXTGnTc/YX7t4jaetZE7oqlZrxy7s\nfoar9A9Fa2Me+OfqaTFnSUpifKHX1OQWfuY/+wzf+/EnId3juickd6OEZdVS24JIuPdNIgltP0Wp\n2qGxg5NHr9VvUsQ5qU55tHCf5Ze/8rDPZB8EgVXG1TGmaUTrJ1UV5Rkzf4xCjWSTsd/RCmmFi0I4\nv+WKsT7/P1LhdS66Po5CmK53oZdi1hn7E+MnvMYuWdQLjDXU1TvXdQrvYPHUWvs3rbXfYK39huPj\n49/bi7Ul9zywbw6UHjL2oLGvmgOSyOWy7HtWNKtmHWMXIjD2umvACEuLuJdi/EU0ZOyLapOxO1dM\nqiJYnaInJ9g2p1IhlGgA7EMfu7c2at1LMSu/G8iTHGHXfdm7VtUaItl/xqB3AyRmxayNKaoW6QdH\nnJ2/1g9eruashFhLFMwTRRMGMHfDIpzGbmVE7Fnv2lDl1Wl/wQ/WyTTh3gDYl1WvpSdaoqRgWbad\nDTK4YsaJpi33eXP10MGhB/a2Lmm8HHCZFGNt3+XruoH7G+t4E9h12oW9AZRVSS0gEsHvrWmbufae\n7gAAIABJREFUEaYe93a8cJz8jf71x19PGrljkqpii7Hv5xHCKiprwPYMfX9y4IA9DLM2FYkxRPF2\n56mTx2KX+H6lK2bEo9oBuy1nvHbxGs82hpnNuu8z/OyNqLder/aFcpC0/iFxUc5Z2KQD9pWpScXg\neKR7HM+cW+lBpFnVLS0FseylGHxmzqOVO5ebrhiAQ7UiN3NWOiVVKY+W7lj9ylcedOd7NmDspWmI\nrOBonND4SVVFedH55a1n7It6g7FrTdLmDggf/I77eaexh6iHRa/lC9Nfb5cw9uBlf6X+eYzw+T9V\n8o5ZHeHtAfZbwNODPz/lf/b7t5rStQHTe727v1rT2D2ANa6Fmigjp8aaeI2xCy/FJE21JcVEMnGW\nq2rZA7vKunFmiw2NP1gOw6sEYG+875d66Vwxpl2XYnz6YDRg7GUdNNgxwrwFKaYxRN6tkA6lmLZG\n24qzNmZZN+jY7WROZ28Qa9W5YkoB0QDYIyURnuUGyas1DRqLjpOuvXwb2NcZDMDJJOHRoreHLqqm\na2sXQnRdesuy97GDY6hVldFYN3M0uGJGzVk3cnA/3X6Q7GURpo0GwF6jBvG+YYrSSB3x9ORpkk6K\n8cXTsqAWAu1BK4sVq6p1riK9rbGDA/YkDE5R5SBts0VZwbU8Bqu7hq/aPyT3JhMyY6h8Q1RlGhIL\nOtkG9mmqwSQuK2UXY7eOydso44HPBzpb3mdWz3imqZn5K7N3xfixd1LAYj0vpmyrrn8izGudlQuW\npEQB2G2zBexpecG0bTnVgmXlgD1R7n0SLSmNs++e+of0ruLpU9IP4tDaMfa5u38ezCvO5n6HNwR2\nvys6HMdUtgf2uQd24xn7YlNjVwoa/5DvgN3tCGL/0FsUs959Q9MD++Smix9JepIA8F3PfRefuvkp\n/umjnyB7+u+44/sORvbC2wPs/xJ4vxDiOSFEDPxp4B+8Da976SrqJeedFFN13mNr7Rpj/8Pj5/nR\nR6fMlh9wwK5TtKmwbdYVbqBn7EldbkkxkUwoBdhq0VXU4/yok2KW9bbGLmTjIgcAOT5GMaKV/rat\nV1RtSWzNevHUM3Gl624LH2SEUZLDhn1v16pbgw52xyFj940wc5txtqwZxw54Txd3PWNvPbBboo2p\nPdrbzJo2zO1sUBaiOO0fgkOAKc56JjNYJxP3Og/m7vMty7aTYMAx0WXVrEk04ed17c7JmdZQnGOt\nZWrOOFOSTEZryX9hTbOItu0Zu6FeG1wRhkzs8WFvX8zWpJi6XFIJQSSC31s5h0fd7mTsAsHHjj9G\n6rfxiSp6YMcB+/4oBqtc00u9dGBkBAfjnNiKrqha2ZrEGneMN1aQYlaY3Yy9XgGWRuXMPHN9df4G\nAM+WKxasM/ZuhqgQsBEEVnp5I40kmOBkcRq79i6SwrZbwA5w0rbMtCuGW1GSevabRoolKRNjunCs\nTWkL4ElcxlMpNYlKOF1WfOQJd2x/501HgmZS9tKZaVBWcDhKqP2OoKhmzPyuyHrwXq51njrGXtae\niDx0ttfA2CMvsRXF3MV2A5Voeinmm/8s/Ic/1zU0hjWNp/zEd/4E3/vUjyL1zL/GOzcWD94GYLfW\nNsCfAz4HfAH4KWvtb/9eX/eqdX/QJVcJwIQ5nL4A6YE9u/ev+eHzGV80z7ktoE6JbIltU87L3scu\nQlNPU3YBYGHFMsEIQVMvu8JLMr7eSTGrjXFqZe2Kp6MwUmt8QmyvgYA3tXK2yc4VM2hQqj2wy6aT\nYqq6RFpLkmTIjYabXatqDEqWaGud+zxo7J5RL0iZFQ2TyAHv2eoBSdRLMZWwzs0zWNrvKoZSjMYS\nx+mAse/W2IfrxA+ZDnLMomo6Vg4OOBdly7JqyCLV+X0nqca27kY6TydQnFO3lmPOOJOSqddbN5eL\nHFbd+dkE9kxnPNF+P+riOwBIonwD2Asq0V9LecfYt4t93/PC9/Cjn/xRJvGEKJ4irCWS1YCxGySO\nsVurqbzkU9sGZSV7WURkRJdvUtmG1FridH33CD750CQUtI6tmo3sIH+uK5F2IP6aHxH57PKCUvmh\n0IGx68DYJSzWYwVq0xBZePbaiEi611rUC5Y2RTVLrLUUGNLhoBMP7NeblrmqWNYO2DPdM/a5zZi2\nphtAsYuxP2H99DPhztXDRcXHntzj2cOc33zN3SsXsg/1qqxBGc/YPbCX9ZyZNy3Yrni6ngx7piRF\nPaWWac/YfedpFHYz1Zy2LiiFwLDhirn+ka3PDs5R9+03/y0WL/0YP/j8f44pnnp3ATuAtfZnrbUf\nsNa+YK39r9+O17xq3akGKX8DXTR0agYphhd/njaa8Jv2eS/FpGhTYk3GeXlO4W8864E9qYstxh7Y\n4KqeU3oASydPdB2Xq41xasu6QghLFnJHRsck1k8C8t2C1Q67Y+6/g1K9Ta5sCrRnx8Gq9zgpRgkX\neYBOeynG3+xL7xY48JG6j4pTYiWpfGxvLSyxWmeJPWMP7hKnF8dJAlK5AnAYMdc2TirZBexTD+ze\nGbMom875Ao6ZL7wrJoBO+Dke2M+SMRTnVK3hSJxzLiX7yfbuALxkYSOWYUAK/U4urG+89r28cW+M\ntZYkyqikwPhz3FQFlRDEMjB2zbJuuuHnw/Xxk4/zQx/9IQBE4prJIlX1nacYpBUc5BEYD+z1ihqD\nss7HH1lJK9yOs6YltpYk3WbsQghimVILH3iwcf2Fc1HKlCUJFsFrxQOkkDxVlVR+fuwmY3dSzLqp\nwTF2wTjVPH0wBQTLeslKpMh6QW1cWvnaoJMBY1+IBbNy5QKxOmBXLEmYGsPME7RdxdMTc5clKSvT\nkOqU00XFwSjmW1845P992X3nuY77zlPboqzkcJxQ+njkojhj5tsNtPGMfQDstlqyEAJrUpbJ8ZbG\nHvtjU1RL6nLldjX0D8PHrXGisc2UZ+I/Csh3LAAM3qWdp/fWgJ2uSakbJScjJ898+ee4ePLTNGg3\nfktnaFNCm611nob43c0GJYDEA1tZL2krn+m+9ySJCcC+ztiXPu0wDwA8OmIsPbDHutPYN+2OWRhg\nPAD2qi2JsERJivA2xKuLpxYpawfskxu9FON9uMERMU1cWNNZNSOJFG1TUpsGI2z3fcOKveWu7rRq\n12wT+4EVxHnP2GcuGpnxdnE8SDGBsS83ADyPXTPHwic7hjVOesZ+lmRQXFDWLUecc6YU+9nhzmMx\nzSLXNVwX1K0F0bimscF67mjMRdHwaFGR+Js4zPlsqhUNosuWybwUs/ITty5dkRumrETZFU8bDNJK\n9vIIY127O9WCCoM0kjSSaO96WtZLGoxj7Ds0dnA1HvATgTblGH8uClJAUOicf7x4lQ9MnycCGr/D\nGcXrdseFkFtSTGVbNM72+jXXxggbs2yWVCp39lgPqqkaAruTS06aloJ5J7eMvYUwiSQLUvaM4fwq\nYG/uclucULZueHxjLIejmG994YhZIVFCMdNR74rBIK3iaBxT2AGwS4m0gkPfRFQMoq+LZkkrBJiE\nVXq9D5nzwJ76z1xUS5pi6R5+7C7W71rBAHD7zB2ndx1jf6fXvbpvl68vY+x3Pw+zN3l4898A6Bl7\nW2DblFk1o/Bjubrh09b28oJfqQffollhfbBRNrreM/ktYPdOlrZ0czqTKeNogq0nvBRFUK98UWqz\nQck9KISsu6yYunHhUXGcdjbExzF2KWtSKxxr7qQYB+xLX1QaJzFTmfKoXfI13EbWfRb7NrCvT/1p\nTYsC0sAm43Gvsd93SZocf+3WZzsaxwgxkGI2AHyUaJ8V03ZsErymHIA9SqE4p2wMx8IB+1667kgI\na5pFYDVFU/regqZLdgzr+WN3rl96sOiAvfQDIAJjj5SXYiInxYSJW5eu2DF2McxjtxaJZOTto5UQ\nsHpEKSXCKlKt0L5P4cIXhyMDabJdOwDIPLAvpdwuoPo/L30R/GemU15u5/zHL3yf+ywe2PMNKWYm\n1FbxtLIt2gpGseLZwxGmjVk1S2qVORdVB+yDz+nrKydWYLGIyMk746hn7Aub8VTdcKe4A9idPvZr\nzR1u2SNXZ/Iy5EEe883PHwKCSIyYqT5SpMIireTaKKbw+fZFec5MSrSJOB57KWYA7MHjbtuUMhuQ\nEV88TTwWlM2Kulq6403/MHzcCsM2bp+5z/gesD9m3R3kPdSDqSjBIROpCL78j92/Pfk04G1WOkPa\nBkzKvJ77eaeqA8vNEDDoPcquPdkBZTK+ThJ+vgHswU6V29ZdIEKQxQpTnvCV2AF7bertBiUbgL3q\nJKLaOGCP4gQhE6R9DLCHLBuEu8G64mnQ2N37ZZHiID/iTEq+pfpVB+yejWQbGeCJv4i78C/P2NPA\n2KO8e3BcBexaSQ5HMfdn7vgsq3ZNYx8lyqc7Np3HGmCSRFg/Yehcu4HWVeOkmDOlLpViQhBYaUqK\n2uf3bBRZnz9yN+7L9xckXlct/Xdp65JKCBL/MHA1gMYx9vgKYI9yHzfR77xa4UAnixXGxlQIWD6k\nFCCtRkrRAXsYyqCtII13b93zIJ/sZOzuXK9sAqLhb44jPiJzPnPgmsGNt2MGxu6ub8FDmWI3XTG2\nRRnJKNE8e5hjTcysXNAot0sL1/4wx6WTYvz7yNjJO1MPlmkkWZDwTFNTmBKh5tuM3VoOytu8ak5Y\ntSusjwi4No45niR88PoE0yZbwC6s4nCUsLIB2GfMpES1Ecfj3N0/Ie4XmHscsSal9s2MQMfYM+92\nKesVTVm4483uhrhdKzD2W6fuHP3/0sf+dq57TW+vqwYZy6F4qoWGL/8c3Px6LrTbqgfGDqDamGUz\nZ1nV3bxToI/tHayQKle2JbR+sO7oBqn/d5tAG/I+Mtt3seaxpq1u8JUooilnNNZpqMMGJY2XkMSA\nsfvwqCTNaGVM/BhgrxuLFQ2pkM5LXqwz9lBMy2LFQX7MWbbHNxS/jKhXbroMO4DdywGNCXG9LRIY\nh4t0KMXc+yKMTrZ8vWEdT1LuXvTF06HG3jP2DSkm1YAiVWPOlOoY+yGnXAjB3obVLKxpGoGJqNrS\nuZ/kNmN/6iAnUoKvPJiTRKHg5r5LW7viaXgYZLFmWbkcn6sZu5Ni1iYoYVFIUq1obUQlBXbxgFKI\nrhs28qy0B3aJviQwahQPLIqXMPaFjYn2/zl3FPwIhwh/DVgPWuGhKoQgEimnIsHsYOzKM/ZnDnOs\nSbioFhg9grbs8uTXrpkA7P6BG4B9z0/lCoz92brxf//QyaTDtTolMUtebQ8pmoLWRxBcy935+9b3\nHVKUcedjt9ZSCRBWcW0UU4qU2FiKeu6kmDbmZJoSWcEqmBrorY/WJLR+LCPQFU9T75wqm4KmWrlO\nV946Yw8DrW+9J8W8tXXXFIy9dFHTA3vH2NsKXv/n8P7Pdqwp2B0BtImwWOb1wk1PagdSzAZjz/3v\nrNoSWicBpZObpGGb1u6WYtK21+sDYy+k5NXC3TzrxVM/2EMlWFF2n7kxNRGWOE4wyg3beFykgBEt\nidBeilm3Oy5sP0DkIDngUTrm+fILXK/f6EaQZRvj2BIPIt1gbeMKgZ0+Ho3WpZiTbbYe1skk4d4s\nFE/X7Y4jz4h3Fk+BVI458z72smnJ1QVGcHnxNNPOM27c8RSiWZcMACUFzx6OHGPXfS0FwHgpJkhx\neaxo/DW3SzroVjQiNRZLz9gbQKJIY0nrAbxZOmAPRXHtbZVh2o6060FjwzXpCp7yUsb+qIH46P/h\nkybhW2rT+f9F6hn74NjHMuNMRFuMvbKuBjBKNM9ec4zdUGJCUbHw0uQQ6Lzd88RLGzLxwB7kDS1Z\nknbALuIH24z9zAWWvW6PqU1N69Mar40csP+hZw4wbcaFAOqiq61hFXmsyCI//D0M2TAJx+NkzXkE\nMAv3bptiArDrDHx+e+7rBVVTYKpVx9jfKrAn2j2c73jDwLvNx/6Or3um5IY/l654GqbAe8Z+74tg\njQf2Ppo3AGjkL5R57baB68C+ftJGHpyLtoTWAWQyvo6MJ0Rmu/O1Y+ymdhcJTp81pdvq/evygX8v\nBlKMZ4UqwYqKxlia1jhgtxYdp1gVE1v72AlKrTAkUnsp5swVkQd2R/CMPT3gVILE8u3VP+mkmHwj\nn2SUBGAPEa6tB3YPDLErpGEt3P/SThkmrJNJwr2LXmPPk6EU47oUZ0W9prEH1hOLCRfCQnlBVdVE\nyj1kdzUnQeg8jWhszapqIMQ8bKznj0ZOY/d/F+aNmrqkEYLEH4+hbHQlY9cxCWBE04/GExaFIlaS\n1qdnVvP7nrG7P4eu2FBsVPby95gmQylmN2P/+YtfQuo5f05ed2w9xDh74B1+n0RmzKTe9rFjEdYB\n+1MHOZgEIauuDrXyO8J0eM9EKeiUa6ObKKGRsXvN0DeQRoo5KTebBoVExg+2H5RnrwLwOs5dVdXr\nwH44irFt4oC9Kbr7VxhNGimyWJNap43PpYA25XiSEBlJSW8PDc2F1iSw58Pbkn5+7sjvVqu2oh0w\n9rdaPBXC3SetscRaXk0I3ub1rgP21rTctxUnfmZVNdTYgyvmzucdY33ykx1rSn3nKUDs86CXzWwH\nY18H9ty3dVcCpJmhrHXe7nhEYukKr2EV4bWaPncmjxVt6Sc6+QLsTsYuo86hU7XG69mAijEqcW3n\nj2HsjfC+4uwArJuMFFwxK3r2eS29xmk14178JN/W/ouueDra+P5jPyYvjHdrTItEdEyaKHdgcv4G\nVLOrgX2a8GDuOjJLP/4urPD/D+dV/9r0rEcz6hIyzfIhVrub8jLGnkbKB5pZFnUJoiHV8da/e+54\nxKsPF11jVkhcNI1j7El46L5VYAcSJEa0nSumxc3PFEIgfGZKtfJSjC+KB499kGIUl79HiBzezdgd\nsL+4epG2uMnXZyfuwVt4GXG8T6zlGktOVMZcSuRy3cceCpIjP80qVRnIcgewbwDdN/1HyI/820yj\na4jIZfyE0LUkkixtigaOxMhJMZcw9jeEewjVjYudCA8j53jKmMMasGM1eexYe2wFq7ZkJiXWuOYg\nbdeBfR6al0yKmgZg7/siRv7er01JWxedK+at2h2h33G+kzIMvAuB/VHxiBY4qt1Brgcaewfst/8V\nvO+PgVRdd+mQsSfGT1Rp5y6LvSlJhHajrDekmIk/uYUQCDN3k4+gcz9sAnsABhdPEKQYDSbn0Ai+\n4Fn/eoOSBw/ptrrgQqga2xBZCypG6MS932MYeyMNiYz7vJbVKVRzbJRj6Ef+HWaHtLbl1/e/hZSq\nA/bxRj7JJPERqD7yoLEGYUUPyvHYgcn9L7k/X8nYU4yF108d+KwXT33YmLFrPxfC+aiVHXHmQ7LU\n6Vc48+zpMo0dekfTeTFDCNv9ebheOBpTt5bzZej2DaFRBRWCWG8z9l3ZJmvvKxQNQ8YukKE702em\nVH5Qs/Cph1EAdg+Wkss9zweZA5+ddke/45i3C2wzQaVT92D3jP37P/0R/s5/8E1rMk+qcpZCIqu+\ndR4CsKsuCXISjxCyQnjwK1buQbAF7J/9r+ADn2U/OUL4YdCHPjAu1arbOV4nR14ixdR6wtwfk1Wp\nOBz1A2D28wjalLkwUBfdPWGtJot6YC9ty0xKWpMxTjSRVRSD4dQLv8O3JkFN/TyBIWOPYoS11KbC\n1gVL8dUxduh3nNP0nfOww7sQ2O8tXUfaQe0jBYbF0yDFFBfw/s8CdO35yYCxZ36+YdHOSSNF0RZO\nvoAtu+PIt3UXQiDtktj7jYndkIn6UsZeDoqn7rO+QMQXrQOOXYw9lZqWfvRbaxsiHLBblZBY0+WL\n71p1a6iEdb7i0CS0OoNqgYjHRMozjlhzlB0B8Pn9r+u+H2wD+9jbGsPN46QYNqSYOdz/gvvzyYcu\n/Xyh+/SVBw581u2O2yDffYZEgxlx7r97fPYS58rnxFzC2KEv6p16eSPbEYP7nLc83r/wiYv+WjJ1\nSSV7YM+i/jM9jrHHQtEI0xdPBV2ypPAdnFXxyDP2AOzuv0GK0fZyIAjsd2fxtFoCgqWZIc0YmYz9\nru0CdMrx/oRveWHd+5/qnFVAggFrL7FgVc860xFClmhfCC38v83iCbvWUdpbCI+ykJgoKESMRXDd\nJMj4YTd7t1unr7IaPYmQYTas5GDU77b2sghrUlbC0jTLXoohQkqXO6QRzKVkJSVNO/I/U5SDskXo\nSqVNSfKRK/wm/WzbNFJE1iVHmmrFQgqUUFuNblet9xj7W1x3lncAOKh82uHAx94xdiw89Y2AA0gp\ncA4Df5OOvMa+svf76UkiSAubjN03O0gBlEQdsI/IrdkC9hDFmg5yZ777ozf4kc+8n/fJlLmPbE02\n7I4AmdA0tp/e3tqWyM9GFVFCak0HPLtWVbdUAlcITNcZO/GoA6QsUh2w3x5d56GdumhiYJKsbzMn\nyboU01qL2CXF3P8ijI4vdcRA33368i5gX5Nl1oEzNCnN25IaSM9f4ky6f3MVsId6wbmXIbIdjP05\nb3m8e+6Hm4RMHO+YiHUvp4W1mRWzuVKpqTGUTYttWxohUJ44SB+GVRfnDtg90AcpqCuecjkQdMAu\ndkgxtUt2LMwMzdgx0HrpHFLJbgDOdUYtvUQxiHquBAijuu9+LZuArNzDAliF4mk8Zdc6zpz8aI1i\n6gmCEIJYayqZcbPVCFkxrzeGfJy9RjV5GqS75haF7PR1cNeD8N2li7bqZrNKEaRGTWQkD/zDv27H\nrrsXvQbsC9t6oqbcvXHwNTA66v4+8cBe2xqagrlQZCq/tKi9awUv+ztZOIV3IbAHxr7vk+sqQTen\nMjD2yNrOdlU2zqsuxDqw70XXKeRLJJF08079NmtTY9/zQFcIQSUsKmyRkzGZNTR2N7DHTQ/s778+\n4ce+8wO8IPvdQLTRoAQB2L0U0xhaWnQnxaTE1q7nSW+u1uVZJCrtpZjizA9eGHdAmsU9sNdizs+1\nf6hrvhhvxBaHG7IMrhhrfEHNg1s8csf/zuevlGGg7z7tgH2YFTPsQt1g7JNUYxqfFyMl+cVLnEmJ\nRDC5hC0CjKIAlg7Y82gb2A9HMdNU8+aZ+37h/LVtAPavsngKJDKi9oy98Yxae8YeBfdNcUYpJco3\nG8X+v53dUWzXA8K6NsqwVl3C2BeUUe6jcif9DvTizcuBPRpRS+8WKdbjOsSAsR+PpwhhIPPypP+s\n6SWvG+YhYBPkwLqZRopSZtz0dbJbi8E4B2vh7FWa6dNdhtOiEGvALoQg817yi7ag8q6vAOxZrFBW\ncT9Ee5tRB+yFEC76ApjTktHfE/w7fwv++F/s3ifRksgKGtNg64KZVGRfhQwDPXl5j7E/Zt1b3kNZ\nyBsfTrWLsVu6i7ioB0H+nsElVDyRfogmfonUu2ISdgP7OPX+ZiHdjRiYVOyAvd0EdhMY+3KL/T8f\n9cwm8YAN9FKMUD2w18YzdgsyQkapG0HWXC7F6HpOKYRzKaxJMXP3IIq3GXvFOX+l+X6+9IzrTAx+\n47D2PLA3/tg2GARynbED3P3txwL7cZBiHjpgzzeyYnb9f/hzXXv2rSTj+cvuv9EUKS6/hIOsNPcx\nAfmOtEQhBM8dj3njoR8w7b9n65l75AtlQ5b+OHdDKmMqYalaQ+07WUNkcGi/X3gZQHmACkmCQYqR\n4nIgmKQRmHh38bReuugFIBWTbhoQsztrMsNw5TqnDcPCvcZvm9rthm0/COXmxJElmQXt2z0wk0sY\n+82xA/Yw7zSsREtKmfGEH035+mwA7MuHUC8x06fBSzGz5Tqwu8/svtfc1F1TmRK9OUBZyYUHdtPm\njBJFJCJW0pst2oa5gNTfz6lWcPgCBK0dt8tXFmofuDYXassO/Lg1eQ/Y39q6u7jLsYEGDRbXxbep\nscsIVEhDbPv0OH9SUiquxx8ENcOoR27GpJCA6OURv0ZxhDCKQggKIVCBScUjMmMwdnPQh9s9JPVq\nC9jfl/QyxWakAEAmJJUZSDEYFAKkREaOsYd0R2stf+rv/yl+8gs/2b2maufeopevSzGlk2JGsZuS\nHilBrnMynVHaMx6wx+3EXdDTjXySvdRLMf7YtliEFT2rDozQ1Fd62MEB4jTVvPLAsczNrJj+/9eB\nc5RoqtIzWqmYrG7xQEbsXWJ1DGvsd1szD+yjHRo7wAtHI15/GBi7l5wCsEdh4lD/+R4H7ImKKYWr\neYTguJBTk/iC+dw/kCIvzUiVkhjba+xie3cR1tRH985ltNMVc+YfaJN4v2fps9uXMvZxPMIqF+gV\nGHt4IFkTdd/9+UN3/X78BUcKimpOZgwi3u0SeXLigZ31455GikLk3KgrsJJXL17t/u7v/tb/zA88\ncZ3F9HrH2Jel7JqT+s/sgH1mqo6xh1F2eawRZkAO2pRRrIlE7Bh7U0KzYi4lMRGxlms7irCEEERW\nusbHpmAu5VdVOIWepLyTXafwLgT2e8t7HLeWhghBtNsVM2DdZWN6F0MUmHHNUfRBAM7tlyn9VHii\nHDb0szRSLnNEiLViV3DFbDL2Dtir5Rb730/23AgughSzrrGniE6zLxuDwaCt+zwySkhsH2HwqHjE\ni2cv8uXTL3evr9qQbz1ygCujgRQzIosVuZelhBAcpocUxjG0RbXCWsUk3XiwJRHSOpsjOGDHSsbx\nBrDDYxk7wMk05fZ5GPm3m6VvFk8nqWZVus91piTSNtyXMftXOGIAJv4hNfcgNd7B2MHp7HeCxh4Y\nuz8P8SBSIKzHauyhxV40bkgD/cT7EFEw80DSOXWilNSa7rNqebkU4+x+CQupdvjYF5xG7ncP0oOe\nsS8f9sOXN1aQ3woh+pmy/mFord4aytH6+6molz6GYzewPzVxTT9qA9gTLVmJlKRZIdpDXpu95t/L\n8r+98o/4fJLw105/sWPs1kZcG68fjz2/S5gJ231WJQf1EDM4RzYjixRJAPa2grpgIQWxja+U1rQV\n1BhEW7KQcssO/LgVNPb3GPtj1t3lXY5bQ2UjlHCBSnYjBCwaMIiyNv3EmwFjj80T2DYTrm3VAAAg\nAElEQVThUfs7rngKWwwbQv61ppSOsUvZh1+l1na+87BqWyGsdgd28/WijBd8x10itIu9hZ6xW+E7\nWa1j7MLgTZjo2DH2yj84Xrl4BehBC0CbsDV2GTVkPi+mWkA8YRSrtZyTo+yIZeuBvV6BibamA40S\njbZ9XEPjgb3vPB1c6MeXO2LCOpkkYS7KGgtea1aKt6WY+dLr5f7cnirFwWMY+77PMw+535vNV2E9\nfzzuBpmEnYnx1sphumNYj9XYw05M1Fws/IR7D+iZ1/lnvqYRGqPEIC8IQMvdnxW8hc7EDtirbVfM\nmXYgcpgerD94L2Hsk3gwQzQAu+99sAMppstutzWomFWzIjN2a5xkWDe8xh4cP93H0IqFyEnbGbo9\n6Rj775z+Dq+UD/hgWfF/PfgVov1/6X7BxFuMfd93hc6kpAq7HN270Gw72GHJEVIKIpWykhJbFx1j\n11wN7ApJTYtoSlZCMP7dMvZ3MLIX3mXAbq3l7vIuJ01DhUbLiFIImmqdsevB8IWyaXvG7gF0JGsu\nVoZ29Qz3qi86u6Nl5wWaRcoNRRYB2P2JjUc7gb0xZa/Db+pxUc4L/rPGQ0YWCnQILAZE6zR2DMqf\nIh2n3jfvvuPL5y8DMK+GwO47YwNLC92n1QziEdMs6ny14IB90Tpnw6pxYUubbHQUK6czdowdJG6k\nnfsi/pjlRzDaHaE7XMHyCJeD+VCiAVzGdumOUZAZZlJc6YgB2PP1kVA8zXYUTyE4YwQRksIGb/Mm\nYx9KMVffNgHYhWw4X64z9uDMmcv1bB4ZpQ4kAWFtJyvsfn2FsInrhNzReXrqCcPJ6HCt4eZyYHfX\n9LnOO1dMJ8UQdTWqboxecN60xZWMPY9yMGk3pCOsNJLcE8cc1HeI7Qmvz17HWsvnXvkcCsHfOK/4\n+PEn0KOvuM9g9ZbGfi0bALt/GIU6RRYrrO0ZctDjE+88Kuo51E5a0Ta58nwqK2mwiLZkJQXj+HcH\n7O8x9ivWvHZRodebhkZERDKmEIqycMW4Lism7i/m9eKp10tlw0VR066e5UH1Ko9Wj0it2XmBprHE\nGLeFK6VA+UEFHbCL9Zmrra1Rl1gniTI+XFYoYCIHT3AVAaJnbKJyrhhh0R2wZ2vA/sr5KwDMBhHG\nygZg91vukBfjpZgf/WMf4K/8u1/f/fvD7JB544C9aAuw8VaziFbOfdLSd1HK4Si0wGCu8K8P18m0\nZ2/5gCkpKTrmlG8w9pNJAjZGy8hF9wILZR8L7AeZO59zf4x2RQpAb3mMrHSDpo2hDYzdA7KSovNb\nb+5qNlfqmaMQJbOlb0jr/PDrjD31f5ZR0p3/2IJQlzN2AC3SS9MdH/rXvjG+1gVaAZcWT0Mn63k0\nHkgx7oEhRdLZ+zrG3iwhHruxeNbs3Ol2b1l9jGP9wfWfacVteZ3cLJjaPVbNivur+3zulc/xTWQc\n7T3LX/z0f9OlemLiLWA/yntgL30RV/tCdB6pLhFSWBh5MA459svi3DF2IREmubJmolDUwqDagpX8\n6pqT4A/O7vjO7g9+jytYHa83FQ91QqxiCiRlUTBmkO4YrzP2DiikAhmR25qLVU27fBawPCwekojp\nzgs0VhJrY0qvse/pAbAbS0uDtba7+FtbEYfDuvmgiHL+5HzBx258Iweq18bxVswwlUnIiqJuMUNg\nTxyw17bGWsvLFzsYu3UPuNRvU8n24ex1NzowGfPc0Yjn6C/Mo+yIZXMBNE4CstFOj662gtan4rXC\nIobt7oGxH39w6/d2rcDYwyDr4RolilXdbrlinE1SMNJTzvXKFSblWwF23/rezkFxaWNJFiuOJwkK\n5YtrK6wHdi3Xi7oCdhbahitEHSdihc8861h8Hq8DewiZU3FGZn38xbD+csmKLgP2eskDa7FtxtE4\ng+Hu5xLGHrJnznTWuWJKn6A6lISOc9dw9OXTL/OZeMRKlDujrofrJ777r26BchpJblv3Wk+aiDeB\nz73yOV6bvcYPLRUcfi1PT5+kfvMHiaa/BTsY+0GeolrNTAr2vMYejnueaIzP5MmsYJz4UZlRCi1c\nLC+4hnLSU5NeWTNRKGos0pQU4q0HgIX1kSf2eP/JmPefXG7L/f1Y7yrGftfPbrzZVK7FXsWUQtKU\nfk7lDsZeNmadhUYZuaw4W9a0xTMIfwgSY7bmnYKrjAtiVlKyErIfHec1dgSd7g3QUhEF4Nti7DkK\neF+x3L5xB8M2kLXbUWDRnh3HHtjBvd+mFGOtRQvHspJkwNjP3+g+7+Y69NOHhF5Qt2UXSLW5JIKG\nYe7J9vBiTj6883c3V7A8boI3OD1fim2pI/xOKiecKcW5lxquihMAOBy571wbB1KXMXaAo3GCtKob\nW2dx19JQMssj9ZaCnALApHLJsvCuGL/TGHWMfT10TcW9xp5Y81hgj1XGyg/FXlvVkkcYbDNy3Zpv\nQWMPyYvnKhu4YjywD3YOR9kRn7z+SX725Z/FxjmFEE4+ugLsPvHMAc8ert9XiVa8bp2z5nk/s/Vv\nf/5vo4XmMw9vuUYhIK4/xPzW9yGEYH9DY59mEcpETorx90Ac95HExvi6lZEdsQtzV8+LGavqHOPH\n4l2tsWtqAbYtaeRXlxMDbjf4cz/27d01/E6tdxewLxywX28aVJSSqIhSSOrKAXvH2AcXcNlsDB/W\nKZmoOV/VYBKeyJ8H1mN2N5ckZiVcO3IP7KPuRhx6y82VwO7/vDrtrY7Dz+V1bCErThcVrQDlbXFx\nkrkYApx979b8lvt/LzNUrSGWPqcmFBXTfaevw05gP0rdzSX0jNpeDuzKSzHGGowQSDEA5f1n4E//\nXfj4D+783c0VmpTyZPtmymPNKNZbu4bA8jVjziRdTszjGPth7gDFCHd9XAXsx5PE2VqlA3YTgH2Q\n4Z7F6rGFU4AkCprukoUH9jSkREbrGnuIrFBR1j3YE2uRjwH2VGUu92TI2K2FesEj22Db3BUch2Tl\nEldMkBfOZdIz9mDT3Dhmf+K5P8HL5y/zxch5wjMfUvfVrERLXmkdY3+fWaKl5t7qHp86/jj7TdUB\ne7hv97OoG24e1n4WoUzMXEoq/1lDbSmPFa0H9sTqjkRk/iF3sZox99+zNeljpBhNJaDyO7ivVor5\ng1rvKmAPUsxJ26KThFQnVELSVr5ByTN2PWByRd1uMPaUPAA78P69jwKQmPpSYFe4C8gI0YNDPHLM\nik1gr4nF5YwdcMC+eTPohMyzFyFqHi1LrIDIg2icZN2D5MWzFzHW8DXTr2HVrGhNS91atPATbULR\ndjhUekfRJzQpCT2jsZVPQ9xe2kpaazo5RsmNG+Fr/80rt+PDdd3HCmw6XwDGidoJ+Pt5RKQE0o44\nx76lnBiAo5H/zsp3kV4BQMfjBMyQsTdbv5PH+rFWR6DL6k/EilXpQafzwzsg76QY/3OdZgPGbjtr\n7mUrUzmlsNi6HzpDU4I1nNka2444GEUgZf9Qv4Sx98Ae9xq7D0PTap2hfvbZz6Kl5v9ULjguFXLL\nIvy4lUSKB03Ogoyb7T2eGj8FwB8/8HWag2fdd/SAezDaPm97WYRoE2ZSsqoXKGtJfHNdFmkaP3sg\nHkQi5P68zIoZc++kqZv8MYw9ohbQ+oynr5ax/0Gtdx2w78dTEutcIlmUUApBWwdgr5DWojYY+1oa\nn05JRcVF4YD9QwcuBCttm53gB6BlwnkodgWm7QcqQD/VyFo3YCH53TJ23+osVcXZKrSiu9dK0p6x\nf+mRS1L86JF7KM3rOVVjUNIHkAW2lw2Abxdj98Au9Qwrqq5zb3MpIWiE7eyk6qsIQdpcoXi62YTk\nfqZ3Ar4QguNxgmkyzmzzlpIdAfayGGs0QvnwsCuA/WgS07YDjX2HFJPFb1GK8R7rWBaswuAV7+YZ\nbwB7SA+N4qyrsbwVxp5HGVZAUQ8a5Lwsc25rx9gDID4G2INufC4jWAXA88dssxcj3efTT3yaf2gv\nWEpJKh5/PDZXoiVlY7gjTjhq7/Ls9Fm01HyH8g18HWN3x+hwF7DnERgH7GW9JB4M/85jRW38UJ02\n6hj72H//RbVg4XeyZZ1emdaphHPeWXwMxnuM/e1fP/yxH+a//9R/CbgbIVauQcnWvvO0WbrGnyGw\n14POU3DATt15qb/u6OMAjAcxu5tLi4SL0FDSjbOLiT2Ah6G+/197Zx4kSXbX988v76yqPmZ6rp3d\nnZ1d7a72YnWNYHUfi7yLJFjAGGSQEBZYQNhcskMhWTa2Al8REASEkY0VCENgQpjgsBQYhCRMIBsF\nAiGwkLSHhLSLtPfsTM90d3VVZVY+/5HvZWVV111d21017xMxMV3dVVkvqzK/+cvfmWYqH+hgLJhe\n36O5cKh2Xx+7ccX4Xsrmrq58NBZ73PGxP3Ahny3aK+yeFvaiQKac5x0O8bG7W4gk+AOE3cGhjaKt\n11cOKE5KLcx7ZvcWIQG87raTvOHOq/q+7vhqRJLEXMpaPK4D2KMsdscRRHmILpcf6oqphagsv+1W\nrTpKT9opB1yff+06z7tm+MUEOr1TPGeXpr6bjHUVbDUMENUR9qoOpvphjytmQGqmwaTd1cvdPls7\nKGBbNXGyWscSNcfdkJYCoAdaNy9DlhWTpHqFHeANN7yBp1TCBdftNM+bgMh3aaYZX+MEG+mTfN/t\n38dP3vWTrG09CeLA2rVAyWKv7BX29TiAdsSWIzTTBoFSeFrYq2FH2J22X/jYazqpYKe1w5ZOgb3U\nCPfkyJfxnICWCC19DE0aPD0oFior5nTtNKca2pIIIwI36B5mndTzDPKwJ3jqdwdPI+lUi16zcppf\nufdXuOUD3zwwCOQ7QZ43BcQl8TeNmoywN9MMnITQ5LEPstihr8Ue6Q6Knq+FPQBfi2gQdCz2By48\nwMnKSU5U8u55261tKhzBEV31agRshCsmcANW/FVa3jY4g4XdwyUTRVu7umax2AGuWov65vW++a7r\nBr7mxErI+Z2ItJLxgZVvAz4ycHpSGSFAoS+SQ9Z9fCVEKZ+GOCSNHZToCmG385p/8frxUjpDLaC+\nNGimAn7Hvxv7Lo6SfLIPnW6afljJUwfJhd0dUCVrMF1H6+0GG0rl7pCkzq4IqWTE3monVhEOt9g9\nxwPlsyUC6ClVpv10nzu9V137KiriUldt4imOhdBzaLUzvpod467W57nq5Is4d+ocfPb3YfWaoh2I\nmYW6UevvismyOA+etpuEoggj09fHI8nyc021A2ravbeixwLWkzo7rfx3W60KV60PdiO6kgt7MRbP\numLmw452UYRRTOAEpE6ppUCym3dD1AejUkpnxXRb7GGpqCgOXF504oVUW3ubdhn80u14VBJkMxxh\nR1tlrTRDJOkUUO8pUCoLex+LXbtiPDfhsr6AGWERPyos9ocvPczZtbPUPvpvgDyY2kyzon91scYu\nV0z/W8ij0QbibSHSKlrH9uKJS4oiMZ0K3dmE/eff9ALeec/o9gNljq+E7OzqAhN5EkcFQy1wg2l/\nK+wNynZtvxaiMp+WQNKso7SFNsx9MwgTvPacZtFStqJFJ/LzBlVNbbGb3jxhVHLFZAp3hI99RYt1\nHQVtMydyh4s6/lD1SncWJpd9gLBD3qhrW1c509gs1h31OW5iL+buWFvV0wi7NrS+nB4jzOqd2byb\njxT+dehY7L2pjpC7aSSr5MHTrEWgOsJe8V0aKv9cpR0WfY3WKvlnspvssq3TOVtZjdNrgz9r1wlo\ni3TusKwrZj7s7ORfSBDF+K5PKuDodMM03dWumNxi6hpkbfBjQtUR9sh39YVhcNpWUMrlLVcvuvr3\n24mx2NvgpERFp8gBwVPoI+wRsb5AeV7aEXZz4ngdYU9VyvUr17Hy9EP6/XNXjKOLpfpb7AOGIcTH\ncLwtcBKCIcKeCVze1WllU4hdmTuuXuPMxmSWz4mVkK26nhcqT+LLXkuyH+aualBg2HB8JaStAhri\nkDZ3ULqNbTCkZ8sgIu0icp0WiW4qVtFTjyLfRVTneFwphL1SVJ5GSuGNEPZ1HSjcLbfuTepFn/q1\noI8bboArBvJGXUUYtnGpaA8dR/0/59ev5XUL41xcezEFXl/TKY9mxikXH+4SduNj7+eKERE8Z4W2\nCJdUSqgUYdiZ6ftUlrczeCS9rtM6t5p/Jo20wY7+zJrtFU4NEXZP799Ft7tfzmFn8YS9nn8hcRTj\nO34+xMC4YtJG7lvSB3JfYfciAsrCXirLHvCllQ/eakmsfV3JtqPTLTsWu3bgTyTsYeGKcd2Eemoq\nH7WwO17ejlhzfXSMms6i2U62SdoZqlfYo9EW+/HKMcTfRETlfdz74DkuqcCFS7onx4wW+zScWImg\nnX9+qfv0+MKuhXlYG1zIhT1TAS2BdqNeuGKmstj1BdWRJqkW9poW9twVk4tEPmIwX1c566nsLx7E\nemimKJVa97bqhcV+JOpxw/kVcAd7Xj1C6mYeaOMSuzooWxkQoL5r7Sa+f/MSr/aO9P37MIzF/jVd\npMTFR/KeN9tPFoFT6Fjs/VwxAIGXX6iekQxfCRVdiBR6DqpdQ/DZTK4p+v4fXcnX2siabJm5BlnI\n6SGuGFOgVdwJWYt9PtS1K6YSVwjcgFQUrrbAk7SRW+yBEfbc6urNYw/08x3JK0s7wt7/C45Kgldu\n/Wp6YJSFHUnyW2o37DT5Kl4w3Mfup008x8NxkyLgF5i5qCKFIACc9WrUtE92u7lFq52hnDYBTsfl\nYFwx4gzct+OVY4inm4cNEnbxSBCe2Xyme03PIsdXQpQWdiQhlPEq+cxdiDdkIhHocWsqpCEO7dYu\nmRb2aQLFfriKqxSOJKSmP7/xsQcuor9HJ3MLP7JTKlCKlMIb4WM/WjFTlMoW+w4XtcvgePlubf1M\n/m8IrsQ0TE/23U12dUJCrdLfyvfCVX784iXOBKODyb10LHYt7Jt/VwywZv1s53lDgqf53/Pj+4Lr\n4KlOppVIXqV8U+M/kG7dVljs1aiGqwfC77QbRJnCczyO1QbfdRjjbbOoFLYW+1zYNcJeqRA4AYmA\nrxLamSJtt/BKQza65p0a/AhfC3sxWclYPAOs2rJfvVLym7tu/vwd/frdJEWcNjEDyqxdP2+lC32F\nnbRJ7MY4TgKyN4/aLQn7DRJS07fu29uP0UozlLSLTJ18m2FuqZluj304FncGDkfuoBiDRypw8XJe\n1BGOyNiYByfKwg5E7pjCXvR6GS7sIoLvVXKLvbWTt0xGhg7yGIgXEiiFOAnKFM3pu7W8qZyx2EsF\nT15EpO/AAqXwRwj7RkX72Hss9k3tMjhRKzVke9W74G1/OHR7vkQ06UxR2tVuwUo84HM258oUrglj\nsV+mStNbyUX94sP5H0sWuxH2jWr/462q7yYuuC6ekq589DhwubgdAk6RQuvoDprNrMV2u0FVwcnV\naE/xUxnTWOyi64Ji4kEbB8XiCXtd9/KuVPAdnzaKgLz3S9JudqU7Gos97LLYY3xtRRUHwiiLvSTC\n1dKwZ5MxUNcn1rZJbRvWGMmcCH0KlEgbxF6MOC0Qc7fROahNIDD2Yk60WoRK4SnF1uVHSVoNUkcR\n9uYVR+sDL1jQyWWHwd0PfSdPK93ayoNcwUEI+2rYaQpF3l5gHMxdyLBRc8U2gypNx6Hd3KEtCn/a\n00OESOWFZqIv0H7RBKws7F4xYBw3KIKnkVL40XDBPF7LLemufjGJdsUo4WS15Ibzo+5Aeh98JyYx\nDe0al2i2W4SZohYN+K5Nj6BphL2UzFCvXJ372IcI+5Fq/4vyuo4ZtEXwMukqHqsELue38vO86Bbq\nOMSZoqlabLdbxFmeoTUMM0FrUw/lmGTe6UGycMLeaGhhr1Z0uqMiJOFivUXabuWDrLXgNgZa7Pr2\n2Ah7a7iwxyVhr5WE3dfd5DrCrv3/Q4Vd/36Qxe7HiNNxxYQlX7xpB3x29SxO/WkEWMkytneeJGts\n0RAh6LVM4yNDhX0j6lh2g6wR3/VIEbbruY89GFE8Mw9yq80lEN3BzxscCCxjBln4YwRBK0Fn0HTi\ngD9F8Y0hRMBJiwu0CYKHvsOuHsScqqgjFCK4uid8OIbFfryqi23KwdPWDpuOkxcn1Ya/vpfAiUhJ\nAIHGJs12C1+pvvUG+Qt0jGPMiuMy5V5Ajeo12hXzSH6RKA2TroUuIoMtdtOTHcBV0jMMxeNyI7+o\nlvchVNBSCdtZQpzJ0MApdIb2XHQdghFxmsPEQuWxAzR1w68ozLNiMhQeCRd3WiRZK584pE9mEzyN\neix2V6U4ZJ0DrLDY+wtg2a++UgpqecEqblNR16lhO4XF3h5syRhLp0/wlLRB5EZsSQsw6XYlYddW\n59m1s7BzHryYWqbY2n0G1cznnYa9Ahav5217B1C22CsDTtJAZx/t1rcg6FRRPpsEXj6p3qNGix2q\nYwq7uSiPI+yrYcyjLWg1N0lFOj1/piBCUJIWF2jjqw89h0SF+sTrFhVPT/0JlSKMhn/GG3ENFNTF\n6bLYN12XrF0dWnTTj8CJUdLI+8k0LtFsJwRqb2/8zgtmcMWULPZm7Rp4+JM6I+Zsl8vwO89dy40n\nVga2cdiI1yAPD+Eqp6udclnk9wp7SqoSgrYMDZxCp0Br03EJh4wrPGzMZLGLyD8Qkc+LSCYi5/Zr\nUcNoaWEXLyysoLYDl3bqpO0Uv9S7opkYq7fbYod8ilJxwJgTY4CwlQWvbLGbRmAmNayuMwmiLB3t\niulnsWcJsRfp8n5drFRy2XjiE2bCrUdvhe2noHaCmhOw07yMalymIVKMXit42Y/BK97Rfy10qk97\n97NMqAvBmo083TEODuYAP7ESgrZ2xxV205tlLGHXYpo0L9FC8HqD3xMQipOnTEpbB1LzY1BEcLXl\n5/QEdD19IQkzRRAOtyRd10FUQL3HYr/gerqz42TWZeTEKKdVCHtLJfhq75jCgkLYZ7PY09Vr8/U/\n+pew3l2gdmI14t47Tg3czvFqJ3Dr0j23tEvYSz+HClqkbKuUMHM4tTr8czYN3XKL/QoRduBzwLcD\nn9iHtYxF2tQl1F5Y5Bi3BC5v75BkSVcWwyf/Ns/i6Kpy9DrCXlzhR6Q7lqfb10oWu4R5vxgzJd0I\ne6U9TNiNK6aPxQ5EToiSFp4Wdr8k1OKE/KvH1vieW78HdrSwezHbyQ4Yi703s+Xme+C2+/qvBTgS\nHgE9V7U2wBIPvIAUodXMLf94hJtgXhxfCUlaelCzP142hklPHSdt8Yj2azebm7QEginK5Q2huGRO\nPg3LU91/M4LeK+y+iaEohTOGYDoqbyfd7WN3u/vEjEnkVRBp04pWc2HPUjzVv1lb/oJ1QEb67vtR\nttjTVZ2t05PqOA4b1Sq+ztAsJxZAt7CXB7eESkhos6Xa+JnL6fURwq5dTruOQ+AsRuAUZhR2pdT9\nSqkH92sx45Do0XK4QXGythC2trdJVLvwi37kc4/zC3/8Jb79hVdzy6lSoM3rZ7FrYR8wu9EM+1XK\noeKXThjdk90MJTDCHmetvVWnhoHCroNrbkBGE1cHsvzSdpQbcFXSzvd75zxUj1MLVtjKmjitS7mw\n994JjMB1XFyVfz7VQRa7H5AKpLqTYGVEjvW8OL4S0tAj8lZHNAAzmClK1TEuRkd0m9+0eZlWv3jF\nBETi0pYMJdkef6eZsOX2BHQ3soD3nL/Aa+t1GCOl1CXsTnds1bWPvTowRXDgek2/mGgVdjdpqRRP\nyWBXTOUovPXDcOd3TfQ+0HMHXU7DnFDY12KfihF2eoVdt+JwnWLyFUCIQ0va7KDwM5er1oYfy1HU\n0Y5wyBzaw8bCBU/TpGSxa2FPRNjZ2SFVKZ7j8cATl3nHb/4/nn/tOv/+276uO5KtxSuUpON7H+GK\nMR34yPyug9IJa0Qqo6UFz6SIRe3BLYA7rpj+Fnvs+HlP9z4We+aG+Gaox/ZTUD3OSnyUbUeoXf5S\nPvhgCp+nTy6Spm9JL7EXkojgKF0NO+HFY784sRLRTvPPdXVMi/2YHrZx9droLBrTvz1tb2thn8Fi\nd3xSMRZ7dyaFccX0ZupkTsCbtrZZy3QdxAhciXW6Yy7sqrXNpiN41MbqQlmmqhurbQYVaFwiUfm6\ne8cUdnH9K4cG5gdRXpt0CfvgXkH9WK/4xJke9t4j7MZo670wBcqhJRk7ovAyb2RWTCXs7F+0IDns\nMIawi8jHReRzff4Nvr/vv523i8inReTTTz/99NQLNi16cTs+9kSEnXqdJMtwxeMHfvXTrEQe73/L\ni/Ye4GVXTJEVo4OLA0SxELye0XFOWCNUimbb5LHrwG67NTioNMzHTi7s7S6LvfQ8N8QnIUkSqJ/P\nXTHVE2yLw5FL99N0pG9vj1EEooV9wB2L7wWkIgTkFy73ALJiQBcppfkax2kABp2sonFK303/dlH1\nfH9naJ0QiU8iGYq9Frs5br2eO4K2di1mSNEIaxiexF3pjlutbTKB2B0v/lDmeHQ1AA+7bu6KIcNT\nMjTHe1rKxpFfWYW4u13vuKzFPmGWb8vtcWuZebq9F6YIh20nQ4ngZcHQ4qR8O52/h+5iVJ3CGFkx\nSqlv3I83Ukq9H3g/wLlz59SIpw8kS5pkODiuVzTIaomwW98hlYx6S3jiUoPf+uGXdg1OLtCWdESL\nuMiK2c2rMwecyKslYS/jRrkrZku3TjUDN+K0Mb3FLh5t1cRz+rliQkISmlvn8VWWu2KUy7YjbGw9\nQCMS4j7d+EYRynr3fvYuWYui65iA7kFZ7CHJpReh2lXWz41nsRtBH0ekT6yYz65FS+JOO4cpCN18\nQEPuiukWRyPovd0027pKNsHvtH4eQuDG7BiL/dHPcPHRP4fTx1jxJ/d7n9JNvb4sGXc3NknWwmLe\n7n5TNrZC383dMbsX9gRPR7Ea+4SZC7Q7A+Q1laIHe/fvQ1x23DwNMpJw5Pzact1K5E9+bh0UC+WK\nyTIFaZO2PjHKwdP6bp1EZWw34bu/4QzPv3bAwW0sdmmVCpR281THASeTSXGUnnKNDBIAABw/SURB\nVEZSXlQjzhQtXfDU0H1BwqQ5Onjaa0EW63JJ1QCL3QsJSGheeiJ/XD1OrXoSJULY/CpNEaIphD12\njqCUFFkhvZjMHEf3e3cPoEAJdPVpuk6yeVeX33QYRtjHsdjXdfA077/dJ8NoAkI3pCUqF/YeV4zJ\n0PF7tq/0xT0ZM186civsiJvPtf3N7+WirjbtagA2JjceO0mWVnkwbUBSJ1UZnpqPPJS/u8hz4Oj1\nUDs1MMY1iLXYJ2jn57DX85kVU5N6XDHlAr7KGFWktVKqc+w/uwOpZ2HWdMdvE5GvAS8B/peIDK9b\nnpGtRopPSmZOjJIrptHYJUEBPj/y2psGb0QLZVh2xSSDW/YCrIQhSjk4eyz2FY612zyVPsNuultY\n7GG6O0WBkrHYXTLaRW91r/Q88SNCSWhfzkcEUjtRTIXZcpx8VNkUJc/XB3+P3a+9hUowYDSeEXZ9\nsfEOyMdeHggcTijsw3qxG0yFcUMLezBF58Ly+7ZEtMXevVazlt42yZlrhH28C0rFq+QFSp/7bdh+\nks1X/AQAR8d0U5V53rXrZK3jfEnHi1Jpz81idx0pKm5D34XX/Ev4jl+eeDuh5xLooq5Bwr7HYi9Z\n9rUxGnrVSkH3eECH1MPIrFkxv6uUukYpFSqlTiql7tmvhfXj0m5CSILSQlNkxYhwfvMSKYqjtdrw\nieBFHnsytrDHgQuZv2fYs19Z5b7tHXZp8pGvfISmdskMG4w90BWjnx/r23bRczr9kq9evICAhPaW\nFvbqcVa0FbHtODorZnIxWg02aG/fNnD2o29cL44R9oPJiim71rp67A9hEovdPKflCAmz+dhDNx/b\nqCTD67kTNHeavcVkSj9uj2mxV/wKu2bbr/8ZLlZzX/WxytGJ17sW+6w4p3ksyyt+2mR7ApL7iUk1\nDj0Hjt0IZ1823XaUMfK6v9+48LF370NUusCvjnF3WwtLnV2vFGF/trm0mxBIiurxmyYi1NqXSUQ4\nvT4icKRFaY/FPiToGPsuSvl7C0qiFc41mlzFKh984IOdqTNqcG/3TuVpj9WrA0iRGZqghb3LYvci\nQhKykrDXgk4BhRLp6kQ5LoFuSTook6IoktIW+0EFT81YPRjfYjdW+CTC3hAhEZmpdULkxbQcQfWx\nfM1xu6fmQK81HVPYq36FXUe4/MIfhhe9lWf0wIpT1Y0Rr+zPmZWz7DpNNh2HloA/x8L00HdwBLwZ\ng7Oh9Bd2EzTtLbCKShfT9Wi0UFeCAEdHBGsL0osdFlHYSRCd42ssnwQ4KlukIsU0+IEYi12S7uDp\nkNdFvguZtyfv2I9rCPCq7Az3X7ifp5L78+erAd0doWSx91iDlfxkjJPcBaOcJq5SuCVhd4OIgDQv\nTnI8iI9Q0wGd87qr3zSDD0y3vUEWu6cFTjl6mPUBdrgzd2OTumLGsb4Li10kF7YZgsSh/p6bTlYM\nJC/+ZjJ1BgTQ2wMGnvSyGlbJBB45988BeGLnGVTmFQ3CJuXOk7kL82HfoymCO0O65yhCz+10V52B\nWPJj0XN7hV2nO/ZmxZSOg43q6F7ygecUcxBWpohfHRQLJeybuy0CUkSfAKbKtCXC3dc5JCL4o4Td\nK2XFmNu0Vn1oz4vQc1DpEQK6b3HFC0mUy13JEap+lSfbnwEluZ0zaHtnXgI33wsrPUObdf/sWKeu\nZa7uVFk6YB0/wpc2zs6TUD0OIoWwP6OFPZrC/320ElAJ3IFiaTJz2qbvyQEK+wkt7JMGT8cSdq9j\nsbdEimygaYj0cVh3ZI/FboKy0R53XP7dtccsjFrROdZPb2/l/+9cQLUrbIxI4RvEK87eBsBXfD+P\nMczZYh/34jyMWFeD9tZWFMLeY7HHpe/0xMoYwu46RfB7JVycdMeFEnZjsTv6Syz72F9xtS5UGCU6\n+uS5Yd3h667WQaZkuLCLCOrxH+B09p29f2CXiFra4p7r3phvXnRy2yCBPXUHfPf/2Otjd3MLPNY5\n9ZnTyi2FUsqdq1Ovgu3HcmGHwhXzjHanTGOxv+Ul1/Hhf/rygalfxhpqO1nX44PgxIp2rczTx26C\npzMEiUNd8LPtyJ4ukZG+W4t7ti/6cTbmOL51PSnsmXou7M/sXpyq6tTwsutuBuUWwt6bZ7+fhJ47\n9nc4jKrOLQ97ZgkUrpgeH3u1dBd2anW0y8pzHVwj7FPUiBwUiynsQbcV1hIh2cmLnvxRfjB98rz1\nxae47bS+ZR3higGI/bivNVyXCC+tc62Xp/sXt3rT+OMqG0S6H0vm6hbE5SZgOkIf7jwGtRMArOiA\nzhPugNv7cd428LjxxODbTGMNZdpid4eMWJs3k7piTlVPcTQ6yvVr1498riMOHk7Jxz79nUmoRWDb\ncfB7XBqx199id3QaaTbmhfOIHrd3YTcX9kd3HkElaxP3iTEEnkekTvDFwKctMtW813GJfKdwAc7C\n9f5VvG6nzpmw+w44HmSxa8MuzjLWV8ZzWaUqf83agPmvh5GFE/ZIUhxvb7pjups3/Bop7I6bTzEy\nbQRgpMUOuf/ZBBnLNLSwf+GRCNm9iXVzgZii6x2VDeJmnpWQOi08pbp88b7Op493n+DxdIX7fuH/\n8tt/8RSOOPxNdBZgquDpKIyw7+rxYNOMi9svrjkSE7jOwFauvRyJjvAn3/Un3HHsjrGeH4iXD7Se\nUdijUvOoXh+7GZ7S201T9ONszGwcMx7v4u42j24/yvnm10h3bpy4s2OZE/F1PBh0Z53Ng9DbH1fM\neniEn33qPKs9I/qO1QJCz+HqI92fsWnBvZJlOGO2n66Tf85r4eII+0L1Y68328RuGzF+UxM8FUjq\nF6AC3jjDZv0YdF+XfAPD0x0hP1A2+lhCuxLjtXf5xENP85IbfoR33PQVuP8zUwr7MeLNL0MFUjfF\nT+m22PVB6aqED30p4QvqMg/9wTbrt1Z5oq2LtuZwMnqlbBHIZ6AeFN/zDdfxkudsTNwLZVxCx2db\nu6Rm6YkTBh1rsDeHPvaMsHdb5q7JmR7zO9zQwr7Z2OZPH/1TANo7N0/tigF47tHn8LHmnwHzFfZB\nwzMmJdBN3ryequn1SsAn3/XaPZ+FaehXzdTgRn09OMonY7wsmsPCQgn7T33rHWSPhYV/uuyKSXcv\nQCXAH8cP5kWQli323ZFVb//1Lee6+kgbmhLD7hbn6y3uueVWnhvkKWfTCftRosf/EioBSpQOnnYO\nzKBUGfqCW27iD+95Jff+/P8hTUJSPXFgHha7sdCbWtjdGfqUz0ocuNx+evIByuMSuT5b+s7En6Wl\nQNgRgd5mYhV9ga71HCMmhqLG/A6NBbnV3OGTj32G2NkAuQq/z53luLz46lv42OP5z9PEa8blvffd\nTtqeurNIQairhf0+HUf7BZFX9GdWy7Ii3jYKRzwyYC2yPva54bRbhdiZEy8Rh0Tn8HrjpCT5EZhm\nYkqN5Yo5tRax3scSajoRfpr3Y3/lzcdBV59O7YqpXygeegN87ADf8HW3cMPxGj/0yhvYaXikkvtZ\np/Gxj8J8zg0RpDQ0YhmJ3JBtLewzFSiV2gr39py5Zf1Omudfxc1Hbu/6vWcGmIzRshfyAiWAS61L\nfOrxT3GEOzgyoyVsUh6hk9kzD47VwpFj6cahort3BmO2kjafWU1l41vs+KjMI/IWZzTe4p2h7VZh\nsRuXQMv1SZSeLTmOK6ZssY9o2TuKLfcIG3KZO69ZyzvFjRjaMZTqMeK0VTzMs2JKJ3nZNaCzYn7o\n1c/BlwoiegzgPCx2/TnvOrKnodWyEbpBYbHPEjyMoo6w91r+N5/YgAtv4IaN7tJ/b1AfoQFUdBvZ\nzzz1F2wn2zzy6LUcmTJwaji7drb4edLe/gfBrS++my+f+Q5ufsErxnp+ZIQ9U2Nb7K74kIULM8ga\nFlXYtdiJjty3HJfU+H/HydjwShb7jq7ijCcvwwa47B7lGJd49U06dWqWC0VlAx9wtUXc64rp+lln\nxVQCj5uOd+aWzuP2ueyKWSjf3RSEXtQR9lks9lL5ea8v/aaTKzz4b+/lzEb3xf/oWu6XP7UxnqvJ\nWJ/R2oOA8LqzL+cHX3nD1GsGqPpVNrL8W44PYLbtpHiVdW542wdw43Fn4OaGXzUb32J3CRC1OGPx\nYMF87EAe9Cy5GwI3IHE8En0xHafZUx481QJ8/ov5/8dunmo52/5RPMm4+6x+XyPs02RUVDYQIHZC\nttu7+ZdTvlD1sdgBnrNxjAf1UN9pCpRGYT7Thsgcu4ccDkI37ARPZ/Cxl5ux9dtOP+vP+NhXquP5\ncs3dWaLqPO/48/hPr5+u30ovN/g1nmlvctXq/GIZB4UfVPCUmsjHvs6d7O4eG/3EQ8SCWuwdYfcd\nn8RxSbWLYCxh96JOVsz5h/L/pxT2nSD/wu9Y7cydxPG7BXlcKvm2Im0h+71uj8L3KsVzoZPLDvO1\n2BtyBbhivLjwsY91LA3cTukYHdfyN68Z8zt0HZdYX0Bednp/RB3gOVF+91pdoErLsXEDfvTiJm/c\nro9tfF3lvppa/ZvnvLD9ZfGEPW1253a7Pi3HJTGumHFyrP24Y1mffyjv0zJl46RXvTDPj3aNSydp\nTOdfh3yOJHnrXugn7FHneaULR7UUV5hnVkxDHNwrQNjb+liaycde+h78cbdjvt8xg6dAIewvvfql\nY79mFNfHuZsvmGPw9MDwQv7RpS1uT7Oxja9K4A4e6n1IWazVZhlkSZdFk/vYHdJJXDFe1Mleefqh\nqa11gNtuuhE+Qj6DFMbKiR+IbgQW6evtHmE3ll/JDQMdi90RZy7FQ+YzbTmy/MJeuijP2rbX4E1q\nsU/gTqt4FdIg5Y6N8QqwxuGu9edy02Of4Ezt6n3b5qHBfBcTuEp/4nU3c3k3mdOC5sNiCbueUFS2\naAI3yNMdZUJXjAmenn8Ibnn99Guqncz/334y/3+M9gQDCVfADYh1eq/fm1ZoTvgeYTeNwEJ3PpH7\n8sXCX8CbvEkISyf8LMLuOi6eglQYv5mY+X4ncKddv3Y9J6sn97W24Ia1G/idR5+AeLq72ENN0fJj\n/Ivnc44vTsWpYcGEXfvFe3zsLcchLT0eia/THesX8qHQM1jshDUIaiVhH50TPxCRPJdd5amLe0TU\nWHQ6I8ZgGoHNww0D3cLuLlDK1zSUU/xm7ZUSibCNmsDHHnT/Pwbvu/t9KGYv9Oni+lfCC94MJ27b\n3+0eBoq7oiV0M5VYLGE3Od69WTG6aROM6WP34txiLzJinjvbumoneiz2GQS2skHc1jn5vVaY2e9e\nV4yeojSP4qR8HaUOk8tusZddKDO6tQIcoI0/7veydgbOfT/c8Jqx30NEkP12j1WPwX3v299tHham\nsNgXkcUS9sJiLwVPnbzF6ESuGGOxn38wf3xsyIzUcaid7PjY0xmCp6CF/XFw2Dtz0oshWt+zXhM8\nnZfFXv5MvSWuOoVuYZ+1V0okLqj2+G2OXQ/e+LMzvadlBFPEMRaRxRJ2k6LYlUrm0xSK4OnYFnuW\nwlP3526d9TOzrat2Ep78fP5zUt9jUU9EZYNo82EI+ljsrgc/+lcQdhdjmODpvHp7lD9TK+wTbEs8\nUK2ZmolZ9pnCYl9uV8xinaUmeFpujOUEtIBkkjx2cxv2xN/Axo15K99ZKFvsswRPIbfYdWC3d0BD\n/veje9K0jI99Xq4YKeWvX0mumJl97KYeYU53UpYpuEIs9sU6S/tY7IGbC3s6kY9df6mPfxaOzxA4\nNdROQPNSLuqzBE9htLD3e3t/vsFToBB27wA7Oz4b7KvFro0Ma7EfIsz3ay32Q8QAiz0VNVlLAXOi\nNS/NlhFjKFIen8qDsrOcyNVj+TBs8jF7Y729FvZ59s8uLPZld8V43RlXM21LW/zekgfqFooi82i5\nv5PFOkvTPsFT16eF6rQUGKe/R/lqvR/CvnIq/3/7Se2KmcViP0qcaWEfMyvDdVwqXmWuFrvJqe+d\nBrRslC32WXrFQN4CGMD3ZjgeLPuLtdgPIe0+wVPHp6XUhOmOJQGcNSMGOnnlW0/MVnkKOo99MmEH\nWA1Xi25/88C7woRdkJknRYVHzgLgVxergdRS414ZFvtiZcW0dVmv21152iIrXDFjnYxl4d3YD2HX\nrpjLj4Jqzyjsx4oCpUnyqN/70vdysnJy+vcdgRH2g5ye9GxghD1wg5mreCOdvTSrS8eyjzgOON7S\nW+yLJez90h0dn0RlJCI4OOMJj7lar50ZORJvLCrHAIGLD+tFzRg8LVwx4wvCS0/vXxOofphArnuA\n806fDcKeebr7sS0r7IeMl78Dbrz7oFcxVxbrLC2Cpz2Vp2Sk62fw3cZ42zHCvh9uGMjTD6vHS8I+\nS+Xp0U7w9BAJgnHBzKPJ2GGiEOMZ/evlbS37Z7ZwvPY9B72CubNYPvbCYu/JilEZrVvfOP4JZIR3\nPwKnhtrJ/bHYvbBoxTp2j5FnAWOxL72P3ds/K9ta7JaDYrGEvV8TMG1Z1ZP6+CeQmSB/fMYeMWVq\nJ0rCPpv/7jD6Zk3+urvk1mfZxz7ztrz9s/4tlklYLGFP97btNeJXTycQ9iNn4Tt/DZ73pv1bW+1k\np8f7jNkp14cbvKy+yx3xidFPfpYwQWnvEF1s5sF++thN+ulhukBbrgwWS9j7WOzGsqon9cl8mbd9\ny/5GxldKGSkzplJVKsf5xSef5tpwugHb88CkXlqLfXxOVk4SuVHXhCuL5dlgsc7Sfm17tWW1k+wc\nrGVUKwn7rPnkepISh+gWvmOxL9YhMyn7GTy95+w9fP1VX2+F3fKsM5PFLiI/LSIPiMhnReR3RWR9\nvxbWl3YTxO1q2mVOwN1092BFpzz8YtY7AT37dJJJOvPGfLbuIbrYzIP9DHi6jsux2BYnWZ59ZnXF\nfAy4Qyl1J/AQ8O7ZlzSEtNllrUPHYp/Ixz4Puiz2GYXdVCoepqwY/dkuu4/ddVw8x9sXH7vFclDM\nJOxKqY8qpcxUuj8Drpl9SUNot/aInbHYd5KdA7bYT3V+XkZXTOFjPzxrmhehG861oZrFMm/2M3j6\nNuAP9nF7ezlxK9x8b9eviqyYSdId50GXK2bGPhRG2OfUX30aPH1H4q2ePuCVzB8r7JZFZ6SJKyIf\nB071+dN7lFIf0s95D5ACvz5kO28H3g5w5syUE4vOvS3/V8KcgAfuYw9X8slM6YzdHUG3KOBwuWK0\ne8lb8uZJkKcpWmG3LDIjlVAp9Y3D/i4i3we8EbhbKTVwXLpS6v3A+wHOnTu3b2PVjS9UoQ7WYhfJ\nrfbLj87uQjn+XLj6HJy6c3/Wtg+Yz9Zd8spTgHe++J2cqvazZSyWxWAmE1dE7gXeCbxKKVXfnyVN\nRtDTm/1AWTkFuxdn3068Dv/4j2bfzj7S8bEvv7Dffd1yN4iyLD+z+th/AVgBPiYify0iv7gPa5qI\nspU+a//smamdWNp2oOZztlWUFsvhZyYlVErduF8LmZaylX7gFvsd3wHH9rH/zCGisNivAFeMxbLo\nLHwZYTnf+MCrIm//1vzfEnIluWIslkVnsXrF9KHsGrBugvnRKVBaeFvAYll6Fl7Yy8FTKzrzw3y2\nBx7HsFgsI1kqYbcW+/ywrhiLZXFYfGF3rLA/GxSuGGuxWyyHnoUX9rL7xbpi5oe12C2WxWHhhV1E\nbI71s4D1sVssi8PCCzt0/OzWYp8fNivGYlkclkLYrcU+f6wrxmJZHJZC2E0A1VqT88NWnlosi8NS\nCLtpJWAt9vlhXTEWy+KwFMJufezzxwq7xbI4LIWwWx/7/Ll943buPnM3N64feN83i8UygqUwv4yP\n/cC7Oy4xG/EGP/eanzvoZVgsljFYCovdumIsFoulw1IIu3XFWCwWS4flEHabFWOxWCwFSyHsNo/d\nYrFYOiyHsGsfu7XYLRaLZUmE3frYLRaLpcNSCLu12C0Wi6XDUgi7rYq0WCyWDssh7DYrxmKxWAqW\nQthtVozFYrF0WA5htz52i8ViKVgKYbc+dovFYumwFMJuLXaLxWLpsBQm7t1n7qaRNlgL1w56KRaL\nxXLgLIWwX7NyDT/4vB886GVYLBbLoWApXDEWi8Vi6WCF3WKxWJYMK+wWi8WyZFhht1gsliVjJmEX\nkZ8Skc+KyF+LyEdF5PR+LcxisVgs0zGrxf7TSqk7lVLPB34P+Ml9WJPFYrFYZmAmYVdKXS49rAJq\ntuVYLBaLZVZmzmMXkX8HfC9wCXjNkOe9HXg7wJkzZ2Z9W4vFYrEMQJQabmSLyMeBU33+9B6l1IdK\nz3s3ECml/vXINxV5GnhkwrUajgHnp3ztInMl7veVuM9wZe73lbjPMPl+X6eUOj7qSSOFfVxE5Azw\n+0qpO/Zlg4Pf59NKqXPzfI/DyJW431fiPsOVud9X4j7D/PZ71qyYm0oP7wMemG05FovFYpmVWX3s\n/1FEngtk5K6VH5p9SRaLxWKZhZmEXSn19/drIRPw/gN4z8PAlbjfV+I+w5W531fiPsOc9nvffOwW\ni8ViORzYlgIWi8WyZCyUsIvIvSLyoIh8SUTeddDrmQcicq2I/LGIfEFEPi8iP6Z/f1REPiYiX9T/\nHznote43IuKKyF+JyO/px1fCPq+LyG+JyAMicr+IvGTZ91tEfkIf258TkQ+KSLSM+ywivywiT4nI\n50q/G7ifIvJurW0Pisg9s7z3wgi7iLjA+4BvAm4D/qGI3Hawq5oLKfDPlFK3AXcB/0Tv57uAP1JK\n3QT8kX68bPwYcH/p8ZWwzz8PfEQpdQvwPPL9X9r9FpGrgR8FzunUaBd4E8u5z78C3Nvzu777qc/x\nNwG369f8Z615U7Ewwg58PfAlpdSXlVIt4DfIUyyXCqXU40qpz+ift8hP9KvJ9/VX9dN+FfjWg1nh\nfBCRa4A3AL9U+vWy7/Ma8ErgAwBKqZZSapMl32/ypI1YRDygAjzGEu6zUuoTwIWeXw/az/uA31BK\nNZVSXwG+RK55U7FIwn418NXS46/p3y0tInIWeAHwKeCkUupx/acngJMHtKx58XPAO8lTZw3Lvs/X\nA08D/027oH5JRKos8X4rpR4Ffgb4O+Bx4JJS6qMs8T73MGg/91XfFknYryhEpAb8NvDjPc3WUHkq\n09KkM4nIG4GnlFJ/Oeg5y7bPGg94IfBflFIvAHbocUEs235rn/J95Be100BVRN5cfs6y7fMg5rmf\niyTsjwLXlh5fo3+3dIiITy7qv66U+h396ydF5Cr996uApw5qfXPgZcC3iMjD5C6214rIf2e59xly\nq+xrSqlP6ce/RS70y7zf3wh8RSn1tFIqAX4HeCnLvc9lBu3nvurbIgn7XwA3icj1IhKQBxo+fMBr\n2ndERMh9rvcrpX629KcPA2/VP78V+FDvaxcVpdS7lVLXKKXOkn+v/1sp9WaWeJ8BlFJPAF/V1dsA\ndwNfYLn3+++Au0Skoo/1u8njSMu8z2UG7eeHgTeJSCgi1wM3AX8+9bsopRbmH/B64CHgb8m7Sx74\nmuawjy8nvz37LPDX+t/rgQ3yKPoXgY8DRw96rXPa/1cDv6d/Xvp9Bp4PfFp/3/8TOLLs+w28l7yv\n1OeAXwPCZdxn4IPkcYSE/O7s+4ftJ/AerW0PAt80y3vbylOLxWJZMhbJFWOxWCyWMbDCbrFYLEuG\nFXaLxWJZMqywWywWy5Jhhd1isViWDCvsFovFsmRYYbdYLJYlwwq7xWKxLBn/H/rt6SemxodhAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1cec0b9780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# lets build a 3D dataset\n",
    "\n",
    "rnd.seed(4)\n",
    "m = 100\n",
    "w1, w2 = 0.1, 0.3\n",
    "noise = 0.1\n",
    "\n",
    "angles = rnd.rand(m) * 3 * np.pi / 2 - 0.5\n",
    "X_train = np.empty((m, 3))\n",
    "X_train[:, 0] = np.cos(angles) + np.sin(angles)/2 + noise * rnd.randn(m) / 2\n",
    "X_train[:, 1] = np.sin(angles) * 0.7 + noise * rnd.randn(m) / 2\n",
    "X_train[:, 2] = X_train[:, 0] * w1 + X_train[:, 1] * w2 + noise * rnd.randn(m)\n",
    "\n",
    "# normalize it\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "scaler = StandardScaler()\n",
    "X_train = scaler.fit_transform(X_train)\n",
    "\n",
    "plt.plot(X_train)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# build AE\n",
    "\n",
    "from tensorflow.contrib.layers import fully_connected\n",
    "\n",
    "n_inputs = 3 # 3D inputs\n",
    "n_hidden = 2 # 2D codings\n",
    "n_outputs = n_inputs\n",
    "\n",
    "learning_rate = 0.01\n",
    "\n",
    "X = tf.placeholder(\n",
    "    tf.float32, shape=[None, n_inputs])\n",
    "\n",
    "#\n",
    "# set activation_fn=None & use MSE for cost function\n",
    "# to perform simple PCA.\n",
    "#\n",
    "\n",
    "hidden = fully_connected(\n",
    "    X, \n",
    "    n_hidden, \n",
    "    activation_fn=None)\n",
    "\n",
    "outputs = fully_connected(\n",
    "    hidden, \n",
    "    n_outputs, \n",
    "    activation_fn=None)\n",
    "\n",
    "# MSE\n",
    "reconstruction_loss = tf.reduce_mean(\n",
    "    tf.square(outputs - X))\n",
    "\n",
    "optimizer = tf.train.AdamOptimizer(\n",
    "    learning_rate)\n",
    "\n",
    "training_op = optimizer.minimize(\n",
    "    reconstruction_loss)\n",
    "\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# run the AE\n",
    "\n",
    "n_iterations = 10000\n",
    "codings = hidden\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    for iteration in range(n_iterations):\n",
    "        training_op.run(feed_dict={X: X_train})\n",
    "    codings_val = codings.eval(feed_dict={X: X_train})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARMAAADbCAYAAABOZXXVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFPZJREFUeJzt3X2MZXV9x/H3d2d2hqSlVXctWHVcjabRSkPJhDrWthsh\nPBiVCvGpwVnEMBCXJhhb49bSkpiG1iZkhVXLILvupD6k6YKiQlgwTNVkfNhVfKQ+1MoKQcFtq9LK\n4u58+8fvHubM5T6cO/d3z++cez6vZDL34cy937kP3/P9PZzfMXdHRGRYm1IHICLjQclERKJQMhGR\nKJRMRCQKJRMRiULJRESiUDIRkSiUTEQkCiUTEYliMnUARW3dutW3bduWOgyRxjl8+PBP3f3p/bar\nTTLZtm0bhw4dSh2GSOOY2f1FtlMzR0SiUDIRkSiUTEQkirFLJisrcO214beIlKc2HbBFrKzAWWfB\n44/D1BR85jMwN5c6KpFmGKvKZGkJHnsMTpwICWV5OXVEIs0xNslkZQX27YNs4biJCdi+PWlIIo0y\nNslkeRmOHw+XzeDSS9XEESnT2CST7dtDP8nEBJx0EszPp45IpFnGpgN2bi50uC4vh8SiqkSkXGOT\nTCAkECURkTTGppkjImkpmYhIFEomaNasSAxj1WeyEZo1KxJH4yuT5eWQSDRrVmQ4jU8m+fkpU1Oa\nNSuyUY1v5mh+ikgcjU8moPkpIjE0vpkjInEomUSgoWWRBjdzVlbi9JNoaFkkSJJMzOzZwBJwCuDA\noru/t4znXlkJiyjt2xeWLBg2AXQaWlYykSZKVZkcB97u7l8xs5OBw2Z2l7t/e5RPmlURjz22tojS\nsAkgG1rOKhMNLUtTJUkm7v4Q8FDr8i/M7D7gmcBIk0lWRWSJxGz4BKChZZEgeZ+JmW0Dfh/4Yof7\nFoAFgJmZmaGfK19FTEyE1djm54dPABpaFgHzbDed4snNfh34N+Dv3P2WXtvOzs56jNODxup4FWkK\nMzvs7rP9tktWmZjZZuAA8OF+iSQmVREio5FknomZGXAzcJ+7X5cihqI0h0SkmFSVyR8CbwK+YWb3\ntm77K3e/PVE8HWkOiUhxqUZzPg9YiucehOaQiBSn6fQ9aHkCkeKSDw1XmeaQiBSnZNJB+/CxkohI\nf0ombWJ3umpeizSFkkmbmJ2uGg2SJlEHLOvnksTsdNVi1dIkja9MOlUPsTpddUSxNEnjk0mn6mHX\nrjjNEY0GSZM0PpmMunrQaFC51OGdTuOTiaqH8aEO77Qan0xguOohvycEJaWUdPhDWkomHfQrlbP7\nt2yBq64KH9zJybCC24kTG9srqjwfnjq801IyaZMvlScn4c1vXr8aW/7+TZtC8lhdDT8QEsqge0WV\n53GoyZqW5pm0yZfKx47BjTeGL3q2nkn+/hMnQkLJ5qRs3ryx+SmajxLP3Fy80TgZjCqTNlmpnK1g\nn680AI4cCRULhO1274ajR4frM1F5LuNAyaRNViovLcHevWt9IFu2rDVFJibgsss6L0a9kT2iyvNy\ndOuXUn9VHEomHWSjO/Pzax+yfFMEYGYm7gdP81FGq1u/1OIiXHlleF+np9VfNQwlkx7av+BqitRX\nt36pnTvDmR0h9JFpOHnjlEwKUlOk3jr1Sy0vr43CQWi+aiexcUomA8gqlewoYyWV+ui2M5ieDhXJ\npk2wZ4/ez2EomQxIc0Lqq73ZqmozLiWTAWnK9nhRx3c8SiYD2rIllMTu6ogVydMM2AGsrIRjcbKZ\nr7t3a68mklEyGUDWxFldDZXJ0aOpI5JUdNrYJ0t54vK9wCuBh939xaniGISmvQuoE76blJXJh4Dz\nEj7/wLLe/3e/Wx+gJtOBmZ0lq0zc/bNmti3V82+Uev9FFWpnlR7NMbMFYAFgZmYmcTQigeandGbu\nnu7JQ2XyqSJ9JrOzs37o0KGRx7RROvJUxpWZHXb32X7bVboyqQt1yAloh6JkEkG3Drkmf7CaRjuU\ntEPDHwW2A1vN7AHgb9395lTxDKO9Qy6/kFJTP1hN075DWVpq3s4k5WjOG1M9d2ztHXLdjt9pehlc\nd73ev/wOZXJy/Sp9TdmZFEomZjYFPAps7rLJre5+YbSoaqjfQkoqg+ut3/uX36EcOQI33dS8g0GL\nViabgUs73P424Azgk9EiGgOdhg6vvVZHG9dZkaPF8+vd7N/fvHkohZKJu/8v8M/528zsPYRE8nZ3\n3zeC2GqtvVLRRKd6G+T9a+o8lIH7TMzMgOuBncBOd39/9KjG1I4d4XenVe2l2gZNEE2cKT1QMjGz\nTcCNhCbPW7KKxMymgT3AWcDTgYeAG9z9hrjh1lN7e3t+PnVEshEbSRBN6nQvfKCfmU0AS8AlwMVt\nTZtJ4MfAOcBvAq8D/trMXhcv1Prqd2DY4iKce274LeMj24lcffX6s0KOq6KjOZuBjwCvBl7v7rfk\n72/1qVydu+leM7sNeBnwL5Fira1e7e3FRbj88nD54MHwe2Gh7AhlFJq2xGffyqTVhLmFsPbIhe2J\npMvfbAb+CPj60BGOgV5LFxw4sH7b9utSX9lOZCPnn66jIpXJEiGRfAh4qpld3Hb/be7+87bb9gC/\naP2t0L29fdFFaxVJdl3GQ9NGdXomk9bIzfmtq5e0fvJWgZPb/uY6YA54ubs/HiXKMZY1aQ4cCIlE\nTRypq6hLEJjZbsKIzsvd/ZFoD0z1lyAQaTcus56LLkEQbdlGM7seOJsRJBKROmra8o5RkomZPQf4\nc+D5wH+a2aOtnztiPL5IHakDdgPc/X7AYjyWyLhQB6yIRNOkafU6CZdICZpw0i5VJiIjNi6jOv2o\nMqmwJuzNmqApozqqTCokf4QpNGNv1gRNWctGyaQi2kvhHTuadZDYONvoqE7dli9QMqmI9lIYmrE3\na4pBR3Xq2M+iPpOKaJ/gND+vk6Q3WR37WVSZVES3UnjQvVmdyuImGfS9qWM/i5JJhQwzwamOZXFT\n9HtvOiWaOs6eTXlGv/OA9wITwAfd/e9TxTIOmraqV530em96JZq6zZ5N0mfSWk/2fYS1Ul4EvNHM\nXpQilnHRtIPK6qTXezNI30jV5x2lqkzOBL7v7j8AMLOPARcA304UT+31KovVl5JWr/emaN9IHZqx\nqZLJM4Ef5a4/APxB+0ZmtgAsAMzMzJQTWY11Kovr8CFsgm5NlqJ9I3Voxla6A9bdF4FFCCutJQ6n\nlurwIWy6In0jdRjdSZVMHgSenbv+rNZtElkdPoTSX76C2bJlfd9KdtvRo2mbsqmSyZeBF5jZcwlJ\n5A3AnyWKZazVcYhROsveu6zZOjEBZvCrX8HqKmzaBNPT6ZqySZKJux83syuBOwlDw3vd/VspYmmC\nug0xSnf5ZuvqargtWxN+dTVtUzZZn4m73w7cnur5ZY1Ge+oj32ztVJmkbMpWugNWRk+jPfXS3mwF\n9ZlIRbSP9iwtqUqpuvZma1XeJyWThmsvm/ftg+PH11cpagZJEUomDZcvm48cgZtuClXKsWNwzTXh\nlKVXXaVmkPSn9UyEuTnYtSusoTI1FTryVlfh7rvhrW+FX/6yXutqSBpKJvKErEo5++y1hHLixNr9\nk5Oho6/KB5tJOkomss7cXGjeTE+HYceMGZx/fmjyXH11GAFSQqmGxUU499zwOyX1mciTZBXK0hLs\n3Ruqk6kpOPVUjfxUzeIiXH55uHzwYPi9sJAmFnOvx/Fzs7OzfujQodRhNE63029MToaZl8ePh6rl\nVa+Cd7xDSWUUeo2mnXvuWhIBOOccuPPOuM9vZofdfbbfdqpMpKf2OQ35kZ/FxbUp3R//ONxxB9xz\njxJKTP0mFV500fpkctFF5ceYUTKRQvJ7x127wvWbb15LJrB+tEfNnzj6LSGRNWkOHAiJJFUTB5RM\npIBue8c9e8LQcTbiMzUVRns0PT+eIktILCykTSIZjeZIX93WKV1YgM99Dq64Ivzcc084PmSj53up\n+hqnKWSd4XU4f5IqE+mr196x0/IGgyzGtLgYSvTTT4cbblBF00ldlpBQMpG+uq3yNcyaptB5WBO0\nvGRdKZlIIe2rfPWqHnrtSfMduQcOdN5mYkLLS9aR+kyksGHPf5t15GYzaE8//cnbmMGll45PVdKk\nfiBVJlLYsItTtyejpzwFbrwxDDF/9athmDk7afs4qNLCU2UsI6FkIoUNuzh1p2Q0NxdGhcZxzZRY\npxkZ9rUpK6kpmchAhhlZ6JWMNvK4VU9AMU4zEiMRlHXuJCUTKVWsYc5UTYhBEliM04zESARlnTtJ\nyURqKcWZCjeSwIZNnjESQaekNoqqTslEShXrQ5ziTIUpElisk6jlk9qoqjolEylNzA9xijMVpjrV\nauwZsKNKikomUprYH+JBv2S9qqIiFdO4nGp1VEmx9GRiZq8FrgFeCJzp7lrxqCHK3rN3W9ipvSoa\npGIapkookrBiNAP7PcaokmKKyuSbwIXAjQmeWxIqc8/eniB27OheFY2q7C+azLrFvJFmYNHHGMXB\ng6UnE3e/D8DyqxVLY5R1BGx7goDuVdEoKqZBklm3mDeS1FJ0Emcq3WdiZgvAAsDMzEziaKRO2hPE\n/Hz46TZhLnbFNEgy6xbzRpJaqk5iGNGC0mZ2N3Bqh7ve5e6faG2zDPxF0T4TLSgtg0o5Q7ZTcwOq\n0WcyqKILSidbnV7JRMbdMF/qKh0qoNXpRRLbaP9QlY42HkTp65mY2WvM7AFgDvi0mUU+y4dIWu1r\nmAy6pkm+v+Wxx8LJzuogxWjOrcCtZT+vSBnaq4rdu8MpVfPXjx7t3XzZvj2sNnfiRDjR2b59ofO4\n6tWJVlqTxhnl6mdLS6GayEZxDhxYqzKOHYOdO/ufq3luLqw2l82eOH588FXtUlAykUZpXzoyZkJZ\nWQlVRDamMTERTow1NRUuT0yE1eSKLHs5Pw8nnRT+puwh3o1SB6w0yigndS0vhyoC1tayXViA005b\nW9k/3+TplCDyozh1Ow5IyUQapdOkrlhzO44cCSd0h/Vr2eZHdU47rXuHaqdRnF27ej9nlZKNkok0\nSvtsV4h7PMzEBFx2We8O0/37w7b7969/vkGqpioOH6vPRBpnbi7s8efm1n+Bjx2Da64ZvB8l/xgn\nTsDMTPcvdqeEkcmqpiL9JL0eJxUlE2m07Au8aVPoHD14cK3pM+hjFEkCvbYd5LzCgzxnWZJNpx+U\nptPLqKyshI7RL31p7bYrroAPfGCwxyjafxGrr6OsPhNNpxcpaG4OzjhjfTLZyGMU/ULHWoahaic0\nVzNHhNBhOj0dhnSnp9dGYpp0es9hqTIRIezh77nnyaeDGMWISdWGdGNRMhFpyZoNWTVy5Ej8CW5V\nHNKNRclEJCf/ZZ+cDKMlEG/EJOWyiqOmZCKSk/+yQ5iANjMTr0mSclnFUVMyEcnptHZszMphXM69\n04mSiUhOGV/2qg3pxqJkItJmXL/so6Z5JiIShZKJiEShZCISSVVmy6aKQ30mIhFUZTJayjhUmYhE\nUJX1RVLGoWQiEkFV1hdJGYeaOSIRVGUyWso4Sl8cycz+EXgV8DjwH8Cb3f1/+v2dFkcSSaPo4kgp\nmjl3AS92998Dvgv0WH9bROqi9GTi7gfdvXV2Eb4APKvsGEQkvtQdsJcCd3S708wWzOyQmR165JFH\nSgxLRAY1kg5YM7sbOLXDXe9y90+0tnkXcBz4cLfHcfdFYBFCn8kIQhWRSJKsTm9mlwCXA2e5+/8V\n/JtHgPv7bLYV+Olw0Y2E4hpMVeOC6sY2yrie4+5P77dRitGc84DrgD9x96htFzM7VKTXuWyKazBV\njQuqG1sV4krRZ7IHOBm4y8zuNbN/ShCDiERW+qQ1d39+2c8pIqOXejQntsXUAXShuAZT1bigurEl\nj6s2pwcVkWobt8pERBJRMhGRKGqbTMzstWb2LTNbNbOuQ2Jm9kMz+0Zr5KiUIwUHiO08M/uOmX3f\nzN5ZQlxPM7O7zOx7rd9P7bJdKa9Zv//fgutb93/dzM4YVSwDxrXdzH7Wen3uNbO/KSmuvWb2sJl9\ns8v9SV6vJ7h7LX+AFwK/AywDsz22+yGwtWqxAROEo6afB0wBXwNeNOK43gO8s3X5ncA/pHrNivz/\nwCsIh1sY8BLgiyW8d0Xi2g58qszPVOt5/xg4A/hml/tLf73yP7WtTNz9Pnf/Tuo4OikY25nA9939\nB+7+OPAx4IIRh3YBsL91eT/wpyN+vl6K/P8XAEsefAF4ipk9owJxJeHunwX+q8cmKV6vJ9Q2mQzA\ngbvN7LCZLaQOJueZwI9y1x9o3TZKp7j7Q63LPwZO6bJdGa9Zkf8/xWtU9Dlf2mpK3GFmvzvimIpK\n8Xo9odIrrRU5YLCAl7n7g2b2W4RZt//eyvBViC26XnHlr7i7m1m3eQEjec3GyFeAGXd/1MxeAXwc\neEHimJKrdDJx97MjPMaDrd8Pm9mthDJ26C9GhNgeBJ6du/6s1m1D6RWXmf3EzJ7h7g+1yt+HuzzG\nSF6zNkX+/5G8RsPG5e4/z12+3czeb2Zb3T31AYApXq8njHUzx8x+zcxOzi4D5wAde8IT+DLwAjN7\nrplNAW8Abhvxc94G7Ghd3gE8qYIq8TUr8v/fBsy3RileAvws10wblb5xmdmpZmaty2cSvkdHRxxX\nESlerzVl90hH7Nl+DaFNeAz4CXBn6/bfBm5vXX4eoTf+a8C3CE2QSsTma73v3yWMHow8NmAL8Bng\ne8DdwNNSvmad/n/gCuCK1mUD3te6/xv0GLUrOa4rW6/N1wirBb60pLg+CjwE/Kr1+XpLFV6v7EfT\n6UUkirFu5ohIeZRMRCQKJRMRiULJRESiUDIRkSiUTEQkCiUTEYlCyUREolAykSjMbMrMHjcz7/Jz\nS+oYZbQqfaCf1Mpmwrmj272NsKDPJ8sNR8qm6fQyMmb2HuAvgbe7+3Wp45HRUmUi0bWOqL0e2Ans\ndPf3Jw5JSqA+E4nKzDYRTgj1VuAt+URiZq8zs8+b2aNm9sNUMcpoqDKRaMxsgrC27OuBi939o22b\n/DfhXNOnEPpSZIwomUgUZrYZ+AjwauD17v6k0Rt3v6u1bcqFrGVElExkaGY2DfwrcDZwobt/OnFI\nkoCSicSwBLwS+BDwVDO7uO3+2zy3bqqMJyUTGUpr5Ob81tVLWj95q8DJJYYkiSiZyFA8TFT6jdRx\nSHpKJlKa1mjP5taPmdlJhHx0LG1kEoOSiZTpTcC+3PVfAvcD25JEI1FpOr2IRKEZsCIShZKJiESh\nZCIiUSiZiEgUSiYiEoWSiYhEoWQiIlH8P2qZkrL9cDGnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1cb95fbdd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure(figsize=(4,3))\n",
    "plt.plot(codings_val[:,0], codings_val[:, 1], \"b.\")\n",
    "plt.xlabel(\"$z_1$\", fontsize=18)\n",
    "plt.ylabel(\"$z_2$\", fontsize=18, rotation=0)\n",
    "#ave_fig(\"linear_autoencoder_pca_plot\")\n",
    "plt.show()\n",
    "\n",
    "# plot: 2D projection with max variance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stacked Autoencoders\n",
    "* AEs with multiple hidden layers - for more complex model learning\n",
    "![stacked-AE](pics/stacked-AE.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tf.reset_default_graph()\n",
    "\n",
    "n_inputs      = 28 * 28 # for MNIST\n",
    "n_hidden1     = 300\n",
    "n_hidden2     = 150 # codings\n",
    "n_hidden3     = n_hidden1\n",
    "n_outputs     = n_inputs\n",
    "learning_rate = 0.01\n",
    "l2_reg        = 0.0001\n",
    "\n",
    "X = tf.placeholder(tf.float32, \n",
    "                   shape=[None, n_inputs])\n",
    "\n",
    "with tf.contrib.framework.arg_scope(\n",
    "    [fully_connected],\n",
    "    activation_fn=tf.nn.elu,\n",
    "    weights_initializer=tf.contrib.layers.variance_scaling_initializer(),\n",
    "    weights_regularizer=tf.contrib.layers.l2_regularizer(l2_reg)):\n",
    "\n",
    "    hidden1 = fully_connected(X,       n_hidden1)\n",
    "    hidden2 = fully_connected(hidden1, n_hidden2) # codings\n",
    "    hidden3 = fully_connected(hidden2, n_hidden3)\n",
    "    outputs = fully_connected(hidden3, n_outputs, activation_fn=None)\n",
    "\n",
    "# MSE\n",
    "reconstruction_loss = tf.reduce_mean(\n",
    "    tf.square(outputs - X))\n",
    "\n",
    "reg_losses = tf.get_collection(\n",
    "    tf.GraphKeys.REGULARIZATION_LOSSES)\n",
    "\n",
    "loss = tf.add_n(\n",
    "    [reconstruction_loss] + reg_losses)\n",
    "\n",
    "optimizer = tf.train.AdamOptimizer(\n",
    "    learning_rate)\n",
    "\n",
    "training_op = optimizer.minimize(loss)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n",
      "0 Train MSE: 0.02705\n",
      "1 Train MSE: 0.0137857\n",
      "2 Train MSE: 0.0113694\n",
      "3 Train MSE: 0.0107478\n"
     ]
    }
   ],
   "source": [
    "# use MNIST dataset \n",
    "\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "mnist = input_data.read_data_sets(\"/tmp/data/\")\n",
    "\n",
    "# train the net. digit labels (y_batch) = unused.\n",
    "\n",
    "n_epochs = 4\n",
    "batch_size = 150\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    for epoch in range(n_epochs):\n",
    "        n_batches = mnist.train.num_examples // batch_size\n",
    "        for iteration in range(n_batches):\n",
    "            print(\"\\r{}%\".format(100 * iteration // n_batches), end=\"\")\n",
    "            sys.stdout.flush()\n",
    "            X_batch, y_batch = mnist.train.next_batch(batch_size)\n",
    "            sess.run(training_op, feed_dict={X: X_batch})\n",
    "        mse_train = reconstruction_loss.eval(feed_dict={X: X_batch})\n",
    "        print(\"\\r{}\".format(epoch), \"Train MSE:\", mse_train)\n",
    "        saver.save(sess, \"./my_model_all_layers.ckpt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# utility: plot grayscale 28x28 image\n",
    "\n",
    "def plot_image(image, shape=[28, 28]):\n",
    "    plt.imshow(image.reshape(shape), cmap=\"Greys\", interpolation=\"nearest\")\n",
    "    plt.axis(\"off\")\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# load model, eval on test set (measure reconstruction error, display original & reconstruction)\n",
    "\n",
    "def show_reconstructed_digits(X, outputs, model_path = None, n_test_digits = 2):\n",
    "    with tf.Session() as sess:\n",
    "        if model_path:\n",
    "            saver.restore(sess, model_path)\n",
    "        X_test = mnist.test.images[:n_test_digits]\n",
    "        outputs_val = outputs.eval(feed_dict={X: X_test})\n",
    "\n",
    "    fig = plt.figure(figsize=(8, 3 * n_test_digits))\n",
    "    for digit_index in range(n_test_digits):\n",
    "        plt.subplot(n_test_digits, 2, digit_index * 2 + 1)\n",
    "        plot_image(X_test[digit_index])\n",
    "        plt.subplot(n_test_digits, 2, digit_index * 2 + 2)\n",
    "        plot_image(outputs_val[digit_index])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAFpCAYAAAAvL3eVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG+RJREFUeJzt3V2MVeX1x/EHeRtmBgaGt0FksA46osTXKFDQIk28Ia1R\nYxPrRetLojHRRBM1mpioN3qn3Bg1GrDoBYnaNia+tRoCtNQqoGCJdhQUFBBhHIYZZnjTXvwv/gnr\nt/TZs885sDjfz+XKfvbec2YOy539cz3DfvzxxwQAQASnnegbAAAgF00LABAGTQsAEAZNCwAQBk0L\nABAGTQsAEAZNCwAQBk0LABAGTQsAEAZNCwAQxogaX4+ZUai0YSf6BurR7t27s7/LJ+OoOHVPp51W\n+f+G/+GHH2R92DD7Z6tqntzPtOw5vfXq2CLrVb2trS3rZnnSAgCEQdMCAIRB0wIAhFHrd1oATgHq\nXY33Tki9vyjy/kPJfafinVfda9l3b9W4/yLvhLz3Z7mK3H/uOznvMy3zWfOkBQAIg6YFAAiDpgUA\nCIOmBQAIg6YFAAiD9CCAwopMj8hNtRVJrxWRm8rzEm3Dhw/PPjbnOinpzyQ35ejVVe3YsWOF7itX\nkfRopa/PkxYAIAyaFgAgDJoWACAMmhYAIAyCGACqquxL/2qMV1KhgREj9D+HR48ezaqpcMKRI0dy\nbjGlpO9z1KhR8lh1r0VGW5W5J0/Z0Vy5eNICAIRB0wIAhEHTAgCEQdMCAIRBEANARRR56V/LcIaa\naKECDt7kjsbGxqzr9/X1ZR2Xkp5UoWpekEMFSdTP6YVL1H0VCY3kTr8ossdZ9rWHvBIAgBqjaQEA\nwqBpAQDCoGkBAMKgaQEAwiA9CKCwIiN71LG5e2x5x6r0mpd+U3V1zjFjxsj1uXtvHTp0yNSamprk\nORsaGkytp6fH1LyU3ujRo7NqHpVUVElDbz+u3KRhkb+JXDxpAQDCoGkBAMKgaQEAwqBpAQDCIIgB\noCLKjnFSe1SlpAMOhw8fNrWBgQG5XoUuVMCgtbVVrh87dqypqdDC5MmTTa25uVmeU/38KkjhBSHU\nyKYi4RSlSGAm93fNfloAgLpG0wIAhEHTAgCEQdMCAIRRV0GMf/3rX6a2dOlSeez06dNNTf0f83/4\nwx/kevVS13vRC5wKiuydVGQ/pwMHDpjaV199ZWrffPONXP/ll1+amvouT5o0Sa4fN26cqamJFm1t\nbVlrU9KhExUkKTJRQoU2vJ9JTepoaWnJuk+vroIg3v3n7scl1w55JQAANUbTAgCEQdMCAIRB0wIA\nhEHTAgCEMazMviZDUNOLHa+zs9PUurq6qnItlcSZN29eVa5VaWeeeaasP/jgg6bW3t5e5bv5WZWf\nE4OftXv37uzvsvo3Ro1hUrWUUuru7ja1Tz/91NQ+/PBDuV4dq1J93t5XKim3detWU1OjnbwxTvv2\n7TO13t5eU+vo6JDr1Xe0r6/P1KZMmSLXL1q0yNTOPfdcUxs1apRcr8Y7jRw50tS89KCqT5s2Leu7\nzJMWACAMmhYAIAyaFgAgDJoWACCMuhrj9Je//MXUPvroI3ns+eefb2r/+c9/TO3999+X6//617+a\n2ttvv21qv/jFL0xt27Zt8py51AvhlFKaNm2aqe3YsSP7vOrl7wMPPJC9HqcOb5+lXLmjnTxq7ynv\n737WrFmmpvbI8q7/+eefZ92TCiIcPHhQHvvtt9+amgo9TJw4Ua7fuXOnqa1Zs8bULrzwQrlefSaz\nZ882NS9IofbzKrJ3VpkAIE9aAIAwaFoAgDBoWgCAMGhaAIAw6iqIoV40qprnggsuMLUbb7xRHvvE\nE0+YmtrXRwUx1P9tX4T3f7GrIIa6/nfffSfXq/9jHvVJ7YdUJJyhXsR7QQo1Xebss882NW+ihTqv\n+o709/fL9eo7oqjpE2pKRUr6OzZ58mRTO+OMM+T61157zdTUfmQ9PT1yvfqs1B5jKvDiXUvxAhcE\nMQAAdYGmBQAIg6YFAAiDpgUACIOmBQAIo67Sg7XU0NBgarnpuyKJxiLUyKm9e/ea2ty5c+X6q6++\nuuL3hJhU+ssb45ObFPSSao2Njaam9qmaOnWqXK+upcYQeSOX1PdRJRrVGCe1R1ZKeu8wlR48cOCA\nXK8+K3X9SZMmyfUqlah+T9UY11UWT1oAgDBoWgCAMGhaAIAwaFoAgDAIYpyCvHE01157rampF61P\nPfWUXK/GvKA+ld0PSwUhVC0lHTBQ11ejpby6+rv3RhOp66uaun/vnsaPH29q6vu1efNmuV7t8dXW\n1mZqV155pVzf2tpqal4QJpf6TL2fv0xAgyctAEAYNC0AQBg0LQBAGDQtAEAYBDFOQcuXL5f13bt3\nm9rEiRNNbebMmZW+JZxiyuyH5PFe2ivqRb4XpFDTJxTvZ1IBi8HBwazreBMl1Dl37NhhaitXrpTr\nVUBj/vz5prZw4UK5Xn3v1T0dPXpUrleKhCvYTwsAUBdoWgCAMGhaAIAwaFoAgDAIYgT3xRdfmNq9\n996bvX7dunWmpv7PemCoym5P4QUsco9ToQkVBPAmcqh67kQQb4qM2gZl9erVpvbuu+/K9Wq7ljlz\n5phae3u7XK+mXxw7dkweqxSZfqEwEQMAUBdoWgCAMGhaAIAwaFoAgDBoWgCAMEgPBvf666+bmpei\nuuGGG0ztrLPOqvg94dRXjZE9XnpNpfcOHTpkaiol6F1f3f+oUaPkem8U0/Gam5tNTaX8Ukppz549\nprZlyxZT6+npkevVeKZf//rXpqb2zUqp2H5iivpdqfSg97snPQgAqAs0LQBAGDQtAEAYNC0AQBgE\nMQJRL0r//Oc/m5oa0ZJSSo8//ripeaNrgJ9SZD+k3Jfu3nG5ezo1NDTIurpXFbrwvgv9/f1Z51TX\n9/by2rBhg6l98sknpuaNVFu8eLGpzZo1y9S8n2lgYMDUioy2UqELFe4oO8JLXrviZwQAoEpoWgCA\nMGhaAIAwaFoAgDAIYgTywgsvmNqaNWtM7fe//71cz/QLVEo1JmKoKRcp6TBA7pSLlPSeVuqc3kQO\nFYAaMcL+06muv23bNnnO9957z9TURIxFixbJ9are1NRkaipwkZKeHuIFuJTc/bS8PbaKBHnMOYe8\nEgCAGqNpAQDCoGkBAMKgaQEAwqBpAQDCID14Evroo49k/a677jK18ePHm9pjjz1W8XsCfk6RvZNU\n+sxL76nzqvSeqqWkRymp0VDe9VWqbuzYsabW19dnauvWrZPnfOONN0xNfU5eenDSpEmmpn6mAwcO\nyPXKyJEjTc0bQ6U+6yKJQNKDAIC6QNMCAIRB0wIAhEHTAgCEQRDjBFNjVm688UZ5rHpRfNNNN5ka\n45pQbWVepHvrvZE/Za+vwgSq5p1ThS5UEGH9+vWm9sorr8hzfv/996Z23XXXmdqcOXPkejWGSo3B\nUvuGpaT/LVHn9EY7eQGN4xUJ5+TiSQsAEAZNCwAQBk0LABAGTQsAEAZBjBpSUwCWLFliap999plc\nP3v2bFN79NFHy98YUFCRIIU6Vn0XVC2l/H2evJf+uddqbm6W61WYoaury9RWrlxpav/+97/lORcu\nXGhq1157rampyRcppXTw4EFTUxMxvMCDCpKofcO89UVCM5XGkxYAIAyaFgAgDJoWACAMmhYAIAya\nFgAgDNKDNdTd3W1qq1atyl6/YsUKU2ttbS1zS8CQqFRZkZE9amSQSr+lpFNtuddJSe8TpRKJjY2N\ncr3ak2rjxo2mtnr1alObOnWqPOfvfvc7U5s7d66peWOY1N5dKtFXZI8xlahUn513rLqWlwhlPy0A\nQF2gaQEAwqBpAQDCoGkBAMIgiFEl+/fvN7V58+ZlrX3ppZdk/eKLLy51T0A1eUEI9dJdHeuNa1JB\nDBU6UOEO71h1LbW3XUopffrpp6b2+uuvm9qOHTtM7eabb5bnXLx4sam1tLSYWn9/v1yv9s7yfn4l\nd+8sLzChPlO1R1eZfbM8PGkBAMKgaQEAwqBpAQDCoGkBAMIgiFEly5YtM7WtW7dmrVV77aRUnZea\nwFAUmWig/m69SQmKmsqgzunt8aTWq+kbe/fulevfe+89U/vb3/5maipI0dnZKc/Z0NBgampKRW9v\nr1yvqJ/Tm2ihPqvccIWnVv8+8aQFAAiDpgUACIOmBQAIg6YFAAiDpgUACIP0YEldXV2y/sgjj9T2\nRoAaKpsUK5IoVMcWSQ+q86rxSGvXrpXr//nPf5qa2s/qjDPOMDVvPy11T4ODg6bmjWbyRl7lUudV\n91Rm3yvvnCmV+/vhSQsAEAZNCwAQBk0LABAGTQsAEAZBjJLWrFkj67njV2bPnm1qY8aMKXVPQLWp\nF+neS/vcl/lekEKNEvKOzb2+ClKoWko6YLFkyRJTO//8803NG+OUG0TxxjDlfqZeECJ3vXdc7u+/\nyB5ruXjSAgCEQdMCAIRB0wIAhEHTAgCEQRCjhn75y1+amtqXhyAGTnZlJyUU4U2FOJ7aIyslfa/q\nnBdeeKFcr6ZaDAwMmFpHR4epTZgwIfueVK3IvmPqWC8IoYIstQpS/NR5c/CkBQAIg6YFAAiDpgUA\nCIOmBQAIg6YFAAhjWC1TQCmlml4MdaHcxk4Ykp07d5rvcjX22EqpdklFL32oUnmqNmKEDWN7ycfc\n9J73s+euL6JIejD3d11kDFRbW1vWSXnSAgCEQdMCAIRB0wIAhEHTAgCEUesgBgAAQ8aTFgAgDJoW\nACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAg\nDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAya\nFgAgDJoWACAMmhYAIAyaFgAgDJoWACCMETW+3o81vh5OfcNO9A3Uo+3bt5903+Uffyx3S8OGVf5P\nqcg9nXZa/jNE7nmL/ExlPz+13ru+qs+YMSPrZnnSAgCEQdMCAIRB0wIAhFHrd1oATgFF3l+Upc5b\n9vpl3/Womnon5b0nUtf/4Ycfso7z5N5nkfNW63da5v0ZT1oAgDBoWgCAMGhaAIAwaFoAgDBoWgCA\nMEgPAiisyPQGlYpTyibVyib1vPsscuzxhg8fLusjRth/enNTkl69SHqxbPqybNKxzO+aJy0AQBg0\nLQBAGDQtAEAYNC0AQBgEMUp6+eWXZb2/v9/U1q9fb2rPPfdc9rUefvhhU1u8eLGpLVq0KPucQKUU\nCUIUGeOTG3o4evSorB85csTUBgcHTe3QoUNyfXd3t6l9//33prZ//35Ta2pqkuccN26cqY0dO9bU\nGhoa5Hp1XhXuGDVqlFw/cuRIU1NBjrLbpXiBC8Y4AQDqAk0LABAGTQsAEAZNCwAQxrAyL8SGoKYX\nq7Q777zT1J599tkTcCf/77zzzjO1tWvXymNbWlqqfTsnQnU2/MFP2r59u/kuV2vvpmPHjpna4cOH\nTa2np0euV0GKL7/80tQ2bNgg12/atMnUvvjiC3ns8VTgIqWU2traTK2zs9PUfvWrX8n1F1xwgalN\nmDDB1LwghgqnqNCFN9FD/a7V76mI9vb2rD8UnrQAAGHQtAAAYdC0AABh0LQAAGHQtAAAYTDGyVGN\npODFF19satdff72pdXV1yfUvvviiqW3ZssXUXnnlFbn+1ltv/blbBIasyB5JRUb+qKRgb2+vqe3a\ntUuu37x5c1ZNpQRT0uOZOjo6TG3mzJmmNmbMGHnOzz//3NS2bdtmaipRmFJKc+fONbXW1lZT80Zb\nKSo9WGSPsbJ7dOXiSQsAEAZNCwAQBk0LABAGTQsAEEbdBzG2b98u688//3zW+ssuu0zW33rrLVNr\nbGw0NTVmxRuHol7e/uMf/zC1vXv3yvXAycx76a9GDg0MDJia2iPLO6/6jl166aVy/YIFC0ztiiuu\nMDU1RskbDfXqq6+amvp+e0EGNV5J7QfmfaYqoOHtJ6ao0EXuHl0plQto8KQFAAiDpgUACIOmBQAI\ng6YFAAij7oMYXmhBvWhUoYu///3vcn1zc/OQ72n58uWy/sEHH2Stv+aaa4Z8bSBHkX341Ev3Ins3\nqRf8o0ePNrUpU6bI9fv27TO1q666ytRmz54t16s969R+WGpKh9q3K6WU1qxZY2pFwhHq81eTQ7yJ\nGF5AI+ecKenfVe7vuSyetAAAYdC0AABh0LQAAGHQtAAAYdC0AABh1H168JJLLpF1lSpUI5e8/XLK\n8EZIeUkeoNZUUswbzZN7rEoJpqSTuGq9lz4855xzTE2lD9vb2+X6cePGmZpKCm7cuNHUvHTx1q1b\nTW3OnDmmNmPGDLle/bujaj09PXK9SvUVSXQqajRWkb+JXDxpAQDCoGkBAMKgaQEAwqBpAQDCqPsg\nhqelpaUm11mxYoWpffzxx9nrr776alPr6OgodU/Az1FjhLzRTrkv/T0NDQ2mpkIbI0bof85UkCJ3\nP6iU9H5eKuCwbt06U/P26zvzzDNN7corrzQ1b7SUCpL09fWZmveZqPFQKmjmBSaKhC6UImPAjseT\nFgAgDJoWACAMmhYAIAyaFgAgDIIYNaT+j/nbb7/d1Lw9dKZNm2ZqS5cuNTXvhTJQKWUmGqSk93Py\n9njKnX7h7WGn9pTq7+83NRUuSEnvx/XZZ5+Z2vr1601tz5498pxqb7558+aZ2qRJk+T6/fv3m5q6\nf29iT2Njo6mpz8lbr6bz5O7RVRZPWgCAMGhaAIAwaFoAgDBoWgCAMGhaAIAwSA/WkBrz4iUFlTvu\nuMPU1F5BwMlOjXHyEonqWJVe8/Z+Uqm63HOmlNKOHTtM7Z133jG1TZs2mdrUqVPlOc8++2xTU/t5\neYk8da8q6eetV2OUBgYGTM1LIpdNj7KfFgCgLtC0AABh0LQAAGHQtAAAYRDEqJJbbrnF1FauXJm1\n9p577pH1+++/v9Q9AZWiXuR7L9dz905S+1allD8eyAs1eXtKHW/Xrl2y/tZbb5maCmKMHz/e1ObO\nnSvPedFFF5najBkzTM37TFSQRO075n12KnShgizeerX31uDgoDy20njSAgCEQdMCAIRB0wIAhEHT\nAgCEQRCjpL6+Pll/8803TU29qFT/x/xDDz0kz6lefgInQtmJCGpKhXfO3L231H5QKem9o9T6rq4u\nuf799983NbWf1aWXXmpqCxYskOecP3++qbW2tpqatx+XCmKo0IYKXKSk/y0aN26cPFZRn58K3Kj7\nLIsnLQBAGDQtAEAYNC0AQBg0LQBAGAQxSrrhhhtk3XuBery7777b1NQLWSAq9YK+yESN3Jf53uQN\nFRro7e01tQ0bNsj1W7ZsMbXTTz/d1M477zxTO/fcc+U51fSKAwcOmJq3XcrYsWOz1h88eFCuV+EU\nxZuIoUIfZcM5uXjSAgCEQdMCAIRB0wIAhEHTAgCEQdMCAIRBerCA9evXm9qqVauy11933XWmdu+9\n95a5JeCkp1JlquYl1cqmB9Uoo08++cTUPvjgA7lepfI6OztNTaUHR44cKc+p0sU9PT2m5o1W6u/v\nNzU1msnbS0ylF9Wx3h5ZagzX6NGj5bGVxpMWACAMmhYAIAyaFgAgDJoWACAMghgO9fL2wQcfNDVv\nzIqi9tthjyxE5IUeFBWkyA1npKS/IyoI4O2ntXnzZlN75513so5LKaUpU6aY2uLFi03tnHPOMTUv\nnKDGK6nQRUtLi1yf+5mqcU8p6c9P/VvmfabqWBXu8H6nRf5+jseTFgAgDJoWACAMmhYAIAyaFgAg\nDIIYjmeeecbU3n333ez1t9xyi6kx/QKnirJ7J6kggDc9QgUxVBBATZRISU+/UHtneaGo3/zmN6a2\ncOFCU2trazM1te9USjqgMXz4cFPzAguHDh0yNRW6aGpqkuvVZ9Xd3W1qXtBszJgxpuZNNFHK/P3w\npAUACIOmBQAIg6YFAAiDpgUACIOmBQAIg/Sg46GHHiq1/sknnzQ1RjYB/0cl5bz0oNrnSY1B+vrr\nr+X6rq4uU9u1a5epTZs2Ta6//PLLTW3ChAlZ96RSkinp8UgqUef9m6FSiepaai+wlFLavn27qXnp\nS6WjoyP7WIUxTgCAukDTAgCEQdMCAIRB0wIAhEEQo0r6+vpMTe2BU1buOJiU9ItaNQ7Go/YYW7p0\nafZ6Rd2rF4LxXtTj5OC9XM/dO0sFLlLSoQUVeujt7ZXr9+zZY2r9/f3yWGXnzp2m9t///jfrnOo+\nU9I/kwpdeP9mqCCG2s9KhVBSSmnTpk2m1traamoXXXSRXD84OGhqarSTuqeUGOMEAKgTNC0AQBg0\nLQBAGDQtAEAYBDGqZPr06TW5zh133GFqp59+ujx29+7dpvb0009X/J7K8j672267rcZ3Ao8KXRR5\nuV7k2NzpCV4ASQWIVGhDfT9SSuntt982tY0bN5ravn37TM0LD6nQgwpXeHtUqc9Ehb+++eYbuV6F\nsq644gpT8/bTUr+/IkEzghgAgLpA0wIAhEHTAgCEQdMCAIRB0wIAhEF60HHTTTeZ2rJly07Anfy0\nZ555puLn9MbpeOms4/3xj3+U9fnz52etX7BgQdZxOHHKpv9Uek2NNkpJJ+jU9b39sC677LKse9qy\nZYtcrxJ4KmmoRht56Tv186ufqb29Xa5X49vUHmHed3bKlClZtba2NrlepSJVzfs78VKROXjSAgCE\nQdMCAIRB0wIAhEHTAgCEMSx3REqF1PRilfanP/3J1LwXrbk+/vhjUys7Wum+++6T9VmzZmWt/+1v\nfyvr6kXtSWDo82AwZF999ZX5LhcJZ6iRP15oQB2rXuR7e1epIIUKXezdu1eu7+7uNjX1vVf7dnnU\n+vHjx5va1KlT5Xo1mkrtjTd58mS5fuLEiabW2dmZvb65udnU1N5ZXqhL/f5mzpyZ9QfEkxYAIAya\nFgAgDJoWACAMmhYAIAyCGIiOIMYJUDaIoXjr1Qt+Fc5QUya8upq+oa6Tkg49qOkXKsih9tjy7qml\npSX7ntRnlVtLSQck1LW8cIxa7+0dlqu9vZ0gBgDg1ELTAgCEQdMCAIRB0wIAhEHTAgCEwX5aACqi\nbBLZW6+Seiq95o0MUknDIiOHmpqaTE0l/YqMOVNjjNR9ep+JSvWplKO3/siRI1n3VIRKKpY9p8KT\nFgAgDJoWACAMmhYAIAyaFgAgDIIYAAorO7JJBQRUEME7VgUJVM1br8YoeaEBFdBQQYgin4kaeaTW\ne+EQFU4pG4RQ670gR+6x3mdSJrTDkxYAIAyaFgAgDJoWACAMmhYAIAyCGAAKK/LSvcw5vfOqY731\nXsAjd70KM+TuXaX27UpJh0ZU6OLw4cNyvfqZ1H16e4wp6v69zy73MylyrVw8aQEAwqBpAQDCoGkB\nAMKgaQEAwqBpAQDCID0IoLAiI3/Kyj1vkZFBKhXnjUwqcq3cc+au9352lQosO0apbCKzWr9/c+2a\nXAUAgAqgaQEAwqBpAQDCoGkBAMIYVquXZwAAlMWTFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYA\nIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACAM\nmhYAIAyaFgAgDJoWACAMmhYAIAyaFgAgDJoWACCM/wGjU+vJN6GzgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1ca51e07b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_reconstructed_digits(X, outputs, \"./my_model_all_layers.ckpt\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tying Weights\n",
    "* Used when AE is symmetrical. Tying decoder layer weights to encoder layers' weights cuts number of weights by 50% (speedup & less memory).\n",
    "* Tied weights in TF is cumbersome. Easier to define layers manually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tf.reset_default_graph()\n",
    "\n",
    "activation = tf.nn.elu\n",
    "\n",
    "regularizer = tf.contrib.layers.l2_regularizer(l2_reg)\n",
    "\n",
    "initializer = tf.contrib.layers.variance_scaling_initializer()\n",
    "\n",
    "X = tf.placeholder(tf.float32, shape=[None, n_inputs])\n",
    "\n",
    "weights1_init = initializer([n_inputs, n_hidden1])\n",
    "weights2_init = initializer([n_hidden1, n_hidden2])\n",
    "\n",
    "weights1 = tf.Variable(weights1_init, dtype=tf.float32, name=\"weights1\")\n",
    "weights2 = tf.Variable(weights2_init, dtype=tf.float32, name=\"weights2\")\n",
    "# weights 3,4 not vars!\n",
    "weights3 = tf.transpose(weights2, name=\"weights3\") # tied weights\n",
    "weights4 = tf.transpose(weights1, name=\"weights4\") # tied weights\n",
    "\n",
    "biases1 = tf.Variable(tf.zeros(n_hidden1),name=\"biases1\")\n",
    "biases2 = tf.Variable(tf.zeros(n_hidden2),name=\"biases2\")\n",
    "biases3 = tf.Variable(tf.zeros(n_hidden3),name=\"biases3\")\n",
    "biases4 = tf.Variable(tf.zeros(n_outputs),name=\"biases4\")\n",
    "\n",
    "hidden1 = activation(tf.matmul(X, weights1) + biases1)\n",
    "hidden2 = activation(tf.matmul(hidden1, weights2) + biases2)\n",
    "hidden3 = activation(tf.matmul(hidden2, weights3) + biases3)\n",
    "outputs = tf.matmul(hidden3, weights4) + biases4\n",
    "\n",
    "reconstruction_loss = tf.reduce_mean(\n",
    "    tf.square(outputs - X))\n",
    "\n",
    "reg_loss = regularizer(weights1) + regularizer(weights2)\n",
    "\n",
    "loss = reconstruction_loss + reg_loss\n",
    "\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate)\n",
    "\n",
    "training_op = optimizer.minimize(loss)\n",
    "\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training one Autoencoder at a time\n",
    "* Often faster to train each shallow AE individually, then stack them.\n",
    "* Simplest approach = use separate TF graph for each phase\n",
    "\n",
    "![one-ae-atatime](pics/training-one-AE-atatime.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_autoencoder(\n",
    "    X_train, \n",
    "    n_neurons, \n",
    "    n_epochs, \n",
    "    batch_size, \n",
    "    learning_rate = 0.01, \n",
    "    l2_reg = 0.0005, \n",
    "    activation_fn=tf.nn.elu):\n",
    "    \n",
    "    graph = tf.Graph()\n",
    "    with graph.as_default():\n",
    "        n_inputs = X_train.shape[1]\n",
    "\n",
    "        X = tf.placeholder(tf.float32, shape=[None, n_inputs])\n",
    "        \n",
    "        with tf.contrib.framework.arg_scope(\n",
    "            [fully_connected],\n",
    "            activation_fn=activation_fn,\n",
    "            weights_initializer=tf.contrib.layers.variance_scaling_initializer(),\n",
    "            weights_regularizer=tf.contrib.layers.l2_regularizer(\n",
    "                l2_reg)):\n",
    "            hidden = fully_connected(\n",
    "                X, n_neurons, scope=\"hidden\")\n",
    "            outputs = fully_connected(\n",
    "                hidden, n_inputs, activation_fn=None, scope=\"outputs\")\n",
    "\n",
    "        mse = tf.reduce_mean(tf.square(outputs - X))\n",
    "\n",
    "        reg_losses = tf.get_collection(\n",
    "            tf.GraphKeys.REGULARIZATION_LOSSES)\n",
    "        \n",
    "        loss = tf.add_n([mse] + reg_losses)\n",
    "\n",
    "        optimizer = tf.train.AdamOptimizer(learning_rate)\n",
    "        \n",
    "        training_op = optimizer.minimize(loss)\n",
    "\n",
    "        init = tf.global_variables_initializer()\n",
    "\n",
    "    with tf.Session(graph=graph) as sess:\n",
    "        init.run()\n",
    "        \n",
    "        for epoch in range(n_epochs):\n",
    "            n_batches = len(X_train) // batch_size\n",
    "            \n",
    "            for iteration in range(n_batches):\n",
    "                print(\"\\r{}%\".format(100 * iteration // n_batches), end=\"\")\n",
    "                sys.stdout.flush()\n",
    "                \n",
    "                indices = rnd.permutation(\n",
    "                    len(X_train))[:batch_size]\n",
    "                \n",
    "                X_batch = X_train[indices]\n",
    "                \n",
    "                sess.run(\n",
    "                    training_op, feed_dict={X: X_batch})\n",
    "                \n",
    "            mse_train = mse.eval(\n",
    "                feed_dict={X: X_batch})\n",
    "            \n",
    "            print(\"\\r{}\".format(epoch), \"Train MSE:\", mse_train)\n",
    "            \n",
    "        params = dict(\n",
    "            [(var.name, var.eval()) for var in tf.get_collection(\n",
    "                tf.GraphKeys.TRAINABLE_VARIABLES)])\n",
    "        \n",
    "        hidden_val = hidden.eval(\n",
    "            feed_dict={X: X_train})\n",
    "        \n",
    "        return hidden_val, params[\"hidden/weights:0\"], params[\"hidden/biases:0\"], params[\"outputs/weights:0\"], params[\"outputs/biases:0\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 Train MSE: 0.0193591\n",
      "1 Train MSE: 0.0190697\n",
      "2 Train MSE: 0.0188801\n",
      "3 Train MSE: 0.0192353\n",
      "0 Train MSE: 0.00428287\n",
      "1 Train MSE: 0.00438113\n",
      "2 Train MSE: 0.00464872\n",
      "3 Train MSE: 0.00457076\n"
     ]
    }
   ],
   "source": [
    "# train two AEs\n",
    "\n",
    "hidden_output, W1, b1, W4, b4 = train_autoencoder(\n",
    "    mnist.train.images, \n",
    "    n_neurons=300, \n",
    "    n_epochs=4, \n",
    "    batch_size=150)\n",
    "\n",
    "_, W2, b2, W3, b3 = train_autoencoder(\n",
    "    hidden_output, \n",
    "    n_neurons=150, \n",
    "    n_epochs=4, \n",
    "    batch_size=150)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create stacked AE by reusing weights &and biases from above\n",
    "\n",
    "tf.reset_default_graph()\n",
    "\n",
    "n_inputs = 28*28\n",
    "\n",
    "X = tf.placeholder(tf.float32, shape=[None, n_inputs])\n",
    "hidden1 = tf.nn.elu(tf.matmul(X, W1) + b1)\n",
    "hidden2 = tf.nn.elu(tf.matmul(hidden1, W2) + b2)\n",
    "hidden3 = tf.nn.elu(tf.matmul(hidden2, W3) + b3)\n",
    "outputs = tf.matmul(hidden3, W4) + b4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing Reconstructions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAC3CAYAAAChbAF+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADOBJREFUeJzt3c+LltX7B/BjOo4zOqNOU+SPzCQyEYaySImgVW1aBbWw\nFv0FbdpEu2pTy/oDoqD2FQUREQSWFi0yo0WKVoplmo6OOuPP+my+fD/guc7H++lxRq/x9Vq+uc/z\n3PPoeHHzvD1nwT///FMAIINbrvcNAEBXhhYAaRhaAKRhaAGQhqEFQBqGFgBpGFoApGFoAZCGoQVA\nGoYWAGksmuP3s2cU19qC630DN6OpqSm/y1xTo6OjnX6XPWkBkIahBUAahhYAacz1d1oAN6QFC+Kv\nVLoe3xRd13rNrlrvfcst9fPGzXLMlCctANIwtABIw9ACIA1DC4A0DC0A0tAeBOaFfttzvayPWoEL\nFy7sdF0r76UR+Pfff1fZ5cuXw2u7ro/uqd/242zwpAVAGoYWAGkYWgCkYWgBkIYiBnBD62cbpVbe\nS+kiKkhcunSpyhYtqv85jbLWa0aickdrfZRFhYuWLNtAedICIA1DC4A0DC0A0jC0AEhDEQOY1/rd\n6SG6dvHixVUWFRm6Fi5KKeXixYtV1ipiRAWPXnbkiO4rev9WkaPfHT364UkLgDQMLQDSMLQASMPQ\nAiANQwuANLQHgRta1FSLzo5qNeWia6NWXKupd+HChU7rly5dWmWt9lzUtBsYGAivjXRtL7bef3p6\nusqiz6nVfux3a6x+eNICIA1DC4A0DC0A0jC0AEhDEQO4oZ07d65T1ipiRKWJ6DysqIhQSilDQ0NX\nu8VSSikzMzNVdvbs2c73FGmVQ4aHh6usazmjlLhgsWTJkiqLPqdS4s+6l7O7+uFJC4A0DC0A0jC0\nAEjD0AIgjZuqiPHNN99U2VtvvRVeu2bNmiqLvpB9/vnnw/VjY2OdMpjvuu6UEJUrSinlzJkzVXbi\nxIkqaxUBotc9depUlbV2f4jyFStWVNnRo0erbGpqKnzN06dPV9nk5GSVtcolIyMjVRb9m3XnnXeG\n69euXVtlo6OjVdb6TKKztyKt++/lPLPqnv71SgCYY4YWAGkYWgCkYWgBkIahBUAaC+bqDJT/M6dv\ndqWNGzdW2b59+2blvZYvX15l27Ztm5X3utbWr18f5i+//HKVrVu3bpbv5qr+fQ2Jf21qaqrz73K0\nFVB0ntPx48fD9VH77+TJk1V25MiRcP3evXurrLW9UldRe+7gwYNV9ttvv4XrFy2qi9vRvxmtraWi\nM76idvLmzZvD9U888USV3XfffVUWtRRbemkERteOjo52egFPWgCkYWgBkIahBUAahhYAadxU2zh9\n+OGHVbZ79+7w2ugLzJ9++qnKvv3223D9Rx99VGWfffZZld19991V9ssvv4Sv2VX0JW8ppaxatarK\nDh061Pl1o4LGSy+91Hk9N6eoTBBtzdRL6SDaxqlVxDh27FiVRaWPZcuWhesHBgaqLCoSDA4OVtnK\nlSvD1xwfH+/0/tHPWUope/bsqbIffvihyqJzt0qJizBRFv3spcRnb/WzNVMvPGkBkIahBUAahhYA\naRhaAKRxUxUxNm3a1ClrmZiYqLLt27eH177xxhtV9uuvv1ZZVMQ4cOBA53uKLF68OMyjIkb0/tEX\n16XE/2MeribadWfp0qVV1ioQReujnRpuu+22cH10ztbq1aurrFU6iM6eisohvZQ77rjjjiqLihw7\nd+4M1//4449VFu0c0jqjbGZmpsrOnz/fKSsl/qyis7daOy45TwuAm4KhBUAahhYAaRhaAKRhaAGQ\nxk3VHpxL0TYnXdt3vTQaexFtOfXXX39V2datW8P10Rk8cDVRKzBqn7Xae1HTbOHChVXWag9GW7JF\nrbjR0dFwfdQAjM7Tmpqa6rS29V5RI/H7778P10etvOjfl7vuuqvz+w8NDVVZ688k+vmjP5OoUdgv\nT1oApGFoAZCGoQVAGoYWAGkoYsxDZ8+eDfOnnnqqyqItbt58881wffRFLVxN9AV9pLX9WFSaiAoO\ny5cvD9dHZYDo7K5WESO6NipCRGdntc6zij6TqBT1xx9/hOujotfGjRurbMuWLeH6W2+9tcqiwky0\ntVQp8f07TwsArmBoAZCGoQVAGoYWAGkoYsxD7777bpgfOXKkyqIvZFv/ix7+jegL+qjI0CpiRAWB\nqIjQKj10LWK0igTR7g/Ra0b32XrNqAD1+++/V9n09HS4PjqPKzr3Kzovr5RSxsfHqywqsrR2xIg+\nv4jztAC4qRlaAKRhaAGQhqEFQBqKGMnt37+/yl588cXO63ft2lVl0Ze8cC11LWeUUsrSpUurLCoy\ntIoc0bWXLl2qsla5YHJystN7ReWQlqNHj1bZF198UWWtHTGi0km0I0Zrl48oj14z+uz+V36l2dgl\nw5MWAGkYWgCkYWgBkIahBUAahhYAaWgPJvfxxx9XWbTtTCmlPPPMM1W2YcOGa35PcDVRU7CXLX+i\n9l4vWyZFTcHWlkldm4LRdRcuXAhf8+DBg52yQ4cOhevvv//+Klu9enWVjY2NhetHRkaqLNqGqnX/\n0Z9VtLXVbPCkBUAahhYAaRhaAKRhaAGQhiJGIlHB4oMPPqiywcHBcP3rr79eZQsXLuz/xqBHvWzj\n1PVL/2hrplLiMsHp06evdov/LypYRL9jUeHj8OHD4Wt+8sknVfbdd99V2fnz58P169evr7LoPK3b\nb789XN/a8qqr6M+vlz/TfnjSAiANQwuANAwtANIwtABIQxEjkbfffrvKduzYUWXPPvtsuN7uF9zI\nWjsqRAWH6Av+VmlhZmam03sNDQ2F65ctW9bp/U+dOlVlO3fuDF/z66+/rrLojK2JiYlw/UMPPVRl\nUTkj2uWilLg0EX1+Xc/NKmV2ShcRT1oApGFoAZCGoQVAGoYWAGkYWgCkoT14A9q9e3eYv/DCC1W2\nYsWKKnvttdeu+T3BtdTvNk7Rlma9bEkWvf/AwEDna6Otofbv319lrfZgtL3TPffcU2VPP/10uH7L\nli1V1mo/RqKm4Llz56qstTVW1L5sNRWvNU9aAKRhaAGQhqEFQBqGFgBpKGJcZ9EWM9u3bw+vvXz5\ncpU999xzVWa7Jm500d/llmgroa7nOZVSyvDwcJVFpYElS5aE66OCx/Hjx6ssOg8r2matlFJGRkaq\n7LHHHquyxx9/PFy/atWqKos+017OGOulyBIVYRQxAOAKhhYAaRhaAKRhaAGQhiLGHIq+UH7yySer\n7Oeffw7Xb9q0qcpeffXV/m8M5oGoXFBKXBCIfhdb53mdOXOmyvbu3VtlX375ZZVNTk6Gr3nvvfdW\nWVTEGB8fD9dHu4REP1PrM4lKG1HW2qVkcHCwynrZ5aQfnrQASMPQAiANQwuANAwtANIwtABIQ3tw\nDp04caLKosZRy3vvvVdlY2Nj/dwSXBfRlkGtrZ2ia6PtiaJGW0vUdGs1/f78888q+/TTT6vsq6++\n6nxPDz74YJVFjcLW1lRRKzBqD0ZZKXFTMvqcW+8/G63ArjxpAZCGoQVAGoYWAGkYWgCkoYgxS06d\nOlVl27Zt67T2/fffD/MHHnigr3uCG0X0RX5rG6WoINC6NhKVEc6fP19lx44dC9fv2bOnynbt2lVl\nUZFk69at4Ws+/PDDVbZy5coqa53xNT09XWW9fKbRvUbXtooc0dZYc1XO8KQFQBqGFgBpGFoApGFo\nAZCGIsYseeedd6rswIEDndY++uijYd763+kwH7T+fkdlgK7nSZUS754RFaWickMpcUFjzZo1VRaV\nJjZv3hy+5sTERJUNDw9XWavcEP2s586dq7KoxNISFTF6KbwoYgDAFQwtANIwtABIw9ACIA1DC4A0\ntAf7tG/fvjB/5ZVX5vZGILlW+yxqFUbtuYsXL4brz549W2VRU/Dw4cPh+pMnT1bZ2rVrO2XRuVml\nxFs2RVsrtc4YixqRUXuxlzPKos/5ep6b1eJJC4A0DC0A0jC0AEjD0AIgDUWMPu3YsSPMp6amOq3f\ntGlTlQ0NDfV1TzCfdN2y6cKFC+H6qKDRy/poe6Xx8fEqGxsb63RdKfH2SFE5pLUNU9ctl3rZhimL\n+fcTATBvGVoApGFoAZCGoQVAGooYc+iRRx6pss8//7zKFDHgv6IiRrTTw8DAQLh+ZGSkyhYvXlxl\nGzZsCNevW7euyqJyRvT+rTPCerm2q+hz6mWXkSzn9XnSAiANQwuANAwtANIwtABIw9ACII0Fc3xe\nyo13OAvZ5ag8zTNTU1Nz9rvctT3YEm1lFDXlWq8ZXdt1e6RWIy86D6sXixbNv+L36Ohop99lT1oA\npGFoAZCGoQVAGoYWAGnMdREDAP41T1oApGFoAZCGoQVAGoYWAGkYWgCkYWgBkIahBUAahhYAaRha\nAKRhaAGQhqEFQBqGFgBpGFoApGFoAZCGoQVAGoYWAGkYWgCkYWgBkIahBUAahhYAaRhaAKRhaAGQ\nhqEFQBr/ARqGrzwwyV+fAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c53b89048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAACFCAYAAAAtgP8MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAC5VJREFUeJzt3clrlEsXx/GKY5znOMQYnEFw4bQQN+rCpYLiyo2ooLgQ\nFBQU/BskG1ERnHAhuBQUHHChiIiIiopjElETxzjPw7u51P3VMV1vJ3YSb5/vZ1UP1fbTye0cnnPu\nqaqKX79+BQDwpltXfwAA6AoEPwAuEfwAuETwA+ASwQ+ASwQ/AC4R/AC4RPAD4BLBD4BLPTr5fiwn\n+XtUdPUHKCeNjY18t/8StbW1RX23efID4BLBD4BLBD8ALnV2zQ9AB6io+LfMZXdq0jkd29e2ZYen\nbt3S56afP38W/W//Fjz5AXCJ4AfAJdJeoMz8+PEjudYU1aa93bt3j2Ob9tpUVt/3y5cvBV+r79na\ndbFzHY0nPwAuEfwAuETwA+CS25rf4cOHk+sPHz7E8eXLl5O5PXv2FHyf7du3J9cLFy6M4/nz5//B\nJwRSWnOz9Tit19ma39evXwvO5ei/s//2+/fvyVxlZWUcDxw4MJnr2bNnHOc+t61HdjSe/AC4RPAD\n4FJFJ5/b26U7X6xfvz6Od+/e3SH3mDZtWhyfO3cumRs0aFCH3LOd2NWlhEq1q4v+PX779i2Z01RT\nyzQhhPD48eM4fvHiRTLX0tISx83Nzcnc69ev49imxEOHDk2uBwwYEMfV1dXJXE1NTavjENI02MYb\nmz6rXItODru6AEAGwQ+ASwQ/AC6VdauL1vhCKL7ON2PGjOR62bJlcXz37t1k7sCBA8n1zZs34/jo\n0aPJ3OrVq4u6P/ywNTCtu9m63tu3b+O4oaEhmbt9+3YcP3nyJJnTmp+tB+r9taYXQggfP35Mrvv1\n69fqOIQQxo0bF8f9+/dP5rTmZ99Ta365Nhy7DE5bZuwOM8XiyQ+ASwQ/AC6VXdr78OHDON67d2/B\n182ZMye5PnHiRBz37ds3mevVq1cc20fze/fuJdfnz5+PY5tiACGkqaZd8fDp06c41jQ3hDSdbWxs\nTObevHkTx/b7q9/ZqVOnJnODBw+O42fPniVzjx49Sq7fvXvX6v3sPSxdKdKnT59kTtt5cm0wdq4U\nu8Hw5AfAJYIfAJcIfgBcKruan9bZbJ1A63ynTp1K5uz/ni9k//79yfWlS5cKvnbJkiVFvSd8ybVp\naC3LzvXo8e+f67Bhw5I5bT3RtpMQ0uVmdsnaq1ev4vjMmTPJ3I0bN5LrpqamgvfI1QP171B3fwkh\nXV5n/161Hmhba3r37h3H7a3/8eQHwCWCHwCXyi7tnTlzZhzbVhNtWbH/y71Ytn3GbvgI/D+avtrd\nSnTjT1uKGTFiRBxXVVUlc5rOjhkzJpnTFRa2fUb/RuyhRNaoUaPiWH+GENK/J7sbjb6vTVH1Z7Sf\nLVcCaO+qjuQ9/vgdAOA/iOAHwCWCHwCXyq7mp0q1c/KhQ4fi+OrVq9nXLlq0KI4nTpxYkvvjvy1X\nn7LL27TOZdtCtBXE1qzHjh0bx7ZWqHXply9fJnNXrlyJY90ZJoTfa+bDhw+P4ylTpiRz2ory+fPn\nZE5/Jlvz09+N/Xn1c2stNITSHHzEkx8Alwh+AFwq67S3vTQVCCGEtWvXxrFtBxg9enRyXVdXF8f2\nUR0+2ZULmqbZ74imjDZ9zJ2Nq/d4//59Mvf06dM4PnbsWDKnZ1Q/ePAgmbOHFM2ePTuOp0+fnszl\nNizNtazo527L30spzvjlyQ+ASwQ/AC4R/AC4RM2vFRcuXEiuc8t+1q1bl1zbFgDA1qe0vcUuBbPX\nufdRujTMfl8vXrwYx3YXouvXr8exrfHZVq0FCxbEsa116+e2u7roztK2rqc7V9ufT2uFdo6aHwC0\nE8EPgEukvf9YtWpVHB85cqTg6zZu3Jhcb9mypcM+E8qDbXXJzWk6Z1dqaJuI/Xeaatpze/VQLdvO\novew5/ba++u1vb+mr7mziO2KFr22KbGm0nYXmVLgyQ+ASwQ/AC4R/AC45LbmZ5cAHT9+PI7tsqKR\nI0fG8bZt25I53R0aaE2u5md3MtH6mN0BReuB9vurdb76+vpkTg8f14N/QkgPLddxCL+3s+jPoQcP\nhZAuocsdTG6XvumcPfhca4C532F78eQHwCWCHwCX3Ka9y5cvT66fPXtW8LUbNmyIY3vuKdBWmr7m\nDjDK7YDy4cOHZE7T3oaGhmROU+shQ4Ykc7lDkWpra5NrTcPtYUPa6mLLRpoG53Z8sW0wud9FKfDk\nB8Algh8Alwh+AFxyVfPTXWvPnj1b8HVLly5Nrjdt2tRRHwkO5XYkyR3Mo/UxW/N7/Phxq+MQ0l1e\ntMYXQgjDhg0rOKeHlIeQtsnYmp/S1jDLfm5dtmbbxrQGmFsG2F48+QFwieAHwCWCHwCXyrrmp71H\nIYSwdevWONYDka1Zs2Yl1yxhQylpLcsuYdPeNrvcS7+zTU1NyVxLS0sc2++99vbV1NQkc3rY+dSp\nUwt+lhDSpWh2t2g9vU0PMA/h96V4KrdLuvb25XZybu/SN578ALhE8APgUlmnvbt27UquT58+XfC1\nupMzrS0opVxbhl22lUuJnz9/Hse6i0oI6fJMmwZqO4umuSGEMH78+Di2S9/sDjDa3mIPTdeWFbu8\nTdNeu7xN2d+FTftVKXZ54ckPgEsEPwAuEfwAuFTWNT+763LOjh074pjWFpRSWw7c1tqZrY+9ePEi\nju/fv5/M3bp1K47tDsx66prW/0JI63y2teXdu3fJtbbQ2N2a9bPaNhzdbuv79+/J3PDhw+PYtsjo\na+12V7Ye2h48+QFwieAHwKWyTnvbQv93/J/sGqvtAfbRXP/Xfa6z3Xbo19XVFXVvez9N+21Kg85j\nUzZNbe1/M32t3QFF21vszuPahjJo0KCCc/Y9m5ub49iuxLAHnGt7id0BRr/P169fT+Y0fbYHoevv\nYtKkScmcfmdLkeZaPPkBcIngB8Algh8Al6j5/aO6urok77Nu3bo4HjNmTDKn9ZWdO3eW5H45+jOt\nWbOmw++H1rVlCZvunGLbQrRlxNbj3rx50+r7W9ouY++R2x06hBAmT54cx7bmp4eY21YXrYPb0w+1\nvcX+LvTf5X6H7cWTHwCXCH4AXCrrtHfFihXJ9b59+zr8nnYnmWLl2h/UypUrk+u5c+cWfO28efPa\n9VnQsXJtRzo3YMCAZG7cuHFxbA8UVzZ9vXbtWhyfPHkymdOVGppyh5BfmWLTTm2DsatItPwzceLE\nZG7ChAlxbFdWaWsYrS4AUCIEPwAuEfwAuFRRih1R26BTb2YdPHgwjnMHGFlXr16N47a0qGzevDm5\ntst31OLFi+O4qqqq6Hv8gT8/9RlRY2Njwe+2rZ1p20ZlZWUyp7VfuwRS21Tu3r2bzNXX18dxQ0ND\nMqctK3fu3Cn0MX+rudnDjvR7aXdy1mVrdrdorQHqLi4hpD+/revpMr221Pxqa2uL+m7z5AfAJYIf\nAJdcpb1IkPaWUC7ttTTttelcLiXWVhRNj0NIW03srkB68NGrV68Kvqeu0miNtt7Yw470s9pWHv0Z\nbYtMbuNgfZ/cBrAWaS8AZBD8ALhE8APgUlkvbwP+RrkdSXSXFXtouNbVbD1Qa2K2DUV3YLEHgesB\n47rjcwi/76SSq93llr7pPe2c3qMtdb1S4MkPgEsEPwAukfYCXSiXPtqWFU2Dc6ml3Z1F22LsWbya\ndtr3zO3cYlvktGXFpst6bTdo7eRWuwRPfgBcIvgBcIngB8Alan5AF8q1d9iam+7QbJfFac3PLn3T\ndhZbc9PX2mVp9n20lmhfq3P2Z8q19nR2e4viyQ+ASwQ/AC6R9gJ/kVwaWOwBQja11X9n01Wds6s/\nbBuKXtt2mq5sWWkvnvwAuETwA+ASwQ+AS529kzMA/BV48gPgEsEPgEsEPwAuEfwAuETwA+ASwQ+A\nSwQ/AC4R/AC4RPAD4BLBD4BLBD8ALhH8ALhE8APgEsEPgEsEPwAuEfwAuETwA+ASwQ+ASwQ/AC4R\n/AC4RPAD4BLBD4BLBD8ALv0P5JAlQu/OyR4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1c53b30eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Load model, evaluates it on test set (reconstruction error)\n",
    "# display original & reconstructed images\n",
    "\n",
    "def show_reconstructed_digits(\n",
    "    X, \n",
    "    outputs, \n",
    "    model_path = None, \n",
    "    n_test_digits = 2):\n",
    "    \n",
    "    with tf.Session() as sess:\n",
    "        if model_path:\n",
    "            saver.restore(sess, model_path)\n",
    "            \n",
    "        X_test = mnist.test.images[:n_test_digits]\n",
    "        outputs_val = outputs.eval(feed_dict={X: X_test})\n",
    "\n",
    "    fig = plt.figure(figsize=(8, 3 * n_test_digits))\n",
    "    \n",
    "    for digit_index in range(n_test_digits):\n",
    "        \n",
    "        plt.subplot(n_test_digits, 2, digit_index * 2 + 1)\n",
    "        plot_image(X_test[digit_index])\n",
    "        plt.subplot(n_test_digits, 2, digit_index * 2 + 2)\n",
    "        plot_image(outputs_val[digit_index])\n",
    "        plt.show()\n",
    "        \n",
    "#show_reconstructed_digits(X, outputs, \"./my_model_all_layers.ckpt\")\n",
    "show_reconstructed_digits(X, outputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing Features\n",
    "* simplest method: find training instances that activate each hidden node the most. (best on upper layers, given their tendency to capture high-level features.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unsupervised Pretraining with Stacked Autoencoders"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Denoising Autoencoders"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sparse Autoencoders"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variational Autoencoders"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other Autoencoders"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}