chapter5/5.3-Fashion-MNIST.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.13.0.dev20220609'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch,math\n",
    "from pathlib import Path\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt  \n",
    "import torchvision.datasets as dsets\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "\n",
    "import torch.nn.functional as F\n",
    "import torch.nn as NN\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fashion MNIST进行分类 \n",
    "## Fashion MNIST 介绍\n",
    "Fashion MNIST数据集 是kaggle上提供的一个图像分类入门级的数据集，其中包含10个类别的70000个灰度图像。如图所示，这些图片显示的是每件衣服的低分辨率(28×28像素)\n",
    "\n",
    "数据集的下载和介绍：[地址](https://www.kaggle.com/zalando-research/fashionmnist/)\n",
    "\n",
    "\n",
    "Fashion MNIST的目标是作为经典MNIST数据的替换——通常被用作计算机视觉机器学习程序的“Hello, World”。\n",
    "\n",
    "MNIST数据集包含手写数字(0-9等)的图像，格式与我们将在这里使用的衣服相同，MNIST只有手写的0-1数据的复杂度不高，所以他只能用来做“Hello, World”\n",
    "\n",
    "而Fashion MNIST 的由于使用的是衣服的数据，比数字要复杂的多，并且图片的内容也会更加多样性，所以它是一个比常规MNIST稍微更具挑战性的问题。\n",
    "\n",
    "Fashion MNIST这个数据集相对较小，用于验证算法是否按预期工作。它们是测试和调试代码的好起点。\n",
    "\n",
    "## 数据集介绍\n",
    "\n",
    "### 分类\n",
    "```\n",
    "0 T-shirt/top\n",
    "1 Trouser\n",
    "2 Pullover\n",
    "3 Dress\n",
    "4 Coat\n",
    "5 Sandal\n",
    "6 Shirt\n",
    "7 Sneaker\n",
    "8 Bag\n",
    "9 Ankle boot \n",
    "```\n",
    "### 格式\n",
    "\n",
    "fashion-mnist_test.csv\n",
    "\n",
    "fashion-mnist_train.csv\n",
    "\n",
    "存储的训练的数据和测试的数据，格式如下：\n",
    "\n",
    "label是分类的标签\n",
    "pixel1-pixel784是每一个像素代表的值 因为是灰度图像，所以是一个0-255之间的数值。\n",
    "\n",
    "为什么是784个像素？ 28 * 28 = 784\n",
    "\n",
    "### 数据提交\n",
    "\n",
    "Fashion MNIST不需要我们进行数据的提交，数据集中已经帮助我们将 训练集和测试集分好了，我们只需要载入、训练、查看即可，所以Fashion MNIST 是一个非常好的入门级别的数据集\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#指定数据目录\n",
    "DATA_PATH=Path('./data/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>pixel9</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "      <th>pixel784</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>203</td>\n",
       "      <td>214</td>\n",
       "      <td>166</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   label  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  pixel8  \\\n",
       "0      2       0       0       0       0       0       0       0       0   \n",
       "1      9       0       0       0       0       0       0       0       0   \n",
       "2      6       0       0       0       0       0       0       0       5   \n",
       "3      0       0       0       0       1       2       0       0       0   \n",
       "4      3       0       0       0       0       0       0       0       0   \n",
       "5      4       0       0       0       5       4       5       5       3   \n",
       "6      4       0       0       0       0       0       0       0       0   \n",
       "7      5       0       0       0       0       0       0       0       0   \n",
       "8      4       0       0       0       0       0       0       3       2   \n",
       "9      8       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel9  ...  pixel775  pixel776  pixel777  pixel778  pixel779  pixel780  \\\n",
       "0       0  ...         0         0         0         0         0         0   \n",
       "1       0  ...         0         0         0         0         0         0   \n",
       "2       0  ...         0         0         0        30        43         0   \n",
       "3       0  ...         3         0         0         0         0         1   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "5       5  ...         7         8         7         4         3         7   \n",
       "6       0  ...        14         0         0         0         0         0   \n",
       "7       0  ...         0         0         0         0         0         0   \n",
       "8       0  ...         1         0         0         0         0         0   \n",
       "9       0  ...       203       214       166         0         0         0   \n",
       "\n",
       "   pixel781  pixel782  pixel783  pixel784  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2         0         0         0         0  \n",
       "3         0         0         0         0  \n",
       "4         0         0         0         0  \n",
       "5         5         0         0         0  \n",
       "6         0         0         0         0  \n",
       "7         0         0         0         0  \n",
       "8         0         0         0         0  \n",
       "9         0         0         0         0  \n",
       "\n",
       "[10 rows x 785 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.read_csv(DATA_PATH / \"fashion-mnist_train.csv\");\n",
    "train.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>pixel9</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "      <th>pixel784</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>103</td>\n",
       "      <td>87</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>53</td>\n",
       "      <td>99</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "      <td>53</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>137</td>\n",
       "      <td>126</td>\n",
       "      <td>140</td>\n",
       "      <td>0</td>\n",
       "      <td>133</td>\n",
       "      <td>224</td>\n",
       "      <td>222</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>44</td>\n",
       "      <td>105</td>\n",
       "      <td>44</td>\n",
       "      <td>10</td>\n",
       "      <td>...</td>\n",
       "      <td>105</td>\n",
       "      <td>64</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>174</td>\n",
       "      <td>136</td>\n",
       "      <td>155</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>57</td>\n",
       "      <td>70</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   label  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  pixel8  \\\n",
       "0      0       0       0       0       0       0       0       0       9   \n",
       "1      1       0       0       0       0       0       0       0       0   \n",
       "2      2       0       0       0       0       0       0      14      53   \n",
       "3      2       0       0       0       0       0       0       0       0   \n",
       "4      3       0       0       0       0       0       0       0       0   \n",
       "5      2       0       0       0       0       0      44     105      44   \n",
       "6      8       0       0       0       0       0       0       0       0   \n",
       "7      6       0       0       0       0       0       0       0       1   \n",
       "8      5       0       0       0       0       0       0       0       0   \n",
       "9      0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel9  ...  pixel775  pixel776  pixel777  pixel778  pixel779  pixel780  \\\n",
       "0       8  ...       103        87        56         0         0         0   \n",
       "1       0  ...        34         0         0         0         0         0   \n",
       "2      99  ...         0         0         0         0        63        53   \n",
       "3       0  ...       137       126       140         0       133       224   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "5      10  ...       105        64        30         0         0         0   \n",
       "6       0  ...         0         0         0         0         0         0   \n",
       "7       0  ...       174       136       155        31         0         1   \n",
       "8       0  ...         0         0         0         0         0         0   \n",
       "9       0  ...        57        70        28         0         2         0   \n",
       "\n",
       "   pixel781  pixel782  pixel783  pixel784  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2        31         0         0         0  \n",
       "3       222        56         0         0  \n",
       "4         0         0         0         0  \n",
       "5         0         0         0         0  \n",
       "6         0         0         0         0  \n",
       "7         0         0         0         0  \n",
       "8         0         0         0         0  \n",
       "9         0         0         0         0  \n",
       "\n",
       "[10 rows x 785 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test = pd.read_csv(DATA_PATH / \"fashion-mnist_test.csv\");\n",
    "test.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "label         9\n",
       "pixel1       16\n",
       "pixel2       36\n",
       "pixel3      226\n",
       "pixel4      164\n",
       "           ... \n",
       "pixel780    255\n",
       "pixel781    255\n",
       "pixel782    255\n",
       "pixel783    255\n",
       "pixel784    170\n",
       "Length: 785, dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ubyte文件标识了数据的格式\n",
    "\n",
    "其中idx3的数字表示数据维度。也就是图像为3维，\n",
    "idx1 标签维1维。\n",
    "\n",
    "具体格式详解：http://yann.lecun.com/exdb/mnist/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2051, 60000, 28, 28)\n"
     ]
    }
   ],
   "source": [
    "import struct\n",
    "from PIL import Image \n",
    "\n",
    "with open(DATA_PATH / \"train-images-idx3-ubyte\", 'rb') as file_object:\n",
    "    header_data=struct.unpack(\">4I\",file_object.read(16))\n",
    "    print(header_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2049, 60000)\n"
     ]
    }
   ],
   "source": [
    "with open(DATA_PATH / \"train-labels-idx1-ubyte\", 'rb') as file_object:\n",
    "    header_data=struct.unpack(\">2I\",file_object.read(8))\n",
    "    print(header_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下是训练的图片的二进制格式\n",
    "\n",
    "    [offset] [type]          [value]          [description]\n",
    "    0000     32 bit integer  0x00000803(2051) magic number\n",
    "    0004     32 bit integer  60000            number of images\n",
    "    0008     32 bit integer  28               number of rows\n",
    "    0012     32 bit integer  28               number of columns\n",
    "    0016     unsigned byte   ??               pixel\n",
    "    0017     unsigned byte   ??               pixel\n",
    "    ........\n",
    "    xxxx     unsigned byte   ??               pixel\n",
    "    \n",
    "有四字节的header_data，故使用`unpack_from`进行二进制转换时，偏置offset=16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(28, 28)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAR1UlEQVR4nO3dbYyV5ZkH8P9fXlRe5EVEhpcIVoxsNi6sIxpBU60Q9INQtVg+NBh1aUxN2qQma9wPNfGDRLdt9gNpMlVTunZtmhQixrcS0sRuwMpIWECmrYBYBsYBBIHhbRi49sM8mCnOc13jec45z5H7/0vIzJxr7nPuc878OWfmeu7npplBRC5+l5Q9ARGpD4VdJBEKu0giFHaRRCjsIokYXM8bI6k//YvUmJmxv8sLvbKTXEDyryR3kHyqyHWJSG2x0j47yUEA/gZgHoB2ABsBLDGz7c4YvbKL1FgtXtlnA9hhZrvMrBvAbwEsLHB9IlJDRcI+CcCePl+3Z5f9A5LLSLaSbC1wWyJSUJE/0PX3VuFLb9PNrAVAC6C38SJlKvLK3g5gSp+vJwPYV2w6IlIrRcK+EcB0ktNIDgXwXQBrqjMtEam2it/Gm1kPyScAvANgEICXzezDqs1MRKqq4tZbRTem39lFaq4mB9WIyNeHwi6SCIVdJBEKu0giFHaRRCjsIolQ2EUSobCLJEJhF0mEwi6SCIVdJBEKu0giFHaRRNT1VNJSf2S/C6C+UHTV48iRI9363Llzc2tvvfVWoduO7tugQYNyaz09PYVuu6ho7p5KnzO9soskQmEXSYTCLpIIhV0kEQq7SCIUdpFEKOwiiVCf/SJ3ySX+/+dnz55169ddd51bf+yxx9z6yZMnc2vHjx93x546dcqtv//++269SC896oNHj2s0vsjcvOMHvOdTr+wiiVDYRRKhsIskQmEXSYTCLpIIhV0kEQq7SCLUZ7/IeT1ZIO6z33XXXW797rvvduvt7e25tUsvvdQdO2zYMLc+b948t/7iiy/m1jo7O92x0Zrx6HGLjBgxIrd27tw5d+yJEycqus1CYSe5G8AxAGcB9JhZc5HrE5HaqcYr+51mdrAK1yMiNaTf2UUSUTTsBuAPJD8guay/byC5jGQrydaCtyUiBRR9Gz/HzPaRHA9gLcm/mNm7fb/BzFoAtAAAyWJnNxSRihV6ZTezfdnH/QBWA5hdjUmJSPVVHHaSw0mOPP85gPkAtlVrYiJSXUXexl8NYHW2bncwgP8xs7erMiupmu7u7kLjb775Zrc+depUt+71+aM14e+8845bnzVrllt//vnnc2utrf6fkLZu3erW29ra3Prs2f6bXO9xXb9+vTt2w4YNubWurq7cWsVhN7NdAP6l0vEiUl9qvYkkQmEXSYTCLpIIhV0kEQq7SCJYdMver3RjOoKuJrzTFkfPb7RM1GtfAcDo0aPd+pkzZ3Jr0VLOyMaNG936jh07cmtFW5JNTU1u3bvfgD/3Bx980B27YsWK3FprayuOHj3a7w+EXtlFEqGwiyRCYRdJhMIukgiFXSQRCrtIIhR2kUSoz94Aou19i4ie3/fee8+tR0tYI959i7YtLtoL97Z8jnr8mzZtcuteDx+I79uCBQtya9dee607dtKkSW7dzNRnF0mZwi6SCIVdJBEKu0giFHaRRCjsIolQ2EUSoS2bG0A9j3W40OHDh916tG775MmTbt3blnnwYP/Hz9vWGPD76ABw+eWX59aiPvvtt9/u1m+77Ta3Hp0me/z48bm1t9+uzRnZ9coukgiFXSQRCrtIIhR2kUQo7CKJUNhFEqGwiyRCffbEDRs2zK1H/eKofuLEidzakSNH3LGfffaZW4/W2nvHL0TnEIjuV/S4nT171q17ff4pU6a4YysVvrKTfJnkfpLb+lw2luRakh9lH8fUZHYiUjUDeRv/KwAXnlbjKQDrzGw6gHXZ1yLSwMKwm9m7AA5dcPFCACuzz1cCWFTdaYlItVX6O/vVZtYBAGbWQTL3QF+SywAsq/B2RKRKav4HOjNrAdAC6ISTImWqtPXWSbIJALKP+6s3JRGphUrDvgbA0uzzpQBeq850RKRWwrfxJF8F8E0A40i2A/gJgOUAfkfyUQB/B/CdWk7yYle05+v1dKM14RMnTnTrp0+fLlT31rNH54X3evRAvDe816eP+uRDhw5168eOHXPro0aNcutbtmzJrUXPWXNzc25t+/btubUw7Ga2JKf0rWisiDQOHS4rkgiFXSQRCrtIIhR2kUQo7CKJ0BLXBhCdSnrQoEFu3Wu9PfTQQ+7YCRMmuPUDBw64de90zYC/lHP48OHu2GipZ9S689p+Z86cccdGp7mO7veVV17p1lesWJFbmzlzpjvWm5vXxtUru0giFHaRRCjsIolQ2EUSobCLJEJhF0mEwi6SCNZzu2CdqaZ/UU+3p6en4uu+5ZZb3Pobb7zh1qMtmYscAzBy5Eh3bLQlc3Sq6SFDhlRUA+JjAKKtriPefXvhhRfcsa+88opbN7N+m+16ZRdJhMIukgiFXSQRCrtIIhR2kUQo7CKJUNhFEvG1Ws/urdWN+r3R6Zij0zl765+9NdsDUaSPHnnzzTfd+vHjx9161GePTrnsHccRrZWPntPLLrvMrUdr1ouMjZ7zaO433nhjbi3ayrpSemUXSYTCLpIIhV0kEQq7SCIUdpFEKOwiiVDYRRLRUH32Imuja9mrrrU77rjDrT/wwANufc6cObm1aNvjaE141EeP1uJ7z1k0t+jnwTsvPOD34aPzOERzi0SPW1dXV27t/vvvd8e+/vrrFc0pfGUn+TLJ/SS39bnsGZJ7SW7O/t1b0a2LSN0M5G38rwAs6Ofyn5vZzOyff5iWiJQuDLuZvQvgUB3mIiI1VOQPdE+Q3JK9zR+T900kl5FsJdla4LZEpKBKw/4LAN8AMBNAB4Cf5n2jmbWYWbOZNVd4WyJSBRWF3cw6zeysmZ0D8EsAs6s7LRGptorCTrKpz5ffBrAt73tFpDGE540n+SqAbwIYB6ATwE+yr2cCMAC7AXzfzDrCGyvxvPFjx4516xMnTnTr06dPr3hs1De9/vrr3frp06fdurdWP1qXHe0zvm/fPrcenX/d6zdHe5hH+68PGzbMra9fvz63NmLECHdsdOxDtJ49WpPuPW6dnZ3u2BkzZrj1vPPGhwfVmNmSfi5+KRonIo1Fh8uKJEJhF0mEwi6SCIVdJBEKu0giGmrL5ltvvdUd/+yzz+bWrrrqKnfs6NGj3bq3FBPwl1t+/vnn7tho+W3UQopaUN5psKNTQbe1tbn1xYsXu/XWVv8oaG9b5jFjco+yBgBMnTrVrUd27dqVW4u2iz527Jhbj5bARi1Nr/V3xRVXuGOjnxdt2SySOIVdJBEKu0giFHaRRCjsIolQ2EUSobCLJKLufXavX71hwwZ3fFNTU24t6pNH9SKnDo5OeRz1uosaNWpUbm3cuHHu2Icfftitz58/360//vjjbt1bInvq1Cl37Mcff+zWvT464C9LLrq8NlraG/XxvfHR8tlrrrnGravPLpI4hV0kEQq7SCIUdpFEKOwiiVDYRRKhsIskoq599nHjxtl9992XW1++fLk7fufOnbm16NTAUT3a/tcT9Vy9PjgA7Nmzx61Hp3P21vJ7p5kGgAkTJrj1RYsWuXVvW2TAX5MePSc33XRTobp336M+evS4RVsyR7xzEEQ/T955Hz799FN0d3erzy6SMoVdJBEKu0giFHaRRCjsIolQ2EUSobCLJCLcxbWaenp6sH///tx61G/21ghH2xpH1x31fL2+anSe70OHDrn1Tz75xK1Hc/PWy0drxqNz2q9evdqtb9261a17ffZoG+2oFx6dr9/brjq639Ga8qgXHo33+uxRD9/b4tt7TMJXdpJTSP6RZBvJD0n+MLt8LMm1JD/KPvpn/BeRUg3kbXwPgB+b2QwAtwL4Acl/AvAUgHVmNh3AuuxrEWlQYdjNrMPMNmWfHwPQBmASgIUAVmbfthLAohrNUUSq4Cv9gY7kVACzAPwZwNVm1gH0/ocAYHzOmGUkW0m2Rr+DiUjtDDjsJEcA+D2AH5nZ0YGOM7MWM2s2s+aiiwdEpHIDCjvJIegN+m/MbFV2cSfJpqzeBCD/z+wiUrqw9cbeHsFLANrM7Gd9SmsALAWwPPv4WnRd3d3d2Lt3b249Wm7b3t6eWxs+fLg7NjqlctTGOXjwYG7twIED7tjBg/2HOVpeG7V5vGWm0SmNo6Wc3v0GgBkzZrj148eP59aidujhw4fdevS4eXP32nJA3JqLxkdbNntLi48cOeKOnTlzZm5t27ZtubWB9NnnAPgegK0kN2eXPY3ekP+O5KMA/g7gOwO4LhEpSRh2M/tfAHlHAHyrutMRkVrR4bIiiVDYRRKhsIskQmEXSYTCLpKIui5xPXnyJDZv3pxbX7VqVW4NAB555JHcWnS65Wh732gpqLfMNOqDRz3X6MjCaEtob3lvtFV1dGxDtJV1R0dHxdcfzS06PqHIc1Z0+WyR5bWA38efNm2aO7azs7Oi29Uru0giFHaRRCjsIolQ2EUSobCLJEJhF0mEwi6SiLpu2Uyy0I3dc889ubUnn3zSHTt+fL9nzfpCtG7b66tG/eKoTx712aN+s3f93imLgbjPHh1DENW9+xaNjeYe8cZ7veqBiJ6z6FTS3nr2LVu2uGMXL17s1s1MWzaLpExhF0mEwi6SCIVdJBEKu0giFHaRRCjsIomoe5/dO0951Jss4s4773Trzz33nFv3+vSjRo1yx0bnZo/68FGfPerze7wttIG4D+/tAwD4z2lXV5c7NnpcIt7co/Xm0Tr+6Dldu3atW29ra8utrV+/3h0bUZ9dJHEKu0giFHaRRCjsIolQ2EUSobCLJEJhF0lE2GcnOQXArwFMAHAOQIuZ/RfJZwD8G4Dzm5M/bWZvBtdVv6Z+Hd1www1uveje8JMnT3bru3fvzq1F/eSdO3e6dfn6yeuzD2STiB4APzazTSRHAviA5PkjBn5uZv9ZrUmKSO0MZH/2DgAd2efHSLYBmFTriYlIdX2l39lJTgUwC8Cfs4ueILmF5Mskx+SMWUaylWRrsamKSBEDDjvJEQB+D+BHZnYUwC8AfAPATPS+8v+0v3Fm1mJmzWbWXHy6IlKpAYWd5BD0Bv03ZrYKAMys08zOmtk5AL8EMLt20xSRosKws/cUnS8BaDOzn/W5vKnPt30bwLbqT09EqmUgrbe5AP4EYCt6W28A8DSAJeh9C28AdgP4fvbHPO+6LsrWm0gjyWu9fa3OGy8iMa1nF0mcwi6SCIVdJBEKu0giFHaRRCjsIolQ2EUSobCLJEJhF0mEwi6SCIVdJBEKu0giFHaRRCjsIokYyNllq+kggE/6fD0uu6wRNercGnVegOZWqWrO7Zq8Ql3Xs3/pxsnWRj03XaPOrVHnBWhularX3PQ2XiQRCrtIIsoOe0vJt+9p1Lk16rwAza1SdZlbqb+zi0j9lP3KLiJ1orCLJKKUsJNcQPKvJHeQfKqMOeQhuZvkVpKby96fLttDbz/JbX0uG0tyLcmPso/97rFX0tyeIbk3e+w2k7y3pLlNIflHkm0kPyT5w+zyUh87Z151edzq/js7yUEA/gZgHoB2ABsBLDGz7XWdSA6SuwE0m1npB2CQvANAF4Bfm9k/Z5c9D+CQmS3P/qMcY2b/3iBzewZAV9nbeGe7FTX13WYcwCIAD6PEx86Z12LU4XEr45V9NoAdZrbLzLoB/BbAwhLm0fDM7F0Ahy64eCGAldnnK9H7w1J3OXNrCGbWYWabss+PATi/zXipj50zr7ooI+yTAOzp83U7Gmu/dwPwB5IfkFxW9mT6cfX5bbayj+NLns+Fwm286+mCbcYb5rGrZPvzosoIe39b0zRS/2+Omf0rgHsA/CB7uyoDM6BtvOuln23GG0Kl258XVUbY2wFM6fP1ZAD7SphHv8xsX/ZxP4DVaLytqDvP76Cbfdxf8ny+0EjbePe3zTga4LErc/vzMsK+EcB0ktNIDgXwXQBrSpjHl5Acnv3hBCSHA5iPxtuKeg2ApdnnSwG8VuJc/kGjbOOdt804Sn7sSt/+3Mzq/g/Avej9i/xOAP9Rxhxy5nUtgP/L/n1Y9twAvIret3Vn0PuO6FEAVwJYB+Cj7OPYBprbf6N3a+8t6A1WU0lzm4veXw23ANic/bu37MfOmVddHjcdLiuSCB1BJ5IIhV0kEQq7SCIUdpFEKOwiiVDYRRKhsIsk4v8B1lwxmxAZrsAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with open(DATA_PATH / \"train-images-idx3-ubyte\", 'rb') as file_object:\n",
    "    raw_img=file_object.read()\n",
    "img = struct.unpack_from(\">784B\",raw_img,16)\n",
    "image = np.asarray(img)\n",
    "image = image.reshape((28,28))\n",
    "print(image.shape)\n",
    "plt.imshow(image,cmap = plt.cm.gray)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0,)\n"
     ]
    }
   ],
   "source": [
    "with open(DATA_PATH / \"train-labels-idx1-ubyte\", 'rb') as file_object:\n",
    "    raw_img = file_object.read(1)\n",
    "    label = struct.unpack(\">B\",raw_img)\n",
    "    print(label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里好像有点错误，显示的错位了，但是我的确是按照格式进行处理的。这种格式处理起来比较复杂，并且数据集中的csv直接给出了每个像素的值，所以这里我们可以直接使用csv格式的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据加载\n",
    "\n",
    "为了使用pytorch的dataloader进行数据的加载，需要先创建一个自定义的dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class FashionMNISTDataset(Dataset):\n",
    "    def __init__(self, csv_file, transform=None):\n",
    "        data = pd.read_csv(csv_file)\n",
    "        self.X = np.array(data.iloc[:, 1:]).reshape(-1, 1, 28, 28).astype(float)\n",
    "        self.Y = np.array(data.iloc[:, 0])\n",
    "        del data;  #结束data对数据的引用,节省空间\n",
    "        self.len=len(self.X)\n",
    "\n",
    "    def __len__(self):\n",
    "        #return len(self.X)\n",
    "        return self.len\n",
    "        \n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        item = self.X[idx]\n",
    "        label = self.Y[idx]\n",
    "        return (item, label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于自定义的数据集，只需要实现三个函数：\n",
    "\n",
    "`__init__`： 初始化函数主要用于数据的加载，这里直接使用pandas将数据读取为dataframe，然后将其转成numpy数组来进行索引\n",
    "\n",
    "`__len__`： 返回数据集的总数，pytorch里面的datalorder需要知道数据集的总数的\n",
    "\n",
    "`__getitem__`：会返回单张图片，它包含一个index，返回值为样本及其标签。\n",
    "\n",
    "创建训练和测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataset = FashionMNISTDataset(csv_file=DATA_PATH / \"fashion-mnist_train.csv\")\n",
    "test_dataset = FashionMNISTDataset(csv_file=DATA_PATH / \"fashion-mnist_test.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在使用Pytorch的DataLoader读取数据之前，需要指定一个batch size 这也是一个超参数，涉及到内存的使用量，如果出现OOM的错误则要减小这个数值，一般这个数值都为2的幂或者2的倍数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#因为是常量，所以大写，需要说明的是，这些常量建议都使用完整的英文单词，减少歧义\n",
    "BATCH_SIZE=256 # 这个batch 可以在M250的笔记本显卡中进行训练，不会oom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们接着使用dataloader模块来使用这些数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_loader = torch.utils.data.DataLoader(dataset=train_dataset,\n",
    "                                           batch_size=BATCH_SIZE,\n",
    "                                           shuffle=True) # shuffle 标识要打乱顺序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_loader = torch.utils.data.DataLoader(dataset=test_dataset,\n",
    "                                           batch_size=BATCH_SIZE,\n",
    "                                           shuffle=False) # shuffle 标识要打乱顺序，测试集不需要打乱"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看一下数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1, 28, 28]), torch.Size([28, 28]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=iter(train_loader)\n",
    "data=next(a)\n",
    "img=data[0][0].reshape(28,28)\n",
    "data[0][0].shape,img.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAARWklEQVR4nO3dW4yV53UG4PflZM4Mw2E4iiTIsutTSYVRLaoKK2pkuMGxTBUuIipZnVwkUpC4KHIlxzeVrLpJmosq0qRGgSp1hJTY5sKqghAWji9iY0TN4Ck2BcppzGAw5mDDwLB6MT/WBM9ea7z/fZpZ7yONZmav+Wd/s/Hrf++9/u/7aGYQkbFvXLMHICKNobCLJKGwiyShsIskobCLJDGhkXdGMuVb/zNmzHDrN2/edOvXr1+v5XBGjVmzZrn16HG5ceNGLYczapgZh7u9VNhJPgHg5wDGA/h3M3uhzO8bq1atWuXWz50759a7u7trOZyWMW6c/8RyzZo1bv3w4cNu/ejRo191SGNa1U/jSY4H8G8A1gJ4AMBGkg/UamAiUltlXrOvAnDUzI6ZWT+A3wBYX5thiUitlQn7YgCnhnx/urjtT5DsJLmf5P4S9yUiJZV5zT7cmwBfegPOzLoAdAF536ATaQVlzuynASwd8v0SAGfLDUdE6qVM2N8BcC/Jr5OcBOC7AHbVZlgiUmssM+uN5DoA/4rB1ts2M/un4OdH7dP4Rx55pGLtqaeeco8dGBhw6xMm+K+myGHbpl947rnn3HozeWNbsmSJe2xPT49bnzx5slu/detWxdqLL77oHjua1aXPbmavA3i9zO8QkcbQ5bIiSSjsIkko7CJJKOwiSSjsIkko7CJJlOqzf+U7G8V99q1bt1aszZkzxz324sWLbj2al/3www+79UWLFlWsnTx50j1227Ztbn3t2rVufcOGDW79woULFWs7d+50j719+7Zbj7S3t1esvfnmm+6xb7zxRqn7bqZKfXad2UWSUNhFklDYRZJQ2EWSUNhFklDYRZJo6FLSo9nixV9acesL0VLQkaj9eeDAAbfe29tbsfboo4+6x0bTa6O/7e2333brXgsrmqIatd6i4/v7+yvWFixY4B47FunMLpKEwi6ShMIukoTCLpKEwi6ShMIukoTCLpKE+uyFaJrqtGnTKtauXbvmHustaQzEveyon3zs2LGKtSNHjrjHRn34vr4+tx7tlBptV+0ZP368W4+mBk+ZMqVibdmyZVWNaTTTmV0kCYVdJAmFXSQJhV0kCYVdJAmFXSQJhV0kCfXZC1E/2Ov5Tp061T32888/d+vRlswTJ050621tbRVrZbY1BuK/bd68eW7du4bg/Pnz7rFRnz16XMaNq3wumzVrlnvsWFQq7CRPALgCYADALTNbWYtBiUjt1eLM/riZfVyD3yMidaTX7CJJlA27Afg9yXdJdg73AyQ7Se4nub/kfYlICWWfxq82s7Mk5wPYTfJ/zGzf0B8wsy4AXcDo3utNZLQrdWY3s7PF5z4ArwBYVYtBiUjtVR12ktNIzrjzNYBvA+iu1cBEpLbKPI3vAPBK0SOeAOA/zey/ajKqJpg+fbpbnzRpUsVa1O/11i8HgPnz57v1e+65x617bty44dajsUeisXmPa7QOwOXLl926t8YA4F9j4M11H6uqDruZHQPw5zUci4jUkVpvIkko7CJJKOwiSSjsIkko7CJJaIprwWutRaJppNHvjrZNjnjHR9tBR9NvZ86c6dY/+eQTt+61x8pOr/WmsALAwMBAxVrUDh2LdGYXSUJhF0lCYRdJQmEXSUJhF0lCYRdJQmEXSUJ99kJHR4db96aCRtNEo+mz0VLS0VRQrxceTUGN+uRRrzta7tnrhUePW3T9QdRnv337dtX3PRbpzC6ShMIukoTCLpKEwi6ShMIukoTCLpKEwi6ShPrshahn6/V8o150tORx1Ku+fv26W/dEY4t62WX70ZcuXapYi7Z7jq4BiMbuLRc9Z84c99hoHn+0zHUr0pldJAmFXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAn12UfIWxs+6qNHojnnXq86Eq1pH63dHm1tfPPmTbf+0UcfVawtWrTIPfb06dNuPerTe+v1X7lyxT022kZ7TPbZSW4j2Ueye8ht7SR3k/yw+Dy7vsMUkbJG8jT+VwCeuOu2rQD2mNm9APYU34tICwvDbmb7AFy86+b1ALYXX28H8GRthyUitVbta/YOM+sFADPrJVnxBQ7JTgCdVd6PiNRI3d+gM7MuAF0AQNLfZVBE6qba1ts5kgsBoPjcV7shiUg9VBv2XQA2FV9vAvBabYYjIvUSPo0n+TKANQDmkjwN4McAXgCwk+QzAE4C2FDPQTZCe3u7W/d66dHc52iP9KhP/9lnn7n1tra2irVoPnvUJ4/mjEdr4nv7v0drCETXAHjrwgP+NQZeDx4Ali9f7taPHj3q1ltRGHYz21ih9K0aj0VE6kiXy4okobCLJKGwiyShsIskobCLJKEproVoaWGvTRQtBd3f3+/WZ8yY4dYHBgbcurfcc9R681pjQNzeilpYZVpv0eMWtSSjKbSeaGrvaKQzu0gSCrtIEgq7SBIKu0gSCrtIEgq7SBIKu0gS6rMXon6xJ5rCGk3VjJaSjrZs9vrVUb846qNH9Wjs3t8ePW7RtQ/Xrl1z657o36Ts8uCtSGd2kSQUdpEkFHaRJBR2kSQUdpEkFHaRJBR2kSTUZy9Ec6u9OevRvOqyve5obF69zPUDAEDSrUd/mzfXPuqTR73u3t7equ87unZhLNKZXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAmFXSQJ9dkL0dbFXr/65MmTpe476mVHfXivH+31moF4XnfUp4/WpffmrF+5csU9NlpP/8aNG27d66VH22xHYxuNwjM7yW0k+0h2D7nteZJnSB4sPtbVd5giUtZInsb/CsATw9z+MzNbUXy8XtthiUithWE3s30ALjZgLCJSR2XeoPshyfeKp/mzK/0QyU6S+0nuL3FfIlJStWH/BYDlAFYA6AXwk0o/aGZdZrbSzFZWeV8iUgNVhd3MzpnZgJndBvBLAKtqOywRqbWqwk5y4ZBvvwOgu9LPikhrCPvsJF8GsAbAXJKnAfwYwBqSKwAYgBMAvl+/ITZGNL/Z61e/+uqr7rFLly5169H+7hFv7fbJkye7x5adrx714b0++8WL/vu+y5Ytc+tHjhxx61evXq1Y27x5s3tstG/9aBSG3cw2DnPzS3UYi4jUkS6XFUlCYRdJQmEXSUJhF0lCYRdJQlNcC1Hr7aGHHqpY+/jjj91jV6xY4dajaaYTJvj/TF492lI5mj5btvXmiZbIjgwMDLj148ePV6xFf1fUkhyNdGYXSUJhF0lCYRdJQmEXSUJhF0lCYRdJQmEXSUJ99kJ/f79b93rhUZ88mmYaLYkc9Xy9KbJRjz7aNjnqR9dzq+to6u/8+fPdujcFNro+IPq7RyOd2UWSUNhFklDYRZJQ2EWSUNhFklDYRZJQ2EWSUJ+9EPXKDx48WLEWLYkcbQcd1css9xxt2ewt9QwA7e3tbj163KZPn16xFvXRoz58ma2uP/jgA/fYqIc/GunMLpKEwi6ShMIukoTCLpKEwi6ShMIukoTCLpKE+uyFefPmuXWvnxzNhY/mjLe1tbn1iNdnj3rZ0brxc+bMcesnTpxw695c/mit/mjs0TUEnqhHH61BMBqFZ3aSS0nuJdlD8jDJHxW3t5PcTfLD4vPs+g9XRKo1kqfxtwBsMbM/A/CXAH5A8gEAWwHsMbN7AewpvheRFhWG3cx6zexA8fUVAD0AFgNYD2B78WPbATxZpzGKSA18pdfsJL8G4JsA/gigw8x6gcH/IZAc9mJikp0AOkuOU0RKGnHYSU4H8FsAm83s8kg3vjOzLgBdxe/wZ12ISN2MqPVGciIGg/5rM/tdcfM5kguL+kIAffUZoojUQnhm5+Ap/CUAPWb20yGlXQA2AXih+PxaXUbYIFOnTnXrs2bNqliLllOOWkTeNFAgbhN5yyJH9x3Vvb8biNtn3t8WbXUdPa5Ru9SbIhv9XTNnznTro9FInsavBvA9AIdIHixuexaDId9J8hkAJwFsqMsIRaQmwrCb2R8AVDq1fKu2wxGRetHlsiJJKOwiSSjsIkko7CJJKOwiSWiKa+Hs2bNu3dv6+MCBA+6xW7ZsqWpMd0S9bK9XHm3ZHC1jHU0zjXrhXj87mkYaLVMdXX/gLfEdLVP9/vvvu/XRSGd2kSQUdpEkFHaRJBR2kSQUdpEkFHaRJBR2kSTUZy/s2LGjVN2zcOFCt3748GG3HvXKvaWqT5065R774IMPuvXLly+79WhOuncNQLSMdXQNQBlPP/103X53q9KZXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAmFXSQJ9dkbYPfu3W79/vvvd+uPPfaYW/d64T09Pe6xHR0dbv348eOljvd66ffdd5977KVLl9z6hQsX3LonmgtvNvY2L9KZXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAmFXSSJkezPvhTADgALANwG0GVmPyf5PIC/B3C++NFnzez1eg202bz10aN52Xv37nXrjz/+uFuP5ox/+umnFWvRfPSoHu1THvXZvbXf29ra3GPnzp3r1vft2+fWPRn77CO5qOYWgC1mdoDkDADvkrxzlcjPzOxf6jc8EamVkezP3gugt/j6CskeAIvrPTARqa2v9Jqd5NcAfBPAH4ubfkjyPZLbSM6ucEwnyf0k95cbqoiUMeKwk5wO4LcANpvZZQC/ALAcwAoMnvl/MtxxZtZlZivNbGX54YpItUYUdpITMRj0X5vZ7wDAzM6Z2YCZ3QbwSwCr6jdMESkrDDsH37Z8CUCPmf10yO1Dl0z9DoDu2g9PRGplJO/GrwbwPQCHSB4sbnsWwEaSKwAYgBMAvl+H8bWMMq2YQ4cOufVoKme0lPSUKVMq1pYtW+Yee/XqVbe+YMECtx49Lt6WzwMDA+6x58+fd+vd3dWfX6LW21g0knfj/wBguEdmzPbURcYiXUEnkoTCLpKEwi6ShMIukoTCLpKEwi6SBBs5lY/k2Js3WAOrV69269G2yl4vO/r3jepTp05165MnT3br3nLPZ86ccY9966233Hp0fUJWZjbsRQQ6s4skobCLJKGwiyShsIskobCLJKGwiyShsIsk0eg++3kA/zfkprkA/HWSm6dVx9aq4wI0tmrVcmzLzGzecIWGhv1Ld07ub9W16Vp1bK06LkBjq1ajxqan8SJJKOwiSTQ77F1Nvn9Pq46tVccFaGzVasjYmvqaXUQap9lndhFpEIVdJImmhJ3kEySPkDxKcmszxlAJyRMkD5E82Oz96Yo99PpIdg+5rZ3kbpIfFp+H3WOvSWN7nuSZ4rE7SHJdk8a2lORekj0kD5P8UXF7Ux87Z1wNedwa/pqd5HgAHwD4GwCnAbwDYKOZvd/QgVRA8gSAlWbW9AswSP41gKsAdpjZQ8Vt/wzgopm9UPyPcraZ/UOLjO15AFebvY13sVvRwqHbjAN4EsDfoYmPnTOuv0UDHrdmnNlXAThqZsfMrB/AbwCsb8I4Wp6Z7QNw8a6b1wPYXny9HYP/sTRchbG1BDPrNbMDxddXANzZZrypj50zroZoRtgXAzg15PvTaK393g3A70m+S7Kz2YMZRoeZ9QKD//EAmN/k8dwt3Ma7ke7aZrxlHrtqtj8vqxlhH259rFbq/602s78AsBbAD4qnqzIyI9rGu1GG2Wa8JVS7/XlZzQj7aQBLh3y/BMDZJoxjWGZ2tvjcB+AVtN5W1Ofu7KBbfO5r8ni+0ErbeA+3zTha4LFr5vbnzQj7OwDuJfl1kpMAfBfAriaM40tITiveOAHJaQC+jdbbinoXgE3F15sAvNbEsfyJVtnGu9I242jyY9f07c/NrOEfANZh8B35/wXwj80YQ4VxfQPAfxcfh5s9NgAvY/Bp3U0MPiN6BsAcAHsAfFh8bm+hsf0HgEMA3sNgsBY2aWx/hcGXhu8BOFh8rGv2Y+eMqyGPmy6XFUlCV9CJJKGwiyShsIskobCLJKGwiyShsIskobCLJPH/53JwXgsI9sAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(img,cmap = plt.cm.gray)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这回看着就没问题了，是一个完整的图了，所以我们还是用csv吧\n",
    "\n",
    "## 创建网络\n",
    "\n",
    "三层的简单的CNN网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CNN(NN.Module):\n",
    "    def __init__(self):\n",
    "        super(CNN, self).__init__()\n",
    "        self.layer1 = NN.Sequential(   \n",
    "            NN.Conv2d(1, 16, kernel_size=5, padding=2),\n",
    "            NN.BatchNorm2d(16), \n",
    "            NN.ReLU()) #16, 28, 28\n",
    "        self.pool1=NN.MaxPool2d(2) #16, 14, 14\n",
    "        self.layer2 = NN.Sequential(\n",
    "            NN.Conv2d(16, 32, kernel_size=3),\n",
    "            NN.BatchNorm2d(32),\n",
    "            NN.ReLU())#32, 12, 12\n",
    "        self.layer3 = NN.Sequential(\n",
    "            NN.Conv2d(32, 64, kernel_size=3),\n",
    "            NN.BatchNorm2d(64),\n",
    "            NN.ReLU()) #64, 10, 10\n",
    "        self.pool2=NN.MaxPool2d(2)  #64, 5, 5\n",
    "        self.fc = NN.Linear(5*5*64, 10)\n",
    "    def forward(self, x):\n",
    "        out = self.layer1(x)\n",
    "        #print(out.shape)\n",
    "        out=self.pool1(out)\n",
    "        #print(out.shape)\n",
    "        out = self.layer2(out)\n",
    "        #print(out.shape)\n",
    "        out=self.layer3(out)\n",
    "        #print(out.shape)\n",
    "        out=self.pool2(out)\n",
    "        #print(out.shape)\n",
    "        out = out.view(out.size(0), -1)\n",
    "        #print(out.shape)\n",
    "        out = self.fc(out)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上代码看起来很简单。这里面都是包含的数学的含义。我们只讲pytorch相关的：在函数里使用torch.nn提供的模块来定义各个层，在每个卷积层后使用了批次的归一化和RELU激活并且在每一个操作分组后面进行了pooling的操作（减少信息量，避免过拟合），后我们使用了全连接层来输出10个类别。\n",
    "\n",
    "view函数用来改变输出值矩阵的形状来匹配最后一层的维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.2606,  0.9980,  0.8250, -1.2780,  0.0940, -0.2697,  0.4149, -0.0625,\n",
       "          0.9231,  0.0346]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cnn = CNN()\n",
    "#可以通过以下方式验证，没报错说明没问题，\n",
    "cnn(torch.rand(1,1,28,28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CNN(\n",
      "  (layer1): Sequential(\n",
      "    (0): Conv2d(1, 16, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n",
      "    (1): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (2): ReLU()\n",
      "  )\n",
      "  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (layer2): Sequential(\n",
      "    (0): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
      "    (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (2): ReLU()\n",
      "  )\n",
      "  (layer3): Sequential(\n",
      "    (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1))\n",
      "    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (2): ReLU()\n",
      "  )\n",
      "  (pool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (fc): Linear(in_features=1600, out_features=10, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "#打印下网络，做最后的确认\n",
    "print(cnn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从定义模型开始就要指定模型计算的位置，CPU还是GPU，所以需要加另外一个参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mps\n"
     ]
    }
   ],
   "source": [
    "# DEVICE=torch.device(\"cpu\")\n",
    "# if torch.cuda.is_available():\n",
    "#         DEVICE=torch.device(\"cuda\")\n",
    "DEVICE = torch.device(\"mps\" if torch.backends.mps.is_available() else \"cpu\")\n",
    "print(DEVICE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#先把网络放到gpu上\n",
    "cnn=cnn.to(DEVICE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 损失函数\n",
    "多分类因为使用Softmax回归将神经网络前向传播得到的结果变成概率分布 所以使用交叉熵损失。\n",
    "在pytorch中 \n",
    "NN.CrossEntropyLoss 是将 `nn.LogSoftmax()` 和 `nn.NLLLoss()`进行了整合，[CrossEntropyLoss](https://pytorch.org/docs/stable/nn.html#crossentropyloss) ,我们也可以分开来写使用两步计算，这里为了方便直接一步到位\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "#损失函数也需要放到GPU中\n",
    "criterion = NN.CrossEntropyLoss().to(DEVICE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 优化器\n",
    "Adam 优化器：简单，暴力，最主要还是懒"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "#另外一个超参数，学习率\n",
    "LEARNING_RATE=0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "#优化器不需要放GPU\n",
    "optimizer = torch.optim.Adam(cnn.parameters(), lr=LEARNING_RATE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 开始训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#另外一个超参数，指定训练批次\n",
    "TOTAL_EPOCHS=10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch : 1/10, Iter : 100/234,  Loss: 0.5333\n",
      "Epoch : 1/10, Iter : 200/234,  Loss: 0.3805\n",
      "Epoch : 2/10, Iter : 100/234,  Loss: 0.3817\n",
      "Epoch : 2/10, Iter : 200/234,  Loss: 0.2733\n",
      "Epoch : 3/10, Iter : 100/234,  Loss: 0.3124\n",
      "Epoch : 3/10, Iter : 200/234,  Loss: 0.2351\n",
      "Epoch : 4/10, Iter : 100/234,  Loss: 0.2251\n",
      "Epoch : 4/10, Iter : 200/234,  Loss: 0.2149\n",
      "Epoch : 5/10, Iter : 100/234,  Loss: 0.2804\n",
      "Epoch : 5/10, Iter : 200/234,  Loss: 0.1902\n",
      "Epoch : 6/10, Iter : 100/234,  Loss: 0.1875\n",
      "Epoch : 6/10, Iter : 200/234,  Loss: 0.1961\n",
      "Epoch : 7/10, Iter : 100/234,  Loss: 0.1918\n",
      "Epoch : 7/10, Iter : 200/234,  Loss: 0.1819\n",
      "Epoch : 8/10, Iter : 100/234,  Loss: 0.1497\n",
      "Epoch : 8/10, Iter : 200/234,  Loss: 0.2436\n",
      "Epoch : 9/10, Iter : 100/234,  Loss: 0.1616\n",
      "Epoch : 9/10, Iter : 200/234,  Loss: 0.2363\n",
      "Epoch : 10/10, Iter : 100/234,  Loss: 0.1231\n",
      "Epoch : 10/10, Iter : 200/234,  Loss: 0.1674\n",
      "CPU times: user 1min 34s, sys: 1min 57s, total: 3min 31s\n",
      "Wall time: 1min 14s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "#记录损失函数\n",
    "losses = [];\n",
    "for epoch in range(TOTAL_EPOCHS):\n",
    "    for i, (images, labels) in enumerate(train_loader):\n",
    "        images = images.float().to(DEVICE)\n",
    "        labels = labels.to(DEVICE)\n",
    "        #清零\n",
    "        optimizer.zero_grad()\n",
    "        outputs = cnn(images)\n",
    "        #计算损失函数\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        losses.append(loss.cpu().data.item());\n",
    "        if (i+1) % 100 == 0:\n",
    "            print ('Epoch : %d/%d, Iter : %d/%d,  Loss: %.4f'%(epoch+1, TOTAL_EPOCHS, i+1, len(train_dataset)//BATCH_SIZE, loss.data.item()))\n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练后操作\n",
    "### 可视化损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEVCAYAAADgh5I1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABKiUlEQVR4nO3dd3hUVfoH8O/03tNIQq8hIFIFlCKgRFGiomJQwPXnFlw7K5ZdC66s61pWYRUUV10LsqyKrCDqqmBHpUmNCKEF0qf3cs/vj3ivMyRkSkKSybyf55mHzJ1775x7uPO+99xyjogxxkAIISTjiTu6AIQQQjoHSgiEEEIAUEIghBDyM0oIhBBCAADSji5AJmKMweFwoKGhAQ6HAx6PBw6HAzabDQ0NDXC5XAgEAggGgwgGgwiFQvB6vfB4PPD5fAgGgwiHw4hEIjHrFYlEkEgkkEqlkMvlkMlkkEqlkMlkkMlkUKvVMJvN0Ov10Ol0MBgM0Gg0MBqNMBgMUCqVUCqV0Gg0MBgMkMlkHVRDZ1Y4HIbdbofb7YbH44HT6RTq1ufzwe/3w+12w+Vywev1Cq9gMIhAIAC/349QKIRwOCy8OI4Dx3Hg79EQiUQAINR7dN0qFArIZDJotVoYDAYYDAbo9Xro9Xrh75ycHBgMBmE96cblcsFqtcLj8Qgvr9cLl8sFl8sl1C//N1+nfr8fgUAAoVAIwWAwZh8XiUTCvi2Xy6FSqaDT6YRXdP0ZjUYYjUbhb5PJ1CX250AggJMnT8Jms8FqtaKmpkbYf/1+v7CvBgIBYZ/m99VIJAKO43DWWWfh8ccfb3b9aZkQbrvtNuzZswcqlQpGoxFms1kIcCqVClqtFiaTSdg5zGYzzGYzNBoNpNK22WSO4+Dz+eByueB0OuH1euF0OuF0OuF2u1FTU4OamhpUV1ejoaFB+Mxms6Gqqgp+v7/F9YtEImHH53d+jUYDlUoFhUIBiUQCiUQCkUgEkUgExhgikQgCgQDC4bCQSMLhMEKhkJBU7HY7OI5LaBuVSiWMRiMsFgu0Wi00Gg3MZjOysrKEH1pOTg4sFgs0Go3wg+R/iCqVqs0DWjAYRF1dHaxWqxBMGhoa0NDQIAQWt9sNm80Gp9MJh8MBl8slBCW32436+vqE6wAAVCoVVCoV5HI5FAoFlEqlkGz5l1gsFl5AY9Ln95Gamhoh0Xi9XiH4BYPBFr9XLpcjJycH2dnZyMnJQbdu3ZCbm4vc3Fyo1WoYjUZkZWXBZDIhKysLRqMRWq1WKENrMcYQCASEgxE+qPMHM1VVVaiurhb+ra6uhtVqFf4vEqFQKKDVaqFSqSCVSqFUKoWEKZfLhX0cACKRCPx+v3Cg5Pf7hd+fz+eL+11qtRparRY6nU6oU4vFArPZDLVajezsbGRlZQn7usFggMlkEpJLW9QrYwzBYBBerxdutxtOpxN1dXWw2WzCe36b+IPEqqoq1NXVoba2FnV1dS2uXyKRQK1WQ6FQCPEiel+VSCTwer2nXV6Ujred3nbbbdi6dSv8fj+sVivsdjtcLleTI+bmyGQyKBQKyOVyqNVq4ehNoVAIFSYWi8FxHCKRiPDDDYVCQkDhf9TxSCQS5OTkICcnR0hYRqMReXl56NatG7KysoSjdIPBALPZDJPJBL1eD6lUekaODjmOE47U7HY7PB4P7HY7HA4H/H4//H6/0GLhj/KsVqtwNN3Q0ACr1Qqn04lAIBB3+zUajZDQ+B8932IRi8VCYuN/bJFIBJFIREhqfJmCwSDcbndCgYYPlvzRt06ng1qthkajgU6nE/5PNBqNMI3/8fAvPnAolco2C7CnCoVCcDqdsNvtQiBwOBxwOByoqalBbW0tamtrUV9fLwTd2tpahEKh065TJBIJyZgPqjKZTNjH+QArFoshEomElk0wGITP5xMCFX90GS88iMVi5OTkID8/H3l5ecjKyoLZbEZ+fj4sFotQ7xqNBmq1WmidarVaaLXaNjtqj0QiMQcAdrtdqFe73Q6bzSbECZfLJdRrXV0d7HZ7i0Eyul41Go1Qr3wc4QOuRCIRysLvw4FAAIFAAD6fT2iVJhJypVKpEC9yc3OFui0oKEBBQYFwIJCbmwuDwSDEMZlM1qq4kZYJoTmMMXi9Xvh8PuEI0eFwwOl0or6+HjabTTjC4U/H8M0rvpnKN6sYY8Lpl+gfFb8T80frarVaaK7yR8h6vR5arRbZ2dmwWCxp2+RPhNfrRW1trVC3fDCLDnBut1sINvyRMf/iky5f5wCEJMGfGuBPtcjlcmi1WpjNZuFIjg8sJpMJ2dnZ0Gg0ZzSAdwYcxwmnCPjTBnwLKbr++VMF/MEMv4/zdc2/+OSgUChikiG/f/P7Ov+e388tFouQWLtCfXMch/r6eqF1E30a1263CweeHo9H2H/5AxW+Jc63OqP3YYVCAYVCIRykaLVaKJVKIXbwdWk2m6HVaoWEeSZa17z8/HyUlpZi+fLlTT5L24QwYMAATJo0CStXruzoohBCSNro06cPxo8fj9dff73JZ2mb2uVyOaxWa0cXgxBC0oparT7tNZe0TQgqlSqhC0mEEEJ+0VLsTNuEIJfL417UJIQQEqul2Jm2CYG/E4gQQkjiWoqdaZ0Q0vR6OCGEdJiWYmfaJgT+lrlkcRzDkXrPGSgRIYR0fi3FzrRNCBzHpZQQ3tpWiclPbMbi9/aegVIRQkjn1lLsTNuEkGoL4eP9NQCAl7860sYlIoSQzq9LthAikYjwqHgyRvY0CX/7Q/G7uiCEkK6kpdiZtgkhEAhAoVAkvZxa/ktFuPzhtiwSIYR0ei3FzrRNCH6/H0qlMunllLLohHD6TsIIIaQrail2pm1CCIVCKfWUqJb/0v21k1oIhJAM01LsTNuEEAwGIZfLk15Op/wlIXgDlBAIIZmlpdiZtgkh1RaCNioheIJ0UZkQklm6ZAvB5/NBpVIlvZxe+UtFeIPUQiCEZJaWYmdaJgR+kBCj0Zj0stGnjDwBaiEQQjJHvNiZlgmBH4bOYDAkvaxS+stdRk66y4gQkkHixc60TAh2ux0AUksI8l822UvXEAghGSRe7EzLhFBfXw8AsFgsSS8rl/yyyXZvsM3KRAghnV282JmWCcFmswFILSFE9+FxwkYjrhFCMke82JmWCYHPcmazuXXrcdOIa4SQzBEvdqZlQuDPg5lMppZnjKPBQ6eMCCGZI17sTMuE4PV6AQAajaZV62lwU0IghGSOeLEzLRNCTU0NZDIZ9Hp9SsvzzyL4qPtrQkgGiRc70zYh5OTkQCxOrfg5uuS7zSaEkHQXL3ZKm516Bnm9Xrz00kvYv38/evfujdtuuy3pPomqqqqQl5eXchnMGjkO1dG4yoSQzBIvdrZrQrDZbBg7diw4jsOYMWOwfPlyfP3111i9enVSPZfW1taioKAg5XJoFO2eBwkhpMPFi53tGhlffPFFeDwe/PTTT1CpVDhy5Ah69+6Nb775BpMmTUp4PXV1dRg2bFjK5YgeNY0QQjJFvNjZrgnhwIED6N69uzBaj1arBQBYrdYm8+7fvx/l5eUQi8VQKBQwGo3IyspC3759UVtbi5ycnJTLoZJRC4EQklkYY3FjZ7tGxquuugovvvgi5s+fj/PPPx/PPPMMevfujcmTJzeZd82aNXjooYdipl1++eV46aWXEAwGW5UQlLK0vJZOCCEpczgccWNnuyaEqVOnYvLkyXjttdfw+uuvgzGGhx9+uNmuWCORpreE5uTkoLa2FgCQm5ubcjnkUkoIhJDMkkjsbNeE8Morr2DLli1YuXIlrrzySixfvhz33Xcf1Go1Fi5cGDNvUVERSktLEYlEEAgEYLPZ0LdvXzidTgCp9XTKk0T1Z0QIIZkgkdjZrgnhiSeewO23344bb7wRAHDvvffi+++/x+uvv94kIZSVlaGsrKzJOj755BMArUwIYkoIhJDM4nA4ALQcO9v13MnJkyfRvXv3mGnZ2dlCQRPBZzmdTtemZSOEkK4skdjZrgnhvPPOw9KlS7Fjxw64XC5s2rQJb7/9NmbOnJnwOviNSrXbCgAIcyzlZQkhJB0lEjvbNSEsW7YMubm5GDFiBPR6PaZNm4aSkhIsXrw44XXwrYlUxlPm+akPI0JIhkkkdrbrNYQ+ffrgs88+Q319Perr69G9e/ekeyzlN6o1LQRPIJzysoQQko4SiZ0d8oRWVlYWsrKyUlrW7XZDLpcn3f9RNOrllBCSaRKJnWl3Q34oFGpVMgAAT4ASAiEksyQSO9MuIQQCAaHri1S56JQRISTDJBI70y4heDweqNXqVq3DH6QWAiEksyQSO9MuIfj9/ta3EPyhNioNIYSkh0RiZ1omBJVK1ap11LkDbVQaQghJD4nEzrRLCF6vt9UJIRRpfDBNQ+MiEEIyRCKxM+0SQlvcZcTL0bfu1BMhhKSLLnmXEYDTDhCdrBydok3WQwgh6SBe7Ey7hMBY2/VDlGegFgIhJDMkEjvTLiG0pXxj665FEEJIV5J2CUEkEoHjuDZZV3dT655nIISQdJFI7Ey7hCAWi9ssIfTL0bbJegghpLNLJHZmXELgosZCyDfSNQRCSGbokglBKpUiHE69LyK775enlM0aeVsUiRBCOr1EYmfGJYTjVq/wt1reIb1/E0JIu6OE0IzyamcbloYQQtJDl0wIMpkMoVDqndMdbfDGn4kQQrqYRGJn2iUEpVIJv9+f8vIHa91tWBpCCEkPicTOtEsICoUCgUDqvZUes1ILgRCSeRKJnWmXEORyOYLBYMrL17mo62tCSOZJJHamXUJQq9Xw+XwpL9/gaawQmUTUVkUihJBOL5HYmbYJobVPKxdQP0aEkAySSOxMy4QAoFUXlgHqtoIQklkSiZ1plxB0Oh0AwOVytWo91NMpISSTJBI70y4haLWNR/Zud+tuH+1moIRACMkcicTOtEsISmVjh3StubAMUMd2hJDMkkjsTLuEwA8S3dqEQB3bEUIySSKxM6MSQvQQchoFdWxHCMkcXTIhaDQaAIDH40l62UjUWAg6SgiEkAySSOxMu4Sg1+sBpHaXUTgqIRjVdMqIEJI5EomdaZcQWtNCCEV+eSBDLZe0WZkIIaSz69QthFWrVuG2225Lejn+1qlUEoInEBH+poRACMkkicTODkkIr7/+Oq677jr07Nkz6WWNRiPEYjFqa2uTXtbu+6VjJ5GI+jIihGSORGJnuyeETz75BL/61a+wZMkS3HnnnUkvL5VKkZWVlVJCcPtTH2mNEELSWSKxs11vtWGM4Z577sGtt96Ke++9t8V59+/fj/LycojFYigUChiNRhgMBhQVFUGr1aZ0UdnmTX2kNUIISXfxYme7JoRPP/0Uu3btwjXXXIPrrrsOeXl5uO6663D22Wc3mXfNmjV46KGHYqaZTCZYrVZoNJqUriG4/JQQCCGZK17sbNdTRkuXLkUwGMTf//53HDt2DK+++ipGjhyJDz74oMm8kUikyTR+QzQaDbze5Ec+i77tlBBCMk282NmuLYTy8nLceOONWL58OaRSKfx+P6ZNm4ann34aJSUlMfMWFRWhtLQUkUgEgUAANpsNbrcboVAIOp0upVNGHCUEQkgGixc72zUh+P1+DBo0CFJp49cqlUpccsklePbZZ5vMW1ZWhrKysmbXYzAYUFlZeUbLSgghXU282Nmup4wKCwuxf//+mGknTpxAfn5+UuvR6/VwOBxJf79ETLeaEkIyV7zY2a4JYdasWfj3v/+N7du3AwB27NiBV199FbNnz05qPSaTCXa7Penvj04I0R3dEUJIJogXO9v1lNHvfvc7fPrppxg9ejRyc3NRXV2Na665BjfffHNS69FqtfB6veA4DmJx4jkt+hJCIMxBKaOnlQkhmSNe7GzXhKBWq7F+/Xrs3LkThw8fxtChQ9GvX7+k18MP9OD3+4VxQhMRfVHZ5Q9TQiCEZJR4sbND+oA+++yzm332IFHRfXIkkxAiUaeJAuGmt7USQkhXFi92pl1vpwBgsVgAAHV1dSmvwxukhEAIySzxYmdaJwSbzZbyOvwhSgiEkMwSL3amZULgmz1utzvldbgD1NEdISSzxIudaZkQdDodgORHTYt+CiEQ4k47HyGEdEXxYmdaJgSz2QwAqK+vT2o5cdRzCJ4gtRAIIZklXuxMy4SQnZ0NIPmLyjLJLwnBG6BrCISQzBIvdqZlQpDL5dBqtbBarUktp5D+8tyBzRtsYU5CCOl64sXOtEwIQOPFkWQvKquiHkSzeighEEIyT0uxM20TglwuRzCYXFBXyX9JCNVOf1sXiRBCOr2WYmdSCSEUCqU0DsGZoFQq4fcnF9Tl0l82t94daOsiEUJIp9dS7Ew4ITz22GPQaDRClxMnTpxAWVkZevXqhUWLFoHj2vc2zlQSgiyqM6cqB7UQCCGZp00SgkajQSgUwrvvvgsAmDdvHjZt2oSrrroKjz/+ONatW9cmhU1UKqeMpFF3GdW7qIVACMk8bXLK6IorrgAAHDx4EBUVFfj000/xwgsv4PHHH8fgwYPx1VdftU1pEySVShEOJ/csQfQpI6c/TB3cEUIyTkuxM+GEkJ+fj7Fjx+LPf/4z3nrrLUilUpx//vkAAJlMhkCgfY+4JRIJIpHkAvqp3V27/fRwGiEks7QUO5O6qPzoo4+ioqICd999N2bOnAmdToejR49i3759GD58eJsUNlESiSTp6xZGlSzmfY2TThsRQjJLS7EzqYQwefJk/Pjjj9i0aRNef/11AI2nkC644ALMmTOn9SU9w9Ty2BbCs5sPdlBJCCGk80k4IQSDQezatQtVVVWYPHkyVCoVOI7DqFGjsH79emEknvbCcRxEIlH8GaOcOv+GXVVtWSRCCOn0WoqdCSeE++67D8OGDcM111wDANi2bRuGDh0Ko9GIkpIS+Hy+tiltgiKRCCQSGgKTEEKS0VLsTDgh8M8fbN68GYwxzJ8/H3K5HK+99ho+/fRTrF69uk0KmyhKCIQQkrw2SQiXXnoppFIpNmzYgN27d2Pv3r1YunQprrvuOgwZMgQ7duxoswInguM4iMVp2/MGIYR0iJZiZ8IR1WAwYPr06XjooYfw7LPPQq1WY8yYMQCAQCAAqVTaNqVNUCgUgkwmiz/jKaYX58a8Z4y1VZEIIaTTayl2JnWI/dRTT8FgMOCFF17AggULoFAosG3bNpSXl2PChAltUthEpZoQuhlUMe8DYRo5jRCSOVqKnUkd1g8YMAA//PADnE4nTCZT4wqkUtx7770oLS1tfUmTEA6HU0oIZo08dj0ctRAIIZmjpdiZ9HkesViMqqoqbNmyBWazGUOHDsWSJUtaXchk+Xy+lG511ShiNzkY5gBFW5WKEEI6t5ZiZ1KnjCoqKjBixAgUFxfj0ksvxfjx49GzZ0+sXLmyTQqaDJ/PB5VKFX/GU+iUsQkhHKFTRoSQzNFS7EwqIdx+++2w2+348MMPhbER7rzzTvzmN7/Bd9991yaFTVQwGIRcLo8/4yl0p7QQInRRmRCSQVqKnUklhH379mHWrFm48MILIRKJoFarce+996J///5YtWpVmxQ2EYwxeDweaLXapJfN0cc2lSgfEEIyRbzYmVRCmDp1KlatWoUTJ04I06qrq1FTU4OCgoLWlTQJPp8PkUgEOp0u6WVPvahM+YAQkinixc6kEsIDDzwAhUKBvn37YsaMGZg9ezaGDBkCs9mMX//6121S4EQ4nU4AgF6vT3rZHF3sFWRJkv0hEUJIuooXO5NKCAUFBdizZw8ef/xx6PV6uN1u3H333di6dSuMRmOrC5sou90OACl9p1ougUT8SxKI/psQQrqyeLEz6dtONRoNbrnlFtxyyy0AGk8ZbdmyBTNmzEi5kMlyOBwAGp+eTpZIJIJKJoE70Dg4jlxC3V8QQjJDvNjZ6mi4ceNGXHrppTh58mRSy0UiEdx1110pnWrimz2pJAQA0Efdeho9zjIhhHRl8WJnqxPChAkTwBjD//73v6SWe/zxx/HEE09g3bp1SX+nx+MB0NhaSYUyaqAclYx6TCWEZIZ4sbPVCaFv377o3r07tm7dmvAyu3btwgMPPIDJkyen9J0NDQ0AIHSfkazoNoGYriEQQjJEvNjZ6i5KRSIRiouLceDAgYTmDwQCmDt3Li688EJcfPHF2Lt3b7Pz7d+/H+Xl5RCLxVAoFDAajVCr1SgqKkJtbS0AIDc3t9ll4xHTnUWEkAwUL3a2mBC8Xi82bNgAxhgkEgkkEgkUCgXUajU0Gg20Wi2ys7ORk5ODXbt2JVSgxYsXo6KiAu+99x42bNhw2vnWrFmDhx56qMn0b7/9Fna7HQqFIqWuKwB6OpkQkpnixc4WE8K2bdswe/bshMYMmDRpUtx5vv76azz22GNYsWIFevTo0eK8kUik2ek6nQ5OpzOlZxB44QglBEJI5okXO1tMCBMmTEAoFEI4HEYkEkEkEkEgEIDX64XX64Xb7YbD4YDH48HYsWNbLEhdXR2uvPJKqNVqHDlyBA8//DC2b98Or9eL559/HrNnz465N7aoqAilpaXCd9psNrjdbuTk5KC+vh5mszm5mojCUQuBEJKB4sXOuNcQ+FNFvFS6iwAaz10NHjwYbrcbGzduRCAQQF1dHbxeLx555BH06NEDF110kTB/WVkZysrKml2X1WqFxWJJqRwAwNEYCISQDBQvdrbbuJfFxcX4+OOPY6YtX74cDz74II4fP57UujweT6tOGVE+IIRkonixs0Mf0+W41MYicLvdKfV0ygvRGAiEkAwUL3Z2aEIoLCxM6VmChoaGVl1D8Ieav2BNCCFdWbzY2aEJobS09LTPIbTEbre3KiF4gpQQCCGZJ17s7PCe3aTS5C5jhEIh+P3+lC9uR6OO7QghmSKR2Jl2EbE1PZ2eSiWnfowIIZkhkdiZdgmhtR3bRVNTQiCEZIhEYmfaJQS/3w8AUCqVceaM79ThNAkhpKtKJHZmdELI1bd+HYQQkg66ZEJoy2sI2VpF/JkIIaQL6JLXEFoznvKpcg3UQiCEZIZEYmfaJYTWXlSORPVboVO0W88dhBDSobrkRWW+2ZNqC8HuDQp/ayghEEIyRCKxM+0SgsvlApB6r6tWzy8JQS5Nu80nhJCUJBI70y4iOp1OiMViqNXqlJb3RfVjRAmBEJIpEomdaRcRrVYrjEYjxOLUiu6L6sdIQQmBEJIhEomdaRcRvV5vyq0DAPBGJQR6UpkQkikSiZ1plxBCoRBkMlnKyzt8IeFvuqhMCMkUicTOjEsIrkBY+NtCXVcQQjJEl0wI4XA46S6zozmjWgg6ZeqJhRBC0kkisTPtEkJrWwixCYFOGRFCMkOXbCEEg0HI5amf6ql1BYS/ZTRADiEkQyQSO9MuIrb2lFH0RWVCCMkUXfKUUSQSgUSS+u2ibv8vF5Xvf3cPXH5KEISQri+R2Jl2CYExlvJDaQBgjerL6LUtR/HNoYa2KBYhhHRqicTOtEsIACASiVJe9mCtO+Z9dO+nhBDSlcWLnWmZEBhruyAeCHNtti5CCOnM4sXOjE8I0V1ZEEJIV9blEoJEIkEk0nZBPMxRC4EQ0vUlEjvTLiFIpdI2TQh0DYEQkgkSiZ1plxDkcjkCgUD8GU+jT1bs8HF0yogQkgkSiZ1plxBUKhV8Pl/Ky4/pbY55v/WItbVFIoSQTi+R2Jl2CUGj0QiDRadiXF9LzPu+2drWFokQQjq9RGJn2iUEtVrdqhbC1KLcmPcKWdpVASGEJC2R2Jl20VAmkyEYDMaf8TRCpzx3YFBRF9iEkK4vkdiZdglBLpe3KiH4w7EXkdVy6gKbENL1JRI72z0aMsbw4Ycf4pNPPoHBYEBZWRn69u2b8PL8RjHGUurCInhKC0FLw2gSQjJAIrGzXVsIbrcbV1xxBS6++GJ8/vnneO6551BUVIRNmzYlvA6FQgHGGMLhcPyZm3FqQlDJU+85lRBC0kUisbNdE8KKFSuwfft2fPvtt/j2229RUVGBIUOG4Kmnnkp4HTqdDgDgdDpTKsOpfRfRuMqEkEyQSOxs1/MlCxcuxK233iqM2sNnLKVS2WTe/fv3o7y8HGKxGAqFAkajEQMGDIDF0njbqM1mE/5ORigSmxBMlBAIIRkgkdjZrglBJBIJyYAxhqeffho7d+7EH//4xybzrlmzBg899FDMtCeeeAIDBw4EAFitqT1Qxp3SuZOchtEkhGQAk8kEoOXY2SFXVB0OB2666SasWrUK99xzD2bNmtVknub63LDb7TAYDMI6UnFq10X+EHVdQQjp+hKJne2eEH744QdcdtllCAaD2LBhAy6++OJm5ysqKkJpaSkikQgCgQBsNhs0Gg00msa+iFrztHI06tuOEJIJEomd7ZoQbDYbpk6dirPOOgtvv/220IRpTllZGcrKyppMP3ToEIDUWwin3mxV7w5gIHQprYsQQtJFIi2Edj2BvnbtWrjdbqxevbrFZNAS/mJIfX19SsuLxbEpwe4NpbQeQghJJ4nEznZtIZSXl8NkMuFf//oX6uvr4Xa7IZFIcOWVV2LixIkJrcNgMECpVKKqqiqlMohPeSDDG0zteQZCCEknicTOdk0Iw4cPx6uvvoqVK1fCaDRCq9XC5/OhsrIy4YQgEonQrVs3VFdXp1QGySkJwUcXlQkhGSCR2NmuCeF01wWSZTKZYLfbU1pWcsopI7rLiBCSKeLFzrS8CV+v16d8UfnUrircAUoIhJDMEC92pm1CcLlcKS3bw6yOec8Y3XdKCMkM8WJnWiYEi8WC2tralJaViEV46uphwvuZw/LbqliEENKpxYudaZkQ8vLyUFtbm/LRPb/Y5cML0D+XnkEghGSGeLEzLRNCbm4uIpEIGhoaUlqe7+BOJkl+PAVCCElX8WJn2iYEAKirq0tpef7OU7p8QAjJJPFiZ1omBK1WC6BxwJ1UiH7uwILyASEkk8SLnWmZEPR6PYDUB8nhWwi7Ku24+61d+OxAai0NQghJJ/FiZ1oOKNzahMCPmnagxo0DNW6cdPgwaUB2m5WPEEI6o3ixMy1bCGp147MEqXaBXV4dWxnhCJ08IoR0ffFiZ1omBD7LpfpwmlQcu9kaheQ0cxJCSNcRL3amZULgB4tONSHoVbKY9zIaRpMQkgHixc60jIQqlQoA4PV6U1pefUp/RnJpWlYDIYQkJV7sTMtIKBaLoVQqU76GoDklIXz5U2qD7RBCSDqJFzvTMiEAjRdHfD5fSsv2ytLEvG/wBNuiSIQQ0um1FDvTNiFotdqUH0w7r19Wk2m1Tn9ri0QIIZ1eS7EzbROCRqNJ/UllkQh5emXMtCc/OtAWxSKEkE6tpdiZtglBJpMhFAqlvLxWGftM3sG61JILIYSkk5ZiZ9omBLlcjmAw9XP/ZrU85v22o7bWFokQQjq9lmJn2iaE1rYQLFp5k2lLNuxrTZEIIaTT65ItBIlEgkgk9fGQh/cwNpm28ovDrSgRIYR0fi3FzrRNCGKxuFXjId94Xp9mp//utW2weYJYu6MSDe5AyusnhJDOqKXYmZa9nQIAx3GQSlMvvlgswt+uPAuL3toVM/2DvdX4YG+18H7fw9OhlqdtNRFCSIyWYmfathAikQgkktZ1SjeujyXuPIMf+BC97tmAdTtPtKpFQgghnUFLsTNtE0I4HG5VCwEAupvV+Pa+qQnNe9vqneh97/v4/ogVb22rxN1v7cKYJR/j+yNWAIA3GAYAMMbw3WErjtR7cLTBg4ue+QKPrN8HdyDcqrISQkhbaCl2pu25kEAgAIVC0er15OqVmHNOD6z69lhC81+14pvTvr9+fC98e9iK/VVODMzV4dcT+2B/lRP7q5zYuKca/735XFi0rS8zIYSkqqXYmbYJwe/3Q6lUxp8xAUXd9G2ynle+PiL8/WONC3/4zw/C+xN2H0Y+8jEeLi3GG1uO4eHSYpzTxwKrJ4g1W4/D7g1h4YUD0OAOYvnmgzir0Ah3IIyyMT0gl4rhDYax54QTEY5hXN/mT3UxxrDpx1rsOeFErl6Bi4d2g04pa3ZeQkhmail2pm1C8Hq9wug/rTWtKAf3v9smq4rrgXV7AQCzX9jS5DOOMbzy9REEwxyAowCAB/+7t8l8D146GCN7mhAMcyjqpodGIYXVE8RNb2zDlgqrMN/db+/GlEE5+Pvss2FQnT4xuPwhzH5+C3pa1Fh+3Uhhenm1E099dAAKmQT/d15vdDepYFDJIG1m/AiHL4QGdwB9srUJ10VL/KEIlLL2G7ho53E7/rP1OO67uAgaRdr+LAiJq6XYmbZ7flsmhG4GFSr+cjE2/ViLt7dX4v3d1VDKxLjzggH4y/vlbfIdiXjh84qE5lv8XtMH6M7rlxWTDHifltdi2OKPMKF/FswaORaVDEK2VoGtR61ocAcxvTgPz246hH1VTuyrcuLOf+/EA5cOhlEtR8nTXwjree+Hk8Lfy8qGo8rhg80bQi+LGi98XoFDdY3d6T47ZwR+v2q7MO/hRy+GSCRqUi5/KIIqhx9vb6vEkAIDuptV+P6wFe/tqsKBahdcgTBuPr8f7rhgAOzeIMwaecx6Km1eqOVSeAJhOP0hPLvpIEIRhhfmjmz2+6IxxmLmCUU4XPbsVwCAET1MmDWysMXlASDCMaz69ii0SikuHx5//kTw6xyYp8eY3uY2WSchp2opdopYmt46o9PpcOONN+Lvf/97m6+71uWHNxBBrywN6lwBPPjfPXh/d3X8BUkTMokIoZ/HrH57wTj0y9Hh4301WBh1Oi0RcqkYb/9uPI5aPbh51Y7Tzvf9H6chSyvHR/tqYPME4QlG8Of1jQn0lin9cLDWjfJqF26b2h8WrRxF3fRY/d0xPPFz54bPXHM2Ss8ugN0bhFYhxeYf62DzBtE/V4dBeTowBvzhrR+wYVdVs99fNqYHrh/fC0cbPJg0MBsKaWMr5+1tlfi0vBY3nd8Xf/jPLmRp5fhz6RAUmFSwe0PI1ilQ9sIWfFPRAABYVDIQbn8YI3uaMLUoF0DjjQsqmSQmmXEcQ0W9B32zNXETYbQ6VwARjiHPEP+06zMf/wSnP4T7LxkcM93hDcGg7rhTku5AGAdr3Ti7u7HDynCmeIPhZm93Z4whEOagkIqT+v+O1lLsTNuEIJfLceedd+Kvf/1ru33n1Su+wXdHmh6Fk8zQzaBElSPxbtL/NKMI+0468c6OEy3OJ5OI8NL1ozH3n981+/nt0/rj+yNWfHWwQZj2x4uL4AqEsfSTn5rM/8aN52BsHwsO17vx1cEG7Dxux2cH6mD9edwPhVSMQJgT5n97wXictPtwXr8sKGRiRDiGH4478Na24/jyYAPqm3lAc9KAbHx2oA7PzhmBGWd1i/ksFOHw+paj6GXRoNLmRcmQbsjSyvHNoQas23kSP9W60N2sxjPXDMfBWhfe/O44/u+83ti4pxo2TxALLxwQE+wO1LiwfPMhDO6mx9g+FmTrFBCLgOWfHcLLXx3BrVP7484LBsScZuRbgYfrPXhneyWuHFmInhYNblu9AzVOP16cPxraOKcGGWP4eH8tGGP4tLwW3c1qXD++V6tOKTLGUOcK4I/v7oFRJcNfrhjaZAjfJz78Ef/YdBDn9cvCk1cPQ+7PPTN7AmEcqnNj5j++QnezCutvnpBSQm4pdqZlQgiHw5DJZFi8eDEeeOCBdvveg7UulK38Fmd3N+KOaQMwOF8Pxhg+3FuD372+rcn8OoUU6289D3/74EdcNaoQ4/pacOea0x9dEpKOrhndHQVGFS4bXoC3t1fi6Y+bJqlU9c/R4qfa+D0RX3Z2Pt7d2Xha867pA/H4hz/GfD5/XE/MHdcT0576PGb6pAHZsGjkEItFKM7Xo9rpx4tfHEaESywsjultxpNXDUN38y+nYCrq3Dja4MX/9tegdFg+xvQ240CNG796+TucbOaAYu7YnlArJNhd6YDLH8buE44m88glYgQjXMy09bechyEFhoTKyYsXO9s9IYTDYbz88sv49NNPkZeXhzvvvBPdu3dPah0OhwNGoxFPPPEEFi5ceIZKmrzdlQ7MWv41+udq8btJfXHRkLxmL8C6A2EMefDDmGlLLh+CCMeEi86nunVKP/zq3N6ocwdwx7934nC9BzOGdkOtK4DPDtQBAAZ300OjkGDvSSd+N6kvnvofjfFASHuQS8RQKySwe1PvcDNZux+6MOm7COPFzna9qOxyuTBlyhTs2bMHF110ET766COsWLECGzduxOTJkxNej9XaeNrGZDKdoZKmZmihAbseujDu+T2tQoollw/Bd4etuOHc3hgWdQ503rheiHAMf924H18dbMA7N42HWCSCXNqYWEwaOTbcOiGh8tx8fj8cs3qRZ1DiswN1+O1rTVsxADCqpwnzx/fCLW/uwOBuejx37Qgs/eSnJqc6+uVokadXoodFjQuKcnGozo1HNuxvsr7v7puKXZUO3LFmJ1z+M/dA3tyxPTGipxF3/Du56xGEtLVghEPQy8WfsQ2lckt5vNjZri2EO++8E2+88Qa+/PJL9O/fH5FIBNdeey0OHDiAbdu2JXyRZNu2bRg1ahTWrVuHmTNnnuFSdy2MMbgCYeiVMvhDEcgkYkjE8es9GOaEpHQqjmP4x6aDKDSpcMWIlu+48YciuPiZL2DRyqGQSnDNmO64aEg3SMQiRDgmlKXa4YdCKsamH2sxrq8F5dUu7D3hwMxhBehuVoGxxv6ogMZbXpds2IdgmMOXBxugkIpxwh47Zmx3swrHrc2PIzsoT4c3fz0WJo0cHMfw6jdH8O7Ok7hyZCEm9M/Cm98dx4rPDgFovC5g94agVUoxc1g+7nlnN7I0cgzO1+NwvQdvJPiA4+nIJCJM6J+NT8tr8XBpMfyhSNw73e69aBC8wQjO7ZeFq5//psV5mzOtKAcf769NtcikAzx51bCE7oY7VbzY2W4JgeM45OXl4a677sJdd90lTP/mm28wfvx47Nq1C0OHDhWm79+/H+Xl5RCLxVAoFDAajVCr1ejfvz+2bNmCKVOmYNOmTUm1LEjm4DgmJIxTNbgD8AQisGjlCV8g9ATCSV9MjHAMYhFg94bQ4AmgX44OQGNS3lflxH93nsTwHiaUDMkT+sk63UERYwz+EAepRIS9J50oztc3uRgZbdknP2H3CQeeu3YEjtt8yNMroZL/8lxHhGM42uCBRauIeUblUJ0bU5/8DABw8dA83DV9EPRKKY40eDEgV4uNe6pRaFRhfL8scBzDbf/eifd+OIk+WRrMH98L88f3Qvjnc92vfH0kpgV58/n98I9NBzGihxEPzSxGD7Maz246iCytAp8dqMPfrjwLj6zfL3QueVahAT0tGlx3Tg+8+d0xlAzJg1EtxzU/P8Pz+/P7orzKhW3HbAmdqrl+fC88NLMYAPBtRQP+t68Gh+s9uHxEAcb3zcLBWjc0CglUMgk8gQh+v2o7lDIxxvWxoLjAgGMNXmw9ao25vfuc3mZ8e7jlG02GFRoweWAOAmEOB2vd+OpgPXyh+F33v3fzefj7xwfQ06JuvAV7zQ/4/OfTw6e7nTueTZs2tRw7WTvZs2cPA8B27doVM91mszEAbO3atTHTH3roIQagyeuZZ55ha9euZQDY9u3b26v4hGSML3+qYweqnW2yrkiEY75guE3WxeO406+zzuVnoXCElVc52etbjrBAKNKm392S3ZV29p+tx5k/FGY/VjvZhl0nGcdxCS3r8AVZMNxY1kgksWVSES92tts1hIaGxlvmsrOzY6brdI1HTU6nM2b66QZwMBqNsNkah7vsbNcQCOkKzu2X1WbrEotFUIrb9olzkUh02qfYs37uK2xgng4D83Rt+r3xDCkwCHf9DMjVYUBu4t+vj7oecLqWbVuIFzvbLSHwBXA4HMjLyxOm84nAaDTGzF9UVITS0lJEIhEEAgHYbDa43W4YDAYcO9Z4nlarbZtuEgghJBO43Y238J4udrZbQsjPzwcAHD58GAMHDhSm79mzBwAwfPjwmPnLyspQVlbW7Lr+9re/AQBUKtWZKCohhHRJPl/jjRWni53tNh6CxWLByJEjsXbt2pjp77zzDrp164bCwsSvmDscDkgkkjbry4gQQjJBvNjZrs8h/Pa3v8VNN92EQYMGYebMmfj3v/+NpUuXYvHixUldMXe5XNDpdCn35UEIIZkoXuxs1+cQOI7D008/jfvvvx9erxdKpRK33norlixZktToZ/Pnz8fnn3+Ow4cPn8HSEkJI1xIKNd6eK5M1/1Bbh/RlZLPZcOzYMXTv3h1mc2rd/LbFmModhTEGh8OBhoYGOBwOeDweOBwO2Gw2NDQ0wOVyIRAIIBgMIhgMIhQKwev1wuPxwOfzIRgMIhwON7kTSyQSQSKRQCqVQi6XQyaTQSqVQiaTQSaTQa1Ww2w2Q6/XQ6fTwWAwQKPRwGg0wmAwQKlUQqlUQqPRwGAwnHanSXfhcBh2ux1utxsejwdOp1OoW5/PB7/fD7fbDZfLBa/XK7yCwSACgQD8fj9CoRDC4bDw4jgOHMc1eZ6Ar/foulUoFJDJZNBqtTAYDDAYDNDr9dDr9cLfOTk5MBgMadsKdrlcsFqt8Hg8wsvr9cLlcsHlcgn1y//N16nf70cgEEAoFEIwGIzZx0UikbBvy+VyqFQq6HQ64RVdf0ajEUajUfjbZDJ1if05EAjg5MmTsNlssFqtqKmpEfZfv98v7KuBQEDYp/l9NRKJgOM4nHXWWXj88cebXX+HjIdgMpladcvobbfdhj179kClUsFoNMJsNgsBTqVSQavVwmQyCTuH2WyG2WyGRqNp9TjMPI7j4PP54HK54HQ64fV64XQ64XQ64Xa7UVNTg5qaGlRXV6OhoUH4zGazoaqqCn5/y71mikQiYcfnd36NRgOVSgWFQgGJRAKJpLErZJFIBMaYcEdWOBwWEkk4HEYoFBKSit1uB8cl9oi9UqmE0WiExWKBVquFRqOB2WxGVlaW8EPLycmBxWKBRqMRfpD8D1GlUrV5QAsGg6irq4PVahWCSUNDAxoaGoTA4na7YbPZ4HQ64XA44HK5hKDkdrtRX1+fcB0AjRfgVCoV5HI5FAoFlEqlkGz5l1gsFl5AY9Ln95Gamhoh0Xi9XiH4BYPBFr9XLpcjJycH2dnZyMnJQbdu3ZCbm4vc3Fyo1WoYjUZkZWXBZDIhKysLRqMRWq1WKENrMcYQCASEgxE+qPMHM1VVVaiurhb+ra6uhtVqFf4vEqFQKKDVaqFSqSCVSqFUKoWEKZfLhX0caDwI9Pv9woGS3+8Xfn/8xdKWqNVqaLVa6HQ6oU4tFgvMZjPUajWys7ORlZUl7OsGgwEmk0lILm1Rr4wxBINBeL1euN1uOJ1O1NXVCXdROp1OYZv4g8SqqirU1dWhtrYWdXV1La6fvz6gUCiEeBG9r0okEni93tMun5a9nd52223YunUr/H4/rFYr7HY7XC7XaZ9diCaTyaBQKCCXy6FWq4WjN4VCIVSYWCwGx3GIRCLCDzcUCgkBhf9RxyORSJCTk4OcnBwhYRmNRuTl5aFbt27IysoSjtINBgPMZjNMJhP0ej2kUukZOTrkOE44UrPb7fB4PLDb7XA4HPD7/fD7/UKLhT/Ks1qtwtF0Q0MDrFYrnE4nAoGm3SKfuv0ajUZIaPyPnm+xiMViIbHxP7ZIJIJIJCIkNb5MwWAQbrc7oUDDB0v+6Fun00GtVkOj0UCn0wn/JxqNRpjG/3j4Fx84lEplmwXYU4VCITidTtjtdiEQOBwOOBwO1NTUoLa2FrW1taivrxeCbm1trdDsb45IJBKSMR9UZTKZsI/zAVYsbuxvi2/ZBINB+Hw+IVDxR5fxwoNYLEZOTg7y8/ORl5eHrKwsmM1m5Ofnw2KxCPWu0WigVquF1qlWq4VWq22zo/ZIJBJzAGC324V6tdvtsNlsQpxwuVxCvdbV1cFut7cYJKPrVaPRCPXKxxE+4PJnLKL34UAggEAgAJ/PJ7RKEwm5UqlUiBe5ublC3RYUFKCgoEA4EMjNzYXBYBDimEwma1XcSMuE0BzGGLxeL3w+n3CE6HA44HQ6UV9fD5vNJhzh8Kdj+OYV30zlm1Xs577UJRJJzI+K34n5o3W1Wi00V/kjZL1eD61Wi+zsbFgslk7X5A+Hw7jsssug1+thMpnw7LPPprwur9eL2tpaoW75YBYd4NxutxBs+CNj/sUnXb7OAQhJgj81wJ9qkcvl0Gq1MJvNwpEcH1hMJhOys7Oh0WgSCuAcx+Hdd98VTgktWLAg5TpobxzHCacI+NMGfAspuv75UwX8wQy/j/N1zb/45KBQKGKSIb9/8/s6/57fzy0Wi5BYz1TCbE8cx6G+vl5o3USfxrXb7cKBp8fjEfZf/kCFb4nzrc7ofVihUEChUAgHKVqtFkqlUogdfF2azWZotVohYZ6J1nUiukxCSFQ4HMbu3buFANOzZ8+OLlK7stlswnUbjUaTcNO+K6E6iJXO1+Naw263Y8yYMTAYDMjKysLGjRs7ukgdLv1Te5L27duHESNGYMiQISgpKeno4rS748ePC38nOw5FV0F1AOzatUu4/jRkyJCOLk6HOHbsGH766Sds3boVR44c6ejidAoZlxCizxVmYtcXmb79ANUBQHUAUB00J+MSgt1uF/7OxM7xMn37AaoDgOoAoDpoTsYlhOg7YxQKRQeWpGNk+vYDVAcA1QFAddCcjEsI0fefZ+KFtEzffoDqAKA6AKgOmtMhD6Z1pEGDBmHx4sWIRCIYNGhQRxen3WX69gNUBwDVAUB10JyMu+2UEEJI8zLulBEhhJDmUUIghBACIMMSgtVqxR133IFRo0ahtLQU3333XUcXqc3s2rULZWVlmDdvHmbPno1Zs2bh6quvxuWXXx7T79Lhw4cxf/58jBw5EvPmzcOhQ4di1sMYwxtvvIEJEyZg3LhxWL58eYt953QGbrcb//d//4dnnnkmZnooFMKyZcswduxYTJw4EatXr27Sqd3Bgwdx7bXXYuTIkbjhhhuaPKDEcRxeeeUVnHvuuRg/fjxWrlyJcDh8pjcpaUePHkVJSQk2bdokTFuzZg3Kyspw3XXX4eqrr8asWbMwa9Ys3H333THLfvHFF7joooswatQo/PGPf4TD4Yj53Ov1YvHixRg9ejQuvPBCfPTRR+2yTcn45ptv8Nvf/hZlZWVYsWJFk362ampqcNNNN2HkyJG48sor8cMPP8R8zhjDhg0bMHXqVIwePRqPPvpok/6N7HY77rrrLowaNQqXXHIJvv766zO+Xe0tYxLCsWPHUFxcjLfeegsXXHABwuEwzjnnHKxbt66ji9YmRCIRVq9eLYwRoVAoIBaLcc455wgdiH3zzTcYPHgwdu7ciZKSEpSXl2Po0KHYvXs3gMYfxdy5czF//nwMGDAAo0aNwqJFi3DNNdd02HbFc/ToUYwfPx4vvfRSTE+QoVAIF154Ie6++26MHj0a/fv3x7x583DXXXcJ82zevBnFxcUoLy9HSUkJdu7ciaFDh+LHH38E0FgfV199NX7961+juLgYZ599Nm6//XZcf/317b2ZLfr8888xatQofPjhhzHdcNTU1GD16tXwer0Qi8VCh2xjxowR5nn66acxceJESKVSTJs2Df/6178wduxYeDweAI0jbI0aNQrPPPMMJk+eDLPZjJKSEixbtqzdt7M5jDE88MADGD9+PPbt2wev14ubb74Zc+bMEeY5cOAAioqK8NFHH2H69Omw2WwYMWJETPK85557cMkllyA7OxuTJk3Ck08+ialTpwodZlZVVWHIkCF44403MG3aNIhEIpx77rn4z3/+0+7bfEaxDDF79mxWVFTEHA6HMO3Xv/41KyoqYhzHdWDJ2sbu3bsZALZr165mP+c4jhUXF7MZM2awYDDIGGMsEomw8847j1155ZWMMcY2btzIALD//e9/wnJfffUVA8C2bNly5jciBQsXLmSjR49m+fn57O677xamL1++nCmVSrZ7925h2htvvMGkUimrrKxkkUiE9evXj82aNYuFQiHGGGPhcJiNGjWKzZ07lzHG2DvvvMNEIhH7/PPPhXV8/PHHDAD74Ycf2mkL4zvvvPNYaWkpA8DefvttYfqyZcuY2Ww+7XKVlZVMLpezRx99VJhWX1/PdDod+8c//sEYY+zuu+9mOTk57MSJE8I8S5YsYWazmXk8njOwNck5efIkKygoYG+++aYwbdmyZQwAq6urY4wxNn36dDZ69OiY8l511VVs3LhxjDHGfvjhBwaAvfLKK8Lnhw4dYhKJRKjPefPmsX79+jGbzSbMc/PNN7N+/fqxcDh8JjexXWVEQvD5fEylUrF//etfMdO3bNnCAMQEjXT1ySefMABs1apVbM6cOWzSpEls4cKFwo9i3759DADbunVrzHL//Oc/mUQiYcFgkN1www1s8uTJTdY9aNAgtnDhwnbZjmRxHMc4jmN9+/Zl99xzjzB9ypQp7Fe/+lXMvIFAgOl0OrZs2TK2bds2BoDt3bs3Zp5ly5YxlUrFOI5jZWVl7OKLL27yfT179mR/+tOfztxGJSkSibD6+noGgK1du1aYfv/997Pi4mL27LPPsssvv5ydf/75bNmyZcIBwXPPPccsFgvz+Xwx65s7dy6bOHEiY4yxPn36sMWLF8d8Xl1dzQCw995778xuWIoefvhhJpPJmM/nY1arlYnFYvbuu+/GzPPBBx8wAOz48ePs/vvvZ/37929yYDht2jQ2e/ZsFgwGmU6nY88//3zM5zt27Gj2N5XOMuKU0Y4dO+Dz+TBx4sSY6X369AGALtGxVVVVFQBgzpw5cLlcKC4uxurVqzF9+nQwxvDVV19BrVZjxIgRMcv16dMHkUgElZWV+PLLLzFp0qQm6+7Tp0+nrSO+g7boHkz57T31/1sul6N79+44cuQIvvzyS2RlZWHw4MEx8/Tp0wc+nw+1tbX48ssvm6xDJBJ1uvoQi8Ww2WwAEDMCYVVVFfbu3YtFixZBp9OhV69euOuuu3DPPfcAAL766iucc845UCqVMevjt6+qqgoVFRVN6oAfoKcz1QFv8+bN+Nvf/oY5c+ZAqVRiy5Yt4DiuyX4d/dvn95VTu5vm62H37t1wuVwtrqOryIgH0/jzqkajMWa6Wq0GgIQGu+ns+PPnr776KubOnQsAWLBgAYYOHYovvvgCbrcbRqOxyU4fXQdutxsGg6HJutVqdUIjUnWUSCQCm82G3NxcAI3XDwKBQJP/b6BxW+JtK/BLfbS0js6koaEBAIQ6ABr3CZPJhM2bN+Oss84CAIwYMQKLFi3CI488Enf7Tve7iZ6nswiHw/jLX/6CxYsXY/r06cI4H/w26PX6mPmT+X/m13Hq/tKV4gcvIxKCxWIB0NgPfvR/vNVqjfk8nc2ePRvDhg3D+eefL0wbMmQIsrOzsXfvXlgsFuEoMlp0HWRlZZ12nsLCwjNX+Fay2+1gjAnBUC6XQ6fTnXZb4m0r0NjZWUt1xgfYzoIvd3RCePDBB6FSqWKewp0yZQp8Ph+OHDkCi8UitCxPXZfFYon53UTjOA52u73T/G4aGhowa9YsbN26FUuXLsWCBQuEQXuysrIANO4j0a2n6P2+pf9nfl8BGushLy+v2XV0FRlxyojv8768vDxm+o4dOwCgyWmUdNStW7eYZAA0/nD5Ift69OgBn8+HY8eOxcyzY8cOdO/eHTk5OejRo4dwh030Onbu3IlRo0ad8W1IFd96ie7CuEePHk3+v202Gw4fPoxRo0ahR48ewiDl0Xbs2IEBAwZAr9c3u45QKITdu3d3uvrgb5GMroPhw4c36ZLB6XQCgLBPlJeXNxnScceOHRg1apQwwtepdbBnzx6Ew+FOUwc33ngjfvrpJ2zbtg2///3vY0Zw69GjBwA02a937NgBhUKB4uLiZv+f+XlGjRqFgoICiMXiZtcBACNHjmzrTeo4HXkBoz2NGTOGzZs3T3jPcRy78sorWXFxcQeWqu04HA5WWVkZM+2dd95hAFh5eTkLBALMaDSyxx57TPg8GAyyoUOHsquvvpox1nhnjlarZXa7XZhn/fr1DAD7/vvv22dDUmCz2RgA9tFHHwnTFi1axPr06SNcQGWs8YKxWCxm1dXVzOv1Mo1Gw5YtWyZ87vf7Wf/+/dn8+fMZY4w9+eSTzGw2M7fbLcyzZs0aBoDt2bPnzG9YEviLpNF30R07doy5XK6Y+W699VaWn5/PwuEw+/777xkA9u233wqfHzlyhMnlcvbcc88xxhi7+uqr2cSJE2MuuP7hD39ger1euDurIzU0NDCJRNLkhhEex3Fs4MCB7NZbbxWmRSIRdsEFF7Bzzz2XMcbYf//7XyYSidiRI0eEebZv3x5zkX7ixIls9uzZMeu99tprWb9+/c7AVnWcjEkIq1evZgDYLbfcwt5//3127bXXMgDs1Vdf7eiitYklS5awrKwstnHjRlZRUcFeeuklptfrWUlJiTDPn/70J6ZUKtkjjzzC1q9fzyZOnMgkEokQEGw2G8vLy2MjR45ka9euZcuWLWNqtZpNmTKl096au3v3bvboo48yAOyqq64Sbpk9dOgQU6lU7MILL2Tr169nDz/8MJNIJOyGG24Qll24cCFTq9XsscceY+vXr2fjxo1jMpmM7dy5kzHGWG1tLbNYLGzcuHFs3bp17KmnnmIKhYLNmDGjQ7b1dD766CO2YMECBoDdcccdbN++fYwxxi644AJ2zjnnsG3btrF9+/ax++67j4lEIvbXv/6VMdYY1CZMmMAKCwvZyy+/zFavXs0KCwtZfn4+czqdjLHG245FIhErKytjGzduZLfffjsDwB5++OEO295o/C2j8+bNY/Pnz2eXX345mzFjBlu0aJFw99SKFSuYRCJhixYtYu+//z677LLLGAC2bt06xljj3WdFRUVswIABbPXq1ezll19mZrOZDR48WDigWLt2LQPAbrrpJvb++++z66+/ngFgK1eu7LBtPxMyJiFwHMfefvttVlhYyACwHj16xNx3nO7cbjebN28eE4vFDACTyWTsuuuuiznaD4fDbOnSpUyv1zMAbNiwYezDDz+MWU9FRQW76KKLhHX8/ve/Z/X19e29OQl7/PHH2YgRI9jw4cPZ8OHD2SOPPCJ8tnPnTjZ27FgGgGk0GvanP/0p5l70UCjEnnjiCabVahkANmrUKLZp06aY9R84cIBNmzaNAWAKhYLdfvvtMfeidwbXX399TB3wt1iWl5ezCRMmMAAMADObzezhhx9mkUhEWNZut7ObbrqJSSQSBoCVlpayH3/8MWb9mzdvZkVFRQwAy8rKYk8++WSnuffe4XCwKVOmsLFjx7KZM2eyOXPmsOuvv55NnDhReHaC4zj22muvsZycHAaA9evXj61ZsyZmPdXV1aysrIyJRCImFovZ3Llz2fHjx2PmWbduHevRowcDwAoLC9nKlSs77YFSqjKut1PGGHw+H1QqVZM7brqCkydP4sSJE+jdu7dwMexUfB3wd0k0JxAIQCqVdol+4n0+n/DkdnM4jkMgEIBKpTrtOvx+P2QyWdrVB2MMBw4cgNvtxuDBg0+7jeFwGJFIpMWBYrxeb1r/bhL57QeDQYhEIuHp/lTWkc4yLiEQQghpXkbcZUQIISQ+SgiEEEIAUEIghBDyM0oIhBBCAFBCIIQQ8rOM6MuIkGiMMVRVVcFut0On06Fbt26QSs/sT8HhcEClUkEul5/R7yGkNaiFQDLGihUrUFRUBJPJhIKCAqEfG4vFghUrVpzR7541axYeeOCBlJb95JNPmvSXc++992Lx4sVtUTRCBNRCIBnjxIkTOHLkCBYtWoRhw4bBbDbDZrNhx44dQt/2Z4pOp0u53/zVq1fHdFrHGMMrr7yC+++/v41KR0gjSggkY1gsFiiVyiZH1pdffvkZ/26RSHTaJ6Wb43K5cPjwYTDG8Omnn2LKlCnCwPAVFRWorq5GQUEBKioqzngyI5mDEgLJGIFAoMWuGRhjuOWWW3D77bdDq9WioqICZ599dpMuPmpra7F7924MGTIkZvwBoLHrg9dffx2MMZSUlKCgoED4br4//oMHD8LtdmPo0KGn7QrjjjvuwD//+U/hfUVFBV588cWYeS677DKYTCbU1NSctqsFQpLSER0oEdIR7rvvPta3b19WVVXFduzYwd5//3320ksvseeff56Fw2FhXOJx48YJnQT26NEjpqvrv/zlL0ylUjEATKvVCj1mMtbYUdzIkSOZTqdj3bp1YwqFgm3evJkxxthFF13EbrjhBlZWViZ0Nsd3s90cjuNYMBhkS5cuZYWFhSwQCLBgMMiCwSAbN24c+8Mf/sACgQALBAJnrL5I5qGEQDLGvHnzhGAc/crOzmYNDQ0sEAgwACw/P5+99dZbbPfu3ay4uJjNmTOHMdbYzTQA9uCDD7IDBw6wqVOnsr59+wo9Xt58882sd+/erL6+nnEcx/7zn/+wiooKxhhjJSUlTCaTsfz8fLZu3Tr2yCOPMLlcHre3zIsvvpgtWLBAeG+325lEImEffPDBGaolkskoIZCMMWfOHDZkyBC2du1a9vXXX7NDhw7FDH7DGGNKpZK98MILwvvHHnuMFRYWMsYYu+KKK9h5550nfLZlyxYGgB09epRZrVamVCrZW2+91ex3T58+nYlEIvbdd98xxhj75ptvGIBmu9KORCLs4MGDbO/evUypVLLly5ezAwcOsAMHDrAXXniBiUQitn37dnbo0KGYrqwJaS26hkAyhtfrRd++fXHZZZeddh6NRiMMyQkA2dnZwti5Bw8eRElJifBZr169ADQOS3no0CEwxlBaWtrsehljuOSSSzB69GgAjd1NA2h2gPf3338fl156qfB+wYIFTdbHD/u6bt06zJw587TbQ0gy6DkEkjFEIhE4jmtxHoVCIQRroHEsXv7CsF6vR11dnfDZ/v37IRKJ0KtXL/h8vhbXH4lEkJ2dLbzX6/UA0GRMZwCYMWMGTpw4gdLSUsyYMQOVlZWorKzE8ePHYbFYcM8996CyshInTpyISRyEtBYlBJIxevTogcrKypigzRhDIBAQ3ovFYqxbtw4vvfQSlixZgmeeeQbXX389AGD27Nl444038Oqrr+LDDz/EggULcM0110Cr1eL888+HWq3Gb37zG1RUVGDv3r248cYbsWrVKgCNg/RoNJqYsgDA0aNHm5RTJBIhJycHn3/+Oa644goUFBSgoKAAVqsVDQ0NwrT8/PwuOUgL6UAdfMqKkHbDXxRWq9XMZDIxrVbLZDIZMxgMwpCQhYWFzGg0MpFIxNRqNbv99tuFO3mCwaAwli4ANmnSJFZdXS2s/7PPPmO9evUSPh85cqRwh1JJSQl78803Y8rTp08f9sQTTzRb1rq6OjZgwICY4Us/+ugjNmzYsE4zfCXpemjENJJRfvzxR3z99deor68HYwwKhQKDBg3C9OnTAQCFhYVYuHAhbrnlFojF4mYfJqupqYHD4UD//v2bHKGHw2EcOnQIYrEY/fr1a/EIvrKyEmazucWhTAlpT3RRmWSUgQMHYuDAgaf9nOM4iESiFju7y83NbfJAGk8qlba4/miFhYUJzUdIe6FrCIRE4RMCIZmIEgIhUUpLS4VbOgnJNHQNgRBCCABqIRBCCPkZJQRCCCEAKCEQQgj5GSUEQgghACghEEII+RklBEIIIQCA/wfme/oXIDiWmAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xkcd()\n",
    "plt.xlabel('Epoch #')\n",
    "plt.ylabel('Loss')\n",
    "plt.plot(losses)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 保存模型 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save(cnn.state_dict(), \"fm-cnn3.pth\")\n",
    "# 加载用这个\n",
    "#cnn.load_state_dict(torch.load(\"fm-cnn3.pth\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型评估\n",
    "\n",
    "模型评估就是使用测试集对模型进行的评估，应该是添加到训练中进行了，这里为了方便说明直接在训练完成后评估了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率: 90.0000 %\n"
     ]
    }
   ],
   "source": [
    "cnn.eval()\n",
    "correct = 0\n",
    "total = 0\n",
    "for images, labels in test_loader:\n",
    "    images = images.float().to(DEVICE)\n",
    "    outputs = cnn(images).cpu()\n",
    "    _, predicted = torch.max(outputs.data, 1)\n",
    "    total += labels.size(0)\n",
    "    correct += (predicted == labels).sum()\n",
    "print('准确率: %.4f %%' % (100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型评估的步骤如下：\n",
    "1. 将网络的模式改为eval。\n",
    "2. 将图片输入到网络中得到输出。\n",
    "3. 通过取出one-hot输出的最大值来得到输出的 标签。\n",
    "4. 统计正确的预测值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 进一步优化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch : 1/20, Iter : 100/234,  Loss: 0.0096\n",
      "Epoch : 1/20, Iter : 200/234,  Loss: 0.0124\n",
      "Epoch : 2/20, Iter : 100/234,  Loss: 0.0031\n",
      "Epoch : 2/20, Iter : 200/234,  Loss: 0.0020\n",
      "Epoch : 3/20, Iter : 100/234,  Loss: 0.0013\n",
      "Epoch : 3/20, Iter : 200/234,  Loss: 0.0041\n",
      "Epoch : 4/20, Iter : 100/234,  Loss: 0.0016\n",
      "Epoch : 4/20, Iter : 200/234,  Loss: 0.0023\n",
      "Epoch : 5/20, Iter : 100/234,  Loss: 0.0010\n",
      "Epoch : 5/20, Iter : 200/234,  Loss: 0.0008\n",
      "Epoch : 6/20, Iter : 100/234,  Loss: 0.0017\n",
      "Epoch : 6/20, Iter : 200/234,  Loss: 0.0010\n",
      "Epoch : 7/20, Iter : 100/234,  Loss: 0.0009\n",
      "Epoch : 7/20, Iter : 200/234,  Loss: 0.0009\n",
      "Epoch : 8/20, Iter : 100/234,  Loss: 0.0005\n",
      "Epoch : 8/20, Iter : 200/234,  Loss: 0.0008\n",
      "Epoch : 9/20, Iter : 100/234,  Loss: 0.0005\n",
      "Epoch : 9/20, Iter : 200/234,  Loss: 0.0006\n",
      "Epoch : 10/20, Iter : 100/234,  Loss: 0.0016\n",
      "Epoch : 10/20, Iter : 200/234,  Loss: 0.0011\n",
      "Epoch : 11/20, Iter : 100/234,  Loss: 0.0003\n",
      "Epoch : 11/20, Iter : 200/234,  Loss: 0.0009\n",
      "Epoch : 12/20, Iter : 100/234,  Loss: 0.0010\n",
      "Epoch : 12/20, Iter : 200/234,  Loss: 0.0002\n",
      "Epoch : 13/20, Iter : 100/234,  Loss: 0.0004\n",
      "Epoch : 13/20, Iter : 200/234,  Loss: 0.0005\n",
      "Epoch : 14/20, Iter : 100/234,  Loss: 0.0003\n",
      "Epoch : 14/20, Iter : 200/234,  Loss: 0.0004\n",
      "Epoch : 15/20, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 15/20, Iter : 200/234,  Loss: 0.0005\n",
      "Epoch : 16/20, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 16/20, Iter : 200/234,  Loss: 0.0007\n",
      "Epoch : 17/20, Iter : 100/234,  Loss: 0.0003\n",
      "Epoch : 17/20, Iter : 200/234,  Loss: 0.0002\n",
      "Epoch : 18/20, Iter : 100/234,  Loss: 0.0004\n",
      "Epoch : 18/20, Iter : 200/234,  Loss: 0.0001\n",
      "Epoch : 19/20, Iter : 100/234,  Loss: 0.0003\n",
      "Epoch : 19/20, Iter : 200/234,  Loss: 0.0005\n",
      "Epoch : 20/20, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 20/20, Iter : 200/234,  Loss: 0.0002\n",
      "Wall time: 2min 21s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "#修改学习率和批次\n",
    "cnn.train()\n",
    "LEARNING_RATE=LEARNING_RATE / 10\n",
    "TOTAL_EPOCHS=20\n",
    "optimizer = torch.optim.Adam(cnn.parameters(), lr=0.001)\n",
    "losses = [];\n",
    "for epoch in range(TOTAL_EPOCHS):\n",
    "    for i, (images, labels) in enumerate(train_loader):\n",
    "        images = images.float().to(DEVICE)\n",
    "        labels = labels.to(DEVICE)\n",
    "        #清零\n",
    "        optimizer.zero_grad()\n",
    "        outputs = cnn(images)\n",
    "        #计算损失函数\n",
    "        #损失函数直接放到CPU中，因为还有其他的计算\n",
    "        loss = criterion(outputs, labels).cpu()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        losses.append(loss.data.item());\n",
    "        if (i+1) % 100 == 0:\n",
    "            print ('Epoch : %d/%d, Iter : %d/%d,  Loss: %.4f'%(epoch+1, TOTAL_EPOCHS, i+1, len(train_dataset)//BATCH_SIZE, loss.data.item()))\n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可视化一下损失"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEVCAYAAADdFfNTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hUZdo/8O/0mmnpIXRBAlJUuiAquGJBfMFC7LBrYRdldS3oiou4C/jD1V2wvApYWQv6Sl8RUIqKkSY9oSQQWpJJMr238/tjck5myJTMTAph7s915WI4OXPmmZOZ+z5POc/DYxiGASGEENLC+O1dAEIIIZcmSjCEEEJaBSUYQgghrYISDCGEkFYhbO8CkJbBMAzMZjPq6+thNptht9thNpthNBpRX18Pq9UKt9sNj8cDj8cDr9cLh8MBu90Op9MJj8cDn88Hv98fdlwejweBQAChUAixWAyRSAShUAiRSASRSAS5XA6dTgeVSoWMjAyo1WooFApoNBqo1WpIpVJIpVIoFAqo1WqIRKJ2OkOty+fzwWQywWazwW63w2KxcOfW6XTC5XLBZrPBarXC4XBwPx6PB263Gy6XC16vFz6fj/sJBAIIBAJgx+HweDwA4M576LmVSCQQiURQKpVQq9VQq9VQqVRQqVTc45ycHKjVau44HY3VaoXBYIDdbud+HA4HrFYrrFYrd37Zx+w5dblccLvd8Hq98Hg8YZ9xHo/HfbbFYjFkMhkyMjK4n9Dzp9FooNFouMdarfaS+Dy73W6cP38eRqMRBoMBNTU13OfX5XJxn1W32819ptnPqt/vRyAQwIABA7Bw4cImx6YE02DmzJk4dOgQZDIZNBoNdDodFzBlMhmUSiW0Wi33YdPpdNDpdFAoFBAKW+Y0BgIBOJ1OWK1WWCwWOBwOWCwWWCwW2Gw21NTUoKamBtXV1aivr+d+ZzQaUVVVBZfLFfP4PB6P+yKxXyaFQgGZTAaJRAKBQACBQAAejwcejweGYeD3++F2u+Hz+bjE5PP54PV6uSRlMpkQCASa9R6lUik0Gg0yMzOhVCqhUCig0+mQlZXFfXFzcnKQmZkJhULBfcHZL7ZMJmvxAOnxeFBbWwuDwcAFp/r6etTX13OBymazwWg0wmKxwGw2w2q1ckHOZrOhrq6u2ecAAGQyGWQyGcRiMSQSCaRSKZe82R8+n8/9AMGLCPYzUlNTwyUuh8PBBVOPxxPzdcViMXJycpCdnY2cnBzk5+cjNzcXubm5kMvl0Gg0yMrKglarRVZWFjQaDZRKJVeGVDEMA7fbzV3csEmCvTiqqqpCdXU19291dTUMBgP3t2gOiUQCpVIJmUwGoVAIqVTKJWCxWMx9xgHA7/fD5XJxF14ul4v7/jmdzrivJZfLoVQqkZGRwZ3TzMxM6HQ6yOVyZGdnIysri/usq9VqaLVaLlm1xHllGAYejwcOhwM2mw0WiwW1tbUwGo3c/9n3xF50VlVVoba2Fnq9HrW1tTGPLxAIIJfLIZFIuHgR+lkVCARwOBwRn8ujYcpBM2fOxO7du+FyuWAwGGAymWC1Wptc0UciEokgkUggFoshl8u5q0uJRML9Afh8PgKBAPx+PxcIvF4vF6DYIBGPQCBATk4OcnJyuASo0WiQl5eH/Px8ZGVlcbUItVoNnU4HrVYLlUoFoVDYKlevgUCAu5I0mUyw2+0wmUwwm81wuVxwuVxcjYq9CjUYDNzVfn19PQwGAywWC9xud9z3r1AouATJBhG2RsXn87lEyX55/X4//H4/lyTZMnk8HthstmYFLjb4srWDjIwMyOVyKBQKZGRkcH8ThULBbWO/jOwPG4ikUmmLBewLeb1eWCwWmEwmLrCYzWaYzWbU1NRAr9dDr9ejrq6OC+J6vR5erzfqMXk8Hpfc2SAtEom4zzgbsPl8Png8Hlfz8ng8cDqdXOBjr37jhRw+n4+cnBwUFBQgLy8PWVlZ0Ol0KCgoQGZmJnfeFQoF5HI5V3tWKpVQKpUtVqvw+/1hFxQmk4k7ryaTCUajkYsTVquVO6+1tbUwmUxRg+6F51WhUHDnlY0jbAAXCARcWdjPsNvthtvthtPp5GrNzQnjQqGQixe5ubncue3UqRM6derEXVjk5uZCrVZzcUwkEiUdNyjBxMAwDBwOB5xOJ3cFazabYbFYUFdXB6PRyF2Bsc1PbHWSrZaz1UiGYbjmptAvKfulYGsTcrmcq56zV/AqlQpKpRLZ2dnIzMzssE0czeFwOKDX67lzywbH0IBps9m44MVeubM/bBJnzzkALumwTSFs05JYLIZSqYROp+OuNNlApdVqkZ2dDYVC0aoJ4WIQCAS4JhG2mYStwYWef7ZphL04Yj/j7Llmf9hkI5FIwpIr+/lmP+vs/9nPeWZmJpeoL4XzHQgEUFdXx9W+QputTSYTdyFrt9u5zy974cO2FLC14tDPsEQigUQi4S56lEolpFIpFzvYc6nT6aBUKrkE3Bq1f1ZBQQEmTpyId999N2w7JZgQvXv3xpgxY7BkyZL2LgohhHQYPXr0wMiRI7F8+fKw7R3/MqEFicViGAyG9i4GIYR0KHK5PGKfFSWYEDKZrFkde4QQQhpFi52UYEKIxeK4ncyEEELCRYudlGBCsCO9CCGENF+02EkJJgSfz2/WcD9CCCGNosVOSjAh2CGWyT63vNYGn59qQISQ9BItdlKCCREIBJJKMF5/AH/8z16M/ec2vLn5WCuUjBBCLl7RYiclmBDJ1mC+2Hka3x6qBgBU1NpbuliEEHJRuyhrMDabDatWrcLatWubNXrLbDbj22+/bbKdYRj8+OOPWLFiBc6ePZt0efx+Pzc1QyKO6xunGslUipN+fUII6Yiixc52SzBr1qxB7969MWnSJEycOBF9+vTBjh07ou5fWlqKIUOG4KGHHgrbfvr0aVx33XW49tprce+99+Kyyy7Da6+9llSZ3G43JBJJws8zORrncdLJKcEQQtJLtNjZLgnm6NGjuPvuuzF+/HgYjUbU1tZi8ODBuPvuuyPOBlteXo5hw4ahsrISUqmU2x4IBHDXXXehvr4e+/fvh9vtxqJFizBr1ixs27Yt4XK5XK6w4zeXw9M4IaZCQhNUE0LSS7TY2S4JZvHixejevTuWLFkCtVqNzMxMLFq0CNXV1Vi1alWT/TMzM7Fw4UK8+OKLYQnoxx9/xM6dO/Gf//wHAwYMgEAgwKOPPorrr78eixcvTrhcXq83qZlY3b7GBCOnBEMISTPRYme7JJiNGzfivvvuC2uzy8/PR69evbBv374m+2s0Gjz22GNwu91QKpXc9k2bNqF///4YOHBg2P6jR4/G/v37mxyntLQUK1euxOrVq7FhwwaUlJTgwIEDMJvNAILrgojFiTdxOUNqMEpJ4n04hBDSkUWLnW1+uc0wDMrLy9G9e/cmv8vKykJ1dXXU59bX1yMnJ4f7f3l5OXr06BHxOFVVVU22r1ixAnPmzGmyvXfv3jh69GjSNRhPyL0vUiElGEJIerloajA8Hg8ymSxiX4vT6YzZB1JXV4e8vDzu/4keJ9riYTKZjHse+zgRXn/jHaxSMSUYQkh6iRY726XDIDc3FzU1NWHbGIbBmTNn8MADD0R9ntlsRrdu3bj/5+TkRGxSO336NPr06dNke1FRESZOnMgtA8wuKVpQUMAtuqTRaBJ+P25vY+JSSTv+Gt2EENJcsWJnuySYYcOGYfPmzZg1axa37cCBA9Dr9Rg2bFjU57FrxYceZ+HChaitrUV2djaAYKLavHkzbrnllibPLy4uRnFxccRjWywWMAwDtVqd8PtxhSSYfHXio9AIIaSjYpdsjhQ726WTv7i4GFu2bMGaNWsABJu+ZsyYgcLCQlx11VURn8MwDHQ6HWpra7lmsZtuugkqlQovvvgi3G43AoEAXnnlFRw9ehT/8z//k1CZTCYTACSVYLyBxiYypZRGkRFC0kes2Nku0fC2227D9OnTMXHiRPTt25frkF+xYgU3EmHEiBHo168fli5dit9++w2jRo2Cw+EAEBxVVl9fD7lcjo8++ggPPfQQ1q9fD5lMhpMnT+K5557D6NGjEypTXV0dgOCQ6ER5fI2d/AoxJRhCSPqIFTvbJRryeDy89dZbKC4uxubNm6HVavHggw+GteHp9XooFAoAQL9+/bB+/Xr4fD4wDAOVSsV1KE2cOBFlZWX49NNP4XK5cMcdd6B///4Jl8loNAJILsGYnY138gv4yc3GTAghHVGs2Nmul9vXXHMNrrnmmoi/Ky8v5x6LxWJcd911UY+Tl5eHZ599NqWysFlYp9MlfQwVNY8RQtJMrNhJsyk3YNsRtVpt0segaWIIIekmVuykBNOA7d9hm+WSIRPRPTCEkPQSK3ZSgmlQU1MDkUgElUqV9DEyqImMEJJmYsVOSjANampqkJOTAz4/+VOioan6CSFpJlbspATToKqqKmwammRQDYYQkm5ixU5KMA30ej3y8/NTOgY/ieWWCSGkI4sVOynBNKitrUVWVlZKx2Di70IIIZeUWLGTEgyC09Do9fqwpQCSQfUXQkg6iRc7KcEgOEuzx+NJOcEI6S5+QkgaiRc7KcEg2IYIBJcRSIWfoUYyQkj6iBc7KcEgOFU/kNxMyqFCJ70khJBLXbzYSQkGwWoekHqCcXojr5hJCCGXonixkxIMGrNwRkZGSsfx+amJjBCSPuLFTkowaDxJqUwTAwAMDVQmhKSReLGTEgwaq3mR1pROBPXxE0LSSbzYSQkGjScp1RqMmzr5CSFpJF7spAQDwGazQSwWQyQSpXQcS8jKloQQcqmLFzspwQDwer0pJxcAsLgowRBC0ke82EkJBoDb7YZUKk35OE4PDVMmhKSPeLGTEgwAu90OuVye8nEsLl8LlIYQQjqGeLGTEgwAl8uVUg1GIW5cKtkfoKFkhJD0EC92UoJB8CTJZLKkny8UNJ5GK/XDEELSRLzYSQkGgMPhSCnBCEJmUa6xuFuiSIQQctGLFzspwSD1UWShs/RXmZ0tUCJCCLn40SiyZuLzkz8VoTUYK3X0E0LSSKzY2S4Jxmq1Yv/+/airq2vW/jabDfv370dtbW3UfRiGAZPkXC3JPo8lFjaeRhqqTAhJF/FiZ5smGIZhsHjxYnTp0gWDBg1CYWEhZs2aBb8/elB+77330LVrV27/v/zlL/D5GmsJdrsdM2fORE5ODpRKJaZOnRr2+7YgCsngdLMlIYQEtWmCWbJkCZ566ik89dRTOH78ON577z28/fbbePPNNyPu/8knn2D69On44x//iOPHj+ODDz7AsmXLsGDBAm6fJ598Et9++y2WLFmC//u//0NJSQn+9Kc/JVQuHo+HQCD5ecRCazAuWhOGEJIm4sVOYVsVJBAIYM6cOZg5cyZefvllAMBll12G48eP480338Sf//xnCIWNxWEYBn/7298wffp0vPrqq9z+J0+exKJFi/Dss8+Cz+fj008/xQcffIA77rgDAHDw4EG8/fbbCZWNz+enlGAUksZyu7w04SUhJD3Ei51tVoM5ePAgqqqq8Nhjj4VtHz9+PM6fP48zZ86EbT927BhOnToVcf/a2lpUVFSAz+dDIpFg9+7dYBgGXq8XP/74Y9TV1UpLS7Fy5UqsXr0aGzZsQElJCRiGSTnByESNN1ra3NTJTwhJD/FiZ5vVYA4dOgShUIiePXuGbc/NzQUAnDt3Dt27dw/bHwAuv/zyqPsXFRVh/vz5eOKJJ7Bt2zbo9XowDINvvvkmYhlWrFiBOXPmhG1zOBwQCoUp9duoZI2nsc5G98EQQtJDvNjZZjUYoVCIQCAAHo8Xtp0tXGjzWOj/4+0vlUohEAigUqnQrVs3VFdXY+3atRFHN0QaTODz+VJOMGqZmHtM85ERQtJFvNjZZjWYnJwcBAIBmEwm6HQ6bntVVRUAhNVe2P0BoK6uDgUFBRH3r6urw9NPP42///3veP755wEAb7zxBp555hnceuutGDlyZNgxi4qKMHHiRPj9frjdbhiNRgQCgZQTTGgTmYOayAghaeKiqcFcccUV4PP5+PHHH8O2b9u2DV27duWavlhFRUUQCoUR98/NzUWXLl2wefNmOBwOPPPMM+DxeODxeJgxYwZ4PB527drVpAzFxcVYtWoV1q5di40bN2LXrl1Qq9UQiUTwepMfXiwPmezS5aNRZISQ9BAvdrZZgsnOzsaYMWPw73//G05ncDqVo0eP4n//938xfvz4JvtrNBrceOONWLx4MRwOBwCgvLwcb731FsaPHw8ej8c1u+n1eu55VVVVYBgm6hrRkUilUrhcrqTfm0wcWoOhBEMISQ/xYmeb3gfz+uuv48CBA+jXrx/uvPNODBkyBBKJhBu2bDKZMHToUJw8eRIA8Nprr+H48eMoKirCXXfdhauvvhoMw2Du3LkAgNtuuw2FhYW49tprMW/ePLz66qv43e9+h65du+Kuu+5qdrkkEgnc7uQ75yUh98HQKDJCSLqIFzvbNMFcddVVKCsrw1133QWhUIjZs2ejtLSU62M5dOgQdu3ahe+++w4A0L9/f5SVleG+++6DQCDArFmzcPToUXTp0gVAMHuWlJRg8uTJ2LJlC3bs2IEpU6Zg9+7dCS0gJhaL4fF4kn5foffBuH10HwwhJD3Ei51t1snPysrKwmuvvRbxd6NGjYLb7YZY3DgqS6vVYt68eVGPV1BQEPV4zSWXy7lmu2ToFI3l9VCCIYSkiXix86KbTTk0ubQV9iQle7NlToaEe+z0+mlVS0JIWogXOy+6BNMe2Oa0ZDv689XhC+54/VSLIYRc+uLFTkowADIyMgAElxFIhlYRvuCOj2owhJA0EC92UoIBoFQqAQTXnUmGRCgIG0lG/TCEkHQQL3ZSgkFwNBqAlDr61bLGWowvhYkzCSGko4gXOynBAJDJgn0oqSQYpTR0qYGUi0QIIRe9eLGTEgxaJsGE1mB4MfYjhJBLBSWYZlAoFACCyy8nK3So8oUzQBNCyKUoXuykBANApVIBSH4UGQBo5Y3371B+IYSkg3ixkxIMWr4GIxLQaSWEXPqoBtMM7FC7VBKMJqQGE7o+DCGEXKrixU5KMAguDcDn88Om/U+UMmTCS5GA2sgIIZe+eLGTEgyCq7JlZWWllGBCh45RJz8hJB3Ei52UYBoolcqUOvlB974QQtJQrNhJCaaBQqFIqQ9GSM1ihJA0FCt2UoJpoFAouKWZkyGljn1CSBqKFTspwTTIyMhIqYmMTxUYQkgaihU7KcE0UKvVMJvNST+fTx37hJA0FCt2UoJpoFKpUkownpBFxhia7ZIQkiZixU5KMA20Wi1MJlPSzzc6vNxjr58SDCEkPcSKnZRgGiiVSjgcjqhrS8djdTUmGD+taEkISROxYiclmAbswjnR1paOx+nxc48D1ERGCEkTsWInJZgGqc5HZgppIiOEkHQRK3ZSgmmQmZkJAKitrU3q+XaPryWLQwghHUKs2EkJpgF7koxGY1LPN9o93GMaskwISRexYme7JRi3253QcF632x23A97r9SY9RJit5tlstqSeX2drTDA0bQwhJF3Eip1tnmB27tyJkSNHQiqVIjMzE6+//jr8fn/U/ffu3Ytrr70WUqkUOp0O8+fPh8/XtDnqyJEj6NatG77++uukypWRkQEg+VUtz5sa16QW0m39hJA0ESt2CptsaUV79+7F6NGjMWbMGKxZswZHjhzBnDlz4HK58NJLLzXZ/+DBgxg1ahSGDx+O1atX49ixY5g7dy7sdjv+/ve/c/tVVFRg7NixKCoqwoQJE5Iqm06nAwDU1dUl9fz6kCYymq6fEJIuYsXONk0wL7zwAoYPH44NGzaAz+djwoQJEAgEmDdvHp566ilu+U3WSy+9hAEDBmDTpk0QCIKTSUqlUrz44ot49tlnoVar4fP5cMstt6B///5YtWoVN2QuUdnZ2QCS6+T3hdzFT6tZEkLSSazY2WZNZDabDVu2bMHMmTPB5ze+7KRJk2A0GvHbb7+F7e9yubBp0yY8+eSTXHJh97dardi1axcA4KuvvoLdbsfXX38NuVweswylpaVYuXIlVq9ejQ0bNqCkpAQHDhyAxWKBWCyGUqmEwWBI+L2F9r9IRDRughCSPmLFzjarwRw6dAherxdXXnll2PaCggIAwOnTp8O2l5WVwel0Ntk/NzcXfD4fp0+fBsMwWLBgAXJycjB69GicO3cOI0eOxEsvvYShQ4c2KcOKFSswZ86cJtt79+6No0ePQqlUJtXJX2t1c4+lQqrBEELSS7TY2WaX2+wIMLFYHLad7a+4sN+iOfsfOHAABw4cgMFgwIQJE/D444+jrKwM48aNi5hNow0m8Hg83GuxjxNhcDQ+Ry6hBEMISS/RYmdCNRiv1wuXy8WNGkhETk4OgGBHUKdOnbjtbLsdW5OJtH/Pnj257QaDAYFAAAUFBVyt59ChQ1z/zcyZM9GpUyesWbMGDz/8cNgxi4qKMHHiRPj9frjdbhiNRthsNu7ESKXSpKaKcYbcZCkXU4IhhKSXaLGz2TWY1157DQqFAoMGDQIAnDt3DsXFxejWrRuee+65uPeoFBYWQiaTYc+ePWHbf/nlF/D5fFx99dVh23Nzc6FSqSLuDwBDhgzhajeh975kZ2cjOzsbNTU1TcpQXFyMVatWYe3atdi4cSN27dqF0tJSlJeXA0g+wdjcjTUjCTWREULSTMoJRqFQwOv1YtWqVQCABx98EFu2bMFdd92FhQsXYvXq1XELcPPNN2Pp0qXcfSwOhwP/+te/MGTIEO5mHZZIJMJtt92GDz74AF5vcJ4vl8uFN954AwMHDoROp0OvXr0AADt27OCed/78eVRXV6N///7NfWucZJvIQmswEiF18hNC0ku02NnsaDhp0iQAwIkTJ1BRUYEffvgB77//PhYuXIi+ffvi559/jnuMl19+GQcOHMDQoUMxe/ZsjBgxAiUlJZg/fz6AYE1k5syZqKysBBAcpnzs2DEMHjwYs2fPxsiRI7Ft2zYsWLAAANCjRw/ccMMNePLJJ/HDDz9gx44dmDJlCrp164Ybb7yxuW+NIxQKI97EGY/FRU1khJD0FS12NjvBFBQUYPjw4Xj11Vfx9ddfQygU4vrrrwcQrG243e44RwAGDhyIw4cPo2/fvli3bh2Kioqwb98+7jhnzpzBokWL8MILLwAI9pkcOXIEgwYNwrp169CzZ0/s3bsX48eP5475ySefYNCgQRg3bhxGjx6NrKwsbNy4ESKRqLlvjSMQCGLOKhCN3d14YpWSNr21iBBC2l202JlQNJw/fz7uuOMOPP/885g0aRIyMjJQWVmJI0eO4Mknn2zWMbp27Yrly5dH/F2XLl2wf/9+9OnTh9tWWFiIjz/+OOrxOnXqhC+++AKffPIJeDxeUomFJRAIklpwzOGhPhhCSPqKFjsTSjDXXXcdjh49itLSUgwbNgxAsMnsxhtvxL333tsiBR0wYEBSz7twOHNbCl1sTEo3WhJCCIAEEozH40FZWRkCgQCuu+46AMF7VQYPHox169ZdEvNvBQIBCIWJN3FZQpZL1sjbL9ERQkh7iBY7m325/eKLL2LgwIGYMmUKAGDPnj3o378/NBoNxo8fD6fTGecIFz+/3x82LU1zOb2NNRi1LPkmOkII6Yiixc5mJxj2/petW7eCYRg89NBDEIvF+PTTT/HDDz/giy++aLnStpNkE8zpegf3mEaREULSTcoJZsKECRAKhVi/fj0OHjyIw4cPY9GiRbj//vtxxRVXNJmssiMKBAJhE3E2V0Vd41rUMkowhJA0Ey12NrvDQa1W46abbsKcOXNwyy23QC6XcxNKut3upPouLjZerzelUWgATddPCEk/0WJnQpfrb7zxBtRqNd5//31Mnz4dEokEe/bsQVlZGUaPHt1ihW0vLZFg6D4YQki6iRY7E4qGvXv3xv79+2GxWKDVaoMHEArxwgsvYOLEiS1T0nbk8/lSTjB56uQWPCOEkI4qWuxM+HKbz+ejqqoKJSUl0Ol06N+/P/7xj3+0SCHbm9PpTHhFTH+ACft/jooSDCEkvUSLnQk1kVVUVOCqq65Cv379MGHCBIwcORJdu3bFkiVLWqyg7cnpdEImkyX0HJsrfP4dBXXyE0LSTLTYmVCC+fOf/wyTyYTvvvsOXq8XVqsVTz/9NB599FHs3LmzxQrbXjweT8IzAtg84QnGF2AQuKBWQwghl7JosTOhBHPkyBFMnjwZv/vd78Dj8SCXy/HCCy+gV69e+Oyzz1qssO2BYRjY7fYmywbE4/KGT/B21dxNeHrFvpYsGiGEXLRixc6EEszYsWPx2Wef4dy5c9y26upq1NTUhK1S2RE5nU74/f6EV+t0uMMTjNXtww9l+pYsGiGEXLRixc6EEszLL78MiUSCnj174tZbb8U999yDK664AjqdDo888kiLFbg9WCwWAIBKpUrseSHzkLFoPjJCSLqIFTsTSjCdOnXCoUOHsHDhQqhUKthsNjz//PPYvXs3NBpNy5S2nZhMJgBI+H1YXU0X2bmw2YwQQi5VsWJnwsOUFQoFnnjiCTzxxBMAgk1kJSUluPXWW1MsZvsym80AgjMWJKKiztZkm9ef+JoyhBDSEcWKnSkvXvLtt99iwoQJOH/+fKqHaldsNS/hBFNrb7LN7aMEQwhJD7FiZ8oJZvTo0WAYBps2bUr1UO3Kbg8mCoVCkdDzLM6mfTBUgyGEpItYsTPlBNOzZ0907twZu3fvTvVQ7aq+vh4AuClwmssVobbi9dO9MISQ9BArdqacYHg8Hvr164djx46leqh2pdcHhxbn5uYm9DxvlOYwP0MJhhBy6YsVO2N28jscDqxfvx4Mw0AgEEAgEEAikUAul0OhUECpVCI7Oxs5OTk4cOBA65S+jZhMJkgkkoSniglESSSx8sunJZUAw+D+4V0viaWmCSHpK1bsjJlg9uzZg3vuuQdMM67Gx4wZk3wJLwIWiyXhe2AAINqZYaL8JhBgMHvVIQDAWaMTL9xSlPBrEkLIxSJW7IyZYEaPHg2v1wufzwe/3w+/3w+32w2HwwGHwwGbzQaz2Qy73Y7hw4e3SuHbSl1dHXQ6XcLPi5Z8eYhcM+HzG7e/t70Cz950OeiuIUoAACAASURBVISClFsqCSGkXcSKnXHvg2GbxliJTqXSURgMBmRmZib8vGiVO36Mlq8spRh1Ng8AwOn1I4MSDCGkg4oVOymyNbDb7QkPUQYAoSByJhHEyDChU8mYIwxzJoSQjiJW7KQE08BmsyU8k3IssTrvdSEJxuGhaWUIIR1XrNjZ5gnm3LlzmDp1Krp164bhw4dj7dq1Mfevrq7GI488gu7du2Po0KH45ptvovZ7fP/99+jWrRuWL1+ecLnq6+uT6oMpqTA0e1+23CpZ49KiTkowhJAOLFbsbNMEc/LkSQwYMAA7duzAjBkz0LdvX9xxxx1YtmxZxP3Pnj2LAQMG4IcffsAf//hHDBw4EHfffTfeeeedJvvq9XoUFxejsrISFRUVCZfNZDIllWCaq8rsxOC/b8ad7+5AaZWF2x5pNmZCCOkoYsXOhCe7TMWsWbNQUFCAnTt3cmOmc3NzMWfOHDz44IMQiURh+//1r3+FVqvF3r17uTa+wsJCzJ07F7///e+5NaAZhsFjjz2Gzp07JzWrs9frhcvlatUBDMdrbKi3e1Bv94RtpyYyQkhHFS92tlkNxu12Y926dXjmmWfCbsh54IEHcPbsWezZsydsf5/PhzVr1uDpp58O60B64IEHoNfrUVJSwm379NNPsXbtWixduhRCYfScWVpaipUrV2L16tXYsGEDSkpKUFlZmfRMyomIdr8MTe1PCOmo4sXONqvBHDx4EA6HAyNGjAjb3rVrVwBAZWVl2L00ZWVlMJlMTfbv3Lkztz8AnD59Gk888QSeeeYZXHnllTHLsGLFCsyZMyds24MPPoi5c+cCSHyiy5bgoZmXCSEdVLxJgtusBuN0OgGgyWgDsTg4osrn8zVrf6FQCKFQCJ/PB5/Ph6lTp6KwsLBJ4ojE729aW5BKpXC5XNzj1hJtTJmHZl4mhHRQ8WJnm9VgsrOzAQRvyikoKOC2GwyGsN9H2r9Hjx7cdovFAp/Ph+zsbMydOxc//PADBgwYgIceeghisRjnzp3DmjVroFKpMHPmzLDhwkVFRZg4cSI3I4HRaETXrl3bJMHwowxbplFkhJCO6qJJMIWFhRAKhTh8+DCuuOIKbjs7zf/gwYPD9s/Pz4dEIsHhw4fDfsfuP2TIEDgcDvz+97+H2+2G1+uF3W6H3++HwWDA3r17m5ShuLgYxcXFTbZv374dQOv2wfCj1BW9fpp1mRDSMV00fTBKpRI33HADli9fjnvuuQcAEAgEsHTpUvTt27fJMDeJRIKbbroJy5cvx4MPPggejweGYbBkyRL06NED+fn5mDJlCqZMmRL2vL59+2LKlCl4+eWXm122WGtKt5RoNRgfNZERQjqoeLGzTYcpP/fccxg3bhzuvvtu3H777fjqq6+wZs0afP7559w+n3/+OW655Rao1Wo8++yzuPbaazFp0iRMnjwZK1euxDfffIMPP/ww6msEAokH7GRXs4xHb3EhUymBgM+DMMrUMbRuDCGko7poOvkBYOzYsfjpp59QUVGBRx99FKdPn8bKlSu5Wsjp06dx77334rHHHgMAjBo1Cr/88gvOnz+PRx99FMePH8eKFSvw8MMPR32NUaNGoVevXgmVi63mtWQN5sOfT2LovO/x8urg1PwSoSDifpRfCCEdVbzY2aY1GAC45pproi6v3KVLFyxbtgy33XYbt23YsGH49ddfm338pUuXJlwmq9UKoGVnin5l7REAwNajtQAAuSRygqH7YAghHVW82NnmCSaeadOmtflrWiwW8Pl8yOXyFj92J03wplKZKHKCoaliCCEdVbzYSbMpIzgUWqPRgB9tqFcK3L5gDUUaJcHQMGVCSEcVL3ZSggHgcDhapfYCADZ38AZSiTDyqXZ5aRQZIaRjihc7KcEgOGHbhRNtNpdCHLlmwjptcAAA5GJBxEXIfEmMeiOEkItBvNhJCQapJZjQ1SkjHrvhRkoejxcxGdGNloSQjooSTDP4fL6YszDHkpUhafa+GdKmfwh/gBIMIaRjihc7KcEgtRpMliJ2DQYAnvlqP4BgM9mFaLJLQkhHRTWYZvB4PNyszonqrIs/OODrPWfh8voRiHBXpZum6yeEdFDxYiclGKTWRCaOMjrsQna3L2J/iyjKFDKEEHKxoyayZvD7/RAIYo8Gi4YdhhxPtKWRo02CSQghF7t4sZMSDACGYZK+yfK8ydms/bz+AHXoE0IuKfFiJyWYBrwkaxJ6i7tZ+5VUGKK8blIvSwghF4VYsfOim4usvTBJTmtcZW5eDebFlQcjbhcJKMcTQjquWLGToluDZBOMxdW8PphoVDLK8YSQjosSTBwCgQB+f3KTTqbaryIXU4IhhHRM8WInJRgAQqEw6QSTKrUsuRs8CSGkvcWLnZRgAIjFYrjdzeusb2mqCNPHEEJIRxAvdlKCASCTyeB0Nq+z/kLNvdEyGq2CEgwhpGOKFzspwQBQKBSw2+1JPdeT4lQvNIqMENJRxYudFN0AyOXypGswqaIEQwjpqOLFTopuAEQiETweT7u8drSVLgkh5GIXL3ZSdEOwo6q9EszPJ+pQXmtrl9cmhJBUxIudlGDQeJKSvdkyFV/sOsOtF0MIIR1JvNhJCQaARCIBwzDw+VK7Kz9ZB86a2+V1CSEkFfFiJyUYABkZGQAAi8XSLq+vldNQZUJIxxMvdlKCAZCZmQkAMBqN7fL6dnf7zCJACCGpiBc72zzBOJ1OLFiwADfddBMeeughHDlyJOb+brcbr7/+OsaPH48HHngABw4caLJPTU0NFixYgOnTp+Odd95J+J4WrVYLADAYIk+p39qcXn+79P8QQkgq4sXONk0wBoMBV155JRYsWIAuXbrgxIkT6N+/P/773/9G3N9sNmPw4MGYO3cuCgsLUVlZiUGDBmHlypXcPhs3bkSfPn2wePFinDhxAk8//TRGjhyZ0KgwtVrNvV57cad4wyYhhLS1eLGzTafy/etf/wq73Y4jR46goKAADMPgD3/4A5555hncfPPNTRaumTNnDurq6nD48GF07twZDMNgxowZePbZZ3H77bdDIBBg+vTpuPfee/HGG29AIpFg+/btGDNmDL7//nvcfPPNzSqXQqEAgKTv5m8JHn8AUlFyyzYTQkh7iBc72yzB+P1+fPnll5gzZw4KCgoABFdCmz59Oj744APs27cPV155Jbc/wzD44osv8NRTT6Fz585h+7/zzjvYuXMnRowYgWPHjoWtCc02NSmVyiZlKC0tRVlZGfh8PiQSCTQaDQoLCy+KGoyXajCEkA7moqnBHD58GEajEWPHjg3b3rt3bwDAyZMnwxLM8ePHUV1d3WT/Xr16cfuPGDEiLLnU1NTgySefRI8ePTB8+PAmZVixYgXmzJkTtm3ixIn46KOPAAB1dXVJv79Uef3UB0MI6VjYTv5osbPN+mBMJlNYgVjRqljR9heLxRCJRE3237hxIwYOHAiDwYDVq1dDJGo69DfSugV6vR5qtRpSqRRVVVUJvquW4wtQDYYQ0rHEi51tVoMJHc6Wl5fHbWcTCTsaIdL+3bp147Y7HA54vV5u/0AggNmzZ2PevHm4//77sWjRoibHYhUVFWHixInw+/1wu90wGo2QyWTg8XjIz89HdXV1i73fRFF+IYR0NPFiZ5slGLbfpby8HEVFRdx2dtjx1VdfHbZ/Xl4e+Hw+ysvLw5rOLtz/rbfewvz58/HZZ5+huLg4ZhmKi4uj7qPVarlk1x4CNEyZENIBxYqdbdZEptVqMWLECHz11Vdh27/88kt06dIF+fn5YdsVCgXGjBkTcf+cnByuVrNs2TJMnTo1bnKJR6VSJdXJz+fF36c5eC10HEIIaUuxYmebDlOeMWMG7r//fnTp0gW33347vvzyS7z33nt44403uH2OHTuGHj16QCgUYsaMGZg8eTK6d++OyZMnY+XKlfj3v/+NefPmcUOajx07hp49e2LWrFkwmUxwOp3Iy8vDM888g+zs7GaXTaVSobKyMuH3JBEK4PSmfic+nzIMIaQDihU72/RGy+LiYnzwwQd45513MHToUCxbtgwLFizAzJkzAQDnz5/H5ZdfjkceeQQAMGnSJCxfvhwffPABhg4dirfffhuvvvoqnnvuOe6Y48ePx759+7Bt2zaUl5fDbDZjw4YN2LNnT0Jly8zMhF6vT/g9qWQtk6PZ/EJ39BNCOpJYsbNNazA8Hg8PP/wwiouLodfrkZ2dDalUyv0+NzcXjz76KP70pz9x2+677z7cddddqKmpabI/gLC7+lORl5cHvV4PhmGa3PAZS6ZCghqLO+XXFwn4OHLegvuX/Ypnb7ocxUO7pHxMQghpbbFiZ7tMdimRSNC5c+cmyUIgEOC9997DgAEDwraLxeKI+7ek3Nxc+P1+1NfXJ/Y8laRFXl8i5GPD4WoY7B688M1BODzts3QAIYQkIlbspNmUG+Tm5gIAamtrE3pelrJlEoxIwIdU1PjnMNgjz6X2aUklNh+paZHXJISQVMWKnW3aRHYxY6eWsdkSW75YqxC3yOufrLPD7PA2lkfS9E+jt7owe9UhAMCpBbe2yOsSQkgqYsVOqsE0UKlUABJfdEzTQouFTX53B8qqrdz/M6RNjyvkN/65vH66M5MQ0v5ixU5KMA2STTA6ecvUYNy+APZUNi7aI4hwg41Y2PjnctAiZYSQiwAlmGaQy+UAEp+yXyZuuSn2be7YHfvSkARjcXlj7EkIIW0jVuykBNOAzcJWqzXOnuFaYw0XiTDyn0UoaNyut7pa/HUJISRRsWInJZgGGRkZABJPMLJWSDBuXwDLfjoZc5/jNYkNRiCEkNYQK3ZSgmkgk8kABGdrToRC0jqrUK7Zfx5WlxfLfjqJyvqmVc8TekowhJD2Fyt2UoJpwOfzIZVKE+6DkYtbZ6T3/jMmzF51CK+uO4IxC7fi24Ph6y0cPm/B1qN6+AM0tQwhpP3Eip2UYELI5XI4nc6EnhOtv6QlrNp3nns8/T978dYPx7n//1JRj4c/3IV1B85HeiohhLSZaLGTEkwIpVKZ8I2Wklbog4nm9Y3HmmyjvhhCSHuLFjspwYRQKBQJJxhpK9ZgmqNHtqJdX58QQqLFTkowIUQiEbzexO4vacn7YJLRWn1ApGPz+gPYdKQGJkfkOe0IaUnRYiclmBBisRgeT2JfyNYYppyIx5dHX/emxuLCqbrEBi20lHe3luP3H+3CoXOJrxJKUlNZb8fwed/jkU924/WNR9u7OCQNRIudlGBCJFODSWTtmNZkdnpxQm9DIMDA6vLiH+uPYNi87/G7N7fHnSEgUT5/AIu+P45/xgheS36swPdlety2+Ce8u7W82cd2eHxYve8c7C1c5nTy6rpS1DfMxl1voxoMaX3RYie1r4QQCATw+zveHF+V9XaMWbgVAJCnksLPMKi1BhdB8/gDcHh8EWdnjsbl9WPuuiO4/vIc3Ng3N+x3Pn8AUz/ahR+P1wEAHrm2B1QRJuY0Oxs/bKt+O4fp1/WM+7oMw2DcP7fhvNmFv03oi6nXdG92mZvD5w9g0Q8nsPNkPd697+oWmwn7YlNtaRzNo5a1zGSshMQSLXZSDSYEn89Pasni//xhWCuUpvme/eoA97ja4uKSC0sibGzGqza74IszE/O8/5bis19P45FPdjf5XZXZxSUXAHB7Ix9LEFKzEwmbV8s7WmPFeXNwCpzjLXwjKcMweOzTPVj0/XGUVBhw4BJuunOF/E00LTQZKyGxRIudlGBCBAKBpJq8rrksC3Mn9muFEjXPzlOGmL+XNwxEOKG3Yfj873HXe7/E3H/r0eiLrqkuuCJ2eZtetfj8AXhCktihcxacro8/QwIPjee+pfu26u0efF/WuG64WHDpfvQ9vsZz3959hCQ9RIudl+63LAl+vx8CQXJfSPtFPH3+piM1OGt04GjDejO/nTZh/L+246kv90XcP3Q1zQtrQyqpEMKQpQScERKMy9e0VnPtwi1xy6mUNjbjRUpcqbiwTyeRJsOOJjTByNt5lCNJD9FiJyWYED6fD0JhcoHnvCmxGQDa0h//sxejXtuCP322l9tWVm3Fyt/Owenxg2EYnDc5wTAMvtp9JmxQwF9XHuQeBwIMeDxeWLt+pAEE0ZrgvP4AGIaJ2gypCkswjcf4+UQdZq86hIra5JvNfBdMqROazC41/pDzq7iEEym5eESLnfTpC+F2uyGRSJJ6bmindkfi8Qfwwc8nsfC7o+iRpUDFBcOa9581AQCsLi/6z9nY5PlOT4Qmsijzo+2pNOKLnaex5WgtHh7ZDdNGdQ9LVqG1CmvIejfvba/A9mO1+LSkEn+5sTeeGNsrsTcJwOcPL1NrTVJ6MQgEQhPMpfs+ycUjWuykGkwIl8sFqVSa1HNDm5U6klqrC3sbVtK8MLkAQI0l2ET25a4zEZ9/4WSbRrsHT37+W8R9p7xfglX7zsPs9OLf3x/HjW9s45quGIbBhkPV3L5V5sb1bkKby/79/fGkBmL4Ao01Ih6v5VYivRiFJvj/t+Foiw9TJ+RC0WInJZgQDoeDW50tUdYOusLk7FWHwzq/I/l4xyn8fX1pxN/5GQbHa6xcorh36a/YUV7frNfWW93QN/TxXPnqJkz/T2MT3uHzjaO8QodB+wIM7B5/wn00O040lkkrF4ct3taSfj5Rhzvf3YGno/Rvperbg1W4b2kJ/nuwKqymEioQkoDPmZwY989tSSXljqLW6u6w379LRbTYSQkmRCoJZuKgTi1cmrbxS0X8ZPC3NYej/m7qh7tw45vb8fSKfSirtqC0qum63LFsKdPjznd3wOQIDxABJhgcGYaBSBA+OuWKv32HcW9si7hUgdcfQElFPdy+xgRkc/vwj/82Jshotc1Ub+78+7ojuG/pr9hdacQ3v50DEKyZ7T1tDOt4T8VXe87i5xP1+ON/9uJ/3vk5YmC9MJdUW1wRa6cdnccXwD83HsWQf2zG/Ut/be/ipDVKMM3g8XggFifXdDL1mm4tW5gO5rvDNRj/rx8Tft5HO05hd0MT3YWuWfAD/rJiP74NaTpjnTU6m9RivP4A/uednzHl/RJ8+kslgOCAgwNnTHHL8c7WE+j3t++w+UgNgGCz3NIfK/C7N7fh4Nno98xsOFSNF745gHMmJ7YcDa8Jrtl/HmsPVGHSOzvw5y8jNxte6NOSSjz/9QFuxN+FvCEDKPafNeNYTdP9ItVW6i4YDQg0f6Re6MCMWIM02triH45j8Q8nAATPBWk/0WJnm3fyMwyDVatWYcuWLcjLy8P06dOh1Wpj7r9u3Tps3rwZ2dnZePzxx5GVlRW2j8FgwLvvvouamhqMHTsWt99+e1L3s6TSyc/j8fDWvVdixmfNCyQk6LQh9v0xbE0gkl2nDLi6qxa/nTahrNqCPLUMh84Fa1AHz5kRCDC47K/fRnxut1nrI25/adUhKCRC/PnL37j+pwlv/YRTC24FEPw8+gIMRA1NbO9uPYH9Z834fGfTPqpPfznFTUb634NNkyTrvwer0DNbicvzMvDOlhOoMrvw5e4zmHRVJ7x+50DwQ4aFXzhYQcBveo3ojVCzM10wCOV/t5Vj4XdH8afreuLP43qDz+fB4wvev8QOtth+rBZnjA58d7gGeosLHzw8BNct3Irfj+6O58f3ifheXF4/pG107039BTXRk3V2dM9SYOmPFfhy1xn8flR3DOysQVG+qk3K4/b58fmvp7H0p5MYV5SLObe3371x6w6cxy/l9XhoZDf0zs1o9deLFjt5TBtejrhcLkycOBGbN2/GNddcg/LycjidTmzcuBGDBw9usr/H48HkyZOxfv16XHPNNTh16hQsFgs2bNiAESNGAAC2bt2KyZMnQyKRoFevXvj5559xyy234JtvvkloyLHP54NIJMIrr7yCl19+Oen3yAauXjlKfDh1CEa9Fv/+D3Lx+3jaUEiFfHx7qBrf7D2LTU+PgVomQp/ZG2I+r3uWAicbmqd+P6o7LE4vHhvTA4VaOd7ecgIn6+xYdyC4WmnJC2MxfP73TY4x73/64+7BhRAK+Bj/r+0oC6ndrH9yFPoVqAEER/TJxIKIyfNf9wzCHVc2NuMOn/c9qi2NAyn+eF1PfFoSrPX99ZYi9MxRYsZne7kke6HSueObzCT+8Y5T+Nuaw7iikwoLJg3AFZ3UMc8Ny+3z49uD1bh1QD5EAj7WH6jC5XlKXJYTDIx1Njd8fgZ56sZO5JKKekxfvgdGR/y+l50vjkWOqmkH9IZDVfjw51OYP6k/emQrm1XWaPwBBrf8+0ccbahR5qok+PXFcSkdMxH1Njf2VBoxtigXAj4PN725nSvL5KsK8drk/q3W7xgrdrZpgpk9ezYWL16MzZs3Y/DgwXC5XJg8eTLMZjN++umnJvv/4x//wIIFC7Bp0yYMHz4cHo8HU6ZMQWVlJXbv3g2Hw4FevXph5MiR+Pjjj6FQKLB//34MHToUH3/8MaZMmdLsspnNZmg0Grz++uv4y1/+kvR7PGt0wOdn0C0ruE7L1qN6PPzhLswc2ws/n6iL2hxESDL65GWEJZxYFt45AAMKNdh6VI/535al/Non59/CtRSs3ncOM78IH9hQPLQzhnXPxKl6O/61Obga66398/HanQMgFwm4mtkNr2/l+oieH98Hr20Ilu3UgltRa3XjtsU/osbixronRuGKTmqYnV4MfKXpkPlYjv39Zji9fhw5b8FTX+5DdoYEB0OmCyqfdwt+raiHSibCnkojRvXKQs+QpOPzB7D9eC2Gdc+MeG9RpGH81/bOxifThoJhGO48uX1+CBumVWEDvj/AoNriQieNrFnvZfcpA0QCPk7V27HhUDX+MLoHJr+7A0DwxlpHhFsHpgzpjAWTB+CE3opdp4y4Z3DnsJpxKmLFzjZLMAzDoLCwEI899lhYltu6dSuuv/56lJaWok+f8Gp3z549cc8992DevHnctpKSEowYMQK//fYbKioqcM899+Ds2bPIzW2clPHOO++E2WzGpk2bml2+kydPokePHli2bBmmTZuWwjttyucPQCjg47vD1Xjs0+jT6xNCLn53DCoIW868LfTNV+G58ZfD7PQ2SeSpunVAPs4ZnfjT9Zc1mdy2OWLFzjbrgzl27BjOnz+P22+/PWz7gAEDAADHjx8PSzCVlZWoqKiIuf/27dsxbNiwsOTC7rNs2bImZSgtLUVZWRn4fD4kEgk0Gg3kcjkuu+wyGAzB+bwu7N9pCeyVyk398vDFo8Ox/kAV/jC6O/625jA379ekqzqhq06BQ+fNeOPugXh8+R783DC0dkChGs+P74ORPTPh8gbw2oYyfLTjFPLVUhjsHrgTGKE0qLMG+5rR6U0IiaytkwsAHKmy4OEPd7XKsdc3NNE+8slu/PLCDchXN68mxYoVO9sswdTUBEfn5OXlhW1Xq4PttEajsVn7y+VyiEQiGI1G1NTUID8/v8lraTSaJscDgBUrVmDOnDlNts+ZMwfXXnstAEClat0OweE9MjG8RyYA4KOpQ3G63oFvfjuLaaO6h93v8fJt/fB/e8/iT9ddBrW8cbtMLMCc2/uFdSCaHV6UVlvQI0vBtTWbnV5sP1aLK7toIBUJoJWLIWioEpudXqz67VzY8OPX7xqIod106KSVYcOhaogEPNzYNxf+AINNR2qQo5JCKRFCKuKj2uzCmv3noZKJsO+0CQ6vHy/dWoRCrQyLvj+Oz3eeweanx2DcG9sAAF89PgKvrjuC3rkZeOzaHvjrqkPYebJxgs4MiRBXd9OGTbLZv5M6rAkDAD54eDCmfdR0hufmuG1APtfXQQiJLNLMHPFYLMGBNZFiZ5slGI1GwxUmNGlYrcH244yMjKj7h3K5XPB6vcjIyIBarcbp06ebvJbFYon4ZqOt9aLVamE2B4MZm/DaSpdMOf48rneT7ZfnZeDFW4qadQy1XMQlLW6bTIQJAwsi7y8T4aGR3fDgiK4or7Wja6acGxUFBKvMLKGAh5v7hyfxrpkKDLvg9VjzJw3A/EnBWiY78goA1swYxT1e8diIZr0vlsnhgdfPIDtDEnbMUHa3D15/oMn09IEAw7U1v3Vv5OM7PL6oS0+Htp8DwXsvDpw1QSYWoFAjh1IqhIDP4256ZF/rwudtOarHf0oqMX/SAEhEfO5iwunxw+LyYuPhany28wyu6ZmJJ27oBZVMCL3VjR3lddhTacRfbrwcQgEPtVY3ujf07+0or0edzY2ZX+zD9Ot6ho3s8vkD+Lm8HgML1ThrdKJvvgq/nTFxbfUAsOHPo9EnTwW3z483Nh7De9srcHluBm4fVIDpY3qCAcAD8M9NR/H2luCicTOuvwyPjgmuAXTwrBkT3mrsO31tcn/ccWUn1Ns8WLP/PBaE9PO8de+VuKlfHuptHnh8Ady66EdYG+47GtxVi1k398EVndQQ8HlY8G0Zlv10MuzvUPbqeAQYBnKxEGcMDuit7rD38tS43rhveBdkKsT4YtcZvPBNcA694qFd8IfRwYs3BgyG/qNxEMVdVxfipVv7IsAwqKizYfK74bOM//riWNRa3chVSWF2ejDuje1hvy/UyvDhw0OQkyFFnd2Nsf/cxv3u/905AHdeVQin14/7lv7apNXg51k3oJMmeDEXaVXahXcOwLW9s6GVi9H7paYjId9/4GrYPT7kZkhhc/vwaISm919euAEGuwe3Lgrv3x7aXYclDw5ucpH51r1XJjXYIVbsbLM+GL1ej9zcXGzevBljx47ltv/6668YPnw4ysvL0aNHD24723G0bt063HprY1DZv38/Bg0ahMOHD+Pzzz/H119/jdLS8LvMH3jgARgMBqxfHz6a5vPPP8eXX34Jv98Pt9sNo9EIm82G2bNnw+12Y9q0aTh58iS6devWOieBEEJaSXAy2rZfZffDDz+MGjvb7EbLnJwc9O/fH2vWrAnbvnr1amRlZaF79/DVC9VqNYYMGRJx/4yMDPTpPVwt+AAAHA5JREFU0wfjxo1DWVkZjh8/zv3e6XTiu+++w9ChQ5uUobi4GKtWrcLatWuxceNG7Nq1C6Wlpbj33nthswVn6lUqUxuuSAgh7YHP57XLEu6xYmeb3sk/bdo0vP/++1i+fDksFguWLFmCf/7zn5g2bRp3YkKbsaZNm4YPP/wQH374ISwWCz7++GPMnz8fU6dOBZ/Px6hRo9CrVy88/PDDKCsrw7lz5zBp0iSYTCbcf//9CZXN6QxOty+TJdbBRQgh6Sxm7GTakNfrZWbNmsUIhUIGAMPn85mpU6cyTqeTYRiGqaurYwAwzz//PMMwDOP3+5nZs2czIpGIAcDweDzm/vvvZ+x2O3fMI0eOMEOHDmUAMACYgoIC5ptvvkm4bC+++CIjEAiYQCDQMm+WEELSQKzY2aZTxQiFQsyfPx+PP/44jh8/jp49e4Y1jSkUCgwdOpQb0cXn8zF37lw88sgjOHr0KLp3746ePXuGHbOoqAg7duzAzp074Xa7MWzYsKRqIVarFRkZGe1SxSSEkI4qVuxs0zv5L2YPPfQQtm/fjpMnT8bfmRBCCADA6w1O1yMSiZr8jhJMiGjrSncEDMPAbDajvr4eZrMZdrsdZrMZRqMR9fX1sFqtcLvd8Hg88Hg88Hq9cDgcsNvtcDqd8Hg88Pl8TYZy83g8CAQCCIVCiMViiEQiCIVCiEQiiEQiyOVy6HQ6qFQqbui4QqGARqOBWq2GVCqFVCqFQqGAWq2O+CG8FPh8PphMJthsNtjtdlgsFu7cOp1OuFwu2Gw2WK1WOBwO7sfj8cDtdnPD730+H/cTCAQQCAS42YvZK0T2vIeeW4lEApFIBKVSCbVaDbVaDZVKBZVKxT3OycmBWq3usLV0q9UKg8EAu93O/TgcDlitVlitVu78so/Zc+pyueB2u+H1euHxeMI+4zwej/tsi8ViyGQyZGRkcD+h50+j0UCj0XCPtVrtJfF5drvdOH/+PIxGIwwGA2pqarjPr8vl4j6rbreb+0yzn1W/349AIIABAwZg4cKFTY5NSyY3mDlzJg4dOgSZTAaNRgOdTscFTJlMBqVSCa1Wy33YdDoddDodFApFQpNqxhIIBOB0OmG1WmGxWOBwOGCxWGCxWGCz2VBTU4OamhpUV1ejvr6e+53RaERVVRVcLlfM4/N4PO6LxH6ZFAoFZDIZJBIJBAIBBAIBeLzgaBSGYbgh3T6fj0tMPp8PXq+XS1ImkwmBQPNmE5BKpdBoNMjMzIRSqYRCoYBOp0NWVhb3xc3JyUFmZiYUCgX3BWe/2DKZrMUDpMfjQW1tLQwGAxec6uvrUV9fzwUqm80Go9EIi8UCs9kMq9XKBTmbzYa6urpmnwMg2CEqk8kgFoshkUgglUq55M3+8Pl87gcIXkSwn5GamhoucTkcDi6YejyxV1YVi8XIyclBdnY2cnJykJ+fj9zcXOTm5kIul0Oj0SArKwtarRZZWVnQaDRQKpVcGVLFMAzcbjd3ccMmCfbiqKqqCtXV1dy/1dXVMBgM3N+iOSQSCZRKJWQyGYRCIaRSKZeAxWIx9xkHgheVLpeLu/ByuVzc94/tvI5FLpdDqVQiIyODO6eZmZnQ6XSQy+XIzs5GVlYW91lXq9XQarVcsmqJ88owDDweDxwOB2w2GywWC2pra7nbMCwWC/ee2IvOqqoq1NbWQq/Xo7a2NubxBQIB5HI5JBIJFy9CP6sCgQAOR+RZ0akG02DmzJnYvXs3XC4XDAYDTCYTrFZr1JszQ4lEIkgkEojFYsjlcu7qUiKRcH8APp+PQCAAv9/PBQKv18sFKDZIxCMQCJCTk4OcnBwuAWo0GuTl5SE/Px9ZWVlcLUKtVkOn00Gr1UKlUkEoFLbK1WsgEOCuJE0mE+x2O0wmE8xmM1wuF1wuF1ejYq9CDQYDd7VfX18Pg8EAi8UCtzvy7L2h71+hUHAJkg0ibI2Kz+dziZL98vr9fvj9fi5JsmXyeDyw2WzNClxs8GVrBxkZGZDL5VAoFMjIyOD+JgqFgtvGfhnZHzYQSaXSFgvYF/J6vbBYLDCZTFxgMZvNMJvNqKmpgV6vh16vR11dHRfE9Xo918wRCY/H45I7G6RFIhH3GWcDNp/PB4/H42peHo8HTqeTC3zs1W+8kMPn85GTk4OCggLk5eUhKysLOp0OBQUFyMzM5M67QqGAXC7nas9KpRJKpbLFahV+vz/sgsJkMnHn1WQywWg0cnHCarVy57W2thYmkylq0L3wvCoUCu68snGEDeBsi0roZ9jtdsPtdsPpdHK15uaEcaFQyMWL3Nxc7tx26tQJnTp14i4scnNzoVaruTgmEomSjhuUYGJgGAYOhwNOp5O7gjWbzbBYLKirq4PRaOSuwNjmJ7Y6yVbL2Wok03Bnt0AgCPuSsl8KtjYhl8u56jl7Ba9SqaBUKpGdnY3MzMyLronD5/PhjjvugEqlglarxdtvv530sRwOB/R6PXdu2eAYGjBtNhsXvNgrd/aHTeLsOQfAJR22KYRtWhKLxVAqldDpdNyVJhuotFotsrOzoVAompUQAoEAVq1axTWBTZ8+Pelz0NYCgQDXJMI2k7A1uNDzzzaNsBdH7GecPdfsD5tsJBJJWHJlP9/sZ539P/s5z8zM5BJ1ayXg1sRcMHtDIBBAXV0dV/sKbbY2mUzchazdbuc+v+yFD9tSwNaKQz/DEokEEomEu+hRKpWQSqVc7GDPpU6ng1Kp5BJwa9T+46EEkyKv14s9e/ZwzRxFRc2b3uVSYjQaodPpAARHAja3KeNSku7n4MLgmo6MRiN69OiB7Oxs5OXlYfv27fGfdImjPpgUlZaWcouf9enTp8m0NengzJnG1Rw7d+7cjiVpP+l+Dg4ePIiBAwcCSO/vAduM1lEHC7W0jlcPvciEtrOm6zQzdA7oHKT7+wfoHERCCSZFJlPjLKlarbYdS9J+6BzQOUj39w/QOYiEEkyKQkc9SSSSdixJ+6FzQOcg3d8/QOcgEkowKQq99yFd213pHNA5SPf3D9A5iIQ6+VPUp08fvPLKK/D7/WFLPqcTOgd0DtL9/QN0DiKhYcqEEEJaBTWREUIIaRWUYAghhLQKSjAp+vXXXzHh/7d37kFRVm8c/y6wCuvqossG6LhyGQgZNRUoJYsuErp4S60GvEwyqUxpVCaRl6bSUiq1yciQclLLS+o4lQqatwzG0kZEIVHZhKBUVgFXrsLu9/cHs++v10UyYAH1fGYcZ8953ofzPGd5H857nvc8Y8ciJCQE8+fPR1lZWUcPqdX89ddfGDt2LNLT02Xt169fx8KFCxEWFobRo0fj8OHDdtf+8ssvGDNmzC39UVtbi2XLluHBBx/Ek08+iV27dt3WOUrtybFjxxAfH4+YmBh89tlndmfEmUwmzJ07FyEhIZg4cSJOnDhhp2Pv3r0YOXIkwsLCsHTpUlRVVcn6zWYzkpKSEBoaCoPBgJ9//tmhNv0XqqurkZqailmzZuGdd96B0Wi0kykoKMDUqVMREhKCGTNmoLCwUNZvtVqxfv16jBgxAuHh4UhLS0NDQ4NM5q+//sLMmTMREhKCmJiYTvlyZlVVFZ599lnExcXZ9f2XOY6Ojm5yjrOyshAdHY3Q0FAkJSXJUp3vChxT4+zeIDU1lQAYFRXFpKQk9uvXj76+viwvL+/oobWYo0eP0svLiwC4efNmqd1kMtHPz486nY6JiYmcOHEiAXDdunWSzOeff27nDz8/P1ZUVJAkKysrOXjwYPbo0YOvvvoqY2Nj6eTkxOXLl7e7nU1htVr57rvvUqFQMDw8nOPHj6dSqWR0dLQkU1BQQK1WSx8fH7755puMjIykQqFgRkaGJLN48WIC4OTJk/n6669Tp9MxNDSUN27cIEleunSJer2enp6eTExM5Pjx4wmAGzdubHebb6auro5BQUFUq9WMiopiYGAglUolf/31V0nm8OHD7NKlC4cOHcoFCxZwyJAhVKvVPHPmDMlGP06aNIkuLi6cOXMmX3zxRapUKsbGxko6cnJyqFarGRQUxAULFnDEiBFUKpU8evRou9vcHPHx8QTAwMBAWXtbzPHq1asJgAaDgW+88Qb79u3LgIAAms3mdrXRkYgA00JMJhNVKhUXLVoklQqtqKigh4cHk5OTO3h0Leepp57ihAkT7H4ZXnzxRfbt25elpaVSW1JSEnv37s26ujqWlpZSpVJx8eLFMn9otVp+8MEHJMl33nmH7u7uLCwslHSsWrWK3bt3l4JQR2Iymdi7d29u2LBBalu7di0BsKSkhCQ5btw4Dh48mJWVlZLM1KlTOXToUFqtVv7+++9UKBRMTU2V+v/8808qlUpu2rSJJDlz5kz6+PjwypUrksy8efOo1+tZX1/vaDObxWKxcOXKlfz7779JNgaL0aNHc/r06VJ/QEAAJ06cKI21oaGBYWFhnDp1Kkly586dVCgU/OmnnyS9Bw4cIACePHmSJBkeHs6IiAjW1tZKP8dgMDAyMrLdbP03MjIyqFAoaDAYZAHGNsdr166V2oqKiqhUKqU/yl544YVm5/jixYt0dXXlkiVLpP6ysjJqNBp+/PHH7WBd+yACTAvZsGED1Wo1r1+/LmuPj49nSEhIB42q9VgsFtbU1BAAv/nmG6ntvvvu44oVK2Sy58+fJwAePnyYX331Fbt3727nj9mzZzM0NJQkOWDAACYmJsr6KyoqqFAouHXrVgda1XKSk5Pp5OTE69ev02w2U6lU8ttvv5XJHDp0iABYUFDApUuX0sfHhxaLRSZjMBj49NNPs6Ghgb169eLq1atl/b///jsBMCsry+E2/ResVivHjRvHadOmkSRPnDhBADx9+rRMLiUlha6urrRYLIyNjeXo0aPt9Pj4+HDhwoUsKSkhAB44cEAms337dgLoFE8AysrK2Lt3b7700ktcunSpLMDczhz37Nmz2TlOS0uju7s7q6urZTIzZsxgeHi44wxrZ8QeTAvJzMxESEiI3ZlDfn5+ds+j7yScnJykfRPb6cAFBQUoLS1FRESETNbHxwcKhQKFhYXIyspq1h8VFRXIzc2106HRaKDVajulz7KysvDee+/hueeeg1qtxvHjx1FfX49HH31UJufn5wcAKCwsRGZmJh555BG74+ZtfsjPz0dZWZmdDl9fX0lHZyA3Nxfz5s3DAw88gB9//BHz588H0Pi979WrFwYMGCCT9/PzQ21tLUpLS5GZmWlnn0KhkHyQlZUFhUKBESNG2OkAgKKiIgdadnvMmTMHTk5OeP/99+36bmeOy8vLm53jzMxMPPjgg3Bzc2tSx92CCDAtpLKyEu7u7nbtKpXqtgqHdWauXr0KAPD09AQA6eh5jUYjk7PVV7FVfGzOH7YN0Jt1/FOms2CxWPD+++8jIiICDz30EFJTUwH83w8326lSqQBA8kNzNt5Kh61wV2fxQ3p6OtatW4fTp0/DYDBIN/9bzbPtRnk734XKykqoVCp06dLlljo6kq1bt2LTpk348ssv0aNHD7v+27EPaH6O7+b7xz8RAaaFaLValJeX27WXlZVBq9V2wIjaDtsKxhZgbPbcbK+t0JpWq/1Xf9hWQzfLkOxUPisvL0dUVBSWLl2KFStWICMjA927dwdwaz/Y/KXVauHh4dGsH26lw2w2w2KxdBo/zJ8/H1evXsWOHTuwa9cuvPzyywDQrH1A46r3374LHh4eqKqqsivv/E8/dhTnzp1DXFwctFotdu7ciYSEBKSnp8NkMmH58uWSDa2d47v5/vFPRIBpIXq9HufOnbOrw56dnY3Q0NAOGlXbYKtFbnvc5enpiS5duuDs2bMyuezsbABAaGgo9Ho9zp49e0t/uLm5QafT2ekwGo2orKzsND6Lj4/H6dOncfz4cSQkJMgeg+j1egBAfn6+7Jrs7Gw4OzvjgQcegF6vt+u3yYSGhsLb2xvOzs5N6gDQafwAND4unThxIuLj46WUdb1ej/Lycly+fFkmm52djYCAAPTo0aNJHzQ0NODUqVPSdwVAk98njUYjrZY6gtraWowZMwbDhg2D0WhETk4O/vzzT1RVVWHbtm0wGo1tMsc2HbwpRf9uuH/I6OhNoDsV24bdwYMHpbaLFy+yW7du/PDDDztwZK3nyJEjBMCLFy9KbQaDQZauSzZu4Ht7e9NqtTIvL48AeOjQIan/77//pkql4kcffUSycQMzLCxMyjIjybfeeotubm6sqalxrFG3wbVr16hUKmXZQTczcOBAzp49W/pstVoZHR0tJTLs3buXAHj+/HlJ5vTp07JEhpEjR3LChAkyvTNmzGDfvn3b0pwWcerUKaakpMjaFixYQH9/f5JkdXU1u3Xrxk8++UTqr6urY2BgoJRptnLlSvbs2VOW8LFt2zYpOcBisbBPnz5cuHCh1N/Q0MDhw4czKirKkea1iJs3+TMyMlo9x9nZ2XZJHcXFxXR1dZX59k5HBJhWMGrUKHp5eTEtLY3btm2jr68vdTqdLDXxTuPQoUNMSEggAM6ZM4c5OTkkyX379hEA4+LimJ6eLr0fsGrVKunap556il5eXvziiy9k/rh69SrJxgwkZ2dnTpo0iXv27GFiYiIBMCkpqUNsvZn8/HwC4JQpU/j888/z6aefZnR0NOfNm8eqqiqS5Lp16+jk5MR58+Zxz549nDx5MgFw27ZtJMkbN25w0KBB9Pf356ZNm7h+/XrqdDoGBgayrq6OJLl7924C4KxZs5iens4XXniBAPjpp592mO02bAHyrbfeYmZmJlNSUujm5sZly5ZJMq+//jpVKhWXL1/OXbt2cfjw4VQqlczOziZJlpaW0sPDg8OGDeN3333HVatW0dXVlQaDQdLx4YcfUqlUcvHixdy9ezdHjRpFhULB/fv3t7vN/8aSJUtkAaYt5thqtfLxxx9n7969uW7dOm7dupV6vZ5eXl6dImW/rRABphVcv36dr776Kl1cXAiAo0ePZm5ubkcPq1XEx8dz6NChHDJkCIcMGcItW7ZIfRkZGfT39ycAenp68tNPP5WtRsxmM1955RXJHwaDgXl5eTL9WVlZHDhwIAGwZ8+eXLZsmfRyWkdTWVnJyMhIDhs2jOPGjWNMTAyff/55PvLIIywqKiLZeGPYvHkzvb29CYC+vr5SOreN0tJSTp8+nQqFggqFgjExMbJ3f8jGG5Cvry8B0Nvbm2vWrJH5siNZvXo1dTodAbB79+5MTEyUvZ9TX1/Pjz76iGq1mgAYEhIiW8mTjSnskZGRBMCuXbsyISFBln5ssViYlpbGXr16EQD79+/P77//vt1s/C9s2bLF7v2ctpjja9eucc6cOXR2diYAjh07VnpZ9W5BnKbcBjQ0NMBisdwzRYaqq6vh5uYGhULRZP/t+OPfdHR2SKKmpqZZG27cuAGFQgGlUnlLPZ3VD1arFVeuXIFGo7nlPFqtVtTV1dml2v6T2tpaKJXKW9ZHsfnRlol3p9EWc3w33z9EgBEIBAKBQxBZZAKBQCBwCCLACAQCgcAhiAAjEAgEAocgAoxAIBAIHIIIMAKBQCBwCC4dPQCB4G6grq4OxcXFsFgs8PDwaJfzpK5cuQIPDw+H/xyBoKWIFYxA0Aoee+wxDBw4EN26dUNAQACCgoLg4eGB+++/36HHrhcXF0On07W4zPCsWbOQnJwsfa6trUVQUBDOnTvXVkMUCMQKRiBoDUajETqdDqtWrUJgYCBcXFxQXFyMvLw86RRmR2A7Rr6oqAj9+/f/T9feuHEDX3/9NVJSUqS2n376CQUFBfDy8mrTcQrubUSAEQhagVarRUREBObOnduuP9f2VvjNRa+ao7i4GGVlZcjLy0NNTQ10Oh1ycnIAAJs3b0ZwcDAuXLiAfv36NVmrRCD4r4gAIxC0grq6umaP+Dhz5gzWrl2LlStXIjc3FyQxaNAgO7lTp06hvLwcDz/8MFxc5L+WRqMRu3btgq+vL0aNGoUuXbqgrq4OQGOBqvr6emRnZ6NPnz7o06dPk+OoqalB//79pcJvADB27Fg7ucGDB2Pu3Ln45JNPbst+gaA5RIARCFqB2WxG165dceHCBVy8eBGXLl3C5cuXERwcjIiICOzZswdpaWk4dOiQtFp45plnsHnzZjg7O8NkMiE2Nhb79+8H0HiDP3DggFSgLSMjA5MnT4a7uzsqKirg6+uLkydPSnVE8vPzMX36dFy4cAHOzs5IT09HZGSk3Tjd3Nxw7do1WK1WDB8+HFFRUXj77bcBNJbEDg4OxqlTpxAQEGBXaVIgaDEdd86mQHBnU1tbS6VSSQB2/+Li4kiSqampBMBx48bx2LFjUl2Uffv2kSRjY2Pp7e3N9PR0/vzzz+zatSuXLFlCkqyqquJ9993HhIQEWq1WVlRU8PPPP6fVauWlS5cIgEqlkhMmTOBvv/3GoUOHcubMmc2O2Xbdr7/+KrWlpKTQy8ur05zmLLh7ECsYgaCFODk5ob6+Hi+99BLGjBkDT09PeHp6QqfTSafrajQaAMD27duhVCoRFhaG/v3748iRIwgNDcWWLVuwdu1ajBo1CgAwY8YMHDx4EIsWLcKGDRtgtVqxfPlyKBQKaDQazJ49GwCkFcyAAQPw7bffSrovXbrU5FjNZjMuX76MH374ARqNBhqNBufPnwcA/PDDDxg8eDAKCgrg7u4OnU7nUL8J7h1EgBEIWkh1dTUAYOTIkVKAuJlu3boBaNwDsQUdnU6HsrIyFBYWwmq1Ijw8XJL38fHB8ePHAQBHjx7F+PHj4erqaqfXFmAWLVok6W1oaEBlZWWT44iLi8OOHTukz0FBQXYygYGB6NmzJ65evdrpygcI7kzEezACQQux3YStVustZWwJAA0NDdL/BQUF6NOnj5RqbDKZJPkzZ87A398fQGNQ4i2qaVgsFgCQrTZ69OiBy5cvNym/ceNGnD17Fi4uLli/fj1KSkpQUlKC9evXAwBOnDiBkpIS/PHHHyK4CNoMsYIRCFqIWq2Gu7s7iouLZe319fVwcnKCs7OzlEacnJyM4OBgfP/99zCZTIiJiYGvry9CQkLw2muvYcWKFcjJycHGjRuRnp4OAJg8eTKmT5+OJ598Ek888QSOHTuGNWvWYMOGDaipqQHw/xUSAOj1ehQVFYGkXZBwc3NDXl4eXFxcMGnSJOm6kydPIjg4GEOGDHGYnwT3MB27BSQQ3NlMmTKFAOju7k53d3eqVCo6OTlxzpw5JMn9+/cTALVaLQEwMDCQO3fulK7Pzc1lYGCgVFr47bffljbbrVYrFy9eLCUSKJVKzpo1i/X19czPz6e/vz/NZrOk68iRIwRAk8nU5FjXrFnD+Ph4Wdu0adOYnJzc1m4RCEiKkskCQauwWCw4cOAAjEYjrl27BhcXF6jVakRGRsLf3x/79+9HZGQkysrKoFarmyyta7FYYDQaodVqmzzDzGw2w2g0ol+/flL6clOQREFBAQICAtrURoGgpYgAIxA4kH379iEqKgrl5eXi7XjBPYfY5BcIHIgtAUBsnAvuRUSAEQgcSEBAACZMmCDbjBcI7hXEIzKBQCAQOASxghEIBAKBQxABRiAQCAQOQQQYgUAgEDgEEWAEAoFA4BBEgBEIBAKBQxABRiAQCAQO4X8krNr+/uC5wwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xkcd();\n",
    "plt.xlabel('Epoch #');\n",
    "plt.ylabel('Loss');\n",
    "plt.plot(losses);\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 再次进行评估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率: 91.0000 %\n"
     ]
    }
   ],
   "source": [
    "cnn.eval()\n",
    "correct = 0\n",
    "total = 0\n",
    "for images, labels in test_loader:\n",
    "    images = images.float().to(DEVICE)\n",
    "    outputs = cnn(images).cpu()\n",
    "    _, predicted = torch.max(outputs.data, 1)\n",
    "    total += labels.size(0)\n",
    "    correct += (predicted == labels).sum()\n",
    "print('准确率: %.4f %%' % (100 * correct / total))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch : 1/10, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 1/10, Iter : 200/234,  Loss: 0.0001\n",
      "Epoch : 2/10, Iter : 100/234,  Loss: 0.0001\n",
      "Epoch : 2/10, Iter : 200/234,  Loss: 0.0005\n",
      "Epoch : 3/10, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 3/10, Iter : 200/234,  Loss: 0.0001\n",
      "Epoch : 4/10, Iter : 100/234,  Loss: 0.0003\n",
      "Epoch : 4/10, Iter : 200/234,  Loss: 0.0001\n",
      "Epoch : 5/10, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 5/10, Iter : 200/234,  Loss: 0.0003\n",
      "Epoch : 6/10, Iter : 100/234,  Loss: 0.0002\n",
      "Epoch : 6/10, Iter : 200/234,  Loss: 0.0002\n",
      "Epoch : 7/10, Iter : 100/234,  Loss: 0.0001\n",
      "Epoch : 7/10, Iter : 200/234,  Loss: 0.0002\n",
      "Epoch : 8/10, Iter : 100/234,  Loss: 0.0008\n",
      "Epoch : 8/10, Iter : 200/234,  Loss: 0.0008\n",
      "Epoch : 9/10, Iter : 100/234,  Loss: 0.0005\n",
      "Epoch : 9/10, Iter : 200/234,  Loss: 0.0002\n",
      "Epoch : 10/10, Iter : 100/234,  Loss: 0.0006\n",
      "Epoch : 10/10, Iter : 200/234,  Loss: 0.0002\n",
      "Wall time: 1min 9s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "#修改学习率和批次\n",
    "cnn.train()\n",
    "LEARNING_RATE=LEARNING_RATE / 10\n",
    "TOTAL_EPOCHS=10\n",
    "optimizer = torch.optim.Adam(cnn.parameters(), lr=0.001)\n",
    "losses = [];\n",
    "for epoch in range(TOTAL_EPOCHS):\n",
    "    for i, (images, labels) in enumerate(train_loader):\n",
    "        images = images.float().to(DEVICE)\n",
    "        labels = labels.to(DEVICE)\n",
    "        #清零\n",
    "        optimizer.zero_grad()\n",
    "        outputs = cnn(images)\n",
    "        #计算损失函数\n",
    "        #损失函数直接放到CPU中，因为还有其他的计算\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        losses.append(loss.cpu().data.item());\n",
    "        if (i+1) % 100 == 0:\n",
    "            print ('Epoch : %d/%d, Iter : %d/%d,  Loss: %.4f'%(epoch+1, TOTAL_EPOCHS, i+1, len(train_dataset)//BATCH_SIZE, loss.data.item()))\n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEXCAYAAAAX7LteAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydeXgT5fbHv9nXJmlLNxBB2axQEAQRkU1QcOGiXFFRdnDBiyKK60+9KKJXUNGrV1C4gODFS1Uum6gFWRQUrIjI0rKWvXubpmn2ZH5/1JlOtknSpk2C5/M8fWhn3sy8M0zOmXPes4gYhmFAEARBEHGIONYTIAiCIIhgkJIiCIIg4hZSUgRBEETcQkqKIAiCiFtISREEQRBxCykpgiAIIm6RxvLkeXl5WLlyJSQSCR5++GHccMMNguO3b9+OZcuWQSQSYdq0aRg4cKDXfrvdjldeeQU33HAD7rjjDgDAr7/+ii+//BIymQx2ux0ulwtisRhqtRp///vfuc+Wlpbi7bffRlFREfr164dHH30USqUy+hcdxzAMg5qaGlRWVqKmpgZ1dXWoqalBdXU1KisrUVtbC7vdDofDAYfDAafTCYvFgrq6OlitVjgcDrhcLrjdbq/jikQiSCQSSKVSyOVyyGQySKVSyGQyyGQyqNVqpKSkQKfTISkpCXq9HhqNBgaDAXq9HkqlEkqlEhqNBnq9HjKZLEZ3qHlxuVwwGo0wm82oq6uDyWTi7q3VaoXNZoPZbEZtbS0sFgv343A4YLfbYbPZ4HQ64XK5uB+PxwOPxwM200QkEgEAd9/591ahUEAmk0Gr1UKv10Ov10On00Gn03G/p6enQ6/Xc8dJNGpra1FVVYW6ujrux2KxoLa2FrW1tdz9ZX9n76nNZoPdbofT6YTD4fB6xkUiEfdsy+VyqFQqJCUlcT/8+2cwGGAwGLjfk5OTL4nn2W634+LFi6iurkZVVRVKS0u559dms3HPqt1u555p9ll1u93weDzo3r07FixY4HfsmCgphmHw8MMPY8mSJRg+fDicTif69++P119/Hc8//3zA8TNnzsT777+PYcOGAQAGDRqEl19+Ga+88goAoLi4GH/961/x008/YeHChdxnT506hddffx1XXXUVtFotJBIJPB4PbrrpJm7MDz/8gNtvvx2tWrVCnz598Nprr2Hp0qX48ccfYTAYwr6umTNn4tChQ1CpVDAYDEhJSeGErkqlglarRXJyMvfApqSkICUlBRqNBlJpdP4rPB4PrFYramtrYTKZYLFYYDKZYDKZYDabUVpaitLSUpSUlKCyspLbV11djeLiYthsNsHji0Qi7svIfiE1Gg1UKhUUCgUkEgkkEglEIhFEIhEYhoHb7eZeEFjl5nK54HQ6OUVnNBrh8XjCukalUgmDwYDU1FRotVpoNBqkpKSgVatW3Jc/PT0dqamp0Gg0nJBghYNKpYq6kHU4HCgvL0dVVRUn4CorK1FZWckJO7PZjOrqaphMJtTU1KC2tpYTlGazGRUVFWHfAwBQqVRQqVSQy+VQKBRQKpXcCwD7IxaLuR+g/rvEPiOlpaWc8rNYLJxAdjgcgueVy+VIT09HWloa0tPTkZWVhYyMDGRkZECtVsNgMKBVq1ZITk5Gq1atYDAYoNVquTk0FYZhYLfbuRckVtGwL1jFxcUoKSnh/i0pKUFVVRX3fxEOCoUCWq0WKpUKUqkUSqWSU+JyuZx7xgHA7XbDZrNxL282m437/lmt1pDnUqvV0Gq1SEpK4u5pamoqUlJSoFarkZaWhlatWnHPul6vR3JyMqfwonFfGYaBw+GAxWKB2WyGyWRCeXk5qqurub/Za2JfXIuLi1FeXo6ysjKUl5cLHl8ikUCtVkOhUHDygv+sSiQSWCyWgJ+NiZLasGEDli5dig0bNmDkyJEAgEWLFuGJJ57AlClTkJGR4TU+Ly8P77//Pj7//HPcfffdAIDly5fjoYcewrRp09C2bVs8+eSTqKyshFqt9vos+/ePP/6I5ORkv7k4nU5MmDABQ4cOxerVq6FSqVBRUYFu3brhgw8+wIsvvhjRtdlsNhiNRhw+fBhGoxG1tbV+lkUgZDIZFAoF5HI51Go195arUCi4/0SxWAyPxwO3280JE6fTyQk5VtCEQiKRID09Henp6UhKSkJWVhays7ORmZmJrKwstGrVirNm9Ho9UlJSkJycDJ1OB6lU2ixv0R6Ph3ujNRqNqKurg9FoRE1NDWw2G2w2G2fZsW/DVVVVnNVx8OBBVFVVwWQywW63h7x+jUbDKVlWELGWnVgs5pQtKwDcbjfcbjenaNk5ORwOmM3msIQfK8BZKyUpKQkZGRnQaDRISkri/k80Gg23jf1Csz+sMFMqlVET+r44nU6YTCYYjUZOONXU1KCmpgalpaUoKytDWVkZKioqUFxcjEOHDqGsrAxOpzPoMUUiEfeCwAp6mUzGPeOs0BeLxRCJRJwF6HA4YLVaOeHJvoWHqkEgFouRnp6O1q1bIysrCzk5OUhJSUHr1q2RmpoKtVrN3We1Ws1Z8VqtFlqtNmrWjdvt9nopMRqN3H01Go2orq7m5ERtbS3Kyspw5swZ/PLLLzAajUEFt+991Wg03H1l5QirBCQSCTcX9hm22+2w2+2wWq2c9R5OXQepVMq98GVkZKBLly7o378/2rRpgzZt2nAvJxkZGdDr9Zwck8lkjZYbolhUnBgzZgysVis2bdrEbbPZbMjIyMBrr72Gxx57zGv8+PHjUVxcjK1bt3LbnE4nsrKy8Oyzz+Lpp5+G0WiESqVCRkYG/v73v2PWrFkAgE8++QQzZszAli1bsH79epSVlWHo0KG49957IZFIsHPnTgwePBgnTpxAhw4duOPPnj0bGzduxNGjR73mUlBQgMLCQojFYigUCu4tsWvXrgH/ExiGgcVigdVq5d6ka2pqYDKZUFFRgerqau5NkHWlsaYx62JgTWKGYTjXGf+Lzn6xWKtGrVZzrgbWktDpdNBqtUhLS0NqamrCumvCwWKxoKysjLu3rIDlC12z2cwJQNaCYH/YFwH2ngPgFBfr1mHdZHK5HFqtFikpKdwbLyvskpOTkZaWBo1G06xKJR7weDyce4d1+bCWJP/+s24e9gWLfcbZe83+sApLoVB4KWj2+WafdfZv9jlPTU3llP2lcL89Hg8qKio4K5DvgjcajaiqquJe6tjnl315Yj0WrHXOf4YVCgUUCgX34qTVaqFUKjnZwd7LlJQUaLVaTok3hxeCpXXr1hg1ahQWLVrktT0mltT27du91oOAehfOFVdcgZMnTwYcP3PmTK9tMpkMHTt25MYbDAburYXvoisuLkZdXR369euH3r17w2AwYPLkyfj++++xePFi7NixA507d/ZSUACQnZ2Nf/7zn3C73dybCADk5uZizpw5fnNs27Ytzp49i86dO2PQoEFYsmQJgPo3HfbL1KpVK7Rv3z6ie0VEjlqtRvv27eletyBisZhzP7Vp0ybW07lkYC3C9PT0WE+l2VEqlaitrfXb3uKvGgzDoKqqCmlpaX77dDodTCaT3/Zwx1dXV4NhGC93YWlpKRiGwfLly5Gfn48tW7bgo48+wtKlSzlfdbBjO51OP9dRMNcda5bL5XJUVVUFuXqCIAgiEGq1OuAaXotbUiKRCMnJyaipqfHbV1NTEzBQQWh8x44dub8rKysBwEtJ3XfffRg4cCDuuusubtvo0aMxefJkHDp0CMnJyTAajQGPza5Z8MnOzsaoUaO4YAB2YZE9t0qlCmuxlCAIgmggmOyMibuvdevWKCoq8tpmt9tx7NgxzJ49O6zxLpcLBQUFmDZtGretrq4OQL0VxNK3b1+/47lcLgD11k/r1q1x7tw5OJ1Or8XSQ4cO4ZprrvHzv44dOxZjx44Nem1yuTzkwj1BEAThTTDZGZOVxeHDh+N///ufV7jt5s2bYbPZcP311wccv379ei9X25YtW1BbW+s1ns1r4l8oGxbKZ+PGjZBIJOjbty9uueUWmEwmbNu2jdvvdDqxfv36gAouFGwEHkEQBBE+wWRnTJTUlClTUFRUhMmTJ6OgoAC5ubmYMmUKhg0bhk6dOgGAVyjxpEmTUFxcjPHjx+PIkSNYu3Ytxo0bhwEDBqBbt24AgLNnz2L37t0AgDVr1uD48eMAgMceewyDBw/GxYsX4XA4sH79esyaNQt33XUXMjIy0L59e4wYMQKPPPIIvv76axw8eBC33347zp8/j4ceeijiaxOLxWGFchIEQRANBJWdTIz49ttvmSuuuIIBwABgxowZw5SUlHD7VSoV07p1a+7vbdu2MR07duTG33XXXcz58+e5/ZMnT2YMBgOj0+kYnU7HvPrqqwzDMExBQQHTrVs3BgAjkUgYAMwdd9zBVFVVcZ8tLy9nHnjgAe7YV1xxBbNu3bpGXdeAAQOYwYMHN+qzBEG0HOerLUyV2R7raRB/EEx2xiRPisVut+PEiRMBw1b/7//+D1lZWZgxYwa3zeFw4MSJE9DpdLjsssvCPo/H48GOHTtQXl6Oa665Bl26dAk47vz58zCZTOjYsSPkcnmjrunGG2+EXC73ch8SBBE/lNfa8cbXBVj76wVcnaXD5pkDYj0lAsFlZ0yV1KVI//79oVQq8d1338V6KgRBBODuRT/ilzPV3N+n/3F7DGdDsASTnYmfkh1n+Cb/EgQRXxTXNJQOa2NQCYwkWpJgspOUVJSx2+1QKBSxngZBEEFwuhsiyORSEoHxQjDZSf9DUcZms/3pWnwQRCLh9jSscMglJALjhWCyk/6HooxvUjBBEPEF35JSyck1Hy8Ek52kpKKMw+FodGQgQRDND8+QQrKaXijjhWCyk5RUlCFLiiASB4OaXijjBbKkWgir1epXlJYgiPiBX41TKSN3X7wQTHaSkooibOO3SFrOEwTRsvBrRicpY1Jjm/BBSHaSkooibAtmvV4f66kQBBEEsbhBS+lV5JqPB4RkJympKML2pSIlRRDxi4RnSpElFR8IyU5SUlGkoqICAJCamhrjmRAEEQy+JaVVkJKKB4RkJympKFJdXV8PjJQUQcQvPB1F7r44QUh2kpKKIuzbQEpKSoxnQhBEMKTiBrFHllR8ICQ7SUlFEdavmpycHOOZEAQRDAnPlErVUp3NeEBIdpKSiiIWiwUAoNFoYjwTgiCCwS8qm0ZKKi4Qkp2kpKJIaWkpZDIZdDpdrKdCEEQQNDwXn5ai++ICIdlJSiqKlJaWIj09HWIx3VaCiFf4gRN81x8RO4RkJ0nTKFJcXIzMzMxYT4MgCAE81Is87hCSnaSkokhZWRmysrJiPQ2CIATwkJaKO4RkJympKFJeXo5WrVrFehoEQQjAgJRUvCEkO0lJRQmGYVBWVob09PRYT4UgCAFEoHWoeCKU7IxpaMsPP/yAVatWQSwW46GHHkKvXr0Ex+/ZswfLly8HAEydOhXXXXed136n04k333wTffr0wfDhw732HTp0CJs2bYJMJsOoUaPQsWNHbt/HH3+MqqoqMAwDu90OkUgEi8WCMWPGoHfv3mFdS01NDRwOBykpgohzxBQsEVeEkp0xsaQYhsETTzyBgQMHorCwEAcOHMC1116LhQsXBv3Ms88+i379+uHw4cM4fPgw+vbtizfeeIPbX15ejltuuQUvvfQSCgoKuO0ejwfPP/88cnJy8PHHH2P+/Pm46qqr8OWXX3Jjli5dirlz5+KTTz7B2rVrkZubi127dnGx++FQVlYGAMjIyIjkVhAE0cK4eO3jidgTUnYyMWDTpk0MAGbNmjXctoULFzJKpZIpLy/3G79161YGALNy5Upu26JFixiZTMZcvHiRYRiGGTt2LNO+fXtGpVIxCxcu5MZ98cUXjEajYT777DPG4/Ewdrudufnmm5nu3btzY4YMGcJMnjy5SdeUn5/PAGA2bNjQpOMQBNG8DH17B9Pu2U1Mu2c3ReV4i3ecYJ7+/DfmbGVdVI73ZyOU7IyJJbVy5UoMHz4c99xzD7ft4YcfhlQqxZo1awKOHzRoEMaPH89tmzJlCrRaLVavXg0AePfdd3H48GHI5XJ4PA1vSnfeeSfOnz+P++67DyKRCHK5HMnJyZBIGjpyVlRUIDMzE5988gmeeuopvPjiizh37lzAuRcUFOB///sf1q9fj2+++QZ79uxBaWkpampqAFCbDoKId6wOd1SP98H2E8j95TwGzN+O9b9diOqx/wyEkp0xUVLbt2/Hbbfd5rVNpVLhyiuvxIkTJ8IaL5fL0alTJ258eno6FAqFX3dHiUTi9feWLVvw5ZdfYvTo0dy24uJizJ8/H48++ij27duHTz/9FNnZ2SgsLPSbS25uLkaPHo0777wTt956K/r164f//ve/MJlMAICkpKRG3BGCIFoKhSy6Ys/CU3qf/Xw2qsf+MxBKdra4kmIYBhUVFQEXyfR6PadV+YQ7vrq6GgzDBPRtut1uzJ07FyNGjMDtt9+Op59+GgDgcrlQUVGBDh064OjRo9ixYwcKCwtxxRVX4K233gp4nEDzYG80lUQiiPhGI49uvNhlySrud6ebwtsjJZTsbPHoPpFIBIPBEFAZ1dTUBDT5hMZ36NCB+7uqqgqA/wJcaWkpxowZg19++QXvvPMOHnvsMa78hkQiweuvv45x48bhsssuAwAolUqMGjUKmzdv9jtndnY2Ro0aBbfbDbvdjurqalx22WU4cuQIN1eCIOIXD9OgSFxuD6SSpr2rK6UNSwcUlBE5rGwPJjtjEoKelZWFM2fOeG1zOBw4fvw4nnzyybDGu91uFBYWYsqUKdw2s9kMwNu36fF4cMstt8BqtWL//v3o0qWL13FEIhGef/55v3MyDMMdj8/YsWMxduxYv+0//fQTALKkCCLeMdtdXr8b1PImHU8l5ykpqmYRMaySCiY7Y7Imdcstt2D9+vVgeG80eXl5sFqt6Nu3b9Dx/ICI7du3w2QyeY1XKpUA6hUey48//ojff/8dn376qZ+CYuGPB+oV2zfffBNwLsEwm82Qy+WQyajTJ0HEM/w1pFqbS2BkeGgUDUrKTUoqYkLJzpgoqSlTpuDo0aOYPn06Tp8+jU2bNmHy5MkYNGgQp0hsNhtcrvoHaNKkSTh37hymTZuGoqIifPPNNxg3bhz69u2LHj16AABKSkqwb98+AMC6detw+vRpAMDBgwchk8lw8OBBvPnmm3jxxRfxwgsvYNu2bQDqXYR6vR7Lly8HwzCoqanB9OnT8euvv+LBBx8M+5qcTicpKIJIAByuhpfdGquzycdTyUhJNYVQsjMm7r6cnBysXbsW06dPx0cffQQAuOOOO7B48WKIRPXZ4K1atUJqairOnDmDLl26YP369Xj44Ye5ihPDhw/HRx99xI1/+umnkZubC6lUijlz5sBut+PVV19Fp06dIJfL8cwzzyAlJQVarRYejwdr165FYWEhUlJS8PTTT+Phhx/GY489hrq6OiQlJeGDDz7AwIEDw74mu93OWXIEQcQv/AKzFWZ7k4+nJCXVJELJzpiVRfrLX/6CoUOHoqCgAAaDwatMEVBf9ohfFffWW29FYWEhCgoKoNPp0KlTJ6/xq1atwqpVq/zOM2zYsIBrS3xeffVVTJ06FTt37oRKpcKIESMiDiWvq6uDWq2O6DMEQbQ8/HWjaFtStCYVOaFkZ0xr92k0mqC18d577z2/bWq1Gtdee22zzKVdu3aYMGFCoz9vs9nIkiKIBIBv7URjTUotJ0uqKYSSnVQFPUrYbDaoVKrQAwmCiCn8Vh0mW9MtKX4Len4wGBEeoWQnKakoYbFYSEkRRIJRZ2+6JZXcxBD2PzuhZCcpqShB0X0EkXhYolDHL0kZ01WThCeU7CQlFUXYKhYEQSQGNZamu/vUvDJLbLQxERlCspOkapQgXzRBJAZiniKJ9poUvadGTijZSbeUIIg/FVJeZ16bs+m19vSqBleVhCypqENKKkqIRCKvsk0EQcQnEp6SsjiaHjiRplUEPDYRHqFkJympKCEWi0lJEUQC4K2kmh44oeNZUnJeRXQiPELJTlJSUYKUFEEkBhLewlE0lBQ/mVcuJZEaKaSkWgipVMoVxCUIIn6RSxosKXMU8qRkvH5Udmd0W9P/GQglO0lJRQlSUgSRGPALwkZDSfGJRi3APxukpFoIUlIEkRjwXXIOlyeq9faiUQvwzwYpqRZCJpPB6aS3KIKId3zXjfj9pZoKxfZFTijZSUoqSiiVSthstlhPgyCIECh8lJQ7ion4Hkrqj5hQspOUVJRQKBSw25veQI0giOaFvyYFRLe9hoyi+yImlOykOxol5HI5HA5HrKdBEEQIVD5KykM9oGJKKNlJSipKqNVqWK3WWE+DIIgQaBTeVcuj6e7zVYBEaELJTlJSUYK90ZTQSxDxjVbRfK01+Im9RHiEkp2kpKKEWq0GAAqeIIg4R61oPkXCT+wlwiOU7KQ7GiWSkpIAALW1tTGeCUEQQvi65KIZNk4FZiMnlOwkJRUltFotAMBsNsd4JgRBCKGRe7v7pFG0fsiSipxQsjOmfY/z8/Px6aefQiKRYNq0abj66qsFx+/fvx8rV64EAEyePBndu3f32u92u/Huu++iZ8+euOmmm7z2bd26FevWrUNSUhKmT5+Oyy+/3Gt/bW0tPvzwQ5w+fRrXX389xo0bB4kkfLeAUqkEAAqeIIg4R+vT7l0maZr143Q3rKX45mARoQklO2N2R59//nlcd9112L17N7Zt24acnBwsWrQo6Pg5c+bg2muvxc6dO7Fz505cc801ePfdd7n91dXVuOOOOzB79mwcOHCA2+7xeDB+/HjcfPPN+O2337B27Vp07twZ69ev58b8+uuv6NSpE95++20cP34cjzzyCPr374+6urqwr0elUgEgJUUQ8URVnQPzvjqCvq9vxZYjpQAAnVLmNUbRxPYa/Hp90cy5+rMQSnbGREnl5eXhH//4B1asWIH8/Hzs378f8+bNw+zZs1FVVeU3/vvvv8crr7yCRYsWYd++fdi3bx/eeecdPPfccygtrX/wZsyYgd9//x1KpRIiXnfMZcuWYc2aNfj222+xa9cuFBQUYOrUqZgxYwacTifcbjceeOABdOvWDceOHcPWrVtRWFiI48ePY/HixWFfEykpgogvTpTVYtD87VjyQxFKTXb8eLICAKCUNYg9iVjU5HUko6VBSdlcVAU9UuJSSa1YsQJDhw7FxIkTIRKJIBKJ8Pjjj0MkEmHNmjUBx/fv3x8PP/wwN3769OlQqVRYvXo1AGD+/Pk4cuQIlEqlVyjj8uXLMX78eNxyyy0A6nuXPPvsszh//jy2bNmCvXv3orCwEO+//z4MBgMAoF27dhg3bhyWLVsW9jVpNBoAiMj6Igii+Thwrga1vCrnbNt4fp5UNPKaymsbqiU4XWRJRUoo2RmTNalt27bhueee89qmVqvRoUMHHD9+POD4Rx55xGubQqFA586dufFt2rSB2+1GTU0N9Ho9gPqL3rt3L2bPnu312csvvxx6vR7Hjx+H2WxGhw4dkJ2d7TUmJycHixcvhsfjgZjXJK2goACFhYUQi8VQKBQwGAxo27YtdDodAIruI4h4wdf1xrr1+HlSepW3668xlJsblBTV7oucULKzxZUUwzAoKytDZmam3z69Xg+j0ei3PdzxRqMRDMNwY6uqquB2uwU/W1VVFXS/w+GA1WrlND0A5ObmYs6cOV5jJ06cyG0jS4og4gMG3gqDdfOpeAm3BnXTlZTR0lDSh1RU5ISypFrc3ScSiWAwGGAymfz2mUwmTqvyCXd8ZWUlACA9PZ37HDsu2GeFji2VSjl/KYvb7e9zrqur48IoSUkRRHzCFpY18Kwn32KzjYHfgp4sqcgJJTtj4u7LzMzE2bNnvbY5nU6cOHECM2fODGu8x+PBsWPHMHHiRG4bG2fPKietVgu1Wu332ZKSEhiNRvTo0QNHjx7FuXPn4Ha7vULOCwsL0b17dy9XHwBkZ2dj1KhRcLvdsNvtqK6uhkajgcFggFgsRllZWSPuCEEQ0Ubkk6bLuvkMajm3LRrJt9UWKizdFELJzpgETgwbNgwbNmwAw3vr+O6771BXV4e+ffuGNf77779HdXW113iFQgEAXAMtkUjEfZbP+vXrIRKJ0Lt3bwwbNgxGoxG7du3i9rvdbmzYsAHXXXed31zGjh2LdevWYePGjcjLy0N+fj5WrFgBqVSKVq1akZIiiDhB7KOAkv4IPecrpmg0PDTWNUT3Ub2JyAklO2OipCZPnozDhw/jqaeeQllZGbZt24YpU6bghhtu4AIYXC4Xp5QmTpyIoqIizJgxA6Wlpdi5cyfGjx+PXr16oWfPngCAiooKHDlyBACwefNmXLhwAQAwbdo0bNq0CfPnz0dlZSW++OILPPPMM7j33nuh0+nQqVMnDBw4EI8++ih+/vlnXLx4Effffz+OHz+OqVOnRnRdWq2WAicIIk6Q+igpfug5CxMF91ydoyGCUCwiNdUYhGRnTJRUz5498dlnn+HTTz9FRkYGhg4diuzsbPznP//hcpxSU1PRsWNHAEDXrl2Rm5uLL7/8EpmZmRg8eDCuvPJK5ObmcuP/9re/4Z577gEAPP300/jggw8AACNHjsT777+PuXPnolWrVhgzZgyGDRuGf/3rX9x8/vOf/6Bt27bo27cv2rRpg+3bt2P58uXo3bt3RNel0WhoTYog4gSpTyWJQEm7rigk39bZSUk1FSHZGbOySPfccw+GDx+OAwcOIDk5GTk5OV777777brRu3Zr7+6677sLQoUNx4MAB6PV65OTkeCXtrlmzBqtXr+bejPjrSzNmzMC9996LI0eOoHXr1ujUqZPXuS677DJ8/fXXOHDgAGpra9GzZ09uMS8SNBoNLBZLxJ8jCCL6SH3WkxUBLCmbs+nJt1W8ZF5fFyMRHkKyM6a1+/R6PQYOHBhw37///W+/bTqdDgMGDAh6PKFae2lpaRg0aFDQ/SKRCNdcc43AbEOTlJRE7j6CiBP83H0BLClRFCyfugAJw0RkCMlOqoYYRfR6PWpqamI9DYIg4B9eHihxNxq19uy8UkjUqqNxCMlOUlJRRKfTkZIiiDjBt7mhb/XzaMEvhdTUiup/VsOdnqwAACAASURBVIRkJympKJKcnBywYgZBEC0P370nEYu8knhZkqKguKy8da2mVlT/syIkO0lJRRGtVguLxeJV4JYgiNjAL3+kV8kCBjVEo/+ThReCHijMnQiNkOykOxpF2OZdNpstxjMhCIKvMIIVkm3qkpTHw8DpbjiInJoeNgoh2Ul3NIpQ/T6CiB/4bThSNfKAY9TyprnnHG7vN39y9zUOIdlJSiqKpKamAgDKy8tjPBOCIPjuvmSekopGlQkW3+hAuYREamMQkp10R6MIe6Orq6tjPBOCIPgKgx8gwa9azs9xagy+Vc9pTapxCMlOuqNRhDVZ2WrsBEHEDn6iLl+X8JWUzdm0ICffdX5+118ifIRkJ93RKJKUlASAuvMSRLzBLyzBj8ZzuhuvpLYfLcPOo97uqVgpKY+HQbHJhjYGVejBcYiQ7CQlFUVSUlIA1FdkJwgifuAHNPDzmnwDH8LB4nBh1prf8O3h0gDniY1zav63R7F450n86/5euL17Vkzm0BSEZCe5+6JIWloaAAqcIIh4gx/PUGdvUFIud+RBFN8fqwiooICGxootzeKdJwEA878tjMn5m4qQ7CQlFUXkcjm0Wi2qqqpiPRWCIHjw16RMtoaq5a5GJN4LVU5vrtJL4dJKq4jp+RuLkOwkJRVltFotBU4QRJzB7/NksvKUVCMsKX5BWV9iZUmxXJ2li+n5m0Iw2UlKKsrI5XI4HI5YT4MgCB785oZGXv8neyPaxzsEFJsuSGWLluLKNE1Mz98UgsnOiJSU0+mkyLUQKJVKKotEEHEAP2nXa02KF93XmMAJu4C7T6eMrZJKS0pMdx8QXHaGraTefPNNaDQarjHghQsXMHbsWLRv3x7PPPMMFVX9A1JSBBEf8K0kOy8fysbLk2pM9QkhxRaNquqRwr+GVM2fWElpNBo4nU6sW7cOADBhwgRs374dY8aMwYIFC7B+/frozTaBIXcfQcQHtbYGi6ma5+Kr81JSkR/XI1CV1rfRYktQy6uakZYUuEZhItBkd9/o0aMBACdOnMCpU6ewbds2fPzxx1iwYAGuvvpq7N69O3qzTWCkUilcrqaVWiEIounwk3Yr6+zc7/w8KZeHwao9Z6J2TlUMlFSVuUGwJ6sTV0kFk51hK6nWrVvj+uuvx9y5c/HFF19AKpViyJAhAACZTAa73R7iCH8OJBIJ3O7gPmuCIFoGB8/dV8OzpPjuPgD4cPsJ7neGYVBpbrwsi0Vn3hpetGKsQ+CbQjDZGVHgxBtvvIFTp07h2WefxV/+8hckJSXhzJkzOHLkCHr27Bm1ySYyEomE1ucIIg7g1+WrsjRYGyab99t6VV3Dvrwjpbj2ta34YNvxRp2TXy+wpSivbVCqiVyFPZjsjEjtDh48GEePHkVBQQH69u0LoN79d/PNN+P+++8P6xgejwdr1qxBfn4+OnXqhMmTJ3MNr4KN/+KLL7Bnzx5ceeWVmDp1KlQq7/pUxcXFWLp0KUwmE0aPHo1+/foBqH8r2r17N1wuF+RyORQKBRQKBaRSKUQiEdq2bQulUomVK1fC7XbD5XLBbrdDIpHAbDZj/PjxaN26dSS3iCCIOMHMW6upsTrh9jCQiEV+eU783Kbc/HMAgLfyjmHGTZ0CHjcWikiIcp7lF29ziwZhKymHw4HCwkJ4PB4MHjwYQL0C6d27NzZt2hTWzamursaIESPw22+/4dprr8WyZcuwYMEC7NixA5dffrnfeJPJhNtuuw0///wzevfujRUrVmD+/PnYsWMHrrzySgDAF198gcmTJyM5ORlpaWl46623MHPmTLz77rs4d+4chgwZEnSN6OWXX8ZLL72Ep556CmazGTqdDnK5HG63G5mZmbj55psjVlIejwdSaeKa3ARxqWB1NnzvGaZ+XSo9SelXMYIfBRhOnpMkQBv6WFJSc2lEEweTnWHbhi+88AJ69OiB++67DwCwb98+5OTkwGAwYMSIEbBarSGPMWvWLBQXF+PAgQP48ccfUVRUBJ1OhxdffDHg+GeeeQanTp3Cr7/+yo3PzMzEc889B6Degpo4cSLGjRuH48ePY9++fdiwYQPee+89/Pzzz7j88sthMplQXl6OCxcu4PTp0zh06BDatm2Lq6++Gk8++SSkUik3B3ZcSUkJfvvtN/Tq1Svc28PhdrshkVB3ToKINQ6fRF220oTvdr5lVWoKLfClcaakqi2XRjRxMNkZtpJi86N27NgBhmEwceJEyOVyrFq1Ctu2bcN///tfwc/X1dXhs88+w6uvvoqrrroKAJCcnIyZM2fis88+80sSttvtWLVqFV5++WV069YNAKDX6zFr1ix8+eWXqKqqwpo1a6BSqbBw4UIoFPX5ASNHjkSvXr2wdOlSAIBKpUKrVq3QunVrtGvXDkuXLkVVVRXWrl0LvV4PoL7ybmZmJj7//HPMnTsXH330ESwWS7i3xgtSUgQRH/hWhmDXqJw+251uBq4/cp8qwgiaiDdLymy7NKKJg8nOsP1SI0eOhFQqxVdffYU+ffrg8OHD+P777zFgwAC8/fbb2L9/PyZPnhz087t374bD4cCoUaO8tvfo0QMulwunT59GTk4Ot33v3r2wWCwBx3s8Hpw8eRLbtm3D8OHD/da0evTogWPHjvnNobCwEP/85z/x1ltvoUuXLgDqlafZbMYTTzwBl8uFq666CidPnsSCBQuwb98+TpGxFBQUoLCwEGKxGAqFAgaDAWq1Gm3btkVycjI8Hg/E4sRdvCSISwXfyhBsF95AybgOtwdSiRhltaGVlDTOghMq6y4NSyqY7AxbSen1egwfPhxz5szBbbfdBrVajeuuuw5AvdUTah2mpKQECoUCycnJXtvZv33bBpeUlAAAMjIygo4vKSnhlI3vmEDVdP/+97+jbdu2+Nvf/uZ3ns6dO2Pz5s3IysrChQsXkJOTg6VLl+Kpp57yOkZubi7mzJnjd2y9Xg+j0Qin0wmZLLalUQiCANw+Sbdscm+gZFx2LFvXTy0P7g1RxyAXSgh+VfdEJpjsjOiV4J133oFer8fHH3+M6dOnQ6FQYN++fSgsLMSAAQMEP6vX62G32/0yik0mE4CGzoz88UC9pRNsvF6vD1hL0GQyQafzrgZ87tw5fPHFF5g1axbk8oaEt/T0dDz++OP45ptvkJVV3yysTZs2GDZsGPbu3et37GA5UOzNJSVFEPGBy0cZsUm87gBlJnz1VmuBDrfx1iLe6rg08jKDyc6I7nbnzp1x4MABmEwmzqKRSqV4/vnn/dxyvmRmZgIAzp8/z0XmAcDRo0chlUq5dSrf8efOncPVV1/tNV4kEqFr167IzMzE2bNn/c519OhRdO/e3WvbkiVLoNPpMHXqVK/tSUlJeO+99/yOIZPJUFlZ6bc9Ozsbo0aNgtvtht1uR3V1NcxmM6d8XS4XKSmCiAM8PsqIdff5bgf8a/hl6YOnxejiLGFWqL9VIhFMdkbsXBWLxSguLsbXX3+NvXv3olOnTpg3b17IdZiePXtCr9dj48aNXts3bNiA7t27++U+devWDWlpaQHHZ2dnQ6fT4aabbsL27du9rKnS0lLs2bMHffr04bY5nU4sXboUkydPhlarDXmNdrsdu3fvDpigPHbsWKxbtw4bN25EXl4e8vPzUVBQgJMn6ztjWq1WwbwvgiBaBl93H5s3FSjXftS/dmPfmYYlgrYp6qDHVceZJeUbCJKoBJOdESmpU6dOoVevXujatStGjhyJG264Ae3atcOSJUtCflYul2P8+PGYN28etmzZAovFgjfffBOrV6/GlClTuHFsxrFEIsHEiRMxf/58fP3117BarVi4cCGWL1/OjR81ahSUSiUmT56M0tJSFBUV4c4774RGo8Fdd93FHXPjxo0oLi7GuHHj/OZ16NAhtG7dGvn5+QAAi8WCadOm4fz585gwYUIktwdA/Y32VbgEQcQei4Bb7EylBet/u8j93SEt+MusVhFfa1KXSv5uMNkZ0SvBE088AaPRiG+//RY333wzrFYr3nvvPTz00EPo0aMHF0gRjNdffx21tbW45ZZbANS71GbOnInp06cDAH755Rf06dMHa9aswT333IM5c+bAaDTitttuA1CvuKZPn46ZM2cCAFJSUrB+/XpMmTKFcw9eeeWVWLt2LQwGA3fe1atXo1evXgEto86dO+O6667Dddddh/bt2+PChQuQyWRYsmQJsrOzI7k9AOqTnvlrXgRBxAe++VG+8GW9kLvPEGdFXGVxFm3YWILJzoiU1JEjR/DXv/6VUzJqtRrPP/88li9fjtWrV4dUUklJSVixYgVmz56Ns2fPIicnB23btuX2t2nTBtnZ2ejYsSOA+vYgS5YswZNPPomioiJ07doV7dq18zrmwIEDcfDgQfz4448Qi8Xo37+/34U++OCDyMjICFgVQy6XY926ddi1axf27NmDlJQUjB492kvJhQvDMKirqwvLpUgQRPMi9vm+s6HnwSyPCl4ot0Gg8gR/n1wiblTjxGiikCa+khKSnREpqaFDh2L16tWYNWsW2rRpA6A+hLu0tJT7Oxy6devGJejyycrKwpEjR/y2Z2dnC1o1KpUKQ4cODbp/+PDhIed044034sYbbww5Tgir1Qq32+0XqUgQRMsj9km6ZfOmghVhvWhsqJqTJNBhl5/Mm5akwAVj6Go7zUm8RRs2BiHZGZEKfvnll6FQKNChQwfcfvvtuPfee9GtWzekpKTgwQcfjNqEExU2PN43/J0giJbHt3wRW6PPV3mxlJkaEnk1AutO/MK1vsEZzhhYVbFoDxJthGRnREqqTZs2OHToEBYsWACdTgez2Yxnn30Wv/zyS6PcY5caRqMRAOheEEQcIAni7gu2hlNW21C3Ty/g7uO3xvB19dXZW75EkVwaX4EcjUFIdkZsJ2o0Gjz22GN47LHHANS7+/bs2YPbb7+9idNMfGpqagDAr5QSQRAtj2+NPdbqkQexPPih3EIuNH4/Kt/SS1anGy39iqqSNShdh8uDLUdKMeSqNKjlieMGFJKdTV5x+/rrrzFy5EhcvHgx9OBLHNZkJSVFELFH6qOM2HzdcKLhhIIR+BUe6nzC2mNR7DVF0xAo9sSa/fjb6l/x7tbGNW2MFUKys8lKasCAAWAYBlu2bGnqoRIetoSTRqOJ8UwIglD51Nhj7SR5GNFwQv3xhCo88Fu5txTJvJD4zQfra5F+9Xtxi8+jKQjJziYrqQ4dOqBt27b45ZdfmnqohIcto+RbRJcgiJbH12XHlj7SNjEaziyw7iS0r7kwqP3Xz0wxUJZNQUh2NllJsXX0ArXG+LNRVlYGwL9yO0EQLU8wiymQUI8EoYKusSj2mqpRcL+zdQVrY6Asm4KQ7BR8pbBYLPjqq6/AMAwkEgkkEgkUCgXUajU0Gg20Wi3S0tKQnp6O33//vXlmn0AYjUYoFAoqi0QQcYAySNSbb3X0SBFK3vVdo2oJ+C3vk5Qyr8COREFIdgoqqX379uHee+/1qxAciEGDBjV+hpcIgVqEEAQRG3wtJjbar6kFWX1zo/jUxqC3E99ibKorM1YIyU7BKxowYACcTidcLhfcbjfXnsJiscBiscBsNqOmpgZ1dXW4/vrrm2XyiURFRQVSUlJiPQ2CIACofBoXskrKHagMegQIJewKFbFtLvjJvBl6JY6W+vfYi3eEZGdItcu6+Vio5E9wqqqqkJqaGutpEAQBf6tC9kc7odomusPsAoVqY9HbScmLYszUKQRGxi9CsjPxKxPGEXV1dRR+ThBxgkIqBj+fVyJhLammuft8E3j5xEJJ8ZswJsdZhfZwEZKdpKSiiNlspgroBBEniEQiKHjBE2wtvzCW2AURCryIReAEv7KEbwJzoiAkO0lJRZHKykpakyKIOIKfK8VWmrA20dpxC2i5WISg8+sM8tuTuGLcQiQShGQnKakoYjQaSUkRRByRzIvwY6PgmrNSeSwKzKp5ASL8ihclJlug4XGJkOwkJRUlnE4nbDZbQgeWMAyD3ScqsOT7U7C7Wv6NkCCiTRJvvUb2h7svnLJIjSUWZZH4JZyO8SL7qnhNHOOZULIzMYPq45BEr4B+osyMF9YexM+nqwAAnTK0GNwlPcazIoimwW9eyLr7fFt4RJOmRg42lbOVFu53oyUxSiOFkp1kSUWJRC8uu2jHSU5BAYCriQmPBBEPpPIqhMv+sKCa+mT7tqXn09T1rqZSZWmwnhLFkgolO0lJRQmbrd7/q1QqYzyTxuFb6kWSoFFCBMFHzeuwy67dhJNwK1Rlx7fjLx+HQA5Vc6L8o6eUzdlw/kRRUqFkJympKJHoSsoXeRg9dwgi3uG362Cj4CyO0C45odJJQv2ohBJ9m5O0JP8k3moLKSmCR6KvSfkSTmM4goh3tIqGNSnNH/lE4Vg7QkVkfTv+en0uRgFH7VP9XWWXyppUTAIntm/fjm3btiEzMxOTJk0KuY7zww8/YMuWLUhLS8OkSZP8okDq6uqwYsUKlJSUYOjQoRg8eDC37/Tp0/j+++8hlUrhcDjgcrkgFouhVCpx//33c+PsdjtWrVqF06dPo1+/frjtttsEG5/5YjQaAQAGQ0s3j44Ovt8734ZxBJGIpGob1qTYSL9wQtBrrM6gxVqF1qSaWMyi0WTo/K0QUwyK3TaGULKzRZWU0+nEAw88gM8//xzdu3fHuXPnMG/ePOTl5aFbt25+410uFyZNmoT//Oc/yMnJwYULF/Daa6/hm2++Qc+ePQEAe/fuxejRo2GxWHD55ZfjtddewwMPPICVK1dCLBbj+++/x8SJEyGXy6FUKiEWi+HxeHDLLbdwSqqgoAAjR45EaWkpOnbsiDfeeAPDhw/HunXrIJeHV2Yk0QMnpGJvy4nvyyeIRCVL3yC8tcrwLalKsx1tDIFb7gitSXmaWs6ikVyW7D/XWCQWN4a4CpyYP38+Nm/ejO3bt+PAgQM4ffo0srOz8fjjjwcc/9577+HLL79EXl4efv/9d5w5cwa9evXC3/72NwD1vsx77rkH3bt3x6lTp3DgwAH88MMPWLNmDdatWwcASEtLAwBcvHgRNTU1qK6uRk1NDT7//HMAgMfjwf3334/09HQcP34c+/fvx/79+7Fr1y78+9//DvvaWJM1US0pudT7i5eoJf8Jgg8/BF33x++2MJSUkKtMqPRQjHQUrmjlL+Cb2jerpQglO1tMSTEMg48//hgzZ87k3HE6nQ7PPfcctm/fjhMnTvh95uOPP8ajjz6Km2++GQCg1Wrxwgsv4KeffsKhQ4eQl5eHCxcuYMmSJVzb4RtvvBG33XYblixZAqC+BLxarYbNZsPy5csxf/587Nu3jztHfn4+fvvtNyxatAiZmZkAgO7du+O+++7jjhEOtbX1SXSJmszra0nplE3rXkoQ8QA/mVcll8DjYcKypDYfLMbKn04HjPJTBGmmGEvStP6BE7Gy6iIllOxssdflkydP4uzZsxg9erTX9l69egEAjh49io4dO3Lbz58/j2PHjgmO/+GHH9CnTx9cdtllfmNWrFgBACguLobT6USHDh2gVquh1Wrx7LPP4o033sBzzz2Hbdu24YorrkCPHj38jvHpp5+CYRivtamCggIUFhZCLBZDoVDAYDAgJycHJpMJYrEYarW6aTcqRih4WfhiUUNIK0EkMnyPgEYuFQyI4PPf/HMAgL5XpKJLprfwVDRjxYrGkpTAL5WhZGeL3e2LFy8CANq0aeO1nTXxKisrwxqv0Wggk8lQWVmJixcv+u0HgOTkZO54rJKaPXs2ysrKcPbsWcybNw9z586F0WgUPIbVauXCI1lyc3MxevRo3Hnnnbj11lvRr18/nD59GlVVVTAYDBCL4+8BDgelT6huJEEjBBGv8AvMKmXiiK2LQEEWSoGgIoHlqmbFtwsxILx2Fk+Ekp0tJlHZ1sBms9lrO/u3b5n2YOPtdjucTie0Wi10Op3ffqDefGSPN3jwYMyfPx9z586FVFr/wE6bNg0WiwUHDx4UPIZEIvGL3Xe7/Rcj1Wo1LBZLwlpRgHfIeaL2pCEIIRgm8jWjQHX+1PLgSkooPL05CeSeVyRIhG4o2dli7r6MjAwA9W48vlvv+PHjAICcnJyg4/mRf/zxhw8fxp49e/zOdfz4cXTv3h0AMGrUKL/9CkW9/7a6uhoZGRm4cOGCn1vv+PHjyMnJ8bMosrOzMWrUKLjdbtjtdlRXV0OlUsHpdEImS1yTW8OL5tMqKWiCuDTgrz/V2l0RK5GkAN8FISUlj9F6VaBo3ERJIwklO1vMksrMzESXLl3w1VdfeW3ftGkTkpOTvRQXUO9u69GjR8DxGo0GV199NYYMGYLDhw/j9OnT3H673Y68vDz06dMn6Fx27NgBoD5AYsiQISgvL0d+fj633+Px4Kuvvgp4jLFjx2LdunXYuHEj8vLykJ+fj8zMzIRXUjpeTxp+fxqCSGRqeblCpkZUKC8sqcU7eUdxwWjltukEvh+xWq8KlHyfKBG6caOkRCIRJk2ahEWLFmHjxo3weDzIzc3FggULMH78+IBrIJMmTcLSpUuxdu1aeDwerFu3DvPmzcMDDzwAiUSCQYMG4fLLL8eDDz6IixcvwmQyYdy4cSgtLcUDDzwAAJg9ezbGjx8Ph6O+RMihQ4cwc+ZMDBkyBO3bt0e3bt3Qs2dPPPbYYzh58iSsViseffRRHD58GBMmTAj7+lwuF+dOTET4Xy6hN0WCSCSqeaHkjalQvnjHSfxz2wn0/8c2vLbpCDweRlD4x1Nn3ET5HoeSnS0qVZ988kmcOXOGc8ExDIPRo0dj3rx5AOrdb2lpaZgzZw5efPFFzJgxA0VFRbj77ru58SNHjsSCBQsAADKZDGvXrsWECRO44Ifk5GQsX74cV111FQBg5MiRuOuuu9C6dWsYDAacPHkS2dnZWLZsGYB65fnZZ59h/Pjx6NixI0QiETQaDd59913ceOONYV9boltSfNdAIkcKEQQfvvVktrsQqQrhVzVfuqsI9/RpGzBxliWeal76uu1/O2fEqxsP49VR3dCtTfyUbwslO1tUScnlcixatAgzZsxAYWEhOnXqxK0dAfVrRR06dEDnzp3rJyeV4r333sMjjzyCI0eOoEOHDrjmmmu8jnnttddi//792LZtG+x2OwYNGuSVFDZo0CAUFRVh9erVqKioQK9evTBixAhIJA1CuUuXLvjpp5+wY8cOmEwm9O/fH+npkfVScjgcYVeniEdUvLcuypEiLhUsPCVjdbojWpNSysRQ+qwx2Z0ewbUeVRxZL77f4zv/tRsA8NL6Q/jfo/1jMaWAhJKdMfFPde3aFV27dvXbrlarcfToUb/t2dnZyM7ODno8uVyOESNGBN2v1+sxffp0wTlJJBIMHTpUcIwQie7uM6gaHpIUDSkp4tLAzlNSDpdHsO6eL2lJCvgOd/sEWPkST5YUv5cWH1/FG2tCyc74uaMJjtvt9rLOEg2dquEhIXcfcanAL4HkcHkisqRaaRURrzHFU/eAQO07ACAliPKKFaFkZ/zc0QSHYZiETeQFAAMvNypRFlwJIhT86L46R2Qh6MlquZ9lVFBsEvxMPAVOBFNS6brA22NFKNmZuFI1DknkKg38iKVACYwEkYiYrA0RfRZ7ZNF9OqUUUh8llfvLOcHPxFO5vGBRiPzK8PGCkOwkaRRFhFpOxzv8N0xTI0J1CSIe4XfhDaf6OR+JWOxnSbGNE4PBjwbk89qmI1i2qyii8zeVYEEcmw+WoLzW3qJzCYWQ7CQlFUUSWUnxqYvwjZMg4hV+CLorzOKyLDVWh59XIZQrPFCF9aMltVi6qwivbjoS0fmbSrAoxN/OGdFn3lb8a/uJuJFZpKRaAIlEErCuXyLijPCNkyDiFSNfSUXYX8locfpZI3qVDHM2HA76mUAFbC/yqlW0JKGWHxZ8exQny/3rlrY0oWQnKakoIZVKLxklZXNdGtdBEFV1Du53T4RKqs7hhsZHSWXolNh5rDzoZwJVHj9fbYnovM1BsB5aTnfsLalQspOUVJSQy+Ww2+PLz9tYrA6ypIhLA36HXXeEri2z3enXlsOglgm24whUefx8dWwsKT78tTk+8dDOI5TsTNzs0zhDpVLBao39w9hY+G+ZcfDcEkRUMFoaLKlIl1/cbsbP3dc2RS2YC8V+j1b+dBrvbT2Obm30cdFVgF/DkE+sWovwCSU7Y3/3LhE0Gg3q6upiPY1Gwy8fQ+4+4lLA5fY0KVK13Gz3q86QqpELpmi4GQZ/+8+v+OpgMQBg57FyXJUZuC16S3K6MrBsksZBbmco2Rn7GV4iqNXqhLak+DkkJTWXhtuS+HNT5/B+2WKthmDlgnxxuhnIfBTS7hOVgkVqlVIJp6BYKnnrYrEiWMh5PCQfh5KdpKSihEwm49qBJCIW3hd6a0FpwLbZBJFI2HxyllirITtLF/YxFD6uvYVbj+HA+Zqg4wO1cY9FTlKSTyJvpTmwbNLEQc+pULKTlFSUkMvlCa2kKuu8v0j5p6tiNBOCiA5WH0uKtRqys8J3vylkkYlIIVdg6xas9NDGp51IqckWcFysmjTyCSU7Yz/DSwT2RsdLclyklJm8lVRRReKurxEEANh9wq4XfHsUgxZsx69njWEfwze6D6gvlxSMYKHeQPBaes1BK633uaotgZVAPFRtDyU7Yz/DSwSFQgGGYeByJWa1hiqfhzhWCYgEES0CuazPVFqw70x12McIVGGiXaom6HhhJdVyllSyz7pboK7ESUopxHEQ3RdKdpKSihJJSfUuBJNJuEpyvOJbCul0ZewTEAmiKTiisK4aSEkJdeZ1CiQMt2SftitaeStSc4BSZ77WVqwIJTtJSUWJ1NRUAEB1dfhvafGExcd/f6ykNkYzIYjoEI3yXiqZv2uvfavglpTbE/ycmfrgyi3aZPi04+DXMGQJ5MqMBaFkJympKJGcnAwAqKpKzIADfksDALhAHLo+owAAIABJREFU7j4iwYm0wkQg+M1AWTqla4OOd7qZoFUc2qWomzyfYPiu5/haSb4voUB8BE0AoWVnfMzyEkCv1wMAamqCh6fGM77uvkAPNUEkEu4Ia/UF4vZ/7vLb1lZA2Xg8jFcDUT6p2ubriOtbg883FyxQC5GkOKiEAYSWnaSkooRGU+8CSNSqE5YgfXAIIlFxRaF4ak0AN1lygFwoFg/DID1IFF+qpvnWgOw+VWJ887VsAV46Q/XGailCyU5SUlEi0S2pSLuWEkS8E6htRjTQKYUDIILlVjWnJeVrKaVpvSMJnQHWypJbMJBDiFCyMyaq9KeffsKOHTuQmZmJ+++/HwqF8BtGfn4+vvvuO6SlpeH++++HSuW9AGmz2bB69WqUlpbipptuQt++ff2OUV1djR07dkAmk2Hw4MHQahv8yt999x1qa2vhdrvhcDggEolgsVhw0003oX379mFdE7v4V1FREdb4eCMakVAE8WdApwou3EUiEWRB1qSCNSGMBha7t5LSKLzPFUhfZ+jio418KNnZokrK5XJh6tSpWLlyJTp27Iji4mK8+uqr+Pbbb9G5c2e/8R6PB4888giWLFmCDh06oKysDK+88gq+/vprdO3aFQCwf/9+jB49GuXl5WjdujVeeOEFPPTQQ1i8eDHX9GvZsmV4/PHHYbPZ4PF4kJKSgg0bNuCGG24AADz++OM4cqS+a6ZcLofH44HBYMDSpUvDVlJ6vR5KpRLFxcWhB8chvomPBJHohOj512iESoaJ4N3Dik9zRtOZbN5uSalPkm6ge5ESZg3D5iaU7GxRd9/ChQuRm5uLzZs34/jx4zh79izatm2LGTNmBBz/r3/9C5988gnWrVuHEydO4OzZs+jcuTOmT58OAHA4HBgzZgyuvPJKnDp1CseOHcOWLVvw73//G5s2bQIAbNq0CQ8++CCeeeYZ1NbW4uLFi2jXrh2efvpp7jwZGRmYOHEiPB4P7HY7nE4nysvLMWrUqLCvTSQSISsrCyUlJU24Q7GDavURlxriZtBSBrUM1XWB214A9d1/j5cF7nbbnG0xAq2d8QnUXqQ5LbtICCU7W0xJMQyDDz/8EI8//jhuvfVWAEBKSgpeeOEFbNmyBadOnfL7zKJFi/DII49wysJgMODFF1/EDz/8gIKCAmzZsgVFRUVYtmwZ0tPTAQDDhg3DiBEj8NFHHwEABg4ciAMHDuDll1+GSqVCZmYmcnJyUFvbkAdUWVmJyy+/HHv37sXHH3+MVatWwWYLXOtKiOTkZBiN4ZdciSWr9pzBNa/m4Z0tx2B1uKMSCUUQ8URztKG4LFkFozV4nTmhihPNqaR8o3N9CVT+KB6Ky7IIyc4Wm2VRURFOnz6Nu+++22t77969AQCFhYW48sorue0XLlxAQUEBp2x8xxcUFGDXrl3o06cP2rVr5zdm5cqVAACdTodu3bpx+w4dOoTc3FzOGgOAkpISfPjhh5g7dy6ysrJQUVGBl156CT/99BOysrK8jl1QUIDCwkKIxWIoFAoYDAao1Wrk5ORAp9MlTODEjycqYLQ48c/vjmPT7xcDPsQMw3AuU4JINCItDhsOKRqFoCJSCpwzlpaUSi4BfILngkUhxgIh2dliltT58+cBAJdddpnXdjaRy3fR7MKFCwHHa7VayGQyVFRU4Pz58377gXoLLdAi3H//+1/0798f2dnZeOmllwDUr5OVl5dDLBZjz549uHjxIs6fPw+Px4O3337b7xi5ubkYPXo07rzzTtx6663o168fevToAZPJBJ1O52WhxTN88/9UeR3KArQTCJRbQRCJQnMUT01WywSVVKwcEqHagegDBHukxklZJACCsrPFlBRbn8k3Fp79m42VDzXe4XDA6XRCo9EgKSkpYGy92Wz2it6zWq2YNGkSxo4di/Hjx+OHH36AwWAAUO8PHT9+PPLy8riowPT0dIwZMwa7dvkn8rndgQV3dXU1UlNTUVZWFvwmxBG+Pmq29w6/wrPNSetUROIi1Oa9seiUMsEgo5KayJcJokGoDsTaAK69QNtihZDsbLFZZmRkAAAuXryIjh07cttPnjwJAF4uOd/x/H388YcPH0Z+fr7fuU6ePMl9hmEYjBo1Cvv378fWrVsxdOhQr7ESiQSffPKJ3zHUanXAWlLZ2dkYNWoU3G437HY7qqurYTabUVJSgszMTJSVlSWEm8zXLcG+HWoUUu6Bp2AKIpERcr01liSlVPB7YXHEJt8wWEQhS6AcrUDWVawQkp0tZkllZWWhY8eO2Lx5s9f2r776Cnq9Hp06dfLanpKSgm7dugUcr1ar0bVrVwwaNAgHDx7EuXPnuP1OpxPffvst+vTpA6A+RH3Lli1YtWqVn4IKBsMw2LVrl5/iBICxY8di3bp12LhxI/Ly8pCfn4+CggJcf/31yMjIgNvtRmVlZVjniSVan5Iorj/8FPyqz69uOtKicyKIaNIcloJBLUO5ObhrzRajVA5zCEsqUMVzoQaNLY2Q7GyxWYpEIkyYMAGLFi3Cd999B6A+PHz+/Pm4//77IQ4QiTNhwgQsXboU33zzDQAgLy8Pr732Gu69915IpVIMGTIEbdq0wfTp01FRUQGr1YqpU6fi4sWLGDt2LABg7969kMvl0Gg0+N///odVq1bhk08+QVFREYD6ooZdu3bF7t27AdQrqLfeegs7duzAxIkTI7pG1vorLy9v3E1qQZTSwOGn/FyOr35PzJwvggCApBCVIRrDuv0X8X//OxR0f6yiZIt5nXdlEn8vTrzkRAVDSHa2qFPy6aefxsmTJzFs2DCo1WpYLBaMGDEC//jHPwDUl8VIT0/H3Llz8cwzz2DmzJk4duwYbr31Vm780KFDuYAGuVyOL774AuPGjUNmZibEYjGUSiUWL17MWUGpqalwOBwYOHAggHo3HlBvqZ07dw4GgwFdu3bFwIED0aNHDxQXF6OkpASzZs3CyJEjI7o+dh3MbA6cJxFPBEssjHMvJUGETXO4+44UC/eLk4hEcKPlFdWF6ob+b4EsyPQWbLjYGIRkZ4sqKaVSiRUrVmDGjBkoKChAly5dcN111zVMRipFRkYGMjMzAdQroSVLluDRRx/FoUOH0LFjR/Tr18/rmNdffz0OHjyIr7/+Gna7HTfffDNatWrF7R8zZgz69+8PpVKJ5ORkiMViuN1uLtxRLBYjNzcXO3bswJYtW6BWqzFmzJiAFTBCodPpACRG48PT1B6euMSJxbqwRCwCggTFVprtSNHIm2VeFeaGNSl1gMKxwdafHC4PjFZHzJWYkOyMSXhH7969uXwnPhqNBmfPnvXb3rNnT/Ts2TPo8VQqFUaPHh1wn0gkQps2bby2SSQSpKSkeG0bPHgwBg8eHMbsg5NISmrNL+cCbo9G5WiC+LMSrJcUAFz72lY8MawTnhgW+QtwJATKD/Otig4Aub+cw+ubC1Brc+GbmQPQKSOpWeclhJDsjJ+Vs0sA1pWYqO06gIZQdCC+FlYJIlJ+Lmr5BqShEohPlUdfNnh81sEC5YfxA6JYnvnidxgtTrg9TMg8q+ZGSHaSFIoi7NtAoiT08mHfAPm5UV1i+GZFEE3hRFkt7v34pxY/L9/tFghpgKCGpuKbtxWoskWo4raxfiEVkp3xk811CcAmICeiklLKJDDbXV7VlDsHUVJnKuvw711FuDpLh/uuu7ylpkgQYVNSYw/YniLWyKJcT/BUuRkHL3iXEwq05BXI3ccn1kpKSHaSkooibJ8ri8USYmT8oVHUKyl+2/i0ALW93vr2KD7YfgIAkKVXkpIi4pLmanjYVGTS6FlSZyrrcNPbO/22B7r0UMVkmyNcPxKEZCe5+6IIGwIf72tSTICnONBDvHjnSVTyEhcZhsGHO05wf1eGcG0QBOGNQurbjJDBRztP4oNtxyM+1kff+3eOqD+m/7ZQreJjnUclJDvJkooyarUaVqs11tMQJFDh2GAP8Z0f7saWWYOglEkgEokglYi5Ekr6EC4EgiC8YV8GTTYnVuw+DbvLjX9try/1NuXGKwKGjwcjWJ3AQFZkqArsSXFQxy+Y7Iz9zC4xtFptTJN5iyrq8NHOk1DLpXh55NUBxwSq4hwsj+JclRWHL9bg2nb1IftquUSwCjRBxAPxmpSuVUjg9jAYsmAHKn3q7dmdHqjl9VGJr28uQN8rUvD8bdlBjxWsXp8jwpqbSQopxM3YRiRcgslOcvdFGY1GEzMltXDLMQx7Zyf+m38Oy3YXeYWT8wlkSSULmPt/XfQTTvzRbVTH812H81zbnG4s2nGS+zxBtATN0ZU3GihlEhy8UOOnoADA5nL/f3tnHhd1tf//12wMAwMMDJsiuOKCigouYW7hnqbmVpl52yzv16yr3frdupZW3zK/mmVpat0W9VrmkpqKZWqKmmuiAoLKpqCAwOzMPvP+/YHzkWFmWBRk1PN8POahfM75fOacM+fzfp/lfd5v/N+v2Zi65hjOFqqwJjUPx/MqPMaK8vROmSx2EBEOXy7Dv7el13lwP8DXO+YqnmQnU1KNjEgkgsVSewCy2rhSUYlZ6//CDyeuNnjGsuHElXr5DnN3YFdWh0fk0ctTYbLanGZc9Yl8uul0IRb/mo1hy1w3eBmMpsJLdRR8RQLszXQfJl1rtOLLg7lO15786jj6fLgPmdddAwLqPETjvaYyYOqaY3jmm5PYcOIqtvxVVGuZAu/QG7rOZMXrm87hoY/242rF7RuNeZKdTEk1Mj4+PjCbb9+g4NeMEvyaWYK3t6XjkaUHkV5U/0i/9Y2fk1vmOlpxZ8lXHYuNcLFE6zTjqk+k0SOXXYNPMhhNjbfOpCQigYsicqDSux/cmq12/GPj2QZ9z6mCW2GGHHtUnszQ7yRkBxFh8qo/sfVMEUo0Rhdz+IbgSXYyJdXI3OlMylptJnRNZcB3R/ORXVI/N0viGmcdPL2o649dcblWl5Kqer4AUvEt6yR33par83VqHvZeKOX+dmdVyGgYBrMNj395FG9vS2/uong1dfXN5mLDCdd3z4FKb/a49Hb5hg7/PX7r3ppeJmpDeHPw2jNa5jbdXRiP+mKzE7JLbp1tstpvf7+azaTuEgKBwGP03tvh57RrGPXZYRQqap9GX63Qu0z/Pc10LG46eFg9Omqwn8hpT8pXJEClyYpLpVoYzM51LlEb8WFKltM1vZmFo79Tfjp1FWlXVfjhhKuPS8YtmiIqb2NwPM+zqyat0Vqrwpi/PYMLqlhbdOCaOAavHcKkbtPDAz1/56I9WZi36Sxybrh3UCAU8CGp5s3Ccge+Pz3JTu/YMbuP4PP5dzRj8DT7URssiHZzvVChx0cpWdiT4brO7dBRJWojZH4izjWKxI1/MXeRO2sSKBE5zbhUegu6LviN+7vg4zHc/935Aqs0W+s8VMioHZWHTfSGUKw2IL1IjaFdIuq1ZHsvUvM8krfg7yNApYfBmslqr9MzxHWVAR3CAzwaRbnDoaRayiRu0787WoAJPaPQw81Ma82hqrNY29Ku4ZVHOuD1EZ1c8gRJRJwx1p1Y/nqSnd453LiHsdvtd+SK39MyRc2lPAfPfX/KrYIS8nm4VKrDc9+dxEOL9jutabvz4yVx44CyJr4igdOhv2sq5zMNf+bc2n9yt6mrNzXNTMpktWHNoVwcvlx23y8pRlUTNNYGmhrb7YRVB3ORtOgAXlr/F1IveX9wztvF0/vS3HhSUECV1W1dBkyamxF4rQ1Y7nO877UZSEz48mit/YEI2OBh9l594OqY6d0OnmQnG9Y2MjabDWLx7a/xeprGe1q+8GSGarUTRn6WeitfmQ45N3S4oTW67eCB9XCLklumQ5HS80HlC8Ua9O9QFcvLaHV9Gav7BWxMjucpsGhPNgAgqZ0cS6bEo1WwX5N8V3NTfcZrsNgQ0IBlrT9zK7D412zub5Xh/vUY0ty+6G4Hk9VW595wscoIxDQsArBjn8tdMEQHRFWyZFDHMO5azS0Gk4fZW/UyN2QZsiaeZCdTUo2M1WqFUHj7zeppr+H8NTXWHitAq2A/bDhxBY92a4GZA9vV+7n+PgLM+u9fyLmhc+vK/72dmXU+Y/Tyw7VO5x1LEIUKvVsz24IKPeJbud+8vRO01ZTfsbwKrD9+BW+N9nwI8k65oTXivV8u4G/926Bv25C6b2hEgv1uKSmjxY6GxKqrubJnsd6/s05vnUnVht1OiJLVPria/cMZ/Jkbg9mPdKj3c/19hChS6vHmlnO15vMVCfDd0Xz8mlGCE27CnFSabXh7Wzr+/WgXp2X7iMBbnbAhyrMmnmQnU1KNjMlkuqOZlKfZxsoDObhYemvzcsUfOfhvLZZCNREK+Mi5UWUe6u5E+m+ZpS7XalLXenNeeSX+tfU8Np5yH1BxwY4MDI4NQ/o1NZLay132Q66rDPjn5nMY37MlnugTg+N5FbhUqsWMpDYNKldTB25ccSAHu9OLsTu92Gkfjohgp/qZ5t8u1ZdsTG5mq7VR88B2Qz0T3EtIveSAakMwWe0e942qs+HEVbw0qP4DVKXejCFLDta5RFgfi9EfTlzFiLgIDOkUzl2rbsJ+J0rKk+y8935JL8doNMLX9/ZCMZutdmiN7td0qysoB57OVbjD3eypsfn5zLVa05V6C3q8vxcAsGxqD0xMaOWUfjSnHH/mVuDP3ApcuK7B2pum8gM6hEJtsKBHK5lb9y13ssTQUKw2OzZ5iGr89rYM7Dh7DdMfao2XBrVzsdTKLtFgW9o1PNe/LSKD6u4jJqsNm08XYUCHULQJ9QfgPJNSGyxoFVx3mZWVZsj8RC57EvUVKESEC8UadI4MbDZDi6sVegT5iVzO9OhMVny+/zIGdwzDwzeXmgHvNZyojYxranSODKxX3sFLDtb7ucdyKxq0h1UXNWNmVT+Wciff4kl2MiXVyOj1ei7KZEOZ/p8TjVyaW7hzhdSczNt0Diq9BU8/FIOT+Qos+e0izlc7uLy22lkuRziCt0Z3xsuD2wOoWhr5NbME3x7JR34Nty819+8W/5qNMq0J74/vWqsDT6PFBl9RlW81T8I4JaPEKTAkEXGbvcdyy6E32/BVah5+OlWIA68Phryaovo6NR9bzxRhzaE87JozAN2igtx+x5a/isDnVdVj/vYMAMC3z/ZGcucIp431ugLsAcDu88WY/cMZxLcKwgsD2jql1TecxZwf07DrfDE+mdIDkxJb1X1DI1Ko0OODXRew90Ip+rQJxuZZ/bk0IsKQJX+gXGfGiXwFvgj2Q3mlCQkx9dDcXsgfF8vQqwnK/nNa7YPHhvLPzecw+WY/0BotWLr3Epd2J0MYT7KTKalG5k6U1MmCpgt3fbZQ1WTPvl3e33UB7++6UO/8i/Zk4+XB7WGzE8Z+cQRZxe4POVc3EDKYq3wHAlUbw5890RMAuJmJgxnfnkTqpTIESURQGyx4tn8bvDM2zklZvb0t3WXP8GyhCr1ignEguxQF1VzCqA0WFCkNTkpKpb+lVP5zOA+fPdkLRITNfxVh9cFcvJLcAY/1aIl/bq7aO3iq761DB+/vvIDkzhFOM8nLpVoMrrbR7cBktXEziUs3Z+Dni9R4rYbXgvoIFIvNjl3niwEAJ/Ir6qWkdpy9hj+yb+CZpDZIbH1nQvfp/5zA1Zsb+NUPjQJVe5wORW002zB1zTGUaIzYMisJvdvc3b3CxmLZ75fqzuQFfHkwBzMHtsOU1c7Rj+/E0Ycn2Xnv7S56OWazGT4+DY/NcvDijSYozf1J5nW1RwUFAIKbb8qu89fR64O93PWzhSoMWXoQQ5YexIm8Cu56idrImd86nHl+/2cBVv6RAyLizNrdGbUs+e0iyrQmLN93Kx5QYLX9kLwyHVLSq4R89c1mh4PRt7el480t55FXXon92TecluB+OXud+7/jIHR1s///3Z2Fd3dkcGW22OxYlJKFru/+hq9T8/DLues4Vq2eLu1Uj6W76uf2IoPq3i8BgP8czsf2s9cxadWfeHHt6VpN5e12qtV7QmW1oww1zxBVPysU1zIQJZqq0BUr/8gBo2n5v18v4tPfL7kMHAR3oKU8yc5mmUmdPXsWBw8eRIsWLTBx4kSIRLWbP6enp+PAgQMIDw/HpEmTXCpisViwdetWlJSUIDk5GfHx8S7PuHz5Mvbs2YPAwEBMnTrVRWPb7Xbs2LEDBQUF6N+/P/r163dbdbtdw4mTbqxpGK60+dfuOvOs+CMHsRFSl5lDdbJLtOjXTo6/rigwadUxt3mW/X6JG9lueNF9f/gztwLfHMnHuWpLlY6zLONXHuWuzRve0cni7PDlcuxJL8bBi7fOpoQHiJ2UQvUzNQI+D5/vv+wy0l537ArWHbuCQ28Mwef7c7D1TJUz0ZrePtzxzo5MmKx2tA+TYkinMPyWWYoPUy5g1dOJ3FJkdUVWqNDjw91VM9+XBrV3Mj3WGi1YtCcbZ64onQTXvqxS5JZVolNkgNsyPPv9KVwu1eKTqT2Q2DoYPgK+01kZf7GQU+iFCgN0JitnSl3dT1x1ZfbHxTKoG7Bfy7g93PkgvBPTf0+yk0d38fSj3W7H7NmzsWbNGrRo0QIVFRVo06YNUlJS0K6dq7UKEeEf//gHvvjiC0RGRkKpVCIqKgopKSno2LEjACAjIwOTJ09GQUEB5HI5iouL8eqrr+LTTz/lOvsHH3yADz74AMHBwaisrIRMJsO2bdvQp08fAMCVK1cwceJEnD9/HpGRkSgqKsL06dPx/fffQyCo/was1WqFSCTCe++9h3fffbdBbVMf4cu4v5nQsyVeSY5tdo/xUxJbYcmUHrDZCUaLzcmrSHWCJCK0CpbgqkLv0eDHweJJ3TGhVxQul+qwO70YT/WJwbG8cvy/ra4WZRtfeggPtZMDAEZ+mupiNBQZ6Itvn+2DRz8/7PH7PhjfFe/sqPtYhYM2cj+n5VrG7TF/TBe82ICjMQ5qk513VUktX74cb775Jn744QdMmjQJJSUlePzxxxEcHIyUlBSX/GvWrMErr7yCdevW4cknn0R5eTkmTZoEoVCIAwcOwGKxoHv37ggNDcVPP/2Eli1bYufOnXj88cexa9cujB49Gtu3b8fjjz+OlStXYtasWdBqtZgxYwZyc3ORnl71ggwYMABarRY///wzOnTogNTUVIwaNQpffvklnn322XrXT61WQyaTYenSpXj99dcb1DZMSTEYt/hgfFdM6R2NpEX7oWyiWdHfklrDVySAyWrHm6M64Y0t57H75v4b4/apfiyjvtQmO++qkoqNjcW4cePwySefcNd2796NsWPHIj8/H23atHHK3717dwwaNAgrV67kru3fvx/Dhg1DdnY28vPz8eijjyInJ8dpJjZ69Gj4+Phgx44dGDZsGGQyGbZs2cKlZ2RkoHv37khNTYVUKkVCQgKOHz/utMT37LPPIjs7G8ePH693/fLz89GuXTt88803eP755xvSNExJMRh3mRcGtMU7Y29FrzZZbVh/7Ar+d/etpdJOEQH4/KleTt5b3CEW8nHsraF49ruTTlaqDyK7Xx2Ari3dW656ojbZedf2pAoKCpCTk4OpU6c6Xe/bty8AICsry0lJFRcXIyMjAytWrHCb/8KFCzh69Ch69+7tslTYt29fbNiwAQaDAYcPH8b69eud0rt27QqJRILMzExotVrExMRwz63+jK1btzqZGDvKmZ2dDT6fD7FYDJlMBj8/P3To0AEKRdW+UmhoKO4mfdoEO8WPqS8bXuyHtKtKJxNSd7w1ujP8fAQel086RQS4PcfFYHgzkho+LMVCAV4c2A5P9o3Bh7svoGe0DJMSWkEo4GNYlwicyK/At8/2cbFoO/vucFjthBB/H+yY/TB+OlWIf/3svIwZ4Cusc0m0Pozr0RItZRKsPuQ+JpU3EB3ScOvm2mTnXVNSV69WWUbFxMQ4XQ8JqTIVLStzdm5YWFjoNr9UKoVIJEJZWRmuXr3qkg4AcrkcN27cwI0bN2A2m13y8Hg8yOVylJWV4caNG4iJiXFxbCiXy6HT6WA0GiGR3LJq2rRpExYuXOjynQsXLsSgQYMAAIGB9TuQV539rw/Gqz+mgccDXk2ORb92cuzPKsXIrpFOVmHb064hOkSCxNYhSC9S44qiEmO6twCPx8M1lQHL9l5Cv3YhmNo7GulFalxT6RHXIggxcj/Y7YSfTheiS4tALrZMYutgCAV8HLx4A+EBvnionRyJrYMh4PPw5pZzSCtUoWvLIAyIDcXkxGgnR7Q6kxUSkQACPg83tEZs/esaMq6pMSA2FFN7R+NSqRZKvRk2OyEhJhgaowU+Aj6C/XygMVrQ96P9Tt4inuobAwEf8PMR4qvUPKf2GREXgUUTuwMA/rqixEvr/wIAPNu/DYrVBo8eM775W2+cLFBgfI8orDqUi53nrrvNBwCfPtEDJ/MVyLyuQatgCVLS3UdQ9cTCx+IQ7O+DIR3D8em+S/j+zwKn9Ee7R6JntAwxIX7ILavEkt8uNuj51ZmS2Ao2ojoPUAPAqqcTYLTaMPenW25xtv69P9qH+bv8BvcLreV+sNoI11QGRMkknFXky4PbQa23wGyz4/GEKLf3SsVCLJrobHz19YxE2OxU5bnlw9FY/Gs2vj6cjwCxEIG+Iu5oAI/Hw5N9YzCyayTWHivAyK6R6NIiEESE01eUaCmTIEomgdVmx89p1/DmlvMAgNHdIjlH0RN7RSEiyBc+N5XjYyuOoGe0DEun9ECH8KpwG6FSH3x5MBeju0XileQOEPB42HW+GB3CpZD5iaCoNGNfVinmDe+EG1ojyrVm8PlA79YhWHHgMvzFQvytfxuo9Bbk3NBh+jcn0CkiAPNGdMTIrpFY+EumS/9tG+qPXtEyPPdwWxzNLcfHe7Kd0rtFBWJ0txb18gNaE42mylrXreyku8Rff/1FACgnJ8fpukajIQC0efNmp+uZmZkEgDIyMpyum0wmAkDr16+n5557jkaPHu3Oe3D/AAAgAElEQVTyXR999BFFRkZSeXk5AaD9+/e75AkODqalS5fSv/71L0pISHBJ/+abb4jP55PNZnO6/s477xCqDlY7fZYvX07btm0jAHTmzJl6t4s3Y7fbSVVpbvLvSS9SUaGissm/pzp2u53KtUZKL1KR3mT1mM9ms3tMUxvMpNCZSG+ykt3ums9osdKOs9do48krbtNrlsday3dpjZZa7yci0hktdK5QSeuPFZCy0uQ2T6nGUOszbDY7VZosXHm1N5/pqfwlagN9nZpLlSYLGcxWMpitVKo2kMlic8mbV6ajPenFpDNaqExrJI3BTFqjhQoVlS7PL1Lqad2f+fTBzkw6e1Xpsbwn8iro1R/P0KqDObRs70WyWF2/t6lIu6qstWz1oa5+cbfwVI6rFZV0NKfstu5tCLXJzrs2kwoPr/L1VFxcjPbt23PX8/KqRsxdu3b1mL96Wn5+Ppc/IyMDZ86ccfmuvLw8dO3aFcHBwRAKhSgudt4MVSgUUCqV6Nq1K7KysnD9+nWXZb28vDzExcWBz3c2qezSpQvGjx8Pm80Gk8kEpVIJnU6H0NBQKJVVS27Bwffmifea8Hg8BNUR36Yx8OR5oSnh8XiQS8VOh23d4c4Nk4O6RoxioQDjerSsd3lqCyZbmwdrB/5iIeJbyWp14hteh0daPp/n5JVDevOZnogI9HWx5nIXCgaoGom3vXmIuvrqgLu6RckkeKYOn40A0LdtyF138uvAU6TbhnAnYX0aE0/liA7xq3P5rjHqUJvsvGuHeaOiotC2bVvs2bPH6brj7FKnTs7BtEJDQ9GlSxe3+X19fdGtWzcMHDgQ586dw7Vrt5Y8rFYr9u7di759+4LP52PAgAEuz/jttyqT2j59+mDgwIEoKSnBuXO3lkKICCkpKS77VADw1FNPYfv27di5cyf27t2LU6dOISsrC9OmTYNOVxU2Qyp1HwGTwWAwGK7UKjvveJ7WAN59910KDg6mI0eOEBHR77//TjKZjF588UW3+T/66CMKCAiggwcPEhHRwYMHSS6X0/Tp04mIyGg0UkREBE2cOJFUKhWZTCaaOXMmAaC0tDQiqlq2E4lEtGPHDiIiOnPmDLVq1YqGDx9ORFVT1c6dO1NycjKVlpaS1Wqlt956iwDQb7/91qD6LV68mACQTqdreOMwGAzGA0ptsvOuKqnKykqaMmUKAaDQ0FACQIMGDSKFQkFERGq1miQSCS1fvpyIqpTQtGnTnPInJSXRjRs3uGcePHiQoqOjSSKRkFQqJV9fX/rss8+4dJvNRrNnzyYej0dyuZx4PB7Fx8dTfn4+l+fMmTPUsWNH8vHxocDAQBIKhfTOO+80uH5vv/02CQQCr1lnZjAYjHuB2mTnXXWL5Ofnh02bNuHw4cO4cOECOnXqhMGDB3NrmjweDwEBAZy7drFYjA0bNmD27NlIT09Hhw4dkJyc7LQGOnjwYFy4cAE7duyA0WjE6NGj0bLlrX0APp+PFStW4IUXXsCpU6fQqlUrjBo1ymmvqVevXjh//jx27NgBjUaD5ORktx4w6kKr1SIgIMBr1pkZDAbjXqA22XlXD/Pe7/ztb39DamoqZ9zBYDAYjLqxWKq8irjz48qUVCNjs9ka5O/PmyAiqNVqVFRUQK1Wo7KyEmq1GkqlEhUVFdBqtTCZTDCbzTCbzbBYLNDr9aisrITBYIDZbIbVaoXN5hy7isfjQSAQQCgUwsfHByKRCEKhECKRCCKRCH5+fggJCUFgYCACAgIQFBQEf39/yGQyBAUFwdfXF76+vvD390dQUFCdDonvVaxWK1QqFXQ6HSorK6HRaLi2NRgMMBqN0Ol00Gq10Ov13MdsNsNkMsFoNMJiscBqtXIfu90Ou93OeXJ3jFQd7V69bcViMUQiEaRSKYKCghAUFITAwEAEBgZy/w8PD0dQUNA9u1qg1WqhUChQWVnJffR6PbRaLbRaLde+jv872tRoNMJkMsFiscBsNjv1cR6Px/VtHx8fSCQSBAQEcJ/q7SeTySCTybj/BwcH3xf92WQy4fr161AqlVAoFCgtLeX6r9Fo5PqqyWTi+rSjr9psNtjtdsTHx2PJkiUuz2bxpBqR1157DRkZGZBIJJDJZAgJCeGErkQigVQqRXBwMNdhQ0JCEBISAn9/fwiFjfNT2O12GAwGaLVaaDQa6PV6aDQaaDQa6HQ6lJaWorS0FCUlJaioqODSlEoliouLYTQaa30+j8fjXkbHC+nv7w+JRAKxWAyBQACBQAAejwcejwci4sz1rVYrp9ysVissFgun6FQqFez2+h0q9fX1hUwmg1wuh1Qqhb+/P0JCQhAaGsq9/OHh4ZDL5fD39+eEhEM4SCSSRheyZrMZZWVlUCgUnICrqKhARUUFJ+x0Oh2USiU0Gg3UajW0Wi0nKHU6HcrLy+vdBgAgkUggkUjg4+MDsVgMX19fbgDg+PD5fO4D3Axxf7OPlJaWcspPr9dzAtlsrj2Yoo+PD8LDwxEWFobw8HC0aNECERERiIiIgJ+fH2QyGUJDQxEcHIzQ0FDIZDJIpVKX4xy3CxHBZDJxAySHonEMsIqLi1FSUsL9W1JSAoVCwf0W9UEsFkMqlUIikUAoFMLX15dT4j4+PlwfB6oGpkajkRu8GY1G7v0zGAx1fFPVNohUKkVAQADXpnK5HCEhIfDz80NYWBhCQ0O5vh4UFITg4GBO4TVGuxIRzGYz9Ho9dDodNBoNysrKuCM2Go2Gq5Nj4FpcXMw5RKjpjKEmAoEAfn5+EIvFnLyo3lcFAgH0evcOftlMqhF57bXXcPr0aRiNRigUCqhUKmi1WpeZhTtEIhHEYjF8fHzg5+fHjXLFYjH3I/L5fNjtdthsNk6YWCwWTsg5BE1dCAQChIeHIzw8nFOiMpkMkZGRaNGiBUJDQ7nZTFBQEEJCQhAcHIzAwEAIhcImGUXb7XZuRKtSqVBZWQmVSgW1Wg2j0Qij0cjN7ByjYYVCwc06KioqoFAooNFoYDKZ6qy/v78/p2Qdgsgxs+Pz+ZyydQgAm80Gm83GKVpHmcxmM3Q6Xb2En0OAO2YpAQEB8PPzg7+/PwICArjfxN/fn7vmeKEdH4cw8/X1bTShXxOLxQKNRgOVSsUJJ7VaDbVajdLSUs6bS3l5OacIbty4wS3ZuIPH43EDBIegF4lEXB93CH0+vypUh2MGaDabYTAYOOHpGIXXJbb4fD7Cw8PRsmVLREZGIjQ0FCEhIWjZsiXkcjnX7v7+/vDz8+Nm8VKplPNq0xjYbDanQYlKpeLaVaVSQalUcnJCq9Vy7VpWVgaVSuVRcNdsV39/f65dHXLEoQQcKzvV+7DJZILJZILBYOBm7/VRBUKhkJMXERERXNtGRUUhKiqKG5xEREQgKCiIk2Mikei25QZTUk0MEUGv18NgMHAjabVaDY1Gg/LyciiVSm4k6FhKc0yNHUsMjikx3TxwLBAInF50x4vlmNX4+flxSw2OmURgYCCkUinCwsIgl8u9brnGarViwoQJCAwMRHBwsJNT4Yai1+tx48YNrm0dAra60NXpdJwAdMwgHB/HQMDR5gA4xeVY1nEsk/n4+EAqlSIkJIQb8TqEXXBwMMLCwuDv718vpWK327F9+3ZuOe/vf//7bbfB3cZut3PLO44lH8dMsnr7O5Z5HAMsRx93tLXj41BYYrHYSUE7+rejrzv+dvRzuVzOKfumUuJNCdVwKmC321FeXs7NAqsvwatUKm4wXFlZyfVfx+DJsWLhmJ1X78NisRhisZgbOEmlUvj6+nKyw9GWISEhkEqlnBJvilWIumBKyguwWCz466+/uCWbLl26NHeR7jpKpZLz4+jv71/vZZn7CdYGztzL+7u3i1KpRLt27RAWFobIyEikptbuff1BgO1JeQFZWVlISkoCAHTu3BlZWXVHVb3fcDgUBoDo6OhmLEnzwdoAOH/+PHr06AHgwXwXCgsLuSXBB01Be+Lemw/fh1Rfd35QXSqxNmBtALA2eNDr7w6mpLwAlUrF/f9+cU7bUFgbsDYAWBs86PV3B1NSXkB1azSxuHav3PcrrA1YGwCsDR70+ruDKSkvoPrZmAd1HZq1AWsDgLXBg15/dzDDCS+gc+fOeO+992Cz2dC5c+fmLk6zwNqAtQHA2uBBr787mAk6g8FgMLwWttzHYDAYDK+FKSkGg8FgeC1MSXkBJ06cwGOPPYbExES88cYbUCgUzV2kRmPlypWYNm0apk+fjilTpmDy5MmYMGECPv30Uy4PEWHHjh145JFH0LdvXyxZssTFB6FCocDcuXPRu3dvjBs3DidPnrzbVWkwp0+fxoABA1BQUOB0vaioCC+88AISExMxbdo0ZGdnO6UTEbZs2YIhQ4agX79+WL58uYs/wrKyMsyZMweJiYmYOHEizpw509TVuS1++uknDBkyxOna//zP/+CZZ57BtGnTMHnyZEyZMgVjxozBgQMHuDxmsxmffPIJ+vXrh0ceeQTbtm1z8S2XlZWFJ554AomJiXjppZdQVFR0N6pUb8xmM9asWYMnn3wSL7/8Mv7880+XPIcOHcKoUaPQu3dvzJ8/HxqNxim9srISCxYsQJ8+fTBixAjs27fP5RmnT5/G+PHjkZiYiLlz56K8vLzJ6tQcMCXVzHz11Vd46KGHYLFYMGLECGzevBm9e/d2Oi9xL5OXl4ft27fDYDBwTlwDAgLQs2dPLs/cuXMxYcIEtGzZEgMGDMCiRYswcuRIztKpsLAQXbt2xZYtWzB8+HDYbDb069cPO3bsaK5q1ckPP/yAgQMH4ujRo7Bardz1tLQ0dO7cGcePH8eoUaNw5coV9OjRA6dOneLyzJo1C1OnTkVMTAySkpKwYMECjBs3jhPSubm5iIuLw65duzBy5EjodDr07t0bv/32212vpyfsdjveeustPPnkk0hPT3dKO3HiBA4dOgSz2QwfHx/w+XzExMSgTZs2AACj0YhBgwZhwYIFSEpKQkxMDKZOnYoFCxZwz0hJSUF8fDyuXr2KUaNG4dixY4iPj3cZEDQXpaWlGDBgAF5//XXYbDacOHECDz/8sFOfXbp0KYYMGQKxWIyhQ4fiu+++Q1JSEuc5XaVSISEhAStWrMCQIUMgk8kwYsQIrFq1invG2rVr0adPH1RWVmLEiBH45ZdfkJCQgIqKirte5yajKUIBM+pHWVkZ+fn50fz587mwySqVikJDQ2nx4sXNXLrG4fXXX6eEhASP6adOnSIA9OOPP3LXsrOzicfj0a5du4iI6IknnqAuXbqQWq3m8sycOZO6dOniNtx0c2Oz2SgyMpLGjh1LACgrK4tL69evHyUnJ5PRaCQiIrvdTiNGjKDRo0cTEVFqaioBoO3bt3P3nD17lgDQ/v37iYho3Lhx1LNnT9LpdFye6dOnU0JCgte0x8WLFykoKIiGDh1KAQEBTmmJiYk0b948j/cuWbKEpFIpXbp0ibv21Vdfka+vL5WVlZHZbKZWrVrRjBkzyGazERGR2WymuLg4+vvf/940FWogy5cvp379+lF+fj4REVmtVurbty+NGTOGiIiuXLlCIpGIlixZwt1TWlpK/v7+tGbNGiIimjdvHkVGRlJxcTGX57333qOwsDAyGAykVCopMDCQ3njjDe5312q1FBkZSe+///5dqmnTw5RUM7Ju3TqSSqWk1Wqdrs+aNYsSExObqVSNyzPPPEPDhg2jRYsW0WOPPUbDhg2j9evXcy/Vm2++Sd26dXMRrgMHDqQZM2aQwWAgiURC33//vVP68ePHCQClp6fftbo0BJvNRocPHyYAlJ2dTURVggkAHTp0yCnvxo0bCQBpNBqaM2cO9enTx+V5iYmJNGvWLNJoNCQSiWjTpk1O6X/88QcBoJycnKarVAOx2Wy0dOlSCgoKcroeHR1N8+bNo7lz59Lw4cNp4sSJdPToUS69b9++9MorrzjdU1lZST4+PvTtt99Samoq8Xg8TgE4+Pjjj0kulzdZfe4Eq9VKsbGx9MwzzxAR0eeff07h4eFkMpmc8j355JM0dOhQstvtFB0dTR9++KFTelFREQGgPXv20E8//UQSiYSUSqVTntdee426devWtBW6i7DlvmbkyJEjSExMdPHR1a5dO69ZtrhTiouLsW/fPixduhQREREIDw/HjBkz8MUXXwAAjh49ikGDBrm4/3e0QVpaGgwGAwYPHuySDsBr24nP50OpVAIA59n86NGjEAgE6N+/v1NeR12uXLmCI0eOuNTVkaegoACnTp2CxWLBoEGD3D7Dm9rD0QaO+gNV+20lJSVYtmwZ9uzZg06dOqGiogKDBg3CmTNnYDKZcPr0aZf6+fn5oUWLFigoKMCRI0eclgcdtGvXjosg7U3YbDb885//xOXLl/Hcc88BqHr3k5KS4OPj45TX8TsXFRWhsLDQpR1atmwJHx8frh169OgBmUzm9hl0n5wuYod5mxGdTufSwYCqF7I+wQvvBcrKytC6dWscPnyY8+zdokULfPbZZ3j11VfrbANHuIqgoCCXdABe3U4VFRUQCASQy+UAqn5vqVTqEoW5el10Op1LXR15ysrKuPao2Wbe2h4VFRWIiIjg/lar1bBYLHj00Ufx888/QywWw26346GHHsKKFSuwbNky2O32WvuE1Wr1mA5UuRYKCAhouko1gMLCQkyfPh3Hjh3D6tWr8cgjjwCo6guhoaEu+Wv2+5r15PF4Tnk89RWj0eh1MeNuFzaTakbkcjk32q6OQqHgBNu9zhdffOGkoAAgOTkZ+fn50Ov1dbaB40WumcdhAenN7VRRUYGwsDAu+J5cLodGo3EypACc6xIaGlprezjqe6+0h0KhcFJSQUFB2LZtG7Zs2cL5puPz+RgyZAgyMzMRGBgIgUBQZxt4ShcKhQgMDGy6CjWA33//HfHx8VCr1Th58iRefvllLu12f2ebzQa1Wl2vvnK/wJRUMxITE4NLly45+esCqizAevfu3UylalwGDhzoEhvJYWZLRIiJiXExwQZutUGrVq0AABcvXnRJB4CEhISmKHajYDAYnJZyY2JiQES4fPmyU760tDSEhISgTZs2btuDiJCWlobExETExMQAgEuetLQ0CAQCLhaTt6DX653agMfjYcKECZBIJE75NBoNiAh8Ph/R0dEu9bt+/TpKS0vRu3dvxMTE4Pr161Cr1U550tLSEB8f77KE1hxcv34d48ePx6hRo3Dy5Ekna1agqi/U7NPArX4vl8shkUhc2uHcuXMgIq4dcnJyXAY995P8AJiSalbGjBmDkpISHDp0iLtWUlKCvXv3ckEQ73Vyc3NhNpudrv3444946KGH4O/vjzFjxiA1NRXXr1/n0k+cOIFLly4hKSkJYWFh6NevHzZu3MilExHWr1+Prl27ul3u8BakUqlTfKCEhARERkY61cVms2HDhg1ISkoCj8fDmDFjsG/fPqezLocOHUJRURGSkpIQHR2N7t27u22PXr16uQj/5qZmGxCRi+DV6XTYuXMnhg4dCqDqvdi0aRNsNhuXZ/369RAKhUhMTMSwYcPA4/Hw888/c+l6vR5btmzxmvdm69atEAqF+Prrr90qzTFjxuDSpUvcYAsA8vPzub0qgUCA0aNHY+PGjU57S+vWrUNwcDA6deqEMWPGQKFQOJ2dKi8vR0pKite0Q6PQTAYbjJuMGjWKIiMj6euvv6bNmzdT27ZtKSwsjMrLy5u7aI1CbGwsjR07ljIzM+ncuXM0c+ZMAkAbN24kIiK9Xk8dOnSguLg42rRpE/3nP/8hmUxGPXv2JKvVSkS3rN/mzJlDKSkp9PTTTxMAWrduXXNWzSN2u51+/PFHmjRpEolEIlq4cCGVlJQQUZUFmo+PDy1YsIB2795Nw4cPJx6PRwcPHiSiKhPimJgYio+Ppy1bttDq1aspICCA+vXrx1lAfvfdd8Tn8+n111+nlJQUmjx5MgGgzZs3N1uda2IwGGjlypXUt29fatu2LS1evJgsFgsVFhYSAPr3v/9N+fn5lJqaSv379yc/Pz/OMjEzM5N8fHxo7NixtHv3bpo/fz7xeDyaM2cO9/xZs2ZRQEAALVu2jH755RdKSEggX19fzpKyuZk7dy6Fh4fTnDlz6KmnnqLHHnuMJkyYQFu2bCGiKsvHpKQkiomJobVr19IPP/xAUVFRFB0dzR0tOHToEAGg6dOn0549e2jOnDkEgD766CPue8aPH09hYWG0evVq2rp1K8XGxlJwcDDX3+4HmJJqZrRaLc2dO5eEQiEBoNGjR1NGRkZzF6vROHbsGMXHxxMAAkBRUVG0evVqpzzXr1+nKVOmEI/HIz6fT88++yxdu3aNS7fb7bR161Zq1aoVAaCYmBgXk3RvwmKxUHJyMiUkJFCvXr0oMTGR0tLSiKhKOK1Zs4ZCQkIIAMXFxdHOnTud7r969SqNHz+eAJBQKKSZM2dSaWkpl+5Qgi1atCAA1LZtW9qwYcNdrWNdXL58mfr168e1wYABA0ij0RAR0bfffkvh4eFcn0hISHAxyz958iQlJCQQAAoMDKT333+fO1tGRGQymejDDz8kPz8/AkBJSUlOZuzNzc6dOykxMZGSk5Np0qRJNGPGDJoyZQrNnj2by6NQKOjll18mPp9PAGjixIl0+fJlp+fs37+fOnXqRAAoLCyMPv30U+5sGBGRTqejN954g0QiEQGg4cOH07lz5+5aPe8GzAu6l2C1WmGz2e7LQGd2ux0XLlyAxWJBt27dIBKJ3OYzm83g8Xge04kIBoMBEonknrdcctTFYZHmDpPJBIFA4GINWPMZ92J76PV6ZGVlISAgALGxsR7LbzAYIBaLOeOTmtjtdphMJq9b5mwIFosFRFTrXpper6/1d7bZbLBYLPD19W2qYjYbTEkxGAwGw2thhhMMBoPB8FqYkmIwGAyG18KUFIPBYDC8FqakGAwGg+G1MCXFYDAYDK+FOZhlMLwEk8mEwsJC2Gw2hIaG3hX/a+Xl5W4dnTIY3gKbSTEYzcyQIUPQvXt3+Pv7IzY2Fp07d0ZoaCg6derUpKE3CgsLERYWhqysrNu6/6WXXsLixYu5v41GIzp37oxLly41VhEZDDaTYjCam9zcXISFheHTTz9Fx44dIRQKUVhYiMzMzCYNOeHwFn7lyhV06dKlQfeazWb897//xcqVK7lrhw4dQk5ODiIjIxu1nIwHG6akGIxmRi6XY/DgwZgzZ85d/V6H9wJP3hzcUVhYCIVCgczMTBgMBoSFheHcuXMAqhwHx8XFIT8/H61bt3Yb84nBaChMSTEYzYzJZKrVHVZWVha++uorLFu2DBkZGSAixMfHu+Q7f/48lEolHn74YRdXSrm5udi1axfatm2LUaNGwcfHByaTCUBVkDyLxYK0tDRERUUhKirKbTkMBgO6dOmCyspK7tpjjz3mkq9nz56YM2cOPv/883rVn8GoDaakGIxmRqPRQCwWIz8/H8XFxSgpKUFpaSni4uIwePBgpKSk4Ouvv8Yff/zBzVqmTJmCH3/8EQKBAGVlZZg2bRoXsqFnz57Yv38/F7b9119/xeTJkyGTyaBSqdC2bVucPXuWCwGRnZ2NGTNmID8/HwKBAHv27MHw4cNdyimRSKBWq2G325GUlISRI0di4cKFAICcnBzExcXh/PnziI2N9YqYToz7hObybMtgMIiMRiPnwbrm5/nnnyciojVr1hAAGjduHJ08eZI2b95MAGjv3r1ERDRt2jRq0aIF7dmzhw4fPkxisZg++OADIiKqrKyk8PBweu2118hut5NKpaLVq1eT3W6nkpISAkAikYgmTJhAp0+fpoSEBJo5c2atZXbcd+LECe7aypUrKTIykgsnwmA0FmwmxWA0I3w+HxaLBbNnz8bYsWMRERGBiIgIhIWFcd7gHYEdt2zZApFIhD59+qBLly5ITU1F7969sXHjRnz11VcYNWoUAOC5557DgQMHMH/+fKxbtw52ux0ff/wxeDwegoKCuDDmdHMm1a1bN2zatIl7dklJiduyajQalJaWYufOnQgKCkJQUBAXZXjnzp3o2bMncnJyIJPJEBYW1qTtxnhwYEqKwWhGHFFrhw0bximZmvj7+wOo2hNyKK6wsDAoFAoUFBTAbrejf//+XP42bdrg1KlTAIBjx45h/PjxbkM4OJTU/PnzuedarVbodDq35Xj++eexdetW7u/OnTu75OnYsSOCg4NRUVFxz4UPYXgn7JwUg9GMOAS53W73mMdhVGG1Wrl/c3JyEBUVxZmRl5WVcfmzsrLQvn17AFWKjTxE43GEZ68+6wkMDERpaanb/OvXr8fFixchFAqxdu1aFBUVoaioCGvXrgUAnDlzBkVFRcjLy2MKitFosJkUg9GMSKVSyGQyFBYWOl23WCzg8/kQCAScifjixYsRFxeHX375BWVlZXjqqafQtm1bJCYmYt68efjkk09w7tw5rF+/Hnv27AEATJ48GTNmzMDQoUORnJyMkydPYtWqVVi3bh0MBgOAWzM1AIiJicGVK1dARC6KRiKRIDMzE0KhEJMmTeLuO3v2LOLi4tCrV68mayfGA0zzbokxGIynn36aAJBMJiOZTEZ+fn7E5/PplVdeISKiffv2EQCSy+UEgDp27Ejbtm3j7s/IyKCOHTsSABKLxbRw4ULOgMFut9M777zDGWeIRCJ66aWXyGKxUHZ2NrVv354L605ElJqaSgCorKzMbVlXrVpFs2bNcrr2zDPP0OLFixu7WRgMImLh4xmMZsdms2H//v3Izc2FWq2GUCiEVCrF8OHD0b59e+zbtw/Dhw+HQqGAVCrl9o9qPiM3Nxdyudytzz+NRoPc3Fy0bt2aM013BxEhJycHsbGxjVpHBuN2YUqKwfBy9u7di5EjR0KpVDIvDowHDmY4wWB4OQ6jCmaMwHgQYUqKwfByYmNjMWHCBCcDBwbjQYEt9zEYDAbDa2EzKQaDwWB4LUxJMRgMBsNrYXcEmm4AAAAnSURBVEqKwWAwGF4LU1IMBoPB8FqYkmIwGAyG18KUFIPBYDC8lv8PwyZ5zP8SqEkAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xkcd();\n",
    "plt.xlabel('Epoch #');\n",
    "plt.ylabel('Loss');\n",
    "plt.plot(losses);\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率: 91.0000 %\n"
     ]
    }
   ],
   "source": [
    "cnn.eval()\n",
    "correct = 0\n",
    "total = 0\n",
    "for images, labels in test_loader:\n",
    "    images = images.float().to(DEVICE)\n",
    "    outputs = cnn(images).cpu()\n",
    "    _, predicted = torch.max(outputs.data, 1)\n",
    "    total += labels.size(0)\n",
    "    correct += (predicted == labels).sum()\n",
    "print('准确率: %.4f %%' % (100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "损失小了，但是准确率没有提高，这就说明已经接近模型的瓶颈了，如果再要进行优化，就需要修改模型了。另外还有一个判断模型是否到瓶颈的标准，就是看损失函数，最后一次的训练的损失函数明显的没有下降的趋势，只是在震荡，这说明已经没有什么优化的空间了。\n",
    "\n",
    "通过简单的操作，我们也能够看到Adam优化器的暴力性，我们只要简单的修改学习率就能够达到优化的效果，Adam优化器的使用一般情况下是首先使用0.1进行预热，然后再用0.01进行大批次的训练，最后使用0.001这个学习率进行收尾，再小的学习率一般情况就不需要了。\n",
    "\n",
    "## 总结\n",
    "最后我们再总结一下几个超参数:\n",
    "\n",
    "`BATCH_SIZE`: 批次数量，定义每次训练时多少数据作为一批，这个批次需要在dataloader初始化时进行设置，并且需要这对模型和显存进行配置，如果出现OOM有线减小，一般设为2的倍数\n",
    "\n",
    "`DEVICE`：进行计算的设备，主要是CPU还是GPU\n",
    "\n",
    "`LEARNING_RATE`：学习率，反向传播时使用\n",
    "\n",
    "`TOTAL_EPOCHS`：训练的批次，一般情况下会根据损失和准确率等阈值\n",
    "\n",
    "其实优化器和损失函数也算超参数，这里就不说了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.13 ('MAC-GPU')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "vscode": {
   "interpreter": {
    "hash": "82ebbbba6002e52ffd5c3db618d60685134ba8ae181eb7df3dc9e0c9fc17132c"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}