6_1_using_word_embeddings(김창우)4월21일.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "6_1_using_word_embeddings(김창우)4월21일.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/kcw0331/Deeplearning/blob/main/6_1_using_word_embeddings(%EA%B9%80%EC%B0%BD%EC%9A%B0)4%EC%9B%9421%EC%9D%BC.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "id": "zJiaT_3QXrAj",
        "outputId": "e0fdcc3c-9824-44dd-e548-87973ce25f01"
      },
      "source": [
        "import tensorflow\n",
        "tensorflow.keras.__version__"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.4.0'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w_Felw0jXrAo",
        "outputId": "80b0f0db-e6f6-4ad2-a725-fbe470bd36c4"
      },
      "source": [
        "tensorflow.executing_eagerly()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zz2fIjgtXrAp"
      },
      "source": [
        "# Using word embeddings\n",
        "\n",
        "This notebook contains the second code sample found in Chapter 6, Section 1 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "Another popular and powerful way to associate a vector with a word is the use of dense \"word vectors\", also called \"word embeddings\". \n",
        "While the vectors obtained through one-hot encoding are binary, sparse (mostly made of zeros) and very high-dimensional (same dimensionality as the \n",
        "number of words in the vocabulary), \"word embeddings\" are low-dimensional floating point vectors \n",
        "(i.e. \"dense\" vectors, as opposed to sparse vectors). \n",
        "Unlike word vectors obtained via one-hot encoding, word embeddings are learned from data. \n",
        "It is common to see word embeddings that are 256-dimensional, 512-dimensional, or 1024-dimensional when dealing with very large vocabularies. \n",
        "On the other hand, one-hot encoding words generally leads to vectors that are 20,000-dimensional or higher (capturing a vocabulary of 20,000 \n",
        "token in this case). So, word embeddings pack more information into far fewer dimensions. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UNjv868vXrAp"
      },
      "source": [
        "![word embeddings vs. one hot encoding](https://s3.amazonaws.com/book.keras.io/img/ch6/word_embeddings.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SaRahwBcXrAq"
      },
      "source": [
        "There are two ways to obtain word embeddings:\n",
        "\n",
        "* Learn word embeddings jointly with the main task you care about (e.g. document classification or sentiment prediction). \n",
        "In this setup, you would start with random word vectors, then learn your word vectors in the same way that you learn the weights of a neural network.\n",
        "* Load into your model word embeddings that were pre-computed using a different machine learning task than the one you are trying to solve. \n",
        "These are called \"pre-trained word embeddings\". \n",
        "\n",
        "Let's take a look at both."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LAZeLLnDXrAq"
      },
      "source": [
        "## Learning word embeddings with the `Embedding` layer\n",
        "\n",
        "\n",
        "The simplest way to associate a dense vector to a word would be to pick the vector at random. The problem with this approach is that the \n",
        "resulting embedding space would have no structure: for instance, the words \"accurate\" and \"exact\" may end up with completely different \n",
        "embeddings, even though they are interchangeable in most sentences. It would be very difficult for a deep neural network to make sense of \n",
        "such a noisy, unstructured embedding space. \n",
        "\n",
        "To get a bit more abstract: the geometric relationships between word vectors should reflect the semantic relationships between these words. \n",
        "Word embeddings are meant to map human language into a geometric space. For instance, in a reasonable embedding space, we would expect \n",
        "synonyms to be embedded into similar word vectors, and in general we would expect the geometric distance (e.g. L2 distance) between any two \n",
        "word vectors to relate to the semantic distance of the associated words (words meaning very different things would be embedded to points \n",
        "far away from each other, while related words would be closer). Even beyond mere distance, we may want specific __directions__ in the \n",
        "embedding space to be meaningful. \n",
        "\n",
        "[...]\n",
        "\n",
        "\n",
        "In real-world word embedding spaces, common examples of meaningful geometric transformations are \"gender vectors\" and \"plural vector\". For \n",
        "instance, by adding a \"female vector\" to the vector \"king\", one obtain the vector \"queen\". By adding a \"plural vector\", one obtain \"kings\". \n",
        "Word embedding spaces typically feature thousands of such interpretable and potentially useful vectors.\n",
        "\n",
        "Is there some \"ideal\" word embedding space that would perfectly map human language and could be used for any natural language processing \n",
        "task? Possibly, but in any case, we have yet to compute anything of the sort. Also, there isn't such a thing as \"human language\", there are \n",
        "many different languages and they are not isomorphic, as a language is the reflection of a specific culture and a specific context. But more \n",
        "pragmatically, what makes a good word embedding space depends heavily on your task: the perfect word embedding space for an \n",
        "English-language movie review sentiment analysis model may look very different from the perfect embedding space for an English-language \n",
        "legal document classification model, because the importance of certain semantic relationships varies from task to task.\n",
        "\n",
        "It is thus reasonable to __learn__ a new embedding space with every new task. Thankfully, backpropagation makes this really easy, and Keras makes it \n",
        "even easier. It's just about learning the weights of a layer: the `Embedding` layer."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pcwASkmBO3gM"
      },
      "source": [
        "- 교수님이 word embeding을 하려면 embeding layer를 추가해주는 방법을 사용해준다고 말씀하심."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N7WzU-xDXrAr"
      },
      "source": [
        "from tensorflow.keras.layers import Embedding\n",
        "\n",
        "# The Embedding layer takes at least two arguments:\n",
        "# the number of possible tokens, here 1000 (1 + maximum word index),\n",
        "# and the dimensionality of the embeddings, here 64.\n",
        "embedding_layer = Embedding(1000, 64)  #그래서 모형을 define 할 때, embeding_layer를 넣어주는 방식으로 한다고 말씀하심."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QejDj4KQXrAr"
      },
      "source": [
        "\n",
        "The `Embedding` layer is best understood as a dictionary mapping integer indices (which stand for specific words) to dense vectors. It takes \n",
        "as input integers, it looks up these integers into an internal dictionary, and it returns the associated vectors. It's effectively a dictionary lookup."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5JkxGuznXrAr"
      },
      "source": [
        "\n",
        "The `Embedding` layer takes as input a 2D tensor of integers, of shape `(samples, sequence_length)`, where each entry is a sequence of \n",
        "integers. It can embed sequences of variable lengths, so for instance we could feed into our embedding layer above batches that could have \n",
        "shapes `(32, 10)` (batch of 32 sequences of length 10) or `(64, 15)` (batch of 64 sequences of length 15). All sequences in a batch must \n",
        "have the same length, though (since we need to pack them into a single tensor), so sequences that are shorter than others should be padded \n",
        "with zeros, and sequences that are longer should be truncated.\n",
        "\n",
        "This layer returns a 3D floating point tensor, of shape `(samples, sequence_length, embedding_dimensionality)`. Such a 3D tensor can then \n",
        "be processed by a RNN layer or a 1D convolution layer (both will be introduced in the next sections).\n",
        "\n",
        "When you instantiate an `Embedding` layer, its weights (its internal dictionary of token vectors) are initially random, just like with any \n",
        "other layer. During training, these word vectors will be gradually adjusted via backpropagation, structuring the space into something that the \n",
        "downstream model can exploit. Once fully trained, your embedding space will show a lot of structure -- a kind of structure specialized for \n",
        "the specific problem you were training your model for.\n",
        "\n",
        "Let's apply this idea to the IMDB movie review sentiment prediction task that you are already familiar with. Let's quickly prepare \n",
        "the data. We will restrict the movie reviews to the top 10,000 most common words (like we did the first time we worked with this dataset), \n",
        "and cut the reviews after only 20 words. Our network will simply learn 8-dimensional embeddings for each of the 10,000 words, turn the \n",
        "input integer sequences (2D integer tensor) into embedded sequences (3D float tensor), flatten the tensor to 2D, and train a single `Dense` \n",
        "layer on top for classification."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hKEhk6VpPdGI"
      },
      "source": [
        "- 예제에서 IMDB데이터를 읽어 올 때, 10000개의 most common words로 해당 데이터를 읽어 온다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h0IK7p6MXrAs",
        "outputId": "13550e54-3d97-40bc-f8db-e190d1988691"
      },
      "source": [
        "from tensorflow.keras.datasets import imdb\n",
        "from tensorflow.keras import preprocessing\n",
        "\n",
        "# Number of words to consider as features\n",
        "max_features = 10000\n",
        "# Cut texts after this number of words \n",
        "# (among top max_features most common words)\n",
        "maxlen = 20\n",
        "\n",
        "# Load the data as lists of integers.\n",
        "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
        "\n",
        "# This turns our lists of integers\n",
        "# into a 2D integer tensor of shape `(samples, maxlen)`\n",
        "x_train = preprocessing.sequence.pad_sequences(x_train, maxlen=maxlen)\n",
        "x_test = preprocessing.sequence.pad_sequences(x_test, maxlen=maxlen)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n",
            "17465344/17464789 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "<string>:6: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/datasets/imdb.py:159: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_train, y_train = np.array(xs[:idx]), np.array(labels[:idx])\n",
            "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/datasets/imdb.py:160: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_test, y_test = np.array(xs[idx:]), np.array(labels[idx:])\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2pTn1WBrP_de"
      },
      "source": [
        "- maxlen = 20이걸 사용했기 때문에 x_train, x_test를 해주게 되면 len이 20개가 나오는 것을 볼 수 있다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qZkNE-8PXrAs",
        "outputId": "4dc7741e-b22a-4bea-d2a3-d73f7da09072"
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Flatten, Dense\n",
        "\n",
        "model = Sequential()\n",
        "# We specify the maximum input length to our Embedding layer\n",
        "# so we can later flatten the embedded inputs\n",
        "model.add(Embedding(10000, 8, input_length=maxlen))  #maxlen을 해줘서 20개의 token index를 준다. 그럼 그 token index를 8차원 공간에 embeding시켜가지고 성능을 살펴보는 것이다.\n",
        "# After the Embedding layer, \n",
        "# our activations have shape `(samples, maxlen, 8)`.\n",
        "\n",
        "# We flatten the 3D tensor of embeddings \n",
        "# into a 2D tensor of shape `(samples, maxlen * 8)`\n",
        "model.add(Flatten())  #embeding만 해가지고 바로 Flatten시켜가지고 \n",
        "\n",
        "# We add the classifier on top\n",
        "model.add(Dense(1, activation='sigmoid'))  #output해주는 레이어를 달아서 결과를 냈다. \n",
        "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(x_train, y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=32,\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_1 (Embedding)      (None, 20, 8)             80000     \n",
            "_________________________________________________________________\n",
            "flatten (Flatten)            (None, 160)               0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 1)                 161       \n",
            "=================================================================\n",
            "Total params: 80,161\n",
            "Trainable params: 80,161\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "Epoch 1/10\n",
            "625/625 [==============================] - 2s 2ms/step - loss: 0.6867 - acc: 0.5559 - val_loss: 0.6225 - val_acc: 0.6958\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.5743 - acc: 0.7466 - val_loss: 0.5265 - val_acc: 0.7298\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.4702 - acc: 0.7858 - val_loss: 0.5008 - val_acc: 0.7458\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.4191 - acc: 0.8135 - val_loss: 0.4964 - val_acc: 0.7514\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.3950 - acc: 0.8226 - val_loss: 0.4949 - val_acc: 0.7596\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.3705 - acc: 0.8361 - val_loss: 0.4989 - val_acc: 0.7598\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.3421 - acc: 0.8517 - val_loss: 0.5041 - val_acc: 0.7592\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.3232 - acc: 0.8634 - val_loss: 0.5119 - val_acc: 0.7578\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.3164 - acc: 0.8678 - val_loss: 0.5196 - val_acc: 0.7552\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 1s 1ms/step - loss: 0.2916 - acc: 0.8816 - val_loss: 0.5292 - val_acc: 0.7550\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nnc_2OMAQTcc"
      },
      "source": [
        "- 이런 애들을 딥러닝 모형에 넣는데, embeding_layer를 사용해가지고 word embeding 실습을 해보는 것이다. \n",
        "- 그리고 마지막 결과를 보았을 때, val_acc가 74%가 되는 것을 볼 수 있다. 모든 review에 앞에 20단어만 보았을 때, 74%정도의 정확도로 판단하는게 나쁘지는 않다고 하심. 그래서 이런정도의 성능을 확인 할 수 있었다. \n",
        "- 그리고 나중에 recurrent layers 나 1D convolutional layers를 배우면 더 성능이 좋아지는 것을 기대해 볼 수 있다고 말씀하심."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IIyF7Ju8XrAt"
      },
      "source": [
        "We get to a validation accuracy of ~76%, which is pretty good considering that we only look at the first 20 words in every review. But \n",
        "note that merely flattening the embedded sequences and training a single `Dense` layer on top leads to a model that treats each word in the \n",
        "input sequence separately, without considering inter-word relationships and structure sentence (e.g. it would likely treat both _\"this movie \n",
        "is shit\"_ and _\"this movie is the shit\"_ as being negative \"reviews\"). It would be much better to add recurrent layers or 1D convolutional \n",
        "layers on top of the embedded sequences to learn features that take into account each sequence as a whole. That's what we will focus on in \n",
        "the next few sections."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "npeV-qL1XrAt"
      },
      "source": [
        "## Using pre-trained word embeddings\n",
        "\n",
        "\n",
        "Sometimes, you have so little training data available that could never use your data alone to learn an appropriate task-specific embedding \n",
        "of your vocabulary. What to do then?\n",
        "\n",
        "Instead of learning word embeddings jointly with the problem you want to solve, you could be loading embedding vectors from a pre-computed \n",
        "embedding space known to be highly structured and to exhibit useful properties -- that captures generic aspects of language structure. The \n",
        "rationale behind using pre-trained word embeddings in natural language processing is very much the same as for using pre-trained convnets \n",
        "in image classification: we don't have enough data available to learn truly powerful features on our own, but we expect the features that \n",
        "we need to be fairly generic, i.e. common visual features or semantic features. In this case it makes sense to reuse features learned on a \n",
        "different problem.\n",
        "\n",
        "Such word embeddings are generally computed using word occurrence statistics (observations about what words co-occur in sentences or \n",
        "documents), using a variety of techniques, some involving neural networks, others not. The idea of a dense, low-dimensional embedding space \n",
        "for words, computed in an unsupervised way, was initially explored by Bengio et al. in the early 2000s, but it only started really taking \n",
        "off in research and industry applications after the release of one of the most famous and successful word embedding scheme: the Word2Vec \n",
        "algorithm, developed by Mikolov at Google in 2013. Word2Vec dimensions capture specific semantic properties, e.g. gender.\n",
        "\n",
        "There are various pre-computed databases of word embeddings that can download and start using in a Keras `Embedding` layer. Word2Vec is one \n",
        "of them. Another popular one is called \"GloVe\", developed by Stanford researchers in 2014. It stands for \"Global Vectors for Word \n",
        "Representation\", and it is an embedding technique based on factorizing a matrix of word co-occurrence statistics. Its developers have made \n",
        "available pre-computed embeddings for millions of English tokens, obtained from Wikipedia data or from Common Crawl data.\n",
        "\n",
        "Let's take a look at how you can get started using GloVe embeddings in a Keras model. The same method will of course be valid for Word2Vec \n",
        "embeddings or any other word embedding database that you can download. We will also use this example to refresh the text tokenization \n",
        "techniques we introduced a few paragraphs ago: we will start from raw text, and work our way up."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wgf_-Z98XrAt"
      },
      "source": [
        "## Putting it all together: from raw text to word embeddings\n",
        "\n",
        "\n",
        "We will be using a model similar to the one we just went over -- embedding sentences in sequences of vectors, flattening them and training a \n",
        "`Dense` layer on top. But we will do it using pre-trained word embeddings, and instead of using the pre-tokenized IMDB data packaged in \n",
        "Keras, we will start from scratch, by downloading the original text data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hk9OYkOmXrAu"
      },
      "source": [
        "### Download the IMDB data as raw text\n",
        "\n",
        "\n",
        "First, head to `http://ai.stanford.edu/~amaas/data/sentiment/` and download the raw IMDB dataset (if the URL isn't working anymore, just \n",
        "Google \"IMDB dataset\"). Uncompress it.\n",
        "\n",
        "Now let's collect the individual training reviews into a list of strings, one string per review, and let's also collect the review labels \n",
        "(positive / negative) into a `labels` list:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6Kv9ToLDXrAu"
      },
      "source": [
        "import os\n",
        "\n",
        "#imdb_dir = '/home/Data/aclImdb'\n",
        "imdb_dir = '/Users/김창우/Desktop/data/aclImdb'\n",
        "train_dir = os.path.join(imdb_dir, 'train')\n",
        "\n",
        "labels = []   #여기있는 labels과 texts에 append를 시켜서 데이터를 정리해준다. \n",
        "texts = []\n",
        "\n",
        "for label_type in ['neg', 'pos']:\n",
        "    dir_name = os.path.join(train_dir, label_type)\n",
        "    for fname in os.listdir(dir_name):\n",
        "        if fname[-4:] == '.txt':\n",
        "            f = open(os.path.join(dir_name, fname), encoding='UTF-8')\n",
        "            texts.append(f.read())\n",
        "            f.close()\n",
        "            if label_type == 'neg':\n",
        "                labels.append(0)\n",
        "            else:\n",
        "                labels.append(1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5B1mM2_MrFxO",
        "outputId": "7a3ec9b8-b611-44a6-9847-f40b13eaaf7d"
      },
      "source": [
        "len(labels)   #여기에는 25000개의 labels이있고 "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "25000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w9M345wPrFxO",
        "outputId": "736114de-b744-4c5a-a431-8942d1dd59be"
      },
      "source": [
        "sum(labels)   #25000개 중에서 반은 positive이고 반은 negative이다. 그래서 완전 밸런스된 데이터이다.   "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "12500"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "urlEo5NWrFxO",
        "outputId": "b77f36a3-9c93-4f19-84ae-7562b1c83e95"
      },
      "source": [
        "dir_name   #어떤식으로 파일이 구성되어 있는지를 dir_name으로 해서 볼 수 있다."
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'/Users/김창우/Desktop/data/aclImdb\\\\train\\\\pos'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fCm-zfvdrFxO",
        "outputId": "fb4c8a6c-25f2-48bc-9abc-51fd16fd6ac7"
      },
      "source": [
        "os.listdir(dir_name)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['0_9.txt',\n",
              " '10000_8.txt',\n",
              " '10001_10.txt',\n",
              " '10002_7.txt',\n",
              " '10003_8.txt',\n",
              " '10004_8.txt',\n",
              " '10005_7.txt',\n",
              " '10006_7.txt',\n",
              " '10007_7.txt',\n",
              " '10008_7.txt',\n",
              " '10009_9.txt',\n",
              " '1000_8.txt',\n",
              " '10010_7.txt',\n",
              " '10011_9.txt',\n",
              " '10012_8.txt',\n",
              " '10013_7.txt',\n",
              " '10014_8.txt',\n",
              " '10015_8.txt',\n",
              " '10016_8.txt',\n",
              " '10017_9.txt',\n",
              " '10018_8.txt',\n",
              " '10019_8.txt',\n",
              " '1001_8.txt',\n",
              " '10020_8.txt',\n",
              " '10021_8.txt',\n",
              " '10022_7.txt',\n",
              " '10023_9.txt',\n",
              " '10024_9.txt',\n",
              " '10025_9.txt',\n",
              " '10026_7.txt',\n",
              " '10027_7.txt',\n",
              " '10028_10.txt',\n",
              " '10029_10.txt',\n",
              " '1002_7.txt',\n",
              " '10030_10.txt',\n",
              " '10031_10.txt',\n",
              " '10032_10.txt',\n",
              " '10033_10.txt',\n",
              " '10034_8.txt',\n",
              " '10035_9.txt',\n",
              " '10036_8.txt',\n",
              " '10037_9.txt',\n",
              " '10038_10.txt',\n",
              " '10039_10.txt',\n",
              " '1003_10.txt',\n",
              " '10040_10.txt',\n",
              " '10041_10.txt',\n",
              " '10042_10.txt',\n",
              " '10043_10.txt',\n",
              " '10044_9.txt',\n",
              " '10045_10.txt',\n",
              " '10046_9.txt',\n",
              " '10047_10.txt',\n",
              " '10048_10.txt',\n",
              " '10049_8.txt',\n",
              " '1004_7.txt',\n",
              " '10050_10.txt',\n",
              " '10051_10.txt',\n",
              " '10052_10.txt',\n",
              " '10053_8.txt',\n",
              " '10054_10.txt',\n",
              " '10055_7.txt',\n",
              " '10056_8.txt',\n",
              " '10057_9.txt',\n",
              " '10058_7.txt',\n",
              " '10059_10.txt',\n",
              " '1005_10.txt',\n",
              " '10060_9.txt',\n",
              " '10061_8.txt',\n",
              " '10062_10.txt',\n",
              " '10063_9.txt',\n",
              " '10064_10.txt',\n",
              " '10065_9.txt',\n",
              " '10066_10.txt',\n",
              " '10067_9.txt',\n",
              " '10068_8.txt',\n",
              " '10069_8.txt',\n",
              " '1006_8.txt',\n",
              " '10070_9.txt',\n",
              " '10071_9.txt',\n",
              " '10072_9.txt',\n",
              " '10073_10.txt',\n",
              " '10074_9.txt',\n",
              " '10075_9.txt',\n",
              " '10076_9.txt',\n",
              " '10077_10.txt',\n",
              " '10078_8.txt',\n",
              " '10079_8.txt',\n",
              " '1007_10.txt',\n",
              " '10080_10.txt',\n",
              " '10081_9.txt',\n",
              " '10082_10.txt',\n",
              " '10083_7.txt',\n",
              " '10084_10.txt',\n",
              " '10085_10.txt',\n",
              " '10086_7.txt',\n",
              " '10087_10.txt',\n",
              " '10088_10.txt',\n",
              " '10089_7.txt',\n",
              " '1008_10.txt',\n",
              " '10090_8.txt',\n",
              " '10091_7.txt',\n",
              " '10092_8.txt',\n",
              " '10093_7.txt',\n",
              " '10094_7.txt',\n",
              " '10095_7.txt',\n",
              " '10096_7.txt',\n",
              " '10097_9.txt',\n",
              " '10098_10.txt',\n",
              " '10099_10.txt',\n",
              " '1009_8.txt',\n",
              " '100_7.txt',\n",
              " '10100_10.txt',\n",
              " '10101_8.txt',\n",
              " '10102_7.txt',\n",
              " '10103_8.txt',\n",
              " '10104_10.txt',\n",
              " '10105_8.txt',\n",
              " '10106_8.txt',\n",
              " '10107_8.txt',\n",
              " '10108_10.txt',\n",
              " '10109_10.txt',\n",
              " '1010_10.txt',\n",
              " '10110_10.txt',\n",
              " '10111_7.txt',\n",
              " '10112_7.txt',\n",
              " '10113_10.txt',\n",
              " '10114_10.txt',\n",
              " '10115_10.txt',\n",
              " '10116_10.txt',\n",
              " '10117_8.txt',\n",
              " '10118_7.txt',\n",
              " '10119_7.txt',\n",
              " '1011_10.txt',\n",
              " '10120_7.txt',\n",
              " '10121_8.txt',\n",
              " '10122_7.txt',\n",
              " '10123_10.txt',\n",
              " '10124_8.txt',\n",
              " '10125_8.txt',\n",
              " '10126_10.txt',\n",
              " '10127_8.txt',\n",
              " '10128_9.txt',\n",
              " '10129_7.txt',\n",
              " '1012_10.txt',\n",
              " '10130_10.txt',\n",
              " '10131_10.txt',\n",
              " '10132_9.txt',\n",
              " '10133_7.txt',\n",
              " '10134_7.txt',\n",
              " '10135_7.txt',\n",
              " '10136_7.txt',\n",
              " '10137_7.txt',\n",
              " '10138_8.txt',\n",
              " '10139_8.txt',\n",
              " '1013_9.txt',\n",
              " '10140_8.txt',\n",
              " '10141_9.txt',\n",
              " '10142_8.txt',\n",
              " '10143_8.txt',\n",
              " '10144_8.txt',\n",
              " '10145_8.txt',\n",
              " '10146_7.txt',\n",
              " '10147_10.txt',\n",
              " '10148_10.txt',\n",
              " '10149_9.txt',\n",
              " '1014_9.txt',\n",
              " '10150_9.txt',\n",
              " '10151_8.txt',\n",
              " '10152_9.txt',\n",
              " '10153_9.txt',\n",
              " '10154_8.txt',\n",
              " '10155_9.txt',\n",
              " '10156_10.txt',\n",
              " '10157_10.txt',\n",
              " '10158_7.txt',\n",
              " '10159_7.txt',\n",
              " '1015_10.txt',\n",
              " '10160_7.txt',\n",
              " '10161_9.txt',\n",
              " '10162_9.txt',\n",
              " '10163_8.txt',\n",
              " '10164_7.txt',\n",
              " '10165_7.txt',\n",
              " '10166_7.txt',\n",
              " '10167_7.txt',\n",
              " '10168_8.txt',\n",
              " '10169_7.txt',\n",
              " '1016_8.txt',\n",
              " '10170_8.txt',\n",
              " '10171_7.txt',\n",
              " '10172_8.txt',\n",
              " '10173_8.txt',\n",
              " '10174_7.txt',\n",
              " '10175_10.txt',\n",
              " '10176_7.txt',\n",
              " '10177_9.txt',\n",
              " '10178_10.txt',\n",
              " '10179_9.txt',\n",
              " '1017_8.txt',\n",
              " '10180_7.txt',\n",
              " '10181_8.txt',\n",
              " '10182_8.txt',\n",
              " '10183_7.txt',\n",
              " '10184_9.txt',\n",
              " '10185_10.txt',\n",
              " '10186_8.txt',\n",
              " '10187_7.txt',\n",
              " '10188_8.txt',\n",
              " '10189_7.txt',\n",
              " '1018_8.txt',\n",
              " '10190_7.txt',\n",
              " '10191_10.txt',\n",
              " '10192_8.txt',\n",
              " '10193_9.txt',\n",
              " '10194_10.txt',\n",
              " '10195_8.txt',\n",
              " '10196_10.txt',\n",
              " '10197_7.txt',\n",
              " '10198_8.txt',\n",
              " '10199_7.txt',\n",
              " '1019_10.txt',\n",
              " '101_8.txt',\n",
              " '10200_10.txt',\n",
              " '10201_10.txt',\n",
              " '10202_10.txt',\n",
              " '10203_10.txt',\n",
              " '10204_8.txt',\n",
              " '10205_10.txt',\n",
              " '10206_10.txt',\n",
              " '10207_10.txt',\n",
              " '10208_7.txt',\n",
              " '10209_7.txt',\n",
              " '1020_10.txt',\n",
              " '10210_7.txt',\n",
              " '10211_7.txt',\n",
              " '10212_8.txt',\n",
              " '10213_8.txt',\n",
              " '10214_10.txt',\n",
              " '10215_10.txt',\n",
              " '10216_8.txt',\n",
              " '10217_9.txt',\n",
              " '10218_8.txt',\n",
              " '10219_10.txt',\n",
              " '1021_10.txt',\n",
              " '10220_7.txt',\n",
              " '10221_8.txt',\n",
              " '10222_9.txt',\n",
              " '10223_10.txt',\n",
              " '10224_10.txt',\n",
              " '10225_9.txt',\n",
              " '10226_10.txt',\n",
              " '10227_10.txt',\n",
              " '10228_8.txt',\n",
              " '10229_8.txt',\n",
              " '1022_10.txt',\n",
              " '10230_9.txt',\n",
              " '10231_10.txt',\n",
              " '10232_10.txt',\n",
              " '10233_7.txt',\n",
              " '10234_10.txt',\n",
              " '10235_8.txt',\n",
              " '10236_8.txt',\n",
              " '10237_10.txt',\n",
              " '10238_10.txt',\n",
              " '10239_10.txt',\n",
              " '1023_10.txt',\n",
              " '10240_8.txt',\n",
              " '10241_8.txt',\n",
              " '10242_8.txt',\n",
              " '10243_10.txt',\n",
              " '10244_7.txt',\n",
              " '10245_10.txt',\n",
              " '10246_10.txt',\n",
              " '10247_10.txt',\n",
              " '10248_7.txt',\n",
              " '10249_7.txt',\n",
              " '1024_9.txt',\n",
              " '10250_10.txt',\n",
              " '10251_10.txt',\n",
              " '10252_9.txt',\n",
              " '10253_10.txt',\n",
              " '10254_8.txt',\n",
              " '10255_9.txt',\n",
              " '10256_8.txt',\n",
              " '10257_8.txt',\n",
              " '10258_10.txt',\n",
              " '10259_8.txt',\n",
              " '1025_8.txt',\n",
              " '10260_10.txt',\n",
              " '10261_8.txt',\n",
              " '10262_10.txt',\n",
              " '10263_10.txt',\n",
              " '10264_10.txt',\n",
              " '10265_9.txt',\n",
              " '10266_9.txt',\n",
              " '10267_8.txt',\n",
              " '10268_9.txt',\n",
              " '10269_7.txt',\n",
              " '1026_9.txt',\n",
              " '10270_9.txt',\n",
              " '10271_10.txt',\n",
              " '10272_10.txt',\n",
              " '10273_8.txt',\n",
              " '10274_8.txt',\n",
              " '10275_10.txt',\n",
              " '10276_10.txt',\n",
              " '10277_9.txt',\n",
              " '10278_7.txt',\n",
              " '10279_8.txt',\n",
              " '1027_8.txt',\n",
              " '10280_10.txt',\n",
              " '10281_7.txt',\n",
              " '10282_8.txt',\n",
              " '10283_10.txt',\n",
              " '10284_9.txt',\n",
              " '10285_10.txt',\n",
              " '10286_9.txt',\n",
              " '10287_8.txt',\n",
              " '10288_10.txt',\n",
              " '10289_10.txt',\n",
              " '1028_10.txt',\n",
              " '10290_8.txt',\n",
              " '10291_7.txt',\n",
              " '10292_7.txt',\n",
              " '10293_8.txt',\n",
              " '10294_8.txt',\n",
              " '10295_7.txt',\n",
              " '10296_8.txt',\n",
              " '10297_8.txt',\n",
              " '10298_9.txt',\n",
              " '10299_9.txt',\n",
              " '1029_9.txt',\n",
              " '102_10.txt',\n",
              " '10300_10.txt',\n",
              " '10301_8.txt',\n",
              " '10302_9.txt',\n",
              " '10303_7.txt',\n",
              " '10304_7.txt',\n",
              " '10305_8.txt',\n",
              " '10306_8.txt',\n",
              " '10307_8.txt',\n",
              " '10308_8.txt',\n",
              " '10309_7.txt',\n",
              " '1030_10.txt',\n",
              " '10310_9.txt',\n",
              " '10311_9.txt',\n",
              " '10312_10.txt',\n",
              " '10313_7.txt',\n",
              " '10314_8.txt',\n",
              " '10315_8.txt',\n",
              " '10316_8.txt',\n",
              " '10317_7.txt',\n",
              " '10318_7.txt',\n",
              " '10319_7.txt',\n",
              " '1031_10.txt',\n",
              " '10320_7.txt',\n",
              " '10321_10.txt',\n",
              " '10322_7.txt',\n",
              " '10323_10.txt',\n",
              " '10324_9.txt',\n",
              " '10325_10.txt',\n",
              " '10326_10.txt',\n",
              " '10327_7.txt',\n",
              " '10328_8.txt',\n",
              " '10329_8.txt',\n",
              " '1032_7.txt',\n",
              " '10330_8.txt',\n",
              " '10331_10.txt',\n",
              " '10332_8.txt',\n",
              " '10333_8.txt',\n",
              " '10334_8.txt',\n",
              " '10335_8.txt',\n",
              " '10336_8.txt',\n",
              " '10337_9.txt',\n",
              " '10338_9.txt',\n",
              " '10339_7.txt',\n",
              " '1033_10.txt',\n",
              " '10340_9.txt',\n",
              " '10341_7.txt',\n",
              " '10342_7.txt',\n",
              " '10343_7.txt',\n",
              " '10344_7.txt',\n",
              " '10345_7.txt',\n",
              " '10346_9.txt',\n",
              " '10347_9.txt',\n",
              " '10348_8.txt',\n",
              " '10349_10.txt',\n",
              " '1034_7.txt',\n",
              " '10350_10.txt',\n",
              " '10351_8.txt',\n",
              " '10352_10.txt',\n",
              " '10353_9.txt',\n",
              " '10354_9.txt',\n",
              " '10355_9.txt',\n",
              " '10356_9.txt',\n",
              " '10357_8.txt',\n",
              " '10358_9.txt',\n",
              " '10359_7.txt',\n",
              " '1035_7.txt',\n",
              " '10360_8.txt',\n",
              " '10361_7.txt',\n",
              " '10362_8.txt',\n",
              " '10363_9.txt',\n",
              " '10364_10.txt',\n",
              " '10365_8.txt',\n",
              " '10366_10.txt',\n",
              " '10367_8.txt',\n",
              " '10368_7.txt',\n",
              " '10369_8.txt',\n",
              " '1036_9.txt',\n",
              " '10370_9.txt',\n",
              " '10371_8.txt',\n",
              " '10372_7.txt',\n",
              " '10373_7.txt',\n",
              " '10374_8.txt',\n",
              " '10375_10.txt',\n",
              " '10376_7.txt',\n",
              " '10377_9.txt',\n",
              " '10378_8.txt',\n",
              " '10379_10.txt',\n",
              " '1037_8.txt',\n",
              " '10380_10.txt',\n",
              " '10381_10.txt',\n",
              " '10382_10.txt',\n",
              " '10383_10.txt',\n",
              " '10384_10.txt',\n",
              " '10385_10.txt',\n",
              " '10386_8.txt',\n",
              " '10387_7.txt',\n",
              " '10388_7.txt',\n",
              " '10389_10.txt',\n",
              " '1038_7.txt',\n",
              " '10390_10.txt',\n",
              " '10391_10.txt',\n",
              " '10392_10.txt',\n",
              " '10393_9.txt',\n",
              " '10394_10.txt',\n",
              " '10395_8.txt',\n",
              " '10396_8.txt',\n",
              " '10397_8.txt',\n",
              " '10398_8.txt',\n",
              " '10399_10.txt',\n",
              " '1039_9.txt',\n",
              " '103_7.txt',\n",
              " '10400_10.txt',\n",
              " '10401_10.txt',\n",
              " '10402_10.txt',\n",
              " '10403_7.txt',\n",
              " '10404_9.txt',\n",
              " '10405_8.txt',\n",
              " '10406_10.txt',\n",
              " '10407_8.txt',\n",
              " '10408_10.txt',\n",
              " '10409_10.txt',\n",
              " '1040_10.txt',\n",
              " '10410_10.txt',\n",
              " '10411_9.txt',\n",
              " '10412_8.txt',\n",
              " '10413_10.txt',\n",
              " '10414_10.txt',\n",
              " '10415_7.txt',\n",
              " '10416_9.txt',\n",
              " '10417_8.txt',\n",
              " '10418_9.txt',\n",
              " '10419_10.txt',\n",
              " '1041_9.txt',\n",
              " '10420_10.txt',\n",
              " '10421_7.txt',\n",
              " '10422_7.txt',\n",
              " '10423_9.txt',\n",
              " '10424_9.txt',\n",
              " '10425_9.txt',\n",
              " '10426_9.txt',\n",
              " '10427_8.txt',\n",
              " '10428_10.txt',\n",
              " '10429_10.txt',\n",
              " '1042_10.txt',\n",
              " '10430_9.txt',\n",
              " '10431_10.txt',\n",
              " '10432_10.txt',\n",
              " '10433_9.txt',\n",
              " '10434_10.txt',\n",
              " '10435_7.txt',\n",
              " '10436_8.txt',\n",
              " '10437_7.txt',\n",
              " '10438_9.txt',\n",
              " '10439_8.txt',\n",
              " '1043_10.txt',\n",
              " '10440_9.txt',\n",
              " '10441_10.txt',\n",
              " '10442_10.txt',\n",
              " '10443_9.txt',\n",
              " '10444_9.txt',\n",
              " '10445_10.txt',\n",
              " '10446_10.txt',\n",
              " '10447_10.txt',\n",
              " '10448_10.txt',\n",
              " '10449_9.txt',\n",
              " '1044_8.txt',\n",
              " '10450_10.txt',\n",
              " '10451_10.txt',\n",
              " '10452_10.txt',\n",
              " '10453_10.txt',\n",
              " '10454_9.txt',\n",
              " '10455_10.txt',\n",
              " '10456_10.txt',\n",
              " '10457_8.txt',\n",
              " '10458_10.txt',\n",
              " '10459_9.txt',\n",
              " '1045_8.txt',\n",
              " '10460_10.txt',\n",
              " '10461_9.txt',\n",
              " '10462_7.txt',\n",
              " '10463_10.txt',\n",
              " '10464_7.txt',\n",
              " '10465_8.txt',\n",
              " '10466_8.txt',\n",
              " '10467_10.txt',\n",
              " '10468_9.txt',\n",
              " '10469_10.txt',\n",
              " '1046_10.txt',\n",
              " '10470_9.txt',\n",
              " '10471_10.txt',\n",
              " '10472_7.txt',\n",
              " '10473_10.txt',\n",
              " '10474_9.txt',\n",
              " '10475_8.txt',\n",
              " '10476_9.txt',\n",
              " '10477_9.txt',\n",
              " '10478_8.txt',\n",
              " '10479_10.txt',\n",
              " '1047_8.txt',\n",
              " '10480_10.txt',\n",
              " '10481_8.txt',\n",
              " '10482_10.txt',\n",
              " '10483_8.txt',\n",
              " '10484_8.txt',\n",
              " '10485_8.txt',\n",
              " '10486_7.txt',\n",
              " '10487_7.txt',\n",
              " '10488_10.txt',\n",
              " '10489_10.txt',\n",
              " '1048_8.txt',\n",
              " '10490_7.txt',\n",
              " '10491_7.txt',\n",
              " '10492_10.txt',\n",
              " '10493_9.txt',\n",
              " '10494_10.txt',\n",
              " '10495_7.txt',\n",
              " '10496_10.txt',\n",
              " '10497_8.txt',\n",
              " '10498_10.txt',\n",
              " '10499_10.txt',\n",
              " '1049_7.txt',\n",
              " '104_10.txt',\n",
              " '10500_10.txt',\n",
              " '10501_10.txt',\n",
              " '10502_9.txt',\n",
              " '10503_10.txt',\n",
              " '10504_9.txt',\n",
              " '10505_10.txt',\n",
              " '10506_10.txt',\n",
              " '10507_10.txt',\n",
              " '10508_10.txt',\n",
              " '10509_7.txt',\n",
              " '1050_9.txt',\n",
              " '10510_7.txt',\n",
              " '10511_7.txt',\n",
              " '10512_10.txt',\n",
              " '10513_7.txt',\n",
              " '10514_8.txt',\n",
              " '10515_9.txt',\n",
              " '10516_7.txt',\n",
              " '10517_8.txt',\n",
              " '10518_9.txt',\n",
              " '10519_9.txt',\n",
              " '1051_9.txt',\n",
              " '10520_9.txt',\n",
              " '10521_9.txt',\n",
              " '10522_7.txt',\n",
              " '10523_9.txt',\n",
              " '10524_10.txt',\n",
              " '10525_10.txt',\n",
              " '10526_9.txt',\n",
              " '10527_10.txt',\n",
              " '10528_10.txt',\n",
              " '10529_10.txt',\n",
              " '1052_8.txt',\n",
              " '10530_10.txt',\n",
              " '10531_10.txt',\n",
              " '10532_8.txt',\n",
              " '10533_10.txt',\n",
              " '10534_7.txt',\n",
              " '10535_7.txt',\n",
              " '10536_10.txt',\n",
              " '10537_10.txt',\n",
              " '10538_8.txt',\n",
              " '10539_10.txt',\n",
              " '1053_8.txt',\n",
              " '10540_10.txt',\n",
              " '10541_10.txt',\n",
              " '10542_7.txt',\n",
              " '10543_8.txt',\n",
              " '10544_8.txt',\n",
              " '10545_7.txt',\n",
              " '10546_9.txt',\n",
              " '10547_9.txt',\n",
              " '10548_7.txt',\n",
              " '10549_9.txt',\n",
              " '1054_8.txt',\n",
              " '10550_8.txt',\n",
              " '10551_7.txt',\n",
              " '10552_9.txt',\n",
              " '10553_8.txt',\n",
              " '10554_7.txt',\n",
              " '10555_8.txt',\n",
              " '10556_10.txt',\n",
              " '10557_9.txt',\n",
              " '10558_10.txt',\n",
              " '10559_8.txt',\n",
              " '1055_10.txt',\n",
              " '10560_9.txt',\n",
              " '10561_8.txt',\n",
              " '10562_9.txt',\n",
              " '10563_7.txt',\n",
              " '10564_10.txt',\n",
              " '10565_9.txt',\n",
              " '10566_8.txt',\n",
              " '10567_9.txt',\n",
              " '10568_10.txt',\n",
              " '10569_10.txt',\n",
              " '1056_10.txt',\n",
              " '10570_8.txt',\n",
              " '10571_8.txt',\n",
              " '10572_8.txt',\n",
              " '10573_10.txt',\n",
              " '10574_10.txt',\n",
              " '10575_9.txt',\n",
              " '10576_7.txt',\n",
              " '10577_10.txt',\n",
              " '10578_7.txt',\n",
              " '10579_10.txt',\n",
              " '1057_9.txt',\n",
              " '10580_8.txt',\n",
              " '10581_10.txt',\n",
              " '10582_10.txt',\n",
              " '10583_10.txt',\n",
              " '10584_10.txt',\n",
              " '10585_9.txt',\n",
              " '10586_10.txt',\n",
              " '10587_8.txt',\n",
              " '10588_10.txt',\n",
              " '10589_10.txt',\n",
              " '1058_10.txt',\n",
              " '10590_8.txt',\n",
              " '10591_10.txt',\n",
              " '10592_8.txt',\n",
              " '10593_8.txt',\n",
              " '10594_8.txt',\n",
              " '10595_10.txt',\n",
              " '10596_8.txt',\n",
              " '10597_9.txt',\n",
              " '10598_8.txt',\n",
              " '10599_8.txt',\n",
              " '1059_10.txt',\n",
              " '105_7.txt',\n",
              " '10600_9.txt',\n",
              " '10601_10.txt',\n",
              " '10602_10.txt',\n",
              " '10603_10.txt',\n",
              " '10604_7.txt',\n",
              " '10605_7.txt',\n",
              " '10606_10.txt',\n",
              " '10607_10.txt',\n",
              " '10608_10.txt',\n",
              " '10609_10.txt',\n",
              " '1060_10.txt',\n",
              " '10610_8.txt',\n",
              " '10611_8.txt',\n",
              " '10612_7.txt',\n",
              " '10613_8.txt',\n",
              " '10614_7.txt',\n",
              " '10615_8.txt',\n",
              " '10616_7.txt',\n",
              " '10617_8.txt',\n",
              " '10618_8.txt',\n",
              " '10619_8.txt',\n",
              " '1061_10.txt',\n",
              " '10620_10.txt',\n",
              " '10621_10.txt',\n",
              " '10622_10.txt',\n",
              " '10623_8.txt',\n",
              " '10624_7.txt',\n",
              " '10625_7.txt',\n",
              " '10626_7.txt',\n",
              " '10627_10.txt',\n",
              " '10628_7.txt',\n",
              " '10629_10.txt',\n",
              " '1062_10.txt',\n",
              " '10630_8.txt',\n",
              " '10631_8.txt',\n",
              " '10632_10.txt',\n",
              " '10633_9.txt',\n",
              " '10634_10.txt',\n",
              " '10635_8.txt',\n",
              " '10636_8.txt',\n",
              " '10637_10.txt',\n",
              " '10638_8.txt',\n",
              " '10639_7.txt',\n",
              " '1063_10.txt',\n",
              " '10640_8.txt',\n",
              " '10641_7.txt',\n",
              " '10642_8.txt',\n",
              " '10643_8.txt',\n",
              " '10644_8.txt',\n",
              " '10645_8.txt',\n",
              " '10646_8.txt',\n",
              " '10647_8.txt',\n",
              " '10648_8.txt',\n",
              " '10649_7.txt',\n",
              " '1064_10.txt',\n",
              " '10650_8.txt',\n",
              " '10651_7.txt',\n",
              " '10652_9.txt',\n",
              " '10653_10.txt',\n",
              " '10654_7.txt',\n",
              " '10655_9.txt',\n",
              " '10656_7.txt',\n",
              " '10657_8.txt',\n",
              " '10658_10.txt',\n",
              " '10659_8.txt',\n",
              " '1065_10.txt',\n",
              " '10660_10.txt',\n",
              " '10661_9.txt',\n",
              " '10662_7.txt',\n",
              " '10663_8.txt',\n",
              " '10664_8.txt',\n",
              " '10665_8.txt',\n",
              " '10666_8.txt',\n",
              " '10667_8.txt',\n",
              " '10668_7.txt',\n",
              " '10669_10.txt',\n",
              " '1066_10.txt',\n",
              " '10670_10.txt',\n",
              " '10671_10.txt',\n",
              " '10672_9.txt',\n",
              " '10673_10.txt',\n",
              " '10674_8.txt',\n",
              " '10675_8.txt',\n",
              " '10676_9.txt',\n",
              " '10677_8.txt',\n",
              " '10678_9.txt',\n",
              " '10679_10.txt',\n",
              " '1067_7.txt',\n",
              " '10680_8.txt',\n",
              " '10681_10.txt',\n",
              " '10682_10.txt',\n",
              " '10683_7.txt',\n",
              " '10684_9.txt',\n",
              " '10685_7.txt',\n",
              " '10686_8.txt',\n",
              " '10687_10.txt',\n",
              " '10688_9.txt',\n",
              " '10689_8.txt',\n",
              " '1068_10.txt',\n",
              " '10690_10.txt',\n",
              " '10691_7.txt',\n",
              " '10692_8.txt',\n",
              " '10693_8.txt',\n",
              " '10694_7.txt',\n",
              " '10695_8.txt',\n",
              " '10696_7.txt',\n",
              " '10697_8.txt',\n",
              " '10698_9.txt',\n",
              " '10699_9.txt',\n",
              " '1069_10.txt',\n",
              " '106_10.txt',\n",
              " '10700_8.txt',\n",
              " '10701_10.txt',\n",
              " '10702_10.txt',\n",
              " '10703_7.txt',\n",
              " '10704_10.txt',\n",
              " '10705_7.txt',\n",
              " '10706_7.txt',\n",
              " '10707_8.txt',\n",
              " '10708_8.txt',\n",
              " '10709_10.txt',\n",
              " '1070_8.txt',\n",
              " '10710_9.txt',\n",
              " '10711_10.txt',\n",
              " '10712_8.txt',\n",
              " '10713_9.txt',\n",
              " '10714_8.txt',\n",
              " '10715_8.txt',\n",
              " '10716_7.txt',\n",
              " '10717_10.txt',\n",
              " '10718_10.txt',\n",
              " '10719_10.txt',\n",
              " '1071_8.txt',\n",
              " '10720_9.txt',\n",
              " '10721_9.txt',\n",
              " '10722_10.txt',\n",
              " '10723_8.txt',\n",
              " '10724_8.txt',\n",
              " '10725_9.txt',\n",
              " '10726_7.txt',\n",
              " '10727_7.txt',\n",
              " '10728_10.txt',\n",
              " '10729_8.txt',\n",
              " '1072_10.txt',\n",
              " '10730_10.txt',\n",
              " '10731_7.txt',\n",
              " '10732_8.txt',\n",
              " '10733_7.txt',\n",
              " '10734_10.txt',\n",
              " '10735_10.txt',\n",
              " '10736_10.txt',\n",
              " '10737_10.txt',\n",
              " '10738_9.txt',\n",
              " '10739_10.txt',\n",
              " '1073_9.txt',\n",
              " '10740_8.txt',\n",
              " '10741_10.txt',\n",
              " '10742_9.txt',\n",
              " '10743_9.txt',\n",
              " '10744_8.txt',\n",
              " '10745_10.txt',\n",
              " '10746_10.txt',\n",
              " '10747_10.txt',\n",
              " '10748_10.txt',\n",
              " '10749_8.txt',\n",
              " '1074_10.txt',\n",
              " '10750_8.txt',\n",
              " '10751_10.txt',\n",
              " '10752_10.txt',\n",
              " '10753_10.txt',\n",
              " '10754_10.txt',\n",
              " '10755_10.txt',\n",
              " '10756_8.txt',\n",
              " '10757_10.txt',\n",
              " '10758_8.txt',\n",
              " '10759_9.txt',\n",
              " '1075_10.txt',\n",
              " '10760_8.txt',\n",
              " '10761_10.txt',\n",
              " '10762_10.txt',\n",
              " '10763_8.txt',\n",
              " '10764_9.txt',\n",
              " '10765_10.txt',\n",
              " '10766_7.txt',\n",
              " '10767_10.txt',\n",
              " '10768_7.txt',\n",
              " '10769_10.txt',\n",
              " '1076_8.txt',\n",
              " '10770_7.txt',\n",
              " '10771_10.txt',\n",
              " '10772_10.txt',\n",
              " '10773_9.txt',\n",
              " '10774_8.txt',\n",
              " '10775_8.txt',\n",
              " '10776_8.txt',\n",
              " '10777_9.txt',\n",
              " '10778_8.txt',\n",
              " '10779_10.txt',\n",
              " '1077_8.txt',\n",
              " '10780_10.txt',\n",
              " '10781_10.txt',\n",
              " '10782_7.txt',\n",
              " '10783_10.txt',\n",
              " '10784_10.txt',\n",
              " '10785_10.txt',\n",
              " '10786_10.txt',\n",
              " '10787_10.txt',\n",
              " '10788_10.txt',\n",
              " '10789_10.txt',\n",
              " '1078_8.txt',\n",
              " '10790_8.txt',\n",
              " '10791_9.txt',\n",
              " '10792_9.txt',\n",
              " '10793_10.txt',\n",
              " '10794_10.txt',\n",
              " '10795_7.txt',\n",
              " '10796_9.txt',\n",
              " '10797_8.txt',\n",
              " '10798_8.txt',\n",
              " '10799_7.txt',\n",
              " '1079_7.txt',\n",
              " '107_10.txt',\n",
              " '10800_8.txt',\n",
              " '10801_8.txt',\n",
              " '10802_8.txt',\n",
              " '10803_8.txt',\n",
              " '10804_10.txt',\n",
              " '10805_10.txt',\n",
              " '10806_9.txt',\n",
              " '10807_9.txt',\n",
              " '10808_10.txt',\n",
              " '10809_10.txt',\n",
              " '1080_9.txt',\n",
              " '10810_8.txt',\n",
              " '10811_7.txt',\n",
              " '10812_8.txt',\n",
              " '10813_10.txt',\n",
              " '10814_7.txt',\n",
              " '10815_10.txt',\n",
              " '10816_10.txt',\n",
              " '10817_10.txt',\n",
              " '10818_10.txt',\n",
              " '10819_10.txt',\n",
              " '1081_10.txt',\n",
              " '10820_10.txt',\n",
              " '10821_8.txt',\n",
              " '10822_10.txt',\n",
              " '10823_8.txt',\n",
              " '10824_10.txt',\n",
              " '10825_9.txt',\n",
              " '10826_10.txt',\n",
              " '10827_10.txt',\n",
              " '10828_10.txt',\n",
              " '10829_10.txt',\n",
              " '1082_10.txt',\n",
              " '10830_10.txt',\n",
              " '10831_7.txt',\n",
              " '10832_10.txt',\n",
              " '10833_10.txt',\n",
              " '10834_7.txt',\n",
              " '10835_10.txt',\n",
              " '10836_10.txt',\n",
              " '10837_10.txt',\n",
              " '10838_10.txt',\n",
              " '10839_10.txt',\n",
              " '1083_10.txt',\n",
              " '10840_9.txt',\n",
              " '10841_10.txt',\n",
              " '10842_7.txt',\n",
              " '10843_7.txt',\n",
              " '10844_9.txt',\n",
              " '10845_10.txt',\n",
              " '10846_9.txt',\n",
              " '10847_10.txt',\n",
              " '10848_10.txt',\n",
              " '10849_10.txt',\n",
              " '1084_9.txt',\n",
              " '10850_10.txt',\n",
              " '10851_9.txt',\n",
              " '10852_10.txt',\n",
              " '10853_10.txt',\n",
              " '10854_10.txt',\n",
              " '10855_9.txt',\n",
              " '10856_8.txt',\n",
              " '10857_8.txt',\n",
              " '10858_8.txt',\n",
              " '10859_7.txt',\n",
              " '1085_7.txt',\n",
              " '10860_7.txt',\n",
              " '10861_7.txt',\n",
              " '10862_9.txt',\n",
              " '10863_8.txt',\n",
              " '10864_8.txt',\n",
              " '10865_7.txt',\n",
              " '10866_7.txt',\n",
              " '10867_7.txt',\n",
              " '10868_8.txt',\n",
              " '10869_7.txt',\n",
              " '1086_7.txt',\n",
              " '10870_8.txt',\n",
              " '10871_7.txt',\n",
              " '10872_7.txt',\n",
              " '10873_8.txt',\n",
              " '10874_10.txt',\n",
              " '10875_8.txt',\n",
              " '10876_7.txt',\n",
              " '10877_10.txt',\n",
              " '10878_7.txt',\n",
              " '10879_10.txt',\n",
              " '1087_10.txt',\n",
              " '10880_8.txt',\n",
              " '10881_7.txt',\n",
              " '10882_8.txt',\n",
              " '10883_7.txt',\n",
              " '10884_8.txt',\n",
              " '10885_7.txt',\n",
              " '10886_10.txt',\n",
              " '10887_7.txt',\n",
              " '10888_8.txt',\n",
              " '10889_10.txt',\n",
              " '1088_9.txt',\n",
              " '10890_9.txt',\n",
              " '10891_7.txt',\n",
              " '10892_7.txt',\n",
              " '10893_8.txt',\n",
              " '10894_8.txt',\n",
              " '10895_7.txt',\n",
              " '10896_8.txt',\n",
              " '10897_9.txt',\n",
              " '10898_7.txt',\n",
              " '10899_10.txt',\n",
              " '1089_10.txt',\n",
              " '108_10.txt',\n",
              " ...]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bjiDVA7qrFxP",
        "outputId": "758dc41e-e3cb-45de-dc0a-c81060c763db"
      },
      "source": [
        "!cat /Users/김창우/Desktop/data/aclImdb/train/pos/0_9.txt   #원래는 !cat하게 되면 movie 리뷰에 대한 내용이 나오게 된다."
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'cat'은(는) 내부 또는 외부 명령, 실행할 수 있는 프로그램, 또는\n",
            "배치 파일이 아닙니다.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1-mOZQwcXrAu"
      },
      "source": [
        "### Tokenize the data\n",
        "\n",
        "\n",
        "Let's vectorize the texts we collected, and prepare a training and validation split.\n",
        "We will merely be using the concepts we introduced earlier in this section.\n",
        "\n",
        "Because pre-trained word embeddings are meant to be particularly useful on problems where little training data is available (otherwise, \n",
        "task-specific embeddings are likely to outperform them), we will add the following twist: we restrict the training data to its first 200 \n",
        "samples. So we will be learning to classify movie reviews after looking at just 200 examples...\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NlFUMAuyrFxP"
      },
      "source": [
        "- 여기에서는 keras에 있는 Tokenizer를 사용해서 preprocessing하는 부분이다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sUJO2VUXXrAu",
        "outputId": "b42875fa-61d7-4d5b-b351-04372c10ef02"
      },
      "source": [
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "import numpy as np\n",
        "\n",
        "maxlen = 100  # We will cut reviews after 100 words  #movie리뷰에서 앞에서 100단어를 사용할 것이다. \n",
        "training_samples = 200  # We will be training on 200 samples  #여기에서는 training_saples이 작은데도 pretrain을 잘 할 수 있을까 하는 거다.\n",
        "validation_samples = 10000  # We will be validating on 10000 samples\n",
        "max_words = 10000  # We will only consider the top 10,000 words in the dataset #자주 사용되는 단어는 10000개를 사용할 것이다. \n",
        "\n",
        "tokenizer = Tokenizer(num_words=max_words)\n",
        "tokenizer.fit_on_texts(texts)  #Tokenizer를 정의를 한 다음에 texts에서 25000개의 전체 텍스트로 이 tokenizer에서 정의한 것을 fitting시켜준다. \n",
        "sequences = tokenizer.texts_to_sequences(texts) #text데이터를 sequences로 바꾼게 이부분이다. \n",
        "\n",
        "word_index = tokenizer.word_index\n",
        "print('Found %s unique tokens.' % len(word_index))\n",
        "\n",
        "data = pad_sequences(sequences, maxlen=maxlen)  #pad_sequences이거를 사용해서 maxlen으로 끊겠다는 것이다. \n",
        "#25000개의 데이터를 앞에서 100개로 끊은게 바로 위에있는 data 코드이다. \n",
        "\n",
        "labels = np.asarray(labels)   #여기에서 labels의 정보를 출력해 주었다.\n",
        "print('Shape of data tensor:', data.shape)\n",
        "print('Shape of label tensor:', labels.shape)\n",
        "\n",
        "# Split the data into a training set and a validation set\n",
        "# But first, shuffle the data, since we started from data\n",
        "# where sample are ordered (all negative first, then all positive).\n",
        "indices = np.arange(data.shape[0])\n",
        "np.random.shuffle(indices)\n",
        "data = data[indices]   #여기에서는 shuffle해줘서 데이터를 섞어준 다음에 \n",
        "labels = labels[indices]\n",
        "\n",
        "x_train = data[:training_samples]  #앞에서 200개를 끊어서 train set으로 쓰고 \n",
        "y_train = labels[:training_samples]\n",
        "x_val = data[training_samples: training_samples + validation_samples] #그다음 여기서 부터는 validation_sample로 끊어서 validation set으로 쓰겠다는 것이다. \n",
        "y_val = labels[training_samples: training_samples + validation_samples]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 88582 unique tokens.\n",
            "Shape of data tensor: (25000, 100)\n",
            "Shape of label tensor: (25000,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DxJw2yJMrFxP",
        "outputId": "ff048877-0757-4147-d9b1-504d3baa5c02"
      },
      "source": [
        "len(sequences[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "104"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vW7EvbJTrFxQ",
        "outputId": "adb4288b-140f-46d4-caad-18a02bfaa5db"
      },
      "source": [
        "data.shape  #data의 shape이 25000개에서 100차원으로 "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(25000, 100)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UsdDCUQcXrAu"
      },
      "source": [
        "### Download the GloVe word embeddings\n",
        "\n",
        "\n",
        "Head to `https://nlp.stanford.edu/projects/glove/` (where you can learn more about the GloVe algorithm), and download the pre-computed \n",
        "embeddings from 2014 English Wikipedia. It's a 822MB zip file named `glove.6B.zip`, containing 100-dimensional embedding vectors for \n",
        "400,000 words (or non-word tokens). Un-zip it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ZHydFXWrFxQ"
      },
      "source": [
        "- 여기에서는 gloove모형을 다운 받아서 사용을 해준다. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fJfOw2CEXrAv"
      },
      "source": [
        "### Pre-process the embeddings\n",
        "\n",
        "\n",
        "Let's parse the un-zipped file (it's a `txt` file) to build an index mapping words (as strings) to their vector representation (as number \n",
        "vectors)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-gZJF2m2rFxQ",
        "outputId": "d23af73c-a84f-45c9-92eb-d533da9c03b5"
      },
      "source": [
        "!head /Users/김창우/Desktop/gloveData/glove.6B.100B.txt -n 2\n",
        "#!head이게 리눅스에서 앞에서 몇개를 보여주는 코드이다. \n",
        "#그래서 앞에 -n 2인 두라인을 출력해준다는 것이다. "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'head'은(는) 내부 또는 외부 명령, 실행할 수 있는 프로그램, 또는\n",
            "배치 파일이 아닙니다.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "swBe9x3-XrAv",
        "outputId": "a910564f-9e3e-4ba8-f748-ff6aaf3d8a94"
      },
      "source": [
        "glove_dir = 'C:/Users/김창우/Desktop/gloveData'\n",
        "\n",
        "embeddings_index = {}\n",
        "f = open(os.path.join(glove_dir, 'glove.6B.100d.txt'), encoding='UTF-8')\n",
        "for line in f:\n",
        "    values = line.split()  #split을 하게 되면 해당 라인들이 단어들로 split이 된다. \n",
        "    word = values[0]  #첫번째는 단어라서 단어를 가지고 와준다. \n",
        "    coefs = np.asarray(values[1:], dtype='float32')  #그리고 뒤의 숫자들은 coefs에 들어가게 된다. \n",
        "    embeddings_index[word] = coefs  # embeddings_index이거 해줘서 word에다가 coefs를 집어넣어서 embeddings_index여기에다가 결과를 저장하게 된다. \n",
        "f.close()\n",
        "\n",
        "print('Found %s word vectors.' % len(embeddings_index))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 400000 word vectors.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u-dsIfJGrFxR"
      },
      "source": [
        "- 40000개의 단어들에 대해서 glove 모형이 pretrain된게 담겨 있다는 뜻이다. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PNxtqpNiXrAv"
      },
      "source": [
        "\n",
        "Now let's build an embedding matrix that we will be able to load into an `Embedding` layer. It must be a matrix of shape `(max_words, \n",
        "embedding_dim)`, where each entry `i` contains the `embedding_dim`-dimensional vector for the word of index `i` in our reference word index \n",
        "(built during tokenization). Note that the index `0` is not supposed to stand for any word or token -- it's a placeholder."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rz3wtX9OrFxR"
      },
      "source": [
        "- 우리가 10000개를 사용을 해서 이부분이 10000개에 해당하는 matrix를 저장하는 부분이다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a0cCedCmXrAv"
      },
      "source": [
        "embedding_dim = 100\n",
        "\n",
        "embedding_matrix = np.zeros((max_words, embedding_dim))  #여기에서는 0으로 해줘서 matrix를 저장을 해준다. \n",
        "for word, i in word_index.items():\n",
        "    embedding_vector = embeddings_index.get(word)  #불러온 embeddings_index이거에서 해당 word에 대한 vector를 불러와서 embedding_vector여기에 저장을 하고 \n",
        "    if i < max_words:\n",
        "        if embedding_vector is not None:  #embedding vector가 None이 아니면 \n",
        "            # Words not found in embedding index will be all-zeros.\n",
        "            embedding_matrix[i] = embedding_vector #embedding_matrix i에다가 embedding_vector를 집어 넣어 준다. "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2VQF0ycLrFxR",
        "outputId": "30b9a888-b533-4160-c3c8-d51790c6cf71"
      },
      "source": [
        "embedding_matrix.shape   #10000개의 단어들에 대한 100차원의 embedding을 저장해야 되는데 그거를 전부 0으로 해줘서 저장을 한다. "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10000, 100)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yFd5azpfXrAv"
      },
      "source": [
        "### Define a model\n",
        "\n",
        "We will be using the same model architecture as before:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AaheRCcNXrAw",
        "outputId": "0848e067-eb4e-49af-f3db-5650b4f08a6f"
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Embedding, Flatten, Dense\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Embedding(max_words, embedding_dim, input_length=maxlen)) #모형을 정의 할 때는 Embedding모형을 정의를 하고 \n",
        "model.add(Flatten())\n",
        "model.add(Dense(32, activation='relu'))\n",
        "model.add(Dense(1, activation='sigmoid'))\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_2 (Embedding)      (None, 100, 100)          1000000   \n",
            "_________________________________________________________________\n",
            "flatten_1 (Flatten)          (None, 10000)             0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 32)                320032    \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1)                 33        \n",
            "=================================================================\n",
            "Total params: 1,320,065\n",
            "Trainable params: 1,320,065\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lVQWF5AyXrAw"
      },
      "source": [
        "### Load the GloVe embeddings in the model\n",
        "\n",
        "\n",
        "The `Embedding` layer has a single weight matrix: a 2D float matrix where each entry `i` is the word vector meant to be associated with \n",
        "index `i`. Simple enough. Let's just load the GloVe matrix we prepared into our `Embedding` layer, the first layer in our model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nbu5k5ZgXrAw"
      },
      "source": [
        "model.layers[0].set_weights([embedding_matrix])  #embedding_matrix이것을 넣어줘서 embedding weights를 적용을 해준다. \n",
        "model.layers[0].trainable = False  #False를 해줘서 해당 layers는 trainable하지 않게 선언을 해준다. "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5Sh17Z6prFxS"
      },
      "source": [
        "- 위의 코드는 model.add(Embedding(max_words, embedding_dim, input_length=maxlen))이것을 글로벌 웨이트로 저장을 시켜줘야 한다.\n",
        "- 그 부분이 model.layers[0].set_weights([embedding_matrix])이 부분이다. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M_R0Q21nXrAw"
      },
      "source": [
        "\n",
        "Additionally, we freeze the embedding layer (we set its `trainable` attribute to `False`), following the same rationale as what you are \n",
        "already familiar with in the context of pre-trained convnet features: when parts of a model are pre-trained (like our `Embedding` layer), \n",
        "and parts are randomly initialized (like our classifier), the pre-trained parts should not be updated during training to avoid forgetting \n",
        "what they already know. The large gradient update triggered by the randomly initialized layers would be very disruptive to the already \n",
        "learned features."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CHW5HWZ8XrAw"
      },
      "source": [
        "### Train and evaluate\n",
        "\n",
        "Let's compile our model and train it:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TOiDyv9CXrAw",
        "outputId": "796d30a1-651b-4bde-e38d-0587d8afa41d"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['acc'])\n",
        "history = model.fit(x_train, y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=32,\n",
        "                    validation_data=(x_val, y_val))\n",
        "model.save_weights('pre_trained_glove_model.h5')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "7/7 [==============================] - 1s 71ms/step - loss: 2.3531 - acc: 0.5354 - val_loss: 1.0897 - val_acc: 0.5081\n",
            "Epoch 2/10\n",
            "7/7 [==============================] - 0s 54ms/step - loss: 0.5807 - acc: 0.6791 - val_loss: 2.0555 - val_acc: 0.4922\n",
            "Epoch 3/10\n",
            "7/7 [==============================] - 0s 56ms/step - loss: 0.7808 - acc: 0.7195 - val_loss: 0.6891 - val_acc: 0.5740\n",
            "Epoch 4/10\n",
            "7/7 [==============================] - 0s 51ms/step - loss: 0.2240 - acc: 0.9567 - val_loss: 1.0979 - val_acc: 0.4949\n",
            "Epoch 5/10\n",
            "7/7 [==============================] - 0s 50ms/step - loss: 0.1845 - acc: 0.9720 - val_loss: 1.0197 - val_acc: 0.5024\n",
            "Epoch 6/10\n",
            "7/7 [==============================] - 0s 51ms/step - loss: 0.1118 - acc: 1.0000 - val_loss: 0.7597 - val_acc: 0.5534\n",
            "Epoch 7/10\n",
            "7/7 [==============================] - 0s 50ms/step - loss: 0.1728 - acc: 0.9351 - val_loss: 0.7110 - val_acc: 0.5829\n",
            "Epoch 8/10\n",
            "7/7 [==============================] - 0s 50ms/step - loss: 0.0622 - acc: 1.0000 - val_loss: 0.7361 - val_acc: 0.5813\n",
            "Epoch 9/10\n",
            "7/7 [==============================] - 0s 47ms/step - loss: 0.0422 - acc: 1.0000 - val_loss: 0.7680 - val_acc: 0.5777\n",
            "Epoch 10/10\n",
            "7/7 [==============================] - 0s 52ms/step - loss: 0.0261 - acc: 1.0000 - val_loss: 1.7592 - val_acc: 0.5124\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OuKzRDldrFxT"
      },
      "source": [
        "- 그다음에 여기에서 train을 시켜준다. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DppimNQEXrAx"
      },
      "source": [
        "Let's plot its performance over time:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JcCk2ttYrFxT"
      },
      "source": [
        "- 그리고 여기에서는 plot을 해준다음에 성능을 봐주는 것이다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SV3gBgblXrAx",
        "outputId": "29d52981-eb97-4912-f21d-94c45a51f977"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "acc = history.history['acc']\n",
        "val_acc = history.history['val_acc']\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(1, len(acc) + 1)\n",
        "\n",
        "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.legend()\n",
        "\n",
        "plt.figure()\n",
        "\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAApq0lEQVR4nO3deXxU1f3/8deHsBkWlV2JEGxFxCIQIiJuqKgorigKpgqi4m7V1laL29eWtr9Kq7Vuxb0ai9aqdUGpS61a6xIQqaAoIkKKKKAiEsAAn98fZxImQ5YJJLmTm/fz8ZjH3OXMnc/cST5z7rnnnmvujoiINH7Nog5ARETqhhK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITCihx5iZPWtm4+q6bJTMbJGZDa+H7bqZfT8xfYeZXZ1O2a14nwIz+8fWxilSHVM/9MxiZt8mzWYD64GNiflz3L2w4aPKHGa2CDjL3V+o4+06sJu7L6irsmaWC3wCtHD3DXUSqEg1mkcdgFTk7m3LpqtLXmbWXElCMoX+HjODmlwaCTMbZmbFZvYzM1sG3GtmO5rZ02a23My+SkznJL3mZTM7KzE93sxeM7MpibKfmNmRW1m2l5m9YmarzewFM7vVzB6sIu50YvyFmf07sb1/mFmnpPWnmdmnZrbSzCZVs3+GmNkyM8tKWnaCmc1JTA82s/+Y2ddm9pmZ3WJmLavY1n1m9suk+csTr1lqZhNSyo40s3fM7BszW2Jm1yWtfiXx/LWZfWtm+5bt26TXDzWzt81sVeJ5aLr7ppb7uYOZ3Zv4DF+Z2RNJ644zs9mJz/CxmY1ILK/QvGVm15V9z2aWm2h6OtPMFgMvJZb/NfE9rEr8jeyZ9PrtzOx3ie9zVeJvbDsze8bMLkr5PHPM7PjKPqtUTQm9cekGdAB6AhMJ39+9ifkewFrglmpevw8wH+gE/Ba428xsK8o+BLwFdASuA06r5j3TifFU4AygC9AS+AmAmfUFbk9sf+fE++VQCXd/A1gDHJKy3YcS0xuBSxOfZ1/gUOD8auImEcOIRDyHAbsBqe33a4DTgR2AkcB5SYnowMTzDu7e1t3/k7LtDsAzwM2Jz/Z74Bkz65jyGbbYN5WoaT8/QGjC2zOxrRsTMQwG/gxcnvgMBwKLqniPyhwE7AEckZh/lrCfugCzgOQmwinAIGAo4e/4p8Am4H7gh2WFzKw/0B2YXos4BMDd9cjQB+Efa3hiehjwHdC6mvIDgK+S5l8mNNkAjAcWJK3LBhzoVpuyhGSxAchOWv8g8GCan6myGK9Kmj8feC4xfQ0wLWldm8Q+GF7Ftn8J3JOYbkdItj2rKHsJ8HjSvAPfT0zfB/wyMX0P8Jukcr2Ty1ay3ZuAGxPTuYmyzZPWjwdeS0yfBryV8vr/AONr2je12c/AToTEuWMl5f5UFm91f3+J+evKvuekz7ZrNTHskCizPeEHZy3Qv5JyrYAvCeclICT+2+rjfyruD9XQG5fl7r6ubMbMss3sT4lD2G8Ih/g7JDc7pFhWNuHuJYnJtrUsuzPwZdIygCVVBZxmjMuSpkuSYto5edvuvgZYWdV7EWrjo8ysFTAKmOXunybi6J1ohliWiONXhNp6TSrEAHya8vn2MbN/Jpo6VgHnprndsm1/mrLsU0LttExV+6aCGvbzLoTv7KtKXroL8HGa8VamfN+YWZaZ/SbRbPMNm2v6nRKP1pW9l7uvBx4BfmhmzYCxhCMKqSUl9MYltUvSj4HdgX3cvT2bD/GrakapC58BHcwsO2nZLtWU35YYP0veduI9O1ZV2N3nERLikVRsboHQdPMBoRbYHvj51sRAOEJJ9hDwJLCLu28P3JG03Zq6kC0lNJEk6wH8L424UlW3n5cQvrMdKnndEuB7VWxzDeHorEy3Ssokf8ZTgeMIzVLbE2rxZTGsANZV8173AwWEprAST2mekvQooTdu7QiHsV8n2mOvre83TNR4i4DrzKylme0LHFNPMT4KHG1m+ydOYF5PzX+zDwEXExLaX1Pi+Ab41sz6AOelGcMjwHgz65v4QUmNvx2h9rsu0R59atK65YSmjl2r2PZ0oLeZnWpmzc3sFKAv8HSasaXGUel+dvfPCG3btyVOnrYws7KEfzdwhpkdambNzKx7Yv8AzAbGJMrnAyelEcN6wlFUNuEoqCyGTYTmq9+b2c6J2vy+iaMpEgl8E/A7VDvfakrojdtNwHaE2s8bwHMN9L4FhBOLKwnt1g8T/pErcxNbGaO7zwUuICTpz4CvgOIaXvYXwvmGl9x9RdLynxCS7WrgzkTM6cTwbOIzvAQsSDwnOx+43sxWE9r8H0l6bQkwGfi3hd41Q1K2vRI4mlC7Xkk4SXh0Stzpuonq9/NpQCnhKOULwjkE3P0twknXG4FVwL/YfNRwNaFG/RXwf1Q84qnMnwlHSP8D5iXiSPYT4L/A24Q28/9HxRz0Z6Af4ZyMbAVdWCTbzMweBj5w93o/QpD4MrPTgYnuvn/UsTRWqqFLrZnZ3mb2vcQh+ghCu+kTEYcljViiOet8YGrUsTRmSuiyNboRutR9S+hDfZ67vxNpRNJomdkRhPMNn1Nzs45UQ00uIiIxoRq6iEhMRDY4V6dOnTw3NzeqtxcRaZRmzpy5wt07V7YusoSem5tLUVFRVG8vItIomVnq1cXl1OQiIhITSugiIjGhhC4iEhMZdcei0tJSiouLWbduXc2FJRKtW7cmJyeHFi1aRB2KiKTIqIReXFxMu3btyM3Nper7LkhU3J2VK1dSXFxMr169og5HRFLU2ORiZveY2Rdm9l4V683MbjazBYnbRuVtbTDr1q2jY8eOSuYZyszo2LGjjqBqUFgIubnQrFl4LmzCt/XOlH3RZOKo6Q4YhGFI84D3qlh/FGFoTgOGAG+mc2eNQYMGeap58+ZtsUwyj76nqj34oHt2tjtsfmRnh+VNTabsi7jFARR5FXk1rUv/zSwXeNrdf1DJuj8BL7v7XxLz84FhHsZgrlJ+fr6n9kN///332WOPPWqMR6Kl76lqubnwaSW9hHv2hEWLGjqaaGXKvohbHGY2093zK1tXF71culPxFl3FVLyFVnIgE82syMyKli9fXgdvXbdWrlzJgAEDGDBgAN26daN79+7l89999121ry0qKuLiiy+u8T2GDh1aYxlpvBYvrt3yOMuUfdGU4qiLhF5Zg3el1X53n+ru+e6e37lzpVeu1kpdt0d17NiR2bNnM3v2bM4991wuvfTS8vmWLVuyYcOGKl+bn5/PzTffXON7vP7669sWpGS0Hqk3qKtheZxlyr5oSnHURUIvpuI9F3MI90qsV4WFMHFiOIRxD88TJ9b9SYbx48dz2WWXcfDBB/Ozn/2Mt956i6FDhzJw4ECGDh3K/PnzAXj55Zc5+uijAbjuuuuYMGECw4YNY9ddd62Q6Nu2bVteftiwYZx00kn06dOHgoKCsnMSTJ8+nT59+rD//vtz8cUXl2832aJFizjggAPIy8sjLy+vwg/Fb3/7W/r160f//v254oorAFiwYAHDhw+nf//+5OXl8fHH23JfYKnK5MmQnV1xWXZ2WN7UZMq+aFJxVNW4nvwg3Oy1qpOiI6l4UvStdLa5rSdFe/aseHKh7NGzZ9qbqNa1117rN9xwg48bN85HjhzpGzZscHf3VatWeWlpqbu7P//88z5q1Ch3d//nP//pI0eOLH/tvvvu6+vWrfPly5d7hw4d/LvvvnN39zZt2pSXb9++vS9ZssQ3btzoQ4YM8VdffdXXrl3rOTk5vnDhQnd3HzNmTPl2k61Zs8bXrl3r7u4ffvihl+3P6dOn+7777utr1qxxd/eVK1e6u/vgwYP9sccec3f3tWvXlq/fGjopWr0HHwx/h2bhuSmeEC2TKfsiTnFQzUnRGvuhm1nZPRo7mVkx4eazLRI/BncQbnR7FOF+iyWE+xPWu4ZsFxs9ejRZWVkArFq1inHjxvHRRx9hZpSWllb6mpEjR9KqVStatWpFly5d+Pzzz8nJyalQZvDgweXLBgwYwKJFi2jbti277rpreT/vsWPHMnXqljdxKS0t5cILL2T27NlkZWXx4YcfAvDCCy9wxhlnkJ2oCnTo0IHVq1fzv//9jxNOOAEIFwdJ/SkoCA/JnH3RVOKoscnF3ce6+07u3sLdc9z9bne/I5HMSfxoXODu33P3fu7eIEMoNmS7WJs2bcqnr776ag4++GDee+89nnrqqSr7ZLdq1ap8Oisrq9L298rKeJo3HLnxxhvp2rUr7777LkVFReUnbd19i3786W6zscuUvsYiUWm0Y7lE1S62atUquncPnXjuu+++Ot9+nz59WLhwIYsS/Zgefrjym9OvWrWKnXbaiWbNmvHAAw+wceNGAA4//HDuueceSkpKAPjyyy9p3749OTk5PPHEEwCsX7++fH1cNNQ5FZFM1mgTekEBTJ0a+nCaheepU+v/sOqnP/0pV155Jfvtt195Eq1L2223HbfddhsjRoxg//33p2vXrmy//fZblDv//PO5//77GTJkCB9++GH5UcSIESM49thjyc/PZ8CAAUyZMgWABx54gJtvvpm99tqLoUOHsmzZsjqPPUqTJkHqb1RJSVgu0lREdk9RXVhUtW+//Za2bdvi7lxwwQXstttuXHrppVGHVS4Tv6dmzULNPJUZbNrU8PGI1Jf6vrBI6tidd97JgAED2HPPPVm1ahXnnHNO1CFlvEzpaywSpYwabVGCSy+9NKNq5I3B5MmhzTy52aWp9v+Wpks1dImFqM6piGQS1dAlNjKlr7FIVFRDFxGJCSV0EZGYUEJPMmzYMGbMmFFh2U033cT5559f7WvKul8eddRRfP3111uUue6668r7g1fliSeeYN68eeXz11xzDS+88EItoheRpk4JPcnYsWOZNm1ahWXTpk1j7Nixab1++vTp7LDDDlv13qkJ/frrr2f48OFbtS0RaZqU0JOcdNJJPP3006xfvx4IQ9QuXbqU/fffn/POO4/8/Hz23HNPrr322kpfn5uby4oVKwCYPHkyu+++O8OHDy8fYhdCH/O9996b/v37c+KJJ1JSUsLrr7/Ok08+yeWXX86AAQP4+OOPGT9+PI8++igAL774IgMHDqRfv35MmDChPL7c3FyuvfZa8vLy6NevHx988MEWMWmYXZGmI2N7uVxyCcyeXbfbHDAAbrqp6vUdO3Zk8ODBPPfccxx33HFMmzaNU045BTNj8uTJdOjQgY0bN3LooYcyZ84c9tprr0q3M3PmTKZNm8Y777zDhg0byMvLY9CgQQCMGjWKs88+G4CrrrqKu+++m4suuohjjz2Wo48+mpNOOqnCttatW8f48eN58cUX6d27N6effjq33347l1xyCQCdOnVi1qxZ3HbbbUyZMoW77rqrwuu7dOnC888/T+vWrfnoo48YO3YsRUVFPPvsszzxxBO8+eabZGdn8+WXXwJQUFDAFVdcwQknnMC6devYpMssRRoN1dBTJDe7JDe3PPLII+Tl5TFw4EDmzp1boXkk1auvvsoJJ5xAdnY27du359hjjy1f995773HAAQfQr18/CgsLmTt3brXxzJ8/n169etG7d28Axo0bxyuvvFK+ftSoUQAMGjSofECvZKWlpZx99tn069eP0aNHl8ed7jC72akjoIlIxsrYGnp1Nen6dPzxx3PZZZcxa9Ys1q5dS15eHp988glTpkzh7bffZscdd2T8+PFVDptbJnUI2zLjx4/niSeeoH///tx33328/PLL1W6nprF2yobgrWqI3uRhdjdt2lQ+FnpTHmZXJK5UQ0/Rtm1bhg0bxoQJE8pr59988w1t2rRh++235/PPP+fZZ5+tdhsHHnggjz/+OGvXrmX16tU89dRT5etWr17NTjvtRGlpKYVJY7u2a9eO1atXb7GtPn36sGjRIhYsWACEURMPOuigtD+PhtkVaTqU0CsxduxY3n33XcaMGQNA//79GThwIHvuuScTJkxgv/32q/b1eXl5nHLKKQwYMIATTzyRAw44oHzdL37xC/bZZx8OO+ww+vTpU758zJgx3HDDDQwcOLDCicjWrVtz7733Mnr0aPr160ezZs0499xz0/4sGmZXpOnQ8LlSa6nfU2FhGHd88eIwuuHkyboEX6S+VDd8bsa2oUvjUHanoLKWmbI7BYGSukhDU5OLbBPdKUgkc2RcQldPi8yW+v0sXlx5uaqWi0j9yaiE3rp1a1auXKmknqHcnZUrV5Z3fQTdKUgkk2RUG3pOTg7FxcUsX7486lCkCq1btyYnJ6d8XncKEskcGZXQW7RoQa9evaIOQ2qh7MSnermIRC+jEro0TrpTkEhmyKg2dBER2XpK6CIiMaGELiISE0roIiIxoYQuIhITaSV0MxthZvPNbIGZXVHJ+h3N7HEzm2Nmb5nZD+o+VBERqU6NCd3MsoBbgSOBvsBYM+ubUuznwGx33ws4HfhDXQcqIiLVS6eGPhhY4O4L3f07YBpwXEqZvsCLAO7+AZBrZl3rNFIREalWOgm9O7Akab44sSzZu8AoADMbDPQEclLKYGYTzazIzIp0eb+ISN1KJ6FXdnPM1NGzfgPsaGazgYuAd4AtbnDp7lPdPd/d8zt37lzbWEVEpBrpXPpfDOySNJ8DLE0u4O7fAGcAWLjz8CeJh4iINJB0auhvA7uZWS8zawmMAZ5MLmBmOyTWAZwFvJJI8iIi0kBqrKG7+wYzuxCYAWQB97j7XDM7N7H+DmAP4M9mthGYB5xZjzGLiEgl0hpt0d2nA9NTlt2RNP0fYLe6DU1ERGpDV4qKiMSEErqISEwooYuIxIQSukgMFRZCbi40axaeCwujjkgagm5BJxIzhYUVb9z96adhHnSrwLhTDV0kZiZN2pzMy5SUhOUSb0roIjGzeHHtlkt8KKGLxEyPHrVbLvGhhC4SM5MnQ3Z2xWXZ2WG5xJsSukjMFBTA1KnQsyeYheepU3VCtClQLxeRGCooUAJvilRDFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJibQSupmNMLP5ZrbAzK6oZP32ZvaUmb1rZnPN7Iy6D1VERKpTY0I3syzgVuBIoC8w1sz6phS7AJjn7v2BYcDvzKxlHccqIiLVSKeGPhhY4O4L3f07YBpwXEoZB9qZmQFtgS+BDXUaqWyhsBByc6FZs/BcWBh1RCISpeZplOkOLEmaLwb2SSlzC/AksBRoB5zi7ptSN2RmE4GJAD169NiaeCWhsBAmToSSkjD/6adhHqCgILq4RCQ66dTQrZJlnjJ/BDAb2BkYANxiZu23eJH7VHfPd/f8zp071zJUSTZp0uZkXqakJCwXkaYpnYReDOySNJ9DqIknOwN4zIMFwCdAn7oJUSqzeHHtlotI/KWT0N8GdjOzXokTnWMIzSvJFgOHAphZV2B3YGFdBioVVdVipZYskaarxoTu7huAC4EZwPvAI+4+18zONbNzE8V+AQw1s/8CLwI/c/cV9RW0wOTJkJ1dcVl2dlguIk1TOidFcffpwPSUZXckTS8FDq/b0KQ6ZSc+J00KzSw9eoRkrhOiIk1XWgldMlNBgRK4iGymS/9FRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJfSsUFkJuLjRrFp4LC6OOSEQEmkcdQGNTWAgTJ0JJSZj/9NMwD1BQEF1cIiKqodfSpEmbk3mZkpKwXEQkSkrotbR4ce2Wi4g0FCX0WurRo3bLRUQaihJ6LU2eDNnZFZdlZ4flIiJRUkKvpYICmDoVevYEs/A8dapOiIpI9NTLZSsUFCiBi0jmSauGbmYjzGy+mS0wsysqWX+5mc1OPN4zs41m1qHuwxURkarUmNDNLAu4FTgS6AuMNbO+yWXc/QZ3H+DuA4ArgX+5+5f1EK+IiFQhnRr6YGCBuy909++AacBx1ZQfC/ylLoITEZH0pZPQuwNLkuaLE8u2YGbZwAjgb1Wsn2hmRWZWtHz58trGKiIi1UgnoVsly7yKsscA/66qucXdp7p7vrvnd+7cOd0YRUQkDekk9GJgl6T5HGBpFWXHoOYWEZFIpJPQ3wZ2M7NeZtaSkLSfTC1kZtsDBwF/r9sQRUQkHTX2Q3f3DWZ2ITADyALucfe5ZnZuYv0diaInAP9w9zX1Fq2IiFTJ3KtqDq9f+fn5XlRUFMl7i4g0VmY2093zK1unS/9FRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQm0kroZjbCzOab2QIzu6KKMsPMbLaZzTWzf9VtmCIiUpPmNRUwsyzgVuAwoBh428yedPd5SWV2AG4DRrj7YjPrUk/xiohIFdKpoQ8GFrj7Qnf/DpgGHJdS5lTgMXdfDODuX9RtmCIiUpN0Enp3YEnSfHFiWbLewI5m9rKZzTSz0yvbkJlNNLMiMytavnz51kUsIiKVSiehWyXLPGW+OTAIGAkcAVxtZr23eJH7VHfPd/f8zp071zpYERGpWo1t6IQa+S5J8znA0krKrHD3NcAaM3sF6A98WCdRiohIjdKpob8N7GZmvcysJTAGeDKlzN+BA8ysuZllA/sA79dtqCIiUp0aa+juvsHMLgRmAFnAPe4+18zOTay/w93fN7PngDnAJuAud3+vPgMXEZGKzD21Obxh5Ofne1FRUSTvLdIUbNwIixdDs2bQvDm0aLH5UTaflRV1lFJbZjbT3fMrW5dOG7qINBKbNsEbb8DDD8Nf/wqffVZ9ebOqk31l87UpmzzfqhVkZ0ObNpufk6dTl2Vn68dmayihizRy7vD225uT+JIlIYEeeSSMGBES6oYNUFq6+ZE8n8506vy6dbB6dfqvWb++9p+rVav0fwDSXdezZ9gfcaWELtIIucPs2SGJP/IIfPJJSFSHHw6TJ8Nxx0H79lFHuZl7+BFYsyY8Skqqn65u/VdfQXHxlmXTaT3ebbfwo9e/f/1/5igooTdy99wDX34JP/5xOHyWeHvvvc1J/MMPQ7PE8OFw9dVw/PGw445RR1g5M9huu/Do1Knut1/2g1Hdj8PKlXDddbDPPnDzzXD22fH7n1FCb6Tc4f/+LzwA5s6FO+8MbZYSL/PnhyT+8MMwb144yTlsWPgRHzWqfhJkY5P8g9GxY9Xljj8eTjsNzjkHXn4Z/vQnaNeuoaKsf/r3b4Tc4bLL4KabYPx46NEDrr8eVqwI//TZ2VFHKNtq4cLNSfzdd0PC2n9/uOUWOPFE6NYt6ggbpy5d4Nln4de/hmuugZkzQxPMXntFHVndUEJvZDZuhIkTQ1PLxRfDjTeGGlu3bnDBBaEN9amnMvfQW6q2eHFoSnn4YSjr0TtkSPiOR4+G7qkjKMlWadYMJk0KP5Bjx4YmmD/+Ec48MwZNMO4eyWPQoEEutbN+vfvo0e7gfvXV7ps2VVz/17+6t2zpvuee7sXF0cQotVNc7H7jje5DhoTvFdzz891vuMF90aKoo4u/Zcvchw8P+72gwH316qgjqhlQ5FXkVSX0RmLNGvcjjwzf2JQpVZd78UX3du3ce/Rwf//9hotP0rdsmfstt7gfcIC7WfhO+/d3/9Wv3BcsiDq6pmfDBvfrr3dv1sx9993d58yJOqLqVZfQdQu6RuCbb0Kf4ueeCydxfvzjqsseckg42bNuXTikfOutBgtTqrFiBUydCoceCjvvDBdeuLnXxQcfhC6IV14J3/te1JE2PVlZoZfQCy/AqlUweDDcfXd63SAzjRJ6hluxIiSB11+HwsLQfl6TvDz4979DP+RDDoF//KP+45QtffUV3HtvuLinW7fQs2LJEvj5z+G//w09k665BnbfPepIBeDgg8MP6377wVlnwemnw7ffRh1V7SihZ7ClS+Ggg8I//+OPhxM46fr+90NS//73YeRI+Mtf6i9OqWjmzHBhT9euMGFC6C9++eUwa1bogviLX8APfhB1lFKZrl1hxozQHbiwEPbeO/T9byyU0DPUwoWhyWTx4tDN6uija7+NnXaCf/0r1DhOPTVcTCH1Z/nycAS1997wn/+EXkhvvQUffxy6yQ0cGINeFE1AVlY4cnrhhXCUNXhwONJqDJTQM9C8eXDAAfD11/Dii+FQcGttv31oez/hBPjRj0J3rcbYNpjJNmwI3d569w7/+JdeCh99BFOmhOSuJN44HXJIaILZd99wpDV+fLjiNJMpoWeYmTPhwAPDqHmvvBJqB9uqdetw8cTZZ8OvfhWeN2zY9u0K/POfoeZ98cUhec+ZA7/7XfghlcavW7dwDuraa+HPfw7f8dy5UUdVNSX0DPLqq6FW0LZtmK7LdtasrNBD5qqrwhn8k06CtWvrbvtNzeLFcPLJ4fv69ttwjmPGDNhjj6gjk7qWlRV6Iz3/fOiZtPfecN99UUdVOSX0DPHcc3DEEaFL22uvhZOZdc0snJC7+WZ48snwfl9/XffvE2dr14ZhFvr0gaefDtPz5oUxQtS0Em+HHhqaYPbZB844IzwyrQlGCT0DPPooHHtsSBKvvAI5OfX7fhddBA89FG6EcNBBNd8EQcJ5h8cfh759w+H3MceE/uNXXx0GhJKmYaedwsnSa66B++8PTaLz5kUd1WZK6BG791445ZTwh/HSS9C5c8O875gx8MwzoQfG0KHhJJ5Ubt68MEbOqFGhOeyll8J4Kz16RB2ZRCErK3RrnDEj9Gzae+/Qvp4JlNAj9Ic/hLPnw4eHP44ddmjY9z/ssHBV6bffhq6NM2c27PtnulWrQo+VvfYKg2X98Y/wzjvb1utI4uOww0ITzN57w7hxYXCvkpJoY1JCj4B7aHu95JJQ63vyyXB7rCjk54cLkLKzwxjbL7wQTRyZZNOmMJpl797hR/fMM8PFQRdeqPHmpaKddw7/M1ddFY62Bw+G99+PLh4l9AbmDj/5SWiHHTcuHLq3ahVtTL17h6EFcnPhqKPCEK5N1ZtvhiFrzzwznJguKgq9gxqqKUwan+bNQ2eD556DL74INfYHH4wmFiX0BlQ2lvnvfx9qe/fckzk1vp13Didk99kntK/femvUETWsZctCr4UhQ8L9Kh94IPQ2ysuLOjJpLA4/PDTBDBoU7op01lkRNMFUNQxjfT+a2vC5333nfsopYajUSZO2HMs8U5SUuB97bIjzmmsyN866sn59GI64XTv3Fi3cf/Yz92++iToqacxKS8P/OLj/4Ad1P4w1Gg89WiUl7iNHhr39299GHU3NSkvdJ0wI8Z5zThgvOo5mzHDv0yd8ziOPdJ8/P+qIJE6ee869Uyf3Nm3cH3yw7rZbXUJXk0s9KxvLfPp0uOOOMOpepmveHO66C664IrQfn3xyGF89LhYuDBcCHXFEGALh6afD99O7d9SRSZwccUToFTVwIPzwh6G5tb6vzlZCr0crV4Yuia+9FobiPOecqCNKn1kYIfDGG+Gxx8KP0qpVUUe1bdasCRcC9e0beib8+tdhaNSRI6OOTOIqJyeM93PllXDnneEc1fz59fd+Suj15LPPwlWYc+bUfizzTHLJJeGM/WuvhW6Ny5ZFHVHtuYeeO3vsAb/8JZx4YvinuuKK6HsYSfw1bx4GxZs+PdzjID8/9G6rD40qoRcWQs+eofaYmxvmM9Enn4SxzD/9NIxlfswxUUe0bQoK4KmnQl/s/fYLV5c2FnPmhAuBTjkFOnYMg54VFkL37lFHJk3NkUeGXjD9+9fjGDBVNa7X96O2J0UffNA9O9vL74wO7llZ7ief7P7MM+5LlmRGj4z333fv3t19xx3d33gj6mjq1htvuHfs6N61q/usWVFHU72VK90vvDDc+LdDB/fbb4/vyV1pXDZs2LZcRTUnRc3TuNuBmY0A/gBkAXe5+29S1g8D/g58klj0mLtfX9028/PzvaioKO0fntzcUOOtTocO4dcv+dG3b8MdVs+aFU6EZGWFoTb79WuY921IH3wQ+tt+/TX8/e+Zdxn8xo3hhO6kSeFuM+edF67K7dAh6shE6oaZzXT3/MrW1XhZi5llAbcChwHFwNtm9qS7p44x9qq7b8WN0tKzeHHV6159Fd59d/PjT3/afDY5KyuMYpia6Lt2rdvhTl97LZxc23HHcMKtPoa/zQR9+oSrSo84Itz8+KGHQpt0QygtDYMhffFF1Y/582HBgnD+4uabwzgsIk1FOtcpDgYWuPtCADObBhwHNOigkT16VF5D79kztFfvv//mZRs3hnbe5CT/6qsh+ZTp3HnLJN+nD7RsWfvYZswIt3jbZZeQzHfZpfbbaExycsL+POYYGD0abr9963rwuIeafnUJ+osv4PPPw/NXX1W+nZYtww90ly6h6+Evfxm6Wmp8cmlq0kno3YElSfPFwD6VlNvXzN4FlgI/cfctbtRkZhOBiQA9ajn26OTJoR9n8qW02dlheaqsrPCP3bt3SDhlvvwS/vvfion+lltg/fqwvkWL0BMiNdFXN47H3/4WerDsuWdI7F261OpjNVodOoRmpZNPhnPPDUn36qvDvkw3QS9fHmrdlenYMezLLl3Cd1A2XdmjfXslbxGg5jZ0MxsNHOHuZyXmTwMGu/tFSWXaA5vc/VszOwr4g7vvVt12a9uGDqF3wqRJofmlR4+QzAsKarWJLWzYEHpvvPtu6BFRluiXLt1cZqedwqF7cpLfffcQz4QJYfyPZ55p+OFvM0FpabhH6f33hxEjqzp7v912m2vRqY/U5Z06Zc4YNyKZpro29HQS+r7Ade5+RGL+SgB3/3U1r1kE5Lv7iqrKbE1Cb0grVlSsyc+ZE24OW1ajbNUq1EYPOyz0M49q+NtM4B6OdD7+uOqE3ZT3j0hd2taE3hz4EDgU+B/wNnBqcpOKmXUDPnd3N7PBwKNAT69m45me0CtTWhp6eZQl+datwzjIujhFRBrKNvVycfcNZnYhMIPQbfEed59rZucm1t8BnAScZ2YbgLXAmOqSeWPVokXoitivXxibQUQkk6TVD70+NMYauohI1KqroTeqS/9FRKRqSugiIjGhhC4iEhNK6CIiMaGELiISE0roIiIxoYQuIhITkfVDN7PlQA0jnGe8TkCVwxs0QdofFWl/bKZ9UdG27I+e7l7pkIGRJfQ4MLOiqjr4N0XaHxVpf2ymfVFRfe0PNbmIiMSEErqISEwooW+bqVEHkGG0PyrS/thM+6KietkfakMXEYkJ1dBFRGJCCV1EJCaU0LeCme1iZv80s/fNbK6Z/SjqmKJmZllm9o6ZPR11LFEzsx3M7FEz+yDxN7Jv1DFFycwuTfyfvGdmfzGz1lHH1JDM7B4z+8LM3kta1sHMnjezjxLPO9bFeymhb50NwI/dfQ9gCHCBmfWNOKao/Qh4P+ogMsQfgOfcvQ/Qnya8X8ysO3Ax4R7DPyDc9WxMtFE1uPuAESnLrgBedPfdgBcT89tMCX0ruPtn7j4rMb2a8A/bPdqoomNmOcBI4K6oY4mambUHDgTuBnD379z960iDil5zYLvE/YmzgaURx9Og3P0V4MuUxccB9yem7weOr4v3UkLfRmaWCwwE3ow4lCjdBPwU2BRxHJlgV2A5cG+iCeouM2sTdVBRcff/AVOAxcBnwCp3/0e0UWWEru7+GYQKItClLjaqhL4NzKwt8DfgEnf/Jup4omBmRwNfuPvMqGPJEM2BPOB2dx8IrKGODqcbo0Tb8HFAL2BnoI2Z6Rbr9UQJfSuZWQtCMi9098eijidC+wHHmtkiYBpwiJk9GG1IkSoGit297IjtUUKCb6qGA5+4+3J3LwUeA4ZGHFMm+NzMdgJIPH9RFxtVQt8KZmaENtL33f33UccTJXe/0t1z3D2XcLLrJXdvsjUwd18GLDGz3ROLDgXmRRhS1BYDQ8wsO/F/cyhN+CRxkieBcYnpccDf62KjzetiI03QfsBpwH/NbHZi2c/dfXp0IUkGuQgoNLOWwELgjIjjiYy7v2lmjwKzCL3D3qGJDQNgZn8BhgGdzKwYuBb4DfCImZ1J+NEbXSfvpUv/RUTiQU0uIiIxoYQuIhITSugiIjGhhC4iEhNK6CIiMaGELiISE0roIiIx8f8B+HafztKhiDkAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAu8klEQVR4nO3deXxU9bn48c9DCMQAiiwqmywuIGvAsAiIOLFVUAEpViiCaK8U9br2ulRaoYv3tr/SXsutS3EPiVJXCoLWsgniGpAiKCgiwQhiRNkMO8/vj+8MTMIkmSQzOTNnnvfrNa/JnDlzzpMhPPOd5/s936+oKsYYY5JfHa8DMMYYExuW0I0xxicsoRtjjE9YQjfGGJ+whG6MMT5hCd0YY3zCErqJSEReFZFrYr2vl0Rkk4hcFIfjqoicGfz5ERH5VTT7VuM8Y0Xk9erGWcFxB4tIUayPa2pfXa8DMLEjInvCHmYC+4HDwcc/U9X8aI+lqkPisa/fqeqkWBxHRNoBnwPpqnooeOx8IOp/Q5N6LKH7iKo2DP0sIpuA/1DVBWX3E5G6oSRhjPEPK7mkgNBXahG5W0S+Ap4UkZNF5BURKRaR74I/tw57zRIR+Y/gzxNE5E0RmRbc93MRGVLNfduLyFIR2S0iC0TkQRHJKyfuaGL8rYgsDx7vdRFpFvb8OBEpFJHtIjK5gvenn4h8JSJpYduuEJHVwZ/7iMjbIrJDRLaKyF9FpF45x3pKRH4X9vjO4Gu2iMh1Zfa9VEQ+EJFdIvKFiEwNe3pp8H6HiOwRkfNC723Y6/uLyPsisjN43z/a96YiInJO8PU7RGStiAwLe26oiHwUPOaXIvJfwe3Ngv8+O0TkWxFZJiKWX2qZveGp4zSgCdAWmIj7t38y+Ph0YC/w1wpe3xdYDzQD/h/wuIhINfZ9BngPaApMBcZVcM5oYvwJcC1wClAPCCWYzsDDweO3DJ6vNRGo6jvA90CgzHGfCf58GLg9+PucB+QAN1YQN8EYLgnG8wPgLKBs/f57YDzQGLgUuEFERgSfGxS8b6yqDVX17TLHbgLMA6YHf7c/A/NEpGmZ3+G496aSmNOBucDrwdfdDOSLSMfgLo/jyneNgK7AouD2nwNFQHPgVOBewOYVqWWW0FPHEWCKqu5X1b2qul1VX1TVElXdDdwPXFDB6wtV9VFVPQw8DbTA/ceNel8ROR3oDdynqgdU9U1gTnknjDLGJ1X1E1XdCzwHZAW3jwJeUdWlqrof+FXwPSjPs8AYABFpBAwNbkNVV6jqO6p6SFU3AX+LEEckPw7Gt0ZVv8d9gIX/fktU9UNVPaKqq4Pni+a44D4APlXVmcG4ngXWAZeH7VPee1ORfkBD4PfBf6NFwCsE3xvgINBZRE5U1e9UdWXY9hZAW1U9qKrL1CaKqnWW0FNHsaruCz0QkUwR+VuwJLEL9xW/cXjZoYyvQj+oaknwx4ZV3Lcl8G3YNoAvygs4yhi/Cvu5JCymluHHDibU7eWdC9caHyki9YGRwEpVLQzGcXawnPBVMI7/xrXWK1MqBqCwzO/XV0QWB0tKO4FJUR43dOzCMtsKgVZhj8t7byqNWVXDP/zCj/sj3IddoYi8ISLnBbf/EdgAvC4iG0Xknuh+DRNLltBTR9nW0s+BjkBfVT2RY1/xyyujxMJWoImIZIZta1PB/jWJcWv4sYPnbFrezqr6ES5xDaF0uQVc6WYdcFYwjnurEwOubBTuGdw3lDaqehLwSNhxK2vdbsGVosKdDnwZRVyVHbdNmfr30eOq6vuqOhxXjpmNa/mjqrtV9eeq2gH3LeEOEcmpYSymiiyhp65GuJr0jmA9dkq8Txhs8RYAU0WkXrB1d3kFL6lJjC8Al4nIwGAH5m+o/O/9GeAW3AfH82Xi2AXsEZFOwA1RxvAcMEFEOgc/UMrG3wj3jWWfiPTBfZCEFONKRB3KOfZ84GwR+YmI1BWRq4DOuPJITbyLq+3fJSLpIjIY9280K/hvNlZETlLVg7j35DCAiFwmImcG+0pC2w9HPIOJG0voqesB4ATgG+Ad4LVaOu9YXMfiduB3wN9x4+UjeYBqxqiqa4GbcEl6K/AdrtOuIs8Cg4FFqvpN2Pb/wiXb3cCjwZijieHV4O+wCFeOWFRmlxuB34jIbuA+gq3d4GtLcH0Gy4MjR/qVOfZ24DLct5jtwF3AZWXirjJVPQAMw31T+QZ4CBivquuCu4wDNgVLT5OAq4PbzwIWAHuAt4GHVHVJTWIxVSfWb2G8JCJ/B9apaty/IRjjd9ZCN7VKRHqLyBkiUic4rG84rhZrjKkhu1LU1LbTgJdwHZRFwA2q+oG3IRnjD1ZyMcYYn7CSizHG+IRnJZdmzZppu3btvDq9McYkpRUrVnyjqs0jPedZQm/Xrh0FBQVend4YY5KSiJS9QvgoK7kYY4xPWEI3xhifsIRujDE+YePQjUkhBw8epKioiH379lW+s/FURkYGrVu3Jj09PerXWEI3JoUUFRXRqFEj2rVrR/nrkxivqSrbt2+nqKiI9u3bR/26pCq55OdDu3ZQp467z7flco2pkn379tG0aVNL5glORGjatGmVv0klTQs9Px8mToSS4NIIhYXuMcDYsd7FZUyysWSeHKrz75Q0LfTJk48l85CSErfdGGNMEiX0zZurtt0Yk3i2b99OVlYWWVlZnHbaabRq1ero4wMHDlT42oKCAm655ZZKz9G/f/+YxLpkyRIuu+yymByrtiRNQj+97OJdlWw3xtRcrPutmjZtyqpVq1i1ahWTJk3i9ttvP/q4Xr16HDp0qNzXZmdnM3369ErP8dZbb9UsyCSWNAn9/vshM7P0tsxMt90YE3uhfqvCQlA91m8V68EIEyZM4I477uDCCy/k7rvv5r333qN///707NmT/v37s379eqB0i3nq1Klcd911DB48mA4dOpRK9A0bNjy6/+DBgxk1ahSdOnVi7NixhGaXnT9/Pp06dWLgwIHccsstlbbEv/32W0aMGEH37t3p168fq1evBuCNN944+g2jZ8+e7N69m61btzJo0CCysrLo2rUry5Yti+0bVoGk6RQNdXxOnuzKLKef7pK5dYgaEx8V9VvF+v/dJ598woIFC0hLS2PXrl0sXbqUunXrsmDBAu69915efPHF416zbt06Fi9ezO7du+nYsSM33HDDcWO2P/jgA9auXUvLli0ZMGAAy5cvJzs7m5/97GcsXbqU9u3bM2bMmErjmzJlCj179mT27NksWrSI8ePHs2rVKqZNm8aDDz7IgAED2LNnDxkZGcyYMYOLL76YyZMnc/jwYUrKvolxlDQJHdwfUaIk8Fmz4Ikn4LXX3NdRY/ymNvutrrzyStLS0gDYuXMn11xzDZ9++ikiwsGDByO+5tJLL6V+/frUr1+fU045hW3bttG6detS+/Tp0+fotqysLDZt2kTDhg3p0KHD0fHdY8aMYcaMGRXG9+abbx79UAkEAmzfvp2dO3cyYMAA7rjjDsaOHcvIkSNp3bo1vXv35rrrruPgwYOMGDGCrKysmrw1VWKpqJr+8Af4179gzRqvIzEmPmqz36pBgwZHf/7Vr37FhRdeyJo1a5g7d265Y7Hr169/9Oe0tLSI9fdI+1RnUZ9IrxER7rnnHh577DH27t1Lv379WLduHYMGDWLp0qW0atWKcePGkZubW+XzVZcl9GpYvRpWrXI/L1zoaSjGxI1X/VY7d+6kVatWADz11FMxP36nTp3YuHEjmzZtAuDvf/97pa8ZNGgQ+cHOgyVLltCsWTNOPPFEPvvsM7p168bdd99NdnY269ato7CwkFNOOYXrr7+en/70p6xcuTLmv0N5Kk3oItJGRBaLyMcislZEbo2wj4jIdBHZICKrRaRXfMJNDDNnQt260KoVLFrkdTTGxMfYsTBjBrRtCyLufsaM+Jc977rrLn7xi18wYMAADh8+HPPjn3DCCTz00ENccsklDBw4kFNPPZWTTjqpwtdMnTqVgoICunfvzj333MPTTz8NwAMPPEDXrl3p0aMHJ5xwAkOGDGHJkiVHO0lffPFFbr31uJQZN5WuKSoiLYAWqrpSRBoBK4ARqvpR2D5DgZuBoUBf4C+q2rei42ZnZ2syLnBx6BC0aQN9+0KLFq7H/9tvXYI3JtF9/PHHnHPOOV6H4bk9e/bQsGFDVJWbbrqJs846i9tvv93rsI4T6d9LRFaoanak/SttoavqVlVdGfx5N/Ax0KrMbsOBXHXeARoHPwh8Z+FC+OorGD8eAgHYvRuS8HPJmJT26KOPkpWVRZcuXdi5cyc/+9nPvA4pJqrUrhSRdkBP4N0yT7UCvgh7XBTctrXM6ycCEwFOT9IrgnJz4eST4dJLXTIHV3bp18/buIwx0bv99tsTskVeU1F3iopIQ+BF4DZV3VX26QgvOa6Wo6ozVDVbVbObN4+4xmlC27ULXn4ZRo+G+vWhWTPo0cM6Ro0xiSGqhC4i6bhknq+qL0XYpQhoE/a4NbCl5uEllhdfhL17XbklJCcHli8HWy/AGOO1aEa5CPA48LGq/rmc3eYA44OjXfoBO1V1azn7Jq3cXDjrLNchGhIIwP79kMLTRxhjEkQ0LfQBwDggICKrgrehIjJJRCYF95kPbAQ2AI8CN8YnXO8UFsKSJa51Hj5N8fnnQ1qaDV80xngvmlEub6qqqGp3Vc0K3uar6iOq+khwH1XVm1T1DFXtpqq+G/eRl+fur7669PYTT4Q+fayObkw0Bg8ezD//+c9S2x544AFuvLH8NuDgwYMJDXEeOnQoO3bsOG6fqVOnMm3atArPPXv2bD766Ohoa+677z4WLFhQhegjS6Rpdu1K0SiounLLBRe4KUTLCgTg/fddp6kxpnxjxoxh1qxZpbbNmjUrqgmywM2S2Lhx42qdu2xC/81vfsNFF11UrWMlKkvoUXjvPfjkk9KdoeFycuDwYajFWTKNSUqjRo3ilVdeYf/+/QBs2rSJLVu2MHDgQG644Qays7Pp0qULU6ZMifj6du3a8c033wBw//3307FjRy666KKjU+yCG2Peu3dvevTowY9+9CNKSkp46623mDNnDnfeeSdZWVl89tlnTJgwgRdeeAGAhQsX0rNnT7p168Z11113NL527doxZcoUevXqRbdu3Vi3bl2Fv5/X0+za9Y1RyM2FjAwYNSry8+ed54YxLlzoxqcbkwxuu+3YnESxkpUFDzxQ/vNNmzalT58+vPbaawwfPpxZs2Zx1VVXISLcf//9NGnShMOHD5OTk8Pq1avp3r17xOOsWLGCWbNm8cEHH3Do0CF69erFueeeC8DIkSO5/vrrAfjlL3/J448/zs0338ywYcO47LLLGFXmP/K+ffuYMGECCxcu5Oyzz2b8+PE8/PDD3HbbbQA0a9aMlStX8tBDDzFt2jQee+yxcn8/r6fZtRZ6Jfbvd1Pljhjh6uWRZGTAgAHWMWpMNMLLLuHllueee45evXrRs2dP1q5dW6o8UtayZcu44ooryMzM5MQTT2TYsGFHn1uzZg3nn38+3bp1Iz8/n7Vr11YYz/r162nfvj1nn302ANdccw1Lly49+vzIkSMBOPfcc49O6FWeN998k3HjxgGRp9mdPn06O3bsoG7duvTu3Zsnn3ySqVOn8uGHH9KoUaMKjx0Na6FXYv58N1dLeeWWkJwcN/F/cTEk4TVTJgVV1JKOpxEjRnDHHXewcuVK9u7dS69evfj888+ZNm0a77//PieffDITJkwod9rcEJFI1zO6FZBmz55Njx49eOqpp1iyZEmFx6lsPqvQFLzlTdFb2bFC0+xeeumlzJ8/n379+rFgwYKj0+zOmzePcePGceeddzK+skRTCWuhVyI3F049FX7wg4r3CwTcfSV/O8akvIYNGzJ48GCuu+66o63zXbt20aBBA0466SS2bdvGq6++WuExBg0axMsvv8zevXvZvXs3c+fOPfrc7t27adGiBQcPHjw65S1Ao0aN2B2aryNMp06d2LRpExs2bABg5syZXHDBBdX63byeZtda6BXYvh3mzYObb658NsXsbGjUyNXRr7yyduIzJlmNGTOGkSNHHi299OjRg549e9KlSxc6dOjAgAEDKnx9r169uOqqq8jKyqJt27acf/75R5/77W9/S9++fWnbti3dunU7msRHjx7N9ddfz/Tp0492hgJkZGTw5JNPcuWVV3Lo0CF69+7NpEmTjjtnNKZOncq1115L9+7dyczMLDXN7uLFi0lLS6Nz584MGTKEWbNm8cc//pH09HQaNmwYk4UwKp0+N16SYfrchx6Cm25yHUc9elS+/+WXw/r1bkSMMYnIps9NLjGfPjeV5eZC9+7RJXNwdfRPP4Uvvqh8X2OMiTVL6OVYvx7efbfyztBwoTq6jXYxxnjBEno5Zs6EOnXgJz+J/jVdu7opdS2hm0TmVZnVVE11/p0soUdw5IhL6D/8oVtmLlp16rhW+sKFbroAYxJNRkYG27dvt6Se4FSV7du3k5GRUaXX2SiXCJYuhc2b4fe/r/prAwF47jlXSw9ep2BMwmjdujVFRUUUFxd7HYqpREZGBq1bt67SayyhR5Cb64YgDh9e9dfm5Lj7RYssoZvEk56eTvv27b0Ow8SJlVzKKCmB5593Y8kzM6v++jPOgDZtbDpdY0zts4RexuzZsGdP1Ua3hBNxrfTFi10t3hhjaosl9DJyc6FtW7cSUXUFAu4q0+DMmcYYc9T48fD3v8fn2JbQw2zZAv/6F4wb50asVJeNRzfGRPLRR24E3dY4rbhsCT3MM8+4Mklw9stqa9UKOna0OroxprT8fNdYHD06Pse3hB6kCk8/Df36xWZ0SiDghj8ePFjzYxljkt+RI67ReNFFcNpp8TmHJfSgf/8b1qypfmdoWTk5rnM1wecfM8bUkrfegk2bjl9oPpYsoQfl5kJ6Olx1VWyON3iwu7eyizEGXLklMxOuuCJ+57CEDhw65N7syy+HJk1ic8ymTd36itYxaow5cMCNbBk+HBo2jN95LKEDr78OX38du3JLSE6O+5q1d29sj2uMSS6vvgrffRffcgtYQgdcuaVpUxgyJLbHDQTcItNvvRXb4xpjkkt+vpuJtbKlLGsq5RP6jh3u6tAxY6Bevdge+/zz3dJ1Vkc3JnXt3Alz5rihiunp8T1Xyif0F15wrehYl1vATfDVp4/V0Y1JZS+95HLM2LHxP1fKJ/TcXOjUyS3yHA85OfD+++5T2hiTevLy3KR9ffvG/1wpndA3boRly1zrXCQ+5wgE3AUFS5fG5/jGmMT15Zduor6rr45fjgmX0gk9L8+9yfH8KtSvH2RkWNnFmFT07LPuKvTaKLdACid0VVduufBCOP30+J0nIwMGDrSOUWNSUV6e60c766zaOV/KJvS334bPPotPZ2hZgQB8+KEb626MSQ1r1rgpRWqrdQ4pnNBnznSX4Y4cGf9zhZalW7w4/ucyxiSG/HxIS4vddCLRSMmEvn8/zJrlknmjRvE/X69ecOKJVkc3JlWEZlb8wQ/g1FNr77wpmdBfecVdUFQb5RZwFxcNHmwJ3ZhU8eabsHlz/C/1LyslE3puLrRseWxlodoQCMCGDe4f2Rjjb/n50KABjBhRu+dNuYReXAzz57uOirS02juvLUtnTGrYvx+ee84l8wYNavfcKZfQZ81y0+XWVrklpGtXaN7chi8a43evvupKurVdboEUTOi5udCzp0uwtUnEtdIXLXJj4I0x/pSXB6ec4paaq22VJnQReUJEvhaRNeU8P1hEdorIquDtvtiHGRsffeSWhKvt1nlITg5s2QLr13tzfmNMfO3YAXPnupkV69at/fNH00J/Crikkn2WqWpW8PabmocVHzNnurr5mDHenN/q6Mb424svutWJavNionCVJnRVXQp8WwuxxNXhw+6r0CWX1O640HAdOrhpBiyhG+NPeXnuMv/evb05f6xq6OeJyL9F5FUR6VLeTiIyUUQKRKSguLg4RqeOzpIlUFTkXbkFXB09J8ddMXrkiHdxGGNi74sv4I03am9mxUhikdBXAm1VtQfwf8Ds8nZU1Rmqmq2q2c2bN4/BqaOXmwsnneQWgvZSIADffuvmeDDG+EdoZsWf/MS7GGqc0FV1l6ruCf48H0gXkWY1jiyG9uxxta0f/xhOOMHbWEJ1dBu+aIy/5Oe76bLPPNO7GGqc0EXkNBH3BUNE+gSPub2mx42ll1+G77/3ttwS0rKlWyHJ6ujG+Mfq1e7mVWdoSKUDa0TkWWAw0ExEioApQDqAqj4CjAJuEJFDwF5gtGpijbTOzYX27WHAAK8jcXJy4KmnXG94rBemNsbUPi9mVoyk0oSuqhUO8lPVvwJ/jVlEMVZU5Mob993nXUdFWYEAPPigW2s0UT5kjDHVE5pZ8eKL3dXgXvL9laL5+a6jYtw4ryM5ZvBg9+FiZRdjkt/Spa7h6MWl/mX5OqGHlpkbMMCtup0omjRx0w9Yx6gxyS8/Hxo2hOHDvY7E5wl95Up3uX8idIaWFQi4ZfBKSryOxBhTXfv2wfPPwxVXuBXQvObrhJ6bC/Xrw5VXeh3J8XJyXKfo8uVeR2KMqa7582HnzsQot4CPE/rBg66jYtgwOPlkr6M53sCBbvIeq6Mbk7zy8txUIrW5WE5FfJvQX3sNvvkmMcst4Gpu/fpZQjcmWX33Hcyb5yb782JmxUh8m9Bzc90Qoosv9jqS8gUCbjrfHTu8jsQYU1UvvODtzIqR+DKhf/cdzJnj5lRIT/c6mvIFAm4M69KlXkdijKmqvDzo2BHOPdfrSI7xZUJ/7jn3yZmo5ZaQfv3c3DI2fNGY5LJ5s2uIeTmzYiS+TOi5udClixvrncjq13edo1ZHNya5PPOMu/dyZsVIfJfQP/sM3nrLtc4T6ZOzPDk5sGYNbNvmdSTGmGiounJL//5u0ZpE4ruEPnOmS+SJ1FFRkdBwp8WLvY3DGBOd1ath7drEzDG+SuihS/0vughatfI6muj06uUW3rCyizHJIT/fDVP88Y+9juR4vkroy5fD558nfmdouLQ0N1mXdYwak/gOH3b180sugWYJtYyP46uEnpsLDRq4eRWSSSAAGzfCpk1eR2KMqcgbb8CXXybOpf5l+Sah793rhiuOGuWSejLJyXH3VnYxJrHl50OjRt6vTVwe3yT0uXPdJDnJVG4J6dzZzQdhCd2YxLVvn7s6dOTIxJhZMRLfJPTcXGjTxtWjk42IK7ssXOg6do0xieeVV2DXrsQtt4BPEvq2bW4yrquvhjpJ+hsFAvDVV7BundeRGGMiycuDFi3gwgu9jqR8SZr+Snv2Wdf7nEjLzFVVaDy6lV2MSTzffuvmPh8zxo1MS1S+SOi5uZCdDeec43Uk1dehA7RrZ8MXjUlEzz/v1lhIxIuJwiV9Qv/wQ/jgg+TsDC0rEIAlS9y3DWNM4sjPdw3GRJ8fKukT+syZ7qqt0aO9jqTmcnLc1L+rVnkdiTEmZNMmWLYs8WZWjCSpE/rhw66jYuhQt5hFsgt1tlgd3ZjEkagzK0aS1Al94ULYutUf5RZwPeidO1sd3ZhEEZpZceBA18eV6JI6oefmQuPGcNllXkcSO4GA+3p34IDXkRhjVq2Cjz9O/M7QkKRN6Lt3w0svudp5/fpeRxM7gQCUlMB773kdiTEmP98tY3nllV5HEp2kTegvvujmb/FLuSVk8GDX8WJlF2O8FZpZccgQaNrU62iik7QJPTcXzjzTrcvpJyef7OZIt45RY7y1ZInro0vkS/3LSsqEXljoVvhJlmXmqionB95+G77/3utIjEldeXlw4onJ1UeXlAk9P9/dJ9MnZ1UEAu6qtOXLvY7EmNS0d68r6/7oR3DCCV5HE72kS+ihZeYGDYL27b2OJj4GDnQdMclUR581C/70J/chtHev19EYUzNz57qBF8nWaKzrdQBV9f77sH493Hmn15HET4MGrm8gWerojzwCN9xw7HF6OmRlud/hvPPcfbt2/iyPGX/Ky4OWLeGCC7yOpGqSroVeUgL9+7uVifwsEICVK91UAIls/ny46SZ3te6XX8I//gE//7n7UHr8cXd1XYcO7qKpESPgD39wy3hZ/4BJVN98A6++6v52E3lmxUhEPVpRITs7WwsKCjw5dzJYtsyVlV5+2SXCRPTBB3D++XD22bB0KTRsWPr5Q4dgzRp45x3XyfvOO/DJJ+65tDTo3r10K/7MM60Vb7z38MNw443u7zsry+tojiciK1Q1O+JzltAT04EDbgjjT38K06d7Hc3xvvjCJeE6deDdd93X02hs3+72DyX4d991tUpwY33DE3yfPm79RmNq08CBsGOHm8k1ERsYFSX0pKuhp4p69VzrNxE7RnftgksvdYl4+fLokzm4pD10qLuBu3jj44+PJfi334Z589xzItC1a+kk37Fj8q5KZRLfxo3ub/p//icxk3llLKEnsEAA7r7bLU132mleR+McPOgug/74Y1c/79atZsdLS3NJu2tXuP56t23HDtdyf+cdd3v+eXj0Ufdc48bQt++xBN+3r9tmTCyEZlYcM8bbOKqr0oQuIk8AlwFfq2rXCM8L8BdgKFACTFDVlbEONBWFL0uXCFN3qrra4uuvw2OPwQ9+EJ/zNG4MF1/sbgBHjrjae3gr/te/Prag9jnnHEvw550HXbokZ+vKeEvVXeMyaBC0bet1NNUTzZfXp4BLKnh+CHBW8DYReLjmYRlwq6M0bpw4wxd//3uXyCdPdrX92lKnDnTqBNdeC3/7G6xeDTt3woIF8LvfwRlnuNE1Eye6bwzDh7tavTFVsXKlW6Q9WWZWjKTSFrqqLhWRdhXsMhzIVde7+o6INBaRFqq6NVZBpqq0NDdZVyIk9GefhXvvdd8Ufvtbr6NxnaU5Oe4GrnX12Wfu6r777oMePdzX50GDvI3TJI/8fNd3lSwzK0YSi+6lVsAXYY+LgttMDOTkwOefu5tXli2DCRNcJ+0TTyRmOUPEDXu8+25XlsnMdCtA/frXtkarqdzhw67RMnSoG12WrGKR0CP99444FlJEJopIgYgUFBcXx+DU/hdeR/fC+vVuHHz79jB7dnLMPd+zJ6xY4S7bnjrVfSgWFXkdlUlkixa5wQfJdql/WbFI6EVAm7DHrYEtkXZU1Rmqmq2q2c39sAhoLTjnHDfCxYvhi8XFrsWSluZGtDRpUvsxVFejRvD00+5WUOBKMHPneh2VSVR5eXDSSW44bjKLRUKfA4wXpx+w0+rnsSPiWumLFh0b1VEb9u6FYcNgyxaYM8ddvp+Mxo93nV1t27rf57bbYP9+r6MyiaSkxK1+NmoUZGR4HU3NVJrQReRZ4G2go4gUichPRWSSiEwK7jIf2AhsAB4FboxbtCkqEIBt2+Cjj2rnfEeOwLhxbix4Xl7yLyJy9tluqOOtt8Jf/uKGNoamIDBmzhzYsyf5yy0Q3SiXCofYB0e33BSziMxxQiM5Fi1yY6zj7e673WiRP/3JzQftB/XrwwMPuA/Ha691q0I9/LD74DKpLS8PWrf2x4gou4g6CbRr5zola6Nj9KGHYNo0N4Pi7bfH/3y1bdgw+Pe/4dxzXTnmmmtc68ykpuJi+Oc/3XBcP0wp4YNfITXk5Lg1DuM5BG/ePLj5Zrfk1gMPJObwxFho3dp9OE6d6lpnvXq5mfVM6nnuOTcraDJfTBTOEnqSCATcHCfxSjwrV8JVV7npQp99Fur6fJaftDSYMsWtTVtS4voJpk+v3Y5n4738fHd1cffuXkcSG5bQk0RoPHo8hi9u3uxa5U2awCuvHD+vuZ8NGuRKMBdf7DpNbdqA1PHZZ66z3A+doSGW0JPEqae6DtFY19F37nRjb7//3o01b9EitsdPBk2burlg/vIXV0/t0cMt2GH87ZlnXFkxWWdWjMQSehIJBNxl+AcOxOZ4Bw+6sbfr1rlRLV2Pm0szdYjALbfYtAGpQtX1n1xwAbRpU/n+ycISehLJyXEX/LzzTs2PpQqTJrkZC2fMgIsuqvkx/cCmDUgNBQXuWgS/dIaGWEJPIhdc4IZWxaLs8t//7Sba+tWv3Lhsc0zZaQOyslzfgvGP0MyKflts3hJ6Emnc2I2frmnHaH4+/PKX7qKaX/86JqH5UmjagNNPh8svt2kD/OLQITeS6/LL/bfalSX0JBMIuJLL999X7/VvvAHXXefmWX/sMf+ONY+VSNMGfPqp11GZmli4EL7+2n/lFrCEnnQCAdfCWLas6q9dtw6uuMJNtPXSS+4rp6lcaNqAf/wDCgvdhUgzZ3odlamuvDzXMg8tVO4nltCTzMCBkJ5e9Tr611+7P+D0dDc8MZkn8fdKaNqAXr1s2oBk9f338PLLblWiZJjbv6osoSeZzEz3tb8qCb2kxCWjr75yc4K3bx+/+PzOpg1Ibv/4h0vqfrqYKJwl9CSUk+M66779tvJ9Dx92f7zvvecupOjTJ/7x+V1o2oBFi2zagGSTn+/GnQ8c6HUk8WEJPQkFAi55vPFG5fvedZf7ivm//+uWkjOxc8EFsGoV/PCHNm1AMvj6a3cl8Nix/phZMRKf/lr+1qcPNGhQ+fDFv/4V/vxndwXkrbfWTmypplkzt0CCTRuQ2FTdN9TQN1a/8vmcev5Urx6cf37FdfS5c10SHzbMJXUTP6FpAwYOhNGj3bQBkye7i1Y6dEityc5qk6orO27bVvnt66/dNQQ9etTOIjFeEfWo8Jedna0FBQWenNsP/vhHV0758kto2bL0cytWuFkEO3d2c6g3aOBJiClp9263OEj4sMZTT4Uzzjh2O/PMYz83a2bXAoQ7cgS++Sb6JH3o0PHHqFsXTjnFve9lb0OGuIXXk5mIrFDV7IjPWUJPTitXuqtG8/JKXyBRWOg66erXdxcgnXaadzGmstWr3bj/DRvcNK2hW9l5YRo1Kj/Zt27tOmCT3cGD0Sfp4mKX1MtKT4+coE891f2Nhz8++WT/1sih4oRuJZck1aOH+8NdtOhYQt+xw02Fu3evq69bMvdO9+6RF03Ytw8+//xYgg8l/A8/dLX4gweP7Vuvnlt+MFKyb9++9leo37/flTgqu23fXvrx7t2Rj5eRcSwJt23r+obKS9qNG9s3mWhYQk9SaWmuVrtwoaslHjzoFnT+5BN47TVXbjGJJyPDfeWP9LX/8GHXgg9v0YcS/ptvlk6MItCqVeRkf8YZFc9RsndvdIm47K2i6SbS0twCKaFby5ZuOuamTd3jpk2PT9KNGlmSjjVL6EksEHCX8G/cCL/7nWutP/30sdWNTHJJS3Mt1bZtj/83VHVli/BkH0r48+a5ckW4Jk1cYj/tNNi1q3TC3rev/BjS048l4SZNXCw9e5ZO1uHPh26WnBODJfQklpPj7q+6ynWETp3qLkk3/iMCzZu7W79+xz+/Z4/7YC+b8AsLXWnuzDNdSaNsIi6bpDMzLTEnM0voSaxjR7dk3IoVLpHfd5/XERmvNGxYft3epA5L6ElMBK6/HtasgUcftZaVManOEnqSswUqjDEhPh6taYwxqcUSujHG+IQldGOM8QlL6MYY4xOW0I0xxicsoRtjjE9YQjfGGJ+whG6MMT5hCd0YY3zCEnoSy89382XXqePu8/O9jsgY4yW79D9J5efDxIlQUuIeFxa6x1B6BSNjTOqwFno1JELLePLkY8k8pKTEbTfGpCZroVdRorSMN2+u2nZjjP9F1UIXkUtEZL2IbBCReyI8P1hEdorIquDNtzNzJ0rL+PTTq7bdGON/lSZ0EUkDHgSGAJ2BMSISacXKZaqaFbz9JsZxJoxEaRnff79bXSZcZqbbboxJTdG00PsAG1R1o6oeAGYBw+MbVuJKlJbx2LEwY4Zb81HE3c+YYR2ixqSyaBJ6K+CLsMdFwW1lnSci/xaRV0WkS6QDichEESkQkYLi4uJqhOu9RGoZjx0LmzbBkSPu3pK5MaktmoQeaWEzLfN4JdBWVXsA/wfMjnQgVZ2hqtmqmt28efMqBZoorGVsjElU0YxyKQLahD1uDWwJ30FVd4X9PF9EHhKRZqr6TWzCTCxjx1oCN8Yknmha6O8DZ4lIexGpB4wG5oTvICKnibglikWkT/C422MdrDHGmPJV2kJX1UMi8p/AP4E04AlVXSsik4LPPwKMAm4QkUPAXmC0qpYtyxhjjIkj8SrvZmdna0FBgSfnNsaYZCUiK1Q1O9Jzdum/Mcb4hCV0Y4zxCUvoxhjjE5bQjTHGJyyhG2OMT1hCN8YYn7CEbowxPmEJ3RhjfMISujHG+IQldGOM8QlL6MYY4xOW0I0xxicsoRtjjE9YQjfGGJ+whG6MD+XnQ7t2UKeOu8/P9zoiUxuiWYLOGJNE8vNh4kQoKXGPCwvdY7ClE/3OWuimxqw1mFgmTz6WzENKStx242/WQjc1Yq3BxLN5c9W2G/+wFrqpEWsNJp7TT6/aduMfltBNjVhrMPHcfz9kZpbelpnptht/s4RuasRag4ln7FiYMQPatgURdz9jhpXAUoEldFMj1hpMTGPHwqZNcOSIu7dknhosoZsaSaTWoI22ManORrmYGhs71vsWoI22McZa6MYnbLSNMZbQjU/YaBtjLKEbn7DRNsZYQjc+YaNtjLGEbnzCRtsYY6NcjI/YaBuT6qyFbkwM2Wgb4yVL6MbEkI22MV6yhG5MDNloG+MlS+jGxJCNtinNOohrlyV0Y2IokUbbeC3UQVxYCKrHOogtqcePJXRjYsxmOnQSqYM4Vb4pWEI3xsRFonQQJ9I3hXh/sFhCN8bERaJ0ECfKN4Xa+GCxhG6MiYtE6SBOlG8KtfHBElVCF5FLRGS9iGwQkXsiPC8iMj34/GoR6RW7EI0xyShROogT5ZtCbXywVJrQRSQNeBAYAnQGxohI5zK7DQHOCt4mAg/HLkRjTLJKhA7iRPmmUBsfLNG00PsAG1R1o6oeAGYBw8vsMxzIVecdoLGItIhdmMYYUz2J8k2hNj5YoknorYAvwh4XBbdVdR9EZKKIFIhIQXFxcVVjNcaYakmEbwq18cESzWyLEmGbVmMfVHUGMAMgOzv7uOeNMcbP4j0jaDQt9CKgTdjj1sCWauxjjDEmjqJJ6O8DZ4lIexGpB4wG5pTZZw4wPjjapR+wU1W3xjhWY4wxFai05KKqh0TkP4F/AmnAE6q6VkQmBZ9/BJgPDAU2ACXAtfEL2RhjTCRRrVikqvNxSTt82yNhPytwU2xDM8YYUxV2pagxxviEuMa1BycWKQYKPTl57DQDvvE6iARi70dp9n4cY+9FaTV5P9qqavNIT3iW0P1ARApUNdvrOBKFvR+l2ftxjL0XpcXr/bCSizHG+IQldGOM8QlL6DUzw+sAEoy9H6XZ+3GMvRelxeX9sBq6Mcb4hLXQjTHGJyyhG2OMT1hCrwYRaSMii0XkYxFZKyK3eh2T10QkTUQ+EJFXvI7FayLSWEReEJF1wb+R87yOyUsicnvw/8kaEXlWRDK8jqk2icgTIvK1iKwJ29ZERP4lIp8G70+OxbksoVfPIeDnqnoO0A+4KcIqTqnmVuBjr4NIEH8BXlPVTkAPUvh9EZFWwC1Atqp2xc0HNdrbqGrdU8AlZbbdAyxU1bOAhcHHNWYJvRpUdauqrgz+vBv3H/a4BT1ShYi0Bi4FHvM6Fq+JyInAIOBxAFU9oKo7PA3Ke3WBE0SkLpBJik2trapLgW/LbB4OPB38+WlgRCzOZQm9hkSkHdATeNfjULz0AHAXcMTjOBJBB6AYeDJYgnpMRBp4HZRXVPVLYBqwGdiKm1r7dW+jSginhqYYD96fEouDWkKvARFpCLwI3Kaqu7yOxwsichnwtaqu8DqWBFEX6AU8rKo9ge+J0dfpZBSsDQ8H2gMtgQYicrW3UfmXJfRqEpF0XDLPV9WXvI7HQwOAYSKyCbeAeEBE8rwNyVNFQJGqhr6xvYBL8KnqIuBzVS1W1YPAS0B/j2NKBNtEpAVA8P7rWBzUEno1iIjgaqQfq+qfvY7HS6r6C1VtrartcJ1di1Q1ZVtgqvoV8IWIdAxuygE+8jAkr20G+olIZvD/TQ4p3EkcZg5wTfDna4B/xOKgUS1wYY4zABgHfCgiq4Lb7g0uBGLMzUB+cMnGjaTwCl6q+q6IvACsxI0O+4AUmwZARJ4FBgPNRKQImAL8HnhORH6K+9C7Mibnskv/jTHGH6zkYowxPmEJ3RhjfMISujHG+IQldGOM8QlL6MYY4xOW0I0xxicsoRtjjE/8fwavccr26vrnAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6K5nkmjbrFxT"
      },
      "source": [
        "- train acc는 높은데 validation acc는 50%로 낮은 것을 볼 수 있다. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YARo6zfmXrAx"
      },
      "source": [
        "\n",
        "The model quickly starts overfitting, unsurprisingly given the small number of training samples. Validation accuracy has high variance for \n",
        "the same reason, but seems to reach high 50s.\n",
        "\n",
        "Note that your mileage may vary: since we have so few training samples, performance is heavily dependent on which exact 200 samples we \n",
        "picked, and we picked them at random. If it worked really poorly for you, try picking a different random set of 200 samples, just for the \n",
        "sake of the exercise (in real life you don't get to pick your training data).\n",
        "\n",
        "We can also try to train the same model without loading the pre-trained word embeddings and without freezing the embedding layer. In that \n",
        "case, we would be learning a task-specific embedding of our input tokens, which is generally more powerful than pre-trained word embeddings \n",
        "when lots of data is available. However, in our case, we have only 200 training samples. Let's try it:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oF6n-0pJrFxU"
      },
      "source": [
        "- 여기에서는 똑같은 예제를 Embedding layers를 풀어가지고 하면 어떻게 되는지를 본다. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LPvaTxaRXrAx",
        "outputId": "ac5edd78-d3a7-4c9e-8dd4-f9545d84d7ea"
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Embedding, Flatten, Dense\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Embedding(max_words, embedding_dim, input_length=maxlen))\n",
        "model.add(Flatten())\n",
        "model.add(Dense(32, activation='relu'))\n",
        "model.add(Dense(1, activation='sigmoid'))\n",
        "model.summary()\n",
        "\n",
        "model.compile(optimizer='rmsprop',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['acc'])\n",
        "history = model.fit(x_train, y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=32,\n",
        "                    validation_data=(x_val, y_val))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_3 (Embedding)      (None, 100, 100)          1000000   \n",
            "_________________________________________________________________\n",
            "flatten_2 (Flatten)          (None, 10000)             0         \n",
            "_________________________________________________________________\n",
            "dense_3 (Dense)              (None, 32)                320032    \n",
            "_________________________________________________________________\n",
            "dense_4 (Dense)              (None, 1)                 33        \n",
            "=================================================================\n",
            "Total params: 1,320,065\n",
            "Trainable params: 1,320,065\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "Epoch 1/10\n",
            "7/7 [==============================] - 1s 77ms/step - loss: 0.6897 - acc: 0.5586 - val_loss: 0.6940 - val_acc: 0.5032\n",
            "Epoch 2/10\n",
            "7/7 [==============================] - 0s 58ms/step - loss: 0.4914 - acc: 0.9584 - val_loss: 0.7071 - val_acc: 0.5038\n",
            "Epoch 3/10\n",
            "7/7 [==============================] - 0s 56ms/step - loss: 0.2708 - acc: 0.9931 - val_loss: 0.7707 - val_acc: 0.4961\n",
            "Epoch 4/10\n",
            "7/7 [==============================] - 0s 60ms/step - loss: 0.1229 - acc: 0.9950 - val_loss: 0.7071 - val_acc: 0.5318\n",
            "Epoch 5/10\n",
            "7/7 [==============================] - 0s 59ms/step - loss: 0.0604 - acc: 1.0000 - val_loss: 0.7554 - val_acc: 0.5086\n",
            "Epoch 6/10\n",
            "7/7 [==============================] - 0s 55ms/step - loss: 0.0308 - acc: 1.0000 - val_loss: 0.7485 - val_acc: 0.5157\n",
            "Epoch 7/10\n",
            "7/7 [==============================] - 0s 54ms/step - loss: 0.0160 - acc: 1.0000 - val_loss: 0.7592 - val_acc: 0.5171\n",
            "Epoch 8/10\n",
            "7/7 [==============================] - 0s 53ms/step - loss: 0.0099 - acc: 1.0000 - val_loss: 0.7574 - val_acc: 0.5203\n",
            "Epoch 9/10\n",
            "7/7 [==============================] - 0s 58ms/step - loss: 0.0058 - acc: 1.0000 - val_loss: 0.8067 - val_acc: 0.5129\n",
            "Epoch 10/10\n",
            "7/7 [==============================] - 0s 53ms/step - loss: 0.0035 - acc: 1.0000 - val_loss: 0.7779 - val_acc: 0.5219\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "030ikkBXXrAy",
        "outputId": "5cde1f2b-06dc-4364-cc1e-23d386e7210b"
      },
      "source": [
        "acc = history.history['acc']\n",
        "val_acc = history.history['val_acc']\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(1, len(acc) + 1)\n",
        "\n",
        "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.legend()\n",
        "\n",
        "plt.figure()\n",
        "\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAidklEQVR4nO3de3RU9bn/8fdDQDCgIFeFCMEWRCkSYoqKWLHiKVYqanEJ0ipyKiq1Hu2xra1tte3hnP5az6l1VaRUUau01NqWosdLhWrpsReJgFYUFeViRDBCBeQiCTy/P747YTLMJJNkkpnsfF5r7TX7Ps/sJJ9857v37DF3R0RE2r4OuS5ARESyQ4EuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUCPMTN73Mwuz/a6uWRm681sfAvs183so9H4XDP7VibrNuF5ppnZH5pap0h9TNeh5xcz+yBhshD4ENgfTV/l7gtav6r8YWbrgS+4+5Is79eBIe6+NlvrmlkxsA7o5O7VWSlUpB4dc12A1OXu3WrG6wsvM+uokJB8od/H/KAulzbCzMaZWYWZfc3MNgP3mtlRZvaomVWa2T+j8aKEbZ4xsy9E49PN7P/M7LZo3XVmdm4T1x1sZsvMbKeZLTGzO83swTR1Z1Lj98zs2Wh/fzCz3gnLP29mG8xsq5ndXM/xOdXMNptZQcK8C83sxWh8tJn91czeN7N3zOwnZnZYmn3dZ2b/kTD9lWibTWY2I2nd88xspZntMLO3zOzWhMXLosf3zewDMzut5tgmbD/GzJab2fbocUymx6aRx7mnmd0bvYZ/mtmihGWTzGxV9BreMLMJ0fw63VtmdmvNz9nMiqOup381s43AH6P5v45+Dtuj35HhCdsfbmb/Hf08t0e/Y4eb2f+a2ZeSXs+LZnZBqtcq6SnQ25ajgZ7AIGAm4ed3bzQ9ENgD/KSe7U8BXgV6Az8A7jEza8K6vwCeA3oBtwKfr+c5M6nxUuAKoC9wGHAjgJmdCNwV7b9/9HxFpODufwN2AZ9M2u8vovH9wA3R6zkNOBuYVU/dRDVMiOo5BxgCJPff7wIuA3oA5wHXJATRJ6LHHu7ezd3/mrTvnsD/AndEr+1/gP81s15Jr+GQY5NCQ8f5AUIX3vBoXz+KahgN/Bz4SvQaPgGsT/McqZwJnAB8Kpp+nHCc+gIrgMQuwtuAk4ExhN/jrwIHgPuBz9WsZGYjgQHAY42oQwDcXUOeDoQ/rPHR+DhgH9ClnvVLgH8mTD9D6LIBmA6sTVhWCDhwdGPWJYRFNVCYsPxB4MEMX1OqGr+ZMD0LeCIa/zawMGFZ1+gYjE+z7/8A5kfjRxDCdlCada8Hfpcw7cBHo/H7gP+IxucD309Yb2jiuin2ezvwo2i8OFq3Y8Ly6cD/ReOfB55L2v6vwPSGjk1jjjNwDCE4j0qx3k9r6q3v9y+avrXm55zw2o6rp4Ye0TrdCf9w9gAjU6zXGdhGOC8BIfjntMTfVNwHtdDblkp331szYWaFZvbT6C3sDsJb/B6J3Q5JNteMuPvuaLRbI9ftD2xLmAfwVrqCM6xxc8L47oSa+ifu2913AVvTPRehNX6RmXUGLgJWuPuGqI6hUTfE5qiO/yS01htSpwZgQ9LrO8XMno66OrYDV2e435p9b0iat4HQOq2R7tjU0cBxPpbwM/tnik2PBd7IsN5Uao+NmRWY2fejbpsdHGzp946GLqmey90/BB4CPmdmHYCphHcU0kgK9LYl+ZKkfweOB05x9yM5+BY/XTdKNrwD9DSzwoR5x9azfnNqfCdx39Fz9kq3sru/TAjEc6nb3QKh62YNoRV4JPCNptRAeIeS6BfAYuBYd+8OzE3Yb0OXkG0idJEkGgi8nUFdyeo7zm8RfmY9Umz3FvCRNPvcRXh3VuPoFOskvsZLgUmEbqnuhFZ8TQ3vAXvrea77gWmErrDdntQ9JZlRoLdtRxDexr4f9cfe0tJPGLV4y4FbzewwMzsN+EwL1fgwMNHMxkYnML9Lw7+zvwCuIwTar5Pq2AF8YGbDgGsyrOEhYLqZnRj9Q0mu/whC63dv1B99acKySkJXx3Fp9v0YMNTMLjWzjmZ2CXAi8GiGtSXXkfI4u/s7hL7tOdHJ005mVhP49wBXmNnZZtbBzAZExwdgFTAlWr8MmJxBDR8S3kUVEt4F1dRwgNB99T9m1j9qzZ8WvZsiCvADwH+j1nmTKdDbttuBwwmtn78BT7TS804jnFjcSui3/hXhDzmV22lije6+GvgiIaTfAf4JVDSw2S8J5xv+6O7vJcy/kRC2O4GfRTVnUsPj0Wv4I7A2ekw0C/iume0k9Pk/lLDtbmA28KyFq2tOTdr3VmAioXW9lXCScGJS3Zm6nfqP8+eBKsK7lHcJ5xBw9+cIJ11/BGwH/sTBdw3fIrSo/wl8h7rveFL5OeEd0tvAy1EdiW4E/gEsJ/SZ/z/qZtDPgRGEczLSBPpgkTSbmf0KWOPuLf4OQeLLzC4DZrr72FzX0laphS6NZmYfN7OPRG/RJxD6TRfluCxpw6LurFnAvFzX0pYp0KUpjiZcUvcB4Rrqa9x9ZU4rkjbLzD5FON+whYa7daQe6nIREYkJtdBFRGIiZzfn6t27txcXF+fq6UVE2qTnn3/+PXfvk2pZzgK9uLiY8vLyXD29iEibZGbJny6upS4XEZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJiQYD3czmm9m7ZvZSmuVmZneY2droa6NKs1+mSMMWLIDiYujQITwuyNHXaedDHflQg+rIQR0NfQMG4TakpcBLaZZ/mnBrTgNOBf6eyTdrnHzyyS6SLQ8+6F5Y6A4Hh8LCML+91ZEPNaiOlqsDKPd0eZ1uQZ2Vwo3q0wX6T4GpCdOvAsc0tE8FevM9+KD7oEHuZuGxtX9B86mOQYPq/qHUDIMGtb868qEG1dFydbR0oD8KjE2YXgqUpVl3JuHLEcoHDhzY5AMj8Wt1NJdZ6j8Ws/ZXRz7UoDparo76Aj0bJ0VTfY1Xyjt+ufs8dy9z97I+fVJ+clUydPPNsHt33Xm7d4f57bGOgclfDNfA/DjXkQ81qI7c1JGNQK+g7ncuFhG+K1Fa0MaNjZsf9zpmz4bCwrrzCgvD/PZWRz7UoDpyVEe6pnviQP1dLudR96Toc5nsU33ozRO3fsFsyIe+/HypIx9qUB0tUwf1dLk0eD90M6v5jsbehBvQ3wJ0iv4ZzDUzA34CTAB2A1e4e4N33SorK3PdnKvpFiyAmTPrdncUFsK8eTBtWvurQ6S9MLPn3b0s1bIG77bo7lMbWO6EL/KVVlQTljffHLo3Bg4Mb91aO0TzpQ4RyeE3FqmFLiLSePW10PXRfxGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQmyBf7q0sIpKowQ8WSV3Jn4zcsCFMgz5MIyK5pRZ6I+XL3QVFRJIp0BspX+4uKCKSTIHeSPlyb2URkWQK9EbKl3sri4gkU6A30rRp4dawgwaBWXjUrWJFJB/oKpcmmDZNAS4i+UctdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYiKjQDezCWb2qpmtNbObUiw/ysx+Z2YvmtlzZvax7JcqIiL1aTDQzawAuBM4FzgRmGpmJyat9g1glbufBFwG/DjbhYqISP0yaaGPBta6+5vuvg9YCExKWudEYCmAu68Bis2sX1YrFRGRemUS6AOAtxKmK6J5iV4ALgIws9HAIKAoeUdmNtPMys2svLKysmkVi4hISpkEuqWY50nT3weOMrNVwJeAlUD1IRu5z3P3Mncv69OnT2NrFRGRemTyJdEVwLEJ00XApsQV3H0HcAWAmRmwLhpERKSVZNJCXw4MMbPBZnYYMAVYnLiCmfWIlgF8AVgWhbyIiLSSBlvo7l5tZtcCTwIFwHx3X21mV0fL5wInAD83s/3Ay8C/tmDNIiKSQiZdLrj7Y8BjSfPmJoz/FRiS3dJERKQx9ElREZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGIio0A3swlm9qqZrTWzm1Is725mj5jZC2a22syuyH6pIiJSnwYD3cwKgDuBc4ETgalmdmLSal8EXnb3kcA44L/N7LAs1yoiIvXIpIU+Gljr7m+6+z5gITApaR0HjjAzA7oB24DqrFYqIiL1yiTQBwBvJUxXRPMS/QQ4AdgE/AP4N3c/kLwjM5tpZuVmVl5ZWdnEkkVEJJVMAt1SzPOk6U8Bq4D+QAnwEzM78pCN3Oe5e5m7l/Xp06eRpYqISH0yCfQK4NiE6SJCSzzRFcBvPVgLrAOGZadEERHJRCaBvhwYYmaDoxOdU4DFSetsBM4GMLN+wPHAm9ksVERE6texoRXcvdrMrgWeBAqA+e6+2syujpbPBb4H3Gdm/yB00XzN3d9rwbpFpBmqqqqoqKhg7969uS5F0ujSpQtFRUV06tQp423MPbk7vHWUlZV5eXl5Tp5bpL1bt24dRxxxBL169SJcnCb5xN3ZunUrO3fuZPDgwXWWmdnz7l6Wajt9UlSkHdq7d6/CPI+ZGb169Wr0OygFukg7pTDPb035+SjQRaTVbd26lZKSEkpKSjj66KMZMGBA7fS+ffvq3ba8vJzrrruuwecYM2ZMtsptMxo8KSoismAB3HwzbNwIAwfC7NkwbVrT99erVy9WrVoFwK233kq3bt248cYba5dXV1fTsWPqeCorK6OsLGUXch1/+ctfml5gG6UWuojUa8ECmDkTNmwA9/A4c2aYn03Tp0/ny1/+MmeddRZf+9rXeO655xgzZgyjRo1izJgxvPrqqwA888wzTJw4EQj/DGbMmMG4ceM47rjjuOOOO2r3161bt9r1x40bx+TJkxk2bBjTpk2j5mKQxx57jGHDhjF27Fiuu+662v0mWr9+PWeccQalpaWUlpbW+Ufxgx/8gBEjRjBy5Ehuuinct3Dt2rWMHz+ekSNHUlpayhtvvJHdA1UPtdBFpF433wy7d9edt3t3mN+cVnoqr732GkuWLKGgoIAdO3awbNkyOnbsyJIlS/jGN77Bb37zm0O2WbNmDU8//TQ7d+7k+OOP55prrjnkUr+VK1eyevVq+vfvz+mnn86zzz5LWVkZV111FcuWLWPw4MFMnTo1ZU19+/blqaeeokuXLrz++utMnTqV8vJyHn/8cRYtWsTf//53CgsL2bZtGwDTpk3jpptu4sILL2Tv3r0cOHDIXVBajAJdROq1cWPj5jfHxRdfTEFBAQDbt2/n8ssv5/XXX8fMqKqqSrnNeeedR+fOnencuTN9+/Zly5YtFBUV1Vln9OjRtfNKSkpYv3493bp147jjjqu9LHDq1KnMmzfvkP1XVVVx7bXXsmrVKgoKCnjttdcAWLJkCVdccQWFhYUA9OzZk507d/L2229z4YUXAuFa8takLhcRqdfAgY2b3xxdu3atHf/Wt77FWWedxUsvvcQjjzyS9hK+zp07144XFBRQXX3ojV5TrZPpZ3B+9KMf0a9fP1544QXKy8trT9q6+yFXouTqcz01FOgiUq/ZsyFqhNYqLAzzW9L27dsZMCDc2PW+++7L+v6HDRvGm2++yfr16wH41a9+lbaOY445hg4dOvDAAw+wf/9+AP7lX/6F+fPnszvqj9q2bRtHHnkkRUVFLFq0CIAPP/ywdnlrUKCLSL2mTYN582DQIDALj/PmZb//PNlXv/pVvv71r3P66afXhmg2HX744cyZM4cJEyYwduxY+vXrR/fu3Q9Zb9asWdx///2ceuqpvPbaa7XvIiZMmMD5559PWVkZJSUl3HbbbQA88MAD3HHHHZx00kmMGTOGzZs3Z732dPTRf5F26JVXXuGEE07IdRk598EHH9CtWzfcnS9+8YsMGTKEG264Iddl1Ur1c9JH/0VEUvjZz35GSUkJw4cPZ/v27Vx11VW5LqlZdJWLiLRbN9xwQ161yJtLLXQRkZhQoIuIxIQCXUQkJhToIiIxoUAXkVY3btw4nnzyyTrzbr/9dmbNmlXvNjWXOn/605/m/fffP2SdW2+9tfZ68HQWLVrEyy+/XDv97W9/myVLljSi+vylQBeRVjd16lQWLlxYZ97ChQvT3iAr2WOPPUaPHj2a9NzJgf7d736X8ePHN2lf+UaBLiKtbvLkyTz66KN8+OGHQLhF7aZNmxg7dizXXHMNZWVlDB8+nFtuuSXl9sXFxbz3Xvge+tmzZ3P88cczfvz42lvsQrjG/OMf/zgjR47ks5/9LLt37+Yvf/kLixcv5itf+QolJSW88cYbTJ8+nYcffhiApUuXMmrUKEaMGMGMGTNq6ysuLuaWW26htLSUESNGsGbNmkNqyofb7Oo6dJF27vrrIfquiawpKYHbb0+/vFevXowePZonnniCSZMmsXDhQi655BLMjNmzZ9OzZ0/279/P2WefzYsvvshJJ52Ucj/PP/88CxcuZOXKlVRXV1NaWsrJJ58MwEUXXcSVV14JwDe/+U3uuecevvSlL3H++eczceJEJk+eXGdfe/fuZfr06SxdupShQ4dy2WWXcdddd3H99dcD0Lt3b1asWMGcOXO47bbbuPvuu+tsnw+32VULXURyIrHbJbG75aGHHqK0tJRRo0axevXqOt0jyf785z9z4YUXUlhYyJFHHsn5559fu+yll17ijDPOYMSIESxYsIDVq1fXW8+rr77K4MGDGTp0KACXX345y5Ytq11+0UUXAXDyySfX3tArUVVVFVdeeSUjRozg4osvrq0709vsFibfAa0J1EIXaefqa0m3pAsuuIAvf/nLrFixgj179lBaWsq6deu47bbbWL58OUcddRTTp09Pe9vcGum+THn69OksWrSIkSNHct999/HMM8/Uu5+G7mtVcwvedLfoTbzN7oEDB2rvhd6at9lVC11EcqJbt26MGzeOGTNm1LbOd+zYQdeuXenevTtbtmzh8ccfr3cfn/jEJ/jd737Hnj172LlzJ4888kjtsp07d3LMMcdQVVXFgoTvyzviiCPYuXPnIfsaNmwY69evZ+3atUC4a+KZZ56Z8evJh9vsKtBFJGemTp3KCy+8wJQpUwAYOXIko0aNYvjw4cyYMYPTTz+93u1LS0u55JJLKCkp4bOf/SxnnHFG7bLvfe97nHLKKZxzzjkMGzasdv6UKVP44Q9/yKhRo+qciOzSpQv33nsvF198MSNGjKBDhw5cffXVGb+WfLjNrm6fK9IO6fa5bYNunysi0k4p0EVEYkKBLiISEwp0kXYq199QL/Vrys8no0A3swlm9qqZrTWzm1Is/4qZrYqGl8xsv5n1bHQ1ItIqunTpwtatWxXqecrd2bp1a+217Jlq8INFZlYA3AmcA1QAy81ssbvXfnzL3X8I/DBa/zPADe6+rVGViEirKSoqoqKigsrKylyXIml06dKFoqKiRm2TySdFRwNr3f1NADNbCEwC0n0edyrwy0ZVISKtqlOnTgwePDjXZUiWZdLlMgB4K2G6Ipp3CDMrBCYAv0mzfKaZlZtZuVoGIiLZlUmgp7pRQrqOt88Az6brbnH3ee5e5u5lffr0ybRGERHJQCaBXgEcmzBdBGxKs+4U1N0iIpITmQT6cmCImQ02s8MIob04eSUz6w6cCfw+uyWKiEgmGjwp6u7VZnYt8CRQAMx399VmdnW0fG606oXAH9x9V4tVKyIiaenmXCIibYhuziUi0g4o0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYyCnQzm2Bmr5rZWjO7Kc0648xslZmtNrM/ZbdMERFpSMeGVjCzAuBO4BygAlhuZovd/eWEdXoAc4AJ7r7RzPq2UL0iIpJGJi300cBad3/T3fcBC4FJSetcCvzW3TcCuPu72S1TREQakkmgDwDeSpiuiOYlGgocZWbPmNnzZnZZqh2Z2UwzKzez8srKyqZVLCIiKWUS6JZinidNdwROBs4DPgV8y8yGHrKR+zx3L3P3sj59+jS6WBERSa/BPnRCi/zYhOkiYFOKdd5z913ALjNbBowEXstKlSIi0qBMWujLgSFmNtjMDgOmAIuT1vk9cIaZdTSzQuAU4JXslgoLFkBxMXToEB4XLMj2M4iItF0NttDdvdrMrgWeBAqA+e6+2syujpbPdfdXzOwJ4EXgAHC3u7+UzUIXLICZM2H37jC9YUOYBpg2LZvPJCLSNpl7cnd46ygrK/Py8vKM1y8uDiGebNAgWL8+a2WJiOQ1M3ve3ctSLWsznxTduLFx80VE2ps2E+gDBzZuvohIe9NmAn32bCgsrDuvsDDMFxGRNhTo06bBvHmhz9wsPM6bpxOiIiI1MrkOPW9Mm6YAFxFJp8200EVEpH4KdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEy0qfuhS/6pqoJFi+Cee6BTJzjnnDAMGxa+iEREWo8CXZqkoiJ8Y9TPfgabN4dvkOrYER59NCwfMOBguJ99NvTrl9t6RdoDBbpkzB2WLoU5c2DxYjhwAM49F2bNggkToKAA1q2DJUvgqafCOvfdF7Y96aSDAX/GGYd+P6yINJ+5e06euKyszMvLy3Py3NI4778fgvmuu+C116BXL/jCF+Cqq2Dw4PTb7d8PK1eGcH/qKXj2Wdi3Dw47DMaOPRjwo0ZBB53NkXbEveldkmb2vLuXpVymQJd0VqwIIb5gAezZA6edFlrjkydDly6N39+uXfDnPx8M+H/8I8zv1Qs++cmDAV9cnNWXIS1s//7w+7F7d/rHDh2gb9+DQ9euua66ZezfD+++C5s2wdtvHxySp6+7Dr7znaY9R32Bri4XqWPvXnjooRDkf/tb6Br53OfgmmtCS7o5unYNXTMTJoTpzZtDF05NwP/612H+Rz96MNzPOgt69Gje87YX7qEbrLr64LBvX8Nhm+ljumX79jW+1sLCugFf39C7dzjhnms7d6YP6Zrxd94JoZ6oQwc45hjo3x+GDIFx4+CUU1qmRrXQBYA334S5c2H+fNi6FYYODa3xyy9vnUB1h1deORjuzzwTWvQdOsDo0SHcx4+HU08NXTb5Ys8eqKw8dNi6NQRdYrhWV4ergpLnZbo8k2XN1akTHH54CNxsPu7fH47Lu+8eOiTOr6pKXVfPnpn/A+jRo3HdGVVVoXGRLqRrhg8+OHTbHj1CUA8YEIZU4/36hfNL2aIuF0lp/3544gm4887w2KEDXHBBaI1/8pO5vexw377wDuGpp8JJ1ueeC63Pbt3gzDMPtuBPOCF7dbqHfyKpAromeJKHXbtS76ugIPzj6dQpXP2TamjqssZs29iA7pjD9+zusH176tBPNWzdmno/HTvWDfg+fQ6OV1cf2sresiU8d6JOnRoO6v79c9N1pECXOiorQ0t87lxYvz68HZw5E668Mvyi5qP334ennz7Ygl+7Nszv3/9guI8fX/fySHfYsSN1EKcL6b17Uz9/584hGFINNaGROHTvruvwW1pVVQj1TP8B1Pzz7d27/qAeMCCc18nXE/UKdME9tHjnzAl95Pv2hf7pWbNg0qT86KNsjPXrD4b70qWwbVuYP3x4aKFVVsJ776Xv3y0sTB/QqUK6WzcFdFu3a1f43ejcOdeVNE+zA93MJgA/BgqAu939+0nLxwG/B9ZFs37r7t+tb58K9Naxaxf84hchyFetgiOPDP3iV18NJ56Y6+qyo+byyCVLYNmy0N3RUFDrOnhpq5p1lYuZFQB3AucAFcByM1vs7i8nrfpnd5/Y7GobsHx5CKcDB8JQc2Y/cWjpeTXTNX11/fqlH/r2DX2TrW3NmnCc7r8/dDucdBL89Kdw6aWhtRknBQVQVhaGm27KdTUiuZPJKZDRwFp3fxPAzBYCk4DkQG8VW7bAH/8Y+rfMwmPi0JR5HTs2bdsPPwx9c6tWhbq2b09d8xFHpA/75HnNeWtfVRU+nTlnTjhGhx0GF18culVOO01dBiJxl0mgDwDeSpiuAFJdRXmamb0AbAJudPfVySuY2UxgJsDAgQMbXy0wcWIY8tHevSHgt2ypOyTOW7MG/vSn9GfoDz88fdgn/yM46qgQ0ps2hXuqzJsXxgcNgv/6L5gxI6wrIu1DJoGeql2X3PG+Ahjk7h+Y2aeBRcCQQzZynwfMg9CH3rhS81+XLjBwYBgaUlUVTtylCv2aYcOGcLleZWXo4knWqVMI7M2bw/IJE0K3yrnnZve6VxFpGzIJ9Arg2ITpIkIrvJa770gYf8zM5phZb3d/Lztlxk/Nda79+ze87v794SqO5MCvGY4+Otxb5SMfafm6RSR/ZRLoy4EhZjYYeBuYAlyauIKZHQ1scXc3s9GEL85I06kgjZV41cbHPpbrakQkXzUY6O5ebWbXAk8SLluc7+6rzezqaPlcYDJwjZlVA3uAKZ6rC9xFRNopfbBIRKQNqe869Dz9cKuIiDSWAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGIiZ5ctmlklsCEnT549vQF9GvYgHY+6dDwO0rGoqznHY5C790m1IGeBHgdmVp7uetD2SMejLh2Pg3Qs6mqp46EuFxGRmFCgi4jEhAK9eebluoA8o+NRl47HQToWdbXI8VAfuohITKiFLiISEwp0EZGYUKA3gZkda2ZPm9krZrbazP4t1zXlmpkVmNlKM3s017Xkmpn1MLOHzWxN9DtyWq5ryiUzuyH6O3nJzH5pZl1yXVNrMrP5Zvaumb2UMK+nmT1lZq9Hj0dl47kU6E1TDfy7u58AnAp80cxOzHFNufZvwCu5LiJP/Bh4wt2HASNpx8fFzAYA1wFl7v4xwpfkTMltVa3uPmBC0rybgKXuPgRYGk03mwK9Cdz9HXdfEY3vJPzBDshtVbljZkXAecDdua4l18zsSOATwD0A7r7P3d/PaVG51xE43Mw6AoUkfSdx3Ln7MmBb0uxJwP3R+P3ABdl4LgV6M5lZMTAK+HuOS8ml24GvAgdyXEc+OA6oBO6NuqDuNrOuuS4qV9z9beA2YCPwDrDd3f+Q26ryQj93fwdCAxHom42dKtCbwcy6Ab8Brnf3HbmuJxfMbCLwrrs/n+ta8kRHoBS4y91HAbvI0tvptijqG54EDAb6A13N7HO5rSq+FOhNZGadCGG+wN1/m+t6cuh04HwzWw8sBD5pZg/mtqScqgAq3L3mHdvDhIBvr8YD69y90t2rgN8CY3JcUz7YYmbHAESP72Zjpwr0JjAzI/SRvuLu/5PrenLJ3b/u7kXuXkw42fVHd2+3LTB33wy8ZWbHR7POBl7OYUm5thE41cwKo7+bs2nHJ4kTLAYuj8YvB36fjZ12zMZO2qHTgc8D/zCzVdG8b7j7Y7krSfLIl4AFZnYY8CZwRY7ryRl3/7uZPQysIFwdtpJ2dhsAM/slMA7obWYVwC3A94GHzOxfCf/0Ls7Kc+mj/yIi8aAuFxGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURi4v8DFJBJVW2kPy8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoGklEQVR4nO3de3RU9bn/8fdjUCCCaLmo5a5F8crFERUsYqtHvFQs6hFIVUp7KFZrq23VlmNh2dLVVk9/tkeUplZtJYhWLaLFS9UqVO0pAQEFwaKCRFARBYKAEHh+f3x3YDJMkkkyyUx2Pq+1Zs3sy+z9zAQ+s/d3f/fe5u6IiEjzt1+uCxARkexQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0CUtM3vSzK7M9ry5ZGarzOysRlium9kXotfTzOzmTOatx3qKzOyZ+tZZw3KHmVlZtpcrTa9VrguQ7DGzLUmDhcBnwK5o+FvuXpLpstz93MaYN+7cfUI2lmNmvYB3gP3dvSJadgmQ8d9QWh4Feoy4e7vK12a2Cvimuz+bOp+ZtaoMCRGJDzW5tACVu9RmdqOZvQ/ca2aHmNkTZrbezD6JXndLes8LZvbN6PVYM/uHmd0WzfuOmZ1bz3l7m9lcMys3s2fNbKqZTa+m7kxq/KmZvRQt7xkz65Q0/XIzW21mG8xsYg3fz6lm9r6ZFSSN+6qZLYleDzKzV8xso5mtM7M7zOyAapZ1n5n9LGn4h9F71prZuJR5zzezV81ss5mtMbPJSZPnRs8bzWyLmZ1W+d0mvX+wmc03s03R8+BMv5uamNkx0fs3mtlSM7swadp5ZrYsWuZ7ZvaDaHyn6O+z0cw+NrN5ZqZ8aWL6wluOw4DPAT2B8YS//b3RcA9gG3BHDe8/BVgBdAJ+BfzBzKwe884A/gV0BCYDl9ewzkxqHAN8HegCHABUBsyxwF3R8j8fra8babj7P4FPgS+lLHdG9HoXcF30eU4Dvgx8u4a6iWoYHtVzNtAHSG2//xS4AjgYOB+4yswuiqYNjZ4Pdvd27v5KyrI/B/wV+G302X4N/NXMOqZ8hn2+m1pq3h94HHgmet93gBIzOzqa5Q+E5rv2wPHA89H47wNlQGfgUODHgK4r0sQU6C3HbmCSu3/m7tvcfYO7P+LuW929HJgCnFHD+1e7++/dfRfwR+Bwwn/cjOc1sx7AycBP3H2Hu/8DmF3dCjOs8V53f9PdtwEPAf2j8ZcAT7j7XHf/DLg5+g6q8wAwGsDM2gPnReNw9wXu/k93r3D3VcDv0tSRzn9G9b3u7p8SfsCSP98L7v6au+929yXR+jJZLoQfgH+7+/1RXQ8Ay4GvJM1T3XdTk1OBdsAvor/R88ATRN8NsBM41swOcvdP3H1h0vjDgZ7uvtPd57kuFNXkFOgtx3p33145YGaFZva7qEliM2EX/+DkZocU71e+cPet0ct2dZz388DHSeMA1lRXcIY1vp/0emtSTZ9PXnYUqBuqWxdha3ykmbUGRgIL3X11VMdRUXPC+1EdPydsrdemSg3A6pTPd4qZ/T1qUtoETMhwuZXLXp0ybjXQNWm4uu+m1prdPfnHL3m5FxN+7Fab2Ytmdlo0/lZgJfCMmb1tZjdl9jEkmxToLUfq1tL3gaOBU9z9IPbu4lfXjJIN64DPmVlh0rjuNczfkBrXJS87WmfH6mZ292WE4DqXqs0tEJpulgN9ojp+XJ8aCM1GyWYQ9lC6u3sHYFrScmvbul1LaIpK1gN4L4O6altu95T27z3Ldff57j6C0Bwzi7Dlj7uXu/v33f0Iwl7C9Wb25QbWInWkQG+52hPapDdG7bGTGnuF0RZvKTDZzA6Itu6+UsNbGlLjw8AFZnZ6dADzFmr/9z4DuJbww/HnlDo2A1vMrC9wVYY1PASMNbNjox+U1PrbE/ZYtpvZIMIPSaX1hCaiI6pZ9hzgKDMbY2atzOwy4FhC80hD/B+hbf8GM9vfzIYR/kYzo79ZkZl1cPedhO9kF4CZXWBmX4iOlVSO35V2DdJoFOgt1+1AW+Aj4J/AU0203iLCgcUNwM+ABwn95dO5nXrW6O5LgasJIb0O+IRw0K4mDwDDgOfd/aOk8T8ghG058Puo5kxqeDL6DM8TmiOeT5nl28AtZlYO/IRoazd671bCMYOXop4jp6YsewNwAWEvZgNwA3BBSt115u47gAsJeyofAXcCV7j78miWy4FVUdPTBOBr0fg+wLPAFuAV4E53f6EhtUjdmY5bSC6Z2YPAcndv9D0EkbjTFro0KTM72cyONLP9om59IwhtsSLSQDpTVJraYcCjhAOUZcBV7v5qbksSiQc1uYiIxISaXEREYiJnTS6dOnXyXr165Wr1IiLN0oIFCz5y987ppuUs0Hv16kVpaWmuVi8i0iyZWeoZwnuoyUVEJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMKdBGRmMgo0M1suJmtMLOV6e5EYmYdzOxxM1sc3VT269kvVUSkfjZuhPvugzvvhNJS2LEj1xU1jlpPLIpu9zWVcKPbMmC+mc2O7vBS6Wpgmbt/xcw6AyvMrCS6trKISJP77DN48kkoKYHHHw/DlQ44APr3h0GD4OSTw/NRR8F+zbzNIpMzRQcBK939bQAzm0m45GlyoDvQPrpbSTvgY6Aiy7WKiNRo9254+WWYPh0eegg++QS6dIEJE+BrX4POnWH+fPjXv8LjvvvgjjvCew86CBKJEO6VQd+1K1hj3pQxyzIJ9K5UvdFtGXBKyjx3EO6NuJZwW63LUm4yC4CZjQfGA/TokXp7RRGR+nnjjbAlXlICq1ZBYSGMHBlC/MtfhlZJSdezJ1xySXi9axcsXx7CvTLob7sNKqLN0cMPr7oVn0jAIYc0+cfLWCaBnu73KfWau+cAi4AvAUcCfzOzee6+ucqb3IuBYoBEIqHr9opIva1bBzNnhhBfsCA0l/zHf8DPfgYjRkC7drUvo6AAjjsuPL4eHfnbvh0WL967FT9/Pjz22N739OlTNeT794e2bRvlI9ZZJoFeRtU7l3cjbIkn+zrwCw8XV19pZu8AfYF/ZaVKERFgyxb4y19Ck8qzz4YmlkQCbr8dLrsMDjus4eto0wZOOSU8Km3cGA6mVm7F//3v4YcEwtb/CSfsbaoZNAiOOSb8WDS1Wm9wYWatgDeBLwPvAfOBMdFNeCvnuQv4wN0nm9mhwEKgX003rE0kEq6rLYpIbSoq4G9/CyE+axZs3Qq9eoXmlKIi6Ns3N3W9997egJ8/Pzw2bQrTDjwQTjpp71b8oEGhqScb7fFmtsDdE+mm1bqF7u4VZnYN8DRQANzj7kvNbEI0fRrwU+A+M3uN0ERzY0PvPi4iLZd72CKePj00q3z4IXzuc3DllSHITzst9wcru3YNj4suCsO7d8PKlVWbau64Y2/vmk6d9ob7ueeG52zL6Hro7j4HmJMyblrS67XAf2S3NBFpad56KzRlTJ8O//43tG4NF14YQnz48NDdMF/tt1/o+njUUaFeCP3dX3utas+aJ58Mex2NEeg5u6eomlwa7pNP4Pe/h23bYPz4cERepLn56KPQxXD6dHjllbDlPWxYCMWLL4YOHXJdYXaVl8POnWGPoz4a1OQi+WfNmnAQqLg4HCQyg5//POyO/uAHYQtBJJ9t2xZO9pk+fe8W6/HHwy9/CaNHQ/futS+juWrfvvGW3czPi2pZli6FsWPhiCPgN78JXbMWL4Y334Rx4+BPfwoHiC6+OOzaieSTXbvguedC98BDDw29UhYuhOuuC/+OX3sNbrgh3mHe2NTkkufc4R//gF/9Cp54Ipww8c1vwvXXh6PmyT74AP73f2Hq1NDN6owz4MYbQ9tjrg8gSfa4h932jz6CDRvCc+qjcvzmzaFtt6AgdK9Lfm7I67q+b9kymDED1q4NZ2RefHFoUjnjjNx072vOampyUaDnqd27YfbsEOSvvBKOkH/nO3D11dCxY83vLS+Hu++GX/8ayspCH9kbbghbRPvv3zT1N7U33giBMWtWOBDVvn14tGu393Xyo7bxBx7YND+C7vDpp/sGcXUBXfnYuTP98goKwr+PTp3C46CDwr+lXbvCo6Ki6nN1rzOdnqlWreC880KIX3BB/pyI0xwp0JuRzz4L7Yq33gorVkDv3vD974fd1MLCui1rx47Q5etXvwrNNT16hC37b3wjs7Po8t2aNeHzzZgBixaFLdFhw0KQlZeHx5Yte1+Xl4e220yY7Q33uvwoVI5r2zb0SU4XzqkBnXzRqNQaksO5U6d9h1PHd+jQtBeY2r07sx+EQw6Bgw9uurriTIHeDGzaBL/7XTjYuW4dDBgQtqovuaTqdSjqY/dumDMnBPu8eeE/1zXXhEeXLlkpv8ls2AAPPxxCfO7cMO6UU2DMGPjP/6z9TMGKihDyqUFf3Q9ATeO3bKnbD8Qhh1QfxOnGH3ywmiNkXwr0PLZuXQjxadNCe+dZZ4UgP+usxtnlf+WVEOyzZoVTnMeNC3sARxyR/XVly6efhuanGTPgqadCKPftG84SHDUKvvCF3NVW+QORLugPPnhvQB9yiMJZskOBnodWrAjNKvffH0Lh0kvhhz8Mpws3heXLw1Xl/vSnsFt86aXhh2TgwKZZf2127IBnngkh/thj4XTvbt1Cl7YxY6BfPx3olZZJgZ5H/vnP0Nf2scfCWXDjxoV27SOPzE09a9eGLpBNtYdQk927Q4+eGTPgz3+Gjz8OJ19cemkI8dNPb/43IBBpKAV6ju3eHU6e+OUv87cNuzHb8GviHvogz5gBDzwQeuUUFoY+9mPGhMuh5vPp3iJNTYGeI6m9TLp3D+3V+dzLJJu9bGry1lshwGfMCF0OW7WCc84JIX7hhfn7/YjkWk2Brh3YRlBeDv/v/4VmlCuvDM0E998fQuy7383vsGrdOvzgLFsWrjt96KFhT6JnT7jlltDLpL7efx9++1s49dRwIPPmm8MBw7vuCnsFTzwRAj2fvx+RfKYt9CxKPVNz2LDQbNGcz9SsPFP1l7+Ev/615jNV09m0KfwwzJgRTvvevTsc0BwzJvRQ0Z0IRepGTS6NbOVK+J//gXvvDc0sI0eGHivJdzyJg9dfD00xM2aEoB81KvxgnXhi1fm2bw/hP2NGeP7ss9AtcvTo8DjuuNzULxIHDQ50MxsO/IZwg4u73f0XKdN/CBRFg62AY4DO7v5xdcuMQ6CXlob28UceCW3AY8eG9ua4X+1wzZrQpFRcHPqIDx8egr2iIoT4o4+GHjNduoTQHzMmXPu5ue6liOSTBgW6mRUQbkF3NuH+ovOB0e6+rJr5vwJc5+5fqmm5+Rro7qG73Pvvh3bd6p7XrQvNCR06wLe/Dddem537GTYnn3wCd94Zuj2uXx/GtW8fLrw0ZgyceWbj9pARaYkaej30QcBKd387WthMYASQNtCB0cAD9Sm0Me3YEdq4U0M5XWCnu/BRYWG4gcRhh4XrNp91Vjhb8fLLwwWQWqJDDoGJE0N7+p//HC5odd55uvCSSK5kEuhdgTVJw2VA2tZhMysEhgPXNLy02rmHXfuawrnyubreGZ077w3qY44Jz5XDyc/t2qnJoDpt28IVV+S6ChHJJNDTxVh17TRfAV6qru3czMYD4wF61KN7Q0lJ2Br88MOwK2+Wfmu6deu9YXzUUTB0aPqQ7tIlvpeTFZGWJ5NALwOS7yHSDVhbzbyjqKG5xd2LgWIIbegZ1giEMB8/PlzTA8IBuFatQlvt+efvDenDDw/t2tqaFpGWJpNAnw/0MbPewHuE0B6TOpOZdQDOAL6W1QojEyfuDfNKFRXw0ksh7EVEWrpaA93dK8zsGuBpQrfFe9x9qZlNiKZPi2b9KvCMu3/aGIW++27dxouItDQZdSpz9znAnJRx01KG7wPuy1ZhqXr0gNWr048XEZFmdC2XKVP2vThUYWEYLyIizSjQi4rCmYk9e4YDnj17huGiotrfKyLSEjSr8/iKihTgIiLVaTZb6CIiUjMFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYyCjQzWy4ma0ws5VmdlM18wwzs0VmttTMXsxumSIiUptaL59rZgXAVOBswg2j55vZbHdfljTPwcCdwHB3f9fMujRSvSIiUo1MttAHASvd/W133wHMBEakzDMGeNTd3wVw9w+zW6aIiNQmk0DvCqxJGi6LxiU7CjjEzF4wswVmdkW6BZnZeDMrNbPS9evX169iERFJK5NAtzTjPGW4FXAScD5wDnCzmR21z5vci9094e6Jzp0717lYERGpXia3oCsDuicNdwPWppnnI3f/FPjUzOYC/YA3s1KliIjUKpMt9PlAHzPrbWYHAKOA2SnzPAZ80cxamVkhcArwRnZLFRGRmtS6he7uFWZ2DfA0UADc4+5LzWxCNH2au79hZk8BS4DdwN3u/npjFi4iIlWZe2pzeNNIJBJeWlqak3WLiDRXZrbA3RPppulMURGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISExkFupkNN7MVZrbSzG5KM32YmW0ys0XR4yfZL1VERGpS6y3ozKwAmAqcTbgZ9Hwzm+3uy1JmnefuFzRCjSIikoFMttAHASvd/W133wHMBEY0blkiIlJXmQR6V2BN0nBZNC7VaWa22MyeNLPj0i3IzMabWamZla5fv74e5YqISHUyCXRLMy71ztILgZ7u3g/4X2BWugW5e7G7J9w90blz5zoVKiIiNcsk0MuA7knD3YC1yTO4+2Z33xK9ngPsb2adslaliIjUKpNAnw/0MbPeZnYAMAqYnTyDmR1mZha9HhQtd0O2i80XJSXQqxfst194LinJdUUiIhn0cnH3CjO7BngaKADucfelZjYhmj4NuAS4yswqgG3AKHdPbZaJhZISGD8etm4Nw6tXh2GAoqLc1SUiYrnK3UQi4aWlpTlZd0P06hVCPFXPnrBqVVNXIyItjZktcPdEumk6U7SO3n23buNFRJqKAr2OevSo23gRkaaiQK+jKVOgsLDquMLCMF5EJJcU6HVUVATFxaHN3Cw8FxfrgKiI5F6tvVxkX0VFCnARyT/aQhcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISExkFOhmNtzMVpjZSjO7qYb5TjazXWZ2SfZKFBGRTNQa6GZWAEwFzgWOBUab2bHVzPdLwp2NRESkiWWyhT4IWOnub7v7DmAmMCLNfN8BHgE+zGJ9IiKSoUwCvSuwJmm4LBq3h5l1Bb4KTKtpQWY23sxKzax0/fr1da1VRERqkEmgW5pxqTcivR240d131bQgdy9294S7Jzp37pxhiSIikolMrodeBnRPGu4GrE2ZJwHMNDOATsB5Zlbh7rOyUaSIiNQuk0CfD/Qxs97Ae8AoYEzyDO7eu/K1md0HPKEwFxFpWrUGurtXmNk1hN4rBcA97r7UzCZE02tsNxcRkaaR0S3o3H0OMCdlXNogd/exDS9LRETqSmeKiojEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYyCjQzWy4ma0ws5VmdlOa6SPMbImZLTKzUjM7PfuliohITWq9Y5GZFQBTgbMJN4yeb2az3X1Z0mzPAbPd3c3sROAhoG9jFCwiIullsoU+CFjp7m+7+w5gJjAieQZ33+LuHg0eCDgiItKkMgn0rsCapOGyaFwVZvZVM1sO/BUYl25BZjY+apIpXb9+fX3qFRGRamQS6JZm3D5b4O7+F3fvC1wE/DTdgty92N0T7p7o3LlznQoVEZGaZRLoZUD3pOFuwNrqZnb3ucCRZtapgbWJiEgdZBLo84E+ZtbbzA4ARgGzk2cwsy+YmUWvBwIHABuyXayIiFSv1l4u7l5hZtcATwMFwD3uvtTMJkTTpwEXA1eY2U5gG3BZ0kFSERFpApar3E0kEl5aWpqTdYuINFdmtsDdE+mm6UxREZGYUKCLiMSEAl1EJCYU6M1YSQn06gX77ReeS0pyXZGI5FKtvVwkP5WUwPjxsHVrGF69OgwDFBXlri4RyR1toTdTEyfuDfNKW7eG8SLSMinQm6l3363beBGJPwV6M9WjR93Gi0j8KdCbqSlToLCw6rjCwjBeRFomBXozVVQExcXQsyeYhefiYh0QFWnJ1MulGSsqUoCLyF7aQhcRiQkFuohITCjQRURiQoEuIhITGQW6mQ03sxVmttLMbkozvcjMlkSPl82sX/ZLFRGRmtQa6GZWAEwFzgWOBUab2bEps70DnOHuJxJuEF2c7UJFRKRmmWyhDwJWuvvb7r4DmAmMSJ7B3V9290+iwX8SbiQtIiJNKJNA7wqsSRoui8ZV5xvAkw0pSkRE6i6TE4sszbi0NyI1szMJgX56NdPHA+MBeuiiIyIiWZXJFnoZ0D1puBuwNnUmMzsRuBsY4e4b0i3I3YvdPeHuic6dO9enXhERqUYmgT4f6GNmvc3sAGAUMDt5BjPrATwKXO7ub2a/TBERqU2tTS7uXmFm1wBPAwXAPe6+1MwmRNOnAT8BOgJ3mhlAhbsnGq9sERFJZe5pm8MbXSKR8NLS0pysW0SkuTKzBdVtMOtMURGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUCXBispgV69YL/9wnNJSa4rEmmZMrljkUi1Skpg/HjYujUMr14dhgGKinJXl0hLpC10aZCJE/eGeaWtW8N4EWlaebWFvnPnTsrKyti+fXuuS5FatGnThm7duvHuu/unnf7uu01ckIhkFuhmNhz4DeGORXe7+y9SpvcF7gUGAhPd/bb6FFNWVkb79u3p1asX0Z2PJA+5Oxs2bKCsrIwePXqzevW+8+ge4CJNr9YmFzMrAKYC5wLHAqPN7NiU2T4GrgXqFeSVtm/fTseOHRXmec7M6NixI9u3b2fKFCgsrDq9sBCmTMlNbSItWSZt6IOAle7+trvvAGYCI5JncPcP3X0+sLOhBSnMm4fKv1NRERQXQ8+eYBaei4t1QFQkFzJpcukKrEkaLgNOqc/KzGw8MB6gh/bJY6OoSAEukg8y2UJPt8lcrztLu3uxuyfcPdG5c+f6LKKKbPd/3rBhA/3796d///4cdthhdO3adc/wjh07anxvaWkp1157ba3rGDx4cMOKjLzwwgtccMEFWVmWiMRDJlvoZUD3pOFuwNrGKSdzjdH/uWPHjixatAiAyZMn065dO37wgx/smV5RUUGrVum/skQiQSKR9kbcVbz88sv1K05EpBaZbKHPB/qYWW8zOwAYBcxu3LJq11T9n8eOHcv111/PmWeeyY033si//vUvBg8ezIABAxg8eDArVqwAqm4xT548mXHjxjFs2DCOOOIIfvvb3+5ZXrt27fbMP2zYMC655BL69u1LUVER7mHHZ86cOfTt25fTTz+da6+9ttYt8Y8//piLLrqIE088kVNPPZUlS5YA8OKLL+7ZwxgwYADl5eWsW7eOoUOH0r9/f44//njmzZuX3S9MRHKm1i10d68ws2uApwndFu9x96VmNiGaPs3MDgNKgYOA3Wb2PeBYd9/cWIVX18+5Mfo/v/nmmzz77LMUFBSwefNm5s6dS6tWrXj22Wf58Y9/zCOPPLLPe5YvX87f//53ysvLOfroo7nqqqvYf/+qfbZfffVVli5dyuc//3mGDBnCSy+9RCKR4Fvf+hZz586ld+/ejB49utb6Jk2axIABA5g1axbPP/88V1xxBYsWLeK2225j6tSpDBkyhC1bttCmTRuKi4s555xzmDhxIrt27WJr6q+iiDRbGfVDd/c5wJyUcdOSXr9PaIppMj160GT9ny+99FIKCgoA2LRpE1deeSX//ve/MTN27kzfsef888+ndevWtG7dmi5duvDBBx/QrVvVr2jQoEF7xvXv359Vq1bRrl07jjjiCHr37g3A6NGjKS4urrG+f/zjH3t+VL70pS+xYcMGNm3axJAhQ7j++uspKipi5MiRdOvWjZNPPplx48axc+dOLrroIvr379+Qr0ZE8kizPfW/Kfs/H3jggXte33zzzZx55pm8/vrrPP7449We1dq6des9rwsKCqioqMhonspml7pI9x4z46abbuLuu+9m27ZtnHrqqSxfvpyhQ4cyd+5cunbtyuWXX86f/vSnOq9PRPJTsw30XPV/3rRpE127dgXgvvvuy/ry+/bty9tvv82qVasAePDBB2t9z9ChQymJuvi88MILdOrUiYMOOoi33nqLE044gRtvvJFEIsHy5ctZvXo1Xbp04b/+67/4xje+wcKFC7P+GUQkN5ptoEMI71WrYPfu8NwUfaFvuOEGfvSjHzFkyBB27dqV9eW3bduWO++8k+HDh3P66adz6KGH0qFDhxrfM3nyZEpLSznxxBO56aab+OMf/wjA7bffzvHHH0+/fv1o27Yt5557Li+88MKeg6SPPPII3/3ud7P+GXJFl/GVls7qs4ufDYlEwktLS6uMe+ONNzjmmGNyUk8+2bJlC+3atcPdufrqq+nTpw/XXXddrsvaRz79vVK7sUJogtNZqxI3ZrbA3dP2kW7WW+hx9fvf/57+/ftz3HHHsWnTJr71rW/luqS8p8v4iuTZ5XMluO666/JyizyfNWU3VpF8pS10iYXquqvqkkHSkijQJRZ0GV8RBbrEhC7jK6JAlxjJRTfWdNR9UnJFgZ5k2LBhPP3001XG3X777Xz729+u8T2V3S/PO+88Nm7cuM88kydP5rbbar6Z06xZs1i2bNme4Z/85Cc8++yzdag+PV1mt2lVdp9cvRrc914FVKEuTUGBnmT06NHMnDmzyriZM2dmdIEsCFdJPPjgg+u17tRAv+WWWzjrrLPqtSzJHXWflFzK20D/3vdg2LDsPr73vZrXeckll/DEE0/w2WefAbBq1SrWrl3L6aefzlVXXUUikeC4445j0qRJad/fq1cvPvroIwCmTJnC0UcfzVlnnbXnErsQ+piffPLJ9OvXj4svvpitW7fy8ssvM3v2bH74wx/Sv39/3nrrLcaOHcvDDz8MwHPPPceAAQM44YQTGDdu3J76evXqxaRJkxg4cCAnnHACy5cvr/Hz6TK7jS9fuk+q2adlyttAz4WOHTsyaNAgnnrqKSBsnV922WWYGVOmTKG0tJQlS5bw4osv7gnDdBYsWMDMmTN59dVXefTRR5k/f/6eaSNHjmT+/PksXryYY445hj/84Q8MHjyYCy+8kFtvvZVFixZx5JFH7pl/+/btjB07lgcffJDXXnuNiooK7rrrrj3TO3XqxMKFC7nqqqtqbdapvMzukiVL+PnPf84VV1wBsOcyu4sWLWLevHm0bduWGTNmcM4557Bo0SIWL16sqzJmKB+6T+ZTs49+WJpW3p5YdPvtuVlvZbPLiBEjmDlzJvfccw8ADz30EMXFxVRUVLBu3TqWLVvGiSeemHYZ8+bN46tf/SqFUT+6Cy+8cM+0119/nf/+7/9m48aNbNmyhXPOOafGelasWEHv3r056qijALjyyiuZOnUq34t2N0aOHAnASSedxKOPPlrjsnSZ3cY3ZUr6SxA0ZffJmpp9mvJAcWPcVawhtUycGPaUevQIf4849oDSFnqKiy66iOeee46FCxeybds2Bg4cyDvvvMNtt93Gc889x5IlSzj//POrvWxuJbN0t2INd0C64447eO2115g0aVKty6ntWjuVl+Ct7hK9tS1Ll9nNrnzoPpkvzT75cjyhJe2xZBToZjbczFaY2UozuynNdDOz30bTl5jZwOyW2XTatWvHsGHDGDdu3J6DoZs3b+bAAw+kQ4cOfPDBBzz55JM1LmPo0KH85S9/Ydu2bZSXl/P444/vmVZeXs7hhx/Ozp0791zyFqB9+/aUl5fvs6y+ffuyatUqVq5cCcD999/PGWecUa/PpsvsNo1cd5/Mh2Yf0A9Lqqb4Yak10M2sAJgKnAscC4w2s2NTZjsX6BM9xgN30YyNHj2axYsXM2rUKAD69evHgAEDOO644xg3bhxDhgyp8f0DBw7ksssuo3///lx88cV88Ytf3DPtpz/9Kaeccgpnn302ffv23TN+1KhR3HrrrQwYMIC33nprz/g2bdpw7733cumll3LCCSew3377MWHChHp9Ll1mt2XIl7Nm9cNSVZP8sLh7jQ/gNODppOEfAT9Kmed3wOik4RXA4TUt96STTvJUy5Yt22ec5C/9vfLX9OnuPXu6m4Xn6dNzU0NhoXvYHg2PwsKmr6Vnz6o1VD569mzaOszS12FWt+UApV5NrmbS5NIVWJM0XBaNq+s8mNl4Mys1s9L169dnsGoRqY9cN/tU1pDr4wnQsvZYMgn0dEf3Uo+uZTIP7l7s7gl3T3Tu3DmT+kSkGdMPy15N8cOSSbfFMqB70nA3YG095smIu1fbQ0Tyh+foTlci9VFUlPtuipXrb8zuk5lsoc8H+phZbzM7ABgFzE6ZZzZwRdTb5VRgk7uvq2sxbdq0YcOGDQqLPOfubNiwgTZt2uS6FJFmpbH3WGrdQnf3CjO7BngaKADucfelZjYhmj4NmAOcB6wEtgJfr08x3bp1o6ysDLWv5782bdrQrVu3XJchIkny6ibRIiJSM90kWkSkBVCgi4jEhAJdRCQmctaGbmbrgdU5WXn2dAI+ynUReUTfR1X6PvbSd1FVQ76Pnu6e9kSenAV6HJhZaXUHJ1oifR9V6fvYS99FVY31fajJRUQkJhToIiIxoUBvmOJcF5Bn9H1Upe9jL30XVTXK96E2dBGRmNAWuohITCjQRURiQoFeD2bW3cz+bmZvmNlSM2vx92czswIze9XMnsh1LblmZgeb2cNmtjz6N3JarmvKJTO7Lvp/8rqZPWBmLeoynWZ2j5l9aGavJ437nJn9zcz+HT0fko11KdDrpwL4vrsfA5wKXJ3mPqstzXeBN3JdRJ74DfCUu/cF+tGCvxcz6wpcCyTc/XjCFVtH5baqJncfMDxl3E3Ac+7eB3guGm4wBXo9uPs6d18YvS4n/Ifd55Z7LYWZdQPOB+7OdS25ZmYHAUOBPwC4+w5335jTonKvFdDWzFoBhdTz5jfNlbvPBT5OGT0C+GP0+o/ARdlYlwK9gcysFzAA+L8cl5JLtwM3ALtzXEc+OAJYD9wbNUHdbWYH5rqoXHH394DbgHeBdYSb3zyT26rywqGVNwGKnrtkY6EK9AYws3bAI8D33H1zruvJBTO7APjQ3RfkupY80QoYCNzl7gOAT8nS7nRzFLUNjwB6A58HDjSzr+W2qvhSoNeTme1PCPMSd3801/Xk0BDgQjNbBcwEvmRm03NbUk6VAWXuXrnH9jAh4Fuqs4B33H29u+8EHgUG57imfPCBmR0OED1/mI2FKtDrwcJdrP8AvOHuv851Pbnk7j9y927u3otwsOt5d2+xW2Du/j6wxsyOjkZ9GViWw5Jy7V3gVDMrjP7ffJkWfJA4yWzgyuj1lcBj2VhorfcUlbSGAJcDr5nZomjcj919Tu5KkjzyHaAkuqn629TzHrtx4O7/Z2YPAwsJvcNepYVdBsDMHgCGAZ3MrAyYBPwCeMjMvkH40bs0K+vSqf8iIvGgJhcRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYuL/A5qjkD5jeISTAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GTd33Z2AXrAy"
      },
      "source": [
        "\n",
        "Validation accuracy stalls in the low 50s. So in our case, pre-trained word embeddings does outperform jointly learned embeddings. If you \n",
        "increase the number of training samples, this will quickly stop being the case -- try it as an exercise.\n",
        "\n",
        "Finally, let's evaluate the model on the test data. First, we will need to tokenize the test data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UpVevFtWXrAy",
        "outputId": "94e7fac5-31a9-478c-85da-a74656e0a8eb"
      },
      "source": [
        "test_dir = os.path.join(imdb_dir, 'test')\n",
        "\n",
        "labels = []\n",
        "texts = []\n",
        "\n",
        "for label_type in ['neg', 'pos']:\n",
        "    dir_name = os.path.join(test_dir, label_type)\n",
        "    for fname in sorted(os.listdir(dir_name)):\n",
        "        if fname[-4:] == '.txt':\n",
        "            f = open(os.path.join(dir_name, fname))\n",
        "            texts.append(f.read())\n",
        "            f.close()\n",
        "            if label_type == 'neg':\n",
        "                labels.append(0)\n",
        "            else:\n",
        "                labels.append(1)\n",
        "\n",
        "sequences = tokenizer.texts_to_sequences(texts)\n",
        "x_test = pad_sequences(sequences, maxlen=maxlen)\n",
        "y_test = np.asarray(labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "UnicodeDecodeError",
          "evalue": "'cp949' codec can't decode byte 0xe2 in position 32: illegal multibyte sequence",
          "traceback": [
            "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[1;31mUnicodeDecodeError\u001b[0m                        Traceback (most recent call last)",
            "\u001b[1;32m<ipython-input-48-d5f93f97acef>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      9\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mfname\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m4\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'.txt'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     10\u001b[0m             \u001b[0mf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdir_name\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 11\u001b[1;33m             \u001b[0mtexts\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mread\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     12\u001b[0m             \u001b[0mf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     13\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mlabel_type\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'neg'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
            "\u001b[1;31mUnicodeDecodeError\u001b[0m: 'cp949' codec can't decode byte 0xe2 in position 32: illegal multibyte sequence"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VFSEYNuMXrAy"
      },
      "source": [
        "And let's load and evaluate the first model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "udMAfTe-XrAy",
        "outputId": "9155a16b-38c4-4d97-9232-ebf9738d798a"
      },
      "source": [
        "model.load_weights('pre_trained_glove_model.h5')\n",
        "model.evaluate(x_test, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\r25000/1 [================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 2s 61us/sample - loss: 0.9205 - acc: 0.5361\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.8344906200027465, 0.53608]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WjdQLofjXrAz"
      },
      "source": [
        "We get an appalling test accuracy of 54%. Working with just a handful of training samples is hard!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "32ie8wnsrFxV"
      },
      "source": [
        "- 왜? 이렇게 성능이 안좋으냐면 샘플의 사이즈가 작기 때문에 성능이 좋지 않다는 것이 결론이다. "
      ]
    }
  ]
}