static_representation.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/nicolashernandez/teaching_nlp/blob/main/static_representation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Foreword\n",
        "\n",
        "ne pas consulter les appendix. Elles contiennent des éléments de réponse."
      ],
      "metadata": {
        "id": "W1YngDYXt_wM"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VEzct6qpjorn"
      },
      "source": [
        "# Introduction\n",
        "\n",
        "L'objectif de ce notebook est de vous faire manipuler, observer et construire différentes modélisations de données statiques clairesemées et denses sur une tâche de classification : à savoir une tâche de classification binaire en sentiment (positif/négatif).\n",
        "\n",
        "Le notebook est découpé en activités de 1 à 6. Les deux premières activités conduisent à observer les performances d'un classifieur entraîné sur des données clairsemées. L'activité 3 est un travail d'observation de modèles dense. Les activités 4 à 6 sont destinés à l'observation des performances d'un classifieur entraîné sur des données denses.\n",
        "\n",
        "Plusieurs modélisations de données seront explorées\n",
        "1. term frequency model\n",
        "2. TF-IDF\n",
        "3. pretrained word2vec, glove, FastText models\n",
        "4. word2vec and FastText models trained from scratch on the target domain\n",
        "5. doc2vec model trained from scratch on the target domain\n",
        "\n",
        "Il vous est demandé de réaliser les tâches précédées par l'étiquette TODO. Quand toutes celles-ci auront été faites, s'il reste du temps vous pourrez faire les tâches TODO_FURTHER.  \n",
        "\n",
        "La construction de certains modèles peut prendre du temps. Afin de  \n",
        "- jouer avec les algorithmes de construction d'embeddings\n",
        "- jouer avec les hyperparamètres des modèles (dimensions, ...)\n",
        "- jouer avec les types de vectorisation\n",
        "- jouer avec les prétraitements\n",
        "\n",
        "je vous propose de travailler en petits groupes de 2-3 et de vous répartir des configurations à expérimenter.\n",
        "\n",
        "On utilisera la base de données IMDB qui vient avec 3 partitions : train, test et unsupervised. La construction des modèles de données utilisera le jeu unsupervised. La construction des modèles de classification utilisera le jeu train tandis que l'évaluation se fera sur le jeu de test.\n",
        "\n",
        "Pour les approches  de la vectorization 1 et 2, qui ne manipulent pas des vecteurs denses, vous pouvez travailler avec les jeux de données train et test au complet. Pour les autres approches, si les délais dépassent le quart d'heures alors travaillez sur l'échantillon (sample) de données de train et de test construit à partir du train original.\n",
        "\n",
        "L'ordre de travail suggéré est le suivant :\n",
        "1. Activité 1\n",
        "2. Activité 2\n",
        "3. Mener sur deux notebooks distincts d'un côté les activités 4, 5 et 6 selon les configurations de votre répartition et de l'autre l'activité 3 qui vous permet de patienter quand la machine calcule...\n",
        "\n",
        "Ci-dessous un tableau non exhaustif des expériences à réaliser.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##  Experimental planning\n",
        "\n",
        "TODO remplir la colonne *Acc*. Editez au besoin les autres colonnes et détaillez éventuellement les approches en plusieurs lignes.\n",
        "\n",
        "TODO En fin de séance, réaliser une synthèse comparative des différentes approches et configuration. Cherchez notamment à observer et à expliquer les différences\n",
        "* performance of sparse models\n",
        "* sparse models vs dense models performance wrt number of dimension\n",
        "* pretrained w2v-like model vs trained on domain data w2v-like models\n",
        "* number of dimensions and performance for dense models\n",
        "* difference of performance between w2v, glove, fastText and doc2vec models\n",
        "* rôle of stemming preprocessing\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "| Model    | Pretrain           |  #Dim | Hyperparam | Train/Predict | Acc.  |\n",
        "|----------|--------------------|-------|------------|---------------|-------|\n",
        "| tf       | imdb-unsupervised  |    50 |                 | all    |       |\n",
        "| tf       | imdb-unsupervised  |   100 |                 | all    |       |\n",
        "| tf       | imdb-unsupervised  |   500 |                 | all    |       |\n",
        "| tf       | imdb-unsupervised  |  1000 |                 | all    |       |\n",
        "| tf       | imdb-unsupervised  |  5000 |                 | all    |       |\n",
        "| tf       | imdb-unsupervised  | 10000 |                 | all    |       |\n",
        "| tf       | imdb-unsupervised  | best  | stem            | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  |    50 |                 | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  |   100 |                 | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  |   500 |                 | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  |  1000 |                 | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  |  5000 |                 | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  | 10000 |                 | all    |       |\n",
        "| tf-idf   | imdb-unsupervised  | best  | stem            | all    |       |\n",
        "|----------|--------------------|-------|-----------------|--------|-------|\n",
        "| glove    | twitter            |    25 |                 | sample |       |\n",
        "| glove    | twitter            |    25 |                 | sample |       |\n",
        "| glove    | twitter            |    50 |                 | sample |       |\n",
        "| glove    | twitter            |   100 |                 | sample |       |\n",
        "| fasttext | wiki-news-subwords |   300 |                 | sample |       |\n",
        "| word2vec | google-news        |   300 |                 | sample |       |\n",
        "|----------|--------------------|-------|-----------------|--------|-------|\n",
        "| word2vec | imdb-unsupervised  |   100 |                 | sample |       |\n",
        "| word2vec | imdb-unsupervised  |   100 |ws epoch min_count negative| sample |       |\n",
        "| word2vec | imdb-unsupervised  |   300 |best             | sample |       |\n",
        "| word2vec | imdb-unsupervised  |   500 |best             | sample |       |\n",
        "| fasttext | imdb-unsupervised  |   100 |                 | sample |       |\n",
        "| fasttext | imdb-unsupervised  |   100 |ws epoch min_count negative| sample |       |\n",
        "| fasttext | imdb-unsupervised  |   300 |best             | sample |       |\n",
        "| fasttext | imdb-unsupervised  |   500 |best             | sample |       |\n",
        "|----------|--------------------|-------|-----------------|--------|-------|\n",
        "| doc2vec  | imdb-unsupervised  |   100 |                 | sample |       |\n",
        "| doc2vec  | imdb-unsupervised  |   300 |ws epoch min_count negative             | sample |       |\n",
        "| doc2vec  | imdb-unsupervised  |   500 |best             | sample |       |\n",
        "\n"
      ],
      "metadata": {
        "id": "D1yorOPtpohG"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pXBCWTTCjr_o"
      },
      "source": [
        "# A bit of theory"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmNr9PDicuOn"
      },
      "source": [
        "## Modèle \"sac de mots\" clairsemé\n",
        "\n",
        "Le [**sac de mots** ou *BOW* pour *Bag Of Words* en anglais](https://fr.wikipedia.org/wiki/Sac_de_mots) est un modèle classique utilisé en Recherche d'Information (RI) pour représenter le contenu d'un document. Chaque document est décrit vis-à-vis d'un vocabulaire donné commun.\n",
        "\n",
        "Différentes vues sont possibles :\n",
        "- compter binairement si les mots du vocabulaire sont présents dans le document,\n",
        "- compter les occurrences des mots du vocabulaire dans le document,\n",
        "- pondérer les mots en tenant compte de leurs spécificités dans le document vis-à-vis des autres documents (on parle de pondération _tf-idf_).\n",
        "\n",
        "La **vectorisation** est le processus qui désigne transformation des textes en vecteurs (de mots selon une modélisation *bow*).\n",
        "\n",
        "On doit à [Karen Spärck Jones](https://fr.wikipedia.org/wiki/Karen_Sp%C3%A4rck_Jones) la proposition de la pondération _tf-idf_ des termes.\n",
        "> « La spécificité d'un terme peut être quantifiée comme une fonction inverse du nombre de documents dans lesquels il apparaît. »\n",
        "\n",
        "[Gérard Salton](https://fr.wikipedia.org/wiki/Gerard_Salton), quant à lui, est reconnu comme étant le père de la recherche d'information en ayant proposé une modélisation des documents dans un espace vectoriel.\n",
        "\n",
        "\n",
        "* Karen Spärck Jones, « A statistical interpretation of term specificity and its application in retrieval », Journal of Documentation, vol. 28, no 1,‎ 1972, p. 11–21 (DOI 10.1108/eb026526)\n",
        "* G. Salton , A. Wong , C. S. Yang, A vector space model for automatic indexing, Communications of the ACM, v.18 n.11, p. 613-620, novembre 1975\n",
        "\n",
        "Par la suite nous utiliserons le module python sklearn qui offre des facilités pour pré-traiter (normaliser) et vectoriser aisément les textes.\n",
        "\n",
        "On utilisera néanmoins un pré-traitement extérieur pour comparer différentes vectorisations concurrentes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rKxD0JXTjZvM"
      },
      "source": [
        "## Modèle dense statique\n",
        "\n",
        "Les [**plongements de mots** (_word embeddings_ en anglais)](https://fr.wikipedia.org/wiki/Word_embedding) désignent le résultat de techniques récentes de vectorisation qui produisent des vecteurs denses de dimensions réduites, prédéfinies, et non corrélées avec la taille du vocabulaire (e.g. 100, 300, 500...).\n",
        "Ces techniques reposent sur l'hypothèse distributionnelle de Harris qui veut que les mots apparaissant dans des contextes similaires ont des significations apparentées.\n",
        "\n",
        "La méthode de référence est connue sous le nom [**Word2Vec** est attribuée à Mikolov et ses collègues (Google) en 2013](https://github.com/tmikolov/word2vec).\n",
        "> Tomas Mikolov, Kai Chen, Greg Corrado et Jeffrey Dean, « Efficient Estimation of Word Representations in Vector Space », arXiv:1301.3781 [cs],‎ 16 janvier 2013\n",
        "\n",
        "Les auteurs proposent deux architectures neuronales à 2 couches [CBOW (_continuous bag of words_) et SkipGram](https://fr.wikipedia.org/wiki/Word2vec#/media/Fichier:CBOW_eta_Skipgram.png).\n",
        "\n",
        "> Le CBOW vise à prédire un mot étant donné son contexte. Skip-gram a une architecture symétrique visant à prédire les mots du contexte étant donné un mot en entrée. En pratique, le modèle CBOW est plus rapide à apprendre, mais le modèle skip-gram donne généralement de meilleurs résultats.\n",
        "\n",
        "> La couche cachée contient quelques centraines de neurones et constitue à l'issue de l'entraînement le plongement représentant un mot. La couche de sortie permet d'implémenter une tâche de classification au moyen d'une softmax.\n",
        "\n",
        "> L'apprentissage ne nécessite néanmoins aucun label, la vérité terrain étant directement déduite des données et plus particulièrement de la proximité des mots au sein du corpus d'entraînement. En ce sens, l'apprentissage de Word2vec constitue un apprentissage auto-supervisé.\n",
        "\n",
        "Les vecteurs obtenus sont dits statiques (ou non contextuels) car ils restent tel quel quelle que soit l'occurrence du mot en contexte.\n",
        "\n",
        "Un second modèle bien connu est celui de [**FastText** (Facebook)](https://github.com/facebookresearch/fastText) qui propose de traiter la variabilité morphologique des mots en construisant des vecteurs non pas pour des mots mais pour des sous-mots (séquence de caractères). Le lecteur d'un mot est la somme de tous les vecteurs des sous-mots le composant.\n",
        "\n",
        "Cette approche est indépendante de la langue, et montre de meilleurs résultats que word2vec sur des tâches syntaxiques, surtout quand le corpus d'entraînement est petit. Word2vec est légèrement meilleur pour des tâches sémantiques. Un des avantage de FastText est de pouvoir fournir des vecteurs mêmes pour les mots hors vocabulaires.\n",
        "\n",
        "Plusieurs **librairies permettent de créer, charger, sauver et manipuler des modèles de plongements de mots**. Nous allons utiliser _gensim_ qui permet aussi bien de travailler avec des modèles [word2vec](https://radimrehurek.com/gensim/auto_examples/tutorials/run_word2vec.html#sphx-glr-auto-examples-tutorials-run-word2vec-py) que [fasttext](https://radimrehurek.com/gensim/auto_examples/tutorials/run_fasttext.html#sphx-glr-auto-examples-tutorials-run-fasttext-py).  \n",
        "\n",
        "Ici une [Comparison of FastText and Word2Vec](https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/Word2Vec_FastText_Comparison.ipynb)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2NL-4rAEsmOz"
      },
      "source": [
        "# Environment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "-MoKnfAdsoq2"
      },
      "outputs": [],
      "source": [
        "from tqdm import tqdm\n",
        "import pandas as pd\n",
        "pd.options.display.max_colwidth = 200\n",
        "import numpy as np\n",
        "import re\n",
        "import json\n",
        "import sys"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-eiThtk3ieMI"
      },
      "source": [
        "# Dataset: IMDB\n",
        "\n",
        "For our experiment, we use the [Large Movie Review Dataset](https://huggingface.co/datasets/stanfordnlp/imdb) from [Learning Word Vectors for Sentiment Analysis](https://aclanthology.org/P11-1015) (Maas et al., ACL 2011).\n",
        "\n",
        "The dataset was developed for binary sentiment classification task.\n",
        "\n",
        "It contains\n",
        "* a set of 25,000 highly polar movie reviews for training,\n",
        "* 25,000 for testing\n",
        "* and additional 50,000 unlabeled data for use as well."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S7wjXSENuddb"
      },
      "source": [
        "## Load and preprocess the raw dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eMokGsZUhJrF"
      },
      "source": [
        "**load the dataset online**\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000,
          "referenced_widgets": [
            "d77a502cab7245e7844b6eb512ec2bac",
            "95c82cf664894953b68dcfcb59570c2d",
            "e42d26fd80a64b56bcbebb177e10623e",
            "83416178bb2a4e45a84f54f32dc40298",
            "c6dcbccceaa04a28a3ce6b46eb1b7ef2",
            "f21c313672ee430c9d1676ca50ebf5c7",
            "49f93406f14341e487837b96d1c1297c",
            "6c43aae432784b74bfbfab624269241d",
            "85e761a53a2d40489289d677cec60ffb",
            "6f30f1734c5f4065af5286e0c70f5157",
            "552e76fa353c4fe8b7a73688dd0fde10",
            "91016fa3b3fd44d4ae24fd9b17e748ed",
            "78ca7fc76e6b4f9aacc20b7a6f29dc34",
            "7cfae29822b6432cba222f0979b6533e",
            "fe25d3d63912413687e52550ba096aa8",
            "6cddfe098c4a4f90bbe9aa717807aa18",
            "136179a5945446c79a499bd0a4adfdf5",
            "a0b199fc37144f97b1ba4922bd3534e2",
            "fb008639ccab46e4af58934cb9c28f99",
            "9268c965e47840f0bf919bfb2c8e1162",
            "a0897276a6df44aeabc550bac4858c25",
            "660a18580d9742b7a32e35aff84607de",
            "d5ede4e9468149d6af8da57ad0cc3cd4",
            "9703145cb59a4f7f91b39aba192c81e5",
            "b388061acbef42a188465e825c07d4d0",
            "9ca07343a6dd41459227f83e1d7a4738",
            "1cfbe1e9b07543a695fa1fc07806973d",
            "3ee6d7ad91f14a419a0f3fb6bcfdb7b4",
            "5f387440d57f47068b6bb7df8c38109c",
            "e6cab835fc85448599677208bc24c2d1",
            "f7d119f4aabb49e8b24bc9c3ec72981c",
            "0222832ed417478496546330becdbddf",
            "ea57bfea55a248c9a2c1d3d59635a4d0",
            "9cb58b762aee44b89e71fe2b241c4b2e",
            "54c16b32ff5941a4a78f7afb2f1d8671",
            "c2a92a65f47d43dab28326bfce0aa543",
            "06fca36f17694e19b031e6b528a353ac",
            "de886875d455478a80bfa42ee8b295bd",
            "b9bdd5aec6c84eea8d5b3790604c21d7",
            "e1b167bbf59b44bfa757993a01f626d2",
            "24f5b462e68a40089f7b4618367859b5",
            "ba54d6c3fcb74d35bba73f9759b98a72",
            "e904d3e6eb4b42cd9f9e9b1179b21685",
            "6dca763ecba0497cb993a73c163d7de4",
            "8ca06d0929ff458abf0ba3b234e5ff0d",
            "15c830b03a724d399bc9b05e45fce7f7",
            "a9ad58e893a94c5f92c557a7e071be07",
            "4b1d5de8fdeb4516bbbcc3b6416f8ff7",
            "e7301dd2f8e548be978564930152cee4",
            "6eb0f32fb619407d832c3277ef3b5244",
            "f140d271f1c646e8a19b06f300ec4e98",
            "96c1f79d82af452e8975d3dfc5f19a75",
            "7ccbfb90a0c7456585bc69990295c1ba",
            "4d51f72c64c14974859d8d99ab0a4c60",
            "1145a4fb1e7346a6bd8ac66edaa6a04d",
            "61530322251744e789553a585cf4da51",
            "a0d30f77b77d47adae0d8289266cdea9",
            "cf2bacc80176433aa2cbc0f6409f49b5",
            "507b2e8f01d3405eb61db0448e9d44fb",
            "0a8e6e2e2f2a4fd9ba526c823648f9c2",
            "2947ed1b13334895b68bcf79a7ee9fce",
            "6adef6eca2e746aca4cec0a56c6cba26",
            "c8f9a0c747cf456e82d4ee354336b723",
            "66097bbb677d4cbcab24decb8b257657",
            "995227da41444e8e9c6adcca3b7859f9",
            "63c3c29aba704df8aded6873c5aa4a4f",
            "8bd32eb2ce4b462887d11c57dc50a9cd",
            "c97b98daf84748c2820d12fca3f4a8c4",
            "37b1d9114d834467898053070770a536",
            "d13b49a7e45f430e9ddedda32f8421dc",
            "61f65dc2a82b4f6685c20b27931de74a",
            "e34f3b756a994c699450b089f07062e2",
            "7dcff6e6fb3a4750a54f60e56d6ec0d6",
            "a81fe2e090424809ab65c6b015490ccb",
            "abfaaf18f2c54ed0a7c8c666d0526f5c",
            "38492cac60ef45a4ba20416a2ad975c1",
            "eaff87e842b5459487c602eedda5b550"
          ]
        },
        "id": "6yWRx2VViHCM",
        "outputId": "b72aeadf-1749-441d-cc2c-7e796f3715b4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting datasets\n",
            "  Downloading datasets-3.1.0-py3-none-any.whl.metadata (20 kB)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets) (3.16.1)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.26.4)\n",
            "Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (17.0.0)\n",
            "Collecting dill<0.3.9,>=0.3.0 (from datasets)\n",
            "  Downloading dill-0.3.8-py3-none-any.whl.metadata (10 kB)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (2.2.2)\n",
            "Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.32.3)\n",
            "Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.6)\n",
            "Collecting xxhash (from datasets)\n",
            "  Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
            "Collecting multiprocess<0.70.17 (from datasets)\n",
            "  Downloading multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)\n",
            "Collecting fsspec<=2024.9.0,>=2023.1.0 (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets)\n",
            "  Downloading fsspec-2024.9.0-py3-none-any.whl.metadata (11 kB)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.10.10)\n",
            "Requirement already satisfied: huggingface-hub>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.26.2)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (24.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.2)\n",
            "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (2.4.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (24.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.5.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.1.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.17.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.23.0->datasets) (4.12.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.4.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2.2.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2024.8.30)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.2)\n",
            "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n",
            "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from yarl<2.0,>=1.12.0->aiohttp->datasets) (0.2.0)\n",
            "Downloading datasets-3.1.0-py3-none-any.whl (480 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m480.6/480.6 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading dill-0.3.8-py3-none-any.whl (116 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading fsspec-2024.9.0-py3-none-any.whl (179 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m179.3/179.3 kB\u001b[0m \u001b[31m2.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading multiprocess-0.70.16-py310-none-any.whl (134 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: xxhash, fsspec, dill, multiprocess, datasets\n",
            "  Attempting uninstall: fsspec\n",
            "    Found existing installation: fsspec 2024.10.0\n",
            "    Uninstalling fsspec-2024.10.0:\n",
            "      Successfully uninstalled fsspec-2024.10.0\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "gcsfs 2024.10.0 requires fsspec==2024.10.0, but you have fsspec 2024.9.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed datasets-3.1.0 dill-0.3.8 fsspec-2024.9.0 multiprocess-0.70.16 xxhash-3.5.0\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "README.md:   0%|          | 0.00/7.81k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d77a502cab7245e7844b6eb512ec2bac"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "train-00000-of-00001.parquet:   0%|          | 0.00/21.0M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "91016fa3b3fd44d4ae24fd9b17e748ed"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "test-00000-of-00001.parquet:   0%|          | 0.00/20.5M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d5ede4e9468149d6af8da57ad0cc3cd4"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "unsupervised-00000-of-00001.parquet:   0%|          | 0.00/42.0M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9cb58b762aee44b89e71fe2b241c4b2e"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating train split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8ca06d0929ff458abf0ba3b234e5ff0d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating test split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "61530322251744e789553a585cf4da51"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating unsupervised split:   0%|          | 0/50000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8bd32eb2ce4b462887d11c57dc50a9cd"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['text', 'label'],\n",
              "        num_rows: 25000\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['text', 'label'],\n",
              "        num_rows: 25000\n",
              "    })\n",
              "    unsupervised: Dataset({\n",
              "        features: ['text', 'label'],\n",
              "        num_rows: 50000\n",
              "    })\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "!pip install datasets\n",
        "\n",
        "from datasets import load_dataset\n",
        "\n",
        "imdb_dataset = load_dataset(\"stanfordnlp/imdb\")\n",
        "imdb_dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**pandas dataframe creation for each splits** (train, test and unsupervised)"
      ],
      "metadata": {
        "id": "nlIgmjnaOHXk"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "I2kH4e5Hp4Ku",
        "outputId": "88cc42b0-bb9e-4d9c-bcf8-d32fbdba9525"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                                                                                                                                                                          text  \\\n",
              "0      I rented I AM CURIOUS-YELLOW from my video store because of all the controversy that surrounded it when it was first released in 1967. I also heard that at first it was seized by U.S. customs if i...   \n",
              "1      \"I Am Curious: Yellow\" is a risible and pretentious steaming pile. It doesn't matter what one's political views are because this film can hardly be taken seriously on any level. As for the claim t...   \n",
              "2      If only to avoid making this type of film in the future. This film is interesting as an experiment but tells no cogent story.<br /><br />One might feel virtuous for sitting thru it because it touc...   \n",
              "3      This film was probably inspired by Godard's Masculin, féminin and I urge you to see that film instead.<br /><br />The film has two strong elements and those are, (1) the realistic acting (2) the i...   \n",
              "4      Oh, brother...after hearing about this ridiculous film for umpteen years all I can think of is that old Peggy Lee song..<br /><br />\"Is that all there is??\" ...I was just an early teen when this s...   \n",
              "...                                                                                                                                                                                                        ...   \n",
              "24995  A hit at the time but now better categorised as an Australian cult film. The humour is broad, unsubtle and, in the final scene where a BBC studio fire is extinguished by urinating on it, crude. Co...   \n",
              "24996  I love this movie like no other. Another time I will try to explain its virtues to the uninitiated, but for the moment let me quote a few of pieces the remarkable dialogue, which, please remember,...   \n",
              "24997  This film and it's sequel Barry Mckenzie holds his own, are the two greatest comedies to ever be produced. A great story a young Aussie bloke travels to england to claim his inheritance and meets ...   \n",
              "24998  'The Adventures Of Barry McKenzie' started life as a satirical comic strip in 'Private Eye', written by Barry Humphries and based on an idea by Peter Cook. McKenzie ( 'Bazza' to his friends ) is a...   \n",
              "24999  The story centers around Barry McKenzie who must go to England if he wishes to claim his inheritance. Being about the grossest Aussie shearer ever to set foot outside this great Nation of ours the...   \n",
              "\n",
              "       label  \n",
              "0          0  \n",
              "1          0  \n",
              "2          0  \n",
              "3          0  \n",
              "4          0  \n",
              "...      ...  \n",
              "24995      1  \n",
              "24996      1  \n",
              "24997      1  \n",
              "24998      1  \n",
              "24999      1  \n",
              "\n",
              "[25000 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-60ba2e68-d4d1-4dc5-97b2-054968122d7f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>I rented I AM CURIOUS-YELLOW from my video store because of all the controversy that surrounded it when it was first released in 1967. I also heard that at first it was seized by U.S. customs if i...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>\"I Am Curious: Yellow\" is a risible and pretentious steaming pile. It doesn't matter what one's political views are because this film can hardly be taken seriously on any level. As for the claim t...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>If only to avoid making this type of film in the future. This film is interesting as an experiment but tells no cogent story.&lt;br /&gt;&lt;br /&gt;One might feel virtuous for sitting thru it because it touc...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>This film was probably inspired by Godard's Masculin, féminin and I urge you to see that film instead.&lt;br /&gt;&lt;br /&gt;The film has two strong elements and those are, (1) the realistic acting (2) the i...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Oh, brother...after hearing about this ridiculous film for umpteen years all I can think of is that old Peggy Lee song..&lt;br /&gt;&lt;br /&gt;\"Is that all there is??\" ...I was just an early teen when this s...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24995</th>\n",
              "      <td>A hit at the time but now better categorised as an Australian cult film. The humour is broad, unsubtle and, in the final scene where a BBC studio fire is extinguished by urinating on it, crude. Co...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24996</th>\n",
              "      <td>I love this movie like no other. Another time I will try to explain its virtues to the uninitiated, but for the moment let me quote a few of pieces the remarkable dialogue, which, please remember,...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24997</th>\n",
              "      <td>This film and it's sequel Barry Mckenzie holds his own, are the two greatest comedies to ever be produced. A great story a young Aussie bloke travels to england to claim his inheritance and meets ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24998</th>\n",
              "      <td>'The Adventures Of Barry McKenzie' started life as a satirical comic strip in 'Private Eye', written by Barry Humphries and based on an idea by Peter Cook. McKenzie ( 'Bazza' to his friends ) is a...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24999</th>\n",
              "      <td>The story centers around Barry McKenzie who must go to England if he wishes to claim his inheritance. Being about the grossest Aussie shearer ever to set foot outside this great Nation of ours the...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>25000 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-60ba2e68-d4d1-4dc5-97b2-054968122d7f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-60ba2e68-d4d1-4dc5-97b2-054968122d7f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-60ba2e68-d4d1-4dc5-97b2-054968122d7f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-d87ff3f1-0e22-480d-92f2-74527532ebad\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d87ff3f1-0e22-480d-92f2-74527532ebad')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-d87ff3f1-0e22-480d-92f2-74527532ebad button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_bcbb1c11-61d6-42c9-8c2a-23bbbcc2861f\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('train_df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_bcbb1c11-61d6-42c9-8c2a-23bbbcc2861f button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('train_df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "train_df",
              "summary": "{\n  \"name\": \"train_df\",\n  \"rows\": 25000,\n  \"fields\": [\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 24904,\n        \"samples\": [\n          \"Although I didn't like Stanley & Iris tremendously as a film, I did admire the acting. Jane Fonda and Robert De Niro are great in this movie. I haven't always been a fan of Fonda's work but here she is delicate and strong at the same time. De Niro has the ability to make every role he portrays into acting gold. He gives a great performance in this film and there is a great scene where he has to take his father to a home for elderly people because he can't care for him anymore that will break your heart. I wouldn't really recommend this film as a great cinematic entertainment, but I will say you won't see much bette acting anywhere.\",\n          \"Yet again one of the most misunderstood Goddesses of my country has been twisted by \\\"Westerners\\\" who cannot understand the esoteric symbolism of the Mother Goddess in her dark forms. The Mother takes on the frightening form of Kali Mata to destroy our inner demons, and to terrify our egos. And though blood sacrifice is given to Kali and Durga, the events depicted in this film are just absurd. The Mother takes on a wrathful form to be wrathful to our inner demons, limitations, and ego when no other form will suffice. It's also in her wrathful form that she burns away all your Karmas in the \\\"Smashan\\\" fires that you cultivate in your heart for her to dance on if you love her, and she will bring you to reality and truth. Reality and truth has a dark side as well as light, which serves a purpose. The Mother is the embodiment of the physical Universe as well, she is Nature. Nature can be cruel and destructive to maintain balance. You cannot have growth and life without death and destruction. Kali represents the force of destruction for the purpose of new growth and life both mundane and spiritual in the universe. It's very outrageous to me that people who know nothing of India or it's divinity can just take one of our beloved Goddesses and use her like a cheap prostitute to make some low-budget, talentless horror film. How dare they take our beloved Mother and portray her as a horror that makes people chop their eyelids off!? She is only horrific to those who are attached to their ego and who live in delusion , greed, anger, and other inner-demons. It's very clear to me that the person who wrote this movie must have a very serious self-deluding ego, and serious inner-demons to see Kali as so horrible and terrible. When the ego drops away she becomes a form that is enchanting, beautiful, and young, a beauty that is so enchanting to behold that she enchants the entire Universe with it. Kali Maa is an ancient Mother, not to be trifled with for the sake of entertainment, let's just hope that in her endless compassion and mercy that she does not take on wrathful form to those involved with this movie.<br /><br />The audacity that Westerners have in using religions like my own, or the religions of the Caribbean Islands such as Santeria, and Vodou which are actually very positive, and other such religions to twist and exaggerate misunderstood elements that the Western mind cannot comprehend, is totally ridiculous. It's clear that there is no respect for what people live, breathe and believe in when it comes to these kind of flicks.<br /><br />Kali Maa in reality is a caring and compassionate mother, whom we shed tears at her beautiful feet in devotion and love for. And I am happy that my Mother takes on wrathful form sometimes to protect her devotees from themselves and from outside forces.<br /><br />Many Praises to the REAL Kali Maa, who has shown MANY the path of God and realization.\",\n          \"Paris is the place to be to enjoy beautiful art and music, and to fall madly in love - as is the case in this film. Boy meets girl, they fall in love, but something stands in their way of eternal happiness, the classic story.<br /><br />The wonderful music of George Gerschwin complements the great dancing by Gene Kelly and Leslie Caron. \\\"An American in Paris\\\" is a humorous, light-hearted, loving film well worth watching.<br /><br />8/10<br /><br />\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"label\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1,\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "train_df = imdb_dataset['train'].to_pandas()\n",
        "test_df = imdb_dataset['test'].to_pandas()\n",
        "unsupervised_df = imdb_dataset['unsupervised'].to_pandas()\n",
        "\n",
        "train_df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "snGBplFjvs_S"
      },
      "source": [
        "**preprocessing**\n",
        "\n",
        "sklearn vectorizers can perform preprocessing while vectorizing a text (see the documentation). gensim offer external methods to do so (see the documentation [preprocessing gensim methods](https://radimrehurek.com/gensim/parsing/preprocessing.html)).\n",
        "\n",
        "To compare vectorization, it is important to apply the same preprocessing.\n",
        "\n",
        "Instead of using a home made method we choose to use a gensim method to be compliant with the gensim word2vec pretrained models.\n",
        "\n",
        "The method applies a list of chosen filters. By default: strip_tags(), strip_punctuation(),  strip_multiple_whitespaces(), strip_numeric(), remove_stopwords(), strip_short(), stem_text().\n",
        "\n",
        "\n",
        "TODO_FURTHER: you may want to change the preprocessing later. For example, add `stem_text` !"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# create nlp preprocessing filters\n",
        "from gensim.parsing.preprocessing import strip_tags\n",
        "from gensim.parsing.preprocessing import strip_punctuation\n",
        "from gensim.parsing.preprocessing import strip_multiple_whitespaces\n",
        "from gensim.parsing.preprocessing import remove_stopwords\n",
        "from gensim.parsing.preprocessing import strip_numeric\n",
        "from gensim.parsing.preprocessing import strip_short\n",
        "from gensim.parsing.preprocessing import stem_text\n",
        "my_filters = [lambda x: x.lower(), strip_tags, strip_punctuation, strip_multiple_whitespaces, remove_stopwords, strip_numeric, strip_short] # , stem_text\n",
        "\n",
        "# preprocess\n",
        "from gensim.parsing.preprocessing import preprocess_string\n",
        "%time train_df['preprocessed_text'] = train_df['text'].apply(lambda s : ' '.join(preprocess_string(s, filters = my_filters)))\n",
        "%time test_df['preprocessed_text'] = test_df['text'].apply(lambda s : ' '.join(preprocess_string(s, filters = my_filters)))\n",
        "%time unsupervised_df['preprocessed_text'] = unsupervised_df['text'].apply(lambda s : ' '.join(preprocess_string(s, filters = my_filters)))\n",
        "\n",
        "train_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 528
        },
        "id": "lsU7yFBdZT_3",
        "outputId": "6f95864c-697b-4d76-d9e2-34145a7fdd6e"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 8.28 s, sys: 25.4 ms, total: 8.31 s\n",
            "Wall time: 8.83 s\n",
            "CPU times: user 8.21 s, sys: 41 ms, total: 8.25 s\n",
            "Wall time: 8.35 s\n",
            "CPU times: user 19.7 s, sys: 137 ms, total: 19.8 s\n",
            "Wall time: 22.3 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                                                                                                                                                                          text  \\\n",
              "0      I rented I AM CURIOUS-YELLOW from my video store because of all the controversy that surrounded it when it was first released in 1967. I also heard that at first it was seized by U.S. customs if i...   \n",
              "1      \"I Am Curious: Yellow\" is a risible and pretentious steaming pile. It doesn't matter what one's political views are because this film can hardly be taken seriously on any level. As for the claim t...   \n",
              "2      If only to avoid making this type of film in the future. This film is interesting as an experiment but tells no cogent story.<br /><br />One might feel virtuous for sitting thru it because it touc...   \n",
              "3      This film was probably inspired by Godard's Masculin, féminin and I urge you to see that film instead.<br /><br />The film has two strong elements and those are, (1) the realistic acting (2) the i...   \n",
              "4      Oh, brother...after hearing about this ridiculous film for umpteen years all I can think of is that old Peggy Lee song..<br /><br />\"Is that all there is??\" ...I was just an early teen when this s...   \n",
              "...                                                                                                                                                                                                        ...   \n",
              "24995  A hit at the time but now better categorised as an Australian cult film. The humour is broad, unsubtle and, in the final scene where a BBC studio fire is extinguished by urinating on it, crude. Co...   \n",
              "24996  I love this movie like no other. Another time I will try to explain its virtues to the uninitiated, but for the moment let me quote a few of pieces the remarkable dialogue, which, please remember,...   \n",
              "24997  This film and it's sequel Barry Mckenzie holds his own, are the two greatest comedies to ever be produced. A great story a young Aussie bloke travels to england to claim his inheritance and meets ...   \n",
              "24998  'The Adventures Of Barry McKenzie' started life as a satirical comic strip in 'Private Eye', written by Barry Humphries and based on an idea by Peter Cook. McKenzie ( 'Bazza' to his friends ) is a...   \n",
              "24999  The story centers around Barry McKenzie who must go to England if he wishes to claim his inheritance. Being about the grossest Aussie shearer ever to set foot outside this great Nation of ours the...   \n",
              "\n",
              "       label  \\\n",
              "0          0   \n",
              "1          0   \n",
              "2          0   \n",
              "3          0   \n",
              "4          0   \n",
              "...      ...   \n",
              "24995      1   \n",
              "24996      1   \n",
              "24997      1   \n",
              "24998      1   \n",
              "24999      1   \n",
              "\n",
              "                                                                                                                                                                                             preprocessed_text  \n",
              "0      rented curious yellow video store controversy surrounded released heard seized customs tried enter country fan films considered controversial plot centered young swedish drama student named lena w...  \n",
              "1      curious yellow risible pretentious steaming pile matter political views film hardly taken seriously level claim frontal male nudity automatic isn true seen rated films male nudity granted offer fl...  \n",
              "2      avoid making type film future film interesting experiment tells cogent story feel virtuous sitting touches important issues discernable motive viewer comes away new perspectives comes mind wanders...  \n",
              "3      film probably inspired godard masculin féminin urge film instead film strong elements realistic acting impressive undeservedly good photo apart strikes endless stream silliness lena nyman annoying...  \n",
              "4      brother hearing ridiculous film umpteen years think old peggy lee song early teen smoked fish hit young theater manage sneak goodbye columbus screening local film museum beckoned finally film old ...  \n",
              "...                                                                                                                                                                                                        ...  \n",
              "24995  hit time better categorised australian cult film humour broad unsubtle final scene bbc studio extinguished urinating crude contains cliche traditional australian pilgrimage old country cliche rapa...  \n",
              "24996  love movie like time try explain virtues uninitiated moment let quote pieces remarkable dialogue remember tongue cheek aussies poms understand title song lyric sink beer pick queer latest double b...  \n",
              "24997  film sequel barry mckenzie holds greatest comedies produced great story young aussie bloke travels england claim inheritance meets mates loveable innocent chock block great sayings needs drink bad...  \n",
              "24998  adventures barry mckenzie started life satirical comic strip private eye written barry humphries based idea peter cook mckenzie bazza friends lanky loud hat wearing australian main interests life ...  \n",
              "24999          story centers barry mckenzie england wishes claim inheritance grossest aussie shearer set foot outside great nation culture clash fun games ensue songs barry mckenzie barry crocker highlights  \n",
              "\n",
              "[25000 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e574fd83-ac58-40e9-ae9d-3b28d7db8286\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>label</th>\n",
              "      <th>preprocessed_text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>I rented I AM CURIOUS-YELLOW from my video store because of all the controversy that surrounded it when it was first released in 1967. I also heard that at first it was seized by U.S. customs if i...</td>\n",
              "      <td>0</td>\n",
              "      <td>rented curious yellow video store controversy surrounded released heard seized customs tried enter country fan films considered controversial plot centered young swedish drama student named lena w...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>\"I Am Curious: Yellow\" is a risible and pretentious steaming pile. It doesn't matter what one's political views are because this film can hardly be taken seriously on any level. As for the claim t...</td>\n",
              "      <td>0</td>\n",
              "      <td>curious yellow risible pretentious steaming pile matter political views film hardly taken seriously level claim frontal male nudity automatic isn true seen rated films male nudity granted offer fl...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>If only to avoid making this type of film in the future. This film is interesting as an experiment but tells no cogent story.&lt;br /&gt;&lt;br /&gt;One might feel virtuous for sitting thru it because it touc...</td>\n",
              "      <td>0</td>\n",
              "      <td>avoid making type film future film interesting experiment tells cogent story feel virtuous sitting touches important issues discernable motive viewer comes away new perspectives comes mind wanders...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>This film was probably inspired by Godard's Masculin, féminin and I urge you to see that film instead.&lt;br /&gt;&lt;br /&gt;The film has two strong elements and those are, (1) the realistic acting (2) the i...</td>\n",
              "      <td>0</td>\n",
              "      <td>film probably inspired godard masculin féminin urge film instead film strong elements realistic acting impressive undeservedly good photo apart strikes endless stream silliness lena nyman annoying...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Oh, brother...after hearing about this ridiculous film for umpteen years all I can think of is that old Peggy Lee song..&lt;br /&gt;&lt;br /&gt;\"Is that all there is??\" ...I was just an early teen when this s...</td>\n",
              "      <td>0</td>\n",
              "      <td>brother hearing ridiculous film umpteen years think old peggy lee song early teen smoked fish hit young theater manage sneak goodbye columbus screening local film museum beckoned finally film old ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24995</th>\n",
              "      <td>A hit at the time but now better categorised as an Australian cult film. The humour is broad, unsubtle and, in the final scene where a BBC studio fire is extinguished by urinating on it, crude. Co...</td>\n",
              "      <td>1</td>\n",
              "      <td>hit time better categorised australian cult film humour broad unsubtle final scene bbc studio extinguished urinating crude contains cliche traditional australian pilgrimage old country cliche rapa...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24996</th>\n",
              "      <td>I love this movie like no other. Another time I will try to explain its virtues to the uninitiated, but for the moment let me quote a few of pieces the remarkable dialogue, which, please remember,...</td>\n",
              "      <td>1</td>\n",
              "      <td>love movie like time try explain virtues uninitiated moment let quote pieces remarkable dialogue remember tongue cheek aussies poms understand title song lyric sink beer pick queer latest double b...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24997</th>\n",
              "      <td>This film and it's sequel Barry Mckenzie holds his own, are the two greatest comedies to ever be produced. A great story a young Aussie bloke travels to england to claim his inheritance and meets ...</td>\n",
              "      <td>1</td>\n",
              "      <td>film sequel barry mckenzie holds greatest comedies produced great story young aussie bloke travels england claim inheritance meets mates loveable innocent chock block great sayings needs drink bad...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24998</th>\n",
              "      <td>'The Adventures Of Barry McKenzie' started life as a satirical comic strip in 'Private Eye', written by Barry Humphries and based on an idea by Peter Cook. McKenzie ( 'Bazza' to his friends ) is a...</td>\n",
              "      <td>1</td>\n",
              "      <td>adventures barry mckenzie started life satirical comic strip private eye written barry humphries based idea peter cook mckenzie bazza friends lanky loud hat wearing australian main interests life ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24999</th>\n",
              "      <td>The story centers around Barry McKenzie who must go to England if he wishes to claim his inheritance. Being about the grossest Aussie shearer ever to set foot outside this great Nation of ours the...</td>\n",
              "      <td>1</td>\n",
              "      <td>story centers barry mckenzie england wishes claim inheritance grossest aussie shearer set foot outside great nation culture clash fun games ensue songs barry mckenzie barry crocker highlights</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>25000 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e574fd83-ac58-40e9-ae9d-3b28d7db8286')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e574fd83-ac58-40e9-ae9d-3b28d7db8286 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e574fd83-ac58-40e9-ae9d-3b28d7db8286');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-67606e8f-7479-4d16-8759-03228cf45bac\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-67606e8f-7479-4d16-8759-03228cf45bac')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-67606e8f-7479-4d16-8759-03228cf45bac button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_263e52c9-9c0e-495b-baff-bc3aa4f5e3b0\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('train_df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_263e52c9-9c0e-495b-baff-bc3aa4f5e3b0 button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('train_df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "train_df",
              "summary": "{\n  \"name\": \"train_df\",\n  \"rows\": 25000,\n  \"fields\": [\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 24904,\n        \"samples\": [\n          \"Although I didn't like Stanley & Iris tremendously as a film, I did admire the acting. Jane Fonda and Robert De Niro are great in this movie. I haven't always been a fan of Fonda's work but here she is delicate and strong at the same time. De Niro has the ability to make every role he portrays into acting gold. He gives a great performance in this film and there is a great scene where he has to take his father to a home for elderly people because he can't care for him anymore that will break your heart. I wouldn't really recommend this film as a great cinematic entertainment, but I will say you won't see much bette acting anywhere.\",\n          \"Yet again one of the most misunderstood Goddesses of my country has been twisted by \\\"Westerners\\\" who cannot understand the esoteric symbolism of the Mother Goddess in her dark forms. The Mother takes on the frightening form of Kali Mata to destroy our inner demons, and to terrify our egos. And though blood sacrifice is given to Kali and Durga, the events depicted in this film are just absurd. The Mother takes on a wrathful form to be wrathful to our inner demons, limitations, and ego when no other form will suffice. It's also in her wrathful form that she burns away all your Karmas in the \\\"Smashan\\\" fires that you cultivate in your heart for her to dance on if you love her, and she will bring you to reality and truth. Reality and truth has a dark side as well as light, which serves a purpose. The Mother is the embodiment of the physical Universe as well, she is Nature. Nature can be cruel and destructive to maintain balance. You cannot have growth and life without death and destruction. Kali represents the force of destruction for the purpose of new growth and life both mundane and spiritual in the universe. It's very outrageous to me that people who know nothing of India or it's divinity can just take one of our beloved Goddesses and use her like a cheap prostitute to make some low-budget, talentless horror film. How dare they take our beloved Mother and portray her as a horror that makes people chop their eyelids off!? She is only horrific to those who are attached to their ego and who live in delusion , greed, anger, and other inner-demons. It's very clear to me that the person who wrote this movie must have a very serious self-deluding ego, and serious inner-demons to see Kali as so horrible and terrible. When the ego drops away she becomes a form that is enchanting, beautiful, and young, a beauty that is so enchanting to behold that she enchants the entire Universe with it. Kali Maa is an ancient Mother, not to be trifled with for the sake of entertainment, let's just hope that in her endless compassion and mercy that she does not take on wrathful form to those involved with this movie.<br /><br />The audacity that Westerners have in using religions like my own, or the religions of the Caribbean Islands such as Santeria, and Vodou which are actually very positive, and other such religions to twist and exaggerate misunderstood elements that the Western mind cannot comprehend, is totally ridiculous. It's clear that there is no respect for what people live, breathe and believe in when it comes to these kind of flicks.<br /><br />Kali Maa in reality is a caring and compassionate mother, whom we shed tears at her beautiful feet in devotion and love for. And I am happy that my Mother takes on wrathful form sometimes to protect her devotees from themselves and from outside forces.<br /><br />Many Praises to the REAL Kali Maa, who has shown MANY the path of God and realization.\",\n          \"Paris is the place to be to enjoy beautiful art and music, and to fall madly in love - as is the case in this film. Boy meets girl, they fall in love, but something stands in their way of eternal happiness, the classic story.<br /><br />The wonderful music of George Gerschwin complements the great dancing by Gene Kelly and Leslie Caron. \\\"An American in Paris\\\" is a humorous, light-hearted, loving film well worth watching.<br /><br />8/10<br /><br />\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"label\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1,\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"preprocessed_text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 24900,\n        \"samples\": [\n          \"eddie izzard million comic genius goes squirrels wwii stonehenge religion englebert humperdink absolutely hilarious makes sense copy won regret\",\n          \"delectable fusion new age babble luridly bad film making open borrow film favorite verbs leave jaw slack belly sore laughter retching based best selling book james redfield self published cornucopia kitsch tracks spiritual awakening american history teacher matthew settle traveling deepest darkest phoniest peru sniffing air illegal discovers schlock shangri populated smiling zombies nuts heavily medicated accompanied panpipe flourish occasional shout celestial choir lot talk energy quality decidedly missing motley cast numbers include thomas kretschmann annabeth gish hector elizondo jurgen prochnow firmly ensconced camp pantheon care plot involves military terrorists roman catholic church armand mastroianni provided inept direction redfield barnet bain dan gordon wrote hoot script short easily worst film seen years viewing movies\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**create split sets**"
      ],
      "metadata": {
        "id": "iHdqtmWX0CKE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#\n",
        "X_train = train_df['preprocessed_text']\n",
        "Y_train = train_df['label']\n",
        "X_test = test_df['preprocessed_text']\n",
        "Y_test = test_df['label']\n",
        "X_unsupervised = unsupervised_df['preprocessed_text']\n",
        "\n",
        "print (X_train[:5], Y_train[:5])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mw2UDmE-z_Zh",
        "outputId": "7ba55255-3379-4e66-e9fb-e3361a508d43"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0    rented curious yellow video store controversy surrounded released heard seized customs tried enter country fan films considered controversial plot centered young swedish drama student named lena w...\n",
            "1    curious yellow risible pretentious steaming pile matter political views film hardly taken seriously level claim frontal male nudity automatic isn true seen rated films male nudity granted offer fl...\n",
            "2    avoid making type film future film interesting experiment tells cogent story feel virtuous sitting touches important issues discernable motive viewer comes away new perspectives comes mind wanders...\n",
            "3    film probably inspired godard masculin féminin urge film instead film strong elements realistic acting impressive undeservedly good photo apart strikes endless stream silliness lena nyman annoying...\n",
            "4    brother hearing ridiculous film umpteen years think old peggy lee song early teen smoked fish hit young theater manage sneak goodbye columbus screening local film museum beckoned finally film old ...\n",
            "Name: preprocessed_text, dtype: object 0    0\n",
            "1    0\n",
            "2    0\n",
            "3    0\n",
            "4    0\n",
            "Name: label, dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# to numpy\n",
        "X_train_np = np.array(X_train)\n",
        "Y_train_np = np.array(Y_train)\n",
        "X_test_np = np.array(X_test)\n",
        "Y_test_np = np.array(Y_test)\n",
        "X_unsupervised_np = np.array(X_unsupervised)\n",
        "\n",
        "print (X_train_np[:5], '\\n', Y_train_np[:5])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-NoT3rk40Hsc",
        "outputId": "b36f187f-be83-4444-dcb8-8981dd662c5f"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['rented curious yellow video store controversy surrounded released heard seized customs tried enter country fan films considered controversial plot centered young swedish drama student named lena wants learn life particular wants focus attentions making sort documentary average swede thought certain political issues vietnam war race issues united states asking politicians ordinary denizens stockholm opinions politics sex drama teacher classmates married men kills curious yellow years ago considered pornographic sex nudity scenes far shot like cheaply porno countrymen mind shocking reality sex nudity major staple swedish cinema ingmar bergman arguably answer good old boy john ford sex scenes films commend filmmakers fact sex shown film shown artistic purposes shock people money shown pornographic theaters america curious yellow good film wanting study meat potatoes pun intended swedish cinema film plot'\n",
            " 'curious yellow risible pretentious steaming pile matter political views film hardly taken seriously level claim frontal male nudity automatic isn true seen rated films male nudity granted offer fleeting views rated films gaping vulvas flapping labia exist goes crappy cable shows schlongs swinging breeze clitoris sight pretentious indie movies like brown bunny treated site vincent gallo throbbing johnson trace pink visible chloe sevigny crying implying double standard matters nudity mentally obtuse account unavoidably obvious anatomical difference men women genitals display actresses appears nude said man fact generally won female genitals american film short porn explicit erotica alleged double standard double standard admittedly depressing ability come terms culturally insides women bodies'\n",
            " 'avoid making type film future film interesting experiment tells cogent story feel virtuous sitting touches important issues discernable motive viewer comes away new perspectives comes mind wanders invariably pointless film better spend time staring window tree growing'\n",
            " 'film probably inspired godard masculin féminin urge film instead film strong elements realistic acting impressive undeservedly good photo apart strikes endless stream silliness lena nyman annoying actress world acts stupid nudity film unattractive comparing godard film intellectuality replaced stupidity going far subject follows difference ideals french swedish society movie time place'\n",
            " 'brother hearing ridiculous film umpteen years think old peggy lee song early teen smoked fish hit young theater manage sneak goodbye columbus screening local film museum beckoned finally film old parents schlepped reason film condemned anonymous sands time obscenity case sparked release millions people flocked stinker thinking going sex film instead got lots closeups gnarly repulsive swedes street interviews bland shopping malls asinie political pretension feeble cares simulated sex scenes saggy pale actors cultural icon holy grail historic artifact thing shred burn stuff ashes lead box elite esthetes scrape value boring pseudo revolutionary political spewings weren censorship scandal ignored forgotten instead blank blank rhythymed title repeated endlessly years titilation porno films curious lavender gay films curious black blaxploitation films years thing rises dead viewed new generation suckers want naughty sex film revolutionized film industry yeesh avoid like plague rent video fast forward dirty parts'] \n",
            " [0 0 0 0 0]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Sample\n",
        "\n",
        "In order to make the process faster, work on a sample"
      ],
      "metadata": {
        "id": "1pQfuq2gF8F1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# select a sample then split it into training and testing sets\n",
        "from sklearn.utils import resample\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "sample_data = resample(train_df, n_samples = 1000, random_state = 42)\n",
        "print (sample_data.shape)\n",
        "print (sample_data.columns)\n",
        "\n",
        "X_train, X_test, Y_train, Y_test = train_test_split(sample_data['preprocessed_text'], sample_data['label'], test_size=0.2, random_state=45 )\n",
        "X_train_np = np.array(X_train)\n",
        "Y_train_np = np.array(Y_train)\n",
        "X_test_np = np.array(X_test)\n",
        "Y_test_np = np.array(Y_test)\n",
        "\n",
        "# FURTHER  perform your own preprocess on the raw dataset"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GIWlx6fvF8SJ",
        "outputId": "2a74e3b3-7403-4e40-e77f-864c588e1ce1"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1000, 3)\n",
            "Index(['text', 'label', 'preprocessed_text'], dtype='object')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from collections import Counter\n",
        "print ('Y_train distribution', list(Counter(Y_train).items()))\n",
        "print ('Y_test distribution', list(Counter(Y_test).items()))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1ZhT5AlAKMGo",
        "outputId": "ecd9a976-51f1-4709-eba0-8e05a5f5467a"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Y_train distribution [(0, 388), (1, 412)]\n",
            "Y_test distribution [(1, 92), (0, 108)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jkr1RKkDipqo"
      },
      "source": [
        "# Activity 1: Vectorization 1 via term frequency model\n",
        "\n",
        "Given train, test and unsupervised datasets.\n",
        "\n",
        "1. vectorize X_train and X_test using vocabulary learnt on X_unsupervised\n",
        "  - First, fit the vectorizer on the raw data (here *imdb-unsupervised*) to learn a vocabulary dictionary of tokens.\n",
        "  - Then, transform the train and the test documents to document-term matrix. This will extract token counts out of the train and the test text documents using the vocabulary fitted with the fit method on the raw data. See [CountVectorizer API](https://scikit-learn.org/1.5/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html) to read more on the fit, transform and the fit_transform methods\n",
        "2. train a classification model (here a logistic) on the train data\n",
        "3. use the model learnt to predict the classes of the tests instances\n",
        "4. evaluate the prediction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YBUAsxjCKEe9"
      },
      "source": [
        "1. Vectorize the datasets\n",
        "\n",
        "Le code suivant réalise une vectorisation du corpus en comptant les occurrences des mots. La vectorisation prend en charge la construction d'un vocabulaire sur le corpus ainsi que quelques pré-traitements de normalisation linguistiques. Enfin le résultat de la vectorisation est affiché sous la forme d'une matrice **document-terme** (les documents sont en ligne et les mots en colonne).\n",
        "Nous aurons une matrice de dimension _nb de documents * taille du vocabulaire_.\n",
        "\n",
        "Here we limit the number of dimensions to ```max_features=1000```.\n",
        "\n",
        "Do not hesite to have a look at all the parameters of [CountVectorizer](https://scikit-learn.org/1.5/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 493
        },
        "id": "rJlCIP-3r_Vi",
        "outputId": "d3444e5f-4397-4d03-e275-55c128749d36"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CountVectorizer(max_features=1000)\n",
            "X_train dimensions: (25000, 1000) Taille mémoire 48\n",
            "X_test dimensions: (25000, 1000) Taille mémoire 48\n",
            "Vocabulary size: 1000\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       ability  able  absolutely  accent  act  acted  acting  action  actor  \\\n",
              "0            0     0           0       0    0      0       0       0      0   \n",
              "1            1     0           0       0    0      0       0       0      0   \n",
              "2            0     0           0       0    0      0       0       0      0   \n",
              "3            0     0           0       0    0      0       1       0      0   \n",
              "4            0     0           0       0    0      0       0       0      0   \n",
              "...        ...   ...         ...     ...  ...    ...     ...     ...    ...   \n",
              "24995        0     0           0       0    0      0       0       0      0   \n",
              "24996        0     0           0       0    0      0       0       0      0   \n",
              "24997        0     0           0       0    0      0       1       0      0   \n",
              "24998        0     0           0       0    0      0       0       0      0   \n",
              "24999        0     0           0       0    0      0       0       0      0   \n",
              "\n",
              "       actors  ...  wrong  wrote  yeah  year  years  yes  york  young  \\\n",
              "0           0  ...      0      0     0     0      1    0     0      1   \n",
              "1           0  ...      0      0     0     0      0    0     0      0   \n",
              "2           0  ...      0      0     0     0      0    0     0      0   \n",
              "3           0  ...      0      0     0     0      0    0     0      0   \n",
              "4           1  ...      0      0     0     0      3    0     0      1   \n",
              "...       ...  ...    ...    ...   ...   ...    ...  ...   ...    ...   \n",
              "24995       1  ...      0      0     0     0      0    0     0      0   \n",
              "24996       0  ...      0      0     0     0      0    0     0      1   \n",
              "24997       0  ...      0      0     0     0      0    0     0      1   \n",
              "24998       0  ...      0      0     0     0      1    1     0      1   \n",
              "24999       0  ...      0      0     0     0      0    0     0      0   \n",
              "\n",
              "       younger  zombie  \n",
              "0            0       0  \n",
              "1            0       0  \n",
              "2            0       0  \n",
              "3            0       0  \n",
              "4            0       0  \n",
              "...        ...     ...  \n",
              "24995        0       0  \n",
              "24996        0       0  \n",
              "24997        0       0  \n",
              "24998        0       0  \n",
              "24999        0       0  \n",
              "\n",
              "[25000 rows x 1000 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-321b1913-d94c-459b-9fdb-450872dd4abf\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>ability</th>\n",
              "      <th>able</th>\n",
              "      <th>absolutely</th>\n",
              "      <th>accent</th>\n",
              "      <th>act</th>\n",
              "      <th>acted</th>\n",
              "      <th>acting</th>\n",
              "      <th>action</th>\n",
              "      <th>actor</th>\n",
              "      <th>actors</th>\n",
              "      <th>...</th>\n",
              "      <th>wrong</th>\n",
              "      <th>wrote</th>\n",
              "      <th>yeah</th>\n",
              "      <th>year</th>\n",
              "      <th>years</th>\n",
              "      <th>yes</th>\n",
              "      <th>york</th>\n",
              "      <th>young</th>\n",
              "      <th>younger</th>\n",
              "      <th>zombie</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24995</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24996</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24997</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24998</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24999</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>25000 rows × 1000 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-321b1913-d94c-459b-9fdb-450872dd4abf')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-321b1913-d94c-459b-9fdb-450872dd4abf button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-321b1913-d94c-459b-9fdb-450872dd4abf');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-f92f88cd-124d-4fa0-b081-9e28c66fabe3\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f92f88cd-124d-4fa0-b081-9e28c66fabe3')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-f92f88cd-124d-4fa0-b081-9e28c66fabe3 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe"
            }
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "# 1\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "# define the vectorizer\n",
        "DIMENSION_SIZE = 1000\n",
        "count_vectorizer = CountVectorizer(stop_words= None, max_features=DIMENSION_SIZE) #stop_words='english')\n",
        "\n",
        "# display the configuration of the vectorizer\n",
        "print (count_vectorizer)\n",
        "\n",
        "# 1.1 Learn the vocabulary (fit)\n",
        "X_unsupervised_counter_matrix = count_vectorizer.fit(X_unsupervised_np)\n",
        "\n",
        "# 1.2 Transform documents to document-term matrix (i.e. perform the vectorization) by using Vocabulary learned\n",
        "X_train_counter_matrix = count_vectorizer.transform(X_train_np)\n",
        "X_test_counter_matrix = count_vectorizer.transform(X_test_np)\n",
        "print ('X_train dimensions:', X_train_counter_matrix.get_shape(), 'Taille mémoire', sys.getsizeof(X_train_counter_matrix))\n",
        "print ('X_test dimensions:', X_test_counter_matrix.get_shape(), 'Taille mémoire', sys.getsizeof(X_test_counter_matrix))\n",
        "\n",
        "# get all unique words in the corpus (the vocabulary and also the names of the matrix columns/features)\n",
        "vocab = count_vectorizer.get_feature_names_out() # .get_feature_names() # for sklearn_version >= 1.0\n",
        "print ('Vocabulary size:', len(vocab))\n",
        "\n",
        "# show document-term matrix\n",
        "X_train_counter_matrix_array = X_train_counter_matrix.toarray()\n",
        "pd.DataFrame(X_train_counter_matrix_array, columns=vocab)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wsTaiOQZvytd"
      },
      "source": [
        "2. **train** a classifier"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "kqbtU5O9vy25"
      },
      "outputs": [],
      "source": [
        "# 2.\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "LR_counter_model = LogisticRegression().fit(X_train_counter_matrix, Y_train_np)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M8FxZeANv2wk"
      },
      "source": [
        "3. use the classifier to **predict**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "Kjy7onedv24Y"
      },
      "outputs": [],
      "source": [
        "# 3.\n",
        "Y_test_prediction = LR_counter_model.predict(X_test_counter_matrix)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "peUGEht3v7OF"
      },
      "source": [
        "4. evaluate"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bwt7SFzQv7XD",
        "outputId": "ae1974ef-a79e-4093-e237-a7f9e1a63451"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.8504\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.86      0.84      0.85     12500\n",
            "           1       0.84      0.86      0.85     12500\n",
            "\n",
            "    accuracy                           0.85     25000\n",
            "   macro avg       0.85      0.85      0.85     25000\n",
            "weighted avg       0.85      0.85      0.85     25000\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# 4.\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "print(accuracy_score(Y_test,Y_test_prediction))\n",
        "print(classification_report(Y_test,Y_test_prediction))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HIuWsNULPULG"
      },
      "source": [
        "# Activity 2: Vectorization 2 via TF-IDF model\n",
        "\n",
        "La vectorisation avec occurrences présente des limites lorsqu'elle est utilisée sur de large corpus. En effet, le modèle présume que l'importance des mots est fonction de sa fréquence et qu'un mot plus fréquent qu'un autre dans un document est plus discriminant que l'autre. Le problème intervient quand un mot fréquent, supposé important, apparaît dans plusieurs documents. Le fait qu'il apparaisse dans plusieurs documents peut au final le rendre moins discriminant que d'autres pourtant moins fréquents. Le problème vient du fait que l'on prenne des valeurs absolues.\n",
        "\n",
        "Le modèle **TF-IDF** vise à solutionner ce problème en normalisant le compte des occurrences. TF-IDF correspond à _Term Frequency-Inverse Document Frequency_.\n",
        "\n",
        "On définit le TF-IDF comme suit: `tfidf = tf x idf`\n",
        "* `tfidf(w, D)` est le score TF-IDF du mot `w` dans le document `D`\n",
        "* `tf(w, D)` représente le nombre d'occurrence du terme `w` dans le document `D`\n",
        "* `idf(w, D)` représente la fréquence inverse documentaire du terme `w`, qui peut être calculée comme le log du nombre total de documents dans le corpus `C` divisé par la fréquence documentaire du terme `w` (i.e. le nombre de documents du corpus `C` dans lequel le terme `w` se produit).\n",
        "\n",
        "\n",
        "Here the definition of the [TfidfVectorizer](https://scikit-learn.org/1.5/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html)."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO Implement the TfidfVectorizer vectorizer, train, predict and evaluate a LogisticRegression using this vectorizer\n",
        "\n",
        "TODO Play with the number of dimensions (50, 100, 500, 1000, 5000, 10000...)  and observe the performance"
      ],
      "metadata": {
        "id": "wPmYdiZ08ujp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# 1. define the vectorizer\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "DIMENSION_SIZE = 50 #\n",
        "# TODO\n",
        "%time tfidf_vectorizer = None\n",
        "\n",
        "# 1.1 Learn the vocabulary (fit)\n",
        "# TODO\n",
        "X_unsupervised_tfidf_matrix = None\n",
        "\n",
        "# 1.2 transform documents to document-term matrix (i.e. perform the vectorization)\n",
        "# TODO\n",
        "X_train_tfidf_matrix = None\n",
        "X_test_tfidf_matrix = None\n",
        "\n",
        "# 2. train\n",
        "# TODO\n",
        "\n",
        "# 3. predict\n",
        "# TODO\n",
        "\n",
        "# 4. evaluate\n",
        "# TODO\n"
      ],
      "metadata": {
        "id": "qZo9kxRG2BXi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Activity 3: Play with static dense models"
      ],
      "metadata": {
        "id": "pWZNA21jNomG"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_aW9RPQRD32D"
      },
      "source": [
        "## Load existing pretrain models\n",
        "\n",
        "Le dépôt [gensim-data](https://github.com/RaRe-Technologies/gensim-data) contient quelques corpus et modèles pré-entraînés librement disponibles.\n",
        "Jean-Philippe Fauconnier met des [modèles word2vec à disposition pour le français ](https://fauconnier.github.io/#data).\n",
        "\n",
        "\n",
        "The model size depends on the number of dimensions. Higher dimensions model will take more time to download.\n",
        "\n",
        "TODO select a model (start with a small model). Optionaly, uncomment the code to show info about the available gensim models/datasets.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L23OV9ZgblQp",
        "outputId": "e4c2fa46-5bc0-4d61-e2e5-aee231a8e711"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[==================================================] 100.0% 387.1/387.1MB downloaded\n",
            "CPU times: user 4min 4s, sys: 14.7 s, total: 4min 19s\n",
            "Wall time: 6min 49s\n"
          ]
        }
      ],
      "source": [
        "import gensim.downloader as gensim_api\n",
        "\n",
        "# uncomment to show info about available models/datasets\n",
        "# print(json.dumps(gensim_api.info(), indent=4))\n",
        "\n",
        "# load a w2v_model\n",
        "#%time glove_twitter_25_model = gensim_api.load(\"glove-twitter-25\")         # 1193514 vectors # 104 MB\n",
        "#%time glove_twitter_50_model = gensim_api.load(\"glove-twitter-50\")         # 1193514 vectors # 199 MB\n",
        "%time glove_twitter_100_model = gensim_api.load(\"glove-twitter-100\")        # 1193514 vectors # 387 MB\n",
        "#%time glove_wiki_gigaword_100_model = gensim_api.load(\"glove-wiki-gigaword-100\") # 400000  vectors # 128 MB\n",
        "#%time ft_wiki_news_300_model = gensim_api.load(\"fasttext-wiki-news-subwords-300 \") # 999999  vectors # 958 MB\n",
        "#%time w2v_gn_300_model = gensim_api.load('word2vec-google-news-300') # 3000000 vectors # 1662 MB\n",
        "\n",
        "# generic variable in the following codes\n",
        "w2v_model = glove_twitter_100_model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zgpFk5jWNomP"
      },
      "source": [
        "## Get most similar words and search for bias\n",
        "\n",
        "Pour chaque question ci-dessous, jouez le jeu et prenez le temps faire des propositions de réponses avant d'exécuter le code qui permettra de consulter la connaissance du modèle et connaître ce qu'il répondrait."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vp_ldrADNomQ"
      },
      "source": [
        "**Si je vous dis 'king', vous pensez à quoi ?**\n",
        "\n",
        "TODO Faire quelques propositions de synonymes ou de mots substituables sémantiquement proches. La méthode `most_similar` affichera les 10 mots les plus proches d'un mot donné, du plus similaire au moins similaire, avec pour chacun un score de similarité avec le mot donné (scores décroissants donc)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7Xbtcw7BNomR"
      },
      "outputs": [],
      "source": [
        "# obtenir les mots similaires à 1 mot\n",
        "w2v_model.most_similar(\"king\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ouRFN2n7NomS"
      },
      "source": [
        "Si je vous demande de me donner des mots relatifs à des 'dessert' et 'cake', à quoi pensez-vous ? Pour information, la méthode accepte une liste de mots en paramètres."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "A1PVZRVlNomT"
      },
      "outputs": [],
      "source": [
        "# obtenir les mots similaires relatifs à une liste\n",
        "w2v_model.most_similar(['dessert', 'cake'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OASGU-73NomU"
      },
      "source": [
        "Si j'ajoute les vecteurs de roi et de femme et que je retire le vecteur homme qu'est ce que j'obtiens ? Répondez avant d'exécuter le code ci-dessous.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dCm2Mo4kNomU"
      },
      "outputs": [],
      "source": [
        "# Si j'ajoute les vecteurs de roi et de femme et que je retire le vecteur homme qu'est ce que j'obtiens ?\n",
        "w2v_model.most_similar(positive = ['king', 'woman'], negative = ['man'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZAcHPLGrNomU"
      },
      "source": [
        "One of the open science value is the access to knowledge to anyone regardless his/her location, nationality, race, age, gender, income, socio-economic circumstance, career stage, discipline, language, religion, disability, ethnicity, migratory status...\n",
        "\n",
        "Below an example of association which desmonstrates some gender bias:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print (w2v_model.most_similar(positive =['strength', 'man'], negative = ['woman']))\n",
        "print (w2v_model.most_similar(positive =['strength', 'woman'], negative = ['man']))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1FLHb-A_zuBv",
        "outputId": "45e9d347-683f-4956-ab83-28b836c01ace"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[('faith', 0.589231550693512), ('god', 0.5837271213531494), ('lord', 0.5627992153167725), ('rest', 0.5500725507736206), ('training', 0.5458858013153076), ('joy', 0.5310255885124207), ('power', 0.5269313454627991), ('determination', 0.5251309871673584), ('patience', 0.5244085192680359), ('motivation', 0.5203278064727783)]\n",
            "[('confidence', 0.6551448702812195), ('humility', 0.643629252910614), ('requires', 0.6390582919120789), ('faith', 0.6357606649398804), ('ability', 0.6332043409347534), ('discipline', 0.6117353439331055), ('inner', 0.6078113913536072), ('patience', 0.6021292209625244), ('powerful', 0.6009049415588379), ('qualities', 0.5983656048774719)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO: by using the `most_similar` method (with/without negative), show that the model contains some bias in at least three dimensions. Vous pouvez vous aider du visualiseur 2D ci-dessous pour cartographier les mots qui vous interrogent.\n"
      ],
      "metadata": {
        "id": "mDsK9n6mzt6K"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PGxJJaDSNomW"
      },
      "source": [
        "## Visualiser les plongements lexicaux dans un graph en 2D\n",
        "\n",
        "Pour ce faire, il faut transformer les vecteurs de n dimensions à des vecteurs à 2 dimensions. La réduction des dimensions est effectuée à l'aide d'une [analyse en composantes principales (ACP ou PCA pour _Principal Component Analysis_ en anglais)](https://fr.wikipedia.org/wiki/Analyse_en_composantes_principales)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q80hGn82NomX"
      },
      "source": [
        "TODO Observer si la synonymie, la morphologie, les biais sont \"marqués\" spatialement (même zone ou position relative). Vous pouvez tester avec d'autres listes de mots.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "DIuwMKPPNomX",
        "outputId": "545fce82-150f-456d-80b9-72272fb52126"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x800 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1MAAAKTCAYAAAD8J1KcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQ1ElEQVR4nO3deXhUhd238e/JACGQBcKSjJKFfZOggPgATzBI2EQKRMBSK4sIVYEakCJqy+YCRQygqLhF8HFFDOibikgigQgIBElEGlDSwKAMBhGyUCEwM+8fyGgkbEcmMwn357pylTlz5pzfZKztzVnGcLlcLgEAAAAALouftwcAAAAAgMqImAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADChmrcHuBCn06mDBw8qKChIhmF4exwAAAAAXuJyuVRcXKxrrrlGfn6+cUzIp2Pq4MGDioiI8PYYAAAAAHzEgQMH1KhRI2+PIcnHYyooKEjSmV9YcHCwl6cBKlb//v3Vrl07zZ0712P7aNeune677z7df//9kqSQkBC9+eabuu222zy2TwAAADOKiooUERHhbgRf4NMxdfbUvuDgYGIKVx2LxaIaNWp49J99wzBUs2ZN9z7sdrvq1q0rf39/j+0TAADg9/Cly398OqYAVKzw8HBvjwAAAFBp+MaVWwDK5XQ6NXXqVIWGhio8PFwzZ850P3fs2DHdc889atCggYKDg3XLLbcoJyfH/XxeXp4GDhyosLAwBQYG6sYbb1RaWtoF92cYhlatWiVJ2rdvnwzDUEpKinr06KFatWqpffv22rx5c5nXvPzyy4qIiFCtWrU0ePBgJSUlqU6dOlfqVwAAAOCziCnAhy1btky1a9fWli1bNG/ePM2ePVtr166VJA0dOlQFBQVavXq1tm/frg4dOqhnz5768ccfJUklJSW69dZblZ6erh07dqhv374aMGCAbDbbZc3w6KOPasqUKcrOzlaLFi00fPhwnT59WpK0ceNG3XvvvXrggQeUnZ2tXr166YknnriyvwQAAAAfxWl+gA+LiYnRjBkzJEnNmzfX4sWLlZ6eroCAAG3dulUFBQXu65vmz5+vVatWacWKFRo3bpzat2+v9u3bu7f12GOPaeXKlfrwww81YcKES55hypQp6t+/vyRp1qxZatu2rfbu3atWrVrp2WefVb9+/TRlyhRJUosWLbRp0yalpqZeqV8BAACAz+LIFODDYmJiyjy2Wq0qKChQTk6OSkpKVK9ePQUGBrp/8vPzlZeXJ+nMkakpU6aodevWqlOnjgIDA5Wbm3vZR6Z+PYPVapUkFRQUSJL27Nmjzp07l1n/t48BAACqKo5MAT6sevXqZR4bhiGn06mSkhJZrVZlZGSc85qz1ytNmTJFa9eu1fz589WsWTMFBARoyJAhKi0tNT3D2bvnOJ3Oy3sjAAAAVRAxBVRCHTp00KFDh1StWjVFR0eXu87GjRs1atQoDR48WNKZI1X79u27onO0bNlS27ZtK7Pst48BAACqKk7zAyqh+Ph4denSRYMGDdInn3yiffv2adOmTXr00UeVlZUl6cw1VikpKcrOzlZOTo7+9Kc/XfEjShMnTtRHH32kpKQkffPNN3rxxRe1evVqn/r+BwAAAE8hpoBKyDAMffTRR+revbtGjx6tFi1a6I9//KP279+vsLAwSVJSUpLq1q2rrl27asCAAerTp486dOhwRefo1q2blixZoqSkJLVv314ff/yxJk2apJo1a17R/QAAAPgiw+Vyubw9xPkUFRUpJCREhYWFCg4O9vY4AC7B2LFjtXv3bmVmZnp7FAAAUIX4YhtwzRSA32X+/Pnq1auXateurdWrV2vZsmV6/vnnvT0WAACAxxFTAH6XrVu3at68eSouLlaTJk30zDPP6J577vH2WAAAAB5HTAH4XZYvX+7tEQAAALyCG1AAAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAl8zxTgwxxOhzJtmbIX22UNsio2MlYWP4u3xwIAAICIKcBnpeSmaNKaSbIV2tzLIkMitaDPAiW0TvDiZAAAAJCIKcAnpeSmaMjyIWrbuK0SeyfKWs8q+xG70rLSNGT5EK0YtoKgAgAA8DLD5XK5vD3E+RQVFSkkJESFhYUKDg729jhAhXA4HWryTBMFhwbr7tvulp/xy6WNTpdTyanJKj5arLyJeZzyBwAArhq+2AbcgALwMZm2TNkKbYrvFF8mpCTJz/BTz049tf/YfmXaMr00IQAAACRiCvA59mK7JMlaz1ru82eXn10PAAAA3kFMAT7GGvRzLB0pP5bOLj+7HgAAALyDmAJ8TGxkrCJDIpWWlSany1nmOafLqfSsdEXViVJsZKyXJgQAAIBETAE+x+Jn0YI+C7Qrf5eSU5OVb8/XidITyrfnKzk1WbvydympdxI3nwAAAPAy7uYH+Kjyvmcqqk6UknoncVt0AABw1fHFNiCmAB/mcDqUacuUvdgua5BVsZGxHJECAABXJV9sA760F/BhFj+L4qLjvD0GAAAAysE1UwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACZ4NKZeeOEFxcTEKDg4WMHBwerSpYtWr17tyV0CAAAAQIXwaEw1atRIc+fO1fbt25WVlaVbbrlFAwcO1K5duzy5WwAAAADwOMPlcrkqcoehoaF66qmnNGbMmIuuW1RUpJCQEBUWFio4OLgCpgMAAADgi3yxDapV1I4cDofee+89HT9+XF26dCl3nZMnT+rkyZPux0VFRRU1HgAAAABcFo/fgGLnzp0KDAyUv7+/7r33Xq1cuVJt2rQpd905c+YoJCTE/RMREeHp8QAAAADAFI+f5ldaWiqbzabCwkKtWLFCr7zyitavX19uUJV3ZCoiIsKnDuUBAAAAqHi+eJpfhV8zFR8fr6ZNm+rFF1+86Lq++AsDAAAAUPF8sQ0q/HumnE5nmaNPAAAAAFAZefQGFA8//LD69eunyMhIFRcX66233lJGRobWrFnjyd0CAAAAgMd5NKYKCgo0YsQI2e12hYSEKCYmRmvWrFGvXr08uVsAAAAA8DiPxtSrr77qyc0DAAAAgNdU+DVTAAAAAFAVEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmEFMAAAAAYAIxBQAAAAAmXPUxFRcXp8TERI/uIzo6WgsXLvToPgAAAABUrKs+pgAAAADADGIKAAAAAEwgpiQ5nU5NnTpVoaGhCg8P18yZM93PHTt2TPfcc48aNGig4OBg3XLLLcrJyXE/n5eXp4EDByosLEyBgYG68cYblZaW5oV3AQAAAKAiEVOSli1bptq1a2vLli2aN2+eZs+erbVr10qShg4dqoKCAq1evVrbt29Xhw4d1LNnT/3444+SpJKSEt16661KT0/Xjh071LdvXw0YMEA2m82bbwkAAACAhxkul8vl7SHOp6ioSCEhISosLFRwcLBH9hEXFyeHw6HMzEz3ss6dO+uWW27Rbbfdpv79+6ugoED+/v7u55s1a6apU6dq3Lhx5W7zuuuu07333qsJEyZIOnMDisTERI/f6AIAAACoqiqiDS5XNW8P4AtiYmLKPLZarSooKFBOTo5KSkpUr169Ms//9NNPysvLk3TmyNTMmTP1r3/9S3a7XadPn9ZPP/3EkSkAAACgiiOmJFWvXr3MY8Mw5HQ6VVJSIqvVqoyMjHNeU6dOHUnSlClTtHbtWs2fP1/NmjVTQECAhgwZotLS0gqYHAAAAIC3EFMX0KFDBx06dEjVqlVTdHR0uets3LhRo0aN0uDBgyWdOVK1b9++ihsSAAAAgFdwA4oLiI+PV5cuXTRo0CB98skn2rdvnzZt2qRHH31UWVlZkqTmzZsrJSVF2dnZysnJ0Z/+9Cc5nU4vTw4AAADA04ipCzAMQx999JG6d++u0aNHq0WLFvrjH/+o/fv3KywsTJKUlJSkunXrqmvXrhowYID69OmjDh06eHlyAAAAAJ521d/NDwAAAIDv88U24MgUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJjAl/ZeAofToUxbpuzFdlmDrIqNjJXFz+LtsQAAAAB4ETF1ESm5KZq0ZpJshTb3ssiQSC3os0AJrRO8OBkAAAAAbyKmLiAlN0VDlg9R28Ztldg7UdZ6VtmP2JWWlaYhy4doxbAVBBUAAABwleJLe8/D4XSoyTNNFBwarLtvu1t+xi+XlzldTiWnJqv4aLHyJuZxyh8AAADgYXxpbyWSacuUrdCm+E7xZUJKkvwMP/Xs1FP7j+1Xpi3TSxMCAAAA8CZi6jzsxXZJkrWetdznzy4/ux4AAACAqwsxdR7WoJ9j6Uj5sXR2+dn1AAAAAFxdiKnziI2MVWRIpNKy0uR0Ocs853Q5lZ6Vrqg6UYqNjPXShAAAAAC8iZg6D4ufRQv6LNCu/F1KTk1Wvj1fJ0pPKN+er+TUZO3K36Wk3kncfAIAAAC4SnE3v4so73umoupEKal3ErdFBwAAACqIL7TBbxFTl8DhdCjTlil7sV3WIKtiI2M5IgUAAABUIF9pg1/jS3svgcXPorjoOG+PAQAAAOAiXC6XHA6HqlXzfOpwzRQAAAAAr4mLi9Nf//pXTZ06VaGhoQoPD9fMmTMlSfv27ZNhGMrOznavf+zYMRmGoYyMDElSRkaGDMPQ6tWr1bFjR/n7++uzzz5TTk6OevTooaCgIAUHB6tjx47Kyspyb+ezzz5TbGysAgICFBERob/+9a86fvz4Zc1OTAEAAADwqmXLlql27drasmWL5s2bp9mzZ2vt2rWXtY1p06Zp7ty5ys3NVUxMjO688041atRI27Zt0/bt2zVt2jRVr15dkpSXl6e+ffvq9ttv15dffql3331Xn332mSZMmHBZ++Q0PwAAAABeFRMToxkzZkiSmjdvrsWLFys9PV3Nmze/5G3Mnj1bvXr1cj+22Wz629/+platWrm3e9acOXN05513KjEx0f3cM888o5tvvlkvvPCCataseUn75MgUAAAAAK+KiYkp89hqtaqgoOCyttGpU6cyjydPnqx77rlH8fHxmjt3rvLy8tzP5eTkaOnSpQoMDHT/9OnTR06nU/n5+Ze8T2IKAAAAgFedPf3uLMMw5HQ65ed3Jld+fQPy06dPl7uN2rVrl3k8c+ZM7dq1S/3799enn36qNm3aaOXKlZKkkpIS/eUvf1F2drb7JycnR998842aNm16yXNzmh8AAAAAn9SgQQNJkt1ud0fOl19+ecmvb9GihVq0aKFJkyZp+PDheu211zR48GB16NBB//73v9WsWbPfNR9HpgAAAAD4pICAAP3P//yP5s6dqz179kiSHn/88Yu+7qefftKECROUkZGh/fv3a+PGjdq2bZtat24tSXrooYe0adMmTZgwQdnZ2frmm2/0wQcfXPYNKIgpAAAAAD4rOTlZp0+f1s033yxJ+vvf/37R11gsFh05ckQjRoxQixYtNGzYMPXr10+zZs2SdOYarfXr1+vrr79WbGysbrjhBk2fPl3XXHPNZc1muH59AqKP8cVvOQYAAABQ8XyxDTgyBQAAAAAmEFMAAAAAYAIxBQAAAAAmcGt0AAAAAOdwOB3KtGXKXmyXNciq2MhYWfws3h7LpxBTAAAAAMpIyU3RpDWTZCu0uZdFhkRqQZ8FSmid4MXJfAsxBQAAAMAtJTdFQ5YPUdvGbZXYO1HWelbZj9iVlpWmIcuHaMWwFQTVz7g1OgAAAABJZ07ta/JMEwWHBuvu2+6Wn/HLLRacLqeSU5NVfLRYeRPzKvyUP19sA25AAQAAAECSlGnLlK3QpvhO8WVCSpL8DD/17NRT+4/tV6Yt00sT+hZiCgAAAIAkyV5slyRZ61nLff7s8rPrXe2IKQAAAACSJGvQz7F0pPxYOrv87HpXO2IKAAAAgCQpNjJWkSGRSstKk9PlLPOc0+VUela6oupEKTYy1ksT+hZiCgAAAIAkyeJn0YI+C7Qrf5eSU5OVb8/XidITyrfnKzk1WbvydympdxLfN/Uz7uYHAAAAoIzyvmcqqk6Uknonee226L7YBsQUAAAAgHM4nA5l2jJlL7bLGmRVbGSsV49I+WIb8KW9AAAAAM5h8bMoLjrO22P4NK6ZAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMMGjMTVnzhzdeOONCgoKUsOGDTVo0CDt2bPHk7sEAAAAgArh0Zhav369xo8fr88//1xr167VqVOn1Lt3bx0/ftyTuwUAAAAAj6vQ75k6fPiwGjZsqPXr16t79+4XXd8X7yUPAAAAoOL5YhtU6PdMFRYWSpJCQ0PLff7kyZM6efKk+3FRUVGFzAUAAAAAl6vCbkDhdDqVmJiobt266brrrit3nTlz5igkJMT9ExERUVHjAQAAAMBlqbDT/O677z6tXr1an332mRo1alTuOuUdmYqIiPCpQ3kAAAAAKt5Ve5rfhAkTlJqaqg0bNpw3pCTJ399f/v7+FTESAAAAAPwuHo0pl8uliRMnauXKlcrIyFDjxo09uTsAAAAAqDAejanx48frrbfe0gcffKCgoCAdOnRIkhQSEqKAgABP7hoAAAAAPMqj10wZhlHu8tdee02jRo266Ot98bxIAAAAABXPF9vA46f5AQAAAEBVVGG3RgcAAACAqoSYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAuArFxcUpMTHR22MAlRoxBQAAgN+FMMPVipgCAACATygtLfX2CMBlIaYAAACquOPHj2vEiBEKDAyU1WrV008/Xeb5o0ePasSIEapbt65q1aqlfv366ZtvvimzzsaNGxUXF6datWqpbt266tOnj44ePapRo0Zp/fr1WrRokQzDkGEY2rdvnyRp/fr16ty5s/z9/WW1WjVt2jSdPn3avc24uDhNmDBBiYmJql+/vvr06ePx3wVwJRFTAAAAVdzf/vY3rV+/Xh988IE++eQTZWRk6IsvvnA/P2rUKGVlZenDDz/U5s2b5XK5dOutt+rUqVOSpOzsbPXs2VNt2rTR5s2b9dlnn2nAgAFyOBxatGiRunTporFjx8put8tutysiIkLfffedbr31Vt14443KycnRCy+8oFdffVWPP/54mdmWLVumGjVqaOPGjVqyZEmF/l6A36uatwcAAACA55SUlOjVV1/VG2+8oZ49e0o6EzCNGjWSJH3zzTf68MMPtXHjRnXt2lWS9OabbyoiIkKrVq3S0KFDNW/ePHXq1EnPP/+8e7tt27Z1/7lGjRqqVauWwsPD3cuef/55RUREaPHixTIMQ61atdLBgwf10EMPafr06fLzO/N3+s2bN9e8efM8/nsAPIEjUwAAAFVYXl6eSktLddNNN7mXhYaGqmXLlpKk3NxcVatWrczz9erVU8uWLZWbmyvplyNTlyM3N1ddunSRYRjuZd26dVNJSYm+/fZb97KOHTuael+ALyCmAAAAcEEBAQEe23bt2rU9tm3A04gpAACAKqxp06aqXr26tmzZ4l529OhRff3115Kk1q1b6/Tp02WeP3LkiPbs2aM2bdpIkmJiYpSenn7efdSoUUMOh6PMstatW7uvvzpr48aNCgoKcp9iCFR2xBQAAEAVFhgYqDFjxuhvf/ubPv30U3311VcaNWpUmWuWBg4cqLFjx+qzzz5TTk6O/vznP+vaa6/VwIEDJUkPP/ywtm3bpvvvv19ffvmldu/erRdeeEE//PCDJCk6OlpbtmzRvn379MMPP8jpdOr+++/XgQMHNHHiRO3evVsffPCBZsyYocmTJ7v3DVR2/JMMAADggxxOhzL2ZejtnW8rY1+GHE7HxV90Hk899ZRiY2M1YMAAxcfH63//93/LXKv02muvqWPHjrrtttvUpUsXuVwuffTRR6pevbokqUWLFvrkk0+Uk5Ojzp07q0uXLvrggw9UrdqZe5lNmTJFFotFbdq0UYMGDWSz2XTttdfqo48+0tatW9W+fXvde++9GjNmjP7+97//vl8M4EMM16+PvfqYoqIihYSEqLCwUMHBwd4eBwAAoEKk5KZo0ppJshXa3MsiQyK1oM8CJbRO8OJkgPf4Yhtwa3QAAAAfkpKboiHLh6ht47ZK7J0oaz2r7EfsSstK05DlQ7Ri2AqCCvARHJkCAADwEQ6nQ02eaaLg0GDdfdvd8jN+uSLD6XIqOTVZxUeLlTcxTxY/ixcnBSqeL7YB10wBAAD4iExbpmyFNsV3ii8TUpLkZ/ipZ6ee2n9svzJtmV6aEMCvEVMAAAA+wl5slyRZ61nLff7s8rPrAfAuYgoAAMBHWIN+jqUj5cfS2eVn1wPgXcQUAACAj4iNjFVkSKTSstLkdDnLPOd0OZWela6oOlGKjYz10oQAfo2YAgAA8BEWP4sW9FmgXfm7lJyarHx7vk6UnlC+PV/Jqcnalb9LSb2TuPkE4CO4mx8AAICPKe97pqLqRCmpdxK3RcdVyxfbgJgCAADwQQ6nQ5m2TNmL7bIGWRUbGcsRKVzVfLEN+NJeAAAAH2TxsyguOs7bYwC4AK6ZAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMIGYAgAAAAATiCkAAAAAMMGjMbVhwwYNGDBA11xzjQzD0KpVqzy5OwAAAACoMB6NqePHj6t9+/Z67rnnPLkbAAAAAKhw1Ty58X79+qlfv36XvP7Jkyd18uRJ9+OioiJPjAUAAAAAv5tPXTM1Z84chYSEuH8iIiK8PRIAAAAAlMunYurhhx9WYWGh++fAgQPeHgkAAAAAyuXR0/wul7+/v/z9/b09BgAAAABclE8dmQIAAACAyoKYAgAAAAATPHqaX0lJifbu3et+nJ+fr+zsbIWGhioyMtKTuwYAAAAAj/JoTGVlZalHjx7ux5MnT5YkjRw5UkuXLvXkrgEAAADAozwaU3FxcXK5XJ7cBQAAAAB4BddMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMAQAAAIAJxBQAAAAAmEBMSYqLi1NiYmKF7c8wDK1atarC9gcAAADgyiOmPGjmzJm6/vrrz1lut9vVr1+/ih8IAAAAwBVTzdsDVEUul0sOh+O8z4eHh1fgNAAAAAA8gSNTP3M6nZo6dapCQ0MVHh6umTNnSpL27dsnwzCUnZ3tXvfYsWMyDEMZGRmSpIyMDBmGodWrV6tjx47y9/fXG2+8oVmzZiknJ0eGYcgwDC1dulRS2dP8zm4/JSVFPXr0UK1atdS+fXtt3ry5zHwvv/yyIiIiVKtWLQ0ePFhJSUmqU6eOZ38pAAAAAM6LmPrZsmXLVLt2bW3ZskXz5s3T7NmztXbt2svaxrRp0zR37lzl5uaqV69eevDBB9W2bVvZ7XbZ7Xbdcccd533to48+qilTpig7O1stWrTQ8OHDdfr0aUnSxo0bde+99+qBBx5Qdna2evXqpSeeeOJ3vV8AAAAAvw+n+f0sJiZGM2bMkCQ1b95cixcvVnp6upo3b37J25g9e7Z69erlfhwYGKhq1apd0ml9U6ZMUf/+/SVJs2bNUtu2bbV37161atVKzz77rPr166cpU6ZIklq0aKFNmzYpNTX1ct4iAAAAgCuII1M/i4mJKfPYarWqoKDA/djhdChjX4be3vm2PrN9Vu42OnXqdEX2b7VaJcm9/z179qhz585l1v/tYwAAAAAViyNTP6tevXqZx4ZhyOl0ys/vTG/e9tZtOhR06MyTx8/8x4b9GxSnOPdrateufUX2bxiGpDPXcQEAAADwTRyZuoiNRzZKkvxd/kocmqh/3vtPDW4xWJI0Y90MpeSmnPe1NWrUuOBd/S5Vy5YttW3btjLLfvv4auZyuTRu3DiFhoaec7OQ8pR3U5HyVPT3jwEAAKByIaYuwOVyadr6aQpoGiDLZotqFtXUgW0HtP2F7ZKkaGu0Jn8yWQ5n+cEUHR2t/Px8ZWdn64cfftDJkydNzTFx4kR99NFHSkpK0jfffKMXX3xRq1evdh/Butp9/PHHWrp0qVJTU2W323XdddddcP2IiIgy6529G+OxY8cqYFoAAABUFcTUBXx//HvZCm0aumConKedevqWp7XykZXq/+iZG0V0bNlR+4/t186CneW+/vbbb1ffvn3Vo0cPNWjQQG+//bapObp166YlS5YoKSlJ7du318cff6xJkyapZs2apt9bVZKXlyer1aquXbsqPDxc1apd+OxVi8VySesBAAAAF2K4XC6Xt4c4n6KiIoWEhKiwsFDBwcEVvv+3d76tP6X8Sf+895/yr+F/zvMnSk9o2pJpeivhLQ1vN7xCZxs7dqx2796tzMzMCt2vrxk1apSWLVvmfhwVFaUlS5bo8ccf11dffSWLxaIuXbpo0aJFatq0qaQzp/k1btxYO3bsUJ06ddS4ceMy2xw5cqSWLl2quLg4xcTEqGbNmnrllVdUo0YN3Xvvve7vIAMAAEDF8XYblIcjUxdgDTpzVz37EXu5z59dfnY9T5o/f75ycnK0d+9ePfvss1q2bJlGjhzp8f36ukWLFmn27Nlq1KiR7Ha7tm3bpuPHj2vy5MnKyspSenq6/Pz8NHjw4HJv6BEREaH3339f0pm7Jtrtdi1atMj9/JX4/jEAAABUTcTUBcRGxioyJFJpWWlyusr+H3Gny6n0rHRF1YlSbGSsx2fZunWrevXqpXbt2mnJkiV65plndM8993h8v74uJCREQUFB7lP3GjRooEGDBym0Q6i2/bRNx+oc08uvvKydO3fq3//+9zmvt1gsCg0NlSQ1bNhQ4eHhCgkJcT9/9vvHmjdvrhEjRqhTp05KT0+vsPcHAAAA31UhF40899xzeuqpp3To0CG1b99ezz77bKX4niSLn0UL+izQkOVDlJyarJ6despazyr7EbvSs9K1K3+XVgxbIYufxeOzLF++3OP7qApSclM04Y0Jsn9ol76V9F/J0JkbddhstovenOK3Lvb9YwAAALh6eTym3n33XU2ePFlLlizRTTfdpIULF6pPnz7as2ePGjZs6Ond/24JrRO0YtgKTVozSYve++X0r6g6UVoxbIUSWid4cTr8WkpuioYsH6Lqr1VXZONI9Xm5j076n9SmnZu099G9Wp+3Xrfq1sva5vm+fwwAAADweEwlJSVp7NixGj16tCRpyZIl+te//qXk5GRNmzatzLonT54sc/vwoqIiT493SRJaJ2hgy4HKtGXKXmyXNciq2MjYCjkihUs3ac0ktWzQUrvtuzXwlYFq2uXMDSeCLcFarMV6Lfs1Pel88pzX1ahRQ5KuyHeCAQAA4Orh0WumSktLtX37dsXHx/+yQz8/xcfHa/PmzeesP2fOHIWEhLh/IiIiPDneZbH4WRQXHafh7YYrLjqOkPIxJ06fkK3Qpj7d+6h2aG1tXrZZh/9zWF9v+Fof/uNDSdLh44eVaTv37odRUVEyDEOpqak6fPiwSkpKKnp8AAAAVEIejakffvhBDodDYWFhZZaHhYXp0KFD56z/8MMPq7Cw0P1z4MABT46HKsThOnNU6ZoG12jEKyN0IPuA/tntn1r16Cr9YdYf3OvZi8+9M+O1116rWbNmadq0aQoLC9OECRMqbG4AAABUXh79nqmDBw/q2muv1aZNm9SlSxf38qlTp2r9+vXasmXLBV/vi/eSh2/K2JehHst6KHFooqKt0ec8n2/P16L3FmndyHWKi46r8PkAAADw+/hiG3j0yFT9+vVlsVj0/fffl1n+/fffKzw83JO7xlXGl25jDwAAgKuDR2OqRo0a6tixY5nv5XE6nUpPTy9zpAr4vc7exn5X/i4lpyYr356vE6UnlG/PV3Jqsnbl71JS7ySudQMAAMAV4/G7+U2ePFkjR45Up06d1LlzZy1cuFDHjx93390PuFK4jT0AAAAqksdj6o477tDhw4c1ffp0HTp0SNdff70+/vjjc25KAVwJ3MYeAAAAFcWjN6D4vXzxIjMAAAAAFc8X28Cj10wBAAAAQFVFTAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAEAAACACcQUAAAAAJhATAE+7L///a9uv/12BQcHyzAMHTt2TNHR0Vq4cOEFX2cYhlatWlUhMwIAAFytqnl7AADnt2zZMmVmZmrTpk2qX7++QkJCtG3bNtWuXdvbowEAAFz1iCnAh+Xl5al169a67rrr3MsaNGjgxYkAAABwFqf5Ab+D0+nUvHnz1KxZM9Xwr6H61vq6Z8o9cjgd2rlzp2655RYFBASoXr16GjdunEpKStyvHTVqlAYNGqT58+fLarWqXr16Gj9+vE6dOiVJiouL09NPP60NGzbIMAzFxcVJ0jmn+X3zzTfq3r27atasqTZt2mjt2rXnzHngwAENGzZMderUUWhoqAYOHKh9+/Zd8iySdPLkST300EOKiIiQv7+/mjVrpldffdX9/FdffaV+/fopMDBQYWFhuuuuu/TDDz9cod80AACA7yGmgN/h4Ycf1mNPPKajnY/q1L2ndKTfEb369auKfipaN/e8WXXr1tW2bdv03nvvKS0tTRMmTCjz+nXr1ikvL0/r1q3TsmXLtHTpUi1dulSSlJKSorFjx6pLly6y2+1KSUk5Z/9Op1MJCQmqUaOGtmzZoiVLluihhx4qs86pU6fUp08fBQUFKTMzUxs3blRgYKD69u2r0tLSS5pFkkaMGKG3335bzzzzjHJzc/Xiiy8qMDBQknTs2DHdcsstuuGGG5SVlaWPP/5Y33//vYYNG3aFftMAAAC+h9P8AJOKi4u1YOECnepzStf1vU4jOo2QtZ5V9iN2LZ+/XEeLj+r2R253n6K3ePFiDRgwQP/85z8VFhYmSapbt64WL14si8WiVq1aqX///kpPT9fYsWMVGhqqWrVqqUaNGgoPDy93hrS0NO3evVtr1qzRNddcI0l68skn1a9fP/c67777rpxOp1555RUZhiFJeu2111SnTh1lZGSod+/eF53l66+/1vLly7V27VrFx8dLkpo0aeLex+LFi3XDDTfoySefdC9LTk5WRESEvv76a7Vo0eJK/doBAAB8BkemAJO+2vWVTpWeUoubW+ju2+5WtDVa/jX8FW2NVvOazVU7urYe+ewROZwOSVK3bt3kdDq1Z88e9zbatm0ri8Xifmy1WlVQUHDJM+Tm5ioiIsIdUpLUpUuXMuvk5ORo7969CgoKUmBgoAIDAxUaGqoTJ04oLy/vkmbJzs6WxWLRzTffXO4cOTk5WrdunXv7gYGBatWqlSSV2QcAAEBVwpEpwKTcY7mSpG7tusnPOPfvJeoG1dX+Y/uVactUXHRcuduoXr16mceGYcjpdF7ROUtKStSxY0e9+eab5zz365tZXGiWgICAi+7j7FG337JarWbGBgAA8HnEFGCSpb5FqiYVfVUk3Vj2ufAW4dr69lapVLIX2yVJGzdulJ+fn1q2bHnFZmjdurUOHDggu93ujpbPP/+8zDodOnTQu+++q4YNGyo4ONjUftq1ayen06n169e7T/P77T7ef/99RUdHq1o1/rUCAACuDpzmB5gUVT9K+l8pdVaqtr6zVT/k/6B92/bp8//7XB2HdpRfdT9ppXTi4AmtW7dOEydO1F133eW+XupKiI+PV4sWLTRy5Ejl5OQoMzNTjz76aJl17rzzTtWvX18DBw5UZmam8vPzlZGRob/+9a/69ttvL2k/0dHRGjlypO6++26tWrXKvY3ly5dLksaPH68ff/xRw4cP17Zt25SXl6c1a9Zo9OjRcjgcV+z9AgAA+BJiCjApNjJWEQMiFNIrRKvnrNac/5mjZWOWqfiHYlULqCbrA1b5n/bX/YPu15AhQ9SzZ08tXrz4is7g5+enlStX6qefflLnzp11zz336IknniizTq1atbRhwwZFRkYqISFBrVu31pgxY3TixInLOlL1wgsvaMiQIbr//vvVqlUrjR07VsePH5ckXXPNNdq4caMcDod69+6tdu3aKTExUXXq1JGfH/+aAQAAVZPhcrlc3h7ifIqKihQSEqLCwkLTpycBnpSSm6Ihy4eobeO26tmpp/tufulZ6dqVv0srhq1QQusEb48JAABQ6fliGxBTwO+UkpuiSWsmyVZocy+LqhOlpN5JhBQAAMAV4ottQEwBV4DD6VCmLVP2YrusQVbFRsbK4me5+AsBAABwSXyxDbjtFnAFWPws5739OQAAAKomrgwHAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABOIKR8RFxenxMREb48BAAAA4BIRU1XczJkzdf3113t7DAAAAKDKqebtAXBxDqdDmbZM2YvtsgZZFRsZK4uf5YKvcblccjgcFTQhAAAAcPXhyJQPcTqdmjp1qkJDQxUeHq6ZM2cqJTdFkTMj1aNxD/3puT+px7IeavJME/3flv+TYRjKyMiQJGVkZMgwDK1evVodO3aUv7+/3njjDc2aNUs5OTkyDEOGYWjp0qVefY8AAABAVcGRKR+ybNkyTZ48WVu2bNHmzZs1atQouf7jUov2LSRJDwx9QIbVUFpWmkasHFHuNqZNm6b58+erSZMmqlmzph588EF9/PHHSktLkySFhIRU2PsBAAAAqjKOTPmQmJgYzZgxQ82bN9edf75T1SOqq/6P9TWs5zBJUvXq1RVtjdbdt92t1lGtJZ05BfDXZs+erV69eqlp06a69tprFRgYqGrVqik8PFzh4eEKCAio8PcFAAAAVEXElA+JiYlx/znTlqnSgFI1rNFQhmGUWc/P8NPNN9wsSdpZsLPMc506dfL8oAAAAACIKV9SvXp195/txXZJUkD1APn5/fwxuX5Zt35QfUnSkf8eKbON2rVre3ZIAAAAAJKIKZ9lDbJKkv578r+qXe9MIBV9X+R+/qvNX0mS6tWqd8Ht1KhRg7v6AQAAAB7ADSh8VGxkrAKqB8h+xK5qNaspqlOU0hamKTQyVEWHi7RmzhpJUruG7S64nejoaOXn5ys7O1uNGjVSUFCQ/P39K+ItAAAAAFUaR6Z8lMXPousaXqdjxceUnJqs7tO76/Sp05rfY76SJyTrp24/ude7kNtvv119+/ZVjx491KBBA7399tsVMT4AAABQ5Rkul8t18dW8o6ioSCEhISosLFRwcLC3x/GKlNwUTVozSbZCm3tZVJ0oJfVOUkLrBC9OBgAAAFQcX2wDYqoScDgdyrRlyl5slzXIqtjI2IsekQIAAACqEl9sA66ZqgQsfhbFRcd5ewwAAAAAv8I1UwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABgAjEFAAAAACYQUwAAAABggsdi6oknnlDXrl1Vq1Yt1alTx1O7AQAAAACv8FhMlZaWaujQobrvvvs8tQsAAAAA8JpqntrwrFmzJElLly695NecPHlSJ0+edD8uKiq60mMBAAAAwBXhU9dMzZkzRyEhIe6fiIgIb48EAAAAAOXyqZh6+OGHVVhY6P45cOCAt0cCAAAAgHJdVkxNmzZNhmFc8Gf37t2mh/H391dwcHCZHwAAAADwRZd1zdSDDz6oUaNGXXCdJk2a/J55AAAAAKBSuKyYatCggRo0aOCpWQAAAACg0vDY3fxsNpt+/PFH2Ww2ORwOZWdnS5KaNWumwMBAT+0WAAAAACqEx2Jq+vTpWrZsmfvxDTfcIElat26d4uLiPLVbAAAAAKgQhsvlcnl7iPMpKipSSEiICgsLuRkFAAAAcBXzxTbwqVujAwAAAEBlQUwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYQEwBAAAAgAnEFAAAAACYUM3bAwAAAM9yOB3KtGXKXmyXNciq2MhYWfws3h4LACo9YgoAgCosJTdFk9ZMkq3Q5l4WGRKpBX0WKKF1ghcnA4DKj5gCAKCKSslN0ZDlQ9S2cVsl9k6UtZ5V9iN2pWWlacjyIVoxbAVBBQC/g+FyuVzeHuJ8ioqKFBISosLCQgUHB3t7HAAAKg2H06EmzzRRcGiw7r7tbvkZv1wm7XQ5lZyarOKjxcqbmMcpfwAqBV9sA25AAQBAFZRpy5St0Kb4TvFlQkqS/Aw/9ezUU/uP7VemLdNLEwJA5UdMAQBQBdmL7ZIkaz1ruc+fXX52vbNSU1NVp04dORwOSVJ2drYMw9C0adPc69xzzz3685//LEl6//331bZtW/n7+ys6OlpPP/10me1FR0fr8ccf14gRIxQYGKioqCh9+OGHOnz4sAYOHKjAwEDFxMQoKyvL/ZojR45o+PDhuvbaa1WrVi21a9dOb7/9dpntxsXF6a9//aumTp2q0NBQhYeHa+bMmSZ+UwBgHjEFAEAVZA36OZaO2Mt9/uzys+udFRsbq+LiYu3YsUOStH79etWvX18ZGRnuddavX6+4uDht375dw4YN0x//+Eft3LlTM2fO1D/+8Q8tXbq0zDYXLFigbt26aceOHerfv7/uuusujRgxQn/+85/1xRdfqGnTphoxYoTOXnlw4sQJdezYUf/617/01Vdfady4cbrrrru0devWMttdtmyZateurS1btmjevHmaPXu21q5da/p3BgCXi2umAACogn7PNVMdO3bU8OHDNWXKFA0ePFg33nijZs2apSNHjqiwsFCNGjXS119/rZkzZ+rw4cP65JNP3K+dOnWq/vWvf2nXrl2SzhyZio2N1f/93/9Jkg4dOiSr1ap//OMfmj17tiTp888/V5cuXWS32xUeHl7u+7ntttvUqlUrzZ8/X9KZI1MOh0OZmb+cpti5c2fdcsstmjt37hX4DQLwNb7YBhyZAgCgCrL4WbSgzwLtyt+l5NRk5dvzdaL0hPLt+UpOTdau/F1K6p1U7s0nbr75ZmVkZMjlcikzM1MJCQlq3bq1PvvsM61fv17XXHONmjdvrtzcXHXr1q3Ma7t166ZvvvnGfZqgJMXExLj/HBYWJklq167dOcsKCgokSQ6HQ4899pjatWun0NBQBQYGas2aNbLZfrm9+2+3K0lWq9W9DQCoCNwaHQCAKiqhdYJWDFuhSWsmadF7i9zLo+pEXfC26HFxcUpOTlZOTo6qV6+uVq1aKS4uThkZGTp69Khuvvnmy5qjevXq7j8bhnHeZU6nU5L01FNPadGiRVq4cKHatWun2rVrKzExUaWlpefd7tntnN0GAFQEYgoAgCosoXWCBrYcqExbpuzFdlmDrIqNjL3g7dDPXje1YMECdzjFxcVp7ty5Onr0qB588EFJUuvWrbVx48Yyr924caNatGghi8X87dY3btyogQMHum9y4XQ69fXXX6tNmzamtwkAnkBMAQBQxVn8LIqLjrvk9evWrauYmBi9+eabWrx4sSSpe/fuGjZsmE6dOuUOrAcffFA33nijHnvsMd1xxx3avHmzFi9erOeff/53zdu8eXOtWLFCmzZtUt26dZWUlKTvv/+emALgc7hmCgAAnOPmm2+Ww+FQXFycJCk0NFRt2rRReHi4WrZsKUnq0KGDli9frnfeeUfXXXedpk+frtmzZ2vUqFG/a99///vf1aFDB/Xp00dxcXEKDw/XoEGDft8bAgAP4G5+AAAAAHyeL7YBR6YAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwARiCgAAAABMIKYAAAAAwIRq3h4AwNXN4XQo05Ype7Fd1iCrYiNjZfGzeHssAACAiyKmAHhNSm6KJq2ZJFuhzb0sMiRSC/osUELrBC9OBgAAcHHEFACvSMlN0ZDlQ9S2cVsl9k6UtZ5V9iN2pWWlacjyIVoxbAVBBQAAfBrXTAGocA6nQ5PWTFLbxm119213Kyo8SqumrtJL3V7SVyO/kjHP0Kh7R8nhdFzyNg3D0KpVqy57lri4OCUmJl726wAAADwWU/v27dOYMWPUuHFjBQQEqGnTppoxY4ZKS0s9tUsAlUSmLVO2QpviO8XLz/BTblqutr69VWPfHqvZubM15uMxKu5WrExb5hXbZ0ZGhgzD0LFjx8osT0lJ0WOPPXbF9gMAAK4eHjvNb/fu3XI6nXrxxRfVrFkzffXVVxo7dqyOHz+u+fPne2q3ACoBe7FdkmStZ5UkHdl3RMFhwWp8U2NJUtO6TaVPf1nPk0JDQz2+DwAAUDV57MhU37599dprr6l3795q0qSJ/vCHP2jKlClKSUnx1C4BVBLWoDMRZT9i15vj39T7D72vo98eVWJooma1n6VnBzwrrf7Vena7+vfvr4CAADVu3FhvvfWWoqOjtXDhwjLb/eGHHzR48GDVqlVLzZs314cffijpzJHyHj16SJLq1q0rwzA0atQoSeee5hcdHa0nn3xSd999t4KCghQZGamXXnqpzH42bdqk66+/XjVr1lSnTp20atUqGYah7OzsK//LAgAAPqtCr5kqLCy84N8Cnzx5UkVFRWV+AFQ9sZGxigyJVFpWmgY9OUj9Hu6nOtfU0ezc2ZqUNklHi48qyD9IsZGxkqQRI0bo4MGDysjI0Pvvv6+XXnpJBQUF52x31qxZGjZsmL788kvdeuutuvPOO/Xjjz8qIiJC77//viRpz549stvtWrRo0Xnne/rpp9WpUyft2LFD999/v+677z7t2bNHklRUVKQBAwaoXbt2+uKLL/TYY4/poYce8sBvCQAA+LoKi6m9e/fq2Wef1V/+8pfzrjNnzhyFhIS4fyIiIipqPAAVyOJn0YI+C7Qrf5fe3vC2jjuPS37SEecRLf98uf7703/VPaq7LH4W7d69W2lpaXr55Zd10003qUOHDnrllVf0008/nbPdUaNGafjw4WrWrJmefPJJlZSUaOvWrbJYLO6/yGnYsKHCw8MVEhJy3vluvfVW3X///WrWrJkeeugh1a9fX+vWrZMkvfXWWzIMQy+//LLatGmjfv366W9/+5tnflEAAMCnXXZMTZs2TYZhXPBn9+7dZV7z3XffqW/fvho6dKjGjh173m0//PDDKiwsdP8cOHDg8t8RgEohoXWCVgxboaIfi7Qhe4OOFR/TovcWqfhosdo2bKtmoc0knTmSVK1aNXXo0MH92mbNmqlu3brnbDMmJsb959q1ays4OLjcI1gX8+vtGIah8PBw93b27NmjmJgY1axZ071O586dL3sfAACg8rvsG1A8+OCD7msNzqdJkybuPx88eFA9evRQ165dz7nu4Lf8/f3l7+9/uSMBqKQSWidoYMuBeuDHB/Tuznf13sj3FBsZq54re5raXvXq1cs8NgxDTqfTa9sBAABV22XHVIMGDdSgQYNLWve7775Tjx491LFjR7322mvy8+NrrQCUZfGzqFloM9WuXltx0XHnPN+yZUudPn1aO3bsUMeOHSWdOW346NGjl7WfGjVqSJIcjkv/7qrytGzZUm+88YZOnjzp/sufbdu2/a5tAgCAysljdfPdd98pLi5OkZGRmj9/vg4fPqxDhw7p0KFDntolgCqoVatWio+P17hx47R161bt2LFD48aNU0BAgAzDuOTtREVFyTAMpaam6vDhwyopKTE1z5/+9Cc5nU6NGzdOubm5WrNmjfvrHi5nHgAAUPl5LKbWrl2rvXv3Kj09XY0aNZLVanX/AMDleP311xUWFqbu3btr8ODBGjt2rIKCgspct3Qx1157rWbNmqVp06YpLCxMEyZMMDVLcHCw/t//+3/Kzs7W9ddfr0cffVTTp0+XpMuaBwAAVH6Gy+VyeXuI8ykqKlJISIgKCwsVHBzs7XEA+Ihvv/1WERERSktLU8+e5q6vupLefPNNjR49WoWFhQoICPD2OAAAVEm+2AaXfc0UAFS0Tz/9VCUlJWrXrp3sdrumTp2q6Ohode/e3SvzvP7662rSpImuvfZa5eTk6KGHHtKwYcMIKQAArjLEFACfd+rUKT3yyCP6z3/+o6CgIHXt2lVvvvnmOXfdqyiHDh3S9OnTdejQIVmtVg0dOlRPPPGEV2YBAADew2l+AAAAAHyeL7YB9yoHAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABO4NTqAq5LD6VCmLVP2YrusQVbFRsbK4mfx9lgAAKASIaYAXHVSclM0ac0k2Qpt7mWRIZFa0GeBEloneHEyAABQmRBTAK4qKbkpGrJ8iNo2bqvE3omy1rPKfsSutKw0DVk+RCuGrSCoAADAJeFLewFcNRxOh5o800TBocG6+7a75Wf8ctmo0+VUcmqyio8WK29iHqf8AQDgY3yxDbgBBYCrRqYtU7ZCm+I7xZcJKUnyM/zUs1NP7T+2X5m2TC9NCAAAKhNiCsBVw15slyRZ61nLLHe5XHo38V291O0laabUP6a/EhMTL2vbhmFo1apVV2ZQAABQKRBTAK4a1qAzEWU/Yi+zPDctV1vf3qrbFt8mPSi98ekbeuyxx67ovjMyMmQYho4dO3ZFtwsAALyHmAJw1YiNjFVkSKTSstLkdDndy4/sO6LgsGDlOnIVFRGlP3T4g4KCgrw4KQAAqAyIKQBXDYufRQv6LNCu/F1KTk1Wvj1fr9/7ut5/6H0d/faovhr5lf4777/qeUvPMqf52e129e/fXwEBAWrcuLHeeustRUdHa+HChWW2/8MPP2jw4MGqVauWmjdvrg8//FCStG/fPvXo0UOSVLduXRmGoVGjRlXQuwYAAJ5CTAG4qiS0TtCKYStU9GORFr23SF80+ULqIVnqWPTK+le0K3vXOa8ZMWKEDh48qIyMDL3//vt66aWXVFBQcM56s2bN0rBhw/Tll1/q1ltv1Z133qkff/xRERERev/99yVJe/bskd1u16JFizz+XgEAgGfxPVMArjoJrRM0sOVAZdoyZS+2a2P1jUr9T6rGdB9zzrq7d+9WWlqatm3bpk6dOkmSXnnlFTVv3vycdUeNGqXhw4dLkp588kk988wz2rp1q/r27avQ0FBJUsOGDVWnTh3PvTkAAFBhiCkAVyWLn0Vx0XGSpO/Tvz/venv27FG1atXUoUMH97JmzZqpbt2656wbExPj/nPt2rUVHBxc7hEsAABQNXCaHwBcIdWrVy/z2DAMOZ3O86wNAAAqO2IKAC6gZcuWOn36tHbs2OFetnfvXh09evSytlOjRg1JksPhuKLzAQAA7yGmAOACWrVqpfj4eI0bN05bt27Vjh07NG7cOAUEBMgwjEveTlRUlAzDUGpqqg4fPqySkhIPTg0AACoCMQUAF/H6668rLCxM3bt31+DBgzV27FgFBQWpZs2al7yNa6+9VrNmzdK0adMUFhamCRMmeHBiAABQEQyXy+Xy9hDnU1RUpJCQEBUWFio4ONjb4wCAJOnbb79VRESE0tLS1LNnT2+PAwDAVcEX24C7+QHARXz66acqKSlRu3btZLfbNXXqVEVHR6t79+7eHg0AAHgRMQUAF3Hq1Ck98sgj+s9//qOgoCB17dpVb7755jl37wMAAFcXTvMDAAAA4PN8sQ24AQUAAAAAmEBMAQAAAIAJxBQAAAAAmMANKAAA8BKH06FMW6bsxXZZg6yKjYyVxc/i7bEAAJeImAIAwAtSclM0ac0k2Qpt7mWRIZFa0GeBEloneHEyAMClIqYAAKhgKbkpGrJ8iNo2bqvE3omy1rPKfsSutKw0DVk+RCuGrSCoAKAS4NboAABUIIfToSbPNFFwaLDuvu1u+Rm/XL7sdDmVnJqs4qPFypuYxyl/APArvtgG3IACAIAKlGnLlK3QpvhO8WVCSpL8DD/17NRT+4/tV6Yt00sTAgAuFTEFAEAFshfbJUnWetZynz+7/Ox6AADfRUwBAFCBrEE/x9KR8mPp7PKz6wEAfBcxBQBABYqNjFVkSKTSstLkdDnLPOd0OZWela6oOlGKjYz10oQAgEtFTAEAUIEsfhYt6LNAu/J3KTk1Wfn2fJ0oPaF8e76SU5O1K3+XknoncfMJAKgEiCkAACpYQusErRi2QkU/FmnRe4s0bck0LXpvkYqPFl/wtuhxcXGaOHGiEhMTVbduXYWFhenll1/W8ePHNXr0aAUFBalZs2ZavXq1JMnhcGjMmDFq3LixAgIC1LJlSy1atKjMNkeNGqVBgwZp/vz5slqtqlevnsaPH69Tp055/PcAAJUd3zMFAIAXJLRO0MCWA5Vpy5S92C5rkFWxkbEXPSK1bNkyTZ06VVu3btW7776r++67TytXrtTgwYP1yCOPaMGCBbrrrrtks9lUvXp1NWrUSO+9957q1aunTZs2ady4cbJarRo2bJh7m+vWrZPVatW6deu0d+9e3XHHHbr++us1duxYT/8aAKBS43umAACoJOLi4uRwOJSZeea26Q6HQyEhIUpISNDrr78uSTp06JCsVqs2b96s//mf/zlnGxMmTNChQ4e0YsUKSWeOTGVkZCgvL08Wy5mQGzZsmPz8/PTOO+9U0DsDgIvzxTbgyBQAAJVITEyM+88Wi0X16tVTu3bt3MvCwsIkSQUFBZKk5557TsnJybLZbPrpp59UWlqq66+/vsw227Zt6w4pSbJardq5c6cH3wUAVA1cMwUAQCVSvXr1Mo8NwyizzDAMSZLT6dQ777yjKVOmaMyYMfrkk0+UnZ2t0aNHq7S09KLbdDrL3mkQAHAujkwBAFBFbdy4UV27dtX999/vXpaXl+fFiQCgauHIFAAAVVTz5s2VlZWlNWvW6Ouvv9Y//vEPbdu2zdtjAUCVQUwBAFBF/eUvf1FCQoLuuOMO3XTTTTpy5EiZo1QAgN+Hu/kBAAAA8Hm+2AYcmQIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADCBmAIAAAAAE4gpAAAAADChmrcHuBCXyyVJKioq8vIkAAAAALzpbBOcbQRf4NMxVVxcLEmKiIjw8iQAAAAAfEFxcbFCQkK8PYYkyXD5Utr9htPp1MGDBxUUFCTDMLw9jk8pKipSRESEDhw4oODgYG+Pgwvgs6oc+JwqDz6ryoHPqfLgs6oc+JzOHJEqLi7WNddcIz8/37hayaePTPn5+alRo0beHsOnBQcHX7X/haps+KwqBz6nyoPPqnLgc6o8+Kwqh6v9c/KVI1Jn+UbSAQAAAEAlQ0wBAAAAgAnEVCXl7++vGTNmyN/f39uj4CL4rCoHPqfKg8+qcuBzqjz4rCoHPiff5NM3oAAAAAAAX8WRKQAAAAAwgZgCAAAAABOIKQAAAAAwgZgCAAAAABOIKQAAAAAwgZiqAvbt26cxY8aocePGCggIUNOmTTVjxgyVlpZ6ezT8xhNPPKGuXbuqVq1aqlOnjrfHwa8899xzio6OVs2aNXXTTTdp69at3h4Jv7FhwwYNGDBA11xzjQzD0KpVq7w9EsoxZ84c3XjjjQoKClLDhg01aNAg7dmzx9tj4TdeeOEFxcTEKDg4WMHBwerSpYtWr17t7bFwEXPnzpVhGEpMTPT2KPgZMVUF7N69W06nUy+++KJ27dqlBQsWaMmSJXrkkUe8PRp+o7S0VEOHDtV9993n7VHwK++++64mT56sGTNm6IsvvlD79u3Vp08fFRQUeHs0/Mrx48fVvn17Pffcc94eBRewfv16jR8/Xp9//rnWrl2rU6dOqXfv3jp+/Li3R8OvNGrUSHPnztX27duVlZWlW265RQMHDtSuXbu8PRrOY9u2bXrxxRcVExPj7VHwK3zPVBX11FNP6YUXXtB//vMfb4+CcixdulSJiYk6duyYt0eBpJtuukk33nijFi9eLElyOp2KiIjQxIkTNW3aNC9Ph/IYhqGVK1dq0KBB3h4FF3H48GE1bNhQ69evV/fu3b09Di4gNDRUTz31lMaMGePtUfAbJSUl6tChg55//nk9/vjjuv7667Vw4UJvjwVxZKrKKiwsVGhoqLfHAHxeaWmptm/frvj4ePcyPz8/xcfHa/PmzV6cDKgaCgsLJYn/TfJhDodD77zzjo4fP64uXbp4exyUY/z48erfv3+Z/62Cb6jm7QFw5e3du1fPPvus5s+f7+1RAJ/3ww8/yOFwKCwsrMzysLAw7d6920tTAVWD0+lUYmKiunXrpuuuu87b4+A3du7cqS5duujEiRMKDAzUypUr1aZNG2+Phd9455139MUXX2jbtm3eHgXl4MiUD5s2bZoMw7jgz2//z953332nvn37aujQoRo7dqyXJr+6mPmcAOBqMH78eH311Vd65513vD0KytGyZUtlZ2dry5Ytuu+++zRy5Ej9+9//9vZY+JUDBw7ogQce0JtvvqmaNWt6exyUgyNTPuzBBx/UqFGjLrhOkyZN3H8+ePCgevTooa5du+qll17y8HQ463I/J/iW+vXry2Kx6Pvvvy+z/Pvvv1d4eLiXpgIqvwkTJig1NVUbNmxQo0aNvD0OylGjRg01a9ZMktSxY0dt27ZNixYt0osvvujlyXDW9u3bVVBQoA4dOriXORwObdiwQYsXL9bJkydlsVi8OCGIKR/WoEEDNWjQ4JLW/e6779SjRw917NhRr732mvz8OOhYUS7nc4LvqVGjhjp27Kj09HT3zQycTqfS09M1YcIE7w4HVEIul0sTJ07UypUrlZGRocaNG3t7JFwip9OpkydPensM/ErPnj21c+fOMstGjx6tVq1a6aGHHiKkfAAxVQV89913iouLU1RUlObPn6/Dhw+7n+Nv1n2LzWbTjz/+KJvNJofDoezsbElSs2bNFBgY6N3hrmKTJ0/WyJEj1alTJ3Xu3FkLFy7U8ePHNXr0aG+Phl8pKSnR3r173Y/z8/OVnZ2t0NBQRUZGenEy/Nr48eP11ltv6YMPPlBQUJAOHTokSQoJCVFAQICXp8NZDz/8sPr166fIyEgVFxfrrbfeUkZGhtasWePt0fArQUFB51xvWLt2bdWrV4/rEH0EMVUFrF27Vnv37tXevXvPOZWCO9/7lunTp2vZsmXuxzfccIMkad26dYqLi/PSVLjjjjt0+PBhTZ8+XYcOHdL111+vjz/++JybUsC7srKy1KNHD/fjyZMnS5JGjhyppUuXemkq/NYLL7wgSef8O+2111676CnRqDgFBQUaMWKE7Ha7QkJCFBMTozVr1qhXr17eHg2oVPieKQAAAAAwgQtrAAAAAMAEYgoAAAAATCCmAAAAAMAEYgoAAAAATCCmAAAAAMAEYgoAAAAATCCmAAAAAMAEYgoAAAAATCCmAAAAAMAEYgoAAAAATCCmAAAAAMCE/w+SNeONYb1GdQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "from sklearn.decomposition import PCA\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "#plt.style.use('ggplot')\n",
        "#words = sum([[k] + v for k, v in similar_words.items()], [])\n",
        "#words = ['roi', 'reine']\n",
        "# soit une liste de mots à projeter\n",
        "#words = ['palais', 'église', 'cathédrale', 'monastère', 'route', 'train', 'bateau', 'calèche', 'voiture', 'armée', 'soldat', 'bataille']\n",
        "words = ['man', 'woman', 'doctor', 'nurse', 'faith', 'confidence', 'heal', 'hurt', 'healing', 'hurting', 'fight', 'fighting']\n",
        "\n",
        "wvs = w2v_model[words]\n",
        "\n",
        "# Application de la transformation PCA qui réduit les vecteurs à 2 dimensions\n",
        "pca = PCA(n_components=2)\n",
        "#np.set_printoptions(suppress=True)\n",
        "P = pca.fit_transform(wvs)\n",
        "labels = words\n",
        "\n",
        "# Affichage\n",
        "fig = plt.figure(figsize=(10, 8))\n",
        "plt.scatter(P[:, 0], P[:, 1], c='lightgreen', edgecolors='g')\n",
        "for label, x, y in zip(labels, P[:, 0], P[:, 1]):\n",
        "    plt.annotate(label, xy=(x*1.05, y*1.05), xytext=(0, 0), textcoords='offset points')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_dLIrOt1NomY"
      },
      "source": [
        "## Visualiser les plongements lexicaux en 3D dynamique à l'aide du _projector de tensorflow_\n",
        "\n",
        "Le _[projector de tensorflow](http://projector.tensorflow.org/)_ de tensorflow vous permet de visualiser vos modèles ou des modèles existants.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z5L2DpskNomZ"
      },
      "source": [
        "\n",
        "TODO utiliser un modèle disponible comme Word2Vec 10K, observer les voisins de mots éthiquement incorrects (search puis isolate en haut à droite). Relevez vous des associations douteuses ?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZuUDZHuYjSy6"
      },
      "source": [
        "# Activity 4: Vectorization 3 using existing pretrained word static dense models\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jU90qlkkvCPR"
      },
      "source": [
        "## Load existing pretrain models\n",
        "\n",
        "Le dépôt [gensim-data](https://github.com/RaRe-Technologies/gensim-data) contient quelques corpus et modèles pré-entraînés librement disponibles.\n",
        "Jean-Philippe Fauconnier met des [modèles word2vec à disposition pour le français ](https://fauconnier.github.io/#data).\n",
        "\n",
        "\n",
        "The model size depends on the number of dimensions. Higher dimensions model will take more time to download.\n",
        "\n",
        "TODO select a model (start with a small model). Optionaly, uncomment the code to show info about the available gensim models/datasets.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9b1aeec4-bd4a-4c2f-aebb-2c8d00f0e559",
        "id": "EpYkYVGIvCPc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 1min 42s, sys: 2.03 s, total: 1min 44s\n",
            "Wall time: 1min 45s\n"
          ]
        }
      ],
      "source": [
        "import gensim.downloader as gensim_api\n",
        "\n",
        "# uncomment to show info about available models/datasets\n",
        "# print(json.dumps(gensim_api.info(), indent=4))\n",
        "\n",
        "# load a w2v_model\n",
        "#%time glove_twitter_25_model = gensim_api.load(\"glove-twitter-25\")         # 1193514 vectors # 104 MB\n",
        "#%time glove_twitter_50_model = gensim_api.load(\"glove-twitter-50\")         # 1193514 vectors # 199 MB\n",
        "%time glove_twitter_100_model = gensim_api.load(\"glove-twitter-100\")        # 1193514 vectors # 387 MB\n",
        "#%time glove_wiki_gigaword_100_model = gensim_api.load(\"glove-wiki-gigaword-100\") # 400000  vectors # 128 MB\n",
        "#%time ft_wiki_news_300_model = gensim_api.load(\"fasttext-wiki-news-subwords-300 \") # 999999  vectors # 958 MB\n",
        "#%time w2v_gn_300_model = gensim_api.load('word2vec-google-news-300') # 3000000 vectors # 1662 MB\n",
        "\n",
        "# generic variable in the following codes\n",
        "w2v_model = glove_twitter_100_model"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Build a document vectorizer based on a word dense model"
      ],
      "metadata": {
        "id": "r1x1zNYb3g-V"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Simple w2v methods to learn to handle the API**\n",
        "\n",
        "TODO Allign each instruction with the right comment"
      ],
      "metadata": {
        "id": "s9Kjeau_kmIX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "a_word = \"king\"\n",
        "\n",
        "# comments\n",
        "print ('Model shape (|Vocabulary|,|dimensions|):')\n",
        "print ('A sample of the dictionary of word2index:')\n",
        "print ('Index of a given word \"', a_word, '\":')\n",
        "print ('Vector of the word')\n",
        "print ('Vector of the word by direct access to the matrix')\n",
        "print ('Mean of a list of vectors:')\n",
        "print ('List of words similar to the given vector')\n",
        "\n",
        "# instructions\n",
        "print (w2v_model.get_vector('word'))\n",
        "a_mean_vector = w2v_model.get_mean_vector([\"animal\", \"water\"])\n",
        "print (a_mean_vector)\n",
        "print (w2v_model.vectors.shape)\n",
        "print (w2v_model.similar_by_vector( a_mean_vector))\n",
        "print(list(w2v_model.key_to_index.items())[:15])\n",
        "print(w2v_model.vectors[w2v_model.key_to_index['word']])\n",
        "print(w2v_model.key_to_index[a_word])"
      ],
      "metadata": {
        "id": "wq4VmRnXkxqd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Document vectorizer based on a w2v-like model**\n",
        "\n",
        "TODO implement the vectorizer method using a w2v-like model (word2vec, glove, fastText)\n",
        "\n",
        "The basic idea of the algorithm is to parse the list of documents to vectorize. For each document,\n",
        "- tokenize the document\n",
        "- create a list of token present in the dictionary of the model\n",
        "- turn this list of tokens into a list of vectors of these tokens\n",
        "- compute the mean of these vectors\n",
        "the resulting mean will be used to represent the document"
      ],
      "metadata": {
        "id": "BcKIuKvVw8lB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def w2v_vectorizer(w2v_model, X):\n",
        "  \"\"\"\n",
        "  Vectorize each document of the X list.\n",
        "\n",
        "  Args:\n",
        "      param1: the w2v like model to use.\n",
        "      param2: the list of documents (eachone a single string).\n",
        "\n",
        "  Returns:\n",
        "      Description of the return the numpy 'stack' of the document vectors.\n",
        "  \"\"\"\n",
        "  # Your code here\n",
        "  pass\n"
      ],
      "metadata": {
        "id": "iGbPgCBypwjN"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "1. vectorize (backup option)\n",
        "2. train\n",
        "3. predict\n",
        "4. evaluate"
      ],
      "metadata": {
        "id": "Xl4lFBGryFzj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#w2v_model = glove_twitter_100_model\n",
        "\n",
        "# Vectorize\n",
        "%time X_train_w2v_matrix = w2v_vectorizer(w2v_model, X_train)\n",
        "print (X_train_w2v_matrix.shape)\n",
        "\n",
        "%time X_test_w2v_matrix = w2v_vectorizer(w2v_model, X_test)\n",
        "print (X_test_w2v_matrix.shape)\n",
        "\n",
        "print (Y_train_np.shape)\n",
        "\n",
        "# Backup\n",
        "#import pickle\n",
        "#with open('X_train_w2v_matrix.pkl', 'wb') as f:\n",
        "#  pickle.dump(X_train_w2v_matrix, f)\n",
        "#\n",
        "#with open('X_test_w2v_matrix.pkl', 'wb') as f:\n",
        "#  pickle.dump(X_test_w2v_matrix, f)\n",
        "\n",
        "# train\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "LR_w2v_model = LogisticRegression().fit(X_train_w2v_matrix, Y_train_np)\n",
        "\n",
        "# predict\n",
        "Y_test_prediction = LR_w2v_model.predict(X_test_w2v_matrix)\n",
        "\n",
        "# evaluate\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "print(accuracy_score(Y_test,Y_test_prediction))\n",
        "print(classification_report(Y_test,Y_test_prediction))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xoP1AUcXrFqs",
        "outputId": "56b97884-d210-49e3-981a-13a558d29cc6"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [00:27<00:00, 919.54it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 25.3 s, sys: 185 ms, total: 25.5 s\n",
            "Wall time: 27.3 s\n",
            "(25000, 100)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [00:42<00:00, 593.84it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 30.5 s, sys: 247 ms, total: 30.7 s\n",
            "Wall time: 42.2 s\n",
            "(25000, 100)\n",
            "(25000,)\n",
            "0.79068\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.78      0.80      0.79     12500\n",
            "           1       0.80      0.78      0.79     12500\n",
            "\n",
            "    accuracy                           0.79     25000\n",
            "   macro avg       0.79      0.79      0.79     25000\n",
            "weighted avg       0.79      0.79      0.79     25000\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " # Activity 5: Vectorization 4 by pretraining a word static dense model\n"
      ],
      "metadata": {
        "id": "Nl2UZ8TUPIJr"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iqjuUMhvdXhq"
      },
      "source": [
        "Word2Vec et FastText prennent tous deux un corpus normalisé segmenté en phrases et tokenisés en mots.\n",
        "\n",
        "Pour ce faire nous allons utiliser le dataset *unsupervised*.\n",
        "\n",
        "TODO tokeniser le dataset unsupervised\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wZ6uO08_1JQJ",
        "outputId": "d1707fcd-8556-49ed-8678-21a5da4fb3f9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[['preciou', 'littl', 'diamond', 'plai', 'script', 'excel', 'compar', 'movi', 'mayb', 'movi', 'leon', 'wonderfulli', 'plai', 'jean', 'reno', 'natali', 'portman', 'best', 'movi', 'ann', 'parillaud', 'plai', 'franki', 'starlight', 'speak', 'english', 'mean', 'stori', 'young', 'punk', 'girl', 'nikita', 'taken', 'deprav', 'world', 'secret', 'govern', 'forc', 'exception', 'american', 'mind', 'point', 'return', 'especi', 'femm', 'nikita', 'seri', 'compar', 'origin', 'believ', 'trash', 'video', 'bui', 'rent', 'bui', 'btw', 'bewar', 'subtitl', 'compani', 'translat', 'releas', 'disgrac', 'understand', 'french', 'dub', 'version', 'regret', 'later'], ['favourit', 'film', 'time', 'comment', 'taken', 'lightli', 'probabl', 'watch', 'far', 'film', 'healthi', 'love', 'saw', 'femm', 'nikita', 'nearli', 'year', 'ago', 'manag', 'absolut', 'favourit', 'incred', 'stylish', 'sexi', 'thriller', 'luc', 'besson', 'great', 'flair', 'impecc', 'direct', 'fashion', 'appropri', 'usag', 'music', 'make', 'watchabl', 'film', 'ann', 'parillaud', 'perfect', 'render', 'complex', 'charact', 'transform', 'heartless', 'killer', 'compassion', 'vibrant', 'young', 'woman', 'make', 'film', 'beauti', 'ey', 'screen', 'seen', 'luc', 'besson', 'film', 'includ', 'subwai', 'profession', 'irrit', 'fifth', 'element', 'nikita', 'doubt', 'far', 'superior', 'film', 'tragic', 'element', 'ultim', 'extrem', 'hope', 'stori', 'person', 'cruel', 'merciless', 'ultim', 'come', 'realiz', 'human', 'person', 'power', 'extrem', 'inspir', 'hope', 'nikita', 'hope']]\n"
          ]
        }
      ],
      "source": [
        "X_tokenized_unsupervised = None\n",
        "#print (X_tokenized_unsupervised[:2])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UI2LwobJfVqO"
      },
      "source": [
        "Hyper-paramètres les plus communs du constructeur de la [classe Word2Vec](https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.Word2Vec) :\n",
        "\n",
        "* `corpus`: List of tokenized sentences\n",
        "* `size` : Dimensionality of the word vectors (default: 100)\n",
        "* `window` : Maximum distance between the current and predicted word within a sentence\n",
        "* `sg` : Training algorithm: 1 for skip-gram; otherwise CBOW\n",
        "* `iter` :  Number of iterations (epochs) over the corpus\n",
        "* `workers` Use these many worker threads to train the model (=faster training with multicore machines)\n",
        "\n",
        "\n",
        "\n",
        "cette définition peut servir à la fois pour Word2Vec ou FastText\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "id": "dUUD2EMae1y9"
      },
      "outputs": [],
      "source": [
        "# Set values for various parameters\n",
        "lr = 0.05   # Learning rate\n",
        "dim = 100   # Word vector dimensionality\n",
        "ws = 5      # Context window size\n",
        "epoch = 5\n",
        "minCount = 5 # Minimum word count\n",
        "neg = 5\n",
        "loss = 'ns'\n",
        "t = 1e-4\n",
        "#sample = 1e-3   # Downsample setting for frequent words\n",
        "sg=1\n",
        "\n",
        "params = {\n",
        "    'alpha': lr,\n",
        "    'vector_size': dim,\n",
        "    'window': ws,\n",
        "    'epochs': epoch,\n",
        "    'min_count': minCount,\n",
        "    'sample': t,\n",
        "    'sg': 1,\n",
        "    'hs': 0,\n",
        "    'negative': neg\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v1R9Ee81fRw7"
      },
      "source": [
        "Construction des modèles. Observez la rapidité compte tenu du nombre de mots."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c1V66cxT8jyt"
      },
      "source": [
        "## pretrain a w2v\n",
        "\n",
        "peut prendre de 3 à 5 minutes pour 100 à 300 dimensions\n",
        "\n",
        "TODO implementer et exécuter le modèle Word2Vec  sur le corpus unsupervised (tokenisé)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fvJPnIjme657",
        "outputId": "07d74b35-d609-4572-8d82-1bab42fd6c9c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 17min 22s, sys: 2.12 s, total: 17min 24s\n",
            "Wall time: 10min 46s\n"
          ]
        }
      ],
      "source": [
        "from gensim.models import Word2Vec, KeyedVectors\n",
        "\n",
        "#\n",
        "%time w2v_model =  None # TODO\n",
        "\n",
        "# save the model\n",
        "# !mkdir -p models\n",
        "# w2v_model_path = 'models/w2v_imdb_unsupervised_HYPER_PARAM.bin'\n",
        "# w2v_model.save(w2v_model_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vectorize, train, predict and evaluate"
      ],
      "metadata": {
        "id": "rDl2fHH5QQQB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "w2v_model = w2v_model.wv"
      ],
      "metadata": {
        "id": "2k_EkMR12Gra"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Vectorization\n",
        "%time X_train_w2v_matrix = w2v_vectorizer(w2v_model, X_train)\n",
        "%time X_test_w2v_matrix = w2v_vectorizer(w2v_model, X_test)\n",
        "\n",
        "print (X_train_w2v_matrix.shape)\n",
        "print (X_test_w2v_matrix.shape)\n",
        "print (Y_train_np.shape)\n",
        "\n",
        "# train # TODO\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "%time LR_w2v_model = LogisticRegression().fit(X_train_w2v_matrix, Y_train_np)\n",
        "\n",
        "# predict # TODO\n",
        "%time Y_test_prediction = LR_w2v_model.predict(X_test_w2v_matrix)\n",
        "\n",
        "# evaluate # TODO\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "print(accuracy_score(Y_test,Y_test_prediction))\n",
        "print(classification_report(Y_test,Y_test_prediction))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 769
        },
        "outputId": "abb6142a-7e6d-4e81-d170-596f79e9db6c",
        "id": "duGqYpLKQQQD"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "  0%|          | 0/25000 [00:00<?, ?it/s]\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "'Word2Vec' object has no attribute 'get_mean_vector'",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<timed exec>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-41-dace88f4dbd3>\u001b[0m in \u001b[0;36mw2v_vectorizer\u001b[0;34m(w2v_model, X_train)\u001b[0m\n\u001b[1;32m      2\u001b[0m   \u001b[0md2v_matrix\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m   \u001b[0;32mfor\u001b[0m \u001b[0md\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtqdm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m     \u001b[0md2v_matrix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw2v_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_mean_vector\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mw\u001b[0m \u001b[0;32min\u001b[0m \u001b[0md\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mw\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mw2v_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkey_to_index\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      5\u001b[0m   \u001b[0;32mreturn\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvstack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md2v_matrix\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'Word2Vec' object has no attribute 'get_mean_vector'"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "  0%|          | 0/25000 [00:00<?, ?it/s]\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "'Word2Vec' object has no attribute 'get_mean_vector'",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<timed exec>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-41-dace88f4dbd3>\u001b[0m in \u001b[0;36mw2v_vectorizer\u001b[0;34m(w2v_model, X_train)\u001b[0m\n\u001b[1;32m      2\u001b[0m   \u001b[0md2v_matrix\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m   \u001b[0;32mfor\u001b[0m \u001b[0md\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtqdm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m     \u001b[0md2v_matrix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw2v_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_mean_vector\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mw\u001b[0m \u001b[0;32min\u001b[0m \u001b[0md\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mw\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mw2v_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkey_to_index\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      5\u001b[0m   \u001b[0;32mreturn\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvstack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md2v_matrix\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'Word2Vec' object has no attribute 'get_mean_vector'"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(25000, 100)\n",
            "(25000, 100)\n",
            "(25000,)\n",
            "CPU times: user 549 ms, sys: 127 ms, total: 677 ms\n",
            "Wall time: 386 ms\n",
            "CPU times: user 11 ms, sys: 14.8 ms, total: 25.8 ms\n",
            "Wall time: 12.9 ms\n",
            "0.79068\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.78      0.80      0.79     12500\n",
            "           1       0.80      0.78      0.79     12500\n",
            "\n",
            "    accuracy                           0.79     25000\n",
            "   macro avg       0.79      0.79      0.79     25000\n",
            "weighted avg       0.79      0.79      0.79     25000\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Go1MIx3y8qMd"
      },
      "source": [
        "## pretrain FastText\n",
        "\n",
        "Peut prendre pour 100 dimensions 8-15 minutes, 300 dimensions 11-22 minutes, 500 dimensions 23-40 minutes.\n",
        "\n",
        "TODO implementer et exécuter le modèle FastText sur le corpus unsupervised (tokenisé)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P_SvnXRufQk5",
        "outputId": "e08faabf-4c1f-4719-df14-ab5e4d22272a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 39min 12s, sys: 7.47 s, total: 39min 19s\n",
            "Wall time: 23min 30s\n"
          ]
        }
      ],
      "source": [
        "from gensim.models.fasttext import FastText\n",
        "\n",
        "#\n",
        "%time ft_model = None # TODO\n",
        "\n",
        "# save the model\n",
        "#!mkdir -p models\n",
        "#ft_model_path = 'models/ft_imdb_unsupervised_HYPER_PARAM.bin'\n",
        "#ft_model.save(ft_model_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vectorize, train, predict and evaluate"
      ],
      "metadata": {
        "id": "2gqNm1iOEFvF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "w2v_model = ft_model.wv\n",
        "\n",
        "# Vectorization\n",
        "#%time X_train_w2v_matrix = w2v_vectorizer(w2v_model, X_train)\n",
        "#%time X_test_w2v_matrix = w2v_vectorizer(w2v_model, X_test)\n",
        "%time X_train_w2v_matrix = my_w2v_vectorizer(w2v_model, X_train)\n",
        "%time X_test_w2v_matrix = my_w2v_vectorizer(w2v_model, X_test)\n",
        "\n",
        "\n",
        "# train # TODO\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "%time LR_w2v_model = LogisticRegression().fit(X_train_w2v_matrix, Y_train_np)\n",
        "\n",
        "# predict # TODO\n",
        "%time Y_test_prediction = LR_w2v_model.predict(X_test_w2v_matrix)\n",
        "\n",
        "# evaluate # TODO\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "print(accuracy_score(Y_test,Y_test_prediction))\n",
        "print(classification_report(Y_test,Y_test_prediction))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qFqQ3zsr5A_G",
        "outputId": "122b2686-dc3e-49d5-c27d-9c601fdb4341"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [00:30<00:00, 832.59it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 29.2 s, sys: 428 ms, total: 29.6 s\n",
            "Wall time: 30.1 s\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [00:28<00:00, 888.37it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 27.5 s, sys: 340 ms, total: 27.8 s\n",
            "Wall time: 28.2 s\n",
            "CPU times: user 563 ms, sys: 134 ms, total: 696 ms\n",
            "Wall time: 415 ms\n",
            "CPU times: user 53.8 ms, sys: 235 ms, total: 289 ms\n",
            "Wall time: 160 ms\n",
            "0.84536\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.85      0.84      0.84     12500\n",
            "           1       0.84      0.85      0.85     12500\n",
            "\n",
            "    accuracy                           0.85     25000\n",
            "   macro avg       0.85      0.85      0.85     25000\n",
            "weighted avg       0.85      0.85      0.85     25000\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Activity 6: Vectorization 5 using doc2vec\n",
        "\n",
        "La documentation de [doc2vec](https://radimrehurek.com/gensim/models/doc2vec.html) décrit comment construire un modèle doc2vec à partir d'une collection de documents et d'inférer (vectoriser) un document.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "jx-EHios-gMl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO construire un modèle doc2vec à partir de la collection X_unsupervised. Les documents ont besoin des d'être tokenisé en entrée de TaggedDocument. Vous êtes libre de tester différents hyperparamètres (e.g. 100, 300, 500 dimensions et différents min_count de 1 à 5) :  Tokenize the raw data, turn it into TaggedDocument then build a doc2vec model from it"
      ],
      "metadata": {
        "id": "THVY_Mnc-6wv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from gensim.test.utils import common_texts\n",
        "from gensim.models.doc2vec import Doc2Vec, TaggedDocument\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wdSohdMq_dKI",
        "outputId": "fc707f98-040f-413c-b7ea-f6bb0cbbf467"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 8min 7s, sys: 15.7 s, total: 8min 23s\n",
            "Wall time: 6min 21s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vectorize\n",
        "\n",
        "TODO vectorize X_train and X_test documents"
      ],
      "metadata": {
        "id": "wcGNTLVJEbnf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Infer vector for new documents:\n",
        "#\n",
        "X_train_d2v_matrix = list()\n",
        "X_train_d2v_matrix_np = None\n",
        "\n",
        "#\n",
        "X_test_d2v_matrix = list()\n",
        "X_test_d2v_matrix_np = None"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DPmwYf7DEaN6",
        "outputId": "f0373e42-1821-4558-9532-04b928f64664"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 800/800 [00:09<00:00, 85.06it/s] \n",
            "100%|██████████| 200/200 [00:02<00:00, 79.04it/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Train, predict, and evaluate"
      ],
      "metadata": {
        "id": "92pzGdg7Efrx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# train\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "%time LR_w2v_model = LogisticRegression().fit(X_train_d2v_matrix_np, Y_train_np)\n",
        "\n",
        "# predict\n",
        "%time Y_test_prediction = LR_w2v_model.predict(X_test_d2v_matrix_np)\n",
        "\n",
        "# evaluate\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "#print(accuracy_score(Y_test_np,Y_test_prediction))\n",
        "print(classification_report(Y_test_np,Y_test_prediction))"
      ],
      "metadata": {
        "id": "mDI0Av1dEUSl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Appendix\n",
        "\n",
        "**DO NOT READ**"
      ],
      "metadata": {
        "id": "3pUU4xs0-O-S"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Quantization**\n",
        "\n",
        "Quantifier un float32 en float16 ne change pas le temps de la vectorisation ; cela signifie que ce n'est pas l'opération mean qui pose problème ou qu'avec peu de dimensions on ne voit pas la différence en termes de temps de traitement\n",
        "\n",
        "```\n",
        "> print('w2v_model.vectors[0][0]', w2v_model.vectors[0][0], 'type(w2v_model.vectors[0][0]', type(w2v_model.vectors[0][0]))\n",
        "> w2v_model.vectors[0][0] 0.62415 type(w2v_model.vectors[0][0] <class 'numpy.float32'>\n",
        "\n",
        "# quantiser de float32 à float16\n",
        "> w2v_model_vectors_float16 = np.float16(w2v_model.vectors)\n",
        "\n",
        "> print('w2v_model_vectors_int16[0][0]', w2v_model_vectors_int16[0][0], 'type(w2v_model_vectors_int16[0][0]', type(w2v_model_vectors_int16[0][0]))\n",
        "> w2v_model_vectors_float16[0][0] 0.624 type(w2v_model_vectors_float16[0][0] <class 'numpy.float16'>\n",
        "\n",
        "> # changer les poids du modèles\n",
        "> w2v_model.vectors = w2v_model_vectors_float16\n",
        "```\n",
        "\n"
      ],
      "metadata": {
        "id": "tdHSh6ERW5EV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "**Get the size of (in bytes)**\n",
        "```\n",
        "print ('X_train dimensions:', X_train_w2v_gn_300_matrix.shape, 'Taille mémoire', sys.getsizeof(X_train_w2v_gn_300_matrix))\n",
        "print ('X_test dimensions:', X_test_w2v_gn_300_matrix.shape, 'Taille mémoire', sys.getsizeof(X_test_w2v_gn_300_matrix))\n",
        "```\n",
        "\n"
      ],
      "metadata": {
        "id": "OaaevTL1m370"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Comparer et évaluer deux word models**\n",
        "\n",
        "[`gensim` implémente la comparaison de modèles](https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/Word2Vec_FastText_Comparison.ipynb) selon la tâche de **raisonnement analogique** telle que décrite à la [section 4.1 du papier de 2013 de Mikolov et al.](https://arxiv.org/pdf/1301.3781v3.pdf).\n",
        "\n",
        "D'autres évaluations intrinsèques sont possibles comme le [calcul d'un coefficient de corrélation entre un taux de similarité calculée sur la base d'une appréciation humaine et un score de similarité cosinus entre des représentations Word2Vec](https://nlp-ensae.github.io/materials/course2/)."
      ],
      "metadata": {
        "id": "IPFQNrKKqzLF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**tf-idf approach**"
      ],
      "metadata": {
        "id": "T56PPuFbuUiI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# 1. define the vectorizer\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "DIMENSION_SIZE = 1000 # 1000 seems good !\n",
        "%time tfidf_vectorizer = TfidfVectorizer(stop_words = None, max_features=DIMENSION_SIZE)\n",
        "# TODO\n",
        "\n",
        "# 1.1 Learn the vocabulary (fit)\n",
        "# TODO\n",
        "X_unsupervised_tfidf_matrix = tfidf_vectorizer.fit(X_unsupervised_np)\n",
        "# 1.2 transform documents to document-term matrix (i.e. perform the vectorization)\n",
        "# TODO\n",
        "X_train_tfidf_matrix = tfidf_vectorizer.transform(X_train_np)\n",
        "X_test_tfidf_matrix = tfidf_vectorizer.transform(X_test_np)\n",
        "print ('X_train dimensions:', X_train_tfidf_matrix.get_shape(), 'Taille mémoire', sys.getsizeof(X_train_tfidf_matrix))\n",
        "print ('X_test dimensions:', X_test_tfidf_matrix.get_shape(), 'Taille mémoire', sys.getsizeof(X_test_tfidf_matrix))\n",
        "\n",
        "# 2. train\n",
        "# TODO\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "%time LR_tfidf_model = LogisticRegression().fit(X_train_tfidf_matrix, Y_train_np)\n",
        "\n",
        "# 3. predict\n",
        "# TODO\n",
        "%time Y_test_prediction = LR_tfidf_model.predict(X_test_tfidf_matrix)\n",
        "\n",
        "# 4. evaluate\n",
        "# TODO\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "print(accuracy_score(Y_test,Y_test_prediction))\n",
        "print(classification_report(Y_test,Y_test_prediction))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ucaf8bHUuUvr",
        "outputId": "27f69458-3177-4b19-b5d9-624c868ca82d"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 109 µs, sys: 0 ns, total: 109 µs\n",
            "Wall time: 126 µs\n",
            "X_train dimensions: (25000, 1000) Taille mémoire 48\n",
            "X_test dimensions: (25000, 1000) Taille mémoire 48\n",
            "CPU times: user 115 ms, sys: 0 ns, total: 115 ms\n",
            "Wall time: 145 ms\n",
            "CPU times: user 4.88 ms, sys: 0 ns, total: 4.88 ms\n",
            "Wall time: 3.93 ms\n",
            "0.8544\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.86      0.85      0.85     12500\n",
            "           1       0.85      0.86      0.86     12500\n",
            "\n",
            "    accuracy                           0.85     25000\n",
            "   macro avg       0.85      0.85      0.85     25000\n",
            "weighted avg       0.85      0.85      0.85     25000\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observation de biais dans les modèles pré-entraînés**"
      ],
      "metadata": {
        "id": "Lzt1WFzvvopg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "* nationality bias"
      ],
      "metadata": {
        "id": "73bT91kywpyE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "w2v_model.most_similar(['country', 'alcoholic'])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7DcNCs8cvFkT",
        "outputId": "c8097eb3-ab5e-4133-c7ac-f02b3ef2af7c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('drinking', 0.7099121809005737),\n",
              " ('drink', 0.7013658881187439),\n",
              " ('either', 0.6966495513916016),\n",
              " ('drunk', 0.6830408573150635),\n",
              " ('rich', 0.6809152960777283),\n",
              " ('any', 0.677390456199646),\n",
              " ('canadian', 0.6682177782058716),\n",
              " ('there', 0.6656902432441711),\n",
              " ('enough', 0.6633507609367371),\n",
              " ('mexican', 0.6610922813415527)]"
            ]
          },
          "metadata": {},
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* religion bias"
      ],
      "metadata": {
        "id": "eD2PlRL4w36D"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print('', w2v_model.most_similar(positive = ['religious', 'banker']))"
      ],
      "metadata": {
        "id": "YLJ5dG-zw4Ej",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fe76ae4c-eef2-49dc-b8de-efd0cfa2ad65"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " [('jewish', 0.7112376689910889), ('political', 0.709888219833374), ('atheist', 0.6896890997886658), ('politician', 0.6885793805122375), ('lawyer', 0.6672108769416809), ('corporate', 0.6595140695571899), ('rights', 0.6554924249649048), ('catholic', 0.6550735831260681), ('conservative', 0.6545618772506714), ('claims', 0.6487963199615479)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* gender bias"
      ],
      "metadata": {
        "id": "UrcZ_SaNzRQ_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print('', w2v_model.most_similar(positive = ['job', 'woman'], negative = ['man']))\n",
        "print('', w2v_model.most_similar(positive = ['job', 'man'], negative = ['woman']))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wWuwwVWMzRaD",
        "outputId": "027513fa-9ee3-49b2-c40c-fcd5f477a1ed"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " [('jobs', 0.6884946823120117), ('customer', 0.6854056119918823), ('business', 0.6830732822418213), ('hiring', 0.6667071580886841), ('assistant', 0.6455451846122742), ('position', 0.6424741744995117), ('client', 0.6418120861053467), ('nurse', 0.6407575607299805), ('looking', 0.6397485136985779), ('company', 0.6283044219017029)]\n",
            " [('manager', 0.6876481771469116), ('senior', 0.6037998199462891), ('work', 0.6011847257614136), ('jobs', 0.5986695885658264), ('training', 0.5969608426094055), ('bro', 0.5880462527275085), ('good', 0.5779356360435486), ('game', 0.5760307312011719), ('hiring', 0.5741257667541504), ('well', 0.5699031352996826)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Simple w2v methods to learn to handle the API**"
      ],
      "metadata": {
        "id": "ORidNX6LwQSs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print ('Model shape (|Vocabulary|,|dimensions|):')\n",
        "print (w2v_model.vectors.shape)\n",
        "print ('A sample of the dictionary of word2index:')\n",
        "print(list(w2v_model.key_to_index.items())[:15])\n",
        "\n",
        "a_word = \"king\"\n",
        "print ('Index of a given word \"', a_word, '\":')\n",
        "print(w2v_model.key_to_index[a_word])\n",
        "\n",
        "print ('Vector of the word')\n",
        "print (w2v_model.get_vector('word'))\n",
        "\n",
        "print ('Vector of the word by direct access to the matrix')\n",
        "print(w2v_model.vectors[w2v_model.key_to_index['word']])\n",
        "\n",
        "a_mean_vector = w2v_model.get_mean_vector([\"animal\", \"water\"])\n",
        "print ('Mean of a list of vectors:')\n",
        "print (a_mean_vector)\n",
        "\n",
        "print ('List of words similar to the given vector')\n",
        "print (w2v_model.similar_by_vector( a_mean_vector))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g4d2aTVSPlpd",
        "outputId": "50dd1f10-38ae-490d-c934-9a0949c5e779"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model shape (|Vocabulary|,|dimensions|):\n",
            "(1193514, 100)\n",
            "A sample of the dictionary of word2index: [('<user>', 0), ('.', 1), (':', 2), ('rt', 3), (',', 4), ('<repeat>', 5), ('<hashtag>', 6), ('<number>', 7), ('<url>', 8), ('!', 9), ('i', 10), ('a', 11), ('\"', 12), ('the', 13), ('?', 14)]\n",
            "Index of a given word \" king \": 1696\n",
            "Vector of the word [ 0.57479   0.27959  -0.17003   1.0926   -0.5678    0.13946  -0.22845\n",
            "  0.27979   0.1436    0.25408   0.14175   0.47737  -4.1063   -0.45932\n",
            " -0.78775  -0.061295  0.28098   0.55691   0.040097 -0.33675   0.10952\n",
            "  0.32482  -0.60996   0.77837   1.0855    0.092512 -0.34347  -0.52561\n",
            " -0.32974  -0.45062  -0.33763   0.26943  -0.7608   -0.013459 -0.097348\n",
            " -0.40263   0.22523   0.40602   0.34765  -1.2264   -0.81516  -0.57451\n",
            "  0.084248  0.36518   0.24649  -0.26708   0.074     0.73033  -0.34619\n",
            "  0.29964   0.49903   0.46251  -0.68305  -0.92597   0.075895 -0.51661\n",
            " -0.67615  -0.017943 -1.1911   -0.12817   0.27478  -0.77928  -0.35465\n",
            "  0.39712   0.22347   0.38169  -0.067566 -0.24608   0.34249  -0.26701\n",
            " -0.78815  -0.79426  -0.57019   0.14404   0.23621  -0.067121  0.31948\n",
            "  0.06233  -0.3619   -0.012909  0.91253   0.21408   0.12472  -0.64596\n",
            "  0.12799   0.11704  -0.66266   0.31085  -0.3327   -0.2197    0.4885\n",
            "  0.42261  -0.19855  -0.081162  0.015766 -0.12767  -0.062801  0.67944\n",
            "  0.65676  -0.19021 ]\n",
            "Vector of the word by direct access to the matrix [ 0.57479   0.27959  -0.17003   1.0926   -0.5678    0.13946  -0.22845\n",
            "  0.27979   0.1436    0.25408   0.14175   0.47737  -4.1063   -0.45932\n",
            " -0.78775  -0.061295  0.28098   0.55691   0.040097 -0.33675   0.10952\n",
            "  0.32482  -0.60996   0.77837   1.0855    0.092512 -0.34347  -0.52561\n",
            " -0.32974  -0.45062  -0.33763   0.26943  -0.7608   -0.013459 -0.097348\n",
            " -0.40263   0.22523   0.40602   0.34765  -1.2264   -0.81516  -0.57451\n",
            "  0.084248  0.36518   0.24649  -0.26708   0.074     0.73033  -0.34619\n",
            "  0.29964   0.49903   0.46251  -0.68305  -0.92597   0.075895 -0.51661\n",
            " -0.67615  -0.017943 -1.1911   -0.12817   0.27478  -0.77928  -0.35465\n",
            "  0.39712   0.22347   0.38169  -0.067566 -0.24608   0.34249  -0.26701\n",
            " -0.78815  -0.79426  -0.57019   0.14404   0.23621  -0.067121  0.31948\n",
            "  0.06233  -0.3619   -0.012909  0.91253   0.21408   0.12472  -0.64596\n",
            "  0.12799   0.11704  -0.66266   0.31085  -0.3327   -0.2197    0.4885\n",
            "  0.42261  -0.19855  -0.081162  0.015766 -0.12767  -0.062801  0.67944\n",
            "  0.65676  -0.19021 ]\n",
            "Mean of a list of vectors: [-2.84165852e-02 -4.65126485e-02 -5.42706177e-02  2.71266848e-02\n",
            "  4.67017144e-02  2.96881925e-02  8.48957151e-02  2.26917062e-02\n",
            "  7.14888349e-02 -6.32957369e-02  7.81830121e-03  4.79734614e-02\n",
            " -5.34279406e-01  1.99160576e-02  1.65823065e-02  6.06554076e-02\n",
            "  1.08105816e-01 -2.09957492e-02  6.08230792e-02 -6.22347370e-02\n",
            " -4.66787443e-02  4.73911464e-02  3.71996313e-04 -1.04883527e-02\n",
            "  1.08489752e-01 -9.09926742e-02 -1.04809389e-01  2.21639685e-02\n",
            " -9.10187140e-03  8.48755985e-02 -3.94055210e-02 -5.23091070e-02\n",
            " -1.25540748e-01 -7.09626824e-04 -5.61059676e-02 -8.17931518e-02\n",
            "  1.91915892e-02  2.17252336e-02  4.62724529e-02  3.98465469e-02\n",
            " -4.52225059e-02 -2.94087082e-03 -1.03422515e-02 -5.77086285e-02\n",
            "  8.73177797e-02  9.46319252e-02  3.14802900e-02  1.86731130e-01\n",
            "  6.39428273e-02 -7.74758309e-02  9.76051614e-02 -6.43095672e-02\n",
            "  3.84059101e-02 -1.80073678e-02 -4.54078168e-02  4.55794111e-02\n",
            "  5.67157045e-02  6.59534037e-02 -4.76490110e-02 -4.84654680e-04\n",
            " -5.81441894e-02  5.14275543e-02 -3.80062498e-02 -3.30481678e-02\n",
            "  9.33406875e-02 -4.15995978e-02  2.43585408e-02 -3.10825929e-02\n",
            "  5.58519140e-02 -6.40074611e-02  4.92549427e-02 -4.79293428e-02\n",
            "  7.93773960e-03  4.17947806e-02  5.80032542e-02  1.30256508e-02\n",
            " -1.32077187e-01 -3.99520807e-02  3.77940200e-02 -3.99651565e-02\n",
            "  2.06028968e-01  5.16950246e-03 -6.37452751e-02 -3.22060697e-02\n",
            "  7.40248263e-02 -3.95117924e-02  3.48142013e-02 -6.64118975e-02\n",
            " -1.17016062e-01  1.47274107e-01 -1.10579506e-02  1.07272994e-02\n",
            " -7.40909297e-03 -3.57680432e-02  9.85512882e-03 -1.43557340e-02\n",
            "  6.79346127e-03 -3.58869284e-02  3.55782965e-03  1.08046725e-01]\n",
            "List of words similar to the given vector [('water', 0.8271138668060303), ('animal', 0.8271138072013855), ('fish', 0.7437265515327454), ('dog', 0.7357937097549438), ('animals', 0.7304122447967529), ('cat', 0.7167508602142334), ('dogs', 0.7055673003196716), ('food', 0.6977534294128418), ('pet', 0.6904383301734924), ('space', 0.6900535821914673)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**vectorizer method using a w2v-like model (word2vec, glove, fastText)**\n",
        "\n",
        "The basic idea of the algorithm is to parse the list of documents to vectorize. For each document,\n",
        "- tokenize the document\n",
        "- create a list of token present in the dictionary of the model\n",
        "- turn this list of tokens into a list of vectors of these tokens\n",
        "- compute the mean of these vectors\n",
        "the resulting mean will be used to represent the document"
      ],
      "metadata": {
        "id": "T7yVw0cT0oOw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def w2v_vectorizer(w2v_model, X):\n",
        "  \"\"\"\n",
        "  Vectorize each document of the X list.\n",
        "\n",
        "  Args:\n",
        "      param1: the w2v like model to use.\n",
        "      param2: the list of documents (eachone a single string).\n",
        "\n",
        "  Returns:\n",
        "      Description of the return the numpy 'stack' of the document vectors.\n",
        "  \"\"\"\n",
        "  # Your code here\n",
        "  pass\n",
        "  d2v_matrix = list()\n",
        "  for d in tqdm(X):\n",
        "    d2v_matrix.append(w2v_model.get_mean_vector([w for w in d.split() if w in w2v_model.key_to_index.keys()]))\n",
        "  return np.vstack(d2v_matrix)\n"
      ],
      "metadata": {
        "id": "EnSNVthl0oO9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**dense doc vectorizer variant from w2v**"
      ],
      "metadata": {
        "id": "iH8in5b2wpbQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from gensim.models import Word2Vec\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "def w2v_vectorizer(model, documents):\n",
        "  # Create a CountVectorizer to build a document-term matrix\n",
        "  print ('vocabulary size:', len(model.key_to_index.keys()))\n",
        "  vectorizer = CountVectorizer(vocabulary=model.key_to_index.keys()) # Use the model's vocabulary\n",
        "  dtm = vectorizer.fit_transform(documents).toarray()\n",
        "  print (dtm.shape)\n",
        "  #dtm\n",
        "\n",
        "  # Initialize an array to hold document vectors\n",
        "  num_documents = dtm.shape[0]\n",
        "  print ('num_documents:', num_documents)\n",
        "  vector_size = model.vector_size\n",
        "  print ('vector size:', vector_size)\n",
        "  doc_vectors = np.zeros((num_documents, vector_size))\n",
        "  print('doc_vectors.shape', doc_vectors.shape)\n",
        "\n",
        "  # Build document vectors from the DTM\n",
        "  for i in tqdm(range(num_documents)):\n",
        "    # Get the indices of the words in the current document\n",
        "    word_indices = np.nonzero(dtm[i])[0]\n",
        "    #print ('num_documents:', i, '; len(word_indices):', len(word_indices), 'word_indices:',word_indices)\n",
        "\n",
        "    if len(word_indices) > 0:\n",
        "        # Get the corresponding word vectors\n",
        "        word_vecs = model[vectorizer.get_feature_names_out()[word_indices]]\n",
        "        # Average the word vectors\n",
        "        doc_vectors[i] = np.mean(word_vecs, axis=0)\n",
        "  # Now `doc_vectors` contains the averaged word vectors for each document\n",
        "  #print(doc_vectors)\n",
        "  print('doc_vectors.shape', doc_vectors.shape)\n",
        "  return doc_vectors\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0be00c3e-4320-40a8-c3b3-e29a0d89e4af",
        "id": "o13tIgpCwpbV"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "vocabulary size: 39101\n",
            "(800, 39101)\n",
            "num_documents: 800\n",
            "vector size: 100\n",
            "doc_vectors.shape (800, 100)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 800/800 [00:28<00:00, 27.90it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "doc_vectors.shape (800, 100)\n",
            "CPU times: user 28.3 s, sys: 339 ms, total: 28.6 s\n",
            "Wall time: 29 s\n",
            "vocabulary size: 39101\n",
            "(200, 39101)\n",
            "num_documents: 200\n",
            "vector size: 100\n",
            "doc_vectors.shape (200, 100)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 200/200 [00:06<00:00, 29.96it/s]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "doc_vectors.shape (200, 100)\n",
            "CPU times: user 6.68 s, sys: 66.2 ms, total: 6.75 s\n",
            "Wall time: 6.79 s\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F9tqpClk1hab"
      },
      "source": [
        "**tokeniser le dataset unsupervised**\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "758544af-e871-410d-a121-23923b141e4b",
        "id": "tf94ihMY1hae"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[['precious', 'little', 'diamond', 'play', 'script', 'excellent', 'compare', 'movie', 'maybe', 'movie', 'leon', 'wonderfully', 'played', 'jean', 'reno', 'natalie', 'portman', 'best', 'movie', 'anne', 'parillaud', 'played', 'frankie', 'starlight', 'speaking', 'english', 'mean', 'story', 'young', 'punk', 'girl', 'nikita', 'taken', 'depraved', 'world', 'secret', 'government', 'forces', 'exceptionally', 'americans', 'mind', 'point', 'return', 'especially', 'femme', 'nikita', 'series', 'compare', 'original', 'believe', 'trash', 'videos', 'buy', 'rent', 'buy', 'btw', 'beware', 'subtitles', 'company', 'translate', 'release', 'disgrace', 'understand', 'french', 'dubbed', 'version', 'regret', 'later'], ['favourite', 'film', 'time', 'comment', 'taken', 'lightly', 'probably', 'watch', 'far', 'films', 'healthy', 'loved', 'saw', 'femme', 'nikita', 'nearly', 'years', 'ago', 'manages', 'absolute', 'favourite', 'incredibly', 'stylish', 'sexy', 'thriller', 'luc', 'besson', 'great', 'flair', 'impeccable', 'direction', 'fashion', 'appropriate', 'usage', 'music', 'makes', 'watchable', 'film', 'anne', 'parillaud', 'perfect', 'rendering', 'complex', 'character', 'transforms', 'heartless', 'killer', 'compassionate', 'vibrant', 'young', 'woman', 'makes', 'film', 'beautiful', 'eyes', 'screen', 'seen', 'luc', 'besson', 'films', 'including', 'subway', 'professional', 'irritating', 'fifth', 'element', 'nikita', 'doubt', 'far', 'superior', 'film', 'tragic', 'elements', 'ultimately', 'extremely', 'hopeful', 'story', 'person', 'cruel', 'merciless', 'ultimately', 'comes', 'realize', 'humanity', 'personal', 'power', 'extremely', 'inspiring', 'hope', 'nikita', 'hope']]\n"
          ]
        }
      ],
      "source": [
        "X_tokenized_unsupervised = [text.split() for text in X_unsupervised]\n",
        "print (X_tokenized_unsupervised[:2])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6iB6YB6I2f_N"
      },
      "source": [
        "** pretrain a w2v model:word2vec and fastText**\n",
        "\n",
        "de 3 à 5 minutes pour 100 à 300 dimensions"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from gensim.models import Word2Vec, KeyedVectors\n",
        "\n",
        "#\n",
        "%time w2v_model = Word2Vec(X_tokenized_unsupervised, **params)\n",
        "\n",
        "# save the model\n",
        "# !mkdir -p models\n",
        "# w2v_model_path = 'models/w2v_imdb_unsupervised_HYPER_PARAM.bin'\n",
        "# w2v_model.save(w2v_model_path)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TF5Me3cN2qoQ",
        "outputId": "ceb00ebb-da7c-45dd-dc97-cc77787b3da8"
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 4min 44s, sys: 861 ms, total: 4min 45s\n",
            "Wall time: 3min 48s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from gensim.models.fasttext import FastText\n",
        "\n",
        "#\n",
        "%time ft_model = FastText(X_tokenized_unsupervised, **params)\n",
        "\n",
        "# save the model\n",
        "#!mkdir -p models\n",
        "#ft_model_path = 'models/ft_imdb_unsupervised_HYPER_PARAM.bin'\n",
        "#ft_model.save(ft_model_path)"
      ],
      "metadata": {
        "id": "1JbDb6jg3g4_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Vectorization 5 using doc2vec**\n",
        "\n",
        "La documentation de [doc2vec](https://radimrehurek.com/gensim/models/doc2vec.html) décrit comment construire un modèle doc2vec à partir d'une collection de documents et d'inférer (vectoriser) un document.\n",
        "\n",
        "TODO\n",
        "- construire un modèle doc2vec à partir de la collection X_unsupervised. Les documents ont besoin des d'être tokenisé en entrée de TaggedDocument. Vous êtes libre de tester différents hyperparamètres (e.g. 100, 300, 500 dimensions et différents min_count de 1 à 5)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "2S4-xM-B36G4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Initialize & train a doc2vec model:\n",
        "\n"
      ],
      "metadata": {
        "id": "0KNjXErF36G7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from gensim.test.utils import common_texts\n",
        "from gensim.models.doc2vec import Doc2Vec, TaggedDocument\n",
        "\n",
        "# une liste de documents tokekiné\n",
        "common_texts = [d.split() for d in X_unsupervised]\n",
        "\n",
        "documents = [TaggedDocument(doc, [i]) for i, doc in enumerate(common_texts)]\n",
        "\n",
        "%time d2v_model = Doc2Vec(documents, vector_size=500, window=5, min_count=1, negative =5, workers=16)\n",
        "\n",
        "# Persist a model to disk:\n",
        "# from gensim.test.utils import get_tmpfile\n",
        "#fname = get_tmpfile(\"my_doc2vec_model\")\n",
        "#model.save(fname)\n",
        "#model = Doc2Vec.load(fname)  # you can continue training with the loaded model!"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fc707f98-040f-413c-b7ea-f6bb0cbbf467",
        "id": "1a6gCy0Y36G8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 8min 7s, sys: 15.7 s, total: 8min 23s\n",
            "Wall time: 6min 21s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vectorize"
      ],
      "metadata": {
        "id": "_S_f6YuO36G9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Infer vector for new documents:\n",
        "#\n",
        "X_train_d2v_matrix = list()\n",
        "for d in tqdm(X_train):\n",
        "  X_train_d2v_matrix.append(d2v_model.infer_vector(d.split()))\n",
        "X_train_d2v_matrix_np = np.vstack(X_train_d2v_matrix)\n",
        "print ('X_train_d2v_matrix_np.shape:', X_train_d2v_matrix_np.shape)\n",
        "\n",
        "#\n",
        "X_test_d2v_matrix = list()\n",
        "for d in tqdm(X_test):\n",
        "  X_test_d2v_matrix.append(d2v_model.infer_vector(d.split()))\n",
        "X_test_d2v_matrix_np = np.vstack(X_test_d2v_matrix)\n",
        "print ('X_test_d2v_matrix_np.shape:', X_test_d2v_matrix_np.shape)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f0373e42-1821-4558-9532-04b928f64664",
        "id": "qfaZvxCO36G-"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 800/800 [00:09<00:00, 85.06it/s] \n",
            "100%|██████████| 200/200 [00:02<00:00, 79.04it/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Train, predict, and evaluate"
      ],
      "metadata": {
        "id": "vQOE3f6u36G_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# train\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "%time LR_w2v_model = LogisticRegression().fit(X_train_d2v_matrix_np, Y_train_np)\n",
        "\n",
        "# predict\n",
        "%time Y_test_prediction = LR_w2v_model.predict(X_test_d2v_matrix_np)\n",
        "\n",
        "# evaluate\n",
        "from sklearn.metrics import accuracy_score, classification_report\n",
        "#print(accuracy_score(Y_test_np,Y_test_prediction))\n",
        "print(classification_report(Y_test_np,Y_test_prediction))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e15817a7-b085-4767-c6fa-b8ddb215c5de",
        "id": "ywCDfCsd36G_"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 30.1 ms, sys: 7.05 ms, total: 37.1 ms\n",
            "Wall time: 55.9 ms\n",
            "CPU times: user 1.47 ms, sys: 3 ms, total: 4.47 ms\n",
            "Wall time: 6.32 ms\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.79      0.82      0.81       108\n",
            "           1       0.78      0.74      0.76        92\n",
            "\n",
            "    accuracy                           0.79       200\n",
            "   macro avg       0.78      0.78      0.78       200\n",
            "weighted avg       0.78      0.79      0.78       200\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "300\n",
        "```\n",
        "precision    recall  f1-score   support\n",
        "\n",
        "           0       0.81      0.81      0.81       108\n",
        "           1       0.78      0.78      0.78        92\n",
        "\n",
        "    accuracy                           0.80       200\n",
        "   macro avg       0.80      0.80      0.80       200\n",
        "weighted avg       0.80      0.80      0.80       200\n",
        "```\n",
        "\n"
      ],
      "metadata": {
        "id": "dAr13OeK36HA"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyO+nOidC/P9IkXXmFBUG01U",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "d77a502cab7245e7844b6eb512ec2bac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_95c82cf664894953b68dcfcb59570c2d",
              "IPY_MODEL_e42d26fd80a64b56bcbebb177e10623e",
              "IPY_MODEL_83416178bb2a4e45a84f54f32dc40298"
            ],
            "layout": "IPY_MODEL_c6dcbccceaa04a28a3ce6b46eb1b7ef2"
          }
        },
        "95c82cf664894953b68dcfcb59570c2d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f21c313672ee430c9d1676ca50ebf5c7",
            "placeholder": "​",
            "style": "IPY_MODEL_49f93406f14341e487837b96d1c1297c",
            "value": "README.md: 100%"
          }
        },
        "e42d26fd80a64b56bcbebb177e10623e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6c43aae432784b74bfbfab624269241d",
            "max": 7809,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_85e761a53a2d40489289d677cec60ffb",
            "value": 7809
          }
        },
        "83416178bb2a4e45a84f54f32dc40298": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6f30f1734c5f4065af5286e0c70f5157",
            "placeholder": "​",
            "style": "IPY_MODEL_552e76fa353c4fe8b7a73688dd0fde10",
            "value": " 7.81k/7.81k [00:00&lt;00:00, 63.5kB/s]"
          }
        },
        "c6dcbccceaa04a28a3ce6b46eb1b7ef2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f21c313672ee430c9d1676ca50ebf5c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "49f93406f14341e487837b96d1c1297c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6c43aae432784b74bfbfab624269241d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "85e761a53a2d40489289d677cec60ffb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6f30f1734c5f4065af5286e0c70f5157": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "552e76fa353c4fe8b7a73688dd0fde10": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "91016fa3b3fd44d4ae24fd9b17e748ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_78ca7fc76e6b4f9aacc20b7a6f29dc34",
              "IPY_MODEL_7cfae29822b6432cba222f0979b6533e",
              "IPY_MODEL_fe25d3d63912413687e52550ba096aa8"
            ],
            "layout": "IPY_MODEL_6cddfe098c4a4f90bbe9aa717807aa18"
          }
        },
        "78ca7fc76e6b4f9aacc20b7a6f29dc34": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_136179a5945446c79a499bd0a4adfdf5",
            "placeholder": "​",
            "style": "IPY_MODEL_a0b199fc37144f97b1ba4922bd3534e2",
            "value": "train-00000-of-00001.parquet: 100%"
          }
        },
        "7cfae29822b6432cba222f0979b6533e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fb008639ccab46e4af58934cb9c28f99",
            "max": 20979968,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9268c965e47840f0bf919bfb2c8e1162",
            "value": 20979968
          }
        },
        "fe25d3d63912413687e52550ba096aa8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a0897276a6df44aeabc550bac4858c25",
            "placeholder": "​",
            "style": "IPY_MODEL_660a18580d9742b7a32e35aff84607de",
            "value": " 21.0M/21.0M [00:00&lt;00:00, 30.3MB/s]"
          }
        },
        "6cddfe098c4a4f90bbe9aa717807aa18": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "136179a5945446c79a499bd0a4adfdf5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a0b199fc37144f97b1ba4922bd3534e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fb008639ccab46e4af58934cb9c28f99": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9268c965e47840f0bf919bfb2c8e1162": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a0897276a6df44aeabc550bac4858c25": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "660a18580d9742b7a32e35aff84607de": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d5ede4e9468149d6af8da57ad0cc3cd4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9703145cb59a4f7f91b39aba192c81e5",
              "IPY_MODEL_b388061acbef42a188465e825c07d4d0",
              "IPY_MODEL_9ca07343a6dd41459227f83e1d7a4738"
            ],
            "layout": "IPY_MODEL_1cfbe1e9b07543a695fa1fc07806973d"
          }
        },
        "9703145cb59a4f7f91b39aba192c81e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3ee6d7ad91f14a419a0f3fb6bcfdb7b4",
            "placeholder": "​",
            "style": "IPY_MODEL_5f387440d57f47068b6bb7df8c38109c",
            "value": "test-00000-of-00001.parquet: 100%"
          }
        },
        "b388061acbef42a188465e825c07d4d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6cab835fc85448599677208bc24c2d1",
            "max": 20470363,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f7d119f4aabb49e8b24bc9c3ec72981c",
            "value": 20470363
          }
        },
        "9ca07343a6dd41459227f83e1d7a4738": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0222832ed417478496546330becdbddf",
            "placeholder": "​",
            "style": "IPY_MODEL_ea57bfea55a248c9a2c1d3d59635a4d0",
            "value": " 20.5M/20.5M [00:00&lt;00:00, 75.9MB/s]"
          }
        },
        "1cfbe1e9b07543a695fa1fc07806973d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3ee6d7ad91f14a419a0f3fb6bcfdb7b4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5f387440d57f47068b6bb7df8c38109c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e6cab835fc85448599677208bc24c2d1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f7d119f4aabb49e8b24bc9c3ec72981c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0222832ed417478496546330becdbddf": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ea57bfea55a248c9a2c1d3d59635a4d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9cb58b762aee44b89e71fe2b241c4b2e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_54c16b32ff5941a4a78f7afb2f1d8671",
              "IPY_MODEL_c2a92a65f47d43dab28326bfce0aa543",
              "IPY_MODEL_06fca36f17694e19b031e6b528a353ac"
            ],
            "layout": "IPY_MODEL_de886875d455478a80bfa42ee8b295bd"
          }
        },
        "54c16b32ff5941a4a78f7afb2f1d8671": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b9bdd5aec6c84eea8d5b3790604c21d7",
            "placeholder": "​",
            "style": "IPY_MODEL_e1b167bbf59b44bfa757993a01f626d2",
            "value": "unsupervised-00000-of-00001.parquet: 100%"
          }
        },
        "c2a92a65f47d43dab28326bfce0aa543": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_24f5b462e68a40089f7b4618367859b5",
            "max": 41996509,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ba54d6c3fcb74d35bba73f9759b98a72",
            "value": 41996509
          }
        },
        "06fca36f17694e19b031e6b528a353ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e904d3e6eb4b42cd9f9e9b1179b21685",
            "placeholder": "​",
            "style": "IPY_MODEL_6dca763ecba0497cb993a73c163d7de4",
            "value": " 42.0M/42.0M [00:00&lt;00:00, 86.8MB/s]"
          }
        },
        "de886875d455478a80bfa42ee8b295bd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b9bdd5aec6c84eea8d5b3790604c21d7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e1b167bbf59b44bfa757993a01f626d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "24f5b462e68a40089f7b4618367859b5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ba54d6c3fcb74d35bba73f9759b98a72": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e904d3e6eb4b42cd9f9e9b1179b21685": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6dca763ecba0497cb993a73c163d7de4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8ca06d0929ff458abf0ba3b234e5ff0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_15c830b03a724d399bc9b05e45fce7f7",
              "IPY_MODEL_a9ad58e893a94c5f92c557a7e071be07",
              "IPY_MODEL_4b1d5de8fdeb4516bbbcc3b6416f8ff7"
            ],
            "layout": "IPY_MODEL_e7301dd2f8e548be978564930152cee4"
          }
        },
        "15c830b03a724d399bc9b05e45fce7f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6eb0f32fb619407d832c3277ef3b5244",
            "placeholder": "​",
            "style": "IPY_MODEL_f140d271f1c646e8a19b06f300ec4e98",
            "value": "Generating train split: 100%"
          }
        },
        "a9ad58e893a94c5f92c557a7e071be07": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_96c1f79d82af452e8975d3dfc5f19a75",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7ccbfb90a0c7456585bc69990295c1ba",
            "value": 25000
          }
        },
        "4b1d5de8fdeb4516bbbcc3b6416f8ff7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4d51f72c64c14974859d8d99ab0a4c60",
            "placeholder": "​",
            "style": "IPY_MODEL_1145a4fb1e7346a6bd8ac66edaa6a04d",
            "value": " 25000/25000 [00:00&lt;00:00, 40177.90 examples/s]"
          }
        },
        "e7301dd2f8e548be978564930152cee4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6eb0f32fb619407d832c3277ef3b5244": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f140d271f1c646e8a19b06f300ec4e98": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "96c1f79d82af452e8975d3dfc5f19a75": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7ccbfb90a0c7456585bc69990295c1ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4d51f72c64c14974859d8d99ab0a4c60": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1145a4fb1e7346a6bd8ac66edaa6a04d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "61530322251744e789553a585cf4da51": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a0d30f77b77d47adae0d8289266cdea9",
              "IPY_MODEL_cf2bacc80176433aa2cbc0f6409f49b5",
              "IPY_MODEL_507b2e8f01d3405eb61db0448e9d44fb"
            ],
            "layout": "IPY_MODEL_0a8e6e2e2f2a4fd9ba526c823648f9c2"
          }
        },
        "a0d30f77b77d47adae0d8289266cdea9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2947ed1b13334895b68bcf79a7ee9fce",
            "placeholder": "​",
            "style": "IPY_MODEL_6adef6eca2e746aca4cec0a56c6cba26",
            "value": "Generating test split: 100%"
          }
        },
        "cf2bacc80176433aa2cbc0f6409f49b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c8f9a0c747cf456e82d4ee354336b723",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_66097bbb677d4cbcab24decb8b257657",
            "value": 25000
          }
        },
        "507b2e8f01d3405eb61db0448e9d44fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_995227da41444e8e9c6adcca3b7859f9",
            "placeholder": "​",
            "style": "IPY_MODEL_63c3c29aba704df8aded6873c5aa4a4f",
            "value": " 25000/25000 [00:00&lt;00:00, 39886.25 examples/s]"
          }
        },
        "0a8e6e2e2f2a4fd9ba526c823648f9c2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2947ed1b13334895b68bcf79a7ee9fce": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6adef6eca2e746aca4cec0a56c6cba26": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c8f9a0c747cf456e82d4ee354336b723": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "66097bbb677d4cbcab24decb8b257657": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "995227da41444e8e9c6adcca3b7859f9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "63c3c29aba704df8aded6873c5aa4a4f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8bd32eb2ce4b462887d11c57dc50a9cd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c97b98daf84748c2820d12fca3f4a8c4",
              "IPY_MODEL_37b1d9114d834467898053070770a536",
              "IPY_MODEL_d13b49a7e45f430e9ddedda32f8421dc"
            ],
            "layout": "IPY_MODEL_61f65dc2a82b4f6685c20b27931de74a"
          }
        },
        "c97b98daf84748c2820d12fca3f4a8c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e34f3b756a994c699450b089f07062e2",
            "placeholder": "​",
            "style": "IPY_MODEL_7dcff6e6fb3a4750a54f60e56d6ec0d6",
            "value": "Generating unsupervised split: 100%"
          }
        },
        "37b1d9114d834467898053070770a536": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a81fe2e090424809ab65c6b015490ccb",
            "max": 50000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_abfaaf18f2c54ed0a7c8c666d0526f5c",
            "value": 50000
          }
        },
        "d13b49a7e45f430e9ddedda32f8421dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_38492cac60ef45a4ba20416a2ad975c1",
            "placeholder": "​",
            "style": "IPY_MODEL_eaff87e842b5459487c602eedda5b550",
            "value": " 50000/50000 [00:00&lt;00:00, 67770.79 examples/s]"
          }
        },
        "61f65dc2a82b4f6685c20b27931de74a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e34f3b756a994c699450b089f07062e2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7dcff6e6fb3a4750a54f60e56d6ec0d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a81fe2e090424809ab65c6b015490ccb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "abfaaf18f2c54ed0a7c8c666d0526f5c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "38492cac60ef45a4ba20416a2ad975c1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eaff87e842b5459487c602eedda5b550": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}