Example)Generative QG Dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example) Build Generative QG Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Without Tokenization\n",
    "\n",
    "1. Create a `SquadGuru` object who is an NLP expert. Let him `gather` the complex squad json dataset organized to make it usable in an NLP task.\n",
    "\n",
    "   - Constructor Signature\n",
    "\n",
    "     ```python\n",
    "     SquadGuru(parser: SquadParser, #parser which implement SquadParser\n",
    "               tokenizer=None, #tokenizer which implement .tokenize(text: str)\n",
    "               tags=SQUAD_TAGS, #iterable of str\n",
    "               versions=SQUAD_VERSIONS #iterable of float\n",
    "     )\n",
    "     ```\n",
    "\n",
    "   - Inject a `parser`, which the guru will use to extract X and Y data from the original suqad dataset.\n",
    "   - Inject a `tokenizer` that will be used to create tokenized X and Y.\n",
    "   - Inject an iterable of `tags` that describes the tags of the dataset to load.\n",
    "   - Inject an iterable of `versions` that describes the versions of the dataset to load. Version's datatype is `float`.\n",
    "\n",
    "\n",
    "   Here we're gonna use `GenerativeQGParser`. In order to create an instance of it, use static factory method pattern like: `SquadParser.from_nlp_task('GEN_QG')`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from prep_squad import SquadGuru\n",
    "from prep_squad import SquadParser\n",
    "\n",
    "squad_parser = SquadParser.from_nlp_task('GEN_QG')\n",
    "guru = SquadGuru(squad_parser, tags=['dev'], versions=(1.1, 2.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Use `squadGuru.gather()` to let the guru remember extracted X and Y.\n",
    "\n",
    "   - Method Signature\n",
    "\n",
    "     ```python\n",
    "     squadGuru.gather(only_first_answer=False, \n",
    "                      verbose=False)\n",
    "```\n",
    "     \n",
    "     - Set `only_first_answer` to extract the first answer in each of question-answers sets.\n",
    "     - Set `verbose` to print some logs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SQuAD-v1.1 dev dataset has been parsed.\n",
      "SQuAD-v2.0 dev dataset has been parsed.\n"
     ]
    }
   ],
   "source": [
    "guru.gather(only_first_answer=True, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get DataFrame Table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Input</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Which NFL team represented the AFC at Super Bo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Which NFL team represented the NFC at Super Bo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Where did Super Bowl 50 take place?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Which NFL team won Super Bowl 50?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>What color was used to emphasize the 50th anni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16493</th>\n",
       "      <td>The pound-force has a metric counterpart, less...</td>\n",
       "      <td>What is the metric term less used than the New...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16494</th>\n",
       "      <td>The pound-force has a metric counterpart, less...</td>\n",
       "      <td>What is the kilogram-force sometimes reffered ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16495</th>\n",
       "      <td>The pound-force has a metric counterpart, less...</td>\n",
       "      <td>What is a very seldom used unit of mass in the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16496</th>\n",
       "      <td>The pound-force has a metric counterpart, less...</td>\n",
       "      <td>What seldom used term of a unit of force equal...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16497</th>\n",
       "      <td>The pound-force has a metric counterpart, less...</td>\n",
       "      <td>What is the seldom used force unit equal to on...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>16498 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   Input  \\\n",
       "0      Super Bowl 50 was an American football game to...   \n",
       "1      Super Bowl 50 was an American football game to...   \n",
       "2      Super Bowl 50 was an American football game to...   \n",
       "3      Super Bowl 50 was an American football game to...   \n",
       "4      Super Bowl 50 was an American football game to...   \n",
       "...                                                  ...   \n",
       "16493  The pound-force has a metric counterpart, less...   \n",
       "16494  The pound-force has a metric counterpart, less...   \n",
       "16495  The pound-force has a metric counterpart, less...   \n",
       "16496  The pound-force has a metric counterpart, less...   \n",
       "16497  The pound-force has a metric counterpart, less...   \n",
       "\n",
       "                                                  Target  \n",
       "0      Which NFL team represented the AFC at Super Bo...  \n",
       "1      Which NFL team represented the NFC at Super Bo...  \n",
       "2                    Where did Super Bowl 50 take place?  \n",
       "3                      Which NFL team won Super Bowl 50?  \n",
       "4      What color was used to emphasize the 50th anni...  \n",
       "...                                                  ...  \n",
       "16493  What is the metric term less used than the New...  \n",
       "16494  What is the kilogram-force sometimes reffered ...  \n",
       "16495  What is a very seldom used unit of mass in the...  \n",
       "16496  What seldom used term of a unit of force equal...  \n",
       "16497  What is the seldom used force unit equal to on...  \n",
       "\n",
       "[16498 rows x 2 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guru.to_dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Numpy Array "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24–10 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi\\'s Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50. [SEP] Denver Broncos',\n",
       "        'Which NFL team represented the AFC at Super Bowl 50?'],\n",
       "       ['Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24–10 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi\\'s Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50. [SEP] Carolina Panthers',\n",
       "        'Which NFL team represented the NFC at Super Bowl 50?'],\n",
       "       ['Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24–10 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi\\'s Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50. [SEP] Santa Clara, California',\n",
       "        'Where did Super Bowl 50 take place?'],\n",
       "       ...,\n",
       "       ['The pound-force has a metric counterpart, less commonly used than the newton: the kilogram-force (kgf) (sometimes kilopond), is the force exerted by standard gravity on one kilogram of mass. The kilogram-force leads to an alternate, but rarely used unit of mass: the metric slug (sometimes mug or hyl) is that mass that accelerates at 1 m·s−2 when subjected to a force of 1 kgf. The kilogram-force is not a part of the modern SI system, and is generally deprecated; however it still sees use for some purposes as expressing aircraft weight, jet thrust, bicycle spoke tension, torque wrench settings and engine output torque. Other arcane units of force include the sthène, which is equivalent to 1000 N, and the kip, which is equivalent to 1000 lbf. [SEP] slug',\n",
       "        'What is a very seldom used unit of mass in the metric system?'],\n",
       "       ['The pound-force has a metric counterpart, less commonly used than the newton: the kilogram-force (kgf) (sometimes kilopond), is the force exerted by standard gravity on one kilogram of mass. The kilogram-force leads to an alternate, but rarely used unit of mass: the metric slug (sometimes mug or hyl) is that mass that accelerates at 1 m·s−2 when subjected to a force of 1 kgf. The kilogram-force is not a part of the modern SI system, and is generally deprecated; however it still sees use for some purposes as expressing aircraft weight, jet thrust, bicycle spoke tension, torque wrench settings and engine output torque. Other arcane units of force include the sthène, which is equivalent to 1000 N, and the kip, which is equivalent to 1000 lbf. [SEP] kip',\n",
       "        'What seldom used term of a unit of force equal to 1000 pound s of force?'],\n",
       "       ['The pound-force has a metric counterpart, less commonly used than the newton: the kilogram-force (kgf) (sometimes kilopond), is the force exerted by standard gravity on one kilogram of mass. The kilogram-force leads to an alternate, but rarely used unit of mass: the metric slug (sometimes mug or hyl) is that mass that accelerates at 1 m·s−2 when subjected to a force of 1 kgf. The kilogram-force is not a part of the modern SI system, and is generally deprecated; however it still sees use for some purposes as expressing aircraft weight, jet thrust, bicycle spoke tension, torque wrench settings and engine output torque. Other arcane units of force include the sthène, which is equivalent to 1000 N, and the kip, which is equivalent to 1000 lbf. [SEP] sthène',\n",
       "        'What is the seldom used force unit equal to one thousand newtons?']],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guru.to_numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save to files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#passage-answer\n",
    "#question\n",
    "guru.to_file('examples/genqg.pa.example.txt', 'examples/genqg.q.example.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Applying Bert Tokenization\n",
    "\n",
    "1. Create a `SquadGuru` object who is an NLP expert. Here we give him a pretrained `BertTokenizer`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from prep_squad import SquadGuru\n",
    "from prep_squad import SquadParser\n",
    "from transformers import BertTokenizer\n",
    "\n",
    "task = SquadParser.from_nlp_task('GEN_QG')\n",
    "tok = BertTokenizer.from_pretrained('bert-large-cased')\n",
    "guru = SquadGuru(task, tok, tags=['dev'], versions=(1.1, 2.0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SQuAD-v1.1 dev dataset has been parsed.\n",
      "SQuAD-v2.0 dev dataset has been parsed.\n"
     ]
    }
   ],
   "source": [
    "guru.gather(only_first_answer=True, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Input</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Which NFL team represented the AFC at Super Bo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Which NFL team represented the NFC at Super Bo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Where did Super Bowl 50 take place ?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>Which NFL team won Super Bowl 50 ?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Super Bowl 50 was an American football game to...</td>\n",
       "      <td>What color was used to emphasize the 50th anni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16493</th>\n",
       "      <td>The pound - force has a metric counterpart , l...</td>\n",
       "      <td>What is the metric term less used than the New...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16494</th>\n",
       "      <td>The pound - force has a metric counterpart , l...</td>\n",
       "      <td>What is the k ##ilo ##gram - force sometimes r...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16495</th>\n",
       "      <td>The pound - force has a metric counterpart , l...</td>\n",
       "      <td>What is a very seldom used unit of mass in the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16496</th>\n",
       "      <td>The pound - force has a metric counterpart , l...</td>\n",
       "      <td>What seldom used term of a unit of force equal...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16497</th>\n",
       "      <td>The pound - force has a metric counterpart , l...</td>\n",
       "      <td>What is the seldom used force unit equal to on...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>16498 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   Input  \\\n",
       "0      Super Bowl 50 was an American football game to...   \n",
       "1      Super Bowl 50 was an American football game to...   \n",
       "2      Super Bowl 50 was an American football game to...   \n",
       "3      Super Bowl 50 was an American football game to...   \n",
       "4      Super Bowl 50 was an American football game to...   \n",
       "...                                                  ...   \n",
       "16493  The pound - force has a metric counterpart , l...   \n",
       "16494  The pound - force has a metric counterpart , l...   \n",
       "16495  The pound - force has a metric counterpart , l...   \n",
       "16496  The pound - force has a metric counterpart , l...   \n",
       "16497  The pound - force has a metric counterpart , l...   \n",
       "\n",
       "                                                  Target  \n",
       "0      Which NFL team represented the AFC at Super Bo...  \n",
       "1      Which NFL team represented the NFC at Super Bo...  \n",
       "2                   Where did Super Bowl 50 take place ?  \n",
       "3                     Which NFL team won Super Bowl 50 ?  \n",
       "4      What color was used to emphasize the 50th anni...  \n",
       "...                                                  ...  \n",
       "16493  What is the metric term less used than the New...  \n",
       "16494  What is the k ##ilo ##gram - force sometimes r...  \n",
       "16495  What is a very seldom used unit of mass in the...  \n",
       "16496  What seldom used term of a unit of force equal...  \n",
       "16497  What is the seldom used force unit equal to on...  \n",
       "\n",
       "[16498 rows x 2 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guru.to_dataframe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['Super Bowl 50 was an American football game to determine the champion of the National Football League ( NFL ) for the 2015 season . The American Football Conference ( AFC ) champion Denver Broncos defeated the National Football Conference ( NFC ) champion Carolina Panthers 24 – 10 to earn their third Super Bowl title . The game was played on February 7 , 2016 , at Levi \\' s Stadium in the San Francisco Bay Area at Santa Clara , California . As this was the 50th Super Bowl , the league emphasized the \" golden anniversary \" with various gold - themed initiatives , as well as temporarily su ##sp ##ending the tradition of naming each Super Bowl game with Roman n ##ume ##rals ( under which the game would have been known as \" Super Bowl L \" ) , so that the logo could prominently feature the Arabic n ##ume ##rals 50 . [SEP] Denver Broncos',\n",
       "        'Which NFL team represented the AFC at Super Bowl 50 ?'],\n",
       "       ['Super Bowl 50 was an American football game to determine the champion of the National Football League ( NFL ) for the 2015 season . The American Football Conference ( AFC ) champion Denver Broncos defeated the National Football Conference ( NFC ) champion Carolina Panthers 24 – 10 to earn their third Super Bowl title . The game was played on February 7 , 2016 , at Levi \\' s Stadium in the San Francisco Bay Area at Santa Clara , California . As this was the 50th Super Bowl , the league emphasized the \" golden anniversary \" with various gold - themed initiatives , as well as temporarily su ##sp ##ending the tradition of naming each Super Bowl game with Roman n ##ume ##rals ( under which the game would have been known as \" Super Bowl L \" ) , so that the logo could prominently feature the Arabic n ##ume ##rals 50 . [SEP] Carolina Panthers',\n",
       "        'Which NFL team represented the NFC at Super Bowl 50 ?'],\n",
       "       ['Super Bowl 50 was an American football game to determine the champion of the National Football League ( NFL ) for the 2015 season . The American Football Conference ( AFC ) champion Denver Broncos defeated the National Football Conference ( NFC ) champion Carolina Panthers 24 – 10 to earn their third Super Bowl title . The game was played on February 7 , 2016 , at Levi \\' s Stadium in the San Francisco Bay Area at Santa Clara , California . As this was the 50th Super Bowl , the league emphasized the \" golden anniversary \" with various gold - themed initiatives , as well as temporarily su ##sp ##ending the tradition of naming each Super Bowl game with Roman n ##ume ##rals ( under which the game would have been known as \" Super Bowl L \" ) , so that the logo could prominently feature the Arabic n ##ume ##rals 50 . [SEP] Santa Clara , California',\n",
       "        'Where did Super Bowl 50 take place ?'],\n",
       "       ...,\n",
       "       ['The pound - force has a metric counterpart , less commonly used than the new ##ton : the k ##ilo ##gram - force ( kg ##f ) ( sometimes k ##ilo ##po ##nd ) , is the force ex ##ert ##ed by standard gravity on one k ##ilo ##gram of mass . The k ##ilo ##gram - force leads to an alternate , but rarely used unit of mass : the metric s ##lug ( sometimes mug or h ##yl ) is that mass that accelerate ##s at 1 m · s ##− ##2 when subjected to a force of 1 kg ##f . The k ##ilo ##gram - force is not a part of the modern S ##I system , and is generally de ##p ##re ##cated ; however it still sees use for some purposes as expressing aircraft weight , jet thrust , bicycle spoke tension , torque w ##rench settings and engine output torque . Other arc ##ane units of force include the s ##th ##ène , which is equivalent to 1000 N , and the k ##ip , which is equivalent to 1000 lb ##f . [SEP] s ##lug',\n",
       "        'What is a very seldom used unit of mass in the metric system ?'],\n",
       "       ['The pound - force has a metric counterpart , less commonly used than the new ##ton : the k ##ilo ##gram - force ( kg ##f ) ( sometimes k ##ilo ##po ##nd ) , is the force ex ##ert ##ed by standard gravity on one k ##ilo ##gram of mass . The k ##ilo ##gram - force leads to an alternate , but rarely used unit of mass : the metric s ##lug ( sometimes mug or h ##yl ) is that mass that accelerate ##s at 1 m · s ##− ##2 when subjected to a force of 1 kg ##f . The k ##ilo ##gram - force is not a part of the modern S ##I system , and is generally de ##p ##re ##cated ; however it still sees use for some purposes as expressing aircraft weight , jet thrust , bicycle spoke tension , torque w ##rench settings and engine output torque . Other arc ##ane units of force include the s ##th ##ène , which is equivalent to 1000 N , and the k ##ip , which is equivalent to 1000 lb ##f . [SEP] k ##ip',\n",
       "        'What seldom used term of a unit of force equal to 1000 pound s of force ?'],\n",
       "       ['The pound - force has a metric counterpart , less commonly used than the new ##ton : the k ##ilo ##gram - force ( kg ##f ) ( sometimes k ##ilo ##po ##nd ) , is the force ex ##ert ##ed by standard gravity on one k ##ilo ##gram of mass . The k ##ilo ##gram - force leads to an alternate , but rarely used unit of mass : the metric s ##lug ( sometimes mug or h ##yl ) is that mass that accelerate ##s at 1 m · s ##− ##2 when subjected to a force of 1 kg ##f . The k ##ilo ##gram - force is not a part of the modern S ##I system , and is generally de ##p ##re ##cated ; however it still sees use for some purposes as expressing aircraft weight , jet thrust , bicycle spoke tension , torque w ##rench settings and engine output torque . Other arc ##ane units of force include the s ##th ##ène , which is equivalent to 1000 N , and the k ##ip , which is equivalent to 1000 lb ##f . [SEP] s ##th ##ène',\n",
       "        'What is the seldom used force unit equal to one thousand new ##tons ?']],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guru.to_numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#passage-answer\n",
    "#question\n",
    "guru.to_file('examples/genqg.pa.tok.example.txt', 'examples/genqg.q.tok.example.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}