chatbot_using_RNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chatbot using RNN\n",
    "### Northwestern University - Fall 2017\n",
    "### Student: Danilo Neves Ribeiro\n",
    "### E-mail: daniloribeiro2021@u.northwestern.edu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "The idea of this project is to create a simple chatbot by training a Recurrent Neural Networks. \n",
    "\n",
    "## Chatbots\n",
    "There are many ways one can go about creating a chat-bot. For example, many chatbots rely on pre-defined rules to answer questions. Those can work well but requires intese human work to create as many rules as possible. \n",
    "\n",
    "Machine learning greately simplify this task by enableing to learn from pre-existing conversation corpus. The two main types of ML chatbots are:\n",
    "\n",
    "- Retrieval-based: answer questions by choosing from one of the answers available in the data-set.\n",
    "- Generative: generates the conversation dialog word by word based on the query. The generated sentense is normally not included in the original data-set.\n",
    "\n",
    "For this project, I decided to create a chatbot using the generative approch, which normally makes more mistakes, such as grammar mistakes, but can respond a broader set of questions and contexts.\n",
    "\n",
    "## Dataset\n",
    "The model was trained using the [Cornell Movie Dialog Corpus](http://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html), that contains a collection of fictional conversations extracted from raw movie scripts. The data was split in 108136 conversation pairs for training, and 30000 conversation pairs for testing.\n",
    "\n",
    "## Implementation Architecture\n",
    "Here I use a Recurrent Neural Network to train on the data set. More specifically I use a seq2seq model with bucketing and attention mechanism, which is described in more details below:\n",
    "\n",
    "### Seq2Seq:\n",
    "Sequence to Sequence RNN models are composed of two main components: encoder and decoder. The encoder is responsible for reading the input, word by word, and generating a hidden state that \"represents\" the input. The decoder outputs words according to the hidden states generated by the encoder. The following image gives a general idea of this architecture:\n",
    "<img src=\"seq2seq.png\" alt=\"seq2seq\" style=\"width: 700px;\"/>\n",
    "\n",
    "### Padding and Bucketing: \n",
    "One of the limitations of the simple Seq2Seq arquitectures is that it has fixed size input and output. Therefore we need to use padding and special symbols to deal with the fact that both input and output sentences can have different length (the ones used here are: EOS = \"End of sentence\", PAD = \"Filler\", GO = \"Start decoding\", plus a special symbol for unknown words: UNK).\n",
    "\n",
    "To efficiently handle sentenses with different lengths the bucketing method is used. This model uses buckets = [(5, 10), (10, 15), (20, 25), (40, 50)]. This means that if the input is a sentence with 3 tokens, and the corresponding output is a sentence with 6 tokens, then they will be put in the first bucket and padded to length 5 for encoder inputs, and length 10 for decoder inputs.\n",
    "\n",
    "### Attention mechanism:\n",
    "The attention mechanism tries to address the following limitations:\n",
    "- The decoder is not aware of which parts of the encoding are relevant at each step of the generation.\n",
    "- The encoder has limited memory and can't \"remember\" more than a single fixed size vector.\n",
    "\n",
    "The attention model comes between the encoder and the decoder and helps the decoder to pick only the encoded inputs that ar important for each step of the decoding process.\n",
    "\n",
    "<img src=\"attention.jpg\" alt=\"Attention mechanism\" style=\"width: 400px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Code\n",
    "\n",
    "The code will be split between: \n",
    "- Preprocessing data (tokenizing, creating vobabulary, transforming input from words to word ids)\n",
    "- Training\n",
    "- Testing\n",
    "\n",
    "##### Software requirements\n",
    "- Python 3.6.2\n",
    "- Numpy\n",
    "- TensorFlow \n",
    "\n",
    "**_Note_**: the following code is largely based on the code from the chatbot tutorial by [Suriyadeepan Ram](http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/) and uses the more general seq2seq model provided by the [Google Tensorflow tutorial on NMT](https://github.com/tensorflow/nmt), which is imported from a separate code file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# IMPORTS\n",
    "\n",
    "import os\n",
    "import numpy as np\n",
    "import re\n",
    "import tensorflow as tf\n",
    "import math\n",
    "from seq2seq_model import Seq2SeqModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# GLOBAL VARIABLES AND PARAMS\n",
    "\n",
    "# encoding and decoding paths\n",
    "TRAIN_END_PATH = os.path.join('data', 'train.enc')\n",
    "TRAIN_DEC_PATH = os.path.join('data', 'train.dec')\n",
    "TEST_END_PATH = os.path.join('data', 'test.enc')\n",
    "TEST_DEC_PATH = os.path.join('data', 'test.dec')\n",
    "\n",
    "TRAIN_END_ID_PATH = os.path.join('data', 'train.enc.id')\n",
    "TRAIN_DEC_ID_PATH = os.path.join('data', 'train.dec.id')\n",
    "TEST_END_ID_PATH = os.path.join('data', 'test.enc.id')\n",
    "TEST_DEC_ID_PATH = os.path.join('data', 'test.dec.id')\n",
    "\n",
    "# vocabulary paths\n",
    "VOCAB_ENC_PATH = os.path.join('data', 'vocab.enc')\n",
    "VOCAB_DEC_PATH = os.path.join('data', 'vocab.dec')\n",
    "MAX_VOCAB_SIZE = 20000\n",
    "\n",
    "# data utils\n",
    "SPLIT_REGEX = re.compile(\"([.,!?\\\"':;)(])\")\n",
    "PAD_TOKEN = \"_PAD\"\n",
    "START_TOKEN = \"_GO\"\n",
    "END_TOKEN = \"_EOS\"\n",
    "UNKNOWEN_TOKEN = \"_UNK\"\n",
    "INIT_VOCAB = [PAD_TOKEN, START_TOKEN, END_TOKEN, UNKNOWEN_TOKEN]\n",
    "\n",
    "# args\n",
    "BUCKETS = [(5, 10), (10, 15), (20, 25), (40, 50)]\n",
    "LSTM_LAYES = 3\n",
    "LAYER_SIZE = 256\n",
    "BATCH_SIZE = 64\n",
    "LEARNING_RATE = 0.5\n",
    "LEARNING_RATE_DECAY_FACTOR = 0.99\n",
    "MAX_GRADIENT_NORM = 5.0\n",
    "STEP_CHECKPOINTS = 5\n",
    "MAX_ITERATIONS = 1000\n",
    "\n",
    "# pre training\n",
    "TRAINED_MODEL_PATH = 'pre_trained'\n",
    "TRAINED_VOCAB_ENC = os.path.join('pre_trained', 'vocab.enc')\n",
    "TRAINED_VOCAB_DEC = os.path.join('pre_trained', 'vocab.dec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# SIMPLE TOKENIZER\n",
    "\n",
    "def tokenize(sentense):\n",
    "    tokens = []\n",
    "    for token in sentense.strip().split():\n",
    "        tokens.extend([x for x in re.split(SPLIT_REGEX, token) if x])\n",
    "    return tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# CREATING VOCABULARY\n",
    "\n",
    "def create_vocab(data_path, vocab_path):\n",
    "    vocab = {}    \n",
    "    # only creates new file if file doesn't exist\n",
    "    if os.path.exists(vocab_path):\n",
    "        print(\"file \", vocab_path, \" already exists\") \n",
    "    else:\n",
    "        with open(data_path, 'r') as data_file:\n",
    "            for line in data_file:\n",
    "                tokens = tokenize(line)\n",
    "                for token in tokens:\n",
    "                    if token not in vocab:\n",
    "                        vocab[token] = 1\n",
    "                    else:\n",
    "                        vocab[token] += 1\n",
    "        # use the default tokens as initial vocabulity words\n",
    "        vocab_list = INIT_VOCAB + sorted(vocab, key=vocab.get, reverse=True)\n",
    "        # trim vocabulary\n",
    "        vocab_list = vocab_list[:MAX_VOCAB_SIZE]\n",
    "        print(\"final vacabulary size for \", data_path, \" = \", len(vocab_list))\n",
    "        # save to file\n",
    "        with open(vocab_path, 'w') as vocab_file:\n",
    "            for word in vocab_list:\n",
    "                vocab_file.write(word + \"\\n\")   \n",
    "        # update vocab with new order\n",
    "        vocab = dict([(y, x) for (x, y) in enumerate(vocab_list)])\n",
    "        return vocab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TRANSFORM WORDS IN DATA TO IDS\n",
    "\n",
    "def from_text_data_to_id_list(data_path, ouput_path, vocab):\n",
    "    # only creates new file is file doesn't exist\n",
    "    if os.path.exists(ouput_path):\n",
    "        print(\"file \", ouput_path, \" already exists\") \n",
    "    else:\n",
    "        with open(data_path, 'r') as data_file:\n",
    "            with open(ouput_path, 'w') as ouput_file:\n",
    "                for line in data_file:\n",
    "                    tokens = tokenize(line)\n",
    "                    id_list = [str(vocab.get(word, vocab.get(UNKNOWEN_TOKEN))) for word in tokens]\n",
    "                    ouput_file.write(\" \".join(id_list) + \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DATA PREPROCESSING\n",
    "\n",
    "def preprocess_data():\n",
    "    encoding_vocab = create_vocab(TRAIN_END_PATH, VOCAB_ENC_PATH)\n",
    "    decoding_vocab = create_vocab(TRAIN_DEC_PATH, VOCAB_DEC_PATH)\n",
    "    from_text_data_to_id_list(TRAIN_END_PATH, TRAIN_END_ID_PATH, encoding_vocab)\n",
    "    from_text_data_to_id_list(TRAIN_DEC_PATH, TRAIN_DEC_ID_PATH, decoding_vocab)\n",
    "    from_text_data_to_id_list(TEST_END_PATH, TEST_END_ID_PATH, encoding_vocab)\n",
    "    from_text_data_to_id_list(TEST_DEC_PATH, TEST_DEC_ID_PATH, decoding_vocab)\n",
    "    print(\"Data preprocessing complete.\")\n",
    "\n",
    "preprocess_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def read_data(source_path, target_path):\n",
    "    data_set = [[] for _ in BUCKETS]\n",
    "    with tf.gfile.GFile(source_path, mode=\"r\") as source_file:\n",
    "        with tf.gfile.GFile(target_path, mode=\"r\") as target_file:\n",
    "            source, target = source_file.readline(), target_file.readline()\n",
    "            while source and target:\n",
    "                source_ids = [int(x) for x in source.split()]\n",
    "                target_ids = [int(x) for x in target.split()]\n",
    "                target_ids.append(INIT_VOCAB.index(END_TOKEN))\n",
    "                for bucket_id, (source_size, target_size) in enumerate(BUCKETS):\n",
    "                    if len(source_ids) < source_size and len(target_ids) < target_size:\n",
    "                        data_set[bucket_id].append([source_ids, target_ids])\n",
    "                        break\n",
    "                source, target = source_file.readline(), target_file.readline()\n",
    "                return data_set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# CREATE MODEL\n",
    "\n",
    "def create_model(forward_only):\n",
    "    # TODO: remove\n",
    "    return Seq2SeqModel(\n",
    "        MAX_VOCAB_SIZE, MAX_VOCAB_SIZE, BUCKETS, LAYER_SIZE, LSTM_LAYES, MAX_GRADIENT_NORM, \n",
    "        BATCH_SIZE, LEARNING_RATE, LEARNING_RATE_DECAY_FACTOR, forward_only)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TRAIN MODEL\n",
    "\n",
    "def train():\n",
    "    # setup config to use BFC allocator\n",
    "    config = tf.ConfigProto()\n",
    "    config.gpu_options.allocator_type = 'BFC'\n",
    "    with tf.Session(config=config) as sess:\n",
    "        print(\"creating model...\")\n",
    "        model = create_model(forward_only = False)\n",
    "        sess.run(tf.global_variables_initializer())\n",
    "\n",
    "        # Read data into buckets and compute their sizes.\n",
    "        dev_set = read_data(TEST_END_ID_PATH, TEST_DEC_ID_PATH)\n",
    "        train_set = read_data(TRAIN_END_ID_PATH, TRAIN_DEC_ID_PATH)\n",
    "        train_bucket_sizes = [len(train_set[b]) for b in range(len(BUCKETS))]\n",
    "        train_total_size = float(sum(train_bucket_sizes))\n",
    "\n",
    "        # A bucket scale is a list of increasing numbers from 0 to 1 that we'll use\n",
    "        # to select a bucket. Length of [scale[i], scale[i+1]] is proportional to\n",
    "        # the size if i-th training bucket, as used later.\n",
    "        train_buckets_scale = [sum(train_bucket_sizes[:i + 1]) / train_total_size\n",
    "                               for i in range(len(train_bucket_sizes))]\n",
    "\n",
    "        print(\"Running main loop...\")\n",
    "        for current_step in range(MAX_ITERATIONS):\n",
    "            # Choose a bucket according to data distribution. We pick a random number\n",
    "            # in [0, 1] and use the corresponding interval in train_buckets_scale.\n",
    "            random_number = np.random.random_sample()\n",
    "            bucket_id = min([i for i in range(len(train_buckets_scale))\n",
    "                           if train_buckets_scale[i] > random_number])\n",
    "\n",
    "            # Get a batch and make a step.\n",
    "            encoder_inputs, decoder_inputs, target_weights = model.get_batch(\n",
    "              train_set, bucket_id)\n",
    "            _, step_loss, _ = model.step(sess, encoder_inputs, decoder_inputs,\n",
    "                                       target_weights, bucket_id, False) \n",
    "\n",
    "            # Print statistics.\n",
    "            perplexity = math.exp(step_loss) if step_loss < 300 else float('inf')\n",
    "            print (\"global step %d perplexity %.2f\" % (model.global_step.eval(), perplexity))\n",
    "\n",
    "# train model\n",
    "train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# LOAD PRE-TRAINED MODEL\n",
    "\n",
    "def load_vocabulary_list(vocabulary_path):\n",
    "    with open(vocabulary_path, mode=\"r\") as vocab_file:\n",
    "        return [line.strip() for line in vocab_file.readlines()]\n",
    "\n",
    "def load_pre_trained_model(session):\n",
    "    print(\"Loading vocab...\")\n",
    "    enc_vocab_list = load_vocabulary_list(TRAINED_VOCAB_ENC)\n",
    "    dec_vocab_list = load_vocabulary_list(TRAINED_VOCAB_DEC)\n",
    "    enc_vocab = dict([(x, y) for (y, x) in enumerate(dec_vocab_list)])\n",
    "    rev_dec_vocab = dict(enumerate(dec_vocab_list))\n",
    "    \n",
    "    print(\"Creting model...\")\n",
    "    model = create_model(forward_only = True)\n",
    "\n",
    "    print(\"Loading saved model...\")\n",
    "    ckpt = tf.train.get_checkpoint_state(TRAINED_MODEL_PATH)\n",
    "    model.saver.restore(session, ckpt.model_checkpoint_path)\n",
    "    return (model, enc_vocab, rev_dec_vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# DECODING\n",
    "\n",
    "def decode(sentence, model, session, enc_vocab):\n",
    "    # Get token-ids for the input sentence.\n",
    "    token_ids = [enc_vocab.get(w, INIT_VOCAB.index(UNKNOWEN_TOKEN)) for w in tokenize(sentence)]\n",
    "    bucket_id = min([b for b in range(len(BUCKETS)) if BUCKETS[b][0] > len(token_ids)])\n",
    "    # Get a 1-element batch to feed the sentence to the model.\n",
    "    encoder_inputs, decoder_inputs, target_weights = model.get_batch(\n",
    "      {bucket_id: [(token_ids, [])]}, bucket_id)\n",
    "    # Get output logits for the sentence.\n",
    "    _, _, output_logits = model.step(session, encoder_inputs, decoder_inputs,\n",
    "                                   target_weights, bucket_id, True)\n",
    "    # This is a greedy decoder - outputs are just argmaxes of output_logits.\n",
    "    outputs = [int(np.argmax(logit, axis=1)) for logit in output_logits]\n",
    "    return outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CHATBOT MAIN APP\n",
    "\n",
    "def run_chatbot():\n",
    "    print(\"Starting chatbot...\")\n",
    "    with tf.Session() as sess:\n",
    "        model, enc_vocab, rev_dec_vocab = load_pre_trained_model(sess)\n",
    "        model.batch_size = 1  # We decode one sentence at a time.\n",
    "        # Decode from standard input.\n",
    "        sentence = input(\"Chatbot started, ask anything!\\n> \")\n",
    "        while sentence:\n",
    "            outputs = decode(sentence, model, sess, enc_vocab)\n",
    "            # If there is an EOS symbol in outputs, cut them at that point.\n",
    "            if INIT_VOCAB.index(END_TOKEN) in outputs:\n",
    "                outputs = outputs[:outputs.index(INIT_VOCAB.index(END_TOKEN))]\n",
    "                print(\" \".join([tf.compat.as_str(rev_dec_vocab[output]) for output in outputs]))\n",
    "                sentence = input(\"> \")\n",
    "            \n",
    "run_chatbot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation\n",
    "In the last stop of training the model reported global [perplexity](https://www.tensorflow.org/tutorials/recurrent#loss_function) around 8.3\n",
    "\n",
    "Follows an image with a sample conversation, to help evaluate the qualitative side of the model:\n",
    "\n",
    "<img src=\"chat_sample.png\" alt=\"Sample conversation\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "This project showed how a simple generative chatbot can be created using a Recurrent Neural Net. The final results indicates that the model can perform reasonably well for a open conversation chatbot, even though it still makes grammar mistakes and sometimes gives very vague or unrelated answers.\n",
    "\n",
    "## Project Challenges\n",
    "Here I present some of the challenges I faced when tring to train the model for this project:\n",
    "\n",
    "- Initially I tried to use the [ubuntu-dialog corpus](http://dataset.cs.mcgill.ca/ubuntu-corpus-1.0/). The dataset proved to be very large (a few Gb) and it took several hours just to preprocess the data. I decided that this corpus would be to complex to train on and decided to use the The Cornell Movie Dialog Corpus.\n",
    "- The Cornell Movie Dialog Corpus is a smaller and more manageble dataset, but training the model still took several hours (almost two days), while consuming a big part of my computer's resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}