Hybrid Code Networks (HCN)

Hybrid Code Networks

• Jason D. Williams, Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning

HCN combines an RNN with domain-specific knowledge encoded as software and system action templates.

4 Major components of HCN are

1. RNN
2. Domain Specific Software
3. Domain Specific Action Template
4. Entity Extraction Module

Both the RNN and the developer code maintain dialog state. Each action template can be a textual communicative action or an API call.

1. Utterance : The user provides an utterance as text.
2. BoW : Bag of words representation of the utterance is done.
3. Utterance Embedding : Average word embedding of utterance is calculated. 300 dimensional word2vec is used. The update of the word embeddings is forbidden during training.
4. Entity Extraction : Identify entities in the utterance, like name, place, time, etc,.
5. Entity Tracking : Map entities in the utterance, to a row in database; override existing entities (in buffer) with new entities.
6. Action Mask : A bit vector representation of which actions are allowed given the context at current timestamp (turn)
   • Context Features : Handwritten features extracted from entities that may be useful for distinguishing among actions
   • A feature vector is formed based on information extracted from (1) to (5).
7. The feature vector is passed to an RNN
8. RNN computes and maintains a hidden(internal) state
9. Based on the hidden state, an output distribution over the list of distinct system action templates
10. The action mask from (6), is applied as elementwise multiplication which should elimante non-permitted actions in the probability distribution over all actions
11. An action is selected from the output distribution

• In a supervised learning setting, the action that has the maximum probability is selected
• In a reinforcement learning setting, an action is sampled from the distribution

12. Entity Output : Based on the selected action, entities are filled in; fully-formed actions are produced