MultiWOZ turn based dataset

[oplatek]

• whole conversation view with reference instructions to users. See Multiwoz #63
• Turn based view:
  • Not straightforward using huggingface Dataset & tabgenie: I need 1 - to n mapping between
    • Need to pregenerate all prefixes - bad for storage
    • Need to create completely new class DialogueDataset / equivalent to TabularDataset
      • computing of get_example_count will differ - turns instead of conversations (as tables)
      • the turn prefixes will be generated on the fly from the stored conversation
  • system view
  • user view
• consider adding multiple annotation. See "all_version" multiwoz version https://huggingface.co/datasets/pietrolesci/multiwoz_all_versions/viewer/pietrolesci--multiwoz_all_versions/test

See also budzianowski/multiwoz#119 (comment)

[oplatek]

Harmonize with https://storage.googleapis.com/gresearch/dstc11/dstc11.2022-10-17a.html ?