Extend Tokenizer to Support Single Strings and Lists of Strings#258

Merged

avidale merged 4 commits intofacebookresearch:MLH-devfrom Paulooh007:tokenize-function

Nov 7, 2023

+20-5

Commits on Nov 2, 2023

Commits on Nov 5, 2023

Handle string input in call method
Paulooh007
committed