Add special_tokens_in_strings Arg to byte_pair_tokenizer. #1546
Open
abuelnasr0 wants to merge 4 commits intokeras-team:masterfrom abuelnasr0:byte_pair_tokenizer
+194-171
special_tokens_in_strings Arg to byte_pair_tokenizer. #1546