Skip to content

Commit

Permalink
Tokenizer resolution: logging
Browse files Browse the repository at this point in the history
  • Loading branch information
stefanik12 committed Apr 18, 2024
1 parent c7f35e2 commit fab201e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion adaptor/lang_module.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def _find_and_load_tokenizer(model_name_or_path) -> PreTrainedTokenizer:
if any(f.startswith("tokenizer") for f in os.listdir(os.path.join(root, subdir)))]
if not subdirs_with_tokenizer:
raise OSError("Could not find a tokenizer in any of the subdirectories %s "
"of given model_name_or_path='%s'", subdirs, root)
"of given model_name_or_path='%s'" % (subdirs, root))
tokenizer_dir = subdirs_with_tokenizer[0]

tokenizer = AutoTokenizer.from_pretrained(tokenizer_dir)
Expand Down

0 comments on commit fab201e

Please sign in to comment.