test: OpenAI frontend invalid chat tokenizer network issue WAR

[kthui]

What does the PR do?

Allow test_chat_completions_invalid_chat_tokenizer test to be skipped if there is a network issue when accessing the invalid tokenizer.

Checklist

• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

• build
• ci
• docs
• feat
• fix
• perf
• refactor
• revert
• style
• test

Related PRs:

N/A

Where should the reviewer start?

N/A

Test plan:

The affected test should continue to pass on existing pipeline, and skipped when there is a network issue.

• CI Pipeline ID: N/A

Caveats:

N/A

Background

N/A

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

N/A

[kthui]

When there is a network issue:

tests/test_chat_completions.py::TestChatCompletionsTokenizers::test_chat_completions_invalid_chat_tokenizer
...
transformers.__version__='4.45.1'
SKIPPED (HuggingFace network issues)

[rmccorm4]

This is fine for now, but just be wary if the message changes slightly in the future it might not be an exact match.

[kthui]

Yes. I want to be cautious on this first, to avoid skipping the test for anything else.

If the message varies in the future, I think we can update the condition to detect if the network communication is disabled, i.e. HF_HUB_OFFLINE, and skip the test based on that.

[rmccorm4]

Thanks for the fix Jacky!

For future reference, this error is probably due to some env var or change to transformers build to disable network communication with HF and only allow pre-downloaded local files, such as via HF_HUB_OFFLINE