Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huggingface InferenceAPI stopped working after 0.9.0.13 due to name parameter not being set #998

Closed
ppietikainen opened this issue Nov 12, 2024 · 1 comment · Fixed by #1011
Assignees
Labels
bug Something isn't working generators Interfaces with LLMs

Comments

@ppietikainen
Copy link

python3 -m garak --model_type huggingface.InferenceAPI --model_name microsoft/Phi-3-mini-4k-instruct --probes malwaregen.Evasion

stopped working after 0.9.0.13 (it just gets stuck). Logging https requests and digging into the code:

send: b'POST /models/ HTTP/1.1\r\nHost: api-inference.huggingface.co\r\nUser-A
gent: python-requests/2.32.3\r\nAccept-Encoding: gzip, deflate, zstd\r\nAccept
: /\r\nConnection: keep-alive\r\nAuthorization: Bearer ...

when it had the model name in the URI before.

class InferenceAPI(Generator):
...
self.uri = self.URI + name

If I change name to config_root.plugins.model_name it works like before, but maybe not the correct fix? I'm having a hard time figuring out the call chain (_plugins.py through klass() without name being passed, apparently). 0.9.0.13 inherited from HFCompatible, which seems to do some magic in this area.

@ppietikainen ppietikainen added the bug Something isn't working label Nov 12, 2024
@leondz
Copy link
Collaborator

leondz commented Nov 12, 2024

Thank you! Will take a look

@leondz leondz added the generators Interfaces with LLMs label Nov 12, 2024
@jmartin-tech jmartin-tech self-assigned this Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working generators Interfaces with LLMs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants