New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Increase timeouts for local LLMs #57

Merged

kaspermarstal merged 5 commits into main from http-timeout

Nov 19, 2024

Contributor

kaspermarstal commented Nov 17, 2024 •

edited

Loading

Assume local model by default, so significantly increase default timeout. It's really sad to run a large model locally and have the response cut short by a timeout.
Set shorter timeout for hosted model configurations.
Set HttpClients' timeouts to one hour so it doesn't interfere with polly

kaspermarstal added 5 commits

November 17, 2024 22:42


          fix: Increase timeouts for local LLMs

201400b


          fix: Set CellConfiguration

5f0eeeb


          Merge branch 'main' into http-timeout

be37f00


          fix: Set HttpClients' timeouts to one hour so it doesn't interfere wi…

99f92cb

…th polly


          style: dotnet format

8ccd71a

kaspermarstal merged commit 303274d into main

2 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet