Skip to content

[v0.24.1] Handle [DONE] signal from TGI + remove logic for "non-TGI servers"
Compare
Choose a tag to compare
@Wauplin Wauplin released this 23 Jul 14:44
· 139 commits to main since this release
v0.24.1
1dd68a9
This commit was signed with the committer’s verified signature.
Wauplin Lucain
GPG key ID: 9838FE02BECE1A02
Learn about vigilant mode.

This release fixes 2 things:

  • handle "[DONE]" message in chat stream (related to TGI update huggingface/text-generation-inference#2221)
  • remove the "non-TGI" logic in chat completion since all models support server-side rendering now that even transformers-backed models are TGI-server.

See #2410 for more details.

Full Changelog: v0.24.0...v0.24.1

Assets 2
Loading