Skip to content

0.0.1-dev2 - Speedups

Pre-release
Pre-release
Compare
Choose a tag to compare
@michaelfeil michaelfeil released this 12 Oct 01:46
· 927 commits to main since this release
0.0.1-dev2
3ed24bb
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

adds new dependency (orjson) for faster response serialization - 300%
uses torch.inference_mode() and delayed moving to CPU - 10%
adds uvicorn[standard] - slightly faster 2-5%?
Updates readme

#2

Assets 2
Loading