What's Changed
- [BUG] Fix path to support windows machines by @not-lain in #12
- support local tiktokenizer loading by @not-lain in #13
- Fix #10: Add local tokenizer support by @bhavnicksm in #16
- Bump version to 0.2.2 and update init.py to reflect the new version by @bhavnicksm in #17
New Contributors
Full Changelog: v0.2.1...v0.2.2