v0.4.5

Dan-wanna-M released this 06 Oct 18:48

· 33 commits to master since this release

718b02e

Fix quality degradation when duplicate tokens(tokens that correspond to the same bytes but different token ids) are present in vocabulary.

Assets 2