v0.4.0

Summary

• faster Formatter compilation
• Refactors to make internal implementation and public API cleaner
• Native support for basic json schema(all type keyword constraints and recursion)
• Basic support for substring extraction

What's changed

• Removed grammar_generators package completely.
  • JsonExtractor class is moved to formats.json package.
• FormatterBuilder.schema() is removed.
• Added FormatterBuilder.json() which is simpler to use than FormatterBuilder.schema().
• RegexExtractor is moved from extractor to formats.regex
• kbnf_representation is renamed to kbnf_reference and nonterminal is renamed to kbnf_definition for clarity in Extractor.
• NonterminalExtractor is added to better illustrate the differences in interfaces&implementation between LiteralExtractor and other extractors.
• LiteralExtractor is private now,
• Bug fix: JsonExtractor will no longer produce None when parsing json where some keys/values contain { or }.
• Make Formatron compatible with the newest vllm==0.6.0 and exllamav2==0.2.1.
• Faster constrained decoding with exllamav2 integration thanks to @turboderp 's optimizations in his exllamav2 library.
• Faster kbnf.Vocabulary creation by @turboderp in #10
• Update docs, benchmarks and comments accordingly.

Full Changelog: v0.3.4...v0.4.0

v0.3.4

• Bug fix: directly import formatron.formatter with pip installed package no longer produces ModuleNotFoundError

v0.3.3

• Bug fix: exllamav2 integration will now correctly end generation with exactly one eos token

v0.3.2

• Bug fix: literals passed to FormatterBuilder now implements str correctly

v0.3.1

• Fix submodule imports

v0.3.0

• Add utilities in FormatterBuilder to make using custom CFG easier
• Fix a bug when JsonExtractor is used together with ChoiceExtractor
• Fix readme example

v0.2.0

• logits_processors first run execution time improved
• Remove not_contain parameter from FormatterBuilder.str() method
• improve huggingface integration's speed significantly

v0.1.3

• Llama3 huggingface vocabulary is now handled correctly for non-ascii characters.
• Vllm integrations no longer crash on batch of different generated sequence length
• logits_processors execution time improved

v0.1.2

• Fixed import namespace issue in readme.md examples
• Fixed #TODO in readme.md function call examples
• Added manual reset API to integrations' logits_processor so users can manually reset formatter state if the generation does not complete normally
• Fixed a bug in exllamav2 integrations so Exllamav2Filter can be reused across generations
• Significantly improved the speed of formatter.mask_logits on torch tensors