Skip to content

Commit

Permalink
Improved JSONSchemaParser performance (#93)
Browse files Browse the repository at this point in the history
* Improving backslash escaping performance, based on github discussion

* Taking the finished parser's last parsed string before discarding it
  • Loading branch information
noamgat authored Apr 20, 2024
1 parent 47d545a commit f4d6496
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions lmformatenforcer/jsonschemaparser.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,15 @@ def add_character(self, new_character: str) -> CharacterLevelParser:
option_json_schema_parsers.append(option_parser)
return UnionParser(option_json_schema_parsers)

# For some performance optimizations to work, we want to make sure we don't leave irrelevant
# objects at the top of the stack, which we know will be passed over next timestep
new_object_stack = updated_parser.object_stack
while new_object_stack and new_object_stack[-1].can_end() and new_object_stack[-1].get_allowed_characters() == '':
finished_receiver = new_object_stack[-1]
if isinstance(finished_receiver, StringParsingState):
updated_parser.last_parsed_string = finished_receiver.parsed_string
del new_object_stack[-1]

return updated_parser

def get_allowed_characters(self) -> str:
Expand Down

0 comments on commit f4d6496

Please sign in to comment.