feat(jsonish): Add support for curly quotes in JSON parsing

[revidious]

Description

LLMs will sometimes output JSON with curly quotes (U+201C and U+201D) instead of straight quotes (U+0022). This PR adds automatic normalization of these quotes during parsing, making BAML more robust when handling LLM outputs.

Changes

• Added normalize_quotes() function that converts:
  • Left double quotation mark (U+201C) → Basic quotation mark (U+0022)
  • Right double quotation mark (U+201D) → Basic quotation mark (U+0022)
• Applied normalization before all parsing attempts:
  • Standard JSON parsing (serde_json)
  • Markdown-embedded JSON
  • Multiple JSON objects
  • JSON fixing parser
• Preserved original string for error messages and output

Fixes #1074

---

Important

Add support for curly quotes in JSON parsing and implement custom SIGINT handling for graceful shutdowns.

• JSON Parsing:
  • Added normalize_quotes() in entry.rs to convert curly quotes (U+201C, U+201D) to standard quotes (U+0022).
  • Applied normalization in parse() before parsing JSON, Markdown-embedded JSON, multiple JSON objects, and JSON fixing parser.
• Signal Handling:
  • Implemented custom SIGINT handling in lib.rs using ctrlc crate to ensure graceful shutdown across Python/Rust boundary.
  • Added dependencies libc, ctrlc, and tokio-util in Cargo.toml for signal handling support.

^{This description was created by for 0513ae7. It will automatically update as commits are pushed.}

[vercel]

@revidious is attempting to deploy a commit to the Gloo Team on Vercel.

A member of the Team first needs to authorize it.

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 0513ae7 in 1 minute and 6 seconds

More details

• Looked at 151 lines of code in 3 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. engine/baml-lib/jsonish/src/jsonish/parser/entry.rs:41

• Draft comment:
  Consider normalizing quotes once and reusing the result to avoid redundant operations.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The normalize_quotes function is correctly implemented to replace curly quotes with straight quotes. However, the function is called multiple times in the parse function, which could lead to unnecessary performance overhead. It would be more efficient to call normalize_quotes once and reuse the result.

2. engine/language_client_python/src/lib.rs:32

• Draft comment:
  Consider documenting any potential limitations or side effects of using the ctrlc crate for SIGINT handling.
• Reason this comment was not posted:
  Confidence changes required: 33%
  The ctrlc crate is used to handle SIGINT signals, which is a good approach for ensuring graceful shutdowns. However, the comment mentions that this is a hacky solution. It would be beneficial to document any potential limitations or side effects of this approach in the code comments.

Workflow ID: wflow_2kgUvhKu6kb7Dltf

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[imalsogreg]

@revidious Thanks for this PR, it looks great!

Regarding curly quotes, do you think it's better to do normalization as a preprocessing step, as you have done in the PR? Or should we add curly quotes as another type of quotation mark, alongside the parsers for quoted string, single-quoted string, and backtick-string, here: https://github.com/BoundaryML/baml/blob/canary/engine/baml-lib/jsonish/src/jsonish/parser/fixing_parser/json_parse_state.rs#L540 ?