feat: allow optional lists and maps

[revidious]

Allow optional lists and maps

Resolves #948

Description

This PR adds support for optional lists and maps in BAML, allowing users to declare properties like string[]? and map<string, int>?. Previously these types were not supported, and the parser would coerce unset parameters into empty lists/maps.

Changes

1. Parser (parse_types.rs):
   • Added support for optional arrays with string[]? syntax

• Added optional list and map support using syntax of map<string, int>?
  • Plenty of test cases covered, both optional list and optional map

2. Json Schema Generation(json_schema.rs)
   • Supporting the optional array and maps in the generation
   • Correct JSON schema is generated in, with
     • appropriate type definitions([\"array\", \"null\"], or [\"object\",\"null\"])
     • Null default values

• Documentation

3. Grammar (datamodel.pest):
   • Changed array notation to allow optional token
   • Changed map syntax to allow optional token

Example Usage

class Foo {
    optional_array string[]?
    optional_map map<string, int>?
}

Testing

• Added parser tests for optional arrays and maps
• Verified JSON schema generation
• Tested with various type combinations

---

Important

Adds support for optional lists and maps in BAML, updating parser, JSON schema generation, and grammar to handle string[]? and map<string, int>? syntax.

• Behavior:
  • Support for optional lists and maps added in parse_types.rs using string[]? and map<string, int>? syntax.
  • Parser now distinguishes between required and optional types, updating FieldArity accordingly.
  • JSON schema generation in json_schema.rs updated to handle optional arrays and maps, allowing null values.
• Grammar:
  • Updated datamodel.pest to allow optional token for arrays and maps.
• Testing:
  • Added tests for optional arrays and maps in parse_types.rs to verify correct parsing and schema generation.

^{This description was created by for 8a31ab9. It will automatically update as commits are pushed.}

[vercel]

@revidious is attempting to deploy a commit to the Gloo Team on Vercel.

A member of the Team first needs to authorize it.

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 8a31ab9 in 20 seconds

More details

• Looked at 306 lines of code in 3 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. engine/baml-lib/baml-core/src/ir/json_schema.rs:235

• Draft comment:
  The required field is being populated with optional fields instead of required ones. It should be populated with fields that are not optional.

if let FieldType::Optional(_) = t {
    // Do nothing for optional fields
} else {
    required_props.push(name.clone());
}

• Reason this comment was not posted:
  Comment was not on a valid diff hunk.

Workflow ID: wflow_hG9ygX9vO60xfENi

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[aaronvg]

nice work on all these PRs! This one LGTM, will ping someone else on the team to also take a look.

[antoniosarosi]

@revidious LGTM! I made some changes to your PR, mainly removed the "Maps/Lists are not allowed to be optional" error, added validation and integ tests for the new feature and then I made some changes to the spelling in the docs you wrote because it looks like you used some kind of automatic spell-checker that messes up the punctuation and capitalization.